178440a39a
Correct login request example
...
In the example log is not imported
and not found in the global name space
2015-05-05 10:36:02 +02:00
3f3263dd83
Update request-response.rst
...
Fixed minor typo
2014-09-14 10:54:43 -04:00
8bdb6e2e3e
Elaborated request priority value.
2014-05-07 19:14:45 +09:00
eb07e09166
Merge pull request #663 from pawl/patch-1
...
fixed typo
2014-04-24 17:59:36 -04:00
b4593c2ae7
document shortcuts in TextResponse class
2014-04-24 00:15:00 -03:00
2d3803672b
DOC use top-level shortcuts in docs
2014-04-15 01:09:35 +06:00
a1ee354609
fixed typo
2014-03-20 15:16:48 -05:00
ca1c1a82b5
FormRequest doc improvements
...
Clickdata doc enhancements:
* Fix xml attributes mention
* nr is 0-indexed reference
2014-03-12 12:34:50 -03:00
e29ab4d112
New doc: clickdata in Formrequest.from_response
...
Documentation about:
* clickdata parameter in Formrequest.from_response
* nr attribute in clickdata dict
* default behaviour when clickdata is None
2014-03-12 06:43:50 -03:00
72b6c96d9a
Running lucasdemarchi/codespell to fix typos in docs
2014-03-06 12:40:55 -03:00
a27d91f0a6
Rename BaseSpider to Spider. See GH-495.
2013-12-30 19:46:41 +06:00
e42e3743fe
quick documentation for #475
2013-12-24 12:19:15 -02:00
086b8a20d4
typo fix in TextResponse docs
2013-10-17 04:50:30 +06:00
a3b711bdea
Move callback blob; mention errback
2013-10-15 12:19:42 +01:00
0ba0d85685
Parameters to Request() in wrong order
...
Implied that callback wasn't the first optional unnamed parameter.
2013-10-15 11:50:43 +01:00
37c24e01d7
document bindaddress request meta
2013-10-02 17:13:17 -03:00
a9c3519897
updated required twisted version to 10.0
2013-10-01 14:07:38 -03:00
71b320914a
Update request-response.rst
...
Fix small doc typo (too many backticks)
2013-09-18 11:45:25 -03:00
1a1c93fafe
tiny FormRequest doc fix
2013-07-15 15:47:34 +06:00
70179c7c0c
doc: remove trailing spaces
2013-03-21 13:57:39 -06:00
0d7747d353
doc: Response.replace() cannot take meta argument
...
>>> response.replace(meta={'foo':1})
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/srv/scrapy/scrapy-fork/scrapy/scrapy/http/response/text.py", line 45, in replace
return Response.replace(self, *args, **kwargs)
File "/srv/scrapy/scrapy-fork/scrapy/scrapy/http/response/__init__.py", line 77, in replace
return cls(*args, **kwargs)
File "/srv/scrapy/scrapy-fork/scrapy/scrapy/http/response/text.py", line 22, in __init__
super(TextResponse, self).__init__(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'meta'
2013-03-21 13:49:55 -06:00
aae6aed4fb
fixes spelling errors in documentation
2013-01-22 14:52:18 -08:00
897195186a
document new FormRequest parameter named `formxpath` that matches forms using xpath
2013-01-08 18:36:20 -02:00
39274a2457
doc: removed obsolete references to ClientForm
2012-11-23 19:06:47 -02:00
babfc6e79b
Updated documentation after singleton removal changes.
...
Also removed some unused code and made some minor additional
refactoring.
2012-08-28 18:35:57 -03:00
177c81745d
domain and path support for request cookies
2012-06-25 20:17:59 +02:00
da826aa13d
fixed minor mistake in Request objects documentation
2012-03-21 10:25:41 +08:00
26c8004125
added documentation for the new cookiejar Request.meta key
2012-02-27 19:58:58 -02:00
41fd3c4f6c
doc: removed duplicated callback argument from Request.replace()
2011-12-23 15:55:46 -02:00
6a31ab667d
minor fix to doc
2011-09-01 15:08:23 -03:00
d98b058c21
no longer recommend using labmda's in the doc, as they're not friendly with scheduler persistence
2011-09-01 15:06:49 -03:00
5c6b0631e2
minor doc fix
2011-08-19 11:42:03 -03:00
9d97e73a24
fixed priority handling on the new scheduler so that it's backwards compatible (ie. bigger priorities are higher). also fixed a few documentation bugs related to requests priority
2011-08-19 08:26:41 -03:00
3ee2c94e93
Improved cookies middleware by making COOKIES_DEBUG nicer and documenting it
2011-04-06 14:54:48 -03:00
91a7c25797
* Made Response.meta attribute map to Request.meta attribute. Closes #290
...
* Record redirected URLs in redirect middleware. Closes #291
2010-11-18 12:51:54 -02:00
9288f622f9
Added formname parameter for FormRequest.from_response
2010-09-20 08:33:24 -03:00
bf467fc37a
Check 'dont_merge_cookies' membership in request.meta, instead of getting its value
2010-09-10 15:29:15 -03:00
7d14a52234
Reference dont_merge_cookies in list of special Request.meta keys
2010-09-09 21:54:26 -03:00
7f21a6384f
Documented handle_httpstatus_list request.meta key
2010-09-09 21:50:40 -03:00
f1c943543a
Added dont_retry request.meta key to make RetryMiddleware ignore requests. Closes #234
2010-09-09 21:43:44 -03:00
9f01e3e79e
Added dont_redirect request.meta key to make RedirectMiddleware ignore requests. Closes #233
2010-09-09 21:37:35 -03:00
7da79b90fe
Make url/body attributes of Request/Response objects read-only - use replace() to change them. Deprecation warning left for backwards compatibilty.
2010-09-08 00:15:11 -03:00
c1aab2f58e
Copy callback/errback attributes when copying Requests
2010-09-08 00:15:09 -03:00
9aefa242d5
Applied documentation patch provided by Lucian Ursu ( closes #207 )
2010-08-21 01:26:35 -03:00
73305b1eb3
Added support for Requests without callbacks ( #166 ) - the Spider.parse() method
...
is used in those cases.
Also removed Request.deferred attribute.
2010-06-08 18:18:02 -03:00
c0d45846b8
Automated merge with ssh://hg.scrapy.org/scrapy-0.8
2010-04-26 22:29:45 -03:00
5d03405cac
FormRequest.from_response doc fix. closes #155
...
--HG--
extra : rebase_source : d54979f6a15e5e997072dcbbc6d43b426189312b
2010-04-26 22:28:07 -03:00
db5c3df679
SEP12 implementation
...
* Rename BaseSpider.domain_name to BaseSpider.name
This patch implements the domain_name to name change in BaseSpider class and
change all spider instantiations to use the new attribute.
* Add allowed_domains to spider
This patch implements the merging of spider.domain_name and
spider.extra_domain_names in spider.allowed_domains for offsite checking
purposes.
Note that spider.domain_name is not touched by this patch, only not used.
* Remove spider.domain_name references from scrapy.stats
* Rename domain_stats to spider_stats in MemoryStatsCollector
* Use ``spider`` instead of ``domain`` in SimpledbStatsCollector
* Rename domain_stats_history table to spider_data_history and rename domain
field to spider in MysqlStatsCollector
* Refactor genspider command
The new signature for genspider is: genspider [options] <domain_name>.
Genspider uses domain_name for spider name and for the module name.
* Remove spider.domain_name references
* Update crawl command signature <spider|url>
* docs: updated references to domain_name
* examples/experimental: use spider.name
* genspider: require <name> <domain>
* spidermanager: renamed crawl_domain to crawl_spider_name
* spiderctl: updated references of *domain* to spider
* added backward compatiblity with legacy spider's attributes
'domain_name' and 'extra_domain_names'
2010-04-01 18:27:22 -03:00
1330697c3d
Some improvements to Response encoding support:
...
* added encoding aliases, configurable through a new ENCODING_ALIASES setting
* Response.encoding now returns the real encoding detected for the body
* simplified TextResponse API by removing body_encoding() and
headers_encoding() methods
* Response.encoding now tries to infer the encoding from the body always (it
was done before only on HtmlResponse and TextResponse)
* removed scrapy.utils.encoding.add_encoding_alias() function
* updated implementation of scrapy.utils.response function to reflect these API
changes
* updated documentation to reflect API changes
2010-03-25 15:47:10 -03:00
180c091fb2
Fixed encoding issue (reported in #135 ) when the encoding declared in the HTTP header is unknown. This is the patch proposed by Rolando, with an update to the Request/Response documentation.
2010-02-24 14:01:29 -02:00
Vicente Jimenez Aguilar
John-Scott Atlakson
Rendaw
Pablo Hoffman
Daniel Graña
Mikhail Korobov
Paul Brown
Julia Medina
Capi Etheriel
Pablo Hoffman
Dragon Dave
cacovsky
Steven Almeroth
Chris Tilden
Alexandru Cepoi
Jason Yeo
Shuaib
Rolando Espinoza La fuente