RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,357 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

3357 Commits

Author SHA1 Message Date
Pablo Hoffman bb20907254 minor updated to faq 2013-03-14 16:43:00 -03:00
Pablo Hoffman 098ccff862 added FAQ about error: "cannot import name crawler" 2013-03-14 12:57:59 -03:00
Pablo Hoffman a862f23376 added Nicolas Ramirez to AUTHORS 2013-03-14 12:44:39 -03:00
Pablo Hoffman 46b305ef7d Merge pull request #267 from nramirezuy/formrequest 
Override Request 'method' in FormRequest
 2013-03-14 08:43:53 -07:00
Nicolás Ramírez f043bba049 Override request method in FormRequest 
Changes proposed in the comments
 2013-03-14 10:39:17 -03:00
Pablo Hoffman a2e9d031f9 removed duplicated test 2013-03-14 10:32:19 -03:00
Pablo Hoffman 8391b36251 minor updates to contributing doc 2013-03-13 03:24:25 -03:00
Pablo Hoffman 51c301b3a2 added link to python binary libs, for windows installation 2013-03-13 03:18:33 -03:00
Pablo Hoffman 8e72730792 Merge pull request #261 from stav/allowed_domains 
allow spider allowed_domains to be set/tuple, #259
 2013-03-12 20:44:51 -07:00
Pablo Hoffman 296db1dc09 log (just once) when duplicate requests are filtered out. closes #105, #249 2013-03-12 19:28:43 -03:00
Pablo Hoffman a4507f62c3 sep-019: fixed typo 2013-03-12 18:44:09 -03:00
Steven Almeroth 650eda68da doc: add comment about commit history cleanliness 2013-03-10 18:51:04 -06:00
Steven Almeroth 5828179c5f remove over-testing and dict testing for test_utils_url.py 2013-03-10 18:30:49 -06:00
Steven Almeroth 1514b3b5db pylint clean-ups for test_utils_url.py 2013-03-10 16:47:24 -06:00
Steven Almeroth a613e15154 add tests for url_is_from_spider() with allowed_domains 2013-03-10 16:41:27 -06:00
Pablo Hoffman b2b256dc37 added nicolas to sep-019 authors, changed main title formatting for sep-019 and sep-020 2013-03-10 14:22:08 -03:00
Pablo Hoffman b11a1326b9 Merge pull request #264 from stav/sep 
add new sep-20 and update sep-19 with table at the top
 2013-03-08 11:23:15 -08:00
Steven Almeroth e66db6fa56 add new sep-20 and update sep-19 with table at the top 2013-03-08 13:04:11 -06:00
Pablo Hoffman eeb69d2f70 added #260 to release notes 2013-03-08 11:59:38 -02:00
Pablo Hoffman 42499c72b1 updated AUTHORS 2013-03-08 11:57:54 -02:00
Pablo Hoffman b5b37197bf Merge pull request #260 from llonchj/entry_point 
Commands module
 2013-03-08 05:49:08 -08:00
Pablo Hoffman b560f4d1c1 sep-019: remove TODO section (settled down with class method) 2013-03-08 09:57:12 -02:00
Pablo Hoffman a44757179b sep-019: other minor fixes 2013-03-07 19:38:12 -02:00
Pablo Hoffman c7add1e5bd sep-019: fix typos 2013-03-07 19:24:44 -02:00
Pablo Hoffman d806acdf50 sep-019: minor tide up 2013-03-07 12:39:40 -02:00
Pablo Hoffman 6f24b46115 added SEP-019 (per-spider settings) - first draft 2013-03-07 12:34:17 -02:00
Pablo Hoffman a027da0a2c Some changes to scrapy deploy command: 
- use branch names when constructing HG/GIT versions
- added -d flag for debugging (keeps build directory)
- improved command output (redirected setup.py stderr)
 2013-03-06 11:41:34 -02:00
Steven Almeroth b48ec1dce4 allow spider allowed_domains to be set/tuple, #259 2013-03-06 00:19:47 -06:00
Jordi Llonch 5b118ff4ab added documentation (experimental feature) 2013-03-06 06:36:23 +11:00
Jordi Llonch d9261f6c54 pluggable sub-commands for scrapy comand-line 2013-03-06 05:25:38 +11:00
Pablo Hoffman 19d0942c74 Merge branch 'shell' of git://github.com/stav/scrapy into stav-shell 2013-03-04 02:14:01 -02:00
Pablo Hoffman 3c8eef99cb docs/contributing: added note explaining what Scrapy contrib is 2013-03-04 01:35:17 -02:00
Pablo Hoffman 7dd360f39f Merge pull request #257 from stav/cleanups 
doc: fix typo in spider middleware
 2013-03-03 19:17:46 -08:00
Steven Almeroth 81111dd39a fetch command should catch IgnoreRequest exception 2013-03-02 20:13:14 -06:00
Steven Almeroth f62b6660d4 doc: fix typo in spider middleware 2013-03-02 19:46:31 -06:00
Pablo Hoffman d5d944fa44 Log overriden Scrapy settings when Scrapy starts. 
This is useful for debugging, to quickly find out which settings where
used for a specific spider run. dict settings are omitted for brevity.
DEBUG level is used for consistency with "Enabled
extensions/middlewares" lines (which share a similar purpose).
 2013-02-28 11:35:11 -02:00
Pablo Hoffman 8f3a509d44 remove debugging code 2013-02-27 03:52:55 -02:00
Pablo Hoffman 3a6b80259d arg_to_iter: replaced double call to isinstance with a single one. refs #248 2013-02-27 03:51:23 -02:00
Pablo Hoffman 2bbd92742b arg_to_iter: treat items the same way as dicts (ie. non iterables). fixes #248 2013-02-27 02:39:31 -02:00
Pablo Hoffman 7400ceb1ed added 502 to RETRY_HTTP_CODES 2013-02-22 19:12:59 -02:00
Pablo Hoffman e3f50b97b2 made scrapy/item.py pep8 compliant 2013-02-20 11:38:50 -02:00
Pablo Hoffman a038f46859 doc: fixed rst title 2013-02-14 11:11:17 -02:00
Pablo Hoffman 22edc44c6c doc: remove links to diveintopython.org, which is no longer available. closes #246 2013-02-14 11:09:40 -02:00
Pablo Hoffman aeb7fbe221 Merge pull request #240 from llonchj/spider_log 
spider.log to pass keyword arguments into twisted log.msg
 2013-02-12 18:13:26 -08:00
Pablo Hoffman 07669947c7 Merge pull request #244 from morty/master 
scrapy genspider allows you to create a spider module with the same name as the project
 2013-02-12 18:11:49 -08:00
Tom Mortimer-Jones 2efd859525 Added check so that genspider cannot create a spider with the same name as the project. 2013-02-12 12:26:38 +00:00
Pablo Hoffman 1ff8b4f831 updated release notes with previous commit 2013-02-12 00:59:25 -02:00
Rolando Espinoza La fuente 5971333a93 added --pdb command option to enable pdb debugger on failure. 2013-02-11 12:34:29 -04:00
Pablo Hoffman d9043ffa6e reverted previous changed as it broke tests 2013-02-11 11:43:25 -02:00
Pablo Hoffman 2df010a972 Merge pull request #241 from zuhao/master 
Fix url_has_any_extension bug
 2013-02-11 05:29:14 -08:00