RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,339 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

3339 Commits

Author SHA1 Message Date
Steven Almeroth e66db6fa56 add new sep-20 and update sep-19 with table at the top 2013-03-08 13:04:11 -06:00
Pablo Hoffman eeb69d2f70 added #260 to release notes 2013-03-08 11:59:38 -02:00
Pablo Hoffman 42499c72b1 updated AUTHORS 2013-03-08 11:57:54 -02:00
Pablo Hoffman b5b37197bf Merge pull request #260 from llonchj/entry_point 
Commands module
 2013-03-08 05:49:08 -08:00
Pablo Hoffman b560f4d1c1 sep-019: remove TODO section (settled down with class method) 2013-03-08 09:57:12 -02:00
Pablo Hoffman a44757179b sep-019: other minor fixes 2013-03-07 19:38:12 -02:00
Pablo Hoffman c7add1e5bd sep-019: fix typos 2013-03-07 19:24:44 -02:00
Pablo Hoffman d806acdf50 sep-019: minor tide up 2013-03-07 12:39:40 -02:00
Pablo Hoffman 6f24b46115 added SEP-019 (per-spider settings) - first draft 2013-03-07 12:34:17 -02:00
Pablo Hoffman a027da0a2c Some changes to scrapy deploy command: 
- use branch names when constructing HG/GIT versions
- added -d flag for debugging (keeps build directory)
- improved command output (redirected setup.py stderr)
 2013-03-06 11:41:34 -02:00
Jordi Llonch 5b118ff4ab added documentation (experimental feature) 2013-03-06 06:36:23 +11:00
Jordi Llonch d9261f6c54 pluggable sub-commands for scrapy comand-line 2013-03-06 05:25:38 +11:00
Pablo Hoffman 19d0942c74 Merge branch 'shell' of git://github.com/stav/scrapy into stav-shell 2013-03-04 02:14:01 -02:00
Pablo Hoffman 3c8eef99cb docs/contributing: added note explaining what Scrapy contrib is 2013-03-04 01:35:17 -02:00
Pablo Hoffman 7dd360f39f Merge pull request #257 from stav/cleanups 
doc: fix typo in spider middleware
 2013-03-03 19:17:46 -08:00
Steven Almeroth 81111dd39a fetch command should catch IgnoreRequest exception 2013-03-02 20:13:14 -06:00
Steven Almeroth f62b6660d4 doc: fix typo in spider middleware 2013-03-02 19:46:31 -06:00
Pablo Hoffman d5d944fa44 Log overriden Scrapy settings when Scrapy starts. 
This is useful for debugging, to quickly find out which settings where
used for a specific spider run. dict settings are omitted for brevity.
DEBUG level is used for consistency with "Enabled
extensions/middlewares" lines (which share a similar purpose).
 2013-02-28 11:35:11 -02:00
Pablo Hoffman 8f3a509d44 remove debugging code 2013-02-27 03:52:55 -02:00
Pablo Hoffman 3a6b80259d arg_to_iter: replaced double call to isinstance with a single one. refs #248 2013-02-27 03:51:23 -02:00
Pablo Hoffman 2bbd92742b arg_to_iter: treat items the same way as dicts (ie. non iterables). fixes #248 2013-02-27 02:39:31 -02:00
Pablo Hoffman 7400ceb1ed added 502 to RETRY_HTTP_CODES 2013-02-22 19:12:59 -02:00
Pablo Hoffman e3f50b97b2 made scrapy/item.py pep8 compliant 2013-02-20 11:38:50 -02:00
Pablo Hoffman a038f46859 doc: fixed rst title 2013-02-14 11:11:17 -02:00
Pablo Hoffman 22edc44c6c doc: remove links to diveintopython.org, which is no longer available. closes #246 2013-02-14 11:09:40 -02:00
Pablo Hoffman aeb7fbe221 Merge pull request #240 from llonchj/spider_log 
spider.log to pass keyword arguments into twisted log.msg
 2013-02-12 18:13:26 -08:00
Pablo Hoffman 07669947c7 Merge pull request #244 from morty/master 
scrapy genspider allows you to create a spider module with the same name as the project
 2013-02-12 18:11:49 -08:00
Tom Mortimer-Jones 2efd859525 Added check so that genspider cannot create a spider with the same name as the project. 2013-02-12 12:26:38 +00:00
Pablo Hoffman 1ff8b4f831 updated release notes with previous commit 2013-02-12 00:59:25 -02:00
Rolando Espinoza La fuente 5971333a93 added --pdb command option to enable pdb debugger on failure. 2013-02-11 12:34:29 -04:00
Pablo Hoffman d9043ffa6e reverted previous changed as it broke tests 2013-02-11 11:43:25 -02:00
Pablo Hoffman 2df010a972 Merge pull request #241 from zuhao/master 
Fix url_has_any_extension bug
 2013-02-11 05:29:14 -08:00
Zuhao Wan 27ca25472f Fix url_has_any_extension bug 2013-02-11 17:19:31 +08:00
Jordi Llonch 6d4b764f29 spider.log to pass keyword arguments into twisted log.msg 2013-02-11 05:00:06 +11:00
Daniel Graña c0a7040f16 add PHONYs so build does not match build/ path 2013-02-08 15:07:00 -02:00
Daniel Graña 910effd145 get scrapy version from package data 2013-02-06 11:44:26 -02:00
Daniel Graña 79cb0318d0 update travis-ci conf 2013-02-06 10:36:08 -02:00
Daniel Graña 5db45b3825 remove scrapyd, it was migrated to its own repository 2013-02-06 05:24:07 +00:00
Daniel Graña 33ca295129 Merge pull request #235 from whodatninja/patch-1 
Fix typo labeling attrs type bool instead of list
 2013-02-05 17:27:10 -08:00
whodatninja 8e3b5baac5 Fix typo labeling attrs type bool instead of list 2013-02-05 15:10:41 -05:00
Daniel Graña f293d08e7a Merge pull request #234 from darkrho/httpcache-dbm-module-fullpath 
allow to use full path in HTTPCACHE_DBM_MODULE setting
 2013-02-01 09:01:27 -08:00
Rolando Espinoza La fuente d00e6cf1d7 tests: dummydbm.error should be KeyError. 2013-02-01 12:52:59 -04:00
Rolando Espinoza La fuente 6ed44a3a67 httpcache: added tests for custom dbm module. 
And removed the break in the line to improve readability as there
is no strict 80-chars line width convention.
 2013-02-01 12:41:52 -04:00
Rolando Espinoza La fuente a2602df90b httpcache: allow to import submodules within packages as HTTPCACHE_DBM_MODULE setting. 2013-01-31 23:49:36 -04:00
Daniel Graña e5edb8ec3e Merge branch 'dangra/issue-24' 2013-01-30 16:55:53 -02:00
Daniel Graña 3af240f5f9 Merge branch 'dangra/issue-12' 2013-01-30 16:55:16 -02:00
Daniel Graña 0d3e4b4c43 do not unquote slash and question mark in url paths. fix #24 2013-01-30 11:22:03 -02:00
Daniel Graña 872a22df68 pep8ize sgml link extractors 2013-01-29 18:08:32 -02:00
Daniel Graña cc69b3aa4c Fix #199 encoding error concatenating link text 2013-01-29 17:56:34 -02:00
Daniel Graña 95fde0a498 register namespaces when XMLFeedSpider uses iternodes mode. fixes #12 2013-01-29 12:00:13 -02:00