RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,316 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

3316 Commits

Author SHA1 Message Date
Pablo Hoffman a038f46859 doc: fixed rst title 2013-02-14 11:11:17 -02:00
Pablo Hoffman 22edc44c6c doc: remove links to diveintopython.org, which is no longer available. closes #246 2013-02-14 11:09:40 -02:00
Pablo Hoffman aeb7fbe221 Merge pull request #240 from llonchj/spider_log 
spider.log to pass keyword arguments into twisted log.msg
 2013-02-12 18:13:26 -08:00
Pablo Hoffman 07669947c7 Merge pull request #244 from morty/master 
scrapy genspider allows you to create a spider module with the same name as the project
 2013-02-12 18:11:49 -08:00
Tom Mortimer-Jones 2efd859525 Added check so that genspider cannot create a spider with the same name as the project. 2013-02-12 12:26:38 +00:00
Pablo Hoffman 1ff8b4f831 updated release notes with previous commit 2013-02-12 00:59:25 -02:00
Rolando Espinoza La fuente 5971333a93 added --pdb command option to enable pdb debugger on failure. 2013-02-11 12:34:29 -04:00
Pablo Hoffman d9043ffa6e reverted previous changed as it broke tests 2013-02-11 11:43:25 -02:00
Pablo Hoffman 2df010a972 Merge pull request #241 from zuhao/master 
Fix url_has_any_extension bug
 2013-02-11 05:29:14 -08:00
Zuhao Wan 27ca25472f Fix url_has_any_extension bug 2013-02-11 17:19:31 +08:00
Jordi Llonch 6d4b764f29 spider.log to pass keyword arguments into twisted log.msg 2013-02-11 05:00:06 +11:00
Daniel Graña c0a7040f16 add PHONYs so build does not match build/ path 2013-02-08 15:07:00 -02:00
Daniel Graña 910effd145 get scrapy version from package data 2013-02-06 11:44:26 -02:00
Daniel Graña 79cb0318d0 update travis-ci conf 2013-02-06 10:36:08 -02:00
Daniel Graña 5db45b3825 remove scrapyd, it was migrated to its own repository 2013-02-06 05:24:07 +00:00
Daniel Graña 33ca295129 Merge pull request #235 from whodatninja/patch-1 
Fix typo labeling attrs type bool instead of list
 2013-02-05 17:27:10 -08:00
whodatninja 8e3b5baac5 Fix typo labeling attrs type bool instead of list 2013-02-05 15:10:41 -05:00
Daniel Graña f293d08e7a Merge pull request #234 from darkrho/httpcache-dbm-module-fullpath 
allow to use full path in HTTPCACHE_DBM_MODULE setting
 2013-02-01 09:01:27 -08:00
Rolando Espinoza La fuente d00e6cf1d7 tests: dummydbm.error should be KeyError. 2013-02-01 12:52:59 -04:00
Rolando Espinoza La fuente 6ed44a3a67 httpcache: added tests for custom dbm module. 
And removed the break in the line to improve readability as there
is no strict 80-chars line width convention.
 2013-02-01 12:41:52 -04:00
Rolando Espinoza La fuente a2602df90b httpcache: allow to import submodules within packages as HTTPCACHE_DBM_MODULE setting. 2013-01-31 23:49:36 -04:00
Daniel Graña e5edb8ec3e Merge branch 'dangra/issue-24' 2013-01-30 16:55:53 -02:00
Daniel Graña 3af240f5f9 Merge branch 'dangra/issue-12' 2013-01-30 16:55:16 -02:00
Daniel Graña 0d3e4b4c43 do not unquote slash and question mark in url paths. fix #24 2013-01-30 11:22:03 -02:00
Daniel Graña 872a22df68 pep8ize sgml link extractors 2013-01-29 18:08:32 -02:00
Daniel Graña cc69b3aa4c Fix #199 encoding error concatenating link text 2013-01-29 17:56:34 -02:00
Daniel Graña 95fde0a498 register namespaces when XMLFeedSpider uses iternodes mode. fixes #12 2013-01-29 12:00:13 -02:00
Daniel Graña 8e77f27897 Find form nodes in invalid html5 documents 
lxml fails to parse invalid html5 documents
This error was reported in scrapy/loginform#3
 2013-01-24 17:47:53 -02:00
Daniel Graña ff04480675 fix exporting nested items as xml. fixes #66 2013-01-24 14:57:53 -02:00
Pablo Hoffman 71d7df9273 simplify InitSpider implementation, fixing one bug (closes #228) and adding support generators return from start_requests() (which the previous version didn't) 2013-01-23 16:28:41 -02:00
Daniel Graña 3cf7f4975b Add 0.16.4 to release notes 
Conflicts:
	docs/news.rst
 2013-01-23 11:29:38 -02:00
Daniel Graña c40f947dc1 Merge pull request #229 from christilden/master 
fixes spelling errors in documentation
 2013-01-22 20:27:40 -08:00
Chris Tilden aae6aed4fb fixes spelling errors in documentation 2013-01-22 14:52:18 -08:00
Pablo Hoffman 27583922a7 remove unused imports 2013-01-21 14:01:12 -02:00
Pablo Hoffman 65258e3621 Merge pull request #227 from tonal/correct-old-cache 
Correct init bag for load FilesystemCacheStorage from old location
 2013-01-21 04:57:19 -08:00
Alexandr N Zamaraev (aka tonal) 2c51266a40 Correct init bag for load old scrapy.contrib.httpcache.FilesystemCacheStorage 2013-01-21 13:54:36 +07:00
Pablo Hoffman 6ab8afb992 improve documentation about removing namespaces 2013-01-18 12:35:30 -02:00
Pablo Hoffman 1ba04b1fc3 added remove_namespaces() method to XmlXPathSelector objects 2013-01-18 12:20:03 -02:00
Pablo Hoffman b7eeeff410 get rid of assertDictEqual (since it's python 2.7+ only) 2013-01-17 13:18:23 -02:00
Pablo Hoffman c31441a273 revert default HTTP cache policy to dummy (instead of RFC2616) 2013-01-17 13:08:29 -02:00
Daniel Graña 897195186a document new FormRequest parameter named `formxpath` that matches forms using xpath 2013-01-08 18:36:20 -02:00
Daniel Graña 7527ef97ba Merge pull request #185 from notsobad/master 
Added xpath support in FormRequest.from_response
 2013-01-08 12:34:06 -08:00
Daniel Graña 75563b3f00 Add list of supported and missing RFC2616 caching features 2013-01-08 18:16:44 -02:00
Daniel Graña 3cbc4d0b94 django is an optional_features, its imports must not fail 2013-01-08 17:56:46 -02:00
Daniel Graña d8a760bf57 Merge branch 'http-cache-middleware' 
Conflicts:
	scrapy/contrib/downloadermiddleware/httpcache.py
	scrapy/contrib/httpcache.py
	scrapy/tests/test_downloadermiddleware_httpcache.py
 2013-01-08 17:34:48 -02:00
Daniel Graña 864a7aef87 More httpcache updates 
* Change default cache policy to RFC2616
* Update HttpCacheMiddleware documentation
* Move policies to scrapy.contrib.httpcache
* remove a lint error for .has_key() usage in DBM storage backend
 2013-01-08 17:26:32 -02:00
Daniel Graña 487299e068 TakeFirst doc says it returns first non-null/non-empty value, zero is a valid value. closes #59 2013-01-08 15:47:33 -02:00
Daniel Graña 672d09ea2e add meta-refresh changes to release notes 2013-01-08 12:30:36 -02:00
Daniel Graña 9527c5819a pep8ize settings 2013-01-08 11:48:36 -02:00
Daniel Graña defc4f89b5 update metarefresh settings 2013-01-08 11:41:19 -02:00