RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,903 Commits 49 Branches 132 Tags 187 MiB
Commit Graph

3903 Commits

Author SHA1 Message Date
Daniel Graña faac8557de Merge pull request #694 from kmike/0.22 
backport xml fixes to 0.22
 2014-04-17 12:06:54 -04:00
Claudio Salazar 99c8c209f9 Added resolve_entities to kwargs in SafeXMLParser 2014-04-17 18:18:49 +06:00
Claudio Salazar 912e4f7dbd Added test against XXE attacks for Sitemap 2014-04-17 18:17:12 +06:00
Claudio Salazar 259f66305b Fixed XML selector against XXE attacks 2014-04-17 18:15:03 +06:00
Claudio Salazar 9e0a6a877b Fixed XXE flaw in sitemap reader 2014-04-17 18:10:03 +06:00
Pablo Hoffman deda5a11d4 Merge pull request #617 from cash/patch-1 
Added missing word in practices.rst
 2014-02-27 07:03:57 -08:00
Cash Costello b24b16d87f Added missing word in practices.rst 2014-02-27 09:53:11 -05:00
Mikhail Korobov 91646846d2 Merge pull request #610 from je64u/patch-1 
Update item-pipeline.rst
 2014-02-23 18:29:01 +05:00
je64u b1d2b34a60 Update item-pipeline.rst 2014-02-22 23:50:11 -05:00
Daniel Graña 23582fa136 Add 0.22.2 release notes 2014-02-14 15:44:57 -02:00
Daniel Graña 8833a71ede bumped version to 0.22.2 2014-02-14 15:42:54 -02:00
Daniel Graña 13c099a104 fix a reference to unexistent engine.slots. closes #593 2014-02-14 15:41:15 -02:00
Nikolaos-Digenis Karagiannis 8ae11bf945 downloaderMW doc typo (spiderMW doc copy remnant) 2014-02-14 15:41:08 -02:00
Pablo Hoffman 5925f34e9b Merge pull request #587 from tracicot/patch-1 
Correct typos
 2014-02-10 05:41:36 -08:00
Daniel Graña 273343533b Add 0.22.1 release notes 2014-02-08 17:08:27 -02:00
tracicot 1346037ba5 Correct typos 2014-02-08 12:08:24 -07:00
Daniel Graña c85c3ceac3 bumped version to 0.22.1 2014-02-08 17:07:02 -02:00
Daniel Graña 2ec22793cb localhost666 can resolve under certain circumstances 
localhost666 is resolvable if the host running the tests contains a
"search mydomain.com" in /etc/resolv.conf and a wildcard dns entry
exists for *.mydomain.com
 2014-02-08 17:04:52 -02:00
Daniel Graña cc3eda36d8 test inspect.stack failure 2014-02-08 17:04:49 -02:00
Mikhail Korobov 8cb44f90ab Handle cases when inspect.stack() fails 2014-02-08 17:04:43 -02:00
Daniel Graña 46d98d625a Fix wrong checks on subclassing of deprecated classes. closes #581 2014-02-05 20:05:45 -02:00
Paul Tremberth 13846ded0f Docs: 4-space indent for final spider example 2014-02-05 20:05:31 -02:00
Paul Tremberth 368a9467aa Fix HtmlParserLinkExtractor and tests after #485 merge 2014-02-05 20:05:06 -02:00
Benoit Blanchon b566388122 BaseSgmlLinkExtractor: Fixed the missing space when the link has an inner tag 2014-02-05 20:03:59 -02:00
Benoit Blanchon c1cb418dee BaseSgmlLinkExtractor: Added unit test of a link with an inner tag 2014-02-05 20:03:57 -02:00
Benoit Blanchon 7e4d62778e BaseSgmlLinkExtractor: Fixed unknown_endtag() so that it only set current_link=None when the end tag match the opening tag 2014-02-05 20:03:52 -02:00
Paul Tremberth 76c7e200aa Fix tests for Travis-CI build 2014-02-05 20:01:53 -02:00
Daniel Graña 5f87b17047 replace unencodeable codepoints with html entities. fixes #562 and #285 2014-02-05 20:01:13 -02:00
Paul Tremberth d0ee545a8f RegexLinkExtractor: encode URL unicode value when creating Links 2014-02-05 20:01:00 -02:00
Rolando Espinoza 8da65deda1 Updated the tutorial crawl output with latest output. 2014-02-05 20:00:33 -02:00
Rolando Espinoza 875b9abe8a Updated shell docs with the crawler reference and fixed the actual shell output. 
Also updated the shell example with a reproducible code example.
 2014-02-05 20:00:28 -02:00
Rolando Espinoza f89efaf667 PEP8 minor edits. 2014-02-05 20:00:23 -02:00
Rolando Espinoza 5349cecd0e Expose current crawler in the scrapy shell. 2014-02-05 20:00:18 -02:00
Rolando Espinoza 387f414894 Unused re import and PEP8 minor edits. 2014-02-05 19:59:37 -02:00
Rolando Espinoza 06325467d8 Ignore None's values when using the ItemLoader. 2014-02-05 19:59:22 -02:00
Rolando Espinoza cde9a8c68d DOC Fixed HTTPCACHE_STORAGE typo in the default value which is now Filesystem instead Dbm. 2014-01-30 22:54:26 +06:00
Daniel Graña fb5c9c552f show ubuntu setup instructions as literal code 2014-01-20 16:23:12 -02:00
Daniel Graña 70fb1058ff Update Ubuntu installation instructions 2014-01-20 15:32:26 -02:00
Daniel Graña 6f70b6a196 Merge pull request #550 from stray-leone/patch-1 
modify the version of scrapy ubuntu package
 2014-01-20 04:08:39 -08:00
stray-leone 725900df44 modify the version of scrapy ubuntu package 
latest version is 0.22.
with scrapy-0.18, tutorial project provides error
relative issue : https://github.com/scrapy/scrapy/issues/511
 2014-01-20 12:32:09 +09:00
Daniel Graña af0219a561 fix 0.22.0 release date 2014-01-17 17:55:34 -02:00
Mikhail Korobov b7f58f47a8 fix typos in news.rst and remove (not released yet) header 2014-01-18 01:44:21 +06:00
Daniel Graña 8fee933c15 bumped version to 0.22.0 2014-01-17 15:49:39 -02:00
Mikhail Korobov 7b7a1d8dfd Make Filesystem storage backend default again. See GH-500. 2014-01-17 04:32:08 +06:00
Daniel Graña f0851e41ec Merge pull request #478 from redapple/offsitetests 
Add tests for OffsiteMiddleware() + use re.escape() in domains regexp
 2014-01-16 14:02:26 -08:00
Daniel Graña 151f9478c1 Merge pull request #538 from kmike/ajaxcrawlable-rename 
[MRG] Rename AjaxCrawlableMiddleware to AjaxCrawlMiddleware
 2014-01-16 10:33:46 -08:00
Mikhail Korobov b03fe04999 Rename AjaxCrawlableMiddleware to AjaxCrawlMiddleware 2014-01-16 23:09:37 +06:00
Pablo Hoffman ed6fd4933f Merge pull request #524 from hobsonlane/master 
documentation code example corrections per pablohoffman
 2014-01-16 06:44:51 -08:00
Pablo Hoffman 71ada5476e Merge pull request #472 from redapple/exslt 
Register EXSLT namespaces by default (resolves #470)
 2014-01-16 06:32:05 -08:00
Daniel Graña b9bb9bed6b Merge pull request #343 from kmike/ajax-crawlable 
[MRG] AjaxCrawlableMiddleware
 2014-01-16 05:07:39 -08:00