RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,947 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

527 Commits

Author SHA1 Message Date
Capi Etheriel 72b6c96d9a Running lucasdemarchi/codespell to fix typos in docs 2014-03-06 12:40:55 -03:00
Alexey Bezhan 210a0a6fe1 Fix some typos, whitespace and small errors in docs 2014-02-27 18:02:22 +00:00
Alexey Bezhan 2e59d77580 Clarify MapCompose documentation 2014-02-27 17:56:14 +00:00
Cash Costello c532e13dcd Added missing word in practices.rst 2014-02-27 13:08:36 -02:00
Pablo Hoffman 6d8b7b29ef remove unused setting: DOWNLOADER_DEBUG 2014-02-27 12:16:05 -02:00
Pablo Hoffman 822adb1d5f Merge pull request #602 from Blender3D/email 
Added a mimetype parameter to `MailSender.send`
 2014-02-20 12:05:14 -08:00
Capi Etheriel 1ec1100b27 fixes dynamic itemclass example usage of type() 2014-02-18 20:59:05 -03:00
Nikita Nikishin a676017fe4 Fixed #441. 2014-02-18 03:50:43 -05:00
Paul Tremberth 41765ca18d DupeFilter: add setting for verbose logging + stats counter for filtered requests 2014-02-17 13:42:42 +01:00
Rolando Espinoza 28f946b05f DOC Use pipelines module name instead of pipieline following default project files. 2014-02-15 11:01:26 -04:00
Daniel Graña 6ca49ce76a Merge pull request #594 from dangra/593-engineslots 
fix a reference to unexistent engine.slots.
 2014-02-14 09:39:30 -08:00
Daniel Graña b58285b635 fix a reference to unexistent engine.slots. closes #593 2014-02-14 15:31:05 -02:00
Nikolaos-Digenis Karagiannis 43a797e2f7 downloaderMW doc typo (spiderMW doc copy remnant) 2014-02-11 22:30:00 +02:00
tracicot b2f4b296df Correct typos 2014-02-10 11:46:23 -02:00
Rolando Espinoza a6279fe95b DOC Fixed HTTPCACHE_STORAGE typo in the default value which is now Filesystem instead Dbm. 2014-01-30 11:53:42 -04:00
Rolando Espinoza 9aab9224cb Updated shell docs with the crawler reference and fixed the actual shell output. 
Also updated the shell example with a reproducible code example.
 2014-01-23 18:04:57 -04:00
Daniel Graña 677afe7e54 show ubuntu setup instructions as literal code 2014-01-20 16:22:53 -02:00
Daniel Graña ebfb5b7096 replace warning about updating package lists by a note on package upgrade 2014-01-20 15:18:34 -02:00
Daniel Graña 52c3ff9190 fix apt-get line 2014-01-20 14:39:26 -02:00
Daniel Graña eb73ddd301 Update Ubuntu installation instructions 2014-01-20 14:15:01 -02:00
stray-leone 7f30a671c3 modify the version of scrapy ubuntu package 
latest version is 0.22.
with scrapy-0.18, tutorial project provides error
relative issue : https://github.com/scrapy/scrapy/issues/511
 2014-01-20 10:24:34 -02:00
Mikhail Korobov 7b7a1d8dfd Make Filesystem storage backend default again. See GH-500. 2014-01-17 04:32:08 +06:00
Mikhail Korobov b03fe04999 Rename AjaxCrawlableMiddleware to AjaxCrawlMiddleware 2014-01-16 23:09:37 +06:00
Pablo Hoffman ed6fd4933f Merge pull request #524 from hobsonlane/master 
documentation code example corrections per pablohoffman
 2014-01-16 06:44:51 -08:00
Pablo Hoffman 71ada5476e Merge pull request #472 from redapple/exslt 
Register EXSLT namespaces by default (resolves #470)
 2014-01-16 06:32:05 -08:00
Daniel Graña b9bb9bed6b Merge pull request #343 from kmike/ajax-crawlable 
[MRG] AjaxCrawlableMiddleware
 2014-01-16 05:07:39 -08:00
Paul Tremberth 827c0cf51f Rename "regexp" prefix to "re" 2014-01-15 15:00:25 +01:00
Paul Tremberth 88c8a523a7 Add warning in docs on performance when using EXSLT regexp functions 2014-01-15 12:52:10 +01:00
Paul Tremberth a3eba68aca Drop EXSLT strings and math extensions 2014-01-15 12:28:25 +01:00
Pablo Hoffman ea2f897b81 Merge pull request #502 from scrapy/doc-fixes 
DOWNLOAD_DELAY docs clarification
 2014-01-14 21:07:42 -08:00
Paul Tremberth 2cc26e6f56 Fix typo error 2014-01-14 13:09:18 +01:00
Paul Tremberth 29fc9f3466 Update selectors documentation and tests 2014-01-14 12:56:37 +01:00
Hobson Lane 6ba0857a5c documentation code example correction corrections per pablohoffman 2014-01-10 10:37:27 -08:00
malcolm m 962e5ef702 Clarify return value from extract_links 2014-01-05 14:42:48 -08:00
Yuri Prezument 060891c01c Remove unused import from code sample 
Item pipeline docs - removed unused import from code sample
 2014-01-03 15:44:17 +02:00
Mikhail Korobov a27d91f0a6 Rename BaseSpider to Spider. See GH-495. 2013-12-30 19:46:41 +06:00
Mikhail Korobov e713733edf minor fixes to scrapy shell docs 
* better IPython links;
* MDC link instead of w3schools;
* small formatting fixes;
* show quoted URL in example
 2013-12-30 10:27:39 +06:00
Mikhail Korobov 9a999daa2a DOWNLOAD_DELAY docs clarification: 
* delay is enforced per website, not per spider;
* document download_delay attribute (it was previously documented only in FAQ about 999 error codes);
* document how CONCURRENT_REQUESTS_PER_IP affects download delays.
 2013-12-28 06:30:34 +06:00
Pablo Hoffman e42e3743fe quick documentation for #475 2013-12-24 12:19:15 -02:00
Mikhail Korobov e0cebbfc8f add a remark about 1% 2013-12-20 23:12:37 +06:00
Mikhail Korobov 943a0bd264 AjaxCrawlableMiddleware in Broad Crawl docs 2013-12-19 01:01:26 +06:00
Mikhail Korobov a87b3bd1c8 AjaxCrawlableMiddleware 2013-12-19 00:06:47 +06:00
Pablo Hoffman 339861367e Merge pull request #425 from audiodude/master 
DownloaderMiddleware docs: Update process_request and minor cleanups.
 2013-11-25 10:33:35 -08:00
Paul Tremberth 14f5817d6b Modify ItemLoader to support XPath and CSS selectors 
Deprecate XPathItemLoader (now an alias to the new ItemLoader)
 2013-11-21 18:05:24 +01:00
Pablo Hoffman f87be371a2 better names for HANDLE_* settings, and added doc 2013-11-21 14:33:17 -02:00
Brian Lange e4c1d8d37d Elaborate on use of order numbers 2013-11-19 17:51:50 -06:00
Brian Lange b878f60b5a Add note to item-pipeline documentation explaining order in the ITEM_PIPELINES setting. 2013-11-19 16:12:54 -06:00
tntC4stl3 b51d5d81e4 duplicate 'use' in line 87 2013-11-15 13:56:44 +08:00
Daniel Graña 2df8156431 Drop Python 2.6 support 2013-10-29 13:44:00 -02:00
Pablo Hoffman 911c8082b0 simplified description of crawl command 2013-10-21 14:42:51 -02:00