RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,905 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

800 Commits

Author SHA1 Message Date
Paul Tremberth 57f30bcb04 Docs: 4-space indent for final spider example 2014-02-01 23:34:55 +01:00
Rolando Espinoza a6279fe95b DOC Fixed HTTPCACHE_STORAGE typo in the default value which is now Filesystem instead Dbm. 2014-01-30 11:53:42 -04:00
Rolando Espinoza 4255e12bc7 Updated the tutorial crawl output with latest output. 2014-01-23 18:18:56 -04:00
Rolando Espinoza 9aab9224cb Updated shell docs with the crawler reference and fixed the actual shell output. 
Also updated the shell example with a reproducible code example.
 2014-01-23 18:04:57 -04:00
Daniel Graña 677afe7e54 show ubuntu setup instructions as literal code 2014-01-20 16:22:53 -02:00
Daniel Graña ebfb5b7096 replace warning about updating package lists by a note on package upgrade 2014-01-20 15:18:34 -02:00
Daniel Graña 52c3ff9190 fix apt-get line 2014-01-20 14:39:26 -02:00
Daniel Graña eb73ddd301 Update Ubuntu installation instructions 2014-01-20 14:15:01 -02:00
stray-leone 7f30a671c3 modify the version of scrapy ubuntu package 
latest version is 0.22.
with scrapy-0.18, tutorial project provides error
relative issue : https://github.com/scrapy/scrapy/issues/511
 2014-01-20 10:24:34 -02:00
Daniel Graña 431f2d109f fix 0.22.0 release date 2014-01-17 17:54:01 -02:00
Mikhail Korobov dfd13f9941 fix typos in news.rst and remove (not released yet) header 2014-01-18 01:50:19 +06:00
Daniel Graña d1f1b074e1 Merge 0.22.0 release notes 2014-01-17 15:55:46 -02:00
Mikhail Korobov 7b7a1d8dfd Make Filesystem storage backend default again. See GH-500. 2014-01-17 04:32:08 +06:00
Mikhail Korobov b03fe04999 Rename AjaxCrawlableMiddleware to AjaxCrawlMiddleware 2014-01-16 23:09:37 +06:00
Pablo Hoffman ed6fd4933f Merge pull request #524 from hobsonlane/master 
documentation code example corrections per pablohoffman
 2014-01-16 06:44:51 -08:00
Pablo Hoffman 71ada5476e Merge pull request #472 from redapple/exslt 
Register EXSLT namespaces by default (resolves #470)
 2014-01-16 06:32:05 -08:00
Daniel Graña b9bb9bed6b Merge pull request #343 from kmike/ajax-crawlable 
[MRG] AjaxCrawlableMiddleware
 2014-01-16 05:07:39 -08:00
Hobson Lane 85a80d0752 remove "for brevity's sake" line and correct "Torrent item" 
Torrent item -> TorrentItem class
 2014-01-15 17:29:23 -08:00
Paul Tremberth 827c0cf51f Rename "regexp" prefix to "re" 2014-01-15 15:00:25 +01:00
Paul Tremberth 88c8a523a7 Add warning in docs on performance when using EXSLT regexp functions 2014-01-15 12:52:10 +01:00
Paul Tremberth a3eba68aca Drop EXSLT strings and math extensions 2014-01-15 12:28:25 +01:00
Pablo Hoffman ea2f897b81 Merge pull request #502 from scrapy/doc-fixes 
DOWNLOAD_DELAY docs clarification
 2014-01-14 21:07:42 -08:00
Hobson Lane a3db95985b another import name correction by pablo 2014-01-14 21:04:15 -08:00
Pablo Hoffman 3d308d7732 Merge pull request #517 from hoprocker/patch-1 
Clarify return value from extract_links
 2014-01-14 21:01:32 -08:00
Paul Tremberth 2cc26e6f56 Fix typo error 2014-01-14 13:09:18 +01:00
Paul Tremberth 29fc9f3466 Update selectors documentation and tests 2014-01-14 12:56:37 +01:00
Ferdy Rodriguez 807dd25324 fixed error on tor's name 2014-01-13 00:03:58 -06:00
Ferdy Rodriguez 8b9348cfaf Changed TOR Info as previous was removed from www.mininova.org 2014-01-12 23:46:04 -06:00
Hobson Lane 6ba0857a5c documentation code example correction corrections per pablohoffman 2014-01-10 10:37:27 -08:00
malcolm m 962e5ef702 Clarify return value from extract_links 2014-01-05 14:42:48 -08:00
Yuri Prezument 060891c01c Remove unused import from code sample 
Item pipeline docs - removed unused import from code sample
 2014-01-03 15:44:17 +02:00
Mikhail Korobov a27d91f0a6 Rename BaseSpider to Spider. See GH-495. 2013-12-30 19:46:41 +06:00
Mikhail Korobov e713733edf minor fixes to scrapy shell docs 
* better IPython links;
* MDC link instead of w3schools;
* small formatting fixes;
* show quoted URL in example
 2013-12-30 10:27:39 +06:00
Mikhail Korobov f18ac02987 remove duplicated link extractors link 
Check http://doc.scrapy.org/en/latest/topics/link-extractors.html - two menu items are highlighted at the left.
 2013-12-28 05:40:10 +05:00
Mikhail Korobov 9a999daa2a DOWNLOAD_DELAY docs clarification: 
* delay is enforced per website, not per spider;
* document download_delay attribute (it was previously documented only in FAQ about 999 error codes);
* document how CONCURRENT_REQUESTS_PER_IP affects download delays.
 2013-12-28 06:30:34 +06:00
Pablo Hoffman e42e3743fe quick documentation for #475 2013-12-24 12:19:15 -02:00
Mikhail Korobov e0cebbfc8f add a remark about 1% 2013-12-20 23:12:37 +06:00
Mikhail Korobov 943a0bd264 AjaxCrawlableMiddleware in Broad Crawl docs 2013-12-19 01:01:26 +06:00
Mikhail Korobov a87b3bd1c8 AjaxCrawlableMiddleware 2013-12-19 00:06:47 +06:00
RasPat1 ff21281b95 Note about selector class import 
This is the salient point of this code compared to the last example.  We have a selector now and this is how we use it.  Especially since the user has just come from the shell where the pre-instantiated selector is taken for granted.
 2013-12-15 13:46:42 -05:00
Daniel Graña 8a7c5b5d81 Add 0.20.2 release notes 
Conflicts:
	docs/news.rst
 2013-12-09 18:33:46 -02:00
Pablo Hoffman f2741c413e fix method name in tutorial. closes GH-480 2013-12-02 13:24:12 -02:00
Daniel Graña e34ffc0f42 Add 0.20.1 release notes 
Conflicts:
	docs/news.rst
 2013-11-28 16:25:57 -02:00
Pablo Hoffman 339861367e Merge pull request #425 from audiodude/master 
DownloaderMiddleware docs: Update process_request and minor cleanups.
 2013-11-25 10:33:35 -08:00
Paul Tremberth 14f5817d6b Modify ItemLoader to support XPath and CSS selectors 
Deprecate XPathItemLoader (now an alias to the new ItemLoader)
 2013-11-21 18:05:24 +01:00
Pablo Hoffman f87be371a2 better names for HANDLE_* settings, and added doc 2013-11-21 14:33:17 -02:00
Brian Lange e4c1d8d37d Elaborate on use of order numbers 2013-11-19 17:51:50 -06:00
Brian Lange b878f60b5a Add note to item-pipeline documentation explaining order in the ITEM_PIPELINES setting. 2013-11-19 16:12:54 -06:00
Pablo Hoffman afe6eaa2fe Merge pull request #460 from tntC4stl3/master 
duplicate 'use' in line 87
 2013-11-15 04:10:49 -08:00
tntC4stl3 b51d5d81e4 duplicate 'use' in line 87 2013-11-15 13:56:44 +08:00