RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,029 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

114 Commits

Author SHA1 Message Date
Alexey Bezhan 210a0a6fe1 Fix some typos, whitespace and small errors in docs 2014-02-27 18:02:22 +00:00
Paul Tremberth 57f30bcb04 Docs: 4-space indent for final spider example 2014-02-01 23:34:55 +01:00
Rolando Espinoza 4255e12bc7 Updated the tutorial crawl output with latest output. 2014-01-23 18:18:56 -04:00
Rolando Espinoza 9aab9224cb Updated shell docs with the crawler reference and fixed the actual shell output. 
Also updated the shell example with a reproducible code example.
 2014-01-23 18:04:57 -04:00
Pablo Hoffman ed6fd4933f Merge pull request #524 from hobsonlane/master 
documentation code example corrections per pablohoffman
 2014-01-16 06:44:51 -08:00
Hobson Lane 85a80d0752 remove "for brevity's sake" line and correct "Torrent item" 
Torrent item -> TorrentItem class
 2014-01-15 17:29:23 -08:00
Hobson Lane a3db95985b another import name correction by pablo 2014-01-14 21:04:15 -08:00
Ferdy Rodriguez 807dd25324 fixed error on tor's name 2014-01-13 00:03:58 -06:00
Ferdy Rodriguez 8b9348cfaf Changed TOR Info as previous was removed from www.mininova.org 2014-01-12 23:46:04 -06:00
Hobson Lane 6ba0857a5c documentation code example correction corrections per pablohoffman 2014-01-10 10:37:27 -08:00
Mikhail Korobov a27d91f0a6 Rename BaseSpider to Spider. See GH-495. 2013-12-30 19:46:41 +06:00
RasPat1 ff21281b95 Note about selector class import 
This is the salient point of this code compared to the last example.  We have a selector now and this is how we use it.  Especially since the user has just come from the shell where the pre-instantiated selector is taken for granted.
 2013-12-15 13:46:42 -05:00
Pablo Hoffman f2741c413e fix method name in tutorial. closes GH-480 2013-12-02 13:24:12 -02:00
Daniel Graña 2df8156431 Drop Python 2.6 support 2013-10-29 13:44:00 -02:00
Pablo Hoffman e8ee449a2a Merge pull request #432 from darkrho/crawl-url 
Removed URL reference in crawl command and .tld suffix in docs for spider names
 2013-10-21 09:40:58 -07:00
Rolando Espinoza La fuente 34543c2b2e DOCS removed .tld suffix for spider names for the sake of consistency. 2013-10-19 23:03:20 -04:00
Daniel Graña 155ea08ea1 use `sel` name for Selector's instances in docs, internals and shell 2013-10-15 15:58:42 -02:00
Daniel Graña 1abb1af0c6 fix typos and wording on selector's introduction 2013-10-15 10:13:43 -02:00
Daniel Graña 4645f9e03c Updates docs to reflect unified selectors api 2013-10-14 16:31:20 -02:00
Pablo Hoffman e1683ddf9b fix doc typo 2013-10-09 17:24:12 -02:00
Pablo Hoffman b1d1a36a1e add note about enclosing urls with quotes when running from command-line. closes GH-384 2013-09-18 18:01:28 -03:00
Kumara Tharmalingam bbb0603091 Fixed directory location for dmoz_spider.py file 
It should be under 'tutorial/spiders' not 'dmoz/spiders'
 2013-09-15 21:55:52 -07:00
Daniel Graña 0400b18efa docs: list lxml as installation prerequisite 2013-09-09 12:44:26 -03:00
Hart c00c4d7148 correction to description of example XPath retrieval in overview doc 2013-08-03 17:08:58 -07:00
Juan M Uys 4de3aa4932 Update overview.rst 2013-04-08 14:13:15 +02:00
Pablo Hoffman 51c301b3a2 added link to python binary libs, for windows installation 2013-03-13 03:18:33 -03:00
Pablo Hoffman 22edc44c6c doc: remove links to diveintopython.org, which is no longer available. closes #246 2013-02-14 11:09:40 -02:00
Pablo Hoffman d6867f7984 removed sphinx warnings about duplicate reference names 'this page' 2012-09-13 15:26:45 -03:00
Pablo Hoffman f4a17ec272 removed references to Scrapy Snippets site 2012-09-03 22:19:15 -03:00
Pablo Hoffman 4a5f70278f minor tidy up to installation guide windows notes 2012-08-29 15:44:24 -03:00
Pablo Hoffman 098d892c03 simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users 2012-08-29 15:37:05 -03:00
Daniel Graña abcc8c9f63 Recommend pypi as single way to install on Windows 2012-08-06 10:21:13 -03:00
Valentin-Costel Hăloiu 00bfb37e79 Update master 2012-07-04 06:55:01 +03:00
Pablo Hoffman 2fb5e62c39 doc: update overview page to point to the genspider command. refs #107 2012-04-19 02:37:22 -03:00
Pablo Hoffman 4f28ffcb2c removed no longer needed dependency on simplejson 2012-04-10 16:01:36 -03:00
Pablo Hoffman 6e8edbd72e switched default selectors backend to lxml 2012-04-10 15:52:14 -03:00
Pablo Hoffman b6ae266546 Removed (very old and possibly broken) backwards compatibility support for Twisted 2.5 2012-03-15 00:28:24 -03:00
Pablo Hoffman e521da2e2f Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25 2012-03-01 08:18:12 -02:00
Pablo Hoffman 0be421fbf0 fixed reference to tutorial directory 2011-12-23 18:57:11 -02:00
Daniel Graña bcb31988f2 change tutorial to follow changes on dmoz site 2011-12-14 13:03:31 -02:00
Pablo Hoffman ade5efdc61 added -o option to scrapy crawl, a convenient shortcut for using feed exports 2011-10-22 20:53:49 -02:00
Pablo Hoffman 431441cb52 updated documentation to remove references to old issue tracker and mercurial repos 2011-09-25 13:06:24 -03:00
Pablo Hoffman 76af0cdd44 updated documentation and code to use -s instead of --set option 2011-09-01 14:35:37 -03:00
Pablo Hoffman a3697421c0 some minor updates to documentation 2011-08-11 09:19:59 -03:00
Pablo Hoffman 5da6ffb57b Automated merge with ssh://hg.scrapy.org:2222/scrapy-0.12 2011-08-11 09:11:19 -03:00
Pablo Hoffman bc2d2183e9 fixed import in doc 2011-08-11 09:11:08 -03:00
Pablo Hoffman c59340150f Added cached DNS resolver based on old caching resolver extension from scrapy.contrib.resolver. This new one is *not* an extension, it comes builtin and always enabled. 2011-07-27 03:45:15 -03:00
Pablo Hoffman 57c43fdce6 added SitemapSpider, with tests and doc 2011-06-15 11:54:34 -03:00
Pablo Hoffman 5bf733b6f6 Changed default representation of items to pretty-printed dicts. This improves 
default logging by making log more readable in the default case, for both Scraped and Dropped lines.

Projects can still customize how items are represented by overriding the item's __str__ method, as usual.
 2011-06-03 01:13:01 -03:00
Pablo Hoffman 951ba507f9 Removed support for default values in Scrapy items, which have proven confusing in the past 2011-05-19 21:42:46 -03:00