Pawel Miech
07d1605586
[docs] warnings about windows + python 3 in faq and install
2016-06-17 13:28:51 +02:00
Matt O'Connell
0bb1288969
Typo fixes
2016-06-13 16:21:02 -04:00
Paul Tremberth
149c4cd41e
[DOC][Overview] Use idiomatic .extract_first()
2016-05-17 17:53:40 +02:00
nyov
5876b9aa30
Update documentation links
2016-03-03 16:28:33 +00:00
Rolando Espinoza
f8ae99d18f
DOC Removed pywin32 from install instructions as it's already declared as dependency.
2015-12-05 09:48:17 -04:00
Rolando Espinoza
cc8ddb68b0
Added installation notes about using Conda for Windows and other OSes.
2015-11-26 13:08:56 -04:00
Mikhail Korobov
600f27c39b
Merge pull request #1350 from scrapy/installation-docs-update
...
Updates in Installation and Contributing sections in the docs
2015-09-09 17:54:35 +05:00
Julia Medina
eb7b9d506f
Add note to ubuntu install section about debian compatibility
2015-09-08 18:49:44 -03:00
Robert Weindl
833efbfcbd
Update tutorial.rst
...
Add missing "scrapy." prefix.
2015-09-02 20:35:23 +05:00
Rick
aa88fd797a
Small grammatical change
2015-08-26 15:01:50 -03:00
Julia Medina
d3c3408cd4
Replace alternative OSX install workaround with virtualenv
2015-08-19 03:19:27 -03:00
Julia Medina
46ee1c6ee2
Reference Homebrew's homepage for installation instructions
2015-08-19 02:21:08 -03:00
Mikhail Lyundin
ea4199f8e1
Minor method name fix
2015-08-05 22:42:53 +03:00
Julia Medina
439b376d5b
Note in install docs about pip being already included in python>=2.7.9
2015-07-10 20:34:43 -03:00
Julia Medina
5723e6b457
Add non-python dependencies to Ubuntu install section in the docs
...
Closes #1314 and closes #1198 .
2015-07-10 20:34:43 -03:00
Julia Medina
290ebee090
Add OS X installation section to docs
...
Closes #1342 and (possibly) #1126
2015-07-10 20:34:43 -03:00
Nikola Pavlović
7bef61dbbd
Fix reference.
2015-07-10 02:35:27 +02:00
Marco DallaG
44c8ef2d47
DOC Bring Ubuntu and Archlinux outside of Windows subsection
...
In the installation guide, in the "Platform specific installation notes" section, Ubuntu and ArchLinux are currently subsections of Windows, which does not make sense imho.
This commit changes the section tree from:
Platform specific installation notes
- Windows
-- Archlinux
-- Ubuntu
To:
Platform specific installation notes
- Windows
- Ubuntu
- Archlinux
2015-06-24 11:44:48 +02:00
Daniel Graña
1195f906d6
Merge pull request #1218 from Curita/move-base-to-packages
...
Move base classes to their packages
2015-05-14 18:28:17 -03:00
Julia Medina
42c2c4b6ce
Wrong topic link in docs/intro/overview.rst
2015-05-09 05:23:12 -03:00
Julia Medina
d3f576a816
Move scrapy/spider.py to scrapy/spiders/__init__.py
2015-05-09 04:20:09 -03:00
Julia Medina
fc346cba4d
Move scrapy/contrib/spiders to scrapy/spiders
2015-04-29 21:27:19 -03:00
Pablo Hoffman
3d2b74a6ff
Merge pull request #1188 from eliasdorneles/favoring_web_scraping_over_screen_scraping
...
[MRG+1] Favoring web scraping over screen scraping in the descriptions
2015-04-29 16:49:43 -03:00
Elias Dorneles
3d3633f3d2
favoring web scraping over screen scraping in the descriptions
2015-04-25 11:20:20 -03:00
Julia Medina
7a92dae4c8
Change Scrapy log output through docs
2015-04-22 17:27:24 -03:00
Elias Dorneles
d63567531d
change data extraction in crawl example to be consistent with tutorial, removed statement implying mandatory usage of Item
2015-04-21 11:30:48 -03:00
Elias Dorneles
f7da69d116
fixing example CSS expr
2015-04-21 11:19:10 -03:00
Elias Dorneles
ff007afb9d
expanded crawling primer with examples, and applied other suggestions from the review
2015-04-21 10:57:44 -03:00
Elias Dorneles
595146e158
some improvements for Scrapy tutorial
2015-04-20 21:10:07 -03:00
Mikhail Korobov
1794a893f4
Merge pull request #1172 from bagratte/docs
...
minor corrections in documentation.
2015-04-19 21:41:29 +05:00
bagratte
beea9267a1
minor corrections in documentation.
2015-04-18 19:48:25 +04:00
rajathkumarmp
02629b5f7b
Added link to ipython in doc.
2015-04-18 13:00:34 +05:30
bagratte
8d339da4e5
add some minor stylistic and grammar corrections to tutorial.rst.
2015-04-17 20:55:02 +04:00
Elias Dorneles
18c5bc75c2
some more minor grammar fixes
2015-03-27 21:00:21 -03:00
Elias Dorneles
7135c4e694
some minor grammar fixes
2015-03-27 17:56:56 -03:00
Julia Medina
f4e241a018
Merge pull request #1106 from eliasdorneles/overview-page-improvements
...
[MRG+1] some improvements to overview page
2015-03-27 15:16:33 -03:00
Peter Bronez
475766c73a
Converted sel.xpath() calls to response.xpath() in Extracting the data
2015-03-26 15:34:30 -04:00
Elias Dorneles
4dcecc98f9
moved example data to a better place
2015-03-26 15:45:17 -03:00
Elias Dorneles
7402e27230
fix community link
2015-03-26 15:35:31 -03:00
Elias Dorneles
729861c864
fixing indentation
2015-03-26 15:31:42 -03:00
Elias Dorneles
13d0ecde77
addressing more review comments, to avoid ambiguity on desired reading flow
2015-03-26 15:26:16 -03:00
Elias Dorneles
76e3bf1250
addressing comments from the review plus further editing
2015-03-26 14:26:20 -03:00
Elias Dorneles
8f4a268f37
added bit about async requests, improved phrasing
2015-03-26 12:14:56 -03:00
Elias Dorneles
32423d4a33
some improvements to overview page
2015-03-25 19:27:52 -03:00
Shadab Zafar
5a58d64131
Fix some redirection links in documentation
...
Fixes #606
2015-03-18 19:41:26 -03:00
Daniel Graña
2c67bd6c57
pywin32 is required by Twisted. closes #937
...
see:
* http://twistedmatrix.com/trac/ticket/6032
* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2028
2014-11-05 23:05:54 -02:00
Daniel Graña
6cb8995731
Update install.rst
...
fixes #937
2014-11-05 22:50:52 -02:00
Daniel Graña
a9292cfab7
jsonrpc webservice moved to https://github.com/scrapy/scrapy-jsonrpc repository
2014-08-15 23:28:13 -03:00
Daniel Graña
0772201ac8
Update installation docs
2014-08-08 18:07:38 -03:00
Zbigniew Siciarz
0466e8cb7a
Fixed Python syntax in tutorial.
2014-07-04 10:38:01 +02:00
Daniel Graña
2ad8db6ae6
Merge pull request #761 from dangra/lxmlextractor
...
Promote LxmlLinkExtractor as LxmlExtractor
2014-06-25 15:07:02 -03:00
Daniel Graña
a9ecef5662
promote LxmlLinkExtractor as default in docs
2014-06-25 14:34:30 -03:00
Daniel Graña
5b2faf61c3
recognize jl extension as jsonlines exporter and update docs
2014-06-25 13:55:15 -03:00
Julia Medina
bdca06240c
Fix settings repr on the logs of the shell and tutorial docs topics
2014-06-10 11:26:50 -03:00
Carlos Rivera
946b854ddf
grammatical issue
2014-05-06 15:41:59 -05:00
Daniel Graña
1117687c47
update docs
2014-04-23 23:39:58 -03:00
Mikhail Korobov
2d3803672b
DOC use top-level shortcuts in docs
2014-04-15 01:09:35 +06:00
Julia Medina
80081054a2
Fix broken links in documentation
2014-04-09 18:57:52 -03:00
Alexey Bezhan
210a0a6fe1
Fix some typos, whitespace and small errors in docs
2014-02-27 18:02:22 +00:00
Paul Tremberth
57f30bcb04
Docs: 4-space indent for final spider example
2014-02-01 23:34:55 +01:00
Rolando Espinoza
4255e12bc7
Updated the tutorial crawl output with latest output.
2014-01-23 18:18:56 -04:00
Rolando Espinoza
9aab9224cb
Updated shell docs with the crawler reference and fixed the actual shell output.
...
Also updated the shell example with a reproducible code example.
2014-01-23 18:04:57 -04:00
Pablo Hoffman
ed6fd4933f
Merge pull request #524 from hobsonlane/master
...
documentation code example corrections per pablohoffman
2014-01-16 06:44:51 -08:00
Hobson Lane
85a80d0752
remove "for brevity's sake" line and correct "Torrent item"
...
Torrent item -> TorrentItem class
2014-01-15 17:29:23 -08:00
Hobson Lane
a3db95985b
another import name correction by pablo
2014-01-14 21:04:15 -08:00
Ferdy Rodriguez
807dd25324
fixed error on tor's name
2014-01-13 00:03:58 -06:00
Ferdy Rodriguez
8b9348cfaf
Changed TOR Info as previous was removed from www.mininova.org
2014-01-12 23:46:04 -06:00
Hobson Lane
6ba0857a5c
documentation code example correction corrections per pablohoffman
2014-01-10 10:37:27 -08:00
Mikhail Korobov
a27d91f0a6
Rename BaseSpider to Spider. See GH-495.
2013-12-30 19:46:41 +06:00
RasPat1
ff21281b95
Note about selector class import
...
This is the salient point of this code compared to the last example. We have a selector now and this is how we use it. Especially since the user has just come from the shell where the pre-instantiated selector is taken for granted.
2013-12-15 13:46:42 -05:00
Pablo Hoffman
f2741c413e
fix method name in tutorial. closes GH-480
2013-12-02 13:24:12 -02:00
Daniel Graña
2df8156431
Drop Python 2.6 support
2013-10-29 13:44:00 -02:00
Pablo Hoffman
e8ee449a2a
Merge pull request #432 from darkrho/crawl-url
...
Removed URL reference in crawl command and .tld suffix in docs for spider names
2013-10-21 09:40:58 -07:00
Rolando Espinoza La fuente
34543c2b2e
DOCS removed .tld suffix for spider names for the sake of consistency.
2013-10-19 23:03:20 -04:00
Daniel Graña
155ea08ea1
use `sel` name for Selector's instances in docs, internals and shell
2013-10-15 15:58:42 -02:00
Daniel Graña
1abb1af0c6
fix typos and wording on selector's introduction
2013-10-15 10:13:43 -02:00
Daniel Graña
4645f9e03c
Updates docs to reflect unified selectors api
2013-10-14 16:31:20 -02:00
Pablo Hoffman
e1683ddf9b
fix doc typo
2013-10-09 17:24:12 -02:00
Pablo Hoffman
b1d1a36a1e
add note about enclosing urls with quotes when running from command-line. closes GH-384
2013-09-18 18:01:28 -03:00
Kumara Tharmalingam
bbb0603091
Fixed directory location for dmoz_spider.py file
...
It should be under 'tutorial/spiders' not 'dmoz/spiders'
2013-09-15 21:55:52 -07:00
Daniel Graña
0400b18efa
docs: list lxml as installation prerequisite
2013-09-09 12:44:26 -03:00
Hart
c00c4d7148
correction to description of example XPath retrieval in overview doc
2013-08-03 17:08:58 -07:00
Juan M Uys
4de3aa4932
Update overview.rst
2013-04-08 14:13:15 +02:00
Pablo Hoffman
51c301b3a2
added link to python binary libs, for windows installation
2013-03-13 03:18:33 -03:00
Pablo Hoffman
22edc44c6c
doc: remove links to diveintopython.org, which is no longer available. closes #246
2013-02-14 11:09:40 -02:00
Pablo Hoffman
d6867f7984
removed sphinx warnings about duplicate reference names 'this page'
2012-09-13 15:26:45 -03:00
Pablo Hoffman
f4a17ec272
removed references to Scrapy Snippets site
2012-09-03 22:19:15 -03:00
Pablo Hoffman
4a5f70278f
minor tidy up to installation guide windows notes
2012-08-29 15:44:24 -03:00
Pablo Hoffman
098d892c03
simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users
2012-08-29 15:37:05 -03:00
Daniel Graña
abcc8c9f63
Recommend pypi as single way to install on Windows
2012-08-06 10:21:13 -03:00
Valentin-Costel Hăloiu
00bfb37e79
Update master
2012-07-04 06:55:01 +03:00
Pablo Hoffman
2fb5e62c39
doc: update overview page to point to the genspider command. refs #107
2012-04-19 02:37:22 -03:00
Pablo Hoffman
4f28ffcb2c
removed no longer needed dependency on simplejson
2012-04-10 16:01:36 -03:00
Pablo Hoffman
6e8edbd72e
switched default selectors backend to lxml
2012-04-10 15:52:14 -03:00
Pablo Hoffman
b6ae266546
Removed (very old and possibly broken) backwards compatibility support for Twisted 2.5
2012-03-15 00:28:24 -03:00
Pablo Hoffman
e521da2e2f
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
2012-03-01 08:18:12 -02:00
Pablo Hoffman
0be421fbf0
fixed reference to tutorial directory
2011-12-23 18:57:11 -02:00
Daniel Graña
bcb31988f2
change tutorial to follow changes on dmoz site
2011-12-14 13:03:31 -02:00
Pablo Hoffman
ade5efdc61
added -o option to scrapy crawl, a convenient shortcut for using feed exports
2011-10-22 20:53:49 -02:00
Pablo Hoffman
431441cb52
updated documentation to remove references to old issue tracker and mercurial repos
2011-09-25 13:06:24 -03:00
Pablo Hoffman
76af0cdd44
updated documentation and code to use -s instead of --set option
2011-09-01 14:35:37 -03:00
Pablo Hoffman
a3697421c0
some minor updates to documentation
2011-08-11 09:19:59 -03:00
Pablo Hoffman
5da6ffb57b
Automated merge with ssh://hg.scrapy.org:2222/scrapy-0.12
2011-08-11 09:11:19 -03:00
Pablo Hoffman
bc2d2183e9
fixed import in doc
2011-08-11 09:11:08 -03:00
Pablo Hoffman
c59340150f
Added cached DNS resolver based on old caching resolver extension from scrapy.contrib.resolver. This new one is *not* an extension, it comes builtin and always enabled.
2011-07-27 03:45:15 -03:00
Pablo Hoffman
57c43fdce6
added SitemapSpider, with tests and doc
2011-06-15 11:54:34 -03:00
Pablo Hoffman
5bf733b6f6
Changed default representation of items to pretty-printed dicts. This improves
...
default logging by making log more readable in the default case, for both Scraped and Dropped lines.
Projects can still customize how items are represented by overriding the item's __str__ method, as usual.
2011-06-03 01:13:01 -03:00
Pablo Hoffman
951ba507f9
Removed support for default values in Scrapy items, which have proven confusing in the past
2011-05-19 21:42:46 -03:00
Pablo Hoffman
503f302010
removed remaining references to scheduler middleware from doc, as it will be removed on next release
2011-05-18 19:48:48 -03:00
Pablo Hoffman
7f97259ba7
added w3lib to requirements, in installation guide
2011-05-01 11:14:57 -03:00
Pablo Hoffman
bb2b67c862
updated tutorial to use 'dmoz' as the name of the spider instead of 'dmoz.org', so that it's more similar to the dirbot example project
2011-04-28 09:31:57 -03:00
Pablo Hoffman
bf73002428
removed googledir example, replaced by dirbot project on github. updated docs accordingly
2011-04-28 02:28:39 -03:00
Pablo Hoffman
181d1c09ae
Fixed typo and code indentation in the doc. Closes #307 and #308
2011-02-09 11:19:46 -02:00
Pablo Hoffman
426b6fa100
docs/intro/install.rst: added -U flag to easy_install command
2010-11-22 13:50:19 -02:00
Pablo Hoffman
ac007802d6
Simplified installation guide, including lxml as alternative dependency to libxml2. Closes #280
2010-11-17 21:32:23 -02:00
Pablo Hoffman
5a5364d0c1
Updated documentation to point out that simplejson is now required if using Python 2.5, and to recommended switching to Python 2.6
2010-11-16 03:31:04 -02:00
Pablo Hoffman
5f65c26080
Some minor improvements to feature list in Scrapy at a Glance documentation page
2010-10-16 19:02:08 -02:00
Pablo Hoffman
b4fbc6c5fa
Updated Scrapy Tutorial to reference feed exports, instead a custom written pipeline, and extended item pipeline documentation to include a JSON writer.
2010-10-10 20:31:05 -02:00
Pablo Hoffman
f4accb6c7f
Updated dmoz xpaths of Scrapy tutorial
2010-10-07 18:22:01 -02:00
Pablo Hoffman
e3d67d74f7
docs/intro/overview.rst: add example of scraped data and introduce loaders
2010-09-06 10:04:00 -03:00
Pablo Hoffman
00d55fbbd1
Updated 'Scrapy at a glance' document replacing item pipeline example by a simpler usage of feed exports
2010-09-05 23:38:37 -03:00
Pablo Hoffman
e2ed27e4fd
Added documentation for Ubuntu packages. Refs #211
2010-08-23 21:28:32 -03:00
Pablo Hoffman
9aefa242d5
Applied documentation patch provided by Lucian Ursu ( closes #207 )
2010-08-21 01:26:35 -03:00
Pablo Hoffman
1d3b9e2ca8
Scrapy shell refactoring
2010-08-20 11:26:14 -03:00
Pablo Hoffman
7858244dca
Scrapy shell: moved python console starting code to scrapy.utils.console and get rid of noisy console banners
2010-08-20 01:33:02 -03:00
Pablo Hoffman
34554da201
Deprecated scrapy-ctl.py command in favour of simpler "scrapy" command. Closes #199 . Also updated documenation accordingly and added convenient scrapy.bat script for running from Windows.
...
--HG--
rename : debian/scrapy-ctl.1 => debian/scrapy.1
rename : docs/topics/scrapy-ctl.rst => docs/topics/cmdline.rst
2010-08-18 19:48:32 -03:00
Pablo Hoffman
e741a807d2
Added new Feed exports extension with documentation and storage tests. Closes #197 .
...
Also deprecated File export pipeline (to be removed in Scrapy 0.11).
Still need to add tests for FeedExport main extension code.
2010-08-17 14:27:48 -03:00
Pablo Hoffman
43d47e5d9b
Some improvements to Item Pipeline ( closes #195 ):
...
* Made Item Pipeline Manager a subclass of scrapy.middleware.MiddlewareManager
* Added open_spider/close_spider methods with support for returning deferreds from them
* Inverted the process_item() arguments to be more friendly with deferred
callbacks (backwards compatibility kept through arguments introspection)
* Updated documentation with new methods and process_item() arguments change
2010-08-12 10:48:37 -03:00
Ismael Carnales
e145ec686c
Replaced default spider manager (TwistedPluginSpiderManger) with a simpler one that doesn't depend on Twisted Plugins infrastructure.
2010-07-30 17:30:32 -03:00
Pablo Hoffman
9e37ec4230
fixed documentation typo ( closes #151 )
2010-07-13 19:03:02 -03:00
Pablo Hoffman
6a33d6c4d0
* Added Scrapy Web Service with documentation and tests.
...
* Marked Web Console as deprecated.
* Removed Web Console documentation to discourage its use.
2010-06-09 13:46:22 -03:00
Pablo Hoffman
81f6502e37
Automated merge with http://hg.scrapy.org/scrapy-0.8/
2010-04-24 18:22:13 -03:00
Pablo Hoffman
2121a30c74
added note about installing Zope.Interface in windows platforms
2010-04-24 18:19:52 -03:00
Rolando Espinoza La fuente
db5c3df679
SEP12 implementation
...
* Rename BaseSpider.domain_name to BaseSpider.name
This patch implements the domain_name to name change in BaseSpider class and
change all spider instantiations to use the new attribute.
* Add allowed_domains to spider
This patch implements the merging of spider.domain_name and
spider.extra_domain_names in spider.allowed_domains for offsite checking
purposes.
Note that spider.domain_name is not touched by this patch, only not used.
* Remove spider.domain_name references from scrapy.stats
* Rename domain_stats to spider_stats in MemoryStatsCollector
* Use ``spider`` instead of ``domain`` in SimpledbStatsCollector
* Rename domain_stats_history table to spider_data_history and rename domain
field to spider in MysqlStatsCollector
* Refactor genspider command
The new signature for genspider is: genspider [options] <domain_name>.
Genspider uses domain_name for spider name and for the module name.
* Remove spider.domain_name references
* Update crawl command signature <spider|url>
* docs: updated references to domain_name
* examples/experimental: use spider.name
* genspider: require <name> <domain>
* spidermanager: renamed crawl_domain to crawl_spider_name
* spiderctl: updated references of *domain* to spider
* added backward compatiblity with legacy spider's attributes
'domain_name' and 'extra_domain_names'
2010-04-01 18:27:22 -03:00
Pablo Hoffman
1dfc79b5d0
Automated merge with http://hg.scrapy.org/scrapy-0.8
2010-03-20 20:48:11 -03:00
Pablo Hoffman
99a876754c
Improved "What else?" section of "Scrapy at a glance" overview
2010-03-20 20:24:18 -03:00
Pablo Hoffman
264cd2e035
Automated merge with http://hg.scrapy.org/scrapy-0.8
2010-03-19 10:32:42 -03:00
Daniel Grana
17091902f3
Explicity say where to save item class in "Defining our item" section of tutorial
2010-03-12 14:12:49 -02:00
Rolando Espinoza La fuente
7235040936
merged upstream
2010-02-19 17:41:45 -04:00
Pablo Hoffman
48739ae60c
install.rst: added explanation about why libxml2 2.6.28 or above is required
2010-01-13 12:20:24 -02:00
Rolando Espinoza La fuente
1402da31c5
docs: fixed typos and updated code examples
2010-01-11 12:28:22 -04:00
Pablo Hoffman
d60412ce19
titlecased Scrapy easy_install and some fixes to sign_release.sh script
2009-12-13 14:23:31 -02:00
Pablo Hoffman
7728a23e99
Changed item pipeline API to pass spider references (instead of domain names) to process_item() method
2009-11-06 13:46:36 -02:00
Pablo Hoffman
9b5fef4f48
fixed typo in intro/install doc (thanks phaithful)
2009-10-28 09:34:31 -02:00
Pablo Hoffman
37d9e015bb
minor fix to tutorial
2009-10-07 20:15:49 -02:00
Pablo Hoffman
a0eec7eaf6
some typos fixes and updates to install doc
2009-09-29 09:44:02 -03:00
Ismael Carnales
1646482bef
reformatted installation guide
2009-09-29 08:41:34 -03:00
Ismael Carnales
5862ba7db7
modified doc to reflect the new spider callback return policy (lists not needed)
2009-09-22 11:25:40 -03:00
Pablo Hoffman
6e93872955
updated installation guide for using releases
2009-09-17 11:06:55 -03:00
Pablo Hoffman
8a074c9cb5
removed scrapy-admin.py command, and left only scrapy-ctl as the only scrapy command
2009-08-24 15:43:36 -03:00
Ismael Carnales
39540b188a
changed torrent in overview doc
2009-08-24 15:11:04 -03:00
Ismael Carnales
4b5aa30867
minor update to tutorial
2009-08-24 14:34:17 -03:00
Pablo Hoffman
9635a7839c
rearranged documentation into a better organization
...
--HG--
rename : docs/topics/index.rst => docs/index.rst
2009-08-21 21:49:54 -03:00
Ismael Carnales
c08d3aa9cc
updated tutorial to use new items api
2009-08-21 14:16:27 -03:00
Pablo Hoffman
33b53c59d5
moved scrapy.xpath to scrapy.selector
...
--HG--
rename : scrapy/xpath/__init__.py => scrapy/selector/__init__.py
rename : scrapy/xpath/document.py => scrapy/selector/document.py
rename : scrapy/xpath/factories.py => scrapy/selector/factories.py
2009-08-19 21:50:52 -03:00
Pablo Hoffman
e8504a054c
moved scrapy.newitem to scrapy.item and declared newitem api officially stable. updated docs and example project. deprecated old ScrapedItem
2009-08-19 21:39:58 -03:00
Ismael Carnales
48b40bd620
renamed x method of selectors to select
2009-08-17 15:58:06 -03:00
Pablo Hoffman
5aeab5b291
converted scrapy.item package to module
...
--HG--
rename : scrapy/item/models.py => scrapy/item.py
2009-08-12 21:31:50 -03:00
Pablo Hoffman
b296d4169e
minor doc update for making it more windows-friendly
2009-08-09 17:08:42 -03:00
Pablo Hoffman
38c3f7d0b4
Some changes to logging of scraped items:
...
1. "Scraped Item" log level changed to DEBUG
2. "Dropped Item" log level changed to WARNING
3. added "Passed Item" log message with INFO level
2009-07-23 11:49:48 -03:00
Pablo Hoffman
e43e28bf1d
minimal doc improvement
2009-07-23 09:12:49 -03:00
Ismael Carnales
6d24ae5920
added reference to working with relative xpaths in the tutorial
2009-07-23 09:05:14 -03:00
Pablo Hoffman
1125825996
doc: minor updates to tutorial
2009-07-15 22:10:00 -03:00
Pablo Hoffman
ae7333d598
added simplejson optional dependency to doc
2009-07-09 16:49:20 -03:00
Pablo Hoffman
0c4c153819
improved Scrapy documentation index for better usability
2009-07-01 09:51:57 -03:00
Pablo Hoffman
80cd534f92
removed redundant botname from log lines
2009-06-25 16:48:04 -03:00
Pablo Hoffman
b1dad251ae
Deprecated Common Downloader Middleware and added DefaultHeaders Downloader
...
Middleware
2009-05-25 14:41:06 -03:00
Pablo Hoffman
befd28eef4
docs/tutorial: added reminder about adding pipeline to ITEM_PIPELINES settings (thanks jamie)
2009-05-20 00:57:44 -03:00
Pablo Hoffman
04610a25dc
fixed bug in tutorial regarding csv writer pipeline, and other minor corrections
2009-05-19 03:07:08 -03:00
Daniel Grana
abfc52cd17
docs: modify install document to mercurial based installation instructions
2009-05-19 01:50:44 -03:00
Pablo Hoffman
86498abdf1
Sorted out Link Extractors organization by moving all them to
...
scrapy.contrib.linkextractors.
The most relevant being:
scrapy.link.extractors.RegexLinkExtractor
which was moved to:
scrapy.contrib.linkextractors.sgml.SgmlLinkExtractor
The old location still works but throws a deprecation warning. It will be
removed before the 0.7 release.
Documentation and tests were also updated.
Also, in this changeset, a new regex-based link extractor was added to
scrapy.contrib.linkextractors.regex.
--HG--
rename : scrapy/tests/sample_data/link_extractor/regex_linkextractor.html => scrapy/tests/sample_data/link_extractor/sgml_linkextractor.html
rename : scrapy/tests/test_link.py => scrapy/tests/test_contrib_linkextractors.py
2009-05-18 19:19:37 -03:00
Daniel Grana
ba69c29e37
mv scrapy/trunk to root as part of svn2hg migration
...
--HG--
rename : scrapy/trunk/AUTHORS => AUTHORS
rename : scrapy/trunk/INSTALL => INSTALL
rename : scrapy/trunk/LICENSE => LICENSE
rename : scrapy/trunk/README => README
rename : scrapy/trunk/bin/runtests.sh => bin/runtests.sh
rename : scrapy/trunk/docs/Makefile => docs/Makefile
rename : scrapy/trunk/docs/README => docs/README
rename : scrapy/trunk/docs/_ext/scrapydocs.py => docs/_ext/scrapydocs.py
rename : scrapy/trunk/docs/_static/items_adaptors-sample1.html => docs/_static/items_adaptors-sample1.html
rename : scrapy/trunk/docs/_static/scrapydoc.css => docs/_static/scrapydoc.css
rename : scrapy/trunk/docs/_static/selectors-sample1.html => docs/_static/selectors-sample1.html
rename : scrapy/trunk/docs/conf.py => docs/conf.py
rename : scrapy/trunk/docs/faq.rst => docs/faq.rst
rename : scrapy/trunk/docs/index.rst => docs/index.rst
rename : scrapy/trunk/docs/intro/index.rst => docs/intro/index.rst
rename : scrapy/trunk/docs/intro/install.rst => docs/intro/install.rst
rename : scrapy/trunk/docs/intro/overview.rst => docs/intro/overview.rst
rename : scrapy/trunk/docs/intro/tutorial.rst => docs/intro/tutorial.rst
rename : scrapy/trunk/docs/media/scrapy-architecture.dia => docs/media/scrapy-architecture.dia
rename : scrapy/trunk/docs/misc/api-stability.rst => docs/misc/api-stability.rst
rename : scrapy/trunk/docs/misc/index.rst => docs/misc/index.rst
rename : scrapy/trunk/docs/proposed/_images/scrapy_architecture.odg => docs/proposed/_images/scrapy_architecture.odg
rename : scrapy/trunk/docs/proposed/_images/scrapy_architecture.png => docs/proposed/_images/scrapy_architecture.png
rename : scrapy/trunk/docs/proposed/index.rst => docs/proposed/index.rst
rename : scrapy/trunk/docs/proposed/introduction.rst => docs/proposed/introduction.rst
rename : scrapy/trunk/docs/proposed/newitem.rst => docs/proposed/newitem.rst
rename : scrapy/trunk/docs/proposed/spiders.rst => docs/proposed/spiders.rst
rename : scrapy/trunk/docs/ref/downloader-middleware.rst => docs/ref/downloader-middleware.rst
rename : scrapy/trunk/docs/ref/email.rst => docs/ref/email.rst
rename : scrapy/trunk/docs/ref/exceptions.rst => docs/ref/exceptions.rst
rename : scrapy/trunk/docs/ref/extension-manager.rst => docs/ref/extension-manager.rst
rename : scrapy/trunk/docs/ref/extensions.rst => docs/ref/extensions.rst
rename : scrapy/trunk/docs/ref/index.rst => docs/ref/index.rst
rename : scrapy/trunk/docs/ref/link-extractors.rst => docs/ref/link-extractors.rst
rename : scrapy/trunk/docs/ref/logging.rst => docs/ref/logging.rst
rename : scrapy/trunk/docs/ref/request-response.rst => docs/ref/request-response.rst
rename : scrapy/trunk/docs/ref/selectors.rst => docs/ref/selectors.rst
rename : scrapy/trunk/docs/ref/settings.rst => docs/ref/settings.rst
rename : scrapy/trunk/docs/ref/signals.rst => docs/ref/signals.rst
rename : scrapy/trunk/docs/ref/spiders.rst => docs/ref/spiders.rst
rename : scrapy/trunk/docs/topics/_images/adaptors_diagram.png => docs/topics/_images/adaptors_diagram.png
rename : scrapy/trunk/docs/topics/_images/adaptors_diagram.svg => docs/topics/_images/adaptors_diagram.svg
rename : scrapy/trunk/docs/topics/_images/firebug1.png => docs/topics/_images/firebug1.png
rename : scrapy/trunk/docs/topics/_images/firebug2.png => docs/topics/_images/firebug2.png
rename : scrapy/trunk/docs/topics/_images/firebug3.png => docs/topics/_images/firebug3.png
rename : scrapy/trunk/docs/topics/_images/scrapy_architecture.odg => docs/topics/_images/scrapy_architecture.odg
rename : scrapy/trunk/docs/topics/_images/scrapy_architecture.png => docs/topics/_images/scrapy_architecture.png
rename : scrapy/trunk/docs/topics/adaptors.rst => docs/topics/adaptors.rst
rename : scrapy/trunk/docs/topics/architecture.rst => docs/topics/architecture.rst
rename : scrapy/trunk/docs/topics/downloader-middleware.rst => docs/topics/downloader-middleware.rst
rename : scrapy/trunk/docs/topics/extensions.rst => docs/topics/extensions.rst
rename : scrapy/trunk/docs/topics/firebug.rst => docs/topics/firebug.rst
rename : scrapy/trunk/docs/topics/firefox.rst => docs/topics/firefox.rst
rename : scrapy/trunk/docs/topics/index.rst => docs/topics/index.rst
rename : scrapy/trunk/docs/topics/item-pipeline.rst => docs/topics/item-pipeline.rst
rename : scrapy/trunk/docs/topics/items.rst => docs/topics/items.rst
rename : scrapy/trunk/docs/topics/link-extractors.rst => docs/topics/link-extractors.rst
rename : scrapy/trunk/docs/topics/robotstxt.rst => docs/topics/robotstxt.rst
rename : scrapy/trunk/docs/topics/selectors.rst => docs/topics/selectors.rst
rename : scrapy/trunk/docs/topics/settings.rst => docs/topics/settings.rst
rename : scrapy/trunk/docs/topics/shell.rst => docs/topics/shell.rst
rename : scrapy/trunk/docs/topics/spider-middleware.rst => docs/topics/spider-middleware.rst
rename : scrapy/trunk/docs/topics/spiders.rst => docs/topics/spiders.rst
rename : scrapy/trunk/docs/topics/stats.rst => docs/topics/stats.rst
rename : scrapy/trunk/docs/topics/webconsole.rst => docs/topics/webconsole.rst
rename : scrapy/trunk/examples/experimental/googledir/googledir/__init__.py => examples/experimental/googledir/googledir/__init__.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/items.py => examples/experimental/googledir/googledir/items.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/pipelines.py => examples/experimental/googledir/googledir/pipelines.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/settings.py => examples/experimental/googledir/googledir/settings.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/spiders/__init__.py => examples/experimental/googledir/googledir/spiders/__init__.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/spiders/google_directory.py => examples/experimental/googledir/googledir/spiders/google_directory.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_basic.tmpl => examples/experimental/googledir/googledir/templates/spider_basic.tmpl
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_crawl.tmpl => examples/experimental/googledir/googledir/templates/spider_crawl.tmpl
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_csvfeed.tmpl => examples/experimental/googledir/googledir/templates/spider_csvfeed.tmpl
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_xmlfeed.tmpl => examples/experimental/googledir/googledir/templates/spider_xmlfeed.tmpl
rename : scrapy/trunk/examples/experimental/googledir/scrapy-ctl.py => examples/experimental/googledir/scrapy-ctl.py
rename : scrapy/trunk/examples/googledir/googledir/__init__.py => examples/googledir/googledir/__init__.py
rename : scrapy/trunk/examples/googledir/googledir/items.py => examples/googledir/googledir/items.py
rename : scrapy/trunk/examples/googledir/googledir/pipelines.py => examples/googledir/googledir/pipelines.py
rename : scrapy/trunk/examples/googledir/googledir/settings.py => examples/googledir/googledir/settings.py
rename : scrapy/trunk/examples/googledir/googledir/spiders/__init__.py => examples/googledir/googledir/spiders/__init__.py
rename : scrapy/trunk/examples/googledir/googledir/spiders/google_directory.py => examples/googledir/googledir/spiders/google_directory.py
rename : scrapy/trunk/examples/googledir/scrapy-ctl.py => examples/googledir/scrapy-ctl.py
rename : scrapy/trunk/extras/sql/scraping.sql => extras/sql/scraping.sql
rename : scrapy/trunk/profiling/priorityqueue/pq_classes.py => profiling/priorityqueue/pq_classes.py
rename : scrapy/trunk/profiling/priorityqueue/run.py => profiling/priorityqueue/run.py
rename : scrapy/trunk/profiling/priorityqueue/test_cases.py => profiling/priorityqueue/test_cases.py
rename : scrapy/trunk/scrapy/__init__.py => scrapy/__init__.py
rename : scrapy/trunk/scrapy/bin/scrapy-admin.py => scrapy/bin/scrapy-admin.py
rename : scrapy/trunk/scrapy/command/__init__.py => scrapy/command/__init__.py
rename : scrapy/trunk/scrapy/command/cmdline.py => scrapy/command/cmdline.py
rename : scrapy/trunk/scrapy/command/commands/__init__.py => scrapy/command/commands/__init__.py
rename : scrapy/trunk/scrapy/command/commands/crawl.py => scrapy/command/commands/crawl.py
rename : scrapy/trunk/scrapy/command/commands/download.py => scrapy/command/commands/download.py
rename : scrapy/trunk/scrapy/command/commands/genspider.py => scrapy/command/commands/genspider.py
rename : scrapy/trunk/scrapy/command/commands/help.py => scrapy/command/commands/help.py
rename : scrapy/trunk/scrapy/command/commands/list.py => scrapy/command/commands/list.py
rename : scrapy/trunk/scrapy/command/commands/log.py => scrapy/command/commands/log.py
rename : scrapy/trunk/scrapy/command/commands/parse.py => scrapy/command/commands/parse.py
rename : scrapy/trunk/scrapy/command/commands/shell.py => scrapy/command/commands/shell.py
rename : scrapy/trunk/scrapy/command/commands/start.py => scrapy/command/commands/start.py
rename : scrapy/trunk/scrapy/command/commands/stats.py => scrapy/command/commands/stats.py
rename : scrapy/trunk/scrapy/command/models.py => scrapy/command/models.py
rename : scrapy/trunk/scrapy/conf/__init__.py => scrapy/conf/__init__.py
rename : scrapy/trunk/scrapy/conf/commands/__init__.py => scrapy/conf/commands/__init__.py
rename : scrapy/trunk/scrapy/conf/commands/crawl.py => scrapy/conf/commands/crawl.py
rename : scrapy/trunk/scrapy/conf/commands/help.py => scrapy/conf/commands/help.py
rename : scrapy/trunk/scrapy/conf/commands/list.py => scrapy/conf/commands/list.py
rename : scrapy/trunk/scrapy/conf/commands/log.py => scrapy/conf/commands/log.py
rename : scrapy/trunk/scrapy/conf/commands/scrape.py => scrapy/conf/commands/scrape.py
rename : scrapy/trunk/scrapy/conf/commands/shell.py => scrapy/conf/commands/shell.py
rename : scrapy/trunk/scrapy/conf/commands/stats.py => scrapy/conf/commands/stats.py
rename : scrapy/trunk/scrapy/conf/commands/test.py => scrapy/conf/commands/test.py
rename : scrapy/trunk/scrapy/conf/default_settings.py => scrapy/conf/default_settings.py
rename : scrapy/trunk/scrapy/contrib/__init__.py => scrapy/contrib/__init__.py
rename : scrapy/trunk/scrapy/contrib/aws.py => scrapy/contrib/aws.py
rename : scrapy/trunk/scrapy/contrib/closedomain.py => scrapy/contrib/closedomain.py
rename : scrapy/trunk/scrapy/contrib/cluster/__init__.py => scrapy/contrib/cluster/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/crawler/__init__.py => scrapy/contrib/cluster/crawler/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/crawler/manager.py => scrapy/contrib/cluster/crawler/manager.py
rename : scrapy/trunk/scrapy/contrib/cluster/hooks/__init__.py => scrapy/contrib/cluster/hooks/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/hooks/svn.py => scrapy/contrib/cluster/hooks/svn.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/__init__.py => scrapy/contrib/cluster/master/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/manager.py => scrapy/contrib/cluster/master/manager.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/web.py => scrapy/contrib/cluster/master/web.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/ws_api.txt => scrapy/contrib/cluster/master/ws_api.txt
rename : scrapy/trunk/scrapy/contrib/cluster/tools/scrapy-cluster-ctl.py => scrapy/contrib/cluster/tools/scrapy-cluster-ctl.py
rename : scrapy/trunk/scrapy/contrib/cluster/tools/test-worker.py => scrapy/contrib/cluster/tools/test-worker.py
rename : scrapy/trunk/scrapy/contrib/cluster/worker/__init__.py => scrapy/contrib/cluster/worker/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/worker/manager.py => scrapy/contrib/cluster/worker/manager.py
rename : scrapy/trunk/scrapy/contrib/codecs/__init__.py => scrapy/contrib/codecs/__init__.py
rename : scrapy/trunk/scrapy/contrib/codecs/x_mac_roman.py => scrapy/contrib/codecs/x_mac_roman.py
rename : scrapy/trunk/scrapy/contrib/debug.py => scrapy/contrib/debug.py
rename : scrapy/trunk/scrapy/contrib/delayedclosedomain.py => scrapy/contrib/delayedclosedomain.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/__init__.py => scrapy/contrib/downloadermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/cache.py => scrapy/contrib/downloadermiddleware/cache.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/common.py => scrapy/contrib/downloadermiddleware/common.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/cookies.py => scrapy/contrib/downloadermiddleware/cookies.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/debug.py => scrapy/contrib/downloadermiddleware/debug.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/errorpages.py => scrapy/contrib/downloadermiddleware/errorpages.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/httpauth.py => scrapy/contrib/downloadermiddleware/httpauth.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/httpcompression.py => scrapy/contrib/downloadermiddleware/httpcompression.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/redirect.py => scrapy/contrib/downloadermiddleware/redirect.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/retry.py => scrapy/contrib/downloadermiddleware/retry.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/robotstxt.py => scrapy/contrib/downloadermiddleware/robotstxt.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/stats.py => scrapy/contrib/downloadermiddleware/stats.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/useragent.py => scrapy/contrib/downloadermiddleware/useragent.py
rename : scrapy/trunk/scrapy/contrib/groupsettings.py => scrapy/contrib/groupsettings.py
rename : scrapy/trunk/scrapy/contrib/item/__init__.py => scrapy/contrib/item/__init__.py
rename : scrapy/trunk/scrapy/contrib/item/models.py => scrapy/contrib/item/models.py
rename : scrapy/trunk/scrapy/contrib/itemsampler.py => scrapy/contrib/itemsampler.py
rename : scrapy/trunk/scrapy/contrib/link_extractors.py => scrapy/contrib/link_extractors.py
rename : scrapy/trunk/scrapy/contrib/memdebug.py => scrapy/contrib/memdebug.py
rename : scrapy/trunk/scrapy/contrib/memusage.py => scrapy/contrib/memusage.py
rename : scrapy/trunk/scrapy/contrib/pipeline/__init__.py => scrapy/contrib/pipeline/__init__.py
rename : scrapy/trunk/scrapy/contrib/pipeline/images.py => scrapy/contrib/pipeline/images.py
rename : scrapy/trunk/scrapy/contrib/pipeline/media.py => scrapy/contrib/pipeline/media.py
rename : scrapy/trunk/scrapy/contrib/pipeline/s3images.py => scrapy/contrib/pipeline/s3images.py
rename : scrapy/trunk/scrapy/contrib/pipeline/show.py => scrapy/contrib/pipeline/show.py
rename : scrapy/trunk/scrapy/contrib/prioritizers.py => scrapy/contrib/prioritizers.py
rename : scrapy/trunk/scrapy/contrib/response/__init__.py => scrapy/contrib/response/__init__.py
rename : scrapy/trunk/scrapy/contrib/response/soup.py => scrapy/contrib/response/soup.py
rename : scrapy/trunk/scrapy/contrib/schedulermiddleware/__init__.py => scrapy/contrib/schedulermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib/schedulermiddleware/duplicatesfilter.py => scrapy/contrib/schedulermiddleware/duplicatesfilter.py
rename : scrapy/trunk/scrapy/contrib/spider/__init__.py => scrapy/contrib/spider/__init__.py
rename : scrapy/trunk/scrapy/contrib/spider/profiler.py => scrapy/contrib/spider/profiler.py
rename : scrapy/trunk/scrapy/contrib/spider/reloader.py => scrapy/contrib/spider/reloader.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/__init__.py => scrapy/contrib/spidermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/depth.py => scrapy/contrib/spidermiddleware/depth.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/duplicatesfilter.py => scrapy/contrib/spidermiddleware/duplicatesfilter.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/limit.py => scrapy/contrib/spidermiddleware/limit.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/offsite.py => scrapy/contrib/spidermiddleware/offsite.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/referer.py => scrapy/contrib/spidermiddleware/referer.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/restrict.py => scrapy/contrib/spidermiddleware/restrict.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/urlfilter.py => scrapy/contrib/spidermiddleware/urlfilter.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/urllength.py => scrapy/contrib/spidermiddleware/urllength.py
rename : scrapy/trunk/scrapy/contrib/spiders/__init__.py => scrapy/contrib/spiders/__init__.py
rename : scrapy/trunk/scrapy/contrib/spiders/crawl.py => scrapy/contrib/spiders/crawl.py
rename : scrapy/trunk/scrapy/contrib/spiders/feed.py => scrapy/contrib/spiders/feed.py
rename : scrapy/trunk/scrapy/contrib/spiders/generic.py => scrapy/contrib/spiders/generic.py
rename : scrapy/trunk/scrapy/contrib/web/__init__.py => scrapy/contrib/web/__init__.py
rename : scrapy/trunk/scrapy/contrib/web/http.py => scrapy/contrib/web/http.py
rename : scrapy/trunk/scrapy/contrib/web/json.py => scrapy/contrib/web/json.py
rename : scrapy/trunk/scrapy/contrib/web/service.py => scrapy/contrib/web/service.py
rename : scrapy/trunk/scrapy/contrib/web/site.py => scrapy/contrib/web/site.py
rename : scrapy/trunk/scrapy/contrib/web/stats.py => scrapy/contrib/web/stats.py
rename : scrapy/trunk/scrapy/contrib/webconsole/__init__.py => scrapy/contrib/webconsole/__init__.py
rename : scrapy/trunk/scrapy/contrib/webconsole/enginestatus.py => scrapy/contrib/webconsole/enginestatus.py
rename : scrapy/trunk/scrapy/contrib/webconsole/livestats.py => scrapy/contrib/webconsole/livestats.py
rename : scrapy/trunk/scrapy/contrib/webconsole/scheduler.py => scrapy/contrib/webconsole/scheduler.py
rename : scrapy/trunk/scrapy/contrib/webconsole/spiderctl.py => scrapy/contrib/webconsole/spiderctl.py
rename : scrapy/trunk/scrapy/contrib/webconsole/spiderstats.py => scrapy/contrib/webconsole/spiderstats.py
rename : scrapy/trunk/scrapy/contrib/webconsole/stats.py => scrapy/contrib/webconsole/stats.py
rename : scrapy/trunk/scrapy/contrib_exp/__init__.py => scrapy/contrib_exp/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/__init__.py => scrapy/contrib_exp/adaptors/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/date.py => scrapy/contrib_exp/adaptors/date.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/extraction.py => scrapy/contrib_exp/adaptors/extraction.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/markup.py => scrapy/contrib_exp/adaptors/markup.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/misc.py => scrapy/contrib_exp/adaptors/misc.py
rename : scrapy/trunk/scrapy/contrib_exp/downloadermiddleware/__init__.py => scrapy/contrib_exp/downloadermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/downloadermiddleware/decompression.py => scrapy/contrib_exp/downloadermiddleware/decompression.py
rename : scrapy/trunk/scrapy/contrib_exp/history/__init__.py => scrapy/contrib_exp/history/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/history/history.py => scrapy/contrib_exp/history/history.py
rename : scrapy/trunk/scrapy/contrib_exp/history/middleware.py => scrapy/contrib_exp/history/middleware.py
rename : scrapy/trunk/scrapy/contrib_exp/history/scheduler.py => scrapy/contrib_exp/history/scheduler.py
rename : scrapy/trunk/scrapy/contrib_exp/history/store.py => scrapy/contrib_exp/history/store.py
rename : scrapy/trunk/scrapy/contrib_exp/link/__init__.py => scrapy/contrib_exp/link/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/__init__.py => scrapy/contrib_exp/newitem/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/adaptors.py => scrapy/contrib_exp/newitem/adaptors.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/fields.py => scrapy/contrib_exp/newitem/fields.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/models.py => scrapy/contrib_exp/newitem/models.py
rename : scrapy/trunk/scrapy/contrib_exp/pipeline/shoveitem.py => scrapy/contrib_exp/pipeline/shoveitem.py
rename : scrapy/trunk/scrapy/core/__init__.py => scrapy/core/__init__.py
rename : scrapy/trunk/scrapy/core/downloader/__init__.py => scrapy/core/downloader/__init__.py
rename : scrapy/trunk/scrapy/core/downloader/dnscache.py => scrapy/core/downloader/dnscache.py
rename : scrapy/trunk/scrapy/core/downloader/handlers.py => scrapy/core/downloader/handlers.py
rename : scrapy/trunk/scrapy/core/downloader/manager.py => scrapy/core/downloader/manager.py
rename : scrapy/trunk/scrapy/core/downloader/middleware.py => scrapy/core/downloader/middleware.py
rename : scrapy/trunk/scrapy/core/downloader/responsetypes/__init__.py => scrapy/core/downloader/responsetypes/__init__.py
rename : scrapy/trunk/scrapy/core/downloader/responsetypes/mime.types => scrapy/core/downloader/responsetypes/mime.types
rename : scrapy/trunk/scrapy/core/downloader/webclient.py => scrapy/core/downloader/webclient.py
rename : scrapy/trunk/scrapy/core/engine.py => scrapy/core/engine.py
rename : scrapy/trunk/scrapy/core/exceptions.py => scrapy/core/exceptions.py
rename : scrapy/trunk/scrapy/core/manager.py => scrapy/core/manager.py
rename : scrapy/trunk/scrapy/core/prioritizers.py => scrapy/core/prioritizers.py
rename : scrapy/trunk/scrapy/core/scheduler/__init__.py => scrapy/core/scheduler/__init__.py
rename : scrapy/trunk/scrapy/core/scheduler/middleware.py => scrapy/core/scheduler/middleware.py
rename : scrapy/trunk/scrapy/core/scheduler/schedulers.py => scrapy/core/scheduler/schedulers.py
rename : scrapy/trunk/scrapy/core/scheduler/store.py => scrapy/core/scheduler/store.py
rename : scrapy/trunk/scrapy/core/signals.py => scrapy/core/signals.py
rename : scrapy/trunk/scrapy/dupefilter/__init__.py => scrapy/dupefilter/__init__.py
rename : scrapy/trunk/scrapy/extension/__init__.py => scrapy/extension/__init__.py
rename : scrapy/trunk/scrapy/fetcher/__init__.py => scrapy/fetcher/__init__.py
rename : scrapy/trunk/scrapy/http/__init__.py => scrapy/http/__init__.py
rename : scrapy/trunk/scrapy/http/cookies.py => scrapy/http/cookies.py
rename : scrapy/trunk/scrapy/http/headers.py => scrapy/http/headers.py
rename : scrapy/trunk/scrapy/http/request/__init__.py => scrapy/http/request/__init__.py
rename : scrapy/trunk/scrapy/http/request/form.py => scrapy/http/request/form.py
rename : scrapy/trunk/scrapy/http/request/rpc.py => scrapy/http/request/rpc.py
rename : scrapy/trunk/scrapy/http/response/__init__.py => scrapy/http/response/__init__.py
rename : scrapy/trunk/scrapy/http/response/html.py => scrapy/http/response/html.py
rename : scrapy/trunk/scrapy/http/response/text.py => scrapy/http/response/text.py
rename : scrapy/trunk/scrapy/http/response/xml.py => scrapy/http/response/xml.py
rename : scrapy/trunk/scrapy/http/url.py => scrapy/http/url.py
rename : scrapy/trunk/scrapy/item/__init__.py => scrapy/item/__init__.py
rename : scrapy/trunk/scrapy/item/adaptors.py => scrapy/item/adaptors.py
rename : scrapy/trunk/scrapy/item/models.py => scrapy/item/models.py
rename : scrapy/trunk/scrapy/item/pipeline.py => scrapy/item/pipeline.py
rename : scrapy/trunk/scrapy/link/__init__.py => scrapy/link/__init__.py
rename : scrapy/trunk/scrapy/link/extractors.py => scrapy/link/extractors.py
rename : scrapy/trunk/scrapy/log/__init__.py => scrapy/log/__init__.py
rename : scrapy/trunk/scrapy/mail/__init__.py => scrapy/mail/__init__.py
rename : scrapy/trunk/scrapy/management/__init__.py => scrapy/management/__init__.py
rename : scrapy/trunk/scrapy/management/telnet.py => scrapy/management/telnet.py
rename : scrapy/trunk/scrapy/management/web.py => scrapy/management/web.py
rename : scrapy/trunk/scrapy/patches/__init__.py => scrapy/patches/__init__.py
rename : scrapy/trunk/scrapy/patches/monkeypatches.py => scrapy/patches/monkeypatches.py
rename : scrapy/trunk/scrapy/spider/__init__.py => scrapy/spider/__init__.py
rename : scrapy/trunk/scrapy/spider/manager.py => scrapy/spider/manager.py
rename : scrapy/trunk/scrapy/spider/middleware.py => scrapy/spider/middleware.py
rename : scrapy/trunk/scrapy/spider/models.py => scrapy/spider/models.py
rename : scrapy/trunk/scrapy/stats/__init__.py => scrapy/stats/__init__.py
rename : scrapy/trunk/scrapy/stats/corestats.py => scrapy/stats/corestats.py
rename : scrapy/trunk/scrapy/stats/statscollector.py => scrapy/stats/statscollector.py
rename : scrapy/trunk/scrapy/store/__init__.py => scrapy/store/__init__.py
rename : scrapy/trunk/scrapy/store/db.py => scrapy/store/db.py
rename : scrapy/trunk/scrapy/templates/project/module/__init__.py => scrapy/templates/project/module/__init__.py
rename : scrapy/trunk/scrapy/templates/project/module/items.py.tmpl => scrapy/templates/project/module/items.py.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/pipelines.py.tmpl => scrapy/templates/project/module/pipelines.py.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/settings.py.tmpl => scrapy/templates/project/module/settings.py.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/spiders/__init__.py => scrapy/templates/project/module/spiders/__init__.py
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_basic.tmpl => scrapy/templates/project/module/templates/spider_basic.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_crawl.tmpl => scrapy/templates/project/module/templates/spider_crawl.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_csvfeed.tmpl => scrapy/templates/project/module/templates/spider_csvfeed.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_xmlfeed.tmpl => scrapy/templates/project/module/templates/spider_xmlfeed.tmpl
rename : scrapy/trunk/scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/root/scrapy-ctl.py
rename : scrapy/trunk/scrapy/tests/__init__.py => scrapy/tests/__init__.py
rename : scrapy/trunk/scrapy/tests/run.py => scrapy/tests/run.py
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-ascii.html => scrapy/tests/sample_data/adaptors/enc-ascii.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-cp1252.html => scrapy/tests/sample_data/adaptors/enc-cp1252.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-latin1.html => scrapy/tests/sample_data/adaptors/enc-latin1.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-utf8-meta-latin1.html => scrapy/tests/sample_data/adaptors/enc-utf8-meta-latin1.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-utf8.html => scrapy/tests/sample_data/adaptors/enc-utf8.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/extr_unquoted.xml => scrapy/tests/sample_data/adaptors/extr_unquoted.xml
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.tar => scrapy/tests/sample_data/compressed/feed-sample1.tar
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.xml => scrapy/tests/sample_data/compressed/feed-sample1.xml
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.xml.bz2 => scrapy/tests/sample_data/compressed/feed-sample1.xml.bz2
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.xml.gz => scrapy/tests/sample_data/compressed/feed-sample1.xml.gz
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.zip => scrapy/tests/sample_data/compressed/feed-sample1.zip
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/html-gzip.bin => scrapy/tests/sample_data/compressed/html-gzip.bin
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/html-rawdeflate.bin => scrapy/tests/sample_data/compressed/html-rawdeflate.bin
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/html-zlibdeflate.bin => scrapy/tests/sample_data/compressed/html-zlibdeflate.bin
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample1.xml => scrapy/tests/sample_data/feeds/feed-sample1.xml
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample2.xml => scrapy/tests/sample_data/feeds/feed-sample2.xml
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample3.csv => scrapy/tests/sample_data/feeds/feed-sample3.csv
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample4.csv => scrapy/tests/sample_data/feeds/feed-sample4.csv
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample5.csv => scrapy/tests/sample_data/feeds/feed-sample5.csv
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/image_linkextractor.html => scrapy/tests/sample_data/link_extractor/image_linkextractor.html
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/linkextractor_latin1.html => scrapy/tests/sample_data/link_extractor/linkextractor_latin1.html
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/linkextractor_noenc.html => scrapy/tests/sample_data/link_extractor/linkextractor_noenc.html
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/regex_linkextractor.html => scrapy/tests/sample_data/link_extractor/regex_linkextractor.html
rename : scrapy/trunk/scrapy/tests/sample_data/test_site/index.html => scrapy/tests/sample_data/test_site/index.html
rename : scrapy/trunk/scrapy/tests/sample_data/test_site/item1.html => scrapy/tests/sample_data/test_site/item1.html
rename : scrapy/trunk/scrapy/tests/sample_data/test_site/item2.html => scrapy/tests/sample_data/test_site/item2.html
rename : scrapy/trunk/scrapy/tests/test_adaptors.py => scrapy/tests/test_adaptors.py
rename : scrapy/trunk/scrapy/tests/test_aws.py => scrapy/tests/test_aws.py
rename : scrapy/trunk/scrapy/tests/test_c14nurls.py => scrapy/tests/test_c14nurls.py
rename : scrapy/trunk/scrapy/tests/test_contrib_response_soup.py => scrapy/tests/test_contrib_response_soup.py
rename : scrapy/trunk/scrapy/tests/test_dependencies.py => scrapy/tests/test_dependencies.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_cookies.py => scrapy/tests/test_downloadermiddleware_cookies.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_decompression.py => scrapy/tests/test_downloadermiddleware_decompression.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_httpcompression.py => scrapy/tests/test_downloadermiddleware_httpcompression.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_redirect.py => scrapy/tests/test_downloadermiddleware_redirect.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_retry.py => scrapy/tests/test_downloadermiddleware_retry.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_useragent.py => scrapy/tests/test_downloadermiddleware_useragent.py
rename : scrapy/trunk/scrapy/tests/test_dupefilter.py => scrapy/tests/test_dupefilter.py
rename : scrapy/trunk/scrapy/tests/test_engine.py => scrapy/tests/test_engine.py
rename : scrapy/trunk/scrapy/tests/test_http_cookies.py => scrapy/tests/test_http_cookies.py
rename : scrapy/trunk/scrapy/tests/test_http_headers.py => scrapy/tests/test_http_headers.py
rename : scrapy/trunk/scrapy/tests/test_http_request.py => scrapy/tests/test_http_request.py
rename : scrapy/trunk/scrapy/tests/test_http_response.py => scrapy/tests/test_http_response.py
rename : scrapy/trunk/scrapy/tests/test_http_url.py => scrapy/tests/test_http_url.py
rename : scrapy/trunk/scrapy/tests/test_item.py => scrapy/tests/test_item.py
rename : scrapy/trunk/scrapy/tests/test_itemadaptor.py => scrapy/tests/test_itemadaptor.py
rename : scrapy/trunk/scrapy/tests/test_libxml2.py => scrapy/tests/test_libxml2.py
rename : scrapy/trunk/scrapy/tests/test_link.py => scrapy/tests/test_link.py
rename : scrapy/trunk/scrapy/tests/test_newitem.py => scrapy/tests/test_newitem.py
rename : scrapy/trunk/scrapy/tests/test_pipeline_images.py => scrapy/tests/test_pipeline_images.py
rename : scrapy/trunk/scrapy/tests/test_responsetypes.py => scrapy/tests/test_responsetypes.py
rename : scrapy/trunk/scrapy/tests/test_robustscrapeditem.py => scrapy/tests/test_robustscrapeditem.py
rename : scrapy/trunk/scrapy/tests/test_schedulermiddleware_duplicatesfilter.py => scrapy/tests/test_schedulermiddleware_duplicatesfilter.py
rename : scrapy/trunk/scrapy/tests/test_serialization.py => scrapy/tests/test_serialization.py
rename : scrapy/trunk/scrapy/tests/test_spidermiddleware_duplicatesfilter.py => scrapy/tests/test_spidermiddleware_duplicatesfilter.py
rename : scrapy/trunk/scrapy/tests/test_spidermonkey.py => scrapy/tests/test_spidermonkey.py
rename : scrapy/trunk/scrapy/tests/test_spiders/__init__.py => scrapy/tests/test_spiders/__init__.py
rename : scrapy/trunk/scrapy/tests/test_spiders/testspider.py => scrapy/tests/test_spiders/testspider.py
rename : scrapy/trunk/scrapy/tests/test_stats.py => scrapy/tests/test_stats.py
rename : scrapy/trunk/scrapy/tests/test_storedb.py => scrapy/tests/test_storedb.py
rename : scrapy/trunk/scrapy/tests/test_utils_datatypes.py => scrapy/tests/test_utils_datatypes.py
rename : scrapy/trunk/scrapy/tests/test_utils_defer.py => scrapy/tests/test_utils_defer.py
rename : scrapy/trunk/scrapy/tests/test_utils_iterators.py => scrapy/tests/test_utils_iterators.py
rename : scrapy/trunk/scrapy/tests/test_utils_markup.py => scrapy/tests/test_utils_markup.py
rename : scrapy/trunk/scrapy/tests/test_utils_misc.py => scrapy/tests/test_utils_misc.py
rename : scrapy/trunk/scrapy/tests/test_utils_python.py => scrapy/tests/test_utils_python.py
rename : scrapy/trunk/scrapy/tests/test_utils_request.py => scrapy/tests/test_utils_request.py
rename : scrapy/trunk/scrapy/tests/test_utils_response.py => scrapy/tests/test_utils_response.py
rename : scrapy/trunk/scrapy/tests/test_utils_url.py => scrapy/tests/test_utils_url.py
rename : scrapy/trunk/scrapy/tests/test_webclient.py => scrapy/tests/test_webclient.py
rename : scrapy/trunk/scrapy/tests/test_xpath.py => scrapy/tests/test_xpath.py
rename : scrapy/trunk/scrapy/tests/test_xpath_extension.py => scrapy/tests/test_xpath_extension.py
rename : scrapy/trunk/scrapy/utils/__init__.py => scrapy/utils/__init__.py
rename : scrapy/trunk/scrapy/utils/c14n.py => scrapy/utils/c14n.py
rename : scrapy/trunk/scrapy/utils/datatypes.py => scrapy/utils/datatypes.py
rename : scrapy/trunk/scrapy/utils/db.py => scrapy/utils/db.py
rename : scrapy/trunk/scrapy/utils/defer.py => scrapy/utils/defer.py
rename : scrapy/trunk/scrapy/utils/display.py => scrapy/utils/display.py
rename : scrapy/trunk/scrapy/utils/http.py => scrapy/utils/http.py
rename : scrapy/trunk/scrapy/utils/iterators.py => scrapy/utils/iterators.py
rename : scrapy/trunk/scrapy/utils/markup.py => scrapy/utils/markup.py
rename : scrapy/trunk/scrapy/utils/misc.py => scrapy/utils/misc.py
rename : scrapy/trunk/scrapy/utils/python.py => scrapy/utils/python.py
rename : scrapy/trunk/scrapy/utils/request.py => scrapy/utils/request.py
rename : scrapy/trunk/scrapy/utils/response.py => scrapy/utils/response.py
rename : scrapy/trunk/scrapy/utils/serialization.py => scrapy/utils/serialization.py
rename : scrapy/trunk/scrapy/utils/test.py => scrapy/utils/test.py
rename : scrapy/trunk/scrapy/utils/url.py => scrapy/utils/url.py
rename : scrapy/trunk/scrapy/xlib/BeautifulSoup.py => scrapy/xlib/BeautifulSoup.py
rename : scrapy/trunk/scrapy/xlib/ClientForm.py => scrapy/xlib/ClientForm.py
rename : scrapy/trunk/scrapy/xlib/__init__.py => scrapy/xlib/__init__.py
rename : scrapy/trunk/scrapy/xlib/lrucache.py => scrapy/xlib/lrucache.py
rename : scrapy/trunk/scrapy/xlib/lsprofcalltree.py => scrapy/xlib/lsprofcalltree.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/__init__.py => scrapy/xlib/pydispatch/__init__.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/dispatcher.py => scrapy/xlib/pydispatch/dispatcher.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/errors.py => scrapy/xlib/pydispatch/errors.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/license.txt => scrapy/xlib/pydispatch/license.txt
rename : scrapy/trunk/scrapy/xlib/pydispatch/robust.py => scrapy/xlib/pydispatch/robust.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/robustapply.py => scrapy/xlib/pydispatch/robustapply.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/saferef.py => scrapy/xlib/pydispatch/saferef.py
rename : scrapy/trunk/scrapy/xlib/spidermonkey/INSTALL.scrapy => scrapy/xlib/spidermonkey/INSTALL.scrapy
rename : scrapy/trunk/scrapy/xlib/spidermonkey/__init__.py => scrapy/xlib/spidermonkey/__init__.py
rename : scrapy/trunk/scrapy/xlib/spidermonkey/sm_settings.py => scrapy/xlib/spidermonkey/sm_settings.py
rename : scrapy/trunk/scrapy/xlib/spidermonkey/spidermonkey.py => scrapy/xlib/spidermonkey/spidermonkey.py
rename : scrapy/trunk/scrapy/xpath/__init__.py => scrapy/xpath/__init__.py
rename : scrapy/trunk/scrapy/xpath/constructors.py => scrapy/xpath/constructors.py
rename : scrapy/trunk/scrapy/xpath/document.py => scrapy/xpath/document.py
rename : scrapy/trunk/scrapy/xpath/extension.py => scrapy/xpath/extension.py
rename : scrapy/trunk/scrapy/xpath/selector.py => scrapy/xpath/selector.py
rename : scrapy/trunk/scrapy/xpath/types.py => scrapy/xpath/types.py
rename : scrapy/trunk/scripts/rpm-install.sh => scripts/rpm-install.sh
rename : scrapy/trunk/setup.cfg => setup.cfg
rename : scrapy/trunk/setup.py => setup.py
2009-05-06 15:55:17 -03:00