Commit Graph

2967 Commits

Author SHA1 Message Date
Pablo Hoffman 9dc7a3a373 some minor adjustments to logging doc 2009-08-29 18:10:28 -03:00
Pablo Hoffman 60cbf24c89 more cleanups to startproject and project templates
--HG--
rename : scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/scrapy-ctl.py
2009-08-29 04:29:47 -03:00
Pablo Hoffman c87216f02b doc: added missing :synopsis: to some modules 2009-08-29 03:46:58 -03:00
Pablo Hoffman 2f55d7932b replaced :ref: by :doc: links in doc index 2009-08-29 03:37:59 -03:00
Pablo Hoffman 924219dd5a - added reference documentation about scrapy-ctl.py script
- yet another refactor to cmdline module
- removed help command (use -h to help now)

--HG--
rename : docs/experimental/scripts.rst => docs/topics/scrapy-ctl.rst
2009-08-28 20:32:55 -03:00
Pablo Hoffman 609aed4553 added inspect_response() function for inspecting responses from spiders 2009-08-27 19:33:33 -03:00
Pablo Hoffman ea4f16bae6 refactored scrapy shell implementation, dropping IPython dependency, and adding a new 'view' shortcut 2009-08-27 18:24:08 -03:00
Ismael Carnales fb39bca24f added djangoitem doc 2009-08-26 08:44:22 -03:00
Pablo Hoffman 65ab611fb0 more updates to HttpErrorMiddleware doc 2009-08-26 08:30:18 -03:00
Pablo Hoffman 44783a3a06 minor improvements to FAQ entry 2009-08-26 00:18:58 -03:00
Pablo Hoffman 4dba75a73f updated HttpErrorMiddleware doc 2009-08-25 20:13:58 -03:00
Pablo Hoffman 8a074c9cb5 removed scrapy-admin.py command, and left only scrapy-ctl as the only scrapy command 2009-08-24 15:43:36 -03:00
Ismael Carnales 39540b188a changed torrent in overview doc 2009-08-24 15:11:04 -03:00
Ismael Carnales 4b5aa30867 minor update to tutorial 2009-08-24 14:34:17 -03:00
Pablo Hoffman 0363040884 doc: added FAQ entry about Accept-Language 2009-08-24 13:56:44 -03:00
Ismael Carnales 85282a4b76 added scrapy commandline scripts doc 2009-08-24 12:02:44 -03:00
Pablo Hoffman ead8cb9fac removed documentation about ugly DontCloseDomain exception (which will be removed in the future) 2009-08-24 11:57:21 -03:00
Pablo Hoffman 74f706b356 renamed "parse_item" method of XMLFeedSpider to "parse_node", keeping backwards compatibility 2009-08-24 10:54:34 -03:00
Pablo Hoffman 31693eb90f dropped "cache" attribute of Request and Response objects 2009-08-24 10:34:05 -03:00
Pablo Hoffman 0186c6937a HTTP auth middleware: added doc and unittest 2009-08-24 08:07:20 -03:00
Pablo Hoffman 46aaea9f32 doc: improved documentation about debugging leaks 2009-08-23 20:36:00 -03:00
Pablo Hoffman 0ed849248f some improvements to item exporters
- passed previous class attributes to instances attributes
- better handling of constructor arguments
- better coverage on unittets (including encoding)
- updated documentation with new changes
2009-08-23 05:48:35 -03:00
Pablo Hoffman 7461710ef1 added some missing dots 2009-08-21 21:54:10 -03:00
Pablo Hoffman 9635a7839c rearranged documentation into a better organization
--HG--
rename : docs/topics/index.rst => docs/index.rst
2009-08-21 21:49:54 -03:00
Pablo Hoffman 4761e0c88d minor doc correction 2009-08-21 19:11:59 -03:00
Pablo Hoffman 012fd1cc2f moved api-stability.rst doc to root and updated it
--HG--
rename : docs/misc/api-stability.rst => docs/api-stability.rst
2009-08-21 16:29:23 -03:00
Pablo Hoffman ef6f04eb06 moved doc about debugging memory leaks to its own topic and added doc about trackref module 2009-08-21 16:07:16 -03:00
Pablo Hoffman 4980eb4968 added titles to signals doc 2009-08-21 15:07:52 -03:00
Pablo Hoffman daea3f8a0c sphinx docs: replaced custom :exception: xref by standard :exc: 2009-08-21 15:05:06 -03:00
Ismael Carnales c08d3aa9cc updated tutorial to use new items api 2009-08-21 14:16:27 -03:00
Pablo Hoffman 37760eb13f improved consistency of logging settings to use LOG_* 2009-08-21 08:54:12 -03:00
Ismael Carnales f0689a9ee8 fixed error in link extractors doc, thanks tarasm 2009-08-21 08:34:53 -03:00
Pablo Hoffman 41f1972e47 removed unused TRACE log level and improved logging documentation 2009-08-20 18:17:48 -03:00
Pablo Hoffman 20e82335e2 rename some exporter methods and complete exporter tests refactoring 2009-08-20 14:09:24 -03:00
Pablo Hoffman 3152e66fec deprecate domain_open signal and handle stats domain open/close directly from the engine 2009-08-20 10:54:58 -03:00
Pablo Hoffman 33b53c59d5 moved scrapy.xpath to scrapy.selector
--HG--
rename : scrapy/xpath/__init__.py => scrapy/selector/__init__.py
rename : scrapy/xpath/document.py => scrapy/selector/document.py
rename : scrapy/xpath/factories.py => scrapy/selector/factories.py
2009-08-19 21:50:52 -03:00
Pablo Hoffman dcc90fc196 declared loaders api stable and updated example project to use them
--HG--
rename : docs/experimental/loaders.rst => docs/topics/loaders.rst
2009-08-19 21:39:59 -03:00
Pablo Hoffman e8504a054c moved scrapy.newitem to scrapy.item and declared newitem api officially stable. updated docs and example project. deprecated old ScrapedItem 2009-08-19 21:39:58 -03:00
Ismael Carnales 314e8dea43 added new item exporter tests, introduced some api changes 2009-08-19 19:05:33 -03:00
Pablo Hoffman 907fad6da8 make sure input processors always receive iterables as input 2009-08-19 16:49:49 -03:00
Pablo Hoffman 741a6d7863 item exporters refactoring
--HG--
rename : scrapy/contrib/exporter/jsonexporter.py => scrapy/contrib/exporter/jsonlines.py
2009-08-19 13:09:39 -03:00
Pablo Hoffman 82e4b6adcf merge with ismael repo 2009-08-18 15:38:20 -03:00
Pablo Hoffman e01d31c498 another improvement to doc navbar 2009-08-18 15:35:53 -03:00
Ismael Carnales be4226cec5 merge 2009-08-18 15:21:39 -03:00
Ismael Carnales f88fb27851 fixed error in xpath selectors doc 2009-08-18 15:18:49 -03:00
Ismael Carnales 67c0c6a9e4 corrected indentation in xpath selectors doc 2009-08-18 15:13:23 -03:00
Pablo Hoffman ff837e5a45 doc: improved top navbar 2009-08-18 15:12:44 -03:00
Ismael Carnales 428dfe0d4a corrected the style of spiders documentation 2009-08-18 15:06:33 -03:00
Pablo Hoffman 0192282d07 reorganized doc and moved robotstxt doc inside downloader middlewares doc 2009-08-18 14:36:18 -03:00
Ismael Carnales 33089d287d merged topics and reference doc 2009-08-18 14:05:15 -03:00
Pablo Hoffman 7f30461410 added support for defining EXTENSIONS setting using dicts, like middleware settings
--HG--
rename : scrapy/tests/test_utils_middleware.py => scrapy/tests/test_utils_conf.py
rename : scrapy/utils/middleware.py => scrapy/utils/conf.py
2009-08-18 11:05:36 -03:00
Ismael Carnales 1bbe7991dc added documentation for ImagesPipeline 2009-08-18 09:35:32 -03:00
Ismael Carnales 48b40bd620 renamed x method of selectors to select 2009-08-17 15:58:06 -03:00
Pablo Hoffman 59e0a83ad4 removed more obsolete adaptors code 2009-08-17 14:48:11 -03:00
Ismael Carnales 8c2f62ba9c added Item Exporters documentation 2009-08-14 09:16:29 -03:00
Pablo Hoffman 02b01c3d2b loaders doc: fixed outdated line 2009-08-13 23:24:24 -03:00
Daniel Grana 6a74b51371 remove dupes words in loaders doc, and unused import in example
--HG--
extra : rebase_source : ea7886725af7a54bd1031cb28f92efbdfe921d9e
2009-08-13 23:23:08 -03:00
Ismael Carnales ecca7f6478 fixes to Item Loader doc 2009-08-13 13:30:50 -03:00
Pablo Hoffman 8131d674d4 fixed outdated documentation (refs #97) 2009-08-13 09:24:41 -03:00
Pablo Hoffman 5aeab5b291 converted scrapy.item package to module
--HG--
rename : scrapy/item/models.py => scrapy/item.py
2009-08-12 21:31:50 -03:00
Pablo Hoffman 991afdf6b3 some minor fixes to loaders doc 2009-08-12 19:23:04 -03:00
Pablo Hoffman e6dd4d0955 renamed ApplyConcat processor to MapCompose 2009-08-12 18:43:08 -03:00
Pablo Hoffman 4e74f32401 renamed Pipe processor to Compose and documented it 2009-08-12 18:09:36 -03:00
Pablo Hoffman 958c3d9940 fixed some links to item loaders doc 2009-08-12 17:42:01 -03:00
Pablo Hoffman d94b633565 renamed ItemLoader method populate_item() to load_item() 2009-08-12 17:40:27 -03:00
Pablo Hoffman 1dc592882b Moved Item Loader to its final location in scrapy.contrib.loader, and updated doc/tests
--HG--
rename : docs/experimental/itemparser.rst => docs/experimental/loaders.rst
rename : scrapy/contrib/itemparser/__init__.py => scrapy/contrib/loader/__init__.py
rename : scrapy/contrib/itemparser/common.py => scrapy/contrib/loader/common.py
rename : scrapy/contrib/itemparser/parsers.py => scrapy/contrib/loader/processor.py
rename : scrapy/tests/test_itemparser.py => scrapy/tests/test_contrib_loader.py
2009-08-12 16:49:07 -03:00
Pablo Hoffman 7cbbc3ffb0 Renamed Loader to ItemParser (SEP-8 proposal). Documentation and unittests also updated.
--HG--
rename : docs/experimental/loaders.rst => docs/experimental/itemparser.rst
rename : scrapy/newitem/loader/__init__.py => scrapy/contrib/itemparser/__init__.py
rename : scrapy/newitem/loader/expanders.py => scrapy/contrib/itemparser/parsers.py
rename : scrapy/tests/test_itemloader.py => scrapy/tests/test_itemparser.py
2009-08-12 16:49:05 -03:00
Pablo Hoffman d4aa72d2bb removed obsolete documentation about Robust Scraped Item and Adaptors 2009-08-12 13:50:15 -03:00
Pablo Hoffman e69311ef38 added missing text to new stats collector methods 2009-08-11 16:37:52 -03:00
Daniel Grana 45cbe1333f fix typo in stats docs 2009-08-11 16:30:04 -03:00
Daniel Grana 8f16e7f9fa remove `default` parameter from max_value/min_value stats methods, update docs, and add new methods to dummy collector 2009-08-11 16:23:23 -03:00
Daniel Grana cd8eed77c5 stats collector gains two new methods to store values only if greater/lower than previous value 2009-08-11 15:46:44 -03:00
Ismael Carnales 3614998053 fixed error in doc 2009-08-11 09:23:45 -03:00
Pablo Hoffman 7ac7597950 removed obsolete scrapy.contrib.cluster 2009-08-10 21:02:46 -03:00
Pablo Hoffman c0532712f9 removed unnecesary response ResponseSoup extension, and replaced by a utils function with cache support 2009-08-10 21:02:24 -03:00
Pablo Hoffman d334c035c0 removed unnecesary ResponseLibxml2 extension and moved libxml2 document caching functionality to Libxml2Document using weak references 2009-08-10 20:52:43 -03:00
Pablo Hoffman 49bab4777b XPathSelector: added 're' argument to add_xpath method, exposed selector attribute 2009-08-10 19:42:20 -03:00
Pablo Hoffman 50fcc67278 improved reducers examples 2009-08-10 10:13:30 -03:00
Pablo Hoffman 5be0cee577 added TreeExpander example 2009-08-09 20:54:22 -03:00
Pablo Hoffman 38f82e3993 loaders doc: added information about expanders/reducers declaration precendece, and other minor improvements 2009-08-09 18:06:12 -03:00
Pablo Hoffman b296d4169e minor doc update for making it more windows-friendly 2009-08-09 17:08:42 -03:00
Pablo Hoffman 91eea82eef added XPathLoader for working with XPath Selectors more conveniently 2009-08-08 07:26:20 -03:00
Pablo Hoffman db90e26a8b renamed ItemLoader class to Loader 2009-08-07 14:39:30 -03:00
Pablo Hoffman e585c6cac4 relocated experimental newitems/loaders doc, and added example for extending fields metadata
--HG--
rename : docs/experimental/newitem-loader.rst => docs/experimental/loaders.rst
rename : docs/experimental/newitem.rst => docs/experimental/newitems.rst
2009-08-07 14:28:58 -03:00
Pablo Hoffman d95e99f585 Added documentation for Items and Loaders, removed obsolete Item Adaptors documentation
--HG--
rename : docs/experimental/topics/newitem/index.rst => docs/experimental/newitem.rst
2009-08-07 03:50:09 -03:00
Ismael Carnales 32894643a0 added ListField documentation, ordered field reference alphabetically 2009-08-03 15:00:04 -03:00
Pablo Hoffman c3427e075c added domain_stats parameter to stats_domain_closed signal 2009-07-31 16:36:35 -03:00
Pablo Hoffman 73172b244d added from_unicode_list() method to Field objects 2009-07-30 16:58:24 -03:00
Pablo Hoffman 8ec7c9e01c WEBCONSOLE_PORT setting now defaults to 6080 2009-07-29 18:59:34 -03:00
Pablo Hoffman fcc91901eb finished cleaning up closedomain documentation, and updated default settings 2009-07-27 15:42:35 -03:00
Pablo Hoffman 09ba6927d7 Some changes to CloseDomain extension:
- added support for closing by item passed count (CLOSEDOMAIN_ITEMPASSED)
- removed support for sending notification emails (since that's the job of
  another extension)
2009-07-27 15:23:50 -03:00
Pablo Hoffman 9da66698f3 moved httprepr() method (from Request and Response objects) to scrapy.utils functions 2009-07-25 18:56:12 -03:00
Pablo Hoffman c615ace6bd doc: updated google directory links in firebug guide 2009-07-24 13:14:36 -03:00
Daniel Grana 96c3cdbec2 improve OffsiteMiddleware reference docs
--HG--
extra : rebase_source : 3ed3f23fc1ec63b521ead029c5749898f3ab05d7
2009-07-24 12:59:38 -03:00
Pablo Hoffman 38c3f7d0b4 Some changes to logging of scraped items:
1. "Scraped Item" log level changed to DEBUG
2. "Dropped Item" log level changed to WARNING
3. added "Passed Item" log message with INFO level
2009-07-23 11:49:48 -03:00
Pablo Hoffman e43e28bf1d minimal doc improvement 2009-07-23 09:12:49 -03:00
Ismael Carnales 6d24ae5920 added reference to working with relative xpaths in the tutorial 2009-07-23 09:05:14 -03:00
Pablo Hoffman 9baa6bb2a8 minor selectors doc fix 2009-07-23 01:56:56 -03:00
Ismael Carnales 3c4afb23be moved newitem from scrapy.contrib_exp to scrapy.newitem 2009-07-22 15:13:36 -03:00
Ismael Carnales 202894dd8f fixes to newitem doc 2009-07-22 10:25:22 -03:00
Pablo Hoffman 7d8ba0542c fixed typo in doc 2009-07-21 17:38:46 -03:00
Pablo Hoffman aa345e116a Added spider middleware documentation 2009-07-21 17:19:19 -03:00
Ismael Carnales 90f1d9e489 Added Scheduler middleware reference documentation 2009-07-21 16:52:27 -03:00
Ismael Carnales 00d9ce6608 fixes to newitem doc 2009-07-21 12:22:49 -03:00
Ismael Carnales 894a5d81d3 changed the newitem API to a dict-like interface 2009-07-21 12:20:49 -03:00
Pablo Hoffman 9867e7b2d6 removed references to obsolete ENABLED_SPIDERS_FILE setting 2009-07-21 01:11:48 -03:00
Pablo Hoffman 92b746d866 SimpledbStatsCollector: moved domain creation to constructor 2009-07-17 12:49:53 -03:00
Pablo Hoffman 292757f312 added link to architecture overview and fixed old link 2009-07-16 19:15:19 -03:00
Pablo Hoffman 136014d718 Added section about relative xpaths to XPathSelectors doc 2009-07-16 17:29:29 -03:00
Pablo Hoffman 0fd603e27e Automated merge with http://hg.scrapy.org/users/ismael/scrapy-newitem/ 2009-07-16 11:16:46 -03:00
Pablo Hoffman c2f12327d6 added dumping of global scrapy stats at engine shutdown 2009-07-16 10:48:10 -03:00
Ismael Carnales a9069e8eaa reorganized experimental doc in topics and ref, slitted long documents, introduced minor changes 2009-07-16 09:58:44 -03:00
Pablo Hoffman 1125825996 doc: minor updates to tutorial 2009-07-15 22:10:00 -03:00
Pablo Hoffman 5582305648 renamed old STATS_DEBUG setting to STATS_DUMP 2009-07-15 21:02:51 -03:00
Pablo Hoffman 7f281cf295 some minor updates to doc 2009-07-15 01:40:44 -03:00
Pablo Hoffman 8480f49cf6 Complete stats refactoring and documentation
--HG--
rename : scrapy/stats/statscollector.py => scrapy/stats/collector/__init__.py
2009-07-15 00:30:40 -03:00
Ismael Carnales 0227c53b13 reorganize newitem documentation 2009-07-14 15:55:52 -03:00
Pablo Hoffman 7fb889349e fixed link to experimental doc 2009-07-14 08:46:15 -03:00
Pablo Hoffman f1dc4e4af2 corrected TextField examples 2009-07-14 08:39:38 -03:00
Pablo Hoffman a8aae41a13 removed unused files 2009-07-13 22:16:43 -03:00
Pablo Hoffman 5c39d173a5 merged proposed and experimental documentation, as it didn't make sense to keep two separate sections
--HG--
rename : docs/proposed/_images/scrapy_architecture.odg => docs/experimental/_images/scrapy_architecture.odg
rename : docs/proposed/_images/scrapy_architecture.png => docs/experimental/_images/scrapy_architecture.png
rename : docs/proposed/index.rst => docs/experimental/index.rst
rename : docs/proposed/newitem-fields.rst => docs/experimental/newitem-fields.rst
rename : docs/proposed/newitem.rst => docs/experimental/newitem.rst
2009-07-13 22:15:54 -03:00
Pablo Hoffman 26bb8ef608 doc: improved newitem fields reference 2009-07-13 22:05:48 -03:00
Pablo Hoffman 2bf39b7cdb minor layout cleanups to newitem doc 2009-07-13 22:05:18 -03:00
Pablo Hoffman 74fcfc2cfc deprecated old adaptors documentation 2009-07-13 22:03:56 -03:00
Ismael Carnales 47d937f36b only accept unicode strings in text fields 2009-07-13 15:54:48 -03:00
Ismael Carnales d75afaa161 renamed StringField to TextField 2009-07-13 15:54:46 -03:00
Pablo Hoffman dff510384b doc: updated SCHEDULER_MIDDLEWARES_BASE setting 2009-07-13 13:33:47 -03:00
Ismael Carnales b44409a203 added TimeField to newitem 2009-07-13 10:31:32 -03:00
Pablo Hoffman 5054b67a02 improved newitems doc and marked robust scraped items as deprecated 2009-07-11 21:26:52 -03:00
Daniel Grana d5d2c5c924 update documentation to recent pydispatcher import path change 2009-07-09 17:13:30 -03:00
Daniel Grana 18fbd7c7eb Automated merge with ssh://hg.scrapy.org/scrapy 2009-07-09 16:58:07 -03:00
Daniel Grana eff8ea6173 remove response from item_passed and item_dropped signal api 2009-07-09 16:57:03 -03:00
Pablo Hoffman 5da32d9f6d fixed Sphinx warning 2009-07-09 16:50:13 -03:00
Pablo Hoffman ae7333d598 added simplejson optional dependency to doc 2009-07-09 16:49:20 -03:00
Pablo Hoffman 60e7b80798 removed signal docs from core.signals module, to leave them only in once place (the doc) 2009-07-09 12:57:10 -03:00
Ismael Carnales 9e3e41f946 added more newitem documentation in proposed 2009-07-09 11:29:04 -03:00
Pablo Hoffman b071681cd4 removed duplicated spiders doc (which used autodoc) 2009-07-09 11:14:33 -03:00
Pablo Hoffman 4f19115a80 removed old setting from default_settings.py, updated doc of CONCURRENT_ITEMS setting 2009-07-09 10:56:15 -03:00
Pablo Hoffman 8b26e49636 Added new ItemProcessor component to Scraper component 2009-07-08 23:48:06 -03:00
Pablo Hoffman 0c4c153819 improved Scrapy documentation index for better usability 2009-07-01 09:51:57 -03:00
Pablo Hoffman 80cd534f92 removed redundant botname from log lines 2009-06-25 16:48:04 -03:00
Pablo Hoffman 834ac9fca0 Some telnet console changes:
- added telnet console documentation
- added documentation for debugging memory leaks with guppy
- sorted out shell alises
- set default port (TELNETCONSOLE_PORT) to 6023
2009-06-23 16:08:58 -03:00
Pablo Hoffman a8fd107ad4 engine: some extra simplifications and removed debug mode 2009-06-22 23:01:41 -03:00
Pablo Hoffman 400a54bf7c Added reasons when closing domains ('reason' argument to engine close_domain method), replaced old 'status' parameter by new 'reason' parameter in domain_closed signal. updated and improved signals doc 2009-06-21 22:00:16 -03:00
Pablo Hoffman adccd9a04e Sorted out Duplicate Filter API.
--HG--
rename : scrapy/dupefilter/__init__.py => scrapy/contrib/dupefilter.py
2009-06-20 20:29:07 -03:00
Daniel Grana 47970e91bc core: Invert request priority meaning, a higher request.priority value means more priority 2009-06-20 19:23:26 -03:00
Daniel Grana 18b6fecc47 Remove custom redirection priority of request returned by downloadermiddleware 2009-06-20 19:19:07 -03:00
Pablo Hoffman 161335fe78 Added domain schedulers (whose functionality was previously mixed with the
Scrapy Scheduler) and removed domain prioritizers whose functionality became
duplicated by the new domain schedulers.
2009-06-19 17:55:54 -03:00
Pablo Hoffman 3cb12e5dd3 Moved init_domain functionality out of the engine (refs #88) and into the
spider level. A new spider (InitSpider, not yet documented) was added to
provide initialization facilities.

Also renamed make_request_from_url to make_requests_from_url and allowed it to
return iterables.
2009-06-18 14:43:56 -03:00
Pablo Hoffman fd0e490157 added StatsMailer extension 2009-06-12 15:38:21 -03:00
Pablo Hoffman 4a1a01354b Added 'priority' attribute to Requests and removed old 'priority' argument passed through engine, scheduler and scheduler middleware calls 2009-06-11 22:25:47 -03:00
Pablo Hoffman 635ac1ca64 Simplified domain prioritizers, so that they don't receive domains in the
constructor (domain prioritizers will be refactored later anyway) and
simplified Scrapy Manager code thanks to this.

Added make_request_from_url method to BaseSpider, splitting funtionality to
create requests from URLs which was previously done all in start_requests.
2009-06-10 14:21:36 -03:00
Pablo Hoffman 1aac694343 updated settings doc 2009-05-28 13:52:56 -03:00
Pablo Hoffman 04e7f8f5f6 merged with Daniel's HttpException-removal branch 2009-05-28 13:45:26 -03:00
Daniel Grana 7eaa3ed24d stop raising HttpException at download handlers and adapt download middlewares 2009-05-27 16:51:36 -03:00
Daniel Grana c8827552b6 fix typo at WEBCONSOLE_ENABLED setting documentaion of default value. thanks dzen 2009-05-26 15:48:34 -03:00
Pablo Hoffman 6d1ffa7137 renamed CrawlDebug downloader middleware to DebugMiddleware 2009-05-25 20:14:50 -03:00
Pablo Hoffman b1dad251ae Deprecated Common Downloader Middleware and added DefaultHeaders Downloader
Middleware
2009-05-25 14:41:06 -03:00
Pablo Hoffman 90d408b04f Some changes to HTTP cache middleware:
* documented
* moved from scrapy.contrib.downloadermiddleware.cache.CacheMiddleware to
  scrapy.contrib.downloadermiddleware.httpcache.HttpCacheMiddleware
* settings prefix changed from CACHE2_ to HTTPCACHE_

--HG--
rename : scrapy/contrib/downloadermiddleware/cache.py => scrapy/contrib/downloadermiddleware/httpcache.py
2009-05-24 19:13:06 -03:00
Pablo Hoffman befd28eef4 docs/tutorial: added reminder about adding pipeline to ITEM_PIPELINES settings (thanks jamie) 2009-05-20 00:57:44 -03:00
Pablo Hoffman 04610a25dc fixed bug in tutorial regarding csv writer pipeline, and other minor corrections 2009-05-19 03:07:08 -03:00
Daniel Grana abfc52cd17 docs: modify install document to mercurial based installation instructions 2009-05-19 01:50:44 -03:00
Pablo Hoffman 86498abdf1 Sorted out Link Extractors organization by moving all them to
scrapy.contrib.linkextractors.

The most relevant being:
    scrapy.link.extractors.RegexLinkExtractor

which was moved to:
    scrapy.contrib.linkextractors.sgml.SgmlLinkExtractor

The old location still works but throws a deprecation warning. It will be
removed before the 0.7 release.

Documentation and tests were also updated.

Also, in this changeset, a new regex-based link extractor was added to
scrapy.contrib.linkextractors.regex.

--HG--
rename : scrapy/tests/sample_data/link_extractor/regex_linkextractor.html => scrapy/tests/sample_data/link_extractor/sgml_linkextractor.html
rename : scrapy/tests/test_link.py => scrapy/tests/test_contrib_linkextractors.py
2009-05-18 19:19:37 -03:00
Pablo Hoffman 766c8a4ea8 fixed some doc typos reported by phaithful 2009-05-14 08:32:48 -03:00
Pablo Hoffman 314a1c2bc2 improved configuration of middlewares using dicts and orders (closes #85) 2009-05-11 01:40:40 -03:00
Pablo Hoffman c1c7b2d6c6 Sorted exceptions reference alphabetically
--HG--
extra : rebase_source : 1c6a192a76fcc90103ea324f6baf4387ba65e14a
2009-05-07 14:52:32 -03:00
Daniel Grana ba69c29e37 mv scrapy/trunk to root as part of svn2hg migration
--HG--
rename : scrapy/trunk/AUTHORS => AUTHORS
rename : scrapy/trunk/INSTALL => INSTALL
rename : scrapy/trunk/LICENSE => LICENSE
rename : scrapy/trunk/README => README
rename : scrapy/trunk/bin/runtests.sh => bin/runtests.sh
rename : scrapy/trunk/docs/Makefile => docs/Makefile
rename : scrapy/trunk/docs/README => docs/README
rename : scrapy/trunk/docs/_ext/scrapydocs.py => docs/_ext/scrapydocs.py
rename : scrapy/trunk/docs/_static/items_adaptors-sample1.html => docs/_static/items_adaptors-sample1.html
rename : scrapy/trunk/docs/_static/scrapydoc.css => docs/_static/scrapydoc.css
rename : scrapy/trunk/docs/_static/selectors-sample1.html => docs/_static/selectors-sample1.html
rename : scrapy/trunk/docs/conf.py => docs/conf.py
rename : scrapy/trunk/docs/faq.rst => docs/faq.rst
rename : scrapy/trunk/docs/index.rst => docs/index.rst
rename : scrapy/trunk/docs/intro/index.rst => docs/intro/index.rst
rename : scrapy/trunk/docs/intro/install.rst => docs/intro/install.rst
rename : scrapy/trunk/docs/intro/overview.rst => docs/intro/overview.rst
rename : scrapy/trunk/docs/intro/tutorial.rst => docs/intro/tutorial.rst
rename : scrapy/trunk/docs/media/scrapy-architecture.dia => docs/media/scrapy-architecture.dia
rename : scrapy/trunk/docs/misc/api-stability.rst => docs/misc/api-stability.rst
rename : scrapy/trunk/docs/misc/index.rst => docs/misc/index.rst
rename : scrapy/trunk/docs/proposed/_images/scrapy_architecture.odg => docs/proposed/_images/scrapy_architecture.odg
rename : scrapy/trunk/docs/proposed/_images/scrapy_architecture.png => docs/proposed/_images/scrapy_architecture.png
rename : scrapy/trunk/docs/proposed/index.rst => docs/proposed/index.rst
rename : scrapy/trunk/docs/proposed/introduction.rst => docs/proposed/introduction.rst
rename : scrapy/trunk/docs/proposed/newitem.rst => docs/proposed/newitem.rst
rename : scrapy/trunk/docs/proposed/spiders.rst => docs/proposed/spiders.rst
rename : scrapy/trunk/docs/ref/downloader-middleware.rst => docs/ref/downloader-middleware.rst
rename : scrapy/trunk/docs/ref/email.rst => docs/ref/email.rst
rename : scrapy/trunk/docs/ref/exceptions.rst => docs/ref/exceptions.rst
rename : scrapy/trunk/docs/ref/extension-manager.rst => docs/ref/extension-manager.rst
rename : scrapy/trunk/docs/ref/extensions.rst => docs/ref/extensions.rst
rename : scrapy/trunk/docs/ref/index.rst => docs/ref/index.rst
rename : scrapy/trunk/docs/ref/link-extractors.rst => docs/ref/link-extractors.rst
rename : scrapy/trunk/docs/ref/logging.rst => docs/ref/logging.rst
rename : scrapy/trunk/docs/ref/request-response.rst => docs/ref/request-response.rst
rename : scrapy/trunk/docs/ref/selectors.rst => docs/ref/selectors.rst
rename : scrapy/trunk/docs/ref/settings.rst => docs/ref/settings.rst
rename : scrapy/trunk/docs/ref/signals.rst => docs/ref/signals.rst
rename : scrapy/trunk/docs/ref/spiders.rst => docs/ref/spiders.rst
rename : scrapy/trunk/docs/topics/_images/adaptors_diagram.png => docs/topics/_images/adaptors_diagram.png
rename : scrapy/trunk/docs/topics/_images/adaptors_diagram.svg => docs/topics/_images/adaptors_diagram.svg
rename : scrapy/trunk/docs/topics/_images/firebug1.png => docs/topics/_images/firebug1.png
rename : scrapy/trunk/docs/topics/_images/firebug2.png => docs/topics/_images/firebug2.png
rename : scrapy/trunk/docs/topics/_images/firebug3.png => docs/topics/_images/firebug3.png
rename : scrapy/trunk/docs/topics/_images/scrapy_architecture.odg => docs/topics/_images/scrapy_architecture.odg
rename : scrapy/trunk/docs/topics/_images/scrapy_architecture.png => docs/topics/_images/scrapy_architecture.png
rename : scrapy/trunk/docs/topics/adaptors.rst => docs/topics/adaptors.rst
rename : scrapy/trunk/docs/topics/architecture.rst => docs/topics/architecture.rst
rename : scrapy/trunk/docs/topics/downloader-middleware.rst => docs/topics/downloader-middleware.rst
rename : scrapy/trunk/docs/topics/extensions.rst => docs/topics/extensions.rst
rename : scrapy/trunk/docs/topics/firebug.rst => docs/topics/firebug.rst
rename : scrapy/trunk/docs/topics/firefox.rst => docs/topics/firefox.rst
rename : scrapy/trunk/docs/topics/index.rst => docs/topics/index.rst
rename : scrapy/trunk/docs/topics/item-pipeline.rst => docs/topics/item-pipeline.rst
rename : scrapy/trunk/docs/topics/items.rst => docs/topics/items.rst
rename : scrapy/trunk/docs/topics/link-extractors.rst => docs/topics/link-extractors.rst
rename : scrapy/trunk/docs/topics/robotstxt.rst => docs/topics/robotstxt.rst
rename : scrapy/trunk/docs/topics/selectors.rst => docs/topics/selectors.rst
rename : scrapy/trunk/docs/topics/settings.rst => docs/topics/settings.rst
rename : scrapy/trunk/docs/topics/shell.rst => docs/topics/shell.rst
rename : scrapy/trunk/docs/topics/spider-middleware.rst => docs/topics/spider-middleware.rst
rename : scrapy/trunk/docs/topics/spiders.rst => docs/topics/spiders.rst
rename : scrapy/trunk/docs/topics/stats.rst => docs/topics/stats.rst
rename : scrapy/trunk/docs/topics/webconsole.rst => docs/topics/webconsole.rst
rename : scrapy/trunk/examples/experimental/googledir/googledir/__init__.py => examples/experimental/googledir/googledir/__init__.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/items.py => examples/experimental/googledir/googledir/items.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/pipelines.py => examples/experimental/googledir/googledir/pipelines.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/settings.py => examples/experimental/googledir/googledir/settings.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/spiders/__init__.py => examples/experimental/googledir/googledir/spiders/__init__.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/spiders/google_directory.py => examples/experimental/googledir/googledir/spiders/google_directory.py
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_basic.tmpl => examples/experimental/googledir/googledir/templates/spider_basic.tmpl
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_crawl.tmpl => examples/experimental/googledir/googledir/templates/spider_crawl.tmpl
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_csvfeed.tmpl => examples/experimental/googledir/googledir/templates/spider_csvfeed.tmpl
rename : scrapy/trunk/examples/experimental/googledir/googledir/templates/spider_xmlfeed.tmpl => examples/experimental/googledir/googledir/templates/spider_xmlfeed.tmpl
rename : scrapy/trunk/examples/experimental/googledir/scrapy-ctl.py => examples/experimental/googledir/scrapy-ctl.py
rename : scrapy/trunk/examples/googledir/googledir/__init__.py => examples/googledir/googledir/__init__.py
rename : scrapy/trunk/examples/googledir/googledir/items.py => examples/googledir/googledir/items.py
rename : scrapy/trunk/examples/googledir/googledir/pipelines.py => examples/googledir/googledir/pipelines.py
rename : scrapy/trunk/examples/googledir/googledir/settings.py => examples/googledir/googledir/settings.py
rename : scrapy/trunk/examples/googledir/googledir/spiders/__init__.py => examples/googledir/googledir/spiders/__init__.py
rename : scrapy/trunk/examples/googledir/googledir/spiders/google_directory.py => examples/googledir/googledir/spiders/google_directory.py
rename : scrapy/trunk/examples/googledir/scrapy-ctl.py => examples/googledir/scrapy-ctl.py
rename : scrapy/trunk/extras/sql/scraping.sql => extras/sql/scraping.sql
rename : scrapy/trunk/profiling/priorityqueue/pq_classes.py => profiling/priorityqueue/pq_classes.py
rename : scrapy/trunk/profiling/priorityqueue/run.py => profiling/priorityqueue/run.py
rename : scrapy/trunk/profiling/priorityqueue/test_cases.py => profiling/priorityqueue/test_cases.py
rename : scrapy/trunk/scrapy/__init__.py => scrapy/__init__.py
rename : scrapy/trunk/scrapy/bin/scrapy-admin.py => scrapy/bin/scrapy-admin.py
rename : scrapy/trunk/scrapy/command/__init__.py => scrapy/command/__init__.py
rename : scrapy/trunk/scrapy/command/cmdline.py => scrapy/command/cmdline.py
rename : scrapy/trunk/scrapy/command/commands/__init__.py => scrapy/command/commands/__init__.py
rename : scrapy/trunk/scrapy/command/commands/crawl.py => scrapy/command/commands/crawl.py
rename : scrapy/trunk/scrapy/command/commands/download.py => scrapy/command/commands/download.py
rename : scrapy/trunk/scrapy/command/commands/genspider.py => scrapy/command/commands/genspider.py
rename : scrapy/trunk/scrapy/command/commands/help.py => scrapy/command/commands/help.py
rename : scrapy/trunk/scrapy/command/commands/list.py => scrapy/command/commands/list.py
rename : scrapy/trunk/scrapy/command/commands/log.py => scrapy/command/commands/log.py
rename : scrapy/trunk/scrapy/command/commands/parse.py => scrapy/command/commands/parse.py
rename : scrapy/trunk/scrapy/command/commands/shell.py => scrapy/command/commands/shell.py
rename : scrapy/trunk/scrapy/command/commands/start.py => scrapy/command/commands/start.py
rename : scrapy/trunk/scrapy/command/commands/stats.py => scrapy/command/commands/stats.py
rename : scrapy/trunk/scrapy/command/models.py => scrapy/command/models.py
rename : scrapy/trunk/scrapy/conf/__init__.py => scrapy/conf/__init__.py
rename : scrapy/trunk/scrapy/conf/commands/__init__.py => scrapy/conf/commands/__init__.py
rename : scrapy/trunk/scrapy/conf/commands/crawl.py => scrapy/conf/commands/crawl.py
rename : scrapy/trunk/scrapy/conf/commands/help.py => scrapy/conf/commands/help.py
rename : scrapy/trunk/scrapy/conf/commands/list.py => scrapy/conf/commands/list.py
rename : scrapy/trunk/scrapy/conf/commands/log.py => scrapy/conf/commands/log.py
rename : scrapy/trunk/scrapy/conf/commands/scrape.py => scrapy/conf/commands/scrape.py
rename : scrapy/trunk/scrapy/conf/commands/shell.py => scrapy/conf/commands/shell.py
rename : scrapy/trunk/scrapy/conf/commands/stats.py => scrapy/conf/commands/stats.py
rename : scrapy/trunk/scrapy/conf/commands/test.py => scrapy/conf/commands/test.py
rename : scrapy/trunk/scrapy/conf/default_settings.py => scrapy/conf/default_settings.py
rename : scrapy/trunk/scrapy/contrib/__init__.py => scrapy/contrib/__init__.py
rename : scrapy/trunk/scrapy/contrib/aws.py => scrapy/contrib/aws.py
rename : scrapy/trunk/scrapy/contrib/closedomain.py => scrapy/contrib/closedomain.py
rename : scrapy/trunk/scrapy/contrib/cluster/__init__.py => scrapy/contrib/cluster/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/crawler/__init__.py => scrapy/contrib/cluster/crawler/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/crawler/manager.py => scrapy/contrib/cluster/crawler/manager.py
rename : scrapy/trunk/scrapy/contrib/cluster/hooks/__init__.py => scrapy/contrib/cluster/hooks/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/hooks/svn.py => scrapy/contrib/cluster/hooks/svn.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/__init__.py => scrapy/contrib/cluster/master/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/manager.py => scrapy/contrib/cluster/master/manager.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/web.py => scrapy/contrib/cluster/master/web.py
rename : scrapy/trunk/scrapy/contrib/cluster/master/ws_api.txt => scrapy/contrib/cluster/master/ws_api.txt
rename : scrapy/trunk/scrapy/contrib/cluster/tools/scrapy-cluster-ctl.py => scrapy/contrib/cluster/tools/scrapy-cluster-ctl.py
rename : scrapy/trunk/scrapy/contrib/cluster/tools/test-worker.py => scrapy/contrib/cluster/tools/test-worker.py
rename : scrapy/trunk/scrapy/contrib/cluster/worker/__init__.py => scrapy/contrib/cluster/worker/__init__.py
rename : scrapy/trunk/scrapy/contrib/cluster/worker/manager.py => scrapy/contrib/cluster/worker/manager.py
rename : scrapy/trunk/scrapy/contrib/codecs/__init__.py => scrapy/contrib/codecs/__init__.py
rename : scrapy/trunk/scrapy/contrib/codecs/x_mac_roman.py => scrapy/contrib/codecs/x_mac_roman.py
rename : scrapy/trunk/scrapy/contrib/debug.py => scrapy/contrib/debug.py
rename : scrapy/trunk/scrapy/contrib/delayedclosedomain.py => scrapy/contrib/delayedclosedomain.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/__init__.py => scrapy/contrib/downloadermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/cache.py => scrapy/contrib/downloadermiddleware/cache.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/common.py => scrapy/contrib/downloadermiddleware/common.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/cookies.py => scrapy/contrib/downloadermiddleware/cookies.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/debug.py => scrapy/contrib/downloadermiddleware/debug.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/errorpages.py => scrapy/contrib/downloadermiddleware/errorpages.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/httpauth.py => scrapy/contrib/downloadermiddleware/httpauth.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/httpcompression.py => scrapy/contrib/downloadermiddleware/httpcompression.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/redirect.py => scrapy/contrib/downloadermiddleware/redirect.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/retry.py => scrapy/contrib/downloadermiddleware/retry.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/robotstxt.py => scrapy/contrib/downloadermiddleware/robotstxt.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/stats.py => scrapy/contrib/downloadermiddleware/stats.py
rename : scrapy/trunk/scrapy/contrib/downloadermiddleware/useragent.py => scrapy/contrib/downloadermiddleware/useragent.py
rename : scrapy/trunk/scrapy/contrib/groupsettings.py => scrapy/contrib/groupsettings.py
rename : scrapy/trunk/scrapy/contrib/item/__init__.py => scrapy/contrib/item/__init__.py
rename : scrapy/trunk/scrapy/contrib/item/models.py => scrapy/contrib/item/models.py
rename : scrapy/trunk/scrapy/contrib/itemsampler.py => scrapy/contrib/itemsampler.py
rename : scrapy/trunk/scrapy/contrib/link_extractors.py => scrapy/contrib/link_extractors.py
rename : scrapy/trunk/scrapy/contrib/memdebug.py => scrapy/contrib/memdebug.py
rename : scrapy/trunk/scrapy/contrib/memusage.py => scrapy/contrib/memusage.py
rename : scrapy/trunk/scrapy/contrib/pipeline/__init__.py => scrapy/contrib/pipeline/__init__.py
rename : scrapy/trunk/scrapy/contrib/pipeline/images.py => scrapy/contrib/pipeline/images.py
rename : scrapy/trunk/scrapy/contrib/pipeline/media.py => scrapy/contrib/pipeline/media.py
rename : scrapy/trunk/scrapy/contrib/pipeline/s3images.py => scrapy/contrib/pipeline/s3images.py
rename : scrapy/trunk/scrapy/contrib/pipeline/show.py => scrapy/contrib/pipeline/show.py
rename : scrapy/trunk/scrapy/contrib/prioritizers.py => scrapy/contrib/prioritizers.py
rename : scrapy/trunk/scrapy/contrib/response/__init__.py => scrapy/contrib/response/__init__.py
rename : scrapy/trunk/scrapy/contrib/response/soup.py => scrapy/contrib/response/soup.py
rename : scrapy/trunk/scrapy/contrib/schedulermiddleware/__init__.py => scrapy/contrib/schedulermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib/schedulermiddleware/duplicatesfilter.py => scrapy/contrib/schedulermiddleware/duplicatesfilter.py
rename : scrapy/trunk/scrapy/contrib/spider/__init__.py => scrapy/contrib/spider/__init__.py
rename : scrapy/trunk/scrapy/contrib/spider/profiler.py => scrapy/contrib/spider/profiler.py
rename : scrapy/trunk/scrapy/contrib/spider/reloader.py => scrapy/contrib/spider/reloader.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/__init__.py => scrapy/contrib/spidermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/depth.py => scrapy/contrib/spidermiddleware/depth.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/duplicatesfilter.py => scrapy/contrib/spidermiddleware/duplicatesfilter.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/limit.py => scrapy/contrib/spidermiddleware/limit.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/offsite.py => scrapy/contrib/spidermiddleware/offsite.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/referer.py => scrapy/contrib/spidermiddleware/referer.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/restrict.py => scrapy/contrib/spidermiddleware/restrict.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/urlfilter.py => scrapy/contrib/spidermiddleware/urlfilter.py
rename : scrapy/trunk/scrapy/contrib/spidermiddleware/urllength.py => scrapy/contrib/spidermiddleware/urllength.py
rename : scrapy/trunk/scrapy/contrib/spiders/__init__.py => scrapy/contrib/spiders/__init__.py
rename : scrapy/trunk/scrapy/contrib/spiders/crawl.py => scrapy/contrib/spiders/crawl.py
rename : scrapy/trunk/scrapy/contrib/spiders/feed.py => scrapy/contrib/spiders/feed.py
rename : scrapy/trunk/scrapy/contrib/spiders/generic.py => scrapy/contrib/spiders/generic.py
rename : scrapy/trunk/scrapy/contrib/web/__init__.py => scrapy/contrib/web/__init__.py
rename : scrapy/trunk/scrapy/contrib/web/http.py => scrapy/contrib/web/http.py
rename : scrapy/trunk/scrapy/contrib/web/json.py => scrapy/contrib/web/json.py
rename : scrapy/trunk/scrapy/contrib/web/service.py => scrapy/contrib/web/service.py
rename : scrapy/trunk/scrapy/contrib/web/site.py => scrapy/contrib/web/site.py
rename : scrapy/trunk/scrapy/contrib/web/stats.py => scrapy/contrib/web/stats.py
rename : scrapy/trunk/scrapy/contrib/webconsole/__init__.py => scrapy/contrib/webconsole/__init__.py
rename : scrapy/trunk/scrapy/contrib/webconsole/enginestatus.py => scrapy/contrib/webconsole/enginestatus.py
rename : scrapy/trunk/scrapy/contrib/webconsole/livestats.py => scrapy/contrib/webconsole/livestats.py
rename : scrapy/trunk/scrapy/contrib/webconsole/scheduler.py => scrapy/contrib/webconsole/scheduler.py
rename : scrapy/trunk/scrapy/contrib/webconsole/spiderctl.py => scrapy/contrib/webconsole/spiderctl.py
rename : scrapy/trunk/scrapy/contrib/webconsole/spiderstats.py => scrapy/contrib/webconsole/spiderstats.py
rename : scrapy/trunk/scrapy/contrib/webconsole/stats.py => scrapy/contrib/webconsole/stats.py
rename : scrapy/trunk/scrapy/contrib_exp/__init__.py => scrapy/contrib_exp/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/__init__.py => scrapy/contrib_exp/adaptors/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/date.py => scrapy/contrib_exp/adaptors/date.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/extraction.py => scrapy/contrib_exp/adaptors/extraction.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/markup.py => scrapy/contrib_exp/adaptors/markup.py
rename : scrapy/trunk/scrapy/contrib_exp/adaptors/misc.py => scrapy/contrib_exp/adaptors/misc.py
rename : scrapy/trunk/scrapy/contrib_exp/downloadermiddleware/__init__.py => scrapy/contrib_exp/downloadermiddleware/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/downloadermiddleware/decompression.py => scrapy/contrib_exp/downloadermiddleware/decompression.py
rename : scrapy/trunk/scrapy/contrib_exp/history/__init__.py => scrapy/contrib_exp/history/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/history/history.py => scrapy/contrib_exp/history/history.py
rename : scrapy/trunk/scrapy/contrib_exp/history/middleware.py => scrapy/contrib_exp/history/middleware.py
rename : scrapy/trunk/scrapy/contrib_exp/history/scheduler.py => scrapy/contrib_exp/history/scheduler.py
rename : scrapy/trunk/scrapy/contrib_exp/history/store.py => scrapy/contrib_exp/history/store.py
rename : scrapy/trunk/scrapy/contrib_exp/link/__init__.py => scrapy/contrib_exp/link/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/__init__.py => scrapy/contrib_exp/newitem/__init__.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/adaptors.py => scrapy/contrib_exp/newitem/adaptors.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/fields.py => scrapy/contrib_exp/newitem/fields.py
rename : scrapy/trunk/scrapy/contrib_exp/newitem/models.py => scrapy/contrib_exp/newitem/models.py
rename : scrapy/trunk/scrapy/contrib_exp/pipeline/shoveitem.py => scrapy/contrib_exp/pipeline/shoveitem.py
rename : scrapy/trunk/scrapy/core/__init__.py => scrapy/core/__init__.py
rename : scrapy/trunk/scrapy/core/downloader/__init__.py => scrapy/core/downloader/__init__.py
rename : scrapy/trunk/scrapy/core/downloader/dnscache.py => scrapy/core/downloader/dnscache.py
rename : scrapy/trunk/scrapy/core/downloader/handlers.py => scrapy/core/downloader/handlers.py
rename : scrapy/trunk/scrapy/core/downloader/manager.py => scrapy/core/downloader/manager.py
rename : scrapy/trunk/scrapy/core/downloader/middleware.py => scrapy/core/downloader/middleware.py
rename : scrapy/trunk/scrapy/core/downloader/responsetypes/__init__.py => scrapy/core/downloader/responsetypes/__init__.py
rename : scrapy/trunk/scrapy/core/downloader/responsetypes/mime.types => scrapy/core/downloader/responsetypes/mime.types
rename : scrapy/trunk/scrapy/core/downloader/webclient.py => scrapy/core/downloader/webclient.py
rename : scrapy/trunk/scrapy/core/engine.py => scrapy/core/engine.py
rename : scrapy/trunk/scrapy/core/exceptions.py => scrapy/core/exceptions.py
rename : scrapy/trunk/scrapy/core/manager.py => scrapy/core/manager.py
rename : scrapy/trunk/scrapy/core/prioritizers.py => scrapy/core/prioritizers.py
rename : scrapy/trunk/scrapy/core/scheduler/__init__.py => scrapy/core/scheduler/__init__.py
rename : scrapy/trunk/scrapy/core/scheduler/middleware.py => scrapy/core/scheduler/middleware.py
rename : scrapy/trunk/scrapy/core/scheduler/schedulers.py => scrapy/core/scheduler/schedulers.py
rename : scrapy/trunk/scrapy/core/scheduler/store.py => scrapy/core/scheduler/store.py
rename : scrapy/trunk/scrapy/core/signals.py => scrapy/core/signals.py
rename : scrapy/trunk/scrapy/dupefilter/__init__.py => scrapy/dupefilter/__init__.py
rename : scrapy/trunk/scrapy/extension/__init__.py => scrapy/extension/__init__.py
rename : scrapy/trunk/scrapy/fetcher/__init__.py => scrapy/fetcher/__init__.py
rename : scrapy/trunk/scrapy/http/__init__.py => scrapy/http/__init__.py
rename : scrapy/trunk/scrapy/http/cookies.py => scrapy/http/cookies.py
rename : scrapy/trunk/scrapy/http/headers.py => scrapy/http/headers.py
rename : scrapy/trunk/scrapy/http/request/__init__.py => scrapy/http/request/__init__.py
rename : scrapy/trunk/scrapy/http/request/form.py => scrapy/http/request/form.py
rename : scrapy/trunk/scrapy/http/request/rpc.py => scrapy/http/request/rpc.py
rename : scrapy/trunk/scrapy/http/response/__init__.py => scrapy/http/response/__init__.py
rename : scrapy/trunk/scrapy/http/response/html.py => scrapy/http/response/html.py
rename : scrapy/trunk/scrapy/http/response/text.py => scrapy/http/response/text.py
rename : scrapy/trunk/scrapy/http/response/xml.py => scrapy/http/response/xml.py
rename : scrapy/trunk/scrapy/http/url.py => scrapy/http/url.py
rename : scrapy/trunk/scrapy/item/__init__.py => scrapy/item/__init__.py
rename : scrapy/trunk/scrapy/item/adaptors.py => scrapy/item/adaptors.py
rename : scrapy/trunk/scrapy/item/models.py => scrapy/item/models.py
rename : scrapy/trunk/scrapy/item/pipeline.py => scrapy/item/pipeline.py
rename : scrapy/trunk/scrapy/link/__init__.py => scrapy/link/__init__.py
rename : scrapy/trunk/scrapy/link/extractors.py => scrapy/link/extractors.py
rename : scrapy/trunk/scrapy/log/__init__.py => scrapy/log/__init__.py
rename : scrapy/trunk/scrapy/mail/__init__.py => scrapy/mail/__init__.py
rename : scrapy/trunk/scrapy/management/__init__.py => scrapy/management/__init__.py
rename : scrapy/trunk/scrapy/management/telnet.py => scrapy/management/telnet.py
rename : scrapy/trunk/scrapy/management/web.py => scrapy/management/web.py
rename : scrapy/trunk/scrapy/patches/__init__.py => scrapy/patches/__init__.py
rename : scrapy/trunk/scrapy/patches/monkeypatches.py => scrapy/patches/monkeypatches.py
rename : scrapy/trunk/scrapy/spider/__init__.py => scrapy/spider/__init__.py
rename : scrapy/trunk/scrapy/spider/manager.py => scrapy/spider/manager.py
rename : scrapy/trunk/scrapy/spider/middleware.py => scrapy/spider/middleware.py
rename : scrapy/trunk/scrapy/spider/models.py => scrapy/spider/models.py
rename : scrapy/trunk/scrapy/stats/__init__.py => scrapy/stats/__init__.py
rename : scrapy/trunk/scrapy/stats/corestats.py => scrapy/stats/corestats.py
rename : scrapy/trunk/scrapy/stats/statscollector.py => scrapy/stats/statscollector.py
rename : scrapy/trunk/scrapy/store/__init__.py => scrapy/store/__init__.py
rename : scrapy/trunk/scrapy/store/db.py => scrapy/store/db.py
rename : scrapy/trunk/scrapy/templates/project/module/__init__.py => scrapy/templates/project/module/__init__.py
rename : scrapy/trunk/scrapy/templates/project/module/items.py.tmpl => scrapy/templates/project/module/items.py.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/pipelines.py.tmpl => scrapy/templates/project/module/pipelines.py.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/settings.py.tmpl => scrapy/templates/project/module/settings.py.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/spiders/__init__.py => scrapy/templates/project/module/spiders/__init__.py
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_basic.tmpl => scrapy/templates/project/module/templates/spider_basic.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_crawl.tmpl => scrapy/templates/project/module/templates/spider_crawl.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_csvfeed.tmpl => scrapy/templates/project/module/templates/spider_csvfeed.tmpl
rename : scrapy/trunk/scrapy/templates/project/module/templates/spider_xmlfeed.tmpl => scrapy/templates/project/module/templates/spider_xmlfeed.tmpl
rename : scrapy/trunk/scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/root/scrapy-ctl.py
rename : scrapy/trunk/scrapy/tests/__init__.py => scrapy/tests/__init__.py
rename : scrapy/trunk/scrapy/tests/run.py => scrapy/tests/run.py
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-ascii.html => scrapy/tests/sample_data/adaptors/enc-ascii.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-cp1252.html => scrapy/tests/sample_data/adaptors/enc-cp1252.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-latin1.html => scrapy/tests/sample_data/adaptors/enc-latin1.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-utf8-meta-latin1.html => scrapy/tests/sample_data/adaptors/enc-utf8-meta-latin1.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/enc-utf8.html => scrapy/tests/sample_data/adaptors/enc-utf8.html
rename : scrapy/trunk/scrapy/tests/sample_data/adaptors/extr_unquoted.xml => scrapy/tests/sample_data/adaptors/extr_unquoted.xml
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.tar => scrapy/tests/sample_data/compressed/feed-sample1.tar
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.xml => scrapy/tests/sample_data/compressed/feed-sample1.xml
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.xml.bz2 => scrapy/tests/sample_data/compressed/feed-sample1.xml.bz2
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.xml.gz => scrapy/tests/sample_data/compressed/feed-sample1.xml.gz
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/feed-sample1.zip => scrapy/tests/sample_data/compressed/feed-sample1.zip
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/html-gzip.bin => scrapy/tests/sample_data/compressed/html-gzip.bin
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/html-rawdeflate.bin => scrapy/tests/sample_data/compressed/html-rawdeflate.bin
rename : scrapy/trunk/scrapy/tests/sample_data/compressed/html-zlibdeflate.bin => scrapy/tests/sample_data/compressed/html-zlibdeflate.bin
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample1.xml => scrapy/tests/sample_data/feeds/feed-sample1.xml
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample2.xml => scrapy/tests/sample_data/feeds/feed-sample2.xml
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample3.csv => scrapy/tests/sample_data/feeds/feed-sample3.csv
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample4.csv => scrapy/tests/sample_data/feeds/feed-sample4.csv
rename : scrapy/trunk/scrapy/tests/sample_data/feeds/feed-sample5.csv => scrapy/tests/sample_data/feeds/feed-sample5.csv
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/image_linkextractor.html => scrapy/tests/sample_data/link_extractor/image_linkextractor.html
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/linkextractor_latin1.html => scrapy/tests/sample_data/link_extractor/linkextractor_latin1.html
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/linkextractor_noenc.html => scrapy/tests/sample_data/link_extractor/linkextractor_noenc.html
rename : scrapy/trunk/scrapy/tests/sample_data/link_extractor/regex_linkextractor.html => scrapy/tests/sample_data/link_extractor/regex_linkextractor.html
rename : scrapy/trunk/scrapy/tests/sample_data/test_site/index.html => scrapy/tests/sample_data/test_site/index.html
rename : scrapy/trunk/scrapy/tests/sample_data/test_site/item1.html => scrapy/tests/sample_data/test_site/item1.html
rename : scrapy/trunk/scrapy/tests/sample_data/test_site/item2.html => scrapy/tests/sample_data/test_site/item2.html
rename : scrapy/trunk/scrapy/tests/test_adaptors.py => scrapy/tests/test_adaptors.py
rename : scrapy/trunk/scrapy/tests/test_aws.py => scrapy/tests/test_aws.py
rename : scrapy/trunk/scrapy/tests/test_c14nurls.py => scrapy/tests/test_c14nurls.py
rename : scrapy/trunk/scrapy/tests/test_contrib_response_soup.py => scrapy/tests/test_contrib_response_soup.py
rename : scrapy/trunk/scrapy/tests/test_dependencies.py => scrapy/tests/test_dependencies.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_cookies.py => scrapy/tests/test_downloadermiddleware_cookies.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_decompression.py => scrapy/tests/test_downloadermiddleware_decompression.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_httpcompression.py => scrapy/tests/test_downloadermiddleware_httpcompression.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_redirect.py => scrapy/tests/test_downloadermiddleware_redirect.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_retry.py => scrapy/tests/test_downloadermiddleware_retry.py
rename : scrapy/trunk/scrapy/tests/test_downloadermiddleware_useragent.py => scrapy/tests/test_downloadermiddleware_useragent.py
rename : scrapy/trunk/scrapy/tests/test_dupefilter.py => scrapy/tests/test_dupefilter.py
rename : scrapy/trunk/scrapy/tests/test_engine.py => scrapy/tests/test_engine.py
rename : scrapy/trunk/scrapy/tests/test_http_cookies.py => scrapy/tests/test_http_cookies.py
rename : scrapy/trunk/scrapy/tests/test_http_headers.py => scrapy/tests/test_http_headers.py
rename : scrapy/trunk/scrapy/tests/test_http_request.py => scrapy/tests/test_http_request.py
rename : scrapy/trunk/scrapy/tests/test_http_response.py => scrapy/tests/test_http_response.py
rename : scrapy/trunk/scrapy/tests/test_http_url.py => scrapy/tests/test_http_url.py
rename : scrapy/trunk/scrapy/tests/test_item.py => scrapy/tests/test_item.py
rename : scrapy/trunk/scrapy/tests/test_itemadaptor.py => scrapy/tests/test_itemadaptor.py
rename : scrapy/trunk/scrapy/tests/test_libxml2.py => scrapy/tests/test_libxml2.py
rename : scrapy/trunk/scrapy/tests/test_link.py => scrapy/tests/test_link.py
rename : scrapy/trunk/scrapy/tests/test_newitem.py => scrapy/tests/test_newitem.py
rename : scrapy/trunk/scrapy/tests/test_pipeline_images.py => scrapy/tests/test_pipeline_images.py
rename : scrapy/trunk/scrapy/tests/test_responsetypes.py => scrapy/tests/test_responsetypes.py
rename : scrapy/trunk/scrapy/tests/test_robustscrapeditem.py => scrapy/tests/test_robustscrapeditem.py
rename : scrapy/trunk/scrapy/tests/test_schedulermiddleware_duplicatesfilter.py => scrapy/tests/test_schedulermiddleware_duplicatesfilter.py
rename : scrapy/trunk/scrapy/tests/test_serialization.py => scrapy/tests/test_serialization.py
rename : scrapy/trunk/scrapy/tests/test_spidermiddleware_duplicatesfilter.py => scrapy/tests/test_spidermiddleware_duplicatesfilter.py
rename : scrapy/trunk/scrapy/tests/test_spidermonkey.py => scrapy/tests/test_spidermonkey.py
rename : scrapy/trunk/scrapy/tests/test_spiders/__init__.py => scrapy/tests/test_spiders/__init__.py
rename : scrapy/trunk/scrapy/tests/test_spiders/testspider.py => scrapy/tests/test_spiders/testspider.py
rename : scrapy/trunk/scrapy/tests/test_stats.py => scrapy/tests/test_stats.py
rename : scrapy/trunk/scrapy/tests/test_storedb.py => scrapy/tests/test_storedb.py
rename : scrapy/trunk/scrapy/tests/test_utils_datatypes.py => scrapy/tests/test_utils_datatypes.py
rename : scrapy/trunk/scrapy/tests/test_utils_defer.py => scrapy/tests/test_utils_defer.py
rename : scrapy/trunk/scrapy/tests/test_utils_iterators.py => scrapy/tests/test_utils_iterators.py
rename : scrapy/trunk/scrapy/tests/test_utils_markup.py => scrapy/tests/test_utils_markup.py
rename : scrapy/trunk/scrapy/tests/test_utils_misc.py => scrapy/tests/test_utils_misc.py
rename : scrapy/trunk/scrapy/tests/test_utils_python.py => scrapy/tests/test_utils_python.py
rename : scrapy/trunk/scrapy/tests/test_utils_request.py => scrapy/tests/test_utils_request.py
rename : scrapy/trunk/scrapy/tests/test_utils_response.py => scrapy/tests/test_utils_response.py
rename : scrapy/trunk/scrapy/tests/test_utils_url.py => scrapy/tests/test_utils_url.py
rename : scrapy/trunk/scrapy/tests/test_webclient.py => scrapy/tests/test_webclient.py
rename : scrapy/trunk/scrapy/tests/test_xpath.py => scrapy/tests/test_xpath.py
rename : scrapy/trunk/scrapy/tests/test_xpath_extension.py => scrapy/tests/test_xpath_extension.py
rename : scrapy/trunk/scrapy/utils/__init__.py => scrapy/utils/__init__.py
rename : scrapy/trunk/scrapy/utils/c14n.py => scrapy/utils/c14n.py
rename : scrapy/trunk/scrapy/utils/datatypes.py => scrapy/utils/datatypes.py
rename : scrapy/trunk/scrapy/utils/db.py => scrapy/utils/db.py
rename : scrapy/trunk/scrapy/utils/defer.py => scrapy/utils/defer.py
rename : scrapy/trunk/scrapy/utils/display.py => scrapy/utils/display.py
rename : scrapy/trunk/scrapy/utils/http.py => scrapy/utils/http.py
rename : scrapy/trunk/scrapy/utils/iterators.py => scrapy/utils/iterators.py
rename : scrapy/trunk/scrapy/utils/markup.py => scrapy/utils/markup.py
rename : scrapy/trunk/scrapy/utils/misc.py => scrapy/utils/misc.py
rename : scrapy/trunk/scrapy/utils/python.py => scrapy/utils/python.py
rename : scrapy/trunk/scrapy/utils/request.py => scrapy/utils/request.py
rename : scrapy/trunk/scrapy/utils/response.py => scrapy/utils/response.py
rename : scrapy/trunk/scrapy/utils/serialization.py => scrapy/utils/serialization.py
rename : scrapy/trunk/scrapy/utils/test.py => scrapy/utils/test.py
rename : scrapy/trunk/scrapy/utils/url.py => scrapy/utils/url.py
rename : scrapy/trunk/scrapy/xlib/BeautifulSoup.py => scrapy/xlib/BeautifulSoup.py
rename : scrapy/trunk/scrapy/xlib/ClientForm.py => scrapy/xlib/ClientForm.py
rename : scrapy/trunk/scrapy/xlib/__init__.py => scrapy/xlib/__init__.py
rename : scrapy/trunk/scrapy/xlib/lrucache.py => scrapy/xlib/lrucache.py
rename : scrapy/trunk/scrapy/xlib/lsprofcalltree.py => scrapy/xlib/lsprofcalltree.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/__init__.py => scrapy/xlib/pydispatch/__init__.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/dispatcher.py => scrapy/xlib/pydispatch/dispatcher.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/errors.py => scrapy/xlib/pydispatch/errors.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/license.txt => scrapy/xlib/pydispatch/license.txt
rename : scrapy/trunk/scrapy/xlib/pydispatch/robust.py => scrapy/xlib/pydispatch/robust.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/robustapply.py => scrapy/xlib/pydispatch/robustapply.py
rename : scrapy/trunk/scrapy/xlib/pydispatch/saferef.py => scrapy/xlib/pydispatch/saferef.py
rename : scrapy/trunk/scrapy/xlib/spidermonkey/INSTALL.scrapy => scrapy/xlib/spidermonkey/INSTALL.scrapy
rename : scrapy/trunk/scrapy/xlib/spidermonkey/__init__.py => scrapy/xlib/spidermonkey/__init__.py
rename : scrapy/trunk/scrapy/xlib/spidermonkey/sm_settings.py => scrapy/xlib/spidermonkey/sm_settings.py
rename : scrapy/trunk/scrapy/xlib/spidermonkey/spidermonkey.py => scrapy/xlib/spidermonkey/spidermonkey.py
rename : scrapy/trunk/scrapy/xpath/__init__.py => scrapy/xpath/__init__.py
rename : scrapy/trunk/scrapy/xpath/constructors.py => scrapy/xpath/constructors.py
rename : scrapy/trunk/scrapy/xpath/document.py => scrapy/xpath/document.py
rename : scrapy/trunk/scrapy/xpath/extension.py => scrapy/xpath/extension.py
rename : scrapy/trunk/scrapy/xpath/selector.py => scrapy/xpath/selector.py
rename : scrapy/trunk/scrapy/xpath/types.py => scrapy/xpath/types.py
rename : scrapy/trunk/scripts/rpm-install.sh => scripts/rpm-install.sh
rename : scrapy/trunk/setup.cfg => setup.cfg
rename : scrapy/trunk/setup.py => setup.py
2009-05-06 15:55:17 -03:00