RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,565 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

52 Commits

Author SHA1 Message Date
Pablo Hoffman ead8cb9fac removed documentation about ugly DontCloseDomain exception (which will be removed in the future) 2009-08-24 11:57:21 -03:00
Pablo Hoffman 74f706b356 renamed "parse_item" method of XMLFeedSpider to "parse_node", keeping backwards compatibility 2009-08-24 10:54:34 -03:00
Pablo Hoffman 31693eb90f dropped "cache" attribute of Request and Response objects 2009-08-24 10:34:05 -03:00
Pablo Hoffman 0186c6937a HTTP auth middleware: added doc and unittest 2009-08-24 08:07:20 -03:00
Pablo Hoffman 46aaea9f32 doc: improved documentation about debugging leaks 2009-08-23 20:36:00 -03:00
Pablo Hoffman 9635a7839c rearranged documentation into a better organization 
--HG--
rename : docs/topics/index.rst => docs/index.rst
 2009-08-21 21:49:54 -03:00
Pablo Hoffman 4761e0c88d minor doc correction 2009-08-21 19:11:59 -03:00
Pablo Hoffman ef6f04eb06 moved doc about debugging memory leaks to its own topic and added doc about trackref module 2009-08-21 16:07:16 -03:00
Pablo Hoffman 4980eb4968 added titles to signals doc 2009-08-21 15:07:52 -03:00
Pablo Hoffman daea3f8a0c sphinx docs: replaced custom :exception: xref by standard :exc: 2009-08-21 15:05:06 -03:00
Pablo Hoffman 37760eb13f improved consistency of logging settings to use LOG_* 2009-08-21 08:54:12 -03:00
Ismael Carnales f0689a9ee8 fixed error in link extractors doc, thanks tarasm 2009-08-21 08:34:53 -03:00
Pablo Hoffman 41f1972e47 removed unused TRACE log level and improved logging documentation 2009-08-20 18:17:48 -03:00
Pablo Hoffman 3152e66fec deprecate domain_open signal and handle stats domain open/close directly from the engine 2009-08-20 10:54:58 -03:00
Pablo Hoffman 33b53c59d5 moved scrapy.xpath to scrapy.selector 
--HG--
rename : scrapy/xpath/__init__.py => scrapy/selector/__init__.py
rename : scrapy/xpath/document.py => scrapy/selector/document.py
rename : scrapy/xpath/factories.py => scrapy/selector/factories.py
 2009-08-19 21:50:52 -03:00
Pablo Hoffman dcc90fc196 declared loaders api stable and updated example project to use them 
--HG--
rename : docs/experimental/loaders.rst => docs/topics/loaders.rst
 2009-08-19 21:39:59 -03:00
Pablo Hoffman e8504a054c moved scrapy.newitem to scrapy.item and declared newitem api officially stable. updated docs and example project. deprecated old ScrapedItem 2009-08-19 21:39:58 -03:00
Ismael Carnales f88fb27851 fixed error in xpath selectors doc 2009-08-18 15:18:49 -03:00
Ismael Carnales 67c0c6a9e4 corrected indentation in xpath selectors doc 2009-08-18 15:13:23 -03:00
Ismael Carnales 428dfe0d4a corrected the style of spiders documentation 2009-08-18 15:06:33 -03:00
Pablo Hoffman 0192282d07 reorganized doc and moved robotstxt doc inside downloader middlewares doc 2009-08-18 14:36:18 -03:00
Ismael Carnales 33089d287d merged topics and reference doc 2009-08-18 14:05:15 -03:00
Pablo Hoffman 7f30461410 added support for defining EXTENSIONS setting using dicts, like middleware settings 
--HG--
rename : scrapy/tests/test_utils_middleware.py => scrapy/tests/test_utils_conf.py
rename : scrapy/utils/middleware.py => scrapy/utils/conf.py
 2009-08-18 11:05:36 -03:00
Ismael Carnales 48b40bd620 renamed x method of selectors to select 2009-08-17 15:58:06 -03:00
Pablo Hoffman d4aa72d2bb removed obsolete documentation about Robust Scraped Item and Adaptors 2009-08-12 13:50:15 -03:00
Pablo Hoffman e69311ef38 added missing text to new stats collector methods 2009-08-11 16:37:52 -03:00
Daniel Grana 45cbe1333f fix typo in stats docs 2009-08-11 16:30:04 -03:00
Daniel Grana 8f16e7f9fa remove `default` parameter from max_value/min_value stats methods, update docs, and add new methods to dummy collector 2009-08-11 16:23:23 -03:00
Daniel Grana cd8eed77c5 stats collector gains two new methods to store values only if greater/lower than previous value 2009-08-11 15:46:44 -03:00
Ismael Carnales 3614998053 fixed error in doc 2009-08-11 09:23:45 -03:00
Pablo Hoffman b296d4169e minor doc update for making it more windows-friendly 2009-08-09 17:08:42 -03:00
Pablo Hoffman 91eea82eef added XPathLoader for working with XPath Selectors more conveniently 2009-08-08 07:26:20 -03:00
Pablo Hoffman d95e99f585 Added documentation for Items and Loaders, removed obsolete Item Adaptors documentation 
--HG--
rename : docs/experimental/topics/newitem/index.rst => docs/experimental/newitem.rst
 2009-08-07 03:50:09 -03:00
Pablo Hoffman c3427e075c added domain_stats parameter to stats_domain_closed signal 2009-07-31 16:36:35 -03:00
Pablo Hoffman c615ace6bd doc: updated google directory links in firebug guide 2009-07-24 13:14:36 -03:00
Ismael Carnales 6d24ae5920 added reference to working with relative xpaths in the tutorial 2009-07-23 09:05:14 -03:00
Pablo Hoffman 9baa6bb2a8 minor selectors doc fix 2009-07-23 01:56:56 -03:00
Pablo Hoffman 92b746d866 SimpledbStatsCollector: moved domain creation to constructor 2009-07-17 12:49:53 -03:00
Pablo Hoffman 292757f312 added link to architecture overview and fixed old link 2009-07-16 19:15:19 -03:00
Pablo Hoffman 136014d718 Added section about relative xpaths to XPathSelectors doc 2009-07-16 17:29:29 -03:00
Ismael Carnales a9069e8eaa reorganized experimental doc in topics and ref, slitted long documents, introduced minor changes 2009-07-16 09:58:44 -03:00
Pablo Hoffman 5582305648 renamed old STATS_DEBUG setting to STATS_DUMP 2009-07-15 21:02:51 -03:00
Pablo Hoffman 7f281cf295 some minor updates to doc 2009-07-15 01:40:44 -03:00
Pablo Hoffman 8480f49cf6 Complete stats refactoring and documentation 
--HG--
rename : scrapy/stats/statscollector.py => scrapy/stats/collector/__init__.py
 2009-07-15 00:30:40 -03:00
Pablo Hoffman 74fcfc2cfc deprecated old adaptors documentation 2009-07-13 22:03:56 -03:00
Pablo Hoffman 5054b67a02 improved newitems doc and marked robust scraped items as deprecated 2009-07-11 21:26:52 -03:00
Daniel Grana d5d2c5c924 update documentation to recent pydispatcher import path change 2009-07-09 17:13:30 -03:00
Pablo Hoffman 834ac9fca0 Some telnet console changes: 
- added telnet console documentation
- added documentation for debugging memory leaks with guppy
- sorted out shell alises
- set default port (TELNETCONSOLE_PORT) to 6023
 2009-06-23 16:08:58 -03:00
Pablo Hoffman 90d408b04f Some changes to HTTP cache middleware: 
* documented
* moved from scrapy.contrib.downloadermiddleware.cache.CacheMiddleware to
  scrapy.contrib.downloadermiddleware.httpcache.HttpCacheMiddleware
* settings prefix changed from CACHE2_ to HTTPCACHE_

--HG--
rename : scrapy/contrib/downloadermiddleware/cache.py => scrapy/contrib/downloadermiddleware/httpcache.py
 2009-05-24 19:13:06 -03:00
Pablo Hoffman 86498abdf1 Sorted out Link Extractors organization by moving all them to 
scrapy.contrib.linkextractors.

The most relevant being:
    scrapy.link.extractors.RegexLinkExtractor

which was moved to:
    scrapy.contrib.linkextractors.sgml.SgmlLinkExtractor

The old location still works but throws a deprecation warning. It will be
removed before the 0.7 release.

Documentation and tests were also updated.

Also, in this changeset, a new regex-based link extractor was added to
scrapy.contrib.linkextractors.regex.

--HG--
rename : scrapy/tests/sample_data/link_extractor/regex_linkextractor.html => scrapy/tests/sample_data/link_extractor/sgml_linkextractor.html
rename : scrapy/tests/test_link.py => scrapy/tests/test_contrib_linkextractors.py
 2009-05-18 19:19:37 -03:00