RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,171 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

608 Commits

Author SHA1 Message Date
Pablo Hoffman 1f89eb59fe fixed doc reference to topics-contracts 2012-10-09 16:02:12 -02:00
Pablo Hoffman 9e4c6aba2f updated release notes with new stats recently added 2012-10-09 12:50:42 -02:00
Pablo Hoffman 7458092eef added spider contracts to release notes and warn that its API is still subject to change 2012-09-29 03:06:30 -03:00
Pablo Hoffman c380910b40 Merge pull request #167 from alexcepoi/sep-017 
Spider contracts (SEP-017)
 2012-09-28 13:57:07 -07:00
Pablo Hoffman a5f8943d3e added change introduced in previous commit to release notes 2012-09-26 13:17:01 -03:00
Alex Cepoi 11d29c7005 SEP-017 contracts: add tests and minor improvements 2012-09-21 00:12:46 +02:00
Pablo Hoffman b46b5a6ef0 Documented AutoThrottle extension and added to extensions available by 
default. Also deprecated concurrency and delay settings, in favour of
using the standard Scrapy ones.
 2012-09-20 18:52:57 -03:00
Pablo Hoffman c7f8219901 - removed scrapy.conf singleton from scrapy.log, scrapy.responsetypes, 
scrapy.http.response.text, scrapy.selector
- fixed bug with scrapy.conf.settings backwards compatibility support
- added facility to notify (and provide some guidelines) about deprecated/obsolete settings
 2012-09-19 03:03:34 -03:00
stav 303e13f616 selector documentation typos 2012-09-18 12:56:52 -05:00
Pablo Hoffman 3d736e657f fixed typo in doc 2012-09-18 10:51:01 -03:00
Pablo Hoffman eed6eb49da make DBM the new default storage backend for HTTP cache middleware, simplified DBM storage backend code to avoid dealing with many spiders at once (not needed), and update httpcache stats names (hit -> hits, miss -> misses) 2012-09-17 10:11:07 -03:00
Pablo Hoffman 8f2dda12cc removed another instance of scrapy.conf.settings singleton, this time from scrapy.utils.trackref. From now on, trackrefs functionality will be always enabled as it imposes a very minimal performance overhead 2012-09-16 21:21:44 -03:00
Pablo Hoffman 9685c24059 restored stats_spider_{opened,closing,closed} signals in backwards compatibility mode 2012-09-14 12:38:00 -03:00
Pablo Hoffman 81ed2d2d0b major Stats Collection refactoring: removed separation of global/per-spider stats, removed stats-related signals (stats_spider_opened, etc). Stats are much simpler now, backwards compatibility is kept on the Stats Collector API. 2012-09-14 12:31:33 -03:00
Pablo Hoffman d6867f7984 removed sphinx warnings about duplicate reference names 'this page' 2012-09-13 15:26:45 -03:00
Pablo Hoffman a874964ad4 renamed 'XPath Selectors' title to just 'Selectors' 2012-09-13 15:24:44 -03:00
Pablo Hoffman acb8895e1a changed note in scrapyd doc to use sphinx notes 2012-09-13 15:22:59 -03:00
Pablo Hoffman 26f1d5cb48 Merge pull request #171 from artem-dev/scrapyd_job_times 
added start and stop times for scrapyd list jobs web service
 2012-09-13 11:16:01 -07:00
Artem Bogomyagkov 4d8f253912 commited doc file missed from prev commit 2012-09-12 11:12:59 +03:00
Pablo Hoffman 7ef593c5c2 refactored MailSender to get rid of scrapy.conf singleton, also removed ill-designed scrapy.mail.mail_sent signal 2012-09-11 16:27:19 -03:00
Alex Cepoi bf8dc61fb7 SEP-017 contracts: pretty-printing and docs 2012-09-10 23:17:27 +02:00
Pablo Hoffman fff2871828 added doc section (and FAQ) about spider arguments 2012-09-04 14:49:30 -03:00
Pablo Hoffman f4a17ec272 removed references to Scrapy Snippets site 2012-09-03 22:19:15 -03:00
Pablo Hoffman b901e64044 replaced memory usage acounting with (more portable) resource module, removed scrapy.utils.memory module. closes #161 2012-09-03 19:28:16 -03:00
Pablo Hoffman e2f9daac67 fixed formatting in scrapyd release notes 2012-09-03 16:58:58 -03:00
Pablo Hoffman dd1398b280 removed another instance of scrapy.conf.settings singleton, this time from download handlers 2012-09-03 16:44:18 -03:00
Pablo Hoffman 7c8af83138 added issue role to documentation 2012-08-31 18:39:30 -03:00
Daniel Graña a2d22307ad update news.rst 2012-08-31 17:38:40 -03:00
Pablo Hoffman be206ca5ab added process_start_requests method to spider middlewares 2012-08-31 16:41:50 -03:00
Pablo Hoffman b920674666 corrected minor issue with doc references 2012-08-31 16:41:49 -03:00
Pablo Hoffman 4ec99117d3 fixed minor doc typo 2012-08-30 11:56:30 -03:00
Pablo Hoffman 4a5f70278f minor tidy up to installation guide windows notes 2012-08-29 15:44:24 -03:00
Pablo Hoffman 098d892c03 simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users 2012-08-29 15:37:05 -03:00
Pablo Hoffman 70f8e517a1 promoted DjangoItem to main contrib 2012-08-29 11:23:11 -03:00
Pablo Hoffman da234af456 Merge branch 'singleton_removal' 2012-08-28 18:42:33 -03:00
Pablo Hoffman babfc6e79b Updated documentation after singleton removal changes. 
Also removed some unused code and made some minor additional
refactoring.
 2012-08-28 18:35:57 -03:00
Pablo Hoffman 5fa5c4545b use ScrapyJSONEncoder JsonItemExporter & JsonLinesItemExporter, to support nested items properly 2012-08-22 13:46:50 -03:00
Pablo Hoffman 36f47a4aec Removed per-spider settings concept, and scrapy.conf.settings singleton from many extensions and middlewares. There are some still remaining, that will be removed in future commits 2012-08-21 17:27:45 -03:00
Daniel Graña abcc8c9f63 Recommend pypi as single way to install on Windows 2012-08-06 10:21:13 -03:00
Daniel Graña b7b0c49520 append parse command to example code sections in docs. closes #162 2012-08-06 09:10:16 -03:00
Pablo Hoffman 832e45073b fixed typo in stats documentation. closes #159 2012-07-20 17:13:06 -03:00
Valentin-Costel Hăloiu 00bfb37e79 Update master 2012-07-04 06:55:01 +03:00
Daniel Graña 277ed0ae23 Merge pull request #145 from alexcepoi/cookies-changes 
domain and path support for request cookies
 2012-06-25 11:29:04 -07:00
Alexandru Cepoi 177c81745d domain and path support for request cookies 2012-06-25 20:17:59 +02:00
Pablo Hoffman 179e3810dc fixed links to doc. closes #150 2012-06-24 01:00:33 -03:00
Alexandru Cepoi f4faa19e31 added docs topic debugging spiders 2012-06-21 20:03:33 +02:00
Alexandru Cepoi 3e05a2ecf6 update docs for parse command 2012-06-12 18:28:10 +02:00
Pablo Hoffman e1be9c01bc updated FAQ about bot bans 2012-06-08 18:33:53 -03:00
Daniel Graña ae2ff4d33a update news file with 0.14.4 release notes 2012-05-15 16:16:02 -03:00
Pablo Hoffman 9686f97242 added precise to supported ubuntu distros 2012-05-12 19:54:36 -03:00