RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,129 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

3129 Commits

Author SHA1 Message Date
Pablo Hoffman 81ed2d2d0b major Stats Collection refactoring: removed separation of global/per-spider stats, removed stats-related signals (stats_spider_opened, etc). Stats are much simpler now, backwards compatibility is kept on the Stats Collector API. 2012-09-14 12:31:33 -03:00
Pablo Hoffman 8b48420559 better names for scheduler stats 2012-09-13 20:07:11 -03:00
Pablo Hoffman 263e302b6a added more scheduler stats 2012-09-13 19:46:45 -03:00
Pablo Hoffman d6867f7984 removed sphinx warnings about duplicate reference names 'this page' 2012-09-13 15:26:45 -03:00
Pablo Hoffman a874964ad4 renamed 'XPath Selectors' title to just 'Selectors' 2012-09-13 15:24:44 -03:00
Pablo Hoffman acb8895e1a changed note in scrapyd doc to use sphinx notes 2012-09-13 15:22:59 -03:00
Pablo Hoffman 26f1d5cb48 Merge pull request #171 from artem-dev/scrapyd_job_times 
added start and stop times for scrapyd list jobs web service
 2012-09-13 11:16:01 -07:00
Artem Bogomyagkov 4d8f253912 commited doc file missed from prev commit 2012-09-12 11:12:59 +03:00
Artem Bogomyagkov 5dde26d3c8 simplified code for finished jobs times, updated docs for scrapyd 2012-09-12 11:10:49 +03:00
Pablo Hoffman 7ace23c385 removed another instance of scrapy.conf singleton, this time from test_cmdline 2012-09-11 16:37:55 -03:00
Pablo Hoffman 8642645109 removed another instance of scrapy.conf singleton 2012-09-11 16:34:58 -03:00
Pablo Hoffman 7ef593c5c2 refactored MailSender to get rid of scrapy.conf singleton, also removed ill-designed scrapy.mail.mail_sent signal 2012-09-11 16:27:19 -03:00
Artem Bogomyagkov a42a8cafef added start and stop times for scrapyd list jobs web service 2012-09-11 21:06:36 +03:00
Pablo Hoffman 1e2efe5664 scrapy deploy: retry setup.py bdist_egg on EINTR. closes #9 2012-09-07 15:40:26 -03:00
Pablo Hoffman 3c05fbb6a9 removed redundant comment 2012-09-05 15:54:23 -03:00
Pablo Hoffman 44fe4c6b78 set missing 'settings' attribute in FeedExports 2012-09-05 15:54:13 -03:00
Pablo Hoffman 0135a5364e removed debugging print 2012-09-05 15:53:30 -03:00
Pablo Hoffman 7bf0af7a58 fixed some bugs in get_func_args() and added more test cases 2012-09-04 19:01:20 -03:00
Pablo Hoffman 251d9a9033 backwards compatibility support for FeedExport's that don't receive a settings object in their constructor 2012-09-04 18:13:27 -03:00
Pablo Hoffman caa64908d2 added tests for get_func_args() and support for more cases 2012-09-04 18:07:44 -03:00
Pablo Hoffman fff2871828 added doc section (and FAQ) about spider arguments 2012-09-04 14:49:30 -03:00
Pablo Hoffman 241abcb8aa scrapyd: log errors in API calls to the scrapyd log 2012-09-04 14:22:21 -03:00
Pablo Hoffman f4a17ec272 removed references to Scrapy Snippets site 2012-09-03 22:19:15 -03:00
Pablo Hoffman b901e64044 replaced memory usage acounting with (more portable) resource module, removed scrapy.utils.memory module. closes #161 2012-09-03 19:28:16 -03:00
Pablo Hoffman e2f9daac67 fixed formatting in scrapyd release notes 2012-09-03 16:58:58 -03:00
Pablo Hoffman dd1398b280 removed another instance of scrapy.conf.settings singleton, this time from download handlers 2012-09-03 16:44:18 -03:00
Daniel Graña 3b768ae989 oops. remove santas debug line 2012-08-31 20:10:32 -03:00
Pablo Hoffman c6435c5aa7 restored scrapyd log message 2012-08-31 18:49:20 -03:00
Pablo Hoffman 7c8af83138 added issue role to documentation 2012-08-31 18:39:30 -03:00
Pablo Hoffman 3891c20840 Merge pull request #164 from dangra/lazy-log-formatting 
format log lines lazily in case they are dropped by loglevels
 2012-08-31 14:33:41 -07:00
Daniel Graña a2d22307ad update news.rst 2012-08-31 17:38:40 -03:00
Daniel Graña dcef7b03c1 format log lines lazily in case they are dropped by loglevels 2012-08-31 17:14:35 -03:00
Pablo Hoffman be206ca5ab added process_start_requests method to spider middlewares 2012-08-31 16:41:50 -03:00
Pablo Hoffman b920674666 corrected minor issue with doc references 2012-08-31 16:41:49 -03:00
Pablo Hoffman a2d5a304b1 Merge pull request #168 from alexcepoi/parse-fixes 
parse command: fix crash in case of bad callback
 2012-08-31 09:12:19 -07:00
Alex Cepoi 0350bed8fd parse command: fix crash in case of bad callback 2012-08-31 16:43:04 +02:00
Pablo Hoffman 4ec99117d3 fixed minor doc typo 2012-08-30 11:56:30 -03:00
Pablo Hoffman 94b40162a9 fixed tests to work on windows 2012-08-30 11:24:29 -03:00
Pablo Hoffman 3904f5f55a more reliable way to set subprocess PYTHONPATH in tests 2012-08-30 10:09:20 -03:00
Pablo Hoffman 4a5f70278f minor tidy up to installation guide windows notes 2012-08-29 15:44:24 -03:00
Pablo Hoffman 098d892c03 simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users 2012-08-29 15:37:05 -03:00
Pablo Hoffman 6217f108cc avoid random "Interrupted system call" errors 2012-08-29 11:44:00 -03:00
Pablo Hoffman 8aa46a4b5d removed another usage of scrapy.conf.settings singleton 2012-08-29 11:43:01 -03:00
Pablo Hoffman 280d9d4094 added file missed in previous commit 2012-08-29 11:26:52 -03:00
Pablo Hoffman 70f8e517a1 promoted DjangoItem to main contrib 2012-08-29 11:23:11 -03:00
Pablo Hoffman 52151e8703 fixed bug in FeedExports extension, introduced in previous commit 2012-08-28 19:32:43 -03:00
Pablo Hoffman da234af456 Merge branch 'singleton_removal' 2012-08-28 18:42:33 -03:00
Pablo Hoffman babfc6e79b Updated documentation after singleton removal changes. 
Also removed some unused code and made some minor additional
refactoring.
 2012-08-28 18:35:57 -03:00
Pablo Hoffman c95f717ef2 fixed typo in tests 2012-08-22 13:50:23 -03:00
Pablo Hoffman 5fa5c4545b use ScrapyJSONEncoder JsonItemExporter & JsonLinesItemExporter, to support nested items properly 2012-08-22 13:46:50 -03:00