RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,628 Commits 42 Branches 132 Tags 165 MiB
Commit Graph

1628 Commits

Author SHA1 Message Date
Pablo Hoffman 8fab524978 moved engine.getstatus() method to scrapy.utils.engine function, to leave reporting logic out of engine code. added est() shortcut to telnet console 2009-08-31 12:44:32 -03:00
Pablo Hoffman dd80f6acdf MemoryUsage: changed .virtual property to methodd. SpiderProfiler: removed dependency on MemoryUsage extension 2009-08-31 12:14:30 -03:00
Pablo Hoffman e8e760c974 more simplifications to scrapy engine: removed addtasks method 2009-08-31 12:04:01 -03:00
Pablo Hoffman 1dd996f0b6 added missing import 2009-08-31 11:17:28 -03:00
Pablo Hoffman 8f013bf227 imported ismael patch for depreacting old SCRAPYSETTINGS_MODULE envvar 2009-08-31 10:17:12 -03:00
Pablo Hoffman 268acc30f5 merge with trunk 2009-08-31 09:44:55 -03:00
Pablo Hoffman 4f00c12a0a removed unneeded logic from engine 2009-08-31 09:44:31 -03:00
Daniel Grana 91df6fcc7c test genspider in a single testcase 2009-08-31 08:58:29 -03:00
Pablo Hoffman bcf81cd3c4 minor simplification to how default settings are loaded 2009-08-31 07:36:09 -03:00
Pablo Hoffman 062761166c simpledb collector: moved to_sdb_value function to utils.simpledb, and added unittests 2009-08-30 12:37:30 -03:00
Daniel Grana 1f50c24b6c one minus command testing line 2009-08-30 01:55:39 -03:00
Daniel Grana 8ef2cf0add more command testing simplifications 2009-08-30 01:51:34 -03:00
Daniel Grana 719fc2e7df sdb stats: extend type serializations, allow timestamp to be other than now 2009-08-30 01:08:23 -03:00
Daniel Grana 7cb546ba28 use explicit relative import on djangoitem tests 
--HG--
extra : rebase_source : f4f72679106bd9f72ddd9ee7d1f27209248f59be
 2009-08-29 21:04:32 -03:00
Pablo Hoffman 884f0c878f Stats collectin: fixed race condition between stats persistance and population of stats on domain close 2009-08-29 19:44:13 -03:00
Pablo Hoffman 895c70e036 doc: fixed some links to scrapy-ctl topic 2009-08-29 18:23:55 -03:00
Pablo Hoffman 64572124e0 added doc about SCRAPY_SETTINGS_MODULE 2009-08-29 18:20:13 -03:00
Pablo Hoffman 9dc7a3a373 some minor adjustments to logging doc 2009-08-29 18:10:28 -03:00
Pablo Hoffman 91ee71222f some minor changes to test_commands.py 2009-08-29 05:35:22 -03:00
Pablo Hoffman 5ba466a6e1 fixed unnapropiate handling of missing django module in djangitem tests 2009-08-29 05:03:40 -03:00
Pablo Hoffman 6e028fd91b fixed JsonLinesItemExporterTest 2009-08-29 04:59:51 -03:00
Pablo Hoffman 60cbf24c89 more cleanups to startproject and project templates 
--HG--
rename : scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/scrapy-ctl.py
 2009-08-29 04:29:47 -03:00
Pablo Hoffman c87216f02b doc: added missing :synopsis: to some modules 2009-08-29 03:46:58 -03:00
Pablo Hoffman 2f55d7932b replaced :ref: by :doc: links in doc index 2009-08-29 03:37:59 -03:00
Pablo Hoffman 924219dd5a - added reference documentation about scrapy-ctl.py script 
- yet another refactor to cmdline module
- removed help command (use -h to help now)

--HG--
rename : docs/experimental/scripts.rst => docs/topics/scrapy-ctl.rst
 2009-08-28 20:32:55 -03:00
Pablo Hoffman 566758eeb7 removed useless --restrict command line argument 2009-08-28 19:38:04 -03:00
Pablo Hoffman 046e63b1cc added --version command line option 2009-08-28 18:26:29 -03:00
Pablo Hoffman 18e9fd9771 some better code reusage 2009-08-28 18:07:35 -03:00
Pablo Hoffman 7dd43da9a5 removed unnecesary assertEqual's 2009-08-28 18:01:06 -03:00
Pablo Hoffman 83a2b580f7 renamed test_scrapy_ctl.py to test_commands.py 
--HG--
rename : scrapy/tests/test_scrapy_ctl.py => scrapy/tests/test_commands.py
 2009-08-28 17:58:50 -03:00
Pablo Hoffman ee40929df4 some line wrapping at 80 cols 2009-08-28 17:52:44 -03:00
Pablo Hoffman 3e7e321a07 sorted out import order and removed unused imports from previous changeset 2009-08-28 17:47:45 -03:00
Ismael Carnales 0b68d4bbef added tests for scrapy-ctl commands 2009-08-28 14:42:07 -03:00
Ismael Carnales 3616ccb479 changed SCRAPYSETTINGS_MODULE to SCRAPY_SETTINGS_MODULE 2009-08-28 11:20:05 -03:00
Pablo Hoffman 0bfa1a910c removed wrong docstring 2009-08-28 02:21:32 -03:00
Pablo Hoffman da20cf53f5 added compatibility with python 2.5 2009-08-27 20:05:11 -03:00
Pablo Hoffman 609aed4553 added inspect_response() function for inspecting responses from spiders 2009-08-27 19:33:33 -03:00
Pablo Hoffman ea4f16bae6 refactored scrapy shell implementation, dropping IPython dependency, and adding a new 'view' shortcut 2009-08-27 18:24:08 -03:00
Pablo Hoffman 13be33f0e7 added open_in_browser function to scrapy.utils.response 2009-08-27 18:20:51 -03:00
Pablo Hoffman 02de8cb6fa some other minor code cleanups for Settings class 2009-08-27 14:41:49 -03:00
Pablo Hoffman 90e6aefb4d some refactoring to Settings class 2009-08-27 14:08:48 -03:00
Pablo Hoffman 78481bf4d3 fixed bug when spider returns None 2009-08-27 12:10:31 -03:00
Pablo Hoffman ab0def3f8b made scrapy.command.cmdline module executable from command line 2009-08-27 11:49:35 -03:00
Pablo Hoffman 625137464f fixed bug in help command (thanks slav0nic for reporting) 2009-08-26 16:18:25 -03:00
Ismael Carnales 0e163a9816 added override field save test to DjangoItem 2009-08-26 11:58:40 -03:00
Ismael Carnales 62e03419b7 fixed bug in DjangoItem 2009-08-26 11:45:59 -03:00
Ismael Carnales 8efe6eb55e made DjangoItem a descendant of Item and its metaclass 2009-08-26 11:38:49 -03:00
Ismael Carnales fb39bca24f added djangoitem doc 2009-08-26 08:44:22 -03:00
Pablo Hoffman 65ab611fb0 more updates to HttpErrorMiddleware doc 2009-08-26 08:30:18 -03:00
Pablo Hoffman 44783a3a06 minor improvements to FAQ entry 2009-08-26 00:18:58 -03:00