RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,647 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

1647 Commits

Author SHA1 Message Date
Pablo Hoffman db5420f8d5 reload spider modules silently 2009-09-02 00:48:29 -03:00
Pablo Hoffman 596d2c4479 moved CoreStats extension to scrapy.contrib.corestats 
--HG--
rename : scrapy/stats/corestats.py => scrapy/contrib/corestats.py
 2009-09-01 23:00:49 -03:00
Pablo Hoffman 5ce2b66c56 added proper recycling of spider resources to spider manager 2009-09-01 22:53:04 -03:00
Pablo Hoffman 34d559003d removed empty scrapy.contrib.spider module 2009-09-01 22:50:36 -03:00
Pablo Hoffman 6a50af05d7 removed useless SpiderReloader extension 2009-09-01 22:49:15 -03:00
Pablo Hoffman 79851aefa6 moved SpiderProfiler extension to scrapy.contrib_exp and removed references from documentation 
--HG--
rename : scrapy/contrib/spider/profiler.py => scrapy/contrib_exp/spiderprofiler.py
 2009-09-01 22:38:37 -03:00
Pablo Hoffman d3c51fd6f2 improved images pipeline documentation 2009-09-01 21:07:47 -03:00
Pablo Hoffman 18fd635124 another doc typo 2009-09-01 12:52:40 -03:00
Pablo Hoffman 538cc9803a fixed doc typo 2009-09-01 12:47:53 -03:00
Pablo Hoffman e1b33ef165 added missing module from previous commit 2009-09-01 09:04:41 -03:00
Pablo Hoffman df0e1f005f exporters doc: fixed example and some typos 2009-09-01 08:56:54 -03:00
Pablo Hoffman f68dedb284 commented out code that raises sporadically 2009-08-31 22:48:01 -03:00
Pablo Hoffman ac8f46ce9e added File Export Pipeline reference to Exporters doc 2009-08-31 21:01:35 -03:00
Pablo Hoffman 8d006e9ea1 moved item exporters doc to stable doc 
--HG--
rename : docs/experimental/exporters.rst => docs/topics/exporters.rst
 2009-08-31 20:47:12 -03:00
Pablo Hoffman 0b152c99b5 added File Export Pipeline, a wrapper to use Item Exporters as Item Pipelines 2009-08-31 20:40:41 -03:00
Pablo Hoffman 3c6deaeffc Some additional improvements to scrapy.command.cmdline logic: 
- calling scrapymanager.configure() for all commands
- finally added some unittests to check cmdline behaviour!
 2009-08-31 18:53:55 -03:00
Pablo Hoffman 6c58d06f0f wrapped some big lines 2009-08-31 18:50:20 -03:00
Pablo Hoffman b6c1392f9b added 'settings' command for querying scrapy settings 2009-08-31 18:43:51 -03:00
Pablo Hoffman 60299c49ca raise NotConfigured in web/telnetconsole when disabled 2009-08-31 13:42:44 -03:00
Pablo Hoffman 8fab524978 moved engine.getstatus() method to scrapy.utils.engine function, to leave reporting logic out of engine code. added est() shortcut to telnet console 2009-08-31 12:44:32 -03:00
Pablo Hoffman dd80f6acdf MemoryUsage: changed .virtual property to methodd. SpiderProfiler: removed dependency on MemoryUsage extension 2009-08-31 12:14:30 -03:00
Pablo Hoffman e8e760c974 more simplifications to scrapy engine: removed addtasks method 2009-08-31 12:04:01 -03:00
Pablo Hoffman 1dd996f0b6 added missing import 2009-08-31 11:17:28 -03:00
Pablo Hoffman 8f013bf227 imported ismael patch for depreacting old SCRAPYSETTINGS_MODULE envvar 2009-08-31 10:17:12 -03:00
Pablo Hoffman 268acc30f5 merge with trunk 2009-08-31 09:44:55 -03:00
Pablo Hoffman 4f00c12a0a removed unneeded logic from engine 2009-08-31 09:44:31 -03:00
Daniel Grana 91df6fcc7c test genspider in a single testcase 2009-08-31 08:58:29 -03:00
Pablo Hoffman bcf81cd3c4 minor simplification to how default settings are loaded 2009-08-31 07:36:09 -03:00
Pablo Hoffman 062761166c simpledb collector: moved to_sdb_value function to utils.simpledb, and added unittests 2009-08-30 12:37:30 -03:00
Daniel Grana 1f50c24b6c one minus command testing line 2009-08-30 01:55:39 -03:00
Daniel Grana 8ef2cf0add more command testing simplifications 2009-08-30 01:51:34 -03:00
Daniel Grana 719fc2e7df sdb stats: extend type serializations, allow timestamp to be other than now 2009-08-30 01:08:23 -03:00
Daniel Grana 7cb546ba28 use explicit relative import on djangoitem tests 
--HG--
extra : rebase_source : f4f72679106bd9f72ddd9ee7d1f27209248f59be
 2009-08-29 21:04:32 -03:00
Pablo Hoffman 884f0c878f Stats collectin: fixed race condition between stats persistance and population of stats on domain close 2009-08-29 19:44:13 -03:00
Pablo Hoffman 895c70e036 doc: fixed some links to scrapy-ctl topic 2009-08-29 18:23:55 -03:00
Pablo Hoffman 64572124e0 added doc about SCRAPY_SETTINGS_MODULE 2009-08-29 18:20:13 -03:00
Pablo Hoffman 9dc7a3a373 some minor adjustments to logging doc 2009-08-29 18:10:28 -03:00
Pablo Hoffman 91ee71222f some minor changes to test_commands.py 2009-08-29 05:35:22 -03:00
Pablo Hoffman 5ba466a6e1 fixed unnapropiate handling of missing django module in djangitem tests 2009-08-29 05:03:40 -03:00
Pablo Hoffman 6e028fd91b fixed JsonLinesItemExporterTest 2009-08-29 04:59:51 -03:00
Pablo Hoffman 60cbf24c89 more cleanups to startproject and project templates 
--HG--
rename : scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/scrapy-ctl.py
 2009-08-29 04:29:47 -03:00
Pablo Hoffman c87216f02b doc: added missing :synopsis: to some modules 2009-08-29 03:46:58 -03:00
Pablo Hoffman 2f55d7932b replaced :ref: by :doc: links in doc index 2009-08-29 03:37:59 -03:00
Pablo Hoffman 924219dd5a - added reference documentation about scrapy-ctl.py script 
- yet another refactor to cmdline module
- removed help command (use -h to help now)

--HG--
rename : docs/experimental/scripts.rst => docs/topics/scrapy-ctl.rst
 2009-08-28 20:32:55 -03:00
Pablo Hoffman 566758eeb7 removed useless --restrict command line argument 2009-08-28 19:38:04 -03:00
Pablo Hoffman 046e63b1cc added --version command line option 2009-08-28 18:26:29 -03:00
Pablo Hoffman 18e9fd9771 some better code reusage 2009-08-28 18:07:35 -03:00
Pablo Hoffman 7dd43da9a5 removed unnecesary assertEqual's 2009-08-28 18:01:06 -03:00
Pablo Hoffman 83a2b580f7 renamed test_scrapy_ctl.py to test_commands.py 
--HG--
rename : scrapy/tests/test_scrapy_ctl.py => scrapy/tests/test_commands.py
 2009-08-28 17:58:50 -03:00
Pablo Hoffman ee40929df4 some line wrapping at 80 cols 2009-08-28 17:52:44 -03:00