RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,751 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

215 Commits

Author SHA1 Message Date
Ismael Carnales 5862ba7db7 modified doc to reflect the new spider callback return policy (lists not needed) 2009-09-22 11:25:40 -03:00
Ismael Carnales 802f918b69 removed obsolete doc static file 2009-09-22 11:25:38 -03:00
Pablo Hoffman 6e93872955 updated installation guide for using releases 2009-09-17 11:06:55 -03:00
Pablo Hoffman 132557dd14 some deployment changes in preparation for the 0.7.0 release candidate 2009-09-16 22:40:36 -03:00
Ismael Carnales fd41f06056 added doc on how to enable an Item Pipeline component 2009-09-16 14:19:16 -03:00
Ismael Carnales 404e7e09d7 changed spider doc references in BaseSpider class 2009-09-16 14:10:11 -03:00
Daniel Grana 062730cbd8 fix csv exporter documentation 2009-09-16 00:17:50 -03:00
Pablo Hoffman 56b292e057 XmlItemExporter: added built-in support for exporting multi-valued fields (for convenience) 2009-09-14 22:05:52 -03:00
Pablo Hoffman e8960bf616 added runspider command to run spiders directly, without having to create a project 2009-09-14 22:05:14 -03:00
Pablo Hoffman 99467d4e6e Changed (unstable) scheduler middleware API to receive spider (instead of domain) in enqueue_request method 2009-09-13 20:51:43 -03:00
Pablo Hoffman 921fc4f3bf Big Scrapy core refactoring to pass around spider references instead of domains. 
This is to avoid accessing the scrapy.spider.spiders singleton for "resolving"
spiders, which is considered an "evil" practice because it ties us to the
singleton model for the spider resolver, which is a bad thing.

This change will also work as the foundation for the API cleaning that we'll
perform for 0.8. We decided to introduce this change now to have a more common
basecode between 0.7 and 0.8, which will allow us to better support 0.7 until
0.8 is released.

However, this change doesn't modify the stable/documented API, nor does it
change the core logic. Those changes will land on the 0.8 branch, after 0.7 is
released.

--HG--
rename : scrapy/contrib/domainsch.py => scrapy/contrib/spiderscheduler.py
 2009-09-12 14:34:18 -03:00
Pablo Hoffman 8d49dc2fb5 changed IMAGES_THUMBS setting to a dict instead of a list of tuples, and more improvements to images pipeline doc 2009-09-11 17:36:00 -03:00
Pablo Hoffman e20f766792 fixed some typos 2009-09-11 16:55:37 -03:00
Pablo Hoffman c2fe350f72 more changes to images pipeline doc 2009-09-11 16:53:36 -03:00
Ismael Carnales ada46a2dbb styled imagesp doc 2009-09-11 15:30:46 -03:00
Pablo Hoffman be0f2beef0 more cleanup to scheduler middelware doc, and permanentely moved to experimental doc 2009-09-11 13:27:31 -03:00
Pablo Hoffman 0af052b68f removed confusing title 2009-09-11 12:19:18 -03:00
Pablo Hoffman f3240748cb changed link to scheduler middleware doc, now in experimental 2009-09-11 12:03:23 -03:00
Ismael Carnales 3998a0cb58 added more scheduler middleware documentation, and moved it to experimental 
--HG--
rename : docs/topics/scheduler-middleware.rst => docs/experimental/scheduler-middleware.rst
 2009-09-11 11:58:53 -03:00
Pablo Hoffman d242a20573 updated images pipeline doc 2009-09-11 11:47:12 -03:00
Pablo Hoffman f1bb8dc2a3 first cleanup of spider manager api 
- removed asdict() and reload() methods
- added list() method
- removed default spider
 2009-09-10 19:06:46 -03:00
Pablo Hoffman f85813cd94 added FAQ entry about scrapy recipes and community spiders 2009-09-10 18:32:50 -03:00
Pablo Hoffman 269724a2b7 added Debugger extension, removed StackTraceDump from extensions available by default 2009-09-08 22:32:17 -03:00
Ismael Carnales 4ddfa9a2a3 stlyed downloaded middleware doc 2009-09-07 12:18:57 -03:00
Ismael Carnales e3df11e5bb added module directive to spidermw documentation 2009-09-07 12:03:24 -03:00
Pablo Hoffman 827aa19c6e removed obsolete scrapy.utils.db module 2009-09-04 17:38:14 -03:00
Pablo Hoffman 861a803cc3 removed obsolete RestrictMiddleware 2009-09-04 17:22:56 -03:00
Ismael Carnales 7e2587169b added missing middleware docs 2009-09-04 12:39:02 -03:00
Pablo Hoffman aefb94063a more updates to spider middleware doc 2009-09-04 13:46:04 -03:00
Pablo Hoffman d04640be5c some improvements to spider middleware doc 2009-09-04 13:29:16 -03:00
Pablo Hoffman 96bb223c13 removed (pretty useless) DebugMiddleware 2009-09-04 12:59:58 -03:00
Pablo Hoffman 8a715701ec fixed another doc typo 2009-09-03 14:31:00 -03:00
Ismael Carnales 3c1bb7bc40 fixed typo in djangoitems doc (thanks anibal) 2009-09-03 11:23:25 -03:00
Daniel Grana 0e7b2a6da5 write header line by default when using csv exporter 
--HG--
extra : rebase_source : 2d2d7153dde5e3f77e682e16d2e4408f732f234e
 2009-09-03 13:58:39 -03:00
Pablo Hoffman 596d2c4479 moved CoreStats extension to scrapy.contrib.corestats 
--HG--
rename : scrapy/stats/corestats.py => scrapy/contrib/corestats.py
 2009-09-01 23:00:49 -03:00
Pablo Hoffman 6a50af05d7 removed useless SpiderReloader extension 2009-09-01 22:49:15 -03:00
Pablo Hoffman 79851aefa6 moved SpiderProfiler extension to scrapy.contrib_exp and removed references from documentation 
--HG--
rename : scrapy/contrib/spider/profiler.py => scrapy/contrib_exp/spiderprofiler.py
 2009-09-01 22:38:37 -03:00
Pablo Hoffman d3c51fd6f2 improved images pipeline documentation 2009-09-01 21:07:47 -03:00
Pablo Hoffman 18fd635124 another doc typo 2009-09-01 12:52:40 -03:00
Pablo Hoffman 538cc9803a fixed doc typo 2009-09-01 12:47:53 -03:00
Pablo Hoffman df0e1f005f exporters doc: fixed example and some typos 2009-09-01 08:56:54 -03:00
Pablo Hoffman ac8f46ce9e added File Export Pipeline reference to Exporters doc 2009-08-31 21:01:35 -03:00
Pablo Hoffman 8d006e9ea1 moved item exporters doc to stable doc 
--HG--
rename : docs/experimental/exporters.rst => docs/topics/exporters.rst
 2009-08-31 20:47:12 -03:00
Pablo Hoffman 0b152c99b5 added File Export Pipeline, a wrapper to use Item Exporters as Item Pipelines 2009-08-31 20:40:41 -03:00
Pablo Hoffman 8fab524978 moved engine.getstatus() method to scrapy.utils.engine function, to leave reporting logic out of engine code. added est() shortcut to telnet console 2009-08-31 12:44:32 -03:00
Pablo Hoffman 884f0c878f Stats collectin: fixed race condition between stats persistance and population of stats on domain close 2009-08-29 19:44:13 -03:00
Pablo Hoffman 895c70e036 doc: fixed some links to scrapy-ctl topic 2009-08-29 18:23:55 -03:00
Pablo Hoffman 64572124e0 added doc about SCRAPY_SETTINGS_MODULE 2009-08-29 18:20:13 -03:00
Pablo Hoffman 9dc7a3a373 some minor adjustments to logging doc 2009-08-29 18:10:28 -03:00
Pablo Hoffman 60cbf24c89 more cleanups to startproject and project templates 
--HG--
rename : scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/scrapy-ctl.py
 2009-08-29 04:29:47 -03:00