RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,814 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

69 Commits

Author SHA1 Message Date
James McKinney 3c5668d0db
docs: Clarify there's one extension instance per spider 2021-03-01 22:00:33 -05:00
OfirD1 1a597d5e3d
moved the sentence about processing pending requests when a spider is closed onto a generic note. 2020-10-06 21:54:42 +03:00
Adrián Chaves 3f0a677c04
Cover version directive usage in the documentation policy (#4310) 
* Cover version directives in the documentation policy

* Remove version directives in preparation for Scrapy 2.0

* Update the policy based on the deprecation policy

* Only remove version directives after 3 years
 2020-08-27 23:56:58 +05:00
Aditya 80c69d68ad [docs] refactor python docs links using intersphinx 2020-03-24 18:35:42 +05:30
Aditya f37b1bdc56 [docs] update redirect links to python3 2020-03-20 05:31:41 +05:30
Aditya ca08e04198 [docs] update redirect links python2 -> python3 2020-03-20 02:31:35 +05:30
elacuesta dfbe1d9507
Remove object base class (#4430) 2020-03-16 20:12:46 +01:00
Akshay Sharma 182445f9d9
Fix a spelling error: ie. → i.e. (#4338) 2020-02-18 17:58:31 +01:00
Ammar Najjar 68a7d05ed8
docs: use __init__ method instead of constructor 
Issue #4086
 2019-10-21 15:42:24 +02:00
Adrián Chaves 2828cb769f Provide complete API documentation coverage of scrapy.extensions 2019-08-30 14:29:15 +02:00
Adrián Chaves 75d6f56c8a Switch from ` to `` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Aditya 9cdf34b7c7 Link "Debugging in Python" article to its new location 
Reference: https://web.archive.org/web/20170203104051/http://www.ferg.org/papers/debugging_in_python.html
 2017-10-10 22:49:22 +05:30
Paul Tremberth 39ad0d0bdd Fix setting name reference 2017-06-06 10:48:30 +02:00
Paul Tremberth e7061f7a41 Reformat a bit 2017-06-06 10:47:43 +02:00
Chuan Jin 3f8542eb56 Update extensions.rst 
#2759
 2017-06-05 23:31:37 +02:00
Eli Atzaba b74b98fa3e cleanup: removed unused MEMUSAGE_REPORT 
Signed-off-by: Eli Atzaba <eliat123@gmail.com>
 2017-05-16 13:59:58 +03:00
nyov 5876b9aa30 Update documentation links 2016-03-03 16:28:33 +00:00
Julia Medina 48539af163 Merge pull request #1601 from mvj3/minor-documentation-fixes 
Fixed minor grammar issues.
 2015-11-16 14:58:07 -03:00
David Chen 0025d5a943 Fixed minor grammar issues. 2015-11-16 07:30:17 +08:00
Jakob de Maeyer e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Julia Medina 72eeead6db Merge pull request #1524 from Digenis/relocate_telnet_console 
[MRG+1] Relocate telnetconsole to extensions/
 2015-10-30 12:10:38 -03:00
Jakob de Maeyer 26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Νικόλαος-Διγενής Καραγιάννης d523c75f5e Relocate telnetconsole to extensions/ 2015-10-05 21:16:04 +03:00
Chris Nilsson eae25a04d9 Added MEMUSAGE_CHECK_INTERVAL_SECONDS to Memory usage extension options. 
Kept the default as it was, at 60.0 seconds. But added a setting to
allow this to be changed as desired.
 2015-06-06 00:39:14 +10:00
Julia Medina 9a3e3ba505 Move scrapy/contrib remaining top-level files to scrapy/extensions 2015-04-29 21:27:19 -03:00
Julia Medina bd0b639b21 Fix logging usage across docs 2015-04-22 17:24:41 -03:00
Shadab Zafar 5a58d64131 Fix some redirection links in documentation 
Fixes #606
 2015-03-18 19:41:26 -03:00
Sudhanshu Shekhar e42a1ac1a1 Reset items_scraped instead of item_count 
items_scraped is the counter that needs to be reset each time we have scraped a specific number of items in the code instead of item_count (which represents the specific number of items needed before a message is logged). Updating the source code to reflect this.
Removed some irrelevant words from the log message.
Signed-off-by: Sudhanshu Shekhar <sudshekhar02@gmail.com>
 2015-01-30 23:13:06 +05:30
Lazar-T 38dcf50cd6 comma instead of fullstop 2014-10-25 09:19:50 +06:00
Mikhail Korobov e435b3e3a3 DOC simplify extension docs 2014-09-21 00:19:24 +06:00
Daniel Graña a9292cfab7 jsonrpc webservice moved to https://github.com/scrapy/scrapy-jsonrpc repository 2014-08-15 23:28:13 -03:00
Daniel Graña bf37f78572 Drop libxml2 selectors backend 2013-10-11 18:02:35 -02:00
Pablo Hoffman 227a1d666b add doc about disabling an extension. refs #132 2013-01-07 13:16:19 -02:00
Ilya Baryshev 097aea04a4 Fixed docs typo in SpiderOpenCloseLogging example 2012-11-10 12:24:53 +04:00
Pablo Hoffman babfc6e79b Updated documentation after singleton removal changes. 
Also removed some unused code and made some minor additional
refactoring.
 2012-08-28 18:35:57 -03:00
Pablo Hoffman 7fe7c3f3b1 MemoryUsage extension: close the spiders (instead of stopping the engine) when the limit is exceeded, providing a descriptive reason for the close. Also fixed default value of MEMUSAGE_ENABLED setting to match the documentation. 2012-02-23 17:05:06 -02:00
Pablo Hoffman 7b8942a648 updated StackTraceDump extension doc 2012-02-16 15:14:17 -02:00
Pablo Hoffman 91dc46539f added LogStats extension for periodically logging basic stats (like crawled pages and scraped items) 2011-06-14 00:50:05 -03:00
Pablo Hoffman 841e9913db renamed CLOSESPIDER_ITEMPASSED setting to CLOSESPIDER_ITEMCOUNT, to follow the refactoring done in r2630 2011-06-13 16:58:51 -03:00
Pablo Hoffman 474cba512c simplified MemoryDebugger extension to use stats for dumping memory debugging info 2011-06-06 03:13:28 -03:00
Pablo Hoffman d72d3f4607 stack trace dump extension: also dump engine status, and support triggering it with SIGQUIT, besides SIGUSR2 2011-05-20 03:25:00 -03:00
Pablo Hoffman 2557777c39 Updated doc referring to HTTP cache middleware 2010-11-24 13:27:44 -02:00
Pablo Hoffman 7d8f922df9 Added documentation for CLOSESPIDER_ERRORCOUNT setting. Refs #254 2010-10-18 22:36:30 -02:00
Pablo Hoffman 7826869cb2 Added missing colon 2010-09-28 16:44:53 -03:00
Martin Santos 0bf9e4627c added support to CloseSpider extension, for close the spider after N pages have been crawled. Using the CLOSESPIDER_PAGECOUNT setting. closes #253 2010-09-28 16:29:37 -03:00
Pablo Hoffman 9fccc11363 Moved scrapy.extension.extensions singleton to a "extensions" attribute of the scrapy.project.crawler singleton. Refs #189 2010-08-22 02:15:11 -03:00
Pablo Hoffman 784722774b moved scrapy.core.signals to scrapy.signals, keeping backwards compatibility 2010-08-10 17:40:53 -03:00
Pablo Hoffman c359a34d7d moved scrapy.core.exceptions to scrapy.exceptions, keeping backwards compatibility 
--HG--
rename : scrapy/core/exceptions.py => scrapy/exceptions.py
 2010-08-10 17:36:48 -03:00
Pablo Hoffman 6a33d6c4d0 * Added Scrapy Web Service with documentation and tests. 
* Marked Web Console as deprecated.
* Removed Web Console documentation to discourage its use.
 2010-06-09 13:46:22 -03:00
Daniel Grana 658e6f15e9 Automated merge with ssh://hg.scrapy.org/scrapy-0.8 2010-04-18 23:44:59 -03:00