RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,969 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

77 Commits

Author SHA1 Message Date
ajbmachon 5941413af2 Add lifecycle documentation to toctree 
Fixes sphinx warning about orphaned document by adding lifecycle.rst
to the Extending Scrapy section of the documentation index.
 2025-12-30 08:35:22 +01:00
sstegmanbsd 2d073a9c0d
Fix typo in logging documentation (#7109) 2025-10-20 15:52:46 +02:00
Andrey Rakhmatullin ce5a132f12
Run and fix linkcheck. (#6524) 2024-11-04 11:40:07 +01:00
Andrey Rakhmatullin f582246d7b More doc fixes. 2023-06-15 18:49:08 +04:00
Andrey Rakhmatullin 898e3045a1 Merge branch 'master' into addons 2023-06-14 15:37:08 +04:00
Andrey Rahmatullin 0f1112f3e2
Update docs/index.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2022-07-27 23:12:43 +05:00
Andrey Rahmatullin a0681fb811
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2022-07-22 18:30:38 +05:00
Rotzbua 1c7ed4f2e5
[doc] Remove incompatible web service project 
* Abandoned since 2017
* Not compatible with Python3
 2022-07-12 13:09:20 +02:00
Adrián Chaves b95c634b86 Document how to enforce Scrapy versions on Scrapy components 2022-03-17 22:23:25 +01:00
Adrián Chaves aa0306a167
Cover 2.6.0 in the release notes (#5399) 2022-03-01 12:16:37 +01:00
Gowtham Chowdary 3b42ccfeba
Add a link to Discord (#5422) 2022-02-16 21:33:56 +01:00
Andrey Rakhmatullin cd17c829cf Revert "Move documentation about avoiding bans into a topic of its own (#4039)" 
This reverts commit 2d2581c68f.
 2021-08-23 19:55:35 +05:00
Adrián Chaves 2d2581c68f
Move documentation about avoiding bans into a topic of its own (#4039) 2021-08-18 12:46:42 +02:00
Eugenio Lacuesta ddea6b7bfa
Scheduler: minimal interface, API docs (#3559) 2021-04-26 16:16:14 -03:00
Habeeb Shopeju f47b120e2b
Documentation of link extractor usage (#4775) 
* Added description when using link extractor outside crawlspiders and created reference documentation for scrapy.link.Link class

* Added link.rst to toctree

* Corrected spelling errors, moved docs to Link doctstring to use autoclass

* Moved link docs to link_extractors

* Update docs/topics/link-extractors.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update link.py

Improvements to URL description

* Update link.py

* Update link.py

Fixed line length Flake issue

* Update link.py

Fixed trailing whitespace

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-10-01 23:50:11 +05:00
Adrián Chaves 6aa0ba4553
Write release notes for Scrapy 2.0.0 (#4329) 2020-03-03 09:11:11 +01:00
Marc Hernandez Cabot 23a67cec27 fix first letter capitalization for Raring and Scrapy 2019-12-19 09:57:17 +01:00
Adrián Chaves 8a3b15eb91 Document how to select dynamically-loaded content 2019-06-24 16:06:21 +02:00
Adrián Chaves b1063d9b2c Use the description from README.rst on index.rst 2019-03-08 17:22:49 +01:00
Raphael Wuillemier af555cab23 Added general guide for developer tools instead of Firefox and Firebug-sections 2018-08-22 14:15:53 +02:00
Valdir Stumm Junior 80b160d0d7 include references to scrapy subreddit in the docs 2017-05-29 14:56:49 -03:00
Paul Tremberth 11fe3751cf DOC Remove "Ubuntu" section from sidebar/ToC 2016-11-16 11:55:09 +01:00
Elias Dorneles 31260cf02f mentions stackoverflow as help channel (fixes #2255) 2016-09-16 16:05:36 -03:00
orangain 4be4ef038e DOC: Add captions to toctrees which appear in sidebar 2015-12-12 16:21:00 +09:00
Jakob de Maeyer 2946b67414 Document add-ons 2015-11-11 21:08:19 +01:00
Pablo Hoffman b84b3585f2 remove empty experimental section from doc 2015-05-01 01:53:51 -03:00
Julia Medina d47a7edc65 Update Logging topic on docs 2015-04-22 17:24:40 -03:00
Pablo Hoffman 06e1ca9ef2 Merge pull request #1177 from Curita/remove-djangoitem 
Remove djangoitem
 2015-04-21 15:35:47 -03:00
Pablo Hoffman e4122cdd79 Merge pull request #1150 from eliasdorneles/docs-files-pipeline 
Documenting Files Pipeline together with Images Pipeline
 2015-04-21 15:28:47 -03:00
Julia Medina 6da091f162 Remove djangoitem topic from index 2015-04-20 20:19:28 -03:00
Elias Dorneles a1f3b3c7f0 images.rst -> media-pipeline.rst 2015-04-15 14:26:08 -03:00
Elias Dorneles dce48b8654 documenting Files Pipeline together with Images Pipeline 2015-04-11 13:57:55 -03:00
Pablo Hoffman 1a12922b77 improve scrapy deploy documentation 2015-04-09 17:39:01 -03:00
Pablo Hoffman d8184a7239 Merge pull request #1124 from rdowinton/deployment-doc 
Added deployment section covering scrapyd-deploy and shub
 2015-04-09 16:53:25 -03:00
Richard Dowinton 2d142d6401 Added deployment section covering scrapyd-deploy and shub 2015-03-31 12:17:31 +01:00
Pablo Hoffman bb4c922d85 Merge pull request #1081 from scrapy/dict-items 
Allow spiders to return dicts.
 2015-03-27 15:19:27 -03:00
Mikhail Korobov f16a33f34e DOC change structure of spider docs: 
* start with scrapy.Spider, then mention spider arguments,
  then describe generic spiders;
* change wording regarding start_urls/start_requests;
* show an example of start_requests vs start_urls;
* show an example of dicts as items;
* as defining Item is an optional step now, docs for Items are
  moved below Spider docs.
 2015-03-19 05:25:15 +05:00
Shadab Zafar 5a58d64131 Fix some redirection links in documentation 
Fixes #606
 2015-03-18 19:41:26 -03:00
Mikhail Korobov a07b4353d6 DOC reorder topics 
* duplicate "topics/commands" link is removed;
* Request/Response docs are moved to "Basic Concepts";
* settings docs are moved to "Basic Concepts";
* exceptions docs are moved to "Basic Concepts";
* "signals" and "exporters" docs are moved to "Extending Scrapy";
* "Reference" section is dropped because it is empty now.
 2015-01-22 22:58:10 +05:00
Mikhail Korobov f18ac02987 remove duplicated link extractors link 
Check http://doc.scrapy.org/en/latest/topics/link-extractors.html - two menu items are highlighted at the left.
 2013-12-28 05:40:10 +05:00
Pablo Hoffman 76087e336a add scrapy bench command for benchmarking, with documentation 2013-05-16 13:15:25 -03:00
Pablo Hoffman 1e2ee76df2 add documentation topics: Broad Crawls & Common Practies 2012-12-26 14:02:13 -02:00
Pablo Hoffman c380910b40 Merge pull request #167 from alexcepoi/sep-017 
Spider contracts (SEP-017)
 2012-09-28 13:57:07 -07:00
Alex Cepoi 11d29c7005 SEP-017 contracts: add tests and minor improvements 2012-09-21 00:12:46 +02:00
Pablo Hoffman b46b5a6ef0 Documented AutoThrottle extension and added to extensions available by 
default. Also deprecated concurrency and delay settings, in favour of
using the standard Scrapy ones.
 2012-09-20 18:52:57 -03:00
Pablo Hoffman a874964ad4 renamed 'XPath Selectors' title to just 'Selectors' 2012-09-13 15:24:44 -03:00
Alex Cepoi bf8dc61fb7 SEP-017 contracts: pretty-printing and docs 2012-09-10 23:17:27 +02:00
Pablo Hoffman 70f8e517a1 promoted DjangoItem to main contrib 2012-08-29 11:23:11 -03:00
Pablo Hoffman babfc6e79b Updated documentation after singleton removal changes. 
Also removed some unused code and made some minor additional
refactoring.
 2012-08-28 18:35:57 -03:00
Alexandru Cepoi f4faa19e31 added docs topic debugging spiders 2012-06-21 20:03:33 +02:00