RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,396 Commits 49 Branches 132 Tags 131 MiB
Commit Graph

57 Commits

Author SHA1 Message Date
Valdir Stumm Junior bfcc2395e3 add xpath playground support for docs 2017-06-14 10:51:53 -03:00
Valdir Stumm Junior 80b160d0d7 include references to scrapy subreddit in the docs 2017-05-29 14:56:49 -03:00
Paul Tremberth 11fe3751cf DOC Remove "Ubuntu" section from sidebar/ToC 2016-11-16 11:55:09 +01:00
Elias Dorneles 31260cf02f mentions stackoverflow as help channel (fixes #2255) 2016-09-16 16:05:36 -03:00
orangain 4be4ef038e DOC: Add captions to toctrees which appear in sidebar 2015-12-12 16:21:00 +09:00
Pablo Hoffman b84b3585f2 remove empty experimental section from doc 2015-05-01 01:53:51 -03:00
Julia Medina d47a7edc65 Update Logging topic on docs 2015-04-22 17:24:40 -03:00
Pablo Hoffman 06e1ca9ef2 Merge pull request #1177 from Curita/remove-djangoitem 
Remove djangoitem
 2015-04-21 15:35:47 -03:00
Pablo Hoffman e4122cdd79 Merge pull request #1150 from eliasdorneles/docs-files-pipeline 
Documenting Files Pipeline together with Images Pipeline
 2015-04-21 15:28:47 -03:00
Julia Medina 6da091f162 Remove djangoitem topic from index 2015-04-20 20:19:28 -03:00
Elias Dorneles a1f3b3c7f0 images.rst -> media-pipeline.rst 2015-04-15 14:26:08 -03:00
Elias Dorneles dce48b8654 documenting Files Pipeline together with Images Pipeline 2015-04-11 13:57:55 -03:00
Pablo Hoffman 1a12922b77 improve scrapy deploy documentation 2015-04-09 17:39:01 -03:00
Pablo Hoffman d8184a7239 Merge pull request #1124 from rdowinton/deployment-doc 
Added deployment section covering scrapyd-deploy and shub
 2015-04-09 16:53:25 -03:00
Richard Dowinton 2d142d6401 Added deployment section covering scrapyd-deploy and shub 2015-03-31 12:17:31 +01:00
Pablo Hoffman bb4c922d85 Merge pull request #1081 from scrapy/dict-items 
Allow spiders to return dicts.
 2015-03-27 15:19:27 -03:00
Mikhail Korobov f16a33f34e DOC change structure of spider docs: 
* start with scrapy.Spider, then mention spider arguments,
  then describe generic spiders;
* change wording regarding start_urls/start_requests;
* show an example of start_requests vs start_urls;
* show an example of dicts as items;
* as defining Item is an optional step now, docs for Items are
  moved below Spider docs.
 2015-03-19 05:25:15 +05:00
Shadab Zafar 5a58d64131 Fix some redirection links in documentation 
Fixes #606
 2015-03-18 19:41:26 -03:00
Mikhail Korobov a07b4353d6 DOC reorder topics 
* duplicate "topics/commands" link is removed;
* Request/Response docs are moved to "Basic Concepts";
* settings docs are moved to "Basic Concepts";
* exceptions docs are moved to "Basic Concepts";
* "signals" and "exporters" docs are moved to "Extending Scrapy";
* "Reference" section is dropped because it is empty now.
 2015-01-22 22:58:10 +05:00
Mikhail Korobov f18ac02987 remove duplicated link extractors link 
Check http://doc.scrapy.org/en/latest/topics/link-extractors.html - two menu items are highlighted at the left.
 2013-12-28 05:40:10 +05:00
Pablo Hoffman 76087e336a add scrapy bench command for benchmarking, with documentation 2013-05-16 13:15:25 -03:00
Pablo Hoffman 1e2ee76df2 add documentation topics: Broad Crawls & Common Practies 2012-12-26 14:02:13 -02:00
Pablo Hoffman c380910b40 Merge pull request #167 from alexcepoi/sep-017 
Spider contracts (SEP-017)
 2012-09-28 13:57:07 -07:00
Alex Cepoi 11d29c7005 SEP-017 contracts: add tests and minor improvements 2012-09-21 00:12:46 +02:00
Pablo Hoffman b46b5a6ef0 Documented AutoThrottle extension and added to extensions available by 
default. Also deprecated concurrency and delay settings, in favour of
using the standard Scrapy ones.
 2012-09-20 18:52:57 -03:00
Pablo Hoffman a874964ad4 renamed 'XPath Selectors' title to just 'Selectors' 2012-09-13 15:24:44 -03:00
Alex Cepoi bf8dc61fb7 SEP-017 contracts: pretty-printing and docs 2012-09-10 23:17:27 +02:00
Pablo Hoffman 70f8e517a1 promoted DjangoItem to main contrib 2012-08-29 11:23:11 -03:00
Pablo Hoffman babfc6e79b Updated documentation after singleton removal changes. 
Also removed some unused code and made some minor additional
refactoring.
 2012-08-28 18:35:57 -03:00
Alexandru Cepoi f4faa19e31 added docs topic debugging spiders 2012-06-21 20:03:33 +02:00
Pablo Hoffman 7cca916ed5 added release notes to official documentation, including all release notes since Scrapy 0.7 2012-04-11 15:53:23 -03:00
Pablo Hoffman 431441cb52 updated documentation to remove references to old issue tracker and mercurial repos 2011-09-25 13:06:24 -03:00
Pablo Hoffman 40f7075f11 added initial documentation about suspend and resume crawls 2011-09-02 13:12:27 -03:00
Pablo Hoffman bfda9ec319 added clarification about scrapy versioning including the recently adopted odd/even versioning scheme 
--HG--
rename : docs/api-stability.rst => docs/versioning.rst
 2011-07-12 19:53:23 -03:00
Pablo Hoffman cd85c12c33 Some Link extractor improvements: 
* added support for ignoring common file extensions that are not followed if
  they occur in links
* fixed link extractor documentation issues
* slighly improved performance of applying filters
* added link to link extractors doc from documentation index
 2011-05-18 12:32:34 -03:00
Pablo Hoffman bf73002428 removed googledir example, replaced by dirbot project on github. updated docs accordingly 2011-04-28 02:28:39 -03:00
Pablo Hoffman 37e9c5d78e Added new Scrapy service with support for: 
* multiple projects
* uploading scrapy projects as Python eggs
* scheduling spiders using a JSON API

Documentation is added along with the code.

Closes #218.

--HG--
rename : debian/scrapy-service.default => debian/scrapyd.default
rename : debian/scrapy-service.dirs => debian/scrapyd.dirs
rename : debian/scrapy-service.install => debian/scrapyd.install
rename : debian/scrapy-service.lintian-overrides => debian/scrapyd.lintian-overrides
rename : debian/scrapy-service.postinst => debian/scrapyd.postinst
rename : debian/scrapy-service.postrm => debian/scrapyd.postrm
rename : debian/scrapy-service.upstart => debian/scrapyd.upstart
rename : extras/scrapy.tac => extras/scrapyd.tac
 2010-09-03 15:54:42 -03:00
Pablo Hoffman e2ed27e4fd Added documentation for Ubuntu packages. Refs #211 2010-08-23 21:28:32 -03:00
Pablo Hoffman 94ead94bf6 Improved documentation of Scrapy command-line tool 
--HG--
rename : docs/topics/cmdline.rst => docs/topics/commands.rst
 2010-08-19 00:04:52 -03:00
Pablo Hoffman 34554da201 Deprecated scrapy-ctl.py command in favour of simpler "scrapy" command. Closes #199. Also updated documenation accordingly and added convenient scrapy.bat script for running from Windows. 
--HG--
rename : debian/scrapy-ctl.1 => debian/scrapy.1
rename : docs/topics/scrapy-ctl.rst => docs/topics/cmdline.rst
 2010-08-18 19:48:32 -03:00
Pablo Hoffman e741a807d2 Added new Feed exports extension with documentation and storage tests. Closes #197. 
Also deprecated File export pipeline (to be removed in Scrapy 0.11).

Still need to add tests for FeedExport main extension code.
 2010-08-17 14:27:48 -03:00
Pablo Hoffman 6a33d6c4d0 * Added Scrapy Web Service with documentation and tests. 
* Marked Web Console as deprecated.
* Removed Web Console documentation to discourage its use.
 2010-06-09 13:46:22 -03:00
Pablo Hoffman 076cdfd585 Added documentation about contributing to Scrapy 2010-05-17 20:10:46 -03:00
Pablo Hoffman 99a876754c Improved "What else?" section of "Scrapy at a glance" overview 2010-03-20 20:24:18 -03:00
Pablo Hoffman 9d50604d24 added |version| to documentation title 2009-12-12 16:51:59 -02:00
Pablo Hoffman bd481751d8 moved images pipeline documentation to stable doc 
--HG--
rename : docs/experimental/images.rst => docs/topics/images.rst
 2009-10-07 22:57:25 -02:00
Pablo Hoffman be0f2beef0 more cleanup to scheduler middelware doc, and permanentely moved to experimental doc 2009-09-11 13:27:31 -03:00
Pablo Hoffman f3240748cb changed link to scheduler middleware doc, now in experimental 2009-09-11 12:03:23 -03:00
Ismael Carnales 3998a0cb58 added more scheduler middleware documentation, and moved it to experimental 
--HG--
rename : docs/topics/scheduler-middleware.rst => docs/experimental/scheduler-middleware.rst
 2009-09-11 11:58:53 -03:00
Pablo Hoffman 8d006e9ea1 moved item exporters doc to stable doc 
--HG--
rename : docs/experimental/exporters.rst => docs/topics/exporters.rst
 2009-08-31 20:47:12 -03:00