RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,809 Commits 49 Branches 132 Tags 145 MiB
Commit Graph

1655 Commits

Author SHA1 Message Date
Lucy Wang 57b0e6b695 improve document about functions as processors 2018-04-19 13:35:46 +08:00
rhoboro 6ef6585b5a update docs 2018-04-13 19:06:29 +09:00
rhoboro 464973489e Using bucket's default object ACL 2018-04-13 12:06:39 +09:00
grammy-jiang cb76b88331 fix a mistake in topic spider-middleware.rst 2018-04-04 05:56:05 -04:00
rhoboro 74a9c65290 update docs for support gcs acl 2018-04-03 18:20:37 +09:00
Steven Almeroth c6d20bdd82 Doc: update wording for COOKIES_ENABLED 2018-03-27 16:21:07 -04:00
Anjali Jain bbc2a3569f further edited 2018-02-16 23:33:10 +05:30
Anjali Jain 6954da1366
Updated contributing.rst 
Rectified grammatical errors
 2018-02-15 23:27:40 +05:30
Mikhail Korobov dc0304fde1 fix docs building with recent sphinx: don't use deprecated sphinx options and imports 2018-02-13 19:47:41 +05:00
Mikhail Korobov 786144e0c7
Merge pull request #3058 from jesuslosada/fix-link 
Fix link in news.rst
 2018-01-11 01:33:38 +05:00
Jesús Losada a0836b8fd9 Fix link in news.rst 2018-01-01 15:59:38 +00:00
Daniel Graña 2dee191374
Merge branch 'master' into extending-s3-files-store 2017-12-31 16:44:38 -03:00
Mikhail Korobov d07fe11981 set release date 2017-12-30 02:09:41 +05:00
Mikhail Korobov c107059ef8 DOC fix rst syntax 2017-12-29 07:07:43 +05:00
Mikhail Korobov d4e5671d07 make release docs more readable, add highlights 2017-12-29 07:06:00 +05:00
Mikhail Korobov 45b0e1a0e4 DOC draft 1.5 release notes 2017-12-28 07:33:43 +05:00
Paul Tremberth 461f9daff5 Update release notes for upcoming 1.4.1 version 2017-12-28 05:50:18 +05:00
Raphael a1cc5a63d3 Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999) (#3030) 
Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999)
 2017-12-27 17:54:17 -03:00
Konstantin Lopuhin bb1f311891 Add PyPy3 support to faq and install doc 2017-12-25 15:46:05 +03:00
Konstantin Lopuhin ea41114cf0 Mention PyPy support, add PyPy to install docs 2017-12-25 12:35:04 +03:00
Hugo cbcf80b98f
Fix typo 
[CI skip]
 2017-12-20 17:34:13 +02:00
Hugo 44623687ab Drop support for EOL Python 3.3 2017-12-19 17:59:05 +02:00
Mikhail Korobov 9aa9dd8d45 DOC mention an easier way to track pull requests locally. 
Thanks @eliasdorneles!
 2017-12-12 19:17:00 +05:00
Mikhail Korobov f716843a66 DOC update "Contributing" docs: 
* suggest Stack Overflow for Scrapy usage questions;
* encourage users to submit test-only pull requests with reproducable examples;
* encourage users to pick up stalled pull requests;
* we don't use AUTHORS file as a main acknowledgement source;
* suggest using Sphinx autodocs extension
 2017-12-12 16:19:43 +05:00
Daniel Graña 3cf0332ec3
Merge pull request #2957 from ScrapingLab/add_meta_json_to_parse_command 
[MRG+1] Scrapy Command: add --meta/-m to the "parse" command to pass additional meta data into the request
 2017-11-29 16:26:48 -03:00
IAlwaysBeCoding 6af323d7c8
Fix spelling mistake on scrapy parse command docs 
Fixed spelling mistake from "will be pass" to "will be passed"
 2017-11-26 00:24:52 +01:00
Jesse Bakker 0b14cb44aa Added from_crawler to middleware docs 2017-11-23 15:25:43 +01:00
IAlwaysBeCoding 846fd83512 removed commented out code, wrapped line to pep-8 and removed backlashes 2017-11-11 18:30:01 -05:00
Daniel Graña b8870ee8a1
Merge pull request #2989 from colinmorris/ItemExporterDocsExample 
[MRG+1] Revise/modernize item exporter example in docs
 2017-11-03 13:15:31 -02:00
colinmorris 23e571e860 fix issues identified in review 2017-10-31 18:08:47 -04:00
colinmorris 8a7552370d revise/modernize item exporter example in docs 2017-10-31 17:14:53 -04:00
Aditya df7e0a4315 Use https link in default user agent 2017-10-28 23:37:44 +05:30
Aditya 9d9d83a8c3 Use https links wherever possible 2017-10-28 16:24:40 +05:30
Aditya 9dd680d5c9 Use https for external links wherever possible in docs 2017-10-26 23:33:45 +05:30
Mikhail Korobov 98fb03e8bd Merge pull request #2958 from codeaditya/update-links 
Link "Debugging in Python" article to its new location
 2017-10-26 17:01:47 +05:00
Mikhail Korobov 496fc60b25 Merge pull request #2963 from djunzu/mention_request_meta_depth_on_depthmiddleware_doc 
Add note about request.meta['depth'] in DepthMiddleware
 2017-10-26 16:15:36 +05:00
Mikhail Korobov 73c40eb324 Merge pull request #2964 from weldon0405/patch-1 
Update tutorial.rst startproject files
 2017-10-26 16:05:31 +05:00
Weldon Malbrough 95815d27e8 updated file structure to include middlewares.py 2017-10-25 23:16:30 -04:00
Weldon Malbrough 169dc2860e Update tutorial.rst startproject files 
Added middlewares.py to accurately reflect the file structure created by "scrapy startproject tutorial"
 2017-10-16 22:46:32 -04:00
djunzu 8288f78a39 Add note about request.meta['depth'] in DepthMiddleware 2017-10-16 21:34:37 -02:00
Aditya 9cdf34b7c7 Link "Debugging in Python" article to its new location 
Reference: https://web.archive.org/web/20170203104051/http://www.ferg.org/papers/debugging_in_python.html
 2017-10-10 22:49:22 +05:30
Paul Tremberth 0d8a33fddc Update docs 2017-10-05 10:16:42 +02:00
Daniel Graña 5fac2d7b90 Merge pull request #2923 from rhoboro/fixes-685 
[MRG+2] Fixes #685 FilesPipeline support for Google Cloud Storage.
 2017-10-02 13:52:20 -03:00
Mikhail Korobov 92fa74fb43 Merge pull request #2922 from stav/doc.response-body 
[Doc] Update Response.body type
 2017-10-02 10:26:54 +02:00
Mikhail Korobov 3431870b53 Merge pull request #2924 from superyyrrzz/patch-1 
minor fix typo
 2017-10-02 10:25:40 +02:00
rhoboro d4555b2bcc update docs for supporting google cloud storage 2017-09-29 13:04:17 +09:00
Renze Yu 088b80d41a minor fix typo 2017-09-13 23:29:22 +08:00
Steven Almeroth 3637b75a67 [Doc] Update Response.body type 2017-09-12 17:41:47 -04:00
cclauss b8fabeed86 ur'string' not needed in Py 2, syntax error in Py3 
This instance was missed in #2909 --> ur'Scrapy developers' --> u'Scrapy developers'
 2017-09-04 14:29:23 +02:00
cclauss b702236082 ur'string' not needed in Py 2, syntax error in Py3 
Convert `u'(.*)\:\d+\:\s\[(.*)\]\s(?:(.*)\sto\s(.*)|(.*))'`--> `u'(.*)\:\d+\:\s\[(.*)\]\s(?:(.*)\sto\s(.*)|(.*))'`to be compatible with both Python 2 and Python 3.  See #2891
 2017-09-01 11:56:09 +02:00
cclauss 9f16f040b6 ur'string' not needed in Py 2, syntax error in Py3 
Convert `ur'Scrapy Documentation'`--> `u'Scrapy Documentation'`to be compatible with both Python 2 and Python 3.  See #2891
 2017-09-01 11:53:59 +02:00
Pablo Hoffman a429d78019 update scrapinghub.com urls to use https 2017-08-24 16:03:36 -03:00
david watson 71d5b7d75a fix typo (#2867) 2017-08-01 14:49:22 -03:00
simik-ru a65fec050a Small fix in description of startproject arguments 2017-07-31 13:03:49 +02:00
Daniel Graña b6d036ede9 Merge pull request #2837 from dguo/patch-2 
Fix a typo in the Items documentation
 2017-07-24 15:55:30 -03:00
Daniel Graña 6de106e7a8 Merge pull request #2763 from scrapy/dataloss-typo 
DOC fixed rst syntax in DOWNLOAD_FAIL_ON_DATALOSS docs
 2017-07-24 15:48:03 -03:00
Daniel Graña d1e948cef8 Merge pull request #2791 from starrify/doc-DontCloseSpider 
[MRG+1] Added doc for `scrapy.exceptions.DontCloseSpider`. Also fixes inaccurate doc for `scrapy.signals.spider_idle`.
 2017-07-24 15:29:38 -03:00
Daniel Graña 8e0b640d2b Merge pull request #2826 from dguo/patch-1 
[MRG+1] Tweak the CSVFeedSpider documentation
 2017-07-24 15:28:15 -03:00
Danny Guo 26c488970c Fix a typo in the Items documentation 2017-07-18 19:56:51 -05:00
Claus Conrad 18b96dd82a Spelling mistake 2017-07-15 11:31:09 +02:00
Danny Guo dedc4a8b8f Tweak the CSVFeedSpider documentation 2017-07-13 22:58:10 -05:00
Pengyu CHEN f712513ed7 Added doc for `scrapy.exceptions.DontCloseSpider`. Also fixes inaccurate doc for `scrapy.signals.spider_idle`. 2017-06-15 11:09:17 +08:00
Mikhail Korobov 5aebdac45d Merge pull request #2781 from crasker/patch-1 
use suggest method instead of DEPRECATED one
 2017-06-14 03:29:45 +05:00
Casker ae679f6499 Create item-pipeline.rst 2017-06-09 16:12:20 +08:00
Paul Tremberth 39ad0d0bdd Fix setting name reference 2017-06-06 10:48:30 +02:00
Paul Tremberth e7061f7a41 Reformat a bit 2017-06-06 10:47:43 +02:00
Chuan Jin 3f8542eb56 Update extensions.rst 
#2759
 2017-06-05 23:31:37 +02:00
Paul Tremberth fad6b70d92 Use https:// for readthedocs links 2017-06-01 16:41:52 +02:00
Mikhail Korobov 083880888b DOC fixed rst syntax in DOWNLOAD_FAIL_ON_DATALOSS docs 2017-05-30 00:44:11 +05:00
Valdir Stumm Junior 80b160d0d7 include references to scrapy subreddit in the docs 2017-05-29 14:56:49 -03:00
Kurt Peek af2963d0eb Update autothrottle.rst 
Added missing bullet point for the AUTOTHROTTLE_TARGET_CONCURRENCY setting.
 2017-05-24 19:59:36 +05:00
Paul Tremberth fc2846d637 Set release date for v1.4.0 2017-05-18 22:59:46 +02:00
Mikhail Korobov 76e5b0f65c DOC 1.4 deprecations and backwards incompatible changes, add recent commits to news. 2017-05-18 22:37:15 +02:00
Mikhail Korobov edcde7a2cf DOC tweak release notes: promote response.follow, mention logging/stats changes 2017-05-18 22:37:15 +02:00
Paul Tremberth a3d3cd4cb7 Update with latest merges 2017-05-18 22:37:15 +02:00
Paul Tremberth 432668acf7 Mention implementation of #667 2017-05-18 22:37:15 +02:00
Paul Tremberth 896c30a8eb Reference items pretty-printing issue number 2017-05-18 22:37:15 +02:00
Paul Tremberth 55d1082360 Reference recent fixes and commits 2017-05-18 22:37:15 +02:00
Paul Tremberth 7d72394794 Reword mention of new response.follow() shortcut 2017-05-18 22:37:15 +02:00
Paul Tremberth c6464cc4f5 Add verbose introduction to new features 2017-05-18 22:37:14 +02:00
Paul Tremberth e139d990fc Fix sphinx-build warning on deprecated latex_paper_size 2017-05-18 22:37:14 +02:00
Paul Tremberth cba55cd190 Rephrase other sections 2017-05-18 22:37:14 +02:00
Paul Tremberth 8729a91f7a Rephrase "New features" section 2017-05-18 22:37:14 +02:00
Paul Tremberth 851adcedf2 List merged pull requests since 1.3.3 2017-05-18 22:37:14 +02:00
Paul Tremberth 8aa2e4f997 Merge pull request #1829 from nyov/nyov/editor 
[MRG+1] Remove dependency on os.environ from default settings
 2017-05-18 21:32:51 +02:00
Daniel Graña b9d3b447a5 Merge pull request #2670 from qhuang872/master 
[MRG+1] Update spiders.rst
 2017-05-18 16:31:01 -03:00
Mikhail Korobov 532400f993 Merge pull request #2643 from harshasrinivas/set-retry-times-per-request 
[MRG+1] Add feature to set RETRY_TIMES per request (#2642)
 2017-05-17 15:46:33 +05:00
Mikhail Korobov 1acacab93b Merge pull request #2741 from eliat123/2576_cleanup_MEMUSAGE_REPORT 
[FIX #2576] cleanup: removed unused MEMUSAGE_REPORT
 2017-05-17 13:48:00 +05:00
Daniel Graña 73668ce407 Merge pull request #2721 from HarrisonGregg/feature-drop-from-response-field 
[MRG+1] Allow dropping field in from_response formdata
 2017-05-16 09:47:45 -03:00
Eli Atzaba b74b98fa3e cleanup: removed unused MEMUSAGE_REPORT 
Signed-off-by: Eli Atzaba <eliat123@gmail.com>
 2017-05-16 13:59:58 +03:00
Harrison Gregg 45a323024c Add documentation for dropping fields in from_response request body 2017-05-15 12:25:32 -04:00
Paul Tremberth dfe6d3d59a Merge pull request #2456 from elacuesta/feed_export_beautify 
[MRG+1] Feed exports: beautify JSON and XML
 2017-05-12 18:12:48 +02:00
Paul Tremberth 3a0a86ed31 Clarify FEED_EXPORT_INDENT section 2017-05-12 17:26:17 +02:00
Eugenio Lacuesta 25535dba9c Feed exports: edit note, fix typos 2017-05-10 16:45:15 -03:00
Kurt Peek 548a432951 Minor grammatical changes 2017-05-10 23:48:01 +05:00
yandongxu 4966dd7a7f Fix doc: open file with "wb" mode will get an error in python 3 2017-05-08 18:50:30 +08:00
Liu Siyuan 6c1cacb5d5 [MRG+1] doc: fix documentation error in link-extractor.rst (#2676) 
* fix doc error in link-extractor.rst

* remove the import clause

* update based on suggestion
 2017-05-06 02:47:06 +05:00
Martín Gaitán 4bc0c6b0f4 Update practices.rst 
fix a typo
 2017-04-26 14:12:57 +05:00
Paul Tremberth aa59cf4241 Merge pull request #2668 from harshasrinivas/docs-sphinx-rtd-theme 
[MRG+1] Add sphinx_rtd_theme to docs setup readme
 2017-04-24 12:35:40 +02:00
Tiago Cardoso 00ee9eaeaf Mention how to disable request filtering in documentation of DUPEFILTER_CLASS setting 2017-04-22 14:36:44 +01:00