RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,808 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

1082 Commits

Author SHA1 Message Date
Adrián Chaves 9c314800e4 Document the SCRAPY_PROJECT environment variable 
Fixes #1109
 2019-03-22 15:40:36 +03:00
jfflisikowski edaf74bfae Correct the unclear comments by adding <# < processing code not shown > 2018-10-02 19:48:48 +02:00
Mikhail Korobov ffbd33edac DOC mention gotcha with `foo::text` selector and empty `foo` elements 
also, move "Selecting attributes" reference closer to `a::atr(href)` example
 2018-09-18 05:03:35 +05:00
Mikhail Korobov 2c3b2158c9 DOC address @stummjr's review comments 
* fixed several small issues
* re-written "Creating Selectors" section
* fixed remaining .extract usage in tests
 2018-09-18 05:02:17 +05:00
Mikhail Korobov 9db21e5502 DOC fix remove_namespaces example 
See https://github.com/scrapy/parsel/pull/119
 2018-09-15 02:43:37 +05:00
Mikhail Korobov dc95ecbe25 DOC use autodocs for selectors; document more methods and attributes; suggest get/getall 2018-09-12 18:36:25 +05:00
Mikhail Korobov 7fdfdb7fa2 DOC reorganize selectors tutorial, port more topics from parsel docs, adjust wording in the introduction 2018-09-12 17:57:27 +05:00
Mikhail Korobov bdcc045f62 DOC switch from .extract to get/getall API in docs 
Also, response.urljoin is added in a few places, for robustness.
 2018-09-12 17:57:27 +05:00
Mikhail Korobov 2c48d156db DOC cleanup references in tutorials: 
* remove unused link
* fix ReST syntax
* fix a link to regular expression docs
 2018-09-12 17:57:27 +05:00
Mikhail Korobov d32c4deaa9 DOC update Scrapy selectors tutorial to match parsel's tutorial better 2018-09-12 17:57:27 +05:00
Mikhail Korobov ca27010d4f DOC .extract_first() -> .get() 2018-09-12 17:57:27 +05:00
Mikhail Korobov 0ccead6681 DOC more Python 3 in examples 2018-09-12 17:57:27 +05:00
Daniel Graña ae8a0dc77c
Merge pull request #3383 from StasDeep/feature/issue-3382 
[MRG+1] Add ability to use FormRequest in contracts
 2018-09-05 11:21:26 -03:00
Daniel Graña 4da0b59cd7
Merge pull request #3393 from whalebot-helmsman/singal-request-added-to-downloader-slot 
[MRG+1] New signal for reqeuests reached downloader
 2018-09-05 11:17:13 -03:00
Stas Glubokiy 8dbbbd1395 Use request_cls attribute in contract definition 2018-09-03 20:07:37 +03:00
Vostretsov Nikita c02cfa574c remove comma 2018-08-29 11:21:55 +00:00
Raphael Wuillemier 79de3d569a Removed obsolete firebug-images 2018-08-23 16:19:13 +02:00
testingcan e98e7f8506
Added missing curly brace 2018-08-23 14:50:49 +02:00
Raphael Wuillemier 4d3aaabbca Updated code, added code snippets and improved readability 2018-08-23 12:40:31 +02:00
testingcan 3a71e7dbce
Increased length of "=" 2018-08-22 16:57:51 +02:00
Raphael Wuillemier af555cab23 Added general guide for developer tools instead of Firefox and Firebug-sections 2018-08-22 14:15:53 +02:00
Stas Glubokiy 9ab85fe788 Merge branch 'master' of github.com:StasDeep/scrapy into feature/issue-3382 2018-08-19 17:21:28 +03:00
Vostretsov Nikita 597b8a97ad documentation for new signal 2018-08-17 14:39:42 +00:00
Stas Glubokiy b4b1e48343 Add ability to use FormRequest in contracts 2018-08-11 22:18:43 +03:00
Valdir Stumm Junior c61e8a617f
[doc] update default RETRY_HTTP_CODES 2018-07-13 11:55:16 -07:00
Daniel Graña f8f5f463e6
Merge pull request #3253 from rpkilby/depth-stats 
[MRG+1] Update depth middleware stats (fixes #3245)
 2018-07-11 12:00:08 -03:00
chainly 74ce156154 add item_error to be catchable (#3256) 2018-07-03 16:00:59 -03:00
Daniel Graña 594a441eb5
Merge pull request #3294 from rennerocha/feed-exporter-docs 
[MRG+1] Improve docs of S3 Storage to make Python version more explicit
 2018-06-27 11:16:07 -03:00
Renne Rocha e2bb218e9a Include Python version indication to each required library used in S3 storage 2018-06-13 18:11:43 -03:00
mugayoshi d4511667fb Update debugging memory leaks section in the docs 
Add Python3 tools description.
 2018-06-09 18:17:11 +09:00
Mikhail Korobov 847b50ce9c
Merge pull request #3201 from grammy-jiang/master 
[MRG+1] fix a mistake in topic spider-middleware.rst
 2018-06-02 01:09:59 +05:00
Mikhail Korobov 13b15dc9a8
Merge pull request #3279 from lewoudar/patch-1 
Update spiders.rst
 2018-06-01 22:20:41 +05:00
Kevin Tewouda ffa7bede17
Update spiders.rst 
I changed URLs to :class:`~scrapy.http.Request` in start_urls explanation of the default spider
 2018-05-30 06:33:18 +02:00
Ryan P Kilby 2dfc5d128b Update DEPTH_STATS refs to DEPTH_STATS_VERBOSE 2018-05-17 11:00:21 -04:00
Konstantin Lopuhin c4f096d3a5
Merge pull request #3224 from lucywang000/better-processors-doc 
[MRG+1] Improve document about functions as processors
 2018-04-26 09:36:39 +03:00
Konstantin Lopuhin f36e1b53d4
Merge pull request #3199 from rhoboro/gcs_acl 
[MRG+1] FilesPipeline supports ACL for Google Cloud Storage
 2018-04-25 19:42:19 +03:00
Lucy Wang 57b0e6b695 improve document about functions as processors 2018-04-19 13:35:46 +08:00
rhoboro 6ef6585b5a update docs 2018-04-13 19:06:29 +09:00
rhoboro 464973489e Using bucket's default object ACL 2018-04-13 12:06:39 +09:00
grammy-jiang cb76b88331 fix a mistake in topic spider-middleware.rst 2018-04-04 05:56:05 -04:00
rhoboro 74a9c65290 update docs for support gcs acl 2018-04-03 18:20:37 +09:00
Steven Almeroth c6d20bdd82 Doc: update wording for COOKIES_ENABLED 2018-03-27 16:21:07 -04:00
Daniel Graña 2dee191374
Merge branch 'master' into extending-s3-files-store 2017-12-31 16:44:38 -03:00
Raphael a1cc5a63d3 Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999) (#3030) 
Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999)
 2017-12-27 17:54:17 -03:00
Daniel Graña 3cf0332ec3
Merge pull request #2957 from ScrapingLab/add_meta_json_to_parse_command 
[MRG+1] Scrapy Command: add --meta/-m to the "parse" command to pass additional meta data into the request
 2017-11-29 16:26:48 -03:00
IAlwaysBeCoding 6af323d7c8
Fix spelling mistake on scrapy parse command docs 
Fixed spelling mistake from "will be pass" to "will be passed"
 2017-11-26 00:24:52 +01:00
Jesse Bakker 0b14cb44aa Added from_crawler to middleware docs 2017-11-23 15:25:43 +01:00
IAlwaysBeCoding 846fd83512 removed commented out code, wrapped line to pep-8 and removed backlashes 2017-11-11 18:30:01 -05:00
Daniel Graña b8870ee8a1
Merge pull request #2989 from colinmorris/ItemExporterDocsExample 
[MRG+1] Revise/modernize item exporter example in docs
 2017-11-03 13:15:31 -02:00
colinmorris 23e571e860 fix issues identified in review 2017-10-31 18:08:47 -04:00