RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,844 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

2265 Commits

Author SHA1 Message Date
Adrián Chaves c5b1ee8101
Make Twisted[http2] installation optional (#5113) 
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2021-05-11 09:04:53 +02:00
Shinichi Takayanagi bd60c3f41f
More documentation for setting spider atributes 
* docs: require sphinx-rtd-theme>=0.5.2 and the latest pip to prevent installing breaking docutils>=0.17

* Update feed-exports.rst

* Update feed-exports.rst

* Reflects the comments

* Remove redundant newline

* Update docs/topics/feed-exports.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Apply suggestions from code review

Co-authored-by: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2021-05-11 00:58:04 +05:00
Eugenio Lacuesta cec36a9284
Refactor request to/from dict (#5130) 2021-05-10 18:00:08 +02:00
Renne Rocha 34b216289c
Update link for reasoning value of URLLENGTH_LIMIT (#5134) 2021-05-06 16:34:05 +02:00
Eugenio Lacuesta ddea6b7bfa
Scheduler: minimal interface, API docs (#3559) 2021-04-26 16:16:14 -03:00
Andrey Rahmatullin 7095df1558
Merge pull request #5082 from storymode7/shell-fix 
Find bash from PATH instead of /bin/bash
 2021-04-21 14:30:50 +05:00
Eugenio Lacuesta 7e23677b52
Engine: deprecations and type hints (#5090) 2021-04-20 08:45:28 -03:00
Andrey Rahmatullin 5b78a64fca
Merge pull request #5070 from ric2b/patch-1 2021-04-15 14:50:59 +05:00
Mayank Singhal a4415e4e6f Add DataURI download handler in DOWNLOAD_HANDLERS_BASE documentation 2021-04-13 17:56:44 +05:30
Adrián Chaves 5a75b14a5f docs: require sphinx-rtd-theme>=0.5.2 and the latest pip to prevent installing breaking docutils>=0.17 2021-04-07 14:04:00 +02:00
Ricardo Amendoeira 8603f9d7a5
Apply changes to other examples in the same section. 2021-04-06 20:23:07 +01:00
Adrián Chaves a71d6ef29d
2.5.0 release notes (#5028) 
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
 2021-04-06 16:09:07 +02:00
Mayank Singhal e7d51886ef Find bash from PATH instead of /bin/bash 2021-04-06 02:21:18 +05:30
Mikhail Korobov f0c8d31193
Merge pull request #4694 from Jgaldos/improve-httpstatus-all-meta 
Improve http status all on http error middleware
 2021-04-01 22:43:45 +05:00
Akshay Sharma cc095aa895
Windows pip installation guide (#4736) 
* added initial steps

* fixing link

* python3 -> python

* remaining steps

* steps updated

* Update docs/intro/install.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* added link to Visual Studio

* removed 'install V'

* Update docs/intro/install.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-04-01 22:39:33 +05:00
anay2103 5492972d8a
added customized filtering examples in logging.rst (#4965) 
* added customized filtering examples in logging.rst

* Update logging.rst

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-04-01 22:30:48 +05:00
Mikhail Korobov b09ff99d81
Merge pull request #4974 from Gallaecio/ul-margin 
Move lists closer to their introducing paragraph
 2021-04-01 22:27:59 +05:00
Mikhail Korobov e0a2d2b3c9
Merge pull request #5014 from jpmckinney/patch-1 
docs: Clarify there's one extension instance per crawler
 2021-04-01 22:26:20 +05:00
Mikhail Korobov d0e23486b4
Merge pull request #4902 from Gallaecio/retry-request 
Implement a retry request function
 2021-04-01 22:23:56 +05:00
Veniamin Gvozdikov 90fe494ba2 Rebranding, updated GA code 2021-04-01 11:11:28 +03:00
Ricardo Amendoeira b247fa9982
Include loading settings in `Running multiple spiders in the same process` section 
The example in the documentation doesn't take into account the project settings
 2021-03-29 01:48:28 +01:00
Pratik Mahankal f0e1a33225
Sort the list of Request.meta alphabetically #5061 (#5065) 2021-03-23 22:16:50 +05:00
Adrián Chaves 0dad0fce72
Use pip<20.3 to fix ReadTheDocs builds (#5052) 2021-03-19 11:13:05 +01:00
Adrián Chaves 1d83697990
Merge pull request #4769 from scrapy/http2 
Implement experimental HTTP/2 support
 2021-03-18 22:18:12 +01:00
Mikhail Korobov ab037ab7c9
Merge pull request #5016 from deepang17/scrapy-4996 
DOCS: Cover scrapy-bench in the documentation
 2021-03-12 04:18:56 +05:00
Eugenio Lacuesta 0c16088230
headers_received signal (#4897) 2021-03-11 11:52:35 -03:00
Adrián Chaves 5b2d3e17a3
Merge branch 'master' into http2 2021-03-09 14:59:17 +01:00
deepang17 2d41c77e6a Merge branch 'master' of https://github.com/scrapy/scrapy into scrapy-4996 2021-03-09 17:33:05 +05:30
deepang17 3d88ac605b FIX: Updated benchmarking.rst 2021-03-09 17:19:34 +05:30
Andrey Rahmatullin 954c4b48e6
Merge pull request #3667 from Gallaecio/long-allowed-domains 
Add a FAQ entry on how to deal with long lists of allowed domains
 2021-03-05 20:16:37 +05:00
deepang17 36f1dbf665 DOCS: Covered scrapy-bench 2021-03-02 22:12:44 +05:30
James McKinney 3c5668d0db
docs: Clarify there's one extension instance per spider 2021-03-01 22:00:33 -05:00
deepang17 b25616d107 DOCS: Cover scrapy-bench in the documentation 2021-02-28 16:26:46 +05:30
deepang17 7a54580679 DOCS:Cover scrapy-bench in the documentation 2021-02-28 15:33:09 +05:30
Adrián Chaves f689615e8d Close files in the PerYearXmlExportPipeline documentation example 2021-02-24 12:54:56 +01:00
Wehzie a36f952198
fixed typo "an quotes.json" -> "a quotes.json" (#5005) 2021-02-24 08:15:44 +01:00
Adrián Chaves 6ab990181c Document get_retry_requests 2021-02-22 14:48:03 +01:00
Adrián Chaves 4c801551fa Document that the bytes_received signal is not yet implemented for HTTP/2 2021-02-03 21:11:46 +01:00
Adrián Chaves 1773eaf5dc Move lists closer to their introducing paragraph 2021-02-03 11:43:18 +01:00
Adrián Chaves 1a7bde0d8e Document that HTTP/2 server pushes are ignored 2021-02-03 10:55:11 +01:00
Adrián Chaves c8d8b180bf Merge remote-tracking branch 'upstream/master' into http2 2021-02-03 09:37:35 +01:00
Adrián Chaves 2ce8e0c742 Document the (hard-coded) maximum HTTP/2 frame size accepted from servers 2021-02-03 09:09:53 +01:00
Adrián Chaves f30f53b3cc Scrapinghub → Zyte 2021-02-02 15:03:20 +01:00
M Ikram Ullah Khan 0a1e2fefab
Docs: fix typo in news.rst (#4942) 2021-01-04 10:30:23 -03:00
Eugenio Lacuesta e494a3f733
protocol attribute for h2 responses 2020-12-31 11:50:15 -03:00
Eugenio Lacuesta d698b5147b
Merge branch 'master' into http2 2020-12-31 11:13:25 -03:00
Adrián Chaves d0af008608
Merge pull request #4878 from elacuesta/response-protocol-attribute 
Response.protocol attribute
 2020-12-16 15:27:40 +01:00
gunadhya db10aaf9eb
Update links in installation guide (#4899) 2020-12-03 10:56:36 +01:00
etimoz 95d39d5cb4 removed wrong super argument in overriding serialize_fields code example 2020-11-29 13:24:04 +01:00
Adrián Chaves 63becd1bc8
Update news.rst 2020-11-17 21:58:08 +01:00