RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,805 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

8805 Commits

Author SHA1 Message Date
Adrián Chaves bab173eec0 Cover 1.8.1 in the release notes 2021-10-06 12:31:09 +02:00
Adrián Chaves 61130c8aad Bump version: 2.5.0 → 2.5.1 2021-10-05 15:30:25 +02:00
Andrey Rahmatullin 98d21738a2 Pin the libxml2 version in CI as a newer one breaks lxml (#5208) 2021-10-05 15:01:59 +02:00
Eugenio Lacuesta 47fb908a0e [CI] fail-fast: false (#5200) 2021-10-05 14:33:48 +02:00
Adrián Chaves 6d7179b6b7 tests: freeze pylint==2.7.4 2021-10-05 14:28:32 +02:00
Adrián Chaves d06dcb8246 tests: force queuelib < 1.6.0 2021-10-05 14:04:27 +02:00
Adrián Chaves d99b1a189c Cover 2.5.1 in the release notes 2021-10-05 13:34:31 +02:00
Andrey Rakhmatullin c9485a5151 Small documentation fixes. 2021-10-05 13:34:23 +02:00
Andrey Rakhmatullin a172844971 Add http_auth_domain to HttpAuthMiddleware. 2021-10-05 13:34:14 +02:00
Adrián Chaves 5fd75f865f docs: require sphinx-rtd-theme>=0.5.2 and the latest pip to prevent installing breaking docutils>=0.17 2021-04-07 14:04:51 +02:00
Andrey Rakhmatullin e63188cbf7 Bump version: 2.4.1 → 2.5.0 2021-04-06 19:13:32 +05:00
Adrián Chaves a71d6ef29d
2.5.0 release notes (#5028) 
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
 2021-04-06 16:09:07 +02:00
Andrey Rahmatullin 099fb6ead0
Merge pull request #5076 from wRAR/qualname 
Use __qualname__ in middleware handling.
 2021-04-04 15:52:32 +05:00
Andrey Rakhmatullin 9e3b868dd8 Use __qualname__ in middleware handling. 2021-04-03 17:04:09 +05:00
Mikhail Korobov 8c5a3a5189
Merge pull request #5073 from elacuesta/fix-urlencode-doseq-arg 
Fix type for urlencode's doseq argument
 2021-04-02 00:51:15 +05:00
Eugenio Lacuesta 9e7cbc05ae
Fix type for urlencode's doseq argument 2021-04-01 15:22:51 -03:00
Mikhail Korobov 127a8586e1
Merge pull request #5063 from noviluni/update_usageError_message 
Update UsageError message
 2021-04-01 22:47:59 +05:00
Mikhail Korobov 12f2006b7b
Merge pull request #4799 from GeorgeA92/patch-2 
httpcompression stats added
 2021-04-01 22:47:24 +05:00
Mikhail Korobov f0c8d31193
Merge pull request #4694 from Jgaldos/improve-httpstatus-all-meta 
Improve http status all on http error middleware
 2021-04-01 22:43:45 +05:00
Akshay Sharma cc095aa895
Windows pip installation guide (#4736) 
* added initial steps

* fixing link

* python3 -> python

* remaining steps

* steps updated

* Update docs/intro/install.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* added link to Visual Studio

* removed 'install V'

* Update docs/intro/install.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-04-01 22:39:33 +05:00
Mikhail Korobov 0007a4c117
Merge pull request #4895 from elacuesta/minimal-type-hints 
Some type hints
 2021-04-01 22:36:33 +05:00
Kader DJEHAF ad74567469
Fix argument type (int -> bool) (#4950) 
* Fix warning: Expected type 'bool', got 'int' instead

* Update defer.py

* Fix warning: Expected type 'bool', got 'int' instead

Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2021-04-01 22:33:56 +05:00
anay2103 5492972d8a
added customized filtering examples in logging.rst (#4965) 
* added customized filtering examples in logging.rst

* Update logging.rst

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update logging.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-04-01 22:30:48 +05:00
Mikhail Korobov b09ff99d81
Merge pull request #4974 from Gallaecio/ul-margin 
Move lists closer to their introducing paragraph
 2021-04-01 22:27:59 +05:00
Mikhail Korobov e0a2d2b3c9
Merge pull request #5014 from jpmckinney/patch-1 
docs: Clarify there's one extension instance per crawler
 2021-04-01 22:26:20 +05:00
Mikhail Korobov d0e23486b4
Merge pull request #4902 from Gallaecio/retry-request 
Implement a retry request function
 2021-04-01 22:23:56 +05:00
Eugenio Lacuesta d458ccff3b
Retry request: priority_adjust cannot be float (Request.priority is int) 2021-04-01 12:27:35 -03:00
Mikhail Korobov 9a91586f6f
Merge pull request #5072 from zloidemon/rebranding_update_ga 
Updated GA code for docs, remove Segment tracking
 2021-04-01 20:26:11 +05:00
Veniamin Gvozdikov 90fe494ba2 Rebranding, updated GA code 2021-04-01 11:11:28 +03:00
Eugenio Lacuesta 1d200258a5
Adjust h2 version requirement (#5066) 2021-03-26 14:45:26 +01:00
Eugenio Lacuesta 9c9e1a318d
[HTTP/1.1] Skip Content-Length header if its value is UNKNOWN_LENGTH (#5062) 2021-03-25 15:58:39 +01:00
Pratik Mahankal f0e1a33225
Sort the list of Request.meta alphabetically #5061 (#5065) 2021-03-23 22:16:50 +05:00
Marc 64d4ae1a19 Update UsageError message 2021-03-22 21:46:05 +01:00
Eugenio Lacuesta 72e8cea8af
Avoid exceptions in is_generator_with_return_value (#4935) 2021-03-22 11:51:11 -03:00
Adrián Chaves ec5a7918ec
Include Content-Length in HTTP/1.1 responses (#5057) 2021-03-22 11:25:40 +01:00
Adrián Chaves 2973d8d51a
Remove unnecessary reference to private parsel.Selector._default_type (#5006) 2021-03-22 11:24:10 +01:00
Mikhail Korobov 0e7ee1256c
Merge pull request #5036 from dmiwell/urllength-dont-skip-silently 
UrlLengthMiddleware: don't skip silently
 2021-03-20 00:55:33 +05:00
Adrián Chaves 308a58aa27
Update CI to support Twisted 21.2.0 (#5027) 2021-03-19 18:39:44 +01:00
Adrián Chaves 0dad0fce72
Use pip<20.3 to fix ReadTheDocs builds (#5052) 2021-03-19 11:13:05 +01:00
Adrián Chaves 8e302f2906
Merge pull request #5053 from Gallaecio/constraints 
Fix master CI issues due to pip backtracking
 2021-03-19 10:34:36 +01:00
Adrián Chaves a390b934de Do not install mitmproxy in Python 3.9 2021-03-18 23:53:58 +01:00
Adrián Chaves 8e73e1dfb5 upper-constraints.txt: restrict botocore further 2021-03-18 23:42:29 +01:00
Adrián Chaves 94201612bc Simplify the get_retry_request code example 2021-03-18 23:35:47 +01:00
Adrián Chaves 1d83697990
Merge pull request #4769 from scrapy/http2 
Implement experimental HTTP/2 support
 2021-03-18 22:18:12 +01:00
vinayak 42e4dbb23d
Support Python 3.9 (#4759) 
* Update .travis.yml

* Update .travis.yml

* updage travis.yml

* Make 3.9 support official

* Upgrade mitmproxy for Python 3.9

* Restore the Pylint job

* Undo unintended change to mitmproxy requirement

* Enable Python 3.9 in GitHub Actions

* Work around reppy’s Python version limitation

* Disable tests in Windows / Python 3.9 due to a Twisted bug

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-03-18 09:40:03 -03:00
Eugenio Lacuesta 2f61d7cc03
Remove unnecesary del statement 2021-03-15 14:25:46 -03:00
Dmitriy Pomazunovskiy 9cc4513bd6 simpler stats access 2021-03-15 21:38:03 +06:00
Adrián Chaves c0f3ca1938 get_retry_request: add typing information 2021-03-12 14:05:44 +01:00
Dmitriy Pomazunovskiy 0f254a6afb Test from_settings 2021-03-12 17:11:50 +06:00
Dmitriy Pomazunovskiy d4b2b61255 Use from_settings for backward compatibility 2021-03-12 16:59:37 +06:00