RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,136 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

9136 Commits

Author SHA1 Message Date
Adrián Chaves aecbccbaa5 Bump version: 2.6.1 → 2.6.2 2022-07-25 13:33:23 +02:00
Adrián Chaves af7dd16d8d
Merge pull request from GHSA-9x8m-2xpf-crp3 
* Enforce matching proxy request meta and Proxy-Authorization header

* Cover proxy credential security fix in the release notes

* Remove extra empty line

* Reword the security issue description

* Address scenario where Proxy-Authorization is unexpectedly removed by a prior middleware

* Set the release date of Scrapy 2.6.2 and 1.8.3
 2022-07-25 13:15:17 +02:00
Mikhail Korobov 4205609051 Fixed intersphinx references 2022-07-21 18:02:56 +02:00
Adrián Chaves e3e69d1209
Pin documentation requirements (#5536) 2022-06-20 11:46:13 +02:00
Adrián Chaves 54bfb9649b
Cover #5525 in the 2.6.2 release notes (#5535) 2022-06-17 11:51:02 +02:00
Adrián Chaves 4ef71829b2
If TWISTED_REACTOR is None, reuse any pre-installed reactor (#5528) 2022-06-17 10:37:27 +02:00
Adrián Chaves 1c1cd5d8ea Update the 2.6.2 release notes 2022-05-20 07:05:26 +02:00
Adrián Chaves 84c29a286f
Unset the release date of still-unreleased 2.6.2 (#5503) 2022-05-20 06:45:38 +02:00
Andrey Rahmatullin b9b9422bb1
Merge pull request #5482 from alexpdev/parse_help_msg 
Fix TypeError in parse command
 2022-04-29 11:10:15 +05:00
Alex 915c288205 edit 2022-04-24 15:17:48 -07:00
Alex 56c9098d6a changed default depth to 1 2022-04-24 15:17:48 -07:00
Alex b0f5503cb8 Fixes Issue #5481 2022-04-24 15:17:48 -07:00
Alex 636127ec1e tests that all CLI help messages don't throw errors 2022-04-22 16:32:58 -07:00
Adrián Chaves aead27bcbd
Add release notes for 2.6.2 (#5448) 2022-04-14 15:06:22 +02:00
Andrey Rahmatullin fa4c85f4ec
Merge pull request #5471 from Gallaecio/2.6-release-notes-request-serialization 
2.6 release notes request serialization
 2022-04-08 18:14:50 +05:00
Adrián Chaves bae3f87455 Cover a backward-incompatible Request serialization change in the 2.6 release notes 2022-04-08 12:04:02 +02:00
Andrey Rahmatullin 4af22bf157
Pin mitmproxy to < 8 for now (#5459) 2022-04-08 11:26:23 +02:00
Andrey Rahmatullin 35b44f3b98
Merge pull request #5436 from Gallaecio/fix-crawlerprocess-regression 
CrawlerProcess: initialize the reactor only once
 2022-03-25 13:21:21 +05:00
Andrey Rahmatullin 1e1bf353bf
Merge pull request #5440 from Gallaecio/fix-closespider-exception 
engine: prevent slot method call after unsetting the slot
 2022-03-24 19:43:31 +05:00
Adrián Chaves b59a69be17 Test that a low CLOSEPIDER_TIMEOUT does not raise an exception 2022-03-15 12:10:41 +01:00
Laerte Pereira c1d4be8cb5
Restore ‘-o -’ support (#5445) 2022-03-15 11:30:30 +01:00
Adrián Chaves 62a0081266 engine: prevent slot method call after unsetting the slot 2022-03-07 12:00:44 +01:00
Adrián Chaves 3bf6baeaa7 initiated → initialized 2022-03-02 17:30:05 +01:00
Adrián Chaves 96fc4dae15 CrawlerProcess: test a multi-spider scenario 2022-03-02 17:30:05 +01:00
Adrián Chaves 3ecbea4b87 CrawlerProcess: initiate the reactor only once 2022-03-02 17:30:05 +01:00
Adrián Chaves 23537a0f95 Bump version: 2.6.0 → 2.6.1 2022-03-01 13:48:40 +01:00
Adrián Chaves fab3e90729 Cover 2.6.1 in the release notes 2022-03-01 13:41:20 +01:00
Adrián Chaves d60636d0de Fix redirect handling regression 2022-03-01 13:06:58 +01:00
Adrián Chaves 84853c4fa6 bandit: allow-list B324 for the time being 2022-03-01 13:01:20 +01:00
Adrián Chaves 6b63e7c147 Bump version: 2.5.0 → 2.6.0 2022-03-01 12:43:11 +01:00
Adrián Chaves e865c4430e
Merge pull request from GHSA-mfjm-vh54-3f96 
* Ignore cookies with a public suffix as domain unless it matches the request domain

* Fix the merge of 1.8.2 release notes

* Re-apply removal of tldextract restriction
 2022-03-01 12:38:19 +01:00
Adrián Chaves 8ce01b3b76
Merge pull request from GHSA-cjvr-mfj7-j4j8 
* Do not carry over cookies to a different domain on redirect

* Cover the cookie-domain redirect fix in the release notes

* Cover 1.8.2 in the release notes

* Fix redirect Cookie handling when the cookie middleware is disabled

* Update the 1.8.2 release date
 2022-03-01 12:26:05 +01:00
Adrián Chaves aa0306a167
Cover 2.6.0 in the release notes (#5399) 2022-03-01 12:16:37 +01:00
Andrey Rahmatullin 08557e09db
Pin old markupsafe when we pin old mitmproxy (#5427) 2022-02-23 19:52:18 +01:00
Gowtham Chowdary 3b42ccfeba
Add a link to Discord (#5422) 2022-02-16 21:33:56 +01:00
Andrey Rahmatullin 8840403b70
Merge pull request #5412 from Laerte/master 2022-02-15 18:15:39 +05:00
Andrey Rahmatullin 0b0eea3601
Merge pull request #5419 from PendalF89/patch-2 
Update downloader-middleware.rst
 2022-02-15 10:57:35 +05:00
Abhishek K M 187b5c8876
Update the documentation link for robots.txt (#5415) 2022-02-14 19:16:53 +01:00
Boris Zabolotskikh bbb693d046
Update downloader-middleware.rst 
Added a link to the method
 2022-02-14 12:07:45 +03:00
Laerte Pereira befb6df119 Remove Python 2 code from WrappedRequest 2022-02-11 06:19:27 -03:00
Andrey Rahmatullin 5d7c0a5f86
Use toscrape.com instead of example.com in test_command_check. (#5407) 2022-02-10 10:50:12 +01:00
Alex e2e2ffd0d1
Move from optparse to argparse (#5374) 2022-02-09 20:52:07 +01:00
Andrey Rahmatullin 1356836e7e
Merge pull request #5405 from wRAR/tests-twisted-22.1-fix 
Fix running tests with Twisted 22.1.0
 2022-02-09 00:06:25 +05:00
Andrey Rakhmatullin 77547a1ab5 Revert "Temporarily pin Twisted to an older version in CI (#5401)" 
This reverts commit b282a7af01.
 2022-02-08 21:06:02 +05:00
Andrey Rakhmatullin 1e1cfc26db Copy resource classes from twisted.web.test.test_webclient. 2022-02-08 21:01:16 +05:00
Raihan Nismara fd55f62207
Update Logo in README.rst (#5258) 2022-02-08 15:36:25 +01:00
Laerte 4bda0976b2
Fix csviter call, add parse_rows test (#5394) 2022-02-08 14:57:19 +01:00
Andrey Rahmatullin 840362496c
Merge pull request #5396 from peter-gy/master 
docs: use https scheme for each quotes.toscrape.com url occurrence
 2022-02-08 11:05:52 +05:00
Andrey Rahmatullin b282a7af01
Temporarily pin Twisted to an older version in CI (#5401) 2022-02-07 21:25:08 +01:00
Andrey Rahmatullin f7bf7414f0
Merge pull request #5398 from elacuesta/remove-deprecated-item-classes 
Remove deprecated DictItem/BaseItem classes
 2022-02-07 12:34:35 +05:00