RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,430 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

1909 Commits

Author SHA1 Message Date
Adrián Chaves f8490cd20d Fix broken doc reference 2024-02-14 19:24:22 +01:00
Adrián Chaves 505f525c66 Set the release date of version 1.8.4 2024-02-14 18:18:35 +01:00
Adrián Chaves ee7bd9d217 Merge remote-tracking branch 'cw9j-q3vf-hrrv/1.8.4-authorization' into 1.8 2024-02-14 14:35:26 +01:00
Adrián Chaves 71b8741e36 Merge remote-tracking branch '7j7m-v7m3-jqm7/1.8-compression-bomb' into 1.8 2024-02-14 14:33:49 +01:00
Adrián Chaves 73e7c0ed01 Backport the latest 2.11 implementation 2024-02-14 14:25:03 +01:00
Adrián Chaves 978407c938 Warn against using scrapy.downloadermiddlewares.decompression 2023-12-13 13:42:37 +01:00
Adrián Chaves 684d4effd4 Set an arbitrary upper limit on ReDoS-vulnerable regexps 2023-11-29 12:39:41 +01:00
Adrián Chaves 1c3d289798 Protect against compression bombs 2023-11-23 16:24:22 +01:00
Adrián Chaves ca832a4360 Drop the Authorization header on cross-domain redirect 2023-11-17 14:08:10 +01:00
Adrián Chaves 2023a6bdcb
Fix the CI (#6149) 2023-11-17 10:12:29 +01:00
Adrián Chaves ced7e4649a
Merge pull request from GHSA-9x8m-2xpf-crp3 
* Backport the 2.6.2 security fix

* Force the same Python version for docs as CI in Tox configuration

* Re-enable TLS 1.2 in cipher tests.

* tox.ini: ensure pypy runs pypy2

* Set the release date of Scrapy 1.8.3

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2022-07-25 13:15:17 +02:00
Adrián Chaves 91706f58b7
Merge pull request from GHSA-mfjm-vh54-3f96 2022-03-01 12:38:19 +01:00
Adrián Chaves 290e2a2726
Merge pull request from GHSA-cjvr-mfj7-j4j8 
* Do not carry over cookies to a different domain on redirect

* Cover the cookie-domain redirect fix in the release notes

* Restore accidentally-removed release notes content

* Fix redirect Cookie handling when the cookie middleware is disabled

* Update the 1.8.2 release date
 2022-03-01 12:26:05 +01:00
nsirletti 47c4a3282a Suppress Sphinx warnings about MIME types other than .rst 2021-10-06 12:22:13 +02:00
Adrián Chaves 99ac4dbd6b Cover 1.8.1 in the release notes 2021-10-05 21:10:49 +02:00
Andrey Rakhmatullin 16351344a6 Small documentation fixes. 2021-10-05 21:07:52 +02:00
Andrey Rakhmatullin b01d69a1bf Add http_auth_domain to HttpAuthMiddleware. 2021-10-05 21:07:45 +02:00
Adrián Chaves 4183925b4b Travis CI → GitHub Actions 2021-10-05 21:04:36 +02:00
Adrián Chaves 94f060fcc8
Cover Scrapy 1.8.0 in the release notes (#3952) 2019-10-29 12:53:46 +01:00
Andrey Rahmatullin 18b808b2e9
Merge pull request #4092 from further-reading/master 
Add Python 3.8 official support
 2019-10-29 16:30:58 +05:00
Adrián Chaves 3d0df419c4 Mark the LevelDB storage backend as deprecated 2019-10-28 11:24:47 +01:00
elacuesta 7731814cc2 ItemLoader: improve handling of initial item (#4036) 2019-10-28 10:53:53 +01:00
Adrián Chaves bb91f9c78c Cover Scrapy 1.7.4 in the release notes 2019-10-28 09:58:13 +01:00
WinterComes 5d75ed4cba Remove an old note about contracts (#4093) 2019-10-22 12:19:07 +02:00
illgitthat 07822935ec Updating link for miniconda (#4089) 2019-10-22 12:05:34 +02:00
Mikhail Korobov 8cb53441b0
Merge pull request #4056 from Gallaecio/documentation-build 
Fix internal links in the tutorial and release notes
 2019-10-19 00:00:02 +02:00
Bulat 84be6a941e Refactor sentence. 2019-10-16 14:04:07 +03:00
Bulat c9614a5bdd Fixed BOT_NAME documentation 2019-10-16 12:07:19 +03:00
Adrián Chaves d72ed46fe0 Improve how extra Item API members are introduced in the documentation 2019-10-15 16:04:26 +02:00
Baron Hou 7b1e69dec4 reponse → response (#4079) 2019-10-15 14:51:15 +02:00
John Bampton 39c9a3cc1c Fix case of GitHub. 2019-10-05 10:09:14 +10:00
Adrián Chaves a4aa5b8926 Fix internal links in the tutorial and release notes 2019-10-02 14:51:06 +02:00
elacuesta 6ad5a89cb0 [Doc] Use autoclass in topics/request-response.rst (#4055) 2019-10-02 12:18:36 +02:00
Mikhail Korobov 74b4a5c77c
Merge pull request #4006 from whalebot-helmsman/protego-default 
Use protego as a default robots.txt parser
 2019-10-02 00:03:50 +05:00
Kristobal Junta c232bbdc42 fix typo in docs/topics/spiders 2019-10-01 17:41:38 +03:00
Mikhail Korobov 31c631f9ee
Merge pull request #3999 from Gallaecio/documentation-coverage 
Provide complete API documentation coverage of scrapy.item
 2019-09-25 16:13:13 +01:00
Adrián Chaves 1236e9e81e Provide complete API documentation coverage of scrapy.item 2019-09-25 13:24:56 +02:00
Mikhail Korobov 17c0d72d04
Merge pull request #4016 from elacuesta/rule_default_link_extractor 
Crawling rules: make link extractors optional
 2019-09-20 15:27:17 +01:00
Adrián Chaves c26a9015ad Clarify the effects of dont_merge_cookies 2019-09-19 11:08:06 +02:00
Adrián Chaves 9b65f9aa5b
Fix the item exporter example (#4022) 2019-09-19 09:17:23 +02:00
Vostretsov Nikita 2438ac529a
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-09-17 12:27:22 +05:00
Vostretsov Nikita d1d0bf8491
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-09-17 12:27:12 +05:00
Vostretsov Nikita 57e6f4c750 add link to performance comparison 2019-09-17 07:18:37 +00:00
Vostretsov Nikita d39ef77e6b add link to google description of lenght-based rule 2019-09-17 06:34:33 +00:00
Vostretsov Nikita b792dba528 remove periods 2019-09-17 06:28:33 +00:00
Vostretsov Nikita 50c85e3114 Merge branch 'protego-default' of github.com:whalebot-helmsman/scrapy into protego-default 2019-09-16 14:24:31 +00:00
Vostretsov Nikita 5197b39e01 fix capitalization, remove commas 2019-09-16 14:24:25 +00:00
Eugenio Lacuesta 21ad8e20b9
Crawling rules: make link extractors optional 2019-09-13 17:32:42 -03:00
Vostretsov Nikita 66145b4eaf
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Mikhail Korobov <kmike84@gmail.com>
 2019-09-12 18:51:00 +03:00
Vostretsov Nikita 171fa1cd10 documentation rework 2019-09-10 09:59:36 +00:00