RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,142 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

2384 Commits

Author SHA1 Message Date
Adrián Chaves cd0dc45ca2 Release notes: add missing mentions to related issues 2022-09-26 22:01:36 +02:00
Adrián Chaves 0a6bf5ca7f Cover minimum dependency version upgrades on the release notes 2022-09-26 22:00:18 +02:00
Adrián Chaves f92db7f6b8 Mention removing SSLv3 support 2022-09-26 21:50:30 +02:00
Adrián Chaves 662633c125 Add 2.6.3 to the release notes 2022-09-26 21:47:18 +02:00
Gábor Lipták 7f1a0d579b Cleanup METHOD_SSLv3 
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
 2022-09-26 21:44:03 +02:00
Adrián Chaves af7dd16d8d
Merge pull request from GHSA-9x8m-2xpf-crp3 
* Enforce matching proxy request meta and Proxy-Authorization header

* Cover proxy credential security fix in the release notes

* Remove extra empty line

* Reword the security issue description

* Address scenario where Proxy-Authorization is unexpectedly removed by a prior middleware

* Set the release date of Scrapy 2.6.2 and 1.8.3
 2022-07-25 13:15:17 +02:00
Mikhail Korobov 4205609051 Fixed intersphinx references 2022-07-21 18:02:56 +02:00
Adrián Chaves e3e69d1209
Pin documentation requirements (#5536) 2022-06-20 11:46:13 +02:00
Adrián Chaves 54bfb9649b
Cover #5525 in the 2.6.2 release notes (#5535) 2022-06-17 11:51:02 +02:00
Adrián Chaves 4ef71829b2
If TWISTED_REACTOR is None, reuse any pre-installed reactor (#5528) 2022-06-17 10:37:27 +02:00
Adrián Chaves 1c1cd5d8ea Update the 2.6.2 release notes 2022-05-20 07:05:26 +02:00
Adrián Chaves 84c29a286f
Unset the release date of still-unreleased 2.6.2 (#5503) 2022-05-20 06:45:38 +02:00
Adrián Chaves aead27bcbd
Add release notes for 2.6.2 (#5448) 2022-04-14 15:06:22 +02:00
Adrián Chaves bae3f87455 Cover a backward-incompatible Request serialization change in the 2.6 release notes 2022-04-08 12:04:02 +02:00
Adrián Chaves fab3e90729 Cover 2.6.1 in the release notes 2022-03-01 13:41:20 +01:00
Adrián Chaves e865c4430e
Merge pull request from GHSA-mfjm-vh54-3f96 
* Ignore cookies with a public suffix as domain unless it matches the request domain

* Fix the merge of 1.8.2 release notes

* Re-apply removal of tldextract restriction
 2022-03-01 12:38:19 +01:00
Adrián Chaves 8ce01b3b76
Merge pull request from GHSA-cjvr-mfj7-j4j8 
* Do not carry over cookies to a different domain on redirect

* Cover the cookie-domain redirect fix in the release notes

* Cover 1.8.2 in the release notes

* Fix redirect Cookie handling when the cookie middleware is disabled

* Update the 1.8.2 release date
 2022-03-01 12:26:05 +01:00
Adrián Chaves aa0306a167
Cover 2.6.0 in the release notes (#5399) 2022-03-01 12:16:37 +01:00
Gowtham Chowdary 3b42ccfeba
Add a link to Discord (#5422) 2022-02-16 21:33:56 +01:00
Andrey Rahmatullin 0b0eea3601
Merge pull request #5419 from PendalF89/patch-2 
Update downloader-middleware.rst
 2022-02-15 10:57:35 +05:00
Abhishek K M 187b5c8876
Update the documentation link for robots.txt (#5415) 2022-02-14 19:16:53 +01:00
Boris Zabolotskikh bbb693d046
Update downloader-middleware.rst 
Added a link to the method
 2022-02-14 12:07:45 +03:00
Andrey Rahmatullin 840362496c
Merge pull request #5396 from peter-gy/master 
docs: use https scheme for each quotes.toscrape.com url occurrence
 2022-02-08 11:05:52 +05:00
Eugenio Lacuesta fca49cca92
Remove deprecated DictItem class 2022-02-06 18:31:55 -03:00
Péter Ferenc Gyarmati 38d2a154ec docs: use https scheme for each quotes.toscrape.com url occurrence 2022-02-06 18:52:15 +01:00
Sixuan (Cherie) Wu 30d5779ea9
Fix FEED_URI_PARAMS: custom params throws KeyError (#4966) 
* fix FEED_URI_PARAMS: custom params throws KeyError closes #4962

* another try FEED_URI_PARAMS

* add warning message and change default function

* Add tests for FEED_URI_PARAMS

* FEED_URI_PARAMS: warn if the params dict has been modified in-place

* [Doc] FEED_URI_PARAMS: modifying params in-place is deprecated

* Remove whileline

* Rename parameters for lambda function

* Type hints for FeedExporter._get_uri_params

Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2022-01-28 14:30:30 -03:00
Mikhail Korobov 9dd77b42b5
Merge pull request #1582 from starrify/add_doc_textresponse_urljoin 
added: Doc for `scrapy.http.TextResponse.urljoin`
 2022-01-23 21:40:21 +05:00
Andrey Rahmatullin 5059b8c493
Merge branch 'master' into doc-fix 2022-01-21 13:58:43 +05:00
Andrey Rahmatullin 1608b1a7fa
Merge branch 'master' into patch-1 2022-01-21 13:24:51 +05:00
Andrey Rakhmatullin 3577c00569 Merge remote-tracking branch 'origin/master' into crawler-process-reactor-later 2022-01-03 13:59:26 +05:00
Andrey Rahmatullin c5ab58056c
Set WindowsSelectorEventLoopPolicy on Windows (#5315) 2021-12-31 20:38:10 +01:00
Andrey Rakhmatullin 6eaceec735 Implement docs suggestions. 2021-12-31 20:14:24 +05:00
Andrey Rakhmatullin 60800ee5f2 Merge remote-tracking branch 'origin/master' into crawler-process-reactor-later 2021-12-31 20:12:30 +05:00
Andrey Rahmatullin 57dc58123b
Remove the experimental note about asyncio (#5332) 2021-12-31 13:15:08 +01:00
Andrey Oskin b81938684b
Docs: correct process repetition start step (#5356) 
The process repeats from step 3, the scheduler feeds request to the engine. Steps 1 and 2 are not parts of the loop as their incarnations steps 7 and 8 are parts of the loop.
 2021-12-31 11:49:18 +01:00
Andrey Rakhmatullin a986792def Add more docs for TWISTED_REACTOR. 2021-12-24 19:43:14 +05:00
Andrey Rakhmatullin 940cc0776f Add docs about TWISTED_REACTOR and other per-process settings. 2021-12-24 17:12:50 +05:00
azzamsa 67994d1ddd fix: `CodeBlockParser` has been renamed to `PythonCodeBlockParser` 2021-10-27 22:19:07 +07:00
Andrey Rahmatullin 144d1eb834
Add Deferred-to-Future helpers (#5288) 2021-10-22 18:46:01 +02:00
Andrey Rahmatullin afa5881ada
Merge pull request #5279 from erikkemperman/option-logfile-truncate 
Add LOG_FILE_APPEND to settings
 2021-10-18 11:51:42 +05:00
raphaelts3 027ecd8686
Update docs/intro/overview.rst 
Co-authored-by: azzamsa <17734314+azzamsa@users.noreply.github.com>
 2021-10-16 10:52:54 -03:00
Raphael Tomé Santana aec7146e2f Add how Scrapy is pronounced to the docs 2021-10-15 20:38:53 -03:00
Erik Kemperman ca320feb2a
Add LOG_FILE_APPEND to settings 2021-10-15 16:36:41 +02:00
谭九鼎 3243aa2cd5
docs: fix typo 2021-10-14 10:18:26 +08:00
Jake Herbst d08199f631
Removing unnecessary line from docs to prevent test failure (#5274) 2021-10-12 19:20:09 +02:00
Andrey Rahmatullin 1ae53a9d7c
Merge pull request #5256 from rydwhelchel/documentation-changes 
docs: restructed phrasing for clarity
 2021-10-12 16:49:31 +05:00
Andrey Rahmatullin d90b2a2bfc
Merge pull request #5250 from iDeepverma/URLLENGTH_LIMIT-doc 
Documentation update for URLLENGTH_LIMIT
 2021-10-12 16:48:42 +05:00
Andrey Rahmatullin 8a743ae43a
Merge pull request #5266 from MarvinPetzoldt/patch-1 
Fixed documentation example
 2021-10-12 16:46:51 +05:00
Andrey Rahmatullin ce96a961e1
Merge pull request #5271 from kianmeng/fix-typos 
Fix typos
 2021-10-12 16:46:05 +05:00
Andrey Rahmatullin eb0e99dc20
Merge pull request #5254 from PeterMorrison1/FilePipeline-expire-doc 
Document file expiration method in media-pipeline (#5120)
 2021-10-12 16:41:16 +05:00