RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,291 Commits 49 Branches 132 Tags 131 MiB
Commit Graph

9291 Commits

Author SHA1 Message Date
Adrián Chaves 77c055ee28 Relax Proxy-Authorization restrictions 2022-09-14 14:47:14 +02:00
Mikhail Korobov 90b8503789
Merge pull request #5611 from mtabbasi/issue-5601 
BOM should take precedence over Content-Type header when detecting the encoding

Fixes GH-5601.
 2022-09-13 15:07:50 +05:00
Andrey Rahmatullin 3f060aeb56
Merge pull request #5617 from Laerte/fix/tests-w3lib 
Fix failed tests related to w3lib
 2022-09-07 12:52:41 +05:00
Laerte Pereira 582a6bf6db
refactor: Use `safe_url_string` to standardize url output 2022-09-06 10:03:18 -03:00
Laerte Pereira 1289422284
chore: Skip `test_ipv6_default_name_resolver` test if w3lib version >= 2.0.0 2022-09-06 08:17:58 -03:00
Laerte Pereira f4bcc3e67d
fix: failed tests 2022-09-06 01:15:41 -03:00
Mohammadtaher Abbasi a988c4b78b add test 2022-08-29 17:08:30 +04:30
Mohammadtaher Abbasi e411ea94eb BOM should take precedence over Content-Type header when detecting the encoding closes #5601 2022-08-28 20:28:13 +04:30
Mikhail Korobov 52d93490f5
Merge pull request #4978 from wRAR/asyncio-parse-asyncgen-proper-rebased 
Support for async callbacks
 2022-07-30 00:34:37 +05:00
Andrey Rakhmatullin c7b90c6e1e Extract more common code. 2022-07-28 13:44:36 +05:00
Andrey Rahmatullin 0f1112f3e2
Update docs/index.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2022-07-27 23:12:43 +05:00
Andrey Rahmatullin 83ecdf1bca
Update docs/topics/components.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2022-07-27 23:12:31 +05:00
Andrey Rahmatullin faa12d1fbe
Merge pull request #2 from scrapy/asyncio-parse-fixes 
Asyncio parse fixes
 2022-07-27 12:30:09 +05:00
Aftab Alam 67011cd957
updated README.rst , added hyperlink to banner (#5284) 
Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2022-07-27 09:50:26 +05:00
Mikhail Korobov 56e2eeac10 fix typing issues by upgrading mypy 2022-07-27 09:41:12 +05:00
Mikhail Korobov 4dde2f2c36 Merge branch 'master' into asyncio-parse-fixes 2022-07-27 09:32:54 +05:00
Adrián Chaves 5862f6b8e1
Update frozen CI packages (#5574) 2022-07-25 17:25:10 +02:00
Adrián Chaves 737f39613c
Merge pull request #5573 from Gallaecio/2.6-into-master 
Merge 2.6 into the main branch
 2022-07-25 14:52:39 +02:00
Adrián Chaves 00ccb02f1f Merge remote-tracking branch 'upstream/2.6' 2022-07-25 14:13:22 +02:00
Adrián Chaves aecbccbaa5 Bump version: 2.6.1 → 2.6.2 2022-07-25 13:33:23 +02:00
Adrián Chaves af7dd16d8d
Merge pull request from GHSA-9x8m-2xpf-crp3 
* Enforce matching proxy request meta and Proxy-Authorization header

* Cover proxy credential security fix in the release notes

* Remove extra empty line

* Reword the security issue description

* Address scenario where Proxy-Authorization is unexpectedly removed by a prior middleware

* Set the release date of Scrapy 2.6.2 and 1.8.3
 2022-07-25 13:15:17 +02:00
Andrey Rakhmatullin b21c16099e Fix flake8 issues. 2022-07-22 19:18:33 +05:00
Andrey Rahmatullin a0681fb811
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2022-07-22 18:30:38 +05:00
Andrey Rahmatullin dc67100a8f
Merge pull request #1 from Gallaecio/asyncio-parse-asyncgen-proper-rebased 
Refactor the asynchronous process_spider_output documentation
 2022-07-22 18:28:56 +05:00
Mikhail Korobov 4205609051 Fixed intersphinx references 2022-07-21 18:02:56 +02:00
Mikhail Korobov f60c7ae768
Fixed heading levels in downloader middleware docs (#5567) 2022-07-20 11:01:22 +02:00
Mikhail Korobov d78f505f3d
Merge pull request #5450 from kinoute/master 
fixed detection of extensions like ".tar.gz" in URL
 2022-07-20 01:06:11 +05:00
Mikhail Korobov b103664bf4
Address 2/3 of warnings from tests (#5561) 2022-07-19 17:39:26 +02:00
Andrey Rahmatullin 44580851ff
Merge pull request #5560 from scrapy/non-existing-resolvable 
make Scrapy testing suite more robust in environments where non-existing hosts are resolvable
 2022-07-19 11:28:38 +05:00
Andrey Rahmatullin bb61b03b49
Merge pull request #5563 from scrapy/dataclasses-ftw 
remove compatibility code from tests for the case dataclasses module is not available
 2022-07-19 10:37:20 +05:00
Mikhail Korobov 1054689593 remove unused imports 
thanks flake8!
 2022-07-18 23:53:30 +05:00
Mikhail Korobov e248360e6e remove compatibility code from tests for the case dataclasses module is not available 
It was Python 3.6 compat code, and Python 3.6 support is dropped.
 2022-07-18 23:49:08 +05:00
Andrey Rahmatullin 3ef432156c
Merge pull request #5559 from scrapy/fix-intersphinx 
Fixed intersphinx references
 2022-07-17 22:15:10 +05:00
Mikhail Korobov 26c70318cb make Scrapy testing suite more robust in environments where non-existing hosts are resolvable 2022-07-17 16:47:20 +05:00
Mikhail Korobov 9b33b82a8b Fixed intersphinx references 2022-07-17 15:50:40 +05:00
Andrey Rahmatullin ebc6f9c4eb
Merge pull request #5558 from eltociear/patch-1 
Fix typo in sep-014.rst
 2022-07-17 01:12:16 +05:00
Mikhail Korobov 77a3b02523
Merge pull request #5553 from Rotzbua/fix-doc-typo 
Fix doc: `scrapy.exporter` to `scrapy.exporters`
 2022-07-17 00:21:50 +05:00
Mikhail Korobov 94a0324570
Merge pull request #5554 from Rotzbua/doc-remove-outdated 
[doc] Remove incompatible web service project
 2022-07-17 00:20:32 +05:00
Ikko Ashimine 2f13f23d92
Fix typo in sep-014.rst 
requets -> requests
 2022-07-15 18:16:23 +09:00
Rotzbua 1c7ed4f2e5
[doc] Remove incompatible web service project 
* Abandoned since 2017
* Not compatible with Python3
 2022-07-12 13:09:20 +02:00
Rotzbua 09c3a4ad08
Fix doc: `scrapy.exporter` to `scrapy.exporters` 2022-07-12 12:45:15 +02:00
Mikhail Korobov 49942026d8
Merge pull request #5547 from Laerte/master 
Deprecate the `scrapy.downloadermiddlewares.decompression` module
 2022-07-05 12:44:25 +05:00
Laerte Pereira fe08a119d9
chore: import only used function 2022-06-30 10:46:00 -03:00
Laerte Pereira c4c816624f
chore: Deprecate the `scrapy.downloadermiddlewares.decompression` module 2022-06-30 10:42:01 -03:00
Mikhail Korobov b51b52ff37
Merge pull request #5526 from mtabbasi/headers-multiple-values 
Merge values of multiple headers with same name
 2022-06-27 23:03:01 +05:00
Mikhail Korobov 500dae82e2
Merge pull request #3696 from Gallaecio/allow-customizing-export-column-names 
Allow customizing export column names
 2022-06-27 00:26:05 +05:00
Mikhail Korobov 7120c3df33
Merge pull request #5542 from cyai/cyai-patch-2 
MAINT: Updated f-string format
 2022-06-26 23:46:26 +05:00
Vardhaman 387326fad4
MAINT: Updated f-string format 
Updated the code with the f-string method for better and cleaner understanding.
 2022-06-23 14:40:49 +05:30
Emanuele d8223adfac
Typo: cleanup (verb) → clean up (#5538) 2022-06-20 11:54:05 +02:00
Adrián Chaves e3e69d1209
Pin documentation requirements (#5536) 2022-06-20 11:46:13 +02:00