RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,225 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

11225 Commits

Author SHA1 Message Date
Mikhail Korobov 49942026d8
Merge pull request #5547 from Laerte/master 
Deprecate the `scrapy.downloadermiddlewares.decompression` module
 2022-07-05 12:44:25 +05:00
Laerte Pereira fe08a119d9
chore: import only used function 2022-06-30 10:46:00 -03:00
Laerte Pereira c4c816624f
chore: Deprecate the `scrapy.downloadermiddlewares.decompression` module 2022-06-30 10:42:01 -03:00
Mikhail Korobov b51b52ff37
Merge pull request #5526 from mtabbasi/headers-multiple-values 
Merge values of multiple headers with same name
 2022-06-27 23:03:01 +05:00
Mikhail Korobov 500dae82e2
Merge pull request #3696 from Gallaecio/allow-customizing-export-column-names 
Allow customizing export column names
 2022-06-27 00:26:05 +05:00
Mikhail Korobov 7120c3df33
Merge pull request #5542 from cyai/cyai-patch-2 
MAINT: Updated f-string format
 2022-06-26 23:46:26 +05:00
Vardhaman 387326fad4
MAINT: Updated f-string format 
Updated the code with the f-string method for better and cleaner understanding.
 2022-06-23 14:40:49 +05:30
Adrián Chaves 34e4ed72ea Document how DOWNLOAD_DELAY affects per-domain concurrency 2022-06-21 12:46:54 +02:00
Emanuele d8223adfac
Typo: cleanup (verb) → clean up (#5538) 2022-06-20 11:54:05 +02:00
Adrián Chaves e3e69d1209
Pin documentation requirements (#5536) 2022-06-20 11:46:13 +02:00
Adrián Chaves 54bfb9649b
Cover #5525 in the 2.6.2 release notes (#5535) 2022-06-17 11:51:02 +02:00
Adrián Chaves 4ef71829b2
If TWISTED_REACTOR is None, reuse any pre-installed reactor (#5528) 2022-06-17 10:37:27 +02:00
Adrián Chaves ec5cf3e9ce test_feedexport: solve ordered comparison issues 2022-06-17 09:10:18 +02:00
Adrián Chaves bc285f393c Revert "Revert "test_feedexport: remove ordered=False"" 
This reverts commit 516e2d6ec0.
 2022-06-17 09:00:39 +02:00
Adrián Chaves 516e2d6ec0 Revert "test_feedexport: remove ordered=False" 
This reverts commit 24f382fa45.
 2022-06-17 08:55:45 +02:00
Michel Ace 6e878490e8
Support and prefer the .jsonl file extension (#4848) 2022-06-17 08:37:14 +02:00
Adrián Chaves 3729c6d266 Remove unused import and redundant import 2022-06-17 08:33:34 +02:00
Adrián Chaves 24f382fa45 test_feedexport: remove ordered=False 2022-06-17 08:31:45 +02:00
Adrián Chaves 1b9ed22bec Remove Python < 3.7 leftover 2022-06-17 08:27:17 +02:00
Adrián Chaves 3e994bda45 Update for Python 3.7+ 2022-06-16 20:53:14 +02:00
Kromitvs 9e265a2c1f
Mind body to choose response class in cache, FTP and HTTP/1.0 (#4873) 2022-06-16 20:52:19 +02:00
Adrián Chaves e8503217fb Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2022-06-16 20:19:52 +02:00
Mohammadtaher Abbasi 892c2a4655 delete unnecessary test 2022-06-13 23:46:42 +04:30
Mohammadtaher Abbasi a135d6caf0
Move Mapping import line up 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2022-06-13 15:24:30 +04:30
Mikhail Korobov de0e2ccd7b
Merge pull request #5451 from FJMonteroInformatica/master 
make test files follow HTML conventions
 2022-06-12 00:46:45 +05:00
Mohammadtaher Abbasi 6a0bcf97cc Merge values of multiple headers with same name (#5515) 2022-06-11 23:52:21 +04:30
Mohammadtaher Abbasi ddfd192b70 add tests for multiple headers with same name 2022-06-11 23:51:34 +04:30
Mikhail Korobov e2769cfe48
Merge pull request #5524 from Laerte/master 
Remove minimal versions that Scrapy is tested against, cleanup
 2022-06-09 22:07:46 +05:00
Laerte Pereira 197aca2c94
docs: Remove leftover 2022-06-09 10:11:49 -03:00
Laerte Pereira c4c5c9f258
docs: Remove minimal versions paragraphs 2022-06-09 10:00:44 -03:00
Laerte Pereira 6770d1ec62
chore(tests): Remove validations for unsupported modules versions 2022-06-09 09:08:09 -03:00
Laerte Pereira 2e6721fd86
docs: Update minimal versions that Scrapy is tested against 2022-06-09 08:37:01 -03:00
Laerte Pereira 407562b38b
Drop Python 3.6 support (#5514) 
* chore: Drop Python 3.6 support

* Attend PR comments

* Tweak versions

* Update dependencies version

* fix: Ubuntu workflow

* fix windows workflow

* chore: Remove comment

* update `install_requires` dependencies versions

* move lxml to main pinned requirements

* Attend code-review comments

* remove non-pinned 3.7 from windows workflow

* simplify condition

* lint

* remove paragraph

* refactor

* remove leftover
 2022-06-09 00:25:03 -03:00
Adrián Chaves c5627af15b
Centralize request fingerprints (#4524) 
Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2022-06-07 18:44:54 +02:00
Mikhail Korobov 517cbc8d71
Merge pull request #5508 from mtabbasi/thumb_path-access-item 
pass on item to thumb_path function as additional argument
 2022-06-05 03:42:45 +05:00
Mikhail Korobov bafe874ecc
Merge pull request #5358 from burakcank/signal_coroutine 
Document coroutines for signals.
 2022-05-28 13:38:05 +05:00
Mikhail Korobov 99cddecd28
Merge pull request #5497 from AngelikiBoura/issue_3264/command_parser_unhandled_error 
Issue #3264, fix error handling when spider is not matched
 2022-05-28 13:26:23 +05:00
Eugenio Lacuesta 2c65066ad9
Avoid exceptions on copy 2022-05-27 19:56:42 -03:00
Eugenio Lacuesta 1b1d518e2b
Merge remote-tracking branch 'upstream/master' into case-insensitive-dict 2022-05-27 18:45:10 -03:00
Mohammadtaher Abbasi 896f16f2de make thumb_path method backwards compatible 2022-05-25 23:59:25 +04:30
Mohammadtaher Abbasi 5c586d78f0 add tests 2022-05-25 23:58:09 +04:30
Mohammadtaher Abbasi f39def4492 add docs 2022-05-25 23:57:38 +04:30
Mohammadtaher Abbasi 8633190012 pass on item to thumb_path function as additional argument resolves #5504 2022-05-24 14:51:17 +04:30
Andreas Tziortziortziopoulos b5c15d87ff [issue3264] Separate test for not matched spider to a url 2022-05-22 12:19:20 +03:00
Maxime Nannan 078622cfb0
Fix file expiration issue with GCS (#5318) 2022-05-20 08:30:06 +02:00
Andrey Rahmatullin 965fde24a4 Pin mitmproxy to < 8 for now (#5459) 2022-05-20 07:17:09 +02:00
Adrián Chaves 1c1cd5d8ea Update the 2.6.2 release notes 2022-05-20 07:05:26 +02:00
Adrián Chaves 84c29a286f
Unset the release date of still-unreleased 2.6.2 (#5503) 2022-05-20 06:45:38 +02:00
Eugenio Lacuesta 1c031b8a8d
Underscore CaseInsensitiveDict normkey/normvalue 2022-05-12 13:10:08 -03:00
Eugenio Lacuesta 7297ae566f
Merge remote-tracking branch 'upstream/master' into case-insensitive-dict 2022-05-12 12:43:41 -03:00