RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,937 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

81 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 804ae167df
Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
Andrew Murray 5afc9b0221
Automatically transpose images using EXIF data (#6975) 
* Automatically transpose images using EXIF data

* Added test

---------

Co-authored-by: Andrew Murray <radarhere@users.noreply.github.com>
 2025-10-24 16:08:23 +05:00
M Ikram Ullah Khan 9e83a58643
Add type check in get_media_requests to enforce image / file urls as a list (#6949) 2025-07-19 11:54:35 +02:00
Andrey Rakhmatullin 4279f2837c
Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
Andrey Rakhmatullin b4d11b8b25
Further reduce deps on unittest. (#6884) 2025-06-11 01:28:09 +02:00
Andrey Rakhmatullin eb654aa1a8
Convert remaining unittest assert* calls, use the tmp_path fixture. (#6725) 2025-03-11 18:00:36 +01:00
Andrey Rakhmatullin 02ed71d887
Converting tests to plain asserts, part 6. (#6709) 2025-03-09 20:20:24 +01:00
Andrey Rakhmatullin 8d92c28a16
Switch to pytest.raises(). (#6680) 
* Switch to pytest.raises().

* Add matches= to broad pytest.raises().

* Adjust the test_nonserializable_object() regex for Python <= 3.11.

* Adjust the test_nonserializable_object() regex for PyPy.

* Adjust other test exception regexes for PyPy.

* Cleanup.
 2025-02-27 23:13:04 +05:00
Andrey Rakhmatullin 93644f2c30 Add flake8-pie rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin 6aa4d2b4ab Prefer crawler.settings over settings in media pipelines. 2024-11-14 02:03:43 +05:00
Andrey Rakhmatullin 28fafbb8c5 Modernize the media pipeline initialization API. 2024-11-14 01:29:51 +05:00
Andrey Rakhmatullin ce5a132f12
Run and fix linkcheck. (#6524) 2024-11-04 11:40:07 +01:00
Andrey Rakhmatullin d85c39f5bc
Deprecation removals. (#6500) 
* Deprecation removals.

* Clean up the default pytest filterwarnings.

* Remove test_get_images_old().

* Redo boto-requiring test filtering.

* Remove an unused function.

* Improve the Crawler.crawl() error message.

* Fix the test.
 2024-10-31 18:06:22 +05:00
Andrey Rakhmatullin c9095ef927 Remove --keep-runtime-typing from pyupgrade. 2024-10-17 21:26:02 +05:00
Vsevolod Breus 5391663072
Drop Python 3.8 Support (#6472) 2024-10-16 10:03:16 +02:00
Andrey Rakhmatullin feb0b8f7dc Add pyupgrade. 2024-06-05 22:57:18 +05:00
Laerte Pereira 26a16f2c43 Fix tests 2024-02-28 16:36:19 -03:00
Andrey Rakhmatullin b4acf5c827
Fix and remove most of the entries from the mypy ignore list (#6137) 2023-11-07 09:34:35 +01:00
freddiewanah c4f0aa4fdf
Refactor test cases to improve unit test quality (#5986) 2023-07-26 13:09:03 +02:00
Adrián Chaves b07d3f85a3 Upgrade CI tools 2023-02-02 06:37:40 +01:00
Andrey Rakhmatullin ccd1385e11
Merge branch 'master' into no-callback 2023-01-30 15:43:05 +05:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Adrián Chaves 9272c4af0c Merge remote-tracking branch 'scrapy/master' into no-callback 2023-01-25 17:41:41 +01:00
Emmanuel Rondan 8ee4817471 reaplying black, fixing conflicts and ignoring bandit checks on test directory 2023-01-20 10:55:46 -03:00
Adrián Chaves 50500a6b28 Implement a NO_CALLBACK value for Request.callback 2023-01-19 17:14:18 +01:00
Josh Soref 1e44d4614e spelling: straight 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Josh Soref 6aa5374bd3 spelling: received 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Josh Soref 1a5cf00db7 spelling: overridden 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Andrey Rahmatullin bbe24d79a5
Fix test issues 2022-11-07 17:08:54 +05:00
Andrey Rahmatullin 6c0890ff54
Simplify the changes after the merge 2022-11-07 16:36:54 +05:00
Andrey Rahmatullin b03d84e9fb
Merge branch 'master' into 3689-update 2022-11-07 16:33:21 +05:00
Andrey Rakhmatullin ca50af6453 Remove an unused import. 2022-10-27 17:12:26 +06:00
Andrey Rahmatullin 5780ccba55
Merge branch 'master' into deprecate-NoimagesDrop 2022-10-27 15:46:32 +05:00
Mikhail Korobov 1054689593 remove unused imports 
thanks flake8!
 2022-07-18 23:53:30 +05:00
Mikhail Korobov e248360e6e remove compatibility code from tests for the case dataclasses module is not available 
It was Python 3.6 compat code, and Python 3.6 support is dropped.
 2022-07-18 23:49:08 +05:00
Mohammadtaher Abbasi 5c586d78f0 add tests 2022-05-25 23:58:09 +04:30
PluT00 cc16af35af Add deprecation warning test for scrapy.pipelines.images.NoimagesDrop 2022-05-03 11:29:21 +03:00
Adrián Chaves 308a58aa27
Update CI to support Twisted 21.2.0 (#5027) 2021-03-19 18:39:44 +01:00
D R Siddhartha 0c24cdb257 Improved warning messages a little 2020-10-04 02:09:21 +05:30
drs-11 35e2d25689 Merge remote-tracking branch 'upstream/master' into 3689-update 2020-09-02 20:51:09 +05:30
drs-11 6565adc471 added test case for get_images 2020-09-02 20:44:26 +05:30
Ammar Najjar 58ca8bbf6d Use f-strings (#4307) 2020-08-26 07:11:05 +02:00
drs-11 1f9598ada4 Update branch to latest 2020-08-25 02:14:22 +05:30
drs-11 8b84a65a6b cleaned up code relating to issue #3689 2020-08-25 00:30:17 +05:30
Eugenio Lacuesta ec98dabfab
Support for dataclass and attrs items (#3881) 2020-06-14 11:45:27 +02:00
Eugenio Lacuesta 3eeecb4273
Merge branch 'master' into flake8-max-line-length 2020-05-15 11:11:20 -03:00
Marc Hernández fffb0a5b6a
replace set constructor with set literals (#4573) 2020-05-15 15:23:09 +02:00
Eugenio Lacuesta 6f8758624c
Flake8: remove F841 2020-05-11 13:50:34 -03:00
Eugenio Lacuesta 182394bcec
E501 compliance (tests) 2020-03-22 23:29:30 -03:00