RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,668 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

10668 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 7dfbecd392 Fix tracking of coverage in subprocesses. 2025-01-07 20:41:48 +05:00
Andrey Rakhmatullin 59fcb9b93c
Improve internal refs to scrapy.Request and scrapy.Selector (#6526) 
* Improve internal refs to scrapy.Selector.

* Improve internal refs to scrapy.Request.

* More scrapy.http fixes.

* Fix FormRequest refs.

* More fixes.

* Simplifications.

* Last fixes.

* Add the parsel intersphinx.
 2025-01-07 16:18:18 +05:00
Andrey Rakhmatullin 5d3aa80ad1
Switch CI to codecov/codecov-action and enable it on Windows. (#6609) 2025-01-07 10:52:26 +01:00
Andrey Rakhmatullin 4869315d10
Install libjpeg-dev on pinned envs to be able to install Pillow. (#6607) 2025-01-07 10:46:12 +01:00
Laerte Pereira f2234c5b96
Fix Crawler.request_fingerprinter typing (#6605) 2025-01-07 10:40:49 +01:00
Andrey Rakhmatullin 4d31277bc6
Explicitly mark re-exports. (#6579) 2025-01-02 23:48:14 +01:00
Andrey Rakhmatullin c330a399dc
Merge pull request #6601 from wRAR/ruff-rules-5 
Add more Ruff rules, do some pylint cleanups
 2025-01-02 17:38:15 +04:00
Andrey Rakhmatullin 176ae348c5 Reformat long REFERRER_POLICY. 2025-01-02 18:14:18 +05:00
Andrey Rakhmatullin 6ae5b92671
Drop the remaining unittest.main() blocks. (#6602) 2025-01-02 12:45:04 +01:00
Arthur b10d46d280
Fix the calculate_final_stats method (#6599) 2025-01-02 12:36:28 +01:00
Andrey Rakhmatullin dc706d4fc3 Remove useless pylint: disable lines. 2025-01-02 12:32:25 +05:00
Andrey Rakhmatullin b70443f2d0 Split ruff and pylint ignores into two categories, some pylint cleanup. 2025-01-02 01:03:42 +05:00
Andrey Rakhmatullin c87354cd46 Enable SIM Ruff rules. 2025-01-02 01:03:36 +05:00
Andrey Rakhmatullin 273620488c Enable PTH Ruff rules. 2025-01-01 22:21:51 +05:00
Andrey Rakhmatullin f44ca39fa2 Enable FLY Ruff rules. 2025-01-01 21:50:02 +05:00
Andrey Rakhmatullin 838ff99f37 Enable RUF Ruff rules. 2025-01-01 21:41:08 +05:00
Adrián Chaves ee239d2451
Merge pull request #6595 from wRAR/update-tools 
Update tool versions
 2024-12-30 09:56:08 +01:00
Andrey Rakhmatullin f7af7b282d Bump mypy and stubs. 2024-12-29 16:45:26 +05:00
Andrey Rakhmatullin 4a0c05749c Bump tool versions. 2024-12-29 14:29:27 +05:00
Andrey Rakhmatullin cc484efd43 Replace isort with the ruff isort rules. 2024-12-29 14:15:16 +05:00
Mikhail Korobov ba33a40365
Merge pull request #6587 from Gallaecio/settings-addons 
Add ADDONS to the settings template for new projects
 2024-12-18 23:05:14 +05:00
Adrián Chaves c5ed0fd45c Add ADDONS to the settings template for new projects 2024-12-18 17:05:51 +01:00
Laerte Pereira a195af304d
Deprecate w3lib objects importable from scrapy.utils.url (#6586) 2024-12-18 11:50:44 +05:00
Adrián Chaves 21b9ba717c
Allow customizing logged software versions (#6582) 
Co-authored-by: Grammy Jiang <grammy.jiang@gmail.com>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-12-16 14:46:23 +01:00
Andrey Rakhmatullin 7dd92e6e43
Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584) 2024-12-16 09:44:36 +01:00
Andrey Rakhmatullin 57a5460529
Merge pull request #6581 from wRAR/ruff-rules-3 
Ruff: enable other useful rules, turn on autofixing
 2024-12-13 13:14:19 +04:00
Andrey Rakhmatullin c003fc0841 Add flake8 warning rules to ruff. 2024-12-12 21:07:01 +05:00
Andrey Rakhmatullin 1e4c81e9dc Add Perflint rules to ruff. 2024-12-12 21:03:34 +05:00
Andrey Rakhmatullin c2832ed131 Add flake8-pyi rules to ruff. 2024-12-12 20:44:28 +05:00
Andrey Rakhmatullin 93644f2c30 Add flake8-pie rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin e7595837a6 Add flake8-raise rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin 897e124a27 Add flake8-return rules to ruff. 2024-12-12 20:32:25 +05:00
Andrey Rakhmatullin 802c67072c Enable ruff --fix. 2024-12-12 20:12:55 +05:00
Adrián Chaves 5c2df5cf2a
Contributing: add a section on finding work (#6575) 2024-12-12 11:38:30 +01:00
Andrey Rakhmatullin cde0845ab2
Ruff: migrate pyupgrade and bandit, enable some other rules (#6577) 2024-12-10 19:53:27 +01:00
Andrey Rakhmatullin b423e971ae
Merge pull request #6576 from wRAR/ruff-rules-1 
Add flake8 plugin rules to ruff
 2024-12-10 14:34:06 +04:00
Andrey Rakhmatullin f4d8d6d8ac Tidy up noqa comments. 2024-12-10 14:58:45 +05:00
Andrey Rakhmatullin ba30f64268 Remove flake8. 2024-12-10 14:52:16 +05:00
Andrey Rakhmatullin 0d7a5e760d Fix building docs. 2024-12-10 01:42:03 +05:00
Andrey Rakhmatullin d47f142d0f Add flake8-type-checking rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin d6bf1464b8 Add pydocstyle/flake8-docstrings rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin e53d6f09bc Add flake8-comprehensions and flake8-debugger rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin c184f12ab5 Add flake8-bugbear rules to ruff. 2024-12-10 01:27:49 +05:00
Emery Berger 5680bee968
Made path absolute to enable running pytest from a different directory. (#6567) 2024-12-09 11:01:00 +01:00
Andrey Rakhmatullin cc146b9df7
Add ruff with basic rules. (#6565) 2024-12-09 10:47:47 +01:00
Andrey Rakhmatullin 37be9da4e4
Merge pull request #6560 from wRAR/tls-method-tests 
Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code.
 2024-11-25 19:13:36 +04:00
Andrey Rakhmatullin 4dcc04be48 Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code. 2024-11-24 12:44:48 +04:00
Swayam Gupta 8c23da943c
Integrating configs into pyproject.toml (#6547) 2024-11-19 19:21:15 +05:00
Andrey Rakhmatullin efb53aafdc Fix a typo that broke PDF builds. 2024-11-18 15:44:52 +05:00
Andrey Rakhmatullin b1f9e56693 Bump version: 2.11.2 → 2.12.0 2024-11-18 13:08:05 +05:00