RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,652 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

10652 Commits

Author SHA1 Message Date
Adrián Chaves ee239d2451
Merge pull request #6595 from wRAR/update-tools 
Update tool versions
 2024-12-30 09:56:08 +01:00
Andrey Rakhmatullin f7af7b282d Bump mypy and stubs. 2024-12-29 16:45:26 +05:00
Andrey Rakhmatullin 4a0c05749c Bump tool versions. 2024-12-29 14:29:27 +05:00
Andrey Rakhmatullin cc484efd43 Replace isort with the ruff isort rules. 2024-12-29 14:15:16 +05:00
Mikhail Korobov ba33a40365
Merge pull request #6587 from Gallaecio/settings-addons 
Add ADDONS to the settings template for new projects
 2024-12-18 23:05:14 +05:00
Adrián Chaves c5ed0fd45c Add ADDONS to the settings template for new projects 2024-12-18 17:05:51 +01:00
Laerte Pereira a195af304d
Deprecate w3lib objects importable from scrapy.utils.url (#6586) 2024-12-18 11:50:44 +05:00
Adrián Chaves 21b9ba717c
Allow customizing logged software versions (#6582) 
Co-authored-by: Grammy Jiang <grammy.jiang@gmail.com>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-12-16 14:46:23 +01:00
Andrey Rakhmatullin 7dd92e6e43
Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584) 2024-12-16 09:44:36 +01:00
Andrey Rakhmatullin 57a5460529
Merge pull request #6581 from wRAR/ruff-rules-3 
Ruff: enable other useful rules, turn on autofixing
 2024-12-13 13:14:19 +04:00
Andrey Rakhmatullin c003fc0841 Add flake8 warning rules to ruff. 2024-12-12 21:07:01 +05:00
Andrey Rakhmatullin 1e4c81e9dc Add Perflint rules to ruff. 2024-12-12 21:03:34 +05:00
Andrey Rakhmatullin c2832ed131 Add flake8-pyi rules to ruff. 2024-12-12 20:44:28 +05:00
Andrey Rakhmatullin 93644f2c30 Add flake8-pie rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin e7595837a6 Add flake8-raise rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin 897e124a27 Add flake8-return rules to ruff. 2024-12-12 20:32:25 +05:00
Andrey Rakhmatullin 802c67072c Enable ruff --fix. 2024-12-12 20:12:55 +05:00
Adrián Chaves 5c2df5cf2a
Contributing: add a section on finding work (#6575) 2024-12-12 11:38:30 +01:00
Andrey Rakhmatullin cde0845ab2
Ruff: migrate pyupgrade and bandit, enable some other rules (#6577) 2024-12-10 19:53:27 +01:00
Andrey Rakhmatullin b423e971ae
Merge pull request #6576 from wRAR/ruff-rules-1 
Add flake8 plugin rules to ruff
 2024-12-10 14:34:06 +04:00
Andrey Rakhmatullin f4d8d6d8ac Tidy up noqa comments. 2024-12-10 14:58:45 +05:00
Andrey Rakhmatullin ba30f64268 Remove flake8. 2024-12-10 14:52:16 +05:00
Andrey Rakhmatullin 0d7a5e760d Fix building docs. 2024-12-10 01:42:03 +05:00
Andrey Rakhmatullin d47f142d0f Add flake8-type-checking rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin d6bf1464b8 Add pydocstyle/flake8-docstrings rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin e53d6f09bc Add flake8-comprehensions and flake8-debugger rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin c184f12ab5 Add flake8-bugbear rules to ruff. 2024-12-10 01:27:49 +05:00
Emery Berger 5680bee968
Made path absolute to enable running pytest from a different directory. (#6567) 2024-12-09 11:01:00 +01:00
Andrey Rakhmatullin cc146b9df7
Add ruff with basic rules. (#6565) 2024-12-09 10:47:47 +01:00
Andrey Rakhmatullin 37be9da4e4
Merge pull request #6560 from wRAR/tls-method-tests 
Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code.
 2024-11-25 19:13:36 +04:00
Andrey Rakhmatullin 4dcc04be48 Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code. 2024-11-24 12:44:48 +04:00
Swayam Gupta 8c23da943c
Integrating configs into pyproject.toml (#6547) 2024-11-19 19:21:15 +05:00
Andrey Rakhmatullin efb53aafdc Fix a typo that broke PDF builds. 2024-11-18 15:44:52 +05:00
Andrey Rakhmatullin b1f9e56693 Bump version: 2.11.2 → 2.12.0 2024-11-18 13:08:05 +05:00
Adrián Chaves 10089c6fe2
2.12 release notes (#6226) 
* Cover 2.12 in the release notes up to 9bb973dc54

* Add one more highlight

* Better merge of the news entries.

* Cover 2.12 in the release notes up to 642af40.

* Cover 2.12 in the release notes up to 7a0a34b.

* Cover 2.12 in the release notes up to b4bad97.

* Add not yet merged PRs #6463, #6507, #6511 to the 2.12 release notes.

* Cover 2.12 in the release notes up to d85c39f, small fixes.

* Cover 2.12 in the release notes up to d215669.

* Cover #6527 in the release notes.

* Address PR feedback.

* Cover recent PRs.

* Finalize the 2.12.0 release notes, small additional fixes.

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-11-18 13:07:32 +05:00
Andrey Rakhmatullin 212e848402
Merge pull request #6545 from wRAR/mitmproxy-dhparam 
Commit mitmproxy-dhparam.pem.
 2024-11-15 22:24:46 +05:00
Andrey Rakhmatullin feea3a0f67 Commit mitmproxy-dhparam.pem. 2024-11-15 21:08:18 +05:00
Andrey Rakhmatullin 87b2300831
Merge pull request #6543 from scrapy/pep740-attestations 
Verified PyPI releases (a.k.a. PEP740)
 2024-11-15 12:41:43 +05:00
Daniel Graña dc4d6d16ea Verified PyPI releases (a.k.a. PEP740) 2024-11-15 00:09:00 -03:00
Andrey Rakhmatullin 30fb54f47e
Merge pull request #6542 from nicklaustrup/Tests-contracts-swallowed-exceptions 
Fix - return in finally can swallow exception (#6505)
 2024-11-15 00:48:06 +05:00
Nicholas Laustrup bfcee452b0 Added failing test cases to tests/test_contracts.py and fixed corresponding methods + removed pylint comments 2024-11-14 10:40:12 -08:00
Andrey Rakhmatullin ab5cb7c7d9
Merge pull request #6540 from wRAR/build_from_settings 
Remove build_from_settings() and deprecate from_settings() code
 2024-11-14 20:40:54 +05:00
Andrey Rakhmatullin 929d665a74 Address PR feedback. 2024-11-14 19:35:56 +05:00
Andrey Rakhmatullin 2ad5f0c12b Extract duplicated code. 2024-11-14 13:03:04 +05:00
Andrey Rakhmatullin 6aa4d2b4ab Prefer crawler.settings over settings in media pipelines. 2024-11-14 02:03:43 +05:00
Andrey Rakhmatullin 28fafbb8c5 Modernize the media pipeline initialization API. 2024-11-14 01:29:51 +05:00
Robert Palmer 261c4b61dc
Enhancement: Update docs to include IgnoreRequest details (#6506) 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-11-13 22:47:39 +05:00
Andrey Rakhmatullin 8700a5b7a9 Deprecate build_from_crawler() calling from_settings(). 2024-11-13 00:20:10 +05:00
Andrey Rakhmatullin eda1a8a7c5 Deprecate MiddlewareManager.from_settings(). 2024-11-12 22:57:39 +05:00
Andrey Rakhmatullin 499e7e8aa6 Add from_crawler() to components that only had from_settings(). 2024-11-12 21:12:32 +05:00