RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,191 Commits 49 Branches 132 Tags 155 MiB
Commit Graph

11191 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 1e4c81e9dc Add Perflint rules to ruff. 2024-12-12 21:03:34 +05:00
Andrey Rakhmatullin c2832ed131 Add flake8-pyi rules to ruff. 2024-12-12 20:44:28 +05:00
Andrey Rakhmatullin 93644f2c30 Add flake8-pie rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin e7595837a6 Add flake8-raise rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin 897e124a27 Add flake8-return rules to ruff. 2024-12-12 20:32:25 +05:00
Andrey Rakhmatullin 802c67072c Enable ruff --fix. 2024-12-12 20:12:55 +05:00
Adrián Chaves 5c2df5cf2a
Contributing: add a section on finding work (#6575) 2024-12-12 11:38:30 +01:00
Andrey Rakhmatullin cde0845ab2
Ruff: migrate pyupgrade and bandit, enable some other rules (#6577) 2024-12-10 19:53:27 +01:00
Andrey Rakhmatullin b423e971ae
Merge pull request #6576 from wRAR/ruff-rules-1 
Add flake8 plugin rules to ruff
 2024-12-10 14:34:06 +04:00
Andrey Rakhmatullin f4d8d6d8ac Tidy up noqa comments. 2024-12-10 14:58:45 +05:00
Andrey Rakhmatullin ba30f64268 Remove flake8. 2024-12-10 14:52:16 +05:00
Andrey Rakhmatullin 0d7a5e760d Fix building docs. 2024-12-10 01:42:03 +05:00
Andrey Rakhmatullin d47f142d0f Add flake8-type-checking rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin d6bf1464b8 Add pydocstyle/flake8-docstrings rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin e53d6f09bc Add flake8-comprehensions and flake8-debugger rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin c184f12ab5 Add flake8-bugbear rules to ruff. 2024-12-10 01:27:49 +05:00
Emery Berger 5680bee968
Made path absolute to enable running pytest from a different directory. (#6567) 2024-12-09 11:01:00 +01:00
Andrey Rakhmatullin cc146b9df7
Add ruff with basic rules. (#6565) 2024-12-09 10:47:47 +01:00
Andrey Rakhmatullin 37be9da4e4
Merge pull request #6560 from wRAR/tls-method-tests 
Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code.
 2024-11-25 19:13:36 +04:00
Andrey Rakhmatullin 4dcc04be48 Add tests for DOWNLOADER_CLIENT_TLS_METHOD, remove dead code. 2024-11-24 12:44:48 +04:00
Swayam Gupta 8c23da943c
Integrating configs into pyproject.toml (#6547) 2024-11-19 19:21:15 +05:00
Andrey Rakhmatullin efb53aafdc Fix a typo that broke PDF builds. 2024-11-18 15:44:52 +05:00
Andrey Rakhmatullin b1f9e56693 Bump version: 2.11.2 → 2.12.0 2024-11-18 13:08:05 +05:00
Adrián Chaves 10089c6fe2
2.12 release notes (#6226) 
* Cover 2.12 in the release notes up to 9bb973dc54

* Add one more highlight

* Better merge of the news entries.

* Cover 2.12 in the release notes up to 642af40.

* Cover 2.12 in the release notes up to 7a0a34b.

* Cover 2.12 in the release notes up to b4bad97.

* Add not yet merged PRs #6463, #6507, #6511 to the 2.12 release notes.

* Cover 2.12 in the release notes up to d85c39f, small fixes.

* Cover 2.12 in the release notes up to d215669.

* Cover #6527 in the release notes.

* Address PR feedback.

* Cover recent PRs.

* Finalize the 2.12.0 release notes, small additional fixes.

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-11-18 13:07:32 +05:00
Andrey Rakhmatullin 212e848402
Merge pull request #6545 from wRAR/mitmproxy-dhparam 
Commit mitmproxy-dhparam.pem.
 2024-11-15 22:24:46 +05:00
Andrey Rakhmatullin feea3a0f67 Commit mitmproxy-dhparam.pem. 2024-11-15 21:08:18 +05:00
Andrey Rakhmatullin 87b2300831
Merge pull request #6543 from scrapy/pep740-attestations 
Verified PyPI releases (a.k.a. PEP740)
 2024-11-15 12:41:43 +05:00
Daniel Graña dc4d6d16ea Verified PyPI releases (a.k.a. PEP740) 2024-11-15 00:09:00 -03:00
Andrey Rakhmatullin 30fb54f47e
Merge pull request #6542 from nicklaustrup/Tests-contracts-swallowed-exceptions 
Fix - return in finally can swallow exception (#6505)
 2024-11-15 00:48:06 +05:00
Nicholas Laustrup bfcee452b0 Added failing test cases to tests/test_contracts.py and fixed corresponding methods + removed pylint comments 2024-11-14 10:40:12 -08:00
Andrey Rakhmatullin ab5cb7c7d9
Merge pull request #6540 from wRAR/build_from_settings 
Remove build_from_settings() and deprecate from_settings() code
 2024-11-14 20:40:54 +05:00
Andrey Rakhmatullin 929d665a74 Address PR feedback. 2024-11-14 19:35:56 +05:00
Andrey Rakhmatullin 2ad5f0c12b Extract duplicated code. 2024-11-14 13:03:04 +05:00
Andrey Rakhmatullin 6aa4d2b4ab Prefer crawler.settings over settings in media pipelines. 2024-11-14 02:03:43 +05:00
Andrey Rakhmatullin 28fafbb8c5 Modernize the media pipeline initialization API. 2024-11-14 01:29:51 +05:00
Robert Palmer 261c4b61dc
Enhancement: Update docs to include IgnoreRequest details (#6506) 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-11-13 22:47:39 +05:00
Andrey Rakhmatullin 8700a5b7a9 Deprecate build_from_crawler() calling from_settings(). 2024-11-13 00:20:10 +05:00
Andrey Rakhmatullin eda1a8a7c5 Deprecate MiddlewareManager.from_settings(). 2024-11-12 22:57:39 +05:00
Andrey Rakhmatullin 499e7e8aa6 Add from_crawler() to components that only had from_settings(). 2024-11-12 21:12:32 +05:00
Andrey Rakhmatullin f796d8780c Add tests for MediaPipeline.from_crawler() and related code. 2024-11-12 21:08:04 +05:00
Andrey Rakhmatullin 83d4939d41 Deprecate scrapy.extensions.feedexport.build_storage() and simplify _get_storage(). 2024-11-11 22:15:08 +05:00
Andrey Rakhmatullin eda3a89b3f Remove build_from_settings(). 2024-11-11 16:44:47 +05:00
Andrey Rakhmatullin b042ad255d
Address some previously ignored pylint messages. (#6531) 2024-11-11 11:49:52 +01:00
Mikhail Korobov bcef96570b
Merge pull request #6527 from Gallaecio/dont-throttle-meta 
Replace Slot.throttle with Request.meta['dont_throttle']
 2024-11-08 14:25:37 +05:00
Adrián Chaves dc3ebb6cf7 Refactor the docs 2024-11-07 16:38:48 +01:00
Adrián Chaves 2a4b7fe0f8 dont_throttle → autothrottle_dont_adjust_delay 2024-11-07 16:17:16 +01:00
Adrián Chaves b244ea7ac0 Add the missing import to the docs example 2024-11-05 20:05:58 +01:00
Adrián Chaves 5862216bb1 Fix docs example 2024-11-05 19:55:28 +01:00
Adrián Chaves f57fc454be Replace Slot.throttle with Request.meta['dont_throttle'] 2024-11-05 19:44:30 +01:00
Rohit Kumar Singh d2156696c4
Deprecate unused scrapy utils (#6519) 
* Added deprecation warnings for unused Scrapy.utils

* Grammatical corrections

* Exceptions class connected

* Deprecation of ScrapyJSONDecoder

* request_authenticate function deprecation

* Making all warning similar

* Added ignore statements for deprecation warning in tests

* Missing stacklevel attr. added

* Added Deprecation message
 2024-11-04 21:09:45 +05:00