2798c03bb0
Add Python 3.14 to CI. ( #6604 )
...
* Add Python 3.14 (alpha3) to CI.
* Disable mitmproxy on 3.14 for now.
* 3.14.0-alpha.4.
* 3.14.0-alpha.5
* 3.14.0-beta2.
* 3.14 release.
* Fix test_non_pickable_object.
* Fix handling of file:/path feed URIs.
* Better mocking of streams for TextTestResult.
* Do not use .php in test_file_path() as it's now a known extension.
* Fix the URL in TestFeedExporterSignals.
* Fix typing.
* Bump more envs to 3.14.
* Silence pylint.
* Fix another test for .php handling change.
* Remove test_install_asyncio_reactor.
* More bumps to 3.14.
* Revert docs-tests to use 3.13.
* Debug options for Windows.
* Re-enable xdist.
* Revert Windows PYTEST_ADDOPTS.
* Silence loop policy deprecation warnings.
* Restore a lost pylint suppression.
* Update asyncio_enabled_reactor_same_loop.py to new Twisted.
* Fix RobotFileParser tests for Python 3.14.5.
2026-05-12 23:49:37 +05:00
a8e99aeb2e
Fix botocore-pinned and pypy3-* ( #7363 )
2026-03-26 08:21:58 +01:00
6fe27ba33e
Add more no-reactor tests ( #7259 )
...
* Generic changes and scrapy bench.
* scrapy check.
* scrapy crawl.
* scrapy fetch.
* scrapy parse.
* scrapy runspider.
* scrapy shell.
* Skip httpx tests on default-reactor.
* Review requires_reactor marks.
* Make test functions that require an event loop async def.
* Improve test_pending_asyncio_tasks().
* Add Mac OS DNS error.
* Refactor most of test_scheduler.py.
* Finish refactoring DownloaderAwareSchedulerTestMixin.
* Refactor test_engine_loop.py.
* Add the no-reactor-extra-deps tox env, run no-reactor on macos.
* Skip unhandled CancelledError when shutting down the engine.
* Fix typing and pre-commit checks.
* Fix typing problems in master.
---------
Co-authored-by: Adrian <adrian@zyte.com>
2026-03-24 15:29:22 +05:00
6e0a0e476a
Fix running tests with -n auto, enable it on CI ( #7257 )
...
* Make test_start_deprecated_super() more robust.
* Work around the FTPFilesStore.FTP_* initialization.
* Enable xdist on CI.
* Add PYTEST_ADDOPTS to tox passenv.
2026-02-13 22:12:57 +05:00
3186ccf5d5
Reactorless tests 1: the test env ( #6952 )
...
* Reactorless tests.
* Updates after merging master.
* Update a new import.
* Updates after merging master.
* Update testenv:no-reactor-pinned.
* Foundations for the reactorless mode.
* Fixes after merging master.
* Updates after merging reactorless-base.
* Add simple subprocess tests for reactorless AsyncCrawler*.
* More reactorless tests.
* Refactor AsyncCrawlerProcess.start().
* More checks.
* Fix test_reactorless_import_hook.
* More tests.
* Call install_reactor() before asyncio.run().
* Cleanup.
* Rephrase.
* Fix a test.
* Fixes.
* Rephrasing.
* Set TELNETCONSOLE_ENABLED=False in the reactorless mode.
* Fix a new import.
* Clarify --reactor marks.
* Cleanup.
* More docs/comments.
* Update test_fallback_workflow().
* Update TestDeferredFFromCoroF.
* Typing improvements.
2026-02-02 19:25:30 +05:00
49930dfec5
chore: Update outdated GitHub Actions versions ( #7223 )
2026-01-23 12:26:58 +01:00
e30ba7d4ca
( #7180 ) Update codecov test results action to v5 on all CI workflows. ( #7215 )
2026-01-12 08:31:53 +01:00
14737e91ed
Mitigate brotli and deflate decompression bombs DoS ( #7134 )
...
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2025-11-17 14:21:50 +05:00
804ae167df
Update tool versions ( #7127 )
2025-10-27 14:11:31 +01:00
d414d393d4
Remove Python 3.9 support ( #7121 )
2025-10-27 12:37:49 +01:00
393d715205
Drop PyPy 3.10 support, restore coverage for subprocesses ( #7050 )
2025-09-15 11:24:09 +02:00
4899d416e7
Add PyPy 3.11 to CI. ( #6697 )
2025-05-06 12:31:28 +02:00
e50914e0f5
Codecov: Add test analytics ( #6741 )
2025-03-21 12:28:47 +01:00
d0dabbc097
Enable AsyncioSelectorReactor by default. ( #6713 )
...
* Enable AsyncioSelectorReactor by default.
* Improve get_crawler(), switch more tests to it.
* Fix the remaining default-reactor test failures.
* Address documentation feedback.
* Make pinned envs more consistent.
2025-03-12 01:18:30 +05:00
26ecc93228
Run CI only on the main branch, on release branches and on PRs ( #6720 )
2025-03-11 14:12:11 +01:00
d4b152bbf6
Drop PyPy 3.9, add a pypy3-extra-deps CI job. ( #6613 )
2025-01-23 09:22:18 +01:00
5d3aa80ad1
Switch CI to codecov/codecov-action and enable it on Windows. ( #6609 )
2025-01-07 10:52:26 +01:00
4869315d10
Install libjpeg-dev on pinned envs to be able to install Pillow. ( #6607 )
2025-01-07 10:46:12 +01:00
845b1ffd44
Merge remote-tracking branch 'origin/master' into py313
2024-10-16 14:50:12 +05:00
5391663072
Drop Python 3.8 Support ( #6472 )
2024-10-16 10:03:16 +02:00
df6c51af0f
Use the 3.13 release.
2024-10-08 15:37:49 +05:00
1e68d3c0bf
Merge branch 'master' into py313
2024-08-11 13:46:51 +05:00
129dbfa0bf
Bump tool versions.
2024-07-11 12:23:43 +05:00
b6d3d9076f
Help with building lxml on 3.13beta1.
2024-05-22 13:20:48 +05:00
534a66e954
Bump 3.13 to beta1.
2024-05-22 13:16:00 +05:00
c66b517706
Add Python 3.13 alpha to CI
2023-12-01 10:36:27 +01:00
991121fa91
Re-enable uvloop tests on 3.12 ( #6098 )
2023-10-17 19:24:44 +02:00
c65567988d
Change supported PyPy versions to 3.9 and 3.10. ( #6087 )
...
* Change support PyPy versions to 3.9 and 3.10.
* Update the RTD URL for coverage.
* Move pypy3-pinned back to pypy3.8.
2023-10-06 11:27:02 +02:00
5b0b00212e
Merge pull request #6083 from wRAR/py3.12-release
...
Adapt to the Python 3.12 final release
2023-10-03 20:20:12 +04:00
9ae8d97d81
Update GitHub Actions checkout action to version 4 ( #6084 )
2023-10-03 13:04:18 +02:00
60d5f391c4
Use the 3.12 release, enable it for all OSes.
2023-10-02 21:35:45 +04:00
884840e3a3
Use concurrency groups to limit duplicate jobs
...
Leverage concurrency groups along with cancel-in-progress to favor running the most recent job.
Concurrency groups are on a per workflow, per branch/tag basis. So, pushing newer updates to a branch, e.g. as part of a PR, should cancel any in progress runs of workflows that have been retriggered.
See:
- https://docs.github.com/en/enterprise-cloud@latest/actions/using-workflows/workflow-syntax-for-github-actions#concurrency
- https://docs.github.com/en/enterprise-cloud@latest/actions/learn-github-actions/contexts#github-context
2023-10-01 12:12:24 +02:00
da39fbd270
Update tool versions ( #6046 )
2023-09-12 20:07:10 +02:00
b06936f111
Handle Tuple type on getdictorlist method, bump 3.12 python version
2023-08-14 10:33:48 -03:00
21b6dc5f9f
Add 3.12 CI jobs.
2023-07-22 17:55:32 +04:00
2122278d4b
Drop Python 3.7 support.
2023-06-18 18:37:50 +04:00
590955fac8
Provide separate test environments for botocore and boto3 extras; add extra-deps-pinned and remote extras from pinned
2023-03-15 16:03:44 +01:00
016d1de64e
Remove Reppy from CI
2023-01-02 14:13:04 +01:00
c4d7f5e7a9
Use CLang to build Reppy
2023-01-02 13:41:08 +01:00
b654183084
Use Python 3.11 as the default in CI ( #5696 )
2022-10-27 14:00:36 +02:00
22a59d0005
CI: use the latest version of Ubuntu ( #5675 )
2022-10-14 19:41:50 +02:00
ed9bc84d55
Remove a pin on libxml2-dev.
2022-10-08 19:11:16 +06:00
4aea925714
Update action versions.
2022-10-07 15:01:50 +06:00
300c42bfdf
Install PyPy using actions/setup-python.
2022-10-07 14:53:19 +06:00
82d10f0914
Add Ubuntu tests for Python 3.11rc2.
2022-10-06 20:27:06 +06:00
407562b38b
Drop Python 3.6 support ( #5514 )
...
* chore: Drop Python 3.6 support
* Attend PR comments
* Tweak versions
* Update dependencies version
* fix: Ubuntu workflow
* fix windows workflow
* chore: Remove comment
* update `install_requires` dependencies versions
* move lxml to main pinned requirements
* Attend code-review comments
* remove non-pinned 3.7 from windows workflow
* simplify condition
* lint
* remove paragraph
* refactor
* remove leftover
2022-06-09 00:25:03 -03:00
9be878fc09
CI: stop using tox-pip-version ( #5389 )
2022-02-04 12:27:39 +01:00
cfff79cee6
Make Python 3.10 support official ( #5265 )
2021-10-18 22:09:17 +02:00
43ea21e830
Feed post-processing plugin support ( #5190 )
2021-08-24 11:48:01 +02:00
74cee38a4e
Don't run the asyncio tests on 3.9.
2021-08-11 14:19:08 +05:00
Andrey Rakhmatullin
Pádraic Slattery
Santiago Beltrán Ruiz
Rui Xi
Suejung Shin
Adrián Chaves
Vsevolod Breus
Klaus Rettinghaus
Andrew Armbruster
Laerte Pereira
Nirjas Jakilim
Laerte Pereira
Eugenio Lacuesta
D R Siddhartha