RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,107 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

33 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 2798c03bb0
Add Python 3.14 to CI. (#6604) 
* Add Python 3.14 (alpha3) to CI.

* Disable mitmproxy on 3.14 for now.

* 3.14.0-alpha.4.

* 3.14.0-alpha.5

* 3.14.0-beta2.

* 3.14 release.

* Fix test_non_pickable_object.

* Fix handling of file:/path feed URIs.

* Better mocking of streams for TextTestResult.

* Do not use .php in test_file_path() as it's now a known extension.

* Fix the URL in TestFeedExporterSignals.

* Fix typing.

* Bump more envs to 3.14.

* Silence pylint.

* Fix another test for .php handling change.

* Remove test_install_asyncio_reactor.

* More bumps to 3.14.

* Revert docs-tests to use 3.13.

* Debug options for Windows.

* Re-enable xdist.

* Revert Windows PYTEST_ADDOPTS.

* Silence loop policy deprecation warnings.

* Restore a lost pylint suppression.

* Update asyncio_enabled_reactor_same_loop.py to new Twisted.

* Fix RobotFileParser tests for Python 3.14.5.
 2026-05-12 23:49:37 +05:00
Andrey Rakhmatullin 6fe27ba33e
Add more no-reactor tests (#7259) 
* Generic changes and scrapy bench.

* scrapy check.

* scrapy crawl.

* scrapy fetch.

* scrapy parse.

* scrapy runspider.

* scrapy shell.

* Skip httpx tests on default-reactor.

* Review requires_reactor marks.

* Make test functions that require an event loop async def.

* Improve test_pending_asyncio_tasks().

* Add Mac OS DNS error.

* Refactor most of test_scheduler.py.

* Finish refactoring DownloaderAwareSchedulerTestMixin.

* Refactor test_engine_loop.py.

* Add the no-reactor-extra-deps tox env, run no-reactor on macos.

* Skip unhandled CancelledError when shutting down the engine.

* Fix typing and pre-commit checks.

* Fix typing problems in master.

---------

Co-authored-by: Adrian <adrian@zyte.com>
 2026-03-24 15:29:22 +05:00
Andrey Rakhmatullin 6e0a0e476a
Fix running tests with -n auto, enable it on CI (#7257) 
* Make test_start_deprecated_super() more robust.

* Work around the FTPFilesStore.FTP_* initialization.

* Enable xdist on CI.

* Add PYTEST_ADDOPTS to tox passenv.
 2026-02-13 22:12:57 +05:00
Pádraic Slattery 49930dfec5
chore: Update outdated GitHub Actions versions (#7223) 2026-01-23 12:26:58 +01:00
Santiago Beltrán Ruiz e30ba7d4ca
(#7180) Update codecov test results action to v5 on all CI workflows. (#7215) 2026-01-12 08:31:53 +01:00
Andrey Rakhmatullin 804ae167df
Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
Andrey Rakhmatullin d414d393d4
Remove Python 3.9 support (#7121) 2025-10-27 12:37:49 +01:00
Suejung Shin e50914e0f5
Codecov: Add test analytics (#6741) 2025-03-21 12:28:47 +01:00
Adrián Chaves 26ecc93228
Run CI only on the main branch, on release branches and on PRs (#6720) 2025-03-11 14:12:11 +01:00
Andrey Rakhmatullin 5d3aa80ad1
Switch CI to codecov/codecov-action and enable it on Windows. (#6609) 2025-01-07 10:52:26 +01:00
Andrey Rakhmatullin 845b1ffd44 Merge remote-tracking branch 'origin/master' into py313 2024-10-16 14:50:12 +05:00
Vsevolod Breus 5391663072
Drop Python 3.8 Support (#6472) 2024-10-16 10:03:16 +02:00
Andrey Rakhmatullin df6c51af0f Use the 3.13 release. 2024-10-08 15:37:49 +05:00
Andrey Rakhmatullin 1e68d3c0bf Merge branch 'master' into py313 2024-08-11 13:46:51 +05:00
Andrey Rakhmatullin 129dbfa0bf Bump tool versions. 2024-07-11 12:23:43 +05:00
Andrey Rakhmatullin 986d1ee1dd
Move CI from the decommissioned macos-11 to macos-latest. (#6372) 2024-05-28 10:37:19 +02:00
Andrey Rakhmatullin 534a66e954 Bump 3.13 to beta1. 2024-05-22 13:16:00 +05:00
Adrián Chaves c66b517706 Add Python 3.13 alpha to CI 2023-12-01 10:36:27 +01:00
Andrey Rakhmatullin 5b0b00212e
Merge pull request #6083 from wRAR/py3.12-release 
Adapt to the Python 3.12 final release
 2023-10-03 20:20:12 +04:00
Klaus Rettinghaus 9ae8d97d81
Update GitHub Actions checkout action to version 4 (#6084) 2023-10-03 13:04:18 +02:00
Andrey Rakhmatullin 60d5f391c4 Use the 3.12 release, enable it for all OSes. 2023-10-02 21:35:45 +04:00
Andrew Armbruster 884840e3a3 Use concurrency groups to limit duplicate jobs 
Leverage concurrency groups along with cancel-in-progress to favor running the most recent job.

Concurrency groups are on a per workflow, per branch/tag basis. So, pushing newer updates to a branch, e.g. as part of a PR, should cancel any in progress runs of workflows that have been retriggered.

See:
- https://docs.github.com/en/enterprise-cloud@latest/actions/using-workflows/workflow-syntax-for-github-actions#concurrency
- https://docs.github.com/en/enterprise-cloud@latest/actions/learn-github-actions/contexts#github-context
 2023-10-01 12:12:24 +02:00
Andrey Rakhmatullin 2122278d4b Drop Python 3.7 support. 2023-06-18 18:37:50 +04:00
Andrey Rahmatullin b654183084
Use Python 3.11 as the default in CI (#5696) 2022-10-27 14:00:36 +02:00
Johanan Idicula a214147359
ci: Update macos runner 
The GitHub Actions macos-10.15 runner image is now deprecated, and GitHub
Actions has begun to temporarily fail jobs referencing it during brownout
periods. The image will be fully unsupported by 2022-12-01, which is just about
a month away.

This change updates the macOS runner image to the latest generally-available
version, to help reduce spurious CI failures during the brownout periods, and to
stay abreast of the sunsetting of the macos-10.15 image.

See also: actions/runner-images#5583
 2022-10-26 21:44:47 -04:00
Andrey Rahmatullin 029591616c
Merge pull request #5651 from oscard0m/patch-3 
ci(test-macos): upgrade actions/checkout to v3
 2022-10-03 17:12:35 +05:00
Oscar Dominguez af95331296
ci(test-macos): upgrade actions/setup-python to v4 2022-10-01 20:27:23 +02:00
Oscar Dominguez 1a2cb61e22
ci(test-macos): upgrade actions/checkout to v3 2022-10-01 20:27:10 +02:00
Laerte Pereira 407562b38b
Drop Python 3.6 support (#5514) 
* chore: Drop Python 3.6 support

* Attend PR comments

* Tweak versions

* Update dependencies version

* fix: Ubuntu workflow

* fix windows workflow

* chore: Remove comment

* update `install_requires` dependencies versions

* move lxml to main pinned requirements

* Attend code-review comments

* remove non-pinned 3.7 from windows workflow

* simplify condition

* lint

* remove paragraph

* refactor

* remove leftover
 2022-06-09 00:25:03 -03:00
Eugenio Lacuesta cfff79cee6
Make Python 3.10 support official (#5265) 2021-10-18 22:09:17 +02:00
Eugenio Lacuesta c062ed017a
[CI] fail-fast: false (#5200) 2021-07-12 18:34:22 +02:00
vinayak 42e4dbb23d
Support Python 3.9 (#4759) 
* Update .travis.yml

* Update .travis.yml

* updage travis.yml

* Make 3.9 support official

* Upgrade mitmproxy for Python 3.9

* Restore the Pylint job

* Undo unintended change to mitmproxy requirement

* Enable Python 3.9 in GitHub Actions

* Work around reppy’s Python version limitation

* Disable tests in Windows / Python 3.9 due to a Twisted bug

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-03-18 09:40:03 -03:00
Eugenio Lacuesta 80db569aea
Migrate CI to GitHub actions (#4924) 2021-01-01 19:13:39 -03:00