scrapy/tests
Andrey Rakhmatullin 2ce02d417a
TLS code modernization (#7353)
* Import some stuff from service_identity directly.

* Extract _log_tls().

* Sync validating IP addresses.

* Introduce _deps_compat.

* Deprecate getCertificateOptions().

* Deprecate ScrapyClientTLSOptions.

* Add pragma: no cover.

* verify=False and trustRoot=platformTrust() are the defaults.

* Improve docstrings for TLS classes.

* Deprecate AcceptableProtocolsContextFactory.

* Switch from DefaultOpenSSLContextFactory to CertificateOptions.

* Deprecate load_context_factory_from_settings().

* Remove the outdated warning about incompatible factories.

* Simplify _load_context_factory_from_settings().

* Make CertificateOptions and Context once per factory.

* pragma: no cover
2026-03-25 15:42:43 +05:00
..
AsyncCrawlerProcess Foundations for the reactorless mode. (#7199) 2026-01-31 20:48:54 +05:00
AsyncCrawlerRunner Disable mypy allow_untyped_calls. (#7331) 2026-03-24 10:31:17 +01:00
CrawlerProcess Foundations for the reactorless mode. (#7199) 2026-01-31 20:48:54 +05:00
CrawlerRunner Disable mypy allow_untyped_calls. (#7331) 2026-03-24 10:31:17 +01:00
keys Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
mocks Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
mockserver TLS code modernization (#7353) 2026-03-25 15:42:43 +05:00
sample_data Add support for multiple-compressed responses (#6063) 2024-02-22 12:16:24 +01:00
test_cmdline Converting tests to plain asserts, part 2. (#6699) 2025-03-06 20:52:41 +01:00
test_cmdline_crawl_with_pipeline Fix open_spider exception handling to set exit code 1 (#7255) 2026-02-19 11:21:59 +01:00
test_settings Fix override behavior in getwithbase() issue #6912 (#6993) 2026-02-07 00:17:55 +04:00
test_spiderloader Don't use CrawlerProcess in the commands that don't need it. (#6824) 2025-05-28 19:57:33 +05:00
test_utils_misc Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
utils Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
__init__.py TLS code modernization (#7353) 2026-03-25 15:42:43 +05:00
ignores.txt Remove deprecated classes BaseSgmlLinkExtractor, RegexLinkExtractor and SgmlLinkExtractor (#4356) 2020-07-17 12:40:04 +02:00
pipelines.py Deprecate spider args of middlewares and pipelines. (#7006) 2025-08-21 13:23:10 +05:00
spiders.py Disable mypy allow_untyped_calls. (#7331) 2026-03-24 10:31:17 +01:00
test_addons.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_closespider.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_command_check.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_command_crawl.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_command_fetch.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_command_genspider.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_command_parse.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_command_runspider.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_command_shell.py Add warnings to components not working without a reactor (#7355) 2026-03-25 09:21:08 +01:00
test_command_startproject.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_command_version.py Speed up and refactor command tests (#7118) 2025-10-27 12:28:16 +01:00
test_commands.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_contracts.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_core_downloader.py TLS code modernization (#7353) 2026-03-25 15:42:43 +05:00
test_core_scraper.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_crawl.py Disable mypy allow_untyped_calls. (#7331) 2026-03-24 10:31:17 +01:00
test_crawler.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_crawler_subprocess.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_dependencies.py Reduce deps on unittest, unify inlineCallbacks imports in tests. (#6873) 2025-06-06 22:59:09 +02:00
test_downloader_handler_httpx.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_downloader_handler_twisted_ftp.py Add warnings to components not working without a reactor (#7355) 2026-03-25 09:21:08 +01:00
test_downloader_handler_twisted_http2.py Add warnings to components not working without a reactor (#7355) 2026-03-25 09:21:08 +01:00
test_downloader_handler_twisted_http10.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_downloader_handler_twisted_http11.py Add warnings to components not working without a reactor (#7355) 2026-03-25 09:21:08 +01:00
test_downloader_handlers.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_downloader_handlers_http_base.py Add a DOWNLOAD_BIND_ADDRESS setting for download handlers (#7283) 2026-03-19 09:16:40 +01:00
test_downloadermiddleware.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_downloadermiddleware_ajaxcrawlable.py Converting tests to plain asserts, part 3. (#6700) 2025-03-06 20:50:14 +01:00
test_downloadermiddleware_cookies.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_downloadermiddleware_defaultheaders.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_downloadermiddleware_downloadtimeout.py Deprecate spider attributes that can be replaced by settings, round 2 (#7039) 2025-12-01 01:26:17 +05:00
test_downloadermiddleware_httpauth.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_downloadermiddleware_httpcache.py Deprecate ScrapyCommand.set_crawler(), remove/fix some dead code. (#7276) 2026-03-24 10:36:48 +01:00
test_downloadermiddleware_httpcompression.py Restore brotlicffi support (#7160) 2025-11-25 12:06:55 +05:00
test_downloadermiddleware_httpproxy.py Move to mypy --strict with exceptions. (#7300) 2026-03-02 15:47:23 +05:00
test_downloadermiddleware_offsite.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_downloadermiddleware_redirect.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_downloadermiddleware_redirect_base.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_downloadermiddleware_redirect_metarefresh.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_downloadermiddleware_retry.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_downloadermiddleware_robotstxt.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_downloadermiddleware_stats.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_downloadermiddleware_useragent.py Deprecate spider attributes that can be replaced by settings, round 2 (#7039) 2025-12-01 01:26:17 +05:00
test_downloaderslotssettings.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_dupefilters.py Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
test_engine.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_engine_loop.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_engine_stop_download_bytes.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_engine_stop_download_headers.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_exporters.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_extension_periodic_log.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_extension_statsmailer.py Deprecate the mail API (#7263) 2026-03-24 21:31:12 +01:00
test_extension_telnet.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_extension_throttle.py Preparations for switching to direct pytest. (#6678) 2025-02-16 20:28:58 +01:00
test_feedexport.py asyncio alternative to deferToThread() (#7349) 2026-03-24 15:41:15 +01:00
test_feedexport_batch.py asyncio alternative to deferToThread() (#7349) 2026-03-24 15:41:15 +01:00
test_feedexport_postprocess.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_feedexport_storages.py asyncio alternative to deferToThread() (#7349) 2026-03-24 15:41:15 +01:00
test_feedexport_uri_params.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_http2_client_protocol.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_http_cookies.py Converting tests to plain asserts, part 7. (#6710) 2025-03-09 20:23:51 +01:00
test_http_headers.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_http_request.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_http_request_form.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_http_request_json.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_http_response.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_http_response_text.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_item.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_link.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_linkextractors.py Fix invalid XML samples in tests that break with lxml 6. (#6919) 2025-06-27 16:24:09 +05:00
test_loader.py Deprecate ScrapyCommand.set_crawler(), remove/fix some dead code. (#7276) 2026-03-24 10:36:48 +01:00
test_logformatter.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_logstats.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_mail.py Deprecate the mail API (#7263) 2026-03-24 21:31:12 +01:00
test_middleware.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_pipeline_crawl.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_pipeline_files.py asyncio alternative to deferToThread() (#7349) 2026-03-24 15:41:15 +01:00
test_pipeline_images.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_pipeline_media.py Await persist_file() in media pipelines. (#7182) 2026-03-24 10:52:56 +01:00
test_pipelines.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_poet.py Fix getting annotations for _parse_sitemap() at the runtime. (#6671) 2025-02-14 20:40:06 +05:00
test_pqueues.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_proxy_connect.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_request_attribute_binding.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_request_cb_kwargs.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_request_dict.py Converting tests to plain asserts, part 5. (#6712) 2025-03-09 20:24:45 +01:00
test_request_left.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_responsetypes.py Converting tests to plain asserts, part 5. (#6712) 2025-03-09 20:24:45 +01:00
test_robotstxt_interface.py Disable mypy allow_untyped_calls. (#7331) 2026-03-24 10:31:17 +01:00
test_scheduler.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_scheduler_base.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_scrapy__getattr__.py Remove code deprecated in 2.12.0 (#7126) 2025-10-27 17:14:38 +05:00
test_selector.py Test refactoring, part 2. (#6928) 2025-06-29 23:10:38 +02:00
test_signals.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_spider.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_spider_crawl.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_spider_sitemap.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_spider_start.py Fix running tests with -n auto, enable it on CI (#7257) 2026-02-13 22:12:57 +05:00
test_spidermiddleware.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_spidermiddleware_base.py Extract @_warn_spider_arg. (#7033) 2025-09-01 20:17:13 +05:00
test_spidermiddleware_depth.py Deprecate spider args of middlewares and pipelines. (#7006) 2025-08-21 13:23:10 +05:00
test_spidermiddleware_httperror.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_spidermiddleware_output_chain.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_spidermiddleware_process_start.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_spidermiddleware_referer.py Fix test issues with the no-reactor env 2026-03-12 16:46:32 +01:00
test_spidermiddleware_start.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_spidermiddleware_urllength.py Deprecate spider args of middlewares and pipelines. (#7006) 2025-08-21 13:23:10 +05:00
test_spiderstate.py Test refactoring, part 2. (#6928) 2025-06-29 23:10:38 +02:00
test_squeues.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_squeues_request.py Test refactoring, part 2. (#6928) 2025-06-29 23:10:38 +02:00
test_stats.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_toplevel.py Bump tool versions. (#6941) 2025-07-05 10:30:30 +02:00
test_utils_asyncgen.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_utils_asyncio.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_utils_conf.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_utils_console.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_curl.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_utils_datatypes.py Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
test_utils_defer.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_utils_deprecate.py Merge pull request #7145 from Icorebleidd/master 2025-11-28 01:16:02 +05:00
test_utils_display.py Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
test_utils_gz.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_httpobj.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_iterators.py Move to mypy --strict with exceptions. (#7300) 2026-03-02 15:47:23 +05:00
test_utils_log.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_project.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_python.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_utils_reactor.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_utils_request.py Remove code deprecated in 2.12.0 (#7126) 2025-10-27 17:14:38 +05:00
test_utils_response.py Fix get_meta_refresh to correctly resolve relative URLs using <base> tag (#7047) 2025-09-10 15:17:16 +02:00
test_utils_serialize.py Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
test_utils_signal.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_utils_sitemap.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_spider.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_utils_template.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_trackref.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_url.py Bump tool versions. (#6941) 2025-07-05 10:30:30 +02:00
test_webclient.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_zz_resources.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00