..
AsyncCrawlerProcess
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` ( #6817 )
2025-05-28 18:55:44 +05:00
AsyncCrawlerRunner
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` ( #6817 )
2025-05-28 18:55:44 +05:00
CrawlerProcess
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` ( #6817 )
2025-05-28 18:55:44 +05:00
CrawlerRunner
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` ( #6817 )
2025-05-28 18:55:44 +05:00
keys
Commit mitmproxy-dhparam.pem.
2024-11-15 21:08:18 +05:00
mocks
Drop Python 3.8 Support ( #6472 )
2024-10-16 10:03:16 +02:00
sample_data
Add support for multiple-compressed responses ( #6063 )
2024-02-22 12:16:24 +01:00
test_cmdline
Converting tests to plain asserts, part 2. ( #6699 )
2025-03-06 20:52:41 +01:00
test_cmdline_crawl_with_pipeline
Fix and ban the top-level twisted.internet.reactor imports. ( #6835 )
2025-05-28 15:53:52 +05:00
test_settings
Converting tests to plain asserts, part 8. ( #6711 )
2025-03-09 20:24:12 +01:00
test_spiderloader
Don't use CrawlerProcess in the commands that don't need it. ( #6824 )
2025-05-28 19:57:33 +05:00
test_utils_misc
Syntax Error Fixed ( #6738 )
2025-03-20 17:02:10 +05:00
utils
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
__init__.py
Support asynchronous start requests ( #6729 )
2025-05-07 19:04:03 +02:00
ftpserver.py
adding black formatter to all the code
2022-11-29 11:30:46 -03:00
ignores.txt
Remove deprecated classes BaseSgmlLinkExtractor, RegexLinkExtractor and SgmlLinkExtractor ( #4356 )
2020-07-17 12:40:04 +02:00
mockserver.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
pipelines.py
reaplying black, fixing conflicts and ignoring bandit checks on test directory
2023-01-20 10:55:46 -03:00
spiders.py
Support asynchronous start requests ( #6729 )
2025-05-07 19:04:03 +02:00
test_addons.py
Enable AsyncioSelectorReactor by default. ( #6713 )
2025-03-12 01:18:30 +05:00
test_closespider.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_command_check.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_command_crawl.py
Use AsyncCrawlerProcess in commands. ( #6845 )
2025-06-03 17:47:52 +05:00
test_command_fetch.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_command_genspider.py
Split tests/test_commands.py. ( #6836 )
2025-05-28 21:46:04 +02:00
test_command_parse.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_command_runspider.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_command_shell.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_command_startproject.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_command_version.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_commands.py
Remove ProcessTest and SiteTest. ( #6885 )
2025-06-13 22:18:27 +05:00
test_contracts.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_core_downloader.py
Pin older Twisted in tests, update type hints. ( #6882 )
2025-06-09 09:53:42 +02:00
test_crawl.py
Replace most of the @inlineCallbacks test helpers. ( #6883 )
2025-06-10 10:02:27 +02:00
test_crawler.py
Don't use CrawlerProcess in the commands that don't need it. ( #6824 )
2025-05-28 19:57:33 +05:00
test_dependencies.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_downloader_handler_twisted_http2.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_downloader_handler_twisted_http10.py
Rewrite download handler tests to coroutines. ( #6846 )
2025-06-02 16:02:08 +02:00
test_downloader_handler_twisted_http11.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_downloader_handlers.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_downloader_handlers_http_base.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_downloadermiddleware.py
Refactor more Deferred functions. ( #6795 )
2025-05-13 20:47:57 +02:00
test_downloadermiddleware_ajaxcrawlable.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_cookies.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_defaultheaders.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_downloadtimeout.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_httpauth.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_httpcache.py
refactor tests/test_downloadermiddleware_httpcache.py ( #6769 )
2025-05-12 13:04:34 +02:00
test_downloadermiddleware_httpcompression.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_downloadermiddleware_httpproxy.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_offsite.py
Preparations for switching to direct pytest. ( #6678 )
2025-02-16 20:28:58 +01:00
test_downloadermiddleware_redirect.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_retry.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_robotstxt.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_downloadermiddleware_stats.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloadermiddleware_useragent.py
Converting tests to plain asserts, part 3. ( #6700 )
2025-03-06 20:50:14 +01:00
test_downloaderslotssettings.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_dupefilters.py
Converting tests to plain asserts, part 2. ( #6699 )
2025-03-06 20:52:41 +01:00
test_engine.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_engine_loop.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_engine_stop_download_bytes.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_engine_stop_download_headers.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_exporters.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_extension_periodic_log.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_extension_telnet.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_extension_throttle.py
Preparations for switching to direct pytest. ( #6678 )
2025-02-16 20:28:58 +01:00
test_feedexport.py
Remove the chdir fixture, re-enable fancy pytest asserts ( #6888 )
2025-06-14 14:01:20 +05:00
test_http2_client_protocol.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_http_cookies.py
Converting tests to plain asserts, part 7. ( #6710 )
2025-03-09 20:23:51 +01:00
test_http_headers.py
Converting tests to plain asserts, part 7. ( #6710 )
2025-03-09 20:23:51 +01:00
test_http_request.py
Converting tests to plain asserts, part 8. ( #6711 )
2025-03-09 20:24:12 +01:00
test_http_response.py
Converting tests to plain asserts, part 8. ( #6711 )
2025-03-09 20:24:12 +01:00
test_item.py
Converting tests to plain asserts, part 6. ( #6709 )
2025-03-09 20:20:24 +01:00
test_link.py
Converting tests to plain asserts, part 6. ( #6709 )
2025-03-09 20:20:24 +01:00
test_linkextractors.py
Converting tests to plain asserts, part 6. ( #6709 )
2025-03-09 20:20:24 +01:00
test_loader.py
Converting tests to plain asserts, part 8. ( #6711 )
2025-03-09 20:24:12 +01:00
test_logformatter.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_logstats.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_mail.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_middleware.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_pipeline_crawl.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_pipeline_files.py
Remove the chdir fixture, re-enable fancy pytest asserts ( #6888 )
2025-06-14 14:01:20 +05:00
test_pipeline_images.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_pipeline_media.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_pipelines.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_poet.py
Fix getting annotations for _parse_sitemap() at the runtime. ( #6671 )
2025-02-14 20:40:06 +05:00
test_pqueues.py
2.13.1 ( #6832 )
2025-05-28 10:58:34 +02:00
test_proxy_connect.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_request_attribute_binding.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_request_cb_kwargs.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_request_dict.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_request_left.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_responsetypes.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_robotstxt_interface.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_scheduler.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_scheduler_base.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_scrapy__getattr__.py
Split ruff and pylint ignores into two categories, some pylint cleanup.
2025-01-02 01:03:42 +05:00
test_selector.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_signals.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_spider.py
Deprecate _parse_response and implement parse_with_rules ( #6804 )
2025-06-02 17:00:17 +02:00
test_spider_start.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_spidermiddleware.py
Replace most of the @inlineCallbacks test helpers. ( #6883 )
2025-06-10 10:02:27 +02:00
test_spidermiddleware_base.py
Support asynchronous start requests ( #6729 )
2025-05-07 19:04:03 +02:00
test_spidermiddleware_depth.py
Base class for universal spider middlewares ( #6693 )
2025-04-23 19:29:04 +05:00
test_spidermiddleware_httperror.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_spidermiddleware_offsite.py
Base class for universal spider middlewares ( #6693 )
2025-04-23 19:29:04 +05:00
test_spidermiddleware_output_chain.py
Replace most of the @inlineCallbacks test helpers. ( #6883 )
2025-06-10 10:02:27 +02:00
test_spidermiddleware_process_start.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_spidermiddleware_referer.py
Converting tests to plain asserts, part 4. ( #6702 )
2025-03-07 17:25:15 +01:00
test_spidermiddleware_start.py
Support asynchronous start requests ( #6729 )
2025-05-07 19:04:03 +02:00
test_spidermiddleware_urllength.py
Converting tests to plain asserts, part 4. ( #6702 )
2025-03-07 17:25:15 +01:00
test_spiderstate.py
Converting tests to plain asserts, part 4. ( #6702 )
2025-03-07 17:25:15 +01:00
test_squeues.py
Better fix for test_non_pickable_object on Windows.
2025-03-10 13:15:28 +05:00
test_squeues_request.py
Remove the chdir fixture, re-enable fancy pytest asserts ( #6888 )
2025-06-14 14:01:20 +05:00
test_stats.py
Converting tests to plain asserts, part 4. ( #6702 )
2025-03-07 17:25:15 +01:00
test_toplevel.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_urlparse_monkeypatches.py
Converting tests to plain asserts, part 5. ( #6712 )
2025-03-09 20:24:45 +01:00
test_utils_asyncgen.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_asyncio.py
Add AsyncioLoopingCall. ( #6855 )
2025-06-03 20:21:33 +02:00
test_utils_conf.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_console.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_curl.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_datatypes.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_defer.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_utils_deprecate.py
Add is_asyncio_available(). ( #6827 )
2025-05-28 14:46:39 +05:00
test_utils_display.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_gz.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_httpobj.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_iterators.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_log.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_utils_project.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_python.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_reactor.py
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` ( #6817 )
2025-05-28 18:55:44 +05:00
test_utils_request.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_response.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_serialize.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_signal.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
test_utils_sitemap.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_spider.py
Convert tests/test_utils* to plain asserts. ( #6695 )
2025-03-04 09:31:26 +01:00
test_utils_template.py
Convert remaining unittest assert* calls, use the tmp_path fixture. ( #6725 )
2025-03-11 18:00:36 +01:00
test_utils_trackref.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_utils_url.py
Reduce deps on unittest, unify inlineCallbacks imports in tests. ( #6873 )
2025-06-06 22:59:09 +02:00
test_webclient.py
Further reduce deps on unittest. ( #6884 )
2025-06-11 01:28:09 +02:00
upper-constraints.txt
upper-constraints.txt: restrict botocore further
2021-03-18 23:42:29 +01:00