RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
scrapy/tests
History
tanishqtayade b82c04704c fix: LocalCache with limit=0 incorrectly stores items 
When limit=0 is passed to LocalCache (e.g. when DNSCACHE_ENABLED=False),
the condition 'if self.limit' evaluates to False due to Python's truthiness
rules, causing items to be stored despite the cache being disabled.

This leads to an unbounded memory leak during long crawls when DNS caching
is explicitly disabled.

Fix changes the condition to 'if self.limit is not None' and adds an early
return when limit=0 to correctly handle the disabled cache case.
 		2026-06-25 12:24:17 +05:30
..
AsyncCrawlerProcess Disable mypy allow_any_generics. (#7532) 2026-05-18 15:54:06 +02:00
AsyncCrawlerRunner Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. (#7394) 2026-04-02 10:20:54 +02:00
CrawlerProcess Add Python 3.14 to CI. (#6604) 2026-05-12 23:49:37 +05:00
CrawlerRunner Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. (#7394) 2026-04-02 10:20:54 +02:00
keys Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
mocks Disable mypy allow_any_generics. (#7532) 2026-05-18 15:54:06 +02:00
mockserver Fix deprecation warnings with pyOpenSSL 26.3.0. (#7619) 2026-06-15 08:44:33 +02:00
sample_data Add support for multiple-compressed responses (#6063) 2024-02-22 12:16:24 +01:00
test_cmdline Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_cmdline_crawl_with_pipeline Fix open_spider exception handling to set exit code 1 (#7255) 2026-02-19 11:21:59 +01:00
test_settings Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_spiderloader Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_utils_misc Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
utils tests: integration coverage for memusage extension (#7017) 2026-06-19 06:41:26 +02:00
__init__.py TLS code modernization (#7353) 2026-03-25 15:42:43 +05:00
ignores.txt Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
pipelines.py Deprecate spider args of middlewares and pipelines. (#7006) 2025-08-21 13:23:10 +05:00
spiders.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_addons.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_closespider.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_command_check.py Fixes for Python 3.14. (#7460) 2026-04-27 18:47:52 +05:00
test_command_crawl.py Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. (#7394) 2026-04-02 10:20:54 +02:00
test_command_fetch.py Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. (#7394) 2026-04-02 10:20:54 +02:00
test_command_genspider.py Disable mypy allow_any_generics. (#7532) 2026-05-18 15:54:06 +02:00
test_command_parse.py Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. (#7394) 2026-04-02 10:20:54 +02:00
test_command_runspider.py Rename TWISTED_ENABLED to TWISTED_REACTOR_ENABLED. (#7394) 2026-04-02 10:20:54 +02:00
test_command_shell.py Disable mypy allow_any_generics. (#7532) 2026-05-18 15:54:06 +02:00
test_command_startproject.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_command_version.py Speed up and refactor command tests (#7118) 2025-10-27 12:28:16 +01:00
test_commands.py Deprecate ScrapyCommand.help() (#7633) 2026-06-19 06:49:32 +02:00
test_contracts.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_core_downloader.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_core_scraper.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_crawl.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_crawler.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_crawler_subprocess.py Refactor test_crawl.py and test_crawler.py. (#7566) 2026-06-03 14:48:12 +02:00
test_dependencies.py tox.ini: pinned, typing → min, mypy (#7595) 2026-06-11 12:10:48 +05:00
test_downloader_handler_httpx.py Add support for HTTP/2 and for SOCKS proxies to HttpxDownloadHandler, improve handler docs (#7575) 2026-06-09 01:10:44 +05:00
test_downloader_handler_twisted_ftp.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_downloader_handler_twisted_http2.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_downloader_handler_twisted_http11.py Add settings for TLS min/max version as a replacement for the TLS method (#6546) 2026-06-08 11:54:10 +02:00
test_downloader_handlers.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_downloader_handlers_http_base.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_downloadermiddleware.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_downloadermiddleware_cookies.py fix typos (#7564) 2026-06-02 11:04:38 +02:00
test_downloadermiddleware_defaultheaders.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_downloadermiddleware_downloadtimeout.py Deprecate spider attributes that can be replaced by settings, round 2 (#7039) 2025-12-01 01:26:17 +05:00
test_downloadermiddleware_httpauth.py Implement HTTP Auth settings and request metadata keys (#7590) 2026-06-11 15:56:53 +05:00
test_downloadermiddleware_httpcache.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_downloadermiddleware_httpcompression.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_downloadermiddleware_httpproxy.py Move to mypy --strict with exceptions. (#7300) 2026-03-02 15:47:23 +05:00
test_downloadermiddleware_offsite.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_downloadermiddleware_redirect.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_downloadermiddleware_redirect_base.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_downloadermiddleware_redirect_metarefresh.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_downloadermiddleware_retry.py Allow configuring the log level of the retry give-up message (#7567) 2026-06-08 18:44:30 +05:00
test_downloadermiddleware_robotstxt.py Enable mypy warn_return_any. (#7492) 2026-05-04 22:24:47 +02:00
test_downloadermiddleware_stats.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_downloadermiddleware_useragent.py Deprecate spider attributes that can be replaced by settings, round 2 (#7039) 2025-12-01 01:26:17 +05:00
test_downloaderslotssettings.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_dupefilters.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_engine.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_engine_loop.py fix typos (#7564) 2026-06-02 11:04:38 +02:00
test_engine_stop_download_bytes.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_engine_stop_download_headers.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_exporters.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_extension_memusage.py tests: integration coverage for memusage extension (#7017) 2026-06-19 06:41:26 +02:00
test_extension_periodic_log.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_extension_statsmailer.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_extension_telnet.py Validate reversed telnet console port ranges (#7593) 2026-06-11 12:47:00 +05:00
test_extension_throttle.py Preparations for switching to direct pytest. (#6678) 2025-02-16 20:28:58 +01:00
test_feedexport.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_feedexport_batch.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_feedexport_postprocess.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_feedexport_storages.py Close GCS feed temp files after upload (#7546) 2026-06-04 20:11:50 +05:00
test_feedexport_uri_params.py Split long test files (#7329) 2026-03-13 10:30:25 +01:00
test_http2_client_protocol.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_http_cookies.py Converting tests to plain asserts, part 7. (#6710) 2025-03-09 20:23:51 +01:00
test_http_headers.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_http_request.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_http_request_form.py Silence deprecation warnings in TestFormRequest. (#7491) 2026-05-04 22:20:21 +02:00
test_http_request_json.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_http_response.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_http_response_text.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_item.py Add coverage for Item.__delitem__() (#7610) 2026-06-12 15:32:38 +05:00
test_link.py Link.__eq__: return NotImplemented instead of raising NotImplementedError (#7611) 2026-06-12 14:56:15 +02:00
test_linkextractors.py Complete test coverage for linkextractors (#7639) 2026-06-19 14:22:03 +05:00
test_loader.py Disable mypy allow_any_generics. (#7532) 2026-05-18 15:54:06 +02:00
test_logformatter.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_logstats.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_mail.py Deprecate the mail API (#7263) 2026-03-24 21:31:12 +01:00
test_middleware.py Deprecate the spider arg of downloader middleware methods. (#7037) 2025-09-02 21:51:47 +05:00
test_pipeline_crawl.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_pipeline_files.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_pipeline_images.py Fix image store ACL settings for S3 and GCS (#7614) 2026-06-16 00:24:44 +05:00
test_pipeline_media.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_pipelines.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_poet.py Fix getting annotations for _parse_sitemap() at the runtime. (#6671) 2025-02-14 20:40:06 +05:00
test_pqueues.py Improve test coverage for pqueues.py (#7640) 2026-06-19 18:28:32 +05:00
test_request_attribute_binding.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_request_cb_kwargs.py Complete test coverage for linkextractors (#7639) 2026-06-19 14:22:03 +05:00
test_request_dict.py Deprecate FormRequest in favor of form2request (#6438) 2026-05-04 22:38:29 +05:00
test_request_left.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_resolver.py Improve coverage resolver (#7642) 2026-06-19 23:26:54 +05:00
test_responsetypes.py Converting tests to plain asserts, part 5. (#6712) 2025-03-09 20:24:45 +01:00
test_robotstxt_interface.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_scheduler.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_scheduler_base.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_scrapy__getattr__.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_selector.py Test refactoring, part 2. (#6928) 2025-06-29 23:10:38 +02:00
test_signalmanager.py Test SignalManager.disconnect_all() (#7625) 2026-06-15 14:18:29 +05:00
test_signals.py Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
test_spider.py Complete test coverage for linkextractors (#7639) 2026-06-19 14:22:03 +05:00
test_spider_crawl.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_spider_sitemap.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_spider_start.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_spidermiddleware.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_spidermiddleware_base.py Drop start_requests() support. (#7490) 2026-05-04 17:34:24 +05:00
test_spidermiddleware_depth.py Deprecate spider args of middlewares and pipelines. (#7006) 2025-08-21 13:23:10 +05:00
test_spidermiddleware_httperror.py Convert tests that fail with testfixtures 12.0.0 (#7545) 2026-06-03 09:16:24 +02:00
test_spidermiddleware_output_chain.py Remove support for sync process_spider_output(). (#7504) 2026-05-15 16:36:05 +05:00
test_spidermiddleware_process_start.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_spidermiddleware_referer.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_spidermiddleware_start.py Drop start_requests() support. (#7490) 2026-05-04 17:34:24 +05:00
test_spidermiddleware_urllength.py Deprecate spider args of middlewares and pipelines. (#7006) 2025-08-21 13:23:10 +05:00
test_spiderstate.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_squeues.py Fixes for Python 3.14. (#7460) 2026-04-27 18:47:52 +05:00
test_squeues_request.py Test refactoring, part 2. (#6928) 2025-06-29 23:10:38 +02:00
test_stats.py Improve coverage statscollectors (#7641) 2026-06-19 16:59:25 +02:00
test_toplevel.py Bump tool versions. (#6941) 2025-07-05 10:30:30 +02:00
test_utils_asyncgen.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_utils_asyncio.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_utils_conf.py Update tool versions (#7127) 2025-10-27 14:11:31 +01:00
test_utils_console.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_curl.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_utils_datatypes.py fix: LocalCache with limit=0 incorrectly stores items 2026-06-25 12:24:17 +05:30
test_utils_decorators.py Add test coverage for scrapy.utils.decorators (#7645) 2026-06-22 08:50:11 +02:00
test_utils_defer.py Add more no-reactor tests (#7259) 2026-03-24 15:29:22 +05:00
test_utils_deprecate.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_utils_display.py Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
test_utils_gz.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_httpobj.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_utils_iterators.py Move to mypy --strict with exceptions. (#7300) 2026-03-02 15:47:23 +05:00
test_utils_log.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_utils_project.py Address warnings in tests. (#7637) 2026-06-19 06:55:02 +02:00
test_utils_python.py Complete test coverage for linkextractors (#7639) 2026-06-19 14:22:03 +05:00
test_utils_reactor.py Fixes for Python 3.14. (#7460) 2026-04-27 18:47:52 +05:00
test_utils_request.py Fix request_to_curl handling of verbose cookies dictionaries (#7603) 2026-06-15 08:46:30 +02:00
test_utils_response.py Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
test_utils_serialize.py Enable the flake8-builtins rules. (#6942) 2025-07-05 21:43:18 +05:00
test_utils_signal.py Rename async test decorators. (#7234) 2026-02-02 20:39:43 +01:00
test_utils_sitemap.py Refactor and improve catching warnings in tests. (#7643) 2026-06-19 21:04:34 +02:00
test_utils_spider.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
test_utils_template.py Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
test_utils_trackref.py improve trackref (#7375) 2026-04-03 15:15:30 +05:00
test_utils_url.py Fix strip_url() removing default port from password. (#7605) 2026-06-15 08:41:55 +02:00
test_zz_resources.py Assorted test fixes (#7585) 2026-06-10 11:01:06 +02:00