Commit Graph

  • 4ca92f4c49 WIP Adrian Chaves 2026-06-30 16:44:20 +0200
  • c30fcc2c16 Merge remote-tracking branch 'origin/master' into throttling Adrian Chaves 2026-06-30 16:30:24 +0200
  • b3ea60354f WIP Adrian Chaves 2026-06-30 16:28:21 +0200
  • a6d6a48aa6
    Keep Item fields in definition order (#7694) Adrian 2026-06-30 16:26:22 +0200
  • f5b4363170 Support --curl in commands Adrian Chaves 2026-06-30 16:19:54 +0200
  • 0e7c89a6a2 WIP Adrian Chaves 2026-06-30 15:33:10 +0200
  • 00098cb596
    Assorted docstring fixes (#7698) Andrey Rakhmatullin 2026-06-30 18:27:32 +0500
  • 1541419170 Merge branch 'master' into assorted-docstring-fixes Andrey Rakhmatullin 2026-06-30 17:56:59 +0500
  • 49c3ca7bdd Address feedback. Andrey Rakhmatullin 2026-06-30 17:56:24 +0500
  • deb7e2861e
    Fix _get_tag_name() crash for non-string elem.tag (#7686) (#7687) Gaurav Yadav 2026-06-30 18:19:47 +0530
  • 039e06bedb Remove stale module-level docstring from scrapy.mail Wali007-lab 2026-06-30 13:54:06 +0200
  • b8ea9a02af Fix sitemap tag crash in utils/sitemap.py swapnil 2026-06-30 17:17:27 +0530
  • 3ae5ab40c1 WIP Adrian Chaves 2026-06-30 13:32:57 +0200
  • 73da54eed2 Update expectations Adrian Chaves 2026-06-30 13:19:41 +0200
  • ba8bb9e4e9 WIP Adrian Chaves 2026-06-30 12:53:18 +0200
  • 90b078ba12
    Merge 42a17b65ee into 6ad8a043ca Adrian 2026-06-30 05:11:36 -0500
  • 6ad8a043ca
    Fix genspider --editor (#7683) Adrian 2026-06-30 12:05:42 +0200
  • d4a404308d Round 2. Andrey Rakhmatullin 2026-06-30 14:47:55 +0500
  • e0cc7ed7db WIP Adrian Chaves 2026-06-30 11:33:37 +0200
  • 23e67837e3 Response stuff. Andrey Rakhmatullin 2026-06-30 12:23:52 +0500
  • 80f161ef3e Merge remote-tracking branch 'origin/master' into throttling Adrian Chaves 2026-06-30 08:23:04 +0200
  • e543c54b2c Queue stuff. Andrey Rakhmatullin 2026-06-30 01:09:51 +0500
  • 8573026049 Add more code blocks in docstrings. Andrey Rakhmatullin 2026-06-30 00:08:39 +0500
  • 26677fede5 Smaller fixes. Andrey Rakhmatullin 2026-06-15 14:57:46 +0500
  • 092a3e3b3d chore: drop accidental pyproject.toml and uv.lock changes MeGaurav4 2026-06-29 23:23:36 +0530
  • da757c798a
    Merge 821afd5804 into 52147017b4 Adrian 2026-06-30 01:11:59 +0800
  • 52147017b4
    Cleanup cookie handling in request_to_curl() (#7684) Andrey Rakhmatullin 2026-06-29 21:49:09 +0500
  • 6591cb756c
    Fix to LocalCache with limit=0 still stores items rather than disabling the cache (#7663) tanishqtayade 2026-06-29 22:07:06 +0530
  • c9d11d2fc9 Log compressed responses over warn size Xiangyu Li 2026-06-29 22:44:59 +0800
  • b53d402288 Keep Item fields in definition order Adrian Chaves 2026-06-29 16:38:40 +0200
  • 8dfb548415 WIP Adrian Chaves 2026-06-29 16:29:31 +0200
  • e132e77330 test: replace silent call with assert results == [] per Adrian MeGaurav4 2026-06-29 19:41:18 +0530
  • e787fbb139 test: address Adrian+syncrain PR feedback MeGaurav4 2026-06-29 19:31:12 +0530
  • 099b6e47e0 test: use minimal payload for non-string tag test MeGaurav4 2026-06-29 19:28:17 +0530
  • 986b83736c Update test expectations Adrian Chaves 2026-06-29 15:50:17 +0200
  • 4b2b56f384
    Update on Request objects (#7286) Javier 2026-06-29 15:34:12 +0200
  • d7746aa2dc Simplify imports Adrian Chaves 2026-06-29 15:32:08 +0200
  • df5edd14ca Improve the error message when FORCE_CRAWLER_PROCESS is not True and probably should Adrian Chaves 2026-06-29 15:08:39 +0200
  • 9559cbee1e
    Solve timing issues with HTTP cache tests? (#7692) Adrian 2026-06-29 14:21:20 +0200
  • 61a19c5497 Solve timing issues with HTTP cache tests? Adrian Chaves 2026-06-29 12:10:45 +0200
  • 184166d629 Use get_* for getters Adrian Chaves 2026-06-29 10:58:18 +0200
  • 7a211960c9 Merge remote-tracking branch 'origin/master' into throttling Adrian Chaves 2026-06-29 10:47:28 +0200
  • 21daf7c66e Add a leading slash to request URLs with a host and query/fragment but no path Apoorva Verma 2026-06-29 12:05:48 +0530
  • fb98ffc310 Fix get_base_url/get_meta_refresh ignoring base tag beyond 4096 chars JSap0914 2026-06-28 19:43:41 +0900
  • 9f97dcab05
    Scope unscoped log counting to crawler context Mridankan Mandal 2026-06-28 03:01:03 +0530
  • 3b60e59b96 test: improve non-string tag test accuracy and add direct unit test MeGaurav4 2026-06-27 23:59:17 +0530
  • b3cd077b59
    Normalize line endings in log counter fix Mridankan Mandal 2026-06-27 18:21:03 +0000
  • 88a7fc40e7
    Scope LogCounterHandler to the current crawler Mridankan Mandal 2026-06-27 18:19:19 +0000
  • 3977e561c0 Fix _get_tag_name() crash for non-string elem.tag (#7686) MeGaurav4 2026-06-27 22:38:22 +0530
  • 6830609a59 Type hint assertCookieValEqual(). Andrey Rakhmatullin 2026-06-27 14:17:32 +0500
  • ab7d89fc4a Add tests for bytes in Request.cookies. Andrey Rakhmatullin 2026-06-27 14:14:36 +0500
  • 4a72e1cdbe Sync and type hint _cookie_to_set_cookie_value() in tests. Andrey Rakhmatullin 2026-06-27 13:53:53 +0500
  • 78ee47c659 Extract _to_verbose_cookies(). Andrey Rakhmatullin 2026-06-27 13:50:54 +0500
  • 4be8754292 Unify logging. Andrey Rakhmatullin 2026-06-27 13:34:41 +0500
  • 12a5ea1a62 Extract _decode_cookie(). Andrey Rakhmatullin 2026-06-27 13:31:09 +0500
  • ae0a50932f Fix SitemapSpider ignoring sitemaps with query parameters in URL (closes #6293) JSap0914 2026-06-27 13:45:06 +0900
  • 1bda79a521 Address test issues Adrian Chaves 2026-06-26 22:56:58 +0200
  • 8834250059 WIP Adrian Chaves 2026-06-26 22:53:11 +0200
  • 44de6ae3bf Update Adrian Chaves 2026-06-26 22:30:51 +0200
  • 707ab67fe2 Drop list of plain cookie dicts support from request_to_curl(). Andrey Rakhmatullin 2026-06-27 01:28:58 +0500
  • 19d5a106c5 Adjust CookiesT. Andrey Rakhmatullin 2026-06-27 01:28:40 +0500
  • bc33907949 Merge remote-tracking branch 'origin/master' into HassaanNaushahi-BR_fix_issue-1998 Adrian Chaves 2026-06-26 22:10:26 +0200
  • 185d6b9a20
    Fix request_to_curl() corrupting dict cookies with bytes keys/values (#7675) greymoth 2026-06-27 04:53:32 +0900
  • d23c44858a Address issues Adrian Chaves 2026-06-26 21:41:45 +0200
  • 8c96c7cf43 Improve test coverage for crawler.py Adrian Chaves 2026-06-26 20:13:20 +0200
  • 4b40d2d06a
    Close various garbage collectible resources (mostly in tests) (#7644) Andrey Rakhmatullin 2026-06-26 23:05:08 +0500
  • 242a57e9a8 Improve test coverage Adrian Chaves 2026-06-26 19:57:30 +0200
  • ae810f06ce Merge branch 'master' into tests-close-resources Andrey Rakhmatullin 2026-06-26 22:48:33 +0500
  • fb031dee80 Merge remote-tracking branch 'origin/master' into throttling Adrian Chaves 2026-06-26 19:01:20 +0200
  • 5aae8334a5 Improve request and scope delay support Adrian Chaves 2026-06-26 19:00:06 +0200
  • dbee61f2b2 Silence mypy warnings Adrian Chaves 2026-06-26 18:53:45 +0200
  • a212f01160 Improve test coverage for crawler.py Adrian Chaves 2026-06-26 18:36:09 +0200
  • d181c919dc Fix genspider --editor Adrian Chaves 2026-06-26 18:31:32 +0200
  • cf5607f8bc
    Undeprecated the basic FormRequest API (#7671) Adrian 2026-06-26 17:35:46 +0200
  • edc353c975
    Improve test coverage for shell/ (#7680) Adrian 2026-06-26 16:40:36 +0200
  • 42a17b65ee Handle get_event_loop() deprecation Adrian Chaves 2026-06-26 16:06:53 +0200
  • ee8c5f7aa4 Fix two environment-specific test failures Adrian Chaves 2026-06-26 16:00:11 +0200
  • c59d32277d fix: apply _cookie_value_to_unicode to list-branch cookie key/value greymoth 2026-06-26 22:14:33 +0900
  • aeb6b4763c Merge remote-tracking branch 'origin/master' into throttling Adrian Chaves 2026-06-26 14:47:23 +0200
  • de0a31d0ad Make the shell config test more reliable on Windows Adrian Chaves 2026-06-26 14:40:29 +0200
  • 4a7a2d5b5f Improve test coverage Adrian Chaves 2026-06-26 14:39:13 +0200
  • 7ec4622cfc Address test issues Adrian Chaves 2026-06-26 14:14:16 +0200
  • 3774d30b4b Improve test coverage for shell/ Adrian Chaves 2026-06-26 13:51:08 +0200
  • 8caa1ac4db Fix typing issues Adrian Chaves 2026-06-26 13:27:14 +0200
  • 1b940a75ac
    Improve the item pipeline docs (#7676) Adrian 2026-06-26 13:18:22 +0200
  • c889c1057d Improve test coverage Adrian Chaves 2026-06-26 13:09:17 +0200
  • ffcc88e73d LxmlLinkExtractor: add deny_attrs and deny_tags Adrian Chaves 2026-06-26 13:06:24 +0200
  • 627cecc93e Merge remote-tracking branch 'origin/master' into throttling Adrian Chaves 2026-06-26 12:17:22 +0200
  • 6ce90dde3e Minimum queuelib: 1.4.2 → 1.6.1 Adrian Chaves 2026-06-26 12:15:52 +0200
  • 7ab404c725
    Add a documentation page about security (#7678) Adrian 2026-06-26 11:58:37 +0200
  • b2af34a07d Add references to security.rst Adrian Chaves 2026-06-26 11:58:05 +0200
  • 0ccddb4f61
    Improve test coverage for contracts (#7677) Adrian 2026-06-26 11:47:30 +0200
  • 7a6bbd6a8a Cover DOWNLOADER_CLIENT_TLS_CIPHERS Adrian Chaves 2026-06-26 11:37:50 +0200
  • 3e02b9e5ab Merge remote-tracking branch 'origin/master' into throttling Adrian Chaves 2026-06-26 11:32:43 +0200
  • 541d3d01dc Add a documentation page about security Adrian Chaves 2026-06-26 11:28:18 +0200
  • 3c8a654fc9 Improve test coverage for contracts Adrian Chaves 2026-06-26 11:12:33 +0200
  • ffe5953988 Improve the item pipeline docs Adrian Chaves 2026-06-26 10:47:25 +0200
  • 8190b5b703 Add a throttling-aware scheduler Adrian Chaves 2026-06-26 10:44:31 +0200
  • a0d5207846 test: remove unnecessary comment tanishqtayade 2026-06-26 13:37:29 +0530
  • 48913efb7d test: clean up comments in resolver test tanishqtayade 2026-06-26 13:15:44 +0530