Since Python 3.14 (PEP 649) annotations are evaluated lazily, so
inspect.signature() raises NameError for callables whose annotations
reference names imported only under TYPE_CHECKING. This broke
middleware registration (via argument_is_required()) and custom stats
collectors (via _warn_spider_arg) for user code with such annotations.
Use annotation_format=Format.FORWARDREF on 3.14+: parameter names,
kinds and defaults are unchanged, and unresolvable annotations become
ForwardRef proxies instead of raising.
Resolves#7796.
* Store full feed batches before spider closes (#7730)
Start closing and storing each batch as soon as it reaches the configured item count. Track unfinished close tasks so spider shutdown still waits for all deliveries before emitting the exporter-closed signal.
Add an end-to-end regression test that verifies the first batch is stored while the crawl is still running.
* Remove the issue reference
---------
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
* Skip the deprecated scrapy.mail in coverage data
* Improve CrawlSpider coverage
* Improve XMLFeedSpider coverage
* Improve SitemapSpider coverage
* Solve mypy issues
* Align new spider tests with the shared test helper structure
* Fix CaseInsensitiveDict.copy() sharing state with the original
* Address review: don't re-normalise in __copy__, keep _keys in sync in __ior__
UserDict.__ior__ writes self.data directly, bypassing __setitem__, so
_keys never learned about the new keys.
curl merges repeated -d/--data/--data-raw options into a single request
body joined with "&" (e.g. `curl -d a=1 -d b=2` sends `a=1&b=2`). scrapy's
DataAction kept only the last value, silently dropping the earlier ones, so
`Request.from_curl("curl -d a=1 -d b=2 ...")` produced a body of just `b=2`.
Accumulate the values with "&" to match curl. -H/--header and -b/--cookie
already accumulate via action="append"; -d was the only repeatable data flag
that didn't.
Adds state tracking to CsvItemExporter to detect when auto-detected fields drop data on items. Injects a warning instructing users to configure FEED_EXPORT_FIELDS. Includes unit tests. Corrects the false positive logic in stalled PR #7613.
* Fix _get_tag_name() crash for non-string elem.tag (#7686)
* test: improve non-string tag test accuracy and add direct unit test
* test: use minimal payload for non-string tag test
* test: address Adrian+syncrain PR feedback
- remove first docstring line (Adrian: unnecessary)
- replace weak isinstance assert with no-op call
- keep Cython function mention (Adrian: wording is great)
* test: replace silent call with assert results == [] per Adrian
* chore: drop accidental pyproject.toml and uv.lock changes
Adrian
Janit Rajkarnikar
Fandu
Laerte Pereira
Youssef Mohamed
Andrey Rakhmatullin
Shadow_Lu
SpiliosDmk
Mridankan Mandal
Madan kumar
Shaquon Kelley
Fat-Coder-CN
Gaurav Yadav