Address reviewer feedback that spider middleware coverage was incomplete.
Changes include:
- Add new "Start request processing" section explaining process_start()
- Update lifecycle overview to include start processing as phase 2
- Update spider middleware section to show both integration points
- Update sequence diagram to show START PHASE and CRAWL PHASE
- Add missing signal mentions (engine_started, spider_opened,
request_dropped, spider_closed)
Document the end-to-end flow of requests through Scrapy's internals,
including the engine's orchestration role, scheduling, downloading,
spider callback execution, and item pipeline processing.
This conceptual guide complements the existing architecture overview
by providing a detailed narrative of the request lifecycle.
ajbmachon