d8ba1571e7
Assorted docs fixes, part 2. ( #7725 )
...
* Assorted docs fixes, part 2.
* Second pass.
* Address feedback.
2026-07-14 00:49:44 +05:00
bdf3067935
Assorted docs fixes, part 1. ( #7710 )
2026-07-07 18:39:34 +05:00
fb3455304d
Add content-based image filtering example ( #4954 )
2026-06-23 09:04:19 +02:00
beb6c51c17
Update default_settings, deprecate CRAWLSPIDER_FOLLOW_LINKS. ( #7592 )
2026-06-11 00:22:09 +05:00
5149e2c679
docs: switch `scrapy.Item` examples to dataclasses ( #7513 )
...
* docs: switch `scrapy.Item` examples to dataclasses
* make serializer doc generic
* use modern type hints
* docs/spiders: switch TestItem consumer snippets to attribute access
Since the TestItem migration to @dataclass, the existing
item["id"] = ... assignments would raise TypeError on copy-paste.
Switch to item.id = ... to match the new dataclass declaration.
The snippets sit under .. skip: next so docs-tests still pass either
way, but the change keeps the examples runnable for readers.
2026-06-08 12:40:28 +05:00
7e881ce2d7
Release notes for 2.15.0 ( #7373 )
...
* Cover 2.15.0 in the release notes (up to 31bf7c30c6ccf5b68f267
2026-04-09 16:56:30 +05:00
9381ad893d
Remove outdated version marker directives (< 2.8.0) from documentation ( #7198 )
2025-12-22 10:00:39 +01:00
d414d393d4
Remove Python 3.9 support ( #7121 )
2025-10-27 12:37:49 +01:00
d0dabbc097
Enable AsyncioSelectorReactor by default. ( #6713 )
...
* Enable AsyncioSelectorReactor by default.
* Improve get_crawler(), switch more tests to it.
* Fix the remaining default-reactor test failures.
* Address documentation feedback.
* Make pinned envs more consistent.
2025-03-12 01:18:30 +05:00
bee74fb753
Remove trailing whitespace
2025-03-11 11:56:44 +01:00
ce5a132f12
Run and fix linkcheck. ( #6524 )
2024-11-04 11:40:07 +01:00
ee1189512f
Replace urlparse with urlparse_cached where possible ( #6229 )
2024-02-20 12:47:29 +01:00
76eba9977b
Skip or fix failing code block tests ( #5826 )
2023-02-14 09:40:38 +01:00
c1bbb299d7
Add and run pre-commit hook 'blacken-docs'
...
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
2023-02-01 16:30:57 +05:30
40d9ca3bdd
use pathlib
2022-10-17 17:40:10 -04:00
407562b38b
Drop Python 3.6 support ( #5514 )
...
* chore: Drop Python 3.6 support
* Attend PR comments
* Tweak versions
* Update dependencies version
* fix: Ubuntu workflow
* fix windows workflow
* chore: Remove comment
* update `install_requires` dependencies versions
* move lxml to main pinned requirements
* Attend code-review comments
* remove non-pinned 3.7 from windows workflow
* simplify condition
* lint
* remove paragraph
* refactor
* remove leftover
2022-06-09 00:25:03 -03:00
f39def4492
add docs
2022-05-25 23:57:38 +04:30
aa0306a167
Cover 2.6.0 in the release notes ( #5399 )
2022-03-01 12:16:37 +01:00
47533985f4
Document file expiration method in media-pipeline
2021-10-01 12:30:14 -06:00
b22a004398
Document media pipeline file naming ( #5152 )
2021-07-26 20:51:32 +02:00
d7deba7e89
Document all import paths and use the shortest in examples ( #5099 )
2021-07-14 10:34:28 +02:00
015c82b974
Scrapy 2.4 release notes ( #4808 )
2020-10-11 22:09:45 +02:00
85e13aff74
Merge pull request #4782 from sripriyesha/patch-1
...
add mention of FTP server storage in media storage intro
2020-09-15 01:17:46 +05:00
7a83474cc5
add mention of FTP server storage in media storage intro
...
At the beginning of this doc, in "Specifying where to store the media" feature details, FTP server storage mention was missing
2020-09-08 17:16:31 +02:00
1c4b4cc6b0
Support defining file path based on item in media pipelines ( #4686 )
2020-08-11 14:12:44 +02:00
07470e1a3c
Merge pull request #3608 from ejulio/feat-685
...
[MRG+1] Fix for #685 Add Google Cloud Storage Feed Export
2020-07-16 18:11:19 +05:00
6f4c964aa4
Cover Scrapy 2.2.0 in the release notes ( #4630 )
2020-06-24 12:26:38 +02:00
cfd039aeb6
Remove a duplicate GCS_PROJECT_ID reference target
2020-06-22 19:28:33 +02:00
ec98dabfab
Support for dataclass and attrs items ( #3881 )
2020-06-14 11:45:27 +02:00
83a0cc6cdf
Add status to files information
2020-04-12 23:22:17 +03:00
6aa0ba4553
Write release notes for Scrapy 2.0.0 ( #4329 )
2020-03-03 09:11:11 +01:00
752e8f7018
FilesPipeline.file_path has optional arguments
...
Documented signature doesn't match the actual interface in [files.py](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L520 ).
Specifically, it looks like it may be [called](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L422 ) without a response value.
I found this when I was implementing the pipeline with the signature `file_path(self, request, response, info)` and the following error was being return in my results :
[(False, <twisted.python.failure.Failure builtins.TypeError: file_path() missing 1 required positional argument: 'response'>)]
Scrapy==1.8.0
2020-01-28 15:21:15 -06:00
fc98aa6b67
Merge branch 'master' into ftp_files#3928
2020-01-24 14:52:40 +02:00
14d4428e70
Rephrasing documentation for image and file pipelines ( #4252 )
...
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.
* scrapy#4034 rebase master
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.
* scrapy#4034 rebase master
* Rebase master
* Add class to media pipeline docs
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
Co-authored-by: elacuesta <elacuesta@users.noreply.github.com>
2019-12-30 00:56:22 +05:00
c0d84f0962
fix typos
2019-12-18 19:39:21 +01:00
e84cb18ca0
Use InterSphinx to link to the Twisted documentation
2019-11-18 15:50:45 +01:00
229e722a03
Initial Python 2 removal ( #4091 )
2019-10-31 10:46:02 +01:00
175cd2ece5
Update docs/topics/media-pipeline.rst
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2019-10-01 07:27:31 +02:00
28005b2872
Update media-pipeline.rst
2019-09-28 06:21:14 +02:00
b14c3cb612
Add media pipelines FTP documentation
2019-09-19 23:33:57 +02:00
4d4bd0e823
Merge pull request #3609 from Gallaecio/2253
...
Document FilesPipeline.file_path and ImagesPipeline.file_path
2019-07-04 22:05:46 +05:00
431f18a9a1
Document FilesPipeline.file_path and ImagesPipeline.file_path
2019-03-26 12:15:19 +01:00
75d6f56c8a
Switch from ` to `` where inline code formatting is desired
2019-03-01 16:56:58 +01:00
8fca98616a
fix grammar
2019-01-28 10:19:31 -03:00
6ef6585b5a
update docs
2018-04-13 19:06:29 +09:00
464973489e
Using bucket's default object ACL
2018-04-13 12:06:39 +09:00
74a9c65290
update docs for support gcs acl
2018-04-03 18:20:37 +09:00
2dee191374
Merge branch 'master' into extending-s3-files-store
2017-12-31 16:44:38 -03:00
9dd680d5c9
Use https for external links wherever possible in docs
2017-10-26 23:33:45 +05:30
d4555b2bcc
update docs for supporting google cloud storage
2017-09-29 13:04:17 +09:00
Andrey Rakhmatullin
smellslikeml
Omkar Kabde
Adham 3llam
Adrián Chaves
Laerte Pereira
pankaj1707k
Matthew Donoughe
Mohammadtaher Abbasi
Peter Morrison
Rob Banagale
Marlena Chatzigrigoriou
Mikhail Korobov
KAILASA's Sri Nithya Priyeshananda
Ajay Mittur
Eugenio Lacuesta
ilias-ant
Daniel Kimsey
OmarFarrag
1um0s
Marc Hernandez Cabot
Harry Moreno
rhoboro
Daniel Graña
Aditya