RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,743 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

110 Commits

Author SHA1 Message Date
Adrián Chaves bee74fb753 Remove trailing whitespace 2025-03-11 11:56:44 +01:00
Andrey Rakhmatullin ce5a132f12
Run and fix linkcheck. (#6524) 2024-11-04 11:40:07 +01:00
Adrián Chaves 6fc7827042
Do not close the underlying file from compression plugins (#6239) 2024-02-21 14:27:42 +01:00
Adrián Chaves 1864f48e9e Link to Zyte’s export guides 2023-12-20 12:47:34 +01:00
Kevin Lloyd Bernal 9a1bf40c2f expose AWS_REGION_NAME in S3FeedStorage 2023-07-24 16:37:52 +08:00
Mikhail Korobov af2aa4b421
Merge pull request #5847 from namelessGonbai/feat/FixFeedExport 
Fix FeedExporter not to export empty file
 2023-07-22 23:18:36 +05:00
Alex 04ee3303e4
Adding support for Windows of absolute pathlib.Path objects in FeedExporter (#5939) 2023-06-22 07:04:06 +02:00
Andrey Rakhmatullin 85fe88f80f
Merge pull request #5833 from GodelTech/issue_960_s3_feed_export 
Fix #960: S3 Feed Export throws boto error
 2023-06-13 18:44:06 +04:00
Andrey Rakhmatullin 8a64f3e8de
Merge branch 'master' into feat/FixFeedExport 2023-06-13 18:21:06 +04:00
isabela_catanante 5adada5d19 Improve the overwrite feed option documentation 2023-05-12 12:55:24 +02:00
jazzthief cb67bc17b7
Remove `botocore` from docs 2023-03-16 17:25:05 +01:00
namelessGonbai 6ab49e954f
Update docs/topics/feed-exports.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-03-16 18:03:06 +09:00
kenshi kikuchi 50801c7207 Fix Docs 2023-03-16 17:17:20 +09:00
Yegor 29c2477f0a
Document the need to install boto3 2023-02-15 16:40:05 +01:00
Pankaj Kumar 280cd6ce71
Merge branch 'scrapy:master' into blacken-docs 2023-02-03 11:38:26 +05:30
Adrián Chaves 068af85722
Add release notes for Scrapy 2.8 (#5799) 2023-02-02 04:54:02 +01:00
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs' 
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
 2023-02-01 16:30:57 +05:30
Laerte Pereira b611848029
fix(docs): Change `versionchanged` value 2023-01-20 08:35:17 -03:00
Laerte Pereira 973f0cf567
fix: line break 2023-01-20 08:23:05 -03:00
Laerte Pereira 8270df754d
Set `FEED_EXPORT_ENCODING='utf-8'` in the default template 2023-01-20 07:55:16 -03:00
Mikhail Korobov 500dae82e2
Merge pull request #3696 from Gallaecio/allow-customizing-export-column-names 
Allow customizing export column names
 2022-06-27 00:26:05 +05:00
Michel Ace 6e878490e8
Support and prefer the .jsonl file extension (#4848) 2022-06-17 08:37:14 +02:00
Adrián Chaves e8503217fb Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2022-06-16 20:19:52 +02:00
Adrián Chaves aa0306a167
Cover 2.6.0 in the release notes (#5399) 2022-03-01 12:16:37 +01:00
Sixuan (Cherie) Wu 30d5779ea9
Fix FEED_URI_PARAMS: custom params throws KeyError (#4966) 
* fix FEED_URI_PARAMS: custom params throws KeyError closes #4962

* another try FEED_URI_PARAMS

* add warning message and change default function

* Add tests for FEED_URI_PARAMS

* FEED_URI_PARAMS: warn if the params dict has been modified in-place

* [Doc] FEED_URI_PARAMS: modifying params in-place is deprecated

* Remove whileline

* Rename parameters for lambda function

* Type hints for FeedExporter._get_uri_params

Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2022-01-28 14:30:30 -03:00
D R Siddhartha 43ea21e830
Feed post-processing plugin support (#5190) 2021-08-24 11:48:01 +02:00
laggardkernel 8e7b96d8a2 Tweak doc for setting AWS_SESSION_TOKEN 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-07-27 19:40:02 +08:00
laggardkernel 7e4321f201 Add support for temporary security credential in AWS auth 2021-07-27 19:39:59 +08:00
Marlena Chatzigrigoriou d7deba7e89
Document all import paths and use the shortest in examples (#5099) 2021-07-14 10:34:28 +02:00
Türkalp Burak KAYRANCIOĞLU fcc6becc58
S3FeedStorage: allow custom endpoint (#4998) 
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
 2021-07-14 10:00:43 +02:00
D R Siddhartha 4ddc9d6b55
Feeds: Item Filters (#5178) 2021-07-13 17:22:29 +02:00
Shinichi Takayanagi bd60c3f41f
More documentation for setting spider atributes 
* docs: require sphinx-rtd-theme>=0.5.2 and the latest pip to prevent installing breaking docutils>=0.17

* Update feed-exports.rst

* Update feed-exports.rst

* Reflects the comments

* Remove redundant newline

* Update docs/topics/feed-exports.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Apply suggestions from code review

Co-authored-by: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2021-05-11 00:58:04 +05:00
Adrián Chaves 1773eaf5dc Move lists closer to their introducing paragraph 2021-02-03 11:43:18 +01:00
Adrián Chaves 07379cf9b7
Merge branch 'master' into allow-customizing-export-column-names 2020-11-20 20:06:18 +01:00
Adrián Chaves 015c82b974
Scrapy 2.4 release notes (#4808) 2020-10-11 22:09:45 +02:00
Ilia Sergunin d10464ca96
Update docs/topics/feed-exports.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-09-01 10:13:40 +03:00
maranqz a8e895e684 kwargs for Item exporters classes 
test
docs
 2020-08-30 10:57:22 +03:00
Adrián Chaves 3f0a677c04
Cover version directive usage in the documentation policy (#4310) 
* Cover version directives in the documentation policy

* Remove version directives in preparation for Scrapy 2.0

* Update the policy based on the deprecation policy

* Only remove version directives after 3 years
 2020-08-27 23:56:58 +05:00
Adrián Chaves e70975f0bb
Allow overwriting feeds (#4512) 
Co-authored-by: Yuval Hager <yhager@yhager.com>
 2020-08-17 15:10:08 +02:00
Adrián Chaves 65e0abaea5 Document FEED_URI_PARAMS 2020-08-13 09:05:51 +02:00
Adrián Chaves 5138f9a965 Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2020-08-11 13:42:05 +02:00
Adrián Chaves 3600582f56
Cover Scrapy 2.2.1 and 2.3 in the release notes (#4708) 2020-08-04 20:05:56 +02:00
BroodingKangaroo a6c1d79b7c pep8 tiny changes 2020-07-28 11:53:05 +03:00
Adrián Chaves c4f9250220
Merge branch 'master' into ISSUE-4250-add_batch_deliveries 2020-07-16 17:44:18 +02:00
BroodingKangaroo 41263f61c6 Change single quotes to double in example in docs 2020-07-16 18:41:45 +03:00
BroodingKangaroo 8bdcdb0a76 Add quotes to example in docs 2020-07-16 09:13:54 +03:00
BroodingKangaroo 6454d456d2 Make check of placeholder less strict 2020-07-03 10:24:13 +03:00
BroodingKangaroo 1e245046ed Change setting name. Add leading zeroes to batch_id. Minor fixes. 2020-07-02 12:38:08 +03:00
BroodingKangaroo 7b1d3c35ea Minor updates 2020-07-01 12:24:31 +03:00
BroodingKangaroo 05c2587c6a Docs update and tiny fixes 2020-06-28 09:45:45 +03:00