RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,464 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

1490 Commits

Author SHA1 Message Date
Habeeb Shopeju f47b120e2b
Documentation of link extractor usage (#4775) 
* Added description when using link extractor outside crawlspiders and created reference documentation for scrapy.link.Link class

* Added link.rst to toctree

* Corrected spelling errors, moved docs to Link doctstring to use autoclass

* Moved link docs to link_extractors

* Update docs/topics/link-extractors.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Update link.py

Improvements to URL description

* Update link.py

* Update link.py

Fixed line length Flake issue

* Update link.py

Fixed trailing whitespace

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-10-01 23:50:11 +05:00
Andrey Rahmatullin 5a38639359
Merge pull request #4809 from madeny/patch-1 
Correct some typos
 2020-09-28 18:26:43 +05:00
madeny eff96038c7
Correct some typos 
This won't be an issue if **your** spider doesn't rely on cookies.
 2020-09-26 22:50:38 +02:00
Mikhail Korobov 85e13aff74
Merge pull request #4782 from sripriyesha/patch-1 
add mention of FTP server storage in media storage intro
 2020-09-15 01:17:46 +05:00
KAILASA's Sri Nithya Priyeshananda 7a83474cc5
add mention of FTP server storage in media storage intro 
At the beginning of this doc, in "Specifying where to store the media" feature details, FTP server storage mention was missing
 2020-09-08 17:16:31 +02:00
Ammar Najjar defeaacbc2 Merge branch 'master' into 4307-use-f-strings 
Conflicts resolved:
- delete tests/py36/_test_crawl.py
 2020-08-29 07:12:44 +02:00
Adrián Chaves 3f0a677c04
Cover version directive usage in the documentation policy (#4310) 
* Cover version directives in the documentation policy

* Remove version directives in preparation for Scrapy 2.0

* Update the policy based on the deprecation policy

* Only remove version directives after 3 years
 2020-08-27 23:56:58 +05:00
Adrián Chaves 195f738bba
Update Python version references after dropping support for 3.5 (#4742) 
* Update Python version references after dropping support for 3.5

* Remove outdated test

* Undo change affecting collect_asyncgen

* Undo change to be handled by #4743

* Remove unused import

* Remove unused import

* Update tests/requirements-py3.txt

Co-authored-by: Mikhail Korobov <kmike84@gmail.com>

Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2020-08-27 15:43:43 +05:00
Mikhail Korobov a6b67cf4a1
Merge pull request #4755 from maranqz/csv-item-exporter-errors 
Add errors parameter for CsvItemExporter with tests
 2020-08-27 12:51:53 +05:00
Ammar Najjar 7597193dbe
Merge branch 'master' into 4307-use-f-strings 
Resolve Conflicts:
	tests/test_middleware.py
 2020-08-26 13:46:46 +02:00
maranqz 29725e4b58 Fix tabulation of rst and change documentation link. 2020-08-26 14:38:37 +03:00
nyov cf50561b86
Allow passing classes directly in Settings (#3873) 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-08-26 13:08:14 +02:00
maranqz 560c335c07 Add errors parameter in documentation. 2020-08-26 14:00:51 +03:00
Ammar Najjar 58ca8bbf6d Use f-strings (#4307) 2020-08-26 07:11:05 +02:00
sakshamb2113 42383cc267
Add a setting to customize the asyncio event loop (#4414) 2020-08-19 09:18:14 +02:00
Adrián Chaves e70975f0bb
Allow overwriting feeds (#4512) 
Co-authored-by: Yuval Hager <yhager@yhager.com>
 2020-08-17 15:10:08 +02:00
Eugenio Lacuesta 2aa4f3cbf9
Conditional request attribute binding for responses (#4632) 2020-08-17 10:39:59 +02:00
Andrey Rahmatullin acb3b4433a
Merge pull request #4724 from Gallaecio/feed-uri-params 
Document FEED_URI_PARAMS
 2020-08-14 14:47:55 +05:00
Mikhail Korobov 61653418ff
Merge pull request #4090 from Gallaecio/documentation-build 
Fix references to Python types in parameter type fields
 2020-08-14 00:58:06 +05:00
Adrián Chaves 65e0abaea5 Document FEED_URI_PARAMS 2020-08-13 09:05:51 +02:00
Ajay Mittur 1c4b4cc6b0
Support defining file path based on item in media pipelines (#4686) 2020-08-11 14:12:44 +02:00
Adrián Chaves b2f4df5cb7 Merge remote-tracking branch 'upstream/master' into documentation-build 2020-08-11 13:16:14 +02:00
Adrián Chaves 4ee538e44b
Update unicode references from Python 2 times in the documentation (#4703) 2020-08-04 20:34:11 +02:00
Adrián Chaves 3600582f56
Cover Scrapy 2.2.1 and 2.3 in the release notes (#4708) 2020-08-04 20:05:56 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Mikhail Korobov 5e2d1bd187
Merge pull request #4434 from BroodingKangaroo/ISSUE-4250-add_batch_deliveries 
Feed exports: add batch deliveries
 2020-07-29 23:03:28 +05:00
BroodingKangaroo a6c1d79b7c pep8 tiny changes 2020-07-28 11:53:05 +03:00
Andrey Rahmatullin f3372a3753
Merge pull request #4254 from elacuesta/spider.parse 
Change Scraper API to call internal `_parse` method
 2020-07-21 17:37:54 +05:00
Adrián Chaves c4f9250220
Merge branch 'master' into ISSUE-4250-add_batch_deliveries 2020-07-16 17:44:18 +02:00
BroodingKangaroo 41263f61c6 Change single quotes to double in example in docs 2020-07-16 18:41:45 +03:00
Mikhail Korobov 07470e1a3c
Merge pull request #3608 from ejulio/feat-685 
[MRG+1] Fix for #685 Add Google Cloud Storage Feed Export
 2020-07-16 18:11:19 +05:00
BroodingKangaroo 8bdcdb0a76 Add quotes to example in docs 2020-07-16 09:13:54 +03:00
Júlio César Batista 38496a00b7
Use the itemlaoders library (#4516) 2020-07-15 12:08:36 +02:00
Aditya Kumar 3f7e8635f4
Allow the parse command to write data to a file (#4377) 2020-07-11 08:48:24 +02:00
Eugenio Lacuesta 2be2bdd2df
Merge remote-tracking branch 'upstream/master' into spider.parse 2020-07-08 14:00:54 -03:00
BroodingKangaroo 6454d456d2 Make check of placeholder less strict 2020-07-03 10:24:13 +03:00
BroodingKangaroo 1e245046ed Change setting name. Add leading zeroes to batch_id. Minor fixes. 2020-07-02 12:38:08 +03:00
Andrey Rahmatullin af55d23167
Update the OpenSSL cipher list format link 
OpenSSL `ciphers(1)` is now almost empty: https://www.openssl.org/docs/manmaster/man1/ciphers.html

Alternative would be linking to 1.1.1 docs specifically.
 2020-07-01 19:46:54 +05:00
BroodingKangaroo 7b1d3c35ea Minor updates 2020-07-01 12:24:31 +03:00
Eugenio Lacuesta 5b88c522ac
Simplify dataclass example in item loader docs 2020-06-30 12:18:21 -03:00
BroodingKangaroo 05c2587c6a Docs update and tiny fixes 2020-06-28 09:45:45 +03:00
BroodingKangaroo c3cee74fd4 Change default value of FEED_STORAGE_BATCH_ITEM_COUNT to 0 2020-06-26 18:45:21 +03:00
BroodingKangaroo 77333666f1 resolve conflicts 2020-06-26 14:55:46 +03:00
Adrián Chaves 6f4c964aa4
Cover Scrapy 2.2.0 in the release notes (#4630) 2020-06-24 12:26:38 +02:00
Adrián Chaves cfd039aeb6 Remove a duplicate GCS_PROJECT_ID reference target 2020-06-22 19:28:33 +02:00
Eugenio Lacuesta 73b6ce8cb5
Update docs about dataclass items and loaders 2020-06-22 14:13:37 -03:00
Eugenio Lacuesta 1335d9053e
Update docs/topics/loaders.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-06-22 14:05:44 -03:00
Eugenio Lacuesta 3efea98e05
Docs: add note about dataclass items and loaders 2020-06-22 12:41:25 -03:00
Eugenio Lacuesta c7f1c7e3f3
Merge branch 'master' into feat-685 2020-06-19 21:30:54 -03:00
Devi Sandeep 5d54173187
Update docs on accessing callback arguments in errback (#4634) 2020-06-18 12:01:38 +02:00