RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,994 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

2318 Commits

Author SHA1 Message Date
Adrián Chaves 735750c254 Cover 1.8.1 in the release notes 2021-10-06 12:33:57 +02:00
Adrián Chaves f0105a882d Cover 2.5.1 in the release notes 2021-10-06 12:33:44 +02:00
Andrey Rakhmatullin 7ec5f299c4 Small documentation fixes. 2021-10-06 12:33:32 +02:00
Andrey Rakhmatullin b081f18a2f Add http_auth_domain to HttpAuthMiddleware. 2021-10-06 12:33:18 +02:00
Adrián Chaves d91d82b506
Make Scrapy SFW again 2021-10-01 16:31:29 +02:00
Andrey Rahmatullin 7d557f898c
Merge pull request #5239 from maanijou/master 
Improve documentation for spider middlewares
 2021-09-27 15:42:31 +05:00
Adrián Chaves 3c57825b0f
Update docs/topics/spider-middleware.rst 2021-09-26 13:41:26 +02:00
Reza (Milad) Maanijou dfdb779756
Apply review comments 2021-09-26 12:45:44 +03:30
Reza (Milad) Maanijou 1829dd774c
Update spider-middleware.rst 2021-09-25 20:22:09 +03:30
kamran890 e5998fb846 Document spider.state attribute (#5174) 2021-09-22 03:15:12 +05:00
maanijou ac9175964d Improve documentation for spider middlewares 2021-09-12 17:59:20 +02:00
D R Siddhartha 43ea21e830
Feed post-processing plugin support (#5190) 2021-08-24 11:48:01 +02:00
Andrey Rakhmatullin cd17c829cf Revert "Move documentation about avoiding bans into a topic of its own (#4039)" 
This reverts commit 2d2581c68f.
 2021-08-23 19:55:35 +05:00
Andrey Rahmatullin bdf8355ce1
Merge pull request #5231 from umairnsr87/umairnsr87/issues/5226 
updated documentation for python version for reppy
 2021-08-18 21:08:32 +05:00
databender 572d347b3b warning view updated 2021-08-18 16:17:52 +05:30
Adrián Chaves 2d2581c68f
Move documentation about avoiding bans into a topic of its own (#4039) 2021-08-18 12:46:42 +02:00
databender d623ed15d1 indentation updated 2021-08-18 14:51:03 +05:30
databender bcf38a6719 added upstream issue for not supported python version 2021-08-18 14:48:47 +05:30
Andrey Rahmatullin a7bd6fa229
Merge pull request #3669 from Gallaecio/spider-name-collision 2021-08-17 19:42:53 +05:00
databender ebddb77a33 updated suggested changes after review 2021-08-16 18:08:26 +05:30
databender cc1cb2de0c updated suggested changes 2021-08-16 17:21:47 +05:30
databender 1a8b98843a updated documentation for python version for reppy 2021-08-16 17:00:05 +05:30
databender 8bbaea9892 updated documentation for python version for reppy 2021-08-16 16:57:43 +05:30
Aaron Tan 954f303590
Update docs/topics/settings.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-08-09 22:23:23 +10:00
Aaron Tan 8e7d2ef133 Document JOBDIR option issue #5173 
Add JOBDIR setting to the settings page.
Add default JOBDIR setting to global defaults in scrapy.settings.default_settings module.
 2021-08-07 11:44:12 +10:00
Aaron Tan 880a4d9493
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-08-01 11:02:27 +10:00
Aaron Tan d55b6fcad6 Fix for duplicate object description error 2021-07-28 12:10:34 +10:00
Aaron Tan 714aa3970e
Merge branch 'scrapy:master' into docs-extension 2021-07-30 20:36:58 +10:00
Andrey Rahmatullin b406affd5b
Merge pull request #5210 from laggardkernel/feature/aws-session-token 2021-07-30 15:15:55 +05:00
Eugenio Lacuesta cc89f6be38
Response.attributes (#5218) 2021-07-29 22:12:44 +02:00
Eugenio Lacuesta 4b62ac6c3a
Update headless browser docs to mention playwright 2021-07-28 15:00:24 -03:00
Eugenio Lacuesta 9aef9b78eb
Merge remote-tracking branch 'upstream/master' into docs-headless-browser 2021-07-28 14:31:49 -03:00
Eugenio Lacuesta 0e3d50dd18
Update docs/topics/dynamic-content.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-07-28 14:30:16 -03:00
Eugenio Lacuesta 494e0ad8ff
Update docs/topics/dynamic-content.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-07-28 14:29:50 -03:00
laggardkernel 8e7b96d8a2 Tweak doc for setting AWS_SESSION_TOKEN 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-07-27 19:40:02 +08:00
laggardkernel 7e4321f201 Add support for temporary security credential in AWS auth 2021-07-27 19:39:59 +08:00
Rob Banagale b22a004398
Document media pipeline file naming (#5152) 2021-07-26 20:51:32 +02:00
Aaron Tan 209c1fce02
Reference MailSender in StatsMailer 
Added a reference to MailSender in the StatsMailer extension description and included a link to the document detailing how to instantiate MailSender and using Scrapy settings objects.
 2021-07-24 14:50:48 +10:00
Marlena Chatzigrigoriou d7deba7e89
Document all import paths and use the shortest in examples (#5099) 2021-07-14 10:34:28 +02:00
Türkalp Burak KAYRANCIOĞLU fcc6becc58
S3FeedStorage: allow custom endpoint (#4998) 
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
 2021-07-14 10:00:43 +02:00
D R Siddhartha 4ddc9d6b55
Feeds: Item Filters (#5178) 2021-07-13 17:22:29 +02:00
Andrey Rahmatullin 1c46d5aa93
Merge pull request #5191 from ivanprado/master 
CloseSpider can be raised on spider_idle signal handler to set the closing reason
 2021-07-12 18:46:40 +05:00
Iván de Prado Alonso e94d3ac173 Expanded doc for idle signal 2021-06-29 13:40:43 +01:00
Adrián Chaves 73ff9ffd64
spiders.rst: indent warnings into class descriptions 2021-06-26 08:58:29 +02:00
Iván de Prado Alonso 812b4bb518 CloseSpider can be raised on spider_idle signal handler 2021-06-23 17:22:01 +01:00
Adrián Chaves c5b1ee8101
Make Twisted[http2] installation optional (#5113) 
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2021-05-11 09:04:53 +02:00
Shinichi Takayanagi bd60c3f41f
More documentation for setting spider atributes 
* docs: require sphinx-rtd-theme>=0.5.2 and the latest pip to prevent installing breaking docutils>=0.17

* Update feed-exports.rst

* Update feed-exports.rst

* Reflects the comments

* Remove redundant newline

* Update docs/topics/feed-exports.rst

Co-authored-by: Adrián Chaves <adrian@chaves.io>

* Apply suggestions from code review

Co-authored-by: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2021-05-11 00:58:04 +05:00
Eugenio Lacuesta cec36a9284
Refactor request to/from dict (#5130) 2021-05-10 18:00:08 +02:00
Renne Rocha 34b216289c
Update link for reasoning value of URLLENGTH_LIMIT (#5134) 2021-05-06 16:34:05 +02:00
Eugenio Lacuesta ddea6b7bfa
Scheduler: minimal interface, API docs (#3559) 2021-04-26 16:16:14 -03:00