RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,994 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

8994 Commits

Author SHA1 Message Date
Georgiy Zatserklianyi d3f1bf79e8
Use f-strings where appropriate (#5246) 2021-10-07 16:27:20 +02:00
Eugenio Lacuesta 029cab72e8
[CI] fix pypy test (#5264) 2021-10-06 19:34:09 +02:00
Laerte 6c858cec91
Cookies: Cast primitive types to str (#5253) 
* cast primitive types to str

* add tests
 2021-10-06 12:32:04 -03:00
Adrián Chaves 214525c2c1
Merge pull request #5263 from Gallaecio/port-security-fix 
Port security fix from 1.8.1 and 2.5.1 to the main branch
 2021-10-06 13:14:35 +02:00
Adrián Chaves 735750c254 Cover 1.8.1 in the release notes 2021-10-06 12:33:57 +02:00
Adrián Chaves f0105a882d Cover 2.5.1 in the release notes 2021-10-06 12:33:44 +02:00
Andrey Rakhmatullin 7ec5f299c4 Small documentation fixes. 2021-10-06 12:33:32 +02:00
Andrey Rakhmatullin b081f18a2f Add http_auth_domain to HttpAuthMiddleware. 2021-10-06 12:33:18 +02:00
Andrey Rahmatullin cbf9349552
Merge pull request #5255 from raihan71/raihan71-patch-1 
Using Logo Scrapy in Readme.md
 2021-10-03 19:59:47 +05:00
Raihan Nismara ef263042d7
Using Logo Scrapy in Readme.md 
Logo scrapy used in readme.md made looks nicer
 2021-10-03 13:26:20 +07:00
Adrián Chaves d91d82b506
Make Scrapy SFW again 2021-10-01 16:31:29 +02:00
Eugenio Lacuesta 890f884de4
Allow 'callback' key in keyword arguments for request callbacks (#5251) 2021-10-01 09:50:42 +02:00
Andrey Rahmatullin 7d557f898c
Merge pull request #5239 from maanijou/master 
Improve documentation for spider middlewares
 2021-09-27 15:42:31 +05:00
Andrey Rahmatullin 6249175317
Merge pull request #5244 from kamran890/spider-state-doc 
Document spider.state attribute (#5174)
 2021-09-27 15:41:58 +05:00
Adrián Chaves 3c57825b0f
Update docs/topics/spider-middleware.rst 2021-09-26 13:41:26 +02:00
Reza (Milad) Maanijou dfdb779756
Apply review comments 2021-09-26 12:45:44 +03:30
Reza (Milad) Maanijou 1829dd774c
Update spider-middleware.rst 2021-09-25 20:22:09 +03:30
kamran890 e5998fb846 Document spider.state attribute (#5174) 2021-09-22 03:15:12 +05:00
maanijou ac9175964d Improve documentation for spider middlewares 2021-09-12 17:59:20 +02:00
Andrey Rahmatullin 8284de5e76
Fix/silence the Pylint messages added in 2.10 (#5235) 2021-08-24 12:15:29 +02:00
D R Siddhartha 43ea21e830
Feed post-processing plugin support (#5190) 2021-08-24 11:48:01 +02:00
Matsievskiy S.V 3f635eb683
Extract domain from genspider URL (#4439) 2021-08-24 11:05:50 +02:00
Andrey Rahmatullin ada539a63a
Merge pull request #4676 from Gallaecio/startproject-existing-folder 
Fix startproject failing for existing folders
 2021-08-24 12:18:49 +05:00
Andrey Rakhmatullin cd17c829cf Revert "Move documentation about avoiding bans into a topic of its own (#4039)" 
This reverts commit 2d2581c68f.
 2021-08-23 19:55:35 +05:00
Andrey Rahmatullin 731f2d3085
Merge pull request #4178 from elacuesta/remove_spider_make_requests_from_url 2021-08-20 16:16:25 +05:00
Andrey Rahmatullin bdf8355ce1
Merge pull request #5231 from umairnsr87/umairnsr87/issues/5226 
updated documentation for python version for reppy
 2021-08-18 21:08:32 +05:00
databender 572d347b3b warning view updated 2021-08-18 16:17:52 +05:30
Adrián Chaves 2d2581c68f
Move documentation about avoiding bans into a topic of its own (#4039) 2021-08-18 12:46:42 +02:00
databender d623ed15d1 indentation updated 2021-08-18 14:51:03 +05:30
databender bcf38a6719 added upstream issue for not supported python version 2021-08-18 14:48:47 +05:30
Andrey Rahmatullin a7bd6fa229
Merge pull request #3669 from Gallaecio/spider-name-collision 2021-08-17 19:42:53 +05:00
databender 79bad0cf82 updated suggested changes after review 2021-08-16 18:10:23 +05:30
databender ebddb77a33 updated suggested changes after review 2021-08-16 18:08:26 +05:30
umair ansari 013ac90f61
Update docs/topics/downloader-middleware.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-08-16 18:00:06 +05:30
databender cc1cb2de0c updated suggested changes 2021-08-16 17:21:47 +05:30
databender 1a8b98843a updated documentation for python version for reppy 2021-08-16 17:00:05 +05:30
databender 8bbaea9892 updated documentation for python version for reppy 2021-08-16 16:57:43 +05:30
Andrey Rahmatullin 2814e0e197
Disable builtin middlewares in spider middleware tests. (#5229) 2021-08-16 13:22:01 +02:00
Andrey Rahmatullin 47a281d231
Merge pull request #5224 from aaron-tan/jobdir-doc 2021-08-16 12:52:02 +05:00
Andrey Rahmatullin 0915be7300
Merge pull request #5228 from wRAR/test-reqs 
Rename tests/requirements-py3.txt to tests/requirements.txt.
 2021-08-11 21:04:10 +05:00
Andrey Rakhmatullin b63369c148 Rename tests/requirements-py3.txt to tests/requirements.txt. 2021-08-11 20:02:45 +05:00
Andrey Rahmatullin 0590c3756c
Merge pull request #5221 from wRAR/3.10-tests 
Fixing tests for upcoming 3.10.
 2021-08-11 14:19:51 +05:00
Andrey Rakhmatullin 74cee38a4e Don't run the asyncio tests on 3.9. 2021-08-11 14:19:08 +05:00
Andrey Rahmatullin 902fce0640
Merge pull request #5225 from divtiply/patch-3 
Allow comma-separated values in the rel attribute
 2021-08-11 10:58:28 +05:00
Andrey Rakhmatullin 983b89ad4f Fix SpiderLoaderTest on Python 3.10. 2021-08-11 10:39:23 +05:00
Michel Ace ce9d6c658b
Add more rel_has_nofollow tests 2021-08-10 22:21:51 +02:00
Michel Ace 295f0e2bf5
Make flake8 happy 2021-08-10 21:38:29 +02:00
Michel Ace 07d20a8ce4
Fix test_rel_has_nofollow test 2021-08-10 21:21:43 +02:00
Michel Ace 18b6f30a73
Add test for rel_has_nofollow 2021-08-10 21:13:50 +02:00
Michel Ace 1ba0f68483
Allow comma-separated values in the rel tag 
Comma-separated `rel` values are often seen in the wild, because Google allows it (see https://developers.google.com/search/docs/advanced/guidelines/qualify-outbound-links).
 2021-08-10 17:09:37 +02:00