RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,003 Commits 49 Branches 132 Tags 187 MiB
Commit Graph

1733 Commits

Author SHA1 Message Date
Akshay Sharma 0eb1de90e2 fix static checks 2022-06-20 21:10:06 -04:00
Akshay Sharma b9effde875 Refactored _guess_content_type 2022-06-20 20:15:33 -04:00
Akshay Sharma b2181d76d5 Merge branch 'master' of https://github.com/AKSHAYSHARMAJS/scrapy into integrate-mime 2022-05-08 17:48:57 -04:00
Akshay Sharma 66e3b40043 Add more tests 2021-09-20 22:44:13 -04:00
Akshay Sharma c81605abe6 Move tests to post_xtractmime 2021-08-29 13:55:02 -04:00
Andrey Rahmatullin 8284de5e76
Fix/silence the Pylint messages added in 2.10 (#5235) 2021-08-24 12:15:29 +02:00
D R Siddhartha 43ea21e830
Feed post-processing plugin support (#5190) 2021-08-24 11:48:01 +02:00
Matsievskiy S.V 3f635eb683
Extract domain from genspider URL (#4439) 2021-08-24 11:05:50 +02:00
Andrey Rahmatullin ada539a63a
Merge pull request #4676 from Gallaecio/startproject-existing-folder 
Fix startproject failing for existing folders
 2021-08-24 12:18:49 +05:00
Andrey Rahmatullin 731f2d3085
Merge pull request #4178 from elacuesta/remove_spider_make_requests_from_url 2021-08-20 16:16:25 +05:00
Andrey Rahmatullin 2814e0e197
Disable builtin middlewares in spider middleware tests. (#5229) 2021-08-16 13:22:01 +02:00
Akshay Sharma 22c33adb41
Merge branch 'master' into integrate-mime 2021-08-13 01:07:08 +05:30
Akshay Sharma 3475f0049b Change _guess_content_type 2021-08-13 01:05:37 +05:30
Andrey Rakhmatullin b63369c148 Rename tests/requirements-py3.txt to tests/requirements.txt. 2021-08-11 20:02:45 +05:00
Akshay Sharma 4169fd184f More tests 2021-08-11 17:14:38 +05:30
Andrey Rahmatullin 0590c3756c
Merge pull request #5221 from wRAR/3.10-tests 
Fixing tests for upcoming 3.10.
 2021-08-11 14:19:51 +05:00
Andrey Rakhmatullin 983b89ad4f Fix SpiderLoaderTest on Python 3.10. 2021-08-11 10:39:23 +05:00
Michel Ace ce9d6c658b
Add more rel_has_nofollow tests 2021-08-10 22:21:51 +02:00
Michel Ace 295f0e2bf5
Make flake8 happy 2021-08-10 21:38:29 +02:00
Michel Ace 07d20a8ce4
Fix test_rel_has_nofollow test 2021-08-10 21:21:43 +02:00
Michel Ace 18b6f30a73
Add test for rel_has_nofollow 2021-08-10 21:13:50 +02:00
Akshay Sharma 9980a3d134 ADD xtractmime to pinnedtests 2021-08-09 17:28:12 +05:30
Akshay Sharma 2ba2f49ac5 Add more tests 2021-08-08 19:35:48 +05:30
Mannan2812 94baa4b272
Fix FileFeedStoragePreFeedOptionsTest fails in CI/CD pipeline (#5198) 2021-08-05 21:23:11 +02:00
Andrey Rakhmatullin 93bf1ae7e3 Fix tests for the 3.10 TypeError message change. 2021-08-03 20:16:59 +05:00
Akshay Sharma f573d11ec2 Logic fix 2021-08-01 14:40:39 +05:30
Akshay Sharma 180b457f77 Format change 2021-07-31 17:38:28 +05:30
Andrey Rahmatullin b406affd5b
Merge pull request #5210 from laggardkernel/feature/aws-session-token 2021-07-30 15:15:55 +05:00
Eugenio Lacuesta cc89f6be38
Response.attributes (#5218) 2021-07-29 22:12:44 +02:00
laggardkernel 7e4321f201 Add support for temporary security credential in AWS auth 2021-07-27 19:39:59 +08:00
Akshay Sharma 1512ed22b2 Pre n post xtractmime tests 2021-07-25 19:42:27 +05:30
Pascal Corpet 70dddfe2b2 Typing: switch to a newer version of MyPy to check types 2021-07-21 17:10:10 +02:00
Akshay Sharma 763ec07b7b Deprecation Warnings 2021-07-18 03:56:12 +05:30
Akshay Sharma 211fc62b69 Remove commented func 2021-07-15 20:06:56 +05:30
Akshay Sharma 265301215a Add extract_mime 2021-07-15 20:03:11 +05:30
Türkalp Burak KAYRANCIOĞLU fcc6becc58
S3FeedStorage: allow custom endpoint (#4998) 
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
 2021-07-14 10:00:43 +02:00
D R Siddhartha 4ddc9d6b55
Feeds: Item Filters (#5178) 2021-07-13 17:22:29 +02:00
Iván de Prado eca641aa3d
Update tests/test_engine.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-07-08 12:40:20 +01:00
Iván de Prado Alonso f35970778b Test case for raising CloseSpider on spider idle signal handler 2021-06-29 13:21:38 +01:00
Andrey Rahmatullin 016c7e92d1
Merge pull request #3579 from maramsumanth/duplicate_key 
[MRG+1] Issue #2919: Fix FormRequest.formdata with GET method duplicates same key in query string
 2021-06-15 13:36:53 +05:00
ajaymittur28 7d653288e3 Update unittest 2021-06-14 21:39:18 +05:30
ajaymittur28 857e0e225a Merge branch 'master' into http-proxy-schema 2021-06-14 21:18:49 +05:30
Andrey Rahmatullin 66e2004239
Fix a flake8 problem 2021-06-14 18:58:35 +05:00
Andrey Rahmatullin 16be658a31
Merge branch 'master' into duplicate_key 2021-06-14 18:45:33 +05:00
Vostretsov Nikita 23cfdb058e
Reducing amount of warnings during test run (#5162) 
* put flake8 options into separate file to remove pytest warnings

* remove ResourceLeaked warning in pypy

* suppress warnings from twisted

* ignore deprecation warnings here

* ignore deprecation warning in tests of deprecated methods

* ignore deprecation warnings here

* update test classes

* don`t use deprecated method call

* ignore deprecation warnings here

* proper warning class

* more selective ignoring

* Revert "don`t use deprecated method call"

This reverts commit 59216ab560.
 2021-05-28 14:45:06 +05:00
Adrián Chaves c5b1ee8101
Make Twisted[http2] installation optional (#5113) 
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
 2021-05-11 09:04:53 +02:00
Eugenio Lacuesta cec36a9284
Refactor request to/from dict (#5130) 2021-05-10 18:00:08 +02:00
Eugenio Lacuesta 02ae1deaf4
Deprecate unused squeues (#5117) 2021-04-27 09:41:44 -03:00
Eugenio Lacuesta ddea6b7bfa
Scheduler: minimal interface, API docs (#3559) 2021-04-26 16:16:14 -03:00
Eugenio Lacuesta 6837919798
Add peek method to queues (#5112) 2021-04-26 14:55:02 -03:00