RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,208 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

49 Commits

Author SHA1 Message Date
Adrian 9559cbee1e
Solve timing issues with HTTP cache tests? (#7692) 2026-06-29 17:21:20 +05:00
Adrian dd4549e6f9
Improve test coverage for downloader middlewares (#7655) 
* Improve test coverage for downloader middlewares

* Improve coverage further
 2026-06-25 01:48:28 +05:00
Andrey Rakhmatullin 3a36955261
Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
Andrey Rakhmatullin 9f4651151d
Deprecate ScrapyCommand.set_crawler(), remove/fix some dead code. (#7276) 2026-03-24 10:36:48 +01:00
Andrey Rakhmatullin 2ad81a0ef8
Deprecate the spider arg of downloader middleware methods. (#7037) 
* Deprecate the spider arg of downloader middleware methods.

* Add a test for deprecated spider args in downloader mws.
 2025-09-02 21:51:47 +05:00
Andrey Rakhmatullin b68330811b
Deprecate spider arguments of StatsCollector methods (#7011) 2025-08-20 09:52:22 +02:00
Felipe Benevolo 128cb551eb
refactor tests/test_downloadermiddleware_httpcache.py (#6769) 2025-05-12 13:04:34 +02:00
Andrey Rakhmatullin 1843a4f753
Converting tests to plain asserts, part 3. (#6700) 2025-03-06 20:50:14 +01:00
Andrey Rakhmatullin 8d92c28a16
Switch to pytest.raises(). (#6680) 
* Switch to pytest.raises().

* Add matches= to broad pytest.raises().

* Adjust the test_nonserializable_object() regex for Python <= 3.11.

* Adjust the test_nonserializable_object() regex for PyPy.

* Adjust other test exception regexes for PyPy.

* Cleanup.
 2025-02-27 23:13:04 +05:00
Andrey Rakhmatullin a898331d14
Preparations for switching to direct pytest. (#6678) 2025-02-16 20:28:58 +01:00
Andrey Rakhmatullin df688910e0 Remove a duplicate test. 2025-02-02 18:48:26 +05:00
Andrey Rakhmatullin 6ae5b92671
Drop the remaining unittest.main() blocks. (#6602) 2025-01-02 12:45:04 +01:00
Adrián Chaves b07d3f85a3 Upgrade CI tools 2023-02-02 06:37:40 +01:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Mark Mayo f9a29f03d9
Address some issues reported by Pylint (#5677) 
Co-authored-by: Felipe Boff Nunes <51033921+felipeboffnunes@users.noreply.github.com>
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
 2022-11-27 11:00:13 +01:00
Kromitvs 9e265a2c1f
Mind body to choose response class in cache, FTP and HTTP/1.0 (#4873) 2022-06-16 20:52:19 +02:00
Ammar Najjar 58ca8bbf6d Use f-strings (#4307) 2020-08-26 07:11:05 +02:00
Marc Hernández 336f19f5cc
Change super syntax (#4707) 2020-08-04 20:42:01 +02:00
Marc Hernández 91bbc70bc1
fix E30X flake8 (#4355) 2020-02-21 06:05:31 +01:00
Adrián Chaves a2bf340bab Remove unused imports 2019-11-21 14:21:17 +01:00
Adrián Chaves f261cf65e9 Add missing blank lines between functions and classes 
Also fixed 2 unrelated Flake8 issues
 2019-11-18 17:16:09 +01:00
Marc Hernandez Cabot 0e252f5a13 fix E711 and E713 2019-11-15 19:12:43 +01:00
Adrián Chaves 15c55d0c1d
Remove LevelDB support (#4112) 2019-10-31 10:47:29 +01:00
Roy 93e3dc1b82 [test_downloadermiddleware_httpcache.py] Cleaning text 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 16:12:03 +00:00
Roy b73d217de5 [test_downloadermiddleware_httpcache.py] Fixing pytest mark behaviour 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 12:55:54 +00:00
Roy c51fb959e2 [test_downloadermiddleware_httpcache] Fixing pytest skip behaviour 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 12:36:54 +00:00
Roy 4432136fff [test_downloadermiddleware_httpcache] Fixing pytest skip behaviour 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 12:22:21 +00:00
Roy 16bb3ac20d [test_downloadermiddleware_httpcache] Using skipif approach 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-28 11:24:09 +00:00
Roy 20ea912513 [test_downloadermiddleware_httpcache] Making xfails more informative 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:52:01 +00:00
Roy 70b2854590 [test_downloadermiddleware_httpcache] Making xfails more informative 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:51:26 +00:00
Roy b3df0a8415 [test_downloadermiddleware_httpcache] Adding xfails to impacted tests following hack fix 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:28:47 +00:00
Roy 11942c436c [test_downloadermiddleware_httpcache] Trying hack to handle systemerror whjen importing leveldb 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 18:02:13 +00:00
Roy deacd34c8d [test_downloadermiddleware_httpcache] Attempting to add xfail for leveldb related tests 
https://github.com/scrapy/scrapy/issues/4085
 2019-10-27 17:39:47 +00:00
Roy Healy 4068797558
Update test_downloadermiddleware_httpcache.py 
Adding xfail denoting that leveldb is not supported in 3.8
 2019-10-27 17:02:17 +00:00
Paul Tremberth 1fdc10684f HTTP Cache: treat 308 as 301 2017-07-26 19:01:48 +02:00
Konstantin Lopuhin 8330776c21 fix error reporting in test: we can fail in process_request too, so result should always be defined 2016-01-15 15:16:12 +03:00
Konstantin Lopuhin 87849780bc some py3 fixes for RFC2616Policy 2016-01-15 13:13:10 +03:00
Konstantin Lopuhin ddc91dda27 py3: fix _BaseTest in httpcache 2016-01-15 11:49:28 +03:00
Julia Medina d706310d8b Merge pull request #1151 from marven/cache-control 
[MRG+1] RFC2616 policy enhancements + tests
 2015-07-11 08:06:20 -03:00
Marven Sanchez bb3ebf13f9 Add tests for RFC2616 policy enhancements 
Add `scrapy/downloadermiddlewares/httpcache.py` to `tests/py3-ignores.txt
 2015-06-01 18:20:12 +08:00
Julia Medina d3f576a816 Move scrapy/spider.py to scrapy/spiders/__init__.py 2015-05-09 04:20:09 -03:00
Julia Medina 9a3e3ba505 Move scrapy/contrib remaining top-level files to scrapy/extensions 2015-04-29 21:27:19 -03:00
Julia Medina d7c444fefb Move scrapy/contrib/downloadermiddleware to scrapy/downloadermiddlewares 2015-04-29 21:24:30 -03:00
Pablo Hoffman 934584a355 Merge pull request #1020 from jojje/gzip_http_cache 
[MRG+1] add gzip compression to filesystem http cache backend
 2015-03-17 14:32:06 -03:00
nramirezuy c13e23641b httpcache dont_cache meta #19 #689 2015-03-16 11:50:04 -03:00
Jonas Tingeborn bd5d99a2d2 add gzip compression to filesystem http cache backend 2015-01-21 20:18:11 +01:00
Julia Medina 870438e5f4 Update tests utils, fixing get_crawler and removing docrawl 2014-08-12 14:02:56 -03:00
Daniel Graña 242c08553f move tests under root dir 2014-07-30 19:45:08 -03:00