scrapy

Commit Graph

Author	SHA1	Message	Date
Kevin Lloyd Bernal	1fc91bb462	new `allow_offsite` parameter in OffsiteMiddleware (#6151 ) * new 'allow_offsite' parameter in OffsiteMiddleware * document deprecated dont_filter flag in OffsiteMiddleware * avoid deprecating dont_filter in OffsiteMiddleware * Copy the code to the downloader mw. * Add tests for allow_offsite in the downloader mw. * Mark allow_offsite with reqmeta. --------- Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>	2025-01-08 21:28:51 +05:00
Laerte Pereira	26a16f2c43	Fix tests	2024-02-28 16:36:19 -03:00
Jalil SA	a5c1ef8276	sort imports with isort	2023-01-25 14:25:15 -06:00
Emmanuel Rondan	e211ec0aa2	adding black formatter to all the code	2022-11-29 11:30:46 -03:00
Eugenio Lacuesta	bf56517abf	Flake8: Remove E128 (tests, part 2)	2020-05-06 19:18:29 -03:00
Lukas Anzinger	ccc4d88779	Ignore a domain in allowed_domains with port and issue a warning (#4413 )	2020-03-12 20:42:14 +01:00
Lukas Anzinger	9d9dea0d69	Fix handling of None in allowed_domains. Nones in allowed_domains ought to be ignored and there are also tests for that scenario. This commit fixes the handling of None and also the accompanying tests which are now executed again.	2020-03-07 20:34:18 +01:00
Marc Hernandez Cabot	a5de2c64e6	fix W291, W292, W293 (whitespaces)	2019-12-18 16:33:46 +01:00
Eugenio Lacuesta	714e0d3ac9	Merge remote-tracking branch 'upstream/master' into remove-six-code	2019-11-20 12:41:38 -03:00
Adrián Chaves	f261cf65e9	Add missing blank lines between functions and classes Also fixed 2 unrelated Flake8 issues	2019-11-18 17:16:09 +01:00
Eugenio Lacuesta	5ab0f189ce	Remove six.moves occurrences from tests	2019-11-03 00:01:09 -03:00
Jana Cavojska	454d5e5733	checking for subclass of URLWarning instead of checking error message text when URL in allowed_domains	2017-11-26 20:07:04 +01:00
Jana Cavojska	8ec3b476b0	triggering a warning when user puts URL in allowed_domains now covered by test	2017-11-26 16:36:15 +01:00
Chomba Ng'ang'a	4ca61a2051	Update deprecated test aliases - change ``failIf`` to ``assertFalse`` - change ``asertEquals`` to ``assertEqual`` - change ``assert_`` to ``assertTrue`` https://docs.python.org/2/library/unittest.html#deprecated-aliases	2017-08-09 12:52:10 +02:00
Νικόλαος-Διγενής Καραγιάννης	1cffa99e0d	tests+doc for subdomains in offsite middleware	2016-01-26 12:49:43 +02:00
Julia Medina	d3f576a816	Move scrapy/spider.py to scrapy/spiders/__init__.py	2015-05-09 04:20:09 -03:00
Julia Medina	180272c092	Move scrapy/contrib/spidermiddleware to scrapy/spidermiddlewares	2015-04-29 21:26:35 -03:00
Julia Medina	870438e5f4	Update tests utils, fixing get_crawler and removing docrawl	2014-08-12 14:02:56 -03:00
Mikhail Korobov	83432184c8	PY3 enable more tests	2014-08-01 16:53:18 +06:00
Daniel Graña	242c08553f	move tests under root dir	2014-07-30 19:45:08 -03:00

20 Commits