RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,710 Commits 49 Branches 132 Tags 131 MiB
Commit Graph

32 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 7e07d48cc5 Small 3.7 and 3.8 cleanup. 2024-10-17 23:22:37 +05:00
Andrey Rakhmatullin c9095ef927 Remove --keep-runtime-typing from pyupgrade. 2024-10-17 21:26:02 +05:00
Andrey Rakhmatullin 2cb1e10c76 Make shutdown tests more robust. 2023-10-17 23:56:54 +04:00
Andrey Rakhmatullin 029a56384d Increase the timeout. 2023-10-09 00:16:41 +04:00
Andrey Rakhmatullin 4b5fb9b5a6 Add a test for SIGTERM handling. 2023-09-24 22:52:17 +04:00
Andrey Rakhmatullin 6428356584 Use twisted.python.log instead of failure.printTraceback(). 2023-09-13 18:20:42 +04:00
Andrey Rakhmatullin be0e33af92 Don't rely on unhandled exception logging in the expected to fail scripts. 2023-09-13 16:59:54 +04:00
Andrey Rakhmatullin 77f39be407 Add integration tests for modifying settings via args. 2023-09-12 20:16:26 +04:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Mark Mayo f9a29f03d9
Address some issues reported by Pylint (#5677) 
Co-authored-by: Felipe Boff Nunes <51033921+felipeboffnunes@users.noreply.github.com>
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
 2022-11-27 11:00:13 +01:00
Adrián Chaves 960a7f68f6
Verify that the installed asyncio event loop matches ASYNCIO_EVENT_LOOP (#5529) 
Co-authored-by: Laerte Pereira <5853172+Laerte@users.noreply.github.com>
 2022-10-15 11:27:00 +02:00
Andrey Rakhmatullin eeb199adda Fix flake8 issues in previously ignored files. 2022-10-10 11:10:59 +06:00
Adrián Chaves 00ccb02f1f Merge remote-tracking branch 'upstream/2.6' 2022-07-25 14:13:22 +02:00
Adrián Chaves 4ef71829b2
If TWISTED_REACTOR is None, reuse any pre-installed reactor (#5528) 2022-06-17 10:37:27 +02:00
AngelikiBoura b2afcbfe2b
Fix typos in three files for Flake8 check (#5487) 
* Fix typos in extensions files

Made some fixes in files memusage.py and statsmailer.py
in order to pass the flake8 check.

* Fix typos in twisted_reactor_custom_settings_same.py

A small change was needed in order for flake8 check
to pass.
 2022-05-05 15:49:52 +02:00
Adrián Chaves 96fc4dae15 CrawlerProcess: test a multi-spider scenario 2022-03-02 17:30:05 +01:00
Andrey Rakhmatullin 3577c00569 Merge remote-tracking branch 'origin/master' into crawler-process-reactor-later 2022-01-03 13:59:26 +05:00
Andrey Rahmatullin c5ab58056c
Set WindowsSelectorEventLoopPolicy on Windows (#5315) 2021-12-31 20:38:10 +01:00
Andrey Rakhmatullin d4565318c7 Fix a reactor test on Windows. 2021-12-23 17:40:31 +05:00
Andrey Rakhmatullin ebcafdf4a9 Add tests for TWISTED_REACTOR in custom_settings. 2021-12-23 16:35:26 +05:00
Pascal Corpet 70dddfe2b2 Typing: switch to a newer version of MyPy to check types 2021-07-21 17:10:10 +02:00
Eugenio Lacuesta b20cfef1e5
Remove unnecessary line from test 2020-11-09 13:58:52 -03:00
Eugenio Lacuesta 7e98a76ac4
Use deferred_from_coro in asyncio test 2020-11-09 12:17:15 -03:00
Eugenio Lacuesta 5b5478ae9d
Call asyncio.get_event_loop when installing the asyncio reactor 2020-11-05 14:01:34 -03:00
Eugenio Lacuesta 868826b346
CachingHostnameResolver tests 2020-10-09 10:38:07 -03:00
sakshamb2113 42383cc267
Add a setting to customize the asyncio event loop (#4414) 2020-08-19 09:18:14 +02:00
elacuesta 35dafef7f1
Specify Twisted reactor (TWISTED_REACTOR setting) (#4294) 
* Add the ability to install a specific reactor

* Add docs for the TWISTED_REACTOR setting

* Add tests for the TWISTED_REACTOR setting

* Update asyncio reactor test

* Ignore W503 globally

W503 is not PEP8-compliant:
c59c4376ad

* Line length adjustment

* Adjust asyncio reactor tests

* Merge ASYNCIO_ENABLED and TWISTED_REACTOR settings

* More docs about TWISTED_REACTOR

* Fix asyncio reactor test

* Docs: fix title

* Reword docs

* Check the TWISTED_REACTOR setting outside of the installing function

* Remove unrelated change

* Update scrapy/utils/log.py

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/settings.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/settings.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-02-06 22:42:34 +05:00
Eugenio Lacuesta 90e3bd8715
[test] Name resolvers 2020-01-16 20:32:40 -03:00
Andrey Rakhmatullin dc1ee09481 Rename ASYNCIO_ENABLED to ASYNCIO_REACTOR, change the logic accordingly. 2019-12-27 21:56:28 +05:00
Andrey Rakhmatullin bfb78b8dea Add CrawlerProcess tests for ASYNCIO_ENABLED. 2019-12-13 18:12:07 +05:00
Andrey Rahmatullin 02cdc53fb8
Add a test for a CrawlerProcess script. (#4218) 
* Add a test for a CrawlerProcess script.

* Add tests/CrawlerProcess to collect_ignore.

* Remove an extra line.

* Fix/improve conftest.py.
 2019-12-13 18:04:05 +05:00