RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,629 Commits 49 Branches 132 Tags 149 MiB
Commit Graph

8629 Commits

Author SHA1 Message Date
Andrey Rakhmatullin c51ec1ae1c Drop process_iterable_helper, add _process_iterable_universal. 2021-03-23 22:47:30 +05:00
Andrey Rakhmatullin f9a5385146 Revert "Move spider middlewares to process_iterable_helper." 
This reverts commit 92f2c9e308.
 2021-02-19 21:19:21 +05:00
Andrey Rakhmatullin fa8de2d396 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-18 23:06:06 +05:00
Andrey Rakhmatullin 40eab1d473 Drop a duplicate import. 2021-02-18 19:56:12 +05:00
Andrey Rahmatullin fa3ebb1ced
Merge pull request #4987 from wRAR/asyncdef-tests 
Making sure we can run async def tests
 2021-02-18 18:34:25 +05:00
Andrey Rakhmatullin 58706c6598 Merge branch 'asyncdef-tests' into asyncio-parse-asyncgen-proper-rebased 2021-02-18 17:20:25 +05:00
Andrey Rakhmatullin 49af7c4c8b Drop pytest-twisted, use Scrapy code to install the reactor. 2021-02-18 17:10:30 +05:00
Andrey Rakhmatullin abbbfbbb38 Add tests for deferred_f_from_coro_f. 2021-02-12 22:41:16 +05:00
Andrey Rakhmatullin a7557f2a49 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-11 16:09:14 +05:00
Adrián Chaves ea92b49aaa
Merge pull request #4986 from wRAR/uvloop-py36 
Skip uvloop 0.15.0+ on py36.
 2021-02-11 12:07:28 +01:00
Andrey Rakhmatullin 54fd371481 Skip uvloop 0.15.0+ on py36. 2021-02-11 14:24:11 +05:00
Andrey Rakhmatullin 67cff0e8a9 Silence pylint "naked raise" error. 2021-02-10 22:44:14 +05:00
Andrey Rakhmatullin 1cd953e1b0 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-09 13:28:36 +05:00
Adrián Chaves de82ca8547
Merge pull request #4982 from wRAR/refactor-spidermiddlewaremanager 
Refactor SpiderMiddlewareManager.scrape_response.
 2021-02-08 21:27:21 +01:00
Andrey Rakhmatullin 1e9b52c3e0 Refactor SpiderMiddlewareManager.scrape_response. 2021-02-08 22:02:03 +05:00
Andrey Rakhmatullin 5cf403295d Remove a duplicate definition. 2021-02-05 19:40:14 +05:00
Andrey Rakhmatullin 58f8481301 Update docs. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2152a2a508 Add main infrastructure for async callbacks. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2a1e9359ca Add parallel_async. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 92f2c9e308 Move spider middlewares to process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d66d52d3ed Add process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d658552f23 Add only_not_asyncio. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 7e9f498e00 Add MutableAsyncChain. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin acff1eb496 Add aiter_errback. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 76abcedaf4 Add as_async_generator. 2021-02-05 16:21:56 +05:00
Mikhail Korobov 904a50138b
Merge pull request #4973 from Gallaecio/zyte 
Scrapinghub → Zyte
 2021-02-03 01:10:52 +05:00
Adrián Chaves f30f53b3cc Scrapinghub → Zyte 2021-02-02 15:03:20 +01:00
Mikhail Korobov 28262d4b24
Merge pull request #4956 from Gallaecio/readme-contributors 
Mention contributors in the README
 2021-01-14 19:58:52 +05:00
Adrián Chaves 6e7ae789f9
Reuse the text from https://scrapy.org/ 2021-01-14 11:59:38 +01:00
M Ikram Ullah Khan 0a1e2fefab
Docs: fix typo in news.rst (#4942) 2021-01-04 10:30:23 -03:00
Eugenio Lacuesta 80db569aea
Migrate CI to GitHub actions (#4924) 2021-01-01 19:13:39 -03:00
Kader DJEHAF 44a7ab5bf0
Fix warning: Expected type 'bool', got 'int' instead (#4940) 
* Fix warning: Expected type 'bool', got 'int' instead

* Update defer.py
 2020-12-30 11:22:27 -03:00
Adrián Chaves 39bc9a4d28
Merge pull request #4814 from dswij/tox-pip20.2 
add pip 20.2 test for tox
 2020-12-24 15:31:42 +01:00
Adrián Chaves 24e1b350ae
Merge pull request #4936 from timgates42/bugfix_typo_without 
docs: fix simple typo, wihout -> without
 2020-12-22 13:57:03 +01:00
Tim Gates 6dccf82eaa
docs: fix simple typo, wihout -> without 
There is a small typo in scrapy/http/request/form.py.

Should read `without` rather than `wihout`.
 2020-12-22 07:49:13 +11:00
Eugenio Lacuesta b83a1a6fbf Disable test under pypy 2020-12-21 17:21:08 +01:00
Adrián Chaves 45eb099ed1 Maybe it’s about having a newer libssl 2020-12-21 15:37:02 +01:00
Adrián Chaves 1c1255a75d Use sudo for apt-get 2020-12-21 14:41:02 +01:00
Adrián Chaves 798a818caf Move apt-get command from Tox to Travis CI 2020-12-21 13:35:40 +01:00
Adrián Chaves 0dff5781bc Blind attempt to fix the build of the cryptography-provided OpenSSL 2020-12-21 11:13:14 +01:00
Adrián Chaves 0567fdc56e Merge remote-tracking branch 'upstream/master' into tox-pip20.2 2020-12-21 11:02:05 +01:00
Adrián Chaves d0af008608
Merge pull request #4878 from elacuesta/response-protocol-attribute 
Response.protocol attribute
 2020-12-16 15:27:40 +01:00
Mikhail Korobov 212163e199
Merge pull request #4912 from elacuesta/remove_deprecated_pickled_settings_to_override 
Remove deprecated SCRAPY_PICKLED_SETTINGS_TO_OVERRIDE
 2020-12-04 01:36:47 +05:00
gunadhya db10aaf9eb
Update links in installation guide (#4899) 2020-12-03 10:56:36 +01:00
Eugenio Lacuesta 6091f3cc03
Remove unused pickle import 2020-12-01 10:26:21 -03:00
Eugenio Lacuesta a80bafe5cd
Remove deprecated SCRAPY_PICKLED_SETTINGS_TO_OVERRIDE 2020-11-30 19:03:13 -03:00
Kader DJEHAF 7fec9f991f
[Cleaned] PEP 8: E251 unexpected spaces around keyword / parameter equals (#4911) 
[Cleaned] PEP 8: E251 unexpected spaces around keyword / parameter equals
 2020-11-30 17:47:28 -03:00
Adrián Chaves 487e19528b
Merge pull request #4901 from elacuesta/remove_deprecated_stuff 
Remove deprecated stuff
 2020-11-30 14:41:40 +01:00
Mikhail Korobov aed1707f24
Merge pull request #4909 from etimoz/exporters-doc-super-fix 
Removed wrong super argument in overriding serialize_fields code example
 2020-11-29 23:09:20 +05:00
etimoz 95d39d5cb4 removed wrong super argument in overriding serialize_fields code example 2020-11-29 13:24:04 +01:00