RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,618 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

8618 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 67cff0e8a9 Silence pylint "naked raise" error. 2021-02-10 22:44:14 +05:00
Andrey Rakhmatullin 1cd953e1b0 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-02-09 13:28:36 +05:00
Adrián Chaves de82ca8547
Merge pull request #4982 from wRAR/refactor-spidermiddlewaremanager 
Refactor SpiderMiddlewareManager.scrape_response.
 2021-02-08 21:27:21 +01:00
Andrey Rakhmatullin 1e9b52c3e0 Refactor SpiderMiddlewareManager.scrape_response. 2021-02-08 22:02:03 +05:00
Andrey Rakhmatullin 5cf403295d Remove a duplicate definition. 2021-02-05 19:40:14 +05:00
Andrey Rakhmatullin 58f8481301 Update docs. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2152a2a508 Add main infrastructure for async callbacks. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 2a1e9359ca Add parallel_async. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 92f2c9e308 Move spider middlewares to process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d66d52d3ed Add process_iterable_helper. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin d658552f23 Add only_not_asyncio. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 7e9f498e00 Add MutableAsyncChain. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin acff1eb496 Add aiter_errback. 2021-02-05 16:21:56 +05:00
Andrey Rakhmatullin 76abcedaf4 Add as_async_generator. 2021-02-05 16:21:56 +05:00
Mikhail Korobov 904a50138b
Merge pull request #4973 from Gallaecio/zyte 
Scrapinghub → Zyte
 2021-02-03 01:10:52 +05:00
Adrián Chaves f30f53b3cc Scrapinghub → Zyte 2021-02-02 15:03:20 +01:00
Mikhail Korobov 28262d4b24
Merge pull request #4956 from Gallaecio/readme-contributors 
Mention contributors in the README
 2021-01-14 19:58:52 +05:00
Adrián Chaves 6e7ae789f9
Reuse the text from https://scrapy.org/ 2021-01-14 11:59:38 +01:00
M Ikram Ullah Khan 0a1e2fefab
Docs: fix typo in news.rst (#4942) 2021-01-04 10:30:23 -03:00
Eugenio Lacuesta 80db569aea
Migrate CI to GitHub actions (#4924) 2021-01-01 19:13:39 -03:00
Kader DJEHAF 44a7ab5bf0
Fix warning: Expected type 'bool', got 'int' instead (#4940) 
* Fix warning: Expected type 'bool', got 'int' instead

* Update defer.py
 2020-12-30 11:22:27 -03:00
Adrián Chaves 39bc9a4d28
Merge pull request #4814 from dswij/tox-pip20.2 
add pip 20.2 test for tox
 2020-12-24 15:31:42 +01:00
Adrián Chaves 24e1b350ae
Merge pull request #4936 from timgates42/bugfix_typo_without 
docs: fix simple typo, wihout -> without
 2020-12-22 13:57:03 +01:00
Tim Gates 6dccf82eaa
docs: fix simple typo, wihout -> without 
There is a small typo in scrapy/http/request/form.py.

Should read `without` rather than `wihout`.
 2020-12-22 07:49:13 +11:00
Eugenio Lacuesta b83a1a6fbf Disable test under pypy 2020-12-21 17:21:08 +01:00
Adrián Chaves 45eb099ed1 Maybe it’s about having a newer libssl 2020-12-21 15:37:02 +01:00
Adrián Chaves 1c1255a75d Use sudo for apt-get 2020-12-21 14:41:02 +01:00
Adrián Chaves 798a818caf Move apt-get command from Tox to Travis CI 2020-12-21 13:35:40 +01:00
Adrián Chaves 0dff5781bc Blind attempt to fix the build of the cryptography-provided OpenSSL 2020-12-21 11:13:14 +01:00
Adrián Chaves 0567fdc56e Merge remote-tracking branch 'upstream/master' into tox-pip20.2 2020-12-21 11:02:05 +01:00
Adrián Chaves d0af008608
Merge pull request #4878 from elacuesta/response-protocol-attribute 
Response.protocol attribute
 2020-12-16 15:27:40 +01:00
Mikhail Korobov 212163e199
Merge pull request #4912 from elacuesta/remove_deprecated_pickled_settings_to_override 
Remove deprecated SCRAPY_PICKLED_SETTINGS_TO_OVERRIDE
 2020-12-04 01:36:47 +05:00
gunadhya db10aaf9eb
Update links in installation guide (#4899) 2020-12-03 10:56:36 +01:00
Eugenio Lacuesta 6091f3cc03
Remove unused pickle import 2020-12-01 10:26:21 -03:00
Eugenio Lacuesta a80bafe5cd
Remove deprecated SCRAPY_PICKLED_SETTINGS_TO_OVERRIDE 2020-11-30 19:03:13 -03:00
Kader DJEHAF 7fec9f991f
[Cleaned] PEP 8: E251 unexpected spaces around keyword / parameter equals (#4911) 
[Cleaned] PEP 8: E251 unexpected spaces around keyword / parameter equals
 2020-11-30 17:47:28 -03:00
Adrián Chaves 487e19528b
Merge pull request #4901 from elacuesta/remove_deprecated_stuff 
Remove deprecated stuff
 2020-11-30 14:41:40 +01:00
Mikhail Korobov aed1707f24
Merge pull request #4909 from etimoz/exporters-doc-super-fix 
Removed wrong super argument in overriding serialize_fields code example
 2020-11-29 23:09:20 +05:00
etimoz 95d39d5cb4 removed wrong super argument in overriding serialize_fields code example 2020-11-29 13:24:04 +01:00
Andrey Rahmatullin 440e45d172
Merge pull request #4900 from elacuesta/deprecate_utils_py36 
Deprecate scrapy.utils.py36 module
 2020-11-24 23:59:00 +05:00
Adrián Chaves f6879c681e SCRAPY_PINNED → _SCRAPY_PINNED 2020-11-23 23:53:03 +01:00
Adrián Chaves 0dc3e6350c Add a test to check the Twisted version in pinned environments 2020-11-23 22:10:45 +01:00
Eugenio Lacuesta fe8bb6bd90
Fix import 2020-11-23 16:51:04 -03:00
Eugenio Lacuesta 18b05af877
Remove tests/test_utils_http.py 2020-11-23 16:18:58 -03:00
Eugenio Lacuesta 0a93df9efd
Move collect_asyncgen to utils.asyncgen 2020-11-23 16:16:18 -03:00
Eugenio Lacuesta 462014bc57
Scheduler: remove support for deprecated queuelib.PriorityQueue 2020-11-23 15:51:59 -03:00
Eugenio Lacuesta 51ca4d0138
Remove deprecated scrapy.utils.gz.is_gzipped function 2020-11-23 15:47:08 -03:00
Eugenio Lacuesta 4075e1eadd
Remove deprecated modules (utils.http/markup/multipart) 2020-11-23 15:07:56 -03:00
Eugenio Lacuesta 075ab156b0
Deprecate scrapy.utils.py36 module 2020-11-23 11:59:59 -03:00
Adrián Chaves e17c890be1
Merge pull request #4898 from scrapy/2.4 
Scrapy 2.4.1
 2020-11-19 20:31:57 +01:00