RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,654 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

7654 Commits

Author SHA1 Message Date
Eugenio Lacuesta 342bf3cd35
Explicit keyword arguments 2019-12-23 09:52:55 -03:00
Eugenio Lacuesta 931b7e68d3
Update FileDownloadHandler test 2019-12-23 09:50:28 -03:00
Eugenio Lacuesta 310cc081d0
Merge branch 'master' into from_crawler_downloader_handlers 2019-12-21 17:24:45 -03:00
Mikhail Korobov 5577d4d2be
Merge pull request #3869 from elacuesta/detect_return_in_generator_callbacks 
[MRG+1][Py3] Check for 'return' with arguments inside generators
 2019-12-20 22:46:46 +05:00
Mikhail Korobov fb3fb17c28
Merge pull request #4045 from Gallaecio/documentation-coverage 
Deprecate FilteringLinkExtractor direct usage; add API docs for scrapy.linkextractors
 2019-12-20 02:01:45 +05:00
Mikhail Korobov 6a32a36e09
Merge pull request #4143 from Gallaecio/undent-rst-doctests 
Do not indent doctests from the documentation unnecessarily
 2019-12-20 01:59:07 +05:00
Mikhail Korobov 1213c0e5f0
Merge pull request #4247 from noviluni/fix_typos 
fix typos
 2019-12-20 01:56:36 +05:00
Mikhail Korobov a611a7c0f2
Merge pull request #4249 from Gallaecio/docs-py37 
Use Python 3.7 to build the documentation
 2019-12-20 01:54:26 +05:00
Adrián Chaves b4a2d985a7
Merge branch 'master' into undent-rst-doctests 2019-12-19 12:39:17 +01:00
Adrián Chaves e22c0c27d9 Revert "Improve FilteringLinkExtractor.__new__" 
This reverts commit ee9881d270.
 2019-12-19 12:15:54 +01:00
Adrián Chaves f6bc1940a3 Use Python 3.7 to build the documentation 2019-12-19 12:06:15 +01:00
Andrey Rahmatullin c841a1f3e7
Merge pull request #4140 from Gallaecio/docs-py38 
Use the latest Python version to build the documentation
 2019-12-19 14:09:02 +05:00
Marc Hernandez Cabot 16b363de31 Merge branch 'master' into fix_typos 2019-12-19 10:02:21 +01:00
Marc Hernandez Cabot 23a67cec27 fix first letter capitalization for Raring and Scrapy 2019-12-19 09:57:17 +01:00
Mikhail Korobov 3bcb14fe4d
Merge pull request #4164 from Gallaecio/speed-up-tests-2 
Improve the performance of the DOWNLOAD_DELAY test
 2019-12-18 23:56:56 +05:00
Mikhail Korobov 8f7faaa63d
Merge pull request #4190 from Gallaecio/doctest 
Make developer-tools doctests pass
 2019-12-18 23:40:21 +05:00
Marc Hernandez Cabot c0d84f0962 fix typos 2019-12-18 19:39:21 +01:00
Mikhail Korobov c976230b5f
Merge pull request #4246 from noviluni/fix_W293_blank_line_contains_whitespace 
Fix W291, W292 and W293 flake8 issues (whitespaces)
 2019-12-18 23:39:04 +05:00
Marc Hernandez Cabot a5de2c64e6 fix W291, W292, W293 (whitespaces) 2019-12-18 16:33:46 +01:00
elacuesta 916382e109 Add errback parameter to scrapy.spiders.crawl.Rule (#4000) 
* Add errback parameter to scrapy.spiders.crawl.Rule

* CrawlSpider: optimize by reducing iterations

* [test] Rule.errback

* [doc] Rule.errback

* [doc] Use autoclass in docs/topics/spiders.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Rule.process_links takes a list

* Fix aesthetic issue reported by Flake8
 2019-12-18 20:05:33 +05:00
Andrey Rahmatullin 0e8ee22a88
Merge pull request #4244 from wRAR/split-EngineTest 
Split a long test in test_engine.py into three.
 2019-12-18 20:00:36 +05:00
Adrián Chaves 00517ec273
Merge branch 'master' into docs-py38 2019-12-18 15:50:49 +01:00
Andrey Rakhmatullin 7ccb169a27 Split a long test in test_engine.py into three. 2019-12-18 19:41:16 +05:00
Mikhail Korobov 053319334c
Merge pull request #4179 from Gallaecio/user-friendlier-tox 
Make tox configuration more user friendly
 2019-12-18 18:35:19 +05:00
Andrey Rahmatullin e2b5cdeb88
Merge pull request #4242 from whalebot-helmsman/single_place_for_dependencies 
Remove requirements-py3.txt
 2019-12-18 18:27:09 +05:00
Vostretsov Nikita 012533924a remove requirements from here too 2019-12-18 11:13:36 +00:00
Mikhail Korobov e90f276903
Merge pull request #4198 from wRAR/deprecate-noconnect 
Deprecate the HTTPS proxy noconnect mode.
 2019-12-18 16:11:24 +05:00
Adrián Chaves 1f689b50c6 Merge remote-tracking branch 'origin/documentation-coverage' into documentation-coverage 2019-12-18 12:09:34 +01:00
Adrián Chaves 174769a3f0 Use a better name for the LxmlLinkExtractor subclassing test 2019-12-18 12:09:03 +01:00
Adrián Chaves ee9881d270 Improve FilteringLinkExtractor.__new__ 2019-12-18 12:08:34 +01:00
Vostretsov Nikita 12f9ffeb5d remove requirements-py3.txt 2019-12-18 10:53:27 +00:00
Andrey Rakhmatullin ac302c3f61 Fix a flake8 problem. 2019-12-18 15:43:05 +05:00
Andrey Rakhmatullin bb2ff13e4c Skip Http10ProxyTestCase.test_download_with_proxy_https_noconnect 2019-12-18 15:39:08 +05:00
Andrey Rakhmatullin 2d92a39003 Restore test_download_with_proxy_https_noconnect, check for a warning there. 2019-12-18 12:07:08 +05:00
Adrián Chaves 607815dfc2
Merge branch 'master' into documentation-coverage 2019-12-17 16:54:27 +01:00
Adrián Chaves b8cf522916
Merge branch 'master' into user-friendlier-tox 2019-12-17 16:53:08 +01:00
Andrey Rahmatullin 4be2bbfe75
Merge pull request #4239 from Apuyuseng/master 
Fix mail attachs tcmime *** (#4229)
 2019-12-17 19:43:30 +05:00
Andrey Rahmatullin bb3f164280
Merge pull request #4236 from wRAR/pipeline-tests 
Add simple tests for pipelines.
 2019-12-17 19:38:06 +05:00
Marc Hernández 63cf5c75c8 Fix E502: backslash is redundant between brackets (#4238) 2019-12-17 13:53:15 +01:00
apu 7d0096da6e
Fix mail attachs tcmime *** (#4229) 
When the file name consists of alphanumeric characters, it is normal to receive the attachment name.
However，However, problems will occur if the file name is changed to Chinese.
This has nothing to do with the file type
 2019-12-17 09:47:01 +08:00
Andrey Rakhmatullin 5980d3bbff Add simple tests for pipelines. 2019-12-16 22:41:28 +05:00
Mikhail Korobov e9b24d62c6
Merge pull request #4231 from noviluni/docs_fix 
Docs fix
 2019-12-16 21:56:43 +05:00
Eugenio Lacuesta 451e7a616e
Scan callbacks/errbacks for return statements with values different than None 2019-12-16 11:30:40 -03:00
marc a59bb279d1 add year through code 2019-12-15 17:33:00 +01:00
marc 1aab20e1ce update copyright notice year 2019-12-14 10:37:31 +01:00
marc e3a3ad4aaf remove reference to old (Python 2.7) environment 2019-12-14 10:34:31 +01:00
Adrián Chaves a4ef9750f9 Fix Flake8-reported issues 2019-12-13 14:32:06 +01:00
Adrián Chaves b5c4c2cae8
Keep 2 spaces between code and inline comments (#4195) 2019-12-13 14:20:48 +01:00
Andrey Rahmatullin 02cdc53fb8
Add a test for a CrawlerProcess script. (#4218) 
* Add a test for a CrawlerProcess script.

* Add tests/CrawlerProcess to collect_ignore.

* Remove an extra line.

* Fix/improve conftest.py.
 2019-12-13 18:04:05 +05:00
Adrián Chaves 07b8cd28aa
Mark bandit’s 402 check as addressed by #4180 (#4181) 2019-12-05 14:48:31 +01:00