RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

10393 Commits

Author SHA1 Message Date
Adrián Chaves ec0a0eb3ab
Merge branch 'master' into spider-name-collision 2020-04-28 21:13:08 +02:00
Adrián Chaves 1006db1e10
Merge branch 'master' into documentation-build 2020-04-28 21:07:14 +02:00
Adrián Chaves f2bbdb43ee
Merge branch 'master' into remove-deprecated-http-client-factory 2020-04-28 18:24:16 +02:00
Eugenio Lacuesta f787b8483c IPv6 test: check for the absence of DNSLookupError 2020-04-28 17:50:06 +02:00
Adrián Chaves 3a64f3eb29 Remove TOXENV from .travis.yml unless needed 2020-04-28 17:44:19 +02:00
Adrián Chaves 5c0f11b4ef Simplify the asyncio Tox environment 2020-04-28 17:32:53 +02:00
Adrián Chaves 15d96ab8b5 Test the latest Ubuntu along the latest Python 2020-04-28 17:09:05 +02:00
Adrián Chaves e3c3ec2ba9 Run quick tests first in Travis CI 2020-04-28 13:48:50 +02:00
Ashe c207dbf939
Remove the asyncio warning from coroutines page (#4513) 2020-04-27 19:45:19 +02:00
BroodingKangaroo 204737042a Extract the slot closing functionality to the function; minor changes 2020-04-27 12:52:18 +03:00
BroodingKangaroo a18621552f Merge branch 'master' of https://github.com/BroodingKangaroo/scrapy into ISSUE-4250-add_batch_deliveries 2020-04-27 10:34:36 +03:00
BroodingKangaroo 2eee6c8101 Documentation spelling fix 2020-04-27 09:58:14 +03:00
BroodingKangaroo f0f1be76d1 Using time_id instead of time as a timestamp 2020-04-27 09:56:57 +03:00
Adrián Chaves 3878b67a37 Bump version: 2.0.0 → 2.1.0 2020-04-24 11:46:54 +02:00
Adrián Chaves ffe576c4ed
Cover Scrapy 2.1 in the release notes (#4499) 
Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2020-04-24 11:44:36 +02:00
Eugenio Lacuesta efb6f13deb
Remove assertions from production code (#4440) 2020-04-23 12:40:10 +02:00
Aditya Kumar e4750f2fbd
async/deferred signal handlers (#4390) 
* [docs] async/deferred signal handlers

* [docs] update deferred signals example

* [docs] add subsections for built-in signals

* docs(signals): update signal handler example

* docs(signals): update signal handler example
 2020-04-20 20:47:57 +05:00
Eugenio Lacuesta 1fecacbb1a
IPv6 test: check for the absence of DNSLookupError 2020-04-20 12:05:15 -03:00
ilias-ant 773ddf77ba added more tests to cover the (downloaded, uptodate, cached) status functionality 2020-04-19 14:14:17 +03:00
sakshamb2113 bfeb2c8c13
Added warning to use double quotes in Windows for scrapy shell in shell.rst (#4450) 
* modified debugging memory leaks with guppy in leaks.rst

* modified leaks.rst(issue #4285)

* removed guppy from telnet.py

* Fix undefined name error

* removed hpy key from telnet_vars in telnet.py

* updated shell.rst

* Update docs/topics/shell.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-04-18 20:21:26 +05:00
Mikhail Korobov 66ab3347c2
Merge pull request #4507 from elacuesta/docs-update-deprecated-feed-settings 
Docs: replace deprecated FEED_FORMAT and FEED_URI settings
 2020-04-18 20:19:01 +05:00
BroodingKangaroo ec76445dd6 Update tests 2020-04-18 09:29:23 +03:00
Eugenio Lacuesta 04b6295a69
Docs: replace deprecated FEED_* settings 2020-04-17 20:50:17 -03:00
Mikhail Korobov 209ab05315
Merge pull request #4500 from victor-torres/reference-callback 
Serialize requests with callback references as spider attribute
 2020-04-17 18:58:28 +05:00
Victor Torres 1ade3fc723
trying to improve test coverage 2020-04-17 10:34:34 -03:00
Eugenio Lacuesta 7926da4bd0
Merge branch 'master' into flake8-max-line-length 2020-04-16 16:05:09 -03:00
Mikhail Korobov cf39602c30
Merge pull request #4471 from elacuesta/fix-pycodestyle-e731 
Fix pycodestyle E731 ("do not assign a lambda expression, use a def")
 2020-04-16 23:17:31 +05:00
Mikhail Korobov c33fd47a80
Merge pull request #3940 from elacuesta/response_ip_address 
Get server IP address for HTTP/1.1 Responses
 2020-04-16 21:05:41 +05:00
Adrián Chaves ea3e675801 test_utils_iterators: use os.linesep 2020-04-16 17:10:45 +02:00
Adrián Chaves cf41803089 Skip test_reactor_poll on Windows 2020-04-16 17:07:29 +02:00
Adrián Chaves 7cc9601029 Improve reporting on test_ipv6_alternative_name_resolver 2020-04-16 16:57:48 +02:00
Eugenio Lacuesta c922992277
Tests: Move code inside __main__ block 2020-04-16 11:37:37 -03:00
Victor Torres 94c95020b3
add comment to explain the use of __func__ instead of instance method objects 2020-04-16 11:37:03 -03:00
Eugenio Lacuesta 1f2e2a6006
Merge branch 'master' into response_ip_address 2020-04-16 11:32:37 -03:00
Victor Torres e0921cab66
remove not used code 
This code is not needed anymore because we're getting the
already mangled name when matching func with spider attributes.
 2020-04-16 11:18:59 -03:00
Adrián Chaves 94ee68695a Mock server: use 127.0.0.1 also for HTTPS 
Windows throws an error about 0.0.0.0 being external:
https://stackoverflow.com/a/23857995/939364
 2020-04-16 15:37:06 +02:00
Mikhail Korobov fe7043a648
Merge pull request #4503 from Gallaecio/hoverxref-configuration 
Fix the hoverxref configuration
 2020-04-16 18:19:46 +05:00
Adrián Chaves 1d77eac950 Fix Flake8-reported issues 2020-04-16 14:57:55 +02:00
Adrián Chaves 901892dab3 Fix the hoverxref configuration 2020-04-16 14:48:38 +02:00
BroodingKangaroo 5980ae72c6 Some minor fixes and refactoring 2020-04-16 10:13:39 +03:00
BroodingKangaroo cac1f3a6ad Update documentation 2020-04-16 10:06:56 +03:00
Victor Torres 47a992615a
serialize requests with callback references as spider attribute 
You could define a spider attribute that references a callback method
but if this method has a different name than your spider attribute,
the request serializer is not able to find it on the spider class.

With this commit we're fixing this behavior as we're searching for
callback references in the spider object itself instead of looking
for attributes with the same function's name, that could be different.
 2020-04-15 20:03:58 -03:00
BroodingKangaroo 16836e9e77 Merge branch 'ISSUE-4250-add_batch_deliveries' of https://github.com/BroodingKangaroo/scrapy into ISSUE-4250-add_batch_deliveries 2020-04-16 00:41:21 +03:00
BroodingKangaroo 963580463b Update tests 2020-04-15 23:39:46 +03:00
BroodingKangaroo ffa8a533e7 Set batch_id in _get_uri_params 2020-04-15 23:38:27 +03:00
BroodingKangaroo 39d0d13d3f Add partial deliveries tests 2020-04-15 23:38:27 +03:00
BroodingKangaroo d11411b402 fix comments 2020-04-15 23:38:27 +03:00
BroodingKangaroo 0723e3f4f9 add batch_id, add error if uri is specified incorrectly 2020-04-15 23:38:27 +03:00
BroodingKangaroo 8b4566ff93 fix wrong name of first file in partial deliveries 2020-04-15 23:38:27 +03:00
BroodingKangaroo e5b23f4b00 fix #4250: add batch deliveries 2020-04-15 23:38:27 +03:00