RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,710 Commits 49 Branches 132 Tags 145 MiB
Commit Graph

10710 Commits

Author SHA1 Message Date
Eugenio Lacuesta 1f2e2a6006
Merge branch 'master' into response_ip_address 2020-04-16 11:32:37 -03:00
Victor Torres e0921cab66
remove not used code 
This code is not needed anymore because we're getting the
already mangled name when matching func with spider attributes.
 2020-04-16 11:18:59 -03:00
Adrián Chaves 94ee68695a Mock server: use 127.0.0.1 also for HTTPS 
Windows throws an error about 0.0.0.0 being external:
https://stackoverflow.com/a/23857995/939364
 2020-04-16 15:37:06 +02:00
Mikhail Korobov fe7043a648
Merge pull request #4503 from Gallaecio/hoverxref-configuration 
Fix the hoverxref configuration
 2020-04-16 18:19:46 +05:00
Adrián Chaves 1d77eac950 Fix Flake8-reported issues 2020-04-16 14:57:55 +02:00
Adrián Chaves 901892dab3 Fix the hoverxref configuration 2020-04-16 14:48:38 +02:00
BroodingKangaroo 5980ae72c6 Some minor fixes and refactoring 2020-04-16 10:13:39 +03:00
BroodingKangaroo cac1f3a6ad Update documentation 2020-04-16 10:06:56 +03:00
Victor Torres 47a992615a
serialize requests with callback references as spider attribute 
You could define a spider attribute that references a callback method
but if this method has a different name than your spider attribute,
the request serializer is not able to find it on the spider class.

With this commit we're fixing this behavior as we're searching for
callback references in the spider object itself instead of looking
for attributes with the same function's name, that could be different.
 2020-04-15 20:03:58 -03:00
BroodingKangaroo 16836e9e77 Merge branch 'ISSUE-4250-add_batch_deliveries' of https://github.com/BroodingKangaroo/scrapy into ISSUE-4250-add_batch_deliveries 2020-04-16 00:41:21 +03:00
BroodingKangaroo 963580463b Update tests 2020-04-15 23:39:46 +03:00
BroodingKangaroo ffa8a533e7 Set batch_id in _get_uri_params 2020-04-15 23:38:27 +03:00
BroodingKangaroo 39d0d13d3f Add partial deliveries tests 2020-04-15 23:38:27 +03:00
BroodingKangaroo d11411b402 fix comments 2020-04-15 23:38:27 +03:00
BroodingKangaroo 0723e3f4f9 add batch_id, add error if uri is specified incorrectly 2020-04-15 23:38:27 +03:00
BroodingKangaroo 8b4566ff93 fix wrong name of first file in partial deliveries 2020-04-15 23:38:27 +03:00
BroodingKangaroo e5b23f4b00 fix #4250: add batch deliveries 2020-04-15 23:38:27 +03:00
Adrián Chaves 2f510fd47d Fix ShellTest.test_local_file on Windows 2020-04-15 21:10:05 +02:00
Mikhail Korobov 36abe9235c
Merge pull request #4445 from adityaa30/docs-redirect-links 
update redirect links to python3
 2020-04-15 23:13:55 +05:00
Mikhail Korobov 066c02d0c3
Merge pull request #4455 from aditi137/docs-curl2scrapy 
[Docs] mention curl2scrapy in Request.from_curl
 2020-04-15 23:12:33 +05:00
Mikhail Korobov 2d46777d0a
Merge pull request #4489 from elacuesta/remove-returnValue-function 
Remove twisted.internet.defer.returnValue occurrences
 2020-04-15 22:45:52 +05:00
Mikhail Korobov 885b0ad2d4
Merge pull request #4480 from elacuesta/pin-sphinx-versions 
Pin version for Sphinx and extensions, configure hoverxref
 2020-04-15 22:42:49 +05:00
Eugenio Lacuesta ac869181fb
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2020-04-15 13:42:35 -03:00
Adrián Chaves c28dd0cedb
Merge branch 'master' into duplicate_key 2020-04-15 16:22:59 +02:00
Eugenio Lacuesta f242751b62
Remove empty line 2020-04-15 09:39:28 -03:00
Eugenio Lacuesta 01d73dd3f0
Merge branch 'master' into flake8-max-line-length 2020-04-15 09:24:33 -03:00
santoshkosgi c3257dc610
Change Content-type to Content-Type (#4481) 
Co-authored-by: santosh <santosh@acalvio.com>
 2020-04-15 14:24:10 +02:00
Adrián Chaves d323a57381
Merge branch 'master' into fix-pycodestyle-e731 2020-04-15 14:16:15 +02:00
Eugenio Lacuesta 94d7ad76cb
Fix pycodestyle E2XX (whitespace) (#4468) 2020-04-15 14:11:37 +02:00
Eugenio Lacuesta ee4ee486b1
Revert unnecessary changes to docs/conf.py 2020-04-14 15:06:54 -03:00
Eugenio Lacuesta 0a4ef97fa3
Loose restrictions for docs requirements 2020-04-14 14:57:20 -03:00
Eugenio Lacuesta 4023d5db33
Replace _DefGen_Return exception handling 
Handle StopIteration instead
 2020-04-13 06:35:26 -03:00
Eugenio Lacuesta 1bd8f392c9
Initial removal of twisted.internet.defer.returnValue 2020-04-13 06:12:30 -03:00
ilias-ant 83a0cc6cdf Add status to files information 2020-04-12 23:22:17 +03:00
Eugenio Lacuesta 2205f04631
Docs: Add hoverxref_role_types setting 2020-04-10 18:08:04 -03:00
Eugenio Lacuesta 34e81d0d74
Docs: remove duplicated setting definitions 2020-04-10 17:29:02 -03:00
Eugenio Lacuesta 4383f45299
Replace os.path with pathlib in docs config 2020-04-10 16:49:14 -03:00
Eugenio Lacuesta 24a1d9acae
Get version in docs config 2020-04-10 16:48:42 -03:00
Eugenio Lacuesta f97fec5ebd
Pin Sphinx version, including extensions 2020-04-10 16:02:53 -03:00
Eugenio Lacuesta 950a52468c
Merge branch 'master' into flake8-max-line-length 2020-04-10 15:24:22 -03:00
Eugenio Lacuesta 2265c1fda9
Merge branch 'master' into fix-pycodestyle-e731 2020-04-10 14:32:30 -03:00
Mikhail Korobov 39b01b6892
Merge pull request #4469 from elacuesta/fix-pycodestyle-e502 
Fix pycodestyle E502 ("the backslash is redundant between brackets")
 2020-04-10 22:03:39 +05:00
Andrey Rahmatullin 49357cccbd
Merge pull request #4472 from elacuesta/remove-request-body-producer-empty-body 
Remove empty _RequestBodyProducer for POST requests
 2020-04-06 19:48:20 +05:00
Eugenio Lacuesta c4a5e3f0da
Simplify bytes_received signal 
Remove "source" parameter
 2020-04-06 09:26:13 -03:00
Eugenio Lacuesta e3342669ae
Merge branch 'master' into bytes_received_signal 2020-04-06 09:17:35 -03:00
Eugenio Lacuesta 862f0301e2
Remove empty _RequestBodyProducer for POST requests 2020-04-05 00:53:10 -03:00
Eugenio Lacuesta c887fe37ad
Simplify parse command 2020-04-04 22:15:36 -03:00
Eugenio Lacuesta 4270e0a0da
Fix E731: do not assign a lambda expression 2020-04-04 21:54:55 -03:00
Eugenio Lacuesta e2d5d357a7
Fix pycodestyle E502 2020-04-01 14:24:30 -03:00
Adrián Chaves bdb28ac600 Merge remote-tracking branch 'origin/sybil' into azure-pipelines 2020-03-30 20:25:40 +02:00