RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,967 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

7967 Commits

Author SHA1 Message Date
Victor Torres e0921cab66
remove not used code 
This code is not needed anymore because we're getting the
already mangled name when matching func with spider attributes.
 2020-04-16 11:18:59 -03:00
Victor Torres 47a992615a
serialize requests with callback references as spider attribute 
You could define a spider attribute that references a callback method
but if this method has a different name than your spider attribute,
the request serializer is not able to find it on the spider class.

With this commit we're fixing this behavior as we're searching for
callback references in the spider object itself instead of looking
for attributes with the same function's name, that could be different.
 2020-04-15 20:03:58 -03:00
Mikhail Korobov 36abe9235c
Merge pull request #4445 from adityaa30/docs-redirect-links 
update redirect links to python3
 2020-04-15 23:13:55 +05:00
Mikhail Korobov 066c02d0c3
Merge pull request #4455 from aditi137/docs-curl2scrapy 
[Docs] mention curl2scrapy in Request.from_curl
 2020-04-15 23:12:33 +05:00
Mikhail Korobov 2d46777d0a
Merge pull request #4489 from elacuesta/remove-returnValue-function 
Remove twisted.internet.defer.returnValue occurrences
 2020-04-15 22:45:52 +05:00
Mikhail Korobov 885b0ad2d4
Merge pull request #4480 from elacuesta/pin-sphinx-versions 
Pin version for Sphinx and extensions, configure hoverxref
 2020-04-15 22:42:49 +05:00
Eugenio Lacuesta ac869181fb
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2020-04-15 13:42:35 -03:00
santoshkosgi c3257dc610
Change Content-type to Content-Type (#4481) 
Co-authored-by: santosh <santosh@acalvio.com>
 2020-04-15 14:24:10 +02:00
Eugenio Lacuesta 94d7ad76cb
Fix pycodestyle E2XX (whitespace) (#4468) 2020-04-15 14:11:37 +02:00
Eugenio Lacuesta ee4ee486b1
Revert unnecessary changes to docs/conf.py 2020-04-14 15:06:54 -03:00
Eugenio Lacuesta 0a4ef97fa3
Loose restrictions for docs requirements 2020-04-14 14:57:20 -03:00
Eugenio Lacuesta 4023d5db33
Replace _DefGen_Return exception handling 
Handle StopIteration instead
 2020-04-13 06:35:26 -03:00
Eugenio Lacuesta 1bd8f392c9
Initial removal of twisted.internet.defer.returnValue 2020-04-13 06:12:30 -03:00
Eugenio Lacuesta 2205f04631
Docs: Add hoverxref_role_types setting 2020-04-10 18:08:04 -03:00
Eugenio Lacuesta 34e81d0d74
Docs: remove duplicated setting definitions 2020-04-10 17:29:02 -03:00
Eugenio Lacuesta 4383f45299
Replace os.path with pathlib in docs config 2020-04-10 16:49:14 -03:00
Eugenio Lacuesta 24a1d9acae
Get version in docs config 2020-04-10 16:48:42 -03:00
Eugenio Lacuesta f97fec5ebd
Pin Sphinx version, including extensions 2020-04-10 16:02:53 -03:00
Mikhail Korobov 39b01b6892
Merge pull request #4469 from elacuesta/fix-pycodestyle-e502 
Fix pycodestyle E502 ("the backslash is redundant between brackets")
 2020-04-10 22:03:39 +05:00
Andrey Rahmatullin 49357cccbd
Merge pull request #4472 from elacuesta/remove-request-body-producer-empty-body 
Remove empty _RequestBodyProducer for POST requests
 2020-04-06 19:48:20 +05:00
Eugenio Lacuesta 862f0301e2
Remove empty _RequestBodyProducer for POST requests 2020-04-05 00:53:10 -03:00
Eugenio Lacuesta e2d5d357a7
Fix pycodestyle E502 2020-04-01 14:24:30 -03:00
Mikhail Korobov 8845773d44
Merge pull request #4456 from whalebot-helmsman/project_links 
Project URLs in machine-readable format for showing in pypi
 2020-03-27 14:11:03 +05:00
Aditya b1904729d5 [docs] change mod to doc redirect link 2020-03-27 04:37:26 +05:30
Vostretsov Nikita 16f2cb4a83 project URLs in machine-readable format for showing in pypi 2020-03-26 12:57:39 +00:00
Aditi Dutta 010edfe85c [Docs] mention curl2scrapy in Request.from_curl 2020-03-25 14:38:22 -04:00
Aditya 80c69d68ad [docs] refactor python docs links using intersphinx 2020-03-24 18:35:42 +05:30
Mikhail Korobov 0ee04e1e91
Merge pull request #4448 from adityaa30/fix-zope 
[fix] zope interface 5.0.0 unsupported
 2020-03-20 23:18:42 +05:00
Mikhail Korobov c26308dd45
Merge pull request #4438 from seregaxvm/master 
edit zsh completion
 2020-03-20 18:30:50 +05:00
Aditya 532cd1d93e [fix] zope interface 5.0.0 unsupported 2020-03-20 17:36:49 +05:30
Aditya f37b1bdc56 [docs] update redirect links to python3 2020-03-20 05:31:41 +05:30
Aditya ca08e04198 [docs] update redirect links python2 -> python3 2020-03-20 02:31:35 +05:30
Adrián Chaves 6c747953f9 Cover 2.0.1 in the release notes (#4437) 2020-03-18 19:17:16 +01:00
Matsievskiy S.V 9ab45325ff edit zsh completion 
- Fix bug introduced in https://github.com/scrapy/scrapy/pull/4291
- Enforce `[command] [options] [arguments]` syntax. Do not allow options after arguments
- Exclude already used option aliases from completion list
 2020-03-17 18:45:00 +03:00
sakshamb2113 533131a30f
Remove Guppy-specific code and documentation (#4343) 2020-03-17 10:12:49 +01:00
elacuesta dfbe1d9507
Remove object base class (#4430) 2020-03-16 20:12:46 +01:00
elacuesta e5711127b1
Remove deprecated ChunkedTransferMiddleware (#4431) 2020-03-16 19:43:02 +01:00
Andrey Rahmatullin ecd3a976d2
Merge pull request #4398 from nyov/remove-crawler-spiders 
Remove deprecated Crawler.spiders property
 2020-03-16 17:21:07 +05:00
Andrey Rahmatullin 1d862e0d35
Merge pull request #4404 from nyov/obsolete2 
Remove obsolete DEPRECATED_SETTINGS (deprecated.py)
 2020-03-16 17:18:48 +05:00
Aditya Kumar f9bf4b8d4d
Remove all top-level imports for twisted.internet.reactor (#4406) 2020-03-14 10:39:00 +01:00
Andrey Rahmatullin 0afe3da3dd
Merge pull request #4422 from Gallaecio/pin-working-pytest 
Restrict pytest to versions prior to 5.4
 2020-03-13 18:28:11 +05:00
Adrián Chaves 3f6cdcabce Restrict pytest to versions prior to 5.4 2020-03-13 13:25:53 +01:00
Lukas Anzinger ccc4d88779
Ignore a domain in allowed_domains with port and issue a warning (#4413) 2020-03-12 20:42:14 +01:00
nyov 3b0820d747
Deprecate Spider.make_requests_from_url, part 2 (#4412) 2020-03-12 20:15:49 +01:00
Mikhail Korobov db9198bc20
Merge pull request #4409 from nyov/remove-deprecation-warning 
Remove a 'twisted.test.proto_helpers' deprecation warning
 2020-03-12 18:46:47 +05:00
Mikhail Korobov ae5f59856c
Merge pull request #4420 from elacuesta/response_follow_all_empty_generator 
Response.follow_all: return empty generators for empty sequences
 2020-03-12 18:44:49 +05:00
Eugenio Lacuesta 8d30dc0888
Response.follow_all: return empty generators for empty sequences 2020-03-12 09:36:15 -03:00
Andrey Rahmatullin 388f23c30c
Merge pull request #3858 from elacuesta/multiple_feed_export_formats 
[MRG+1] Support for exporting to multiple feeds in a single crawl
 2020-03-12 14:31:57 +05:00
Eugenio Lacuesta c886a70eae
Use dict.setdefault in scrapy.utils.conf.feed_complete_default_values_from_settings 2020-03-11 21:06:51 -03:00
Eugenio Lacuesta f3bab819ab
Add tests for scrapy.utils.conf.feed_complete_default_values_from_settings 2020-03-11 20:57:31 -03:00