7a83474cc5
add mention of FTP server storage in media storage intro
...
At the beginning of this doc, in "Specifying where to store the media" feature details, FTP server storage mention was missing
2020-09-08 17:16:31 +02:00
1278e76d90
Bump version: 2.2.0 → 2.3.0
2020-08-04 20:07:02 +02:00
3600582f56
Cover Scrapy 2.2.1 and 2.3 in the release notes ( #4708 )
2020-08-04 20:05:56 +02:00
015b71d89f
Merge pull request #4704 from Gallaecio/python2-u-prefixes
...
Remove the u prefix from strings
2020-07-31 01:23:00 +05:00
890b2138a6
Remove the u prefix from strings
2020-07-30 13:39:30 +02:00
5e2d1bd187
Merge pull request #4434 from BroodingKangaroo/ISSUE-4250-add_batch_deliveries
...
Feed exports: add batch deliveries
2020-07-29 23:03:28 +05:00
5265853937
Use ItemAdapter.field_names when writing header in CsvItemExporter ( #4668 )
2020-07-28 11:15:14 +02:00
a6c1d79b7c
pep8 tiny changes
2020-07-28 11:53:05 +03:00
ce0c25fc85
Merge pull request #4690 from elacuesta/typing-setup-remove-monkeypatches
...
Remove monkeypatches module from mypy section in setup.cfg
2020-07-23 17:56:08 +05:00
8fae3d5bb7
Remove monkeypatches module from mypy section in setup.cfg
2020-07-22 16:08:35 -03:00
f3372a3753
Merge pull request #4254 from elacuesta/spider.parse
...
Change Scraper API to call internal `_parse` method
2020-07-21 17:37:54 +05:00
de297a3a16
enable ANSI color (instead of ANSI color codes) in the Windows terminal #4393 ( #4403 )
...
* changed ie. -> i.e.(spelling error) on lines 667, 763 (issue scrapy#4332)
* updated all text files for issue #4332 (ie. -> i.e.)
* Apply ie. → i.e. in source comments
* ie → e.g.
* modified scrapy/utils/display.py to stop ANSI color sequences in the Windows terminal (issue #4393 )
* modified scrapy/utils/display.py to stop ANSI color sequences in the Windows terminal (issue #4393 )
* enabled virtual terminal processing (pr #4403 )
* check for specific windows 10 version (pr #4403 )
* fixing flake-8 test (pr #4403 )
* added error handling for terminal info (pr #4403 )
* corrected stderr (pr #4403 )
* changed orientation, removed unwanted spaces (pr #4403 )
* no need for style variable (pr #4403 )
* fixing trailing whitespaces
* commenting windows check
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* small fixes
* Shifting _color_support_info() function
* enabled virtual terminal processing (pr #4403 )
* check for specific windows 10 version (pr #4403 )
* fixing flake-8 test (pr #4403 )
* added error handling for terminal info (pr #4403 )
* corrected stderr (pr #4403 )
* changed orientation, removed unwanted spaces (pr #4403 )
* no need for style variable (pr #4403 )
* fixing trailing whitespaces
* commenting windows check
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Update scrapy/utils/display.py
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* small fixes
* Shifting _color_support_info() function
* error handling
* error handlingy
* raise ValueError
* added in-built function for version comparison
* recommit changes
* changed check -> parse
* version comparison -> parse_version
* added scrapy/utils/display.py in pytest.ini
* Trigger
* Add simple test for scrapy.utils.display._colorize
* Flake8: E501 for tests/test_utils_display.py
* assertEquals -> assertEqual
* Normal formatter for all platforms
* separate test for windows
* all curses under try block
* added global TestStr
* more test added
* small fix
* covering exceptions
* windows test failing
* Refactor output color handling
* Fix pprint test
* fix flake8
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
2020-07-20 17:23:38 +05:00
3e0492741d
Another try to fix test errors on Windows
2020-07-19 00:10:29 +03:00
86f7ac2f2b
Try to fix error at Windows
2020-07-17 18:22:47 +03:00
62a4ede5e9
Remove deprecated classes BaseSgmlLinkExtractor, RegexLinkExtractor and SgmlLinkExtractor ( #4356 )
2020-07-17 12:40:04 +02:00
d29bec60d7
Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest) ( #4504 )
...
* Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest)
* Log a detailed error message to discover why MockServer is not working
* Go for all lines!
* Disable tests based on mitmproxy while running on PyPy
* Fix test_get_func_args for PyPy 3.6+
* Make testPayloadDefaultCiphers work regardless of OpenSSL default ciphers
* Crossing fingers…
* Rename: testPayloadDefaultCiphers → testPayloadDisabledCipher
* Test the PyPy version currently documented as the minimum required version
* Fix the PYPY_VERSION tag
* Update the documentation about supported PyPy versions
* Also test the latest 3.5 Python version with PyPy
* Fix the PYPY_VERSION value for the latest 3.5 version
* Use pinned dependencies for asyncio and PyPy tests against oldest supported Python versions
* Fix PyPy installation for the pypy3-pinned Tox environment
* Try installing Cython
* Maybe PyPy requires lxml 3.6.0?
* install.rst: minor clarification
* lxml 4.0.0 is required on PyPy
* Require setuptools 18.5+
* Revert "Require setuptools 18.5+"
This reverts commit 017ec33ac2
2020-07-17 02:19:24 +05:00
9a74a71c63
Merge pull request #4682 from noviluni/remove_python2_reminiscence
...
remove python 2 reminiscence in cookies
2020-07-16 23:58:58 +05:00
c4f9250220
Merge branch 'master' into ISSUE-4250-add_batch_deliveries
2020-07-16 17:44:18 +02:00
41263f61c6
Change single quotes to double in example in docs
2020-07-16 18:41:45 +03:00
b97a39fda0
deprecate retry_on_eintr ( #4683 )
2020-07-16 17:38:22 +02:00
07470e1a3c
Merge pull request #3608 from ejulio/feat-685
...
[MRG+1] Fix for #685 Add Google Cloud Storage Feed Export
2020-07-16 18:11:19 +05:00
0f2f1acf04
Merge pull request #4458 from scrapy/azure-pipelines
...
Set up CI with Azure Pipelines
2020-07-16 18:09:31 +05:00
0e0d1ad643
remove python 2 reminiscence in cookies
2020-07-16 14:25:46 +02:00
8bdcdb0a76
Add quotes to example in docs
2020-07-16 09:13:54 +03:00
38496a00b7
Use the itemlaoders library ( #4516 )
2020-07-15 12:08:36 +02:00
eb1bc74417
Merge pull request #4674 from ashellunts/delete_deprecated_htmlparserlinkextractor
...
Remove deprecated class HtmlParserLinkExtractor
2020-07-15 12:51:42 +05:00
ed5247ca4c
Remove htmlparser.py from tests/ignore.txt
2020-07-14 18:06:11 +02:00
6e119bd3e3
Merge branch 'master' into azure-pipelines
2020-07-14 11:26:19 +02:00
53c323b19d
_is_path → _is_filesystem_path
2020-07-13 15:29:30 +02:00
d54c4496ee
Refactor guess_scheme
2020-07-13 14:37:08 +02:00
0ebba175ea
Merge pull request #4666 from Gallaecio/execution-access
...
Fix permission handling on project generation from template files
2020-07-13 16:01:06 +05:00
b6f8693db9
Merge pull request #4673 from elacuesta/remove-backslash
...
Remove backslash
2020-07-13 11:14:00 +05:00
3f7e8635f4
Allow the parse command to write data to a file ( #4377 )
2020-07-11 08:48:24 +02:00
a6a5fa91da
Remove deprecated class HtmlParserLinkExtractor
...
Issue #4356
2020-07-10 23:10:49 +02:00
9aea1f0961
Remove backslash (tests)
2020-07-09 11:04:46 -03:00
9e99be982a
Remove backslash
2020-07-09 09:51:28 -03:00
2be2bdd2df
Merge remote-tracking branch 'upstream/master' into spider.parse
2020-07-08 14:00:54 -03:00
7e38615703
Remove unused import
2020-07-07 15:30:19 +02:00
ca77ca1f75
Generate read-only files on the fly
2020-07-07 14:44:03 +02:00
e1450799ce
Remove debug test case variable
2020-07-07 14:11:37 +02:00
a3afff4a0e
Fix style issue
2020-07-07 14:11:02 +02:00
79b4dfc53e
Fix permission handling on project generation from template files
2020-07-07 14:07:04 +02:00
f1020e0e6a
Tiny changes
2020-07-06 15:40:53 +03:00
17aec5944c
Update tests/CrawlerRunner/ip_address.py
...
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
2020-07-06 10:47:25 +02:00
ec06cf79a6
Update tests/CrawlerRunner/ip_address.py
...
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
2020-07-06 10:47:11 +02:00
6454d456d2
Make check of placeholder less strict
2020-07-03 10:24:13 +03:00
eb93774256
TrackrefTestCase.test_get_oldest: protect from lack of precision
2020-07-03 01:41:47 +02:00
3199048520
Complete Azure Pipelines CI setup
2020-07-03 00:56:24 +02:00
6e58da1dcd
Merge branch 'master' into azure-pipelines
2020-07-02 17:49:42 +02:00
1e245046ed
Change setting name. Add leading zeroes to batch_id. Minor fixes.
2020-07-02 12:38:08 +03:00
KAILASA's Sri Nithya Priyeshananda
Adrián Chaves
Mikhail Korobov
Eugenio Lacuesta
BroodingKangaroo
Eugenio Lacuesta
Andrey Rahmatullin
Akshay Sharma
Artur Shellunts
Marc Hernández
Júlio César Batista
Aditya Kumar