RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,909 Commits 49 Branches 132 Tags 131 MiB
Commit Graph

55 Commits

Author SHA1 Message Date
Andrey Rakhmatullin d161d1d47d
Convert tests/test_utils* to plain asserts. (#6695) 2025-03-04 09:31:26 +01:00
Andrey Rakhmatullin 8d92c28a16
Switch to pytest.raises(). (#6680) 
* Switch to pytest.raises().

* Add matches= to broad pytest.raises().

* Adjust the test_nonserializable_object() regex for Python <= 3.11.

* Adjust the test_nonserializable_object() regex for PyPy.

* Adjust other test exception regexes for PyPy.

* Cleanup.
 2025-02-27 23:13:04 +05:00
Andrey Rakhmatullin a898331d14
Preparations for switching to direct pytest. (#6678) 2025-02-16 20:28:58 +01:00
Andrey Rakhmatullin 1e4c81e9dc Add Perflint rules to ruff. 2024-12-12 21:03:34 +05:00
Rohit Kumar Singh d2156696c4
Deprecate unused scrapy utils (#6519) 
* Added deprecation warnings for unused Scrapy.utils

* Grammatical corrections

* Exceptions class connected

* Deprecation of ScrapyJSONDecoder

* request_authenticate function deprecation

* Making all warning similar

* Added ignore statements for deprecation warning in tests

* Missing stacklevel attr. added

* Added Deprecation message
 2024-11-04 21:09:45 +05:00
Andrey Rakhmatullin e139d22db9 Fix expectations for get_func_args() on 3.13. 2024-09-20 19:28:28 +05:00
kokobhara b1fe97dc6c
Fix test expectations (#6316) 2024-04-17 13:14:57 +02:00
pengqiseven 6e84648c07
Fix some comments (#6285) 
Signed-off-by: pengqiseven <i@sohu.com>
Co-authored-by: pengqiseven <i@sohu.com>
 2024-03-11 10:03:06 +01:00
Serhii A c2a31974ff
Improve get_func_args (#5885) 2023-04-13 11:44:20 +02:00
Andrey Rakhmatullin 824641e349 Merge remote-tracking branch 'origin/master' into mypy-imports 2023-02-11 22:02:19 +04:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Andrey Rakhmatullin c0ea7fd4fd Remove obsolete top-level run code in tests. 2023-01-25 22:14:24 +04:00
Emmanuel Rondan 8ee4817471 reaplying black, fixing conflicts and ignoring bandit checks on test directory 2023-01-20 10:55:46 -03:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Laerte Pereira d5b6c236a9
Remove deprecated code (#5719) 2022-11-21 09:32:26 +01:00
Andrey Rakhmatullin b21c16099e Fix flake8 issues. 2022-07-22 19:18:33 +05:00
Andrey Rahmatullin a0681fb811
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2022-07-22 18:30:38 +05:00
Laerte Pereira 407562b38b
Drop Python 3.6 support (#5514) 
* chore: Drop Python 3.6 support

* Attend PR comments

* Tweak versions

* Update dependencies version

* fix: Ubuntu workflow

* fix windows workflow

* chore: Remove comment

* update `install_requires` dependencies versions

* move lxml to main pinned requirements

* Attend code-review comments

* remove non-pinned 3.7 from windows workflow

* simplify condition

* lint

* remove paragraph

* refactor

* remove leftover
 2022-06-09 00:25:03 -03:00
Andrey Rakhmatullin ecfc924ca8 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-06-04 18:55:32 +05:00
Vostretsov Nikita 23cfdb058e
Reducing amount of warnings during test run (#5162) 
* put flake8 options into separate file to remove pytest warnings

* remove ResourceLeaked warning in pypy

* suppress warnings from twisted

* ignore deprecation warnings here

* ignore deprecation warning in tests of deprecated methods

* ignore deprecation warnings here

* update test classes

* don`t use deprecated method call

* ignore deprecation warnings here

* proper warning class

* more selective ignoring

* Revert "don`t use deprecated method call"

This reverts commit 59216ab560.
 2021-05-28 14:45:06 +05:00
Andrey Rakhmatullin 7e9f498e00 Add MutableAsyncChain. 2021-02-05 16:21:56 +05:00
Eugenio Lacuesta f1250177dc
Remove Python 3.5 from CI (#4743) 2020-08-22 09:33:35 +02:00
Adrián Chaves e70975f0bb
Allow overwriting feeds (#4512) 
Co-authored-by: Yuval Hager <yhager@yhager.com>
 2020-08-17 15:10:08 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Adrián Chaves d29bec60d7
Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest) (#4504) 
* Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest)

* Log a detailed error message to discover why MockServer is not working

* Go for all lines!

* Disable tests based on mitmproxy while running on PyPy

* Fix test_get_func_args for PyPy 3.6+

* Make testPayloadDefaultCiphers work regardless of OpenSSL default ciphers

* Crossing fingers…

* Rename: testPayloadDefaultCiphers → testPayloadDisabledCipher

* Test the PyPy version currently documented as the minimum required version

* Fix the PYPY_VERSION tag

* Update the documentation about supported PyPy versions

* Also test the latest 3.5 Python version with PyPy

* Fix the PYPY_VERSION value for the latest 3.5 version

* Use pinned dependencies for asyncio and PyPy tests against oldest supported Python versions

* Fix PyPy installation for the pypy3-pinned Tox environment

* Try installing Cython

* Maybe PyPy requires lxml 3.6.0?

* install.rst: minor clarification

* lxml 4.0.0 is required on PyPy

* Require setuptools 18.5+

* Revert "Require setuptools 18.5+"

This reverts commit 017ec33ac2.

* Maintain lxml as a dependency if setuptools < 18.5 is used
 2020-07-17 02:19:24 +05:00
Eugenio Lacuesta 4270e0a0da
Fix E731: do not assign a lambda expression 2020-04-04 21:54:55 -03:00
elacuesta dfbe1d9507
Remove object base class (#4430) 2020-03-16 20:12:46 +01:00
Marc Hernandez Cabot b49ece0b87 fix E701 and E271 flake8 2020-02-22 12:32:38 +01:00
Marc Hernández 91bbc70bc1
fix E30X flake8 (#4355) 2020-02-21 06:05:31 +01:00
Eugenio Lacuesta 714e0d3ac9
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-20 12:41:38 -03:00
Mikhail Korobov 65e2fb7d89
Merge pull request #4153 from Gallaecio/lgtm 
MutableChain: return self from __iter__
 2019-11-20 19:46:20 +05:00
Eugenio Lacuesta 05785c1c17
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-19 11:01:34 -03:00
Adrián Chaves 99d8b05a0b Deprecate scrapy.utils.python.MutableChain.next 2019-11-18 10:58:47 +01:00
Andrey Rakhmatullin fe3a121f13 Use kwargs when calling get_func_args. 2019-11-14 22:50:53 +05:00
Adrián Chaves 1a4a77d49f Remove Python 2 check from MutableChainTest 2019-11-14 10:24:31 +01:00
Andrey Rakhmatullin 1d7c8cb0b1 Remove six.PY2 and six.PY3 conditionals. 2019-11-14 13:45:37 +05:00
Eugenio Lacuesta b6bbb28197
PEP8 adjustments 2019-11-08 22:13:03 -03:00
Eugenio Lacuesta cca1959c07
Merge remote-tracking branch 'upstream/remove-py2-tests' into remove-six-code 2019-11-03 01:20:08 -03:00
Eugenio Lacuesta 5d8abdde59
Remove six.text_type from tests 2019-11-03 01:01:23 -03:00
Andrey Rakhmatullin bbd9f4be90 Remove six.PY2 and six.PY3 conditionals. 2019-10-31 15:07:10 +05:00
Andrey Rakhmatullin b44bd6f825 Remove Python 2-only tests. 2019-10-31 15:03:09 +05:00
Eugenio Lacuesta e0360e5223 Add tests for MutableChain 2018-10-11 11:55:13 -03:00
Konstantin Lopuhin 041308afe7 Fix get_func_args test for pypy3 
These built-in functions are exposed as methods in PyPy3.
For scrapy this does not matter as:
1) they do not work for CPython at all
2) get_func_args is checked for presense of an argument in scrapy,
   extra "self" does not matter.
But it still makes sense to leave these tests so that we know we
shouldn't use get_func_args for built-in functions/methods.
 2017-12-25 14:48:10 +03:00
Chomba Ng'ang'a fd27cde24d Update asserts to use more generic ones 2017-08-09 12:52:10 +02:00
Chomba Ng'ang'a 4ca61a2051 Update deprecated test aliases 
- change ``failIf`` to ``assertFalse``
- change ``asertEquals`` to ``assertEqual``
- change ``assert_`` to ``assertTrue``

https://docs.python.org/2/library/unittest.html#deprecated-aliases
 2017-08-09 12:52:10 +02:00
Konstantin Lopuhin 7c67047e77 Fix get_func_args tests under PyPy 
On CPython get_func_args does not work correctly for built-in
methods.
 2017-06-15 13:07:59 +03:00
Konstantin Lopuhin 5abb70c8d7 Fix test_weakkeycache on PyPy: run gc.collect() 
One gc.collect() seems to be enough, but it's more reliable
to run it several times (at most 100), until all objects are collected.
 2017-06-15 13:06:30 +03:00
nyov e8ca467572 Rename isbinarytext function to binary_is_text for clarity 
Closes #1389
 2016-03-30 15:44:15 +00:00
Jakob de Maeyer f249b309ab Move scrapy.utils.conf.remove_none_values to s.u.python.without_none_values 2015-10-27 12:43:33 +01:00
Mikhail Korobov a7b4a3e7f9 cleanup 
* run test_utils_python in Python 3;
* make tests for 'errors' argument more explicit
* add missing test_ prefix utf_16_strings_contain_null_bytes;
* cleanup test names.
 2015-07-23 20:03:33 +02:00