RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,204 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

81 Commits

Author SHA1 Message Date
Albert Eduardovich N. 9776a72a6a
chore: simplify some code, get rid of nested fns where it's makes sence + pylint (#7401) 
* chore: simplify some code, get rid of nested fns where it's makes sence

reasoning: https://wemake-python-styleguide.readthedocs.io/en/latest/pages/usage/violations/best_practices.html#wemake_python_styleguide.violations.best_practices.NestedFunctionViolation

* chore: address some pylint rules

* remove `pylint: disable=deprecated-class` from `reactor.py`

* fix typo

* fix another typo :)

* revert backward incompatible arg renames

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2026-05-04 23:32:46 +05:00
Andrey Rakhmatullin f3868e11fb
Drop start_requests() support. (#7490) 2026-05-04 17:34:24 +05:00
Andrey Rakhmatullin 916fe50974
Fix and ban the top-level twisted.internet.reactor imports. (#6835) 2025-05-28 15:53:52 +05:00
Adrián Chaves 036f3e5627
Support asynchronous start requests (#6729) 2025-05-07 19:04:03 +02:00
Andrey Rakhmatullin ce5a132f12
Run and fix linkcheck. (#6524) 2024-11-04 11:40:07 +01:00
Andrey Rakhmatullin d85c39f5bc
Deprecation removals. (#6500) 
* Deprecation removals.

* Clean up the default pytest filterwarnings.

* Remove test_get_images_old().

* Redo boto-requiring test filtering.

* Remove an unused function.

* Improve the Crawler.crawl() error message.

* Fix the test.
 2024-10-31 18:06:22 +05:00
Adrián Chaves b07d3f85a3 Upgrade CI tools 2023-02-02 06:37:40 +01:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Emmanuel Rondan 8ee4817471 reaplying black, fixing conflicts and ignoring bandit checks on test directory 2023-01-20 10:55:46 -03:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Kian-Meng, Ang 3a263280ba Fix typos 2021-10-11 22:32:42 +08:00
Ammar Najjar 58ca8bbf6d Use f-strings (#4307) 2020-08-26 07:11:05 +02:00
Marc Hernández 336f19f5cc
Change super syntax (#4707) 2020-08-04 20:42:01 +02:00
Matsievskiy S.V 9ab45325ff edit zsh completion 
- Fix bug introduced in https://github.com/scrapy/scrapy/pull/4291
- Enforce `[command] [options] [arguments]` syntax. Do not allow options after arguments
- Exclude already used option aliases from completion list
 2020-03-17 18:45:00 +03:00
Matsievskiy S.V c9d3652230 add zsh -h autocomplete option 2020-01-27 18:24:57 +03:00
Eugenio Lacuesta 342e3b5bd5
Merge remote-tracking branch 'upstream/master' into remove-six-code 2019-11-08 12:29:15 -03:00
seregaxvm 698aa704b9 Fix zsh completion file extension (#4122) 2019-11-05 16:30:01 +01:00
Eugenio Lacuesta 415526d922
Remove __future__ imports 2019-11-02 23:26:38 -03:00
Matsievskiy S.V f0db1b4b66 update zsh completion 2019-10-11 00:55:18 +03:00
Aditya 9dd680d5c9 Use https for external links wherever possible in docs 2017-10-26 23:33:45 +05:30
cclauss 33dfac5018 xrange() --> range() for Python 3 
Either this PR or #2845.
 2017-07-24 22:06:17 +02:00
Paul Tremberth 87472346df Update scrapy.1 2017-01-25 11:28:20 +01:00
Michael Fladischer bae12870bb Fix spelling error in scrapy.1 manpage. 
The word "intepreted" should be "interpreted".
 2017-01-24 22:20:37 +01:00
Mikhail Korobov 936266a61a Merge pull request #934 from Dineshs91/zsh-support 
[MRG+1] Added zsh completion for Scrapy command-line tool and updated bash completion
 2015-07-31 03:18:19 +05:00
agusc 140f58251b removed SUFFIX from scrapy name package 2015-06-15 17:02:09 -03:00
Julia Medina d3f576a816 Move scrapy/spider.py to scrapy/spiders/__init__.py 2015-05-09 04:20:09 -03:00
nyov 7ae37d61f9 remove scrapy deploy command 
(closes #1027, #1095, #1102)
 2015-04-20 20:35:06 -03:00
dinesh d55ae53513 Remove deploy and server commands from bash completion 2015-04-05 17:50:12 +05:30
dinesh b1247355dc Remove server command 2015-04-05 17:48:54 +05:30
dinesh 843a9cdbb5 Remove deploy command from arguments 
As scrapy deploy is being removed
 2015-04-03 21:24:31 +05:30
dinesh 7129dc36d1 Move scrapy_zsh_completion inside extras directory 2015-04-03 21:20:06 +05:30
dinesh e2b4fabbf5 Fix zsh completion path 2015-04-03 08:34:16 +05:30
dinesh 9d0ea5a396 Added zsh completion for the Scrapy command-line tool 2015-04-03 08:18:47 +05:30
Pablo Hoffman 8360380db0 removed scrapy-ws.py, moved to scrapy-jsonrpc package 2014-08-19 10:30:25 -03:00
Daniel Graña f35fac11a2 scrapy.bat is not needed anymore 2014-08-01 16:23:26 -03:00
Daniel Graña 242c08553f move tests under root dir 2014-07-30 19:45:08 -03:00
Felix Yan 6f7efa1d1d PY3: use six for robotparser and urlparse 2014-07-14 21:26:37 +08:00
Pablo Hoffman d24212c8e5 remove references to deprecated scrapy-developers list 2014-02-16 21:44:49 -02:00
Mikhail Korobov a27d91f0a6 Rename BaseSpider to Spider. See GH-495. 2013-12-30 19:46:41 +06:00
alexanderlukanin13 ae4a61a2d5 Added "from __future__ import print_function" in all relevant places except xlib 2013-10-20 11:55:07 +06:00
alexanderlukanin13 d381a35732 Python 3 compatible syntax: print, except, raise, octal numbers; removed Python 2.2 boolean compatibility code in xlib/pydispatch/dispatcher.py 2013-10-20 01:59:35 +06:00
Daniel Graña 910effd145 get scrapy version from package data 2013-02-06 11:44:26 -02:00
Daniel Graña 5db45b3825 remove scrapyd, it was migrated to its own repository 2013-02-06 05:24:07 +00:00
Daniel Graña e676ad31d0 Support sending requests trough multiples slots in QPS exmaple spider 2012-12-05 17:06:38 -02:00
Daniel Graña dcc0a540c0 QPS bench server and a useful spider spider to generate requests 2012-12-05 15:08:34 -02:00
Pablo Hoffman c3e91ba73e updated scrapy_bash_completion 2012-10-12 20:02:03 -02:00
Pablo Hoffman 452595790e fix test-scrapyd.sh after changes made to website 2012-10-08 15:30:19 -02:00
Pablo Hoffman 098d892c03 simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users 2012-08-29 15:37:05 -03:00
Pablo Hoffman 52151e8703 fixed bug in FeedExports extension, introduced in previous commit 2012-08-28 19:32:43 -03:00
Pablo Hoffman 9c3b9f2968 fixed bug in json-rpc webservice reported in https://groups.google.com/d/topic/scrapy-users/qgVBmFybNAQ/discussion. also removed no longer supported 'run' command from extras/scrapy-ws.py 2012-05-03 12:05:40 -03:00