RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,191 Commits 49 Branches 132 Tags 152 MiB
Commit Graph

11191 Commits

Author SHA1 Message Date
Andrey Rakhmatullin e7f5ae0b34
Update the outdated Item docstring. (#6427) 
* Update the outdated Item doscstring.

* Fix the reference links in items.html.
 2024-11-04 16:17:56 +05:00
Andrey Rakhmatullin ce5a132f12
Run and fix linkcheck. (#6524) 2024-11-04 11:40:07 +01:00
Rohit Kumar Singh 7701e590fb
Documentation added for Spider State in extensions.rst (#6522) 
* Documentation added for Spider State in extensions.rst

* Made correction in documentation for Spiderstate

* Added appropriate intro for Spider state extension

* Added reference for spiderstate extension

* Added Spiderstate extension hyperlink refrence in jobs.rst
 2024-11-02 10:45:27 +05:00
Andrey Rakhmatullin d85c39f5bc
Deprecation removals. (#6500) 
* Deprecation removals.

* Clean up the default pytest filterwarnings.

* Remove test_get_images_old().

* Redo boto-requiring test filtering.

* Remove an unused function.

* Improve the Crawler.crawl() error message.

* Fix the test.
 2024-10-31 18:06:22 +05:00
Laerte Pereira d2bdbad8c8
Deprecate `scrapy.twisted_version` (#6512) 
* Deprecate scrapy.twisted_version

* fix: typing

* remove typing

* raise default exception if attribute is not found

* remove redudant ()

* add tests

* rollback exception raised

* add filterwarnings again

* change order

* lint
 2024-10-30 00:28:35 +05:00
Rohitkr117 12b087b0f2
Added ignore statements for Windows specific typing issues (#6516) 2024-10-29 19:30:32 +01:00
Rohitkr117 65ecd5d528
Fixes for Twisted Version Check and Typing Issues (#6511) 2024-10-29 19:08:38 +01:00
ThunderMind 5bbf8124ac
Updated deprecated ast.NameConstant with ast.Constant #6305 (#6463) 2024-10-29 10:28:00 +01:00
Andrey Rakhmatullin fcb5ab6cff
Remove code for unsupported Twisted. (#6510) 2024-10-29 10:21:07 +01:00
Andrey Rakhmatullin 0523e1616d
Explictly set html_baseurl on RTD. (#6507) 2024-10-29 10:16:03 +01:00
Andrey Rakhmatullin b4bad97eae
Merge pull request #6166 from Gallaecio/py313 
Add Python 3.13 to CI
 2024-10-22 20:05:56 +05:00
Andrey Rakhmatullin d10c58ff38 Bump pyftpdlib to the version supporting Python 3.13 on Windows. 2024-10-22 19:07:21 +05:00
Andrey Rakhmatullin fffacb9dac Merge remote-tracking branch 'origin/master' into py313 2024-10-22 19:07:10 +05:00
Andrey Rakhmatullin 04d0411bf7
Filter test-time warnings. (#6501) 2024-10-21 12:30:49 +02:00
Andrey Rakhmatullin 6d65708cb7
Merge pull request #6499 from wRAR/3.8-cleanup 
Additional Python 3.7 and 3.8 cleanup, including Reppy support.
 2024-10-18 00:08:11 +05:00
Andrey Rakhmatullin 677e977207 Remove dead links to the Reppy doc from the release notes. 2024-10-18 00:03:32 +05:00
Andrey Rakhmatullin 5759b3f0f2 Drop Reppy. 2024-10-17 23:41:23 +05:00
Andrey Rakhmatullin 7e07d48cc5 Small 3.7 and 3.8 cleanup. 2024-10-17 23:22:37 +05:00
Andrey Rakhmatullin 1138a5cf99
Merge pull request #6498 from wRAR/remove-runtime-typing 
Remove --keep-runtime-typing from pyupgrade.
 2024-10-17 23:18:22 +05:00
Andrey Rakhmatullin 7196a11f53 Reorder unions with None. 2024-10-17 21:52:03 +05:00
Andrey Rakhmatullin c9095ef927 Remove --keep-runtime-typing from pyupgrade. 2024-10-17 21:26:02 +05:00
Julian Ste c8e87ab21a
Fixed typos (#6497) 2024-10-17 17:03:16 +02:00
Andrey Rakhmatullin f65e64a724
Misc typing improvements. (#6494) 2024-10-16 18:38:43 +02:00
Andrey Rakhmatullin 9bd5e5bcdb Revert uvloop restrictions. 2024-10-16 14:50:57 +05:00
Andrey Rakhmatullin 845b1ffd44 Merge remote-tracking branch 'origin/master' into py313 2024-10-16 14:50:12 +05:00
Vsevolod Breus 5391663072
Drop Python 3.8 Support (#6472) 2024-10-16 10:03:16 +02:00
Andrey Rakhmatullin 9736e49b52
Merge pull request #6489 from rettinghaus/pypa_action 
update gh-action-pypi-publish
 2024-10-11 00:24:31 +05:00
Klaus Rettinghaus 5ef5474172
update gh-action-pypi-publish 2024-10-09 20:38:50 +02:00
Andrey Rakhmatullin 7ec6b7e65b
Merge pull request #6487 from wRAR/drop-setuptools-dep 
Remove the installation dependency on setuptools.
 2024-10-08 16:34:11 +05:00
Andrey Rakhmatullin 87651fdf47 Don't use types-setuptools. 2024-10-08 16:04:26 +05:00
Andrey Rakhmatullin 29bb869284 Remove the beta block. 2024-10-08 15:43:17 +05:00
Andrey Rakhmatullin df6c51af0f Use the 3.13 release. 2024-10-08 15:37:49 +05:00
Andrey Rakhmatullin 8c133fcf7e Remove the installation dependency on setuptools. 2024-10-07 23:04:48 +05:00
mmoriniere 46cddc6ecf
Ignore SyntaxError as well when SPIDER_LOADER_WARN_ONLY is set to True (#6484) 2024-10-02 10:04:03 +02:00
Andrey Rakhmatullin e139d22db9 Fix expectations for get_func_args() on 3.13. 2024-09-20 19:28:28 +05:00
Andrey Rakhmatullin ee9ee2d12d Revert bpython restrictions. 2024-09-12 14:32:32 +05:00
Andrey Rakhmatullin b3f562d6a5 Revert "Revert uvloop restrictions." 
This reverts commit c21c4a1850.
 2024-09-12 14:31:50 +05:00
Andrey Rakhmatullin ae967d1c06
Merge pull request #6475 from LucasSD/remove-debug-log-message 
Remove log pollution from core.engine signal handler (#6433)
 2024-09-10 12:01:37 +05:00
LucasSD f260f819e0 Remove debug log message from _schedule_request method 2024-09-09 20:26:02 +01:00
Daniel O'Connor 67ab8d4650
Refactor genspider slightly so template variables can be overridden (#6470) 2024-08-28 21:07:49 +02:00
Andrey Rakhmatullin c9d85faaf2
Merge pull request #6469 from Laerte/master 
Add support for meta in Spider Contracts
 2024-08-28 14:53:39 +05:00
Laerte Pereira ddbdfeb699
Update scrapy/contracts/default.py 2024-08-27 07:24:57 -03:00
Laerte Pereira 4f9b2343c0 Merge branch 'master' of github.com:Laerte/scrapy 2024-08-27 07:16:10 -03:00
Laerte Pereira 3c2a9fa262 update docs 2024-08-27 07:16:01 -03:00
Laerte Pereira f68f29dd13
Update docs/topics/contracts.rst 2024-08-26 23:37:57 -03:00
Laerte Pereira b85e5a66ed Add support for meta in Spider Contracts 2024-08-26 23:21:09 -03:00
Georgiy Zatserklianyi 6ce0342beb
Allow yielding items from start_requests (#6417) 
Co-authored-by: Georgiy Zatserklianyi <george.zatseklyany@gmail.com>
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-08-26 20:53:06 +02:00
Andrey Rakhmatullin 5794071f96
Typing fixes and updates. (#6460) 2024-08-23 12:48:01 +02:00
Andrey Rakhmatullin c21c4a1850 Revert uvloop restrictions. 2024-08-21 01:06:05 +05:00
mlmsmith af15bd1dad
minor changes to items section of docs (#6462) 2024-08-19 13:55:09 +02:00