RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,315 Commits 49 Branches 132 Tags 187 MiB
Commit Graph

10315 Commits

Author SHA1 Message Date
Can Kupeli c513e7d6e5 implemented tests for branch coverage for function printSummary 2024-02-21 23:52:35 +01:00
Andrey Rakhmatullin f798118ac2
Merge pull request #6232 from elramen/linkextractor_tests 
Added tests for LxmlLinkExtractor
 2024-02-21 23:03:20 +05:00
Elias Ram f19045403a test #8 made tests cleaner 2024-02-21 16:03:51 +01:00
Adrián Chaves 6fc7827042
Do not close the underlying file from compression plugins (#6239) 2024-02-21 14:27:42 +01:00
Elias Ram e8e6d28479 test #8 added tests for LxmlLinkExtractor 2024-02-20 20:41:18 +01:00
Laerte Pereira ee1189512f
Replace urlparse with urlparse_cached where possible (#6229) 2024-02-20 12:47:29 +01:00
Mikhail Korobov c4e4b9b56e
Add a SECURITY.md file (#6051) 2024-02-20 10:50:16 +01:00
Andrey Rakhmatullin ba8993ec09
Merge pull request #6223 from jxlil/docs/remove_external_docs 
Fix: Remove docs for other projects from the Scrapy docs
 2024-02-16 23:59:10 +04:00
Jalil SA 36f72877ba update: docs/topics/selectors.rst 2024-02-16 10:39:16 -06:00
Kevin Lloyd Bernal 9bb973dc54
Refactor LogStats extension to log IPM and RPM to stats on spider_close (#4111) 2024-02-16 12:25:38 +01:00
Jalil SA 3e7b704c08 update: docs/topics/selectors.rst 2024-02-15 16:59:32 -06:00
Jalil SA 660e3b1953 update: docs/topics/items.rst 2024-02-15 16:59:31 -06:00
Mikhail Korobov c5fdba9b31
Merge pull request #6222 from Gallaecio/2.11-port 
2.11 port
 2024-02-15 19:26:40 +05:00
Adrián Chaves 6bd45bb6d9 Merge remote-tracking branch 'scrapy/2.11' into 2.11-port 2024-02-15 06:39:50 +01:00
Adrián Chaves 2f1d345e74 Solve test issues 2024-02-14 18:59:01 +01:00
Adrián Chaves 502addc717 Bump version: 2.11.0 → 2.11.1 2024-02-14 18:17:48 +01:00
Adrián Chaves 6b88b3346c Set the release date of versions 2.11.1 and 1.8.4 2024-02-14 18:16:40 +01:00
Adrián Chaves 479619b340 Merge branch '2.11-redos' into 2.11 2024-02-14 17:18:26 +01:00
Adrián Chaves 809bfac489 Merge branch '2.11-compression-bomb' into 2.11 2024-02-14 17:16:25 +01:00
Adrián Chaves 5bcb8fd501 Merge branch '2.11-authorization' into 2.11 2024-02-14 17:14:29 +01:00
Andrey Rakhmatullin a55e933c11
Release notes for 2.11.1 (#6150) 2024-02-14 17:08:40 +01:00
Andy 1c9d308acc
Cover the deprecation and removal of response_httprepr in the release notes (#6216) 2024-02-05 12:52:01 +01:00
Adrián Chaves 5e5a92026e Remove slow leftovers 2024-02-02 14:06:45 +01:00
Adrián Chaves 810aaa637d Undo an unintended change 2024-02-02 14:04:28 +01:00
Adrián Chaves c5dad41190 Speed up tests, remove comments without regexps 2024-02-02 14:03:16 +01:00
Andrey Rakhmatullin 6f73dc0e67
Merge pull request #6213 from Laerte/master 
Cleanup deprecated fingerprint code in scrapy.utils.request
 2024-01-31 21:00:48 +04:00
Laerte Pereira 53ccf0016d Remove empty statement 2024-01-31 13:18:10 -03:00
Laerte Pereira 7001193c80 Simplify the logic 2024-01-31 12:53:08 -03:00
Laerte Pereira 24634f1bb2 Attend PR comments 2024-01-31 12:29:43 -03:00
Laerte Pereira bacaf0db7a Update documentation 2024-01-31 10:14:48 -03:00
Laerte Pereira 019443dd57 Remove settings from default implementation test 2024-01-31 10:08:07 -03:00
Laerte Pereira 2487e3cc03 Cleanup deprecated fingerprint code in scrapy.utils.request 2024-01-31 10:05:50 -03:00
Andrey Rakhmatullin 4229048255
Merge pull request #6208 from kevintomsgithub/add-faq 
Add FAQ on making a blank request
 2024-01-18 21:56:12 +04:00
Kevin Toms 9074c16497 make suggestion 2024-01-18 09:36:25 -05:00
Kevin Toms 46f94ec9cb Fix test 
Wrap the yield line in a function to prevent throwing error when the code snippet is executed
 2024-01-17 15:49:51 -05:00
Kevin Toms 88285e75b6 Add FAQ on making a blank request 2024-01-17 10:05:22 -05:00
Andrey Rakhmatullin 09a7efef7c Remove a defer.returnValue call. 2024-01-15 18:19:29 +04:00
Rotzbua d5233bb57f
chore(docs): update `sphinx` dependencies (#6200) 2024-01-15 14:11:33 +01:00
Adrián Chaves e8dadb9592
scrapy parse: fix the signature of callbacks from the CLI (#6182) 2024-01-15 13:37:03 +01:00
Adrián Chaves fa0c598096
Add component getters to Crawler (#6181) 2024-01-15 13:14:02 +01:00
Mikhail Korobov 8ad17f7476
Merge pull request #6199 from Rotzbua/fix_flake8 
fix(flake8): lint errors
 2024-01-12 22:52:22 +05:00
Mikhail Korobov b7cf30a48e
Merge pull request #6205 from wRAR/fix-pypy-tests 
Remove a defer.returnValue call.
 2024-01-12 22:50:41 +05:00
Andrey Rakhmatullin c2baf4d0da Remove a defer.returnValue call. 2024-01-12 18:30:41 +04:00
Andrey Rakhmatullin becff65ccb
Merge pull request #6201 from wRAR/more-mitmproxy 
Fix and re-enable newer mitmproxy usage in tests.
 2024-01-12 14:26:39 +04:00
Mikhail Korobov 32fda2dc53
Merge pull request #6196 from Rotzbua/fix_typo 
fix(typo): correct `successfully`
 2024-01-12 00:36:16 +05:00
Mikhail Korobov 7eeca55c70
Merge pull request #6197 from Rotzbua/refactor_yield 
refactor(yield): use `yield from` syntax
 2024-01-12 00:35:56 +05:00
Mikhail Korobov 825137399d
Merge pull request #6198 from Rotzbua/refactor_exception 
refactor(): use `OSError` exception
 2024-01-12 00:32:51 +05:00
Andrey Rakhmatullin 68fccb1d58 Fix and re-enable newer mitmproxy usage in tests. 2024-01-06 01:35:56 +04:00
Rotzbua 745b8412f6
fix(flake8): lint errors 
E226 missing whitespace around arithmetic operator
E201 whitespace after '{'
 2024-01-05 14:54:00 +01:00
Rotzbua 42c481cb4a
refactor(): use `OSError` exception 
https://docs.astral.sh/ruff/rules/os-error-alias/
 2024-01-05 12:36:36 +01:00