RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,208 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

56 Commits

Author SHA1 Message Date
Adrian dd10cb8e9a
LxmlLinkExtractor: add deny_attrs and deny_tags (#7679) 2026-07-05 16:46:22 +05:00
Adrian 699c93f6b2
Complete test coverage for linkextractors (#7639) 
* Complete test coverage for linkextractors

* pylint: disable use-implicit-booleaness-not-comparison
 2026-06-19 14:22:03 +05:00
Andrey Rakhmatullin eaae59fbef
Fix invalid XML samples in tests that break with lxml 6. (#6919) 2025-06-27 13:05:02 +02:00
Andrey Rakhmatullin 02ed71d887
Converting tests to plain asserts, part 6. (#6709) 2025-03-09 20:20:24 +01:00
Andrey Rakhmatullin a898331d14
Preparations for switching to direct pytest. (#6678) 2025-02-16 20:28:58 +01:00
Andrey Rakhmatullin b042ad255d
Address some previously ignored pylint messages. (#6531) 2024-11-11 11:49:52 +01:00
Andrey Rakhmatullin c9095ef927 Remove --keep-runtime-typing from pyupgrade. 2024-10-17 21:26:02 +05:00
Andrey Rakhmatullin 1ef9c337ca flake8-docstrings 2024-06-19 23:57:40 +05:00
Andrey Rakhmatullin a5da77d01d
Merge pull request #6221 from jxlil/fix/LxmlLinkExtractor 
fix: LxmlLinkExtractor unique_list missing key
 2024-04-19 19:57:11 +05:00
Laerte Pereira 26a16f2c43 Fix tests 2024-02-28 16:36:19 -03:00
Andrey Rakhmatullin 4f9dd998dc Bump pylint, cleanup the ignored tags. 2024-02-28 14:31:21 +05:00
Elias Ram f19045403a test #8 made tests cleaner 2024-02-21 16:03:51 +01:00
Elias Ram e8e6d28479 test #8 added tests for LxmlLinkExtractor 2024-02-20 20:41:18 +01:00
Adrián Chaves 5e51417a48 Add tests, fix canonicalize passing 2024-02-16 20:10:52 +01:00
Andrey Rakhmatullin b4acf5c827
Fix and remove most of the entries from the mypy ignore list (#6137) 2023-11-07 09:34:35 +01:00
Jeesang Kim dda6feb935
Improve assert readability (#6132) 2023-11-03 09:24:25 +01:00
Adrián Chaves 3f0c2fae5e Skip test_skip_bad_links based on the w3lib version 2023-04-12 09:28:28 +02:00
Samuel Bartlett 618e82dbe1 Exclude test for python versionbs less than 3.8 2023-03-31 15:12:47 +00:00
Samuel Bartlett 608b7de582 Skip new test if python version less than 3.8 2023-03-31 14:38:06 +00:00
Samuel Bartlett 4043560547 remove utf-8 encoding flag from test 2023-03-31 12:29:22 +00:00
Samuel Bartlett 00d93026c8 Fix bad test case 2023-03-31 08:30:19 +00:00
Samuel Bartlett 7cb7cf1ad1 Add link extractor back to test 2023-03-31 08:09:02 +00:00
Samuel Bartlett 9cbcf7724d Add test to make sure spider doesn't crash on bad 2023-03-31 08:07:43 +00:00
Emmanuel Rondan 8ee4817471 reaplying black, fixing conflicts and ignoring bandit checks on test directory 2023-01-20 10:55:46 -03:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Laerte Pereira 1a6408c3fa
Remove `FilteringLinkExtractor` 2022-11-21 07:30:20 -03:00
Marc Hernández 336f19f5cc
Change super syntax (#4707) 2020-08-04 20:42:01 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Eugenio Lacuesta 3eeecb4273
Merge branch 'master' into flake8-max-line-length 2020-05-15 11:11:20 -03:00
Eugenio Lacuesta be39f274b8
Merge branch 'master' into flake8-remove-e128 2020-05-08 18:33:29 -03:00
Eugenio Lacuesta 3ebf2a0d82
Remove lambdas in link extractor 2020-05-08 15:24:37 -03:00
Eugenio Lacuesta bf56517abf
Flake8: Remove E128 (tests, part 2) 2020-05-06 19:18:29 -03:00
Eugenio Lacuesta cc23d1cb58
Flake8: Remove E124 2020-05-06 15:40:37 -03:00
Eugenio Lacuesta 182394bcec
E501 compliance (tests) 2020-03-22 23:29:30 -03:00
Adrián Chaves b4958358e8 Update tests to account for link extractors escaping spaces 2020-02-12 19:00:04 +01:00
Adrián Chaves 35723d76c0 Use canonicalize_url in link extraction 2020-02-07 22:59:53 +01:00
Eugenio Lacuesta 2e405d2d5c
Merge branch 'master' into response_follow_all 2020-01-05 00:33:19 -03:00
Adrián Chaves 174769a3f0 Use a better name for the LxmlLinkExtractor subclassing test 2019-12-18 12:09:03 +01:00
Adrián Chaves a4ef9750f9 Fix Flake8-reported issues 2019-12-13 14:32:06 +01:00
Adrián Chaves 1fc2b140c1
Merge branch 'master' into documentation-coverage 2019-12-05 14:43:36 +01:00
Eugenio Lacuesta 6781d2f5b2
Update sample file references 2019-11-20 09:58:25 -03:00
Adrián Chaves 33ef24c757 Add missing whitespace after ‘,’, ‘;’ or ‘:’ 2019-11-13 10:52:05 +01:00
Adrián Chaves 7f4f98fd38 Provide complete API documentation coverage of scrapy.linkextractors 2019-09-30 18:22:28 +02:00
Eugenio Lacuesta 40086dabb8 Prevent more DeprecationWarnings 2019-07-13 22:14:00 -03:00
Matthieu Grandrie e3b15252c8 New constructor arg *restrict_text* for FilteringLinkExtractor. 
Same as allow and deny args, it holds a string, a regex or an iterable of. Links whose text don't match one of the regex are filtered out.
DOC restrict_text in LxmlLinkExtractor
 2019-02-28 17:21:17 +01:00
nctl144 4c05441450 add ftp to the scheme list 2018-03-03 00:00:03 -05:00
Mikhail Korobov 2b4d46315f TST fixed compatibility with new link extractor whitespace handling 2017-02-21 00:05:40 +05:00
Mikhail Korobov 47f7da8724 canonicalize=False by default for LinkExtractor. Fixes GH-1941. 2017-02-20 22:58:11 +05:00
Mikhail Korobov d079e15fe2 Strip leading/trailing whitespaces in link extractors. Fixes GH-838. 2017-02-16 02:22:17 +05:00
Mikhail Korobov 44bfcbcf0f TST split LinkExtractorTestCase.test_extraction into several methods; remove duplicated test 2015-08-31 00:49:38 +05:00