RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
scrapy/tests/test_utils_misc
History
JSap0914 75cc7b8daa Fix rel_has_nofollow to be case-insensitive per HTML spec 
The HTML specification states that link type keywords like 'nofollow'
are ASCII case-insensitive. Sites using rel="NoFollow" or rel="NOFOLLOW"
were incorrectly treated as follow links, causing Scrapy to crawl pages
it should skip.

Fix: add .lower() before the token split so all casing variants of
'nofollow' are correctly recognized.
 		2026-06-17 15:18:07 +09:00
..
test_walk_modules move tests under root dir 2014-07-30 19:45:08 -03:00
__init__.py Fix rel_has_nofollow to be case-insensitive per HTML spec 2026-06-17 15:18:07 +09:00
test.egg move tests under root dir 2014-07-30 19:45:08 -03:00
test_return_with_argument_inside_generator.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00