scrapy/tests/test_utils_misc
JSap0914 b7824db573
Fix rel_has_nofollow to be case-insensitive per HTML spec (#7632)
The HTML specification states that link type keywords like 'nofollow'
are ASCII case-insensitive. Sites using rel="NoFollow" or rel="NOFOLLOW"
were incorrectly treated as follow links, causing Scrapy to crawl pages
it should skip.

Fix: add .lower() before the token split so all casing variants of
'nofollow' are correctly recognized.

Co-authored-by: JSap0914 <JSap0914@users.noreply.github.com>
2026-06-17 14:48:43 +05:00
..
test_walk_modules move tests under root dir 2014-07-30 19:45:08 -03:00
__init__.py Fix rel_has_nofollow to be case-insensitive per HTML spec (#7632) 2026-06-17 14:48:43 +05:00
test.egg move tests under root dir 2014-07-30 19:45:08 -03:00
test_return_with_argument_inside_generator.py Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00