diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index e90cb0223..6478991d6 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -234,8 +234,6 @@ HttpCacheMiddleware * :setting:`HTTPCACHE_EXPIRATION_SECS` - how many secs until the cache is considered out of date -.. _topics-dlmw-robots: - HttpCompressionMiddleware ------------------------- @@ -293,6 +291,8 @@ RetryMiddleware HTTP protocol. It's included by default because it's a common code used to indicate server overload, which would be something we want to retry. +.. _topics-dlmw-robots: + RobotsTxtMiddleware -------------------