diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst index 918f1cc36..9d4ed6ca6 100644 --- a/docs/topics/spiders.rst +++ b/docs/topics/spiders.rst @@ -703,6 +703,16 @@ SitemapSpider This would retrieve only ``entries`` modified on 2005 and the following years. + Entries are dict objects extracted from the sitemap document. + Usually, the key is the tag name and the value is the text inside it. + + It's important to notice that: + + - as the loc attribute is required, entries without this tag are discarded + - alternate links are stored in a list with the key ``alternate`` + (see ``sitemap_alternate_links``) + - namespaces are removed, so lxml tags named as ``{foo}bar`` become only ``bar`` + If you omit this method, all entries found in sitemaps will be processed, observing other attributes and their settings.