diff --git a/docs/topics/media-pipeline.rst b/docs/topics/media-pipeline.rst index 381a2988a..0ce431ff5 100644 --- a/docs/topics/media-pipeline.rst +++ b/docs/topics/media-pipeline.rst @@ -392,6 +392,36 @@ See here the methods that you can override in your custom Files Pipeline: .. class:: FilesPipeline + .. method:: file_path(request, response, info) + + This method is called once per downloaded item. It returns the + download path of the file originating from the specified + :class:`response `. + + In addition to ``response``, this method receives the original + :class:`request ` and + :class:`info `. + + You can override this method to customize the download path of each file. + + For example, if file URLs end like regular paths (e.g. + ``https://example.com/a/b/c/foo.png``), you can use the following + approach to download all files into the ``files`` folder with their + original filenames (e.g. ``files/foo.png``):: + + import os + from urllib.parse import urlparse + + from scrapy.pipelines.files import FilesPipeline + + class MyFilesPipeline(FilesPipeline): + + def file_path(self, request, response, info): + return 'files/' + os.path.basename(urlparse(request.url).path) + + By default the :meth:`file_path` method returns + ``full/.``. + .. method:: FilesPipeline.get_media_requests(item, info) As seen on the workflow, the pipeline will get the URLs of the images to @@ -475,6 +505,36 @@ See here the methods that you can override in your custom Images Pipeline: The :class:`ImagesPipeline` is an extension of the :class:`FilesPipeline`, customizing the field names and adding custom behavior for images. + .. method:: file_path(request, response, info) + + This method is called once per downloaded item. It returns the + download path of the file originating from the specified + :class:`response `. + + In addition to ``response``, this method receives the original + :class:`request ` and + :class:`info `. + + You can override this method to customize the download path of each file. + + For example, if file URLs end like regular paths (e.g. + ``https://example.com/a/b/c/foo.png``), you can use the following + approach to download all files into the ``files`` folder with their + original filenames (e.g. ``files/foo.png``):: + + import os + from urllib.parse import urlparse + + from scrapy.pipelines.images import ImagesPipeline + + class MyImagesPipeline(ImagesPipeline): + + def file_path(self, request, response, info): + return 'files/' + os.path.basename(urlparse(request.url).path) + + By default the :meth:`file_path` method returns + ``full/.``. + .. method:: ImagesPipeline.get_media_requests(item, info) Works the same way as :meth:`FilesPipeline.get_media_requests` method,