diff --git a/docs/topics/media-pipeline.rst b/docs/topics/media-pipeline.rst index a34f4c053..3da243d29 100644 --- a/docs/topics/media-pipeline.rst +++ b/docs/topics/media-pipeline.rst @@ -77,30 +77,6 @@ PIL. .. _Python Imaging Library: http://www.pythonware.com/products/pil/ -Usage example -============= - -In order to use a media pipeline first, :ref:`enable it -`. - -Then, if a spider returns a dict with the URLs key ('file_urls' or -'image_urls', for the Files or Images Pipeline respectively), the pipeline will -put the results under respective key ('files' or images'). - -If you prefer to use :class:`~.Item`, then define a custom item with the -necessary fields, like in this example for Images Pipeline:: - - import scrapy - - class MyItem(scrapy.Item): - - # ... other item fields ... - image_urls = scrapy.Field() - images = scrapy.Field() - -If you need something more complex and want to override the custom pipeline -behaviour, see :ref:`topics-media-pipeline-override`. - .. _topics-media-pipeline-enabling: Enabling your Media Pipeline @@ -171,6 +147,51 @@ Where: used). For more info see :ref:`topics-images-thumbnails`. +Usage example +============= + +.. setting:: FILES_URLS_FIELD +.. setting:: FILES_RESULT_FIELD +.. setting:: IMAGES_URLS_FIELD +.. setting:: IMAGES_RESULT_FIELD + +In order to use a media pipeline first, :ref:`enable it +`. + +Then, if a spider returns a dict with the URLs key (``file_urls`` or +``image_urls``, for the Files or Images Pipeline respectively), the pipeline will +put the results under respective key (``files`` or ``images``). + +If you prefer to use :class:`~.Item`, then define a custom item with the +necessary fields, like in this example for Images Pipeline:: + + import scrapy + + class MyItem(scrapy.Item): + + # ... other item fields ... + image_urls = scrapy.Field() + images = scrapy.Field() + +If you want to use another field name for the URLs key or for the results key, +it is also possible to override it. + +For the Files Pipeline, set :setting:`FILES_URLS_FIELD` and/or +:setting:`FILES_RESULT_FIELD` settings:: + + FILES_URLS_FIELD = 'field_name_for_your_files_urls' + FILES_RESULT_FIELD = 'field_name_for_your_processed_files' + +For the Images Pipeline, set :setting:`IMAGES_URLS_FIELD` and/or +:setting:`IMAGES_RESULT_FIELD` settings:: + + IMAGES_URLS_FIELD = 'field_name_for_your_images_urls' + IMAGES_RESULT_FIELD = 'field_name_for_your_processed_images' + +If you need something more complex and want to override the custom pipeline +behaviour, see :ref:`topics-media-pipeline-override`. + + Additional features =================== @@ -185,12 +206,14 @@ adjust this retention delay use the :setting:`FILES_EXPIRES` setting (or :setting:`IMAGES_EXPIRES`, in case of Images Pipeline), which specifies the delay in number of days:: - # 90 days of delay for files expiration - FILES_EXPIRES = 90 + # 120 days of delay for files expiration + FILES_EXPIRES = 120 # 30 days of delay for images expiration IMAGES_EXPIRES = 30 +The default value for both settings is 90 days. + .. _topics-images-thumbnails: Thumbnail generation for images @@ -249,7 +272,13 @@ For example:: IMAGES_MIN_HEIGHT = 110 IMAGES_MIN_WIDTH = 110 -Note: these size constraints don't affect thumbnail generation at all. +.. note:: + The size constraints don't affect thumbnail generation at all. + +It is possible to set just one size constraint or both. When setting both of +them, only images that satisfy both minimum sizes will be saved. For the +above example, images of sizes (105 x 105) or (105 x 200) or (200 x 105) will +all be dropped because at least one dimension is shorter than the constraint. By default, there are no size constraints, so all images are processed.