RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

added note about how to revert to old behavior for robotstxt

This commit is contained in:
Elias Dorneles 2016-02-03 20:51:26 -02:00
parent 89a088b3c3
commit 2f697ce59f
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
docs/news.rst
View File

@ -32,7 +32,8 @@ This 1.1 release brings a lot of interesting features and bug fixes:
If you try ``scrapy shell index.html`` it will try to load the URL http://index.html,
use ``scrapy shell ./index.html`` to load a local file.
- Robots.txt compliance is now enabled by default for newly-created projects
(:issue:`1724`).
(:issue:`1724`). If you need old behavior, update :setting:`ROBOTSTXT_OBEY`
in ``settings.py`` file when creating a new project.
- Exporters now work on unicode, instead of bytes by default (:issue:`1080`).
If you use ``PythonItemExporter``, you may want to update your code to
disable binary mode which is now deprecated.