RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

add a remark about 1%

This commit is contained in:
Mikhail Korobov 2013-12-20 23:12:37 +06:00
parent ee46ec8920
commit e0cebbfc8f
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/topics/broad-crawls.rst
View File

@ -122,8 +122,9 @@ To disable redirects use::
Enable crawling of "Ajax Crawlable Pages"
=========================================
Some pages (up to 1%) declare themselves as `ajax crawlable`_. This means they
provide plain HTML version of content that is usually available only via AJAX.
Some pages (up to 1%, based on empirical data from year 2013) declare
themselves as `ajax crawlable`_. This means they provide plain HTML
version of content that is usually available only via AJAX.
Pages can indicate it in two ways:
1) by using ``#!`` in URL - this is the default way;