mirror of https://github.com/scrapy/scrapy.git
Use https for external links wherever possible in docs
This commit is contained in:
Aditya
parent
dc4b36a89e
commit
9dd680d5c9
|
|
@ -7,7 +7,7 @@ Scrapy
|
|||
:alt: PyPI Version
|
||||
|
||||
.. image:: https://img.shields.io/travis/scrapy/scrapy/master.svg
|
||||
:target: http://travis-ci.org/scrapy/scrapy
|
||||
:target: https://travis-ci.org/scrapy/scrapy
|
||||
:alt: Build Status
|
||||
|
||||
.. image:: https://img.shields.io/badge/wheel-yes-brightgreen.svg
|
||||
|
|
@ -15,7 +15,7 @@ Scrapy
|
|||
:alt: Wheel Status
|
||||
|
||||
.. image:: https://img.shields.io/codecov/c/github/scrapy/scrapy/master.svg
|
||||
:target: http://codecov.io/github/scrapy/scrapy?branch=master
|
||||
:target: https://codecov.io/github/scrapy/scrapy?branch=master
|
||||
:alt: Coverage report
|
||||
|
||||
.. image:: https://anaconda.org/conda-forge/scrapy/badges/version.svg
|
||||
|
|
|
|||
|
|
@ -10,10 +10,10 @@ scrapy-logo.jpg
|
|||
|
||||
Main Scrapy logo, in JPEG format.
|
||||
|
||||
qlassik.zip
|
||||
qlassik.zip
|
||||
-----------
|
||||
|
||||
Font used for Scrapy logo. Homepage: http://www.dafont.com/qlassik.font
|
||||
Font used for Scrapy logo. Homepage: https://www.dafont.com/qlassik.font
|
||||
|
||||
scrapy-blog.logo.xcf
|
||||
--------------------
|
||||
|
|
|
|||
|
|
@ -188,7 +188,7 @@ And their unit-tests are in::
|
|||
|
||||
.. _issue tracker: https://github.com/scrapy/scrapy/issues
|
||||
.. _scrapy-users: https://groups.google.com/forum/#!forum/scrapy-users
|
||||
.. _Scrapy subreddit: http://reddit.com/r/scrapy
|
||||
.. _Scrapy subreddit: https://reddit.com/r/scrapy
|
||||
.. _Twisted unit-testing framework: https://twistedmatrix.com/documents/current/core/development/policy/test-standard.html
|
||||
.. _AUTHORS: https://github.com/scrapy/scrapy/blob/master/AUTHORS
|
||||
.. _tests/: https://github.com/scrapy/scrapy/tree/master/tests
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ Python code.
|
|||
In other words, comparing `BeautifulSoup`_ (or `lxml`_) to Scrapy is like
|
||||
comparing `jinja2`_ to `Django`_.
|
||||
|
||||
.. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
|
||||
.. _BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/
|
||||
.. _lxml: http://lxml.de/
|
||||
.. _jinja2: http://jinja.pocoo.org/
|
||||
.. _Django: https://www.djangoproject.com/
|
||||
|
|
|
|||
|
|
@ -188,7 +188,7 @@ solutions:
|
|||
that doesn't conflict with the rest of your system. Here's how to do it using
|
||||
the `homebrew`_ package manager:
|
||||
|
||||
* Install `homebrew`_ following the instructions in http://brew.sh/
|
||||
* Install `homebrew`_ following the instructions in https://brew.sh/
|
||||
|
||||
* Update your ``PATH`` variable to state that homebrew packages should be
|
||||
used before system packages (Change ``.bashrc`` to ``.zshrc`` accordantly
|
||||
|
|
@ -233,9 +233,9 @@ After any of these workarounds you should be able to install Scrapy::
|
|||
.. _pyOpenSSL: https://pypi.python.org/pypi/pyOpenSSL
|
||||
.. _setuptools: https://pypi.python.org/pypi/setuptools
|
||||
.. _AUR Scrapy package: https://aur.archlinux.org/packages/scrapy/
|
||||
.. _homebrew: http://brew.sh/
|
||||
.. _zsh: http://www.zsh.org/
|
||||
.. _homebrew: https://brew.sh/
|
||||
.. _zsh: https://www.zsh.org/
|
||||
.. _Scrapinghub: https://scrapinghub.com
|
||||
.. _Anaconda: http://docs.continuum.io/anaconda/index
|
||||
.. _Miniconda: http://conda.pydata.org/docs/install/quick.html
|
||||
.. _conda-forge: https://conda-forge.github.io/
|
||||
.. _Anaconda: https://docs.anaconda.com/anaconda/
|
||||
.. _Miniconda: https://conda.io/docs/user-guide/install/index.html
|
||||
.. _conda-forge: https://conda-forge.org/
|
||||
|
|
|
|||
|
|
@ -164,4 +164,4 @@ interest!
|
|||
.. _web scraping: https://en.wikipedia.org/wiki/Web_scraping
|
||||
.. _Amazon Associates Web Services: https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html
|
||||
.. _Amazon S3: https://aws.amazon.com/s3/
|
||||
.. _Sitemaps: http://www.sitemaps.org
|
||||
.. _Sitemaps: https://www.sitemaps.org/index.html
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ list of Python resources for non-programmers`_.
|
|||
.. _this list of Python resources for non-programmers: https://wiki.python.org/moin/BeginnersGuide/NonProgrammers
|
||||
.. _Dive Into Python 3: http://www.diveintopython3.net
|
||||
.. _Python Tutorial: https://docs.python.org/3/tutorial
|
||||
.. _Learn Python The Hard Way: http://learnpythonthehardway.org/book/
|
||||
.. _Learn Python The Hard Way: https://learnpythonthehardway.org/book/
|
||||
|
||||
|
||||
Creating a project
|
||||
|
|
|
|||
|
|
@ -142,4 +142,4 @@ available in all future runs should they be necessary again::
|
|||
|
||||
For more information, check the :ref:`topics-logging` section.
|
||||
|
||||
.. _base tag: http://www.w3schools.com/tags/tag_base.asp
|
||||
.. _base tag: https://www.w3schools.com/tags/tag_base.asp
|
||||
|
|
|
|||
|
|
@ -55,5 +55,5 @@ just like ``scrapyd-deploy``.
|
|||
.. _scrapyd-client: https://github.com/scrapy/scrapyd-client
|
||||
.. _shub: https://doc.scrapinghub.com/shub.html
|
||||
.. _scrapyd-deploy documentation: https://scrapyd.readthedocs.io/en/latest/deploy.html
|
||||
.. _Scrapy Cloud documentation: http://doc.scrapinghub.com/scrapy-cloud.html
|
||||
.. _Scrapy Cloud documentation: https://doc.scrapinghub.com/scrapy-cloud.html
|
||||
.. _Scrapinghub: https://scrapinghub.com/
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ In this example, we'll show how to use `Firebug`_ to scrape data from the
|
|||
Project`_ used in the :ref:`tutorial <intro-tutorial>` but with a different
|
||||
face.
|
||||
|
||||
.. _Firebug: http://getfirebug.com
|
||||
.. _Firebug: https://getfirebug.com/
|
||||
.. _Google Directory: http://directory.google.com/
|
||||
.. _Open Directory Project: http://www.dmoz.org
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ when inspecting the page source is not the original HTML, but a modified one
|
|||
after applying some browser clean up and executing Javascript code. Firefox,
|
||||
in particular, is known for adding ``<tbody>`` elements to tables. Scrapy, on
|
||||
the other hand, does not modify the original page HTML, so you won't be able to
|
||||
extract any data if you use ``<tbody>`` in your XPath expressions.
|
||||
extract any data if you use ``<tbody>`` in your XPath expressions.
|
||||
|
||||
Therefore, you should keep in mind the following things when working with
|
||||
Firefox and XPath:
|
||||
|
|
@ -71,11 +71,11 @@ Firecookie
|
|||
|
||||
`Firecookie`_ makes it easier to view and manage cookies. You can use this
|
||||
extension to create a new cookie, delete existing cookies, see a list of cookies
|
||||
for the current site, manage cookies permissions and a lot more.
|
||||
for the current site, manage cookies permissions and a lot more.
|
||||
|
||||
.. _Firebug: http://getfirebug.com
|
||||
.. _Firebug: https://getfirebug.com/
|
||||
.. _Inspect Element: https://www.youtube.com/watch?v=-pT_pDe54aA
|
||||
.. _XPather: https://addons.mozilla.org/en-US/firefox/addon/xpather/
|
||||
.. _XPather: https://addons.mozilla.org/en-US/firefox/addon/xpather/
|
||||
.. _XPath Checker: https://addons.mozilla.org/en-US/firefox/addon/xpath-checker/
|
||||
.. _Tamper Data: https://addons.mozilla.org/en-US/firefox/addon/tamper-data/
|
||||
.. _Firecookie: https://addons.mozilla.org/en-US/firefox/addon/firecookie/
|
||||
|
|
|
|||
|
|
@ -100,4 +100,4 @@ If you wish to log the requests that couldn't be serialized, you can set the
|
|||
:setting:`SCHEDULER_DEBUG` setting to ``True`` in the project's settings page.
|
||||
It is ``False`` by default.
|
||||
|
||||
.. _pickle: http://docs.python.org/library/pickle.html
|
||||
.. _pickle: https://docs.python.org/library/pickle.html
|
||||
|
|
|
|||
|
|
@ -518,8 +518,8 @@ a footer of a page that looks something like:
|
|||
Example::
|
||||
|
||||
<footer>
|
||||
<a class="social" href="http://facebook.com/whatever">Like Us</a>
|
||||
<a class="social" href="http://twitter.com/whatever">Follow Us</a>
|
||||
<a class="social" href="https://facebook.com/whatever">Like Us</a>
|
||||
<a class="social" href="https://twitter.com/whatever">Follow Us</a>
|
||||
<a class="email" href="mailto:whatever@example.com">Email Us</a>
|
||||
</footer>
|
||||
|
||||
|
|
|
|||
|
|
@ -171,7 +171,7 @@ policy::
|
|||
|
||||
For more information, see `canned ACLs`_ in the Amazon S3 Developer Guide.
|
||||
|
||||
.. _canned ACLs: http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
|
||||
.. _canned ACLs: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
|
||||
|
||||
Google Cloud Storage
|
||||
---------------------
|
||||
|
|
|
|||
|
|
@ -249,9 +249,9 @@ If you are still unable to prevent your bot getting banned, consider contacting
|
|||
|
||||
.. _Tor project: https://www.torproject.org/
|
||||
.. _commercial support: http://scrapy.org/support/
|
||||
.. _ProxyMesh: http://proxymesh.com/
|
||||
.. _ProxyMesh: https://proxymesh.com/
|
||||
.. _Google cache: http://www.googleguide.com/cached_pages.html
|
||||
.. _testspiders: https://github.com/scrapinghub/testspiders
|
||||
.. _Twisted Reactor Overview: https://twistedmatrix.com/documents/current/core/howto/reactor-basics.html
|
||||
.. _Crawlera: https://scrapinghub.com/crawlera
|
||||
.. _scrapoxy: http://scrapoxy.io/
|
||||
.. _scrapoxy: https://scrapoxy.io/
|
||||
|
|
|
|||
|
|
@ -734,7 +734,7 @@ HtmlResponse objects
|
|||
which adds encoding auto-discovering support by looking into the HTML `meta
|
||||
http-equiv`_ attribute. See :attr:`TextResponse.encoding`.
|
||||
|
||||
.. _meta http-equiv: http://www.w3schools.com/TAGS/att_meta_http_equiv.asp
|
||||
.. _meta http-equiv: https://www.w3schools.com/TAGS/att_meta_http_equiv.asp
|
||||
|
||||
XmlResponse objects
|
||||
-------------------
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ documents.
|
|||
For a complete reference of the selectors API see
|
||||
:ref:`Selector reference <topics-selectors-ref>`
|
||||
|
||||
.. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
|
||||
.. _BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/
|
||||
.. _lxml: http://lxml.de/
|
||||
.. _ElementTree: https://docs.python.org/2/library/xml.etree.elementtree.html
|
||||
.. _cssselect: https://pypi.python.org/pypi/cssselect/
|
||||
|
|
|
|||
|
|
@ -1002,7 +1002,7 @@ The randomization policy is the same used by `wget`_ ``--random-wait`` option.
|
|||
|
||||
If :setting:`DOWNLOAD_DELAY` is zero (default) this option has no effect.
|
||||
|
||||
.. _wget: http://www.gnu.org/software/wget/manual/wget.html
|
||||
.. _wget: https://www.gnu.org/software/wget/manual/wget.html
|
||||
|
||||
.. setting:: REACTOR_THREADPOOL_MAXSIZE
|
||||
|
||||
|
|
@ -1317,7 +1317,7 @@ Default: ``2083``
|
|||
Scope: ``spidermiddlewares.urllength``
|
||||
|
||||
The maximum URL length to allow for crawled URLs. For more information about
|
||||
the default value for this setting see: http://www.boutell.com/newfaq/misc/urllength.html
|
||||
the default value for this setting see: https://boutell.com/newfaq/misc/urllength.html
|
||||
|
||||
.. setting:: USER_AGENT
|
||||
|
||||
|
|
|
|||
|
|
@ -39,9 +39,9 @@ variable; or by defining it in your :ref:`scrapy.cfg <topics-config-settings>`::
|
|||
[settings]
|
||||
shell = bpython
|
||||
|
||||
.. _IPython: http://ipython.org/
|
||||
.. _IPython installation guide: http://ipython.org/install.html
|
||||
.. _bpython: http://www.bpython-interpreter.org/
|
||||
.. _IPython: https://ipython.org/
|
||||
.. _IPython installation guide: https://ipython.org/install.html
|
||||
.. _bpython: https://www.bpython-interpreter.org/
|
||||
|
||||
Launch the shell
|
||||
================
|
||||
|
|
|
|||
|
|
@ -751,8 +751,8 @@ Combine SitemapSpider with other sources of urls::
|
|||
def parse_other(self, response):
|
||||
pass # ... scrape other here ...
|
||||
|
||||
.. _Sitemaps: http://www.sitemaps.org
|
||||
.. _Sitemap index files: http://www.sitemaps.org/protocol.html#index
|
||||
.. _Sitemaps: https://www.sitemaps.org/index.html
|
||||
.. _Sitemap index files: https://www.sitemaps.org/protocol.html#index
|
||||
.. _robots.txt: http://www.robotstxt.org/
|
||||
.. _TLD: https://en.wikipedia.org/wiki/Top-level_domain
|
||||
.. _Scrapyd documentation: https://scrapyd.readthedocs.io/en/latest/
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Run tests, generate coverage report and open it on a browser
|
||||
#
|
||||
# Requires: coverage 3.3 or above from http://pypi.python.org/pypi/coverage
|
||||
# Requires: coverage 3.3 or above from https://pypi.python.org/pypi/coverage
|
||||
|
||||
coverage run --branch $(which trial) --reporter=text tests
|
||||
coverage html -i
|
||||
|
|
|
|||
Loading…
Reference in New Issue