4.8 KiB
Installation guide
This document describes how to install Scrapy on Linux, Windows and Mac OS X.
Requirements
- Python 2.5, 2.6, 2.7 (3.x is not yet supported)
- Twisted 2.5.0, 8.0 or above (Windows users: you'll need to install Zope.Interface and maybe pywin32 because of this Twisted bug)
- w3lib
- lxml or libxml2 (if using libxml2, version 2.6.28 or above is highly recommended)
- simplejson (not required if using Python 2.6 or above)
- pyopenssl (for HTTPS support. Optional, but highly recommended)
Install Python
First, you need to install Python, if you haven't done so already.
Scrapy works with Python 2.5, 2.6 or 2.7, which you can get at http://www.python.org/download/
System Message: ERROR/3 (<stdin>, line 38)
Unknown directive type "seealso".
.. seealso:: :ref:`faq-python-versions`
System Message: ERROR/3 (<stdin>, line 40)
Unknown directive type "highlight".
.. highlight:: sh
Install Scrapy
There are many ways to install Scrapy. Pick the one you feel more comfortable with.
:ref:`intro-install-release` (requires installing dependencies separately)
System Message: ERROR/3 (<stdin>, line 50); backlink
Unknown interpreted text role "ref".
:ref:`intro-install-easy` (automatically installs dependencies)
System Message: ERROR/3 (<stdin>, line 51); backlink
Unknown interpreted text role "ref".
:ref:`intro-install-pip` (automatically installs dependencies)
System Message: ERROR/3 (<stdin>, line 52); backlink
Unknown interpreted text role "ref".
Download and install an official release
Download Scrapy from the Download page. Scrapy is distributed in two ways: a source code tarball (for Unix and Mac OS X systems) and a Windows installer (for Windows). If you downloaded the tarball, you can install it as any Python package using setup.py:
tar zxf Scrapy-X.X.X.tar.gz cd Scrapy-X.X.X python setup.py install
If you downloaded the Windows installer, just run it.
Warning
In Windows, you may need to add the C:\Python25\Scripts (or C:\Python26\Scripts) folder to the system path by adding that directory to the PATH environment variable from the Control Panel.
Installing with easy_install
You can install Scrapy using setuptools's easy_install with:
easy_install -U Scrapy
Platform specific instructions
Linux
Ubuntu 9.10 or above
If you're running Ubuntu 9.10 (or above), use the official :ref:`Ubuntu Packages <topics-ubuntu>`, which already solve all dependencies for you and are continuously updated with the latest bug fixes.
System Message: ERROR/3 (<stdin>, line 105); backlink
Unknown interpreted text role "ref".Debian or Ubuntu (9.04 or older)
If you're running Debian Linux, run the following command as root:
apt-get install python-twisted python-libxml2 python-pyopenssl python-simplejson
Then:
easy_install -U w3lib
And then follow the instructions in :ref:`intro-install-scrapy`.
System Message: ERROR/3 (<stdin>, line 120); backlink
Unknown interpreted text role "ref".Arch Linux
If you are running Arch Linux, run the following command as root:
pacman -S twisted libxml2 pyopenssl python-simplejson
Then:
easy_install -U w3lib
And then follow the instructions in :ref:`intro-install-scrapy`.
System Message: ERROR/3 (<stdin>, line 133); backlink
Unknown interpreted text role "ref".Other Linux distros
The easiest way to install Scrapy on other Linux distros is through easy_install or pip, which will automatically install Twisted, w3lib and lxml dependencies.
See :ref:`intro-install-easy` or :ref:`intro-install-pip`
System Message: ERROR/3 (<stdin>, line 142); backlink
Unknown interpreted text role "ref".System Message: ERROR/3 (<stdin>, line 142); backlink
Unknown interpreted text role "ref".Another way would be to install dependencies, if you know the packages in your distros that meets them. See :ref:`intro-install-requirements`.
System Message: ERROR/3 (<stdin>, line 144); backlink
Unknown interpreted text role "ref".Mac OS X
The easiest way to install Scrapy on Mac is through easy_install or pip, which will automatically install Twisted, w3lib and lxml dependencies.
See :ref:`intro-install-easy` or :ref:`intro-install-pip`
System Message: ERROR/3 (<stdin>, line 153); backlink
Unknown interpreted text role "ref".System Message: ERROR/3 (<stdin>, line 153); backlink
Unknown interpreted text role "ref".Windows
The easiest way to install Scrapy on Windows is through easy_install or pip, which will automatically install Twisted, w3lib and lxml dependencies.
See :ref:`intro-install-easy` or :ref:`intro-install-pip`
System Message: ERROR/3 (<stdin>, line 161); backlink
Unknown interpreted text role "ref".System Message: ERROR/3 (<stdin>, line 161); backlink
Unknown interpreted text role "ref".