5.1 KiB
Installation guide
This document describes how to install Scrapy on Linux, Windows and Mac OS X.
Requirements
- Python 2.6, 2.7 (3.x is not yet supported)
- Twisted 8.0 or above (Windows users: you'll need to install Zope.Interface and maybe pywin32 because of this Twisted bug)
- w3lib
- lxml or libxml2 (if using libxml2, version 2.6.28 or above is highly recommended)
- pyopenssl (for HTTPS support. Optional, but highly recommended)
Install Python
First, you need to install Python, if you haven't done so already.
Scrapy works with Python 2.6 or 2.7, which you can get at http://www.python.org/download/
System Message: ERROR/3 (<stdin>, line 36)
Unknown directive type "seealso".
.. seealso:: :ref:`faq-python-versions`
System Message: ERROR/3 (<stdin>, line 38)
Unknown directive type "highlight".
.. highlight:: sh
Install Scrapy
There are many ways to install Scrapy. Pick the one you feel more comfortable with.
:ref:`intro-install-release` (requires installing dependencies separately)
System Message: ERROR/3 (<stdin>, line 48); backlink
Unknown interpreted text role "ref".
:ref:`intro-install-easy` (automatically installs dependencies)
System Message: ERROR/3 (<stdin>, line 49); backlink
Unknown interpreted text role "ref".
:ref:`intro-install-pip` (automatically installs dependencies)
System Message: ERROR/3 (<stdin>, line 50); backlink
Unknown interpreted text role "ref".
Download and install an official release
Download Scrapy from the Download page. Scrapy is distributed in two ways: a source code tarball (for Unix and Mac OS X systems) and a Windows installer (for Windows). If you downloaded the tarball, you can install it as any Python package using setup.py:
tar zxf Scrapy-X.X.X.tar.gz cd Scrapy-X.X.X python setup.py install
If you downloaded the Windows installer, just run it.
Warning
In Windows, you may need to add the C:\Python25\Scripts (or C:\Python26\Scripts) folder to the system path by adding that directory to the PATH environment variable from the Control Panel.
Installing with easy_install
You can install Scrapy using setuptools's easy_install with:
easy_install -U Scrapy
Platform specific instructions
Linux
Ubuntu 9.10 or above
If you're running Ubuntu 9.10 (or above), use the official :ref:`Ubuntu Packages <topics-ubuntu>`, which already solve all dependencies for you and are continuously updated with the latest bug fixes.
System Message: ERROR/3 (<stdin>, line 103); backlink
Unknown interpreted text role "ref".Debian or Ubuntu (9.04 or older)
If you're running Debian Linux, run the following command as root:
apt-get install python-twisted python-lxmlxml python-pyopenssl
Then:
easy_install -U w3lib
And then follow the instructions in :ref:`intro-install-scrapy`.
System Message: ERROR/3 (<stdin>, line 118); backlink
Unknown interpreted text role "ref".Arch Linux
If you are running Arch Linux, run the following command as root:
pacman -S twisted python-lxml pyopenssl
Then:
easy_install -U w3lib
And then follow the instructions in :ref:`intro-install-scrapy`.
System Message: ERROR/3 (<stdin>, line 131); backlink
Unknown interpreted text role "ref".Other Linux distros
The easiest way to install Scrapy in other Linux distros is through easy_install, which will automatically install Twisted, w3lib and lxml as dependencies. See :ref:`intro-install-easy`.
System Message: ERROR/3 (<stdin>, line 136); backlink
Unknown interpreted text role "ref".Another way would be to install dependencies, if you know the packages in your distros that meets them. See :ref:`intro-install-requirements`.
System Message: ERROR/3 (<stdin>, line 140); backlink
Unknown interpreted text role "ref".Mac OS X
The easiest way to install Scrapy on Mac is through easy_install or pip, which will automatically install Twisted, w3lib and lxml dependencies.
See :ref:`intro-install-easy`.
System Message: ERROR/3 (<stdin>, line 149); backlink
Unknown interpreted text role "ref".Windows
There are two ways to install Scrapy in Windows:
using easy_install or pip - see :ref:`intro-install-easy` or :ref:`intro-install-pip`
System Message: ERROR/3 (<stdin>, line 156); backlink
Unknown interpreted text role "ref".
System Message: ERROR/3 (<stdin>, line 156); backlink
Unknown interpreted text role "ref".
using the Windows installer, but you need to download and install the dependencies manually:
- Twisted for Windows - you may need to install pywin32 because of this Twisted bug
- Install Zope.Interface (required by Twisted)
- libxml2 for Windows
- PyOpenSSL for Windows
- Download the Windows installer from the Downloads page and install it.