diff --git a/docs/contributing.rst b/docs/contributing.rst index be34bf98c..548d3d18d 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -146,6 +146,14 @@ tests requires `tox`_. Running tests ------------- +Make sure you have a recent enough `tox`_ installation: + + ``tox --version`` + +If your version is older than 1.7.0, please update it first: + + ``pip install -U tox`` + To run all tests go to the root directory of Scrapy source code and run: ``tox`` diff --git a/docs/intro/install.rst b/docs/intro/install.rst index 67507016d..3adb4e6b0 100644 --- a/docs/intro/install.rst +++ b/docs/intro/install.rst @@ -14,7 +14,8 @@ The installation steps assume that you have the following things installed: * `Python`_ 2.7 * `pip`_ and `setuptools`_ Python packages. Nowadays `pip`_ requires and - installs `setuptools`_ if not installed. + installs `setuptools`_ if not installed. Python 2.7.9 and later include + `pip`_ by default, so you may have it already. * `lxml`_. Most Linux distributions ships prepackaged versions of lxml. Otherwise refer to http://lxml.de/installation.html @@ -58,7 +59,8 @@ Windows Be sure you download the architecture (win32 or amd64) that matches your system -* Install `pip`_ from https://pip.pypa.io/en/latest/installing.html +* *(Only required for Python<2.7.9)* Install `pip`_ from + https://pip.pypa.io/en/latest/installing.html Now open a Command prompt to check ``pip`` is installed correctly:: @@ -79,6 +81,21 @@ Instead, use the official :ref:`Ubuntu Packages `, which already solve all dependencies for you and are continuously updated with the latest bug fixes. +If you prefer to build the python dependencies locally instead of relying on +system packages you'll need to install their required non-python dependencies +first:: + + sudo apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev + +You can install Scrapy with ``pip`` after that:: + + pip install Scrapy + +.. note:: + + The same non-python dependencies can be used to install Scrapy in Debian + Wheezy (7.0) and above. + Archlinux --------- @@ -86,6 +103,58 @@ You can follow the generic instructions or install Scrapy from `AUR Scrapy packa yaourt -S scrapy +Mac OS X +-------- + +Building Scrapy's dependencies requires the presence of a C compiler and +development headers. On OS X this is typically provided by Apple’s Xcode +development tools. To install the Xcode command line tools open a terminal +window and run:: + + xcode-select --install + +There's a `known issue `_ that +prevents ``pip`` from updating system packages. This has to be addressed to +successfully install Scrapy and its dependencies. Here are some proposed +solutions: + +* *(Recommended)* **Don't** use system python, install a new, updated version + that doesn't conflict with the rest of your system. Here's how to do it using + the `homebrew`_ package manager: + + * Install `homebrew`_ following the instructions in http://brew.sh/ + + * Update your ``PATH`` variable to state that homebrew packages should be + used before system packages (Change ``.bashrc`` to ``.zshrc`` accordantly + if you're using `zsh`_ as default shell):: + + echo "export PATH=/usr/local/bin:/usr/local/sbin:$PATH" >> ~/.bashrc + + * Reload ``.bashrc`` to ensure the changes have taken place:: + + source ~/.bashrc + + * Install python:: + + brew install python + + * Latest versions of python have ``pip`` bundled with them so you won't need + to install it separately. If this is not the case, upgrade python:: + + brew update; brew upgrade python + +* *(Optional)* Install Scrapy inside an isolated python environment. + + This method is a workaround for the above OS X issue, but it's an overall + good practice for managing dependencies and can complement the first method. + + `virtualenv`_ is a tool you can use to create virtual environments in python. + We recommended reading a tutorial like + http://docs.python-guide.org/en/latest/dev/virtualenvs/ to get started. + +After any of these workarounds you should be able to install Scrapy:: + + pip install Scrapy .. _Python: https://www.python.org/ .. _pip: https://pip.pypa.io/en/latest/installing.html @@ -95,3 +164,6 @@ You can follow the generic instructions or install Scrapy from `AUR Scrapy packa .. _OpenSSL: https://pypi.python.org/pypi/pyOpenSSL .. _setuptools: https://pypi.python.org/pypi/setuptools .. _AUR Scrapy package: https://aur.archlinux.org/packages/scrapy/ +.. _homebrew: http://brew.sh/ +.. _zsh: http://www.zsh.org/ +.. _virtualenv: https://virtualenv.pypa.io/en/latest/