Merge pull request #404 from darkrho/djangoitem-docs

Add a section to DjangoItem docs page regarding setting up Django's settings.
This commit is contained in:
Pablo Hoffman 2013-09-30 11:15:32 -07:00
commit 56275b1b9c
1 changed files with 55 additions and 7 deletions

View File

@ -27,6 +27,7 @@ Let's see some examples:
Creating a Django model for the examples::
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=255)
age = models.IntegerField()
@ -34,14 +35,15 @@ Creating a Django model for the examples::
Defining a basic :class:`DjangoItem`::
from scrapy.contrib.djangoitem import DjangoItem
class PersonItem(DjangoItem):
django_model = Person
:class:`DjangoItem` work just like :class:`~scrapy.item.Item`::
p = PersonItem()
p['name'] = 'John'
p['age'] = '22'
>>> p = PersonItem()
>>> p['name'] = 'John'
>>> p['age'] = '22'
To obtain the Django model from the item, we call the extra method
:meth:`~DjangoItem.save` of the :class:`DjangoItem`::
@ -72,10 +74,12 @@ As said before, we can add other fields to the item::
django_model = Person
sex = Field()
p = PersonItem()
p['name'] = 'John'
p['age'] = '22'
p['sex'] = 'M'
::
>>> p = PersonItem()
>>> p['name'] = 'John'
>>> p['age'] = '22'
>>> p['sex'] = 'M'
.. note:: fields added to the item won't be taken into account when doing a :meth:`~DjangoItem.save`
@ -96,3 +100,47 @@ models, but bear in mind that Django ORM may not scale well if you scrape a lot
of items (ie. millions) with Scrapy. This is because a relational backend is
often not a good choice for a write intensive application (such as a web
crawler), specially if the database is highly normalized and with many indices.
Django settings set up
======================
To use the Django models outside the Django application you need to set up the
``DJANGO_SETTINGS_MODULE`` environment variable and --in most cases-- modify
the ``PYTHONPATH`` environment variable to be able to import the settings
module.
There are many ways to do this depending on your use case and preferences.
Below is detailed one of the simplest ways to do it.
Suppose your Django project is named ``mysite``, is located in the path
``/home/projects/mysite`` and you have created an app ``myapp`` with the model
``Person``. That means your directory structure is something like this::
/home/projects/mysite
├── manage.py
├── myapp
│   ├── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
└── mysite
├── __init__.py
├── settings.py
├── urls.py
└── wsgi.py
Then you need to add ``/home/projects/mysite`` to the ``PYTHONPATH``
environment variable and set up the environment variable
``DJANGO_SETTINGS_MODULE`` to ``mysite.settings``. That can be done in your
Scrapy's settings file by adding the lines below::
import sys
sys.path.append('/home/projects/mysite')
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
Notice that we modify the ``sys.path`` variable instead the ``PYTHONPATH``
environment variable as we are already within the python runtime. If everything
is right, you should be able to start the ``scrapy shell`` command and import
the model ``Person`` (i.e. ``from myapp.models import Person``).