Scrapy, a fast high-level web crawling & scraping framework for Python.
Go to file
Pablo Hoffman e3fe0ef297 Some changes to newitem API and implementation:
- Dropped support for wildcard importing from newitem package (must now import
  from newitem.fields and don't use wildcard)
- Removed assign() method from Fields as it was apparently redundant (with
  to_python() method) and I couldn't find any reason for keeping it (neither in
  the docs nor in the tests)
- Moved deiter() method of Field to StringField, as its both its purpose and
  implementation was specific for strings. if it's really needed as a general
  purpose method, it could be restored. Also, no unittest was broken because of
  this change, which sort-of reinforces my point.
- Renamed (previously mentioned) StringField.deiter() method to
  StringField.to_single(), for better consistency with to_python() method
- Removed Field class as it was useless without the deiter() functionality (now
  belonging to StringField class)
- Moved ansi_date_re module variable to DateField class attribute
- Simplified implementation of DecimalField, FloatField and IntegerField to one
  line of code (using tests to make sure not to break any functionality)
- Renamed ItemMeta class (in models.py) to _ItemMeta to highlight its protected
  state (should not be externally imported)
- Added support for instantiating new items with dicts, to support
  deserializing items with their repr() string
- Added unittests for new functionality introduced
2009-07-11 22:19:56 -03:00
bin Fixed distutils packaging for all setup.py bdist formats 2009-06-16 17:10:43 +01:00
docs improved newitems doc and marked robust scraped items as deprecated 2009-07-11 21:26:52 -03:00
examples Some changes to HTTP cache middleware: 2009-05-24 19:13:06 -03:00
extras/sql mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
profiling/priorityqueue mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
scrapy Some changes to newitem API and implementation: 2009-07-11 22:19:56 -03:00
scripts removed python2.5 from rpm-install.sh script 2009-06-16 13:14:40 -03:00
.hgignore adding .svn to hgignore to help with hg2svn autocommits 2009-05-07 17:29:30 -03:00
AUTHORS mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
INSTALL mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
LICENSE mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
MANIFEST.in Fixed distutils packaging for all setup.py bdist formats 2009-06-16 17:10:43 +01:00
README mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
setup.cfg Fixed distutils packaging for all setup.py bdist formats 2009-06-16 17:10:43 +01:00
setup.py generate dropin.cache for spiders under tests 2009-07-10 05:29:27 +01:00

README

This is Scrapy, an opensource screen scraping framework written in Python.

For more visit the project home page at http://scrapy.org