mirror of https://github.com/scrapy/scrapy.git
bc17e9d412
Ported .get() from _Element and .text_content() from HTMLMixin Add CSS selectors to scrapy shell Documenting CSS Selectors: Constructing selectors Documenting CSS Selectors: Using Selectors Make CSS Selectors a default feature. Adds XPath powers to CSS Selectors and some syntactic sugar. Removes methods copied over from lxml.html.HtmlMixin. Updating docs to use new CSS Selector super powers. Documenting CSS Selectors: Regular Expressions Moving section after Nesting section, since it mentions it. Documenting CSS Selectors: Nesting Selectors Fix XPath specificity in lxml.selector.CSSSelectorMixin.text Cleaning up unused stuff from cssel.py Changing the behavior of lxml.selector.CSSSelectorMixin.text. Concatenating all of the descendant text nodes is more useful than returning it in pieces (there's xpath() if you need that). Documenting CSS Selectors: CSS Selector objects Documenting CSS Selectors: CSSSelectorList objects Documenting CSS Selectors: HtmlCSSSelector objects Documenting CSS Selectors: XmlCSSSelector objects Fixing some documentations typos and errors Enforcing the 80-char width lines Tidying up CSS selectors and CSSSelectorMixin objects Adding some missing references in documentation. Fixing lxml.selector.CSSSelectorList.text |
||
|---|---|---|
| .travis | ||
| artwork | ||
| bin | ||
| debian | ||
| docs | ||
| extras | ||
| scrapy | ||
| sep | ||
| .coveragerc | ||
| .gitignore | ||
| .travis.yml | ||
| AUTHORS | ||
| CONTRIBUTING.md | ||
| INSTALL | ||
| LICENSE | ||
| MANIFEST.in | ||
| Makefile.buildbot | ||
| NEWS | ||
| README.rst | ||
| requirements.txt | ||
| setup.cfg | ||
| setup.py | ||
| tox.ini | ||
README.rst
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
</head>
</html>
Scrapy
Overview
Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
For more information including a list of features check the Scrapy homepage at: http://scrapy.org
Requirements
- Python 2.6 or up
- Works on Linux, Windows, Mac OSX, BSD
Install
The quick way:
pip install scrapy
For more details see the install section in the documentation: http://doc.scrapy.org/en/latest/intro/install.html
Releases
You can download the latest stable and development releases from: http://scrapy.org/download/
Documentation
Documentation is available online at http://doc.scrapy.org/ and in the docs directory.