RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,673 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

745 Commits

Author SHA1 Message Date
Pablo Hoffman 911c8082b0 simplified description of crawl command 2013-10-21 14:42:51 -02:00
Pablo Hoffman e8ee449a2a Merge pull request #432 from darkrho/crawl-url 
Removed URL reference in crawl command and .tld suffix in docs for spider names
 2013-10-21 09:40:58 -07:00
Rolando Espinoza La fuente 34543c2b2e DOCS removed .tld suffix for spider names for the sake of consistency. 2013-10-19 23:03:20 -04:00
Daniel Graña 875b07aef8 fix references to old selector naming in docs 2013-10-17 09:33:15 -02:00
Mikhail Korobov 086b8a20d4 typo fix in TextResponse docs 2013-10-17 04:50:30 +06:00
Pablo Hoffman 951a9f3f4c Merge pull request #226 from scraperdragon/patch-1 
Parameters to Request() in wrong order
 2013-10-16 13:15:52 -07:00
Daniel Graña 1461363809 Replace `contenttype` references by `type` 
The type to choose from is the selector type, not the input type. A
content-type doesn't make sense in this context.
 2013-10-16 17:37:25 -02:00
Daniel Graña 155ea08ea1 use `sel` name for Selector's instances in docs, internals and shell 2013-10-15 15:58:42 -02:00
Daniel Graña 1abb1af0c6 fix typos and wording on selector's introduction 2013-10-15 10:13:43 -02:00
Dragon Dave a3b711bdea Move callback blob; mention errback 2013-10-15 12:19:42 +01:00
scraperdragon 0ba0d85685 Parameters to Request() in wrong order 
Implied that callback wasn't the first optional unnamed parameter.
 2013-10-15 11:50:43 +01:00
Daniel Graña 28999590fa update release notes 2013-10-14 16:41:04 -02:00
Daniel Graña ab9462a251 remove more references to libxml2 2013-10-14 16:37:14 -02:00
Daniel Graña 4645f9e03c Updates docs to reflect unified selectors api 2013-10-14 16:31:20 -02:00
Daniel Graña bf37f78572 Drop libxml2 selectors backend 2013-10-11 18:02:35 -02:00
Daniel Graña 6d598f0d94 Update selectors docs 2013-10-10 18:24:00 -02:00
Capi Etheriel bc17e9d412 Adds HtmlCSSSelector and XmlCSSSelector classes, cssselect as optional dependency. 
Ported .get() from _Element and .text_content() from HTMLMixin

Add CSS selectors to scrapy shell

Documenting CSS Selectors: Constructing selectors

Documenting CSS Selectors: Using Selectors

Make CSS Selectors a default feature.

Adds XPath powers to CSS Selectors and some syntactic sugar.

Removes methods copied over from lxml.html.HtmlMixin.

Updating docs to use new CSS Selector super powers.

Documenting CSS Selectors: Regular Expressions

Moving section after Nesting section, since it mentions it.

Documenting CSS Selectors: Nesting Selectors

Fix XPath specificity in lxml.selector.CSSSelectorMixin.text

Cleaning up unused stuff from cssel.py

Changing the behavior of lxml.selector.CSSSelectorMixin.text.

Concatenating all of the descendant text nodes is more useful
than returning it in pieces (there's xpath() if you need that).

Documenting CSS Selectors: CSS Selector objects

Documenting CSS Selectors: CSSSelectorList objects

Documenting CSS Selectors: HtmlCSSSelector objects

Documenting CSS Selectors: XmlCSSSelector objects

Fixing some documentations typos and errors

Enforcing the 80-char width lines

Tidying up CSS selectors and CSSSelectorMixin objects

Adding some missing references in documentation.

Fixing lxml.selector.CSSSelectorList.text
 2013-10-10 18:23:15 -02:00
Daniel Graña 5b5dd679b0 Add 0.18.4 release notes 
Conflicts:
	docs/news.rst
 2013-10-10 01:04:35 -02:00
Pablo Hoffman e1683ddf9b fix doc typo 2013-10-09 17:24:12 -02:00
Pablo Hoffman 8b9526a8f6 Merge pull request #400 from irgmedeiros/patch-2 
Update the second code example
 2013-10-07 07:57:18 -07:00
Pablo Hoffman 86c6e9433f remove minor reference to 'scrapy server' command 2013-10-04 14:37:55 -03:00
Daniel Graña aad90ec5a2 Add 0.18.3 release notes 
Conflicts:
	docs/news.rst
 2013-10-03 12:56:25 -03:00
Pablo Hoffman 37c24e01d7 document bindaddress request meta 2013-10-02 17:13:17 -03:00
Pablo Hoffman a9c3519897 updated required twisted version to 10.0 2013-10-01 14:07:38 -03:00
Rolando Espinoza d6e3eae527 docs: added section regarding setting up django's settings. 2013-09-30 09:58:10 -04:00
Rolando Espinoza 0cc1d870db docs: minor tidy up sample code and missing shell prompts. 2013-09-30 09:58:10 -04:00
Loren Davie 8af0e89e85 Corrected typo. 2013-09-29 17:06:46 -04:00
Loren Davie f49f5724d5 Added dynamic creation of item classes to practices.rst. 2013-09-28 09:00:48 -04:00
irgmedeiros 9b50409986 Update the second code example 
Update the second code example to reflect the last change in the first example.
 2013-09-27 18:22:33 -03:00
irgmedeiros d9e0fdc9aa Update practices.rst 
With this modification scrapy runs the spider with project settings. The previous example ran only with default settings resulting in ignoring all user settings as pipelines for example.
 2013-09-27 17:56:30 -03:00
Daniel Graña 265910aae6 Merge pull request #363 from taikano/sitemap_alternate 
also fetch alternate URLs from sitemaps, see #360
 2013-09-26 09:15:02 -07:00
Pablo Hoffman 12280c2a95 fix sphinx references in doc 2013-09-25 15:13:17 -03:00
Pablo Hoffman fc388f4636 Make ITEM_PIPELINE setting a dict 
This is for consistency with how spider and downloader middlewares are
defined. ITEM_PIPELINE_BASE was also added and both remain empty.

Backwards compatibility is kept (with a warning) with list-based
ITEM_PIPELINES.
 2013-09-23 17:50:43 -03:00
Pablo Hoffman b1d1a36a1e add note about enclosing urls with quotes when running from command-line. closes GH-384 2013-09-18 18:01:28 -03:00
cacovsky 71b320914a Update request-response.rst 
Fix small doc typo (too many backticks)
 2013-09-18 11:45:25 -03:00
Kumara Tharmalingam bbb0603091 Fixed directory location for dmoz_spider.py file 
It should be under 'tutorial/spiders' not 'dmoz/spiders'
 2013-09-15 21:55:52 -07:00
Daniel Graña 0400b18efa docs: list lxml as installation prerequisite 2013-09-09 12:44:26 -03:00
Stefan 6994959181 renamed to sitemap_alternate_links and added default value, see #360 2013-09-08 10:38:28 +02:00
Stefan 8ed2d0cda1 improved changes to allow retrieval of alternate links in sitemaps, see #360 2013-09-07 12:56:30 +02:00
Daniel Graña e6b3ca0180 Add 0.18.2 release notes 2013-09-03 14:29:40 -03:00
Daniel Graña 0f00b1602a merge 0.18 release notes 2013-08-27 18:47:46 -03:00
Pablo Hoffman 19ff9ac4f9 url/body attributes of Request/Response objects are now immutable 2013-08-23 12:43:22 -03:00
Pablo Hoffman 86230c0ab8 added quantal & raring to support ubuntu releases 2013-08-22 21:49:55 -03:00
Mikhail Korobov 034ffae60f Recommend Pillow instead of PIL. Closes GH-317. 2013-08-18 00:44:01 +06:00
Berend Iwema 32b6364bcd #327 - Support STARTTLS / SSL option in email sender 2013-08-14 12:59:01 +02:00
Pablo Hoffman c0b26e3d49 minor updates to 0.18 release notes 2013-08-14 01:39:44 -03:00
Daniel Graña ed5b9068d2 fix contributters list format 2013-08-12 11:26:16 -03:00
Daniel Graña a6693c9a5c updated release notes and bumped version to 0.18.0 2013-08-09 19:02:28 -03:00
Hart c00c4d7148 correction to description of example XPath retrieval in overview doc 2013-08-03 17:08:58 -07:00
Hart 0ad01c34d4 fixed typo to parallel fix on 0.16 branch 2013-08-03 17:06:10 -07:00