RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,841 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

1339 Commits

Author SHA1 Message Date
Elias Dorneles 8bafcf33a1 improve description of engine 2016-09-30 11:20:30 -03:00
Elias Dorneles e107212043 restore explanation loop to step 1 (see comment in 3ac3ac4) 2016-09-29 19:02:57 -03:00
Elias Dorneles 86eb865598 Merge pull request #2287 from pawelmhm/docs-2230 
[MRG+1][docs/item_pipeline] process_item returning Deferred docs
 2016-09-29 09:15:09 -03:00
Elias Dorneles 60559369b5 Merge pull request #2292 from stummjr/fix-example-section 
[MRG+1] Update examples section in documentation with quotes.toscrape spiders
 2016-09-29 09:11:45 -03:00
Elias Dorneles 3ac3ac4d92 docs: update data flow description and image (fixes: #2278) 
This fixes the explanation to use Requests instead of URLs,
which is what actually happens, and is also consistent with the
new tutorial, which already explains how URLs become Request objects.

I've also changed the "loop", jumping from 9 to step 2.
 2016-09-28 16:38:45 -03:00
Valdir Stumm Junior 5680c61063 Doc: update examples section with quotes.toscrape.com spiders 2016-09-28 15:45:01 -03:00
pawelmhm 39dbd890b0 [docs/item_pipeline] process_item returning Deferred docs 
* quote url
* use hash of url as filename
 2016-09-24 08:36:09 +02:00
Paul Tremberth 80c1e5dc25 Set release date, fix typo and add tutorial improvement issue number 2016-09-23 16:11:50 +02:00
Paul Tremberth a0f87d2f45 Update release notes for upcoming 1.1.3 bugfix release 2016-09-23 16:11:50 +02:00
Pawel Miech b2bfd1e5c5 [docs] document that process_item can return Deferred 2016-09-23 10:36:03 +02:00
Paul Tremberth a975a50558 Merge pull request #2252 from eliasdorneles/tutorial-upgrades 
[MRG+2] Tutorial: rewrite tutorial seeking to improve learning path
 2016-09-22 16:39:21 +02:00
Elias Dorneles f4a2208916 addressing review comments and other minor editing 2016-09-22 11:04:45 -03:00
Paul Tremberth 2e08a9b412 Merge pull request #2271 from redapple/mailsender-lists 
[MRG+1] Add note on "to" and "cc" as lists for sending emails
 2016-09-22 12:00:14 +02:00
Elias Dorneles d636e5baa8 better description for start_requests expected return value 2016-09-21 18:54:12 -03:00
Elias Dorneles 32017a76f8 recommend learn python the hard way for beginners 2016-09-21 11:06:36 -03:00
Elias Dorneles 38266cc949 recommend Dive into Python and Python tutorial instead of LPTHW for non-beginners 2016-09-21 11:02:24 -03:00
Elias Dorneles c126c59361 address more review comments 2016-09-20 18:19:25 -03:00
Elias Dorneles a876ea5bd2 minor grammar fix 2016-09-20 15:10:49 -03:00
Elias Dorneles bc41fdf20e address review comments, add debug log to initial spider 2016-09-20 15:04:08 -03:00
Paul Tremberth 40293551b2 Remove mention of odd-numbered versions for development releases 
Fixes GH-1317
 2016-09-20 18:15:45 +02:00
Elias Dorneles 125b691102 more reviewing and editing, minor restructure, syntax fixes 2016-09-20 12:47:03 -03:00
Paul Tremberth e59d79bf37 Add note on "to" and "cc" as lists for sending emails 
Fixes GH-2244
 2016-09-20 17:18:49 +02:00
Elias Dorneles 8975371a57 Merge branch 'master' into tutorial-upgrades 2016-09-20 09:45:05 -03:00
Elias Dorneles f4f93c5c26 fix tox docs build, adjust title 2016-09-20 09:19:59 -03:00
Valdir Stumm Junior fee07835f2 Completing the data extraction section 2016-09-19 19:19:44 -03:00
Elias Dorneles 2a409d1d95 [wip] changing introduction to scraping with selectors 2016-09-19 17:13:04 -03:00
Elias Dorneles 063315258e Merge pull request #2202 from scrapy/doc-arch-overview2 
[MRG+1] DOC move Data Flow below the picture; add links to components
 2016-09-19 08:11:18 -03:00
Wayne Lovely cc8497abb1 Fix a dict key in the tutorial 2016-09-17 11:09:28 +00:00
Valdir Stumm Junior 233b98d642 include section describing spider arguments 2016-09-16 18:08:10 -03:00
Elias Dorneles 31545a9f84 tutorial: updating extracting data section to introduce CSS and XPath equally 2016-09-16 17:13:24 -03:00
Elias Dorneles 147e75602d update after review comments (thanks @stummjr) 2016-09-16 16:47:24 -03:00
Elias Dorneles 31260cf02f mentions stackoverflow as help channel (fixes #2255) 2016-09-16 16:05:36 -03:00
Elias Dorneles 21de617c77 mention that spiders need to subclass scrapy.Spider 2016-09-16 15:55:14 -03:00
Elias Dorneles b2a5cddbb0 tutorial: update section about following links, expand examples 
adding an AuthorSpider to demonstrate further a different crawling
arrangement.
 2016-09-16 15:49:49 -03:00
Valdir Stumm Junior 0cd9dfcc85 small fixes on tutorial 2016-09-16 15:21:49 -03:00
Valdir Stumm Junior 0da497cf7a updates on the first section (our first spider) 2016-09-16 11:55:23 -03:00
Elias Dorneles c508f40689 use harcoded URLs, remove item reference on second spider 2016-09-15 18:05:09 -03:00
Elias Dorneles 2427791287 tutorial: remove item class definition and present start_requests first 
This changes the tutorial, removing the step of creating an item class
and also starts by presenting the start_requests method instead of
start_urls.
 2016-09-15 17:46:31 -03:00
Elias Dorneles 75531e409e use better condition in example spider 2016-09-15 16:56:13 -03:00
Elias Dorneles 1d159ae6f9 minor grammar fix 2016-09-15 15:37:03 -03:00
Elias Dorneles 18bd0b0886 docs: update overview spider code to use toscrape.com and minor changes 
So, this will replace the spider example code from the overview that
scrapes questions from StackOverflow by a spider scraping quotes (much
like the one in the tutorial), and upates the text around it to be
consistent.

There are also minor wording changes plus a small Sphinx/reST syntax fix
on the features list at the bottom (it was creating a definition list,
causing one line to be bold).
 2016-09-15 15:16:30 -03:00
Paul Tremberth 2f60f2a5a6 Merge pull request #2236 from stummjr/new-tutorial-toscrape 
[MRG+1] Update broken Scrapy tutorial to use quotes.toscrape.com
 2016-09-15 12:05:03 +02:00
Elias Dorneles a9a96bed8f updated tutorial as per review comments 2016-09-14 18:09:39 -03:00
Valdir Stumm Junior bc67cd9edd fix indentation issue 2016-09-14 12:39:29 -03:00
Valdir Stumm Junior 10f8c52f5d changed tutorial examples from dmoz to quotes.toscrape.com 2016-09-13 14:05:26 -03:00
Matti Remes 0ef570f6f0 Update exceptions.rst 
Added the missing dot. (+1 squashed commit)
Squashed commits:
[2198972] Update exceptions.rst

There are namely no constructors in classes in Python but an ``__init__`` method instead.
 2016-09-08 19:38:17 +05:00
Valdir Stumm Junior 9cea6f0730 Add Segment Analytics to Documentation 2016-09-02 14:51:07 -03:00
Paul Tremberth eedb6ce774 Merge pull request #2190 from stummjr/fix-docs 
[MRG+1] Fix RANDOMIZE_DOWNLOAD_DELAY description in the docs
 2016-08-31 11:51:47 +02:00
Mikhail Korobov 495d322691 DOC move Data Flow below the picture; add links to components 2016-08-26 20:16:22 +05:00
Thom Dixon f68dc3026d Fix indentation 2016-08-24 09:11:27 -07:00