RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,396 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

6396 Commits

Author SHA1 Message Date
Paul Tremberth 23abf2d31b Fix title underline 2016-09-28 18:31:27 +02:00
Paul Tremberth 8d130b295a Update installation guide 2016-09-28 17:09:17 +02:00
Paul Tremberth c707c31345 Fix Debian number 2016-09-26 17:20:13 +02:00
pawelmhm 39dbd890b0 [docs/item_pipeline] process_item returning Deferred docs 
* quote url
* use hash of url as filename
 2016-09-24 08:36:09 +02:00
Paul Tremberth bba31b545d Mention new Debian Jessie baseline 2016-09-23 17:33:08 +02:00
Paul Tremberth 7eeee6f616 Mention \uXXXX escapes in regards to FEED_EXPORT_ENCODING 2016-09-23 16:39:09 +02:00
Paul Tremberth d52fbf2045 Merge remote-tracking branch 'origin/master' into release-notes-1.2 2016-09-23 16:34:06 +02:00
Paul Tremberth d867268976 Merge pull request #2284 from redapple/release-notes-1.1.3-master 
Update release notes with 1.1.3 changes
 2016-09-23 16:25:52 +02:00
Paul Tremberth 80c1e5dc25 Set release date, fix typo and add tutorial improvement issue number 2016-09-23 16:11:50 +02:00
Paul Tremberth a0f87d2f45 Update release notes for upcoming 1.1.3 bugfix release 2016-09-23 16:11:50 +02:00
Elias Dorneles c2493c9452 Merge pull request #2282 from pawelmhm/docs-2230 
[docs] document that process_item can return Deferred
 2016-09-23 09:05:34 -03:00
Pawel Miech b2bfd1e5c5 [docs] document that process_item can return Deferred 2016-09-23 10:36:03 +02:00
Paul Tremberth a975a50558 Merge pull request #2252 from eliasdorneles/tutorial-upgrades 
[MRG+2] Tutorial: rewrite tutorial seeking to improve learning path
 2016-09-22 16:39:21 +02:00
Elias Dorneles 24bb91528a Merge pull request #2229 from ahlinc/fix_shell_completion 
[MRG+1] Fix completion in `scrapy shell` for new imports
 2016-09-22 11:15:30 -03:00
Elias Dorneles f4a2208916 addressing review comments and other minor editing 2016-09-22 11:04:45 -03:00
Paul Tremberth 2e08a9b412 Merge pull request #2271 from redapple/mailsender-lists 
[MRG+1] Add note on "to" and "cc" as lists for sending emails
 2016-09-22 12:00:14 +02:00
Elias Dorneles d636e5baa8 better description for start_requests expected return value 2016-09-21 18:54:12 -03:00
Elias Dorneles 32017a76f8 recommend learn python the hard way for beginners 2016-09-21 11:06:36 -03:00
Elias Dorneles 38266cc949 recommend Dive into Python and Python tutorial instead of LPTHW for non-beginners 2016-09-21 11:02:24 -03:00
Elias Dorneles 9c9690c76c add better messages for when response content isn't text (closes #2264) 2016-09-21 10:30:35 -03:00
Elias Dorneles c126c59361 address more review comments 2016-09-20 18:19:25 -03:00
Elias Dorneles a876ea5bd2 minor grammar fix 2016-09-20 15:10:49 -03:00
Elias Dorneles bc41fdf20e address review comments, add debug log to initial spider 2016-09-20 15:04:08 -03:00
Elias Dorneles a19af5b164 Merge pull request #2273 from redapple/version-stability 
[MRG+1] Remove mention of odd-numbered versions for development releases
 2016-09-20 14:15:52 -03:00
Paul Tremberth 40293551b2 Remove mention of odd-numbered versions for development releases 
Fixes GH-1317
 2016-09-20 18:15:45 +02:00
Elias Dorneles 125b691102 more reviewing and editing, minor restructure, syntax fixes 2016-09-20 12:47:03 -03:00
Paul Tremberth e59d79bf37 Add note on "to" and "cc" as lists for sending emails 
Fixes GH-2244
 2016-09-20 17:18:49 +02:00
Elias Dorneles 8975371a57 Merge branch 'master' into tutorial-upgrades 2016-09-20 09:45:05 -03:00
Elias Dorneles f4f93c5c26 fix tox docs build, adjust title 2016-09-20 09:19:59 -03:00
Elias Dorneles 3fd947b30d Merge pull request #2269 from redapple/unserializable-warning 
Log warning when request cannot be serialized (instead of error)
 2016-09-20 09:00:58 -03:00
Paul Tremberth 81a0e3cd93 Raise log level for HttpErrorMiddleware to INFO (from DEBUG) 
Fixes GH-910
 2016-09-20 13:44:21 +02:00
Paul Tremberth a135dbaf19 Log warning when request cannot be serialized (instead of error) 
Fixes GH-2035
 2016-09-20 12:47:33 +02:00
Ashish Kulkarni 8893527a2f temporarily deprecate official Ubuntu packages 
They are not currently updated and fail to install on
Ubuntu 16.04. Also update the instructions to refer to
the earliest supported LTS (Ubuntu 12.04).

fixes #2137 and closes #2076
 2016-09-20 11:10:59 +02:00
Valdir Stumm Junior fee07835f2 Completing the data extraction section 2016-09-19 19:19:44 -03:00
Elias Dorneles 2a409d1d95 [wip] changing introduction to scraping with selectors 2016-09-19 17:13:04 -03:00
Daniel Graña eb49b459c1 Merge pull request #2212 from redapple/debian-jessie-baseline 
Add Debian Jessie test env
 2016-09-19 15:17:45 -03:00
Paul Tremberth 41cd9f401f Merge pull request #2243 from pawelmhm/image-pipeline-2198 
[MRG+1] [image & file pipeline] loading setting for user classes
 2016-09-19 18:43:52 +02:00
Elias Dorneles 063315258e Merge pull request #2202 from scrapy/doc-arch-overview2 
[MRG+1] DOC move Data Flow below the picture; add links to components
 2016-09-19 08:11:18 -03:00
Paul Tremberth bb8740a552 Update 1.2 release notes with latest changes 2016-09-19 12:26:36 +02:00
Mikhail Korobov 490f6e08f3 Merge pull request #2239 from redapple/streamlogger-flush 
[MRG+1] Add flush() method to StreamLogger
 2016-09-19 14:44:45 +06:00
Mikhail Korobov 5657f6b8ef Merge pull request #2258 from redapple/feed-export-started 
[MRG+1] Feed exporter: start exporting only on first item
 2016-09-19 14:40:30 +06:00
Mikhail Korobov 552368727a Merge pull request #2225 from Tethik/parse_command_rules_fix 
[MRG+1] Two small fixes for when using the parse command and the '-r' flag (rules).
 2016-09-19 14:39:09 +06:00
Joakim Uddholm 8c38dde4e8 Moved parse command tests to its own file. Added some checks to check for logged errors. 2016-09-19 05:33:05 +02:00
Joakim Uddholm 88cf86f5f2 Merge pull request #1 from redapple/tethik_parse_command_rules_fix 
Add tests for crawl command non-default cases
 2016-09-19 00:51:36 +02:00
Paul Tremberth 48f6a065b8 Flush StreamLogger handlers 2016-09-17 15:25:45 +02:00
Paul Tremberth 27f88ad9cb Merge pull request #2260 from waynelovely/tutorial-fix-20160917-1 
Fix a dict key in the tutorial
 2016-09-17 14:19:21 +02:00
Wayne Lovely cc8497abb1 Fix a dict key in the tutorial 2016-09-17 11:09:28 +00:00
Mikhail Korobov 992b2517b0 Merge pull request #2248 from redapple/scrapy-shell-import-scrapy 
[MRG+1] Make scrapy available in shell without explicit import statement
 2016-09-17 06:10:06 +06:00
Mikhail Korobov 91fcafde5e Merge pull request #2257 from scrapy/mention-stackoverflow 
Mentions stackoverflow as support channel (fixes #2255)
 2016-09-17 06:06:53 +06:00
Paul Tremberth 03ab077249 Feed exporter: start exporting only on first item 
Fixes GH-872
 2016-09-17 01:36:56 +02:00