RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,206 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

11206 Commits

Author SHA1 Message Date
Paul Tremberth 53769245f5 Use python 3.6 directly 2017-01-06 16:02:14 +01:00
Paul Tremberth 40851a3d4c Use Python 3.6-dev on Travis 2017-01-06 15:44:32 +01:00
Paul Tremberth a214731471 Add Python 3.6 tox env + Travis CI build for it 2017-01-06 15:38:35 +01:00
Elias Dorneles b6ab1ae9c3 docs: installation instructions, mention conda in the beginning (closes #2475) 2017-01-03 15:14:59 -02:00
Paul Tremberth e7dede66f6 Merge pull request #2458 from rolando-contrib/http11-maxsize-logging-tweak 
[MRG+1] ENH Pass arguments to logger rather than formatted message.
 2017-01-03 14:01:34 +01:00
Alex 8a86574394 .devN release suffix must be preceded with a dot 
https://packaging.python.org/distributing/#standards-compliance-for-interoperability
 2016-12-28 14:25:22 +00:00
Elias Dorneles b7dd089bf9 show CoC in its own section 2016-12-27 11:33:23 -02:00
Elias Dorneles dabcb17d72 update code of conduct http://contributor-covenant.org/version/1/4 2016-12-27 11:32:15 -02:00
Elias Dorneles 9922ec15d7 mention contributing document before CoC in the README 2016-12-27 09:52:35 -02:00
nyov 2240f00a13 Remove dependency on os.environ from default settings 
Avoid loading settings from environment in scrapy core.
Instead it's better to populate them from the starting
shell or an embedding script.
 2016-12-26 20:49:16 +00:00
Raul Gallegos e7c7e055ff settings: fixing name of the pipeline template 2016-12-24 11:55:04 -05:00
Mikhail Korobov cc06b6b1f6 Merge pull request #2460 from redapple/filepipe-random-test-fail 
TST: Randomize FILES_EXPIRES above 90 days
 2016-12-21 23:23:50 +05:00
Paul Tremberth 07f9985a94 TST: Randomize FILES_EXPIRES above 90 days 2016-12-21 17:03:11 +01:00
Paul Tremberth ac74d5a467 Bump version: 1.2.2 → 1.3.0 2016-12-21 16:28:44 +01:00
Paul Tremberth f8793e2460 Set release date for 1.3.0 2016-12-21 16:27:01 +01:00
Paul Tremberth 0c27d252d1 Merge pull request #2437 from scrapy/release-notes-1.3.0 
Update changelog for upcoming 1.3.0 release
 2016-12-21 16:25:13 +01:00
Paul Tremberth b9e7ca044c Reword things a tiny bit 2016-12-21 16:11:19 +01:00
Paul Tremberth d60a6899df Merge logging changes into "New features" section 2016-12-21 16:01:36 +01:00
Paul Tremberth 4eeec3e42c Add preamble on why 1.3.0 comes so soon after 1.2.2 2016-12-21 15:59:18 +01:00
Paul Tremberth 49a84c2d41 Add note on HTTP redirects with shell and fetch 2016-12-21 14:49:16 +01:00
Paul Tremberth 9098001888 Add note on LOG_SHORT_NAMES 2016-12-21 14:32:59 +01:00
Paul Tremberth 9d5afd8c35 Add note on HttpErrorMiddleware new logging level 2016-12-21 14:32:59 +01:00
Paul Tremberth e9b3cf01f4 Update changelog for upcoming 1.3.0 release 2016-12-21 14:32:59 +01:00
Elias Dorneles d09ec3db68 Merge pull request #2410 from redapple/fetch-transparent-redirect 
[MRG+1] Transparently handle redirections in fetch and shell
 2016-12-21 09:49:15 -02:00
Rolando Espinoza 6dec4a3ccb ENH Pass arguments to logger rather than formatted message. 
This not only use the standard form but helps error aggregation
libraries (i.e.: Sentry) to avoid duplicating the message.
 2016-12-20 20:02:31 -04:00
Paul Tremberth 140a57d7b0 Amend note on --no-redirect option for shell tool 2016-12-19 17:54:54 +01:00
Mikhail Korobov d19c4c1f80 Merge pull request #2433 from redapple/wrong-spidermodules-warning 
[MRG+1] Warn user instead of failing for wrong SPIDER_MODULES setting
 2016-12-19 21:46:56 +05:00
Paul Tremberth 78d34cc44c Merge pull request #2454 from crasker/fix/low_parsel_version_number 
[MRG+1] Fix/low parsel version number
 2016-12-19 17:11:17 +01:00
Paul Tremberth ed1e4d8df3 Merge pull request #1731 from scrapy/disable-toplevel-2 
[MRG+1] LOG_SHORT_NAMES option to disable TopLevelFormatter
 2016-12-19 16:02:55 +01:00
zhouyc 2b3abdb700 update parsel version which will cause attribute error 2016-12-19 16:42:31 +08:00
Mikhail Korobov da19f0b7b7 DOC how to override log level for a specific Scrapy component 2016-12-16 22:14:54 +05:00
Mikhail Korobov 0fc73a9d55 DOC update examples with long longger names 2016-12-16 21:47:58 +05:00
Elias Dorneles 97e82107b1 Merge pull request #2270 from redapple/httperror-log-info 
[MRG+1] Raise log level for HttpErrorMiddleware to INFO (from DEBUG)
 2016-12-14 12:18:04 -02:00
Paul Tremberth 70a69d2199 Use built-in range() 2016-12-12 22:40:48 +01:00
Paul Tremberth f7e4081414 Add tests for SequenceExclude container 2016-12-12 22:37:53 +01:00
Paul Tremberth f457379a54 Add stacktrace in warning message 2016-12-09 16:56:26 +01:00
Mikhail Korobov 05b4555f39 TST tests for LOG_SHORT_NAMES 2016-12-09 02:19:51 +05:00
Mikhail Korobov e46572d6f2 TST end-to-end test for LOG_LEVEL option 
there were no end-to-end tests for this option
 2016-12-09 02:19:33 +05:00
Mikhail Korobov 6eab59cbac TST cleanup runspider tests 2016-12-09 02:14:12 +05:00
Mikhail Korobov 05cec0f2f3 fixed ReST syntax 2016-12-09 00:04:04 +05:00
Akhil Lb a75ad2bbc6 LOG_SHORT_NAMES option 2016-12-09 00:04:04 +05:00
Paul Tremberth 7d17836032 Update documentation about --no-redirect option 2016-12-08 17:27:25 +01:00
Paul Tremberth 948e3cd003 Warn user instead of failing for wrong SPIDER_MODULES setting 2016-12-08 12:50:26 +01:00
Paul Tremberth 2cd579a774 Add test for fetch(url) within shell with and without redirect 2016-12-07 19:07:32 +01:00
Paul Tremberth 7e54de2455 Add tests for shell command with and without --no-redirect 2016-12-07 18:41:24 +01:00
Mikhail Korobov bd8c293a97 Merge pull request #2411 from redapple/remove-chunked-transfer-mw 
[MRG+1] Remove ChunkedTransferMiddleware from default settings
 2016-12-07 22:05:15 +05:00
Elias Dorneles cce631abec Merge pull request #1887 from nyov/twisted11 
[MRG+2] Bump Twisted dependency to 13.1.0
 2016-12-07 15:00:01 -02:00
Paul Tremberth 778bed07bf Let framework handle only HTTP redirects by default for fetch and shell commands 2016-12-07 17:56:13 +01:00
Mikhail Korobov ff3aec6613 Merge pull request #2331 from moisesguimaraes/fixes-2272 
[MRG+1] Fixes issue #2272 using arg_to_iter() to wrap single values and list() to…
 2016-12-07 20:08:18 +05:00
Elias Dorneles a9c69458ff Merge pull request #2422 from rolando-contrib/nested-spiders-modules 
[MRG+1] DOC State explicitly that spiders are loaded recursively.
 2016-12-07 11:21:15 -02:00