RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,450 Commits 42 Branches 132 Tags 162 MiB
Commit Graph

6450 Commits

Author SHA1 Message Date
Rolando Espinoza 6dec4a3ccb ENH Pass arguments to logger rather than formatted message. 
This not only use the standard form but helps error aggregation
libraries (i.e.: Sentry) to avoid duplicating the message.
 2016-12-20 20:02:31 -04:00
Paul Tremberth 140a57d7b0 Amend note on --no-redirect option for shell tool 2016-12-19 17:54:54 +01:00
Mikhail Korobov d19c4c1f80 Merge pull request #2433 from redapple/wrong-spidermodules-warning 
[MRG+1] Warn user instead of failing for wrong SPIDER_MODULES setting
 2016-12-19 21:46:56 +05:00
Paul Tremberth 78d34cc44c Merge pull request #2454 from crasker/fix/low_parsel_version_number 
[MRG+1] Fix/low parsel version number
 2016-12-19 17:11:17 +01:00
Paul Tremberth ed1e4d8df3 Merge pull request #1731 from scrapy/disable-toplevel-2 
[MRG+1] LOG_SHORT_NAMES option to disable TopLevelFormatter
 2016-12-19 16:02:55 +01:00
zhouyc 2b3abdb700 update parsel version which will cause attribute error 2016-12-19 16:42:31 +08:00
Mikhail Korobov da19f0b7b7 DOC how to override log level for a specific Scrapy component 2016-12-16 22:14:54 +05:00
Mikhail Korobov 0fc73a9d55 DOC update examples with long longger names 2016-12-16 21:47:58 +05:00
Elias Dorneles 97e82107b1 Merge pull request #2270 from redapple/httperror-log-info 
[MRG+1] Raise log level for HttpErrorMiddleware to INFO (from DEBUG)
 2016-12-14 12:18:04 -02:00
Paul Tremberth 70a69d2199 Use built-in range() 2016-12-12 22:40:48 +01:00
Paul Tremberth f7e4081414 Add tests for SequenceExclude container 2016-12-12 22:37:53 +01:00
Paul Tremberth f457379a54 Add stacktrace in warning message 2016-12-09 16:56:26 +01:00
Mikhail Korobov 05b4555f39 TST tests for LOG_SHORT_NAMES 2016-12-09 02:19:51 +05:00
Mikhail Korobov e46572d6f2 TST end-to-end test for LOG_LEVEL option 
there were no end-to-end tests for this option
 2016-12-09 02:19:33 +05:00
Mikhail Korobov 6eab59cbac TST cleanup runspider tests 2016-12-09 02:14:12 +05:00
Mikhail Korobov 05cec0f2f3 fixed ReST syntax 2016-12-09 00:04:04 +05:00
Akhil Lb a75ad2bbc6 LOG_SHORT_NAMES option 2016-12-09 00:04:04 +05:00
Paul Tremberth 7d17836032 Update documentation about --no-redirect option 2016-12-08 17:27:25 +01:00
Paul Tremberth 948e3cd003 Warn user instead of failing for wrong SPIDER_MODULES setting 2016-12-08 12:50:26 +01:00
Paul Tremberth 2cd579a774 Add test for fetch(url) within shell with and without redirect 2016-12-07 19:07:32 +01:00
Paul Tremberth 7e54de2455 Add tests for shell command with and without --no-redirect 2016-12-07 18:41:24 +01:00
Mikhail Korobov bd8c293a97 Merge pull request #2411 from redapple/remove-chunked-transfer-mw 
[MRG+1] Remove ChunkedTransferMiddleware from default settings
 2016-12-07 22:05:15 +05:00
Elias Dorneles cce631abec Merge pull request #1887 from nyov/twisted11 
[MRG+2] Bump Twisted dependency to 13.1.0
 2016-12-07 15:00:01 -02:00
Paul Tremberth 778bed07bf Let framework handle only HTTP redirects by default for fetch and shell commands 2016-12-07 17:56:13 +01:00
Mikhail Korobov ff3aec6613 Merge pull request #2331 from moisesguimaraes/fixes-2272 
[MRG+1] Fixes issue #2272 using arg_to_iter() to wrap single values and list() to…
 2016-12-07 20:08:18 +05:00
Elias Dorneles a9c69458ff Merge pull request #2422 from rolando-contrib/nested-spiders-modules 
[MRG+1] DOC State explicitly that spiders are loaded recursively.
 2016-12-07 11:21:15 -02:00
Mikhail Korobov 862814651f Merge pull request #2427 from redapple/pipe-test-fail 
TST: Randomize IMAGES_EXPIRES above 90 days
 2016-12-06 23:03:09 +05:00
Paul Tremberth 5efd65255c TST: Randomize IMAGES_EXPIRES above 90 days 2016-12-06 18:49:53 +01:00
Paul Tremberth f3d5995329 Bump version: 1.2.1 → 1.2.2 2016-12-06 15:21:00 +01:00
Paul Tremberth ea968c6c3d Merge pull request #2423 from redapple/release-notes-1.2.2 
Update changelog for upcoming 1.2.2 release
 2016-12-06 15:18:31 +01:00
Paul Tremberth 09e310d0b7 Set release date for 1.2.2 2016-12-06 15:03:38 +01:00
Paul Tremberth aa2e1b030d Add reference to fixed scheduler settings documentation 2016-12-06 14:59:11 +01:00
Paul Tremberth 89d5f5acd3 Update changelog for upcoming 1.2.2 release 2016-12-06 14:59:11 +01:00
Mikhail Korobov 97df139119 Merge pull request #2417 from pawelmhm/scheduler-docs 
[MRG+1] Add docs for some scheduler settings
 2016-12-06 18:42:07 +05:00
Moisés Guimarães c08d278c0c removes note from docs. 2016-12-05 16:47:24 -03:00
Moisés Guimarães a4178f99da fixes params types in docs. 2016-12-05 15:24:26 -03:00
Moisés Guimarães c58ea021b8 fixes docs 2016-12-04 11:56:14 -03:00
nyov 67bc2e0b18 Wipe scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
nyov 67cf64edbe Deprecate scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
nyov 534772f6ea Import xlib.tx code from twisted proper 2016-12-02 21:21:51 +00:00
nyov 985755d1fe Remove obsolete xlib code for Twisted 13.1.0 2016-12-01 23:44:15 +00:00
nyov c8cf1a303d Bump Twisted dependency to 13.1.0 (released June 2013) 2016-12-01 22:02:11 +00:00
nyov e1ea0c433a Strip xlib.tx code of Twisted 10 2016-12-01 22:02:10 +00:00
Elias Dorneles c4e67c0696 Merge pull request #2421 from rolando-contrib/tests-bug 
[MRG+1] TST: Fix duplicated test name.
 2016-12-01 18:43:48 -02:00
Elias Dorneles f3a4420750 Merge pull request #2388 from redapple/robotparser-native-str 
[MRG+1] Parse robots.txt content as native str
 2016-12-01 18:43:23 -02:00
Mikhail Korobov 826b7e92b6 Merge pull request #2386 from rolando-contrib/manifest-tweak 
[MRG+1] Ignore explicitly compiled python files.
 2016-12-02 00:20:27 +05:00
Rolando Espinoza 923b974f0a TST Include nested a nested spider in spider loader test. 2016-12-01 13:26:19 -03:00
Rolando Espinoza 6431e7a138 DOC State explicitly that spiders are loaded recursively. 2016-12-01 13:26:19 -03:00
Rolando Espinoza d9f43e21ba TST: Fix duplicated test name. 2016-12-01 11:56:33 -03:00
Elias Dorneles 5f0af90674 Merge pull request #2395 from elacuesta/robots_txt_relative_sitemap_url 
[MRG+1] Handle relative sitemap URLs in robots.txt
 2016-12-01 11:24:54 -02:00