RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,396 Commits 49 Branches 132 Tags 180 MiB
Commit Graph

6396 Commits

Author SHA1 Message Date
Paul Tremberth ed1e4d8df3 Merge pull request #1731 from scrapy/disable-toplevel-2 
[MRG+1] LOG_SHORT_NAMES option to disable TopLevelFormatter
 2016-12-19 16:02:55 +01:00
zhouyc 2b3abdb700 update parsel version which will cause attribute error 2016-12-19 16:42:31 +08:00
Mikhail Korobov da19f0b7b7 DOC how to override log level for a specific Scrapy component 2016-12-16 22:14:54 +05:00
Mikhail Korobov 0fc73a9d55 DOC update examples with long longger names 2016-12-16 21:47:58 +05:00
Elias Dorneles 97e82107b1 Merge pull request #2270 from redapple/httperror-log-info 
[MRG+1] Raise log level for HttpErrorMiddleware to INFO (from DEBUG)
 2016-12-14 12:18:04 -02:00
Paul Tremberth 70a69d2199 Use built-in range() 2016-12-12 22:40:48 +01:00
Paul Tremberth f7e4081414 Add tests for SequenceExclude container 2016-12-12 22:37:53 +01:00
Paul Tremberth f457379a54 Add stacktrace in warning message 2016-12-09 16:56:26 +01:00
Mikhail Korobov 05b4555f39 TST tests for LOG_SHORT_NAMES 2016-12-09 02:19:51 +05:00
Mikhail Korobov e46572d6f2 TST end-to-end test for LOG_LEVEL option 
there were no end-to-end tests for this option
 2016-12-09 02:19:33 +05:00
Mikhail Korobov 6eab59cbac TST cleanup runspider tests 2016-12-09 02:14:12 +05:00
Mikhail Korobov 05cec0f2f3 fixed ReST syntax 2016-12-09 00:04:04 +05:00
Akhil Lb a75ad2bbc6 LOG_SHORT_NAMES option 2016-12-09 00:04:04 +05:00
Paul Tremberth 7d17836032 Update documentation about --no-redirect option 2016-12-08 17:27:25 +01:00
Paul Tremberth 948e3cd003 Warn user instead of failing for wrong SPIDER_MODULES setting 2016-12-08 12:50:26 +01:00
Paul Tremberth 2cd579a774 Add test for fetch(url) within shell with and without redirect 2016-12-07 19:07:32 +01:00
Paul Tremberth 7e54de2455 Add tests for shell command with and without --no-redirect 2016-12-07 18:41:24 +01:00
Mikhail Korobov bd8c293a97 Merge pull request #2411 from redapple/remove-chunked-transfer-mw 
[MRG+1] Remove ChunkedTransferMiddleware from default settings
 2016-12-07 22:05:15 +05:00
Elias Dorneles cce631abec Merge pull request #1887 from nyov/twisted11 
[MRG+2] Bump Twisted dependency to 13.1.0
 2016-12-07 15:00:01 -02:00
Paul Tremberth 778bed07bf Let framework handle only HTTP redirects by default for fetch and shell commands 2016-12-07 17:56:13 +01:00
Mikhail Korobov ff3aec6613 Merge pull request #2331 from moisesguimaraes/fixes-2272 
[MRG+1] Fixes issue #2272 using arg_to_iter() to wrap single values and list() to…
 2016-12-07 20:08:18 +05:00
Elias Dorneles a9c69458ff Merge pull request #2422 from rolando-contrib/nested-spiders-modules 
[MRG+1] DOC State explicitly that spiders are loaded recursively.
 2016-12-07 11:21:15 -02:00
Mikhail Korobov 862814651f Merge pull request #2427 from redapple/pipe-test-fail 
TST: Randomize IMAGES_EXPIRES above 90 days
 2016-12-06 23:03:09 +05:00
Paul Tremberth 5efd65255c TST: Randomize IMAGES_EXPIRES above 90 days 2016-12-06 18:49:53 +01:00
Paul Tremberth f3d5995329 Bump version: 1.2.1 → 1.2.2 2016-12-06 15:21:00 +01:00
Paul Tremberth ea968c6c3d Merge pull request #2423 from redapple/release-notes-1.2.2 
Update changelog for upcoming 1.2.2 release
 2016-12-06 15:18:31 +01:00
Paul Tremberth 09e310d0b7 Set release date for 1.2.2 2016-12-06 15:03:38 +01:00
Paul Tremberth aa2e1b030d Add reference to fixed scheduler settings documentation 2016-12-06 14:59:11 +01:00
Paul Tremberth 89d5f5acd3 Update changelog for upcoming 1.2.2 release 2016-12-06 14:59:11 +01:00
Mikhail Korobov 97df139119 Merge pull request #2417 from pawelmhm/scheduler-docs 
[MRG+1] Add docs for some scheduler settings
 2016-12-06 18:42:07 +05:00
Moisés Guimarães c08d278c0c removes note from docs. 2016-12-05 16:47:24 -03:00
Moisés Guimarães a4178f99da fixes params types in docs. 2016-12-05 15:24:26 -03:00
Moisés Guimarães c58ea021b8 fixes docs 2016-12-04 11:56:14 -03:00
nyov 67bc2e0b18 Wipe scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
nyov 67cf64edbe Deprecate scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
nyov 534772f6ea Import xlib.tx code from twisted proper 2016-12-02 21:21:51 +00:00
nyov 985755d1fe Remove obsolete xlib code for Twisted 13.1.0 2016-12-01 23:44:15 +00:00
nyov c8cf1a303d Bump Twisted dependency to 13.1.0 (released June 2013) 2016-12-01 22:02:11 +00:00
nyov e1ea0c433a Strip xlib.tx code of Twisted 10 2016-12-01 22:02:10 +00:00
Elias Dorneles c4e67c0696 Merge pull request #2421 from rolando-contrib/tests-bug 
[MRG+1] TST: Fix duplicated test name.
 2016-12-01 18:43:48 -02:00
Elias Dorneles f3a4420750 Merge pull request #2388 from redapple/robotparser-native-str 
[MRG+1] Parse robots.txt content as native str
 2016-12-01 18:43:23 -02:00
Mikhail Korobov 826b7e92b6 Merge pull request #2386 from rolando-contrib/manifest-tweak 
[MRG+1] Ignore explicitly compiled python files.
 2016-12-02 00:20:27 +05:00
Rolando Espinoza 923b974f0a TST Include nested a nested spider in spider loader test. 2016-12-01 13:26:19 -03:00
Rolando Espinoza 6431e7a138 DOC State explicitly that spiders are loaded recursively. 2016-12-01 13:26:19 -03:00
Rolando Espinoza d9f43e21ba TST: Fix duplicated test name. 2016-12-01 11:56:33 -03:00
Elias Dorneles 5f0af90674 Merge pull request #2395 from elacuesta/robots_txt_relative_sitemap_url 
[MRG+1] Handle relative sitemap URLs in robots.txt
 2016-12-01 11:24:54 -02:00
Eugenio Lacuesta 5ff64ad015 handle relative sitemap urls in robots.txt 2016-12-01 09:53:40 -03:00
Elias Dorneles 2086ff4065 Merge pull request #2418 from ahlinc/fix_retriggered_issue_396 
[MRG+1] Fix #396 re-triggered issue
 2016-11-30 10:49:41 -02:00
Mikhail Korobov bba4645f21 Merge pull request #2404 from redapple/scrapy-version 
[MRG+1] Print more dependencies versions in "scrapy version" verbose output
 2016-11-30 17:01:37 +05:00
Andrew Hlynskyi 27606aad11 Fix #396 re-triggered issue 
The InteractiveShellEmbed class is a singleton
and we need to drop the instance with its clear_instance() method
to rebuild the instance from scratch with fresh environment
for each subsequent Scrapy's shell drop in.
 2016-11-30 09:52:44 +02:00