RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,308 Commits 49 Branches 132 Tags 180 MiB
Commit Graph

6308 Commits

Author SHA1 Message Date
Julien Palard 30eec55910 [PEDANTIC] FIX trailing whitespaces in LICENSE. 2017-04-22 00:24:18 +02:00
Paul Tremberth 2d7f11ecd5 Merge pull request #2710 from redapple/travis-pypy-portable 
Travis CI: use portable pypy for Linux
 2017-04-18 16:21:56 +02:00
Mikhail Korobov 9731077a54 Merge pull request #2683 from harshasrinivas/docs-SelectorList 
[MRG+1] Remove __nonzero__ from SelectorList docs
 2017-04-12 21:25:00 +05:00
Mikhail Korobov 9e04e77542 Merge pull request #2705 from redapple/docs-selectors-sections 
DOC Rearrange selector sections
 2017-04-12 21:24:25 +05:00
Paul Tremberth f3f7a41861 Travis CI: use portable pypy for Linux 2017-04-12 16:32:21 +02:00
Paul Tremberth 422b38f65c DOC Rearrange selector sections 2017-04-11 16:55:43 +02:00
Paul Tremberth 252819151e Merge pull request #2695 from LMKight/command-list-fix 
[MRG+1] command-list-fix
 2017-04-05 16:21:57 +02:00
LMKight 05ce1296c6 changed code according to request 2017-04-03 19:47:01 +02:00
LMKight 6352c2e9b2 fixed command list 2017-04-02 15:11:13 +02:00
Mikhail Korobov 8753b458fb Merge pull request #2690 from redapple/faq-py3-windows 
FAQ Rewrite note on Python 3 support on Windows
 2017-03-29 15:43:48 +05:00
Paul Tremberth 163618c9b7 FAQ Rewrite note on Python 3 support on Windows 2017-03-29 12:02:44 +02:00
harshasrinivas 2ff6b05723 Remove __nonzero__ from SelectorList docs 2017-03-24 20:43:28 +05:30
Mikhail Korobov 34f4434b17 Merge pull request #2682 from harshasrinivas/compile-docs-open-webbrowser 
[MRG+1] Update Makefile to open webbrowser in MacOS (#2661)
 2017-03-24 19:31:52 +05:00
harshasrinivas 0298bcbe79 Update Makefile to open webbrowser in MacOS (#2661) 2017-03-24 18:13:08 +05:30
Paul Tremberth 776129a951 Merge pull request #2649 from pawelmhm/logformatter-2647 
[MRG+2] [logformatter] 'flags' format spec backward compatibility
 2017-03-21 10:51:31 +01:00
Pawel Miech 4345eaf1b6 [logformatter] backward compat comments 2017-03-17 08:11:20 +01:00
Mikhail Korobov ec55799d5e Merge pull request #2616 from redapple/mediapipeline-redirect-fix-continued 
[MRG] Allow redirections in media files downloads
 2017-03-16 15:31:28 +05:00
Paul Tremberth 1c0da1749d Merge pull request #2646 from woxcab/master 
[MRG+1] Allowed passing objects of Mapping class or its subclass to the CaselessDict initializer
 2017-03-15 13:11:58 +01:00
woxcab a84652e775 Init tests are split by initializer' input 2017-03-15 12:39:48 +03:00
Mikhail Korobov 4e4395f16f Merge pull request #2644 from delftswa2017/fsfilestore_fixme_fix 
[MRG+1] Fixed the FIXME in FSFilesStore
 2017-03-14 15:11:28 +05:00
Pawel Miech 0f2a5cdb8e [logformatter] 'flags' format spec backward compatibility 
pass 'flags' kwarg to logger so that it is compatible with old
format of CRAWLEDMSG.
 2017-03-13 15:16:38 +01:00
woxcab fbb411a805 Allowed passing objects of Mapping class or its subclass to the CaselessDict initializer 2017-03-13 14:16:39 +03:00
jorenham 3cd9185aa1 Fixed the FIXME; more specific exception catching 2017-03-12 23:02:07 +01:00
Paul Tremberth 871134ee22 Refactor to also test FilesPipeline 2017-03-12 17:30:24 +01:00
Paul Tremberth 708f1b009b Add integration tests for MEDIA_ALLOW_REDIRECTS 2017-03-10 21:36:33 +01:00
Paul Tremberth 7dcc86e61a Add file listing resource + redirecting resource to MockServer 2017-03-10 21:35:25 +01:00
Paul Tremberth cfb56400b2 Merge pull request #2641 from redapple/release-notes-1.3.3-master 
Update release notes and versionadded for SPIDER_LOADER_WARN_ONLY
 2017-03-10 17:15:26 +01:00
Paul Tremberth a7f5207e9f Update version added for SPIDER_LOADER_WARN_ONLY 2017-03-10 16:38:09 +01:00
Paul Tremberth b2c505d8ec Set release date in changelog for v1.3.3 2017-03-10 16:34:21 +01:00
Paul Tremberth d8865b3304 Update changelog for upcoming 1.3.3 2017-03-10 16:34:21 +01:00
Mikhail Korobov 9c69e90056 Merge pull request #2632 from redapple/spider-loader-warn-or-fail 
[MRG] Add SPIDER_LOADER_WARN_ONLY to toggle between spiderloader failure or warning
 2017-03-09 23:01:27 +05:00
Paul Tremberth 9628a73972 Update settings docs for new SPIDER_LOADER_WARN_ONLY 2017-03-09 17:40:34 +01:00
Paul Tremberth f2ac24eb7b Do not only warn on wrong spider modules for "scrapy list" 2017-03-09 17:38:15 +01:00
Paul Tremberth 36160f1b0a Merge pull request #2477 from scrapy/recommend-anaconda-for-win 
[MRG+2] docs: installation instructions, mention conda in the beginning (closes #2475)
 2017-03-09 12:56:50 +01:00
Paul Tremberth c0cbaccb7b Merge pull request #2581 from lopuhin/respect-custom-log-level 
[MRG+1] Respect custom log level: fixes GH-1612
 2017-03-09 12:47:20 +01:00
Paul Tremberth db13ab5ec2 Merge pull request #2611 from delftswa2017/httpcachemiddleware_log 
[MRG+1] [logging] HttpCacheMiddleware: log cache directory at instantiation
 2017-03-09 12:44:10 +01:00
Paul Tremberth c5e193d079 Merge pull request #2636 from delftswa2017/contrib_docs_fix 
[MRG+1] Removed contrib section in contribution documentation
 2017-03-09 12:42:59 +01:00
Paul Tremberth 9cfe9ae098 Do not use self.assertRaises() as context manager 2017-03-09 12:21:03 +01:00
jorenham ac63d3a3cf Removed contrib section; contrib is deprecated 2017-03-09 10:56:23 +01:00
Paul Tremberth 7be773e14a Add SPIDER_LOADER_WARN_ONLY to toggle between spiderloader failure and warning 2017-03-07 17:40:40 +01:00
Mikhail Korobov d3f8f3d38a Merge pull request #2612 from redapple/dupe-spider-name-tests 
[WIP] Add warning on duplicate spider name
 2017-03-07 20:08:42 +05:00
Paul Tremberth 0b9a18e1a1 Warn only once for all spiders 2017-03-07 15:41:17 +01:00
Paul Tremberth 978306a223 Fix dupe spider name warning string tests 2017-03-07 14:48:16 +01:00
Paul Tremberth a017f7b932 Warn about modules where duplicate spider names were found 2017-03-07 14:44:27 +01:00
Mikhail Korobov 802ed30e8e Merge pull request #2391 from redapple/always-gunzip-content-enc-gzip 
[MRG] Always decompress Content-Encoding: gzip at HttpCompression stage
 2017-03-07 16:56:54 +05:00
Paul Tremberth b6378c7ef6 Revert to using self.assert methods 2017-03-07 12:28:24 +01:00
Konstantin Lopuhin 6f55ca4643 Revert unneeded test_crawl changes 2017-03-07 14:20:52 +03:00
Paul Tremberth 4caceccd59 Use 3-bytes for gzip archive type sniffing 2017-03-07 11:02:46 +01:00
Paul Tremberth b174744b80 Do not silently fail on gzip unzipping 2017-03-07 11:02:46 +01:00
Paul Tremberth 11cdf58abe Always decompress Content-Encoding: gzip at HttpCompression stage 
Let SitemapSpider handle decoding of .xml.gz files if necessary
 2017-03-07 11:02:46 +01:00