RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,206 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

11206 Commits

Author SHA1 Message Date
harshasrinivas 38e6857c95 Improvise the clarity of test cases 2017-03-23 19:45:04 +05:30
Paul Tremberth 99e3c0d653 Set bodyproducer with empty content for POST 2017-03-23 14:39:56 +01:00
Simon Diviani Gartz 21d794d35a Fixes conversion of transparent PNG with palette images to jpg #2452 2017-03-22 13:29:15 +01:00
Mikhail Korobov b040df5ac0 TST cleanup spider middleware tests 2017-03-21 15:56:18 +05:00
Paul Tremberth 776129a951 Merge pull request #2649 from pawelmhm/logformatter-2647 
[MRG+2] [logformatter] 'flags' format spec backward compatibility
 2017-03-21 10:51:31 +01:00
Qiwei Huang 8ecc307e8f Update spiders.rst 
Added note to allowed_domain attribute with an example explaining what goes in the list
 2017-03-20 19:37:07 -07:00
Qiwei Huang c5f74f7d1a Update spiders.rst 
Added a note to allowed_domains attribute, reminding users not to add urls into the list.
 2017-03-20 18:52:33 -07:00
harshasrinivas 83aa0c5e1a Clarify docs readme 2017-03-20 22:36:09 +05:30
harshasrinivas 4ec07ae764 Create docs/requirements.txt 2017-03-20 22:21:08 +05:30
harshasrinivas a57e49d55b Add sphinx_rtd_theme to docs setup readme 2017-03-20 19:49:31 +05:30
Oto Brglez 11cf6ad425 Comments for AWS_ENDPOINT_URL setting. 2017-03-19 12:48:06 +01:00
Oto Brglez 605691792f Updating media-pipeline docs for S3-like storage. 2017-03-19 12:35:39 +01:00
harshasrinivas 10741aca72 Update docs - improve clarity 2017-03-19 06:17:28 +05:30
harshasrinivas 0d9ebd6e1e Update tests for max_retry_times 2017-03-19 06:15:46 +05:30
harshasrinivas 49c5afc5ff Fix bug involving OR condition 2017-03-19 06:08:35 +05:30
Pawel Miech 4345eaf1b6 [logformatter] backward compat comments 2017-03-17 08:11:20 +01:00
Mikhail Korobov ec55799d5e Merge pull request #2616 from redapple/mediapipeline-redirect-fix-continued 
[MRG] Allow redirections in media files downloads
 2017-03-16 15:31:28 +05:00
Paul Tremberth 1c0da1749d Merge pull request #2646 from woxcab/master 
[MRG+1] Allowed passing objects of Mapping class or its subclass to the CaselessDict initializer
 2017-03-15 13:11:58 +01:00
woxcab a84652e775 Init tests are split by initializer' input 2017-03-15 12:39:48 +03:00
Bernardas 7ba4b0a21b add support for embeded ptpython shell 2017-03-15 08:07:47 +00:00
harshasrinivas 9d97d788c0 Update docs for meta key 2017-03-15 04:13:47 +05:30
harshasrinivas e321ac9931 Update unittests for max_retry_times 2017-03-15 04:12:32 +05:30
harshasrinivas 966bd49c42 Update unittest for meta['max_retry_times'] 2017-03-14 16:23:47 +05:30
harshasrinivas 694c6d3d74 Simplify retry_times assignment statement 2017-03-14 16:14:40 +05:30
Mikhail Korobov 4e4395f16f Merge pull request #2644 from delftswa2017/fsfilestore_fixme_fix 
[MRG+1] Fixed the FIXME in FSFilesStore
 2017-03-14 15:11:28 +05:00
Pawel Miech 0f2a5cdb8e [logformatter] 'flags' format spec backward compatibility 
pass 'flags' kwarg to logger so that it is compatible with old
format of CRAWLEDMSG.
 2017-03-13 15:16:38 +01:00
woxcab fbb411a805 Allowed passing objects of Mapping class or its subclass to the CaselessDict initializer 2017-03-13 14:16:39 +03:00
jorenham 3cd9185aa1 Fixed the FIXME; more specific exception catching 2017-03-12 23:02:07 +01:00
harshasrinivas 0d57b5cd43 Prevent max_retry_times override 2017-03-13 02:10:23 +05:30
Paul Tremberth 871134ee22 Refactor to also test FilesPipeline 2017-03-12 17:30:24 +01:00
harshasrinivas 810658bcc5 Add feature to set RETRY_TIMES per request (#2642) 2017-03-12 05:06:12 +05:30
Paul Tremberth 708f1b009b Add integration tests for MEDIA_ALLOW_REDIRECTS 2017-03-10 21:36:33 +01:00
Paul Tremberth 7dcc86e61a Add file listing resource + redirecting resource to MockServer 2017-03-10 21:35:25 +01:00
Eugenio Lacuesta 4cfbe82044 Downloader middleware: raise _InvalidOutput 
Instead of AssertionError, to make it consistent with spider middleware
 2017-03-10 15:47:50 -03:00
Eugenio Lacuesta 9c256cf693 Undocument _InvalidOutput exception 2017-03-10 15:41:57 -03:00
Paul Tremberth cfb56400b2 Merge pull request #2641 from redapple/release-notes-1.3.3-master 
Update release notes and versionadded for SPIDER_LOADER_WARN_ONLY
 2017-03-10 17:15:26 +01:00
Paul Tremberth a7f5207e9f Update version added for SPIDER_LOADER_WARN_ONLY 2017-03-10 16:38:09 +01:00
Paul Tremberth b2c505d8ec Set release date in changelog for v1.3.3 2017-03-10 16:34:21 +01:00
Paul Tremberth d8865b3304 Update changelog for upcoming 1.3.3 2017-03-10 16:34:21 +01:00
Mikhail Korobov 9c69e90056 Merge pull request #2632 from redapple/spider-loader-warn-or-fail 
[MRG] Add SPIDER_LOADER_WARN_ONLY to toggle between spiderloader failure or warning
 2017-03-09 23:01:27 +05:00
Paul Tremberth 9628a73972 Update settings docs for new SPIDER_LOADER_WARN_ONLY 2017-03-09 17:40:34 +01:00
Paul Tremberth f2ac24eb7b Do not only warn on wrong spider modules for "scrapy list" 2017-03-09 17:38:15 +01:00
Paul Tremberth 36160f1b0a Merge pull request #2477 from scrapy/recommend-anaconda-for-win 
[MRG+2] docs: installation instructions, mention conda in the beginning (closes #2475)
 2017-03-09 12:56:50 +01:00
Paul Tremberth c0cbaccb7b Merge pull request #2581 from lopuhin/respect-custom-log-level 
[MRG+1] Respect custom log level: fixes GH-1612
 2017-03-09 12:47:20 +01:00
Paul Tremberth db13ab5ec2 Merge pull request #2611 from delftswa2017/httpcachemiddleware_log 
[MRG+1] [logging] HttpCacheMiddleware: log cache directory at instantiation
 2017-03-09 12:44:10 +01:00
Paul Tremberth c5e193d079 Merge pull request #2636 from delftswa2017/contrib_docs_fix 
[MRG+1] Removed contrib section in contribution documentation
 2017-03-09 12:42:59 +01:00
Paul Tremberth 9cfe9ae098 Do not use self.assertRaises() as context manager 2017-03-09 12:21:03 +01:00
jorenham ac63d3a3cf Removed contrib section; contrib is deprecated 2017-03-09 10:56:23 +01:00
Eugenio Lacuesta 4090cc3990 Spider middleware: use Mockserver to test process_spider_exception 2017-03-08 18:11:20 -03:00
Paul Tremberth 7be773e14a Add SPIDER_LOADER_WARN_ONLY to toggle between spiderloader failure and warning 2017-03-07 17:40:40 +01:00