RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

aws: sign requests if url hostname ends with s3.amazonaws.com common bucket suffix. This allows a spider to send requests to S3 with out need to sign it inside spider code 

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40542
This commit is contained in:
Daniel Grana 2008-12-23 14:31:58 +00:00
parent 0cd074c94c
commit 6d33e12c8b
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
scrapy/trunk/scrapy/contrib/aws.py
View File

@ -81,6 +81,7 @@ class AWSMiddleware(object):
self.secret_key = settings['AWS_SECRET_ACCESS_KEY']
def process_request(self, request, spider):
if spider.domain_name in 's3.amazonaws.com':
if spider.domain_name == 's3.amazonaws.com' \
or request.url.hostname.endswith('s3.amazonaws.com'):
request.headers['Date'] = time.strftime("%a, %d %b %Y %H:%M:%S GMT", time.gmtime())
sign_request(request, self.access_key, self.secret_key)