RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,489 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

6489 Commits

Author SHA1 Message Date
Daniel Graña 5fac2d7b90 Merge pull request #2923 from rhoboro/fixes-685 
[MRG+2] Fixes #685 FilesPipeline support for Google Cloud Storage.
 2017-10-02 13:52:20 -03:00
Mikhail Korobov 92fa74fb43 Merge pull request #2922 from stav/doc.response-body 
[Doc] Update Response.body type
 2017-10-02 10:26:54 +02:00
Mikhail Korobov 3431870b53 Merge pull request #2924 from superyyrrzz/patch-1 
minor fix typo
 2017-10-02 10:25:40 +02:00
Mikhail Korobov 2e4ddc6912 Merge pull request #2947 from lagenar/fix-tests-typo 
Fix typos in tests
 2017-10-02 10:24:27 +02:00
Lucas Moauro 59c3f6f095 Fix typos in tests 2017-10-01 12:24:56 -03:00
rhoboro d4555b2bcc update docs for supporting google cloud storage 2017-09-29 13:04:17 +09:00
Mikhail Korobov 346310cd10 Merge pull request #2915 from redapple/versions-with-cryptography 
Print cryptography package version
 2017-09-28 13:04:56 +02:00
Paul Tremberth e733f51d4b Fix test 2017-09-25 12:49:27 +02:00
Mikhail Korobov 5d17f38aee Merge pull request #2935 from lopuhin/pypy2.7-build 
Update pypy version regexp to get last release
 2017-09-20 17:59:46 +05:00
Konstantin Lopuhin 84111969c4 Update pypy version regexp to get last release 
PyPy changed naming conention since 5.8 release, not it's called
pypy2.7-x.x.x
 2017-09-20 13:35:48 +02:00
Renze Yu 088b80d41a minor fix typo 2017-09-13 23:29:22 +08:00
rhoboro ee166ec44f Support for ImagesPipeline 2017-09-13 17:35:46 +09:00
rhoboro e5d4364b2a Add tests for GCS Storage 2017-09-13 16:24:16 +09:00
rhoboro d71a063403 Support for Google Cloud Storage 2017-09-13 16:00:26 +09:00
Steven Almeroth 3637b75a67 [Doc] Update Response.body type 2017-09-12 17:41:47 -04:00
Paul Tremberth abaf466bb3 Print cryptography package version 2017-09-07 11:43:20 +02:00
cclauss b8fabeed86 ur'string' not needed in Py 2, syntax error in Py3 
This instance was missed in #2909 --> ur'Scrapy developers' --> u'Scrapy developers'
 2017-09-04 14:29:23 +02:00
Mikhail Korobov 6213fa5175 Merge pull request #2909 from cclauss/patch-2 
ur'string' not needed in Py 2, syntax error in Py3
 2017-09-01 15:42:02 +05:00
cclauss b702236082 ur'string' not needed in Py 2, syntax error in Py3 
Convert `u'(.*)\:\d+\:\s\[(.*)\]\s(?:(.*)\sto\s(.*)|(.*))'`--> `u'(.*)\:\d+\:\s\[(.*)\]\s(?:(.*)\sto\s(.*)|(.*))'`to be compatible with both Python 2 and Python 3.  See #2891
 2017-09-01 11:56:09 +02:00
cclauss 9f16f040b6 ur'string' not needed in Py 2, syntax error in Py3 
Convert `ur'Scrapy Documentation'`--> `u'Scrapy Documentation'`to be compatible with both Python 2 and Python 3.  See #2891
 2017-09-01 11:53:59 +02:00
Mikhail Korobov 65ac0b06df Merge pull request #2894 from redapple/log-custom-overriden-settings 
Move logging of overriden settings to Crawler init
 2017-08-29 19:20:46 +05:00
Pablo Hoffman a429d78019 update scrapinghub.com urls to use https 2017-08-24 16:03:36 -03:00
Paul Tremberth 7a35a1ad4a Remove trailing bracket from components versions log 2017-08-23 17:54:14 +02:00
Paul Tremberth 1968a8ec02 Move logging of overriden settings to Crawler init 2017-08-23 15:08:10 +02:00
Mikhail Korobov 1ba77f0837 Merge pull request #2869 from cclauss/patch-3 
# noqa to close #2836
 2017-08-21 19:15:30 +05:00
Mikhail Korobov 885289f497 Merge pull request #2854 from jenya/2853-sitemap-follow-alternate-fix 
[MRG+1] Follow alternate link for all types of sitemaps #2853
 2017-08-21 19:15:03 +05:00
Mikhail Korobov 984c0c19ee Merge pull request #2884 from iamminji/patch-1 
fix typo
 2017-08-16 16:36:00 +05:00
kim minji 1dcea6a9d4 fix typo 2017-08-16 18:07:52 +09:00
Daniel Graña d5f3543d77 Merge pull request #2865 from kirankoduru/2831-explicit-msg-for-scrapy-parse-callback 
[MRG+1] Explicit message for scrapy parse callback
 2017-08-09 07:56:57 -03:00
Chomba Ng'ang'a fd27cde24d Update asserts to use more generic ones 2017-08-09 12:52:10 +02:00
Chomba Ng'ang'a 4ca61a2051 Update deprecated test aliases 
- change ``failIf`` to ``assertFalse``
- change ``asertEquals`` to ``assertEqual``
- change ``assert_`` to ``assertTrue``

https://docs.python.org/2/library/unittest.html#deprecated-aliases
 2017-08-09 12:52:10 +02:00
Kiran Koduru 2960c9b568 Use self.__class__.__name__ instead of showing generic Spider class name 2017-08-05 16:29:41 -04:00
Kiran Koduru 12409a0cf6 Fix broken encoding on text for py 3 2017-08-05 15:40:38 -04:00
kirankoduru 7adab61a7a Added test for NotImplemented Spider.parse method 2017-08-05 15:40:38 -04:00
kirankoduru be71f98e92 Explicit message for scrapy parse callback 
The scrapy parse method raises a NotImplementedError when not defined,
but for new comers it can be hard to debug what might be going wrong.

Adding an explicit message for NotImplementedError will help new users.
 2017-08-05 15:40:38 -04:00
Paul Tremberth a68a8f8fdc Merge remote-tracking branch 'origin/1.4' 2017-08-05 11:45:50 +02:00
cclauss 0a69a32b5a Force Travis CI to test again 2017-08-04 14:35:43 +02:00
cclauss c016a4309d # noqa to close #2836 
Marks #2836 as will not fix.
 2017-08-04 01:44:23 +02:00
Paul Tremberth 0cb3085f84 Add test for alternate links 2017-08-03 16:55:09 +02:00
Eugene Vorobev 01ac883893 Follow alternate link for all types of sitemaps 2017-08-03 16:55:09 +02:00
david watson 71d5b7d75a fix typo (#2867) 2017-08-01 14:49:22 -03:00
Andrei Petre 6e6b5cc29f Use getfullargspec under the scenes for py3 to stop DeprecationWarning (#2864) 
Use getfullargspec under the scenes for py3 to stop DeprecationWarning. 

Closes #2862
 2017-08-01 11:14:43 -03:00
Daniel Graña 19382c5675 Merge pull request #2755 from redapple/downloader-mdw-template 
[MRG+1] Add template for a downloader middleware
 2017-07-31 11:37:26 -03:00
simik-ru a65fec050a Small fix in description of startproject arguments 2017-07-31 13:03:49 +02:00
Paul Tremberth aaaa4da7a4 Add template for a downloader middleware 2017-07-28 13:48:23 +02:00
Mikhail Korobov 9d34b2fe16 Merge pull request #2857 from redapple/scrapy-components-logs-startup 
Log versions information at startup
 2017-07-28 00:02:06 +05:00
Paul Tremberth bf7ef3e4c3 Move methods to a new scrapy.utils.versions 2017-07-27 20:08:30 +02:00
Paul Tremberth 219c8aa0b6 Log versions information at startup 2017-07-27 17:30:30 +02:00
Daniel Graña 5d9bac789d Merge pull request #2849 from cclauss/patch-2 
[MRG+1] xrange() --> range() for Python 3
 2017-07-26 15:59:37 -03:00
Daniel Graña bbc56e6960 Merge pull request #2852 from starrify/new-http-codes-to-retry-2851 
Added: HTTP status code 522/524 to retry.
 2017-07-26 15:46:35 -03:00