RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,590 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

4590 Commits

Author SHA1 Message Date
Mikhail Korobov af0635e229 fix tests to account for https://github.com/scrapy/w3lib/pull/23 2014-07-22 04:15:13 +06:00
Daniel Graña f6b1e9b7c2 Merge pull request #803 from felixonmars/py3-port 
[MRG] PY3: use six.BytesIO and six.moves.cStringIO
 2014-07-21 14:17:42 -03:00
Daniel Graña dd3b77ea2e Merge pull request #809 from nramirezuy/get_func_args-728 
get_func_args maximum recursion fix #728
 2014-07-21 13:41:03 -03:00
nramirezuy 86f61a9906 get_func_args maximum recursion fix #728 2014-07-21 13:12:59 -03:00
Pablo Hoffman e748ca50ca Merge pull request #804 from darkrho/issue-560 
Updated input/ouput processor example according to #560.
 2014-07-16 17:14:53 -03:00
Pablo Hoffman a17d417281 Merge pull request #805 from darkrho/spiderstate-jobdir 
For consistency, use `job_dir` helper in `SpiderState` extension.
 2014-07-16 17:11:20 -03:00
Rolando Espinoza 85b52b0654 For consistency, use `job_dir` helper in `SpiderState` extension. 
The `SpiderState` extension is the only place not using the `job_dir`
helper.

This might cause an error in case JOBDIR is a nested path which does not
exists, although this is least likely by default given the job dir being
set up earlier.
 2014-07-16 10:34:42 -04:00
Rolando Espinoza e441a01818 Updated input/ouput processor example according to #560. 2014-07-16 09:04:48 -04:00
Felix Yan 1cd9c4d653 fix tests by using a tempfile instead of BytesIO 2014-07-15 21:26:01 +08:00
Felix Yan 0786e84a33 use io.BytesIO and cStringIO instead of six.BytesIO as suggested 2014-07-15 20:51:12 +08:00
Felix Yan ef8872a518 use .seek(0) instead of reset() for compatibility 
Since I don't find docs for the .reset() method, I lookup up the source
code, and it should work just the same as .seek(0).

For reference:
8eeb7e9122/Modules/cStringIO.c (L281)
 2014-07-15 16:24:57 +08:00
Felix Yan 2999fc75b1 PY3: use six.BytesIO and six.moves.cStringIO 2014-07-15 15:52:10 +08:00
Mikhail Korobov 91387b6e55 Merge pull request #801 from felixonmars/py3-port 
PY3: fix xmlrpclib and email imports
 2014-07-14 21:49:40 +06:00
Felix Yan 61717ca075 use six.moves version for MIMEMultipart, MIMEText, and MIMEBase 2014-07-14 23:08:40 +08:00
Felix Yan 0d758e4b30 PY3: fix xmlrpclib and email imports 2014-07-14 22:18:57 +08:00
Mikhail Korobov 5a2f738fac Merge pull request #800 from felixonmars/py3-port 
PY3: use six for robotparser and urlparse
 2014-07-14 19:46:10 +06:00
Felix Yan 6f7efa1d1d PY3: use six for robotparser and urlparse 2014-07-14 21:26:37 +08:00
Mikhail Korobov 2de4b8dc14 TST clean temporary file explicitly 2014-07-14 18:00:20 +06:00
Mikhail Korobov 321a76569d Merge pull request #799 from felixonmars/py3-port 
PY3: use six.iterkeys, six.iteritems, and tempfile
 2014-07-14 17:52:25 +06:00
Felix Yan 4f24e724a3 PY3: use six.iterkeys, six.iteritems, and tempfile 2014-07-14 18:47:22 +08:00
Mikhail Korobov 2f567f94a5 Merge pull request #798 from felixonmars/py3-port 
PY3: fix has_key and use six.moves.configparser
 2014-07-14 16:17:30 +06:00
Felix Yan 3067d6d710 PY3: fix has_key and use six.moves.configparser 2014-07-14 18:09:11 +08:00
Mikhail Korobov 2cddc2a379 Merge pull request #797 from felixonmars/py3-port 
PY3: use six.moves.cPickle
 2014-07-14 15:44:51 +06:00
Felix Yan 389882eb13 PY3: use six.moves.cPickle 2014-07-14 17:15:28 +08:00
Daniel Graña e95e1fb79c Merge pull request #796 from redapple/patch-1 
Add PyPI Pin for Wheels compatibility
 2014-07-12 22:12:09 -03:00
Daniel Graña 0240ff8287 Merge pull request #791 from redapple/xhtmllinks 
Extract links from XHTML documents with MIME-Type "application/xml"
 2014-07-11 17:07:54 -03:00
Paul Tremberth 41ad55de20 Add PyPI Pin for Wheels compatibility 2014-07-11 21:53:10 +02:00
Alexander Chekunkov a2dd5e5ad9 Added referer to "Spider error processing" log message + fixed some pep8 issues 2014-07-11 16:21:57 +03:00
Mikhail Korobov 71685fa2d0 too much drama 2014-07-11 13:29:18 +06:00
Paul Tremberth 54d2b110f1 Extract links from XHTML documents with MIME-Type "application/xml" 
"application/xhtml+xml" is already interpreted as HTML
and link extractor is fine with it to extract links.

Only for XML documents can the namespaces in tags be an issue.

Fixes #780

Do both tag and attribute tests in _iter_links() method
 2014-07-10 23:50:48 +02:00
Roy Crihfield 1a950e8f5a Fix typo in commands.rst 2014-07-10 14:48:15 -03:00
Daniel Graña ef8b3372df Merge pull request #792 from redapple/travis-release-cond 
Fix Travis CI config for conditional releases
 2014-07-10 14:41:30 -03:00
Paul Tremberth 6e04c0b388 Match dot using character class (avoids escaping headaches) 2014-07-10 14:54:08 +02:00
Paul Tremberth 8d5c2fe30b Fix Travis CI config for conditional releases 2014-07-10 13:44:32 +02:00
Daniel Graña 0fd119dfec Bump version: 0.25.0 → 0.25.1 2014-07-09 03:05:39 -03:00
Daniel Graña 9b8e1836c8 deploy to pypi only on even minor version 2014-07-09 02:50:12 -03:00
Daniel Graña ea8e444bd6 better testcase for settings.overrides.setdefault 2014-07-09 01:41:27 -03:00
Pablo Hoffman 4bb0989833 Merge pull request #787 from ivannotes/master 
Using CRLF as line marker according to http 1.1 definition
 2014-07-08 19:37:01 -03:00
Daniel Graña 379f6ee6b2 Add 0.24.2 release notes 2014-07-08 05:53:35 -03:00
Daniel Graña acbde40fe7 Use a mutable mapping to proxy deprecated settings.overrides and settings.defaults attribute 2014-07-08 05:27:13 -03:00
ivan a9c7f8b26d Using CRLF as line marker according to http 1.1 definition 2014-07-08 14:37:44 +08:00
Mikhail Korobov e29eec9c2a Merge pull request #786 from zsiciarz/patch-1 
Fixed Python syntax in tutorial.
 2014-07-04 14:45:45 +06:00
Zbigniew Siciarz 0466e8cb7a Fixed Python syntax in tutorial. 2014-07-04 10:38:01 +02:00
Mikhail Korobov f3f3a440d3 fix a couple of typos in sep-19 2014-07-04 04:03:45 +06:00
Pablo Hoffman 19dc3fa053 fix typo 2014-07-03 17:54:21 -03:00
Mikhail Korobov a653bf38be Merge pull request #779 from Digenis/master 
selector.__repr__ test
 2014-07-03 20:31:15 +06:00
Nikolaos-Digenis Karagiannis aacf1d05e1 selector.__repr__ test, unicode query 2014-07-03 13:11:25 +03:00
Nikolaos-Digenis Karagiannis 9ac639abd3 selector.__repr__ test, rename method 2014-07-03 13:10:55 +03:00
Nikolaos-Digenis Karagiannis d2d014a4e9 selector.__repr__ test, repr type is bytesarray 2014-07-03 09:32:52 +03:00
Pablo Hoffman 36d99a0739 Merge pull request #777 from scrapy/deprecate-sgml 
Deprecate SgmlLinkExtractor
 2014-07-02 19:05:49 -03:00