RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,534 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

3534 Commits

Author SHA1 Message Date
Daniel Graña efb4f32d80 do no include ResponseFailed if http11 handler is not enabled 2013-08-21 19:04:12 -03:00
Daniel Graña 3c64a989ca New HTTP client wraps connection losts in ResponseFailed exception. fix #373 2013-08-21 17:28:26 -03:00
Pablo Hoffman 071172cbd6 fix retry middleware which didn't retry certain connection errors after the upgrade to http1 client, closes GH-373 2013-08-21 14:32:33 -03:00
Pablo Hoffman 6e4ed8b004 Merge pull request #372 from kmike/xml-exporter-fix-for-2.7.4 
fix XmlItemExporter in Python 2.7.4 and 2.7.5
 2013-08-21 09:55:51 -07:00
Mikhail Korobov a6e6ca06b9 fix XmlItemExporter in Python 2.7.4 and 2.7.5 2013-08-21 06:05:40 +06:00
Mikhail Korobov 034ffae60f Recommend Pillow instead of PIL. Closes GH-317. 2013-08-18 00:44:01 +06:00
Pablo Hoffman f95b164d06 Merge pull request #367 from berendiwema/master 
#327 - Support STARTTLS / SSL option in email sender
 2013-08-14 07:17:30 -07:00
Berend Iwema 32b6364bcd #327 - Support STARTTLS / SSL option in email sender 2013-08-14 12:59:01 +02:00
Pablo Hoffman 49952a45ae add docstring to PythonItemExporter 2013-08-14 02:49:14 -03:00
Pablo Hoffman c0b26e3d49 minor updates to 0.18 release notes 2013-08-14 01:39:44 -03:00
Daniel Graña c6157d37e9 Merge pull request #366 from kalessin/exporter 
added a python native classes item exporter
 2013-08-13 06:55:27 -07:00
olveyra 456b6f2ef5 added a python native classes item exporter 2013-08-13 13:48:28 +00:00
Daniel Graña b43b5f575e adjust http11 pool size to per-domain concurrency 2013-08-12 22:43:40 -03:00
Pablo Hoffman 892386ee01 tox.ini: disable sitepackages on windows, as a compiler is often not available 2013-08-12 18:59:41 -03:00
Daniel Graña ed5b9068d2 fix contributters list format 2013-08-12 11:26:16 -03:00
Daniel Graña 80e25c5980 bumped version to 0.19.0 2013-08-09 19:09:47 -03:00
Daniel Graña a6693c9a5c updated release notes and bumped version to 0.18.0 2013-08-09 19:02:28 -03:00
Pablo Hoffman 4dc76e7cca fixed scrapy.utils.gz.gunzip() broken after changes from Python 2.7.3 to 2.7.4 2013-08-09 18:20:04 -03:00
Daniel Graña 29a12af1b0 Merge pull request #361 from hmsimha/master 
Doc fixes: Minor typo (was fixed on 0.16 branch so I figured it could be on master as well) and an error in the overview doc.
 2013-08-05 09:10:22 -07:00
Hart c00c4d7148 correction to description of example XPath retrieval in overview doc 2013-08-03 17:08:58 -07:00
Hart 0ad01c34d4 fixed typo to parallel fix on 0.16 branch 2013-08-03 17:06:10 -07:00
Nicolás Alejandro Ramírez Quiros c2a4046f14 Merge pull request #359 from rocioar/master 
Added COMPRESSION_ENABLED setting
 2013-08-01 10:43:47 -07:00
Rocio Aramberri d227d530f6 Added COMPRESSION_ENABLED setting to enable or disable the HttpCompressionMiddleware 
Added COMPRESSION_ENABLE setting to docs

Added COMPRESSION_ENABLED setting to default settings
 2013-08-01 11:31:28 -03:00
arijitchakraborty fb770852e8 Skipping cookie retrieval for non http requests 2013-07-22 11:33:26 -03:00
Mikhail Korobov e1f0d0d45d Merge pull request #352 from arijitchakraborty/bugfix/cookie_retrieval_for_hosts_with_port 
Fixes for bug - Cookie retrieval for hosts with port + unittests for the...
 2013-07-18 09:21:13 -07:00
arijitchakraborty 66ff34cf05 improving hostname extraction 2013-07-18 20:51:56 +05:30
arijitchakraborty ebc136ddb0 Fixes for bug - Cookie retrieval for hosts with port + unittests for the fix 2013-07-18 19:42:58 +05:30
Daniel Graña d7e11082f2 it is not possible to enforce an upper limit when latency is out of control 2013-07-18 11:07:42 -03:00
Daniel Graña 3eb5d5e931 Merge pull request #330 from nramirezuy/spider-from_crawler 
use from_crawler method if available in spiders
 2013-07-16 12:42:31 -07:00
Daniel Graña d14541c5f1 Merge pull request #205 from joehillen/master 
PyPy support
 2013-07-16 12:34:17 -07:00
Daniel Graña d51f1ff33d Merge pull request #350 from dellis23/master 
This pull request fixes #349
 2013-07-16 12:33:21 -07:00
Daniel Graña d74b60051d take in count response latencies when testing download delays 2013-07-16 16:22:29 -03:00
Dan 1ca31244b0 Fixed ordering of super argument call. 2013-07-16 14:50:10 -04:00
Dan e12b689c4f Updated documentation of spider arguments to include required super call. 2013-07-16 14:26:53 -04:00
Daniel Graña 891fa980d2 Merge pull request #341 from kmike/test-running 
Improved tox test running & fixed django tests
 2013-07-16 11:16:43 -07:00
Daniel Graña 66aa1331fc Twisted pre 10.2 (lucid) can not setup the required ftp server for tests 2013-07-16 00:49:35 -03:00
Daniel Graña 26c7ec7d9f Merge pull request #329 from kalessin/ftphandler 
added ftp handler
 2013-07-15 16:39:46 -07:00
Daniel Graña f8c105d5c5 Merge branch 'gh-347'. thanks @chrisboo 2013-07-15 12:28:24 -03:00
Daniel Graña 9ff82d6695 use lxml parser with recover option to parse invalid sitemaps. closes #347 
thanks chrissboo for reporting the issue and proposing a fix.
 2013-07-15 12:26:16 -03:00
Mikhail Korobov 1a1c93fafe tiny FormRequest doc fix 2013-07-15 15:47:34 +06:00
Daniel Graña aa7b529975 do not log Starting/Stopping factory messages 2013-07-10 12:35:15 -03:00
Mikhail Korobov 3fe2a32683 handle GET parameters for AJAX crawlable URLs: 2013-07-09 01:58:50 +06:00
Mikhail Korobov 8c555fd467 Enable Travis CI for pull requests and feature branches. Fix GH-340. 2013-07-09 01:50:52 +06:00
Mikhail Korobov 0d9f7843ba allow passing arguments to runtests script via tox 2013-07-09 00:41:13 +06:00
Mikhail Korobov f29e5d0ca1 reuse runtests.sh and runtests.bat scripts in tox.ini 2013-07-09 00:38:00 +06:00
Mikhail Korobov 3a6dff3090 make tox test running more consistent with bin/runtests.sh script 2013-07-09 00:30:39 +06:00
Mikhail Korobov 2105ec58eb add django to test requirements and fix test_djangoitem for modern django versions 2013-07-09 00:28:39 +06:00
Mikhail Korobov b98e80cbb8 use tox deps instead of "manual" pip install; add lucid and precise environments to tox.ini 2013-07-09 00:26:59 +06:00
Mikhail Korobov 35fbec2e63 allow passing arguments to tox 
e.g. "tox -- scrapy.tests.test_contrib_loader" runs test_contrib_loader for Python 2.6 and 2.7
 2013-07-08 22:51:03 +06:00
Daniel Graña 6db259c075 Merge pull request #338 from kmike/patch-3 
DownloaderMiddleware docs fix
 2013-07-08 07:12:42 -07:00