RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,001 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

5001 Commits

Author SHA1 Message Date
Daniel Graña 4e2885c9bf Use priority queues for Downloader slot queues 
closes #1731
 2015-08-18 23:53:09 -03:00
Mikhail Korobov 9adb5c31c0 Merge pull request #1433 from scrapy/codecov 
Coverage and reports at codecov.io and coveralls.io
 2015-08-14 05:21:10 +05:00
Daniel Graña 27077d2b6c Forward coverage envvars needed to cover subprocess 2015-08-13 21:07:45 -03:00
Daniel Graña 43fc2f23b8 Increase coverage of version command 2015-08-13 19:30:06 -03:00
Daniel Graña 54f9837804 Document how to get coverage report 2015-08-13 19:02:51 -03:00
Daniel Graña cb9577a6d4 Do not be verbose with coverage report by default 2015-08-13 19:02:36 -03:00
Daniel Graña 108c3e63a5 Add coverage report trough codecov.io 2015-08-13 13:56:24 -03:00
Daniel Graña 12bebb6172 Merge pull request #1431 from dangra/moved-to-parsel 
Replace usage of deprecated class by its parsel\'s counterpart
 2015-08-12 14:29:34 -03:00
Daniel Graña 6c2c9a4baf Merge pull request #1430 from dangra/openssl-version 
Add openssl version to version command
 2015-08-12 14:24:43 -03:00
Daniel Graña 7f634598b6 Add openssl version to version command 2015-08-12 14:24:24 -03:00
Daniel Graña 4ce0f53f7e Replace usage of deprecated class by its parsel\'s counterpart 2015-08-12 14:21:55 -03:00
Daniel Graña 68d2311298 Merge pull request #1427 from scrapy/python3-porting-status 
add Python 3 porting status badge to README
 2015-08-12 11:15:33 -03:00
Mikhail Korobov 1ed6a01c4b Add Python 3 porting status badge to the README 2015-08-12 19:13:54 +05:00
Daniel Graña 15c1300d35 Merge pull request #1409 from eliasdorneles/migrate-parsel 
[MRG+1] Migrating selectors to use parsel
 2015-08-11 16:22:58 -03:00
Elias Dorneles a5abd19e84 make Parsel's Selector more private, remove direct dependency of ParselSelectorList 2015-08-11 15:58:29 -03:00
Elias Dorneles 766c255152 upgrade parsel and add shim for deprecated selectorlist methods 2015-08-11 15:20:33 -03:00
Daniel Graña 457b97c13c Merge pull request #1388 from scrapy/dupefilter-persist 
Dupefilter persistence
 2015-08-11 14:16:34 -03:00
Daniel Graña e2f31f3018 explicit close file on file:// scheme handler 2015-08-11 14:09:49 -03:00
Elias Dorneles e50610bd3a set base_url in kwargs to be fully backward compatible 2015-08-11 14:09:34 -03:00
Elias Dorneles 8ef5aa2ffc using bytes for response body in tests 2015-08-11 14:09:34 -03:00
Elias Dorneles 3a03ef7c08 cleanup tests for selectors and translators 2015-08-11 14:09:34 -03:00
Elias Dorneles 12579b9afa warning when ambiguous root arguments and minor cleanups 2015-08-11 14:09:34 -03:00
Elias Dorneles 26ebccd37a upgrade parsel and use its function to instantiate root for finding form 2015-08-11 14:09:34 -03:00
Elias Dorneles 2fe6d128f5 upgrade parsel and using promoted root attribute 2015-08-11 14:09:16 -03:00
Elias Dorneles 67c98b185b avoid harcoded check for selector type 2015-08-11 14:09:00 -03:00
Elias Dorneles 94c3a345b7 remove deprecated module lxmldocument 2015-08-11 14:09:00 -03:00
Elias Dorneles 6287fc3109 remove lxmldocument dependency from http.request.form 2015-08-11 14:09:00 -03:00
Elias Dorneles 35c1dcdbc2 use response.selector in link extractors instead of instantiating new Selector 2015-08-11 14:09:00 -03:00
Elias Dorneles 17d7347a36 update minimal parsel version, add deprecated classes for csstranslator modules, fix test 2015-08-11 14:09:00 -03:00
Elias Dorneles 01d948f0fd remove selector support for LxmlDocument DOM cache and add deprecation warning for _root argument 2015-08-11 14:08:43 -03:00
Elias Dorneles 3a572e2f3b cleanup csstranslator module, keeping only imports 2015-08-11 14:08:43 -03:00
Elias Dorneles c7b29d118d fix support to legacy _root argument 2015-08-11 14:08:43 -03:00
Elias Dorneles ce21884a97 migrating scrapy Selector to use Parsel 2015-08-11 14:08:43 -03:00
Daniel Graña f67a808d9e Add 1.0.3 release notes 2015-08-11 13:49:49 -03:00
Daniel Graña 342c29af5d Merge pull request #1425 from scrapy/service-identity 
add service_identity to scrapy install_requires. Fix GH-1418 and GH-850.
 2015-08-11 13:34:36 -03:00
Mikhail Korobov 75d3d03382 add service_identity to scrapy install_requires 2015-08-11 20:14:22 +05:00
Mikhail Korobov fa123b335b Merge pull request #1421 from dangra/nyov/lazyload-downloadhandlers 
lazy-load downloadhandlers (continuation of #1357)
 2015-08-11 18:59:47 +05:00
Daniel Graña 7717501ab2 Use log formatting and pass crawler reference 2015-08-11 10:38:31 -03:00
Mikhail Korobov fa3d84b050 Merge pull request #1420 from scrapy/py3-trackrefs 
PY3: port scrapy.utils.trackref
 2015-08-11 18:21:34 +05:00
Daniel Graña 8738521d0a Add docstrings 2015-08-10 19:35:57 -03:00
Daniel Graña eb44152a58 lints 2015-08-10 18:25:11 -03:00
Daniel Graña 15ccf79cad Log errors importing or initializing download handlers 2015-08-10 18:18:13 -03:00
Daniel Graña d3804b3439 log errors importing or instanciating handlers 2015-08-10 16:52:44 -03:00
nyov 8c7997083f lazy-loading for DownloadHandlers 2015-08-10 16:52:44 -03:00
Daniel Graña 666ebfa1d9 py3: port dictionary itervalues call 2015-08-10 16:45:27 -03:00
Mikhail Korobov bdcc78b4dd Merge pull request #1419 from scrapy/httprepr-on-file-scheme 
Do not fail representing non-http requests
 2015-08-11 00:37:00 +05:00
Daniel Graña accd28cc71 PY3: port scrapy.utils.trackref 2015-08-10 15:13:29 -03:00
Daniel Graña 8d45b3c481 Do not fail representing non-http requests 2015-08-10 14:10:58 -03:00
Daniel Graña 5c4666a3d4 Workaround for travis#253 2015-08-10 11:17:20 -03:00
Daniel Graña 1b9b211192 Merge pull request #1415 from scrapy/nyov-py3 
nyov's PY3 changes
 2015-08-08 12:23:44 -03:00