RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,424 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

5424 Commits

Author SHA1 Message Date
stummjr bb2cf7c0d7 Fixed bug on XMLItemExporter with non-string fields in items 2016-01-30 10:00:06 -02:00
Mikhail Korobov a35aec71e9 Merge pull request #1710 from redapple/1550-shell_file-cont 
[MRG+1] shell command's ability to open local files + tests
 2016-01-29 21:03:40 +05:00
Paul Tremberth 78f00401cd Remove unused import in tests 2016-01-29 16:56:05 +01:00
Mikhail Korobov 505e87258c Merge pull request #1742 from stummjr/add-tests-to-exporters 
[MRG+1] Include tests for non-string items to Exporters
 2016-01-29 03:27:38 +05:00
stummjr cf2ebb0687 Include tests for exporters: JSON, JSON-Lines, Pickle and Marshal. 2016-01-28 19:43:53 -02:00
Paul Tremberth e9f6b98816 Amend guess_scheme() docstring 2016-01-28 16:26:45 +01:00
Paul Tremberth c6f374f2eb Merge remote-tracking branch 'origin/master' into 1550-shell_file-cont 2016-01-28 14:02:48 +01:00
Paul Tremberth 481e251775 Move guess_scheme() tests to relevant test module 2016-01-28 13:51:50 +01:00
Paul Tremberth cae268402d Move guess_scheme() to scrapy.utils.url 2016-01-28 13:42:04 +01:00
Mikhail Korobov 4c9a6ef3ce Merge pull request #1740 from scrapy/response-text-micro-optimize 
[MRG+1] micro-optimize response.text
 2016-01-28 01:39:48 +05:00
Mikhail Korobov 3e080c3c52 call .text from .body_as_unicode() and not the other way around 2016-01-28 00:59:27 +05:00
Mikhail Korobov 394b991e3a Merge pull request #1737 from stummjr/fix-py3-exporters-multiple-types 
[MRG+1] Fix PythonItemExporter and CSVExporter for non-string item types
 2016-01-28 00:46:00 +05:00
stummjr 27758f60ad Changes fallback for CSVItemExporter, avoiding to call to_native_str(str()). 2016-01-27 16:28:01 -02:00
Paul Tremberth 27fb200e45 Merge pull request #1727 from redapple/priority-adjust 
[MRG+1] Clarify priority adjust settings docs
 2016-01-27 18:52:39 +01:00
stummjr c55ff110a3 Fix CSV exporter for non string Python types. 2016-01-27 15:43:17 -02:00
stummjr f1d971a5c0 fix PythonItemExporter for non-string types 2016-01-27 14:34:46 -02:00
Mikhail Korobov 7d24df3738 Merge pull request #1681 from redapple/code-of-conduct 
[MRG+1] Add Code of Conduct Version 1.3.0 from http://contributor-covenant.org/
 2016-01-27 17:56:11 +05:00
Mikhail Korobov 7ca9ae1976 DOC typo fix 2016-01-27 17:54:28 +05:00
Elias Dorneles 77951095e0 Merge pull request #1730 from scrapy/response-text 
[MRG+1] response.text
 2016-01-27 10:45:28 -02:00
Paul Tremberth e0f48c486e Add link to CoC mardown file on Github 2016-01-27 13:04:08 +01:00
Paul Tremberth d999e3f7a7 More explicit description of DEPTH_PRIORITY 2016-01-27 12:57:03 +01:00
Paul Tremberth 6ed08d2332 Add note for DEPTH_PRIORITY 2016-01-27 11:53:29 +01:00
Mikhail Korobov 4bcbb77bcc response.text. Fixes GH-1729. 2016-01-27 01:28:11 +05:00
Paul Tremberth 1c83108893 Clarify priority adjust settings docs 
Fixes #1593
 2016-01-26 19:24:11 +01:00
Daniel Graña 85f0596c43 Merge pull request #1725 from redapple/spiderstate-notconfigured 
[MRG+1] Disable SpiderState extension if no JOBDIR set
 2016-01-26 14:32:17 -03:00
Mikhail Korobov 6660175de9 Merge pull request #1726 from redapple/mwlist-logging-long 
Use long classes names for enabled middlewares in startup logs
 2016-01-26 21:29:44 +05:00
Paul Tremberth bb1f4013a3 Rewrite warning about shell with local files as note 2016-01-26 17:23:28 +01:00
Paul Tremberth c22a4e3bb8 Use long classes names for enabled middlewares in startup logs 2016-01-26 16:41:16 +01:00
Paul Tremberth 29695375d1 Add test for raised exception with SpiderState extension when no JOBDIR used 2016-01-26 16:33:24 +01:00
Paul Tremberth 0349bbf9d3 Disable SpiderState extension if no JOBDIR set 2016-01-26 15:25:15 +01:00
Paul Tremberth 0d368c5d6f Merge pull request #1724 from scrapy/robotstxt-default 
[MRG+1] Enable robots.txt handling by default for new projects.
 2016-01-26 14:19:15 +01:00
Mikhail Korobov f30758c246 Enable robots.txt handling by default for new projects. Fixes GH-1668. 
For backwards compatibility reasons the default value is not changed.
 2016-01-26 17:47:46 +05:00
Elias Dorneles 2246280bb6 Merge pull request #1723 from redapple/closespider-notconfigured 
[MRG+1] Disable CloseSpider extension if no CLOSPIDER_* setting set
 2016-01-26 10:35:59 -02:00
Paul Tremberth 52daa1b953 Merge pull request #1720 from scrapy/deprecations 
[MRG+1] deprecated unused and untested code in scrapy.utils.datatypes
 2016-01-26 13:25:21 +01:00
Mikhail Korobov 1e48574576 Merge pull request #1722 from redapple/mwlist-logging 
[MRG+1] Fix logging of enabled middlewares
 2016-01-26 17:21:08 +05:00
Paul Tremberth 6ee8d8650a Disable CloseSpider extension if no CLOSPIDER_* setting set 2016-01-26 13:08:42 +01:00
Paul Tremberth 7608da8868 Fix logging of enabled middlewares 
Wrong middlewares list was being pretty-printed
(introduced in #1263)
 2016-01-26 13:01:12 +01:00
Elias Dorneles de22b6f3ac Merge pull request #1721 from Digenis/offsitemw_subdomains 
[MRG+1] tests+doc for subdomains in offsite middleware
 2016-01-26 09:37:18 -02:00
Νικόλαος-Διγενής Καραγιάννης 1cffa99e0d tests+doc for subdomains in offsite middleware 2016-01-26 12:49:43 +02:00
Paul Tremberth 713e1eee9b Update docs about local files support for "scrapy shell" 2016-01-26 10:44:38 +01:00
Mikhail Korobov 9c2aa50ea2 deprecate unused and untested scrapy.utils.datatypes.MultiValueDict 2016-01-26 13:58:56 +05:00
Mikhail Korobov 7070dae48d deprecate unused and untested scrapy.utils.datatypes.SiteNode 2016-01-26 13:56:16 +05:00
Daniel Graña a7b86137d0 Merge pull request #1499 from scrapy/py3-port-exporters 
[MRG+1] PY3 exporters
 2016-01-26 01:01:30 -03:00
Daniel Graña d0eacfe0f9 Add test case for marshal item exporter 2016-01-26 00:26:27 -03:00
Elias Dorneles 2dfdde3c79 fallback to repr when can't convert to native string 2016-01-25 22:24:35 -02:00
Elias Dorneles 23b3336c1f add test for invalid option 2016-01-25 22:11:04 -02:00
Daniel Graña b20463183c Merge pull request #1716 from lopuhin/py3-test-pipeline-files-images 
[MRG+1] py3: test_pipeline_files and test_pipeline_images
 2016-01-25 18:24:12 -03:00
Paul Tremberth fb8ab2427b Move urlparsing statement in add_http_if_no_scheme() 2016-01-25 13:13:35 +01:00
Elias Dorneles 0c44fac2b5 added tests for feed export marshal and pickle 2016-01-24 19:17:42 -02:00
Elias Dorneles 4fef2c7628 Merge pull request #1717 from lopuhin/py3-test-spidermiddleware-httperror 
[MRG+1] py3: reviewed passing test_spidermiddleware_httperror.py
 2016-01-24 18:49:22 -02:00