RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,505 Commits 49 Branches 132 Tags 144 MiB
Commit Graph

5505 Commits

Author SHA1 Message Date
Elias Dorneles 89a088b3c3 applying review comments 2016-02-03 20:45:55 -02:00
Daniel Graña a214b131fc Merge pull request #1748 from scrapy/py3-crawlerrunner-stop 
[MRG+1] PY3 fixed CrawlerRunner.stop
 2016-02-03 19:41:43 -03:00
Paul Tremberth 972f192568 Blank line before CoC 2016-02-03 22:58:49 +01:00
Paul Tremberth 728e505e53 Merge remote-tracking branch 'origin/master' into release-notes-1.1 2016-02-03 22:47:15 +01:00
Paul Tremberth 725a209e83 Merge pull request #1755 from stummjr/add-overview-to-release-notes 
Add overview to release notes and other edits
 2016-02-03 22:37:02 +01:00
Valdir Stumm Jr dd7b0eb359 Updates to release notes 2016-02-03 19:23:02 -02:00
Paul Tremberth 51dc741fa2 Add 1.0.5 release notes 2016-02-03 21:07:21 +01:00
Valdir Stumm Jr 0f816524e8 edit API changes, Deprecations, and Relocations sections. 2016-02-03 16:21:32 -02:00
Paul Tremberth fd3193da86 Reword some of "New Features and Enhancements" 2016-02-03 18:19:38 +01:00
Paul Tremberth 3b2a6abc0b Format "Code refactoring" subsection 2016-02-03 17:37:58 +01:00
Paul Tremberth ed02455010 Format "Logging" subsection 2016-02-03 17:27:43 +01:00
Elias Dorneles a8a6f050e7 Merge pull request #1735 from ArturGaspar/master 
[MRG+1] Fix for KeyError in robots.txt middleware
 2016-02-03 13:30:06 -02:00
Paul Tremberth e4cd95444c Merge pull request #1752 from stummjr/release-notes-1.1 
Release notes: edits on "Py3 Support" and "New features" sections
 2016-02-03 16:17:59 +01:00
Mikhail Korobov 43a53aca12 Merge pull request #1746 from redapple/shell-settings-logging 
[MRG+1] Remove __str__ and __repr__ from settings, introduce copy_to_dict()
 2016-02-03 19:59:24 +05:00
Valdir Stumm Jr 1fd95c9c9f fixed references to issues and fit text to 80 cols in the first two sections 2016-02-03 12:45:48 -02:00
Valdir Stumm Jr a21c90bee7 edits on Py3 Support and New features sections 2016-02-03 11:54:46 -02:00
Paul Tremberth 44d8df2060 Add versionadded note for MEMUSAGE_CHECK_INTERVAL_SECONDS 2016-02-03 12:35:26 +01:00
Paul Tremberth 2b033eebce Fix recently added HTTPCACHE_ settings versionadded notes 2016-02-03 12:34:44 +01:00
Paul Tremberth 142aa21737 Add AUTOTHROTTLE_TARGET_CONCURRENCY versionadded note 2016-02-03 12:33:10 +01:00
Paul Tremberth db0697bc06 Add 1.1 release notes (draft) 2016-02-03 12:32:40 +01:00
Mikhail Korobov 65fb67f2db PY3 fixed CrawlerRunner.stop 2016-02-03 01:01:16 +05:00
Nicolás Alejandro Ramírez Quiros 8a391a557f Merge pull request #1741 from scrapy/py3-downloader-gc 
[MRG+1] PY3 fix downloader slots GC
 2016-02-02 12:17:52 -03:00
Julia Medina f2668c316f Merge pull request #1747 from stummjr/fix-xml-exporter-for-nonstring-types 
[MRG+1] Fix bug on XMLItemExporter with non-string fields in items
 2016-02-02 01:35:39 -03:00
Paul Tremberth 268e912273 Add pretty-printting of settings as dict if using IPython shell 
Suggested by @digenis
see http://ipython.readthedocs.org/en/stable/api/generated/IPython.lib.pretty.html?#extending
 2016-02-01 12:43:27 +01:00
stummjr bb2cf7c0d7 Fixed bug on XMLItemExporter with non-string fields in items 2016-01-30 10:00:06 -02:00
Mikhail Korobov f9dc02e23a PY3 fix downloader slots GC 2016-01-30 04:37:22 +05:00
Paul Tremberth d843a0aae8 Amend "settings" command to output JSON for dict settings 2016-01-29 21:12:03 +01:00
Paul Tremberth aa78758bc7 Update tests for settings copy_to_dict() 2016-01-29 18:59:12 +01:00
Paul Tremberth a1ebff83d3 Remove __str__ and __repr__ from settings, introduce copy_to_dict() instead 
Settings instances as dict's are easier to print or pretty print in the shell

Fixes #1732
 2016-01-29 18:39:34 +01:00
Mikhail Korobov a35aec71e9 Merge pull request #1710 from redapple/1550-shell_file-cont 
[MRG+1] shell command's ability to open local files + tests
 2016-01-29 21:03:40 +05:00
Paul Tremberth 78f00401cd Remove unused import in tests 2016-01-29 16:56:05 +01:00
Mikhail Korobov 505e87258c Merge pull request #1742 from stummjr/add-tests-to-exporters 
[MRG+1] Include tests for non-string items to Exporters
 2016-01-29 03:27:38 +05:00
stummjr cf2ebb0687 Include tests for exporters: JSON, JSON-Lines, Pickle and Marshal. 2016-01-28 19:43:53 -02:00
Paul Tremberth e9f6b98816 Amend guess_scheme() docstring 2016-01-28 16:26:45 +01:00
Paul Tremberth c6f374f2eb Merge remote-tracking branch 'origin/master' into 1550-shell_file-cont 2016-01-28 14:02:48 +01:00
Paul Tremberth 481e251775 Move guess_scheme() tests to relevant test module 2016-01-28 13:51:50 +01:00
Paul Tremberth cae268402d Move guess_scheme() to scrapy.utils.url 2016-01-28 13:42:04 +01:00
Artur Gaspar 90e3ae1c58 Do not forget failed requests in robots.txt middleware. 2016-01-27 21:00:35 -02:00
Mikhail Korobov 4c9a6ef3ce Merge pull request #1740 from scrapy/response-text-micro-optimize 
[MRG+1] micro-optimize response.text
 2016-01-28 01:39:48 +05:00
Mikhail Korobov 3e080c3c52 call .text from .body_as_unicode() and not the other way around 2016-01-28 00:59:27 +05:00
Mikhail Korobov 394b991e3a Merge pull request #1737 from stummjr/fix-py3-exporters-multiple-types 
[MRG+1] Fix PythonItemExporter and CSVExporter for non-string item types
 2016-01-28 00:46:00 +05:00
stummjr 27758f60ad Changes fallback for CSVItemExporter, avoiding to call to_native_str(str()). 2016-01-27 16:28:01 -02:00
Paul Tremberth 27fb200e45 Merge pull request #1727 from redapple/priority-adjust 
[MRG+1] Clarify priority adjust settings docs
 2016-01-27 18:52:39 +01:00
stummjr c55ff110a3 Fix CSV exporter for non string Python types. 2016-01-27 15:43:17 -02:00
stummjr f1d971a5c0 fix PythonItemExporter for non-string types 2016-01-27 14:34:46 -02:00
Artur Gaspar b2beb3e85d Fix handling of already failed deferreds when downloading page in robots.txt middleware. 2016-01-27 13:09:08 -02:00
Artur Gaspar dc8701ea42 Add test for already failed deferreds when downloading page in robots.txt middleware. 2016-01-27 12:56:42 -02:00
Mikhail Korobov 7d24df3738 Merge pull request #1681 from redapple/code-of-conduct 
[MRG+1] Add Code of Conduct Version 1.3.0 from http://contributor-covenant.org/
 2016-01-27 17:56:11 +05:00
Mikhail Korobov 7ca9ae1976 DOC typo fix 2016-01-27 17:54:28 +05:00
Elias Dorneles 77951095e0 Merge pull request #1730 from scrapy/response-text 
[MRG+1] response.text
 2016-01-27 10:45:28 -02:00