RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,478 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

5478 Commits

Author SHA1 Message Date
Paul Tremberth 2bcaff97b4 Fix typo in comment 2016-02-24 17:21:08 +01:00
Paul Tremberth e25052e4ef Add "Content-Length: 0" for body-less HTTP/1.1 POST requests 
GH-823 was fixed only for HTTP/1.0 (in GH-1089)
 2016-02-24 17:21:08 +01:00
Paul Tremberth a6bc758db1 Merge pull request #1792 from scrapy/backport-1.1-pr1761 
[backport][1.1.x] Py3 S3 botocore
 2016-02-19 11:17:20 +01:00
Paul Tremberth eafd054092 Merge pull request #1793 from scrapy/backport-1.1-pr1786 
[backport][1.1.x] More liberal Content-Disposition header parsing
 2016-02-18 23:23:16 +01:00
Paul Tremberth 3331cea788 More liberal Content-Disposition header parsing 
Fixes #1782
 2016-02-18 22:56:17 +01:00
Konstantin Lopuhin 25e481fd14 [backport][1.1.x] Py3 S3 botocore 
Originally from https://github.com/scrapy/scrapy/pull/1761
 2016-02-18 22:48:37 +01:00
Mikhail Korobov 7ce32422c4 Merge pull request #1774 from scrapy/backport-1.1 
[MRG+1] [1.1.x][backport] #1766, #1770, #1750, #1662, #1765
 2016-02-15 16:05:31 +05:00
Elias Dorneles f928014d91 add deprecation for pydispatch (thanks for the help @redapple) 2016-02-15 11:40:08 +01:00
Paul Tremberth b1fac01258 Update release notes 2016-02-11 18:44:26 +01:00
Nicolas Pennequin 9cdbe177b7 Update the email doc for the charset argument to send(). 2016-02-09 19:52:06 +01:00
Nicolas Pennequin 72229f92de MailSender.send: allow passing a charset. 
Resolves Issue #348
 2016-02-09 19:52:06 +01:00
Mikhail Korobov 8aca66252a more efficient ExecutionEngine.spider_is_idle 2016-02-09 19:52:06 +01:00
orangain 281e9bd046 PY3: Implement some attributes of WrappedRequest required in Python 3 
This will fix #1770.
 2016-02-09 16:10:58 +01:00
orangain 522c71bb8a Fix SitemapSpider to extract sitemap urls from robots.txt properly 
This will fix #1766.
 2016-02-09 16:10:58 +01:00
Daniel Graña 56ad73fe99 Enable travis builds on tag patterns 2016-02-03 23:11:17 -03:00
Elias Dorneles a0b63a7707 Bump version: 1.1.0dev1 → 1.1.0rc1 2016-02-03 21:52:47 -02:00
Elias Dorneles 94b2c47da5 Merge pull request #1751 from scrapy/release-notes-1.1 
[MRG+1] Release notes 1.1
 2016-02-03 21:51:10 -02:00
Elias Dorneles ca08314654 makes more explicit source of problems in Python 3 limitations 2016-02-03 21:38:25 -02:00
Elias Dorneles 2167573b32 fix comment about disabling robots.txt 2016-02-03 21:31:27 -02:00
Elias Dorneles 3180abc800 applying more review comments 2016-02-03 21:28:05 -02:00
Elias Dorneles d40add7b27 add note about robots.txt waiting and make it explicit builtin extensions only are ported 2016-02-03 21:18:25 -02:00
Elias Dorneles 9cfefd5277 favor beta over basic to describe PY3 support 2016-02-03 20:59:56 -02:00
Elias Dorneles 2f697ce59f added note about how to revert to old behavior for robotstxt 2016-02-03 20:51:26 -02:00
Elias Dorneles 89a088b3c3 applying review comments 2016-02-03 20:45:55 -02:00
Daniel Graña a214b131fc Merge pull request #1748 from scrapy/py3-crawlerrunner-stop 
[MRG+1] PY3 fixed CrawlerRunner.stop
 2016-02-03 19:41:43 -03:00
Paul Tremberth 972f192568 Blank line before CoC 2016-02-03 22:58:49 +01:00
Paul Tremberth 728e505e53 Merge remote-tracking branch 'origin/master' into release-notes-1.1 2016-02-03 22:47:15 +01:00
Paul Tremberth 725a209e83 Merge pull request #1755 from stummjr/add-overview-to-release-notes 
Add overview to release notes and other edits
 2016-02-03 22:37:02 +01:00
Valdir Stumm Jr dd7b0eb359 Updates to release notes 2016-02-03 19:23:02 -02:00
Paul Tremberth 51dc741fa2 Add 1.0.5 release notes 2016-02-03 21:07:21 +01:00
Valdir Stumm Jr 0f816524e8 edit API changes, Deprecations, and Relocations sections. 2016-02-03 16:21:32 -02:00
Paul Tremberth fd3193da86 Reword some of "New Features and Enhancements" 2016-02-03 18:19:38 +01:00
Paul Tremberth 3b2a6abc0b Format "Code refactoring" subsection 2016-02-03 17:37:58 +01:00
Paul Tremberth ed02455010 Format "Logging" subsection 2016-02-03 17:27:43 +01:00
Elias Dorneles a8a6f050e7 Merge pull request #1735 from ArturGaspar/master 
[MRG+1] Fix for KeyError in robots.txt middleware
 2016-02-03 13:30:06 -02:00
Paul Tremberth e4cd95444c Merge pull request #1752 from stummjr/release-notes-1.1 
Release notes: edits on "Py3 Support" and "New features" sections
 2016-02-03 16:17:59 +01:00
Mikhail Korobov 43a53aca12 Merge pull request #1746 from redapple/shell-settings-logging 
[MRG+1] Remove __str__ and __repr__ from settings, introduce copy_to_dict()
 2016-02-03 19:59:24 +05:00
Valdir Stumm Jr 1fd95c9c9f fixed references to issues and fit text to 80 cols in the first two sections 2016-02-03 12:45:48 -02:00
Valdir Stumm Jr a21c90bee7 edits on Py3 Support and New features sections 2016-02-03 11:54:46 -02:00
Paul Tremberth 44d8df2060 Add versionadded note for MEMUSAGE_CHECK_INTERVAL_SECONDS 2016-02-03 12:35:26 +01:00
Paul Tremberth 2b033eebce Fix recently added HTTPCACHE_ settings versionadded notes 2016-02-03 12:34:44 +01:00
Paul Tremberth 142aa21737 Add AUTOTHROTTLE_TARGET_CONCURRENCY versionadded note 2016-02-03 12:33:10 +01:00
Paul Tremberth db0697bc06 Add 1.1 release notes (draft) 2016-02-03 12:32:40 +01:00
Mikhail Korobov 65fb67f2db PY3 fixed CrawlerRunner.stop 2016-02-03 01:01:16 +05:00
Nicolás Alejandro Ramírez Quiros 8a391a557f Merge pull request #1741 from scrapy/py3-downloader-gc 
[MRG+1] PY3 fix downloader slots GC
 2016-02-02 12:17:52 -03:00
Julia Medina f2668c316f Merge pull request #1747 from stummjr/fix-xml-exporter-for-nonstring-types 
[MRG+1] Fix bug on XMLItemExporter with non-string fields in items
 2016-02-02 01:35:39 -03:00
Paul Tremberth 268e912273 Add pretty-printting of settings as dict if using IPython shell 
Suggested by @digenis
see http://ipython.readthedocs.org/en/stable/api/generated/IPython.lib.pretty.html?#extending
 2016-02-01 12:43:27 +01:00
stummjr bb2cf7c0d7 Fixed bug on XMLItemExporter with non-string fields in items 2016-01-30 10:00:06 -02:00
Mikhail Korobov f9dc02e23a PY3 fix downloader slots GC 2016-01-30 04:37:22 +05:00
Paul Tremberth d843a0aae8 Amend "settings" command to output JSON for dict settings 2016-01-29 21:12:03 +01:00