RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,838 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

4838 Commits

Author SHA1 Message Date
Daniel Graña ff0c915fec udpate memusage docs 2015-07-01 05:11:12 -03:00
Daniel Graña c9f3decddd Use psutil if available to measure RSS on memusage extension 2015-07-01 04:50:21 -03:00
Daniel Graña 6fa3f2421c Add 1.0.1 release notes 2015-07-01 01:49:44 -03:00
Daniel Graña 07f4f12e8b Merge pull request #1313 from eliasdorneles/support-empty-passwd-for-http-proxy 
[MRG+1] Support empty password for http_proxy config
 2015-06-30 19:58:16 -03:00
Daniel Graña 98707e15f6 Merge pull request #1324 from scrapy/autothrottle 
Autothrottle enhancements
 2015-06-26 22:26:06 -03:00
Mikhail Korobov d850238c22 add AUTOTHROTTLE_TARGET_CONCURRENCY option and expand AutoThrottle docs 2015-06-27 04:59:42 +05:00
Mikhail Korobov 63317531f9 DOC fix authrottle docs 
see https://github.com/scrapy/scrapy/pull/502/files#r8574692
 2015-06-26 20:47:58 +05:00
Mikhail Korobov 1fef9f1132 AutoThrottle: respect download_delay=0 spider attribute 2015-06-26 19:09:26 +05:00
Mikhail Korobov 17cb51315b drop support for AUTOTHROTTLE_MIN_DOWNLOAD_DELAY 
it was deprecated for years
 2015-06-26 19:07:54 +05:00
Mikhail Korobov 584252e8f2 move AutoThrottle default options to default_settings.py 2015-06-26 18:58:29 +05:00
Mikhail Korobov e6272e5209 make AutoThrottle._adjust_delay easier to understand 2015-06-26 18:44:23 +05:00
Daniel Graña 7c98855927 Merge pull request #1321 from yarikoptic/upstream-master 
ENH: include tests/ to source distribution in MANIFEST.in
 2015-06-26 00:32:02 -03:00
Daniel Graña 5bcda9b7d1 Merge pull request #1290 from scrapy/crawler-runner-cleanup 
extract CrawlerRunner._crawl method which always expects Crawler
 2015-06-26 00:21:23 -03:00
Yaroslav Halchenko 5207d44055 ENH: include tests/ to source distribution in MANIFEST.in 2015-06-25 23:00:00 -04:00
Daniel Graña 148fb14040 Merge pull request #1319 from yarikoptic/fix-fail-if-docsbuild-fails 
[MRG+1] BF: fail if docs failed to build
 2015-06-25 23:35:42 -03:00
Yaroslav Halchenko 5f3e1e9a16 ENH: make explicit build-ignore-errors to be used by linkfix 2015-06-25 21:30:35 -04:00
Mikhail Korobov 2e55e65edc Merge pull request #1320 from barraponto/patch-2 
Fix SelectJmes documentation
 2015-06-25 23:20:46 +05:00
Capi Etheriel 4cea1c1231 Fix SelectJmes documentation 2015-06-25 15:11:56 -03:00
Yaroslav Halchenko 49fe915798 BF: fail if docs failed to build 2015-06-25 10:12:56 -04:00
Pablo Hoffman 8a140b6ba1 Merge pull request #1315 from scrapy/downloader-slots-cleanup 
Small downloader slots cleanup
 2015-06-24 16:25:13 -03:00
Mikhail Korobov ea4b175c2d Merge pull request #1254 from scrapy/bugfix-1232 
[MRG +1] Unquote request path before passing to FTPClient, it already escape paths
 2015-06-24 18:04:45 +05:00
Mikhail Korobov 08501bff92 Merge pull request #1316 from dallagi/patch-1 
DOC Bring Ubuntu and Archlinux outside of Windows subsection
 2015-06-24 17:38:12 +05:00
Marco DallaG 44c8ef2d47 DOC Bring Ubuntu and Archlinux outside of Windows subsection 
In the installation guide, in the "Platform specific installation notes" section, Ubuntu and ArchLinux are currently subsections of Windows, which does not make sense imho.
This commit changes the section tree from:

Platform specific installation notes
- Windows
  -- Archlinux
  -- Ubuntu

To:

Platform specific installation notes
- Windows
- Ubuntu
- Archlinux
 2015-06-24 11:44:48 +02:00
Mikhail Korobov cb0445e8b8 Slot.__str__ and __repr__ methods; remove unused import 2015-06-24 03:22:23 +05:00
Mikhail Korobov 8da9836db3 don't access settings from downloader.Slot 2015-06-24 03:20:46 +05:00
Elias Dorneles eb8fed4ecd support empty password for http_proxy config 2015-06-23 00:43:15 -03:00
Pablo Hoffman 38e5bfb61c remove version suffix from ubuntu package 2015-06-22 10:57:24 -03:00
Julia Medina af97c373ea Update release date for 1.0 
(cherry picked from commit c89fa29c41)
 2015-06-19 18:53:54 -03:00
Daniel Graña 47b1b748c2 Generated version as pep440 and dpkg compatible 2015-06-16 00:16:09 +00:00
Daniel Graña ec56ed5870 Merge pull request #1307 from agusc/master 
removed SUFFIX from scrapy package name
 2015-06-15 18:02:14 -03:00
agusc 140f58251b removed SUFFIX from scrapy name package 2015-06-15 17:02:09 -03:00
Julia Medina f958fb9d62 Update release notes for 1.0.0rc3 2015-06-12 16:42:39 -03:00
Julia Medina fa1c25c840 Merge pull request #1286 from scrapy/configure_logging 
configure_logging: change the meaning of settings=None
 2015-06-12 13:22:42 -03:00
Julia Medina 36bc912cdd DOC indent additional docs for configure_logging 2015-06-12 13:00:31 -03:00
Daniel Graña d862f5acff Merge pull request #1297 from darkrho/leveldb-compact 
Do leveldb compactation for httpcache on closing.
 2015-06-12 11:16:16 -03:00
Mikhail Korobov aaaba9f34d Merge pull request #1298 from bcrowe/fix-typos 
Fix a couple typos
 2015-06-12 16:02:53 +05:00
Bryan Crowe 6a4c475e87 Fix a couple typos 2015-06-11 19:47:30 -04:00
Rolando Espinoza 877c25793a Do leveldb compactation for the httpcache on closing. 2015-06-11 13:27:49 -04:00
Daniel Graña 5bd0395be4 Merge pull request #1291 from scrapy/signalmanager-docstrings 
DOC SignalManager docstrings. See GH-713.
 2015-06-10 16:28:35 -03:00
Daniel Graña 6446652635 Merge pull request #1294 from berkerpeksag/ignore-warnings 
Ignore ScrapyDeprecationWarning warnings properly.
 2015-06-10 16:07:21 -03:00
Mikhail Korobov 6c9daf3a95 DOC remove unnecessary links; fix references in send_catch_log_deferred docstring 2015-06-10 01:44:19 +05:00
Mikhail Korobov a611f8dd2d DOC remove FailureFormatter mentions, stop copy-pasting configure_logging docstring 2015-06-09 22:57:18 +05:00
Berker Peksag 8a48d9c6a8 Ignore ScrapyDeprecationWarning warnings properly. 
Conflicts:

	tests/test_utils_deprecate.py
 2015-06-09 18:22:03 +03:00
Daniel Graña 90aa5c075d Merge pull request #1284 from scrapy/crawler-cleanup 
CrawlerProcess cleanup
 2015-06-09 11:21:24 -03:00
Julia Medina f91461c256 Merge pull request #1292 from scrapy/spidererror-docs 
spider_error doesn't support deferreds
 2015-06-09 11:19:10 -03:00
Julia Medina 9ca87c5b6b Merge pull request #1293 from berkerpeksag/fix-typos 
Fix typos in scrapy/commands/setting.py
 2015-06-09 11:08:10 -03:00
Berker Peksag 66e5591eb9 Fix typos in scrapy/commands/setting.py 
* intepreted -> interpreted
* "a list" instead of "an float" in --getlist help
* "an float"-> "a float"

Also, backslashes were redundant. So I removed them.
 2015-06-09 04:29:24 +03:00
Mikhail Korobov 790c67b643 DOC spider_error doesn't support deferreds 2015-06-09 02:20:10 +05:00
Mikhail Korobov 1740fcf1a6 DOC SignalManager docstrings. See GH-713. 
This change is not 100% backwards compatible because of *args changes.
Their usage was not documented, so we're not breaking public interface.
 2015-06-08 21:05:58 +05:00
Mikhail Korobov 9a787893e3 (backwards-incompatible) allow to pass settings=None to configure_logging 
* use explicit argument for disabling root handler;
* handle LOG_STDOUT even if install_root_handler is False
 2015-06-08 19:54:18 +05:00