RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,439 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

1205 Commits

Author SHA1 Message Date
Valdir Stumm Jr dd7b0eb359 Updates to release notes 2016-02-03 19:23:02 -02:00
Valdir Stumm Jr 0f816524e8 edit API changes, Deprecations, and Relocations sections. 2016-02-03 16:21:32 -02:00
Paul Tremberth fd3193da86 Reword some of "New Features and Enhancements" 2016-02-03 18:19:38 +01:00
Paul Tremberth 3b2a6abc0b Format "Code refactoring" subsection 2016-02-03 17:37:58 +01:00
Paul Tremberth ed02455010 Format "Logging" subsection 2016-02-03 17:27:43 +01:00
Valdir Stumm Jr 1fd95c9c9f fixed references to issues and fit text to 80 cols in the first two sections 2016-02-03 12:45:48 -02:00
Valdir Stumm Jr a21c90bee7 edits on Py3 Support and New features sections 2016-02-03 11:54:46 -02:00
Paul Tremberth 44d8df2060 Add versionadded note for MEMUSAGE_CHECK_INTERVAL_SECONDS 2016-02-03 12:35:26 +01:00
Paul Tremberth 2b033eebce Fix recently added HTTPCACHE_ settings versionadded notes 2016-02-03 12:34:44 +01:00
Paul Tremberth 142aa21737 Add AUTOTHROTTLE_TARGET_CONCURRENCY versionadded note 2016-02-03 12:33:10 +01:00
Paul Tremberth db0697bc06 Add 1.1 release notes (draft) 2016-02-03 12:32:40 +01:00
Paul Tremberth c6f374f2eb Merge remote-tracking branch 'origin/master' into 1550-shell_file-cont 2016-01-28 14:02:48 +01:00
Paul Tremberth 27fb200e45 Merge pull request #1727 from redapple/priority-adjust 
[MRG+1] Clarify priority adjust settings docs
 2016-01-27 18:52:39 +01:00
Mikhail Korobov 7ca9ae1976 DOC typo fix 2016-01-27 17:54:28 +05:00
Paul Tremberth d999e3f7a7 More explicit description of DEPTH_PRIORITY 2016-01-27 12:57:03 +01:00
Paul Tremberth 6ed08d2332 Add note for DEPTH_PRIORITY 2016-01-27 11:53:29 +01:00
Mikhail Korobov 4bcbb77bcc response.text. Fixes GH-1729. 2016-01-27 01:28:11 +05:00
Paul Tremberth 1c83108893 Clarify priority adjust settings docs 
Fixes #1593
 2016-01-26 19:24:11 +01:00
Paul Tremberth bb1f4013a3 Rewrite warning about shell with local files as note 2016-01-26 17:23:28 +01:00
Mikhail Korobov f30758c246 Enable robots.txt handling by default for new projects. Fixes GH-1668. 
For backwards compatibility reasons the default value is not changed.
 2016-01-26 17:47:46 +05:00
Νικόλαος-Διγενής Καραγιάννης 1cffa99e0d tests+doc for subdomains in offsite middleware 2016-01-26 12:49:43 +02:00
Paul Tremberth 713e1eee9b Update docs about local files support for "scrapy shell" 2016-01-26 10:44:38 +01:00
Raul Gallegos a06a5f00f4 adding configurable encoding for httpproxy authentication 2016-01-21 09:35:04 -05:00
Elias Dorneles d4c4ca8062 fix version number to appear new feature 2016-01-21 09:42:15 -02:00
Capi Etheriel 659715ecd9 implements FormRequest.from_response CSS support 2016-01-21 01:05:20 -02:00
Daniel Graña b1e44436bc Merge pull request #1642 from orangain/doc-metarefresh 
[MRG+1] DOC: Update MetaRefreshMiddlware's setting variables
 2016-01-19 13:33:50 -03:00
carlosp420 e15f361b05 fixed typo You -> you 2016-01-19 11:12:43 -05:00
Ralph Gutkowski 79147a61a7 Update stats.rst 2016-01-15 19:25:56 +01:00
Ralph Gutkowski bb38400db5 Update Stats Collection documentation 
`pages_crawled` value doesn't exist. Replace it with `downloader/response_count` in order to avoid confusion.
 2016-01-15 19:00:58 +01:00
Valdir Stumm Jr 2abc9bc901 Update deprecated examples 
* update the scrapy.org example to deal with the new layout.
* replaced slashdot.org by reddit.com, because it seems that slashdot is blocking requests.
 2016-01-06 10:29:45 -02:00
Daniel Graña 1b435b2887 Add 1.0.4 release notes 2015-12-30 15:43:54 -03:00
orangain bcce8d3d80 DOC: Update MetaRefreshMiddlware's setting variables 
* `REDIRECT_MAX_METAREFRESH_DELAY` has been deprecated and was
  renamed to `METAREFRESH_MAXDELAY`.
* Merge duplicate documents about `METAREFRESH_MAXDELAY` appeared both
  in the settings page and the downloader-middlewares page.
 2015-12-17 15:25:02 +09:00
orangain 4be4ef038e DOC: Add captions to toctrees which appear in sidebar 2015-12-12 16:21:00 +09:00
Rolando Espinoza f8ae99d18f DOC Removed pywin32 from install instructions as it's already declared as dependency. 2015-12-05 09:48:17 -04:00
palego 2c251000d3 custom project templates 
allow override of TEMPLATES_DIR for startproject
copy full TEMPLATES_DIR/project tree
doc update
 2015-12-01 03:07:27 +01:00
Pablo Hoffman eb4daa34a2 Merge pull request #1544 from rolando/conda-docs 
Added installation notes about using Conda for Windows.
 2015-11-26 14:11:28 -03:00
Rolando Espinoza cc8ddb68b0 Added installation notes about using Conda for Windows and other OSes. 2015-11-26 13:08:56 -04:00
Julia Medina 48539af163 Merge pull request #1601 from mvj3/minor-documentation-fixes 
Fixed minor grammar issues.
 2015-11-16 14:58:07 -03:00
David Chen 0025d5a943 Fixed minor grammar issues. 2015-11-16 07:30:17 +08:00
Jakob de Maeyer e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Julia Medina 72eeead6db Merge pull request #1524 from Digenis/relocate_telnet_console 
[MRG+1] Relocate telnetconsole to extensions/
 2015-10-30 12:10:38 -03:00
Jakob de Maeyer bb6dee611c Move Settings documentation to docstrings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Jakob de Maeyer a769a1ef78 Introduce BaseSettings with full dictionary interface 2015-10-27 12:38:52 +01:00
Valdir Stumm Jr d577c4702d fixed a typo in the documentation. 2015-10-26 00:00:20 -02:00
Christian Pedersen cc76fb3dff Version 1 now exists 2015-10-14 14:07:34 +02:00
Mikhail Korobov 1b6d60c251 DOC fix docs after GH-1289. 2015-10-09 01:26:09 +05:00
hoatle 2869cf8dde fix another invalid xpath error 2015-10-07 16:03:43 +07:00
Hoat Le 4e66955411 fix ValueError: Invalid XPath: //div/[id="not-exists"]/text() on selectors.rst 
>>> response.xpath('//div/[id="not-exists"]/text()').extract_first() is None
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/http/response/text.py", line 109, in xpath
    return self.selector.xpath(query)
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/selector/unified.py", line 100, in xpath
    raise ValueError(msg if six.PY3 else msg.encode("unicode_escape"))
ValueError: Invalid XPath: //div/[id="not-exists"]/text()
 2015-10-07 15:43:02 +07:00
Νικόλαος-Διγενής Καραγιάννης d523c75f5e Relocate telnetconsole to extensions/ 2015-10-05 21:16:04 +03:00