RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,505 Commits 49 Branches 132 Tags 162 MiB
Commit Graph

5505 Commits

Author SHA1 Message Date
palego d03d262f52 indentation 2016-01-04 10:00:13 +01:00
palego c702c53015 change os.mknod() for open() 
os.mknod() is a privileged command on OS X, making the test fail
 2016-01-03 14:33:42 +01:00
Daniel Graña 1b435b2887 Add 1.0.4 release notes 2015-12-30 15:43:54 -03:00
Aron Bordin f57121c77b show download warnsize once 2015-12-30 13:10:13 -02:00
Yaroslav Halchenko 4f49aab7c0 BF: robustify _monkeypatches check for twisted - str() name first (Closes: #1634) 
In my case, while running datalad tests using nose, scrapy was failing
since v was None
 2015-12-18 16:16:05 -05:00
orangain bcce8d3d80 DOC: Update MetaRefreshMiddlware's setting variables 
* `REDIRECT_MAX_METAREFRESH_DELAY` has been deprecated and was
  renamed to `METAREFRESH_MAXDELAY`.
* Merge duplicate documents about `METAREFRESH_MAXDELAY` appeared both
  in the settings page and the downloader-middlewares page.
 2015-12-17 15:25:02 +09:00
Mikhail Korobov a5db7f862c Merge pull request #1639 from seales/master 
Spelling fixes
 2015-12-14 23:33:14 +05:00
Mikhail Korobov 1bc243a876 Merge pull request #1638 from orangain/toc-captions 
[MRG+1] DOC: Add captions to toctrees which appear in sidebar
 2015-12-14 23:32:18 +05:00
seales 719b1353a7 Spelling fixes 2015-12-13 19:39:48 -08:00
orangain 4be4ef038e DOC: Add captions to toctrees which appear in sidebar 2015-12-12 16:21:00 +09:00
Daniel Graña b5285c3f38 Merge pull request #1628 from rolando/conda-docs 
[MRG] Removed pywin32 from conda install instructions as it's already declared as dependency.
 2015-12-05 14:45:17 -03:00
Rolando Espinoza f8ae99d18f DOC Removed pywin32 from install instructions as it's already declared as dependency. 2015-12-05 09:48:17 -04:00
Elias Dorneles 7d187735ff Merge pull request #1359 from nyov/nyov/drop-optional_features 
[MRG+1] drop deprecated "optional_features" set
 2015-12-04 14:08:44 -02:00
Daniel Graña 17aba44f16 Merge pull request #1565 from starrify/fix-1564 
[MRG+1] fixed: Issue #1564 (Incorrectly picked URL in `scrapy.linkextractors.regex.RegexLinkExtractor` when there is a `<base>` tag. )
 2015-12-04 10:31:13 -03:00
Daniel Graña dc65026395 Merge pull request #1597 from Digenis/various_formreq_fixes 
[MRG+1] Various FormRequest tests+fixes
 2015-12-04 09:55:12 -03:00
Daniel Graña 3881eaff45 Merge pull request #1575 from palego/startproject-templates-override 
[MRG+1] Startproject templates override
 2015-12-04 09:54:48 -03:00
Daniel Graña a97f28fbf0 Merge pull request #1598 from Digenis/disable_dupefilter_in_shell 
[MRG+1] Disable dupefilter in shell to prevent hanging.
 2015-12-04 09:51:56 -03:00
Daniel Graña 21f638b139 Merge pull request #1622 from sibiryakov/twisted-1550-fix 
[MRG+1] Ignoring xlib/tx folder, depending on Twisted version.
 2015-12-04 09:40:35 -03:00
Mikhail Korobov 015cd78940 Merge pull request #1618 from sibiryakov/engine-heartbeats 
Nextcall heartbeats in engine (event loop halting)
 2015-12-04 16:28:45 +05:00
Leonid Amirov 37289815ac code formatting fix 2015-12-03 16:15:50 +03:00
Leonid Amirov 016875fd51 added more verbosity for log and for exception when download is cancelled because of a size limit 2015-12-03 15:30:06 +03:00
Alexander Sibiryakov c4d29ecaef Ignoring xlib/tx folder, depending on Twisted version. 2015-12-02 17:08:21 +01:00
Alexander Sibiryakov 5750ce92c0 nextcall repetitive calls (heartbeats). 2015-12-02 11:39:47 +01:00
palego 2c251000d3 custom project templates 
allow override of TEMPLATES_DIR for startproject
copy full TEMPLATES_DIR/project tree
doc update
 2015-12-01 03:07:27 +01:00
Pablo Hoffman eb4daa34a2 Merge pull request #1544 from rolando/conda-docs 
Added installation notes about using Conda for Windows.
 2015-11-26 14:11:28 -03:00
Rolando Espinoza cc8ddb68b0 Added installation notes about using Conda for Windows and other OSes. 2015-11-26 13:08:56 -04:00
Pengyu CHEN 751155eade removed: Unused code 2015-11-19 00:38:05 +08:00
Julia Medina 48539af163 Merge pull request #1601 from mvj3/minor-documentation-fixes 
Fixed minor grammar issues.
 2015-11-16 14:58:07 -03:00
David Chen 0025d5a943 Fixed minor grammar issues. 2015-11-16 07:30:17 +08:00
Νικόλαος-Διγενής Καραγιάννης ebfdb9bb03 readable xpath with exslt 2015-11-14 23:24:07 +02:00
Νικόλαος-Διγενής Καραγιάννης 4f98be60be FormRequest: fix unicode xpath exception 2015-11-13 18:56:05 +02:00
Νικόλαος-Διγενής Καραγιάννης 395ef805eb FormRequest: test unicode xpath expr & exception 2015-11-13 18:54:02 +02:00
Νικόλαος-Διγενής Καραγιάννης 2d25eab0df FormRequest: <input>'s default type must be text 2015-11-13 18:05:07 +02:00
Νικόλαος-Διγενής Καραγιάννης 7a438a51b7 FormRequest: test default <input> type (is text) 2015-11-13 18:03:54 +02:00
Νικόλαος-Διγενής Καραγιάννης 650acad2b7 FormRequest: fix case-insensitive type attributes 2015-11-13 17:57:46 +02:00
Νικόλαος-Διγενής Καραγιάννης 881bf19e68 FormRequest: test case-insensitive type attribute 2015-11-13 17:57:25 +02:00
Julia Medina 54216d7afe Merge pull request #1586 from jdemaeyer/fix/backwards-compatible-per-key-priorities 
[MRG+1] Backwards compatible per key priorities
 2015-11-11 14:08:50 -03:00
Jakob de Maeyer 4f364764ae Simplify BaseSettings.__get__(), .getpriority() 2015-11-11 17:40:42 +01:00
Jakob de Maeyer 44f6ada034 Overwrite, not update, dictionary-like settings 2015-11-11 17:40:42 +01:00
Jakob de Maeyer 52ecee6a62 Replace BaseSettings._getcomposite() with public .getwithbase() method 2015-11-11 17:39:57 +01:00
Jakob de Maeyer b6a023ce98 Add backwards compatibility for build_component_list 2015-11-11 17:39:56 +01:00
Jakob de Maeyer e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Mikhail Korobov 9548691fdd Merge pull request #1563 from starrify/master 
[MRG+1] fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormReques t.from_response` when there is a `<base>` tag)
 2015-11-10 21:33:24 +05:00
Leonid Amirov 97b51ea33b issue GH #1550 - six library is used instead of urlparse for python3 compatibility 2015-11-03 14:57:37 +03:00
Leonid Amirov dd45b31fe4 issue GH #1550 - rewritten add_http_if_no_scheme() 2015-11-03 14:32:30 +03:00
Pengyu CHEN 94486bb294 added: Test case for the fix. 2015-11-02 23:00:42 +08:00
Pengyu CHEN e19bf4aecc added: Test case for the fix 2015-11-02 22:52:41 +08:00
Leonid Amirov bc9db65358 issue GH #1550 - scrapy shell argument fixes: "example.com" requests "http://example.com"; "example" requests "file://example"; "./example.com" requests "file://example.com" 2015-11-02 16:08:19 +03:00
Leonid Amirov a41c64bfb9 issue GH #1550 - fixed bugs in scrapy.utils.url.add_http_if_no_scheme(): when given URI where scheme is present, but not 'http' the function gave bad result 2015-11-02 16:06:21 +03:00
Leonid Amirov 98a2e77a75 issue GH #1550 - fixed error: shell command wasn't accepting files URIs 2015-11-02 15:30:49 +03:00