RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

10393 Commits

Author SHA1 Message Date
Mikhail Korobov f01fd07642 Merge pull request #1661 from redapple/py3-iterators 
[MRG+1] PY3: port utils/iterators
 2016-01-13 10:03:02 +05:00
Paul Tremberth d7d4ef67a6 Changes following comments 2016-01-12 11:08:49 +01:00
Paul Tremberth 6ddd814738 Support unicode tags in xml iterators (fixes #1665) 2016-01-12 10:48:45 +01:00
Paul Tremberth d4872940db PY3: port utils/iterators 2016-01-07 10:51:27 +01:00
Elias Dorneles 95e8ff8ba1 Merge pull request #1660 from stummjr/patch-4 
[MRG+1] Update deprecated examples
 2016-01-06 10:43:08 -02:00
Valdir Stumm Jr 2abc9bc901 Update deprecated examples 
* update the scrapy.org example to deal with the new layout.
* replaced slashdot.org by reddit.com, because it seems that slashdot is blocking requests.
 2016-01-06 10:29:45 -02:00
palego d03d262f52 indentation 2016-01-04 10:00:13 +01:00
palego c702c53015 change os.mknod() for open() 
os.mknod() is a privileged command on OS X, making the test fail
 2016-01-03 14:33:42 +01:00
Daniel Graña 1b435b2887 Add 1.0.4 release notes 2015-12-30 15:43:54 -03:00
Aron Bordin f57121c77b show download warnsize once 2015-12-30 13:10:13 -02:00
Yaroslav Halchenko 4f49aab7c0 BF: robustify _monkeypatches check for twisted - str() name first (Closes: #1634) 
In my case, while running datalad tests using nose, scrapy was failing
since v was None
 2015-12-18 16:16:05 -05:00
orangain bcce8d3d80 DOC: Update MetaRefreshMiddlware's setting variables 
* `REDIRECT_MAX_METAREFRESH_DELAY` has been deprecated and was
  renamed to `METAREFRESH_MAXDELAY`.
* Merge duplicate documents about `METAREFRESH_MAXDELAY` appeared both
  in the settings page and the downloader-middlewares page.
 2015-12-17 15:25:02 +09:00
Mikhail Korobov a5db7f862c Merge pull request #1639 from seales/master 
Spelling fixes
 2015-12-14 23:33:14 +05:00
Mikhail Korobov 1bc243a876 Merge pull request #1638 from orangain/toc-captions 
[MRG+1] DOC: Add captions to toctrees which appear in sidebar
 2015-12-14 23:32:18 +05:00
seales 719b1353a7 Spelling fixes 2015-12-13 19:39:48 -08:00
orangain 4be4ef038e DOC: Add captions to toctrees which appear in sidebar 2015-12-12 16:21:00 +09:00
Daniel Graña b5285c3f38 Merge pull request #1628 from rolando/conda-docs 
[MRG] Removed pywin32 from conda install instructions as it's already declared as dependency.
 2015-12-05 14:45:17 -03:00
Rolando Espinoza f8ae99d18f DOC Removed pywin32 from install instructions as it's already declared as dependency. 2015-12-05 09:48:17 -04:00
Elias Dorneles 7d187735ff Merge pull request #1359 from nyov/nyov/drop-optional_features 
[MRG+1] drop deprecated "optional_features" set
 2015-12-04 14:08:44 -02:00
Daniel Graña 17aba44f16 Merge pull request #1565 from starrify/fix-1564 
[MRG+1] fixed: Issue #1564 (Incorrectly picked URL in `scrapy.linkextractors.regex.RegexLinkExtractor` when there is a `<base>` tag. )
 2015-12-04 10:31:13 -03:00
Daniel Graña dc65026395 Merge pull request #1597 from Digenis/various_formreq_fixes 
[MRG+1] Various FormRequest tests+fixes
 2015-12-04 09:55:12 -03:00
Daniel Graña 3881eaff45 Merge pull request #1575 from palego/startproject-templates-override 
[MRG+1] Startproject templates override
 2015-12-04 09:54:48 -03:00
Daniel Graña a97f28fbf0 Merge pull request #1598 from Digenis/disable_dupefilter_in_shell 
[MRG+1] Disable dupefilter in shell to prevent hanging.
 2015-12-04 09:51:56 -03:00
Daniel Graña 21f638b139 Merge pull request #1622 from sibiryakov/twisted-1550-fix 
[MRG+1] Ignoring xlib/tx folder, depending on Twisted version.
 2015-12-04 09:40:35 -03:00
Mikhail Korobov 015cd78940 Merge pull request #1618 from sibiryakov/engine-heartbeats 
Nextcall heartbeats in engine (event loop halting)
 2015-12-04 16:28:45 +05:00
Leonid Amirov 37289815ac code formatting fix 2015-12-03 16:15:50 +03:00
Leonid Amirov 016875fd51 added more verbosity for log and for exception when download is cancelled because of a size limit 2015-12-03 15:30:06 +03:00
Alexander Sibiryakov c4d29ecaef Ignoring xlib/tx folder, depending on Twisted version. 2015-12-02 17:08:21 +01:00
Alexander Sibiryakov 5750ce92c0 nextcall repetitive calls (heartbeats). 2015-12-02 11:39:47 +01:00
palego 2c251000d3 custom project templates 
allow override of TEMPLATES_DIR for startproject
copy full TEMPLATES_DIR/project tree
doc update
 2015-12-01 03:07:27 +01:00
Pablo Hoffman eb4daa34a2 Merge pull request #1544 from rolando/conda-docs 
Added installation notes about using Conda for Windows.
 2015-11-26 14:11:28 -03:00
Rolando Espinoza cc8ddb68b0 Added installation notes about using Conda for Windows and other OSes. 2015-11-26 13:08:56 -04:00
Pengyu CHEN 751155eade removed: Unused code 2015-11-19 00:38:05 +08:00
Julia Medina 48539af163 Merge pull request #1601 from mvj3/minor-documentation-fixes 
Fixed minor grammar issues.
 2015-11-16 14:58:07 -03:00
David Chen 0025d5a943 Fixed minor grammar issues. 2015-11-16 07:30:17 +08:00
Νικόλαος-Διγενής Καραγιάννης ebfdb9bb03 readable xpath with exslt 2015-11-14 23:24:07 +02:00
Νικόλαος-Διγενής Καραγιάννης 4f98be60be FormRequest: fix unicode xpath exception 2015-11-13 18:56:05 +02:00
Νικόλαος-Διγενής Καραγιάννης 395ef805eb FormRequest: test unicode xpath expr & exception 2015-11-13 18:54:02 +02:00
Νικόλαος-Διγενής Καραγιάννης 2d25eab0df FormRequest: <input>'s default type must be text 2015-11-13 18:05:07 +02:00
Νικόλαος-Διγενής Καραγιάννης 7a438a51b7 FormRequest: test default <input> type (is text) 2015-11-13 18:03:54 +02:00
Νικόλαος-Διγενής Καραγιάννης 650acad2b7 FormRequest: fix case-insensitive type attributes 2015-11-13 17:57:46 +02:00
Νικόλαος-Διγενής Καραγιάννης 881bf19e68 FormRequest: test case-insensitive type attribute 2015-11-13 17:57:25 +02:00
Jakob de Maeyer a65fc0db7d Drop support for providing file paths as add-ons 2015-11-12 18:37:26 +01:00
Jakob de Maeyer 33dfb3e167 Remove unused project path util function 2015-11-11 21:08:20 +01:00
Jakob de Maeyer b7b00fb956 PEP8ify add-ons and tests 2015-11-11 21:08:20 +01:00
Jakob de Maeyer e924d38238 Drop component configuration (copied from default_settings) from built-in add-ons 2015-11-11 21:08:20 +01:00
Jakob de Maeyer 388c5c4b78 Fix component exporting for Addon base class 2015-11-11 21:08:20 +01:00
Jakob de Maeyer 8e5d067af1 Drop BaseSettings.get_addon() prefix magic 2015-11-11 21:08:20 +01:00
Jakob de Maeyer b10caf91a1 Drop support for add-on configuration in scrapy.cfg 2015-11-11 21:08:20 +01:00
Jakob de Maeyer f7ed239fcb Replace INSTALLED_ADDONS tuple setting with ADDONS dictionary setting 2015-11-11 21:08:20 +01:00