RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,178 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

5178 Commits

Author SHA1 Message Date
Jakob de Maeyer a65fc0db7d Drop support for providing file paths as add-ons 2015-11-12 18:37:26 +01:00
Jakob de Maeyer 33dfb3e167 Remove unused project path util function 2015-11-11 21:08:20 +01:00
Jakob de Maeyer b7b00fb956 PEP8ify add-ons and tests 2015-11-11 21:08:20 +01:00
Jakob de Maeyer e924d38238 Drop component configuration (copied from default_settings) from built-in add-ons 2015-11-11 21:08:20 +01:00
Jakob de Maeyer 388c5c4b78 Fix component exporting for Addon base class 2015-11-11 21:08:20 +01:00
Jakob de Maeyer 8e5d067af1 Drop BaseSettings.get_addon() prefix magic 2015-11-11 21:08:20 +01:00
Jakob de Maeyer b10caf91a1 Drop support for add-on configuration in scrapy.cfg 2015-11-11 21:08:20 +01:00
Jakob de Maeyer f7ed239fcb Replace INSTALLED_ADDONS tuple setting with ADDONS dictionary setting 2015-11-11 21:08:20 +01:00
Jakob de Maeyer 9f7fcf5582 Make update_classpath() util function return non-string objects 2015-11-11 21:08:19 +01:00
Jakob de Maeyer d18b6a61d7 Add missing AddonManager tests 2015-11-11 21:08:19 +01:00
Jakob de Maeyer 18c7f3dbe2 Document built-in add-ons 2015-11-11 21:08:19 +01:00
Jakob de Maeyer 4ac6a83072 Fix class signatures in Extensions docs 2015-11-11 21:08:19 +01:00
Jakob de Maeyer 25498c3c21 Remove unused imports in add-ons 2015-11-11 21:08:19 +01:00
Jakob de Maeyer 2946b67414 Document add-ons 2015-11-11 21:08:19 +01:00
Jakob de Maeyer d91647c38b Add built-in add-ons 2015-11-11 21:08:19 +01:00
Jakob de Maeyer 07455b1883 Integrate add-ons into start-up process 2015-11-11 21:08:19 +01:00
Jakob de Maeyer d8af395d76 Introduce add-ons via AddonManager and Addon base class 2015-11-11 21:08:19 +01:00
Jakob de Maeyer e5b8def0b8 Redraft SEP-021 2015-11-11 21:08:19 +01:00
Jakob de Maeyer cfed9b6659 Allow passing Python objects to middleware dict settings 2015-11-11 21:08:19 +01:00
Julia Medina 54216d7afe Merge pull request #1586 from jdemaeyer/fix/backwards-compatible-per-key-priorities 
[MRG+1] Backwards compatible per key priorities
 2015-11-11 14:08:50 -03:00
Jakob de Maeyer 4f364764ae Simplify BaseSettings.__get__(), .getpriority() 2015-11-11 17:40:42 +01:00
Jakob de Maeyer 44f6ada034 Overwrite, not update, dictionary-like settings 2015-11-11 17:40:42 +01:00
Jakob de Maeyer 52ecee6a62 Replace BaseSettings._getcomposite() with public .getwithbase() method 2015-11-11 17:39:57 +01:00
Jakob de Maeyer b6a023ce98 Add backwards compatibility for build_component_list 2015-11-11 17:39:56 +01:00
Jakob de Maeyer e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Mikhail Korobov 9548691fdd Merge pull request #1563 from starrify/master 
[MRG+1] fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormReques t.from_response` when there is a `<base>` tag)
 2015-11-10 21:33:24 +05:00
Pengyu CHEN e19bf4aecc added: Test case for the fix 2015-11-02 22:52:41 +08:00
Julia Medina 57f87b95d4 Merge pull request #1528 from scrapy/create-crawler 
public Crawler.create_crawler method
 2015-10-30 12:36:35 -03:00
Julia Medina 9424ca0fdb Merge pull request #1570 from jdemaeyer/tests/complete-settings-tests 
[MRG+1] Add some missing tests for scrapy.settings
 2015-10-30 12:36:25 -03:00
Mikhail Korobov 0000b6e9ef TST cleanup scrapy.utils.test.get_crawler 2015-10-30 20:27:03 +05:00
Mikhail Korobov a49c82ad62 TST improve CrawlerRunner tests 
* use CrawlerRunner.create_crawler instead of get_crawler helper in test_crawl;
* add a test for loading spiders by name;
* add a test for passing Crawler objects instead of Spider objects;
* add a test for CrawlerRunner.join
 2015-10-30 20:24:37 +05:00
Mikhail Korobov 11b11c9803 CrawlerRunner.create_crawler method 2015-10-30 20:22:31 +05:00
Julia Medina 72eeead6db Merge pull request #1524 from Digenis/relocate_telnet_console 
[MRG+1] Relocate telnetconsole to extensions/
 2015-10-30 12:10:38 -03:00
Jakob de Maeyer 32ff4cc1d7 PEP8ify scrapy.settings tests 2015-10-29 19:52:19 +01:00
Jakob de Maeyer 51ca84c9b4 Add missing tests for scrapy.settings module 2015-10-29 17:34:18 +01:00
Mikhail Korobov caf2080b80 Merge pull request #1423 from jdemaeyer/enhancement/executionengine-close 
[MRG+1] Add ExecutionEngine.close() method
 2015-10-29 19:11:07 +05:00
Jakob de Maeyer 8307c1212f Add ExecutionEngine.close() method 2015-10-29 10:03:29 +01:00
Pengyu CHEN c34dbe955d fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormRequest.from_response` when there is a `<base>` tag) 2015-10-29 14:18:59 +08:00
Daniel Graña dd9f777ba7 Merge pull request #1149 from jdemaeyer/enhancement/settings-per-key-priorities 
[MRG+1] Per-key priorities for dict-like settings by promoting dicts to Settings instances
 2015-10-28 18:21:26 -03:00
Jakob de Maeyer 03f1720afb Fix backwards-compatibility for users who explicitly set _BASE settings 2015-10-27 14:22:56 +01:00
Jakob de Maeyer f249b309ab Move scrapy.utils.conf.remove_none_values to s.u.python.without_none_values 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 90198e5324 Add __repr__ method for BaseSettings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 9eb3597d15 PEP8ify settings module 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 9bd7af8a62 Remove unused import in scrapy.settings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer bb6dee611c Move Settings documentation to docstrings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Jakob de Maeyer a769a1ef78 Introduce BaseSettings with full dictionary interface 2015-10-27 12:38:52 +01:00
Mikhail Korobov 4a9f76ebd3 Merge pull request #1555 from stummjr/patch-1 
fix typo in the documentation
 2015-10-26 12:30:46 +05:00
Valdir Stumm Jr d577c4702d fixed a typo in the documentation. 2015-10-26 00:00:20 -02:00
Pablo Hoffman 8dc400c66d Merge pull request #1541 from chripede/patch-1 
[MRG+1] Version 1 now exists
 2015-10-14 10:58:54 -03:00