RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,160 Commits 49 Branches 132 Tags 180 MiB
Commit Graph

5160 Commits

Author SHA1 Message Date
Νικόλαος-Διγενής Καραγιάννης f4dd8bcdc2 Disable dupefilter in shell 2015-06-16 17:31:37 +03:00
Julia Medina 54216d7afe Merge pull request #1586 from jdemaeyer/fix/backwards-compatible-per-key-priorities 
[MRG+1] Backwards compatible per key priorities
 2015-11-11 14:08:50 -03:00
Jakob de Maeyer 4f364764ae Simplify BaseSettings.__get__(), .getpriority() 2015-11-11 17:40:42 +01:00
Jakob de Maeyer 44f6ada034 Overwrite, not update, dictionary-like settings 2015-11-11 17:40:42 +01:00
Jakob de Maeyer 52ecee6a62 Replace BaseSettings._getcomposite() with public .getwithbase() method 2015-11-11 17:39:57 +01:00
Jakob de Maeyer b6a023ce98 Add backwards compatibility for build_component_list 2015-11-11 17:39:56 +01:00
Jakob de Maeyer e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Mikhail Korobov 9548691fdd Merge pull request #1563 from starrify/master 
[MRG+1] fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormReques t.from_response` when there is a `<base>` tag)
 2015-11-10 21:33:24 +05:00
Pengyu CHEN e19bf4aecc added: Test case for the fix 2015-11-02 22:52:41 +08:00
Julia Medina 57f87b95d4 Merge pull request #1528 from scrapy/create-crawler 
public Crawler.create_crawler method
 2015-10-30 12:36:35 -03:00
Julia Medina 9424ca0fdb Merge pull request #1570 from jdemaeyer/tests/complete-settings-tests 
[MRG+1] Add some missing tests for scrapy.settings
 2015-10-30 12:36:25 -03:00
Mikhail Korobov 0000b6e9ef TST cleanup scrapy.utils.test.get_crawler 2015-10-30 20:27:03 +05:00
Mikhail Korobov a49c82ad62 TST improve CrawlerRunner tests 
* use CrawlerRunner.create_crawler instead of get_crawler helper in test_crawl;
* add a test for loading spiders by name;
* add a test for passing Crawler objects instead of Spider objects;
* add a test for CrawlerRunner.join
 2015-10-30 20:24:37 +05:00
Mikhail Korobov 11b11c9803 CrawlerRunner.create_crawler method 2015-10-30 20:22:31 +05:00
Julia Medina 72eeead6db Merge pull request #1524 from Digenis/relocate_telnet_console 
[MRG+1] Relocate telnetconsole to extensions/
 2015-10-30 12:10:38 -03:00
Jakob de Maeyer 32ff4cc1d7 PEP8ify scrapy.settings tests 2015-10-29 19:52:19 +01:00
Jakob de Maeyer 51ca84c9b4 Add missing tests for scrapy.settings module 2015-10-29 17:34:18 +01:00
Mikhail Korobov caf2080b80 Merge pull request #1423 from jdemaeyer/enhancement/executionengine-close 
[MRG+1] Add ExecutionEngine.close() method
 2015-10-29 19:11:07 +05:00
Jakob de Maeyer 8307c1212f Add ExecutionEngine.close() method 2015-10-29 10:03:29 +01:00
Pengyu CHEN c34dbe955d fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormRequest.from_response` when there is a `<base>` tag) 2015-10-29 14:18:59 +08:00
Daniel Graña dd9f777ba7 Merge pull request #1149 from jdemaeyer/enhancement/settings-per-key-priorities 
[MRG+1] Per-key priorities for dict-like settings by promoting dicts to Settings instances
 2015-10-28 18:21:26 -03:00
Jakob de Maeyer 03f1720afb Fix backwards-compatibility for users who explicitly set _BASE settings 2015-10-27 14:22:56 +01:00
Jakob de Maeyer f249b309ab Move scrapy.utils.conf.remove_none_values to s.u.python.without_none_values 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 90198e5324 Add __repr__ method for BaseSettings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 9eb3597d15 PEP8ify settings module 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 9bd7af8a62 Remove unused import in scrapy.settings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer bb6dee611c Move Settings documentation to docstrings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Jakob de Maeyer a769a1ef78 Introduce BaseSettings with full dictionary interface 2015-10-27 12:38:52 +01:00
Mikhail Korobov 4a9f76ebd3 Merge pull request #1555 from stummjr/patch-1 
fix typo in the documentation
 2015-10-26 12:30:46 +05:00
Valdir Stumm Jr d577c4702d fixed a typo in the documentation. 2015-10-26 00:00:20 -02:00
Pablo Hoffman 8dc400c66d Merge pull request #1541 from chripede/patch-1 
[MRG+1] Version 1 now exists
 2015-10-14 10:58:54 -03:00
Christian Pedersen cc76fb3dff Version 1 now exists 2015-10-14 14:07:34 +02:00
Pablo Hoffman 451318ef7a Merge pull request #1530 from scrapy/retry-400 
[MRG+1] DOC fix docs after GH-1289.
 2015-10-13 16:09:07 -03:00
Daniel Graña 411174cf38 Merge pull request #1535 from scrapy/py3-commands 
PY3 port bench, startproject, genspider, list and runspider commands
 2015-10-12 20:43:21 -03:00
Daniel Graña 65f4ba349c Merge pull request #1537 from scrapy/no-coveralls 
drop coveralls support
 2015-10-12 20:36:12 -03:00
Mikhail Korobov 09fc9b48b6 drop coveralls support 2015-10-12 18:12:56 +05:00
Mikhail Korobov 215905bdb6 PY3 port bench, startproject, genspider, list and runspider commands 2015-10-12 17:59:49 +05:00
Julia Medina 0117c811c9 Merge pull request #1533 from Digenis/xml_iter-nodename_with_dot 
[MRG+1] xmliter nodename with dot
 2015-10-12 09:33:27 -03:00
Mikhail Korobov 1b6d60c251 DOC fix docs after GH-1289. 2015-10-09 01:26:09 +05:00
Mikhail Korobov 9676cf8e53 Merge pull request #1527 from scrapy/no-pillow-300 
TST don't use broken Pillow version in tests
 2015-10-09 01:20:45 +05:00
Νικόλαος-Διγενής Καραγιάννης f56062d045 escape nodename in xmliter regex 2015-10-07 14:58:47 +03:00
Νικόλαος-Διγενής Καραγιάννης d66efb13ba test xml nodename with dots 2015-10-07 14:43:47 +03:00
Mikhail Korobov 4e9a43c228 Merge pull request #1532 from hoatle/docs-fixes 
Docs fixes
 2015-10-07 15:34:21 +05:00
hoatle 2869cf8dde fix another invalid xpath error 2015-10-07 16:03:43 +07:00
Hoat Le 4e66955411 fix ValueError: Invalid XPath: //div/[id="not-exists"]/text() on selectors.rst 
>>> response.xpath('//div/[id="not-exists"]/text()').extract_first() is None
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/http/response/text.py", line 109, in xpath
    return self.selector.xpath(query)
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/selector/unified.py", line 100, in xpath
    raise ValueError(msg if six.PY3 else msg.encode("unicode_escape"))
ValueError: Invalid XPath: //div/[id="not-exists"]/text()
 2015-10-07 15:43:02 +07:00
Mikhail Korobov 669be0a2b0 TST don't use broken Pillow version in tests 2015-10-06 17:24:52 +05:00
Νικόλαος-Διγενής Καραγιάννης d523c75f5e Relocate telnetconsole to extensions/ 2015-10-05 21:16:04 +03:00
Mikhail Korobov 91cbf97415 Merge pull request #1522 from smirecki/fix-typos-in-docs-topics 
DOC Typos corrections
 2015-10-03 13:23:38 +05:00
smirecki 8379bea7ed Typos corrections 
I've made a few small corrections, some spelling changes and typo fixes.
I've tried to respect regional spelling differences and avoided proposing hyphenating compound words.

 Please enter the commit message for your changes. Lines starting
 2015-10-02 23:48:27 -04:00