RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,163 Commits 49 Branches 132 Tags 145 MiB
Commit Graph

301 Commits

Author SHA1 Message Date
Νικόλαος-Διγενής Καραγιάννης 7a438a51b7 FormRequest: test default <input> type (is text) 2015-11-13 18:03:54 +02:00
Νικόλαος-Διγενής Καραγιάννης 881bf19e68 FormRequest: test case-insensitive type attribute 2015-11-13 17:57:25 +02:00
Julia Medina 54216d7afe Merge pull request #1586 from jdemaeyer/fix/backwards-compatible-per-key-priorities 
[MRG+1] Backwards compatible per key priorities
 2015-11-11 14:08:50 -03:00
Jakob de Maeyer 44f6ada034 Overwrite, not update, dictionary-like settings 2015-11-11 17:40:42 +01:00
Jakob de Maeyer 52ecee6a62 Replace BaseSettings._getcomposite() with public .getwithbase() method 2015-11-11 17:39:57 +01:00
Jakob de Maeyer b6a023ce98 Add backwards compatibility for build_component_list 2015-11-11 17:39:56 +01:00
Mikhail Korobov 9548691fdd Merge pull request #1563 from starrify/master 
[MRG+1] fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormReques t.from_response` when there is a `<base>` tag)
 2015-11-10 21:33:24 +05:00
Pengyu CHEN e19bf4aecc added: Test case for the fix 2015-11-02 22:52:41 +08:00
Julia Medina 57f87b95d4 Merge pull request #1528 from scrapy/create-crawler 
public Crawler.create_crawler method
 2015-10-30 12:36:35 -03:00
Mikhail Korobov a49c82ad62 TST improve CrawlerRunner tests 
* use CrawlerRunner.create_crawler instead of get_crawler helper in test_crawl;
* add a test for loading spiders by name;
* add a test for passing Crawler objects instead of Spider objects;
* add a test for CrawlerRunner.join
 2015-10-30 20:24:37 +05:00
Jakob de Maeyer 32ff4cc1d7 PEP8ify scrapy.settings tests 2015-10-29 19:52:19 +01:00
Jakob de Maeyer 51ca84c9b4 Add missing tests for scrapy.settings module 2015-10-29 17:34:18 +01:00
Jakob de Maeyer 8307c1212f Add ExecutionEngine.close() method 2015-10-29 10:03:29 +01:00
Jakob de Maeyer 03f1720afb Fix backwards-compatibility for users who explicitly set _BASE settings 2015-10-27 14:22:56 +01:00
Jakob de Maeyer f249b309ab Move scrapy.utils.conf.remove_none_values to s.u.python.without_none_values 2015-10-27 12:43:33 +01:00
Jakob de Maeyer 26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Jakob de Maeyer a769a1ef78 Introduce BaseSettings with full dictionary interface 2015-10-27 12:38:52 +01:00
Mikhail Korobov 215905bdb6 PY3 port bench, startproject, genspider, list and runspider commands 2015-10-12 17:59:49 +05:00
Νικόλαος-Διγενής Καραγιάννης d66efb13ba test xml nodename with dots 2015-10-07 14:43:47 +03:00
preetwinder 47c8e2ba78 Restructure tests for add_http_if_no_scheme function 2015-09-24 17:57:25 +00:00
preetwinder 8c629eee3e adds docstring, tests and correction 2015-09-24 17:28:44 +00:00
preetwinder c517951a48 add_scheme_if_missing for scrapy shell command 2015-09-24 17:28:44 +00:00
Mikhail Korobov 916141a08f TST pin pytest to 2.7.3 2015-09-21 21:12:52 +05:00
Daniel Graña 51b1249033 Revert "test suite requires recent queuelib to pass because it depends on queuelib test suite" 
This reverts commit a3390afc66.
 2015-09-18 02:27:07 -03:00
Daniel Graña a3390afc66 test suite requires recent queuelib to pass because it depends on queuelib test suite 2015-09-17 19:42:29 -03:00
Daniel Graña 3c596dcf46 Merge pull request #1467 from dacjames/master 
add support for a nested loaders
 2015-09-16 22:03:19 -03:00
Elias Dorneles 430e90f4b2 minor refactor on metarefresh redirect mware test 2015-09-13 12:29:41 -03:00
Elias Dorneles 81950f773d made encoding conversion more explicit, added test for header with utf-8 encoding replicating what browsers do 2015-09-11 18:51:48 -03:00
Elias Dorneles 179c4588ca adding test for latin1 location 2015-09-10 19:36:47 -03:00
Elias Dorneles defa899135 PY3 port redirect middleware 2015-09-10 16:31:59 -03:00
Daniel Graña 9a64d8ff97 fix scrapy squeue tests after recent changes to queuelib 2015-09-09 15:55:55 -03:00
Daniel Collins 311d5cd495 split nested_loader into seperate methods 2015-09-04 13:15:48 -07:00
Daniel Collins 425e35ee90 removed commented code and add test for error handling 2015-09-04 13:15:48 -07:00
Daniel Collins 1f95af3c07 add support for a nested loaders 2015-09-04 13:15:48 -07:00
Daniel Graña dd473145f2 Merge pull request #1473 from ArturGaspar/master 
[MRG+1] Support for returning deferreds in middlewares
 2015-09-02 13:04:26 -03:00
Artur Gaspar dd83f6123d Test for same response object in downloader middleware test. 2015-09-02 11:36:15 -03:00
Artur Gaspar 2748b38592 Test for not calling the download function when downloader middleware returns a response in process_request. 2015-09-02 11:15:37 -03:00
Artur Gaspar 1f4af9d81e Test for robots.txt middleware for logger not being called when the request for robots.txt is ignored. 2015-09-02 10:33:53 -03:00
Artur Gaspar 668e5fd257 Test for robots.txt middleware for processing a request for which the robots.txt parser is ready. 2015-09-02 10:23:24 -03:00
Artur Gaspar a6a629e707 Call actual error logger in robots.txt middleware tests. 2015-09-02 10:23:24 -03:00
Artur Gaspar 9ce9a293a6 Always check robots.txt before making another request in RobotsTxtMiddleware. 2015-09-02 10:23:24 -03:00
Daniel Graña 3919ad64c5 Merge pull request #1470 from scrapy/py3-middlewares 
[MRG] port some downloader middlewares to Python 3
 2015-09-02 00:17:37 -03:00
Olaf Dietsche 2c28b53cc0 Add tests with incomplete buttons 2015-09-01 12:35:49 +02:00
Mikhail Korobov c44cafe4f5 PY3 fix UserAgentMiddleware tests 2015-09-01 04:11:33 +05:00
Mikhail Korobov 3a9c73bc5d PY3 fix DownloaderStats middleware tests 2015-09-01 04:09:15 +05:00
Mikhail Korobov 78a4cd0f1c PY3 fix HttpAuthMiddleware tests 2015-09-01 04:00:26 +05:00
Mikhail Korobov 179a4409fb PY3 enable DownloadTimeoutMiddleware tests 2015-09-01 03:58:25 +05:00
Mikhail Korobov 3cf1911a92 PY3 fix DefaultHeadersMiddleware tests 2015-09-01 03:57:05 +05:00
Mikhail Korobov 45101829a5 PY3 fix AjacCrawlable middleware tests 2015-09-01 03:49:52 +05:00
Olaf Dietsche e5f26078fa Add test: submit form with button control 2015-08-31 19:57:45 +02:00