RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,382 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

411 Commits

Author SHA1 Message Date
Artur Gaspar dc8701ea42 Add test for already failed deferreds when downloading page in robots.txt middleware. 2016-01-27 12:56:42 -02:00
Paul Tremberth 29695375d1 Add test for raised exception with SpiderState extension when no JOBDIR used 2016-01-26 16:33:24 +01:00
Νικόλαος-Διγενής Καραγιάννης 1cffa99e0d tests+doc for subdomains in offsite middleware 2016-01-26 12:49:43 +02:00
Daniel Graña a7b86137d0 Merge pull request #1499 from scrapy/py3-port-exporters 
[MRG+1] PY3 exporters
 2016-01-26 01:01:30 -03:00
Daniel Graña d0eacfe0f9 Add test case for marshal item exporter 2016-01-26 00:26:27 -03:00
Elias Dorneles 2dfdde3c79 fallback to repr when can't convert to native string 2016-01-25 22:24:35 -02:00
Elias Dorneles 23b3336c1f add test for invalid option 2016-01-25 22:11:04 -02:00
Daniel Graña b20463183c Merge pull request #1716 from lopuhin/py3-test-pipeline-files-images 
[MRG+1] py3: test_pipeline_files and test_pipeline_images
 2016-01-25 18:24:12 -03:00
Elias Dorneles 0c44fac2b5 added tests for feed export marshal and pickle 2016-01-24 19:17:42 -02:00
Konstantin Lopuhin 1be90323c2 py3: properly skip s3 tests on py3 2016-01-24 23:44:56 +03:00
Konstantin Lopuhin 4233b3cda4 py3: reviewed passing test_spidermiddleware_httperror.py 2016-01-24 23:10:03 +03:00
Konstantin Lopuhin 097082cffa reviewed py3 compat in pipelines/images.py and pipelines/files.py 2016-01-24 23:05:23 +03:00
Konstantin Lopuhin 333d4c91fb py3: add boto to py3 test requirements, test_pipeline_files and test_pipeline_images passing now 2016-01-24 22:55:21 +03:00
Konstantin Lopuhin 860353b0c0 py3: unskip test_mail and scrapy/mail.py 2016-01-24 13:27:41 +03:00
Konstantin Lopuhin 9704226ee4 py3: fix test_mail - get_payload returns bytes when decode is True 2016-01-24 13:25:14 +03:00
Elias Dorneles 9fbe6f3e81 added feedexport test for xml output 2016-01-23 17:17:40 -02:00
Elias Dorneles c75f1fe46a restore bytes instead of text, for easier reviewing 2016-01-23 16:09:57 -02:00
Elias Dorneles d0955fd083 add back test for latin-1 encoding 2016-01-22 10:07:55 -02:00
Mikhail Korobov 35ada10729 PY3 enable tests for scrapy parse command 
scrapy parse command is already ported
 2016-01-22 13:39:27 +05:00
Elias Dorneles e938752973 add test for PythonItemExporter binary mode 2016-01-21 21:51:59 -02:00
Elias Dorneles 2514973242 re-enable skipped feed export tests 2016-01-21 21:22:12 -02:00
Elias Dorneles b746d85f4c PY3 port csv exporter 2016-01-21 21:12:43 -02:00
Elias Dorneles c76190d491 PY3: ported json(lines), xml exporters 2016-01-21 18:24:06 -02:00
Elias Dorneles b6ef1f19fd make BaseItemExporter export unicode, pushed down previous behavior for classes that need it 2016-01-21 15:36:14 -02:00
Elias Dorneles a76ecd4ef0 remove test_exporters from py3 ignores 2016-01-21 15:36:14 -02:00
Konstantin Lopuhin 5813de8838 py3: unskip test_closespider - it passes after fixing mockserver.Follow resouce on py3 2016-01-21 17:53:08 +03:00
Konstantin Lopuhin 4607f2843e py3: unskip test_crawl 2016-01-21 17:53:08 +03:00
Konstantin Lopuhin bf5f54fa33 py3: fix getarg 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin ad2b3321b9 py3 compat: use range, fixes CrawlTestCase.test_start_requests_bug_yielding 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin 0680950b98 py3: pass CrawlTestCase.test_referer_header, fixing Echo resource in mockserver and json decoding in test 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin 20b839b44b py3: pass first crawl test (test_follow_all): fix mock server 2016-01-21 17:52:12 +03:00
Daniel Graña 0f500a1f82 Merge pull request #1708 from scrapy/fix-scrapy-bench 
PY3 fixed scrapy bench command
 2016-01-21 11:51:27 -03:00
Raul Gallegos a06a5f00f4 adding configurable encoding for httpproxy authentication 2016-01-21 09:35:04 -05:00
Raul Gallegos f042ad0f39 py3 fix HttpProxy and Retry Middlewares 2016-01-21 09:35:04 -05:00
Paul Tremberth c9e046d11d Merge pull request #1657 from palego/os.mknod-fails-on-macos 
[MRG+1] change os.mknod() for open()
 2016-01-21 15:17:57 +01:00
Mikhail Korobov 80c55f19a1 PY3 fixed scrapy bench command 2016-01-21 18:31:58 +05:00
Paul Tremberth da83f2dca7 Merge pull request #1693 from lopuhin/py3-commands-fetch-shell 
[MRG+1] Py3: port commands fetch and shell
 2016-01-21 10:59:32 +01:00
Paul Tremberth 6d8620ffe7 Merge pull request #1691 from lopuhin/py3-test-engine 
[MRG+1] Py3: port test_engine
 2016-01-21 10:30:39 +01:00
Capi Etheriel 659715ecd9 implements FormRequest.from_response CSS support 2016-01-21 01:05:20 -02:00
Konstantin Lopuhin a5da7531c4 py3 backout skipping test_closespider - it was fixed on another branch 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin 47d3c63338 py3: port fetch and shell commands, and review + enable already passing test_closespider.py and tests/test_utils_template.py 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin c1db60188a py3: unskip passing test_engine 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin 0b08b4bfcf fix engine tests - this callback (spider_closed_callback) should accept one argument, but the error was hidden on py2 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin a32b59ac75 py3: fix EngineTest.test_crawler 2016-01-20 22:57:46 +03:00
Elias Dorneles 8a1255d3ba Merge pull request #1680 from lopuhin/py3-downloader-middleware 
[MRG+1] Py3: port downloader cache and compression middlewares
 2016-01-20 17:05:54 -02:00
Daniel Graña 176610f910 optional_features has been removed 2016-01-19 15:34:26 -03:00
Daniel Graña 5ec4319885 Merge pull request #1358 from nyov/nyov/boto-anon-connection 
[MRG+1] Support anonymous S3DownloadHandler (boto) connections
 2016-01-19 13:35:23 -03:00
Konstantin Lopuhin 4c84172840 move leveldb to tests/requirements-py3.txt 2016-01-19 18:24:58 +03:00
Konstantin Lopuhin 07577c432e Merge branch 'master' into py3-http-downloaders 2016-01-19 17:39:22 +03:00
Daniel Graña 49fe631d89 Merge pull request #1676 from lopuhin/py3-webclient 
[MRG+1] py3: fix webclient
 2016-01-19 11:07:54 -03:00