RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,104 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

5104 Commits

Author SHA1 Message Date
Marius Gedminas eaad10facf Typo 2015-09-28 14:44:15 +05:00
Pablo Hoffman 75cd056223 disable log on version command. closes #1426 2015-09-25 16:39:21 -03:00
Daniel Graña 69398fa148 Merge pull request #1505 from scrapy/fix-test-suite 
TST pin pytest to 2.7.3
 2015-09-21 18:19:59 -03:00
Mikhail Korobov 916141a08f TST pin pytest to 2.7.3 2015-09-21 21:12:52 +05:00
Mikhail Korobov 181f8d27d7 Merge pull request #1501 from scrapy/disable-log-on-startproject 
disable log on startproject command
 2015-09-20 02:47:41 +05:00
Daniel Graña 51b1249033 Revert "test suite requires recent queuelib to pass because it depends on queuelib test suite" 
This reverts commit a3390afc66.
 2015-09-18 02:27:07 -03:00
Daniel Graña a3390afc66 test suite requires recent queuelib to pass because it depends on queuelib test suite 2015-09-17 19:42:29 -03:00
Daniel Graña a57efdb008 Add PyPI download stats badge 2015-09-17 14:07:57 -03:00
Pablo Hoffman 22327ef47b disable log on startproject command 2015-09-17 01:05:52 -03:00
Daniel Graña 3c596dcf46 Merge pull request #1467 from dacjames/master 
add support for a nested loaders
 2015-09-16 22:03:19 -03:00
Mikhail Korobov 09f388c328 Merge pull request #1497 from haiiiiiyun/fix_typo_middlware 
fix typo middlware
 2015-09-16 15:39:24 +05:00
hy 14f7f22555 fix typos in downloader-middleware.rst and exceptions.rst, middlware -> middleware 2015-09-16 16:59:23 +08:00
Daniel Collins 036109e7de updte nested loader documentation 2015-09-15 23:49:35 -07:00
Daniel Graña 5279da9916 Merge pull request #1496 from scrapy/default-template 
[MRG+1] style fixes for settings.py created by `scrapy startproject`
 2015-09-14 17:14:15 -03:00
Mikhail Korobov 3e13740a57 Merge pull request #1481 from mlyundin/develop 
cleanup
 2015-09-15 00:36:50 +05:00
Daniel Graña 4d1c5c3d32 Merge pull request #1488 from scrapy/py3-redirect-downloader-mware 
[MRG+1] PY3 redirect downloader mware
 2015-09-14 02:20:13 -03:00
Mikhail Korobov 5a13022713 style fixes for settings.py created by scrapy startproject 2015-09-13 21:46:20 +05:00
Elias Dorneles 430e90f4b2 minor refactor on metarefresh redirect mware test 2015-09-13 12:29:41 -03:00
Elias Dorneles b06d0706e6 refactoring redirect logic 2015-09-13 11:49:59 -03:00
Elias Dorneles 81950f773d made encoding conversion more explicit, added test for header with utf-8 encoding replicating what browsers do 2015-09-11 18:51:48 -03:00
Elias Dorneles 179c4588ca adding test for latin1 location 2015-09-10 19:36:47 -03:00
Elias Dorneles defa899135 PY3 port redirect middleware 2015-09-10 16:31:59 -03:00
Elias Dorneles 44bd01d51c preparatory refactor in redirect mware 2015-09-10 16:19:56 -03:00
Daniel Graña a7787628ff Merge pull request #1263 from chekunkov/readable-middlewares-list-in-logs 
Make list of enabled middlewares more readable
 2015-09-10 12:09:26 -03:00
Alexander Chekunkov 563b1500bf Make list of enabled middlewares more readable 2015-09-10 14:59:55 +00:00
Daniel Graña 9a64d8ff97 fix scrapy squeue tests after recent changes to queuelib 2015-09-09 15:55:55 -03:00
Mikhail Korobov 600f27c39b Merge pull request #1350 from scrapy/installation-docs-update 
Updates in Installation and Contributing sections in the docs
 2015-09-09 17:54:35 +05:00
Julia Medina eb7b9d506f Add note to ubuntu install section about debian compatibility 2015-09-08 18:49:44 -03:00
Mikhail Lyundin be7821a482 Optimization - avoid temporary list objects, unnecessary function call 2015-09-08 22:10:24 +03:00
Daniel Graña 6490cb534e Merge pull request #1477 from mlyundin/master 
equal_attributes function optimization
 2015-09-07 14:32:27 -03:00
Daniel Collins 311d5cd495 split nested_loader into seperate methods 2015-09-04 13:15:48 -07:00
Daniel Collins 88c92cb68b provide documentation for nested loaders 2015-09-04 13:15:48 -07:00
Daniel Collins 425e35ee90 removed commented code and add test for error handling 2015-09-04 13:15:48 -07:00
Daniel Collins 1f95af3c07 add support for a nested loaders 2015-09-04 13:15:48 -07:00
Daniel Graña 70fa3ae9a5 Merge pull request #1479 from scrapy/twisted-15.4 
fixed compatibility with Twisted 15.4.0
 2015-09-04 14:10:59 -03:00
Daniel Graña 16e63040e9 Merge pull request #1480 from scrapy/travis-run-once 
don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch
 2015-09-04 14:09:21 -03:00
Mikhail Korobov d30e539049 don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch 2015-09-04 22:06:46 +05:00
Mikhail Korobov faf9265c91 fixed compatibility with Twisted 15.4.0 2015-09-04 20:50:48 +05:00
Mikhail Lyundin d022d3cb9e equal_attributes function optimization 2015-09-04 18:21:41 +03:00
Nicolás Alejandro Ramírez Quiros 6ae8963256 Merge pull request #1476 from mlyundin/master 
[MRG+1] Avoid creation of temporary list object in iflatten
 2015-09-02 17:31:04 -03:00
Mikhail Lyundin d9fddabe95 Avoid creation of temporary list object in iflatten 2015-09-02 23:12:36 +03:00
Daniel Graña dd473145f2 Merge pull request #1473 from ArturGaspar/master 
[MRG+1] Support for returning deferreds in middlewares
 2015-09-02 13:04:26 -03:00
Robert Weindl 833efbfcbd Update tutorial.rst 
Add missing "scrapy." prefix.
 2015-09-02 20:35:23 +05:00
Artur Gaspar dd83f6123d Test for same response object in downloader middleware test. 2015-09-02 11:36:15 -03:00
Artur Gaspar 2748b38592 Test for not calling the download function when downloader middleware returns a response in process_request. 2015-09-02 11:15:37 -03:00
Artur Gaspar 1f4af9d81e Test for robots.txt middleware for logger not being called when the request for robots.txt is ignored. 2015-09-02 10:33:53 -03:00
Artur Gaspar 668e5fd257 Test for robots.txt middleware for processing a request for which the robots.txt parser is ready. 2015-09-02 10:23:24 -03:00
Artur Gaspar a6a629e707 Call actual error logger in robots.txt middleware tests. 2015-09-02 10:23:24 -03:00
Artur Gaspar 9ce9a293a6 Always check robots.txt before making another request in RobotsTxtMiddleware. 2015-09-02 10:23:24 -03:00
Daniel Graña af0873746f Merge pull request #1289 from pawelmhm/dont_retry_400 
[MRG+1] [settings/default_settings.py] dont retry 400
 2015-09-02 00:39:30 -03:00