RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,096 Commits 49 Branches 132 Tags 182 MiB
Commit Graph

5096 Commits

Author SHA1 Message Date
Daniel Graña a57efdb008 Add PyPI download stats badge 2015-09-17 14:07:57 -03:00
Daniel Graña 3c596dcf46 Merge pull request #1467 from dacjames/master 
add support for a nested loaders
 2015-09-16 22:03:19 -03:00
Mikhail Korobov 09f388c328 Merge pull request #1497 from haiiiiiyun/fix_typo_middlware 
fix typo middlware
 2015-09-16 15:39:24 +05:00
hy 14f7f22555 fix typos in downloader-middleware.rst and exceptions.rst, middlware -> middleware 2015-09-16 16:59:23 +08:00
Daniel Collins 036109e7de updte nested loader documentation 2015-09-15 23:49:35 -07:00
Daniel Graña 5279da9916 Merge pull request #1496 from scrapy/default-template 
[MRG+1] style fixes for settings.py created by `scrapy startproject`
 2015-09-14 17:14:15 -03:00
Mikhail Korobov 3e13740a57 Merge pull request #1481 from mlyundin/develop 
cleanup
 2015-09-15 00:36:50 +05:00
Daniel Graña 4d1c5c3d32 Merge pull request #1488 from scrapy/py3-redirect-downloader-mware 
[MRG+1] PY3 redirect downloader mware
 2015-09-14 02:20:13 -03:00
Mikhail Korobov 5a13022713 style fixes for settings.py created by scrapy startproject 2015-09-13 21:46:20 +05:00
Elias Dorneles 430e90f4b2 minor refactor on metarefresh redirect mware test 2015-09-13 12:29:41 -03:00
Elias Dorneles b06d0706e6 refactoring redirect logic 2015-09-13 11:49:59 -03:00
Elias Dorneles 81950f773d made encoding conversion more explicit, added test for header with utf-8 encoding replicating what browsers do 2015-09-11 18:51:48 -03:00
Elias Dorneles 179c4588ca adding test for latin1 location 2015-09-10 19:36:47 -03:00
Elias Dorneles defa899135 PY3 port redirect middleware 2015-09-10 16:31:59 -03:00
Elias Dorneles 44bd01d51c preparatory refactor in redirect mware 2015-09-10 16:19:56 -03:00
Daniel Graña a7787628ff Merge pull request #1263 from chekunkov/readable-middlewares-list-in-logs 
Make list of enabled middlewares more readable
 2015-09-10 12:09:26 -03:00
Alexander Chekunkov 563b1500bf Make list of enabled middlewares more readable 2015-09-10 14:59:55 +00:00
Daniel Graña 9a64d8ff97 fix scrapy squeue tests after recent changes to queuelib 2015-09-09 15:55:55 -03:00
Mikhail Korobov 600f27c39b Merge pull request #1350 from scrapy/installation-docs-update 
Updates in Installation and Contributing sections in the docs
 2015-09-09 17:54:35 +05:00
Julia Medina eb7b9d506f Add note to ubuntu install section about debian compatibility 2015-09-08 18:49:44 -03:00
Mikhail Lyundin be7821a482 Optimization - avoid temporary list objects, unnecessary function call 2015-09-08 22:10:24 +03:00
Daniel Graña 6490cb534e Merge pull request #1477 from mlyundin/master 
equal_attributes function optimization
 2015-09-07 14:32:27 -03:00
Daniel Collins 311d5cd495 split nested_loader into seperate methods 2015-09-04 13:15:48 -07:00
Daniel Collins 88c92cb68b provide documentation for nested loaders 2015-09-04 13:15:48 -07:00
Daniel Collins 425e35ee90 removed commented code and add test for error handling 2015-09-04 13:15:48 -07:00
Daniel Collins 1f95af3c07 add support for a nested loaders 2015-09-04 13:15:48 -07:00
Daniel Graña 70fa3ae9a5 Merge pull request #1479 from scrapy/twisted-15.4 
fixed compatibility with Twisted 15.4.0
 2015-09-04 14:10:59 -03:00
Daniel Graña 16e63040e9 Merge pull request #1480 from scrapy/travis-run-once 
don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch
 2015-09-04 14:09:21 -03:00
Mikhail Korobov d30e539049 don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch 2015-09-04 22:06:46 +05:00
Mikhail Korobov faf9265c91 fixed compatibility with Twisted 15.4.0 2015-09-04 20:50:48 +05:00
Mikhail Lyundin d022d3cb9e equal_attributes function optimization 2015-09-04 18:21:41 +03:00
Nicolás Alejandro Ramírez Quiros 6ae8963256 Merge pull request #1476 from mlyundin/master 
[MRG+1] Avoid creation of temporary list object in iflatten
 2015-09-02 17:31:04 -03:00
Mikhail Lyundin d9fddabe95 Avoid creation of temporary list object in iflatten 2015-09-02 23:12:36 +03:00
Daniel Graña dd473145f2 Merge pull request #1473 from ArturGaspar/master 
[MRG+1] Support for returning deferreds in middlewares
 2015-09-02 13:04:26 -03:00
Robert Weindl 833efbfcbd Update tutorial.rst 
Add missing "scrapy." prefix.
 2015-09-02 20:35:23 +05:00
Artur Gaspar dd83f6123d Test for same response object in downloader middleware test. 2015-09-02 11:36:15 -03:00
Artur Gaspar 2748b38592 Test for not calling the download function when downloader middleware returns a response in process_request. 2015-09-02 11:15:37 -03:00
Artur Gaspar 1f4af9d81e Test for robots.txt middleware for logger not being called when the request for robots.txt is ignored. 2015-09-02 10:33:53 -03:00
Artur Gaspar 668e5fd257 Test for robots.txt middleware for processing a request for which the robots.txt parser is ready. 2015-09-02 10:23:24 -03:00
Artur Gaspar a6a629e707 Call actual error logger in robots.txt middleware tests. 2015-09-02 10:23:24 -03:00
Artur Gaspar 9ce9a293a6 Always check robots.txt before making another request in RobotsTxtMiddleware. 2015-09-02 10:23:24 -03:00
Daniel Graña af0873746f Merge pull request #1289 from pawelmhm/dont_retry_400 
[MRG+1] [settings/default_settings.py] dont retry 400
 2015-09-02 00:39:30 -03:00
Daniel Graña 3919ad64c5 Merge pull request #1470 from scrapy/py3-middlewares 
[MRG] port some downloader middlewares to Python 3
 2015-09-02 00:17:37 -03:00
Daniel Graña bc499cb552 Merge pull request #1472 from olafdietsche/incomplete-submit-button 
[MRG+1] Incomplete submit button
 2015-09-02 00:16:42 -03:00
Artur Gaspar ca83a0b028 Support for returning deferreds in downloader middleware methods. 2015-09-01 13:22:43 -03:00
Olaf Dietsche 937277e859 Make sure value attribute values are set. 2015-09-01 12:44:04 +02:00
Olaf Dietsche 2c28b53cc0 Add tests with incomplete buttons 2015-09-01 12:35:49 +02:00
Mikhail Korobov c44cafe4f5 PY3 fix UserAgentMiddleware tests 2015-09-01 04:11:33 +05:00
Mikhail Korobov 3a9c73bc5d PY3 fix DownloaderStats middleware tests 2015-09-01 04:09:15 +05:00
Mikhail Korobov 78a4cd0f1c PY3 fix HttpAuthMiddleware tests 2015-09-01 04:00:26 +05:00