RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,653 Commits 49 Branches 132 Tags 179 MiB
Commit Graph

4653 Commits

Author SHA1 Message Date
Elias Dorneles d1053d2a2f automatic settings list for docs [WIP] 2015-05-04 02:17:19 -03:00
Mikhail Korobov fbb1078f58 Merge pull request #1060 from Curita/python-logging 
[MRG+1] Python logging
 2015-04-29 23:20:34 +05:00
Daniel Graña 5eb098a939 Merge pull request #1168 from scrapy/service-identity 
install service_identity package in tests to prevent warnings
 2015-04-28 23:48:58 -03:00
Mikhail Korobov fa1039f5b2 Merge pull request #1187 from Curita/relax-spiderloader-check 
Relax SpiderLoader interface check
 2015-04-24 03:40:10 +05:00
Julia Medina cc4c31e426 Relax SpiderLoader interface check 2015-04-23 15:08:04 -03:00
Julia Medina 1d8f8221e6 Add backward compatibility to LogFormatter 2015-04-22 17:27:24 -03:00
Julia Medina 4858af4e94 Fix backward compatible functions in scrapy.log 2015-04-22 17:27:24 -03:00
Julia Medina 7a92dae4c8 Change Scrapy log output through docs 2015-04-22 17:27:24 -03:00
Julia Medina 6d1205063c Add a filter to replace '__name__' loggers with 'scrapy' 2015-04-22 17:24:41 -03:00
Julia Medina 4f54ca3294 Change 'scrapy' logger for '__name__' on every module 2015-04-22 17:24:41 -03:00
Julia Medina 69a3d58110 Basic example on manually configuring log handlers 2015-04-22 17:24:41 -03:00
Julia Medina bd0b639b21 Fix logging usage across docs 2015-04-22 17:24:41 -03:00
Julia Medina 4811d16f1d Update `logger` attr and `log` method in the Spiders topic on docs 2015-04-22 17:24:41 -03:00
Julia Medina d47a7edc65 Update Logging topic on docs 2015-04-22 17:24:40 -03:00
Julia Medina ccdd8bfbcc Parametrize log formatting strings 2015-04-22 17:24:40 -03:00
Julia Medina 21b9f377d6 Deprecate more frequently used functions from scrapy/log.py 2015-04-22 17:24:40 -03:00
Julia Medina c174d78f12 Deprecate scrapy/log.py 2015-04-22 17:24:40 -03:00
Julia Medina 6acb3848fb Stdout redirect in configure_logging 2015-04-22 17:24:40 -03:00
Julia Medina ffd97f2f1b Set root handlers based on settings in configure_logging 2015-04-22 17:24:40 -03:00
Julia Medina 1c8708eb82 Create a logger for every Spider and adapt Spider.log to log through it 2015-04-22 17:24:40 -03:00
Julia Medina ac40ef611a Custom handler to count log level occurrences in a crawler 2015-04-22 17:24:40 -03:00
Julia Medina b75556ef79 Add a logging filter to mimic Twisted's log.err formating for Failures 2015-04-22 17:24:40 -03:00
Julia Medina 8baad55267 New scrapy/utils/log.py file with basic log helpers 
There are two functions, `configure_logging` and `log_scrapy_info` which
intend to replace scrapy.log.start and scrapy.log.scrapy_info
respectively.

Creating new functions makes evident the backward incompatible change of
using another logging system, and since the Python logging module is a
standard builtin, additional helpers make sense to be on a scrapy/utils
file.
 2015-04-22 17:24:40 -03:00
Julia Medina 6f9b423215 Restructure LogFormatter to comply with std logging calls 2015-04-22 17:24:40 -03:00
Julia Medina c2d716807a Use LogCapture in testfixtures package for tests 
This allows to remove `get_testlog` helper, `flushLoggedErrors` from
twisted.trial.unittest.TestCase and Twisted log observers created for
each test on conftest.py.
 2015-04-22 17:24:40 -03:00
Julia Medina 7a958f90be Replace scrapy.log calls for their equivalents in the logging std module 
Changes:
 - Each module takes 'scrapy' logger and logs through it
 - Lazy string evaluation in all log messages
 - Added missing log messages in scrapy/core/engine.py
 - Contextual data such as crawler or spider instances, and failures
 2015-04-22 17:24:39 -03:00
Daniel Graña 571bf68d7d Merge pull request #353 from nramirezuy/item-multi_inherit 
[MRG] Item multi inheritance fix
 2015-04-22 05:37:41 -03:00
nramirezuy 7871acd665 Item multi inherit fixed 2015-04-21 18:10:52 -03:00
Daniel Graña ded3f9cb0e Merge pull request #1185 from scrapy/fix-spiderloader-backwards-compatibility 
[MRG +1]fixed backwards compatibility for SPIDER_MANAGER_CLASS option
 2015-04-21 17:17:14 -03:00
Mikhail Korobov c3d3a94914 fixed backwards compatibility for SPIDER_MANAGER_CLASS option 2015-04-22 00:29:40 +05:00
Pablo Hoffman 0a5bbbaed3 Merge pull request #1159 from scrapy/feed-export-fields 
[MRG+1] FEED_EXPORT_FIELDS option
 2015-04-21 15:54:24 -03:00
Pablo Hoffman 4c4eb4f77d Merge pull request #1180 from eliasdorneles/tutorial-improvements 
Some improvements for Scrapy tutorial
 2015-04-21 15:49:39 -03:00
Daniel Graña e034947b54 Merge pull request #1166 from scrapy/spider-loader 
[MRG+1] rename SpiderManager to SpiderLoader
 2015-04-21 15:42:17 -03:00
Pablo Hoffman 06e1ca9ef2 Merge pull request #1177 from Curita/remove-djangoitem 
Remove djangoitem
 2015-04-21 15:35:47 -03:00
Pablo Hoffman e4122cdd79 Merge pull request #1150 from eliasdorneles/docs-files-pipeline 
Documenting Files Pipeline together with Images Pipeline
 2015-04-21 15:28:47 -03:00
Elias Dorneles d63567531d change data extraction in crawl example to be consistent with tutorial, removed statement implying mandatory usage of Item 2015-04-21 11:30:48 -03:00
Elias Dorneles f7da69d116 fixing example CSS expr 2015-04-21 11:19:10 -03:00
Elias Dorneles ff007afb9d expanded crawling primer with examples, and applied other suggestions from the review 2015-04-21 10:57:44 -03:00
Mikhail Korobov c8ccb7c5b9 Merge pull request #722 from Digenis/loader-iteration 
[MRG +1]ItemLoader.load_item: iterate over copy of fields
 2015-04-21 12:34:59 +05:00
Elias Dorneles 595146e158 some improvements for Scrapy tutorial 2015-04-20 21:10:07 -03:00
Julia Medina 0bf1ec160d Merge branch 'nyov/remove-deploy' from pull request #1102 2015-04-20 20:36:33 -03:00
nyov 7ae37d61f9 remove scrapy deploy command 
(closes #1027, #1095, #1102)
 2015-04-20 20:35:06 -03:00
Julia Medina 6da091f162 Remove djangoitem topic from index 2015-04-20 20:19:28 -03:00
Nikolaos-Digenis Karagiannis d39722f6a3 ItemLoader.load_item: iterate over copy of fields 2015-04-20 11:05:56 +03:00
Nikolaos-Digenis Karagiannis 017fb25bee loader test with processors that use item's values 2015-04-20 11:04:31 +03:00
Daniel Graña 16a1a938c1 Add 0.24.6 release notes 2015-04-20 01:40:21 -03:00
Mikhail Korobov 1794a893f4 Merge pull request #1172 from bagratte/docs 
minor corrections in documentation.
 2015-04-19 21:41:29 +05:00
Julia Medina ffc60910aa Remove djangoitem since we moved it to scrapy/scrapy-djangoitem 2015-04-19 13:09:25 -03:00
bagratte 1312bcd007 minor corrections in documentation. 2015-04-19 18:58:15 +04:00
bagratte beea9267a1 minor corrections in documentation. 2015-04-18 19:48:25 +04:00