RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,773 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

4773 Commits

Author SHA1 Message Date
Julia Medina 5cd0296a16 Remove deprecated %z formatting from the default LOG_DATEFORMAT 2015-06-04 13:42:01 -03:00
Mikhail Korobov f6ae78f4b1 fixed spider custom_settings 
https://github.com/scrapy/scrapy/pull/1128 moved spidercls.update_settings
call to a later stage; this commit moves it back.
 2015-06-03 22:19:05 +05:00
Mikhail Korobov bd271d8b99 TST a test case to show custom_settings doesn't always work 2015-06-03 22:19:05 +05:00
Julia Medina 2d6450a4ad Add helper to update deprecated class paths 2015-06-01 20:39:29 -03:00
Pablo Hoffman 525b349932 update old crawlera link 2015-06-01 20:38:17 -03:00
Ally Weir 16218d6534 Spelling correction 
incorrect use of "too" instead of "to"
 2015-06-01 22:22:53 +05:00
Mikhail Korobov 19ac046343 fixed CrawlerProcess when settings are passed as dicts 
See https://github.com/scrapy/scrapy/pull/1156

(cherry picked from commit aa6a72707d)
 2015-06-01 12:50:41 -03:00
Mikhail Korobov 0b6c0eaa35 DOC fix non-working link (by removing it). 
See https://github.com/scrapy/scrapy/pull/1260
 2015-05-27 23:10:58 +05:00
Julia Medina e76359f2ae Update 1.0 release notes 
(cherry picked from commit 343d20d791)
 2015-05-27 12:02:56 -03:00
Alexander Chekunkov 6f6a251a9b LogCounterHandler is never removed from root handlers list, fix that 
lambda is garbage collected and because receiver is added as weak reference by default - when signals.engine_stopped is fired logging.root.removeHandler is not executed. Fixed that by assigning lambda to a private argument and not by using connect(..., weak=False) because I belive this lambda function should be collected with crawler object

(cherry picked from commit b2765aabd8)
 2015-05-27 12:02:45 -03:00
Alexander Chekunkov 6c264f0e22 remove CrawlerProcess.stopping as it isn't used any more 
(cherry picked from commit b0ea3e38d1)
 2015-05-27 12:02:34 -03:00
Pablo Hoffman 8d2531fb05 Merge pull request #1253 from scrapy/deferdelay 
Increase delay of defer calls
 2015-05-22 20:13:11 -03:00
Daniel Graña d439c26d76 update docstring and release notes 2015-05-22 20:00:58 -03:00
Alexey Vishnevsky 27ce3225bd Makes scrapy more async by letting to reactor spend another couple of cycles to accomplish its needs. 2015-05-22 17:05:19 -03:00
Julia Medina de6d232a02 Bump version: 0.25.1 → 1.0.0rc1 2015-05-22 13:24:27 -03:00
Julia Medina 29529e5e8e Merge pull request #1244 from Curita/1.0-release-notes 
1.0 release notes
 2015-05-22 13:21:17 -03:00
Julia Medina 600164594c New release cycle in .bumpversion.cfg 
1.0.0dev1 -> 1.0.0rc1 -> 1.0.0 -> 1.1.0dev1 -> ...
 2015-05-22 12:59:21 -03:00
Julia Medina afcf70cdc6 Add 1.0 release notes 2015-05-22 12:53:11 -03:00
Mikhail Korobov cc2258b2bb Merge pull request #1145 from bosnj/master 
[MRG+1] default return value for extract_first
 2015-05-21 22:03:54 +05:00
Daniel Graña 58717472f7 Merge pull request #1250 from chekunkov/scrapy-log-fix-incompatible-change 
[MRG+1] Keep level_names in scrapy.log for backwards compatibility
 2015-05-21 10:46:39 -03:00
Alexander Chekunkov 795ca3945f keep level_names in scrapy.log for backwards compatibility 2015-05-21 08:56:44 +00:00
Daniel Graña ee59112480 Merge pull request #1224 from scrapy/fix-empty-feed-export-fields 
[MRG] fixed FEED_EXPORT_FIELDS handling (see #1223)
 2015-05-19 16:36:05 -03:00
Daniel Graña 5beb9d251c Merge pull request #1243 from scrapy/remove-contrib-from-py3-ignores 
remove unnecessary lines from py3-ignores
 2015-05-19 12:20:51 -03:00
Mikhail Korobov 7a5b5ec4d6 TST remove unnecessary lines from py3-ignores 
scrapy/contrib is already skipped - see https://github.com/scrapy/scrapy/pull/1165
 2015-05-19 00:57:39 +05:00
Mikhail Korobov 21b17734c5 Merge pull request #1242 from Curita/exporters-single-module 
Move exporters/__init__.py to exporters.py
 2015-05-19 00:10:30 +05:00
Julia Medina 044f31cb8d Merge pull request #1240 from scrapy/fix-feedexport-logging 
MRG+1 fixed FeedExporter shutdown log messages
 2015-05-18 14:54:11 -03:00
Julia Medina af0c8f82f4 Move exporters/__init__.py to exporters.py 2015-05-18 14:46:23 -03:00
Mikhail Korobov 60e79db3ee fixed FeedExporter shutdown log messages 2015-05-18 19:28:37 +05:00
Mikhail Korobov 9b0ca1b7a0 drop support for FEED_EXPORT_FIELD=[] meaning "no fields" 2015-05-18 17:13:25 +05:00
Mikhail Korobov 9fb318338b support FEED_EXPORT_FIELDS=[] 2015-05-18 16:44:02 +05:00
Mikhail Korobov c412034864 Fix GH-1223 2015-05-18 16:44:02 +05:00
Mikhail Korobov e1efd19175 TST, DOC document that Scrapy only infers field names for CSV 2015-05-18 16:43:23 +05:00
Mikhail Korobov 8d9e3b7e8d TST add a test for JSON lines item exporter 2015-05-18 16:43:23 +05:00
Daniel Graña 3a4d57b3f5 Merge pull request #1236 from Curita/fix-failure-tracebacks 
Replace FailureFormatter with direct failure to exc_info conversions in log calls
 2015-05-15 15:00:36 -03:00
Julia Medina e7574a809e Use method to get traceback of failures in failure_to_exc_info 2015-05-15 14:51:14 -03:00
Julia Medina 6dccb3a9b3 Replace FailureFormatter with direct exc_info conversions in log calls 2015-05-15 05:46:14 -03:00
Daniel Graña 2d21677197 Merge pull request #1235 from Curita/remove-log-exception-calls 
Replace logger.exception calls to logger.error for Python compatibility
 2015-05-15 01:44:45 -03:00
Julia Medina 157cec7f64 Replace logger.exception calls for Python compatibility 2015-05-15 01:26:45 -03:00
Daniel Graña faba9bb095 Merge pull request #1206 from Curita/clean-test-collection 
Fix remaining warnings from pytest while collecting tests
 2015-05-14 23:19:47 -03:00
Daniel Graña 5e5e44e91e Merge pull request #1228 from nramirezuy/item_fields_defined_on_fields 
item fields defined on fields class attribute fix
 2015-05-14 23:09:24 -03:00
Daniel Graña fc1aaeb74d Merge pull request #1234 from Curita/add-docs-build-to-travis 
Add docs build to travis
 2015-05-14 23:02:49 -03:00
Julia Medina cbebcbddb1 Add docs toxenv to .travis.yml 2015-05-14 20:54:28 -03:00
Julia Medina ec54942925 Create separate testenvs to build docs and check links 2015-05-14 20:54:26 -03:00
Julia Medina a555778013 Build docs in a temporary folder to avoid caching 2015-05-14 20:53:25 -03:00
Mikhail Korobov 564aca11bf Merge pull request #1233 from Curita/docs-fix 
Fix in docs for error introduced in #1218
 2015-05-15 04:31:39 +05:00
Julia Medina 7c61bd897c Fix in docs for error introduced in #1218 2015-05-14 20:09:19 -03:00
Daniel Graña 1195f906d6 Merge pull request #1218 from Curita/move-base-to-packages 
Move base classes to their packages
 2015-05-14 18:28:17 -03:00
nramirezuy 773ea5a5ef item fields defined on fields class attribute fix 2015-05-13 15:51:23 -03:00
Pablo Hoffman 65aa9ccc70 Merge pull request #1220 from eliasdorneles/building-settingslist-from-docs 
Building settings list from docs
 2015-05-13 14:43:19 -03:00
Elias Dorneles 6884b73e19 sort settings list by name and wrap it in a bullet list 2015-05-11 22:11:30 -03:00