Teknium eb6aa03609 feat(analytics): record auxiliary model usage per task in session accounting (#65537) ... * feat(analytics): record auxiliary model usage per task in session accounting Auxiliary LLM calls (vision, compression, title_generation, web_extract, session_search, ...) discarded their token usage, leaving dashboard analytics blind to aux model spend (issue #23270). - hermes_state.py: session_model_usage gains a task PK dimension (''=main loop) via v22 table-rebuild migration (SQLite can't alter a PK); record_auxiliary_usage() writes per-(model,provider,task) deltas WITHOUT touching sessions counters (gateway overwrites those with absolute main-loop totals — folding aux in would double-count or be clobbered). Aux rows never inherit the session's main-loop route. - agent/aux_accounting.py: ContextVar ambient accounting context (mirrors the portal_tags conversation context); record_aux_usage() normalizes usage via usage_pricing.normalize_usage, estimates cost, and is strictly best-effort. moa_reference/moa_aggregator excluded — conversation_loop already folds MoA usage+cost into the main delta. - agent/auxiliary_client.py: _validate_llm_response is the recording chokepoint — every successful non-streaming aux response passes through it exactly once, sync and async, including fallback paths (model read from the response itself stays accurate across fallbacks). - run_agent.py: run_conversation publishes/resets the accounting context; agent/title_generator.py republishes on its bare thread. - hermes_cli/web_server.py: /api/analytics/usage folds aux rows into by_model (aux-only models finally appear) and adds a by_task summary; /api/analytics/models surfaces aux rows on the Models page. Design per review of PR #62850 by @eeksock (thread-local + separate auxiliary_usage table): rebuilt on ContextVar (async-safe — thread-local cross-attributes concurrent coroutines on one event loop) and the existing session_model_usage table instead of a parallel accounting path, extended beyond vision to every aux task, and wired the analytics endpoints so the dashboard actually shows it. Credit to @eeksock for the approach and @tboatman for the detailed root-cause analysis. * test(moa): match _validate_llm_response mock to new accounting-hint signature * test(aux): accept accounting-hint kwargs in remaining _validate_llm_response mocks		2026-07-16 04:23:12 -07:00
..
lsp	…
transports	fix(agent): treat Codex incomplete content filter as refusal	2026-07-15 09:47:37 -07:00
__init__.py	…
test_account_usage.py	feat(codex): redeem banked usage-limit resets via /usage reset (#64280)	2026-07-14 03:23:19 -07:00
test_anthropic_adapter.py	…
test_anthropic_billing_guidance.py	…
test_anthropic_keychain.py	…
test_anthropic_kwargs_sanitize.py	…
test_anthropic_mcp_prefix_strip.py	…
test_anthropic_oauth_pkce.py	…
test_anthropic_oauth_ua_prefix.py	…
test_anthropic_output_field_leak.py	…
test_anthropic_thinking_block_order.py	…
test_arcee_trinity_overrides.py	…
test_async_utils.py	…
test_auxiliary_anthropic_pool_fallback_regression.py	…
test_auxiliary_client.py	feat(analytics): record auxiliary model usage per task in session accounting (#65537)	2026-07-16 04:23:12 -07:00
test_auxiliary_client_anthropic_custom.py	…
test_auxiliary_client_azure_foundry.py	…
test_auxiliary_client_base_url_host_validation_52608.py	…
test_auxiliary_client_bootstrap_skew.py	fix(auxiliary_client): warn once + regression tests for bootstrap version skew	2026-07-15 07:47:18 -07:00
test_auxiliary_client_proxy_env.py	…
test_auxiliary_client_resolve_dedup.py	…
test_auxiliary_client_ssl_verify.py	…
test_auxiliary_client_xai_oauth_recovery.py	…
test_auxiliary_compression_timeout_floor.py	feat(analytics): record auxiliary model usage per task in session accounting (#65537)	2026-07-16 04:23:12 -07:00
test_auxiliary_config_bridge.py	…
test_auxiliary_main_first.py	…
test_auxiliary_named_custom_providers.py	…
test_auxiliary_transient_retry.py	…
test_auxiliary_transport_autodetect.py	…
test_auxiliary_user_default_headers.py	…
test_azure_identity_adapter.py	…
test_bedrock_1m_context.py	…
test_bedrock_adapter.py	test(bedrock): monkeypatch-based bearer routing tests + SigV4 regression	2026-07-15 09:59:38 -07:00
test_bedrock_integration.py	…
test_bedrock_interrupt_post_worker.py	…
test_billing_view.py	…
test_bounded_response.py	…
test_cascading_interrupt_6600.py	…
test_chat_completion_helpers_provider_sort.py	…
test_close_interrupted_tool_sequence.py	…
test_codex_app_server_persist.py	…
test_codex_cloudflare_headers.py	…
test_codex_gpt55_autoraise_notice.py	…
test_codex_responses_adapter.py	fix(agent): treat Codex incomplete content filter as refusal	2026-07-15 09:47:37 -07:00
test_codex_ttfb_watchdog.py	fix(agent): add finite hard ceiling on openai-codex request time (#64507)	2026-07-15 04:55:48 -07:00
test_coding_context.py	…
test_compaction_anti_thrash.py	fix(context): persist fallback compaction breaker	2026-07-14 02:19:40 +05:30
test_compress_focus.py	…
test_compressed_summary_metadata.py	…
test_compression_concurrent_fork.py	fix(agent): distinguish missing from broken compression locks	2026-07-14 00:14:55 +05:30
test_compression_count_warning_36908.py	…
test_compression_fallback_budget.py	fix(compression): give fallback candidates their own timeout budget + escalate repeat-timeout cooldowns (#65143)	2026-07-15 12:28:09 -07:00
test_compression_interrupt_protection.py	…
test_compression_logging_session_context.py	…
test_compression_progress.py	…
test_compression_rotation_state.py	fix(context): persist fallback compaction breaker	2026-07-14 02:19:40 +05:30
test_compression_small_ctx_threshold_floor.py	…
test_compressor_assistant_tail_anchor.py	…
test_compressor_historical_media.py	…
test_compressor_image_tokens.py	…
test_compressor_media_stripping.py	feat(compressor): keep image URLs and unknown-part markers in summarizer serialization	2026-07-15 09:47:04 -07:00
test_compressor_tool_call_budget.py	…
test_compressor_zero_user_guard.py	…
test_context_breakdown.py	…
test_context_compressor.py	fix(context): preserve missing-key compression history	2026-07-16 14:34:03 +05:30
test_context_compressor_cross_session_guard.py	…
test_context_compressor_session_end_clears_state.py	…
test_context_compressor_summary_continuity.py	…
test_context_compressor_temporal_anchoring.py	…
test_context_engine.py	…
test_context_engine_host_contract.py	fix(context): persist fallback compaction breaker	2026-07-14 02:19:40 +05:30
test_context_references.py	…
test_context_refs_concurrent.py	…
test_copilot_acp_client.py	fix(file-safety): distinguish safe-root write denial from credential blocks	2026-07-14 17:09:40 +05:30
test_copilot_acp_deprecation.py	…
test_credential_pool.py	…
test_credential_pool_oat_authtype.py	fix(auth): centralize pool auth normalization	2026-07-13 23:20:48 +05:30
test_credential_pool_oauth_writethrough.py	…
test_credential_pool_provider_boundary.py	…
test_credential_pool_routing.py	…
test_credits_cold_start.py	…
test_credits_fixture_snapshot.py	…
test_credits_policy.py	…
test_credits_tracker.py	…
test_credits_view.py	…
test_cron_inline_api_call_62151.py	…
test_crossloop_client_cache.py	…
test_curator.py	…
test_curator_activity.py	…
test_curator_backup.py	…
test_curator_classification.py	…
test_curator_reports.py	…
test_custom_pool_mismatch_guard.py	…
test_custom_provider_extra_body.py	…
test_custom_provider_extra_body_matching.py	fix(agent): honor custom-provider extra_body for multi-model catalogs	2026-07-12 23:30:52 -07:00
test_custom_providers_vision.py	…
test_deepseek_anthropic_thinking.py	…
test_direct_provider_url_detection.py	…
test_display.py	…
test_display_emoji.py	…
test_display_todo_progress.py	…
test_display_tool_failure.py	…
test_empty_tool_name_loop_dampening.py	…
test_error_classifier.py	fix(plugins): classify stale-call circuit breaker as failover, not retry	2026-07-16 00:27:39 +05:30
test_external_skills.py	…
test_external_skills_dirs_cache.py	…
test_failover_identity.py	…
test_file_safety.py	…
test_file_safety_container_mirror.py	…
test_file_safety_credentials.py	refactor(terminal,file-tools): delete legacy env-side cwd tracking (step 4)	2026-07-16 00:18:38 -07:00
test_file_safety_cross_profile.py	…
test_file_safety_sandbox_mirror.py	…
test_gemini_fast_fallback.py	…
test_gemini_free_tier_gate.py	…
test_gemini_native_adapter.py	feat(gemini): improve request context for support and compatibility	2026-07-13 22:26:19 -07:00
test_gemini_schema.py	…
test_i18n.py	…
test_image_gen_registry.py	…
test_image_routing.py	…
test_insights.py	…
test_intent_ack_continuation.py	…
test_jiter_preload.py	…
test_kanban_stop.py	follow-up: integrate agent nudge + dispatcher retry docs and tests	2026-07-14 16:47:33 +05:30
test_kimi_coding_anthropic_thinking.py	…
test_last_total_tokens.py	…
test_learn_prompt.py	…
test_learning_graph.py	…
test_learning_graph_render.py	…
test_learning_mutations.py	…
test_local_stream_timeout.py	…
test_manual_compression_feedback.py	fix(context): harden compression failure feedback	2026-07-16 14:34:03 +05:30
test_markdown_tables.py	…
test_memory_async_sync.py	…
test_memory_boundary_commit.py	…
test_memory_provider.py	…
test_memory_session_switch.py	…
test_memory_skill_scaffolding.py	…
test_memory_user_id.py	…
test_memory_write_bridge.py	…
test_message_content.py	…
test_minimax_auxiliary_url.py	…
test_minimax_provider.py	…
test_moa_aggregator_cache_control.py	…
test_moa_aggregator_cost_slot.py	…
test_moa_reasoning_effort.py	fix(moa): aggregator resolves reasoning like an acting model when slot is unset (#64756)	2026-07-15 00:11:04 -07:00
test_moa_slot_api_mode.py	…
test_moa_switch_api_mode.py	…
test_moa_trace_streamed_capture.py	…
test_model_extra_type_guard.py	…
test_model_metadata.py	…
test_model_metadata_local_ctx.py	…
test_model_metadata_ssl.py	…
test_models_dev.py	…
test_moonshot_schema.py	…
test_non_stream_stale_timeout.py	…
test_none_deref_guards.py	…
test_nous_credits_gauge.py	…
test_nous_credits_snapshot.py	…
test_nous_oauth_401_guidance.py	…
test_nous_rate_guard.py	…
test_onboarding.py	…
test_oneshot.py	…
test_openrouter_response_cache.py	…
test_pet_engine.py	…
test_pet_generate.py	…
test_platform_hint_desktop.py	…
test_platform_hint_overrides.py	…
test_plugin_llm.py	…
test_portal_tags.py	feat(portal): ambient conversation context entangles aux/MoA/delegate calls	2026-07-16 01:13:43 -07:00
test_preflight_compression_gate.py	…
test_probe_cache_followups.py	…
test_prompt_builder.py	fix(agent): gate Telegram rich-Markdown hint on rich_messages config	2026-07-14 06:48:53 -07:00
test_prompt_caching.py	…
test_proxy_and_url_validation.py	…
test_rate_limit_tracker.py	…
test_reactions.py	…
test_reasoning_stale_timeout_floor.py	…
test_redact.py	…
test_replay_cleanup.py	…
test_restore_primary_pool_reselect.py	…
test_resume_stale_active_task.py	…
test_runtime_cwd.py	…
test_save_url_image.py	…
test_secret_scope.py	…
test_set_runtime_main_custom_provider.py	…
test_shell_hooks.py	…
test_shell_hooks_consent.py	…
test_skill_bundles.py	…
test_skill_commands.py	fix(skills): guard skill slash commands against core-command and slug collisions	2026-07-14 16:41:21 +05:30
test_skill_commands_reload.py	…
test_skill_utils.py	…
test_ssl_ca_guard.py	…
test_ssl_verify.py	…
test_stream_read_timeout_floor.py	…
test_streaming_context_scrubber.py	…
test_subagent_progress.py	…
test_subagent_stop_hook.py	…
test_subdirectory_hints.py	…
test_subdirectory_hints_tilde.py	…
test_summarize_tool_result_type_safety.py	fix: harden remaining display-layer surfaces against non-string tool args	2026-07-15 00:10:22 -07:00
test_summary_prefix_semantics.py	…
test_system_prompt.py	fix(agent): gate Telegram rich-Markdown hint on rich_messages config	2026-07-14 06:48:53 -07:00
test_system_prompt_restore.py	…
test_think_scrubber.py	test(agent): cover think scrubber leak after flush-then-retry	2026-07-16 01:07:29 +05:30
test_thinking_timeout_guidance.py	…
test_thread_scoped_output.py	…
test_title_generator.py	…
test_tool_call_arg_no_redaction.py	…
test_tool_dispatch_helpers.py	…
test_tool_guardrails.py	…
test_tool_result_classification.py	…
test_trace_upload.py	…
test_transcription_registry.py	fix(deepinfra): harden multimodal provider routing	2026-07-14 02:59:39 +05:30
test_tts_registry.py	…
test_turn_context.py	fix(cli): preserve noted staged input on close	2026-07-14 03:32:45 +05:30
test_turn_finalizer_cleanup_guard.py	…
test_turn_finalizer_final_response_persistence.py	test(session): type finalizer clean-history assertions	2026-07-14 03:32:46 +05:30
test_turn_finalizer_interrupt_alternation.py	…
test_turn_finalizer_iteration_limit_exit.py	…
test_turn_retry_state.py	…
test_unsupported_parameter_retry.py	feat(analytics): record auxiliary model usage per task in session accounting (#65537)	2026-07-16 04:23:12 -07:00
test_unsupported_temperature_retry.py	feat(analytics): record auxiliary model usage per task in session accounting (#65537)	2026-07-16 04:23:12 -07:00
test_usage_pricing.py	…
test_verification_evidence.py	…
test_verification_stop.py	…
test_verification_stop_caching.py	…
test_verify_hooks.py	…
test_vertex_adapter.py	…
test_video_gen_registry.py	fix(deepinfra): harden multimodal provider routing	2026-07-14 02:59:39 +05:30
test_vision_resolved_args.py	…
test_vision_routing_31179.py	…