Repository navigation
[AISAGE-733] fix(ai-gateway): bill the upstream cost of BYOK-served transcriptions - #8877
Conversation
The "openrouters cost accounting doesn't make sense" check treated a cost without is_byok as broken accounting for every upstream. Probing showed which upstreams report is_byok next to cost: OpenRouter (chat completions, Messages, Responses, embeddings) and Vercel AI Gateway (chat completions). Martian reports cost but never is_byok, so every stealth-model response raised the report. Billing is unchanged.
OpenRouter transcriptions report only usage.cost, without is_byok or cost_details, so a BYOK-served transcription billed just OpenRouter's 5% BYOK fee. The response's x-generation-id header now becomes the message id, so the existing generation lookup in processTokenData supplies is_byok and upstream_inference_cost. The missing-is_byok report now only fires when there is no generation id; fetchGeneration already reports lookup failures.
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Executive SummaryThe only new commit renames the failed-generation-lookup Sentry message and updates its assertion to match; the wording, tag, severity, and extra payload stay consistent, and no other code references the old string, so no new defects found. Assumptions: Files Reviewed (2 files)
Previous Review Summaries (2 snapshots, latest commit 09ec669)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 09ec669)Status: No Issues Found | Recommendation: Merge Executive SummaryThe incremental change resolves the previously reported gap: Assumptions: Files Reviewed (2 files)
Previous review (commit 3981f39)Status: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)SUGGESTION
Files Reviewed (2 files)
Reviewed by deepseek-flash-latest · Input: 46.3K · Output: 5.1K · Cached: 374.5K Review guidance: REVIEW.md from base branch |
…tatus unknown With a generation id, the transcription parser no longer reports the missing is_byok, because the generation lookup resolves it. If that lookup fails, processTokenData kept the response cost, which for a BYOK-served request is only OpenRouter's 5% fee, with only an info-level log. Report that case at error level.
…tion-byok-cost # Conflicts: # packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts # packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.ts
The report raised when the generation lookup fails reused "openrouters cost accounting doesn't make sense". Say what happened instead: the lookup failed, so a cost without a BYOK status may be only the BYOK fee.
Summary
OpenRouter transcriptions report only
usage.cost, with nois_byokorcost_details. When OpenRouter serves a transcription with Kilo's BYOK keys, thatcostis just its 5% BYOK fee, so we billed 1/20 of the real cost. Production transcription costs match the fee.The response carries an
x-generation-idheader. OpenRouter's/generationendpoint for that id returnsis_byokandupstream_inference_cost. I checked this by hand foropenai/whisper-1andopenai/gpt-4o-mini-transcribe: both returnis_byok: truewithupstream_inference_costat 20 times the reportedcost, and token counts for the token-based model.The header now becomes the message id, so the existing generation lookup in
processTokenDatasupplies the real cost. Transcriptions have no inference provider in their response, so the lookup always runs. The missing-is_byokreport now fires only when there is no generation id. If the lookup fails, the cost kept from the response may be only the BYOK fee, soprocessTokenDatareports it at error level as "SUSPICIOUS: OpenRouter generation lookup failed, so a cost without BYOK status may be only the BYOK fee" (source: generation_lookup_fallback).Billing impact: BYOK-served transcriptions will charge their upstream cost instead of OpenRouter's fee.
Every figure comes from upstream data. No token or cost estimates are added.
Split out of #8791.
Verification
pnpm --filter @kilocode/web-shared typecheckjest packages/web-shared/src/lib/ai-gateway/llm-proxy-helpers.test.ts packages/web-shared/src/lib/ai-gateway/handlers/audio-transcriptionsprocessTokenDatatests: a failed generation lookup is reported, and a successful one billsupstream_inference_costwithout a report.Models: Claude Opus 5.5 (
kilo/anthropic/claude-opus-5.5), no subagents.