A help question might use one search and a short answer. A data question might need name resolution, two queries, a correction, and a PDF. Charging both as "one message" is simple, but it hides the work that determines cost.
This tutorial designs an AI assistant credit system around measured usage. It explains cache-aware token accounting, separate help and data allowances, the concurrency problem in pre-turn checks, and the operational feedback needed to improve an agentic system.
Define the Unit Before the Budget
One question may search help once. Another may use several model steps, resolve names, query records, and render a document. A fixed charge per message hides that difference.
The reviewed credit calculation sums token usage across model calls. Cached reads and writes are included in reported input tokens, so they must be subtracted before weighting uncached input.
This illustrative implementation preserves the formula verified in the credit source:
const uncached = Math.max(0, inputTokens - cacheReadTokens - cacheWriteTokens);
const weighted =
uncached +
5 * outputTokens +
0.1 * cacheReadTokens +
1.25 * cacheWriteTokens;
const multiplier = tier === "advanced" ? 2.5 : 1;
const credits = Math.round((weighted / 10_000) * multiplier * 10_000) / 10_000;For synthetic usage of 8,000 input tokens, including 6,000 cached-read tokens, plus 1,000 output tokens and no cache writes, the weighted total is 7,600. That is 0.76 base-tier credits or 1.9 advanced-tier credits.
These are internal work units, not universal provider pricing. The weights and tier multipliers are configuration choices that need recalibration as models and usage change.
PDF rendering adds five credits. Charts and CSV have no extra flat charge, although their model turns still consume metered work.
Help and data have separate monthly plan allowances, with tenant overrides that can raise or lower the budget. Usage notices warn at 80%. Data questions stop at 100%; help continues past its allowance and stops at 150%. Owner notifications make approaching limits visible outside the chat panel.
The important limitation is concurrency. A pre-turn check can pass for multiple running turns before their final usage is recorded. These thresholds are not atomic reservations, so usage can overshoot. A proposed improvement is to reserve an estimated allowance atomically, then reconcile actual usage when work completes. That requires a policy for cancellation, failed turns, and abandoned reservations as well as successful answers.
Decide When a Turn Consumes an Allowance
Perform allowance checks before starting model work. Build the available tool set from the result: exhausting data credits should remove business-data tools while preserving eligible help and navigation tools. Enforce the same distinction in the backend rather than relying on a disabled browser button.
Monthly windows in the reference start at the beginning of the calendar month in UTC. State this consistently in notices; a user's local midnight and a UTC reset are not always the same instant.
The ledger records model usage across completed steps, including a turn that reached execution but ended in cancellation or failure. A refusal before model work begins is a different event. Do not count "successful answers" as a proxy for provider work: a failed answer can still have consumed tokens.
The reference chat explicitly uses the base tier. The advanced multiplier exists in configuration, but the system does not automatically route difficult questions to that tier. A future routing policy would need its own cost and quality evaluation.
Treat Reservations as a Separate Design
Suppose two requests both see one credit remaining. Each passes the pre-turn check, and each later consumes 0.8 credits. The resulting 1.6-credit usage is possible even if both checks were individually correct.
Atomic reservations are a proposed fix, not a property of the reference system. A reservation design must choose an estimate, claim it atomically, settle actual usage, and release unused capacity. It must also handle a worker crashing after the claim, cancellation after provider work begins, and actual usage exceeding the estimate.
A reservation can bound admission only under the rules you define. Calling it a hard cap without defining overage and recovery behavior creates another misleading guarantee.
Keep Accounting Distinct from Conversation History
A usage ledger answers "what work happened and what did it cost?" A saved conversation answers "what did this user ask and receive?" Step records answer "which model and tool decisions led to this outcome?"
The reference keeps saved user-owned conversations and feedback, while diagnostic steps are retained for 90 days. Usage metadata includes tokens, credits, model, prompt version, outcomes, timing, and tool summaries. Detailed tenant step records support debugging. These have different access and retention needs.
Some accounting writes are best effort. That preserves answer delivery when logging fails, but missing ledger entries can affect budget accuracy. If credits become a strict billing contract, durable reconciliation becomes a separate requirement rather than an incidental logging improvement.
Evaluate Quality and Cost Together
Tests, model evaluations, and human feedback answer different questions. Tests check deterministic calculations and boundaries. Evaluations check interpretation, tool choice, and answer quality. Feedback reveals confusing explanations and unsupported questions from actual users.
The reference paid model-evaluation workflow is manually dispatched and uses database stand-ins. It is not an every-build gate, a production benchmark, or a full end-to-end test. Feedback does not automatically fine-tune the model.
A practical proposed evaluation case records the expected query interpretation and result alongside observed tokens, steps, and credits. Compare equivalent cases before and after a prompt or vocabulary change. A cheaper answer that drops an exclusion is a regression, not an optimization.
The semantic description hash is useful for attributing wording changes, but it does not cover every calculation-function edit. Track implementation revisions as well when comparing runs.
Roll Out a Budget You Can Explain
Begin with explicit credit units and measured usage, then wire plan budgets, tenant overrides, user notices, and owner notifications. Exercise warning and stop states with synthetic ledgers. Check failed and cancelled turns, cache-token accounting, PDF charges, and month boundaries.
Review observed usage before presenting planning assumptions as typical costs. Then decide whether atomic reservations and calibrated evaluation gates are needed for your product's guarantees.
Part 4 examines AI agent security: what these tools are allowed to read, how untrusted content can influence the model, and what must change before adding actions.
About This Series
These tutorials use a reviewed implementation snapshot (ae7c6e6fca71) as a concrete reference, with synthetic examples and generalized identifiers. They do not describe a verified live deployment. Code marked illustrative explains a pattern rather than providing a complete application. Proposed improvements are labeled explicitly.