When a request you have served before comes in again, Cache replays the stored answer and skips the model entirely, so the repeat is near-instant and costs almost nothing. Exact matching is automatic, semantic matching catches paraphrases, and a hit short-circuits the whole pipeline.
exact · semantic · instant
The same questions arrive again and again, often word for word, and each one runs the full pipeline and bills for a fresh model call. You are paying full latency and full cost to recompute answers you already produced minutes ago. A do-it-yourself cache then has to decide what is safe to store and when it goes stale.
Turn it on and caching is automatic. The cache keys on the request as you sent it, before any other service runs.
Only reproducible requests are cached, those with temperature 0 or a fixed seed. Volatile requests pass through untouched, unless you opt in to caching them.
Your cache is scoped to your organization, never shared across tenants, and bounded by a TTL you set from 1 hour to 30 days. Clear it whenever you want.
A hit records a small cache charge and skips the inference charge. That skipped charge is your saving, and the answer comes back in milliseconds.
A blunt cache returns stale or wrong answers and erodes trust. This one is careful about what it stores, how long it keeps it, and what a hit skips.
Create a key, turn on Cache, and serve your repeat traffic from a stored answer in milliseconds.
We use privacy-friendly, EU-hosted analytics to understand how Akumi is used. Accept analytics cookies, or continue without them. See our cookie policy.
