Running Mistral or Llama on your own GPUs sounds like the cleanest residency answer, right up until you price it. It is a serving stack, a GPU budget that bills whether or not anyone sends a request, an on-call rotation, and a governance layer you still have to write yourself. Akumi gives you the EU-sovereign answer without the fleet, and residency you can prove per request rather than per data centre.
Self-hosting an open-weights model is the answer that sounds cleanest in a meeting. Mistral or Llama on your own hardware, nothing leaving your perimeter, no vendor in the middle. As an engineering position it is entirely coherent.
What it actually buys you is a model, not a platform. You still need somewhere to put your documents and something to retrieve them without leaking one team into another team's answers. You still need to mask personal data, catch prompt injection, and produce a record a regulator would accept. And underneath all of it sits a GPU bill that arrives whether or not anyone sent a request that month, plus the person who upgrades the serving stack the next time a better model lands. Akumi gives you the EU-sovereign answer without the fleet, and the governance layer already built.
A managed EU-sovereign AI platform behind one OpenAI-compatible endpoint. EU-resident models, a PII firewall, a guard, a governed knowledge graph for retrieval and memory, a response cache and per-request observability, all metered per use. Nothing to provision, patch or scale, and residency recorded per request in a metadata-only audit trail.
You run open-weights models on hardware you control, usually with an inference server such as vLLM in front of them. You choose the models, the quantization and the serving topology, and you own the capacity planning, the upgrades, the uptime and everything above the completion: retrieval, isolation, redaction, caching and audit.
| Dimension | Akumi | Self-hosted |
|---|---|---|
| Residency evidence | EU-resident by default, and provable per request: every call records the model, provider and region that served it. | Absolute by construction, with the audit trail and the evidence for your regulator left for you to build. |
| Operational burden | None. Serving, scaling, upgrades and capacity are ours. | Yours: GPU capacity, inference servers, model upgrades, and someone carrying the pager. |
| Cost shape | Usage-based credits. Nothing runs, nothing costs. | Fixed GPU spend whether or not anyone sends a request, plus the engineers who keep it up. |
| Governance layer | PII firewall, guard, egress policy, audit trail and retrieval already built and metered. | Every piece is yours to build, test, and keep correct as models and regulations move. |
| Governed retrieval | Conversation memory and your documents share one knowledge graph, so an answer follows the link from a user to their organization to the document that answers them. Each collection is its own graph partition, namespaced to your organization and workspace before the query is built, and every response carries the sources it used. | Serving open weights gets you a model, not a knowledge layer. The store, the ingestion, the ranking and the isolation between teams are all still ahead of you. |
| Keeping up with models | A curated roster behind one endpoint. New models arrive tested and priced, and swapping is a string. | Every upgrade is a project: new weights, new serving config, new benchmarks, and a rollback plan. |
| Time to production | Minutes. Signup, key, one base_url. | Weeks to months, depending on procurement, hardware lead times, and how much of the stack exists. |
Usage-based pricing and reserved capacity behave very differently, and the difference is not a rounding error. A self-hosted deployment sized for your busiest hour is paid for during every quiet one: overnight, at weekends, and across the months where the feature is live but adoption is still building. Size it for the average instead and the busy hour is where your users meet a queue.
Akumi meters per token and per request, so an idle week costs nothing and a launch week scales without a procurement conversation. Governance modules meter separately when they run, and a cache hit removes the inference charge entirely, so the bill tracks the work rather than the capacity you reserved in case of it.
| Dimension | Akumi | Self-hosted |
|---|---|---|
| Quiet periods | Nothing. You pay for requests, not for reserved capacity. | Full cost, whatever the utilisation. |
| A sudden spike | Absorbed. Scaling is ours to worry about. | A queue, or capacity you bought in advance and paid for all year. |
| What the bill tracks | Work done, itemised per request and per module. | Hardware reserved, plus the people keeping it healthy. |
Getting a model to return tokens on your own hardware is a solved problem, and a good engineer will have it running in a week. The remaining two thirds are the part that takes the quarter: a document store and an ingestion path, retrieval that ranks well enough to be useful, isolation so one customer's documents can never surface in another customer's answer, personal-data masking before egress, injection scanning over anything a tool returns, a cache, and an audit trail specific enough that someone can answer where a given request ran.
None of that is exotic, and all of it is load-bearing. It is also the part that keeps needing attention, because the attacks change, the models change and the obligations change. On Akumi it is already there, already metered, and already reviewed as the tenant-boundary code it is.
| Dimension | Akumi | Self-hosted |
|---|---|---|
| Getting tokens back | Included. | A week of work, and the genuinely enjoyable part. |
| Retrieval and isolation | A governed knowledge graph, partitioned per workspace before the query is built. | A store, an ingestion path, ranking, and the isolation logic between tenants. |
| Redaction and injection | PII firewall and guard on the same request, on by default. | Yours to write, and yours to keep current as attacks evolve. |
| Audit and evidence | Metadata-only record of model, provider and region, per request. | Yours to design, store, retain and defend. |
Model generations turn over fast, and on self-hosted infrastructure each one is a project rather than a config change: new weights to evaluate, serving parameters to retune, throughput and memory to re-measure, quality to re-benchmark against your own workload, and a rollback plan for when it is worse in a way the benchmarks missed. Teams that fall behind on this quietly end up running a model a year older than the one their competitor is using.
Behind a managed endpoint that turnover is a string. New models arrive tested and priced, the router handles fallback, and your evaluation is a comparison you run rather than an upgrade you schedule.
| Dimension | Akumi | Self-hosted |
|---|---|---|
| Adopting a newer model | Change a string, run your evals, keep the old one as fallback. | Weights, serving config, retuning, re-benchmarking and a rollback plan. |
| Falling behind | Hard to, because switching costs almost nothing. | Common, because each upgrade competes with product work. |
| Running two side by side | Point some traffic at each and compare from the same trace data. | Twice the capacity, or a serving topology that can host both. |
It depends almost entirely on utilisation. Self-hosted capacity is a fixed cost that arrives whether or not anyone sends a request, so it rewards large, steady, predictable volume. Usage-based pricing rewards variable or growing workloads, where you would otherwise be paying for idle GPUs. The comparison people forget is the engineering time: the serving stack, retrieval, isolation, redaction and audit all have to be built and then maintained.
Yes. The application, the data and the models are EU-resident, and non-EU routing is blocked at a guard that fails closed unless you explicitly allow it. Every request records the model, provider and region that served it, so residency is provable per request rather than asserted once.
Yes. The EU-sovereign roster is centred on them, served behind the same OpenAI-compatible endpoint as everything else, so you get open models without operating the serving stack that runs them.
The API is OpenAI-compatible, so moving in either direction is a base_url change rather than a rewrite. Starting on Akumi while a self-hosting project is still a roadmap item is a reasonable way to hit a compliance deadline that arrives before your hardware does.
No, and neither does Akumi. Retrieval runs on a governed knowledge graph rather than a vector store: conversation memory and your documents share one substrate, so an answer can follow the relationships between them instead of hoping a single chunk contains the whole chain.
On Akumi, we are. New models arrive tested and priced behind the same endpoint, and switching is a string change with the previous model available as fallback. Self-hosted, each upgrade is yours to evaluate, retune, benchmark and roll back.
Base inference is metered per token and request, and each governance module meters on its own line when it runs, so the PII firewall, retrieval, guard and cache are all visible separately. A cache hit skips the inference charge entirely.
If self-hosting is on the roadmap but the compliance deadline is sooner, start on Akumi now. The API is OpenAI-compatible either way, so moving later is a base_url.
We use privacy-friendly, EU-hosted analytics to understand how Akumi is used. Accept analytics cookies, or continue without them. See our cookie policy.
