Sovereign AI at Home: Self-Hosted AI Models for Business
Aleph Alpha's Kolibri-1 shows how a self-hosted AI model lets regulated businesses run frontier AI inside their own infrastructure, with no data ever leaving the building.
The Short Answer
Every serious conversation about AI in regulated business eventually reaches one quiet question: where exactly does the data go? For a hospital group, a law firm, or a bank, the answer decides whether an AI project gets approved at all. That is why a self-hosted AI model has become one of the most closely watched…
Every serious conversation about AI in regulated business eventually reaches one quiet question: where exactly does the data go? For a hospital group, a law firm, or a bank, the answer decides whether an AI project gets approved at all. That is why a self-hosted AI model has become one of the most closely watched ideas in enterprise AI this year. On 3 October 2026, the German AI company Aleph Alpha gave this movement its flagship example with Kolibri-1, an open-weight model built to run entirely inside an organisation's own infrastructure. No prompt leaves the building, no document is shared with a third-party API, and nobody outside the company decides when the model changes or switches off.
This is not a fringe concern anymore. It is part of the same wave we explored in our piece on self-hosted AI inside the office suite: software vendors are moving intelligence back onto the customer's premises. The logic is simple, keep your data at home. Kolibri extends that logic to a frontier-class language model, at a moment when regulators on several continents are tightening the rules on where AI processing can legally happen.
What a self-hosted AI model actually is
A self-hosted AI model is a language model whose weights are downloaded and run on hardware the organisation owns or rents, rather than accessed through a provider's cloud API. With an API model, every prompt and every uploaded document travels over the internet to servers run by a third party, processed under that provider's terms and infrastructure. With a self-hosted deployment, the model lives behind the company's own firewall. Prompts never leave the network. Version changes happen only when the company decides. Logging, access control, and audit trails are the company's own.
"Sovereign AI" is the word Aleph Alpha and others use for this arrangement. It has two layers. The first is operational sovereignty: you control and operate the model yourself, with no dependency on an outside provider's servers. The second is legal sovereignty: Kolibri was trained on infrastructure in Germany and Finland under European and German law, with the EU AI Act and GDPR considered from the start, and Aleph Alpha has signed the EU's General-Purpose AI Code of Practice. That gives a European data protection officer something concrete to read instead of a foreign cloud provider's terms of service.
Kolibri-1 under the hood: the numbers that matter
Kolibri-1 is a mixture-of-experts (MoE) transformer, and the concept is what makes a self-hosted frontier model practical. An MoE model is one large model split into many specialist parts called experts, with a small router selecting a few of them for each token. Kolibri has 50 layers, each with 384 experts plus one shared expert that is always used, and the router sends each token to 6 of the 384. That gives 78.1 billion total parameters but only about 3.46 billion active per token, roughly 4.4 percent. Each token costs about what a 3.5-billion-parameter model costs in compute.
There is one catch that business decision-makers should understand. The router can pick any expert for the next token, so all 78 billion parameters must stay in GPU memory. MoE saves compute, not memory. The weights ship in FP8, about 78 GB, and the minimum hardware listed is 2x A100 80 GB, 2x H100 SXM5, or a single H200, B200, or B300 class GPU. That is data-centre hardware, not a desktop machine. Businesses without it would rent GPU capacity, ideally from a host in their own jurisdiction.
The training story is unusually transparent. Kolibri was trained on 20 trillion tokens, about 62.5 percent English, 23.9 percent German, and 13.6 percent code, across 768 NVIDIA B200 GPUs over 21 days, roughly 392,000 GPU-hours, in Germany and Finland. It handles 262,144 tokens of context natively, validated up to one million, and ships with four reasoning effort levels, none, low, medium, or high, plus tool calling. Its knowledge cutoff is 18 June 2026. Aleph Alpha's own evaluations place it on a strong quality-versus-serving-cost curve against other models in its active-parameter class.
Why compliance leaders are demanding a self-hosted AI model
The demand for a self-hosted AI model is driven by compliance officers, data protection officers, and in-house counsel who cannot sign off on the alternative. In regulated industries the bar is not just having controls, it is being able to evidence them. Data residency and provider contracts are gating requirements, not preferences, because sensitive categories like health records and personal data cannot reach a provider without a compliant contractual basis. And when an examiner asks how a specific AI output was produced, audit trails must be complete enough to reconstruct it, not merely to show that logging exists.
The regulatory pressure is global and rising. The EU AI Act is now in force, with high-risk system requirements phasing in through 2026, imposing obligations around transparency, human oversight, and data governance across finance, healthcare, and legal applications. GDPR enforcement targeting AI data pipelines has accelerated, with authorities in Germany, Italy, France, and Ireland issuing guidance or enforcement actions. Beyond Europe, India's Digital Personal Data Protection Act introduces data localisation, several Gulf states require data residency in their AI frameworks, and US state-level privacy laws keep accumulating. Governments in the EU, UK, Australia, Canada, and the US are also inserting sovereignty and auditability requirements into procurement frameworks, so for firms selling to the public sector, compliant AI infrastructure is becoming contractual.
This is the context that makes Kolibri's design choices look less like marketing and more like product strategy. A model that never sends data off-premise turns many of these compliance conversations into a simpler statement of architecture: the data never leaves.
The Merlin-Arthur protocol: why "I don't know" is a business feature
One of Kolibri's most business-relevant features is not a benchmark score at all. It is trained with Aleph Alpha's Merlin-Arthur abstention protocol, which teaches the model to say "I don't know" when the answer is not present in the supplied context. The target use case is document search over company files, a retrieval-augmented generation pattern where the model answers from internal documents.
Here is the original insight: in a regulated business, the failure mode of an AI assistant matters more than its accuracy score. A model that silently invents a wrong clause in a contract review, or fabricates a precedent in legal research, creates a liability event that a compliance team cannot easily detect. A model that visibly abstains, saying the supplied documents contain no evidence for an answer, turns a failure into an auditable, manageable outcome. A human reviews the flagged gap; nothing incorrect goes into the record. For a law firm running client matter search, or a finance team querying internal policies, abstention is not a weakness. It is the feature that makes the rest of the system deployable. As we argued in our analysis of data privacy and ownership, control over AI systems is what increasingly separates serious deployments from risky experiments.
Self-hosted vs API: the tradeoffs nobody should skip
Self-hosting is not a free lunch, and an honest evaluation compares it directly with the API route.
| Factor | Self-hosted (Kolibri-style) | Cloud API model |
|---|---|---|
| Data location | Never leaves your network or your chosen infrastructure | Every prompt travels to the provider's servers |
| Model control | You choose when to update; nothing changes under you | Provider can update, deprecate, or change pricing at any time |
| Audit and logging | Your own logs, your own retention policy | Visibility limited to what the provider exposes |
| Hardware | Data-centre GPUs required, or GPU rental in your jurisdiction | None, the provider absorbs it |
| Operations burden | Your team runs serving, monitoring, updates, and security patching | Provider handles infrastructure |
| Language coverage | Kolibri: English and German only | Usually dozens of languages |
| Regulatory posture | Strongest story for residency and examiner scrutiny | Depends on contracts, regions, and provider certifications |
The honest summary, echoing the model card's own FAQ: run a self-hosted deployment if your data cannot leave your own infrastructure, your users work in the languages the model supports, and you have, or can rent, the right class of GPU. Skip it for now if you are a small team without data-centre access, need languages beyond English and German, or mainly want an autonomous coding agent, since independent benchmarks show Kolibri trailing some rivals on agent-style coding tasks.
For most businesses, the answer may be hybrid, and the industry is clearly moving that way. IBM's recent self-hosted deployment of its agentic AI platform lets corporations run AI on internal infrastructure while connecting to external services only where they choose, describing the shift as AI moving to where the data resides. The broader lesson: sensitive workloads stay in-house, less sensitive work can use the cloud. Our earlier coverage of in-country AI processing maps exactly this decision framework.
A practical path for businesses considering this route
Adopting a self-hosted model is a staged project. A sensible sequence looks like this:
- Classify your workloads first. Identify which AI use cases touch personal data, client-confidential material, or regulated records. Those are your self-hosting candidates. Generic drafting and public-information research can stay on cloud APIs.
- Start with document search. Retrieval-augmented generation over company files is the lowest-risk, highest-value pilot for a model like Kolibri, and it is exactly the scenario the Merlin-Arthur abstention protocol was designed for.
- Choose your hosting model. Either provision GPU capacity in your own data centre or rent dedicated GPU infrastructure from a provider in your jurisdiction, so residency claims remain simple and defensible.
- Build the evaluation loop before production. Test the model on your own documents and your own question types, measure how often it abstains versus answers, and tune the retrieval layer so the context it sees is the right one.
- Bring compliance in early, not late. Your data protection officer and legal team should review the deployment plan, the logging design, and the license terms. Apache 2.0 covers Kolibri's weights and configuration files for commercial use, but Aleph Alpha retains rights to its training code and methods, so understand what open-weight does and does not include.
One honest caveat for any compliance conversation: the model card notes that some synthetic training data was rephrased using non-European models. "Sovereign" describes who controls the model and where it runs, not a promise that every byte of its training inputs was European. A credible compliance story owns this distinction rather than hiding it.
Models like Kolibri also pair naturally with the always-on agent patterns businesses are adopting. If your organisation is weighing an AI and machine learning strategy that includes autonomous assistants, the same sovereignty question applies to them: where do the agent's tools and memory live, and who can see them? The always-on AI agent trend only increases the stakes, because agents touch more systems and more data than a chat box ever did.
Frequently asked questions
What is Kolibri-1?
Kolibri-1 is an open-weight English and German language model from Aleph Alpha, released on 3 October 2026. It is a mixture-of-experts model with 78.1 billion total parameters and about 3.46 billion active per token, designed to run on hardware the customer controls. Its weights are on Hugging Face under Apache 2.0.
What does "sovereign AI" mean in practice?
It means the organisation controls and operates the AI itself, with no dependency on an outside provider's servers. In Kolibri's case it also means the model was built in Germany under European and German law with the EU AI Act and GDPR in mind. Nobody can change the model under you or switch it off.
What hardware does a self-hosted model like Kolibri need?
The FP8 weights take about 78 GB of GPU memory. The minimum listed is 2x A100 80 GB, 2x H100 SXM5, or a single H200, B200, or B300 class GPU. This is data-centre hardware, so most businesses without it would rent GPU capacity from a host in their own jurisdiction.
Is Kolibri open source?
It is open-weight, not fully open source. The weights and configuration files are under Apache 2.0 for commercial use, but Aleph Alpha retains the rights to its training code, architecture, and training methods.
Which businesses benefit most from self-hosted AI?
Hospitals, law firms, financial institutions, public sector bodies, and any company whose clients or contracts restrict data from reaching third parties. The deciding factor is whether your compliance requirements make API data transfer a blocker rather than a convenience.
Can self-hosted and cloud AI be used together?
Yes, and a hybrid model is often the pragmatic choice. Sensitive workloads, client data, and internal knowledge bases run on the self-hosted model, while general-purpose drafting stays on cloud APIs. The routing decision becomes part of the compliance architecture.
Kolibri-1 is the clearest signal yet that the "keep your data at home" wave has reached frontier language models. It is not the right answer for every business, but for regulated organisations where data residency rules and client confidentiality are tightening, it offers something cloud APIs structurally cannot: AI that never sends your data anywhere. If you are evaluating a self-hosted AI model strategy, from infrastructure planning to compliance review and deployment, talk to our AI team and we will help you design a sovereign AI deployment your compliance officer can approve.
Related Articles
Keep reading — more guides you may find useful.
Related Services
Services connected to this guide.
