Self-Hosted AI Governance: Why Air-Gapped Deployment Matters

Disclosure: I built systemprompt.io. One of its core design decisions was self-hosted deployment. I will explain why that decision was made, where the tradeoffs are, and where SaaS governance genuinely makes more sense. I have tried to be fair about the operational burden self-hosted requires.

The Deployment Problem

Here is an uncomfortable question: where does your AI governance data live?

Not your AI model data. Not your application data. Your governance data specifically. The logs that record which users accessed which AI agents, which tools those agents called, what parameters were passed, what policies were evaluated, and what the outcomes were.

For most enterprises in 2026, the answer is "in someone else's cloud." Because most AI governance platforms are SaaS. You configure policies through a web dashboard, your agents report activity to the vendor's API, and your governance data sits in the vendor's infrastructure.

Think about what that data contains. It is a complete record of how your organisation uses AI. Which departments use AI agents. What data those agents access. Which tools they call. What parameters they send. How much they spend. What gets blocked and what gets through. It is, in effect, a detailed map of your organisation's AI operations.

For a software company shipping a consumer product, this tradeoff might be acceptable. The vendor manages the infrastructure, handles updates, provides support, and the governance data is not particularly sensitive.

For a bank, a hospital, a defence contractor, or a government agency, this is a non-starter. The governance data itself is sensitive. It reveals operational patterns, data access patterns, and security boundaries. Sending it to a third-party cloud is not a technical concern. It is a compliance violation.

This is the deployment problem. The tools that are supposed to help you govern AI responsibly are asking you to send your governance data — the most operationally sensitive data about your AI usage — to an infrastructure you do not control.

Who Needs Self-Hosted

Not everyone. Let me be clear about that. Self-hosted governance adds operational complexity. If you do not need it, do not take on that burden. Here is who genuinely needs it.

Financial Services

Banking regulators in most jurisdictions require that customer data and operational data remain within defined boundaries. The UK's FCA, the EU's DORA regulation, and US federal banking regulators all have data residency requirements that apply to systems processing customer information.

AI governance logs that record "agent X queried customer account Y with parameters Z" contain customer data by reference, even if the actual customer record never leaves your database. The governance log reveals that the query happened, which customer was queried, and what the purpose was. Regulators consider this operational data, and it is subject to the same residency requirements as the underlying customer data.

Financial institutions also face audit requirements that demand complete chain-of-custody for operational data. If your auditor asks "can you prove that no one outside your organisation has accessed the governance logs?" and the logs are in a vendor's cloud, the honest answer is no. You trust the vendor, but you cannot prove it.

Healthcare

HIPAA's Security Rule requires that covered entities maintain physical and technical safeguards for systems that process electronic protected health information (ePHI). An AI agent that can access patient records, clinical notes, or lab results through tool calls generates governance logs that reference that ePHI.

The governance log itself might not contain the patient record, but it contains the query that accessed it, the user who initiated it, and the result summary. Under HIPAA's broad definition of ePHI, this is protected information. It needs the same safeguards as the patient record itself.

Self-hosted governance keeps these logs within your existing HIPAA-compliant infrastructure. The same physical controls, access controls, and audit mechanisms that protect your clinical systems protect your AI governance data.

Government and Defence

Government agencies and defence contractors operate under classification regimes that prohibit certain categories of data from leaving controlled environments. Even unclassified operational data about AI usage patterns may be considered sensitive in a defence context.

The UK's Official Sensitive classification, the US government's Controlled Unclassified Information (CUI) designation, and NATO's classification system all impose handling requirements that are incompatible with storing data in a commercial SaaS platform. Air-gapped deployment is not a preference. It is a requirement.

Data Sovereignty

The EU's GDPR, Brazil's LGPD, and India's DPDP Act all impose data residency requirements for personal data. AI governance logs that contain user identifiers, user actions, and user behavioural patterns are personal data under these regulations.

If your organisation operates in the EU and your AI governance vendor stores data in the US, you have a data transfer problem. Schrems II invalidated the Privacy Shield. Standard Contractual Clauses require a Transfer Impact Assessment. Self-hosted governance on EU infrastructure eliminates the problem entirely. The data never leaves the jurisdiction.

Deployment Model Comparison

Not all self-hosted options are equal. Here is how the current AI governance landscape breaks down by deployment model.

A few clarifications on this table.

Claude Enterprise with Amazon Bedrock hosts the AI model in your AWS account, which addresses data residency for inference. But the governance features (usage controls, permission management, audit logs) remain in Anthropic's cloud. You get data sovereignty for model interactions but not for governance data. This is an important distinction that is easy to miss in the marketing.

Microsoft's Agent Governance Toolkit is genuinely self-hosted and open-source under the MIT licence. You can deploy it entirely within your infrastructure. The "with effort" qualifier on air-gap support reflects the reality that AGT's full deployment involves multiple services (policy engine, identity provider, sandboxing runtime, event store) running on Kubernetes. Getting all of that running in an air-gapped environment is achievable but requires significant platform engineering effort. You are building a governance platform from components, not deploying a finished product.

IBM watsonx.governance can run on IBM Cloud Pak for Data, which supports private cloud deployment. Whether it truly supports air-gapped operation depends on the specific Cloud Pak configuration and IBM's licensing requirements. Consult IBM directly for air-gapped deployments.

What Air-Gapped Means in Practice

The term "air-gapped" gets thrown around loosely. Some vendors use it to mean "deployed in your VPC." That is not air-gapped. Air-gapped means zero outbound network connections. No telemetry. No update checks. No cloud API calls. No licensing server heartbeats. No DNS lookups to external domains. The system operates entirely within a network that has no route to the public internet.

For a software system to run air-gapped, it must satisfy several requirements that most modern software does not.

No Phone Home

No telemetry collection. No usage reporting. No crash reporting to external services. No anonymous analytics. Many SaaS platforms that claim "optional telemetry" still require periodic connection to a licensing server. In an air-gapped environment, periodic means never. The software must function indefinitely without any outbound connection.

Self-Contained Authentication

The system must issue and validate authentication tokens without relying on an external identity provider. In a connected environment, you might delegate authentication to Auth0, Okta, or Azure AD. In an air-gapped environment, the authentication system runs locally. This means the governance platform needs its own user management, its own token issuance, and its own token validation — all without calling external services.

Alternatively, the platform must integrate with whatever identity provider runs inside the air gap. Many government and defence networks run Active Directory or a SAML-based IdP internally. The governance platform needs to work with what is available inside the boundary.

Local Updates

Software needs updates. Security patches. Bug fixes. Feature additions. In a connected environment, you pull updates from a registry or repository. In an air-gapped environment, updates arrive on physical media or through a controlled transfer process. The software must support this update path — typically by accepting a new binary that replaces the existing one without requiring any external download or dependency resolution.

No External Dependencies at Runtime

The software cannot depend on external package registries, CDN-hosted assets, or cloud-based services at runtime. Every dependency must be bundled in the deployment artifact or available within the air-gapped network. This eliminates most solutions that use Node.js (npm install at startup), Python (pip install), or containerised deployments that pull images from external registries.

The systemprompt.io Approach

systemprompt.io compiles to a single statically-linked Rust binary. Approximately 50MB. No runtime dependencies beyond PostgreSQL. No external library loading. No package manager. No container registry. Copy the binary to a Linux server, point it at a PostgreSQL database, and run it.

The binary is its own web server, its own API server, its own authentication system, and its own job runner. It issues and validates JWT tokens locally. It serves its own admin dashboard. It runs database migrations on startup. There is nothing to install, nothing to configure beyond the database connection, and nothing that reaches outside the network boundary.

This is not an accident of implementation. It is a design decision made specifically to support air-gapped deployment. Rust's compilation model produces self-contained binaries. Static linking eliminates dynamic library dependencies. Embedding all assets in the binary eliminates CDN requirements. Building authentication into the platform eliminates external IdP requirements.

The result is a deployment model that works identically on a developer's laptop, a cloud VM, a bare-metal server in a data centre, and a classified workstation on an isolated network.

Infrastructure Requirements

For organisations evaluating self-hosted AI governance, the infrastructure question matters. What do you actually need to run this?

The Minimum

A Linux server with 2GB of RAM. A PostgreSQL 14+ database. The binary. That is the entire infrastructure requirement for systemprompt.io.

The binary consumes approximately 80MB of RAM at idle and scales with concurrent connections. A server handling 50 concurrent users typically uses 200-400MB of RAM. CPU usage is minimal for governance operations — policy evaluation is a string matching and rule evaluation, not compute-intensive work.

PostgreSQL stores all governance data: audit events, policy configurations, user accounts, agent definitions, and skill content. A standard PostgreSQL deployment with daily backups satisfies the data persistence requirement.

Docker (Optional)

If your organisation standardises on containers, systemprompt.io runs as a Docker container. The image is the binary plus a minimal base layer. No runtime installation steps. No entrypoint scripts that download dependencies. The container starts in milliseconds.

services:
  systemprompt:
    image: systemprompt/systemprompt:latest
    environment:
      DATABASE_URL: postgres://user:pass@db:5432/systemprompt
    ports:
      - "8080:8080"
  db:
    image: postgres:16
    volumes:
      - pgdata:/var/lib/postgresql/data

What You Do Not Need

You do not need Kubernetes. You do not need Redis. You do not need a message queue. You do not need a reverse proxy (the binary serves HTTPS directly if configured with TLS certificates). You do not need a separate job runner, a separate worker process, or a separate admin service. You do not need Node.js, Python, Java, or any runtime beyond the operating system kernel.

This is a deliberate design choice. Every additional dependency is a potential failure point, a security surface, and an operational burden. For air-gapped deployments specifically, every dependency is something that needs to be available inside the boundary. The fewer dependencies, the simpler the deployment.

Compare to Microsoft AGT

Microsoft's Agent Governance Toolkit, for comparison, requires Kubernetes for production deployment. The full deployment includes separate services for the policy engine, identity management, sandboxing, event storage, and the admin interface. Each service has its own configuration, its own scaling requirements, and its own failure modes.

This is not a criticism of AGT. It is an architectural difference that reflects different design priorities. AGT prioritises modularity and extensibility — you can replace any component with your own implementation. systemprompt.io prioritises operational simplicity — everything runs as one process with one configuration.

For organisations with mature platform engineering teams that already run Kubernetes in production, AGT's deployment model is familiar. For organisations without Kubernetes expertise, or those operating in constrained environments where Kubernetes is not available, the single-binary model is significantly simpler.

Compliance Implications

Self-hosted deployment is not just an operational preference. For several compliance frameworks, it is the architectural answer to specific regulatory requirements.

SOC 2: Data Residency

SOC 2's Trust Services Criteria include provisions for data processing location. If your SOC 2 scope includes AI agent operations (and it should if agents access customer data), the auditor will ask where governance data is processed and stored. Self-hosted governance gives a straightforward answer: the same infrastructure as everything else, under the same controls, in the same compliance boundary.

ISO 27001: Information Security Controls

ISO 27001 requires organisations to identify and control information assets. AI governance data — audit logs, policy configurations, user activity records — is an information asset. Annex A control A.5.10 (acceptable use of information) requires that information assets are handled according to the organisation's classification scheme.

If your classification scheme says operational security data stays on-premise, self-hosted governance satisfies that requirement by design. If governance data is in a vendor's cloud, you need to demonstrate that the vendor's controls meet your classification requirements — which is possible but adds complexity to every audit cycle.

HIPAA: Technical Safeguards

HIPAA's technical safeguard requirements (45 CFR 164.312) include access controls, audit controls, integrity controls, and transmission security for systems processing ePHI. Self-hosted governance means these safeguards are implemented by your organisation, under your control, auditable by your compliance team.

When governance data is in a vendor's cloud, the vendor's Business Associate Agreement (BAA) covers their obligations. But the covered entity (you) retains ultimate responsibility. Self-hosted governance eliminates the need to rely on a vendor's BAA for your governance data.

FedRAMP and Government Frameworks

For organisations pursuing or maintaining FedRAMP authorisation, the Authorisation Boundary is sacred. Every system within the boundary must meet the baseline's security controls. Adding a SaaS governance platform means either bringing it inside the boundary (which requires the vendor to have FedRAMP authorisation) or keeping it outside (which means governance data crosses the boundary).

Self-hosted governance runs inside the boundary. It is subject to the same controls as everything else. No boundary crossing. No vendor authorisation dependency. No data flow diagrams with arrows pointing outside your infrastructure.

The Tradeoff

I would be dishonest if I did not address this directly. Self-hosted governance means you take on operational responsibility that SaaS platforms handle for you.

You Manage Updates

When a security patch is released, you need to apply it. On a SaaS platform, the vendor patches and you benefit automatically. On self-hosted, you download the new binary, test it in staging, and deploy to production. systemprompt.io simplifies this — replacing the binary and restarting the process is the entire update procedure — but the responsibility is yours.

For air-gapped environments, the update process is slower. The new binary needs to be transferred through whatever controlled media process your security boundary requires. This might take days rather than minutes.

You Manage PostgreSQL

PostgreSQL needs backups. It needs vacuuming. It needs monitoring. It occasionally needs version upgrades. If your organisation already runs PostgreSQL for other applications, this is not a new burden. If you are introducing PostgreSQL solely for AI governance, it is additional operational surface.

PostgreSQL is mature, well-documented, and widely understood. This is not a risky dependency. But it is a dependency that requires attention.

You Manage Availability

If the governance platform goes down on a SaaS product, the vendor's SRE team responds. On self-hosted, your team responds. For organisations that already run production infrastructure, this is standard operating procedure. For organisations that have moved most workloads to SaaS, this is a capability they may need to rebuild.

You Manage Security Patching

Self-hosted means you are responsible for the security posture of the governance platform. When a CVE is published that affects a dependency, you need to apply the patch. For connected deployments, this is straightforward — download the updated binary, test it, deploy it. For air-gapped deployments, the patch needs to traverse your controlled media transfer process, which may involve multiple approval steps and physical media.

The flip side is that you control the patching timeline. On a SaaS platform, the vendor patches when they decide to patch. If a critical vulnerability is disclosed and the vendor takes 72 hours to deploy a fix, you wait. On self-hosted, you can apply the patch as soon as it is available, on your schedule, with your testing process.

When SaaS Makes More Sense

If you are a startup or mid-sized company without data residency requirements, without regulatory mandates for self-hosted infrastructure, and without an air-gapped network, SaaS governance is simpler and probably the right choice. The operational burden of self-hosted deployment is only justified when the compliance or sovereignty requirements demand it.

There is no shame in choosing SaaS governance. The goal is to govern AI agents effectively, not to run infrastructure for its own sake. Choose the deployment model that your organisation can operate reliably.

The Decision Framework

Ask yourself three questions. First: does any regulation or policy require your governance data to remain within a specific boundary? If yes, self-hosted. Second: do you operate air-gapped or classified networks where the governance platform must run? If yes, self-hosted. Third: does your organisation have the operational capacity to manage a Linux server and a PostgreSQL database? If no, start with SaaS and migrate to self-hosted when you build that capacity.

Most organisations that end up self-hosting do not start there. They start with a SaaS platform, discover the data sovereignty implications during their first compliance audit, and migrate. Starting with self-hosted governance avoids that migration, but only if you have the operational maturity to support it from day one.

How systemprompt.io Addresses This

systemprompt.io was designed from the first line of code for self-hosted deployment. Not as an afterthought. Not as a "private cloud option." As the primary deployment model.

Single binary. ~50MB, statically linked, zero runtime dependencies beyond PostgreSQL. Copy it to a server and run it. Millisecond startup. No installation procedure. No package manager. No container orchestrator. It runs on any Linux server from a Raspberry Pi to a cloud VM.

Zero outbound connections. The binary makes no outbound network connections. No telemetry. No update checks. No licensing calls. No CDN requests. Your firewall rules can block all egress from the governance server and it will continue to function indefinitely.

Built-in authentication. The platform issues and validates JWT tokens locally. No external IdP required (though integration with SAML and OIDC providers is supported for connected deployments). In an air-gapped environment, user management is local and self-contained.

Single-process architecture. No separate worker processes. No message queues. No background services to monitor. One process, one configuration file, one log stream. The operational model is as simple as the deployment model.

Upgrade path. Download the new binary. Replace the old one. Restart. The platform runs database migrations automatically on startup. There is no separate migration step, no version compatibility matrix, and no multi-step upgrade procedure.

For organisations evaluating self-hosted AI governance, the total cost of ownership calculation is straightforward. A Linux server, a PostgreSQL database, and the time to manage both. Compare that to the cost of demonstrating compliance when your governance data sits in a vendor's cloud, and for regulated industries, self-hosted wins on total cost even before factoring in the operational benefits.

See the self-hosted platform feature page for deployment documentation and the compliance feature page for framework-specific guidance.