AI Governance Framework: A Practical Build Guide

An AI governance framework is the structured system of principles, policies, controls, enforcement, and audit that an organisation uses to control how AI is built and used. The hard part is not naming the layers. The hard part is the connection between them: a framework only governs if every principle reaches down to a control that can block a violation in the request path, and every block leaves an audit record. Most published frameworks stop at the policy layer and leave that connection to you.

This guide gives you a complete, implementable AI governance framework in five layers, then maps it against the three reference standards a CISO or CTO will be asked about: the NIST AI Risk Management Framework, ISO/IEC 42001:2023, and the EU AI Act. It also separates two terms that get used interchangeably and should not be: AI ethics and AI governance. The reason the distinction matters is operational, not academic, and it shows up the first time a policy needs enforcing.

The five layers of an AI governance framework

Start with the answer. A working AI governance framework has five layers, and each layer exists to make the layer below it accountable to the layer above it.

1. Principles are the values the organisation will not violate. They are stable, few, and human-readable. "AI decisions affecting people must be contestable" is a principle.
2. Policies are the written rules that operationalise a principle. They have owners and scope. "No personal data may be sent to a third-party model without a documented legal basis" is a policy.
3. Controls are the technical and procedural mechanisms that implement a policy. A control is testable. "All outbound AI tool calls pass a secret and PII scan" is a control.
4. Enforcement is where a control actually blocks, gates, or rate-limits an action at runtime. Enforcement is the difference between a control that detects after the fact and one that prevents.
5. Audit is the evidence trail that proves the other four layers operated. Without audit, you have governance you cannot demonstrate, which in a regulated context is the same as none.

The single most common failure is a framework that is strong at layers 1 and 2 and absent at layers 4 and 5. Organisations publish an AI ethics charter and a policy handbook, then discover during an incident that nothing was enforcing the policies and nothing was logging the violations. The framework below is built from the bottom up so that does not happen.

The rest of this guide works through each layer with concrete artefacts, then shows how to map the whole structure to the standards an auditor or regulator will reference.

AI governance and ethics: why they are not the same thing

The phrases "AI governance and ethics" and "AI ethics and governance" get searched and written as if they name one discipline. Treating them as one is the root cause of frameworks that look complete on paper and fail in production.

AI ethics defines the values. The clearest primary statement is UNESCO's Recommendation on the Ethics of Artificial Intelligence, adopted in November 2021 by all member states as the first global standard on AI ethics. It holds that AI must respect human rights and human dignity, grounded in transparency, fairness, environmental sustainability, and human oversight. Those are commitments about what AI should and should not do.

AI governance is the operating system that makes those commitments real. It is the policies, controls, accountability structures, and risk management that turn a value into something an engineer can implement and an auditor can verify. The NIST AI RMF and ISO/IEC 42001 are governance instruments: they describe how to run the system, not what the values should be.

Here is the distinction stated as plainly as it can be: ethics answers "is this acceptable?" and governance answers "how do we enforce it and prove we did?" The two layers are sequential. An ethics principle that fairness must be preserved is worth nothing until a governance control measures disparate impact and an enforcement mechanism blocks a model from going to production when it fails that test. UNESCO frames its own work as moving "from what to how," pairing values with concrete policy recommendations. That is the same move this framework makes by binding every principle to a control.

The practical test: if your AI governance and ethics programme cannot point, for any given principle, to the line in the request path where a violation is stopped, you have an ethics statement, not a governance framework. Keep the two words distinct and the gap becomes visible and fixable.

Layer 1: Write principles you can actually enforce

The temptation at the principles layer is to write twenty aspirational statements. Resist it. Principles should be few enough to memorise and concrete enough that each one implies a policy.

The OECD AI Principles, adopted in 2019 and updated in 2024, are the most widely adopted values-based set and a good starting vocabulary. They define five principles for trustworthy AI: inclusive growth and well-being; respect for the rule of law, human rights and democratic values including fairness and privacy; transparency and explainability; robustness, security and safety; and accountability. Forty-seven adherent countries reference them, which makes them a defensible baseline when a regulator asks where your principles came from.

Translate the abstract set into principles that name a behaviour. Compare:

• Weak: "We are committed to responsible AI." Unenforceable. Implies no policy.
• Strong: "Every AI system that affects a person's access to a service must have a named human owner who can override it." Enforceable. Implies an ownership policy and an override control.

A good principle survives this question: what policy does it force into existence? If the answer is none, the principle is decoration. The output of Layer 1 is a one-page charter of five to eight principles, each of which you can trace forward to at least one policy in Layer 2.

Layer 2: Turn principles into policies with owners

A policy is a principle made specific enough to argue about. It names the systems in scope, the rule, the owner, and the exception process. The owner is the part most policies omit and the part that matters most: a policy without a single accountable owner is a policy nobody enforces.

ISO/IEC 42001:2023, published in December 2023 as the first AI management system standard, exists almost entirely to formalise this layer. It is built on the Plan-Do-Check-Act cycle and specifies requirements to establish, implement, maintain, and continually improve an AI management system. Its Annex A provides a catalogue of reference controls that organisations select from, omit, or extend based on their risk assessment. If you are pursuing certification, ISO/IEC 42001 is the standard auditors will measure your policy layer against. You can read the scope on the official ISO page.

A workable AI governance policy set covers at least these domains, each as a separate owned document:

• Data and privacy: what data may reach which models, legal basis, retention.
• Acceptable use: what AI may and may not be used for, by role.
• Model and tool approval: how a new model, MCP server, or agent gets allowlisted.
• Human oversight: which decisions require a human in the loop and who that human is.
• Incident response: what counts as an AI incident and the escalation path.
• Procurement and third parties: governance requirements for vendor AI.

Each policy ends with the same sentence pattern: "This policy is enforced by control X and audited by record Y." If you cannot complete that sentence, the policy is not ready. That sentence is the seam between the paper layers (1 and 2) and the operating layers (3, 4, and 5).

Layer 3 and 4: Controls and enforcement at the protocol layer

This is where most frameworks thin out, and where a CISO earns the framework's keep. A control is the mechanism that implements a policy. Enforcement is the control operating synchronously, in the request path, with the power to block.

The NIST AI RMF, released on 26 January 2023, gives the cleanest structure for this work. Its core has four functions:

• Govern cultivates a culture of risk management across the organisation and is cross-cutting (this is your Layers 1 and 2).
• Map establishes the context and identifies risks across the AI lifecycle.
• Measure uses quantitative and qualitative methods to analyse, benchmark, and monitor AI risk.
• Manage allocates resources to treat the mapped and measured risks, including blocking and recovery.

Map and Measure tell you what to control. Manage is where enforcement lives. NIST's Generative AI Profile (NIST-AI-600-1), published 26 July 2024, extends this with 12 generative-AI-specific risks and over 200 recommended actions, which is the most useful single inventory for deciding which controls a framework needs.

The architectural decision that separates a real framework from a binder of intentions is where enforcement sits. There are three options, and only one of them scales.

Per-application enforcement fails because it depends on every team implementing the same checks correctly, forever. Retroactive analysis fails because detection is not prevention: a model that exfiltrated a secret yesterday is not helped by a log entry today. The only enforcement point that covers every client, every model, and every tool with one set of rules is the transport layer, where AI requests actually travel. For agent-based systems this means the Model Context Protocol transport: govern at the protocol layer and every client inherits the policy.

A practical synchronous control set, ordered as a pipeline each AI tool call passes through before execution:

1. Scope check. Is the calling identity permitted this tool and this resource? This is role-based access control applied to AI actions, not just to humans.
2. Secret and credential scan. Does the request contain an API key, token, or password pattern? Block if so. This is the control that implements your data-protection policy.
3. Blocklist and allowlist evaluation. Is the target tool, MCP server, or destination approved? An unallowlisted tool is denied by default.
4. Rate limiting. Is this identity within its budget for calls, cost, or data volume? This is the control that bounds runaway agents.

Each of these is a control (Layer 3) that becomes enforcement (Layer 4) the moment it can return "denied" before the action runs. Bind each one back to the policy it serves. The secret scan serves the data policy. The allowlist serves the model-and-tool-approval policy. The scope check serves human-oversight and access policies. When every policy traces to an enforcing control and every control traces back to a policy, the framework is closed.

Governance as code, not as a document

The reason to express controls as code rather than prose is that the AI landscape does not hold still. New model providers, new agent architectures, and new tool protocols ship constantly. A governance framework written as a static document is out of date the quarter after it is signed. A framework expressed as policy-as-code, allowlists, and a versioned enforcement pipeline can be updated, reviewed in pull requests, and deployed like any other software. This is also what makes the framework auditable: a git history of policy changes is itself an evidence trail.

Layer 5: Audit, the evidence that the framework operated

The audit layer is where governance becomes provable. Every other layer is an assertion until the audit layer produces records that an internal auditor, an external assessor, or a regulator can inspect.

The design requirement is structured, machine-readable events, not free-text logs. Each governed action should emit an event with the identity, the tool, the policy evaluated, the decision, and a timestamp, written as structured JSON so it can be ingested directly by a SIEM such as Splunk, Elastic, or Datadog without custom parsing. The events worth capturing as a baseline: session start and end, every tool call, every prompt, configuration changes, permission grants and denials, and agent lifecycle events. Capturing permission denials matters as much as grants: a denied action is proof the enforcement layer worked.

This layer is also where the EU AI Act becomes concrete. The Act (Regulation (EU) 2024/1689) entered into force on 1 August 2024 and takes a risk-based approach with four tiers: unacceptable risk, high risk, transparency risk, and minimal-to-no risk. For high-risk systems it mandates record-keeping, logging, human oversight, and transparency. Its obligations phase in: prohibited practices applied from 2 February 2025, general-purpose AI model obligations from 2 August 2025, and most high-risk obligations become applicable on 2 August 2026. The penalties are why boards pay attention. Under Article 99, breaches of the Article 5 prohibited practices draw fines of up to EUR 35 million or 7 percent of total worldwide annual turnover, whichever is higher. You can read the obligations on the European Commission's regulatory framework page.

A governance framework whose audit layer already emits structured, per-action records covers most of what the Act's logging and record-keeping articles require. A framework whose audit layer is application logs scattered across teams does not, and you will discover the gap during an assessment rather than before it.

The stakes are not theoretical. Stanford HAI's 2025 AI Index reported that the AI Incident Database recorded 233 AI-related incidents in 2024, a record high and a 56.4 percent increase over 2023, drawn from the AI Index responsible-AI chapter. The audit layer is what turns one of those incidents from an unexplained outage into a traceable event with a root cause.

Mapping your framework to the three reference standards

A CISO will be asked, often in the same meeting, "are we NIST-aligned, are we ISO-certifiable, and are we EU AI Act ready?" These are three different questions, and the value of a single framework is that one control structure can answer all three. The standards are complementary, not competing.

The pragmatic sequence for an enterprise: use NIST AI RMF to give your risk work a shared vocabulary and to structure the Map, Measure, and Manage activities; fold in the GenAI Profile's risk list to decide which controls you actually need; build your policy layer as an ISO/IEC 42001 management system if you want third-party certification; pull AI-specific risk method from ISO/IEC 23894:2023; and treat the EU AI Act as a binding floor that your audit layer must satisfy if you operate in or sell into the EU. One control structure, three answers.

An AI governance framework example, end to end

To make the five layers concrete, trace a single principle all the way down, which is what an "AI governance framework example" should actually show rather than a diagram of boxes.

• Principle: Sensitive data must never leave the organisation's control through an AI system.
• Policy: No credential, key, or personal identifier may be included in any AI tool call. Owner: Head of Security. Exceptions: none.
• Control: A secret-and-PII scanner inspects every outbound AI tool call against a pattern set before the call executes.
• Enforcement: The scanner runs synchronously at the protocol-transport layer. A match returns a denial; the call never reaches the external tool.
• Audit: The denial is written as a structured JSON event (identity, pattern matched, tool, timestamp) and shipped to the SIEM, where it satisfies both the internal control test and the EU AI Act logging expectation.

That vertical slice is the whole framework in miniature. Repeat it for each principle and you have a governance framework that is enforced, not merely written. The maturity metric that matters is the ratio of policies with a bound enforcing control to total policies. At 100 percent, every value the organisation holds reaches the request path. Below that, you know exactly which principles are still only aspirations.

AI governance best practices for enterprises

The five-layer model is the structure. The practices below are what separate an enterprise AI governance programme that holds up under audit from one that collapses the first time it is tested. Each is drawn from the same primary frameworks and stated as a rule you can adopt or reject.

Assign one accountable owner per AI system. Shared ownership is no ownership. The NIST AI RMF Govern function exists to establish exactly this accountability, and the EU AI Act assigns obligations to specific roles (provider, deployer). Name a single person who answers for each system's risk posture. If a model causes harm, the question "who owned this?" must have a one-word answer.

Enforce at the protocol layer, not per application. This is the architectural decision from Layer 4, restated as a governance rule because teams keep relearning it the expensive way. Per-app enforcement guarantees drift: ten teams will implement the same policy ten slightly different ways, and the gaps are where incidents live. One enforcement surface, inherited by every client, is the only model that stays consistent as the organisation grows.

Prefer synchronous controls that block over retroactive controls that detect. A control that flags a violation after the fact is a monitoring tool, not a governance control. Governance prevents. The distinction maps directly to the NIST Manage function: managing risk means treating it before harm, not cataloguing it after.

Log every AI action as structured data inside your own perimeter. Two requirements, both load-bearing. Structured because a SIEM cannot reason over free text. Inside your perimeter because audit logs record which users touched which data through which tools, and if those logs leave your network you have created a new exfiltration path while trying to close one. This is also the difference between governance you can certify and governance you merely assert.

Treat the framework as code so it evolves with the landscape. The AI ecosystem ships new models, protocols, and agent patterns continuously. A framework frozen in a signed PDF is stale within a quarter. Policy-as-code, versioned allowlists, and a reviewable enforcement pipeline let governance keep pace, and the change history doubles as evidence.

Measure coverage, not activity. The vanity metric is "number of policies written." The real metric is the percentage of policies bound to an enforcing control and the percentage of AI actions that actually pass through the enforcement pipeline. A programme with forty policies and four enforced is weaker than one with eight policies all enforced.

Govern third-party and agent-to-agent calls, not just human prompts. The fastest-growing risk surface is autonomous agents calling tools and other agents without a human in the loop. The NIST Generative AI Profile flags this class of risk explicitly. Your scope check and allowlist must apply to machine identities exactly as they apply to people, because an agent with unbounded tool access is a more dangerous actor than any single user.

The failure mode every one of these practices guards against is the same: governance that lives in documentation instead of in the request path. A framework is only as strong as its weakest enforced control, and a control that does not run on the real traffic is not a control.

Conclusion

Build the framework bottom-up. Decide where enforcement sits before you write the principles charter, because a principle you cannot enforce is a liability you will defend in an incident review. Start with one vertical slice (one principle, one policy, one control, one enforcement point, one audit record), prove it works in the request path, then widen. Map the result to NIST AI RMF for vocabulary and the risk register, ISO/IEC 42001 for certification, and the EU AI Act for legal floor. For the deployment architecture that keeps audit evidence inside your own perimeter, see the guide on self-hosted AI governance, and for choosing between approaches, the AI governance platform evaluation guide.