AI governance framework: a practical operating model for enterprise AI

What an AI governance framework actually has to do

Most published frameworks describe values. Boards need values; delivery teams need decisions. A usable AI governance framework answers five questions for any given system, and answers them the same way every time.

If your framework cannot answer these questions in a single document per system, it is a policy, not a framework.

• Is this use case permitted, permitted with conditions, or prohibited?
• Who is accountable for its behaviour in production — by name, not by department?
• What evidence must exist before it goes live, and who checks it?
• How is it monitored, and what triggers a rollback or a human takeover?
• When is it reviewed again, and what would cause it to be retired?

The five layers of the framework

We structure AI governance in five layers. Each layer produces a specific artefact, which is what makes the framework auditable rather than aspirational.

Principles set the boundary of acceptable use. Policy translates the boundary into rules. Process embeds the rules into how work already flows. Controls make the rules hard to bypass. Evidence proves all of the above to an auditor, a regulator or a customer's procurement team.

• Layer 1 — Principles: a one-page statement of purpose, fairness, transparency, human oversight, security and accountability commitments.
• Layer 2 — Policy: acceptable use, prohibited use, data classification rules, third-party model rules, and disclosure obligations.
• Layer 3 — Process: intake, risk tiering, design review, pre-deployment gate, change management and periodic review.
• Layer 4 — Controls: access control, data boundary enforcement, evaluation suites, logging, guardrails and rate limits.
• Layer 5 — Evidence: the model register, evaluation results, sign-off records, incident log and monitoring dashboards.

Risk tiering: the mechanism that keeps governance proportionate

Risk tiering is the single highest-leverage element of an AI governance framework. Without it, every use case receives either too much scrutiny or too little. With it, a marketing copy assistant clears intake in a day while an automated credit decisioning model gets the review it deserves.

Tier on impact, not on technology. A simple rules engine that denies a loan is higher risk than a large language model that drafts internal meeting notes. Three variables drive the tier: who is affected by the output, how reversible a wrong output is, and how much human judgement sits between the model and the consequence.

The EU AI Act's risk-based structure, the NIST AI Risk Management Framework's Govern–Map–Measure–Manage functions, and ISO/IEC 42001's management-system requirements all map cleanly onto this tiering. Build the tiering once and you can evidence against all three without maintaining three programmes.

• Tier 1 — Prohibited: uses your organisation will not pursue at any price. Write them down; the list prevents repeated debate.
• Tier 2 — High risk: affects individuals' rights, money, safety, employment or access to services. Full review, named accountable executive, mandatory human-in-the-loop, formal evaluation and monitoring.
• Tier 3 — Limited risk: affects internal operations or customer experience with reversible outcomes. Lightweight review, standard controls, sampled quality monitoring.
• Tier 4 — Minimal risk: internal productivity with human review of every output. Register it, apply baseline data rules, and get out of the way.

Roles and accountability that survive contact with reality

Governance fails when accountability is collective. Committees can advise; only individuals can be accountable. We define four roles for every AI system and require them to be filled before build starts.

The system owner is a business leader, never the data science lead. If the AI changes how underwriting works, underwriting owns it. This one rule removes most of the failure modes we see: orphaned models, unmaintained prompts and evaluation sets nobody updates.

• System owner (business): accountable for outcomes, approves go-live, owns the benefit case and the retirement decision.
• Technical owner (engineering): accountable for the build, evaluations, monitoring and incident response runbook.
• Risk and compliance reviewer: independent challenge, confirms tier, checks evidence completeness, cannot be a member of the delivery team.
• Data steward: confirms lawful basis, data classification, retention and that no restricted data crosses a model boundary it should not.
• An AI governance council owns the framework itself — the policy, the tiering rules and exceptions — but does not review every system. It reviews Tier 2 systems and exceptions only.

The model lifecycle: seven gates from intake to retirement

The framework becomes real when it is expressed as gates in the delivery lifecycle. Each gate has an owner, an input and a binary decision. Nothing is a 'discussion'.

Crucially, gates 3 and 4 happen before significant engineering spend. The most expensive governance failure is discovering at gate 5 that the data you built on cannot lawfully be used.

• Gate 1 — Intake: use case description, expected benefit, affected population, initial tier. Owner: system owner. Output: registry entry.
• Gate 2 — Tier confirmation: independent reviewer confirms or escalates the tier. Prohibited uses stop here.
• Gate 3 — Data clearance: lawful basis, classification, residency, retention, vendor terms and training-data usage rights.
• Gate 4 — Design review: architecture, model choice, data boundary, human oversight design, failure modes and fallback behaviour.
• Gate 5 — Evaluation: performance against an agreed evaluation set built from real cases, plus safety, bias and robustness testing appropriate to the tier.
• Gate 6 — Go-live approval: signed by the system owner on the evidence pack; includes rollback plan and monitoring thresholds.
• Gate 7 — Periodic review and retirement: scheduled by tier — quarterly for Tier 2, annually for Tier 3 — with drift, incident and benefit data reviewed together.

Controls that do the real work

Policies describe intent; controls enforce it. These are the controls we implement most often, and they are the ones auditors ask about first.

Note how many are engineering concerns rather than policy concerns. An AI governance framework that lives only in the risk function will not hold. It has to be wired into the platform teams use to build.

• A single AI gateway: all model calls route through one service, giving you logging, cost attribution, rate limits, model allow-lists and instant provider switching.
• Data boundary enforcement: private endpoints or self-hosted models for restricted data, with contractual and technical guarantees that prompts are not retained or used for training.
• Retrieval with permission inheritance: a user must never see, via a model, a document they could not open directly.
• Evaluation suites in CI: prompt, model or retrieval changes run against the evaluation set automatically, and regressions block deployment like any failing test.
• Guardrails: input filtering, output validation against schemas, refusal behaviour, citation requirements and confidence-based escalation to a human.
• Complete decision logging: input, retrieved context, model version, output, human action and final decision — retained per your records policy.
• Human-in-the-loop design: define the reviewer's actual authority. A reviewer who cannot realistically dissent is a rubber stamp, not a control.

Evaluation and monitoring: governing behaviour, not intentions

Every Tier 2 and Tier 3 system needs an evaluation set built from real historical cases, labelled by the people who own the process. Fifty to two hundred well-chosen cases beat a generic benchmark every time, because they encode your edge cases and your definition of a good answer.

In production, monitor four families of signal: quality, drift, cost and behaviour. Set thresholds before launch and attach an action to each — alert, throttle, force human review, or roll back. A threshold without an attached action is decoration.

Review incidents in the same forum as operational incidents. AI failures are operational failures; giving them a separate process signals that they are somebody else's problem.

• Quality: accuracy against sampled human review, task completion rate, escalation rate and reversal rate.
• Drift: input distribution change, retrieval hit quality, and output distribution shifts against the launch baseline.
• Cost: cost per resolved task, not cost per token — the only number a business owner can act on.
• Behaviour: refusal rate, guardrail trips, prompt-injection attempts, and override frequency by human reviewers.

Third-party and agentic AI: where most frameworks are already out of date

Two categories now break older governance frameworks. The first is embedded AI in software you already buy: your CRM, service desk and HR suite ship AI features that were never reviewed because no project ever started. The second is agentic systems that take actions rather than produce text.

For embedded AI, run a vendor sweep and register every AI feature currently enabled, then apply the same tiering. For agentic systems, govern the actions, not the model: enumerate every tool the agent can call, apply least privilege, require approval for irreversible actions, cap spend and blast radius, and log every invocation with the reasoning that led to it.

• Maintain an AI vendor register alongside the model register, including data-processing terms and training-data usage clauses.
• Require model version pinning and change notification from vendors; silent model upgrades are a change-management event.
• For agents, classify tools as read, write-reversible or write-irreversible, and gate the last category behind human approval.
• Set hard budget and concurrency limits per agent, enforced at the gateway rather than in the agent's own instructions.

A 90-day implementation plan

You do not need a year to stand this up. The sequence below gets a defensible AI governance framework running in a quarter, with the first governed use cases shipping inside it rather than waiting for it.

Deliberately, discovery comes first. Almost every organisation we work with finds more AI already in use than leadership expected, and that inventory shapes the policy far better than a blank page does.

• Days 1–15 — Discover: inventory every AI system, pilot, vendor feature and shadow tool in use. Draft the prohibited-use list.
• Days 16–30 — Define: publish principles, acceptable-use policy and the risk tiering rules. Name the council and the four roles.
• Days 31–50 — Build the spine: stand up the model register, the AI gateway with logging and allow-lists, and the evidence-pack template.
• Days 51–70 — Pilot the gates: run two live use cases end to end through all seven gates. Time each gate and cut whatever adds no decision value.
• Days 71–90 — Operationalise: wire evaluations into CI, launch monitoring dashboards, run a tabletop AI incident exercise, and schedule the first periodic reviews.

How to know the framework is working

Governance programmes are usually measured by activity — policies published, training completed. Measure yours by throughput and evidence instead, because those are the two things a slow framework destroys and a good one protects.

If time-to-approval is rising while the number of governed systems is flat, your framework has become a bottleneck and should be simplified. Governance that nobody can route around and nobody resents is the target.

• Median days from intake to go-live approval, tracked by tier and trending down.
• Percentage of production AI systems with a named owner, current evaluation results and live monitoring — target 100%.
• Percentage of AI spend flowing through the governed gateway rather than direct vendor accounts.
• Number of exceptions granted, with reasons — a rising count means the policy is wrong, not that teams are.
• Mean time to detect and to roll back an AI incident, exercised at least twice a year.

Common failure modes

Finally, the patterns we are called in to fix. Each one is avoidable with a decision made early.

• Governance written by people who have never shipped a model — the rules are unimplementable and get ignored.
• One review process for all use cases, so trivial systems queue behind consequential ones.
• Accuracy targets set after the build, which makes every evaluation result arguable.
• No evaluation set owned by the business, so quality debates become opinion contests.
• Model registers maintained in a spreadsheet nobody updates after the audit.
• A framework that stops at deployment, with no periodic review and no retirement path — leaving unowned models running for years.