How to design an AI operating model that scales

An AI operating model is the deliberate architecture that connects your AI strategy to sustained business value: it defines structure and decision rights, governance and runtime controls, data and platform infrastructure, end-to-end lifecycle processes, talent and ways of working, and the funding and measurement mechanisms that keep AI initiatives accountable to real KPIs. Without it, you have pilots. With it, you have a compounding capability.

The NIST AI Risk Management Framework makes a critical distinction: unlike traditional IT frameworks, an AI operating model must explicitly address non-deterministic outputs through human-in-the-loop accountability, runtime governance, and continuous model monitoring. That is not a technical footnote. It is the design constraint that shapes every other decision.

Six core elements every leader must account for:

• Structure and ownership: who holds decision rights over AI investments, prioritisation, and deployment gates
• Governance and runtime controls: the automated and human checks that keep production models safe, fair, and compliant
• Data and infrastructure: the pipelines, inventories, and platform layers that make models trainable and trustworthy
• Lifecycle and processes: the build-to-deploy-to-monitor-to-retrain loop that treats AI as an ongoing operation, not a project
• Talent and ways of working: the roles, career paths, and collaboration patterns that sustain AI capability
• Measurement and funding: the KPIs and capital allocation mechanisms that connect AI outputs to P&L

Three actions to take in the next 30–90 days:

1. Audit ownership: identify who currently holds decision rights over AI spend and deployment — if the answer is unclear, that gap is your first risk.
2. Select one or two pilot domains where data is ready, the KPI is measurable, and the scope is controllable.
3. Set runtime governance guardrails before your first model goes to production: drift thresholds, access controls, and an incident response owner.

Key takeaways

An effective AI operating model connects strategy to sustained value through six components: structure, governance, data, lifecycle processes, talent, and measurement — all funded as a portfolio, not as isolated projects.

Table of Contents

• What are the core components of an AI operating model?
• How do you design an AI operating model step by step?
• What does effective AI governance actually require?
• Which roles does your organisation need to operate AI?
• What does an AI implementation roadmap look like in practice?
• How do you measure whether your AI operating model is working?
• What pitfalls should leaders watch for, and what does UK compliance require?
• How do you embed ethical and responsible AI into the operating model?
• What do real AI operating model implementations look like?
• Who owns AI risk management and compliance in your organisation?
• The operating model question most leaders are asking wrong
• Oakandnine gives mid-market leaders a live operating model to build on
• Sources

What are the core components of an AI operating model?

Structure and decision rights determine whether AI scales or stalls. A centralised Centre of Excellence (CoE) works well early, giving you shared standards and a talent nucleus. As you move to multiple production use cases, a hybrid model — CoE plus domain-embedded AI engineers — tends to outperform both pure centralisation and full federation, because it preserves governance rigour while keeping AI close to the business problem. Treating AI initiatives as a portfolio, with executive ownership of data and a shared data inventory, is a consistent marker of organisations that actually scale.

Governance and runtime controls are where most mid-market organisations have the sharpest gap. Governance is not a committee that meets quarterly. It is a layered system: strategic oversight at board or executive level, programme oversight through a cross-functional committee, operational governance through the CoE, and automated runtime enforcement in production. Each layer has distinct accountabilities.

Data and infrastructure underpin everything else. A shared, documented data inventory — not a collection of departmental spreadsheets — is the prerequisite for model training, auditability, and reuse. Platform choices matter here too: frameworks such as PyTorch and TensorFlow shape portability and explainability, while orchestration layers for generative AI add integration complexity that must be accounted for in your architecture.

The platform and MLOps layer is the industrialisation engine. It covers model registries, CI/CD pipelines for models, feature stores, and deployment infrastructure. Without it, every new model is a bespoke engineering effort, and with it, time-to-production drops and reuse rates climb.

Lifecycle and processes formalise the build-to-deploy-to-monitor-to-retrain loop. The critical discipline here is treating monitoring and retraining as first-class operational activities, not afterthoughts. Models degrade. Data distributions shift. The lifecycle process is what catches that before it becomes a business incident.

Talent and ways of working covers both the specialist roles (ML engineers, model validators, data stewards) and the broader organisational shift toward judgement and orchestration work. AI workflow automation changes what mid-market teams actually do day-to-day, and the operating model must account for that redesign explicitly.

Funding and portfolio management closes the loop. Project-by-project funding creates the fragmented pilot problem. Treating AI as a portfolio, with dynamic reallocation across lead, lag, and exit positions, is what Deloitte's research on rewiring the operating model identifies as the structural shift most organisations resist longest.

Value measurement connects every component back to the business. Technical metrics alone — model accuracy, latency — do not tell a CFO anything useful. The measurement layer must translate model performance into revenue uplift, cost-to-serve reduction, or margin improvement.

Pro Tip: Assign a named executive sponsor to each component, not just to the AI programme overall. When governance has no named owner, it defaults to nobody.

How do you design an AI operating model step by step?

The design sequence matters as much as the components. MIT Sloan's research is direct on this: the most common failure is planning in reverse — selecting AI use cases before defining the business strategy they are meant to serve. Start from your north star, not from the technology.

Staged design sequence:

1. Assess current state: map existing AI activity, data assets, ownership gaps, and technical debt. Deliverable: an ownership map and capability baseline.
2. Define strategic north star and KPIs: identify the two or three business outcomes AI must move. Deliverable: a KPI register linked to business unit P&Ls.
3. Decide structure and CoE model: choose centralised, federated, or hybrid based on your maturity and the number of active use cases. Deliverable: a decision-rights matrix.
4. Design governance and runtime controls: define the four governance layers and the automated checks that will run in production. Deliverable: a runtime controls checklist.
5. Build the platform and industrialisation layer: select frameworks, establish the MLOps pipeline, and document the feature store and model registry. Deliverable: an MLOps pipeline specification.
6. Pilot and validate: run one or two use cases against pre-agreed success criteria. Deliverable: a pilot scorecard with KPI outcomes.
7. Scale and manage as a portfolio: use dynamic reallocation (lead/lag/exit) to move capital toward what is working. Deliverable: a portfolio dashboard reviewed at executive cadence.

Pilot selection checklist — a use case qualifies when it meets all four:

• Strategic alignment: the KPI it moves is already on the executive agenda
• Data readiness: clean, accessible, and sufficient training data exists today
• Measurable outcome: success can be expressed in a business metric within 90 days
• Controllable scope: failure is recoverable and does not create regulatory or reputational exposure

PwC's AI business strategy guidance frames this well: leaders must shift from static annual planning to a dynamic, real-time operating cadence so capital can follow the highest-impact workflows as they emerge.

What does effective AI governance actually require?

Governance is four layers, not one policy document.

Strategic layer (board and executive): sets risk appetite, approves the AI portfolio, and owns accountability for material AI decisions. This layer reviews outcomes, not model specifications.

Programme layer (oversight committee): manages cross-functional prioritisation, resolves conflicts between business units, and monitors the portfolio against KPIs.

Operational layer (CoE and platform team): enforces standards, runs the model registry, manages the MLOps pipeline, and owns the runtime controls framework.

Runtime layer (automated enforcement): this is where the NIST AI Risk Management Framework has its sharpest practical implication. Production models require automated checks that run continuously, not audits that happen annually.

Runtime controls checklist:

• Drift detection: statistical monitoring of input data distributions and model output distributions against baseline
• Fairness and bias monitoring: automated checks against protected characteristics relevant to the use case and UK Equality Act obligations
• Accuracy and performance thresholds: defined degradation triggers that escalate to human review or automatic rollback
• Security and supply-chain checks: vulnerability scanning for model dependencies and third-party components
• Incident response: a named owner, a defined escalation path, and a documented rollback procedure

Third-party and vendor models require their own governance track. Modern operating models treat vendors as part of the operating ecosystem, which means procurement must require runtime traceability, performance SLAs, and audit rights — not just contractual indemnities.

Which roles does your organisation need to operate AI?

The talent picture for a mid-market AI operating model is more specific than "hire data scientists." Each role has a distinct accountability:

• Executive sponsor: owns the AI portfolio at board or C-suite level; accountable for KPI outcomes and risk appetite
• Head of Data and AI: translates business strategy into the data and model roadmap; owns the CoE
• Platform engineers: build and maintain the MLOps pipeline, model registry, and deployment infrastructure
• ML engineers: design, train, and optimise models; own the build-to-deploy handoff
• Model validators and assurance leads: independent review of model outputs for accuracy, fairness, and compliance before production deployment
• Product owners (AI): define use-case requirements, own the pilot scorecard, and manage stakeholder expectations
• Domain-embedded AI engineers: sit within business units to translate domain knowledge into model features and monitor live performance
• Data stewards: own data quality, lineage, and access governance within their domain
• Change leads: manage adoption, training, and the cultural shifts that improving operational efficiency at scale requires

Staffing options for mid-market organisations:

• Build: graduate and internal talent pipelines for platform and data engineering roles
• Buy: senior hires for Head of Data and AI and model validation, where experience is non-negotiable
• Partner: use specialist consulting for CoE design, governance framework build, and MLOps architecture
• Internal talent marketplace: redeploy analytically strong domain staff into embedded AI engineer roles with structured upskilling

Pro Tip: Design career paths for AI roles before you hire into them. Without a visible progression from ML engineer to principal or to product owner, you will lose mid-career talent to organisations that have already done this work.

What does an AI implementation roadmap look like in practice?

Three phases, with distinct cost profiles and expected outputs:

Cost buckets to plan for:

• Platform and infrastructure: cloud compute, model hosting, MLOps tooling
• Data engineering: pipeline build, data quality remediation, feature store
• Talent: internal hires, upskilling programmes, consulting support
• External models and licences: third-party foundation model APIs, vendor platforms
• Change management: training, communications, adoption measurement
• Governance and assurance: model validation, audit tooling, compliance review

Costs concentrate in data engineering and consulting during the pilot phase, then shift to talent and change management as you scale. Platform costs tend to be more predictable once the architecture is set.

On funding model: project-by-project budgeting is the structural cause of the fragmented pilot problem. The Lead-Lag-Exit framework from PwC offers a practical alternative: classify each AI initiative as a lead investment (scale capital in), a lag investment (hold and monitor), or an exit (redeploy resources). Review the portfolio quarterly, not annually. This is what dynamic capital reallocation looks like in practice for a mid-market organisation.

How do you measure whether your AI operating model is working?

Technical metrics are necessary but not sufficient.

Core KPI categories:

• Business outcome KPIs: revenue attributable to AI-assisted decisions, cost-to-serve reduction, margin improvement per process automated. Link these to margin strategy and existing P&L lines.
• Platform KPIs: time-to-production for new models, model reuse rate across business units, pipeline reliability
• Quality KPIs: drift incidents per quarter, false positive and false negative rates by use case, rollback frequency
• Organisational KPIs: adoption rate by business unit, time-to-value from pilot approval to KPI movement, employee productivity uplift in AI-assisted roles

Five dashboard widgets worth tracking at executive level:

• AI-attributable revenue or cost saving (£ value, updated monthly)
• Model health score (composite of drift, accuracy, and fairness metrics)
• Time to production (days from use-case approval to live deployment)
• Incidents per month (runtime failures, bias flags, rollbacks)
• Reuse rate (percentage of new use cases built on existing model components)

Attribution is the hardest problem. The discipline is to define the counterfactual at pilot design stage — what would the KPI have been without the model? — and hold that baseline through to scale. Organisations that skip this step end up measuring only technical metrics and cannot answer the CFO's question.

What pitfalls should leaders watch for, and what does UK compliance require?

Research-backed failure modes:

• Planning in reverse: selecting AI use cases before the business strategy is defined. MIT Sloan identifies this as the most common pitfall — and it is recoverable only if caught before significant capital is committed.
• Fragmented pilots: running ten pilots simultaneously with no shared platform, no shared governance, and no portfolio view. Bain's research shows that durable leaders make deliberate structural bets on a few domains rather than spreading thinly.
• CoE becoming a bottleneck: a centralised CoE that does not embed capability into business units creates dependency and slows adoption. The fix is a hybrid model with clear handoff protocols.
• Missing runtime governance: deploying models without automated drift detection and incident response. This is the gap the NIST AI Risk Management Framework addresses most directly.
• Weak value measurement: tracking only technical metrics and failing to connect model outputs to P&L. Organisations that do this accumulate proofs-of-concept that do not change competitive economics.

UK compliance checklist:

• ICO: confirm lawful basis for each data use under UK GDPR; document data minimisation and retention policies for training datasets; conduct a Data Protection Impact Assessment for high-risk AI applications
• NCSC Cyber Essentials: apply the five Cyber Essentials controls (firewalls, secure configuration, access control, malware protection, patch management) to AI infrastructure and model serving environments
• Procurement and supplier due diligence: require audit rights, performance SLAs, and data residency confirmation for all third-party model providers; treat vendor platforms as part of the operating model with the same governance standards applied to internal systems

Executive action checklist for your next leadership meeting:

• Confirm a named executive sponsor for the AI portfolio
• Request the ownership map and KPI register — if neither exists, commission them within 30 days
• Ask which production models have automated drift detection in place
• Confirm that the pilot selection checklist has been applied to every active use case
• Review the funding model: is AI budgeted as a portfolio or as individual projects?

How do you embed ethical and responsible AI into the operating model?

Responsible AI is not a values statement appended to a policy. It is a set of design constraints built into every component of the operating model, from data collection through to runtime monitoring.

At the data layer, this means documenting provenance, applying data minimisation principles, and auditing training datasets for representational bias before model training begins. At the model layer, it means requiring explainability outputs for any model whose decisions affect individuals — a standard that UK GDPR's right to explanation makes legally relevant, not just ethically desirable.

At the governance layer, responsible AI requires independent model validation before production deployment. The validator's role is specifically to challenge the model builder's assumptions — on fairness, on accuracy across demographic subgroups, and on the plausibility of the use case's stated benefits. This independence is structural: validators must not report to the team that built the model.

At the runtime layer, fairness monitoring must be continuous, not a one-time pre-launch check. Model behaviour changes as input data drifts, and a model that was fair at launch can develop discriminatory patterns over time without any deliberate change. Automated fairness checks, with defined escalation thresholds, are the operational mechanism that keeps responsible AI commitments real.

The cultural dimension matters too. Leaders who frame responsible AI as a compliance burden tend to get compliance theatre. Leaders who frame it as a trust asset — something that protects the organisation's licence to operate and its relationships with customers and regulators — tend to get genuine embedding.

What do real AI operating model implementations look like?

Two patterns appear consistently in mid-market implementations, and they illustrate the difference between an organisation that scales and one that stalls.

Pattern one: the CoE-first build. A UK-based financial services firm established a small CoE of six people — a head of AI, two ML engineers, a data engineer, a model validator, and a change lead. Rather than attempting broad deployment, the CoE focused exclusively on one use case: automating a high-volume document classification process that consumed significant analyst time. The pilot ran for ten weeks, with a pre-agreed KPI (analyst hours per document) and a defined rollback threshold. The model reached production in week twelve. Within six months, the CoE had documented the MLOps pipeline, the governance checklist, and the pilot scorecard as reusable artefacts. The second use case took half the time to reach production because the infrastructure already existed.

Pattern two: the federated failure. A mid-market retailer ran eight AI pilots simultaneously across four business units, each with its own vendor, its own data pipeline, and no shared governance. Eighteen months in, three pilots had been quietly discontinued, two were producing outputs that nobody trusted, and the remaining three had no clear path to scale because the underlying platforms were incompatible. The cost of rationalising the estate exceeded the original pilot investment.

The difference is not budget or ambition. It is sequencing: the first organisation built shared infrastructure and governance before broadening scope. The second treated pilots as independent experiments rather than as the foundation of a portfolio.

Bain's research on proprietary intelligence captures the underlying logic: emerging leaders make board-level bets on a few domains, building proprietary data, encoded workflows, and learning architectures that compound advantage over time. Breadth without depth produces neither.

Who owns AI risk management and compliance in your organisation?

Risk management for AI is not the CISO's problem alone, and it is not the data science team's problem alone. It is a distributed accountability that the operating model must make explicit.

Role-based accountability map:

• Board and executive sponsor: own the organisation's AI risk appetite and are accountable for material AI failures to regulators and stakeholders
• Head of Data and AI: owns the risk register for the AI portfolio and escalates material risks to the executive sponsor
• Model validators and assurance leads: own pre-production sign-off; their approval is a deployment gate, not a recommendation
• Data stewards: own data quality and access governance; accountable for flagging training data issues before model build begins
• Platform engineers: own infrastructure security, including patching, access controls, and supply-chain vulnerability management
• Domain-embedded AI engineers: own runtime monitoring within their business unit; first line of response to drift or performance alerts
• Legal and compliance: own the regulatory mapping — UK GDPR, sector-specific obligations, and the evolving UK AI regulatory framework — and advise on high-risk use case classification
• Procurement: own supplier due diligence for third-party models and vendor platforms, including audit rights and data residency confirmation

The critical design principle is that no single role owns both model development and model validation. That separation is what gives the governance layer its integrity. Where organisations collapse these roles for cost reasons, they typically discover the gap only after a production incident.

The operating model question most leaders are asking wrong

The framing I encounter most often in mid-market engagements is: "Which AI use cases should we prioritise?" It is the wrong starting question. The right question is: "What does our business model require us to be better at in the next three years, and which of those capabilities can AI compound?"

That reframe changes everything downstream. It means the AI operating model is designed around existing strategic commitments, not around what the technology can currently do. It means the KPIs are already on the executive agenda before the first model is trained. And it means the governance layer is protecting something the organisation genuinely values, rather than enforcing rules around experiments nobody is sure they want.

The organisations I have seen scale AI successfully share one structural characteristic: they invested in the industrialisation layer and governance framework before they invested in breadth of use cases. Data pipelines, MLOps infrastructure, model registries, and runtime controls are not glamorous. They do not make good board presentations. But they are what separates a portfolio of compounding AI capability from a collection of expensive proofs-of-concept.

For mid-market leaders specifically, the practical implication is this: ask your direct reports next week to show you the ownership map, the runtime controls checklist, and the KPI register for your current AI activity. If any of those three artefacts does not exist, you have your 30-day priority.

Oakandnine gives mid-market leaders a live operating model to build on

Mid-market organisations designing an AI operating model face a specific problem: the data, processes, and organisational structures they need to govern and scale AI are rarely visible in one place. Oakandnine solves that directly.

The platform provides live organisational mapping, workflow detection, and data unification across HR, Finance, and Operations — the exact foundation an AI operating model requires. Where most organisations spend months building the data inventory and process map that governance depends on, Oakandnine surfaces it in real time. That means your pilot selection checklist has accurate data behind it, your runtime controls have a live process baseline to monitor against, and your KPI register connects to actual operational metrics rather than spreadsheet estimates.

Typical client fit: mid-market UK organisations with 100–2,000 employees, ready to move from isolated pilots to a governed, scalable AI capability. Engagements follow a discovery-to-pilot-to-scale cadence, with Oakandnine's consulting team supporting operating model design alongside the platform deployment.

Book a discovery call to map your current AI activity and identify the fastest path to a production-ready operating model.

Sources

• NIST AI Risk Management Framework
• 6 questions to guide your AI strategy — MIT Sloan
• Rewiring the op model for AI | Deloitte Insights

Recommended

• Benefits of AI in business operations: a mid-market guide
• How to Improve Operational Efficiency in Business
• AI workflow automation software for mid-market leaders
• Oakandnine — Map and optimise your organisation