TL;DR
An AI Center of Excellence is a small, operating-led group that owns four functions: platform, governance, enablement, and measurement. Most CoEs fail because they get built before the anchor metric is defined, become committees instead of operators, and try to centralize use cases instead of the rails. Build the CoE after the anchor exists and after one pilot has shipped. Staff it for outcomes, not headcount theater.
- A CoE is operating-led, not committee-led. It ships infrastructure, not slide decks.
- Four functions only: platform, governance, enablement, measurement. Everything else is sprawl.
- Centralize the rails. Federate the use cases.
- Build it after the anchor, not before. Otherwise it is a planning function with no scoreboard.
- Migrate from planning to platform by the end of year one, or kill it.
In this article
What an AI Center of Excellence actually is
An AI Center of Excellence, in the working sense, is a small operating group that owns the conditions under which AI gets built and shipped across a company. It is not a committee. It is not a steering group. It is not the place where AI ideas go to die in a quarterly review.
The right mental model is platform engineering plus rules of the road. The CoE builds the rails (shared models, evaluation harness, observability, vendor contracts) and sets the rules (data handling, output review, kill switches, escalation paths). Then it gets out of the way and lets the business units ship use cases on top.
Three concrete tests for whether your CoE is actually a CoE:
- Does it have a measurable output? If the only artifact the CoE produces is meeting minutes, it is a committee. Real CoEs ship platform components, training programs, audit reports, and quarterly numbers.
- Can a business unit lead self-serve through it? If a marketing lead has to wait three weeks for the CoE to approve a pilot, the CoE is a gatekeeper, not an enabler. Self-serve is the test.
- Is it inside the operating cadence? Real CoEs report into the same weekly, monthly, and quarterly business reviews as every other operating function. If the CoE has its own parallel meeting universe, it is theater.
This is the same distinction I made in the AI transformation playbook: AI is an operating-model change, not a tooling rollout. The CoE is the institutional form that change takes.
When to build one (and when not to)
The biggest mistake I see at consumer brands is building the CoE first. The CEO reads three McKinsey articles, hires a Head of AI, gives them a budget, and tells them to build a Center of Excellence. Six months later, the team has produced an AI policy, a tooling roadmap, and a vendor RFP, and shipped exactly nothing into production.
The right sequence is the opposite. You build the CoE after two things are true:
- The anchor metric is defined. One P&L line the AI program will move. Without it, the CoE has nothing to organize itself around. With it, every CoE decision can be tested against whether it moves the anchor.
- At least one pilot has shipped to production. Not a demo. A real pilot, with a real workflow, real users, and a measurable outcome. The first pilot teaches you what platform components are actually needed and what governance questions actually matter. Building the CoE before this turns it into a guessing exercise.
If you build the CoE before these two conditions are met, you will get a planning function. Planning functions are deeply optimized for producing documents. They are not optimized for shipping. Once a CoE becomes a planning function, it is extremely hard to turn it into a platform function later.
A CoE built before the anchor metric becomes a planning function. A CoE built after the first pilot becomes a platform function. The order is everything.
The four functions of a working CoE
A working AI CoE owns exactly four functions. Not three, not seven. Four. Adding more dilutes accountability. Cutting any one of them leaves a structural gap that shows up later as outage, sprawl, or stalled adoption.
1. Platform
The shared technical substrate. The CoE owns the model gateway, the evaluation harness, the prompt registry, the observability stack, the cost telemetry, and the integration patterns that business units use to plug AI into existing workflows. Without platform ownership, every team builds its own everything, costs explode, and quality is uneven.
2. Governance
The rules of the road. Data handling, PII boundaries, output review for customer-facing AI, vendor contract templates, kill switches, escalation paths, and the model selection policy. Governance is not a 60-page document. It is a one-page policy that every team can actually follow. For a deeper look, see AI governance without killing velocity.
3. Enablement
The human side. Training, internal documentation, prompt libraries, office hours, and the practical change-management work that gets the org from "we have AI tools" to "we actually use them." Enablement is where most CoEs underinvest, and it is the single biggest predictor of whether adoption sticks. The work is covered in detail in AI change management for frontline teams.
4. Measurement
The scoreboard. The CoE owns the anchor metric reporting, the adoption telemetry, the contribution-margin attribution, and the vendor-cost reconciliation. Measurement is the function that keeps the program honest. Without it, activity metrics fill the gap, and the program drifts. See measuring ROI on AI initiatives for the methodology.
These four functions are non-negotiable. If your CoE charter has eight functions, six of them are theater. If it has two, you are missing structural pieces that will show up as failures later.
Staffing patterns at SMB, mid-market, and enterprise
The right CoE size is a function of the anchor metric's annual impact, not the company's headcount or revenue. I have seen $30M brands run effective CoEs with one person and seen $500M brands waste a fifteen-person CoE on slide decks. The size is downstream of the work, not the org chart.
SMB pattern: one plus fractional
At a consumer brand under $50M, the CoE is usually one full-time AI lead plus a Fractional CTO or technical advisor. The lead owns enablement and measurement directly. Platform and governance get bought (model gateway, eval tooling, off-the-shelf policy templates) and stitched together. This is the pattern I use at most early-engagement Automatic clients. It works because the anchor metric is usually one or two use cases deep, and the platform requirements are modest.
Mid-market pattern: the core four
Between $50M and $300M, the CoE grows to three to five people. A lead, a platform engineer who owns the model gateway and observability, an enablement owner (often a former training or operations leader), and a part-time governance counterpart in legal or risk. This is where the CoE earns its name. Platform components get built rather than bought, the eval harness is custom to the company's data, and enablement scales beyond one channel.
Enterprise pattern: separated leads
At nine-figure scale, each of the four functions has a named lead with a small team. Platform lead with two to four engineers. Governance lead with a legal counterpart and an audit function. Enablement lead with training and internal comms. Measurement lead with analytics. Total CoE headcount of eight to fifteen, depending on use-case breadth. The enterprise CoE also runs the vendor-relationship function, because at this scale, AI vendor contracts are material.
What kills enterprise CoEs is over-staffing the planning function. If the CoE has more program managers than engineers, it has become a planning shop. Real platform work requires builders. The ratio I look for is at least 60% of CoE headcount in builder or operator roles, not in coordination roles.
The failure modes that kill most CoEs
I have audited or rebuilt enough CoEs to know the failure modes are predictable. Three patterns account for the overwhelming majority of failures.
Theater CoE
The Theater CoE produces a lot of artifacts: a strategy deck, an AI policy, a vendor scorecard, a maturity model. None of them ship. The CoE meets, deliberates, publishes, and meets again. Nothing in production attributable to it after twelve months. The CEO eventually defunds it quietly.
The Theater CoE is what you get when you build a CoE before the anchor exists. There is no scoreboard, so artifacts become the scoreboard, and producing artifacts becomes the work.
Committee CoE
The Committee CoE has the right charter but the wrong staffing. Every function is owned by a representative from a business unit, none of whom have any time to actually do the work. Decisions take weeks because every stakeholder must be consulted. The CoE becomes a coordination overhead on top of the business units, not a platform underneath them.
The fix is named, dedicated headcount. The CoE has to be somebody's day job, not everybody's side project.
Gatekeeping CoE
The Gatekeeping CoE makes itself the approval body for every AI initiative in the company. Business units must submit proposals. Pilots require CoE sign-off. Tools cannot be purchased without CoE review. Velocity collapses. Business unit leaders learn to route around the CoE, and the CoE becomes irrelevant within eighteen months.
The fix is the centralize-rails, federate-use-cases pattern. The CoE owns the platform and the rules. The business units own the application. The CoE measures the outcomes. It does not approve them.
Migrating from planning function to platform function
Most CoEs start as planning functions whether they intended to or not, because the first ninety days produce documents (charter, strategy, policy) before they produce platform. The migration from planning function to platform function is the most important transition in the CoE's first year. Companies that make the migration scale AI. Companies that do not, do not.
The signals that the migration is happening:
- The CoE owns at least one piece of infrastructure that business units depend on (model gateway, eval harness, observability dashboard).
- The CoE's quarterly review includes platform availability and adoption metrics, not just strategic narrative.
- Business unit leaders can name the platform components they use, and they would notice if those components were turned off.
- The CoE has a sprint cadence, not just a planning cadence.
- The CoE has builders on staff, not just program managers.
The signals that the migration has stalled:
- The CoE's deliverables are all documents.
- The CoE's roadmap is dominated by "strategy," "alignment," and "framework."
- Business units treat the CoE as a tax, not as a service.
- The CoE leader cannot name three platform components they have shipped.
If the migration has stalled, the right move is usually a leadership change. The skills required to build a planning function are different from the skills required to operate a platform function. Most CoE leaders who excel at the first phase struggle with the second. Replacing the leader is not a failure of the program. It is a recognition that the work has changed.
The CoE is not done when the charter is written. It is done when business units would notice if you turned it off.
The bottom line
An AI Center of Excellence is the institutional form an AI transformation takes once the anchor is defined and the first pilot has shipped. It owns four functions: platform, governance, enablement, measurement. It centralizes the rails and federates the use cases. It is operating-led, not committee-led, and it ships infrastructure, not slide decks.
Build it after the anchor, not before. Staff it for outcomes, not headcount. Migrate it from planning to platform inside the first year. If you cannot make that migration, kill it and start over. A CoE that produces only documents is a tax on the rest of the business.
FAQ
What is an AI Center of Excellence?
An AI Center of Excellence is a small, operating-led group that owns the AI platform, governance, enablement, and measurement for the company. It is not a committee, not a steering group, and not a research lab. It ships infrastructure, sets the rules, trains the org, and reports the numbers.
When should a company build an AI Center of Excellence?
After the anchor metric is defined and at least one pilot has shipped to production. A CoE built before the company knows what it is anchoring on becomes a planning function that produces slides. Define the anchor first, ship one pilot, then formalize the CoE.
How is an AI Center of Excellence staffed?
At SMB, the CoE is one person plus fractional support. At mid-market, three to five people: a lead, a platform engineer, an enablement owner, and a part-time governance counterpart. At enterprise, a team of eight to fifteen with separate platform, governance, enablement, and measurement leads. The ratio that matters is at least 60% builders.
What does an AI Center of Excellence cost?
Most consumer brands can run a working CoE for under five percent of the anchor metric's annual impact. At SMB scale, that is often one full-time hire plus tooling. At mid-market, three to five fully loaded headcount plus a platform budget. Tooling is usually less than a third of the total cost.
Should an AI Center of Excellence be centralized or federated?
Centralized for platform, governance, and measurement. Federated for application. The CoE owns the rails. The business units own the use cases. Trying to centralize the use cases turns the CoE into a bottleneck and kills velocity. Trying to federate the rails creates sprawl and uneven quality.
What is the difference between a CoE and an AI team?
An AI team builds AI applications. A CoE builds the conditions under which AI applications can be built repeatedly: platform, governance, enablement, measurement. A company can have one AI team without a CoE. It cannot scale AI across multiple teams without one.