AI Model Governance Can't See What Your Agent Did

What is AI model governance?

AI model governance is the lifecycle discipline of registering, validating, documenting, monitoring, and retiring every model an organisation runs, so each stays an accountable asset with an owner and a risk tier. Most programmes structure it against NIST AI RMF 1.0 or ISO/IEC 42001:2023. It governs a model's outputs and performance. It does not govern what an AI agent does with those outputs, which is why a firm can hold a complete model inventory and still have no record of what its agents did last Tuesday.

That gap is where Major fits. On Major, an agent builds deterministic apps for the repeatable parts of its work, and those apps hold scoped credentials, enforce role-based access, and write their own audit log. Model governance stays in place for the model. The app layer governs the action.

What model governance covers well

The standard lifecycle has six parts. Registration puts the model in an inventory with an owner, purpose, and risk level. Versioning tracks each deployed version with its own metrics. Approval gates review intake materials and risk assessments before release. Model cards document intended use, training data, evaluation results, and limitations. Drift monitoring checks performance and fairness against thresholds. Decommissioning retires a model while preserving its history. Atlan's model governance overview is a representative account of how this is practised.

It works. Drift monitoring is often the only control pointed at a model that was sound at approval and has quietly stopped being sound. Nothing here argues for weakening it. The claim is about coverage.

Where the vocabulary comes from, and why SR 11-7 no longer settles it

Most of the field's language descends from the US banking supervisory guidance SR 11-7, issued 4 April 2011. It defines a model as a quantitative method that processes input data into quantitative estimates, built from an input component, a processing component, and a reporting component. Qualitative approaches count only "provided that the output is quantitative in nature."

That clause is the boundary of the discipline. An agent that reads a contract, decides the payment terms were met, drafts a collections note, and updates a CRM record produces no quantitative estimate. It produces a side effect in a system of record.

SR 11-7 is also no longer current. The Federal Reserve, FDIC, and OCC issued SR 26-2, Revised Guidance on Model Risk Management, on 17 April 2026, superseding SR 11-7 and SR 21-8. It is most relevant to banking organisations with over $30 billion in total assets. It narrows the definition rather than widening it, excluding "deterministic rule-based processes and software where there are no statistical, economic, or financial theories underpinning their design or use." Footnote 3 states that "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance."

The regulator that wrote this vocabulary has said agentic AI sits outside it. The control has to live somewhere else. Pages published in 2026 that still cite SR 11-7 as live guidance, or link the old URL, which now returns a 404, are telling a regulated reader something about the rest of the page.

What a model card cannot tell you

Trace one agent action against a model governance programme. The agent reads an overdue invoice, interprets the contract terms, decides they were breached, sends a collections email, and writes a status change to the CRM under a service credential.

• Read scope: data lineage documents training and input data, not the records the agent reached at run time.
• Interpretation: the model card holds evaluation results from a benchmark set months before this contract existed.
• The decision: drift monitoring needs a score and a threshold. There is neither.
• The tool call: the phrase does not appear in the model governance vocabulary.
• The credential: identity logs show a service account, not which agent acted, on which task, under whose authority.

Six weeks later a customer disputes the note. Model card, validation report, drift chart, and inventory row all fail to reconstruct the decision. Every control caught real failures. None pointed at the thing that went wrong.

The three surfaces with no owner

The tool call. An agent's capability is its tool inventory, which changes when someone edits a config file. The model card describes a model whose available actions it does not know.

The credential. Agents usually act under a long-lived service credential scoped wider than any task needs. Model risk does not cover credentials, and identity governance does not cover the agent's decision to use one.

The action record. Model monitoring records latency, output distribution, and evaluation scores. If an agent updated 400 records overnight, the dashboard shows a normal day.

What to ask a vendor selling agent oversight

For an action taken at a specific time, can you reconstruct four things? The tools available, the authority used, the side effects that landed, and a path to replay it. If any answer is no, the gap is open. Telemetry on tokens and latency does not answer any of the four, and an "inventory row" for a GPT-based assistant describes the model while omitting its tools, credentials, and history.

How to close it on Major

Agent controls layer on top of model controls. A regulated organisation needs both. Here is the shape of the action layer on Major:

1. App that holds state. The collections workflow becomes an accounts app with its own managed database: invoices, contract terms, actions taken, approvals. History lives in tables, not a context window.
2. Connectors under scoped credentials. The app reaches NetSuite, Salesforce, Gmail, and Slack through connectors. The agent never holds the key, and each credential is scoped to the task.
3. Agent for judgment only. The agent reasons about the ambiguous clause or the borderline dispute. Matching, lookups, and status writes run as deterministic code.
4. Trigger. A scheduled workflow or connector event starts each run, so execution is repeatable rather than conversational.
5. RBAC and approvals. Role-based access applies to the records the app touches, and high-value actions route to a person in Slack before they land.
6. Audit. The app writes its own log of what ran, under which identity, with which side effects, so "what did it do in March" is answered by inspection.

You can start building free and put one finance workflow through this shape.

What this does not solve

Major is not a model risk platform and does not validate models. If you run credit, pricing, or capital models, you still need validation, drift monitoring, and an inventory. An app layer also does not remove prompt injection or decide which actions a human should approve. It makes actions inspectable, which is the precondition for enforcing every other control. SR 26-2 puts deterministic software on the far side of its scope line, and work that moves from model reasoning into code moves into ordinary software controls. That does not settle anyone's regulatory position, and an examiner will have views.

Start building on Major

Put the agent's repeatable work into governed apps with scoped credentials, role-based access, and an audit log, so the action layer has an owner alongside your model inventory.

• Start building free
• Book a demo

Related articles

• AI Governance Framework: One You Implement, Not Attest To
• Enterprise AI Governance: From Policy to Implementation
• AI Agent Security: The Threat Model and How to Contain It