Model Governance & AI Compliance

Overview

Model governance is the set of policies, processes, and controls that organizations use to manage AI models throughout their lifecycle, from initial development through production deployment, monitoring, and retirement. As AI systems increasingly influence business decisions, customer experiences, and regulatory outcomes, model governance has moved from good practice to regulatory expectation. US banking supervisors (the Federal Reserve, OCC, and FDIC) have long expected model risk management under guidance such as SR 11-7, and the EU AI Act, the NIST AI RMF, and standards like ISO/IEC 42001 now extend similar expectations across industries.

Model governance addresses several concerns. It checks that models are developed using appropriate data and methods, validated by independent parties before deployment, monitored for performance degradation and drift in production, and retired when they no longer meet quality or compliance standards. It also establishes clear accountability by defining who is responsible for model development, validation, approval, and ongoing oversight at each stage of the lifecycle.

The challenge has grown with large language models and foundation models. Traditional model governance frameworks were designed for statistical models with well-defined inputs, outputs, and risk profiles. Modern AI systems are more complex, more capable, and harder to validate fully. Organizations must adapt their governance frameworks to the particular risks of generative AI, including hallucination, prompt injection, training data memorization, and the difficulty of defining complete test suites for open-ended model capabilities.

AI-Specific Requirements

Model governance frameworks typically require a model inventory that catalogs all AI models in development and production, including their purpose, risk classification, development status, deployment location, responsible owners, and validation status. This inventory is the foundation for governance oversight, letting organizations prioritize review efforts by risk and keep every model inside the governance framework. Regulators and auditors frequently request model inventories as a starting point for assessing an organization's AI governance maturity.

Model validation is a cornerstone of governance, requiring independent assessment of model performance, limitations, and risks before deployment. Validation should include conceptual soundness review (evaluating whether the modeling approach is appropriate for the use case), outcome analysis (testing model accuracy, fairness, and robustness using holdout data), and ongoing monitoring validation (verifying that monitoring controls will detect performance degradation). For high-risk models, validation should be performed by parties independent of the development team.

Change management controls govern how models are updated, retrained, and redeployed. Any material change to a model, whether to training data, model architecture, hyperparameters, or deployment configuration, should go through a defined approval workflow that includes impact assessment, testing, validation, and sign-off by authorized stakeholders. Version control for models and their associated artifacts (training data, code, configurations) enables rollback if a new model version exhibits problems in production. Retirement procedures make sure that deprecated models are properly decommissioned and that downstream systems move to replacement models or alternative processes.

How Ertas Helps

Governance starts from an inventory and depends on knowing exactly which model is in production. A custom model trained in Ertas Studio is yours, and its version stays fixed until you retrain, so each entry in your inventory maps to one set of weights you hold. Hub lists the models you have trained, and Studio stores the dataset behind each run and lets you compare runs, which gives validators the training data and the alternatives behind a model choice.

Because you can export the model as GGUF or safetensors and run it in your own environment, deployment, rollback and retirement go through your own change management, and inference data never reaches Ertas. A retrained model is a new version, so you can validate it before you switch. With the adaptive model endpoint, a newly trained model takes over requests only once it passes your evals, and hard cases stay on the frontier model you already use. The endpoint and enterprise builds can run the model in your own cloud or on-premise.

Training runs on Ertas's cloud GPUs, so training data is uploaded to Ertas; minimize sensitive data first and delete datasets you no longer need. Risk classification, independent validation, approval workflows, performance monitoring and drift detection stay with you.

Compliance Checklist