What is Model Card?

Definition

A model card is a structured document that accompanies a machine learning model, providing transparency about its development, capabilities, limitations, and appropriate use. Introduced by Mitchell et al. at Google in 2019, the model card framework standardizes how models are documented, much like nutrition labels standardize food product information. Model cards are now a widely adopted best practice and increasingly a regulatory requirement.

A comprehensive model card includes several standard sections: model description (architecture, size, training approach), intended uses (what the model is designed for and explicitly not designed for), training data (source description, size, composition, preprocessing), evaluation results (performance metrics across different subgroups and test sets), limitations and biases (known failure modes, demographic biases, capability boundaries), ethical considerations (potential harms, mitigation strategies, responsible use guidelines), and maintenance information (who maintains the model, how to report issues, update schedule).

Model cards serve multiple audiences. For developers, they provide the technical information needed to integrate the model correctly. For end users, they set expectations about what the model can and cannot do. For regulators and auditors, they provide the documentation required for compliance assessments. For the organization deploying the model, they serve as a risk management artifact that documents due diligence in model development.

Why It Matters

The EU AI Act mandates documentation of high-risk AI systems that closely mirrors model card requirements — including training data provenance, performance metrics, known limitations, and intended uses. Organizations deploying AI in the EU without adequate documentation face fines and deployment restrictions. Model cards are the practical mechanism for meeting these documentation requirements.

Beyond compliance, model cards prevent misuse. A model trained on English text and evaluated on American cultural contexts might perform poorly or produce harmful outputs when used for other languages or cultures. Without a model card documenting these limitations, downstream users have no way to know the model's boundaries. Model cards make these limitations explicit, enabling informed deployment decisions and preventing foreseeable harms.

How It Works

Model card creation is typically integrated into the model development workflow. During training, metadata about the training process (data sources, hyperparameters, training duration, hardware used) is captured automatically through experiment tracking tools. After training, evaluation results are generated by running the model through standard and domain-specific benchmarks, with results disaggregated by relevant subgroups (language, demographic, difficulty level).

The model developer then writes the narrative sections — intended uses, limitations, ethical considerations — based on their understanding of the model's capabilities and the evaluation results. The completed model card is published alongside the model weights, typically on the model's repository page (e.g., Hugging Face Hub). Responsible teams update the model card when new information about limitations or biases is discovered, treating it as a living document rather than a one-time artifact.

Example Use Case

A healthcare AI company publishes a model card for their clinical note summarization model. The card documents that the model was trained on English-language clinical notes from US hospitals (2015-2024), achieves 91% ROUGE-L on their test set but only 73% on UK clinical notes due to terminology differences, has not been evaluated on pediatric notes, and should not be used as a sole basis for clinical decisions. When an overseas hospital considers deploying the model, the model card helps them understand the performance gap and plan appropriate validation before deployment.

Key Takeaways

• Model cards are standardized documentation describing a model's capabilities, limitations, and appropriate uses.
• They serve developers, end users, regulators, and organizational risk management simultaneously.
• The EU AI Act mandates documentation requirements closely aligned with model card content.
• Key sections include intended uses, training data, evaluation results, limitations, and ethical considerations.
• Model cards are living documents that should be updated as new limitations or biases are discovered.