The question of how to evaluate an AI model has no single answer — and for most of the field’s history, it had no agreed-upon framework at all. Organizations deploying machine learning systems developed internal practices, published benchmark scores of varying reliability, and hoped their internal testing was adequate. That era is ending. Two major standards — the NIST AI Risk Management Framework and ISO/IEC 42001 — now provide structured approaches to AI model evaluation that are being adopted by regulators, procurement offices, and compliance teams worldwide.

Neither standard dictates a specific test suite or numerical threshold. Both instead define the organizational and procedural conditions under which trustworthy evaluation can occur. Understanding what they actually require — and what they leave to implementers — is essential for any technical or policy professional working in this space.

This article unpacks the core elements of each framework, examines how supplementary tools like model cards and red-teaming fit in, and traces what operationalizing these standards looks like in practice. Where standards remain immature or contested, that is noted plainly.

What Is the NIST AI Risk Management Framework?

The NIST AI Risk Management Framework (AI RMF 1.0), published in January 2023, organizes AI risk management into four core functions: Govern, Map, Measure, and Manage. These are not sequential phases but overlapping activities that operate continuously across the AI system lifecycle.

Govern establishes the organizational policies, roles, and culture that make risk management possible. It asks: who is accountable for AI decisions, and does the organization have the internal infrastructure to act on risk information? Without functional governance — clear ownership, escalation paths, and documented policies — the other three functions lack a home.

Map involves identifying the AI system’s context, intended use, and stakeholders. This is where organizations characterize risk: who could be harmed, under what conditions, and with what severity. Mapping is explicitly an ongoing activity, not a one-time intake exercise. As deployment contexts change — new user populations, new data distributions, updated model versions — the risk map must be revisited.

Measure is the function most directly concerned with AI model evaluation standards in a technical sense. It requires organizations to analyze, assess, benchmark, and monitor AI risk using both quantitative and qualitative methods. The framework does not mandate specific metrics; instead, it calls for measurement practices that are appropriate to the system’s context, regularly updated, and traceable. Fairness metrics, reliability testing, adversarial evaluation, and uncertainty quantification are all within scope, but none are compulsory by name.

Manage closes the loop by requiring that identified risks be prioritized, treated, and tracked. Residual risks — those that cannot be fully mitigated — must be documented and accepted with accountability. This function also covers incident response: what happens when a deployed system fails or causes harm?

The AI RMF is voluntary for most organizations in the United States, though its influence on federal procurement and sector-specific regulation (financial services, healthcare) is growing. The accompanying NIST AI RMF Playbook provides suggested actions for each function, organized by outcome. These are illustrative, not prescriptive.

What Does ISO/IEC 42001 Require?

ISO/IEC 42001:2023 is the first international standard specifically addressing artificial intelligence management systems. Published in December 2023, it follows the high-level structure (Annex SL) used by ISO 9001 (quality management) and ISO 27001 (information security), which means organizations already certified to those standards will find the clause structure familiar.

At its core, ISO/IEC 42001 specifies requirements for establishing, implementing, maintaining, and continually improving an AI management system (AIMS). The standard is certifiable — third-party auditors can assess conformance and issue certificates — distinguishing it from voluntary frameworks like the NIST AI RMF.

Key requirements include:

  • Context and scope definition: Organizations must determine internal and external factors relevant to their AI activities and define the scope of the AIMS. This includes identifying AI systems in use, their purposes, and affected stakeholders.

  • Leadership and AI policy: Top management must demonstrate commitment to the AIMS and establish an AI policy that includes commitments to responsible AI use, legal compliance, and continual improvement.

  • Risk and impact assessment: Clause 6 requires organizations to conduct AI risk assessments and, notably, AI impact assessments. The standard distinguishes between risks to the organization (reliability, security, reputational) and impacts on individuals and society (fairness, privacy, safety). Both require documented processes.

  • AI system lifecycle controls: Annex A contains a set of reference controls covering data management, model development, testing and validation, deployment, and decommissioning. Organizations select controls based on their risk assessments and document their applicability and exclusions in a Statement of Applicability — the same mechanism used in ISO 27001.

  • Operational controls: Clause 8 requires that AI system development and deployment follow defined processes, including verification that systems behave as intended and that documented objectives are met.

  • Monitoring, measurement, analysis, and evaluation: Clause 9 requires organizations to determine what needs to be monitored and measured, how, and when — and to analyze and evaluate the results. This is where ongoing model evaluation lives within the standard’s structure.

Certification audits follow a stage-1 (documentation review) and stage-2 (on-site assessment) structure. Surveillance audits typically occur annually, with recertification every three years. As of mid-2025, ISO/IEC 42001 certification is still relatively rare; the auditor ecosystem is developing and interpretive guidance for specific sectors is limited. Organizations pursuing certification should treat the process as a multi-cycle maturation effort rather than a one-time compliance exercise.

Model Cards and System Cards: Documentation as Evaluation

Model cards, introduced in a 2019 paper by Mitchell et al., are structured documentation artifacts intended to accompany trained machine learning models. A model card specifies the model’s intended use, out-of-scope applications, training data, evaluation results across relevant subgroups, ethical considerations, and known limitations.

The mechanism is straightforward: by requiring developers to document evaluation results — disaggregated by demographic group, deployment context, and edge case — model cards create accountability and enable downstream users to make informed deployment decisions. A model card that shows dramatically degraded performance on a particular language variety or demographic group is not a failure of the card; it is the card working as intended.

Neither the NIST AI RMF nor ISO/IEC 42001 mandates model cards by name, but both frameworks require documentation of evaluation results, intended use cases, and limitations in ways that model cards directly satisfy. The EU AI Act’s conformity documentation requirements for high-risk AI systems similarly align with model card content.

System cards extend this concept to deployed AI products composed of multiple models, data pipelines, and human-in-the-loop components. Rather than documenting a single model in isolation, a system card addresses the emergent properties — and risks — of the full assembled system. System cards are increasingly used by organizations deploying large AI products where the behavior of the system cannot be fully attributed to any single model.

The practical challenge with both artifact types is maintenance. A model card written at training time becomes stale as fine-tuning, deployment contexts, and data distributions evolve. Treating these as living documents — updated at each significant model version and reviewed on a defined schedule — is a requirement under the lifecycle management clauses of ISO/IEC 42001, even if the artifact type is not named explicitly.

Red-Teaming and Adversarial Evaluation: What Do Standards Actually Say?

Red-teaming — structured adversarial testing intended to surface failure modes, unsafe outputs, or exploitable behaviors — has become a prominent practice in AI safety evaluation, particularly for large language models and systems with high societal impact. The term is borrowed from security practice, where red teams attempt to breach defenses.

In the AI context, red-teaming typically involves human evaluators (internal staff, external contractors, or crowdsourced contributors) attempting to elicit harmful, biased, or otherwise problematic outputs from a system through adversarial prompting, edge-case inputs, or manipulation of system context. Automated red-teaming — using another model to generate adversarial inputs — is also in active use.

Neither the NIST AI RMF nor ISO/IEC 42001 uses the term “red-teaming” in its normative text. The NIST AI RMF Measure function calls for “adversarial testing” and “stress testing” as evaluation approaches, and the playbook guidance references red-teaming specifically as an example practice. ISO/IEC 42001 Annex A controls covering testing and validation create space for adversarial evaluation without mandating it.

The US Executive Order on Safe, Secure, and Trustworthy AI (October 2023) moved red-teaming closer to a de facto requirement for frontier AI models, directing NIST to develop standards for red-teaming and requiring developers of the most capable models to share red-team results with the government. The EU AI Act includes conformity assessment requirements for general-purpose AI models with systemic risk that reference adversarial testing. Neither constitutes a universal legal mandate, but the regulatory trajectory is clear.

What organizations actually do varies considerably. A minimally compliant approach might involve a structured internal testing session against a defined threat model. A more rigorous practice includes external red teams, defined scope and methodology, documented findings, and remediation tracking. The absence of a mandated methodology means that “red-teaming completed” can mean very different things across organizations — a gap that standards bodies and regulators are working to close. For a closer look at how these adversarial practices are actually structured and where they fall short, see our analysis of AI safety benchmarks and red-teaming.

This connects directly to the challenge of evaluating LLMs through benchmarks and their limits: benchmark performance and adversarial robustness are related but distinct properties. A model that scores well on standard benchmarks may be highly susceptible to adversarial inputs — making red-teaming a necessary complement to, not substitute for, benchmark evaluation.

How Do These Standards Connect to Technical Evaluation in Practice?

For organizations building or deploying AI systems, the frameworks described above translate into a set of concrete organizational activities. The connection to technical evaluation — the actual testing, measurement, and analysis of model behavior — runs through several channels.

Evaluation planning: Both the NIST AI RMF and ISO/IEC 42001 require that evaluation be planned, not improvised. This means defining evaluation objectives, metrics, data, and acceptance criteria before testing begins — and documenting that definition. For teams accustomed to informal evaluation practices, this represents a meaningful change.

Subgroup and fairness analysis: The Measure function of the AI RMF explicitly calls for disaggregated evaluation — examining performance not just in aggregate but across relevant demographic and contextual subgroups. ISO/IEC 42001’s impact assessment requirements create similar obligations. Practically, this means maintaining test datasets with sufficient demographic coverage, selecting metrics appropriate to the harm type, and documenting results that may show performance gaps.

Ongoing monitoring: Both frameworks treat evaluation as continuous, not a one-time pre-deployment gate. This requires integrating evaluation into MLOps pipelines — monitoring production model behavior, tracking data distribution shifts, and triggering re-evaluation when specified thresholds are crossed. Static evaluation at training time is insufficient under either framework’s requirements.

Third-party and independent evaluation: Neither standard mandates external evaluation for most systems, but both create documentation and process requirements that make third-party audits feasible. ISO/IEC 42001 certification explicitly involves external auditors reviewing evaluation processes and results. For high-risk systems, the EU AI Act requires notified body involvement in conformity assessment.

Documentation and traceability: Evaluation results must be documented in a form that supports accountability. This means not just recording outcomes but maintaining traceability between evaluation findings, remediation decisions, and deployment approvals. Without this traceability, the Manage function of the AI RMF — and the corrective action requirements of ISO/IEC 42001 — cannot be implemented effectively.

Where Are AI Model Evaluation Standards Still Maturing?

The landscape described above represents meaningful progress, but significant gaps remain. Acknowledging them is necessary for honest analysis.

Measurement methodology is underdefined. The NIST AI RMF and ISO/IEC 42001 specify that measurement must occur but leave methodology largely to implementers. This creates wide variation in what “compliant” evaluation actually looks like across organizations. Work on standardized evaluation suites, metric definitions, and benchmarking methodology is ongoing but fragmented across standards bodies, government agencies, and academic consortia.

Sector-specific guidance is sparse. Both frameworks are general-purpose. Healthcare AI, financial services AI, autonomous systems, and criminal justice AI each present distinct risk profiles, affected stakeholders, and regulatory contexts. Sector-specific profiles for the NIST AI RMF are under development, and ISO technical committees are working on domain-specific guidance, but coverage remains incomplete.

Conformity assessment for high-capability models is unsettled. The EU AI Act’s provisions for general-purpose AI models with systemic risk introduce requirements — including red-teaming and model evaluation — that do not yet have fully specified assessment procedures. The relevant implementing acts and guidance documents are still in development as of mid-2025.

Automation and scale create new evaluation challenges. Large language models and multimodal systems present evaluation challenges that existing frameworks did not anticipate: emergent behaviors, context-sensitivity, and the sheer scale of possible inputs make exhaustive evaluation impossible. Standards bodies are beginning to address this, but methodological consensus is lacking.

Organizations serious about AI model evaluation standards should treat current frameworks as structuring guides — valuable for establishing process, accountability, and documentation discipline — while actively tracking the technical evaluation literature for methodology. The two are complements, not substitutes.

Frequently Asked Questions

What is the NIST AI Risk Management Framework?

The NIST AI Risk Management Framework (AI RMF 1.0) is a voluntary US government framework published in 2023 that organizes AI risk management into four functions: Govern, Map, Measure, and Manage. It provides structured guidance for identifying, assessing, and responding to AI-related risks across the system lifecycle without mandating specific metrics or test methods.

What does ISO/IEC 42001 cover?

ISO/IEC 42001:2023 is a certifiable international standard for AI management systems. It specifies requirements for AI governance, risk and impact assessment, lifecycle controls, and continual improvement. Unlike voluntary frameworks, it supports third-party certification audits. It follows the same clause structure as ISO 9001 and ISO 27001, making integration with existing management systems more straightforward.

What is a model card?

A model card is a structured documentation artifact accompanying a trained machine learning model. It records intended uses, out-of-scope applications, training data characteristics, evaluation results disaggregated by subgroup, ethical considerations, and known limitations. Originally proposed in a 2019 research paper, model cards are now widely used as a transparency and accountability tool and align with documentation requirements in major AI governance frameworks.

Is AI red-teaming required by any standard?

No current standard universally mandates red-teaming by name, though the NIST AI RMF references adversarial testing as an example evaluation practice, and the US Executive Order on AI directed development of red-teaming standards for frontier models. The EU AI Act’s conformity requirements for high-risk and systemic-risk AI systems create conditions where adversarial evaluation is practically necessary. Regulatory pressure toward red-teaming requirements is increasing, but formal mandates remain limited.

How do ISO/IEC 42001 and the NIST AI RMF relate to each other?

The two frameworks are complementary rather than duplicative. The NIST AI RMF provides a detailed, flexible risk management vocabulary suited to US contexts and voluntary adoption. ISO/IEC 42001 provides a certifiable management system standard with global applicability. Organizations subject to international procurement or regulatory requirements may pursue ISO/IEC 42001 certification while using the NIST AI RMF playbook for implementation guidance. The frameworks share core concepts — risk assessment, lifecycle management, documentation — and are broadly compatible.