📌 Note: This is an independently prepared study aid based on the publicly available AIGP BoK domain outline. It is not an official IAPP publication. Domain III covers governance process and practice rather than law, so the content is stable and not tied to specific regulatory dates.
1. Introduction and How to Use This Material
Domain III is the most process-intensive domain in the AIGP exam. Where Domain I establishes principles and Domain II maps the legal landscape, Domain III asks: what does a governance professional actually do when AI is being built?
It carries 21–25 questions, split across three competencies:
- III.A, Govern the designing and building of the AI system (6–8 questions)
- III.B, Govern the collection and use of data in training and testing (6–8 questions)
- III.C, Govern the release, monitoring, and maintenance of the AI system (8–10 questions)
III.C is the single heaviest competency in the entire exam.
What Domain III is about
The AI governance professional's role in development is not to be the data scientist, it is to ensure that:
- The right questions are asked before work begins (use case, purpose, risk).
- The right controls exist at each stage of the build.
- The right documentation is produced, maintained, and can withstand regulatory scrutiny.
- Incidents and risks are identified, managed, and learned from.
- The system is not deployed until it is ready, and once deployed, is monitored continuously.
How to use this guide
- Domain III overlaps heavily with Domain IV (deployment and use). The distinction: Domain III is about the development and release side, the people building or overseeing the build. Domain IV is about the deployment and use side, the deployer using someone else's (or their own) system.
- Sections 5–10 (lifecycle diagram, comparison tables, exam traps, knowledge check, mnemonics, 25-fact summary) are designed for revision. Work through Sections 2–4 first.
- The governance professional on an exam question about Domain III is typically acting as an AI governance lead, risk officer, or ethics/policy function embedded in or alongside a development team, not the engineer writing the code.
2. Competency III.A: Govern the Designing and Building of the AI System
2.1 Define the Business Context and Use Case
Governance begins before a single line of code is written. The first step is establishing what the system is for with enough precision that governance can be applied to it.
A well-defined use case captures:
| Element | What to establish |
|---|---|
| Business objective | What problem is the AI solving? What does success look like in business terms? |
| Intended purpose | What specifically will the AI do, and what is it explicitly NOT intended to do? |
| Affected populations | Who are the users? Who else might be affected by the outputs (e.g., third parties whose data is scored)? |
| Data availability | What data will the system need? Does it exist? Can it be lawfully used? |
| Ethical considerations | Does the use case involve sensitive decisions, vulnerable populations, or potential for significant harm? |
| Workforce readiness | Does the organization have the skills and resources to develop, oversee, and maintain this system? |
| Performance requirements | What level of accuracy, speed, or reliability is required? What is the acceptable error rate? |
Poorly defined use cases are one of the most common root causes of AI failures. Systems built without a precise purpose tend to be trained on the wrong data, measured against the wrong metrics, and deployed in contexts they were never designed for.
📘 Study tip The BoK performance indicator is "define the business context and use case." On the exam, a question about what happens first in AI development, or what document captures the purpose and scope of a system, traces to this step.
2.2 Impact Assessment at the Design Stage
An impact assessment at the design stage evaluates the system's potential effects on individuals, groups, and the organization before development is complete. This is a governance gate, not a technical review.
The assessment should address:
- Benefits, what value does the system create and for whom?
- Potential harms, what could go wrong and who could be harmed?
- Affected rights, privacy, equality, autonomy, due process, and others as relevant.
- Risk severity and likelihood, using a structured method such as a probability/severity matrix.
- Mitigations available, are there design choices, data choices, or operational controls that reduce identified risks?
- Residual risk, after mitigations, what risk remains? Is it acceptable given the benefit?
- Decision, proceed, proceed with conditions, or do not proceed.
The design-stage impact assessment is distinct from (though complementary to):
- The DPIA (triggered by high-risk personal data processing, a legal obligation under privacy law).
- The FRIA (triggered by deployment of high-risk AI by certain deployers, an EU AI Act obligation).
- The conformity assessment (pre-market technical compliance check under EU AI Act for high-risk systems).
All may be required for the same system; none substitutes for another.
2.3 Applying Policies, Procedures, and Ethical Considerations to Design
Once the use case is defined and an initial impact assessment performed, the governance professional ensures that the organization's existing policies and ethical commitments are actively applied to design decisions.
The BoK lists the following design-stage governance areas:
| Design area | Governance questions |
|---|---|
| Purpose of AI | Is the intended use consistent with the organization's responsible-AI policy and acceptable-use policy? Are any prohibited uses involved? |
| Requirements gathering | Have stakeholders, including affected populations where feasible, been consulted? Have functional and non-functional requirements (accuracy, fairness, explainability, latency) been documented? |
| Architecture and model selection | Is the chosen architecture appropriate for the task and the explainability requirements? Is a simpler, more interpretable model achievable? Has the model selection been documented with rationale? |
| Human oversight | At what points in the system's operation will a human review, override, or be informed of the AI's output? Is human oversight technically built in or does it depend solely on procedure? |
| Data analysis | Has the available data been analyzed for quality, representativeness, potential sources of bias, and lawful availability before training begins? |
| Metric and threshold evaluation | What metrics will be used to evaluate success? Are these metrics well aligned with the intended outcome and with fairness requirements? Who sets and approves thresholds? |
| Stakeholder engagement and feedback | Have affected stakeholders been engaged during design? Is there a mechanism for ongoing feedback? |
| Operational controls | What controls will apply once the system is in operation, access controls, output review, rate limits, fallback procedures? |
Ethics by design is not a separate step at the end, it means embedding fairness, transparency, and human-oversight requirements into design specifications from the outset. A system whose architecture makes explanation impossible cannot have explainability bolted on later.
2.4 Identifying and Managing Design-Stage Risks
The BoK specifically calls out a set of tools and methods for managing risks during the design and build phase.
Probability / severity harms matrix
A structured way to prioritize risks. Each identified risk is assessed on two dimensions:
HIGH SEVERITY
│
HIGH PROB ─────┼───── LOW PROB
(critical) │ (monitor)
│
HIGH PROB ─────┼───── LOW PROB
(manage) │ (accept)
LOW SEVERITY
- High probability + high severity → critical: block or redesign.
- High probability + low severity → manage with controls.
- Low probability + high severity → monitor closely, have mitigation ready.
- Low probability + low severity → accept with periodic review.
Risk mitigation hierarchy
Before accepting any residual risk, governance should confirm that less harmful alternatives have been considered in order:
- Eliminate the risk, redesign the system or abandon the use case.
- Substitute, use a less risky approach (different model, different data, narrower scope).
- Engineering controls, build safeguards into the system (human-in-the-loop, output filters, confidence thresholds).
- Administrative controls, procedures, training, monitoring.
- Accept residual risk, documented, approved at appropriate level.
Stakeholder mapping
Identifies all parties who have an interest in, or could be affected by, the AI system, users, affected third parties, regulators, employees, customers, communities. Used to ensure that impact assessment and design consultation are genuinely comprehensive.
Use-case evaluation
A structured review of whether the proposed use case is appropriate, considering the use context, the population affected, the availability of recourse, and whether the AI's capabilities match the task.
Benchmarking
Comparing the proposed system's performance against existing solutions (human decision-makers, simpler models, industry standards) to calibrate expectations and identify whether AI is genuinely better or merely novel.
Pre-deployment pilots and testing
Small-scale controlled testing in real or realistic conditions before full rollout, surfacing risks that laboratory testing missed and building evidence for the release decision.
2.5 Documenting the Design and Build Process
Documentation is not bureaucracy, it is the evidence base for compliance and the foundation for accountability. The governance professional ensures that the design and build process produces, and retains, a documented record of:
- The use case definition and business justification.
- The impact assessment, including risks identified and mitigations applied.
- Design decisions and the rationale for them (model selection, fairness metric chosen, human-oversight mechanism).
- Data decisions, sources considered, data selected, data rejected and why.
- Stakeholder consultations conducted.
- Gate approvals, who reviewed what at each stage and what they decided.
📘 Study tip Documentation in Domain III serves three purposes the exam may test: (1) establishing compliance, demonstrating that required processes were followed; (2) managing risk, creating a record that enables later root-cause analysis; and (3) enabling transparency, the foundation for model cards, technical documentation, and public disclosures in III.C.
3. Competency III.B: Govern Data Collection, Training, and Testing
3.1 Data Governance Requirements for AI
Data is the raw material of AI. A model's behavior is determined by its training data; data flaws become model flaws. The governance professional's role in the data phase is to ensure that data is lawful, sufficient, and fit for purpose.
Assessing and documenting lawful rights to collect and use data
Before any data is used for AI training, the organization must be able to demonstrate that it has the legal right to:
- Collect the data (was there a lawful basis for original collection?).
- Use it for training (is the training use compatible with the original purpose? Is there a lawful basis for this new use?).
- Retain it for the duration of the training process and any subsequent use.
This analysis intersects directly with Domain II (privacy law, IP law), but the Domain III perspective is operational: the governance professional ensures the analysis is done and documented, not just that the concepts are understood.
Assessing data quality, quantity, integrity, and fit-for-purpose
| Dimension | What to assess |
|---|---|
| Quality | Is the data accurate, up-to-date, and free from systematic errors? Are labels correct (for supervised learning)? |
| Quantity | Is there enough data for the model to generalize? Is the volume proportionate to the complexity of the task? |
| Integrity | Has the data been tampered with, corrupted, or poisoned? Are there chain-of-custody records? |
| Fit-for-purpose | Does the data actually represent the population and conditions the model will encounter in deployment? A model trained on hospital data from one geography may not transfer to another. |
| Representativeness | Are all relevant subgroups represented at proportions that allow the model to perform equitably across them? |
| Bias sources | Are there known historical biases in the data that the model might learn and amplify? |
3.2 Data Lineage and Provenance
Data lineage is the documented record of where data came from, how it was transformed, and how it flows through the system.
Data provenance is the origin and history of the data, who collected it, when, under what conditions, and with what rights.
Together, lineage and provenance answer:
- If the model behaves unexpectedly, can we trace which data caused it?
- If a data subject exercises an erasure right, can we identify and address all copies?
- If a regulator asks for evidence of lawful data use, can we demonstrate it?
What a lineage record typically includes:
| Element | Description |
|---|---|
| Source | Where did the raw data come from (database, API, vendor, web scrape, survey)? |
| Collection date | When was it collected? |
| Rights basis | What is the lawful basis for collection and training use? |
| Transformations | What cleaning, normalization, augmentation, or labeling was applied, by whom, and when? |
| Version | Which version of the dataset was used for which model training run? |
| Retention | When is this data scheduled for deletion? |
3.3 Training and Testing the AI Model
Once data is governed and documented, the model is trained and tested. The governance professional does not conduct training, but does ensure that:
- Testing is comprehensive in scope.
- Results are documented.
- Pass/fail thresholds are defined in advance, not adjusted retroactively to make results look good.
- No model advances to deployment without completing the required test suite.
Types of testing: the full suite
The BoK lists: unit, integration, validation, performance, security, bias, and interpretability.
| Test type | What it checks |
|---|---|
| Unit testing | Individual components of the system function correctly in isolation. |
| Integration testing | Components work correctly when combined; the system functions correctly as a whole. |
| Validation testing | The model performs as expected on held-out data not seen during training (the validation set). Checks that the model generalizes rather than memorizing training data. |
| Performance testing | Accuracy, precision, recall, F1, AUC-ROC, or task-appropriate metrics against defined thresholds on a separate test set. Checks that the model meets required performance standards. |
| Security testing | Robustness against known AI-specific attacks, prompt injection (for LLMs), data poisoning, adversarial examples, model extraction attempts. |
| Bias testing | Whether the model produces systematically different outcomes for identifiable subgroups, across demographic characteristics, geographic regions, or other relevant cuts. Requires defining which fairness metric applies to this use case. |
| Interpretability testing | Whether the model's outputs can be explained in terms meaningful to the relevant audience, regulators, affected individuals, internal reviewers. May use SHAP, LIME, attention analysis, or other explanation methods. |
📘 Study tip The exam may present a scenario and ask which type of testing would surface a described problem. Match the problem to the test type: a subgroup performing worse → bias testing; a component working in isolation but failing in the full system → integration testing; model performing well in the lab but not in the field → validation / performance on representative test set; attacker causing unexpected output → security testing.
Train / validation / test split discipline
A fundamental governance requirement is that training, validation, and test datasets be kept strictly separate:
Full dataset
┌──────────────────────────────────────────┐
│ TRAINING SET │ VALIDATION │ TEST SET │
│ (model learns) │ (tune & │ (final, │
│ │ iterate) │ held out) │
└──────────────────────────────────────────┘
~70-80% ~10-15% ~10-20%
(typical ranges, vary by context)
The test set must remain unseen until final evaluation. If it is used during development to make decisions, it is no longer an unbiased estimate of real-world performance. This is a common and serious data-governance failure.
3.4 Managing Risks During Training and Testing
Common risks that arise during training and testing and require active governance:
| Risk | Description | Governance response |
|---|---|---|
| Overfitting | Model performs well on training data but poorly on new data; it has memorized rather than generalized. | Train/validation/test discipline; regularization; retesting on representative data. |
| Data leakage | Information from the test set inadvertently influences training, inflating apparent performance. | Strict separation; documented pipeline with gate controls. |
| Label errors | Incorrect or inconsistent labels in supervised training create systematic errors the model learns. | Label-quality audit; inter-rater reliability checks. |
| Bias amplification | Existing biases in training data are not only replicated but amplified by the model. | Bias testing across subgroups; representative dataset construction; reweighting or other mitigation techniques. |
| Concept drift in training data | Training data reflects historical conditions that no longer hold, making the model unsuitable for current deployment. | Data recency analysis; retraining on current data. |
| Insufficient security testing | Model is deployed without testing against known attack vectors. | Security test suite as a required gate before promotion. |
3.5 Documenting the Training and Testing Process
Documentation at this stage serves to:
- Validate results, what testing was done, what the results were, and what they mean.
- Establish compliance, demonstrate that required tests were run and thresholds met.
- Manage risks, record known limitations and residual risks that deployers need to know about.
Key artifacts:
| Artifact | Contents |
|---|---|
| Dataset documentation | Source, rights basis, size, date range, preprocessing steps, version. |
| Training run log | Hyperparameters, training iterations, loss curves, compute environment. |
| Evaluation report | Test results across all required test types, broken down by relevant subgroups, against pre-set thresholds. |
| Bias and fairness report | Fairness metrics by subgroup; description of any disparities; mitigations applied and their effect. |
| Known limitations | What the model cannot do reliably; conditions under which performance degrades; populations or contexts where it has not been tested. |
4. Competency III.C: Govern the Release, Monitoring and Maintenance of the AI System
III.C carries 8–10 questions, the single heaviest competency on the AIGP exam. It covers what happens after the model is built: releasing it responsibly, monitoring it continuously, managing incidents, and meeting public-disclosure obligations.
4.1 Assessing Readiness for Release
Release is a governance gate, not an automatic step after testing. The governance professional's role is to ensure that readiness criteria are met before production deployment.
Model card
A model card is a structured document accompanying the model at release. It is the primary communication artifact between the development team and everyone who will use, oversee, or audit the model. Typical contents:
| Section | Contents |
|---|---|
| Model description | What the model does, its architecture (at appropriate level of detail), intended use. |
| Intended use | Use cases the model is designed for; use cases it is explicitly NOT designed for. |
| Training data | Dataset description, date range, key characteristics. |
| Performance metrics | Results across the required test types; breakdown by relevant subgroups. |
| Known limitations | What the model cannot do; conditions where it fails or underperforms. |
| Ethical considerations | Potential for harm; fairness findings; human-oversight requirements. |
| Caveats and recommendations | Instructions for deployers; monitoring requirements; retraining schedule. |
Conformity requirements
Where applicable (notably under the EU AI Act for high-risk systems), release requires that conformity requirements are satisfied, documentation complete, conformity assessment passed, CE marking applied, and instructions for use addressed to deployers.
Release checklist: minimum governance gate
☐ Use case defined and approved
☐ Impact assessment completed and signed off
☐ Data rights documented
☐ Full test suite completed, all thresholds met
☐ Bias and fairness report reviewed and accepted
☐ Known limitations documented
☐ Model card prepared
☐ Human oversight mechanism in place
☐ Monitoring plan defined (metrics, thresholds, frequency)
☐ Incident management procedure in place
☐ Conformity requirements satisfied (where applicable)
☐ Authorization to deploy obtained from designated approver
4.2 Continuous Monitoring and Maintenance
Deployment is not the end of governance, it is the beginning of a new governance phase. AI systems are dynamic; their operating environment changes, the data they encounter drifts from training distributions, and their performance changes over time.
What to monitor continuously
| Monitoring dimension | What to watch |
|---|---|
| Performance | Are accuracy, precision, recall, and other task-appropriate metrics staying within acceptable bounds? |
| Fairness | Are disparities between subgroups stable and within agreed limits, or widening? |
| Data drift | Has the statistical distribution of inputs changed materially from the training distribution? |
| Model drift (concept drift) | Has the relationship between inputs and the correct outputs changed in the real world, making the model's learned patterns less valid? |
| Error rates and patterns | Are errors clustering in particular subgroups, contexts, or time periods, suggesting a systemic problem? |
| Usage patterns | Is the system being used as intended? Are there signs of misuse, out-of-scope use, or use by unintended populations? |
| Infrastructure health | Latency, throughput, availability, and dependency health. |
Maintenance schedule
Continuous monitoring must be paired with a defined schedule for:
- Routine maintenance, dependency updates, infrastructure patches, performance optimization.
- Scheduled retraining, refreshing the model on new data to maintain performance.
- Triggered retraining, when monitoring metrics breach defined thresholds before the scheduled cycle.
📘 Study tip The distinction between data drift and model drift (concept drift) is frequently tested. Data drift = the inputs the model receives have changed distribution. Model/concept drift = the underlying real-world relationship the model was trained to predict has changed. Both can cause performance degradation, but they require different diagnostic approaches.
4.3 Periodic Performance, Reliability, and Safety Assessment
In addition to continuous monitoring, the BoK requires periodic (scheduled) deeper assessments.
| Assessment type | Purpose |
|---|---|
| Audit | Independent review of the system's governance, documentation, and controls, verifying that what is supposed to happen is actually happening. |
| Red teaming | Adversarial testing where a team attempts to find failure modes, bias, safety gaps, or ways to manipulate the system, especially important for generative AI. |
| Threat modeling | Structured analysis of potential attack vectors and failure scenarios, identifying what could go wrong before it does. |
| Security testing | Ongoing penetration testing and adversarial robustness checks post-deployment, not just at initial release. |
Red teaming and threat modeling deserve particular attention in Domain III because they are explicitly named in the BoK and are increasingly required by regulators and frameworks.
Red teaming: what it means in an AI context
Red teaming for AI goes beyond conventional cybersecurity testing. It includes:
- Attempting to elicit harmful, biased, or off-policy outputs from generative models.
- Testing for bias and discrimination in the model's outputs under varied inputs.
- Testing human-oversight mechanisms, can a determined user bypass them?
- Testing fallback and shutdown procedures.
- Simulating the system under adversarial or edge-case conditions.
4.4 Incident and Issue Management
The BoK requires both managing incidents when they occur and collaborating to understand their root causes.
Defining an AI incident
An AI incident is an event in which the AI system produces an output or contributes to an outcome that:
- Causes harm to individuals, groups, or the organization.
- Violates law, regulation, or policy.
- Falls materially outside expected performance parameters.
- Raises significant fairness, safety, or rights concerns.
The definition must be specific enough to distinguish incidents from ordinary errors, and must be communicated to everyone with a role in the system.
Incident response process
DETECT ──▶ CONTAIN ──▶ ASSESS ──▶ REMEDIATE ──▶ REPORT ──▶ LEARN
│ │ │
(monitoring (internal (policy /
triggers / & external process
user report) where req'd) update)
| Step | What it involves |
|---|---|
| Detect | Monitoring trigger, user report, external report, or audit finding. |
| Contain | If the system is causing ongoing harm, determine whether to suspend, restrict, or modify operation immediately. |
| Assess | Determine the scope of impact, the root cause, and the regulatory reporting obligations triggered. |
| Remediate | Fix the underlying issue, model fix, data fix, process fix, or operational change. |
| Report | Internal escalation; external reporting to regulators or affected parties where required. |
| Learn | Document root cause, update risk register, improve detection and prevention for the future. |
Root causes of AI incidents: BoK named list
The BoK explicitly identifies these as causes the governance professional should be able to recognize:
| Root cause | What it means |
|---|---|
| Brittleness | The system performs well under normal conditions but fails in unexpected or edge-case situations, often because it was not tested on sufficiently diverse inputs. |
| Lack of robustness | Related to brittleness, the system's performance degrades significantly when inputs vary slightly from the training distribution. |
| Lack of quality data | Training or operational data that is inaccurate, biased, outdated, or non-representative. |
| Insufficient testing | The test suite did not cover the conditions under which the failure occurred. |
| Model drift | The real-world relationship the model learned has changed, making its predictions less reliable. |
| Data drift | The inputs the model receives in deployment have shifted away from the training distribution. |
Cross-functional collaboration
The BoK specifically calls out cross-functional stakeholder collaboration for understanding incidents. This reflects the reality that AI incidents often have multiple contributing causes spanning technical, data, process, and organizational dimensions. Effective root-cause analysis requires input from:
- Engineering / data science (technical root cause).
- Data governance (data quality / lineage issues).
- Business / product (use-case conditions that were not tested).
- Legal / compliance (regulatory implications).
- Operations / customer teams (how the system was actually being used).
4.5 Public Disclosures and Transparency Obligations
The BoK requires making public disclosures to meet transparency obligations. This connects directly to the EU AI Act's requirements for high-risk AI providers and the broader principle of transparency from Domain I.
Types of public/external disclosure
| Disclosure type | Purpose and audience |
|---|---|
| Technical documentation | Demonstrates compliance; audience is regulators and conformity-assessment bodies. Covers architecture, training, testing, risk management, and governance. |
| Instructions for use | Addressed to deployers, enables them to deploy responsibly, implement required oversight, and understand limitations. |
| Post-market monitoring plan | Documents how the provider will monitor the system after deployment and what will trigger corrective action or regulatory reporting. |
| Model card | Public-facing summary of intended use, performance, limitations, and ethical considerations, increasingly standard for foundational and commercial models. |
| Transparency notices to users | In limited-risk systems: disclosure that the user is interacting with AI. For automated decision-making: information about the logic and rights available. |
| Incident disclosures | Where law requires notifying regulators or affected individuals of incidents involving the AI system. |
5. The AI Development Lifecycle: End-to-End View
Domain III covers the development side of the lifecycle. Here it is in full, with the governance professional's role at each stage.
┌─────────────────────────────────────────────────────────────────┐
│ AI DEVELOPMENT LIFECYCLE │
│ │
│ ┌──────────────┐ │
│ │ 1. USE CASE │ Define purpose, scope, populations, │
│ │ ASSESSMENT │ ethical considerations, data needs. │
│ └──────┬───────┘ │
│ │ ▶ Gate: approved to proceed? │
│ ┌──────▼───────┐ │
│ │ 2. IMPACT │ Assess potential harms, affected rights, │
│ │ ASSESSMENT │ risk severity / likelihood, mitigations. │
│ └──────┬───────┘ │
│ │ ▶ Gate: risk acceptable with mitigations? │
│ ┌──────▼───────┐ │
│ │ 3. DESIGN & │ Requirements gathering, model selection, │
│ │ BUILD │ ethical-by-design, human oversight, │
│ │ │ stakeholder engagement, documentation. │
│ └──────┬───────┘ │
│ │ ▶ Gate: design meets policy and ethical requirements? │
│ ┌──────▼───────┐ │
│ │ 4. DATA │ Lawful rights, quality, quantity, lineage, │
│ │ GOVERNANCE │ provenance, representativeness, bias audit. │
│ └──────┬───────┘ │
│ │ ▶ Gate: data fit-for-purpose and lawfully usable? │
│ ┌──────▼───────┐ │
│ │ 5. TRAINING │ Train model; validate on held-out data; │
│ │ & TESTING │ full test suite (unit, integration, │
│ │ │ validation, performance, security, │
│ │ │ bias, interpretability). │
│ └──────┬───────┘ │
│ │ ▶ Gate: all thresholds met; test suite complete? │
│ ┌──────▼───────┐ │
│ │ 6. RELEASE │ Model card, conformity requirements, │
│ │ READINESS │ release checklist, deployment approvals. │
│ └──────┬───────┘ │
│ │ ▶ Gate: release criteria met? │
│ ┌──────▼───────┐ │
│ │ 7. DEPLOY & │ Continuous monitoring (performance, │
│ │ MONITOR │ fairness, drift, usage); maintenance │
│ │ │ schedule; periodic audits and red teaming. │
│ └──────┬───────┘ │
│ │ │
│ ┌──────▼───────┐ │
│ │ 8. INCIDENT │ Detect → Contain → Assess → Remediate → │
│ │ MANAGEMENT │ Report → Learn. │
│ └──────┬───────┘ │
│ │ │
│ ┌──────▼───────┐ │
│ │ 9. PUBLIC │ Technical documentation, instructions for │
│ │ DISCLOSURE │ use, post-market monitoring plan, │
│ │ │ user transparency notices. │
│ └──────────────┘ │
└─────────────────────────────────────────────────────────────────┘
📘 Study tip The governance gate after each stage is as important as the stage itself. Governance means there are defined checkpoints at which someone with authority reviews the evidence and decides to proceed, proceed with conditions, or stop. "We tested the model" is not governance. "We tested the model against pre-set thresholds, documented the results, and a designated approver reviewed and signed off" is governance.
6. Comparison Tables
6.1 Impact Assessment Types
| Assessment | Who performs it | Legal basis | Scope | When |
|---|---|---|---|---|
| Design-stage impact assessment (AI IA) | Developer / governance team | BoK best practice; some AI laws require it | AI-specific risks to individuals, groups, business | Before design is finalized |
| DPIA | Controller | GDPR (and similar laws) | Privacy and data protection risks | Before high-risk personal data processing |
| FRIA | Deployer | EU AI Act (certain deployers) | Fundamental rights broadly | Before deploying certain high-risk AI |
| Conformity assessment | Provider (sometimes notified body) | EU AI Act (high-risk systems) | Technical compliance with EU AI Act requirements | Before market placement |
6.2 Types of Testing
| Test | What it surfaces | Who is primarily responsible |
|---|---|---|
| Unit | Component-level failures | Engineering |
| Integration | System-level failures when components combine | Engineering |
| Validation | Generalization failure (overfitting) | Data science / ML engineering |
| Performance | Failure to meet accuracy/recall/precision thresholds | Data science / governance |
| Security | Vulnerability to adversarial attack | Security / engineering |
| Bias | Unfair differential outcomes across subgroups | Responsible-AI / governance |
| Interpretability | Inability to explain outputs | Data science / governance |
6.3 Data Drift vs. Model Drift
| Aspect | Data drift | Model / concept drift |
|---|---|---|
| What changes | The statistical distribution of inputs the deployed model receives | The real-world relationship between inputs and correct outputs |
| Example | A fraud-detection model trained on pre-pandemic spending patterns receives post-pandemic inputs with different distributions | A credit model trained when interest rates were low becomes less predictive when rates rise sharply |
| Detection | Statistical tests comparing live input distribution to training distribution | Monitoring prediction accuracy against ground-truth labels over time |
| Response | Retrain on more recent data | Reassess whether the model's learned patterns still reflect reality; may need fundamental redesign |
6.4 Continuous Monitoring vs. Periodic Assessment
| Aspect | Continuous monitoring | Periodic assessment |
|---|---|---|
| Frequency | Real-time or near-real-time | Scheduled (quarterly, annually, or event-triggered) |
| Method | Automated metrics, dashboards, alerting | Human-led audit, red team exercise, security test |
| Purpose | Detect degradation and incidents early | Deep review of governance, robustness, and risk |
| Outputs | Alerts, incident triggers, performance logs | Audit report, red-team findings, updated risk register |
6.5 Model Card vs. Technical Documentation
| Aspect | Model card | Technical documentation (EU AI Act) |
|---|---|---|
| Audience | Downstream users, deployers, public, researchers | Regulators, conformity-assessment bodies |
| Required by | Best practice / some frameworks | EU AI Act (high-risk AI providers) |
| Detail level | Summary and key findings | Comprehensive technical and governance detail |
| Public? | Typically yes (or at least deployer-facing) | Primarily regulatory, not necessarily public |
7. Exam Traps
⚠ Exam Trap 1, Governance in Domain III is not about writing code. The AI governance professional's role is to ensure that the right processes, controls, and documentation exist, not to be the data scientist or engineer. Questions about Domain III are about governance of development, not technical implementation.
⚠ Exam Trap 2, Ethics by design is not a final review step. Embedding ethical considerations at the end of development, as a check before release, is not ethics by design. Ethics by design means fairness, transparency, and oversight requirements are built into design specifications from the start.
⚠ Exam Trap 3, The test set must be held out until final evaluation. If the test set is used during development to make modeling decisions, it is no longer an unbiased measure of real-world performance. Using the test set during iteration is a data governance failure, not a technical shortcut.
⚠ Exam Trap 4, Passing all tests does not automatically authorize deployment. Release requires a formal governance gate, a designated approver reviewing documented evidence that all criteria are met. Passing the test suite is necessary but not sufficient.
⚠ Exam Trap 5, Monitoring is continuous, not one-time. A common candidate error is treating release as the end of the governance cycle. The BoK is explicit: monitoring is continuous, with a regular schedule for maintenance and retraining, and periodic deeper assessments layered on top.
⚠ Exam Trap 6, Data drift and model drift are different things. Data drift = the inputs have changed distribution. Model/concept drift = the real-world relationship the model was trained to capture has changed. Both cause performance problems; they require different diagnoses.
⚠ Exam Trap 7, A model card is not the same as technical documentation. A model card is a summary artifact for users and deployers; technical documentation under the EU AI Act is a comprehensive compliance record for regulators. They serve different audiences and different purposes.
⚠ Exam Trap 8, Red teaming is not the same as security testing. Red teaming in an AI context is adversarial testing that includes not just cybersecurity probes but also bias elicitation, harmful-output testing, and oversight-bypass attempts. Security testing is one component of what red teaming covers.
⚠ Exam Trap 9, "Lack of robustness" and "brittleness" are related but distinct root causes. Brittleness refers to failures in unexpected or edge-case situations. Lack of robustness is the broader property of performing poorly when inputs deviate from training distribution. Both are BoK-named incident root causes.
⚠ Exam Trap 10, Cross-functional collaboration for root-cause analysis is a governance requirement, not optional. The BoK explicitly calls out the need to collaborate with cross-functional stakeholders to understand why incidents arise. Root-cause analysis that involves only the engineering team is insufficient governance.
8. Knowledge Check Questions
1. An AI governance professional reviewing a new credit-scoring AI project at the design stage performs an impact assessment. This assessment is BEST described as:
- A. A conformity assessment confirming CE marking eligibility.
- B. A structured evaluation of potential effects on individuals and groups, used to identify and mitigate risks before development is complete.
- C. A technical performance benchmark comparing the model to human reviewers.
- D. A DPIA triggered by the processing of personal data.
Answer: B. The design-stage impact assessment evaluates potential harms, affected rights, and risk severity/likelihood, a governance gate before development is finalized. A DPIA is a related but legally distinct obligation; a conformity assessment is a pre-market compliance check. Both may also be required, but they are not the same as the design-stage AI impact assessment.
2. Which of the following is the BEST example of "ethics by design"?
- A. Adding a fairness review step immediately before production deployment.
- B. Conducting a bias audit after the model has been trained.
- C. Embedding fairness metric requirements and human-oversight mechanisms into the system's design specifications from the outset.
- D. Publishing a model card after release describing the model's ethical considerations.
Answer: C. Ethics by design means requirements are built in from the start, not reviewed at the end. After-the-fact reviews (A, B) and post-release disclosure (D) are valuable but are not ethics by design.
3. A model performs well on the training set but significantly worse on the test set. This MOST likely indicates:
- A. Data drift.
- B. Model drift.
- C. Overfitting.
- D. Label poisoning.
Answer: C. High training performance and materially lower test performance is the defining signature of overfitting, the model has memorized the training data rather than generalizing. Data drift and model drift occur in deployment, not at the initial test stage.
4. During post-deployment monitoring, the governance team observes that the distribution of inputs the model is receiving has changed significantly from the training distribution, even though the real-world outcome being predicted has not changed. This is BEST described as:
- A. Concept drift.
- B. Data drift.
- C. Brittleness.
- D. Label error.
Answer: B. When the inputs shift but the underlying real-world relationship is stable, that is data drift. Concept drift is when the real-world relationship itself changes. Brittleness is failure in edge cases. Label error is a training-data quality problem.
5. Which of the following test types would BEST surface the risk that a loan-approval model approves applicants from one demographic at a significantly lower rate than equally qualified applicants from another demographic?
- A. Unit testing.
- B. Integration testing.
- C. Security testing.
- D. Bias testing.
Answer: D. Differential outcomes across demographic groups is a fairness / bias problem, surfaced by bias testing that evaluates model outputs broken down by subgroup. Unit and integration testing check technical functionality; security testing checks for adversarial vulnerabilities.
6. The MOST important reason for keeping the test set strictly separate and unseen during the development process is:
- A. To reduce storage costs for the development team.
- B. To ensure that the measured performance on the test set is an unbiased estimate of real-world performance.
- C. To comply with GDPR data minimization requirements.
- D. To satisfy EU AI Act record-keeping obligations.
Answer: B. If the test set is seen during development and used to make decisions, measured test performance will be optimistic and not reflect how the model will actually perform in the field. This is a fundamental data-governance requirement for valid AI evaluation.
7. An AI governance professional learns that a deployed fraud-detection model is generating a disproportionate rate of false positives for customers in a specific region. The FIRST action consistent with the BoK's incident management framework is:
- A. Retrain the model immediately.
- B. Publish a public disclosure.
- C. Detect and contain, determine whether the system is causing ongoing harm and whether immediate operational restrictions are needed.
- D. Escalate to the board of directors.
Answer: C. The BoK incident response sequence is Detect → Contain → Assess → Remediate → Report → Learn. Containment, determining whether to suspend or restrict operation, precedes remediation, reporting, or escalation. Immediate retraining is a remediation action, not a containment action.
8. Which of the following is included in a model card but NOT typically in EU AI Act technical documentation for high-risk AI?
- A. A description of the model's intended use.
- B. Known limitations of the model.
- C. A public-facing plain-language summary of ethical considerations and caveats for downstream users.
- D. Performance metrics on the test set.
Answer: C. The model card is designed for downstream users and deployers, often in relatively accessible language. EU AI Act technical documentation is a compliance record for regulators, comprehensive in technical detail but not primarily written for public consumption. All four elements could appear in either document, but C is the most distinctively model-card oriented in audience and register.
9. Mnemonics and Memory Aids
9.1 Domain III competencies: "ABCs of AI Development Governance"
- A (III.A), Architecture and building: govern the design and build.
- B (III.B), Build with data: govern training and testing.
- C (III.C), Care post-launch: govern release, monitoring, incidents, and disclosure.
9.2 Testing suite: "UIVPSBI"
(Unit · Integration · Validation · Performance · Security · Bias · Interpretability)
Remember: "U I V, P S B I"
- U, Unit
- I, Integration
- V, Validation
- P, Performance
- S, Security
- B, Bias
- I, Interpretability
9.3 Incident response sequence: "DC-AR-RL"
- D, Detect
- C, Contain
- A, Assess
- R, Remediate
- R, Report
- L, Learn
9.4 Root causes of AI incidents: "BLIND-M"
- B, Brittleness
- L, Lack of robustness
- I, Insufficient testing
- N, Non-quality data (lack of quality data)
- D, Drift (model / concept drift)
- M, More drift (data drift)
9.5 Data governance dimensions: "QQIFR"
- Q, Quality
- Q, Quantity
- I, Integrity
- F, Fit-for-purpose
- R, Representativeness
9.6 Risk mitigation hierarchy: "ESEAA"
- E, Eliminate
- S, Substitute
- E, Engineering controls
- A, Administrative controls
- A, Accept (residual risk, documented)
9.7 Release checklist: "DIATCM"
- D, Data rights documented
- I, Impact assessment completed
- A, All tests passed
- T, Thresholds defined and met
- C, Conformity requirements satisfied
- M, Monitoring plan in place
10. If You Remember Only These 25 Facts
For final-day revision.
- Governance begins before code is written, with the use case definition and impact assessment.
- The use case definition captures: business objective, intended purpose, affected populations, data needs, ethical considerations, workforce readiness, and performance requirements.
- A design-stage impact assessment evaluates potential harms, affected rights, risk severity/likelihood, mitigations, and residual risk. It is distinct from a DPIA, FRIA, and conformity assessment, all may be required for the same system.
- Ethics by design means fairness, transparency, and human-oversight requirements are embedded in design specifications from the outset, not added at the end.
- The probability/severity matrix classifies risks by likelihood and severity to prioritize governance response: critical → manage → monitor → accept.
- The risk mitigation hierarchy runs: Eliminate → Substitute → Engineering controls → Administrative controls → Accept.
- Data governance for AI requires assessing lawful rights to collect and use, and assessing data quality, quantity, integrity, fit-for-purpose, representativeness, and bias sources.
- Data lineage = where data came from, how it was transformed. Data provenance = the origin and history of data including rights basis.
- The full testing suite is: Unit, Integration, Validation, Performance, Security, Bias, Interpretability.
- Train/validation/test split discipline: the test set must be held out and remain unseen until final evaluation. Using it during iteration is a governance failure.
- Overfitting = high training performance, materially lower test performance. The model has memorized rather than generalized.
- Bias testing evaluates whether the model produces systematically different outcomes for identifiable subgroups.
- Interpretability testing evaluates whether outputs can be explained in terms meaningful to relevant audiences.
- Release is a governance gate, a designated approver reviews documented evidence that all criteria are met. Passing tests is necessary but not sufficient.
- A model card is the primary communication artifact at release, intended use, performance, known limitations, ethical considerations, caveats for deployers.
- Continuous monitoring covers performance, fairness, data drift, model/concept drift, error patterns, and usage patterns.
- Data drift = inputs have changed distribution from training. Model/concept drift = the real-world relationship the model was trained to capture has changed. Both cause degradation; they have different diagnoses.
- Periodic assessments (audits, red teaming, threat modeling, security testing) layer on top of continuous monitoring and go deeper.
- Red teaming in AI includes bias elicitation, harmful-output testing, oversight-bypass attempts, and edge-case failure testing, not just cybersecurity probing.
- An AI incident is an event where the system causes harm, violates law or policy, falls outside expected performance, or raises significant fairness/rights concerns.
- The incident response sequence is: Detect → Contain → Assess → Remediate → Report → Learn.
- BoK-named incident root causes: brittleness, lack of robustness, lack of quality data, insufficient testing, model drift, data drift.
- Cross-functional collaboration for root-cause analysis is a BoK-explicit requirement, not optional.
- Public disclosures include: technical documentation (for regulators), instructions for use (for deployers), post-market monitoring plan, model card, user transparency notices.
- Governance is continuous, from use-case definition through retirement. Deployment is the beginning of a new governance phase, not the end.
11. Glossary
Bias testing: Evaluation of whether a model produces systematically different outcomes for identifiable subgroups, used to assess fairness.
Brittleness: The property of an AI system that performs well under normal conditions but fails in unexpected or edge-case situations.
Concept drift: A change in the real-world relationship between inputs and the outcome the model was trained to predict, causing performance to degrade over time.
Conformity assessment: A pre-market evaluation that a high-risk AI system meets the requirements of the EU AI Act. May be self-assessed or performed by a notified body.
Data drift: A change in the statistical distribution of inputs a deployed model receives, relative to the distribution it was trained on.
Data governance (for AI): The set of policies, processes, and controls ensuring that data used to train and operate AI systems is lawfully obtained, documented, of sufficient quality, and fit for purpose.
Data lineage: A documented record of where data came from, how it was transformed, and how it flows through the AI system.
Data provenance: The origin and history of data, who collected it, when, under what conditions, and with what rights.
Ethics by design: The practice of embedding fairness, transparency, human-oversight, and other ethical requirements into AI system design specifications from the outset.
Impact assessment (AI): A structured evaluation of an AI system's potential effects on individuals, groups, and the organization, used as a governance gate before and during development.
Incident response: The structured process for detecting, containing, assessing, remediating, reporting, and learning from AI incidents.
Interpretability: The degree to which the internal mechanisms or outputs of an AI model can be understood and explained by humans.
Model card: A structured document accompanying a model at release, summarizing its intended use, training data, performance, limitations, and ethical considerations.
Model drift: See concept drift.
Overfitting: A condition in which a model learns the training data too well, including its noise and idiosyncrasies, and performs significantly worse on new, unseen data.
Periodic assessment: A scheduled, deep-dive review of an AI system's performance, governance, and safety, including audits, red teaming, threat modeling, and security testing.
Post-market monitoring: Ongoing collection and analysis of AI system performance and incidents after deployment, with corrective action where required.
Probability/severity matrix: A tool for prioritizing risks by assessing each identified risk across two dimensions, the likelihood that it will occur and the severity of harm if it does.
Red teaming: Adversarial testing in which a team actively attempts to find failure modes, bias, safety gaps, and ways to manipulate or misuse an AI system.
Risk mitigation hierarchy: An ordered approach to reducing risk, eliminate, substitute, engineering controls, administrative controls, accept residual risk.
Technical documentation (EU AI Act): Comprehensive documentation required of high-risk AI providers, demonstrating compliance with EU AI Act requirements, addressed primarily to regulators.
Train/validation/test split: The division of a dataset into separate subsets for training the model, tuning it during development (validation), and evaluating final performance (test). The test set must remain unseen until final evaluation.
Validation testing: Evaluation of model performance on a held-out validation set during development, used to check generalization and guide iteration.