Model Risk Operating Model
Following an exam finding, lineage, version, and review status for 24 production models moved from four disconnected spreadsheets into one registry — making model risk a property of the model, not a periodic project.
The challenge
A regulated consumer lender's 24 production models were governed in name only:
- 24 production models — credit scoring, pricing, fraud, collections, marketing — were tracked across four disconnected spreadsheets
- An exam found only 11 of the 24 had a documented review within the required 12-month window
- Model lineage — training data, feature versions, retraining history — lived in individual data scientists' notebooks, not a central record
- Version control was informal, so the model actually in production wasn't always the one that had been approved
- A four-person model risk team couldn't keep pace with retraining and redeployment across 24 models by hand
How it works
Making model risk a property of the model, not a periodic project
The remediation had to outlast the exam response, so governance was built into the pipeline itself:
- 01
Built a central model registry capturing every production model, its owner, and its current version
- 02
Wired lineage capture into the training pipeline so data sources, feature versions, and training runs log automatically
- 03
Tied deployment to the registry, so a model can't go to production without a matching approved entry
- 04
Set standing review cadences per model tier, with automatic flags when a review falls due
- 05
Migrated all 24 existing models into the registry, reconstructing lineage history where it was missing
- 06
Gave examiners direct, read-only access to the registry instead of a point-in-time inventory document
What we built
Key capabilities
One registry, 24 models
Every production model, owner, and version lives in a single system instead of four spreadsheets.
Lineage captured automatically
Training data, feature versions, and training runs log themselves as the pipeline executes.
Deployment gated on approval
A model cannot reach production without a matching, approved registry entry.
Review cadence enforced by system
Standing review schedules per model tier flag automatically when a review falls due — not by memory.
Before vs after
What changed in model governance
- Models with current annual review
- 11 → 24
- Model inventory
- 4 spreadsheets → 1 registry
- Lineage capture
- Manual, in notebooks → Automatic, in pipeline
- Deployment control
- Informal → Tied to registry approval
- Examiner access
- Point-in-time document → Live, read-only registry
Business impact
What it changed
24 of 24 models now current
Every production model carries a documented annual review, up from 11 of 24 at the time of the exam.
Lineage generated, not reconstructed
Training data and version history are captured automatically as models train and deploy, not assembled for the next exam.
Deployment gap closed
Production deployment is now gated on registry approval, directly addressing the control gap the exam originally flagged.
Technology stack
“Model risk management stopped being a document the team updated before an exam and became a control the pipeline enforces every day.”
Keep reading