ML Governance — Prerequisites
“Model governance is the process of assigning and assuring organizational accountability, decision rights, risks, policies, and investment decisions for applying AI.”
Just as Organizations have long treated data as assets, as ML becomes more central to an organization’s operations, models will need to be treated as important assets.
Although modeling approaches may differ, there are a set of development and management phases that all models go through in their lifecycle. Formalizing this lifecycle allows specific actions for successful development, management, and governance.
The Risk assessment at the beginning of the Model Dev-Ops Lifecycle is key to address the level of Governance, risk mitigation required throughout the life cycle stages. Governance does not necessarily mean command and control.
General Model Information
Use Case: Brief description of the business objective for the AI / ML project
Type of Project: Research or Production deployable. Will the model be deployed into production or is it a one-off research effort
Return on Investment / Risk: Potential Upside returns and potential downside risk, worst case downside. Helps in prioritizing and relevant guard rails
Minimum Acceptance Criteria: metrics to consider for the model to be deployment-ready. Definition of success of the Model in its application
Decision matrix: Does the Model make consequential decisions — Regulatory, Compliance, Financial, reputational…
Audit requirements: Does the Model require independent Auditable capability — explainability / Interpretability
Financial Impact: Business and financial impact of Model predictions
Ethics: Requirements for Fairness / Bias, AI Imitating Humans, Predictive Policing
Other Considerations: Risk of Bad press, Privacy, and security
Model Stakeholders: Identification of relevant Model Stakeholders — Model Owner/approver, Model Risk Manager, Business Owner, Data Owner / Steward, Platform Ops team, Data Ops team, ML Ops….
Model deployment requirement: Where will this be deployed — Cloud, edge
Model Consumption requirement: Batch AI, real-time AI
Closed world vs Open world Deployment: Deployed for a smaller group vs. company-wide use vs. more global use
Human in the loop: fully automated vs. Human in the loop
Data Requirements
The success of a Model depends on the signal inherent in and extracted from the data. Ensuring the quality, availability, security, privacy requirements…..
Data requirements: all data requirements — short, medium and long term identified
Data Owner Approval: Approval for use of data for model development and considerations documented
Data Availability: where is the data available for the Model Development — Source systems / Data Platform / Lake…
Data Accessibility patterns: How will be data be accessed — in-situ, data copy
Data Privacy / Sensitivity: The sensitivity of the Data for the model to be addressed. Use of PII data, Secret data for model training.
Data Security: Authentication and authorization to use the data
Data Volatility: Data change over time: processes to be considered
The recommendation is to start building scorecards in each of the relevant categories to address the level of Governance requirements to safely and securely operationalize trusted models.
