ML Model Testing

As machine learning (ML) systems continue to take on central roles in business decisions, creating reliable, production-level machine learning systems have become increasingly more critical. Multiple research papers are focussed on effective Model Testing, Model Explainability, and Model interpretability. Nevertheless, ML testing remains at an early stage in its development, with many challenges and open questions lying ahead.

Below are a couple of papers, I came across, that provides an in-depth study that can be readily adopted as a baseline framework for ML Model testing.

This paper provides a comprehensive survey of techniques for testing machine learning systems; Machine Learning Testing (ML testing) research. It covers testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and application scenarios (e.g., autonomous driving, machine translation).

This paper discusses the difference between software testing and machine learning model testing. The paper outlines 28 specific tests and monitoring needs, present an easy to follow road-map to improve production readiness and pay down ML technical debt