The Risks of Machine Learning Systems
A structured ML risk taxonomy spanning safety, privacy, discrimination, and security - directly applicable to APS AI risk assessment and assurance work.
Key points
- The MLSR framework categorises ML risks into first-order (design/development) and second-order (real-world interaction) effects.
- The taxonomy integrates algorithmic impact assessments, incident reports, and ML literature - useful for structured risk assessment work.
- This is a 2022 arXiv paper spotlighted by MIT AI Risk Repository; not new guidance but a curated reference resource.
Summary
The MIT AI Risk Repository has spotlighted the Machine Learning System Risk (MLSR) framework, a 2022 paper by Tan, Taeihagh, and Baxter. The framework distinguishes first-order risks arising from design and implementation choices (including algorithm robustness, misapplication, and emergent behaviour) from second-order risks that emerge when systems interact with the world (safety, privacy, discrimination, security, environmental, and organisational harms). The taxonomy draws on algorithmic impact assessments, software risk literature, incident reports, and professional experience, offering a structured basis for holistic ML risk assessments. It is one of fifteen frameworks catalogued in the MIT repository.
Implications for Australian agencies
- Consider Agencies developing or refreshing AI risk assessment frameworks could consider whether the MLSR first/second-order categorisation adds structure to existing approaches.
- Monitor Teams tracking the MIT AI Risk Repository may want to monitor the full catalogue of fifteen frameworks as a comparative reference for Australian risk taxonomy work.
Implications are AI-generated. Starting points, not advice.
"The Risks of Machine Learning Systems" Source: MIT AI Risk Repository – Blog Published: 23 April 2025 URL: https://airisk.mit.edu/blog/the-risks-of-machine-learning-systems The MIT AI Risk Repository has spotlighted the Machine Learning System Risk (MLSR) framework, a 2022 paper by Tan, Taeihagh, and Baxter. The framework distinguishes first-order risks arising from design and implementation choices (including algorithm robustness, misapplication, and emergent behaviour) from second-order risks that emerge when systems interact with the world (safety, privacy, discrimination, security, environmental, and organisational harms). The taxonomy draws on algorithmic impact assessments, software risk literature, incident reports, and professional experience, offering a structured basis for holistic ML risk assessments. It is one of fifteen frameworks catalogued in the MIT repository. Implications for Australian agencies: - [Consider] Agencies developing or refreshing AI risk assessment frameworks could consider whether the MLSR first/second-order categorisation adds structure to existing approaches. - [Monitor] Teams tracking the MIT AI Risk Repository may want to monitor the full catalogue of fifteen frameworks as a comparative reference for Australian risk taxonomy work. Retrieved from SIMS, 18 May 2026.