Evaluating the Social Impact of Generative AI Systems in Systems and Society
A structured, expert-validated social impact evaluation framework for generative AI - useful reference for APS teams building or reviewing AI risk assessment approaches.
Key points
- MIT AI Risk Repository spotlights a 2023 framework for evaluating generative AI social impacts across eleven categories.
- Framework covers both technical system-level evaluation and broader societal impacts, with modality-specific mitigation guidance.
- Useful reference material for agencies developing AI risk or impact assessment frameworks, though not APS-specific.
Implications for Australian agencies
- Consider Agencies developing or reviewing AI impact assessment frameworks could consider whether this framework's eleven-category structure complements existing APS risk and ethics guidance.
- Monitor Policy teams may want to monitor the MIT AI Risk Repository's broader catalogue of frameworks as a consolidated reference for international AI risk evaluation approaches.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
"Evaluating the Social Impact of Generative AI Systems in Systems and Society"
Source: MIT AI Risk Repository – Blog
Published: 27 February 2025
URL: https://airisk.mit.edu/blog/evaluating-the-social-impact-of-generative-ai-systems-in-systems-and-society
The MIT AI Risk Repository has spotlighted a 2023 academic paper by Solaiman et al. that proposes a structured framework for evaluating the social impacts of generative AI systems. The framework organises evaluation across two domains: six categories applicable to the technical base system (including bias, privacy, disparate performance, and environmental costs) and five categories applicable to broader societal contexts (including inequality, labour displacement, and ecosystem effects). Each category includes modality-specific guidance on what to evaluate, limitations of current evaluative techniques, and harm mitigation recommendations. The framework was developed through expert workshops and is included as one of thirteen risk frameworks catalogued in MIT's AI Risk Repository.
Implications for Australian agencies:
- [Consider] Agencies developing or reviewing AI impact assessment frameworks could consider whether this framework's eleven-category structure complements existing APS risk and ethics guidance.
- [Monitor] Policy teams may want to monitor the MIT AI Risk Repository's broader catalogue of frameworks as a consolidated reference for international AI risk evaluation approaches.
Retrieved from SIMS, 18 July 2026.