AI chatbots are giving out people’s real phone numbers
LLM-based PII leakage is an emerging risk class agencies must account for when deploying or procuring AI tools that process personal information.
Key points
- AI chatbots including Gemini and ChatGPT are exposing real personal phone numbers drawn from training data.
- DeleteMe reports a 400% rise in customer queries specifically referencing generative AI tools exposing personal data.
- PII leakage from LLMs is directly relevant to APS obligations under the Privacy Act and responsible AI policy.
Implications for Australian agencies
- Consider Agencies deploying or evaluating LLM-based tools could assess whether their privacy impact assessments adequately account for PII leakage from training data as a distinct risk vector.
- Monitor OAIC and privacy-focused policy teams may want to monitor how regulators in other jurisdictions respond to this emerging class of AI-driven PII exposure, as it may prompt guidance updates.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
Appeared in:
Weekly digest, 11 May 2026
"AI chatbots are giving out people’s real phone numbers"
Source: MIT Technology Review – AI
Published: 13 May 2026
URL: https://www.technologyreview.com/2026/05/13/1137203/ai-chatbots-are-giving-out-peoples-real-phone-numbers/
MIT Technology Review reports documented cases of AI chatbots—including Google's Gemini—surfacing real personal phone numbers and other PII, likely due to inclusion of such data in model training sets. The privacy data removal firm DeleteMe reports a 400% increase in customer queries about AI-related personal data exposure over seven months, with ChatGPT, Gemini, and Claude most frequently cited. Experts note that the mechanism is poorly understood and there are currently few effective remedies. The pattern covers both accurate PII retrieval and generation of plausible but incorrect contact information attributed to real individuals.
Implications for Australian agencies:
- [Consider] Agencies deploying or evaluating LLM-based tools could assess whether their privacy impact assessments adequately account for PII leakage from training data as a distinct risk vector.
- [Monitor] OAIC and privacy-focused policy teams may want to monitor how regulators in other jurisdictions respond to this emerging class of AI-driven PII exposure, as it may prompt guidance updates.
Retrieved from SIMS, 18 July 2026.