Researchers used Claude to hack OpenAI

We’ve heard of OpenAI’s AI agents running amok and hacking other companies. Now, a cybersecurity company has turned the tables on the ChatGPT operator by using AI to help hack OpenAI itself.

The hack, which also exposed a bug affecting dozens of other major online services, was conducted as security research. OpenAI paid the researchers for reporting a flaw in its systems through its bug bounty program.

Researchers at cybersecurity tools vendor Hacktron wrote up their adventures in mid-September. A few months earlier, they had begun looking for security flaws at companies developing frontier AI models, which are highly capable models such as those powering ChatGPT and Claude.

Using Anthropic’s Claude, the researchers went from investigating an image-processing flaw to accessing an internal OpenAI software repository in less than 72 hours. They deliberately avoided viewing sensitive information.

To get inside OpenAI, researchers Harsh Jaiswal, Mohan Pedhapati, and Rahul Maini found two vulnerabilities and chained them together. The first wasn’t specific to OpenAI. It involved a bug in an image-upload feature in the Discourse community forum software.

This feature processes images uploaded by users and relies on a low-level software library called libheif. Uploading a specially crafted image could trigger a flaw in the library, allowing an attacker to gain control of the Discourse server.

After Hacktron used that vulnerability to compromise OpenAI’s Discourse server, the second vulnerability came into play. This was a flaw in OpenAI’s single sign-on (SSO) system, which lets people access one service using an account from another.

The SSO flaw gave Hacktron access to the ChatGPT and Codex accounts of people who had logged in to OpenAI’s Discourse forum. OpenAI uses the forum for community support, so that potentially covered a large number of accounts. Codex is an AI coding tool that helps software developers work with code.