A Flaw in OpenAI’s Codex Put 97M Monthly Downloads at Risk

96.8 million. That is how many times the command-line version of Codex, OpenAI’s coding assistant, was downloaded in the 30 days to 10 November. Security researchers say one fake bug report could have let an attacker slip harmful code into a future update of it. OpenAI has fixed the problem and paid the researchers a first reward of $1,000.

The researchers work at the security firm Depi. On 11 November they published every step of the attack. The download count comes from the public statistics of the npm software library. Depi says users of Codex have nothing to patch.

How the attack worked

OpenAI keeps the code for Codex in a public project on GitHub, where anyone can file a bug report. A helper tool read each new report and suggested labels for it. That helper was the Codex assistant itself, and it received the text of the report mixed in with its own instructions. So a writer could hide commands in a report. This trick is called prompt injection.

Depi says it got the assistant to run hidden code, broke out of the closed space it works in and read secret access keys from the computer’s memory. With those keys it planted a harmful file in a store of files that automated jobs share. A more powerful job could then run that file. The last step was a trap that would give the attacker a key the npm library accepts for publishing new versions of Codex.

What OpenAI changed

Depi says OpenAI deployed a fix right away and paid a first reward of $1,000, with the report still under review. The highest rewards for the most serious bugs reach $100,000 before bonuses. The company raised its reward rates earlier this month.

Early on 12 November, Paris time, OpenAI replaced the assistant in that labelling tool with a short script. The script asks a model for labels with its tools switched off and only accepts answers in a fixed format. The change is visible in the project’s public list of changes.

There is one caveat. Depi ran the full attack on its own test copy, where the assistant was allowed to write files. OpenAI’s own helper had used a stricter read-only mode since May. The setting appears in the version of the file from that month. Depi does not say whether the attack would have worked in that mode, and it does not report any harmful update reaching users.

Its advice for everyone else is to wait a short time before tools install new updates automatically. Last month another team received a reward of $5,000 for a flaw in shared chat links.

Why it matters: AI assistants that read text from strangers can be talked into acting on it. Here the stranger only had to file a bug report, and the target was a tool that runs code on developers’ computers tens of millions of times a month.