This is an archive article published on May 8, 2024
Premium
OpenAI releases ‘Deepfake’ detector to disinformation researchers
The prominent A.I. start-up is also joining an industrywide effort to spot content made with artificial intelligence.
A reception area at the headquarters of OpenAI in San Francisco on March 10, 2023. OpenAI is in May 2024 releasing to researchers a tool designed to detect content created by its own popular image generator, DALL-E, while acknowledges that this tool is only a small part of what will be needed to fight so-called deepfakes. (Jim Wilson/The New York Times)
As experts warn that images, audio and video generated by artificial intelligence could influence the fall elections, OpenAI is releasing a tool designed to detect content created by its own popular image generator, DALL-E. But the prominent AI startup acknowledges that this tool is only a small part of what will be needed to fight so-called deepfakes in the months and years to come.
Advertisement
On Tuesday, OpenAI said it would share its new deepfake detector with a small group of disinformation researchers so they could test the tool in real-world situations and help pinpoint ways it could be improved.
In an image provided by OpenAI, an image of a cat created by OpenAI’s Dall-E tool is assessed by the company’s “deepfake detector.” OpenAI is in May 2024 releasing to researchers a tool designed to detect content created by its own popular image generator, DALL-E. It is not designed to detect images produced by other popular generators like Midjourney and Stability. (OpenAI via The New York Times)
“This is to kick-start new research,” said Sandhini Agarwal, an OpenAI researcher who focuses on safety and policy. “That is really needed.”
OpenAI said its new detector could correctly identify 98.8% of images created by DALL-E 3, the latest version of its image generator. But the company said the tool was not designed to detect images produced by other popular generators such as Midjourney and Stability.
Because this kind of deepfake detector is driven by probabilities, it can never be perfect. So, like many other companies, nonprofits and academic labs, OpenAI is working to fight the problem in other ways.
Like the tech giants Google and Meta, the company is joining the steering committee for the Coalition for Content Provenance and Authenticity, or C2PA, an effort to develop credentials for digital content. The C2PA standard is a kind of “nutrition label” for images, videos, audio clips and other files that shows when and how they were produced or altered — including with AI.
OpenAI also said it was developing ways of “watermarking” AI-generated sounds so they could easily be identified in the moment. The company hopes to make these watermarks difficult to remove.
Anchored by companies like OpenAI, Google and Meta, the AI industry is facing increasing pressure to account for the content its products make. Experts are calling on the industry to prevent users from generating misleading and malicious material — and to offer ways of tracing its origin and distribution.
In a year stacked with major elections around the world, calls for ways to monitor the lineage of AI content are growing more desperate. In recent months, audio and imagery have already affected political campaigning and voting in places including Slovakia, Taiwan and India.
OpenAI’s new deepfake detector may help stem the problem, but it won’t solve it. As Agarwal put it: In the fight against deepfakes, “there is no silver bullet.”
This article originally appeared in The New York Times.
A reception area at the headquarters of OpenAI in San Francisco on March 10, 2023. OpenAI is in May 2024 releasing to researchers a tool designed to detect content created by its own popular image generator, DALL-E, while acknowledges that this tool is only a small part of what will be needed to fight so-called deepfakes. (Jim Wilson/The New York Times) 
In an image provided by OpenAI, an image of a cat created by OpenAI’s Dall-E tool is assessed by the company’s “deepfake detector.” OpenAI is in May 2024 releasing to researchers a tool designed to detect content created by its own popular image generator, DALL-E. It is not designed to detect images produced by other popular generators like Midjourney and Stability. (OpenAI via The New York Times)
Share
