OpenAI unveils GPT-4o, a powerful free-for-all AI model with vision, text, and voice

The desktop app for ChatGPT showcased during the event was intended to make accessing ChatGPT easy, simple, and easy to integrate into one’s workspace. Besides, the refreshed UI according to Murati has been aimed at making interactions more natural and allowing users to focus more on the collaborative aspect of ChatGPT. Murati said that AI models are becoming increasingly complex, and OpenAI wants its users to have more natural and engaging interactions with AI models.

 Introducing GPT-4o

GPT-4o offers GPT-4 level intelligence and it is much faster and improves its capabilities across text, vision, and audio. Describing the new model, Murati said that it was the first time that OpenAI was making a huge step forward when it came to ease of use. GPT-4o makes human-to-machine interaction much more natural and far easier.

The voice mode on GPT-4o is efficient and intuitively recognises the voice of the speaker or multiple speakers. Until now, the voice mode had three models that came together to deliver the feature. These are – transcription, intelligence, and text-to-speech and they all came together and orchestrated to deliver the voice mode. This led to latency, however, with GPT-4o all of this happens natively. “GPT-4o reasons across voice, text, and vision,” Murati said.

At present, over 100 million people use ChatGPT to learn, create, and work. Until now, the advanced tools were only available to paid users. With GPT-4o, its advanced efficiencies all users can now experience OpenAI’s powerful tools. Starting today, users will be able to use GPTs from the GPT store, essentially gaining access to over a million GPTs. This will open more possibilities for developers as they will have a much bigger audience.

GPT-4o also has a vision, allowing users to upload photos and documents, and can start conversations about the same. One can also use the Memory feature, and browse to search real-time information during conversations. Also, OpenAI has improved the quality and the speed in 50 different languages.

GPT-4o capabilities

During the event, OpenAI’s research leads Mark Chen and Barret Zoph showcased some interesting live demos. They started with real-time conversations with ChatGPT where it was shown how the model can pick up on a user’s emotions, and come up with different kinds of emotive styles of conversation. While demonstrating the Vision capabilities, Zoph wrote a math problem in linear equations asking ChatGPT to solve it in a step-by-step way. The chatbot solved the problem with ease while Zoph wrote the problem while conversing.

ChatGPT desktop app also was demonstrated. The duo asked ChatGPT to help them understand some coding queries. The bot helped them and also came up with a one-sentence overview of complex charts. Besides it is also capable of telling about your feelings by looking at your face in real-time. Murati and Chen also demonstrated the live real-time translation capabilities of ChatGPT.

GPT-4o will be rolled out in iterative deployment in the next few weeks. GPT-4o is also available to API  and it is two times faster, 50 per cent cheaper, and 5 times higher rate limits compared to GPT-4 Turbo.