China’s Baidu claims Ernie outperforms ChatGPT in some metrics

Baidu's Ernie chatbot is claimed to have surpassed ChatGPT and GPT 4 in Chinese language tests and comprehensive ability scores.

Ernie Bot is based on Baidu's foundational AI model called Ernie. (Express image)

Listen to this article

00:00

1x 1.5x 1.8x

Baidu, the Chinese search engine giant, has announced that its Ernie Bot has outperformed OpenAI’s ChatGPT in a few areas.

Ernie Bot is a chatbot that’s based on Baidu’s foundational AI model called Ernie, which has been trained on a massive dataset of text and code. The latest iteration, Ernie 3.5, is claimed to be more capable than previous versions in a number of tasks, including creative writing, Q&A, reasoning, and code generation.

Citing a recent test conducted by the China Science Daily journal, Baidu has claimed that Ernie 3.5 surpassed GPT-3.5-based ChatGPT and even GPT 4 — OpenAI’s latest and more advanced model — in Chinese language tests and “comprehensive ability scores.”

One test cited by the science journal compared Ernie 3.5 with ChatGPT and GPT 4 on standard admission and qualification exams, such as those for college or law school entrance. Ernie 3.5 outperformed both models in Chinese.

However, the chatbot still lagged behind GPT 4 in English, even as it surpassed ChatGPT, in another test released by Microsoft.

Also read | Spotify, Snapchat & LinkedIn: How generative AI is taking over your favourite mobile apps

A second test assessed Ernie 3.5 on over 13,000 multiple-choice questions in Chinese, covering more than 50 different subjects. Ernie 3.5 scored higher than ChatGPT and GPT 4 in this evaluation.

Baidu’s CTO, Haifeng Wang, said that the company’s latest model has achieved “broad enhancements in efficacy, functionality, and performance.” He added that Ernie 3.5 is now able to access “plugins” which are basically apps that can be accessed via the chatbot. Baidu’s search feature has been integrated with Ernie Bot via a plugin, for example.

Story continues below this ad

The advances made by Baidu’s Ernie Bot underscore the intense competition taking place in the area of generative AI. Both China and the United States are investing heavily in the development of these technologies, and the race to create the most capable chatbot is heating up.