Forget HDR, Google’s Ultra HDR captures unprecedented realism: How it works

What is Ultra HDR?

Ultra HDR is a new way of storing and displaying images that enhances the quality of JPEGs in a clever new way. Images in this format still have a .jpg file extension but they’re accompanied by a hidden piece of data called an HDR gain map. This data tells the device how to boost the brightness, contrast, and colour of the image.

Ultra HDR images can be viewed on any device that can show JPEGs, but they look different depending on the type of display. On a standard SDR display, Ultra HDR images look like normal JPEGs, because the HDR gain map is ignored. But on an HDR display, Ultra HDR images look much better, because the HDR gain map is used to create a more realistic image. However, this only works on devices that recognise the Ultra HDR format, which is not yet widely supported (more on supported devices and apps later).

How does Ultra HDR work?

Photos clicked in the standard dynamic range (SDR) format act like they have a single brightness slider. This means that the brightness for these images acts in a more linear manner, meaning you can either expose the bright areas or the dark areas, but not both.

To avoid this problem, standard HDR photos use a process called tone mapping, which combines multiple images taken at different exposure/brightness levels into one. This way, you can preserve the details in both the bright and the dark parts of the scene. This does have a major downside though – people often complain that HDR shots have an unnatural, overprocessed look.

(Image: Google)

That’s where Ultra HDR comes in. Ultra HDR differs from HDR as it does not rely on tone mapping. Instead, it uses gain maps — special data that tells the camera how to adjust the brightness of each pixel in the image. Gain maps are stored in the metadata of the JPEG file, so they do not affect the image quality or size.

So what does Ultra HDR look like?

Since Ultra HDR is a novel format, some operating systems, browsers, and apps don’t support it yet. It also requires an HDR display and for the images to be uploaded in an uncompressed format. That’s why it’s not easy to demonstrate what Ultra HDR can do through a website.

But that’s not going to stop us from sharing some samples and impressions. When the following daylight image is opened in Google Photos, the app takes a second to apply the gain map. Due to this slight delay, you are able to compare the standard HDR image and its Ultra HDR-enhanced version with improved contrast and life-like colours. But more importantly, the phone’s display automatically brightens areas of the photos emitting light. For example, if you check all the boxes for viewing an Ultra HDR image, just note how the sun shines in this photo and the way the car’s surface gleams.

(Image: Zohaib Ahmed/The Indian Express)

The effect is less dramatic in night shots, but still significant. The car lights in the photo below look like they’re really on and the billboards above seem to glow with their own light – something that SDR displays and even regular HDR photos cannot replicate.

(Image: Zohaib Ahmed/The Indian Express)

What do you need to get started with Ultra HDR?

For now, you can only take Ultra HDR photos on a Pixel 8 and Pixel 8 Pro with the newest version of the Camera app. Google has made sure that the format is readable by Android 14 devices, but you still need an HDR display to see the real enhancements, which is something that most mid-range handsets now support. The app you’re viewing the Ultra HDR media in also needs to support the format. Thankfully, Google Photos does so, as do Chrome and most Chromium-based browsers.

But for this format to truly catch on, social media platforms would also need to gain support for it. Ultra HDR is a ‘fragile’ format, meaning any tinkering with the image can corrupt or strip the HDR gain map from the metadata. And social media apps do alter photos by compressing them when you upload. We get it, it’s a lot, but once this format catches on, viewing Ultra HDR content should be seamless.