Thanks to AI advancements in smartphone technology, boring 2D photos in your gallery could soon be history. A new Google Photos feature will soon allow you to view the existing 2D images in your gallery as 3D images. The app’s algorithms will add simulated depth information to your flat pictures using AI, and add a smooth panning motion to the image, giving it sort of a moving, life-like feel. The new ‘Cinematic Photos’ feature by Google Photos, appearing in the app’s Memories section, is a prime example of that.
How does Google Photos pull this off?
The essential differentiator between 2D and 3D photographs is depth information. In non-technical language, depth information is what the camera captures along with the photo, to let post-processing software know various layers of a photograph from each other. This is used by software for certain effects. The background layer could be blurred out to give the subject a lens blur effect.
Alternatively, it can also be entirely separated from the subject/foreground in a way that both layers move independently. This allows panning over the photos with either your mouse or by moving your mobile device around to create a 3D like effect, similar to how the various layers would move at a different pace if they were in front of your eyes.
“We use machine learning to predict an image’s depth and produce a 3D representation of the scene—even if the original image doesn’t include depth information from the camera,” Jamie Aspinall, a Google Photos product manager in a recent blog post. Machine learning allows Google to basically have a good look at your picture, compare it with the abundance of existing 3D pictures in its database, and split the various layers in your 2D photos, based on clever estimation.
Is Google Photos the only platform to do this?
Well, no. The Facebook app has also been doing exactly this since for a while now. After the feature became popular on Google’s Pixel Phones, Facebook aimed to bring 2D to 3D image conversion to the masses with its own app, completely free of charge in early 2020. Notably, the app would allow the benefits to be used by phones with only single camera sensors.
How Facebook do it?
When there is no depth map in pictures, the Facebook app uses algorithms to estimate the distance of all pixels from the camera sensor using four techniques. These allow the app to distinguish a subject in pictures from its foreground.
The first is a network architecture created with neural building blocks optimised for mobile devices. The second is an automated architecture search that can quickly find particular configurations of these blocks in under a second. The third is a large repository of public 3D photos, from where data is sourced. Lastly, quantisation-aware training is implemented, which makes use of high-performance INT8 quantisation on mobile devices to minimise the loss of quality during the pseudo-depth making process.
As per Facebook’s support page, any iPhone launched after the iPhone 6S (including the iPhone 6S itself and the iPhone SE) should be able to make use of the feature. On the Android side, any decently powered flagship phone, along with recent upper mid-range devices, should do the trick.
Google and Facebook aren’t the only players in this game
If you are not a fan of venturing further into the Google or Facebook ecosystem, and are willing to pay for in-app purchases, there are other apps that do this as well. LucidPix 3D photo creator is one popular example. Apart from adding depth to 2D images, turning them into pseudo-3D images, the app also enables adding 3D frames and text to your photos, which you can then share to Facebook, Instagram and other social media platforms.
As we see multiple camera sensors quickly becoming a norm for new phones, 3D pictures may soon be a bigger deal. Now with both Google and Facebook in the race for 3D-dominance, don’t be surprised to see more features to make your existing pictures pop out in the near future.
Express Tech is now on Telegram. Click here to join our channel (@expresstechie) and stay updated with the latest tech news.