Google Project Starline hands-on at I/O 2024: 3D calling felt like a scene from a sci-fi movie

Before I got the demo of Project Starline, I only imagined what 3D video calling might look like. Nothing special to get excited about, but Google’s Project Starline proved me wrong. I had been hearing about Google’s 3D chat booth for many years, and all the while, I considered it to be a sophisticated technology that might never come to the market.

Sitting across a table

Google calls Project Starline a “magic window” that allows users to have a video call in 3D. (Image credit: Google)

At Google’s annual developer conference in Mountain View, California, this week, when I tried Project Starline in person, it was a moment of epiphany and I wanted to spend more time. I wasn’t allowed to take photographs or film the demo, so my words will have to suffice. Here are my thoughts.

As I walked into the Project Starline demo room, I met Andrew Nartker, General Manager, Google, who has been the lead on this project. Nartker asked me to take a seat, assuring me that he would join me on a call in seconds. I couldn’t understand what he meant by that, but I complied. I took the seat, and the screen in front of me lit up, revealing Nartker. He greeted me as if we were meeting face-to-face for the first time in a normal meeting. Within the next few minutes, I completely forgot I had just met Nartker. However, Nartker had gone to the other room and was being projected in front of me through Google’s Project Starline.

We started talking, and I felt as though Nartker and I were sitting across a table. This was all possible due to a 3D effect, and I thought Nartker was popping out of the screen and he was physically there. This, of course, is the difference between a video call in 3D vs 2D. I could see Nartker’s facial expressions, his body language, and the small nuances that we observe when we meet someone in person. You and other people are no longer cut off from each other. You see their face; they see yours.

Perhaps the reason behind Nartker’s convincing appearance, even though it was virtual, is due to the technology that powers Google’s 3D video conferencing. Nartker told me Starline uses a TV-sized display and a screen called a light field display. This display can project the 3D video by beaming the various angles to the recorded footage.

The concept is very similar to how holographic cards work and can display a different image or 3D effect when you look back and forth. As a result, your right eye will see a slightly different image compared to your left eye, creating the illusion that you’re staring at an image with depth. The tech works in tandem with six cameras that record a 3D video of your face, body, and movements you show during the call from various angles.

Endless possibilities

Google gave the first look at Project Starline during Google I/O in 2021. (Image credit: Google)

During the brief demo, Starline managed to create the right illusion. However, it wasn’t perfect. I could see that Nartker was not real, but rather a recreated version of him. His face seemed pixelated, and when he moved his head, you could sense the 3D effect. That being said, there were no lag or latency issues in Nartker’s movements or visual and audio issues I have observed.

Starline is impressive and it does elevate the video calling experience in 3D, without needing to wear those weird 3D glasses. The possibilities are endless with Starline, especially in an office environment, where I feel Starline makes more sense. The main idea behind Project Starline is to have a better video conferencing solution that gives a sense of virtual presence as if you were having a conversation with someone in the same room. But does that ever beat in-person meetings? Probably not. It is if you can believe this, a three-dimensional, live, computer-generated person sitting next to you, holding an apple in hand.

Some may argue that a headset like the Apple Vision Pro would be a better device for conducting video calls in 3D. The answer is both yes and no, but right now, the technology to conduct video calling in three dimensions, and that too in real-time, does not exist. What you can do with the Vision Pro is have a FaceTime call and chat with someone’s 3D persona, not something like what Starline is trying to achieve. Another aspect is comfort; you don’t have to wear a bulky headset nor go through a long setup process like you do with the Vision Pro.

Google has been teasing Project Starline for years, and at I/O, the company announced that it is near to commercializing the technology next year through a partnership with HP. It amazes me to see the potential of Starline, but I am still not sure if offices will plan to deploy something like this, especially when companies have mandated workers to come to the office and work.

The writer is attending I/O 2024 in Mountain View, California, at the invitation of Google India.