© IE Online Media Services Pvt Ltd
Tags:
Journalism of Courage
Is it possible to know a person’s name, phone number, home address, and other personal details just by looking at their faces? It appears so. Using a combination of advanced technology available today, such as Meta’s Ray-Ban smart glasses and large language models (LLMs), two Harvard University students were able to correctly identify someone and find out where they lived.
As part of the project, AnhPhu Nguyen and Caine Ardayfio developed a tech stack called I-XRAY comprising smart glasses, face search engines, LLMs, and public databases. In a video demo uploaded online, the two students reportedly used I-XRAY to identify several classmates, their addresses, and names of relatives – all in real time.
“Initially started as a side project, I-XRAY quickly highlighted significant privacy concerns. The purpose of building this tool is not for misuse, and we are not releasing it,” read a document by Nguyen and Ardayfio. They added that the goal of the project was to raise awareness that “extracting someone’s home address and other personal details from just their face on the street is possible today.”
First, the user wears Meta’s smart glasses and live streams a video of their surroundings to their Instagram handle. This Instagram live stream is analysed by a computer programme to identify people’s faces. Their photos are then processed using facial recognition models such as PimEyes and FaceCheck.id to match the faces with URLs or images available across the internet. “In our testing, PimEyes was the most effective and accurate to a wide range of people,” the college students said.
Once the URLs or images are matched, I-XRAY prompts an LLM to scrape relevant details such as the person’s name. With the LLM-extracted name of the person, I-XRAY queries online tools such as FastPeopleSearch that provides personal data of individuals such as their age or address by accessing publicly available records and social media profiles.
The project also found that I-XRAY can be used to unmask partial social security numbers (SSNs) on a site called Cloaked.com. “This means that if FastPeopleSearch returns an accurate phone number, we can get the first and last digit of someone’s SSN,” the paper read.
“Unfortunately, it seems most Americans have their SSN leaked in data dumps from corporation-level hacks,” it added. The personal data that has been gleaned is fed back into a phone app.
“What makes I-XRAY unique is that it operates entirely automatically, thanks to the recent progress in LLMs,” the developers said. “The [I-XRAY] system leverages the ability of LLMs to understand, process, and compile vast amounts of information from diverse sources–inferring relationships between online sources, such as linking a name from one article to another, and logically parsing a person’s identity and personal details through text,” they said, elaborating on the role of generative AI in I-XRAY.
Smart glasses have posed privacy risks in the past as well. Earlier, the Google Glass had drawn flak over the fear that people in public spaces could be recorded without their consent.
To address this concern, Meta designed its smart glasses with a privacy light that is automatically turned on whenever the wearer is recording a video or taking photos so that people are aware that they are being recorded. However, users have pointed out that the privacy light can sometimes be hard to notice, especially in the outdoors, crowded places, or in bright light.
Meta’s privacy policy cautions smart glass wearers to not “use your glasses to engage in harmful activities like harassment, infringing on privacy rights, or capturing sensitive information like pin codes.” “Show others how the capture LED works so they know when you’re recording. If the capture LED is covered, you’ll be notified to clear it before taking a photo or video or going live,” it adds.
Meanwhile, TechCrunch reported that certain photos and videos captured by Meta’s smart glasses could be used to train its AI model. “[I]n locations where multimodal AI is available (currently US and Canada), images and videos shared with Meta AI may be used to improve it per our Privacy Policy,” Emil Vazquez, Meta’s policy communications manager, was quoted as saying.
