By Sophie Haigney
At the Betty Cuningham gallery on New York’s Lower East Side recently, I noticed an arresting painting: It showed a nude woman curled against a window, asleep, with the old New Yorker Hotel and Empire State Building in view and a fish above her, hanging or floating. I opened a smartphone app called Magnus, snapped a quick picture, and clicked “Use.” Seconds later, I got that addictive, satisfying click. The app had found a match.
The painting was by Philip Pearlstein, according to the app, known for reinvigorating the tradition of realist figure painting. It was titled “Model With Empire State Building.” dated 1992, measured 72 inches by 60 inches, and was for sale for $300,000. In 2010, it had sold for $170,500 at Sotheby’s in New York, the app told me. Magnus then slotted this information into a folder marked “My Art” for digital safekeeping — and future looking.
Magnus is part of a wave of smartphone apps trying to catalog the physical world as a way of providing instantaneous information about songs or clothes or plants or paintings. First came Shazam, an app that allows users to record a few seconds of a song and instantly identifies it. Shazam’s wild success — it boasts more than 1 billion downloads and 20 million uses daily, and was purchased by Apple for a reported $400 million last year — has spawned endless imitations. There is Shazam for plants or Shazam for clothes and now, Shazam, for art.
The art-oriented apps harness image recognition technology, each with a particular twist. Magnus has built a database of more than 10 million images of art, mostly crowdsourced, and aims to help prospective art buyers navigate the notoriously information-lite arena of galleries and fairs.
Other apps are geared toward museumgoers: Smartify, for example, takes an educational approach, teaming up with museums and sometimes galleries to upload digitized versions of their collections, wall texts, and information about artists. Google Lens — Google’s advanced image recognition technology — is making new forays into the art world. In June, Google Lens announced a partnership with the de Young Museum in San Francisco to show parts of the museum’s collection. In July, Google began collaborating with Wescover, a platform oriented toward design objects, public and local art, furniture, and craft — enabling you to learn the name of that anonymous painting in your WeWork space or coffee shop.
There are some barriers particular to creating a Shazam for art. Magnus Resch, founder of the Magnus app, laid out one: “There is a lot more art in the world than there are songs.” Cataloging individual artworks based in unique locations is far more difficult.
Copyright law also poses challenges. The reproduction of artwork can be a violation of the owner’s copyright. Magnus contends that because the images are created and shared by users, the app is protected by the Digital Millennial Copyright Act. Galleries and competitors, Resch said, complained about the uploading of images and data to the app; in 2016, it was removed from the Apple Store for five months, but Apple ultimately reinstated Magnus after some disputed content was removed.
Another issue is that image recognition technology still often lags when it comes to identifying 3D objects; even a well-known sculpture can baffle apps with its angles, resulting in the deflating, endless spin of technology that’s “thinking” ad infinitum.
Then there is a more salient question for these platforms: What information can an app provide that will enhance the user’s experience of looking at art? What can a Shazam for art really add?
Resch’s answer is simple: transparency. Galleries rarely post prices and often don’t provide basic wall text, so one often has to ask for the title or even the artist’s name.
Jelena Cohen, a brand manager for Colgate-Palmolive, bought her first artwork, a photograph, at Frieze after using Magnus. Before trying the app, she said, the lack of information was a barrier. “I used to go to these art fairs, and I felt embarrassed or shy, because nothing’s listed,” Cohen said. “I loved that the app could scan a piece and give you the exact history of it, when it was last sold, and the price it was sold for. That helped me negotiate.”
Magnus doesn’t give you an art history lesson, or even much of a basic summary about a work; like Shazam, it’s a little blip of information in the dark. Smartify, on the other hand, wants to app-ify what was once the purview of an audio guide. Hold it up to a Gustave Caillebotte still life, as I did, and the app provides information that’s already available on the wall, including the chance to click-to-learn-more. Part of the app’s mission is ease of use and accessibility. People with visual impairments can use Smartify with their phones’ native audio settings and the app is working to integrate audio. The app is elegant and straightforward, and the source is generally cited and fact-checked.
Smartify’s major limitation is that because the app teams up directly with museums, it only works well in a few places. London’s National Gallery, where I tested it, was one of them; it didn’t miss a single painting in the permanent collection. But at the Met, where Smartify has uploaded a limited set of images, I spent a frustrating afternoon waving the app at paintings as it failed to return even facts that I could read in the wall texts.
It’s telling, perhaps, that even as these apps build out their databases, some museums themselves are starting to shy away from apps altogether. The Metropolitan Museum, which rolled out its own app with fanfare in 2014, shuttered it last year.
“While the app was doing a lot of things well, we wanted to create something more seamless,” said Sofie Andersen, the interim chief digital officer at the Met. This translates into content that loads directly in your phone browser as a website, no download required. Similarly, the Jewish Museum introduced a new set of audio tours in July, all on a web-based interface.
“A few years ago, there was an app craze, and now everyone’s entering this post-app phase in the museum industry,” said JiaJia Fei, director of digital for the Jewish Museum. She noted that the vast majority of apps that people download sit unused on their phones. “You just end up using your email and Instagram.”
After a few weeks of trying out apps-for-art in museums and galleries, on street corners and in the occasional coffee shop, I found that they did not increase the quality of my visual encounters. Although the caliber of information in Smartify is quite high when it works — I was able to learn more about specific figures in J.M.W. Turner’s “Ulysses Deriding Polyphemus” — the simple act of raising my phone to take a picture transformed a vibrant physical painting into a flattened reproduction. The extra information wasn’t worth mediating my museum experience through a screen.
And phones are already everywhere in museums, transforming a visit into cataloging as we go. Fei referred to this as “screen suck,” and it’s one reason audio is the preferred medium for the Jewish Museum. Like Shazam itself, the apps are best used for quick answers — a lifeline in a contextless gallery. What is that? How much does it cost? Who made it? (Here, Magnus is the leader.)
The Shazamification of art is a product of a time in which information overpowers the naked eye. But the app shouldn’t be our sole guide through the visual world. Walking around the New Museum with the Magnus app, I found myself breezing past paintings, not looking too hard at details because the camera was looking for me, and the app knew much more than I did. There was that little addictive, satisfying click of recognition. It was hard to stop.