When Martin Scorsese’s Irishman released in 2019, a lot of the conversation surrounding the film was how the production used ground-breaking to depict a younger version of Robert De Niro’s character Frank Sheeran. But now, Disney says it has developed a neural network called FRAN (face re-ageing network) which does the exact opposite—it can age characters on screen.
Disney Research Studios explains that it trained FRAN on a database of hundreds of computer-generated synthetic faces. This allowed them to generate thousands of images of people with the same angle, facial expression and lighting conditions; a herculean task if it had to be done with real images.
“Our new face re-ageing network (FRAN) incorporates simple and intuitive mechanisms that provide artists with localized control and creative freedom to direct and fine-tune the re-aging effect, a feature that is largely important in real production pipelines and often overlooked in related research work,” said Disney Research Studios in a statement.
In video examples provided by Disney, the neural network does a good job at ageing the faces of characters in a video, albeit with some artefacts. While it may seem like this algorithm could potentially replace many VFX jobs in the future, it is not likely that it will do so anytime soon. In Disney’s paper on the neural network, it concedes that the neural net has many limitations that will have to be worked on.
For one, large image changes are more difficult to generate, meaning that it would be challenging to re-age a character from a very young age to a very old age. Also, the algorithm does not capture the greying of hair, which means that the artificially aged characters will have the same hair as that in the original footage.
Also, Disney Research Studios says that the re-ageing process also changes the character’s body mass index (BMI) in a way that cannot yet be controlled. Further, FRAN does not provide means for a user to specify what facial structures—moles, wrinkling patterns and other skin signs—are to be added or removed during the ageing process.