The MIT-IBM Watson AI Lab uses what's called generative adversarial network (GAN) models, a popular AI technique which can be observed in deepfakes. GAN works by getting two neural networks to compete each other in order to produce an acceptable outcome: a generator, which looks at various examples and tries to mimic them, and a discriminator, which judges if they are real by comparing them with the same training examples. In the case of this face filter, they used 45,000 classical portraits to train the program, including paintings Titian, van Gogh, and Rembrandt.
Previous AI methods have created similar AI photo filters, but this algorithm does not merely “paint over” your face in a new style. GAN generates new features from scratch and certain elements in the photo prompts the algorithm on which style to use. Researchers say the algorithm “decides upon a Renaissance painting style, highlighting the elegance of the aquiline nose, the smoothness of the forehead.”
“With AI Portraits Ars anyone is able to use GAN models to generate a new painting, where facial lines are completely redesigned. The model decides for itself which style to use for the portrait. Details of the face and background contribute to direct the model towards a style. In style transfer, there is usually a strong alteration of colors, but the features of the photo remain unchanged. AI Portraits Ars creates new forms, beyond altering the style of an existing photo.”
You might notice that in these portraits stars flashing their pearly whites don't appear in paint - there is a reason for that. “Portrait masters rarely paint smiling people because smiles and laughter were commonly associated with a more comic aspect of genre painting, and because the display of such an overt expression as smiling can seem to distort the face of the sitter,” they write. “This inability of artificial intelligence to reproduce our smiles is teaching us something about the history of art.”
With anxiety over privacy implications after the recent FaceApp debacle, people might be less inclined to play with their portrait. The researchers promised users that the pictures uploaded are immediately deleted after processing by their servers and that they won’t be used for any other purpose other than some really old-fashioned fun.
