Researchers have taken another step towards realistic, synthesized video. The team, made up of scientists in Germany, France, the UK and the US, used AI to transfer the head poses, facial expressions, eye motions and blinks of a person in one video onto another entirely different person in a separate video. The researchers say it’s the first time a method has transferred these types of movements between videos and the result is a series of clips that look incredibly realistic.
The neural network created by the researchers only needs a few minutes of the target video for training and it can then translate the head, facial and eye movements of the source to the target. It can even manipulate some background shadows when they’re present. In the video below, you can see how the system does this using videos of Barack Obama, Vladimir Putin and Theresa May, among others, as examples — mouth movements, head rotations and eyebrow wiggles are all transferred between videos. The researchers even use different portions of the same video as both the source and the target and the synthesized result is nearly indistinguishable from the original video.
The team compares its method to a number of others with similar goals, including this one from last year where researchers used a neural network to manipulate an existing video to match different audio recordings. In this case, the newer method creates more realistic and accurate mouth movements than the earlier method.
While the scientists behind the work note that the method could be useful for VR, film dubbing or post-production movie work in general, there’s obviously some concern over more nefarious applications. “I’m aware of the ethical implications of those reenactment projects,” Justus Theis, one of the researchers on the project, told The Register. “That is also a reason why we published our results. I think it is important that the people get to know the possibilities of manipulation techniques.”
The work has been submitted to the annual SIGGRAPH conference taking place in August.
Via: TechCrunch
Source: arXiv
