New Scientist - 26.10.2019

8 | New Scientist | 26 October 2019

FAKE videos created by artificial

intelligence are now so good that

film-makers are taking note.

Israeli tech firm Canny AI is one

of several companies cashing in

on so-called deepfakes, using the

technology to edit videos into

different languages. The firm

is currently using its AI to dub

advertisements and messages

from celebrities for audiences

in different countries. It plans to

use the technology for television

shows and films in the future.

Deepfakes make it easy for

people with a bit of technical

know-how to create fake videos.

So far, they have mostly been

used to make pornographic films

involving celebrities or to create

videos where well-known figures

appear to say something that they

haven’t. Canny AI, for example,

created a satirical deepfake

of Facebook founder Mark

Zuckerberg that went viral in June.

The firm’s technology requires a

voice actor to provide replacement

audio, unlike other algorithms

that have learned to synthesise

convincing fake speech, such as

that of UK-based company Faculty,

which has generated Donald

Trump’s voice. Canny AI’s system

needs about a minute of speaking

footage of both the person being

deepfaked in the video and the

voice actor saying the words

that will be edited in.

The system learns to transfer

the movements of the lower half

of the dubber’s face and neck into

the video being edited. It works for

both footage where the speaker is

facing the camera and footage

where they are side-on. The result

is a video in which the subject

looks and sounds like they are

saying the new dialogue.

The algorithm is trained on

the footage scene by scene, so

dubbing five languages into one

scene requires less processing

power than dubbing for five

separate scenes, says Omer Ben-

Ami, co-founder of Canny AI.

UK tech firm Synthesia also

offers a dubbing AI, based on

facial mapping. Its deepfakes

were behind a malaria awareness

campaign video in which David

Beckham appeared to speak in

nine languages.

Using such deepfake AIs

commercially brings up legal

questions, says Lilian Edwards

at Newcastle University in the

UK. “There’s an underlying issue

there about what parts of yourself

you own,” she says. “Do you own

your face, do you own your image,

do you own the voices coming

out of your face?”

These are questions that

companies and celebrities will

have to consider when entering

into contracts, says Edwards.

“Every client we’re working

with has to declare they are

responsible for copyrights and

liability issues,” says Ben-Ami. The

AI is an example of how deepfakes

can be used positively, he says.

“Any technology is a double-

edged tool,” says Edwards. The

danger is that as AI video editing

becomes more user-friendly

and widespread, people could

use deepfakes “to put words

in the mouths of politicians,

public figures, people they hate,”

ALLSTAR PICTURE LIBRARY/ALAMY STOCK PHOTO she says.  ❚

News

Italian Oscar-winning

film Life is Beautiful was

dubbed in English in 1999

Machine learning

AI could count how

many people are

in large crowds

ARTIFICIAL intelligence may

be able to settle the debate over

how many people attend protests

or gatherings.

Vast numbers of people took to

the streets of London last weekend

to call for a second referendum

on the UK’s membership of the

European Union. But exactly how

many people were there is disputed.

Protest organisers say there were

1 million people, but when similar

claims were made earlier in

the year, they were disputed

by fact-checking organisations.

A method developed by

Reza Bahmanyar at the German

Aerospace Center and his colleagues

that uses artificial intelligence

could improve counts in the future.

To create the system, the

researchers hand-counted nearly

a quarter of a million people in 33

images taken from planes, drones

and helicopters, then used this to

train an algorithm called MRCNet.

MRCNet divides each image into

small squares and analyses how

many people are in each one.

The results are better than

other AI-powered crowd estimation

systems, proving 15 per cent more

accurate than the nearest competitor

at reaching the correct number

in a crowd. The system is much

faster than hand counting, taking

0.03 milliseconds to compute the

number of people in each square

(arxiv.org/abs/1909.12743).

At present, the researchers have

used the AI only in lab conditions,

but Bahmanyar hopes to soon

mount the system onto planes and

helicopters to do real-time counts.

“I think in the right places, this

technology could be really useful,”

says Keith Still at Manchester

Metropolitan University, UK,

who invented one of the best

current manual methods for

estimating crowds.

However, Still questions whether

protest groups or governments

actually want the real figures.

“They want marketing numbers.

Will they invest in something that

punctures their claim?”  ❚

Chris Stokel-Walker

Technology

Dubbing with deepfakes

Artificial intelligence is so good at manipulating videos, companies

are using it to dub adverts and messages, reports Donna Lu

“ The team hopes to soon

mount the system onto

planes and helicopters

to do real-time counts”