AI themed week: Deepfakes could be used to incite genocide, Mandela warns

South African human rights activist Siyabulela Mandela has warned that AI-generated deepfakes could spark a civil war or genocide in areas of Africa rife with ethnic tension – on a scale bigger than the massacre that occurred in Rwanda during the 1990s.

For some time, cyber security experts have been warning enterprises and governments about the dangers of bad actors using smart technology to edit voices, faces and digitally manipulating media to make it seem like someone said or did something that they didn’t do.

However, speaking to TechInformed after a panel at Tech BBQ which explored the role of media and technology in human rights activism, Mandela – whose great grandfather was anti-apartheid activist and president of South Africa Nelson Mandela – warned that the disinformation spread by deepfake voices and images on social media could have a catastrophic impact in some regions of Africa.

“The dangerous thing about the spread of deepfakes is that they are not easy to track because once it hits WhatsApp it can be forwarded to as many people as it possibly can and it is not easy to trace who the original author was,” he said.

“Think of the impact and the damage it could cause. You could rally a particular ethnic group to attack another, in the split of a second,” he added.

Mandela – who trains local journalists across Africa to report on human rights issues for Canadian organisation Journalists for Human Rights (JHR) – compared the potential damage digital disinformation could cause to the Rwandan genocide against Tutsis in the mid 90s.

In 1994, long-standing ethnic tensions between the Hutu and Tutsi people boiled over and saw neighbours turn against neighbours, leading to the slaughter of over 1m people in the space of a few months.

During the slaughter a pro-genocide radio station played a significant role in inciting the killings and dehumanising the Tutsi population, frequently referring to them as “cockroaches”.

Mandela warned: “In Rwanda they used radio to order an attack – now they could use WhatsApp, Facebook, Instagram, YouTube.”

“It could start a civil war. Within the space of 100 days almost one million people were killed in Rwanda – and that was 1994. I mean, 29 years later imagine with the technology we have, if we do not know how to regulate that, it could be weaponised as a tool to rally people against each other.”

In his role as a JHR trainer Mandela teaches journalists in conflict zones and unstable African countries how to deal with misinformation and report on human rights issues, with an emphasis on the rights of women and girls.

Recent deepfake images that have prompted the warning of AI risks includes the fake arrest of Donald Trump in April.

In the UK meanwhile, the Electoral Commission’s director of regulation Louise Edwards, warned that deepfakes could be used to impact elections.

Her concern was in part prompted by the Turkish elections in May, when the opposition candidate Kemal Kilicdaroglu accused Russia of circulating deepfake and defamatory material on social media to influence the outcome.