“The Tip Of A Very Large Iceberg”

BuzzFeed News has uncovered a new network of suspected Twitter propaganda accounts – sharing messages about Brexit, Donald Trump, and Angela Merkel – that have close connections to the Russian-linked bot accounts identified by the social media platform in its evidence to the US Congress.

The 45 suspect accounts were uncovered through basic analysis of those that interacted or retweeted accounts cited by Twitter to Congress, yet none of them appeared on the company’s list.

The relative ease of discovery raises serious questions as to just how many Russian-linked bots may still be active on Twitter, how the company identifies and removes such accounts, and whether its process for identifying accounts for its evidence was inadequate.

Until BuzzFeed News approached Twitter on Tuesday afternoon with details of the accounts, they all remained active on the platform, though dormant. But within 24 hours, all 45 had been suspended.

The network of propaganda accounts was identified using a database of 17 million Brexit-related tweets collected by the University of Sheffield. They tweeted predominantly in German, and were primarily focused on jumping on German hashtag games and other trending topics, often inserting negative messages about the German chancellor, Angela Merkel.

However, 20 of the accounts also tweeted about Trump, and 21 about Brexit – with a huge spike of activity on the day of the referendum, 23 June 2016. The tweets about Trump and Brexit were often the only ones the accounts posted in English.

Damian Collins, the Conservative chair of the UK parliament’s culture, media, and sport committee – which is holding an inquiry into fake news and online propaganda – said the findings showed the information handed over by the social media platform so far was “only the tip of a very large iceberg”.

“This BuzzFeed investigation clearly calls into question the evidence that Twitter provided to the US Senate judiciary and intelligence committees and demonstrates that whatever process the company undertook to identify Russian-backed fake accounts was simply not rigorous enough,” he said.

“The findings of this report raise serious questions about the methodology used by Twitter to identify fake content on their platform, and how seriously they are taking investigations by governments and parliaments around the world.”

He added: “The technology exists to aid the identification of fake accounts based on their shared characteristics and use of language. Twitter should be deploying this to identify other accounts linked to organisations like the Russian Internet Research Agency.”

The existence of a bot network directly connected to the accounts identified by Twitter is also likely to raise concerns among US regulators and members of Congress investigating possible Russian interference in the 2016 presidential election.

Twenty of the 45 suspected bot accounts identified by BuzzFeed News tweeted about Donald Trump a total of 198 times in the summer of 2016, in a mixture of English and German. These included several tweeting a selection of bizarre memes around supporting Trump “because he is gay”, and aiming to boost the hashtag #WirLiebenTrump (#WeLoveTrump).

The data also showed that, like many of the accounts identified by Twitter to Congress, the newly discovered bots were largely ineffective in skewing public debate. Overall, the accounts achieved very few likes and retweets – the most successful message in the sample got just 15 retweets – leading one University of Sheffield researcher to refer to the network as a “failed experiment”.

Like other accounts that have been connected with Russian propaganda efforts, the accounts predominantly posted retweets (particularly of one another), with a small sample of original tweets – suggesting a mix of automation with some human control. Despite the clear human intervention in creating tweets, such accounts are generally referred to as “bots”.

Some of the accounts even tweeted messages about the need to cooperate better with Russia, or links to articles casting doubt on why Russia would want to influence international elections.

The accounts were uncovered through a process of searching the database of more than 17 million Brexit-related tweets sent before 23 June for any tweets from the profiles on the list Twitter supplied to Congress.

BuzzFeed News then analysed the tweets and retweets from the deleted accounts to look for accounts they frequently interacted with – a very simple form of network analysis – and uncovered a number of suspicious accounts, with few followers or retweets outside the network.

Most of the resulting network of accounts had very similar tweeting patterns, many tweeting sporadically on the same days before falling silent after a flurry of activity on 1 September.

Several of the accounts identified by BuzzFeed News also appeared to have taken their profile pictures from other users’ social media accounts – a common tactic of bots and propaganda accounts. The pictures used included those of an Estonian cartographer, a Dutch photographer, a Swedish hardware consultant, and an Estonian shaman.

The accounts’ main focus was jumping on to otherwise innocuous hashtags to insert anti-Merkel messages. Thirty-four of the bots tweeted about Merkel a total of 340 times – often inserting anti-Merkel messages into hashtag games such as #WLANName (Wi-Fi network name jokes) or #IchLiebeHerbst (I love autumn).

The otherwise German-language accounts also showed a huge spike of activity on the day of the Brexit vote. Data from the University of Sheffield research team shows a big increase in tweets from confirmed Russian bot accounts on 23 June 2016 – though these were overwhelmingly from the German-language bots.

This pattern was matched almost exactly by the extra accounts identified by BuzzFeed News, which – having shown no interest in Brexit prior to 23 June – suddenly tweeted English-language image memes relating to the vote with gusto. Of the 45 accounts, 21 put out a total of 370 Brexit-related tweets, all of them on the day of the referendum.

Some of the accounts occasionally tweet overtly pro-Russia messages. BuzzFeed News identified a now-unavailable tweet from @_your_lifestyle – one of the accounts Twitter identified to Congress – that included a link to a Russian-language blog, of the same name as the deleted Twitter account, rubbishing the idea that Russia would interfere in Brexit, titled “Russia Does Not Need Brexit”.

“Foreign journalists, analysts, and even some politicians are trying with all their might to expose Brexit [as] Putin’s hidden strategy for the collapse of the European Union,” its opening says, translated into English.

“It is assumed that the destabilisation and even disintegration of the EU will become a diplomatic victory for Moscow,” the piece continues, before rejecting the idea.

Several of the new accounts identified by BuzzFeed News also tweeted openly pro-Russia messages:

Kalina Bontcheva, the lead University of Sheffield researcher, said the new bots were easy to detect – which raises the question of whether they should have been spotted by the social network earlier.

“The new accounts were very similar to each other content-wise and thus could easily be detected based on the formulaic behaviour they had,” she said. “The tweeting behaviour of the live ones was quite random in many ways, with bursty tweeting on what looked very much like a random set of topics and obsession with shallow hashtag games.”

Bontcheva added that the findings from the new data showed much more study was needed of propaganda and bots online – but suggested sometimes the social networks are lagging behind in this.

“The intersection of bots, propaganda, and fake news is a key area in need of further investigation,” she said. “But…often scientists often lack the much-needed data, while the data keepers seem to still lag behind scientists in terms of knowledge and methodological approaches to discovering and analysing abnormalities in social media data.”

Ultimately, Bontcheva said, the German-language botnet appeared something of a “failed experiment”.

“They did have what seemed like an anti-Merkel political agenda, but were ultimately not successful in propagating these messages beyond the confines of their limited own botnet,” she concluded. “For me, this German botnet seems like a failed experiment, which was quickly abandoned after a few months, as it failed to gain traction on Twitter.”

The data shows how limited the reach of many of the tweets by the newly discovered bots was. The top two tweets – both sharing anti-Muslim sentiments in German – achieved just 15 and 10 retweets respectively (including those from other suspected bots), while the network’s tweets averaged just 0.3 retweets and 0.4 likes each.

The database of Brexit-related tweets also suggests the impact of the bots identified in the list Twitter gave to Congress was limited. Of the 17 million or so tweets collected by researchers in the run-up to the day of the EU vote, only 451 were English-language tweets from accounts on the list – and these were a mixture of pro-Leave and pro-Remain messages, almost all of which were simply tweets of headlines of mainstream news articles.

When headlines and other obviously automated messages are excluded, along with retweets, this count drops even further to just 13 original tweets on Brexit from the accounts Twitter identified – one of which was devoted to attacking a BuzzFeed humour piece, “16 Pictures That Accurately Predict Life In Post-Brexit Britain”, as BuzzFeed’s “worst and most inaccurate article ever”.

BuzzFeed News contacted Twitter with details of the 45 suspected bot accounts, and asked a series of questions on whether the company felt its systems to detect bots were effective, whether it would investigate the accounts, and whether it would update its evidence to Congress if so.

Despite suspending the 45 accounts after BuzzFeed News asked about them, Twitter declined to respond to any of these specific queries, saying “we’ve nothing to share on individual accounts”.

The company instead reissued a previous statement on its dealings with investigators.

While acknowledging the “huge appetite for more transparency into how Twitter fights bots and manipulative networks”, the statement said: “Due to the nature of these inquiries, we may not always be able to publicly share what we discuss with investigators. And there will always be tools or methods we cannot talk about, because doing so would only help bad actors circumvent them.”

Twitter also sent a lengthy response on its general approach to preventing spam and bots on its network, stating its systems “catch more than 3.2 million suspicious accounts globally per week”, that it detects and prevents 130,000 accounts per day attempting to manipulate trending topics, and noting “Russia and other post-Soviet states have been a primary source of automated and spammy content on Twitter for many years”.

BuzzFeed News would like to acknowledge the members of the GATE research team at the University of Sheffield, who worked on data used in this story: Genevieve Gorrell, Mark Greenwood, Ian Roberts, and Mehmet Bakir, led by Kalina Bontcheva.