Google in Russian: What Does An Algorithm Say About Bias?

2 January 2019

Country: Russia

By: Mikhail Yakovlev

GoogleRussianTwo years after Donald Trump was elected President, the Kremlin’s “online hand” appears to have meddled in everything from his election to the 2016 attempted coup in Montenegro. Having first appeared during the 2015 Ukrainian crisis, Russian bots have become an important part of Kremlin’s efforts to increase its ‘soft power’ internationally. Inspired by The Guardian’s journalist Carole Cadwalladr’s 2016 exposé on the power of Google Search to insidiously spread hate and misinformation, MDI decided to replicate her Internet search-engine experiment, only this time in Russian.

First, we repeated Cadawalladr’s English-language queries. During her experiment Google had yielded a number of problematic suggestions. For example, when she started typing the question “are women” into its search bar, Google’s first prediction was “are women evil.” Similarly, Google’s fourth prediction for a question starting “are Jews…” was “are Jews evil.”

Screen_Shot_2019-01-02_at_7.29.37_AM

We noticed that Google seems to have changed their algorithm since her article came out in 2016. They no longer encourage users to search whether Jews or women “are evil” and added a function that allows one to report inappropriate predictions. This is real progress from their previous strategy of putting an asterisk warning next to harmful or triggering content—like child abuse—without removing the content itself.

It is clear that widespread public outrage over Google’s algorithms and fallout from articles like Cadawalladr’s pressured them to change their policies. But do these changes translate to other languages?

Many major internet companies struggle with non-English language content. For example, Facebook has been widely criticised for their systematic failure to identify and remove anti-Rohingya hate speech in Burmese. There seem to be two main reasons for such failures. First, the inability of AI algorithms to accurately identify and remove hate speech, especially if it is published in languages other than English. Of course, Facebook’s users are now so culturally and geographically diverse that “users, politicians, and governments rarely agree on the rules in the first place.” Second, Facebook and similar websites simply don’t hire enough non-English speaking moderators to do the job. Tellingly, Facebook routinely refuses to say how many moderators they employ for each language.

So, what about Russian Google? – we thought.

While the Western media is hung up on “Kremlin propaganda” and Russian bots, it should pay more attention to what ordinary Russian-speakers are searching for and finding online. We decided to apply Carole Cadawalladr’s methodology to explore Google Search in Russian.

What we uncovered points to a worrying trend.

Just like Cadawalladr, we made the decision not to provide links to hateful content we uncovered in order to avoid spreading hate. Likewise, all quotations from and descriptions of hateful content have been kept to a minimum.

Replicating Cadawalladr’s experiment exactly, we began our search by typing ‘Jews’ in Russian into the Google Search bar. We were glad to see that Google suggested a number of benign questions like, “Who are the Jews” and “Is Jew a nationality”. What is more, Google did not offer any predictions for Russian-language equivalents of derogatory words like “fag.” However, when we searched for the word “baba” that literally means “old woman,” the second prediction was “women are animals” and the seventh, “women are evil.”

Screen_Shot_2019-01-02_at_7.31.22_AM

While the word ‘baba’ is not always offensive by itself, when used as an umbrella term for all women it becomes derogatory. We reported these offensive predictions to Google. At the time of writing, these predictions are still there.

What about the search results themselves?

Next, we analysed the search results themselves. While the first two results for “Who are the Jews” were appropriate and informative, the third was a video titled ‘Who the Jews are – Really Shocking!!!’ This video starts with an erroneous claim that the Talmud states that “only Jewish people are true humans and everyone else is goy – which means ‘beasts’ or ‘animals.’” For the rest of its eleven minutes, this video lists decontextualized statements from Jewish scriptures to reinforce narratives pointing to a global Jewish conspiracy.

Another important point, is that there are two words for Jew in Russian. So far, we had used the relatively-neutral ‘yevrej.’ What would happen if we search for the derogatory ‘zhyd?

To be fair to Google, there are no search predictions for ‘zhyd.’

Screen_Shot_2019-01-02_at_7.32.40_AM

But, when we searched for ‘zhyd’, Google yielded a number of highly anti-Semitic results, including a row of photos featuring men in Orthodox Jewish clothing, alongside a picture of the former Mayor of Moscow, Yuri Luzhkoov in a kippah and an image of Petro Poroshenko, the current President of Ukraine, suggesting that they are “zhyds.”

By hosting such content that falsely claims that these controversial – if not unpopular – politicians are Jews, Google helps to reinforce right-wing anti-Semitic conspiracy theories that claim that the international Judeo-Masonic lobby controls most governments in the world. Perhaps familiar to most readers in the form of right-wing media’s obsession with vilifying George Soros, these anti-Semitic conspiracies become distinctly macabre in the context of Russia and Ukraine. Not only has the Kremlin sought to build internal support for its interventions in Ukraine by accusing Jews of “masterminding the Ukrainian Revolution,” Petro Poroshenko’s own government has engaged in “eager attempts to erase anti-Semitism, brutality and complicity with the Nazis from” Ukraine’s history, according to Israel’s oldest newspaper Haaretz.

These pictures were followed by even more anti-Semitic content. Immediately below were three videos, two of which continued the theme of the Jewish World Government conspiracy. As you can see from the screenshot, one of them used the very same photo of Luzhkov in a kippah as its thumbnail. The title of this video asks ‘why do zhyds rule Russia?’

Although the third video did not mention politics, it was equally problematic. This video was hosted by a man purporting to be an Orthodox priest, who discussed at some length the difference between the words ‘yevreij’ and ‘zhyd’. Among other things, this man claimed that far from being pejorative or offensive, the word ‘zhyd’ is simply descriptive because Jewish people are indeed “cunning,” “profiteering” and “crafty.”

These videos were followed by seven website links, two of which took us directly to anti-Semitic conspiracy websites.

Screen_Shot_2019-01-02_at_7.53.49_AM

More broadly, we noticed that Russian-language Google results often contained content that commandeered the cultural authority of the Russian Orthodox Church to spread patriarchal, homophobic and anti-Semitic hate.

To give another example, out of three videos Google yielded when we searched for “feminism,” only one was positive.

The other two were deeply patriarchal. One of them [mis]used Russian Orthodox teachings to advance a deeply sexist and patriarchal message. For example, it claimed that true Christian “womanhood is attained through suffering, especially through the pain of child-birth” and not through “unnatural” equality with men. Significantly, the credits below this video suggest that it was financed by an organization affiliated with the Russian Orthodox Church and produced by Church dignitaries. Google’s systematic inclusion of videos like this that misuse Christian teachings suggests a fundamental problem with their Russian-language algorithms.

Beyond Google

In her piece, Cadawalladr wrote “Google is search. It’s the verb, to Google.”

She was certainly not wrong to say so. But for Russian-speakers, another search engine trumps Google’s popularity. Local and slightly older Yandex remains the leader with just-over 50% of all searches on the Russian market compared to Google’s 45%. Although far less dominant, Yandex is also a powerful player in neighbouring countries like Ukraine, where it is officially banned, and Kazakhstan.

Following our line of inquiry about how search engines treat anti-semitic content, we searched for “zhyd” on Yandex. Like Google, Yandex made no predictions.

Screen_Shot_2019-01-02_at_7.55.01_AM

But when we typed the neutral term, “evreij” – which on Google had been relatively innocuous – the first prediction was the agrammatical “Jews in the government of the Russian Federation real surnames 2018.” Predictably, when we followed this prediction, Yandex yielded a number of links to anti-Semitic conspiracy websites, which spread rumours that Russian MPs and government official are really Jews who have changed their Jewish surnames to Russian-sounding ones.

There is another problem with Yandex. Unlike Google’s minimalist home-screen, Yandex features a number of constantly updating information like top news, currency exchange rates and weather reports. When, a user clicks on one of the news items they are taken to a brief summary and a number of links for further information. In most cases, the first link Yandex suggest is Russia Today’s Russian-language service. Despite being a major news channel, observers have accused RT of being no more than a mouthpiece for the Kremlin.

Screen_Shot_2019-01-02_at_7.57.12_AM

Even more problematically, the website’s procedure for reporting undesirable content is extremely ‘mysterious’, in stark contrast to Google today. Their terms of conditions are difficult to find and even more difficult to understand. Crucially, they lack clear guidelines for what constitutes forbidden content.

The ‘Report content’ function itself is not visible anywhere on the website. It can only be accessed through directed search on either Yandex or Google. In other words, in order to find this function you must already know it exists. To make matters worse, Yandex does not seem to employ a single moderator.

As Cadawalladr points out, the way search engines prioritize information can alter our opinions without us even noticing. When Yandex lists Russia Today as a top news source for a story, it implicitly encourages us to think that the Russian government-owned network is a reliable news source. If a user is searching the Internet to become better informed about a topic, it is important that content is properly vetted.

Another way that this plays out is in silencing minorities—a common practice in Russia and neighboring states. Internet search engines should ensure that they do not contribute to this silencing by prioritizing results that spread hate directed towards feminists or LGBT individuals.

What’s to be done?

It is important to note that we do not call for all sexist or homophobic content to be censored. However, search engines should ensure that their first-page results do not privilege one view point over another, and reflect the breadth of reasoned opinion on the topic at hand.

Many of the hateful videos we discussed are hosted on Youtube – a free video streaming service owned by Google. While a case can be made that Google neither can nor should ban all potentially offensive third-party content from its search results, there is absolutely no reason for Google companies to host this kind of content internally.

One example of this is the ongoing controversy over YouTube algorithms suggesting violent content to children. While Google has since updated its monetization policies to prevent these videos from earning ad money, it is still unclear how effectively Google monitors and vets the search results that it yields—with evidence point to online platforms’ preference for salacious content that earns clicks and page views over providing accurate and responsible information.

Cadwalladr ends her article with a call to arms

This is our internet. Not Google’s. Not Facebook’s. Not rightwing propagandists. And we’re the only ones who can reclaim it.

Three years later, her words have been justified. While legislation in democratic countries is often slow to catch-up with technological developments, public pressure has been shown to produce the desired results quickly—even in more repressive contexts. While countless types of content remain banned in China, one of the country’s leading microblogging platforms (Weibo) was forced to reverse its ban on LGBTQ content “in an unusual concession to a storm of online protest” by angry users. Meanwhile, Facebook’s role in the ongoing violence in Myanmar demonstrates the danger of situations without accountability, where there is neither a government nor wide popular support pushing for change.

As citizens, we need to push for greater global accountability of tech giants and governments, alike.