Expert View: ‘Algorithms of Oppression’ with Safiya Noble

Dr Safiya Noble is Co-Director of the UCLA Center for Critical Internet Inquiry and author of New York University Press Bestseller ‘Algorithms of Oppression: How Search Engines Reinforce Racism’. At the IoAI we seek out expert perspectives and insights into questions at the cutting edge of AI policy. The views of these experts do not necessarily reflect the Institute’s own position.

‘Algorithms of Oppression’ serves as an important wake-up call for anybody who sees search engines as ‘neutral.’ Safiya Noble demonstrates time and again that Google search results are embedded with both racist and sexist assumptions.

Her background in advertising had made clear the role of search engines as advertising platforms. But what was less clear was what happens when Google is used as a source of knowledge.

“It had me thinking about what happens for everything else, and in particular for communities we might think of as being vulnerable in society; racial and ethnic minorities, religious minorities, women. I started collecting searches on a variety of identity groups. And that is when I discovered that many different women and girls of colour identities in the US context were represented by pornography and stereotypes.”

“The idea here is that search engines are supposed to be democratic, to reflect what is most popular. But if you are any kind of minoritized group you can’t use the democracy model because you will always be outvoted. You are subjected to the tyranny of the majority.”

“Also, if you come from a community or group that isn’t highly capitalised and, of course, we know that racial and ethnic minorities in the US have been and continue to be oppressed, and are paid the lowest wages because of racial and gender discrimination, so they cannot compete with industries and companies that can buy and control the keywords associated with their communities.”

Google simply was not designed to function as an objective knowledge source.

“This was the genesis of my work around a decade ago: thinking about who has the power to control ideas about people in our society, and what’s at stake when we think of these platforms as public interest, fair, credible, neutral resources when they in fact are not.”

Another aspect of Safiya’s background made the danger posed by the lack of accountability of these new ‘information gatekeepers’ clearer still.

“In contemporary times, we have always been in these processes of categorising and selecting, of putting people, communities, and creatures on earth into systems. Hierarchical systems are built into how we come to know many things, quite frankly. In library science, many of us are more attuned to the subjective nature of knowledge and information categorization, and who has the power to authoritatively define the truth and the point of view.”

“In this way, librarianship and knowledge stewardship is a mutable, subjective, contentious process of figuring out which voices need to be collected and included that historically have been precluded because of racism, power, or gender. This subjective nature of knowledge really must translate to how we think of things like data, which is also socially constructed and laden with power relations.”

Dodging accountability for this subjective prioritisation is unjustifiable, and yet Safiya notes that this is exactly the route many in Silicon Valley take, laying claim to Section 230 of the Federal Communications Act, that no provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.

“The problem now, in the age of platforms, is that you have platforms that are absolutely monetising, prioritising, deprioritising, and managing all kinds of content. They are all in the media publishing business.”

“They lay claim to section 230, or claim they are just infrastructure with no responsibility for the content, propaganda, hate, disinformation, calls for genocide or racial violence and so forth that they amplify with speed and to the scale of billions of people.”

“Their entire profit model is predicated on the content, on the media. So, we have to reimagine the kinds of accountabilities that platforms have in the media landscape. And we need much greater legislation and protection with respect to the harms against the public that are fomented by that media.”

Since Safiya embarked on this research, progress has been made on one front. Now, when she raises the topic of racist algorithms there is at least a way to hear what she is saying.

“When I was talking ten years ago about things like algorithmic discrimination, saying it could happen at the level of code and in terms of deployment of these technologies, very few people believed it. It was very difficult in those early years to be understood and to have this be part of the mainstream conversation. The mainstream conversation a decade ago was that computer science is just applied math and applied math can’t be racist because math can’t be racist. It was very narrow.”

But when it comes to concrete progress made by big tech, the steps taken continue to be narrow in scope. Google, for instance, are quick to fix individual cases, especially when faced with a media storm, but fundamental issues remain.

“What Google does in response to any of its critics; journalists, scholars, even the public, is take those criticisms like a ticket and fix it. But this doesn’t really shift the underlying logic.”

“For example, I wrote about what happened when a teenager in Baltimore, Kabir Alli, did a search on Black teenagers, and it surfaced all these criminal photos. Google fixed that. They tried to resolve it. And then about six weeks ago a news story broke that when you google four Black teenagers instead, you still get criminal images. So, the underling logics do not get fixed and have not been fixed.”

And it is not just Google whose focus on individual cases points to problems with the fundamentals of ‘content moderation.’

“The speed and scale at which user generated content moves through social media platforms cannot be attended to by the number of human beings that are hired to adjudicate that content.”

“I’ve learned from the ground-breaking work of my colleague, Dr. Sarah Roberts, that content moderation will never be automated, nor resolved fully through AI. Certain things like child sexual exploitation, things where the pixels are known and you can use software like photo DNA to identify images and take them down, that can happen. But more nuanced dimensions of what moves through the platforms requires a labour force that is profound, and which will be impossible to manage as more deep fakes and fraudulent content is put online.”

“There is a lack of understanding on the part of Facebook about the danger of its platform and what it’s doing. So, appointing its own board and hand selecting people is a bit like the fox guarding the hen house. They’re going to be adjudicating case by case. One piece of content at a time. I think it’s a model that doesn’t solve the problem.”

Legislation is certainly important, but there are different ways to come at the problem, each with their own advantages.

“Of course, the human rights model has limits because it leaves other ways of thinking about harm off the table. However, it has at least been a starting point for things like international pressure on oppressive governments.”

“I think there are civil rights models in the US and other places that might be taken up, that might be internationally oriented. I think there are sovereign rights models where indigenous communities and nations legal and cultural definitions around the world could be elevated as a way of framing internet law. These must be taken into consideration, if we want to intervene on the proliferation of propaganda that gives rise to authoritarianism and rollbacks of democratic rights.”

This is part of Safiya’s fundamental approach, taking diverse perspectives into account and emphasising the importance of using multiple models to approach the problem.

Legislators must be part of this, themselves taking diverse perspectives into account. But they must also support other channels for studying these problems.

“Of course, hiring people with the requisite background in gender studies and ethnic studies is very important to inform policy.”

“[But] you cannot think of regulating big tech and at the same time defund libraries, defund public education, public media, and public health–all these democratic, public-interest institutions that serve as critical knowledge and information infrastructure for the well-being of everyone.”

“Trillion-dollar industries don’t pay taxes. They take the best students, the brightest talent, and pay them well while leaving public-oriented social systems to collapse in the U.S. It is important to keep that in mind. We need governments to deeply fund universities and schools and libraries.”

“Legislators need to legislate funding from the sector that extracts so much from us and put it back into the institutions that can help provide knowledge of science, humanities, arts, and social science. Democracies are only as strong as their publics are well-educated and able to critically think and act.”