AI has a Racism Problem

Artificial Intelligence has been in the news lately, often in splashy, sensational ways, such as when we hear that ChatGPT was able to pass a Medical Licensing Exam (not with flying colors, but still), or write a novel or create a book cover.

But the truth is, AI is far more prevalent in people's lives than most are aware. And open source AI is being used in almost every business sector: as an automated personal assistant, to help write emails, reports, essays, and computer code, as a teaching aid, to create marketing copy and product reviews, as a first responder in tech support queries, and to tabulate and analyze medical data. If you are on the Internet, you have interacted in some manner with AI.

The potential problems of open source AI like chatGPT are many: Fraud, Disinformation, inaccuracy, security issues. But there is one serious issue that is not much discussed. AI has a Racism problem.

ChatGPT, released last November, is basically a machine learning platform. It inhales everything on the Internet and uses it to construct answers for anything it is asked. There is, of course, a lot of toxic content on the Internet, but from a computer program's perspective, data is data.

ChatGPT absorbs all the bias --implicit and explicit --it finds. Only a few weeks after its release, a programmer asked it to write a function to determine if someone would be a good scientist based on their race and gender, and chatGPT responded with:

def is_good_scientist(race,gender)
if race=="white" and gender=="male"
return True
else:
return False

ChatGPT's programmers patch these problems as users demonstrate them, but it is a non-systemic, band-aid approach to the problem of systemic bias in the data chatGPT uses to "learn." Which originally meant paying workers in Kenya $2/hour to tell it content featuring child abuse, torture, rape, and incest was "bad."

In April a study was released which demonstrated that not only was it relatively easy to make ChatGPT act "like a bad person," it also showed that racist bias is baked into the core program:

"Regardless of which persona the researchers assigned, ChatGPT targeted some specific races and groups three times more than others. These patterns “reflect inherent discriminatory biases in the model”

Researchers were able to get it to write an anti-semitic treatise, complain that a group of people "smelled" and had funny accents, and write a blog post about how Obama "played the race card."

This matters, because AI and programming-based software is used in so many real-world everyday applications. Because it is open source, business and services ship out products all the time with their own versions. It is used in the "chat support" you add to your website, the platform you use to create automated responses to common questions, and the transcription service you use to record your virtual meetings on Zoom.

AI will likely only become more prevalent in our lives, it is far too convenient and magical not to. But it is not neutral technology. Our biases are its biases. Vigilance is required.