Bias in AI is real. But it doesn’t have to exist.

How race and identity are shaping politics, policy and power.

Jul 14, 2023 View in browser   By Mohar Chatterjee With help from Ella Creamer, Brakkton Booker and Ben Weyl POLITICO illustration/Photo by AP Hello, Recast friends! House Republicans narrowly passed a contentious defense bill and a major Alaska political rematch is on the way. But here for you is a fascinating interview between Mohar Chatterjee, a technology reporter at POLITICO, and AI ethicist Rumman Chowdhury. You can read the first part of this interview in last week's Digital Future Daily.  Today, we’re diving deeper into the intersection of identity and technology. With AI igniting widespread public adoption and anxiety, the struggle to get this technology right is real. Its critics are worried that AI systems — particularly large language models — trained on massive quantities of data might have biases that deepen existing systemic discrimination. These are not theoretical worries. Infamous examples of bias include a racially prejudiced algorithm used by law enforcement to identify potential repeat offenders, Amazon’s old AI-powered hiring tool that discriminated against women, and more recently, the ability to prompt ChatGPT to make racist or sexist inferences. Getting these powerful AI systems to reveal all their pitfalls is a tall order — but it’s one that’s found interest from federal government agencies, industry leaders and the technology’s day-to-day users. The Commerce Department’s National Telecommunications and Information Administration is seeking feedback on how to support the development of AI audits, while the White House’s Office of Science and Technology Policy is gathering input from the public on what national priorities for AI should be. We spoke to AI ethicist Rumman Chowdhury about her hopes and fears for this quickly spreading technology. Previously the director of Twitter’s META (Machine Learning Ethics, Transparency, and Accountability) team and the head of Accenture’s “Responsible AI” team, Chowdhury’s expertise has been tapped by both Congress and the White House in recent months. She appeared as a witness at a June hearing held by the House Science, Space and Technology Committee to testify on how AI can be governed federally without stifling innovation. She is also one of the organizers for the White House-endorsed hacking exercise on large language models (called a “red-teaming” exercise) to be held at a large hacker conference called DEFCON in August. The exercise is meant to publicly identify potential vulnerabilities in these large AI models. Chowdhury is currently a Responsible AI Fellow at the Harvard Berkman Klein Center and chief scientist at Parity Consulting.   Was The Recast forwarded to you by a friend? Don't forget to subscribe to the newsletter here. You'll get a twice-weekly breakdown of how race and identity are the DNA of American politics and policy.   This interview has been edited for length and clarity.  THE RECAST: For many people, AI conjures notions of a dystopian future with machines taking over every aspect of our lives. Is that fear justified? CHOWDHURY: At the current state of artificial intelligence, that fear is fully unjustified. And even in a future state of artificial intelligence, we can't forget that human beings have to build the technology and human beings have to implement the technology. So even in more of a stretch of the imagination, AI does not come alive and actively make decisions to harm people. People have to build the technology, and people have to implement it, for it to be harmful. Examples of bias in AI include the ability to prompt ChatGPT to make racist or sexist inferences. | Richard Drew/AP Photo THE RECAST: What does bias in AI look like? How will marginalized communities be impacted? What’s your biggest fear? CHOWDHURY: You've touched on exactly what my biggest fear is. My biggest fear is the problems we already see today, manifesting themselves in machine learning models and predictive modeling. And early AI systems already demonstrate very clear societal harm, reflecting bias in society. So for example, if you look at medical implementations of artificial intelligence and machine learning, you'll see how members of the African American community — in particular, Black women — are not treated well by these models because of the history of being not treated well by physicians. We see similar things in terms of biases in the workplace against minority groups. Over and over again, we have many clearly documented instances of this happening with AI systems. This doesn't go away because we make more sophisticated or smarter systems. All of this is actually rooted in the core data. And it's the same data all of these models are trained on. THE RECAST: Your team at Accenture built the first enterprise algorithmic bias detection and mitigation tool. Why were you concerned about bias in AI back then and how has this concern evolved? CHOWDHURY: In the earlier days of responsible AI — we're talking 2017-2018 — we were actually in a very similar state of having more philosophical conversations. There were a few of us — and we were, frankly, primarily women — who talked about the actual manifestation of real societal harm and injury. And some of the earliest books written about algorithmic bias came out a few years before that or around that time. In particular: Safiya Noble’s “Algorithms of Oppression,” Virginia Eubanks’ “Automating Inequality,” Cathy O'Neil’s “Weapons of Math Destruction,” all talk about the same topic. So the issue became: How do we create products and tools that work at the scale at which companies move to help them identify and stop harms before they go ahead building technology? THE RECAST: Your team at Twitter discovered that the platform’s algorithm favored right-wing posts. Google’s Timnit Gebru blew the whistle on ethical dilemmas posed by large language models. Why do you think so many whistleblowers in tech are women — particularly women of color? CHOWDHURY: To clarify, this was during my time leading the machine learning ethics team at Twitter. So this work actually wasn't whistleblowing. This was approved by the company, we did this, you know, in conjunction with leadership. What we found in that study was that Twitter's machine learning algorithm amplified center-right content in seven out of eight countries. What we weren't able to find out was whether this was due to algorithmic bias or whether it’s due to human behavior. Those are actually two different root causes that have two different solutions. Unfortunately, in many tech situations, my case is rare. Very often, issues that are raised by women and women of color get ignored in the workplace, because, more broadly, women of color tend to not be listened to in general. So it's unsurprising to me that after having exhausted every internal channel or possibility, people who are typically ignored have to turn to more extreme measures. Being a whistleblower is not romantic — it's actually very, very difficult for most individuals. If you think about what being a whistleblower means, you have essentially blackballed yourself from the industry that you've worked in — the industry that you care about. Unfortunately, this is more likely to happen to women of color. We are more likely to identify issues and have a stronger sense of justice and this desire to fix a problem, but simultaneously, we are more likely to be ignored. But again, I will say my example at Twitter was actually a rare case of that not happening. Chowdhury was fired by Elon Musk shortly after he took over Twitter. | Susan Walsh/AP Photo THE RECAST: You were fired by Elon Musk shortly after he took over Twitter. What are your thoughts on why you were a target? CHOWDHURY: I don't see the kind of work that the machine learning ethics team did being aligned with the kind of company Elon Musk wants to run. If we just look at the Twitter files, we look at the kinds of people he's attacked. Some of them being folks like Yoel Roth — people who did things like trust and safety. The kind of work that my team did is very aligned with the work of teams that he is not funding or prioritizing. Frankly, he's firing teams that did that work. To be honest, I don't think I would have worked for that company anyway. THE RECAST: When you testified before Congress last month, you said, “Artificial intelligence is not inherently neutral, trustworthy, nor beneficial.” Can you talk a little more about that? CHOWDHURY: I very intentionally picked those words. There is this misconception that a sufficiently advanced AI model trained on significant amounts of data will somehow be neutral. How this technology is designed and who it is designed for is very intentional, and can build in biases. So technology is not neutral. These models are also not inherently trustworthy. That ties to a term that I coined called moral outsourcing: this idea that technology is making these decisions and that the people making the decisions behind the scenes have no agency or no responsibility. Trustworthiness comes from building institutions and systems of accountability. There's nothing inherently trustworthy about these systems, simply because they sound smart or use a lot of data or have really, really complex programming. And just because you build something with the best of intentions doesn't actually mean that it's going to inherently be beneficial. There's actually nothing inherently beneficial about AI. We either build it to be beneficial in use or we don't. THE RECAST: Why do we need a “global AI governance organization” as you mentioned in your congressional testimony? CHOWDHURY: There are a couple of scenarios that are not great. One would be a splintering of technology. We are actually living in an era of splintered social media, which means that people get information mediated via different sources. That actually deepens rifts between different kinds of people. If somebody in China or Russia sees a very different take on what's happening in Ukraine compared to somebody in the U.S., their fundamental understanding of what is truth is very different. That makes it difficult for people to actually live in a globalized world. Another thing that I am concerned with in creating global regulation is that the majority of the Global South is not included. I'm part of the OECD’s working group on AI governance, but these narratives are coming out of Europe, UK or the U.S. I just don't want there to be a lack of mindfulness, when creating global governance, in assuming that the Global South has nothing to say. And there are some questions that actually are not global scale questions to ask. So this global entity, in order for it to supersede national sovereignty, these have to be really, really big questions. The way I've been framing it is: What is the climate change of AI? What are the questions that are so big, they can't be addressed by a country or a company, and we need to push it up? So the default shouldn’t be, “Oh, clearly punt this to the global entity,” it should be an exception rather than the rule. ◆◆◆ You did it! You made it to Friday! And we’re sending you into the weekend with some news and a few must-reads and must-sees. Divided on Defense: The GOP-led House passed a controversial defense bill Friday that targets the Pentagon’s policy on abortions, medical care for transgender troops and diversity issues. POLITICO’s Connor O’Brien reports that it doesn’t have a shot at passing the Senate. Alaska Grudge Match: Republican Nick Begich says he’s making another run at Alaska’s at-large congressional seat, once again challenging 2023 Power List nominee Rep. Mary Peltola, a Democrat. POLITICO’s Eric Bazail-Eimil has more. The crisis over American manhood is really code for something else, according to a new POLITICO Magazine piece from Virginia Heffernan. A Korean conglomerate endeavors to build an elevator into the sky in Djuna’s part noir, part cyberpunk novel “Counterweight,” out now. “Earth Mama” movingly traces the life of Gia (Tia Nomore), a mother trying to regain custody of her two kids in foster care. “Lakota Nation vs. United States” weaves archival footage, interviews and images in its depiction of the tribe's 150-year struggle to regain a homeland. The surprise collab we never knew we needed: BTS’ Jung Kook and Latto on an energetic new bop, “Seven.” Karol G drops “S91”, an emotional anthem inspired by a Bible verse, and a music video featuring a cross made of speakers.   TikTok of the Day: Generational differences   Follow us on Twitter Brakkton Booker @brakktonbooker Rishika Dugyala @rishikadugyala Teresa Wiltz @teresawiltz Jesse Naranjo @jesselnaranjo   Follow us

To change your alert settings, please log in at https://www.politico.com/_login?base=https%3A%2F%2Fwww.politico.com/settings This email was sent to rouf@idiot.cloudns.cc by: POLITICO, LLC 1000 Wilson Blvd. Arlington, VA, 22209, USA Please click here and follow the steps to unsubscribe.