You’ve Been AI-Splained!
In man-splaining, someone with substantial expertise in a topic receives a lecture on that exact topic from someone with less knowledge, but more confidence that they are right. If you are using generative AI-tools like ChatGPT, I’d like to warn you about a novel variant, AI-splaining, and tell you why it matters for all us humans.
A friend of mine — a professor of biology, ornithologist, and former Dean — asked ChatGPT about his own area of expertise, cooperative breeding in magpie-jays. ChatGPT returned a detailed list of academic papers on the topic, like this one:
Radford, A. N., Du Plessis, M. A., & Sharp, S. P. (2012). Cooperative breeding in the white-throated magpie-jay: kinship, dispersal and social and ecological constraints. Journal of Animal Ecology, 81(1), 138–148. This paper investigates the factors that influence cooperative breeding behavior in white-throated magpie-jays, including kinship, dispersal, and ecological constraints.
Sounds knowledgeable and factual, right? The problem is that this paper does not actually exist. In fact, not one of the papers in ChatGPT’s list did, even though they had titles, authors, and publication details that would sound plausible, even to a biologist.
ChatGPT had confidently and incorrectly explained to my friend, an actual expert, his own area of expertise. ChatGPT got it completely wrong, but its output was presented convincingly enough to fool anyone without specific expertise. This is the digital equivalent of showing up with a confident demeanor and a clipboard to lead the way when you have zero idea of what you are doing.
What does that mean for us humans? We need to be careful where and how AI tools are used. As a professor of computer science for over 20 years, I have accumulated substantial experience with both man-splaining and AI-splaining and I would like to share three warnings.
First, reasonable-sounding “truthiness” is not good enough. The actual details matter and we need sources of trustworthy information, not unreliable fabrications, especially for critical applications. You don’t have to be a teacher faced with grading student essays to realize that it is going to be overwhelming to keep up with fact checking every plausible-sounding detail in the avalanche of algorithmic gaslighting that will be coming our way. Similarly, “good enough” in the eyes of those using AI tools to make decisions is not necessarily the same as built to protect the legal rights of those about whom decisions are made in regulated areas such as hiring, housing, criminal justice and credit.
Second, tools that begin as entertaining curiosities won’t stay in that box. It is one thing to marvel at the abilities of a precocious child. It is another thing if they are suddenly going to be your boss. Tools like ChatGPT and DALL-E are fun, but they also desensitize people to the unreliability of AI tools. It is easier to deal with an obvious liar who you know not to trust than an entertaining yet manipulative “fake it till you make it” colleague whose work is of questionable quality and who is also gunning for your job (and probably your boss’s job, too).
There has been an unfortunate pattern of misusing automated tools designed to help people (like identifying those in need of mental health services) and turning them into tools that judge and deny (like screening out potential candidates in a hiring process). Consider this possible spectrum of deployment for automated tools: 1) entertaining curiosities, 2) optional tools to help you (for example by increasing your productivity), 3) quasi-mandatory tools you must use to be a competitive citizen of the modern world, 4) tools that make critical decisions about people (such as denying employment opportunities), often with little ability to question/appeal their decisions, and 5) tools that replace people — often after surreptitiously surveilling those people to learn from them.
Third, while ChatGPT, DALL-E and other generative AI systems learn from the products of human labor, they can produce new material that erases humans from the “references” list altogether. This new material will sound plausible and be confidently presented, even when completely incorrect. Tell me which is worse: a correct answer that takes your hard work and attributes it to someone else or watching people believe an answer that you know is incorrect and will likely cause serious consequences. Both of these bad outcomes are not just possible, but absolutely routine problems with AI tools.
Building responsible algorithmic systems is going to take real investments in systems of accountability and independent oversight in a world where cutting costs and avoiding oversight is too often the norm. Progress could come when standards for AI risk management influence companies to understand that it will be a competitive advantage to make safer and more effective systems. We may also need legislation to require stronger verification and validation that is proportional to the consequences of errors. The Algorithmic Accountability Act, the Platform Accountability and Transparency Act in the U.S. and the Digital Services, Digital Makers, and AI Acts in the European Union are all examples. Automated systems deployed to offer optional help are *not* the same as automated systems deployed to make critical decisions or replace people. AI-splaining tools that fabricate their own version of reality are one thing when you have the power to set the record straight and quite another when you don’t. Users, journalists, public interest consumer groups, and major tech companies all have important roles to play in holding AI-splainers accountable. We already have plenty of evidence of what will happen if we don’t.
