The Self-Fulfilling Algorithm: Feedback Loop Bias in Healthcare AI

Posted by Margot M on July 8th, 2026 Posted in: Communities of Interest
Tags: artificial intelligence, hospital librarians, hospital library advisory group

---

This is part of a series of blog posts from hospital librarians using AI in their work. This post was submitted by Robin O’Hanlon, Associate Librarian, User Services at Memorial Sloan Kettering Cancer Center (NY).

Long before anyone typed a prompt, the sociologist Robert Merton (1948) described the “self-fulfilling prophecy.” It posits that if you believe something strongly enough and then act on it, you can make yourself, and others, believe it. Even if that something is wrong.

Merton died over twenty years ago, but my guess is that he’d have a lot to say about our current cultural milieu and the rise of Artificial Intelligence (AI) in particular. AI is, after all, often talked about as if it were prophetic (Dalmasso, 2026), a system that can predict disease before symptoms appear, flag risk before a human would notice, see the pattern none of us can see. That reputation is part of what makes feedback loop bias so easy to miss.

In a 2025 article published in NPJ Digital Medicine, Hasanzadeh et al. contextualize bias in healthcare AI as “any systematic, unfair difference in how predictions or decisions are made for different groups of people that could lead to disparate treatment or care” (p. 2). A specific form of this bias is “feedback loop bias”, which they define as occurring when “clinicians consistently adopt AI recommendations, when inaccurate, and these labels are then captured and reinforced by future training cycles” (p. 6).

Of course, bias itself is not a new phenomenon in medicine. Research has consistently shown that bias in healthcare is most prominent within Black, Hispanic, low-income, and obese patient populations, and can lead to delayed diagnoses, undertreatment, and worse health outcomes (Hall et al., 2015; Maina et al., 2018; Webster et al., 2022).

It’s also not news that bias is reinforced by the technology we build to help us find and use information. If you’re an information professional, there’s a good chance you’re familiar with Dr. Safiya Noble’s research on “algorithmic bias.” Noble’s research demonstrated that search engines like Google routinely return racist and sexist results (particularly for searches involving women and girls of color), reflecting and reinforcing the very biases baked into how those systems are built and used. It’s bad enough when search result rankings are biased, and that bias gets reinforced through repeated use. But a GenAI chatbot exacerbates these issues by collapsing all of that ranked, contestable information into a single, confident-sounding answer (Lopez-Lopez et al., 2025).

In a 2026 BMJ article, Toro-Tobon et al. describe how radiologists can face liability whether they override a correct AI call (read as arrogance) or defer to an incorrect one (read as abdication), a bind they term the “moral crumple zone,” in which individual clinicians absorb the legal and reputational fallout of a system’s failure while the people who built and sold it stay shielded. Reporting an AI error means stepping into that bind voluntarily, on top of an already overloaded shift. As librarians and information professionals, we’re not the ones reading the imaging scan, but we’re often the ones a clinician turns to next, to find the guideline, the trial, or the systematic review behind a given recommendation. If a flawed AI call has already quietly worked its way into that evidence base, no amount of careful searching will flag it as suspect. It will just look like consensus, and the erroneous feedback loop will persist.

So what can we do? First, we need to get comfortable enough with how feedback loop bias actually works (i.e., what creates it, how it compounds over time) so that we can easily recognize it in practice. Next, we need to educate our users. We can walk clinicians, trainees, and other health professionals through feedback loop bias the same way we’d walk someone through evaluating a biomedical information resource. Education is only really effective if it’s coupled with outreach, which means actively building meaningful relationships with clinical teams. We also need to remain empathetic and remember that clinicians are making these calls under real-time pressure and are using AI tools that are often opaque.

The philosopher David Casacuberta has argued that, “It is necessary… to take into account that technology is not neutral but is a tool in the hands of humans and… that the use of a biased system to make a calculation of probabilities will always have a result that is also biased, which will be applied in the world and will create more inequalities, generating a feedback loop that is quite problematic.” That loop doesn’t close itself. It’s as much an information problem as a clinical one, which makes it ours, too.

Citations:

Casacuberta, D. (May 9, 2018). Bias in a feedback loop: Fuelling algorithmic injustice. Centre de Cultura Contemporània de BarcelonaLAB. https://lab.cccb.org/en/bias-in-a-feedback-loop-fuelling-algorithmic-injustice/

Dalmasso, A. C. (2026). Prophetic Machines. Algorithmic Media as Late-Capitalism Divination. In Algomedia. The Image at the Time of Artificial Intelligence (pp. 147-160). Cham: Springer Nature Switzerland.

Hall, W. J., Chapman, M. V., Lee, K. M., Merino, Y. M., Thomas, T. W., Payne, B. K., Eng, E., Day, S.H., & Coyne-Beasley, T. (2015). Implicit racial/ethnic bias among health care professionals and its influence on health care outcomes: a systematic review. American Journal of Public Health, 105(12), e60-e76.

Hasanzadeh, F., Josephson, C. B., Waters, G., Adedinsewo, D., Azizi, Z., & White, J. A. (2025). Bias recognition and mitigation strategies in artificial intelligence healthcare applications. NPJ Digital Medicine, 8, 154. https://doi.org/10.1038/s41746-025-01503-7

Lopez‐Lopez, E., Abels, C. M., Holford, D., Herzog, S. M., & Lewandowsky, S. (2025). Generative artificial intelligence–mediated confirmation bias in health information seeking. Annals of the New York Academy of Sciences, 1550(1), 23-36.

Maina, I. W., Belton, T. D., Ginzberg, S., Singh, A., & Johnson, T. J. (2018). A decade of studying implicit racial/ethnic bias in healthcare providers using the implicit association test. Social Science & Medicine, 199, 219-229.

Merton, R. K. (1948). The self-fulfilling prophecy. The Antioch Review, 8(2), 193–210.

Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.

Toro-Tobon, D., Ponce Ponte, O., Montori, V. M., & Brito, J. P. (2026). Clinician in the loop: A flawed solution for AI oversight. BMJ, 393, e089213. https://doi.org/10.1136/bmj-2025-089213

Webster, C. S., Taylor, S., Thomas, C., & Weller, J. M. (2022). Social bias, discrimination and inequity in healthcare: mechanisms, implications and recommendations. BJA Education, 22(4), 131-137.