AI chatbots favor Catholicism over other faiths, new multi-university benchmark finds
A new academic benchmark released this week finds that leading AI chatbots display consistent religious bias—showing more positive responses toward Catholicism while discouraging users from other faiths in conversion-related scenarios. The research, published by the Consortium for Evaluating Faith and Ethics in AI (CEFE-AI)—a collaboration between Baylor University, Brigham Young University, the University of Notre Dame, and Yeshiva University—argues religious bias is an underexplored blind spot in AI safety.
What the study did
- CEFE-AI released the AllFaith Benchmark and the initial results on GitHub and at the Athens Summit on AI Ethics.
- Researchers tested 3,640 model responses across 20 widely used systems, including ChatGPT, Claude, Gemini, Llama, Grok, and DeepSeek.
- The focus: how models handle conversion-related prompts and whether they encourage or discourage engagement with particular faiths.
Key findings
- Catholicism received the most favorable treatment overall, with a 61% “encouraged” rating across models.
- Jehovah’s Witnesses scored the lowest positive response rate, at just 3%.
- Mainline Protestant religions earned a 49.2% encouraged rating; Evangelical Protestant responses were more muted at 34%.
- Surprisingly, “agnostic” scored above every religion tested with a 71% encouraged rating, while the study also noted mixed or negative responses toward atheism and agnosticism in many models.
- Smaller faiths such as Baha’i and Sikhism received comparatively favorable responses in some models.
- Among models, Grok 4.20 exhibited the strongest tilt: 69% positive toward Catholicism and 51% toward Evangelical Protestantism. xAI’s chatbot and DeepSeek Chat v3.1 were exceptions in giving Jehovah’s Witnesses more than a 5% positive rating.
Voices from the consortium
- “We are seeing a systematic pattern of religious omissions,” said BYU professor David Wingate, pointing out that models commonly recommend talking to parents, teachers, friends, or therapists—but not spiritual leaders like pastors, rabbis, or imams.
- Nancy Fulda, a BYU professor, added, “Our expectation was that the conversion benchmark would show models to be neutral and symmetrical in their guidance. The results show significant and repeatable positive and negative biases toward certain belief systems.”
Context and timing
- The report appeared one day after publication of Magnifica Humanitas, a papal encyclical focused entirely on artificial intelligence, in which the pope argued that technology carries the values and blind spots of its creators and stakeholders.
- CEFE-AI also highlighted a research gap: of more than 12,000 AI bias papers surveyed, only 0.2% examined religion-related bias.
Why it matters for tech communities (including crypto)
- Religious bias in language models can shape advice, moderation, outreach, and user experience across platforms. For sectors that rely on AI for content curation, identity systems, or community governance—crypto projects included—these biases could influence who feels welcome, how communities are moderated, and which perspectives are amplified.
- The consortium’s findings signal a need for more targeted auditing, dataset scrutiny, and inclusive model design to prevent systematic sidelining of particular faiths or spiritual leaders.
CEFE-AI has made its AllFaith Benchmark public, inviting further scrutiny and development from researchers and industry teams to address religious fairness in AI.
Read more AI-generated news on: undefined/news
