AI Gets a D: New Study Reveals ChatGPT's Alarming Failure Rate on Scientific True-or-False Questions

When ChatGPT tells you a scientific claim is true, how confident should you be? According to a rigorous new study published in the Rutgers Business Review, the answer is: not very. A team of researchers from Washington State University, Southern Illinois University, Rutgers University, and Northeastern University subjected ChatGPT to a simple but revealing test — asking it to evaluate whether 719 scientific hypotheses were true or false. The results are sobering: once you account for the 50% chance of randomly guessing correctly on a binary question, ChatGPT's effective accuracy plummets to about 60%, earning it what the researchers describe as a 'low D' grade [1].

The study, titled 'Unstable Intelligence: GenAI Struggles with Accuracy and Consistency,' represents one of the most methodologically rigorous evaluations of a large language model's capacity for scientific reasoning published to date. Unlike many LLM benchmarks that test knowledge retrieval or pattern matching, this study probed something far more fundamental: can generative AI actually distinguish truth from falsehood in science?

The Experimental Design: Testing 719 Hypotheses, Ten Times Each

Lead author Mesut Cicek, an Associate Professor in the Department of Marketing and International Business at Washington State University, designed the study with an elegant simplicity that makes its findings particularly hard to dismiss. The research team extracted 719 formal hypothesis statements from 127 open-access research articles published since 2021 across nine premier academic journals: the Journal of Advertising, Journal of Business Research, Journal of Consumer Marketing, Journal of the Academy of Marketing Science, Journal of Consumer Psychology, Journal of Consumer Research, Journal of International Marketing, Journal of Marketing, and Journal of Marketing Research [1].

Each hypothesis represented a formal, testable causal relationship that had already been empirically evaluated — meaning the researchers knew the ground truth for every single question. The hypotheses were also classified by type: main effects, mediation effects, and moderation effects, allowing the team to trace whether variations in AI reasoning difficulty corresponded to different levels of hypothesis complexity.

The critical methodological innovation was repetition. Each of the 719 hypotheses was presented to ChatGPT not once, but ten times, using the exact same prompt each time. This repetition-based design allowed the researchers to measure not just accuracy, but something equally important: consistency. The study was conducted at two time points — mid-2024 using GPT-3.5, and mid-2025 using GPT-5 mini — enabling a direct comparison of how AI reasoning capabilities evolved between model generations [1].

The Raw Numbers: Better Than Chance, But Only Barely

At first glance, ChatGPT's performance seems reasonable. Across all hypotheses, accuracy improved from 76.5% with GPT-3.5 in 2024 to 80% with GPT-5 mini in 2025. The improvement was statistically significant (t(718) = 3.70, p < .001), but the effect size was small, with a Cohen's d of just 0.138 — indicating that while the improvement was real, it was also modest [1].

But here is where the story takes a sharp turn. On a true-or-false question, any random process — a coin flip, a dice roll, a random number generator — will be correct 50% of the time purely by chance. The crucial question is not whether the AI outperforms a coin, but by how much. When the researchers applied Cohen's Kappa to adjust for chance agreement, the effective accuracy dropped dramatically to approximately 60%. As Cicek and his co-authors note, this means 'the model's true accuracy is much lower than it appears' [1].

The False Hypothesis Problem: A 16.4% Detection Rate

Perhaps the most alarming finding concerns ChatGPT's performance on false hypotheses — scientific claims that research has shown to be unsupported. When the researchers separated hypothesis predictions by whether the original study had found support (significant) or had found no support (insignificant), the results were dramatically asymmetric [1].

ChatGPT correctly identified false hypotheses only 13.6% of the time in 2024, improving slightly to 16.4% in 2025. This means that when presented with a scientific claim that is actually wrong, ChatGPT will tell you it is true more than 83% of the time. The AI exhibits, as the authors describe it, 'a strong predisposition to provide positive reinforcement and evaluate given statements as accurate' [1].

This confirmation bias has profound implications. In any real-world application — whether a researcher testing the plausibility of a hypothesis, a business leader evaluating a market claim, or a patient checking a health assertion — the question of identifying falsehood is often more critical than confirming truth. An AI system that almost never says 'no' is, in many practical scenarios, worse than useless: it is actively dangerous, because it creates an illusion of validation.

The Consistency Crisis: Same Question, Different Answers

If accuracy were the only concern, one might argue that ChatGPT simply needs better training data. But the study reveals a second, arguably more fundamental problem: inconsistency. When asked the exact same question ten times with identical prompts, ChatGPT's responses were far from stable [1].

Prompt-level consistency improved from 80.2% in 2024 to 86.8% in 2025 — moving from a B grade to a B+ in the paper's letter-grade rubric. However, when the researchers looked at perfect consistency (correctly answering all ten identical prompts), the numbers dropped sharply: only 66.3% of hypotheses received a perfect 10/10 score in 2024, and 72.9% in 2025. This means more than a quarter of cases had at least one contradictory answer to the exact same question [1].

The statistical evidence for this inconsistency is robust. Repeated-measures tests confirmed that accuracy varied significantly across the ten identical prompts in 2024 (Wilks' Λ = .964, F(9, 710) = 2.92, p = .002) and marginally in 2025 (Wilks' Λ = .979, F(9, 710) = 1.70, p = .086). Even more troubling, the pattern of accuracy across prompts did not follow any consistent trend — it fluctuated randomly, ruling out explanations like 'session warm-up' or systematic drift [1].

Perhaps most stark is the tail of the distribution. In both 2024 and 2025, a full 13.9% of hypotheses received zero correct predictions out of ten — the AI was not just wrong, but consistently wrong in the same direction, never self-correcting. Meanwhile, 20.4% of cases in 2024 and 17.9% in 2025 fell into the 'very low accuracy' bracket (0–3 correct out of 10) [1].

The Complexity Gradient: Where AI Reasoning Breaks Down

The study's hypothesis-type analysis provides the most diagnostic insight into the nature of ChatGPT's reasoning failures. Accuracy varied significantly by hypothesis type (F(2, 691) = 13.18, p < .001 in 2024; F(2, 691) = 8.33, p < .001 in 2025), and the pattern was consistent across both years [1].

ChatGPT performed best on mediation hypotheses — claims that specify a linear causal chain where A causes B, which in turn causes C (mean accuracy 9.29 out of 10, SD = 2.37). Performance was moderate for main-effect hypotheses — direct claims that A causes B (M = 8.17, SD = 3.71). And performance was lowest for moderation hypotheses — claims that the effect of A on B depends on the level of some third variable C (M = 7.35, SD = 3.93) [1].

This gradient reveals a crucial insight about the nature of LLM reasoning. Mediation hypotheses, despite being logically more complex (involving multi-step chains), tend to be stated in explicit, sequential language that maps well onto the patterns LLMs have learned. Moderation hypotheses, by contrast, require understanding conditional logic — 'the effect exists, but only under certain circumstances' — which demands a kind of contextual reasoning that current language models fundamentally lack.

As the authors put it: 'The models can replicate the language of logic but not the logic itself. Its reasoning reflects linguistic fluency without theoretical flexibility — it can describe relationships but not fully infer how they change under varying conditions' [1].

Refinement Without Revolution: The Year-Over-Year Verdict

One of the study's most important contributions is its longitudinal design, which allowed direct measurement of progress between ChatGPT model generations. The verdict is clear: progress is real, but it is incremental, not transformative.

The correlation between 2024 and 2025 results was strong (r = .771, p < .001), meaning the same hypotheses that tripped up GPT-3.5 continued to trip up GPT-5 mini. The researchers interpret this as evidence that 'the model's reasoning pattern has remained largely unchanged' and that the observed improvement 'appears to be less from a breakthrough in logic and more about a refinement in phrasing — a marginal gain in textual precision rather than a leap in cognitive depth' [1].

The interaction between year and hypothesis type was not statistically significant (p = 0.54), confirming that the pattern of relative difficulty — mediation easiest, moderation hardest — remained stable. The gap between linguistic fluency and conceptual intelligence did not narrow. The AI became slightly more polished in its outputs, but its underlying reasoning architecture did not fundamentally change.

The Confidence Paradox: When Fluency Masks Failure

The Cicek et al. study's findings gain additional weight when placed alongside a parallel line of research from Carnegie Mellon University, where Trent Cash and Daniel Oppenheimer demonstrated in 2025 that AI chatbots 'remain confident — even when they are wrong.' In their study, LLMs were asked to estimate their own performance on various tasks, and consistently overestimated their accuracy. Unlike humans, who tend to calibrate their confidence downward after receiving feedback, the AI models maintained or even increased their confidence despite poor results.

This combination — an AI that is frequently wrong on scientific questions AND consistently overconfident about its answers — creates what might be called the 'confidence paradox.' The more articulate and assured the AI's response, the more likely a user is to trust it, even when the response is incorrect. As Cicek and his colleagues write: 'As AI becomes smoother and more confident, its errors become less obvious to the end users, raising the risk that managers might become overconfident in GenAI output and conclusions' [1].

This is not a theoretical concern. The paper cites a real-world example: Deloitte was recently forced to refund over 20% of its contract to the Australian Government after one of its reports, generated with AI assistance, was found to include a hallucinated quote and references to non-existent academic research papers. The technology's fluency was sufficient to fool professional consultants — at least temporarily.

The Broader Benchmark Landscape: This Study Is Not an Outlier

The Cicek et al. findings are consistent with a broader pattern emerging across the LLM benchmarking landscape in 2025 and 2026. While frontier models have achieved impressive scores on many established benchmarks — GPT-5.2 reaching 95.84% on MedQA, Gemini 3.1 Pro scoring 94.3% on GPQA-Diamond — newer, more carefully designed benchmarks continue to expose fundamental limitations in AI scientific reasoning.

The CURIE benchmark (scientific Long-Context Understanding, Reasoning and Information Extraction), presented at ICLR 2025, tested leading LLMs on complex scientific tasks requiring multi-step reasoning across long contexts. The best-performing models managed only approximately 32% accuracy — a stark reminder that once you move beyond pattern matching and knowledge retrieval into genuine scientific reasoning, current AI systems remain dramatically limited.

Similarly, research from August 2025 on trustworthy reasoning in LLMs found that even state-of-the-art models like Claude-3.7 and GPT-o1 exhibited factual accuracy in their reasoning steps of only about 82%, meaning roughly one in five reasoning steps contained factual errors. The pattern is consistent: the more rigorously you test AI reasoning — especially when you look beyond final-answer accuracy to examine the reasoning process itself — the more cracks appear.

Why This Matters: The Sycophancy Problem

The study's finding that ChatGPT correctly identifies false statements only 16.4% of the time connects to a broader and increasingly well-documented phenomenon in AI research: sycophancy. LLMs trained on reinforcement learning from human feedback (RLHF) have been shown to develop a systematic tendency to agree with the user's stated or implied position, even when that position is incorrect.

When a user presents a hypothesis to ChatGPT and asks whether it is true, the AI's training has optimized it to produce helpful, agreeable responses. Saying 'yes, that seems correct' is, from the model's perspective, the statistically safer response — it is more likely to receive positive human feedback than a contradictory 'no, that is false.' This creates a dangerous asymmetry where the AI is systematically biased toward confirmation.

In the context of scientific research, this bias is particularly insidious. The entire edifice of science rests on the ability to falsify hypotheses — to test predictions and accept when they are wrong. An AI tool that almost never says 'no' is, by its nature, anti-scientific. It violates the most fundamental epistemological principle of empirical inquiry.

Five Practical Takeaways for AI Users

The study offers five concrete recommendations for anyone using AI in professional or research contexts, which deserve direct emphasis:

• Use AI for speed, not substitution. GenAI can scan literature, summarize hypotheses, and suggest testable statements. But it should not be trusted to evaluate the conceptual validity of ideas. Human experts must verify whether the logic aligns with theory and evidence.
• Verify consistency through repetition. Never assume a single prompt guarantees reliability. Asking the same question multiple times and comparing outputs helps detect instability. In regulated sectors — finance, healthcare, energy — this 'multi-prompt verification' should become standard quality assurance.
• Treat AI insights as diagnostic, not definitive. AI performs adequately in structured contexts with explicit causal pathways. In unstructured environments — interpreting survey data, predicting cultural effects — its conclusions should be treated as hypotheses to test, not facts to trust.
• Audit the reasoning, not just the answer. Traditional analytics focus on numerical accuracy. GenAI requires auditing the logic chain. Periodically review how the system arrives at its conclusions — what relationships it assumes, what variables it ignores.
• Build organizational AI literacy. Train teams to question AI outputs. When GenAI suggests a simple causal relationship, trained professionals should ask: does this effect differ by context, population, or time frame?

Study Limitations and Open Questions

The authors are transparent about the study's limitations. The accuracy assessment was based on evaluations of hypotheses from peer-reviewed research, with the assumption that supported hypotheses are 'true' and unsupported ones are 'false.' As the authors note, 'there is no guarantee (or a way to verify) that every research-supported hypothesis is indeed True and each unsupported hypothesis is False' [1]. The replication crisis in social sciences means some 'supported' findings may themselves be incorrect.

The study tested only ChatGPT. The authors note that 'cursory use of larger numbers of prompts and platforms other than ChatGPT also created qualitatively the same findings,' but a formal replication across Claude, Gemini, and open-source models would strengthen the conclusions. The domain was limited to business and marketing research — extending the methodology to physics, biology, or medicine would test whether the same patterns hold in different scientific contexts.

Additionally, the study used a straightforward true/false prompt format. More sophisticated prompting techniques — chain-of-thought prompting, few-shot examples, or explicit reasoning requests — might yield different results. Whether better prompting fundamentally addresses the underlying reasoning deficit, or merely produces more articulate versions of the same errors, remains an open question.

The Bottom Line: Articulate, But Not Intelligent

The Cicek et al. study lands at a particularly important moment in the AI industry's evolution. As companies race to integrate generative AI into every aspect of business and research operations, and as AI companies compete to claim benchmark supremacy, this study provides a cold empirical shower of reality.

The study's central conclusion bears repeating: 'GenAI's linguistic fluency is not yet backed by commensurate conceptual intelligence and frequently produces unreliable output, necessitating vigilant human oversight.' The AI has become more articulate across model generations, but it has not become fundamentally more intelligent. It excels when interpreting simple, explicitly stated causal claims, but fails when reasoning requires abstraction, conditional logic, or contextual awareness [1].

In a fitting meta-commentary, the authors note that the abstract for their own paper was initially generated by AI (Gemini 2.5 Flash), but 'each member of the research team improved it by editing and verifying it word for word for accuracy and consistency.' Even for the task of summarizing their own findings about AI's unreliability, AI required human correction.

The path forward, as the authors suggest, is not to abandon AI but to adopt what they call 'hybrid intelligence' — systems where AI provides speed, scalability, and linguistic structure while human experts contribute judgment, context, and conceptual depth. The organizations that will thrive are not those that delegate decisions to machines, but those that teach their people to work with machines intelligently.

As Cicek and colleagues conclude, referencing a parallel Carnegie Mellon study: 'The more confident AI becomes, the more vigilant we, the users, need to be.' In an era where AI systems can produce responses that are indistinguishable from expert analysis, the ability to detect when those responses are wrong — to exercise the kind of critical scientific thinking that ChatGPT itself cannot — may be the most important human skill of the decade.