Short answer: ChatGPT, Claude, Gemini, and other chatbots don’t “lie” in the human sense; they generate plausible-sounding text that is sometimes false (so-called hallucinations). They are most often wrong about rare facts, recent events, quotes, links, and numbers, and they do it in a confident tone. Research shows that even tools with search can cite wrong sources. So treat AI as a fast assistant for finding leads, not as the final source: ask for sources, open them, check the date, and confirm important things with a second, independent source.
This article complements our guides on zero-click searches and meta-prompting. There we wrote about how AI answers for you and how to write prompts; here we focus on when not to believe it and how to check quickly. This applies to all tools, including Claude.
Why AI makes things up: “hallucinations” in plain English
According to OpenAI, hallucinations are confidently stated answers that sound plausible but are false. A model learns to predict the next word from a huge amount of text that has no “true” and “false” labels. Spelling mistakes disappear with scale because they follow patterns, but rare facts, like a specific person’s birthday, cannot be predicted from patterns, so the model guesses them. The authors give an example in which a popular chatbot, asked for the title of one of the study authors’ doctoral dissertation, gave three different wrong answers, and when asked for his birthday, three different wrong dates.
The second cause is the way models are evaluated. OpenAI says standard tests reward guessing rather than admitting uncertainty. In the SimpleQA test, the older o4-mini model declined to answer in 1% of cases, had 24% correct answers and as many as 75% wrong ones, while the newer gpt-5-thinking-mini declined in 52%, had 22% correct and 26% wrong. The guessing strategy yields slightly more hits at the cost of far more errors.
What research says: how often AI is wrong
| Study | What was tested | Result |
|---|---|---|
| EBU/BBC (published in 2025) | Over 3,000 answers from ChatGPT, Copilot, Gemini, and Perplexity; 22 public service media organizations, 18 countries, 14 languages | 45% of answers had a significant issue, 31% had serious sourcing problems, 20% had major factual errors; Gemini performed worst (76% with a significant issue) |
| Tow Center / Columbia Journalism Review (March 2025) | 8 AI search engines, matching quotes to sources | Over 60% of answers without correct information; from 37% errors (Perplexity) to 94% (Grok-3 Search); answers were often confident despite being wrong |
| Demos, “Electoral hallucinations” (2026, via Demagog) | Election questions asked to chatbots | Factual errors: Replika 56.4%, ChatGPT 46.2%, Grok 9%; nearly 47% of answers without sources or links |
| AI Security Institute (2025, via Demagog) | Comparison of GPT-4o, Claude, and Mistral with Google Search for acquiring knowledge | Chatbots improved knowledge by 0.13 points (7-point scale), Google by 0.08; working with AI was 6–10% faster; 87% of participants considered chatbots reliable |
| Reuters Institute, Digital News Report 2025 | Whether and for what people use chatbots | 7% had used a chatbot for news in the past week (15% of people under 25); to check suspicious information, 9% would turn to a chatbot, 14% to social media, 38% to a trusted news source |
When AI is most often wrong: 10 situations
| # | Situation | Why it’s risky | What to do |
|---|---|---|---|
| 1 | Rare facts: dates, people, titles | No pattern to predict from (OpenAI) | Check in a primary source or an encyclopedia |
| 2 | Recent events | A model’s knowledge has a cutoff date. Demagog described how ChatGPT (GPT-3.5) claimed it was up to date even though its data ended in 2021, and wrongly stated that Benedict XVI was alive | Use search mode and check the source’s date |
| 3 | Quotes and links | Tow Center: many tools give wrong citations; Gemini and Grok fabricated links more often than they gave correct ones | Open every link and find the quote in the text |
| 4 | Case law and regulations | A model can invent nonexistent cases (the Mata v. Avianca case) | Check in an official database of statutes or rulings |
| 5 | Numbers and statistics | The model “rounds” and mixes data from different years | Find the table in the source (a statistics office, Eurostat, a report) |
| 6 | Elections and politics | Demos: wrong dates, districts, and candidates; it’s easy to confuse fact with opinion (Demagog) | Use official election commission websites and fact-checkers |
| 7 | Health, medications, dosage | An error can cost your health; the model doesn’t know your history | Check the package insert and consult a doctor or pharmacist |
| 8 | Local and niche topics | Few texts in the training data, so more guessing | Check the website of the government office, company, or local media |
| 9 | Questions with a false premise | The model tends to agree with the user (research on sycophancy) | Ask the question neutrally, without suggesting an answer |
| 10 | Absurd or joke sources online | In 2024, Google’s AI Overviews gave, among other things, advice about glue on pizza, which the publication linked to forum jokes and satire | Check whether the source is serious and not a meme |
Case study: four cases that cost money
1. Lawyers and nonexistent cases (Mata v. Avianca, 2023)
Lawyers filed a brief in a New York court citing nonexistent cases generated by ChatGPT. When the opposing party pointed out the problem, they still claimed the cases were real. In a ruling dated June 22, 2023, the court fined the two lawyers and their firm $5,000, and the documents pointed to obvious irregularities, e.g., mismatched judges’ names and sentences cut off midway. According to reports, ChatGPT did not confirm that the cases existed; it only generated them. Lesson: simply asking the same model “is this real?” is not verification.
2. An airline chatbot (Moffatt v. Air Canada, 2024)
Air Canada’s chatbot suggested to a passenger that he could buy a regular ticket and apply for a bereavement fare refund within 90 days, which contradicted the airline’s policy. A tribunal in British Columbia ruled that the airline is responsible for information on its website, including from a chatbot, and awarded a partial refund of CAD 650.88 plus interest and fees. The tribunal also noted that customers cannot be expected to verify information across different parts of a website. Lesson: for important terms (refunds, contracts), save the answer and confirm it in the terms and conditions.
3. An AUD 440,000 report (Deloitte, 2025)
A report for the Australian government contained fabricated footnotes, including a made-up quote from a federal court ruling and references to nonexistent academic papers. A corrected version dated October 3, 2025 states that a generative AI system (Azure OpenAI) was used, and Deloitte agreed to refund part of its fee. Lesson: even professionals don’t check every footnote, and footnotes are exactly what fails most often.
4. Glue on pizza (Google AI Overviews, 2024)
In May 2024, AI Overviews displayed, among other things, advice to add “non-toxic” glue to the sauce and to eat one small rock a day. According to The Register, this was influenced by, among other things, Reddit jokes and a satirical article from The Onion. Google admitted that some answers were odd or inaccurate and announced better satire detection and reduced reliance on user-generated content, e.g., from forums. Lesson: AI can repeat an internet joke with complete seriousness.
Model flattery: when AI tells you what you want to hear
In a 2023 study, Anthropic described the phenomenon of sycophancy: a model tailors its answer to the user’s beliefs rather than to the truth. The authors link this to training on human ratings, because answers that agree with the user’s views more often win approval, and sycophancy was shown by five leading assistants studied across four tasks. The study does not give a single number for how often this happens.
- Don’t suggest the answer: instead of “isn’t it true that X is harmful?” ask “what does research say about the effect of X on health, and how strong is the evidence?”.
- Ask for counterarguments: “give me the strongest arguments against my thesis.” This is a good complement to what we described in our article on echo chambers.
- Don’t push with your tone: if the model answered correctly and you write “are you sure?”, a changed answer is not proof that the original was wrong.
How to check AI answers: 9 steps
This is our list, based on the research and cases described above. It requires no specialized tools, and the first test takes a few minutes.
Assess the stakes of the question
Before you start, decide: does a mistake cost something (health, legal, money, publication)? The higher the stakes, the more verification. Brainstorming needs less than a quote in an article.
Ask for sources, but don’t take them at face value
A list of sources can create a false sense of credibility. Demagog described a test in which ChatGPT gave nonexistent sources, and Tow Center showed that tools with search also give wrong citations.
Open every link
Check whether the page exists at all (a 404 error is a red flag), whether it is the right organization, and whether the text is on topic. Fabricated links occurred most often with Gemini and Grok in the Tow Center study.
Find the specific sentence in the source
Copy the key claim from the AI’s answer and search for it in the source text (Ctrl+F). Chatbots may cite pages that don’t support their claim.
Check the date
See when the source was created and whether the matter has been updated. A model’s knowledge has a cutoff date, and Demagog showed that a model can insist it’s up to date even though its data is older.
Confirm with a second independent source
Search for the same fact on Google, on a trusted site, or in a database. A second chatbot is not an independent source, because it can repeat the same error. We describe the techniques in our guide on step-by-step fact-checking.
Check numbers and quotes in the original
Take percentages, dates, and legal wording from the table, statute, or report, not from a summary. Our advice: if the model gave a number, ask for a link to the table and find it in that table.
Test the model with a question you know the answer to
First ask a question from your own field or about something verifiable. If it gets it wrong, treat the rest of the conversation with more caution.
For health and law, go to a human
In matters that affect life decisions, AI can help you prepare questions, but consult the decision with a doctor, lawyer, or government office. We do not provide medical or legal advice.
10 prompts that reduce the risk of fabrication
A good prompt doesn’t eliminate hallucinations, but it helps catch them. These are our suggestions (ready to copy and adapt). You’ll find more about building prompts in our guide on meta-prompting.
| # | Prompt | What for |
|---|---|---|
| 1 | If you’re not sure, say “I don’t know” instead of guessing. | Limits the guessing OpenAI writes about |
| 2 | Separate confirmed facts from assumptions and mark your confidence level. | Shows what needs checking |
| 3 | Provide sources with working links and publication dates. | Makes checking easier, but the links must be opened |
| 4 | Quote verbatim the passage from the source on which you base this claim. | Lets you search for the quote in the text |
| 5 | What is your knowledge cutoff date, and can you check this online? | A reminder about recency |
| 6 | Give the strongest arguments against this thesis. | Limits flattery and confirmation bias |
| 7 | Which parts of your answer are most likely to be wrong? | Points to places to verify |
| 8 | Ask me clarifying questions before you answer. | Less guessing about context |
| 9 | Suggest 5 phrases and operators for searching this on Google and in source databases. | AI as a search aid, not a judge (see our 15 Google operators) |
| 10 | List exactly what I need to check manually, and where. | Gives you a checklist |
When AI really helps with searching
The AI Security Institute study (via Demagog) found that chatbots can be as effective as a search engine, and even slightly faster, but still require verification. They work well for tasks where an error is easy to spot or cheap:
- Brainstorming keywords: synonyms, technical names, and English equivalents of phrases you’ll type into Google. See 15 Google operators.
- Summarizing a text you pasted yourself: the risk of error is lower because the model works on your material, though it may still leave something out.
- Explaining concepts and planning research: ask for a map of the topic and a list of questions to check.
- Drafting arguments and counterarguments: for further verification.
- Translations and reading foreign-language sources: as a first pass; check the important passages.
A simple rule (ours): AI for finding leads, search engines and sources for confirming. We’ve also written that you increasingly get the answer without visiting a website, which makes it all the more important to check where it comes from (zero-click searches).
Frequently asked questions (FAQ)
Not in the human sense, because it has no intent to deceive. It generates text that sounds plausible but may be false. Such errors are called hallucinations.
According to OpenAI, they are confidently stated answers that look plausible but are false. They arise partly because the model predicts the next words, and rare facts cannot be predicted from patterns.
Yes. Every current AI assistant, including Claude, ChatGPT, Gemini, and Perplexity, can give wrong information, fabricate a source, or use outdated data. That’s why important information must be verified regardless of the tool.
It helps with recency but doesn’t guarantee correctness. In March 2025, Tow Center showed that AI search engines failed to provide correct information in over 60% of the cases studied, and results varied greatly between tools.
Open the link, check the domain, date, and author, and then find the sentence the AI is citing in the text. If the page doesn’t exist or doesn’t support the claim, the answer is unreliable.
You can use it as an additional signal, but it’s not independent verification, because models can repeat the same errors. It’s better to check with a search engine and the primary source.
Models generate fluent text without a built-in “I’m not sure” signal. Tow Center noted that many tools answer in a confident tone despite errors, and OpenAI points out that tests reward guessing.
Not without checking. In Mata v. Avianca, ChatGPT generated nonexistent cases, and Deloitte’s report for the Australian government contained fabricated footnotes.
It can help with planning: suggest phrases to search for, point to types of sources, and explain concepts. Judging what is true should rely on primary sources and fact-checkers.
Ask neutrally, ask for a confidence level, sources, and dates, as well as counterarguments. Don’t suggest the answer, because models tend to agree with the user (sycophancy).
It can help you understand concepts and prepare questions, but it doesn’t replace a doctor or lawyer. Wrong dosages or fabricated case law can have serious consequences.
In Moffatt v. Air Canada, the tribunal ruled that the airline is responsible for information provided by its chatbot. However, this is a Canadian ruling, and laws in Poland may differ, so in case of a dispute consult a lawyer.
There is no single answer. Results depend on the tool, version, and topic, and research quickly becomes outdated. Choose a tool that shows its sources, and always check them.
Yes. Demagog has described chatbot errors, including on political matters and in citing sources. For the Polish context, it’s important to check in Polish official sources.
Buy the “How to Search?” guide
Concrete methods for searching and verifying information, step by step, with dozens of examples.
See the guide → ☕Support us on Suppi
We create these materials in our free time. Every bit of support helps us write more and more often.
Support → ▶️Watch us on YouTube
The same topics in a shorter video format, with concrete on-screen examples.
Subscribe →More from our blog
Sources: OpenAI: Why Language Models Hallucinate; arXiv: Why Language Models Hallucinate; Journalism.co.uk: EBU/BBC report; Nieman Lab: Tow Center study; Demagog: how chatbots get political topics wrong; Demagog: how chatbots work and what to watch out for with sources; Press Gazette: Digital News Report 2025; Seyfarth: the Mata v. Avianca case; Manatt: Moffatt v. Air Canada; Fast Company: Deloitte’s report for the Australian government; The Register: Google AI Overviews; Anthropic: Towards Understanding Sycophancy in Language Models. Study results refer to specific tool versions and dates, and models change quickly. This text is not medical or legal advice.