Spot BS With AI: 4 Questions to Check Claims and Vendor Quotes
Published 2026-10-04 · Updated 2026-10-13 · 7 min read · ShooWork (FREECO CO., LTD.)
Don't ask AI 'is this true?' Use it to split claims into facts, opinions and fluff, then generate the questions that make vague promises checkable.
Here's what you actually want to know: can you trust this pitch, this quote, or this line that “everyone in the industry does it this way”? The short answer is that AI can't tell you whether something is true. What it can do in about three minutes is sort a block of text into checkable facts, opinions dressed up as facts, and sentences that say nothing. Then it can write the follow-up questions you should send back. That's where its real value is.
The most common mistake is to paste a paragraph into a chatbot and ask “Is this true?” You'll get a confident answer. It won't be any more reliable than ten minutes of your own careful digging, and sometimes it's worse, because the model may hand you a source that doesn't exist or a date that never happened. Use it the other way around, as a question generator. It turns sales talk into specific questions you can ask on a call or ask for in writing. Whether a claim is true depends on how the other side answers and what the original sources say. How sure the model sounds tells you nothing.
We run our own AI tool platform, a video-editing engine and Google Ads tooling, so we've seen this from both sides. Vendors pitch us, and people ask us about competitors' claims. The four questions and the workflow below are what we use when we negotiate with suppliers, review outsourcing quotes and evaluate partnerships. Feel free to steal them.
Why You Shouldn't Ask AI “Is This True?”
A language model predicts the most plausible next words. It doesn't look things up in a database of facts. That makes it very good at the structure of an argument. It can spot when a definition quietly changes halfway through, point out a premise nobody stated, and come up with strong counterarguments. It's weak on specific facts about the world right now: what a tool costs today, whether a policy changed last week, whether a company really uses the technology it says it uses. Unless the model is connected to live search or your own data, it's answering from memory.
Worse, its tone doesn't match how confident it should be. When it makes something up, it sounds exactly like it does when it's right. So limit AI to three jobs:
- Sort. Split the text into three piles: factual claims, opinions, and filler that carries no information.
- Argue the other side. For every factual claim, write the strongest objection and say what evidence would disprove it.
- Build a checklist. For each claim, say where to check it, which document to ask for and exactly what question to ask.
The model doesn't need to know the right answer for any of these jobs. It only needs to break text down well. That's why this method works even in fields you know nothing about. You don't need expertise, just specific questions.
The Four Kinds of Spin to Recognize First
People usually get misled because someone said something that can't be verified and they treated it as fact, not because someone told a flat lie. These four patterns cover most of it:
| Pattern | What it sounds like | Why it works | What you reply |
|---|---|---|---|
| A number with no baseline | “3x more efficient.” “Cuts costs in half.” | A number feels precise, but it doesn't say compared to what, over how long, or with how many cases | “3x compared to what? Over what period? Averaged across how many projects?” |
| Borrowed authority | “All the big players use this.” “It's industry standard.” | It hands the burden of proof to someone you can't check | “Can you name a specific client or point me to something public?” |
| A wall of jargon | A string of acronyms explaining why it costs so much | It makes you feel like you don't know enough, so you stop asking | “In one sentence, what difference does this make to my results?” |
| A fake deadline | “This price ends Friday.” “Only two slots left.” | It leaves you less time to check, and that's the point | “If I decide next month, what changes? Please put it in writing.” |
Once you can spot these patterns, you'll see that the part worth questioning is almost always the half that was left out. It's the same habit we teach in our 10-minute teardown method for learning from the pros: pay attention to what they chose not to say as much as what they said.
Four Questions That Make Vague Claims Checkable
Memorize these. When something sounds off but you can't say why, run through them in order and the problem usually shows up.
- “What's the checkable version of this sentence?”
Ask for the claim to be rewritten as something you could prove true or false tomorrow. “Our system is very stable” can't be checked. “Average monthly uptime of 99.5% over the last three months, and you can see it on our monitoring dashboard” can. If no one can rewrite it, the original sentence probably had no content to begin with. - “What's the baseline, the time period and the sample size?”
Ask this about every percentage and every multiplier. With a different baseline, the same data can support the opposite conclusion. If they can't tell you the baseline, the number shouldn't be part of your decision. - “If this turns out to be false, how soon would I know?”
This is the strongest question of the four. It makes you decide ahead of time what failure looks like. If the promise isn't kept, which number shows it first, and in which week? If you can't name that signal, you weren't planning to check the result anyway, and then it doesn't matter what you sign. - “Who pays, who carries the risk, and how do I get out?”
Does the person talking stand to gain from your decision? If things go wrong, who pays for it? If you leave, can you take your data, accounts and files with you? Who gains and who pays usually predicts the outcome better than anything they say.
When you give these questions to an AI, give it a role, the source material and an output format, not just a question. Here's a prompt you can use as is:
Copy-paste prompt:
“Below is an explanation someone gave me. Do not judge whether it's true, and do not add your own knowledge. Do three things. 1) Put every sentence in a table as one of: checkable factual claim, opinion or position, or filler with no information. 2) For each factual claim, list the missing baseline, time period or assumption. 3) Write a list of follow-up questions I can email them directly, one sentence each, asking for written answers. Material: …”
Example 1: Taking Apart a Vendor Quote
Quotes are packed with spin because they mix numbers, scope and responsibility, the three things most easily left vague. When we hire contractors or evaluate tool vendors, we run every quote through the same steps:
- Paste in the full quote, or one line item at a time, and have the AI only reorganize it: each deliverable, its unit price and billing unit, whether tax is included, and when each payment is due.
- Ask it to list items this quote doesn't mention but that usually lead to extra charges in similar projects. This is where the model shines. It has seen a huge number of similar documents and is good at noticing what's missing.
- Ask it to rewrite every acceptance criterion as a measurable sentence. Any criterion it can't rewrite is where you'll end up arguing later.
- Have it write the ten questions you need to ask, and send them as they are.
Almost every quote has one or two of these four gray areas: unclear billing units (how a “person-day” is counted, whether meetings are included), out-of-scope charges (how many revision rounds, what overages cost, rush fees), acceptance criteria you can't measure (“good results,” “meets expectations”) and no exit terms (who owns the data, accounts and source files when the contract ends). We wrote up what this costs in practice in the real cost of three content outsourcing routes.
If someone is selling you an AI solution, add one more question: “Can we start with a small pilot, agree on a success threshold, and stop if it doesn't hit it?” Vendors who agree usually know their product works. If they flatly refuse and insist on signing the whole project first, that tells you what you need to know. Don't sign a full-scope AI contract with anyone who won't run a pilot. This one habit saves more money than you'd think, and we explain the math in why every AI project needs a PoC first.
Example 2: Checking Something You Read Online
When a claim seems fishy, these five steps take about ten minutes:
- Find the core claim. Have the AI boil the whole thing down to one checkable sentence. Most rumors fall apart here, because once you shrink them down, there's very little left.
- Ask where claims like this usually come from. Have the AI list the usual sources for this kind of claim: official announcements, research papers, news reports, sponsored content, forum posts. Then figure out which one you're looking at.
- Read the original source yourself. Check prices on the official pricing page, specs in the official docs and regulations on the regulator's own site. Don't hand this step to AI. The links it gives you may not exist.
- Ask for the strongest counterargument. “Give me the three strongest reasons this claim is wrong, and what evidence would disprove it.” If the counterargument is stronger than the claim, you know which way to lean.
- Write down what would prove it false. If you can't settle it yet, note what would show it's false and set it aside. Don't share it, and don't dismiss it yet either.
Three Things AI Can't Do for You
Being clear about the limits matters more than selling you one more use case.
First, it can't decide what's true. The model doesn't know what's happening right now. Prices, versions, regulations and who uses what should always be checked on the official source. Because we build ShooWork Clip and run a tools platform, people regularly forward us competitors' claims and ask if the numbers are reasonable. All we can honestly say is, “This number has no baseline. Go back and ask these three questions.” We don't say the other company is lying, and an AI shouldn't either.
Second, it can't read motives. Was the other person honestly wrong, misled themselves, or trying to mislead you? To answer that you need context: knowing the person and knowing what happened the last three times you worked together. AI doesn't have any of that.
Third, don't use this method to reach conclusions about medical, legal or investment questions. Mistakes there cost too much, and AI is especially likely to get the details wrong. In those areas the method has one job: helping you list the questions to take to a qualified professional. Then actually go ask one. Don't treat a model's answer as a diagnosis, legal advice or a reason to buy or sell.
One more trap people miss: the AI follows the way you frame the question. Ask “Is this a scam?” and it leans toward reasons to be suspicious. Ask “What's good about this offer?” and it leans toward the positives. Ask both, phrased the same way, and look at where the two answers agree. Those overlapping points are usually the facts that hold up.
FAQ
Q: Is it ever okay to just ask AI whether something is true?
Don't base a decision on the answer. How confident a model sounds has nothing to do with whether it's right, and it may invent a source. Use it to flag weak spots, argue the other side and draft follow-up questions. For the actual facts, rely on official pages, written answers or the original source.
Q: The numbers they gave me sound professional. How do I know if something's wrong?
Ask one thing: what's the baseline, how long a period, and how big a sample? People who know their numbers will tell you right away and put it in writing. If they get vague or say “you don't understand this industry,” treat the number as if it doesn't exist and leave it out of your decision.
Q: Is there one question that works almost every time?
“Could you send me what you just said in an email?” Most claims that wouldn't hold up get corrected the moment someone has to put them in writing. It works better than any AI analysis, and it's free.
Q: Is a free AI plan good enough, or do I need one with web search?
For breaking down claims and generating questions, a free plan is usually fine, because it doesn't need current information. To check current prices, policies or versions, you need a mode with web search, and you should still open the sources yourself. For what each plan includes and costs, check the provider's official pricing page.
Q: Won't asking this many questions make me look like I don't trust them?
A written list, asked all at once in a neutral tone, looks professional, not combative. People who want a long-term relationship are glad to spell out the terms, because clear terms protect them too. And if someone gets offended by reasonable follow-up questions, that tells you something useful.