How I check what AI tells me

I use language models throughout my working week, both in my professional role managing technology projects and while writing software in the evenings. Like anyone who uses these tools regularly, I have watched them produce brilliant summaries in one minute and astonishingly convincing errors the next.

Over time, I found that relying on generic skepticism was not particularly useful. If you distrust everything an AI model outputs, you waste the productivity advantage of using it. If you accept the output without examination, you will eventually repeat an embarrassing mistake.

To navigate that tension, I developed a practical mental checklist that I run through before acting on machine-generated text.

Check whether the model was given the source

The first consideration is simple: did the model generate this response from text I provided directly, or did it pull the information from its statistical training weights?

When I paste a twenty-page contract into a prompt and ask for a list of termination clauses, the model is reading the actual text on screen. It is summarizing visible data. While it can still occasionally misinterpret nuanced phrasing, the risk of outright invention is low because the source facts are constrained by the input.

The danger spikes when you ask a model to retrieve factual specifics from memory. If you ask for the case law governing a specific tenancy dispute or the exact technical specifications of an older piece of network hardware, the model generates text that sounds like an authoritative answer.

The most common failure mode here is the fabricated citation. The model will supply a plausible case name, complete with realistic court reporters, volume numbers, and dates. Because the grammatical structure of a legal citation is highly standardized, the model reproduces the pattern effortlessly, even when the underlying case never existed. Recognizing whether the model is summarizing provided text or guessing from memory immediately establishes the baseline level of scrutiny required.

Weigh the verification cost against the risk of error

Not every piece of machine-generated text requires rigorous auditing. The degree of checking should match the consequences of being wrong.

When I use an AI model to draft an outline for a project kickoff meeting or brainstorm alternative headlines for an article, a factual quirk or clumsy phrase is harmless. I will read through the text, revise the wording, and adjust the points naturally during the editing process. The cost of an error is negligible.

However, when a response contains specific, actionable data points, such as financial ratios, calendar dates, statutory deadlines, or configuration parameters, the calculation flips. An error in a project schedule or a technical specification can disrupt an entire team.

In those situations, I look for claims that can be independently confirmed in under two minutes. If the model references a specific tax regulation or software library version, I open the official documentation or statute and confirm it directly. Spending ninety seconds verifying a specific claim is trivial compared to the time required to undo a flawed project decision.

Consider how a failure would manifest

Before accepting a complex recommendation, I ask myself what an erroneous response would look like in practice.

If an AI tool makes an arithmetic calculation or formats a spreadsheet formula, an error is usually obvious. The numbers do not balance, or the formula throws a syntax error. The feedback loop is immediate and visible.

The subtle, dangerous errors occur in qualitative analysis. If you ask a model to summarize the risks of adopting a new software framework, it will readily produce five sensible-sounding risks. But what about the critical flaw it omitted? If the model fails to mention that the framework has severe memory leaks on Windows systems, the omission is invisible unless you have independent domain experience.

When dealing with high-stakes qualitative decisions, I treat the model’s output as an initial brainstorming partner rather than a final assessment. I look specifically for what might be missing, not just what was neatly summarized.

Prompting for uncertainty

One habit I have incorporated into daily use is prompting the model to identify its own weak points. At the end of a complex request, I often append a brief instruction: “Highlight the assumptions behind this recommendation, and state where uncertainty or alternative interpretations exist.”

Language models do not possess true self-awareness, but this prompt nudges the system to search for contradictory patterns in its training data. Often, this prompt will surface critical caveats: that a certain software library is deprecated, that a statutory rule varies by jurisdiction, or that a statistical sample was unusually small.

Using language models effectively does not mean trusting them blindly or dismissing them out of hand. It means treating them like capable, junior assistants: give them clear materials, supervise their output, and verify the details before you put your name on the work.