Skip to main content

Why AI answers change between platforms and days

The same question gives different answers because assistants decide whether to search, and what they find shifts.

Ask an AI assistant the same question twice and you'll often get two different answers. Different brands, different order, sometimes different sources — with nothing having changed in the world between the two attempts.

This is normal behaviour, and understanding it is what separates a useful AI visibility programme from a frustrating one.

The assistant decides whether to search

The biggest single factor, and the least obvious.

When you ask a question, the assistant judges whether it needs to look things up. Sometimes it searches the web and cites what it finds. Sometimes it answers from what the model already knows and cites nothing.

That decision isn't fixed per question. The same question can trigger a search on one attempt and not the next — which alone produces two very different answers, since one draws on live pages and the other on training data.

Phrasing tilts the odds. Questions about current recommendations, comparisons, and prices push toward searching, because the model has learned that its own knowledge goes stale on those. Definitional and how-to questions are answered from memory far more often.

You don't have to guess which way one of your own prompts tends to go. The searches an assistant ran, and the share of answers it produced without searching at all, are recorded for every prompt you track. See What the AIs searched for.

Search results themselves move

When a search does happen, it returns whatever ranks at that moment. Rankings shift, new articles publish, pages get updated. An answer built on live search inherits all of that movement.

The models are non-deterministic

Even given identical inputs, these systems don't produce identical outputs. Given the same three sources, an assistant might name Nike first on one pass and third on the next.

This isn't a defect. It's how the underlying technology works, and no amount of optimisation removes it.

Platforms differ genuinely

Different assistants search differently, weigh sources differently, and were trained differently. It's routine for a brand to be strong on one platform and absent from another.

Those differences are stable enough to be informative — if you're consistently missing from one platform, that's a real finding worth investigating, not noise. Usually it points to a specific source that platform trusts and you're absent from.

Personalisation, if you're testing yourself

Asking in your own account isn't a clean test. Memory of past conversations, custom instructions, and your location all shape the answer. If you've discussed your own company before, the assistant is not a neutral judge of whether it recommends you.

What this means in practice

A single check tells you almost nothing. Seeing your brand once proves it's possible, not that it's typical. Not seeing it once proves nothing at all.

Rates are the only honest measurement. "Mentioned in 30% of answers" is a real statement about your position. "Mentioned when I checked on Tuesday" isn't.

Trends beat snapshots. Day-to-day movement is mostly variance. Movement over weeks, across many checks, is signal.

Don't chase a single bad answer. The instinct after seeing one answer that omits you is to rewrite a page. Check whether the pattern holds across many answers first — otherwise you're optimising against a coin flip.

See What makes a page get cited by AI for what to do once you've established a real pattern.

Did this answer your question?