How Gumshoe Works

AI model responses are non-deterministic. That is a technical way of saying the same question, asked twice, produces different answers. Different brands get mentioned. Different sources get cited. Different recommendations emerge.

This is not a bug. It is how large language models work. They are probabilistic systems, meaning every answer is a weighted likelihood rather than a fixed lookup. And it has a direct consequence for measurement: if your methodology does not account for that variability, your data is meaningless.

A single ChatGPT screenshot is not data. It is an anecdote. Building a GEO (Generative Engine Optimization) strategy on anecdotes is the fastest way to waste time and budget on the wrong priorities.

The good news is that variability is tractable. Individual answers shift run to run, but the proportion of responses where a brand appears converges to a stable signal once you run enough conversations. Visibility behaves like a well-defined probability that can be estimated with standard statistical tools. See our blog article, Exploring Variability in AI-Generated Responses: Consistency or Chaos?, for more detail.

Everything about how Gumshoe is built is designed to turn a probabilistic system into statistically reliable data you can actually act on.

If you, your grandmother, and your 13-year-old niece all asked an AI model the same question, you would get three very different answers. The model factors in who is asking, what it knows about them, and what it thinks they care about. Without persona context, you are not getting an accurate read on what your buyers see. You are getting an averaged answer for a person who does not exist.

This is why every Gumshoe report starts with personas.

We go to AI models and ask who is most likely to be researching your brand or category. They come back with specific buyer profiles, grounded in real situations rather than broad labels like "small business owner." You review them, edit any that do not fit, and confirm they reflect your actual buyers. Then we run hundreds of conversations as those personas, querying every model in their voice, with their context built in.

Other GEO tools either skip personas entirely or label prompts after the fact based on who they guess might have asked. Without persona context built into the question itself, you are running anonymous, logged-out queries that produce more hallucinations and less specific answers. Those tools have to run thousands of generic prompts to average out the noise, and even then, they are measuring the average answer for an average person, which is not a person who exists. Gumshoe builds the persona first and gives the model that context up front, so every conversation reflects what a real buyer in your category sees.

Imagine trying to study how a restaurant cooks its signature dish by watching through the kitchen window. You would catch fragments, miss the ingredients you cannot see, and your view would change every time someone walked in front of the glass. That is browser scraping. It tells you something, but not enough, and not reliably. APIs let us walk through the front door with permission and ask the chef directly.

This is why every Gumshoe report runs through official APIs.

We connect to every AI model through its official API (the structured, permissioned way developers connect software to a model). Through that channel, we can engineer system messages that replicate real user behavior: memory, custom instructions, conversation history, and persona context. The data comes back clean, structured, and free of interface contamination. Gumshoe also validates against the native interfaces to confirm we are seeing what real users see.

Other GEO tools rely on browser scraping, which automates a real browser to read AI answers the way a person would. That approach violates most models' terms of service and pulls in personalization, caching, and UI artifacts that pollute the data. To replicate logged-in personalization, scraping tools need sock puppet accounts (fabricated identities with maintained history and activity), which are detected with 89-95 percent accuracy and are ethically questionable to operate. The productive question in this space is not "API versus scraping." It is whether you can engineer API calls to replicate interface behavior while gaining the nuance of personalized results. We have built our entire methodology around that approach, so Gumshoe data reflects what real users actually experience.

Ask an AI model the same question ten times, and you might get ten different answers. Different brands mentioned. Different sources cited. Different recommendations emerging. Trying to draw conclusions from a single run is like calling a coin biased after one flip. The variability is real, and it is the hardest problem in measuring AI visibility.

This is why every Gumshoe report is built on breadth and statistical aggregation, not single snapshots.

Every report spans three dimensions: a set of prompts (the questions your buyers actually ask), a set of personas (the buyers who ask them), and a set of models. A single evaluation across all combinations is a single report run. The default configuration produces around 800 conversations per run, enough to estimate your overall visibility within ±5 percentage points at 95 percent confidence. That is a defensible statistical claim, not a marketing one. Individual conversation results are then aggregated across all prompts for a persona, across all personas in the report, and across all models you are tracking. The final visibility score represents the probability that your brand appears in AI responses for your target audience, rather than a cherry-picked example. Visit How Much Data Do You Need to Measure AI Visibility with Confidence? for the math.

Other GEO tools run thousands of generic prompts and report a single averaged number with no breakdown. Gumshoe gives you a defensible overall score and the underlying structure to slice into it: by persona, by topic, by model. A single run gives a strong directional signal at the overall level. Per-topic and per-persona breakdowns have wider confidence intervals, and we recommend multiple runs when those slices need to drive a decision.

Most GEO tools tell you where you are invisible. Gumshoe tells you why and gives you the tools to fix it inside the same platform. You can use three levers to take action and change your AI visibility.

Transparency matters in a new category where vendors are making big claims. Here is what we do not do.