Intro
Quarterly retests keep the operating picture honest. They do not, on their own, answer a different question: is the measurement system itself still valid?
The annual AI visibility audit is the strategic checkpoint. It reviews the methodology, refreshes the assumptions baked into the prompt set, recalibrates the competitor set, validates the engine mix, and produces a board-pack summary suitable for asset managers, boards and group executives.
It is not a baseline — that is a one-off start, covered in how to run an AI visibility baseline. It is the annual checkpoint that sits above the quarterly cadence and ahead of next year’s plan.
Why quarterly is not enough
A quarterly retest compares like for like. Same prompts, same competitors, same engines, same scoring rubric. That comparability is exactly what makes it useful for operations — and exactly why it eventually drifts from reality.
Three forces erode the quarterly picture over a full year:
- Market shift. New themes emerge in the buying journey. Sustainability, wellness, family travel, longevity — categories that did not register twelve months ago can move into the top of the funnel. A prompt set frozen last January will under-sample them.
- Competitive shift. New entrants appear, incumbents reposition, brands are acquired or rebranded. A competitor set that was sharp twelve months ago will contain dead entries and miss live threats.
- Engine shift. Models update. New engines enter the consideration set — Mistral, Claude, regional players — and existing engines change how they cite sources. A quarterly retest on last year’s engine mix is no longer measuring the live answer surface.
The annual audit is the moment to surface and price these drifts, rather than absorb them silently into a “comparable” number.
Six annual checkpoints
The Capston Core annual audit runs six checkpoints, in this order.
1. Prompt set evolution
The prompt library is reread end to end. For each prompt, three questions: is the intent still live in the market, is the phrasing still how buyers ask it, and is the answer surface still meaningful. Dead prompts are retired with reasons logged. New prompts are proposed against current buying themes and validated against the prompt set methodology.
2. Competitor set refresh
The competitor list is reviewed against the live market. Exits, mergers and rebrands are removed. New entrants — independents that have scaled, foreign brands that have entered the market, adjacent categories that now compete for the same answer — are added with the same evidence threshold used in the competitor set methodology.
3. Engine behaviour shifts
The engine mix is rebenchmarked. Which models have shipped major updates, which new engines have crossed the relevance threshold, which have lost ground. Answer surfaces are re-probed on a control set of prompts to confirm that citation behaviour, source preference and answer length have not silently shifted underneath the score.
4. Brand positioning shifts
The brand’s own year is documented: new properties, refreshed concept, M&A, ownership change, rebrand, new markets. Each of these changes the questions a buyer will plausibly ask. The prompt set is re-aligned to the brand the audit is actually measuring, not the brand it was measuring last January.
5. Source authority drift
The URLs and domains AI engines cite for the brand and its competitors are pulled and compared year over year. Which sources gained authority, which lost it, which are new in the answer set, which dropped out. This is the upstream view of the answer surface and the input into next year’s content and PR plan.
6. Next-year prompt plan
The audit closes with a forward plan: which prompts are locked for next year’s quarterly retests, which are on watch, which themes are scheduled to enter the set if they cross a defined threshold. The plan is signed off the same way next year’s budget is.
What the annual report contains
The deliverable is a single document built to board-pack standard.
- Executive summary — the year in one page: composite movement, biggest gain, biggest gap, biggest risk.
- Score evolution — each of the eight scoring dimensions, year over year, with commentary on what moved and why, referenced against the AI visibility scoring framework.
- Methodology review — what changed in the prompt set, the competitor set and the engine mix, with reasons logged.
- Competitive landscape — who gained share of answer, who lost it, who entered, who exited.
- Source map — domains and properties cited by AI engines for the brand and its competitors, with year-over-year drift.
- Risk register — commercial risks the answer surface has surfaced over the year: OTA capture, aggregator routing, intermediary substitution.
- Next-year plan — prompt set, competitor set, engine mix, retest cadence, content and PR priorities.
The report is built once a year. The version is dated, archived and citable in subsequent years.
Who runs the annual audit
The annual audit is a senior exercise. Capston Core leads it, with three roles on the brand side:
- A senior brand or marketing sponsor, who validates the prompt set and the positioning narrative.
- A commercial or revenue lead, who validates the competitor set and the risk register.
- A board or asset manager contact, who receives the executive summary and signs off the next-year plan.
The audit window is typically three to four weeks. The output is a single dated report, an updated prompt library, an updated competitor list, and a signed-off next-year plan that resets the quarterly cadence for the following twelve months.
How this fits into Capston Core
The annual audit sits at the top of the Capston Core cadence. The quarterly retest is the operating loop. The annual audit is the strategic loop that recalibrates the operating loop.
It depends on the Capston Core methodology for the five-stage process, on the AI visibility scoring framework for comparability across years, on the baseline as the historical anchor, and on the prompt and competitor methodologies as the standards the audit recalibrates against.
→ Back to Capston Core
FAQ
How is the annual audit different from a baseline?
A baseline is the one-off start, run when there is no prior measurement. The annual audit is the yearly checkpoint that recalibrates an existing measurement system. They use the same methodology; they sit at different points in the lifecycle.
Does the annual audit replace the quarterly retest?
No. The quarterly retests run as scheduled. The annual audit runs over and above them, typically aligned to the brand’s planning cycle, and feeds back into how the next four quarterly retests are run.
What if the brand has changed significantly in the year?
That is precisely what the annual audit is built for. New properties, rebrand, M&A and new markets are reasons to recalibrate the prompt set and the competitor set rather than reasons to skip the audit.
Who is the report written for?
The executive summary is written for boards, asset managers and group executives. The body is written for the brand or marketing sponsor and the commercial lead. Both audiences read the same document.
Final CTA block
Recalibrate the year. Book a Capston Core annual audit.
Book an annual audit
Read the methodology