Searchestradocs

Methodology · beta

AI Narrative methodology

How Searchestra builds the AI Narrative card: claim extraction with verbatim evidence, per-platform clusters, consensus, roles, change types, target gaps, thresholds and limitations.

Accuracy contract

AI Narrative shows how the AI answers Searchestra collects for a Prompt Universe described a brand in one scan: the attributes they attach to it, the needs it is recommended for, who it is offered as an alternative to and how this changed since the previous scan. It describes those answers. It is not a measure of what people, or "the AI" in general, think of the brand.

  • Every statement on the card is built from claims that are backed by a verbatim quote from a collected answer. The evidence drawer shows the quote, the prompt, the platform and the date.
  • Counts are kept per platform. Answers from one platform are never pooled with another, so a platform that produces longer or more answers cannot dominate the result.
  • Mentions and recommendation strength are not measured again: the card reads the same values as the visibility and Persuasion screens, so the numbers match.
  • Summary sentences are templates filled with these counts. No language model writes text shown on the card in this version.

Pipeline

  1. Collection. Each scan collects one answer per prompt and platform.
  2. Claim extraction (L1). Answers that mention a tracked brand are sent once to an extraction model with the list of tracked brands. The model proposes claims: kind, the brand it is about, a canonical English key (for example durable), polarity and a verbatim quote. An answer is processed again only when its text, the extraction method or the tracked brand list changes.
  3. Validation. Searchestra checks every proposed claim in code (next section). Only claims that pass become evidence.
  4. Clusters (L2). Valid claims with the same kind, key and condition form a cluster. Clusters are counted per platform, then compared across platforms.
  5. Card. The card combines the clusters with the existing mention and Persuasion values, decides the card status and the role by fixed rules, and stores an immutable snapshot of the result.

Claims and evidence

Claim kinds: attribute, recommended_for, not_recommended_for, audience, comparison, alternative_to and fact. Comparison and alternative-to claims are stored but do not enter the card, the role, gaps or change yet: in the first quality review they did not reach the precision threshold. A proposed claim is rejected when:

  • the quote does not appear in the answer text (quote_not_found);
  • the brand is not a tracked brand (unknown_entity), or the evidence cannot be tied to it (entity_not_linked, goes to review). A claim is tied to a brand when the brand is named in the quote, in the sentence before it, or in the heading of the section it sits under. A heading counts only when it names this brand alone: category headings and lists ("Hardware stores (Brand A, Brand B)") do not, and another tracked brand named on the way stops the link;
  • the kind or key is not valid (invalid_kind, invalid_key), or a comparison compares a brand with itself (self_comparison).

Second reading (L1). Claims that pass these checks are read once more by a reviewer model together with the whole answer. A claim stays on the card only when the answer says it about that brand. It goes to review when the brand is only named as a contrast outside the group the statement is about (verifier_group), when a recommendation is only how-to advice that names the brand as a place or source, such as "visit a showroom first", or an offer to suggest brands later (verifier_process), or when the answer does not say it (verifier_unsupported). The last case includes a key or condition the quote does not support, a quote that is only a link or source, and a statement about a third party. A statement about a group of named brands ("brands like A and B are premium") or about a product type with the brand named as its example ("ready-made modular kitchens like A or B are the cheapest") applies to each brand named. If the second reading fails, the answer is retried; unchecked claims never reach the card.

Hedged statements ("may be", "some users say") are kept with a qualifier, and statements contradicted later in the same answer are marked. Each answer gets a processing status: processed, no claims, failed or pending. Coverage on the card shows these counts.

Clusters and platforms

A cluster is carried by a platform when at least 10% of that platform's processed answers that mention the brand contain it, and at least 2 answers do. Only platforms with enough material in the scan are eligible; the others are listed as having insufficient data.

The consensus view counts, for each cluster, how many eligible platforms carry it. When some carrying platforms are mostly positive and others mostly negative, the cluster is marked as split and is never summarised as one direction. A cluster needs at least 3 answers to be shown.

Card status

  • Ready: enough processed answers that mention the brand (at least 10).
  • Limited: fewer than 10; shown with a warning.
  • Rarely mentioned: the brand appears in fewer than 5% of answered prompts. No narrative is built; the competitor view shows who carries the category.
  • Processing / failed: more than 1% of the answers that mention the brand are still pending or failed.
  • Not measured / not available: nothing collected, or claims are not extracted for this scan.

Roles

The role is decided by rules applied in order; the first rule that holds decides, and the card shows the rule and its facts. Recommendation values are the Persuasion values for the same scan.

  1. Rarely mentioned: mention rate below 5%.
  2. Unclassified: fewer than 10 answers scored for recommendation.
  3. Default choice: strongly recommended in at least 50% of scored answers and primary in at least 40%.
  4. Budget option / premium alternative: a price-value or premium attribute cluster is carried by at least 2 eligible platforms (1 when only one platform is eligible).
  5. Specialist: at least half of the recommendation claims point to one need, carried as above.
  6. Peripheral: strongly recommended in less than 20% and primary in less than 15%.
  7. Otherwise unclassified. A role is never guessed when the evidence is insufficient.

Change since the previous scan

Two scans are compared only on prompt and platform pairs that were asked with the same prompt revision. Pairs whose prompt was edited are excluded and counted; pairs collected before revisions were recorded are counted as unverifiable. A cluster is reported as moved when its share changes by at least 10 points and it has at least 3 answers. Every change carries a type:

  • Observed change: comparable pairs, same method.
  • Model or collection changed: the platform's model version changed between the scans.
  • Prompts edited, method changed (a different extraction method or extraction model; changes are not compared), comparability unknown (revisions not recorded) and no previous scan.

Desired position and gaps

A brand can approve a target set per market: the attributes it wants answers to associate with the brand, and the ones to avoid. Sets are versioned; an approved set never changes, edits create a new version with an effective date, and each scan is read with the version in effect on its day.

Each target is matched (carried with the wanted polarity), weak (seen but not carried), contradicted (the opposite cluster is carried, for example expensive against affordable) or not observed. Contradicted and not observed targets create an Action Center item with the evidence link.

Thresholds

ValueThreshold
Carried on a platform≥ 10% of mentioning processed answers and ≥ 2 answers
Cluster shown≥ 3 answers
Splitcarried positive on one platform and negative on another
Mixed polarity within a clusterminority polarity ≥ 30%
Ready card≥ 10 processed answers that mention the brand
Rarely mentionedmention rate < 5%
Processing or failed> 1% pending or failed
Reported changeshare change ≥ 10 points, ≥ 3 answers

Models and cost

The extraction model and its configuration are recorded on every processed answer; the method line on the card shows the version used. A change of model or prompt is a method change and is shown as such. Market, brand and organization views give each eligible universe (or market) one vote; the answer-weighted share is shown next to it only for comparison.

The feature is in a closed beta. Extraction runs only for organizations that joined it.

Limitations

  • The card describes a sample: the prompts in the universe, the platforms collected and the day of the scan.
  • A quote proves that an answer said something; it does not prove that the statement is true. Factual errors are tracked separately in Portrayal findings.
  • Canonical keys can split one idea into close synonyms (for example luxury and premium_brand); they are shown as separate clusters.
  • Citations shown next to the narrative are sites cited in the same answers. They are associated with the narrative, not proven to cause it.
  • Answers collected before prompt revisions were recorded cannot be compared with certainty.