Back to research
Research paper

Decision Confidence for Product Audits

Qutis AI Research Team·September 24, 2026·Published

AbstractWhen a product audit is scored, three different signals are on the table: how much of the label the ingredient index can actually vouch for, how much context the user has given us, and — when a skin photo was taken — how sure the vision model is about what it saw. This paper defines decision confidence, the single number that blends those signals per personalization mode, caps it by how much personalization the user's journey actually earned, and maps the result to guidance tone. It complements our confidence estimation framework for skin audits, which derives vision confidence itself from token log-probabilities; here we take that score as an input and show how it combines with ingredient evidence and stated context for cosmetic product review.

Three scalars feed the blend

Before a product audit runs, the agent has already gathered what it can: structured profile facts from intake — goals, allergies, current products, skin type, pregnancy and breastfeeding status — and, when the user took a skin photo, a structured audit of concern, condition signals, and derived needs.

Three scalars come out of that intake, each normalized to [0, 1]. Vision confidence V is the geometric mean of the model's token probabilities on its own observation span, exactly as derived in the companion confidence-estimation paper. Audit completeness K is the share of four structured skin-audit checks that were actually filled in. Profile richness h simply records whether the product request carries any stated user context at all.

Stated profile feeds h. The photo path feeds V and K. None of the three is decision confidence by itself — each enters differently depending on which personalization mode the request falls into.

Choosing the personalization mode

Personalization mode m fixes how much skin and profile context a product audit is allowed to lean on, and which of the three scalars enter the blend at all. It is decided once, up front, from the shape of the user's journey — never from decision confidence itself, which would be circular.

Four rules are checked in order, and the first match wins. If the user explicitly declined a skin snapshot, the mode is generic. Otherwise, if a fresh skin audit produced usable observations alongside the product review, the mode is personalized. Otherwise, if there is no fresh audit but the stated profile is non-empty, the mode is profile_only. If none of that holds — no audit, and a profile too sparse to use — the mode falls back to generic by default.

personalized covers a skin analysis taken together with a product review. profile_only covers a product review on its own, with a profile already on file. generic covers a product review with minimal or no profile, or an explicit skip of the skin photo — the same label for two different reasons a request ends up context-poor.

A ceiling on how personalized the answer may sound

Even with a perfect ingredient match, the copy should never sound more personalized than the user's own journey supports. Each mode therefore caps the reported score before it reaches the user, regardless of how the raw blend comes out.

personalized carries no ceiling of its own — a full skin-audit path can report full confidence. profile_only is capped at 0.65: a stated profile earns real weight, but without a fresh photo the system should never sound fully certain. generic is capped at 0.50, keeping tone hedged whenever the request arrived with little or no context.

πm={1.00m=personalized0.65m=profile_only0.50m=generic\pi_m = \begin{cases} 1.00 & m = \text{personalized} \\[4pt] 0.65 & m = \text{profile\_only} \\[4pt] 0.50 & m = \text{generic} \end{cases}

Mode ceiling πₘ. Applied after the raw blend is computed — see below.

Evidence coverage from the ingredient index

Once the INCI list is read off the label, or supplied by the user, the rules engine looks up every line in the CIR ingredient index. Evidence coverage measures how much of that list resolves to an indexed ingredient with retrievable safety data — not how severe any rule finding is, only whether we can cite indexed evidence for each name on the label.

For N ingredients on the request, let indexed(i) be true when line i resolves to a stable indexed id. Three lines with two indexed, for example, gives E = 2/3.

E={0N=01N∑i=1N1[indexed(i)]N≥1E = \begin{cases} 0 & N = 0 \\[6pt] \dfrac{1}{N}\displaystyle\sum_{i=1}^{N} \mathbb{1}[\mathrm{indexed}(i)] & N \ge 1 \end{cases}

Evidence coverage E, clamped to [0, 1]. In v1, E carries 40% of the blend in personalized mode and 55% in profile_only and generic.

Audit completeness, the other half of context

Profile richness h is the simpler of the two remaining scalars: it is set to 1 if the product request carries any of concerns, symptoms, skin type, age range, notes, escalation notes, or a product name, and 0 otherwise. It feeds context score C only in profile_only mode.

Audit completeness K does the equivalent job for personalized mode. It is the fraction of four structured audit checks that pass — primary concern, condition severity, condition signals such as inflammation, barrier state or pattern type, and at least one derived need — so K can only ever land on one of five values.

K=#{audit checks passed}4,K∈{0, 14, 12, 34, 1}K = \frac{\#\{\text{audit checks passed}\}}{4}, \qquad K \in \left\{0,\ \tfrac14,\ \tfrac12,\ \tfrac34,\ 1\right\}

Audit completeness K, over the four structured checks defined above.

Context score by mode

Context score C is the single number that carries user and audit context into the decision-confidence blend. It draws on h, V and K from above, but is not equal to any one of them — how it is built depends entirely on which mode is active.

In generic mode there is no audit to draw on, so C sits at a fixed neutral prior of 0.5. In profile_only mode, C starts at 0.55 for a sparse request and rises to 0.85 the moment h flips to 1 — a stated profile is worth a meaningful jump even with no fresh photo. In personalized mode, C is the average of vision confidence and audit completeness, capped at 1; if both V and K happen to be zero, C falls back to a floor of 0.35 rather than reporting zero context.

C={0.5m=generic0.55+0.30 hm=profile_only0.35m=personalized, V=0, K=0min⁡ ⁣(1, V+K2)m=personalized, otherwiseC = \begin{cases} 0.5 & m = \text{generic} \\[6pt] 0.55 + 0.30\,h & m = \text{profile\_only} \\[6pt] 0.35 & m = \text{personalized},\ V = 0,\ K = 0 \\[6pt] \min\!\left(1,\ \dfrac{V + K}{2}\right) & m = \text{personalized},\ \text{otherwise} \end{cases}

Context score C. Note that in personalized mode, V also re-enters the blend directly (below) — model certainty affects the score twice, once through C and once on its own.

The raw blend

Decision confidence combines evidence coverage E, context score C, and — only in personalized mode, where a fresh photo makes it meaningful — vision confidence V. The weights themselves depend on mode: personalized mode spreads them across all three signals, while profile_only and generic collapse onto just evidence and context, since there is no vision term to add.

Draw={0.40 E+0.35 C+0.25 Vm=personalized0.55 E+0.45 Cm∈{profile_only, generic}D_{\mathrm{raw}} = \begin{cases} 0.40\,E + 0.35\,C + 0.25\,V & m = \text{personalized} \\[6pt] 0.55\,E + 0.45\,C & m \in \{\text{profile\_only},\ \text{generic}\} \end{cases}

Raw decision-confidence blend Dᵣₐᵥ. Every input lies in [0, 1], so Dᵣₐᵥ does too.

Where the 42.5% vision weight comes from

In personalized mode, when audit data exists, C is itself built from (V + K) / 2. Substituting that into the raw blend collapses the three declared weights into two effective ones on V and K, which is worth making explicit rather than leaving buried in the algebra — it is the same formula, just regrouped.

The declared weight on vision is 25%, but half of the 35% weight on context is also vision underneath. Added together, the model's own certainty about what it saw ends up carrying 42.5% of the final score in personalized mode — nearly double what the headline weight alone would suggest.

Draw=0.40 E+0.425 V+0.175 KD_{\mathrm{raw}} = 0.40\,E + 0.425\,V + 0.175\,K

0.425 = 0.25 (direct vision weight) + 0.35 × ½ (vision\u2019s share inside context). Equivalent to the raw blend above whenever C = (V + K) / 2.

Applying the mode ceiling

The raw blend is not what gets logged or shown. It is capped by the mode ceiling from earlier, so a request can never report more certainty than its own journey earned — a personalized-sounding answer never comes out of a generic request, no matter how strong the evidence happens to be.

Dc=min⁡ ⁣(Draw, πm),Dc∈[0,1]D_c = \min\!\left(D_{\mathrm{raw}},\ \pi_m\right), \qquad D_c \in [0, 1]

Final decision confidence Dc. Logged as decision_confidence and used for the tone routing below.

From decision confidence to guidance tone

Dc never touches the rules engine's own safety findings — a rule that fires still fires regardless of how confident the surrounding report sounds. What it controls is how assertive the user-facing product report is allowed to be, through three tiers.

At 0.75 and above, confidence is high and guidance is full: clear safety themes, and a concise recommendation wherever the evidence actually supports one. Between 0.50 and 0.75, confidence is moderate and guidance turns cautious: hedged language, evidence gaps called out explicitly, no definitive fit verdicts. Below 0.50, confidence is low and guidance shifts to clarify: a short report that stresses the uncertainty and asks the user to confirm the product or send a clearer label photo.

Personalization mode still constrains what the report is allowed to say — vision-based, profile-based, or generic disclaimers. Guidance tone constrains how confidently it says it. The two answer different questions and are set independently.

guidance(Dc)={fullDc≥0.75cautious0.50≤Dc<0.75clarifyDc<0.50\text{guidance}(D_c) = \begin{cases} \text{full} & D_c \ge 0.75 \\[4pt] \text{cautious} & 0.50 \le D_c < 0.75 \\[4pt] \text{clarify} & D_c < 0.50 \end{cases}

Guidance mode as a function of final decision confidence.

A worked example

Three INCI lines on the request, all three indexed: E = 1. A skin photo was taken, and the vision model's geometric mean over its observation span comes out to V = 0.80. All four audit checks passed, so K = 1, which puts context score C at min(1, (0.80 + 1) / 2) = 0.90.

Feeding those into the personalized-mode blend gives 0.40(1) + 0.35(0.90) + 0.25(0.80) = 0.40 + 0.315 + 0.20 = 0.915 — and the expanded form agrees exactly: 0.40(1) + 0.175(1) + 0.425(0.80) also comes to 0.915. personalized mode carries no ceiling, so the final score is Dc = 0.915: high confidence, full guidance.

Draw=0.40(1)+0.35(0.90)+0.25(0.80)=0.915 ⇒ Dc=0.915D_{\mathrm{raw}} = 0.40(1) + 0.35(0.90) + 0.25(0.80) = 0.915 \ \Rightarrow\ D_c = 0.915

Worked example: E = 1, V = 0.80, K = 1, C = 0.90 → high confidence, full guidance.

References

  1. A Confidence Estimation Framework for Skin & Product Audits — Qutis AI Research Team. Derives the vision confidence score V used as an input here.

QUTIS AI IS A SKINCARE AUDIT AND ADVISORY SERVICE. IT IS NOT A MEDICAL ASSESSMENT TOOL AND DOES NOT REPLACE PROFESSIONAL MEDICAL CONSULTATION.