Understanding AI Confidence Scores in Text Analytics¶

Treating every AI-classified response as equally certain is how a few genuinely wrong classifications slip into a report unnoticed. Confidence scores exist to prevent that - a signal attached to each classification telling you how sure the system was about that specific assignment, so your review time goes where it's actually needed instead of being spread evenly across responses that didn't need a second look in the first place.

What a Confidence Score Represents¶

Each classified response can carry a confidence indicator alongside its assigned category - a reading of how closely that response matched the category it was assigned to, relative to the alternatives it could have been assigned instead. A high-confidence classification means the response was a clear, unambiguous fit. A lower-confidence one means the response was a plausible fit, but closer to a judgment call than a clean match.

What It Isn't¶

A confidence score isn't a grade on the response itself, and it isn't a guarantee that a low score means the classification is wrong - it only means the case was less clear-cut than a high-confidence one. Plenty of low-confidence classifications are still correct; they're simply the ones where a human reviewer's judgment adds the most value, because the automated pass itself was less certain.

Using Confidence Scores to Focus Review¶

Rather than reviewing every classified response with equal attention, sort or filter by confidence and start with the lowest-scoring assignments first. This concentrates your limited review time on the responses most likely to need correcting, instead of spending it re-confirming classifications that were already unambiguous. See Reviewing and Correcting Individual AI Classifications for the actual review workflow.

Confidence Scores and Category Design¶

A category set with a lot of low-confidence classifications scattered across it can be a signal about the categories themselves, not just the individual responses - categories that are too similar to each other, or too broadly defined, tend to produce more borderline calls than a tighter, more distinct set. If you're seeing widespread low confidence, it may be worth revisiting your categories (see Editing and Refining Categories in Text Analytics) rather than only correcting responses one at a time.

Including Confidence Scores in Exports¶

When exporting classified results, you can choose to include confidence scores alongside each response's category assignment - useful if you want a reviewer, client, or teammate to see the same certainty signal you used during your own review. See Exporting Classified Responses with Confidence Scores.

FAQ¶

Is a low confidence score always wrong?
No - it means the classification was a closer call, not that it's incorrect. Use it as a prompt to review, not as an automatic flag for an error.

Can I set a confidence threshold below which responses are automatically flagged?
Check your Text Analytics results view for available filtering and sorting options by confidence - this is the mechanism for surfacing lower-confidence responses for review.

Do confidence scores change if I edit my categories and reclassify?
Yes - confidence is computed fresh each time classification runs, so it reflects your current category set, not a prior one.

Does a manually corrected classification keep a confidence score?
A correction you make yourself reflects your own judgment rather than the automated pass - check your results view for how a manually corrected assignment is displayed relative to one still carrying its original confidence score.

Next: see Reviewing and Correcting Individual AI Classifications for how to act on what confidence scores surface.