cat docs/conversation-analysis.md
All docs

Conversation analysis

Conversation analysis reviews finished conversations in the background and tells you whether the assistant was actually helpful — and, when it wasn't, what was missing from your knowledge base. The aggregate signals appear in Analytics; the conversations that fell short collect in the Knowledge gaps work queue.

Turning it on

Conversation analysis is off by default and enabled per agent. Open an agent and switch on Analyze conversations. From then on, once a conversation has been idle for a while, it is reviewed once and given:

  • a helpfulness verdict — helpful, partial, or unhelpful;
  • a knowledge gap flag with a short summary, when the assistant couldn't answer because the material wasn't in its library.

Each review draws from your monthly allowance — one message credit by default, and the agent form shows the exact amount when you turn it on.

Where the results appear

The signals are split between two places. Analytics (/analytics) holds the aggregate view, scoped by the page's date range and agent filter:

  • Headline figures — how many conversations were analysed, the share judged helpful, and how many surfaced a knowledge gap.
  • Gaps per day and % helpful per day — the gap rate and helpful share trended across the range.
  • Helpfulness — the helpful / partial / unhelpful split for the range.
  • Quality by agent — the same breakdown per bot.
  • Content moderation — the moderation breakdown is shown here too, whenever there is data (independent of whether Conversation analysis is on).

Knowledge gaps (/knowledge-gaps) is the work queue: every conversation where the bot fell short, each with the missing-topic summary inline. Filter by Open / Resolved / All, open one to read the transcript, mark a gap resolved once you've added the missing source, or export the list as CSV.

Closing the loop

A knowledge gap is a prompt to extend your library: add the missing answer as a source — a PDF, a GitHub Markdown file, or a web page — and the next visitor who asks gets a grounded reply instead. Then mark the gap resolved. See Libraries and sources.

If Analytics and Knowledge gaps show empty states, no conversations have been analysed yet — switch on Analyze conversations for an agent and let a few conversations finish.

See also: Agents and Libraries and sources.