Skip to main content

​
Analyzing Privilege Errors

Before the Prediction Overview and Error Analysis tabs have data, you need to run a feedback batch. This instructs the system to compare the AI’s predictions against your reviewers’ decisions and generate explanations for any mistakes. To run feedback, select the documents you want to evaluate and click Analyze Errors. The process runs in the background. Once complete, all three tabs will reflect the results. You can re-run feedback any time to refresh the data when additional review work has been completed.

​
Model Metrics

This tab shows how your classification model is performing based on a set of documents that have already been reviewed and coded. Summary cards at the top:
  • Total coded documents: the number of documents that have been manually reviewed.
  • Total coded positive: how many of those were marked as privileged.
  • Total coded negative: how many were marked as not privileged.
Relevance Score chart — a bar chart showing how many documents fall into each confidence range. Bars to the left of the cutoff threshold are classified as negative (not privileged); bars to the right are classified as positive (privileged). You can drag the threshold slider to explore how changing the cutoff affects your results.

​
Prediction Overview

This tab gives you a high-level summary of what the AI predicted across your entire document set, and how much of that work has been reviewed by a human. Summary cards:
  • Total Documents: the total number of documents the AI has scored.
  • Privileged: documents the AI predicted as privileged.
  • Non Privileged: documents the AI predicted as not privileged.
Legal Tags Analysis chart: a stacked bar chart breaking down predictions by legal tag type (e.g., attorney-client privilege, work product, etc.). For each tag:
  • The green portion shows documents that have been manually reviewed.
  • The gray portion shows documents that have not yet been reviewed.
This chart helps you understand which tags still have unreviewed predictions.

​
Error Analysis

This tab is where you can dig into the AI’s mistakes, understanding why errors occurred

​
Filters

Three filters at the top let you focus on a specific subset of results:
  • Tags: narrow down to a specific privilege type (e.g., “Privileged”).
  • Error Type: filter by:
    • All Errors: show everything
    • False Positives: documents the AI called privileged, but a reviewer said were not
    • False Negatives: documents the AI missed (called not privileged, but a reviewer said were)
  • Confidence Level: focus on predictions the AI was Low (0–60%), Medium (60–80%), or High (80–100%) confident about.

​
Error Categories Distribution

A pie chart showing which error categories appear most frequently across all mistakes. Each slice represents a category of reason the AI got it wrong (e.g., “keyword overweighting,” “legal context overlooked”). Hover over any slice to see the exact count and percentage. Error categories can be customized in the Configuration tab.

​
Error Analysis Table

A detailed row-by-row table of each document the AI was evaluated on. Columns include: You can apply additional filters using the filter bar.