Skip to main content
The Metrics tab of a model answers two different questions, and it is worth keeping them apart:
  • How good is this model? Answered by a control set — a set of documents a human has coded, measured as precision and recall.
  • What am I leaving behind if I stop here? Answered by an elusion test — a sample of the documents the model called negative, to estimate how many positives are hiding below the cutoff.
Everything else on the page — the score histogram, the custodian breakdown, the coded-document counts — describes the population the model scored, not how well it scored them.

​
The cutoff threshold

Every classification model produces a score between 0 and 1 for each document. The cutoff threshold is what turns that score into a decision:
If the predicted score is larger than the cutoff, the document is classified as positive; otherwise it is classified as negative.
The threshold is a setting you choose, not a property of the model. Every metric on the page is measured at a threshold, so moving the slider changes precision, recall, elusion, and the document counts together. The sliders on each panel let you explore; use Update threshold in the action bar to save the one the model should actually run at. Score buckets are rounded to two decimals, which gives the histogram a hundred buckets rather than one per distinct score.

​
Reading the top of the page

Below the cards, the Prediction Score histogram shows how many documents fall in each score bucket, colored on either side of the cutoff. Further down, Predicted positive by scores and Predicted positive by custodians break the predicted-positive population down by score band and by custodian, with how many have already been reviewed and confirmed.

​
Control sets

A control set is a held-out slice of coded documents that the model is scored against. It produces the confusion matrix and the precision and recall figures in the Evaluation metrics (Precision & Recall) panel.

​
Creating one

Select Create control set and give it: If you do not know what sample size you need, select Estimate Sample Size and supply a confidence interval (for example 0.95), a margin of error (for example 0.05), and an expected richness — the proportion of documents you expect to be positive. Epiq AI returns the sample size that supports those parameters. A control set can also be built over all labeled data rather than a named sample. That set names no specific documents: every coded document the model scored is in it.

​
What “held out” means, and why it matters

A model scored on documents it was trained on will look better than it is. Epiq AI will not report that number. For each document in a control set, it looks for an honest score:
  • An out-of-fold score, where a k-fold run scored the document using a fold that did not train on it, or
  • The corpus score, for documents that were never in the training set at all.
The labels are read from the documents’ current coding, not from whatever the training run was fitted against. Re-running a control set after reviewers retag documents therefore reports the coding they can see today.
Excluding a control set’s dataset from training is the ordinary way to protect it. Epiq AI reads that set’s labels per document precisely so that protecting it does not empty out the measurement.

​
What the panel reports

The confusion matrix counts four cells at the cutoff: From those:
  • Precision = TP / (TP + FP) — of the documents predicted positive, how many really are.
  • Recall = TP / (TP + FN) — of the documents that really are positive, how many the model found.
Where a denominator is empty, the rate is reported as not available rather than as zero: precision with no predicted positives is undefined, not 0%. Alongside the matrix, the panel draws a Prediction Score histogram and a Precision-Recall Curve for the control set. The curve’s confidence intervals come from a bootstrap that resamples the slice, and it needs at least 100 positive and 100 negative documents to run.
On a control set too small to bootstrap, the curve is skipped and its precision and recall read as zero. The confusion matrix is still valid — it is a straight count and does not need resampling. Read the matrix, not the curve, on a small set.
If a control set shows a Needs Rerun prompt, its evaluation is outdated — the model has a new version, or the coding has moved on. Re-run it before quoting the numbers.

​
Elusion tests

An elusion test answers the question a control set cannot: of everything the model called negative, how much was actually positive? It is what supports a defensible decision to stop reviewing.

​
How it works

  1. Epiq AI takes the documents scoring below the cutoff — the ones you would not review.
  2. It draws a random sample of them.
  3. Reviewers code that sample.
  4. The elusion rate is the proportion of that sample that turned out to be positive.

​
Creating one

Select Create Elusion test set and give it: Selecting Estimate Sample Size shows a slider of candidate sample sizes with the lower and upper elusion rate each one would support, so you can see directly what a larger sample buys you before committing reviewers to it.

​
Reading the result

The panel reports two numbers, and they are not the same thing:
  • Actual elusion — the rate measured in the sample. A plain count: positives found divided by documents sampled.
  • Estimated elusion range — a Clopper-Pearson 95% confidence interval, the range the true rate in the whole predicted-negative population is expected to fall within.
The interval is what a defensibility argument rests on; the point estimate on its own says nothing about how far it might be off. Moving the cutoff slider on this panel re-computes the result for that cutoff, which is the fastest way to see what raising the cutoff would cost you in missed documents.

​
Documents that carry no score

Not every document ends up with a prediction. Those are shown on the Predictions tab behind the Not scored toggle, with a count beside it, and they are excluded from the score arrays entirely. This is deliberate. Writing an unscored document into the predictions as 0.0 would make it indistinguishable from a confident negative — it would be counted as a true negative in the confusion matrix, drag elusion estimates down, and tell reviewers the model had judged a document it never saw. Each unscored document carries a reason, which you can filter and export on: The first two are properties of the document. The last three are things that happened to a particular run, and are worth re-running or raising with support.

​
Suggested order of work

  1. Build a control set from a coded review batch, sized with the estimator rather than guessed.
  2. Read the confusion matrix first. On a small set, the curve may not be available.
  3. Move the cutoff and watch precision and recall trade against each other. Save the one that matches what the matter needs — a defensible production usually favors recall.
  4. Run an elusion test at that cutoff and read the confidence interval, not just the point estimate.
  5. Check the Not scored count so you know what population the numbers actually cover.
  6. Where the model is wrong often enough to matter, run error analysis and fix the instructions or sample a targeted review batch to review and then re-run.