> ## Documentation Index
> Fetch the complete documentation index at: https://labs.laer.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Model metrics
> >- How Epiq AI measures a classification model — the cutoff threshold, control sets, elusion tests, documents that carry no score, and the error categories used to explain mistakes.
The **Metrics** tab of a model answers two different questions, and it is worth keeping them apart:
* **How good is this model?** Answered by a **control set** — a set of documents a human has coded, measured as precision and recall.
* **What am I leaving behind if I stop here?** Answered by an **elusion test** — a sample of the documents the model called negative, to estimate how many positives are hiding below the cutoff.
Everything else on the page — the score histogram, the custodian breakdown, the coded-document counts — describes the population the model scored, not how well it scored them.
***
### The cutoff threshold
Every classification model produces a **score between 0 and 1** for each document. The **cutoff threshold** is what turns that score into a decision:
> If the predicted score is larger than the cutoff, the document is classified as positive; otherwise it is classified as negative.
The threshold is a setting you choose, not a property of the model. Every metric on the page is measured **at a threshold**, so moving the slider changes precision, recall, elusion, and the document counts together. The sliders on each panel let you explore; use **Update threshold** in the action bar to save the one the model should actually run at.
Score buckets are rounded to two decimals, which gives the histogram a hundred buckets rather than one per distinct score.
***
### Reading the top of the page
| Card | What it counts |
| ------------------------- | ---------------------------------------------------- |
| **Total coded documents** | Documents a human has reviewed and coded |
| **Total coded positive** | How many of those were coded positive |
| **Total coded negative** | How many of those were coded negative |
| **Predicted positives** | Documents the model scored at or above the cutoff |
| **Predicted negatives** | Documents the model scored below the cutoff |
| **…including family** | The same counts, expanded to whole document families |
Below the cards, the **Prediction Score** histogram shows how many documents fall in each score bucket, colored on either side of the cutoff. Further down, **Predicted positive by scores** and **Predicted positive by custodians** break the predicted-positive population down by score band and by custodian, with how many have already been reviewed and confirmed.
***
### Control sets
A control set is a **held-out slice of coded documents** that the model is scored against. It produces the confusion matrix and the precision and recall figures in the **Evaluation metrics (Precision & Recall)** panel.
#### Creating one
Select **Create control set** and give it:
| Field | What it does |
| ---------------- | --------------------------------------------------------------------------------------- |
| **Name** | How the set is identified in the dropdown |
| **Review Batch** | One or more review batches to draw the documents from. The combined batch size is shown |
| **Sample size** | How many documents to sample |
If you do not know what sample size you need, select **Estimate Sample Size** and supply a **confidence interval** (for example 0.95), a **margin of error** (for example 0.05), and an **expected richness** — the proportion of documents you expect to be positive. Epiq AI returns the sample size that supports those parameters.
A control set can also be built over **all labeled data** rather than a named sample. That set names no specific documents: every coded document the model scored is in it.
#### What "held out" means, and why it matters
A model scored on documents it was trained on will look better than it is. Epiq AI will not report that number. For each document in a control set, it looks for an **honest score**:
* An **out-of-fold score**, where a k-fold run scored the document using a fold that did not train on it, or
* The **corpus score**, for documents that were never in the training set at all.
The labels are read from the documents' **current coding**, not from whatever the training run was fitted against. Re-running a control set after reviewers retag documents therefore reports the coding they can see today.
Excluding a control set's dataset from training is the ordinary way to protect it. Epiq AI reads that set's labels per document precisely so that protecting it does not empty out the measurement.
#### What the panel reports
The confusion matrix counts four cells at the cutoff:
| Cell | Meaning |
| ------------------------ | ------------------------------------------------------------------- |
| **True positives (TP)** | Coded positive, predicted positive |
| **True negatives (TN)** | Coded negative, predicted negative |
| **False positives (FP)** | Coded negative, predicted positive |
| **False negatives (FN)** | Coded positive, predicted negative — the documents the model missed |
From those:
* **Precision** = TP / (TP + FP) — of the documents predicted positive, how many really are.
* **Recall** = TP / (TP + FN) — of the documents that really are positive, how many the model found.
Where a denominator is empty, the rate is reported as **not available** rather than as zero: precision with no predicted positives is undefined, not 0%.
Alongside the matrix, the panel draws a **Prediction Score** histogram and a **Precision-Recall Curve** for the control set. The curve's confidence intervals come from a bootstrap that resamples the slice, and it needs at least **100 positive and 100 negative** documents to run.
On a control set too small to bootstrap, the curve is skipped and its precision and recall read as zero. The **confusion matrix is still valid** — it is a straight count and does not need resampling. Read the matrix, not the curve, on a small set.
If a control set shows a **Needs Rerun** prompt, its evaluation is outdated — the model has a new version, or the coding has moved on. Re-run it before quoting the numbers.
***
### Elusion tests
An elusion test answers the question a control set cannot: **of everything the model called negative, how much was actually positive?** It is what supports a defensible decision to stop reviewing.
#### How it works
1. Epiq AI takes the documents scoring **below the cutoff** — the ones you would not review.
2. It draws a **random sample** of them.
3. Reviewers code that sample.
4. The **elusion rate** is the proportion of that sample that turned out to be positive.
#### Creating one
Select **Create Elusion test set** and give it:
| Field | What it does |
| ---------------------- | ---------------------------------------------------------------------- |
| **Name** | How the test is identified in the dropdown |
| **Review Batch** | Optional. Which batches the sample is drawn from |
| **Prior Elusion Rate** | The elusion rate you expect, used to size the sample (for example 0.1) |
| **Sample size** | How many predicted-negative documents to sample |
Selecting **Estimate Sample Size** shows a slider of candidate sample sizes with the **lower and upper elusion rate** each one would support, so you can see directly what a larger sample buys you before committing reviewers to it.
#### Reading the result
The panel reports two numbers, and they are not the same thing:
* **Actual elusion** — the rate measured in the sample. A plain count: positives found divided by documents sampled.
* **Estimated elusion range** — a **Clopper-Pearson 95% confidence interval**, the range the true rate in the whole predicted-negative population is expected to fall within.
The interval is what a defensibility argument rests on; the point estimate on its own says nothing about how far it might be off.
Moving the cutoff slider on this panel re-computes the result for that cutoff, which is the fastest way to see what raising the cutoff would cost you in missed documents.
***
### Documents that carry no score
Not every document ends up with a prediction. Those are shown on the **Predictions** tab behind the **Not scored** toggle, with a count beside it, and they are excluded from the score arrays entirely.
This is deliberate. Writing an unscored document into the predictions as 0.0 would make it indistinguishable from a confident negative — it would be counted as a true negative in the confusion matrix, drag elusion estimates down, and tell reviewers the model had judged a document it never saw.
Each unscored document carries a **reason**, which you can filter and export on:
| Reason | What it means |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| **No extractable text** | Nothing could be read from the document, so there was nothing to score. A processing gap |
| **No usable features** | Text was extracted, but the metadata and knowledge blocks needed to represent the document are missing |
| **No instructions to review against** | The document is readable, but the model has no instructions covering it |
| **Review returned nothing usable** | The model was asked and returned nothing that could be parsed. A failure worth chasing |
| **No trained model to score with** | Nothing was trained, so nothing scored the document |
The first two are properties of the document. The last three are things that happened to a particular run, and are worth re-running or raising with support.
***
### Suggested order of work
1. Build a **control set** from a coded review batch, sized with the estimator rather than guessed.
2. Read the **confusion matrix** first. On a small set, the curve may not be available.
3. Move the **cutoff** and watch precision and recall trade against each other. Save the one that matches what the matter needs — a defensible production usually favors recall.
4. Run an **elusion test** at that cutoff and read the **confidence interval**, not just the point estimate.
5. Check the **Not scored** count so you know what population the numbers actually cover.
6. Where the model is wrong often enough to matter, run **error analysis** and fix the instructions or sample a targeted review batch to review and then re-run.