Skip to main content
Use these best practices to improve coding consistency, reduce rework, and keep output usable across matters.

​
Start from the protocol

  • Write or confirm the review protocol before configuring the workflow.
  • Make each issue a distinct tag with its own instruction, rather than one broad tag covering several concepts.
  • State what falls outside each tag as well as what falls inside it.
  • Link the protocol to the workflow and import from it, rather than retyping instructions, so each instruction remains traceable to its source.
The tag is how a finding is counted later. Two issues collapsed into one tag cannot be separated afterwards.

​
Plan the pipeline before you build it

  • Decide which passes the review needs and what each one answers.
  • Keep passes that answer different questions in separate steps writing separate columns.
  • Split a single large catalogue across steps that share a column, so it still reads as one coding decision.
  • Add field extraction only when you need structured values per document, not by default.
A pipeline that mirrors the protocol’s structure is easier to review, correct, and explain.

​
Start with clean inputs

  • Use a dataset that matches the review scope.
  • Verify that documents contain usable text for the coding decisions being made.
  • Remove obvious duplicates before running the workflow.
  • Apply filtering before import rather than coding documents you intend to exclude.

​
Test on a sample first

  • Run the classification steps on a pilot set before using the full dataset.
  • Check the distribution across tags against what you expect from the population.
  • Read the reasoning on a representative mix of documents for each tag.
  • Confirm that documents receiving several tags are ones that genuinely raise several issues.
A tag that catches almost everything, or almost nothing, is an instruction problem. Finding it on a sample costs one run.

​
Configure incrementally

  • Start with the default general instruction and change one thing at a time.
  • Test each change before making further adjustments.
  • Record what changed and why, so a result can be traced to the configuration that produced it.
  • Keep tag names stable once review has started — renaming mid-review makes earlier results hard to reconcile.

​
Review output at each stage

  • Validate the coding before running field extraction over it.
  • Review the documents carrying several tags first; they are where instructions overlap.
  • Check the tags that returned nothing. An issue with no documents is a finding about the matter, not necessarily a failure.
  • Inspect documents where the model’s reasoning does not match the tag it applied.
Early review prevents coding errors from carrying into extracted fields and downstream review.

​
Read metrics per tag

  • Evaluate each tag in its own right. An issue catalogue has no single right answer, so an overall accuracy number says little.
  • Use a control set of reviewed documents so precision and recall are measured against a known baseline.
  • Expect tags to perform differently. Concrete, well-bounded issues outperform broad ones, and the fix for a weak tag is usually its instruction.
  • Prioritize the tags that carry the most consequence for the matter, not the ones with the most documents.

​
Use reviewer feedback to improve results

  • Capture repeated correction patterns from reviewers.
  • Refine an instruction when the same issue appears across runs, rather than correcting documents one at a time.
  • Update the protocol when a definition genuinely changes, then re-import and re-run so the workflow and the protocol stay in step.
  • Recheck edge cases after making changes.

​
Reuse what works

  • Save successful configurations for similar matters.
  • Keep protocols and instruction wording that have proven reliable, and adapt them per matter.
  • Compare results across matters to spot recurring issues.
  • Keep internal notes on validated instruction patterns.