- In the Project space menu, select Workflows.
- Select your Issue Review workflow.
- Select the Input Data tab and import the dataset containing the documents to review.
- Select the Steps tab and configure the workflow’s steps.
Link a review protocol
Link a review protocol
The protocol is the master list of coding instructions for the review. Linking it lets each step import the slice of instructions it is responsible for, and records where each instruction came from.
- Open the step’s instruction input and select the Protocol tab.
- Select Select existing to choose a protocol already in the collection, or Import new to upload one.
- Select the protocol.
Issue Classification
Issue Classification
This step codes documents against a set of tags. Each tag carries an instruction stating when it applies, and a document may receive zero, one, or several tags.
- Select the Issue Classification step.
- Select Instructions and do the following:
- Review the General Instructions. This is the standard applied to every document — edit it to record how the catalogue as a whole should be applied.
- To import from the linked protocol, select the Protocol tab, select the instructions this step should carry, and select Import selected. Use Select all to take the whole protocol where it fits within the step’s limit.
- To enter instructions directly, select the Instructions tab and add each tag with the instruction stating when it applies.
- Edit imported or entered instructions as needed for your matter.
- Select Tags and confirm the tags on the step and their grouping.
- Select Model and do the following:
- Update model settings as needed.
- Select Metrics.
- Modify error categories as desired and select Update Categories.
- Select Update.
- Select Output and choose the column this step writes its tags to. Create a new column, or select an existing column to merge this step’s tags into it.
- Optional. Click Run Step to run this step individually, or wait to deploy the full workflow after you configure all steps.
- Select the back arrow to return to the Configuration tab.
Issue Field Extraction
Issue Field Extraction
This step captures structured fields for each document. It runs after the classification steps and treats each document’s predicted tags as context, so extracted values reflect the coding already applied.
- Select Add Step, then select Issue Field Extraction.
- Select the step, then select Instructions and do the following:
- Review the General Instructions.
- Import field instructions from the linked protocol, or enter the fields directly with a description of what each should contain.
- Edit the fields as needed for your matter.
- Optional. Click Run Step to run this step individually, or wait to deploy the full workflow after you configure all steps.
- Select the back arrow to return to the Configuration tab.
Add or remove a step
Add or remove a step
- On the Configuration tab, select Add Step.
- In the Add a step dialog, select Issue Classification or Issue Field Extraction.
- Configure the new step as described above.
- Select Deploy to run all steps in order.
Running the classification steps on their own first is the recommended sequence. Classification establishes the coding, and field extraction uses that coding as context — so it is worth confirming the tags are being applied correctly before running the extraction pass over the population.
Review and correct results
Issue coding is iterative. Use the following procedure to review results and refine the configuration.- Select the Steps tab and select a classification step.
- On the Results tab, review the tags applied and the model’s reasoning for them.
- Apply or remove tags on documents you have decided on. Reviewer decisions take precedence over the workflow’s predictions.
- If a tag is being applied too broadly or too narrowly, select the Instructions tab and edit that tag’s instruction rather than correcting each document individually.
- Click Run Step to re-run the step with your corrections.
- Re-run any Issue Field Extraction steps afterwards, so the extracted fields reflect the corrected coding.