Skip to main content
You can organize the review of a large document set by review batches. You can create a static batch with a fixed number of documents based on search criteria. You can also create a dynamic batch using an active learning algorithm that automatically samples new documents for review.

​
Create a review batch

To create a new review batch, navigate to the Review section of the menu, click Review Batches and then click the Create review batch button.
There are two types of review batches: Dynamic and Static. A Dynamic batch is an active learning batch whose size can grow continuously as users request more documents to be labeled, while a Static batch is a fixed set of randomly sampled documents using user-defined criteria.

​
Create a static batch

You can create a static batch by randomly sampling documents from a dataset or by uploading a list of Doc IDs. Static batches have fixed sizes.

​
Create a dynamic batch

To create a dynamic batch, select Dynamic batch and enter the setup details. Dynamic review batches use active learning to sample documents on demand. There are three types of sampling strategies: priority, coverage, and balanced

​
Sampling strategies

Aida can utilize different sampling strategies during active learning. Each strategy has a different goal.

​
Priority Sampling

The goal is to prioritize documents that are most likely to be relevant based on model predictions. The sampling process is: The process:
  • 90% of the batch comes from the highest-scoring documents: sort the documents in descending order, apply clustering on the top-ranking documents, and select the centroids to form the sample batch. This is to ensure some degree of diversity in the documents.
  • 10% of the batch comes from a pure random sample.
Use Case: Best suited when the richness is low or the goal is to focus on documents most likely to contain relevant information and to maximize efficiency.

​
Coverage Sampling

The goal is to ensure diverse areas of the document space are covered, avoiding bias toward highly scored documents. The process:
  • 30% of the batch comes from documents with scores above 0.5. Documents are selected as centroids of document clusters to ensure diversity.
  • 30% of the batch comes from documents with scores below 0.5. Documents are selected as centroids of document clusters to ensure diversity.
  • 30% of the batch comes from documents with scores around 0.5. Documents are selected as centroids of document clusters to ensure diversity.
  • 10% of the batch comes from a pure random sample.
Use Case: This approach is useful when aiming for covering diverse information to improve the model’s overall accuracy across different types of documents.

​
Balanced Sampling

The goal is to balance the review process by combining priority and coverage strategies. The process:
  • 70% of the batch is selected from documents with scores above 0.7. Documents are selected as centroids of document clusters to ensure diversity.
  • 20% of the batch comes from documents with scores below 0.7. Documents are selected as centroids of document clusters to ensure diversity.
  • 10% of the batch comes from a pure random sample.
Use Case: Useful when seeking to balance between documents that are most likely to be relevant and the ones that are less likely to be relevant, while ensuring that the model is exposed to diverse and relevant data.