> ## Documentation Index
> Fetch the complete documentation index at: https://labs.laer.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Benchmark results
> 2025/11/26
## Privilege classification performance
### Dataset Overview
The evaluation was conducted on a corpus of 24K documents from a legal privilege review workflow. The dataset exhibits characteristic of real-world privilege review with both First-level (1L) annotations and second-level (2L) annotations. The distribution reveals that approximately 57% of documents received withhold recommendations at the 2L level, while 36% were classified as not privileged. Solicitor-client privilege constitutes the dominant privilege type at 64% of 2L privilege classifications, followed by litigation privilege at 5% and common interest privilege at 4%.
### Human 1L vs Human 2L
To establish a baseline for inter-annotator agreement and assess the consistency of privilege determinations across different reviewer experience levels, we compared first-level (1L) and second-level (2L) human annotations on 21,922 documents where both levels of review were available.
The comparison reveals moderate agreement between 1L and 2L reviewers, with an overall **accuracy of 71% and recall of 80.7%**. The results demonstrate systematic differences in how reviewers at different experience levels approach privilege classification, particularly for non-privileged documents.
### EAIDA vs Human 2L
EAIDA generates probability scores for privilege classification, allowing for flexible threshold selection based on operational requirements. We evaluated EAIDA's performance across multiple operating points to understand the precision-recall tradeoff and identify optimal threshold settings for production deployment.
#### Precision-Recall Curve
The precision-recall curve demonstrates EAIDA's ability to achieve high recall while maintaining acceptable precision across a range of threshold values.
### False Negative Rate Analysis: Critical Risk Assessment
In privilege review, false negatives represent the most consequential error type, as they result in privileged documents being incorrectly classified as non-privileged and potentially produced to opposing parties. Such inadvertent disclosure can constitute privilege waiver, expose confidential legal strategy, and create significant legal and business risk. Therefore, minimizing the false negative rate is the primary performance objective for any privilege classification system.
#### Human 1L Performance: High Miss Rate
Analysis of Human 1L annotations reveals a critically high false negative rate that falls well below acceptable standards for privilege review:
### EAIDA vs general-purpose LLMs
We compare EAIDA's performance over general-purpose LLMs. Results suggest that general-purpose LLMs are unsuitable for privilege review without extensive fine-tuning, and that purpose-built systems like EAIDA are essential for achieving acceptable recall rates.
### EAIDA's Scalability and Efficiency
EAIDA's architecture is purpose-built for large-scale AI review, delivering sustained throughput of 5,000 to 10,000 documents per hour for detailed privilege review — up to 5x performance advantage compared to competitive Privilege review solutions.
Unlike general-purpose LLMs that rely on external API calls with unpredictable throttling and cost structures, EAIDA operates as a self-contained system with predictable throughput and pricing regardless of concurrent load. Critically, EAIDA imposes no character-length restrictions on input documents, processing contracts, expert reports, and complex transaction documents of arbitrary length without exceptions. This architectural decision eliminates the operational overhead of pre-processing oversized documents.
#### Feature Comparison with Relativity aiR
EAIDA's speed is limited only by GPU compute - the system can scale horizontally with more GPU compute spend.
\*LLMs have context window limits (typically 128K tokens)
\*\*Estimated API costs based on typical legal document token counts; actual costs vary by document length