Software requirements
- Linux-based 64-bit OS: Red Hat Enterprise Linux 8 or newer, CentOS 8 or newer, Ubuntu 18.04 or newer, Debian Buster or newer.
- Install Docker and Docker-Compose on the host system.
- For GPU deployment, install NVIDIA GPU drivers and CUDA toolkit on the host system. Install NVIDIA Container Toolkit to run GPU-accelerated Docker containers on NVIDIA GPUs.
AIDA deployment components
This high-level architecture diagram outlines the main components of an AIDA deployment. Each component is running as a separate docker container.AIDA System Components
Main App
The Main App provides the core services in AIDA, including document annotations/reviews, document search/analytics, interactive UI for model prediction/analytics, as well as data import/export.Document processing backend
The document processing backend handles low-level document processing, including metadata extraction, text extraction, document conversion, image OCR, and document parsing.Machine learning backend
The machine learning (ML) backend is responsible for hosting pre-trained models, performing model training and predictions, and supporting continuous active learning and semantic search.PostgreSQL
PostgreSQL is used for storing and managing annotation data, including metadata associated with the annotations and other relevant information that is useful for machine learning.Solr
Solr provides full-text search capabilities and indexing of case documents. AIDA uses Solr to support boolean search (using Lucene or dtSearch syntax) of case text and metadata. AIDA also integrates Solr with its machine learning backend to support semantic search.Qdrant
Qdrant is an open-source vector database designed to store and search high-dimensional vectors. AIDA uses it to store vector embeddings for case documents and to support semantic search, where search queries and documents are represented as vectors in a high-dimensional space.Hardware recommendations
The usage of CPU, RAM, and storage space can vary based on the volume of the case data, the number of users, and AIDA’s features being used. AIDA can be deployed on a single server or multiple servers depending on factors such as the size of the dataset, the number of concurrent users, and the required processing power. For smaller datasets and fewer users, a single server deployment may be sufficient. However, as the size of the dataset and the number of concurrent users increases, deploying AIDA on multiple servers can provide additional processing power and improve performance and scalability. For example, you could run the main application on one server, the data processing and machine learning components on a second server, and the databases on a third server.GPU recommendation
It is strongly recommended that AIDA’s machine learning backend runs on a Graphics Processing Unit (GPU) environment. GPUs can significantly accelerate training and inference times, e.g., 5-10x faster than CPUs. Using AIDA’s pre-trained models requires at least 40 GB of GPU memory, e.g., A100 40GB. AIDA also offers more advanced conversational features that require GPUs with at least 160GB memory (e.g., 2 x NVIDIA A100 80GB) to host the models.Deployment examples
Below are three examples of how AIDA can be deployed:GPU Servers
Below are examples of the GPU deployment. The Small LLM server is required and the Chat LLM server is optional.Processing Time
Below are the expected processing times for loading documents (text and metadata) in AIDA using the system config from Table 2 with 32 vCPUs and 128 GB memory. Note that AIDA utilizes multi-processing to parallelize the processing time. It can be further sped up with more CPUs and Memory resources.General considerations
- For every 100 GB of text, allocate at least 64 GB of memory to the DB server.
- It is highly recommended to allocate at least 64 GB of CPU memory to the Processing/ML server. The Processing/ML components support batch processing so the memory requirements do not necessarily grow linearly as the size of the dataset increases. For example, 512 GB memory may be sufficient for processing 1TB of text.
- The main application and the document processing and ML components share the same data volume using NFS share. It is recommended to reserve 5 times the size of the source text for the DB storage and 10 times the size of the source text for the other storage shared by the main app and the processing/ML components.
- The memory requirement for the main application mainly depends on the number of concurrent users. In general, it is recommended to reserver 32GB RAM for every 100 concurrent users.