Skip to main content

​
OpenAI API

You can set up an Azure OpenAPI by:
  1. Sign in to the Azure Portal: https://portal.azure.com/.
  2. Search for Azure OpenAI
  3. Click +Create to create an Azure OpenAI resource
  4. Go to the Overview of the resource and click “Go to Azure OpenAI Studio”
  5. Click “Deployments” and ”+ Deploy Model” -> “Deploy base model”
  6. Click “gpt-4o” and then “Confirm”
  7. Customize the resource and optimize the “Tokens per Minute Rate Limit” settings.
AIDA leverages OpenAI APIs to support two key use cases: real-time chat support and background instruction parsing/model initialization and alignment. You can add multiple API endpoints for each use case for scaling. AIDA utilizes a round-robin approach to distribute incoming requests across different endpoints and prevent bottlenecks at any single endpoint.

​
Chat API

The Chat API is set up to handle real-time Q&A through the Chat function. Each API endpoint can serve a certain number of requests per minute, and you may scale the service to meet user demand. Assuming each user generates fewer than 10 requests per minute, a single Chat API endpoint with a 1000 requests per minute limit can efficiently serve up to 100 concurrent users.

​
Agent API

The Agent API is used for background tasks such as instruction parsing, model initialization and alignment. Unlike the Chat API, these tasks do not require instant response time. However, the processing requirements are substantial, with each model potentially using around 1 million tokens for initialization and alignment. For the text classification and information extraction models, the token usage only occurs when the model is trained the first time. For the text generation model, the token usage occurs every time the model is ran. When running multiple models simultaneously, consider deploying multiple Agent API endpoints. For instance, when running 5 models in parallel, each requiring significant token usage, you can set up 5 different API endpoints. These endpoints are looped through to ensure that token limits are respected and processing remains uninterrupted. Note: The Agent API can be replaced by a self-hosted Llama API for tasks such as text classification and information extraction models. This option provides higher scalability, allowing the system to better handle large-scale processing, though performance may be slightly lower with the Llama API.

​
Email API

You can use an external email server to send outbound emails.

​
Azure Communication Services (ACS)

You can set up an Email API using the Azure Communication Services (ACS) connection string.
  1. Sign in to the Azure Portal:
  2. Create a Communication Services Resource:
    • Search for Communication Services
    • Click Create.
    • Fill in the required information (Subscription, Resource Group, Region, and Name).
    • Click Review + Create
  3. Go to your newly created ACS resource.
  4. Navigate to Keys and Connection String:
    • In the left-hand menu, click on Keys under Settings.
    • Here, you will see Endpoint, Primary Connection String, and Secondary Connection String.
  5. Open config/apis.json.
    • Set the value of acs_connection_string as Primary Connection String
    • Set the value of api_endpoint to Endpoint

​
Search API

You can use a web search API to enhance AIDA’s knowledge layer. AIDA currently integrates with the Bing Search API.

​
Bing Search API

  1. Sign in to the Azure Portal: https://portal.azure.com/.
  2. Search for Bing Resources
  3. Click Add and select Bing Search
  4. Fill in the required information
  • Subscription: Choose your Azure subscription.
  • Resource group: Either create a new resource group or select an existing one.
  • Region: Choose a region where the resource will be hosted.
  • Name: Choose a unique name for your resource.
  • Pricing tier: Select the pricing tier. You may start with S1 and increase as needed.
  • Click Review + Create and then Create to deploy the resource.
  1. Navigate to Keys and Endpoint under Resource Management.
  • Copy Key 1
  1. Open config/apis.json.