AI hub and 3rd-party LLM Support
In this article, we describe how to set-up the AI Hub to integrate third party models within LILT. This article is meant for anyone interested in using and managing their own LLMs within the LILT platform.
Administrator role or equivalent API Access role permission is required to configure LLMs in LILT.
Configuring and Editing LLM Integrations and Settings
Manage LLMs
LILT enables administrators to set-up and manage LLMs all within the LILT platform. To set-up and manage an LLM:
Use the sidebar to navigate to “Manage” then select “AI Hub”. Navigate between Translation, Generative, Audio-to-text, or Image-to-text depending on which third party model you want to set-up.
From the desired LLM card, select “Configure”
Enter the required configurations for your LLM (e.g. API key, JSON credentials, bucket name, Password, etc). For more information on configuration details, see the relevant pages:
Select from the toggle, “Enable for your organization”, “Use Terms”, and “Allow fine-tuning”. NOTE: Some options may be marked as “[not supported]” to indicate that functionality is not possible within the selected LLM.
Enable for organization = allows any LILT user in your organization to view Models built from this LLM within the Models page and use the model when creating a job or Instant Translate.
Use Terms = allows selected terminology entries from the LILT Data Source attached to the model to be used for fine tuning Models built from this LLM.
Allow fine-turning = allows LILT to share memory entries from selected LILT Data Sources to create more contextualized models.
Click “Save configuration”
Edit LLM Credentials and Settings
Once an LLM is set-up, you can edit the API key, Password, and settings by doing the following:
Use the sidebar to navigate to “Manage” then select “AI Hub”
From the desired LLM, select “Edit”
Edit the API Key or Password, and toggle “Enable for your organization”, “Use Terms”, “Allow fine-tuning” as needed, and Save.