Supported Languages

V3 Languages

Starting in V3, our models will be trained in groups as opposed to single language pairs.
Language GroupingExplanationCodes
Asian: English, Japanese, Korean, Chinese (Simplified), Chinese (Traditional)Any two languages in this groupingEN,JA,KO,ZH,ZT
Eastern European: English, Belarusian, Bulgarian, Czech, Croatian, Lithuanian, Latvian, Polish, Romanian, Russian, Slovak, Slovenian, Albanian, Serbian, UkrainianAny two languages in this groupingEN,BE,BG,CS,HR,LT,LV,PL,RO,RU,SK,SL,SQ,SR,UK
Eastern Hemisphere: English, Arabic, Bengali, Central Kurdish, Persian (Farsi), Dari, Gujarati, Hebrew, Hindi, Kannada, Malayalam, Marathi, Nepali, Panjabi, Pushto, Tamil, Telugu, Tajik, UrduAny two languages in this groupingEN,AR,BN,CKB,FA,FS,GU,HE,HI,KN,ML,MR,NE,PA,PS,TA,TE,TG,UR
European Group 2: English, Afrikaans, Danish, German, Greek, Amharic, Finnish, Hungarian, Armenian, Dutch, Norwegian, Swedish, TurkishAny two languages in this groupingEN,AF,DA,DE,EL,ET,FI,HU,HY,NL,NO,SV,TR
European: English, German, Spanish, French, Italian, PortugueseAny two languages in this groupingEN,DE,ES,FR,IT,PT
**Other: **English, Indonesian, Igbo, Javanese, Malay, Burmese, Thai, Tagalog, Samoan, Somali, Swahili, VietnameseAny two languages in this groupingEN,ID,IG,JV,MS,MY,TH,TL,SM,SO,SW,VI

Unsupported Languages

Enlarges the table by opening it in a full screen dialogOpen
Source LanguageTarget LanguageContextual AI SupportBaseline ModelOn-Premise
AmharicEnglishNoN/ANot available
AzeriEnglishNoN/ANot available
CatalanEnglishNoN/ANot available
EnglishAmharicNoN/ANot available
EnglishAzeriNoN/ANot available
EnglishBosnianNoN/ANot available
EnglishCatalanNoN/ANot available
EnglishCreoleNoN/ANot available
EnglishEnglishNoN/ANot available
EnglishIcelandicNoN/ANot available
EnglishKazakhNoN/ANot available
EnglishKhmerNoN/ANot available
EnglishKinyarwandaNoN/ANot available
EnglishLaoNoN/ANot available
EnglishMaoriNoN/ANot available
EnglishMongolianNoN/ANot available
EnglishSinhalaNoN/ANot available
EnglishZuluNoN/ANot available
KhmerEnglishNoN/ANot available
KinyarwandaEnglishNoN/ANot available
LaoEnglishNoN/ANot available
MaoriEnglishNoN/ANot available
MongolianEnglishNoN/ANot available
SinhalaEnglishNoN/ANot available
ZuluEnglishNoN/ANot available

Large Language Models

LILT is shipped with:
  • META: Llama-3-70B-Chat-AWQ
  • OpenAI: Whisper-large-v3

GPU Requirements

V3 language models require three dedicated T4 GPUs. Two GPUs for Translate and One GPU for Batch requests, or two A10 GPUs, one for translate and one for batch
Number of GPU(s)GPU Support
3V3
If you are using split GPUs, please ensure that each split is allocated 16GB of memory.
While the number of GPUs required has not increased, the configuration of those requirements has changed. For this release, 24GB of VRAM is necessary to run the translate pod successfully. For customers running on T4 GPUs, this means that the GPU node must have at least two (2) T4s attached to it. Put another way, it is not sufficient to have two (2) nodes, each with one (1) T4 GPU attached.

Configuring V3 Models

This is no longer relevant to any releases done in 2025 and beyond, and will be removed from the documentation sometime in the future.
It only applies to releases which still contained V1 and V2 models.

Updating V3 Language Models

  1. Choose desired V3 language pairs
    sudo vi $install_dir/lilt/environments/lilt/values.yaml
    
    Set routing:onpremValues:config:selfManagedV3LanguagePairs to required lang-pairs. Sample config:
    routing:
      onpremValues:
         config:
           selfManagedV3LanguagePairs: "en-ar, en-de, en-es, en-fr, en-id"
    
  2. Update GPU count for translatev3 service as per the compatibility metrics
    sudo vi $install_dir/lilt/environments/lilt/values.yaml
    
    Set translatev3:onpremValues:config:resources:nvidia.com/gpu
  3. Run installer script
    # ./$install_dir/install-lilt.sh
    

Disabling All V3 Language Models

If you wish to completely disable all V3 languages, follow the guide below.
  1. Edit custom values.yaml and enable referred services by setting enabled: false
    sudo vi $install_dir/lilt/environments/lilt/values.yaml
    
    • Services:
      • batch-tb [Leave it enabled if you wish to use v2]
      • translatev3
      • updatev3
      • update-managerv3
      • batchv3
      • batch-worker-gpuv3
  2. Comment or remove below parameter:
    routing:onpremValues:config:selfManagedV3LanguagePairs
    
  3. Run installer script
    # ./$install_dir/install-lilt.sh
    

Upgrading from V1 to V2/V3

If you’d like to upgrade to V2/V3 models from V1, please ensure the following resources exist in your cluster:
MasterNode*GPU Node*
No changes required* 128 vCPUs
  • 1024G RAM
  • 1TB Disk space | - > 1 GPU
  • 16 vCPUs
  • 120G RAM
  • 1TB Disk space |
*We can leverage existing, non-utilized cluster resources on the Node/GPU.

Costs

For accurate figures, make use of the AWS cost calculator or refer to your cloud provider’s infrastructure cost tool.

New Language Support

We add new AI-supported languages to the LILT Platform continuously based on three criteria:
  • community and customer demand
  • availability of training data
  • baseline translation quality that is competitive with Google Translate and Microsoft Translator
Email us at sales@LILT.com to request LILT support for additional languages.