Self-Managed Hardware Requirements
LILT can be installed on custom hardware, whether in a private data center or on a variety of cloud platforms. The tables below provide approximate monthly cost estimates for running the base LILT system running V2 and V3 models in a cloud environment.
Where applicable, long-term pricing discounts for cloud vendors is considered. However, resources (especially storage) may need to be increased to suit your needs. These costs are not a guarantee, rather a reflection of the cloud platform vendor's prices at the time of lookup.
Network, bandwidth, security group, permission management, and other platform costs are not considered in the attached estimate.
Are you interested in installing LILT's self-managed solution? Contact Sales from the LILT homepage!
Last Revised: 24 July 2024
AWS Commercial | ||||||||
Platform | Resource Identifier | Specifications | Quantity | Unit | Unit cost [1] | Cost [1] per month | Cost [2] per month | Notes |
AWS | m5.xlarge | 4vCPU / 16G RAM / 200GB | 1 | Hour | $0.192 | $140.16 | $88.33 | K8s Master |
AWS | r5.16xlarge | 64vCPU / 512G RAM / 1TB | 2 | Hour | $8.064 | $5,886.72 | $3,708.40 | K8s Nodes |
AWS | g4dn.4xlarge | 1 GPU / 16vCPU / 64G RAM / 500GB | 2 | Hour | $2.41 | $1,757.84 | $759.20 | Supports up to 5 V2 or all V3 lang-pairs |
AWS | g4dn.4xlarge | 1 GPU / 16vCPU / 64G RAM / 500GB | 6[4] | Hour | $7.22 | $5,273.52 | $2,277.60 | Supports up to 23 V2 or all V3 lang-pairs |
Total Cost | 5 lang-pairs | $10.66 | $7,784.72 | $4,555.93 | ||||
Total Cost | 20 lang-pairs | $15.48 | $11,300.40 | $6,074.33 | ||||
AWS Top Secret | ||||||||
Platform | Resource Identifier | Specifications | Quantity | Unit | Unit cost [1] | Cost [1] per month | Cost [3] per month | Notes |
AWS | m5.xlarge | 4vCPU / 16G RAM / 200GB | 1 | Hour | $0.4502 | $328.68 | $206.59 | K8s Master |
AWS | r5.16xlarge | 64vCPU / 512G RAM / 1TB | 2 | Hour | $15.424 | $11,259.52 | $6,770.02 | K8s Nodes |
AWS | g4dn.4xlarge | 1 GPU / 16vCPU / 64G RAM / 500GB | 2 | Hour | $5.196 | $3,793.08 | $2,390.02 | Supports up to 5 V2 or all V3 lang-pairs |
AWS | g4dn.4xlarge | 1 GPU / 16vCPU / 64G RAM / 500GB | 6[4] | Hour | $15.588 | $11,379.24 | $7,170.06 | Supports up to 23 V2 or all V3 lang-pairs |
Total Cost | 5 lang-pairs | $21.07 | $15,381.28 | $9,366.63 | ||||
Total Cost | 20 lang-pairs | $31.46 | $22,967.44 | $14,146.67 | ||||
[1] Price calculated for plan: "On-Demand Instances"
[2] Price calculated for plan: "EC2 Instance Savings Plans" (1 year)
[3] Price calculated for plan: "EC2 Reserved Instances" (1 year)
[4] V3 models are expected to require two additional GPUs, for a total of eight (8) GPUs
Source: AWS Pricing Calculator Estimate
For detailed V2/ V3 model GPU requirements, visit V2/V3 Language Models
Using T4 GPUs
Last Revised: 14 Aug 2024
GCP | ||||||||
Platform | Resource Identifier | Specifications | Quantity | Unit | Unit cost [1] | Cost [1] per month | Cost [2] per month | Notes |
GCP | n1-standard-4 | 4vCPU / 15G RAM / 200GB | 1 | Hour | $0.20 | $145.50 | $94.18 | K8s Master |
GCP | n2-highmem-64 | 64vCPU / 512G RAM / 1TB | 2 | Hour | $8.61 | $6,199.77 | $3,934.79 | K8s Nodes |
GCP | n1-highmem-32 | 2 GPU / 32vCPU / 208G RAM / 500GB | 1 | Hour | $2.65 | $1,911.58 | $1,210.48 | Supports up to 5 V2 or all V3 lang-pairs |
GCP | n1-highmem-32 | 2 GPU / 32vCPU / 208G RAM / 500GB | 3[3] | Hour | $7.95 | $5,734.74 | $3,631.44 | Supports up to 20 V2 or all V3 lang-pairs |
Total Cost | 5 lang-pairs | $10.64 | $8,256.85 | $5,239.45 | ||||
Total Cost | 20 lang-pairs | $15.94 | $12,079.01 | $7,660.41 | ||||
[1] Price calculated for plan: "On-Demand Instances"
[2] Price calculated for plan: "Committed use discount options" (1 year)
[3] V3 models are expected to require two additional GPUs, for a total of eight (8) GPUs
Source: GCP Pricing Calculator Estimate
For detailed V2/ V3 model GPU requirements, visit V2/V3 Language Models
Using T4 GPUs
Last Revised: 14 June 2024
Microsoft Azure | ||||||||
|---|---|---|---|---|---|---|---|---|
Platform | Resource Identifier | Specifications | Quantity | Unit | Unit cost [1] | Cost [1] per month | Cost [2] per month | Notes |
Azure | CentOS D3 v2 | 4vCPU / 14G RAM / 128GiB | 1 | Hour | $0.293 | $219.78 | $96.06 | K8s Master |
Azure | CentOS E64as v5 | 64vCPU / 512G RAM / 512GiB | 2 | Hour | $4.266 | $5322.88 | $3158.36 | K8s Nodes |
Azure | NC8as T4 | 1 GPU / 8vCPU / 56G RAM / 128GiB | 2 | Hour | $1.504 | $1109.7 | $657.28 | Supports up to 5 V2 or all V3 lang-pairs |
Azure | NC8as T4 | 1 GPU / 8vCPU / 56G RAM / 128GiB | 6 [3] | Hour | $4.512 | $3329.1 | $1971.84 | Supports up to 23 V2 or all V3 lang-pairs |
Total Cost | 5 lang-pairs | $6.06 | $6652.36 | $3911.70 | ||||
Total Cost | 20 lang-pairs | $9.07 | $8871.76 | $5226.26 | ||||
[1] Price calculated for plan: "Pay as you go"
[2] Price calculated for plan: "1 year reserved (~41% - ~58% discount)"
[3] V3 models are expected to require two additional GPUs, for a total of eight (8) GPUs
Source: https://azure.microsoft.com/en-us/pricing/calculator/#virtual-machines1
For detailed V2/ V3 model GPU requirements, visit V2/V3 Language Models
Using T4 GPUs