Overview
This article details the LILT System Architecture, including LILT-managed components, integrations, and customer-managed services. LILT supports three deployment models: GCP-hosted (multi-region), self-managed on AWS, and self-managed on bare metal infrastructure.Architecture Overview
Architecture Diagrams
GCP Hosted Architecture
The GCP-hosted deployment runs across US, Asia, and Europe regions with Google-managed services for data storage and analytics.
- Multi-region: Cloud Load Balancing, GKE, Front App, Core Services, Neural Services, and Redis are deployed across US, Asia, and Europe
- US-only services: Connectors, Elasticsearch, Cloud SQL, Cloud Storage, BigQuery, GPU Pool
- GCP managed services: Pub/Sub, Cloud SQL (MySQL), Cloud Storage, BigQuery, Salesforce (CRM integration)
- Cluster services: Redis (caching), ClickHouse (analytics), Dataflow
- Analytics pipeline: Dataflow connects Core Services to ClickHouse; Cloud SQL feeds BigQuery via Scheduler; Salesforce data flows into BigQuery
Self-Managed AWS Architecture
For customers deploying on AWS infrastructure with AWS-managed data services.
- AWS managed services: RDS (MySQL), S3, SQS
- Cluster services: Redis, ClickHouse, MongoDB, Dataflow
- Additional components: Dataflow connects Core Services to ClickHouse; SQS provides message queuing
- Neural services include LLM inference workloads (VLLM-based: Emma, Gemma, LLaMA) alongside traditional translation services
Self-Managed Bare Metal Architecture
For air-gapped or on-premises deployments where all services run within a single Kubernetes cluster.
- All services self-hosted: MySQL, MinIO (object storage), Redis, MongoDB, ClickHouse, Elasticsearch
- Additional components: Dataflow connects Core Services to ClickHouse
- Single Kubernetes cluster with client load balancer and ingress
- No external cloud dependencies — suitable for air-gapped environments
LILT-Managed Components
This section details the components managed by LILT. The LILT-managed components include all Application services, the Neural services performing translations, and the Core services responsible for the backend functionalities such as file import/export, audit logs, search and more. All components are integrated with RabbitMQ to communicate via message sharing.Neural services
LILT’s neural services include all of the components that perform language detection and machine translations. The neural services are responsible for detecting languages, performing interactive and batch translations, and memory management.Front services
LILT’s front services include the user-facing applications. Information and actions performed in the front services are transferred to the core and neural via RabbitMQ.Core services
LILT’s core services handle everything outside of translation and language detection. This includes audit file handling, logging, tagging, QA, assignments and more. The core services act as the central management hub for LILT, passing information to the front containers and submitting work for the neural containers.Self-Managed Components
Data Storage and Microservices
All data storage and microservices are managed by the customer in the self-managed implementation.Infrastructure services
For self-managed installation of LILT, the applications below are also installed to enable LILT to function:- MinIO: Object Storage Server
- OpenEBS: Local Persistent Volume Provisioner
- MySQL: Database
- Redis: Data Structure Caching
- RabbitMQ: Queue Management
- Nginx-ingress: Ingress controller
- ElasticSearch: Indexing
- ClickHouse: Analytics Database
- MongoDB: Auditlogs database
- Nvidia-device-plugin: GPU management in Kubernetes