Skip to main content

​
Overview

This article details the LILT System Architecture, including LILT-managed components, integrations, and customer-managed services. LILT supports three deployment models: GCP-hosted (multi-region), self-managed on AWS, and self-managed on bare metal infrastructure.

​
Architecture Overview

Lilt Architecture Overview

​
Architecture Diagrams

​
GCP Hosted Architecture

The GCP-hosted deployment runs across US, Asia, and Europe regions with Google-managed services for data storage and analytics.
LILT GCP Hosted Architecture
Key characteristics:
  • Multi-region: Cloud Load Balancing, GKE, Front App, and Redis are deployed across US, Asia, and Europe
  • US-only services: Core Services, Neural Services, Connectors, Elasticsearch, Cloud SQL, Cloud Storage, BigQuery, GPU Pool, and the LILT Assist / Enterprise MCP backend
  • GCP managed services: Pub/Sub, Cloud SQL (MySQL), Cloud Storage, BigQuery, Salesforce (CRM integration), and Google Cloud Vertex AI (Gemini / Claude Haiku LLM reasoning)
  • Cluster services: Redis (caching), ClickHouse (analytics), Dataflow, and the LILT Assist / Enterprise MCP Server backend
  • Analytics pipeline: Dataflow connects Core Services to ClickHouse; Cloud SQL feeds BigQuery via Scheduler; Salesforce data flows into BigQuery

​
Self-Managed AWS Architecture

For customers deploying on AWS infrastructure with AWS-managed data services.
LILT Self-Managed AWS Architecture
Key characteristics:
  • AWS managed services: RDS (MySQL), S3, SQS
  • Cluster services: Redis, ClickHouse, MongoDB, Dataflow
  • Additional components: Dataflow connects Core Services to ClickHouse; SQS provides message queuing
  • Neural services include LLM inference workloads (VLLM-based: Gemma, LLaMA) alongside traditional translation services

​
Self-Managed Bare Metal Architecture

For air-gapped or on-premises deployments where all services run within a single Kubernetes cluster.
LILT Self-Managed Bare Metal Architecture
Key characteristics:
  • All services self-hosted: MySQL, MinIO (object storage), Redis, MongoDB, ClickHouse, Elasticsearch
  • Additional components: Dataflow connects Core Services to ClickHouse
  • Single Kubernetes cluster with client load balancer and ingress
  • No external cloud dependencies — suitable for air-gapped environments

​
LILT-Managed Components

This section details the components managed by LILT. The LILT-managed components include all Application services, the Neural services performing translations, and the Core services responsible for the backend functionalities such as file import/export, audit logs, search and more. All components are integrated with RabbitMQ to communicate via message sharing.

​
Neural services

LILT’s neural services include all of the components that perform language detection and machine translations. The neural services are responsible for detecting languages, performing interactive and batch translations, and memory management.

​
Front services

LILT’s front services include the user-facing applications. Information and actions performed in the front services are transferred to the core and neural via RabbitMQ.

​
Core services

LILT’s core services handle everything outside of translation and language detection. This includes audit file handling, logging, tagging, QA, assignments and more. The core services act as the central management hub for LILT, passing information to the front containers and submitting work for the neural containers.

​
Assist services

LILT’s Assist services are user-facing AI chat services. They are composed of the Assist UI (an interactive frontend chat interface embedded in the translation editor) and the MCP Server backend (the backend execution coordinator). LILT Assist leverages the Google Agent Development Kit (ADK) to run an iterative reasoning loop, calling external LLMs (such as Claude Haiku 4.5 via Google Vertex AI) and querying LILT Core services for translation memory (TM) and termbase (TB) lookups. All Assist requests and tool executions are securely scoped and restricted to the specific roles, resources, and organization permissions of the current signed-in user.

​
Enterprise MCP Gateway

The LILT Enterprise MCP Gateway is a Model Context Protocol endpoint hosted within the MCP Server backend at https://mcp-enterprise.lilt.com/mcp. It exposes the exact same underlying tools and reasoning capabilities as the LILT Assist chatbot (including translating text, searching translation memories, or querying terminology) to external enterprise AI agents, orchestrators (e.g., LangChain), and IDE extensions (e.g., Cursor). It is fully secured using Ory Hydra OAuth 2.1 with opaque token introspection, allowing secure external execution contexts. All queries and tool executions dispatched through the MCP gateway are securely scoped and restricted to the specific roles, resources, and organization permissions of the current authenticated user session.

​
Self-Managed Components

​
Data Storage and Microservices

All data storage and microservices are managed by the customer in the self-managed implementation.

​
Infrastructure services

For self-managed installation of LILT, the applications below are also installed to enable LILT to function:
  • MinIO: Object Storage Server
  • OpenEBS: Local Persistent Volume Provisioner
  • MySQL: Database
  • Redis: Data Structure Caching
  • RabbitMQ: Queue Management
  • Istio: Service mesh and ingress gateway. Deployed in ambient mode, which uses a per-node ztunnel proxy for transparent mTLS between workloads instead of injecting a sidecar into each pod.
  • ElasticSearch: Indexing
  • ClickHouse: Analytics Database
  • MongoDB: Auditlogs database
  • Qdrant: Vector database used by the neural translation service (rayman) for translation-memory nearest-neighbor retrieval
  • KubeRay: Kubernetes operator that manages the Ray cluster powering the neural translation service (rayman)
  • Nvidia-device-plugin: GPU management in Kubernetes