On-Prem Features and Updates

Multimodal OCR Service

LILT 5.3 introduces a powerful new neural-based OCR service that delivers significantly improved accuracy for image and scanned PDF translation. The multimodal OCR service leverages advanced machine learning models to better extract text from complex documents, including those with mixed content types, tables, and challenging layouts. Key Benefits:

Enhanced accuracy for scanned PDFs and images
Improved handling of complex document layouts
Better recognition of tables and formatted content
Seamless integration with existing translation workflows

Configuration: To enable the multimodal OCR service, update your values.yaml:

ocr:
  enabled: true
  env:
    MQ_PWD: "<your-rabbitmq-password>"  # Add to secrets.yaml

Enhanced LLM Inference Architecture

LILT 5.3 restructures the LLM inference system into dedicated, optimized services for different AI models. This architectural improvement provides better resource management, improved performance, and more granular control over AI capabilities.

New Specialized Inference Services:

Llama vLLM Inference

Optimized inference for Llama language models
GPU-accelerated processing (requires 4 GPUs)
High-performance vLLM backend
Resource requirements: 100G memory, 5-14 CPUs

Whisper Speech-to-Text

Real-time speech-to-text transcription
GPU-accelerated for faster processing
Supports multiple languages
Ideal for audio/video translation workflows

Gemma Model Inference

Dedicated inference for Gemma3 models
Optimized for specific translation tasks
GPU-accelerated processing

Emma-500 Model Inference

Specialized inference for Emma-500 models
Advanced language understanding capabilities

Configuration: Enable specific inference services based on your needs:

llama-vllm-inference:
  enabled: true  # Enable Llama models

llm-inference-whisper:
  enabled: true  # Enable speech-to-text

gemma-vllm-inference:
  enabled: true  # Enable Gemma models

emma-500-vllm-inference:
  enabled: true  # Enable Emma-500 models

GPU Requirements: Ensure GPU nodes are available and labeled with capability: gpu.

Third-Party MT Batch Processing

A new CPU-based batch worker specifically designed for third-party machine translation (3pMT) processing has been added. This dedicated worker improves throughput and reliability for integrations with external MT providers. Features:

Dedicated CPU-based processing for 3pMT
Configurable worker pools
24G memory for large document handling
Improved batch processing reliability

Configuration:

batch-worker-cpu-3pmt:
  enabled: true
  env:
    BATCH_WORKERS_MANAGER_3PMT_CPU_MAX_WORKERS: "1"
    BATCH_WORKERS_MANAGER_3PMT_CPU_STANDBY_WORKERS: "0"

Enhanced Document Conversion

Document conversion timeouts have been made configurable, allowing better handling of large and complex documents. The new timeout settings apply to both the converter and file-translation services. Configuration:

converter:
  env:
    APRYSE_TIMEOUT_MINUTES: "5"  # Adjust based on document complexity

file-translation:
  env:
    APRYSE_TIMEOUT_MINUTES: "5"  # Adjust based on document complexity

Increase timeout values for particularly large or complex documents to prevent conversion failures.

MySQL SSL Support

LILT 5.3 adds comprehensive SSL/TLS support for MySQL connections, enhancing security for database communications. SSL is now enabled by default with flexible configuration options. Configuration:

file-job:
  env:
    MYSQL_SSL_MODE: "PREFERRED"
    MYSQL_SSL_ENABLED: "true"

memory:
  env:
    MYSQL_SSL_MODE: "PREFERRED"
    MYSQL_SSL_ENABLED: "true"

SSL Modes:

REQUIRED: Forces SSL connections (recommended for production)
PREFERRED: Uses SSL if available, falls back to unencrypted
DISABLED: Disables SSL (not recommended)

Core Platform Updates

Internal Architecture Improvements

Service Naming Standardization

AV scanner service renamed to lilt-av-scan for consistency
ConfigMap and Secret resources standardized with lilt- prefix
Improved service discovery and management

Breaking Changes

LLM Inference Service Restructuring

The generic llm-inference service has been removed and replaced with specialized inference services. This change improves resource utilization and provides better control over AI model deployment. Migration Required: If you previously had llm-inference enabled, you must:

Disable the old service:

llm-inference:
  enabled: false  # Remove this service

Enable the appropriate replacement services:

# For Llama models
llama-vllm-inference:
  enabled: true

# For speech-to-text
llm-inference-whisper:
  enabled: true

# For Gemma models
gemma-vllm-inference:
  enabled: true

# For Emma-500 models
emma-500-vllm-inference:
  enabled: true

Update GPU node labels if not already configured:

kubectl label nodes <node-name> capability=gpu

ConfigMap and Secret Name Changes

Several services now reference standardized lilt-configmap and lilt-secrets instead of service-specific resources. Affected Services:

Translation Memory (tm)
Lexicon

Migration Steps:

Ensure your deployment creates resources with the new names:
- lilt-configmap (previously front-configmap)
- lilt-secrets (previously front-secrets)

If using custom Helm templates, update references:

# Old:
configMapName: front-configmap
secretsName: front-secrets

# New:
configMapName: lilt-configmap
secretsName: lilt-secrets

Init Container File Path Changes

Services using the two-stage init container pattern (lexicon, file-job) must update file path configurations. Old Format:

lexicon:
  init:
    filePaths: "s3://lilt/lexicon-data/..."

file-job:
  init:
    filePaths: "s3://lilt/tesseract/..."

New Format:

lexicon:
  init:
    filePaths: "lexicon-data/..."  # Remove s3://lilt/ prefix

file-job:
  init:
    filePaths: "tesseract/..."  # Remove s3://lilt/ prefix

Action Required: If you have customized init container configurations, update file paths to remove the s3://lilt/ prefix.

New Configuration Requirements

Required Secrets

Core-API S3 Credentials

Add to secrets.yaml:

core-api:
  onpremValues:
    env:
      S3_ACCESS_KEY: "<your-minio-access-key>"
      S3_SECRET_KEY: "<your-minio-secret-key>"

Set these to match your MinIO or S3-compatible storage credentials.

Multimodal OCR (If Enabling)

Add to secrets.yaml:

multimodal:
  env:
    MQ_PWD: "<your-rabbitmq-password>"

Optional Configuration Updates

Backend Configuration Toggle

A new toggle allows enabling/disabling backend configuration features:

front:
  onpremValues:
    backend_config:
      enabled: false  # Set to true if needed

BigQuery Integration

For customers using BigQuery analytics:

core-api:
  env:
    BIG_QUERY_ENABLED: "false"  # Set to true if using BigQuery

Batch Worker Memory

Batch worker memory limits have been increased for improved performance:

batchv4:
  resources:
    requests:
      memory: 2G  # Increased from 1G
    limits:
      memory: 2G

Action Required: Ensure worker nodes have sufficient memory capacity.

Infrastructure Requirements

GPU Node Requirements

If enabling LLM inference services, ensure GPU nodes are available:

Service	GPU Count	Memory	CPU
llama-vllm-inference	4	100G	5-14
llm-inference-whisper	Yes	Variable	Variable
gemma-vllm-inference	Yes	Variable	Variable
emma-500-vllm-inference	Yes	Variable	Variable

Required Node Label: capability: gpu Setup:

kubectl label nodes <gpu-node-name> capability=gpu

Worker Node Requirements

CPU-intensive services require worker nodes:

Service	Memory	CPU
batch-worker-cpu-3pmt	24G	1
Other batch workers	Variable	Variable

Required Node Label: node-type: worker Setup:

kubectl label nodes <worker-node-name> node-type=worker

Troubleshooting

LLM Inference Services Not Starting

Symptom: LLM inference pods stuck in Pending state Solution:

Verify GPU nodes are available:
kubectl get nodes -l capability=gpu

Check GPU resources:

kubectl describe nodes -l capability=gpu | grep nvidia.com/gpu

Ensure proper node labels:

kubectl label nodes <node-name> capability=gpu

Multimodal OCR Connection Issues

Symptom: Multimodal service cannot connect to RabbitMQ Solution:

Verify RabbitMQ credentials in secrets.yaml:

multimodal:
  env:
    MQ_PWD: "<correct-rabbitmq-password>"

Check RabbitMQ connectivity:

kubectl logs -n lilt deployment/multimodal-ocr

Batch Worker Memory Issues

Symptom: Batch workers being OOMKilled Solution:

Verify node capacity:

kubectl describe nodes -l node-type=worker | grep -A 5 Allocatable

Increase memory if needed:

batch-worker-cpu-3pmt:
  resources:
    requests:
      memory: 32G  # Increase if needed
    limits:
      memory: 32G

ConfigMap/Secret Not Found Errors

Symptom: Services failing with “configmap ‘lilt-configmap’ not found” Solution: Ensure the main Lilt chart creates these resources. Check your Helm templates include:

apiVersion: v1
kind: ConfigMap
metadata:
  name: lilt-configmap
  namespace: lilt
---
apiVersion: v1
kind: Secret
metadata:
  name: lilt-secrets
  namespace: lilt

S3/MinIO Connection Issues

Symptom: Services cannot access S3/MinIO storage Solution:

Verify S3 credentials in secrets.yaml:

core-api:
  onpremValues:
    env:
      S3_ACCESS_KEY: "<access-key>"
      S3_SECRET_KEY: "<secret-key>"

Verify internal MinIO endpoint is accessible:

kubectl exec -n lilt deployment/core-api -- curl -k https://minio.lilt.svc.cluster.local:9000

​On-Prem Features and Updates

​Multimodal OCR Service

​Enhanced LLM Inference Architecture

​New Specialized Inference Services:

​Third-Party MT Batch Processing

​Enhanced Document Conversion

​MySQL SSL Support

​Core Platform Updates

​Internal Architecture Improvements

​Breaking Changes

​LLM Inference Service Restructuring

​ConfigMap and Secret Name Changes

​Init Container File Path Changes

​New Configuration Requirements

​Required Secrets

​Core-API S3 Credentials

​Multimodal OCR (If Enabling)

​Optional Configuration Updates

​Backend Configuration Toggle

​BigQuery Integration

​Batch Worker Memory

​Infrastructure Requirements

​GPU Node Requirements

​Worker Node Requirements

​Troubleshooting

​LLM Inference Services Not Starting

​Multimodal OCR Connection Issues

​Batch Worker Memory Issues

​ConfigMap/Secret Not Found Errors

​S3/MinIO Connection Issues