Skip to main content

​
On-Prem Features and Updates

​
Multimodal OCR Service

LILT 5.3 introduces a powerful new neural-based OCR service that delivers significantly improved accuracy for image and scanned PDF translation. The multimodal OCR service leverages advanced machine learning models to better extract text from complex documents, including those with mixed content types, tables, and challenging layouts. Key Benefits:
  • Enhanced accuracy for scanned PDFs and images
  • Improved handling of complex document layouts
  • Better recognition of tables and formatted content
  • Seamless integration with existing translation workflows
Configuration: To enable the multimodal OCR service, update your values.yaml: 
multimodal:
  enabled: true
  env:
    MQ_PWD: "<your-rabbitmq-password>"  # Add to secrets.yaml

​
Enhanced LLM Inference Architecture

LILT 5.3 restructures the LLM inference system into dedicated, optimized services for different AI models. This architectural improvement provides better resource management, improved performance, and more granular control over AI capabilities.

​
New Specialized Inference Services:

Llama vLLM Inference
  • Optimized inference for Llama language models
  • GPU-accelerated processing (requires 4 GPUs)
  • High-performance vLLM backend
  • Resource requirements: 100G memory, 5-14 CPUs
Whisper Speech-to-Text
  • Real-time speech-to-text transcription
  • GPU-accelerated for faster processing
  • Supports multiple languages
  • Ideal for audio/video translation workflows
Gemma Model Inference
  • Dedicated inference for Gemma3 models
  • Optimized for specific translation tasks
  • GPU-accelerated processing
Emma-500 Model Inference
  • Specialized inference for Emma-500 models
  • Advanced language understanding capabilities
Configuration: Enable specific inference services based on your needs: 
llama-vllm-inference:
  enabled: true  # Enable Llama models

llm-inference-whisper:
  enabled: true  # Enable speech-to-text

gemma-vllm-inference:
  enabled: true  # Enable Gemma models

emma-500-vllm-inference:
  enabled: true  # Enable Emma-500 models
GPU Requirements: Ensure GPU nodes are available and labeled with capability: gpu.

​
Third-Party MT Batch Processing

A new CPU-based batch worker specifically designed for third-party machine translation (3pMT) processing has been added. This dedicated worker improves throughput and reliability for integrations with external MT providers. Features:
  • Dedicated CPU-based processing for 3pMT
  • Configurable worker pools
  • 24G memory for large document handling
  • Improved batch processing reliability
Configuration: 
batch-worker-cpu-3pmt:
  enabled: true
  env:
    BATCH_WORKERS_MANAGER_3PMT_CPU_MAX_WORKERS: "1"
    BATCH_WORKERS_MANAGER_3PMT_CPU_STANDBY_WORKERS: "0"

​
Enhanced Document Conversion

Document conversion timeouts have been made configurable, allowing better handling of large and complex documents. The new timeout settings apply to both the converter and file-translation services. Configuration: 
converter:
  env:
    APRYSE_TIMEOUT_MINUTES: "5"  # Adjust based on document complexity

file-translation:
  env:
    APRYSE_TIMEOUT_MINUTES: "5"  # Adjust based on document complexity
Increase timeout values for particularly large or complex documents to prevent conversion failures.

​
MySQL SSL Support

LILT 5.3 adds comprehensive SSL/TLS support for MySQL connections, enhancing security for database communications. SSL is now enabled by default with flexible configuration options. Configuration: 
file-job:
  env:
    MYSQL_SSL_MODE: "PREFERRED"
    MYSQL_SSL_ENABLED: "true"

memory:
  env:
    MYSQL_SSL_MODE: "PREFERRED"
    MYSQL_SSL_ENABLED: "true"
SSL Modes:
  • REQUIRED: Forces SSL connections (recommended for production)
  • PREFERRED: Uses SSL if available, falls back to unencrypted
  • DISABLED: Disables SSL (not recommended)

​
Core Platform Updates

​
Internal Architecture Improvements

Service Naming Standardization
  • AV scanner service renamed to lilt-av-scan for consistency
  • ConfigMap and Secret resources standardized with lilt- prefix
  • Improved service discovery and management

​
Breaking Changes

​
LLM Inference Service Restructuring

The generic llm-inference service has been removed and replaced with specialized inference services. This change improves resource utilization and provides better control over AI model deployment. Migration Required: If you previously had llm-inference enabled, you must:
  1. Disable the old service: 
    llm-inference:
  enabled: false  # Remove this service
  2. Enable the appropriate replacement services: 
    # For Llama models
llama-vllm-inference:
  enabled: true

# For speech-to-text
llm-inference-whisper:
  enabled: true

# For Gemma models
gemma-vllm-inference:
  enabled: true

# For Emma-500 models
emma-500-vllm-inference:
  enabled: true
  3. Update GPU node labels if not already configured: 
    kubectl label nodes <node-name> capability=gpu

​
ConfigMap and Secret Name Changes

Several services now reference standardized lilt-configmap and lilt-secrets instead of service-specific resources. Affected Services:
  • Translation Memory (tm)
  • Lexicon
Migration Steps:
  1. Ensure your deployment creates resources with the new names:
    • lilt-configmap (previously front-configmap)
    • lilt-secrets (previously front-secrets)
  2. If using custom Helm templates, update references: 
    # Old:
configMapName: front-configmap
secretsName: front-secrets

# New:
configMapName: lilt-configmap
secretsName: lilt-secrets

​
Init Container File Path Changes

Services using the two-stage init container pattern (lexicon, file-job) must update file path configurations. Old Format: 
lexicon:
  init:
    filePaths: "s3://lilt/lexicon-data/..."

file-job:
  init:
    filePaths: "s3://lilt/tesseract/..."
New Format: 
lexicon:
  init:
    filePaths: "lexicon-data/..."  # Remove s3://lilt/ prefix

file-job:
  init:
    filePaths: "tesseract/..."  # Remove s3://lilt/ prefix
Action Required: If you have customized init container configurations, update file paths to remove the s3://lilt/ prefix.

​
New Configuration Requirements

​
Required Secrets

​
Core-API S3 Credentials

Add to secrets.yaml: 
core-api:
  onpremValues:
    env:
      S3_ACCESS_KEY: "<your-minio-access-key>"
      S3_SECRET_KEY: "<your-minio-secret-key>"
Set these to match your MinIO or S3-compatible storage credentials.

​
Multimodal OCR (If Enabling)

Add to secrets.yaml: 
multimodal:
  env:
    MQ_PWD: "<your-rabbitmq-password>"

​
Optional Configuration Updates

​
Backend Configuration Toggle

A new toggle allows enabling/disabling backend configuration features: 
front:
  onpremValues:
    backend_config:
      enabled: false  # Set to true if needed

​
BigQuery Integration

For customers using BigQuery analytics: 
core-api:
  env:
    BIG_QUERY_ENABLED: "false"  # Set to true if using BigQuery

​
Batch Worker Memory

Batch worker memory limits have been increased for improved performance: 
batchv4:
  resources:
    requests:
      memory: 2G  # Increased from 1G
    limits:
      memory: 2G
Action Required: Ensure worker nodes have sufficient memory capacity.

​
Infrastructure Requirements

​
GPU Node Requirements

If enabling LLM inference services, ensure GPU nodes are available:
ServiceGPU CountMemoryCPU
llama-vllm-inference4100G5-14
llm-inference-whisperYesVariableVariable
gemma-vllm-inferenceYesVariableVariable
emma-500-vllm-inferenceYesVariableVariable
Required Node Label: capability: gpu Setup: 
kubectl label nodes <gpu-node-name> capability=gpu

​
Worker Node Requirements

CPU-intensive services require worker nodes:
ServiceMemoryCPU
batch-worker-cpu-3pmt24G1
Other batch workersVariableVariable
Required Node Label: node-type: worker Setup: 
kubectl label nodes <worker-node-name> node-type=worker

​
Troubleshooting

​
LLM Inference Services Not Starting

Symptom: LLM inference pods stuck in Pending state Solution:
  1. Verify GPU nodes are available: 
    kubectl get nodes -l capability=gpu
  2. Check GPU resources: 
    kubectl describe nodes -l capability=gpu | grep nvidia.com/gpu
  3. Ensure proper node labels: 
    kubectl label nodes <node-name> capability=gpu

​
Multimodal OCR Connection Issues

Symptom: Multimodal service cannot connect to RabbitMQ Solution:
  1. Verify RabbitMQ credentials in secrets.yaml: 
    multimodal:
  env:
    MQ_PWD: "<correct-rabbitmq-password>"
  2. Check RabbitMQ connectivity: 
    kubectl logs -n lilt deployment/multimodal-ocr

​
Batch Worker Memory Issues

Symptom: Batch workers being OOMKilled Solution:
  1. Verify node capacity: 
    kubectl describe nodes -l node-type=worker | grep -A 5 Allocatable
  2. Increase memory if needed: 
    batch-worker-cpu-3pmt:
  resources:
    requests:
      memory: 32G  # Increase if needed
    limits:
      memory: 32G

​
ConfigMap/Secret Not Found Errors

Symptom: Services failing with “configmap ‘lilt-configmap’ not found” Solution: Ensure the main Lilt chart creates these resources. Check your Helm templates include: 
apiVersion: v1
kind: ConfigMap
metadata:
  name: lilt-configmap
  namespace: lilt
---
apiVersion: v1
kind: Secret
metadata:
  name: lilt-secrets
  namespace: lilt

​
S3/MinIO Connection Issues

Symptom: Services cannot access S3/MinIO storage Solution:
  1. Verify S3 credentials in secrets.yaml: 
    core-api:
  onpremValues:
    env:
      S3_ACCESS_KEY: "<access-key>"
      S3_SECRET_KEY: "<secret-key>"
  2. Verify internal MinIO endpoint is accessible: 
    kubectl exec -n lilt deployment/core-api -- curl -k https://minio.lilt.svc.cluster.local:9000