Documentation Index
Fetch the complete documentation index at: https://support.lilt.com/llms.txt
Use this file to discover all available pages before exploring further.
Overview
This article details how to manipulate the helm-chart customization to set the number of GPU workers.
Steps
Any customizations made your installation should be made in the install_dir/lilt/environments/lilt/values.yaml file. In order to edit the GPU worker counts, open this file in a text editor.
V4 Language Model GPU Worker Count
To edit the amount of GPU worker counts, set the following values:
batchv4:
onpremValues:
config:
# Maximum number of GPU workers. Can be set to 0 to disable GPU workers.
# In an environment without autoscaling, this number of workers will
# be up at all times.
BATCH_WORKERS_MANAGER_GPU_MAX_WORKERS: 1
# The standby workers option should only be modified in environments that use autoscaling.
# It controls the minimum number of GPU workers that are kept running at
# any given time, even when there are no batch jobs in progress.
BATCH_WORKERS_MANAGER_GPU_STANDBY_WORKERS: 1
batch-worker-gpu:
onpremValues:
# The number of replicas is controlled by BatchWorkersManager.
replicaCount: 1 # this should match BATCH_WORKERS_MANAGER_GPU_MAX_WORKERS
