​
Overview

This article details how to manipulate the helm-chart customization to set the number of GPU workers.

​
Steps

Any customizations made your installation should be made in the install_dir/lilt/environments/lilt/values.yaml file. In order to edit the GPU worker counts, open this file in a text editor.

​
V3 Language Model GPU Worker Count

To edit the amount of GPU worker counts for V1, set the following values: 
batchv3:
  onpremValues:
    config:
      # Maximum number of GPU workers. Can be set to 0 to disable GPU workers.
      # In an environment without autoscaling, this number of workers will
      # be up at all times.
      BATCH_WORKERS_MANAGER_GPU_MAX_WORKERS: 1
      
      # The standby workers option should only be modified in environments that use autoscaling.
      # It controls the minimum number of GPU workers that are kept running at
      # any given time, even when there are no batch jobs in progress.
      BATCH_WORKERS_MANAGER_GPU_STANDBY_WORKERS: 1
      
batch-worker-gpu:
  onpremValues:
    # The number of replicas is controlled by BatchWorkersManager.
    replicaCount: 1 # this should match BATCH_WORKERS_MANAGER_GPU_MAX_WORKERS