Skip to main content
LILT ships alternative GPU allocations and identity-provider configurations as files you copy into an overlay directory. Nothing is edited in place, so your choices survive an upgrade.

​
The Overlay Directory

install-lilt-charts.sh applies every *.yaml file under lilt/environments/<ENV_NAME>/values.d/ on top of the base values file and the environment overlay. ENV_NAME defaults to lilt, so the directory is usually lilt/environments/lilt/values.d/.
The directory is yours. The installer reads it and never writes to it or prunes it, so one copy holds until you undo it. Delete the file to return to the default. This is the supported way to change values the installer ships. Editing lilt/values.yaml or lilt/eks-values.yaml directly produces the same result at deploy time, but you then have to remember to reapply the edit on every upgrade.
Files are applied in alphabetical order, and a later layer replaces an earlier one. Two files that set the same top-level key conflict silently: the alphabetically last one wins and the other is discarded with no warning. Keep at most one profile per service.Helm replaces list values wholesale rather than merging them, so a profile’s nodeSelectorTerms supersede the default entirely rather than adding to it.
Files that set different top-level keys compose safely. A GPU profile and an SSO profile can sit in the directory together.

​
GPU Profiles

The base values file ships one GPU allocation per service, sized for the smallest hardware that service supports. On larger cards the same model needs fewer of them, and the surplus is not free: the device plugin assigns GPUs exclusively, so a card reserved by a pod that never uses it cannot be used by anything else. The profiles in lilt/gpu-profiles/ are alternative allocations for hardware that allows a different one. They are optional. Installing nothing keeps the defaults, which work on every card the service supports.

​
Read Your Cards First

A profile is matched against the VRAM the driver reports, which is lower than the size the card is sold as. An L4 sold as 24 GB reports 23034 MiB.
These labels come from GPU Feature Discovery, which the NVIDIA GPU Operator provides. They are a hard requirement of any GPU install, not something a profile introduces. If the labels are absent, GPU pods stay Pending because no node satisfies their affinity.

​
Choosing a Profile

Pick the smallest card count your hardware allows. The two defaults differ because the hardware does. An on-premises cluster typically has several smaller cards, so the default takes two of them. An AWS GPU node carries one large card, so the EKS default takes one and raises the floor high enough that the card can hold the whole model.
Every gemma configuration requires compute capability 8.9 or higher, on both flavors. Gemma is FP8-quantised and native FP8 support starts with the Ada generation, so every Ampere card is excluded, including the 80 GB A100. This is a separate condition from the memory floor: a card with plenty of VRAM and an older architecture still does not qualify.
The memory floors are exclusive comparisons in the values files. A floor written as 43399 admits cards reporting 43400 MiB or more. The numbers in the preceding table are the values that actually qualify.

​
When to Use Each Profile

gemma-dual-medium.yaml — your cards are the 24 GB class and gemma stays Pending because no single card clears the EKS default’s floor. Two cards together hold what one cannot. The profile is bounded above, so it will not also match a larger card and reserve two where one would do. gemma-single-large.yaml — your cards are 48 GB or larger and the on-premises default is reserving two where one suffices. llama-dual-medium.yaml — your nodes have fewer than four cards, so the default never schedules. Here the profile is what makes llama runnable, not what makes it faster. llama-single-large.yaml — your cards are 80 GB or larger.
llama-vllm-inference ships disabled on both flavors, as it is being retired in favour of Gemma. A llama profile only takes effect on a cluster that has turned it back on.

​
Why a Profile Is a File

Each profile changes two fields that are only correct together: the GPU count Kubernetes reserves, and the memory floor in the node affinity that decides which cards are eligible. Changing one without the other fails, and the two failures look nothing alike. A count lowered without raising the floor still schedules, then the pod lands on cards too small for the model and dies during startup with a VRAM error. A floor raised without lowering the count never schedules at all, and the pod sits Pending forever. Kubernetes cannot make this choice for you. The GPU count is a resource request, fixed when Helm renders the manifest, and the scheduler needs that number to pick a node, so it cannot depend on which node was picked. Keeping both fields in one file is what makes them impossible to apply by halves.

​
Services Without a Profile

llm-inference-whisper, batch-worker-gpuv4, and translatev4 carry GPU floors of their own but ship no profile. To change their allocation you edit the base values file, or write your own overlay file setting the same two fields.

​
Single Sign-On Profiles

LILT ships with external SSO disabled, so password sign-in works out of the box and no install points at an identity provider you do not control. The files in lilt/sso-profiles/ are complete configurations for one provider each. Apply one the same way:
Then set the client secret, which no profile carries, as the singleOidcClientSecret key of the lilt-secrets secret. Add it to the environment’s secrets.yaml, or provision it through External Secrets Operator.
The panva-sandbox.yaml profile targets a public sandbox provider with a shared test client. Anyone can authenticate as anyone. Use it to check that the SSO path works end to end, never on a customer install.

​
Writing a Profile for Your Own Provider

A profile sets all eight values on three services: front, av-scan, and auth-service. Each reads its own copy of the same block, so a profile covering only one of them leaves the other two disabled.
Set all of them together. front validates the block when the provider is enabled and exits on the first empty field, so a partial profile crashes the pod at startup rather than falling back to no SSO.
To customise TLS for the calls LILT makes to your provider, populate the single-oidc-provider-tls-options secret and set the matching SINGLE_OIDC_PROVIDER_TLS_OPTIONS_PATH_TO_* values.

​
Apply a Profile

Copying a file changes nothing on its own. Re-run the installer:
Confirm the layer was picked up. The installer prints a line for each file it applies:
Then check that the pods scheduled:
A pod stuck in Pending with didn't match Pod's node affinity means no card in the cluster clears the profile’s floor. Compare the profile you chose against the nvidia.com/gpu.memory labels on your nodes.