On this page4 sections
The answer has started appearing on the screen. Then it slows to a crawl. The process serving it has not crashed, which is an improvement over the previous attempt, but the person waiting still does not have a finished answer.
In this hypothetical trial, keeping the process alive has solved one problem and left another open. Kubernetes Memory QoS can change how an application behaves when host memory is tight. To decide whether that helps an inference service, count the answers that finish within the promised time, not just the restarts that disappear.
The September 14, 2026 announcement promotes Memory QoS to beta in Kubernetes v1.37. Its feature gate is enabled by default on the relevant Linux cgroup v2 path, but the default kubelet configuration enables neither throttling nor reservation. Those behaviors require separate opt-in. The kubelet, Kubernetes' agent on each node, needs more than an enabled feature gate before it applies them.
First locate the memory that is under pressure. This feature uses Linux's controller for host RAM, not the memory on a GPU. It cannot make an oversized model fit on that GPU. The surrounding process can still compete for host RAM, however, and that competition can leave a request waiting.
Linux organizes processes into control groups, usually called cgroups, so it can account for and control their resources. Memory QoS changes the pressure or protection applied to those groups. It does not add RAM to the machine. The other work on the node still has to fit alongside the application.
Find the setting the waiting process actually received
Give one container a memory request of 256 MiB and a limit of 1 GiB. The request helps Kubernetes account for the resources the container needs; the limit is a separate upper bound. This example uses only container-level settings, with no Pod-level resource settings. A Pod is the Kubernetes unit containing one or more containers.
Because this container's memory request is below its limit, its Pod falls into the Burstable resource class. With throttling enabled at a factor of 0.9, the threshold is request + factor × (limit − request). Here, 1 GiB is 1,024 MiB, so the result is about 947 MiB. Above that threshold, an allocation can make the application wait while Linux tries to reclaim memory, before the hard limit is reached.
But that calculation describes an enabled policy. The new default, memoryThrottlingFactor: null, means the kubelet does not set memory.high, the throttling threshold. One upgraded node may explicitly retain 0.9 while another leaves the setting unset. The first keeps throttling; the second takes the new default. Having inherited an old default is not the same as having written down a value to preserve.
The resource class matters too. Under the container-level route used here, Guaranteed requires every container to have matching positive CPU requests and limits, and matching positive memory requests and limits. Those containers do not get this throttling threshold.
BestEffort Pods have no CPU or memory requests or limits. Their containers therefore use a zero memory request in the calculation. Without a container memory limit, the calculation uses node allocatable memory: the node's budget for Pods, not all the RAM installed in the machine. These distinctions help explain which policy might be slowing the allocation. They do not tell us whether the wait is acceptable to the person receiving the answer.
Look at the effective settings before and after the rollout, not only the repository template. Management tooling may have supplied a missing value or changed an explicit one. A comparison cannot explain throttling if the configuration being compared never reached the node.
The process gets time to recover. The person keeps waiting.
The application needs another piece of memory before it can continue. Linux tries to make space, and the allocation waits. That is the practical consequence of memory.high: going above it can slow allocations while memory reclaim runs, as the Linux documentation explains.
The Kubernetes QoS documentation separates that throttling from the protection provided by memory.min and memory.low. The hard memory limit is separate as well. Slowing allocations does not promise that the process can never exhaust its memory.
A background job may tolerate that pause. An interactive answer may miss its deadline. In the hypothetical trial, fewer restarts are encouraging, but we still need to count successful responses that arrive within the service's time objective. Keep the pressure conditions beside the results so that the tradeoff remains visible.
Reservation brings the neighbors into the decision. Tiered reservation applies across the node's Pods, rather than letting each Pod opt in separately. Protected memory can include the page cache, where Linux keeps file contents. A Pod reading large files can therefore affect the memory available around it. Compare the serving application alongside its neighboring workloads, not as though it were alone on the machine.
Follow both branches of the comparison
Memory pressure starts with the same representative workload and host constraints. Container behavior tracks throttling and restarts. Request behavior tracks latency, errors and completions. Compare with baseline asks whether the service tradeoff is acceptable. These are proposed observations, not measurements from a Kubernetes trial.
Measure the finish, not just the first words
Before changing policy, decide what useful completion means for this service: for example, a successful response within its agreed time objective. Keep the model, request mix, number of simultaneous requests and neighboring work steady. Record the node settings and change one policy. Otherwise, a quieter neighbor or a shorter answer may get mistaken for a benefit of Memory QoS.
Suppose the process now survives the pressure episode and starts streaming sooner, but the last words arrive after the user's deadline. Time to the first token, the first piece of generated text, improved. Time to the complete answer did not meet the promise. Compare both waits alongside memory pressure and terminations, and retain output length so that a longer answer is not mistaken for slower resource handling.
For retrieval or preprocessing work, follow the finished job and any retry effort instead. The question remains practical: did the extra waiting leave the person with more useful work completed, or less?
A short synthetic pressure episode can show that a configuration works and reveal its immediate cost. It cannot establish what happens after a day of cache growth. Larger reads or changed dependencies may place a different burden on the same process. A bounded rollout needs to examine those conditions before extending the first result to them.
The SLO guide connects these observations to the service's promise, while the Linux tuning guide covers the surrounding resource investigation. Reducing terminations, reducing allocation waits and meeting response deadlines are related goals, but they can point to different choices.
Leave the next operator enough to reverse the change
If the late response is the reason to reverse the trial, “turn Memory QoS off” is not a complete instruction. The announcement warns that disabling the feature requires compatible kubelet settings. Reversing the feature gate alone may leave configuration that still needs repair.
Rehearse the release-specific reversal outside production, then compare effective state with the baseline. Keep the node version and cgroup v2 status with the actual throttling and reservation values. Add the workload mix, service objective and observed result, along with the condition that triggers reversal and the owner authorized to make it. The next operator should be able to restore the intended configuration and check the request again, rather than infer recovery from a reversed flag.
Back at the screen, the answer has finally finished. If fewer restarts came with more on-time responses, that supports keeping the tested policy. If fewer useful requests finished, survival alone did not make the service better. Save the completion time beside the effective settings: the next decision needs to account for the person waiting as well as the process that stayed alive.
Sources & context
Sources linked in this article. Read alongside the author’s analysis; a citation does not independently verify a publisher’s claims.
- September 14 Memory QoS announcementkubernetes.io
- Kubernetes QoS documentationkubernetes.io
