About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
I understand your example. But isn't it that with the soft limit, some memory will be paged out, instead of killing a job? Also we could mitigate this somewhat by increasing the low-memory limit on the host, so that e.g. on a 64 Gbyte host, only 60Gbyte can be reserved?
We did previously consider this when we were adding cgroup2 support, something along the lines of:
memory.high = -M hard_memory_limit
memory.low = rusage[mem=xyz]
From various dev discussion forums, this approach would appear to work well when there were just a few jobs on the machine; but on a high core count machine, with many jobs (and thus many cgroups) there were concerns over how memory management would behave with multiple conflicts.
E.g. if the machine was 64GB, and you have 16 jobs with memory.low=4GB, and they all actually consumed it, such that the system.slice was critically low in memory, it could result in the OOM killer killing processes from all 16, rather than just picking one.
Even with weighting all the cgroups, you could conceivably have the edge case where the user for job1 does a bmod to rusage[mem=16gb] which then starts forcing others to swap or be killed.
That said, there have been quite a few enhancements to the cgroup controller over the last few years, and this is something that would be worth revisiting as a future roadmap project.