IBM Data and AI Ideas Portal for Customers
This portal is to open public enhancement requests against products and services offered by the IBM Data & AI organization. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).
Shape the future of IBM!
We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:
Search existing ideas
Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,
Post your ideas
Post ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,
Post an idea
Upvote ideas that matter most to you
Get feedback from the IBM team to refine your idea
Specific links you will want to bookmark for future use
Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.
IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.
ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.
IBM Employees should enter Ideas at https://ideas.ibm.com
I understand your example. But isn't it that with the soft limit, some memory will be paged out, instead of killing a job? Also we could mitigate this somewhat by increasing the low-memory limit on the host, so that e.g. on a 64 Gbyte host, only 60Gbyte can be reserved?
We did previously consider this when we were adding cgroup2 support, something along the lines of:
memory.high = -M hard_memory_limit
memory.low = rusage[mem=xyz]
From various dev discussion forums, this approach would appear to work well when there were just a few jobs on the machine; but on a high core count machine, with many jobs (and thus many cgroups) there were concerns over how memory management would behave with multiple conflicts.
E.g. if the machine was 64GB, and you have 16 jobs with memory.low=4GB, and they all actually consumed it, such that the system.slice was critically low in memory, it could result in the OOM killer killing processes from all 16, rather than just picking one.
Even with weighting all the cgroups, you could conceivably have the edge case where the user for job1 does a bmod to rusage[mem=16gb] which then starts forcing others to swap or be killed.
That said, there have been quite a few enhancements to the cgroup controller over the last few years, and this is something that would be worth revisiting as a future roadmap project.