Queue level cgroup enablement

CPU Shares is the way to go.  Then, use numa affinity on every job same[numa] basically with the exception for real large memory jobs.

Robert,
From your support ticket, the first three parameters do need to be set globally as the control how we collect resource information from the host.

LSF_LINUX_CGROUP_ACCT=Y 
LSF_REPLACE_PIM_WITH_LINUX_CGROUP=Y 
LSB_RESOURCE_ENFORCE="cpu memory"

if the job itself has no cpulimit or memory limit specified then the 3rd parameter has no effect.

Prior to service pack 14, the job would only be moved to its own group if affinity[core(1)] was added to the resource requirements.

The parameter

LSF_CGROUP_CORE_AUTO_CREATE=Y 

was added to simplify that, in that many users (and admins) often forgot to add it, then wondered why the job not been strictly bound.

So the simplest solution is to leave that parameter as N, and add affinity[core(1)] to the jobs/apps that you want strictly bound.

In service pack 15, cpu.shares and cpu.weight are also set automatically to give the job a minimum cpu time allocation proportional to the cores asked/total cores. This can be scaled by using CGROUP_CPU_SHARES_FACTOR or disabled by LSB_CGROUP_CPU_SHARES_OLD_DEFAULT=Y . This does not require the job to be bound to specific cores.

There will be some additional functionality, possibly later this quarter, which will also enable a maximum percentage share of the host, again, without the need to bind to specific cores (so auto_create=n).


We will consider moving LSF_CGROU_CORE_AUTO_CREATE to an application/queue level parameter in a future release.