Show number of requested slots for pending jobs in bjobs output

It is addressed in LSF 10.1.0.7. You can download LSF 10.1.0.7 from Fix Central.

Fix Central URL(s):
IBM Spectrum LSF:
http://www.ibm.com/support/fixcentral/swg/selectFixes?product=ibm/Other+software/IBM+Spectrum+LSF&release=All&platform=All&function=fixId&fixids=lsf-10.1.0.7-spk-2018-Dec-build509238&includeSupersedes=0

Fix ID: lsf-10.1.0.7-spk-2018-Dec-build509238

The following files are uploaded:
Readme_build509238.htm
checksum.md5
lsf10.1.0.7_fixed_bugs.pdf
lsf10.1_linux2.6-glibc2.3-x86_64-509238.tar.Z
lsf10.1_lnx310-lib217-ppc64le-509238.tar.Z
lsf10.1_lnx310-lib217-x86_64-509238.tar.Z
lsf10.1_lnx312-lib217-armv8-509238.tar.Z

In your example, the pending reason is accurate - the job is not running because of the specific limit being reached.

However, I do agree that the output of bjobs -o for the pending job may be confusing to a user who did not understand what they were asking for.

We'll consider implementing an enhancement in a future LSF 10 serivce pack to update the requested slot count when it can be calculated at submission.

Here is another, more explicit example. Let's say we have a slot limit of 16 slots/user for a given queue, and a user submits this job:

bsub -n 4 -R "2*{affinity[core(4)]} + 2*{affinity[core(7)]}" myapp

This job will never be dispatched, since the total number of slots needed is 22 (2*4 + 2*7), which exceeds the 16 slots/user limit for the queue.

bjobs shows:

$ bjobs -o "jobid: min_req_proc:4 max_req_proc:4 stat: name:"
JOBID MIN_ MAX_ STAT JOB_NAME
14333 4 4 PEND myapp

and 'bjobs -p' says:

Resource limit defined on queue has been reached (Resource: slots, Queue: hpc, Limit Name: hpc_default_slot_limit, Limit Value: 16);

To see the real reason for the pending, I have to analyze the RES_REQ expression "manually", although LSF must have this information available, otherwise the job would not be pending.

This is confusing!

Thanks for the feedback. I agree that for simple usage, the behaviour could be viewed as confusing; but for complex usage, showing the wrong value would be equally confusing.
We'll review this again and see if we can come up with a solution that address the opposing viewpoints.

Thanks for your feedback, and your explanations of the current behavior. However, while your arguments make good sense for somebody with at least some knowledge in job scheduling, and especially how LSF works internally, the normal user (and especially new users) might easily get confused.

In our environment, we have hundreds of users, were the majority is not very experienced. Most users will ask for X cores for a job, assuming that their jobs will run on X cores - no more, no less. Furthermore, we apply certain limits, e.g. the number of cores (slots) that a user might use simultaneously.

For a simple request, i.e. '-n X', the number of requested slots/core is easy to see, for more complicated ones, like the one mentioned in the RFE, or even more complicated compound requests, the user will have a hard time to understand the pending of a job, if the 'total requested' number of slots is not shown. It will also make the live of admins and/or help desk support easier, if this number is easily accessible.

I know there will be situations, where there can an interval, i.e. a min. and a max. number. I'd suggest that the minimum number of requested slots should be shown here, as a default, since this is the minimum requirement that e.g. violates the core limit per user.

I hope the above makes sense. Feel free to get back to me, and we can discuss this further, if needed.

Apologies for the delay in responding as this RFE has resulted in a lot of internal discussion.

In your example: bsub -n 4 -R "span[ptile=1] affinity[core(4)]" : the number of slots to allocate is fixed and can be calculated while the job is pending.

However, in many other cases, the number of slots to be allocated is not known until the job is actually scheduled. For example:

* bsub -x
* if affinity includes exclusive=alljobs or affinity[numa(1)]
* if there are alternative resource requirements
* if we have -n min,max, or using JOB_SIZE_LIST,
* bsub -n 2 -R "affinity[numa(1)]"

the actual allocation will depend on the hosts allocated, so the "correct" value cannot be calculated until the job is actually scheduled & dispatched - which is the current behaviour.

Our rational behind the current behaviour is consistency - easier to explain to a user that they value is only shown once they job is dispatched, rather than to explain why the value would be shown for some jobs, and not for others.

If you think the latter is easier to explain to users, then we can consider making this configurable in a future release.