Run multiple jobarrays in parallel in a queue when queue has UJOB_LIMIT specified

As part of the review process, we strive to be transparent about our intentions with each enhancement suggestion. The product team has carefully reviewed this idea and has decided that it does not fit into our current plans, so the request for enhancement record will be closed. The request will be kept in IBM's ideas repository and might be reassessed or reopened for additional feedback or voting in the future.

What YiSun is saying is that we presently support this using the modified syntax he is using. It's another thing entirely to have another layer of policies at the Queue, User, User Group, or Application profile level to limit the concurrency for each job as YiSun has articulated. That's an interesting approach, that I believe has some legs. In the mean time, you could enforce this parallelism at the ESUB level for sure.

The problem with Fairshare as I see it is that the order, for the same user is still somewhat serial. So, you still hit that per user FIFO behavior. So, in the end, I could see something like the following. First, in lsb.users, add something like MAX_ARRAY_PTASKS:

Begin User
USER_NAME MAX_PEND_JOBS MAX_ARRAY_PTASKS
user1 800 20
ugroup1@ 500 40
ugroup1 1000 ()
default 100 ()
End User

Also, in either lsb.applications, or lsb.queues

Begin Queue
QUEUE_NAME = normal
DESCRIPTION = The normal queue
PRIORITY = 50
USERS = all
MAX_ARRAY_PTASKS = 40
EXCLUSIVE = Y
RERUNNABLE = Y
End Queue

This does make sense, and it allows you to reduce the clutter in the cluster wide ESUB.