Need LSF increase rusage string limit

Delivered as part of LSF10

customer is ok with 2k characters. and they are very interested in scheduler plugin and would like more details. I sent to them the LSF programmers Guide. Please help to provide more information.

Thanks.

I'm checking with end users.

Allowing 10,000 characters would be impractical.

While memory is dynamically allocated, if every job did have 10K rusage string, you would be increasing the memory requirements on the master by 10,000x .

This would also increase the size of the event file - which would significantly increase start up and failover time, as it would be reading a file 10,000x large than before.

Also, allowing thousands of ||'s within the parser would increase scheduling complexity by several orders of magnitude.

We can consider increasing this in a future release, but it needs to be of a sensible size.

An alternative would be developing a scheduling plugin to handle the Palladium logic in much the same way that other plugin's handle complex toplogy.

1> Use case:

This issue came up during development of an external
scheduler by Cadence for the Palladium emulation systems.

2> the reason that needs such a long rusage[] string, 10k charactors should be enough.

There are a large number of unique resources (Palladium
emulator domains) and a given design may be able to run using
any one of literally thousands of different combinations of
those resources. The current workaround is to limit the set of
combinations presented to the LSF scheduler to a very small
subset of the actual set of combinations that the job could be
run with, randomly chosen from the super-set.

There are also combos of resources which is why it grows.

If you use A then you will need X,Y,Z
If you use B then you will need X,Z
If you use C then you also need Y,Z
but you can use either A or B or C

customer found no better way to do that than have rusage = A=1:X=1:Y=1:Z=1||
B=1:X=1:Z=1||C=1:Y=1:Z=1

If there is a better way to present these options then customer would be interested. A-C and X-Z are all dynamic resources.



3) Customer would like to know the LSF scheduler performance impact be
cluster-wide, or only impact those jobs that request the
resources involved?

Creating a new RFE based on Community RFE #63177 in product Platform LSF.