Extend SessionScheduler being able to manage small parallel application (size up to one node)

Today we have applications/software running in serial mode via session scheduler (runtime per task <<10min) with up to 10000 tasks. These applications/software will migrate to parallel usage soon and we want to keep the SessionScheduler functionality running the 10000 tasks on more than one core per task.

Post comment

Guest

Reply
| Jul 23, 2021

As discussed offline, this use case can be satisfied directly without using session scheduler. A reservation can be dynamically scheduled/created, and the set of jobs run within it.

Today, you will be doing something like:
$bsub -q whatever -n 16 ssched taskfile

So you could do something like:
$bsub -q whatever myssched taskfile 16

where myssched is:

#!/bin/bash
#usage: myssched taskfile nslots
this_user=`whoami`
this_rsv=$this_user-$LSB_JOBID
brsvsub -D 5 -u $this_user -o -n $2 -N $this_rsv
while read ss_cmd; do
bsub -U "$this_rsv" "$ss_cmd
done < $1

0 reply Hide replies

By clicking the "Post Comment" or "Submit Idea" button, you are agreeing to the IBM Ideas Portal Terms of Use.
Do not place IBM confidential, company confidential, or personal information into any field.

Please enter your email address

RELATED IDEAS

Extend SessionScheduler being able to manage small parallel application (size up to one node)