Skip to Main Content
IBM Data and AI Ideas Portal for Customers


This portal is to open public enhancement requests against products and services offered by the IBM Data & AI organization. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:


Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,


Post your ideas

Post ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,

  1. Post an idea

  2. Upvote ideas that matter most to you

  3. Get feedback from the IBM team to refine your idea


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

IBM Employees should enter Ideas at https://ideas.ibm.com


Status Not under consideration
Workspace Spectrum LSF
Created by Guest
Created on Feb 19, 2020

Run multiple jobarrays in parallel in a queue when queue has UJOB_LIMIT specified

We currently use LSF to distribute electrical analog design simulations to our cluster. These simulations can be very short (>3 minutes) to very long (<48hrs).

We have a queue system with timelimits, which means users can select a queue based on how long a simulation runs. Most of the time, the users submits multiple jobarrays jobs to one queue.

However, due to how queues now work, the user has to wait for his first submitted job array to finish. Then only the second jobarray will start to run. This does not allow the user to work on multiple simulations in parallel. It would be a massive benefit to us if multiple jobarrays of one user, in one queue can run together if the queue has a job limit set.

  • Guest
    Reply
    |
    Apr 30, 2021

    As part of the review process, we strive to be transparent about our intentions with each enhancement suggestion. The product team has carefully reviewed this idea and has decided that it does not fit into our current plans, so the request for enhancement record will be closed. The request will be kept in IBM's ideas repository and might be reassessed or reopened for additional feedback or voting in the future.

  • Guest
    Reply
    |
    Feb 20, 2020

    What YiSun is saying is that we presently support this using the modified syntax he is using. It's another thing entirely to have another layer of policies at the Queue, User, User Group, or Application profile level to limit the concurrency for each job as YiSun has articulated. That's an interesting approach, that I believe has some legs. In the mean time, you could enforce this parallelism at the ESUB level for sure.

    The problem with Fairshare as I see it is that the order, for the same user is still somewhat serial. So, you still hit that per user FIFO behavior. So, in the end, I could see something like the following. First, in lsb.users, add something like MAX_ARRAY_PTASKS:

    Begin User
    USER_NAME MAX_PEND_JOBS MAX_ARRAY_PTASKS
    user1 800 20
    ugroup1@ 500 40
    ugroup1 1000 ()
    default 100 ()
    End User

    Also, in either lsb.applications, or lsb.queues

    Begin Queue
    QUEUE_NAME = normal
    DESCRIPTION = The normal queue
    PRIORITY = 50
    USERS = all
    MAX_ARRAY_PTASKS = 40
    EXCLUSIVE = Y
    RERUNNABLE = Y
    End Queue

    This does make sense, and it allows you to reduce the clutter in the cluster wide ESUB.