IBM Data Platform Ideas Portal for Customers


This portal is to open public enhancement requests against products and services offered by the IBM Data Platform organization. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:


Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,


Post your ideas

Post ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,

  1. Post an idea

  2. Upvote ideas that matter most to you

  3. Get feedback from the IBM team to refine your idea


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

IBM Employees should enter Ideas at https://ideas.ibm.com



Add a new idea Subscribe
Status Not under consideration
Workspace Spectrum LSF
Components Administration
Created by Guest
Created on Apr 16, 2026

RELATED IDEAS

Job IDLE Detection (JOB_IDLE in lsb.queues) need "differential" idle detection

Sometimes we see interactive jobs that after some computation time (e.g. of several hours)
the computation is finished and they (might) wait for a user interaction. So the job essentially
does nothing but keeping the LSF job running and resources allocated.

Also, for batch jobs we have cases where after the computation has finished due to e.g. 
custom software errors jobs do not finish correctly but wait for "something", also keeping the LSF job running.

In both cases the used CPU time is (nearly) 0.0 from the time the *computation* has finished.
We need to detect those "idle" jobs ("job leftovers") to either send messages or kill them
to perform some custom actions.


There is the exception handling feature  using JOB_IDLE in lsb.queues and eadmin in LSF_SERVERDIR.

The problem here is that the documentation for JOB_IDLE:

JOB_IDLE
   Specify a threshold for idle jobs. The value should be a number between 0.0 and 1.0 representing *CPU time/runtime*. If the job idle factor is less than the specified threshold, ...

So the criteria is current "CPU time" in relation to the "full runtime".


This measure is not good enough to reliably detect real "idle" jobs described above because
the IDLE value is  the relation of the current usage to the accumulated runtime since
job start.

See the attached picture showing the IDLE factor of a job running on 4 CPUs for
about 2000 seconds and then stopping CPU usage completely:
The IDLE factor very slowly decreases and we cannot easily define a useful threshold
of the IDLE factor to determine if the JOB is idle. We would have to wait for a long time
and it depends on the real "used" calculation time (which may vary significantly).


So depending on the runtime of the real calculation using high (or medium) CPU load the
calculated JOB IDLE values are *slowly* decreasing even e.g. if the CPU load of the job 
"over last 20 minutes" is nearly 0.0

I would like to have the IDLE factor calculated e.g. as

"CPU time used over last x minutes" / "max possible CPU time over last x minutes"
or
"CPU time used over last x minutes" / "100% CPU time"

So a kind of a "differental" calculation of the IDLE factor.

IDLE_values.png
IDLE_values.png
Open full size
Needed By Quarter
  • Admin
    Bill McMillan
    Jun 20, 2026

    We understand this specific scenario, but are cognizant that the opposite is also true of different applications - initially consume very little cpu vs wallclock while data is being load, which your proposal could result in those jobs being incorrectly terminated.

    The WATCHDOG parameter in lsb.applications allows you do define a custom script for a specific application - using this would be the better approach to defining a custom idle behaviour.

    Reply

By clicking the "Post Comment" or "Submit Idea" button, you are agreeing to the IBM Ideas Portal Terms of Use.
Do not place IBM confidential, company confidential, or personal information into any field.