Skip to Main Content
IBM Data and AI Ideas Portal for Customers


This portal is to open public enhancement requests against products and services offered by the IBM Data & AI organization. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:


Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,


Post your ideas

Post ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,

  1. Post an idea

  2. Upvote ideas that matter most to you

  3. Get feedback from the IBM team to refine your idea


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

IBM Data & AI Roadmaps (http://ibm.biz/Data-and-AI-Roadmaps) - Use this site to view roadmaps for Data & AI products.

IBM Employees should enter Ideas at https://hybridcloudunit-internal.ideas.aha.io/


Status Delivered
Workspace Spectrum Symphony
Components Version 7.2.0.2
Created by Guest
Created on Jul 15, 2021

SD bypass patch

IBM has suggested to implement SD Bypass patch in order to resolve ongoing issue in Citi grid LNPRD cluster. Details of issue is below in form of investigations:

We have analyzed the data we collected yesterday and have found the delay in authentication process is caused mostly in OS syscalls. Below is snippets from both LND and SW grids.


1. strace
23933 20:04:20.437828 write(3, "2021-06-21 20:04:20.437 GMT DEBUG [23585:139771402884864] sd.ssmManager.SdSsmManager - SsmManager::sdkConnect(): Entered:...for app name: symping7.2\n", 150) = 150 <0.000014>
-> SD thread PID=23933 logs sdkConnect message prior to start authentication and authorization processes of the job submission user.

23933 20:04:20.444712 poll([{fd=26, events=POLLIN}], 1, 15000
23933 20:04:20.580480 <... poll resumed>) = 1 ([{fd=26, revents=POLLIN}]) <0.135755>
-> a delay occurred during poll() syscall. Waiting on file descriptor 26.

23933 20:04:20.584180 write(3, "2021-06-21 20:04:20.584 GMT DEBUG [23585:139771402884864] servers.common.resmngr.VEMResManagerBase - VEMResManagerBase::checkRbacUserPermissionInternal() : cache kit key=Admin#/SymTesting/Symping72#SOAM_APP_LOGIN cache size=489\n", 228) = 228 <0.000022>
-> logged by SD as soon as authentication and authorization process is done.

2. lsof
sd 23585 symadmp 26u IPv4 28674654 0t0 TCP gridmstsw30p.nam.nsroot.net:47690->gmwgtdcpsp07p.nam.nsroot.net:44443 (ESTABLISHED)
-> checking SD's lsof output, 26u file descriptor is TCP socket between SD<->Siteminder. This indicates SD was waiting for data arrival at FD=26 from Siteminder port 44443. (POLLIN event waits until data is ready at given FD.)


1. strace
25048 19:18:56.989822 write(3, "2021-06-21 19:18:56.989 GMT DEBUG [24709:139991709902592] sd.ssmManager.SdSsmManager - SsmManager::sdkConnect(): Entered:...for app name: symping7.2\n", 150) = 150 <0.000016>
-> SD thread PID=25048 logs sdkConnect message prior to start authentication and authorization processes of the job submission user.

25048 19:18:56.989893 futex(0x34bf760, FUTEX_WAKE_PRIVATE, 1) = 1 <0.000012>
25048 19:18:56.989940 futex(0x3bf4750, FUTEX_WAIT_PRIVATE, 2, NULL
25047 19:18:57.645953 futex(0x34bf760, FUTEX_WAKE_PRIVATE, 1) = 1 <0.000012>
25047 19:18:57.646001 futex(0x3bf4750, FUTEX_WAKE_PRIVATE, 1
25048 19:18:57.646037 <... futex resumed>) = 0 <0.656088>
25048 19:18:57.650643 poll([{fd=26, events=POLLIN}], 1, 15000
25048 19:18:57.777387 <... poll resumed>) = 1 ([{fd=26, revents=POLLIN}]) <0.126732>
25048 19:18:57.780711 poll([{fd=25, events=POLLIN}], 1, 15000
25048 19:18:57.894638 <... poll resumed>) = 1 ([{fd=25, revents=POLLIN}]) <0.113916>
25048 19:18:57.897883 poll([{fd=26, events=POLLIN}], 1, 15000
25048 19:18:58.024320 <... poll resumed>) = 1 ([{fd=26, revents=POLLIN}]) <0.126426>
-> delay occurred during 3 poll() sequential syscalls. SD waiting on file descriptor 26 and 25.

25048 19:18:58.027782 write(3, "2021-06-21 19:18:58.027 GMT DEBUG [24709:139991709902592] servers.common.resmngr.VEMResManagerBase - VEMResManagerBase::checkRbacUserPermissionInternal() : cache kit key=Admin#/SymTesting/Symping72#SOAM_APP_LOGIN cache size=167\n", 228
-> logged by SD as soon as authentication and authorization process is done.

2. lsof
sd 24709 symadmp 25u IPv4 136235231 0t0 TCP gridmstln30p.eur.nsroot.net:53580->gmwmwdcpsp07p.nam.nsroot.net:44443 (ESTABLISHED)
sd 24709 symadmp 26u IPv4 133881256 0t0 TCP gridmstln30p.eur.nsroot.net:40578->gmwgtdcpsp07p.nam.nsroot.net:44443 (ESTABLISHED)
-> checking SD's lsof output, those FDs are TCP sockets between SD<->Siteminder. It contacted 2 different siteminder server.


• We observe delays are related to siteminder authentications.
• In LND grid case, during authentication phase SD thread had to contact two different siteminder servers. Please turn off second siteminder server (we believe it is gmwmwdcpsp07p based on comparison with SW case), as it may help avoid attempt sending the user query to the second siteminder.
• Or check with Siteminder team if two siteminder servers are in synced in terms of user database. (i.e. if user A is not query-able in siteminder A, it may reach out to B)
• Siteminder plugin code in symphony is owned in Citi, suggest to investigate plugin code to remediate the delays.

  • Guest
    Aug 16, 2021

    .The required enhancement is available on IBM Fix Center: sym-7.2.0.2-build600609-citi