Skip to Main Content
IBM Data and AI Ideas Portal for Customers


This portal is to open public enhancement requests against products and services offered by the IBM Data & AI organization. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:


Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,


Post your ideas

Post ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,

  1. Post an idea

  2. Upvote ideas that matter most to you

  3. Get feedback from the IBM team to refine your idea


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

IBM Employees should enter Ideas at https://ideas.ibm.com


Status Under review
Workspace Planning Analytics
Created by Guest
Created on Jul 16, 2024

improve MDX performance in TI

Dear Sirs,

we wanted to check the performance in v12 concerning MDX and ElComp-function concerning one dimension.


In v11 the performance with ElComp was much better, we thought that with v12 the performance concerning MDX will be improved, however, it is not the case.


Can you say, if there will be a performance improvement in the next versions?


We tested the case with v12, PAW_96, on Linux, 28 GB RAM, 7 Cores.

We created one big dimension with about 5 Mio elements, without attributes.


We created 2 small processes: one with MDX-set and one with ElComp-function.


For the MDX-process the TI needed about 25 seconds,

For the ElComp-process the TI needed about 5 seconds.


###################################


Explanation for the files:

###################################

z_VR_Dim_with_simple_Hierarchy.txt

==> this process creates a simple dimension with one main c-element and different c-elements inbetween.


z_CP_ElComp_Test.txt

==> this process goes with ElComp-function through the WHILE-loop


z_CP_MDX_Test_Test.txt

==> this process goes with MDX through the WHILE-loop.


We also use the undocumented function MilliTime (), to get the time for some actions.


This function is not recognized by the process, however, you can still execute the process.

#####################################


Thank you in advance.


With best regards,


Vitalij

Needed By Yesterday (Let's go already!)
  • Guest
    Reply
    |
    Aug 24, 2024

    hello again, 

    we made a comparable example and for now the time was almost the same. at the end we had for both processes the result of 25 seconds with the same big dimension. 

    Maybe one of the "ideas" for this example was, is it possible for you to get more out of set editing or set and mdx processing? 

    I will send our 2. process to Hubert for the review. 

    Thanks in advance!

  • Guest
    Reply
    |
    Jul 22, 2024

    Stuart, just FYI, Vitalij reached out to me by e-mail, here's the essence of what I replied with to him after taking a look at the processes he provided and realizing what is actually happening:

    When you set up an MDX query (for every record in the source), it involves multiple steps: expressing the query as a string, parsing it, building an execution plan, executing it, and then converting the resulting MDX set of members into a subset of elements. After that's done, you ask to create a static version of this subset, effectively copying the list of elements once again. This process is quite expensive.

    In contrast, the ElComp function directly addresses the list of components, which is ultimately what the MDX expression ends up doing during its execution as well, but without any of the overhead. Given this complexity, it’s impressive that the MDX approach is 'only' five times slower than ElComp.

    The reason I'm saying 'only' is that TI processes aren't highly optimized for repetitive lookups of elements. Every execution of ElComp looks up the dimension and finds the same member repeatedly. Without this repetitive lookup, the performance difference would likely have been even greater.

    I ended with asking if he could explain why he'd prefer using an MDX-based subset whereas this task can easily and much more efficiently be handled with direct TI functions?

  • Guest
    Reply
    |
    Jul 17, 2024

    I created the same processes also at v11-version and got more or less similar relation between mdx- and elcomp-process: 5:1, it means, the process with MDX lasted 5 times longer than with ElComp.

  • Guest
    Reply
    |
    Jul 16, 2024

    it sounds great, thanks.

    In our example we use the process with Subset as datasource.
    I forgot to add this to my description.

  • Admin
    Stuart King
    Reply
    |
    Jul 16, 2024

    IBM will investigate.

    Note that improving performance of MDX views as data sources for TI processes is already planned for the current TM1 v12 roadmap.