IBM Data Platform Ideas Portal for Customers
This portal is to open public enhancement requests against products and services offered by the IBM Data Platform organization. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).
Shape the future of IBM!
We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:
Search existing ideas
Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,
Post your ideas
Post ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,
Post an idea
Upvote ideas that matter most to you
Get feedback from the IBM team to refine your idea
Specific links you will want to bookmark for future use
Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.
IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.
ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.
IBM Employees should enter Ideas at https://ideas.ibm.com
this is exactly the plan in Provides logs to debug Streaming Pipelines
One idea is that in the interface there could be alerts that persist until the user closes them or the pipeline status changes. That alert could lead the user to the log data that corresponds to the failure. I'm sure there are other ways to tackle it -- please feel free to solicit input from the team in the channel, here, or in a call.
@brad can you describe what is it exactly you are expecting as "much bigger heads-up"?
@erel, I don't agree this is a duplicate. The other cases are about having visibility into pipeline status from the index of streaming pipelines, and logs for debugging, but this case relates to pipelines giving users a much bigger heads-up when they fail.
This seems to be a duplicate (or an aggregation) of the following ideas:
Indicate the Streaming Pipeline Status from the Analytics assets page
Provides logs to debug Streaming Pipelines
In addition when the DSX notification system is in place we will publish pipeline status change event to the notification and user can get alerts when something is unhealthy with the pipeline
closing for now
Changes have been requested to the Streams product to provide some improvements to allow Streaming pipelines to handle debug better but these were developed in the code line for the new WAMs based Streaming analytics service that is delayed. Due to that we are using the existing Streaming Analytics service so we currently plan to deliver the underlying streaming analytics service improvements August 31st.
Meanwhile we need to see what improvements can be made in the interim. From a brief conversation with Brad they often see data just stops flowing or object storage files do not get created. It can be hard to determine if the issue is with streaming pipelines or message hub feeding it or object storage etc. SP can show flows, but I don't believe it shows operator health/failures. If a user saw that things are healthy with no data they should then know to look at the source but it needs to be clear that the flow is healthy. It may also be useful to show a metric on the number of messages read from message hub so a user can see if its not getting any new messages in (but this is already implied if no data comes out of the source so may not add much).
I believe that the team are planning to deliver some improvements next week. Erel, can you please describe the pending improvements here.