Skip to Main Content
IBM Data and AI Ideas Portal for Customers


This portal is to open public enhancement requests against products and services offered by the IBM Data & AI organization. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:


Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,


Post your ideas

Post ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,

  1. Post an idea

  2. Upvote ideas that matter most to you

  3. Get feedback from the IBM team to refine your idea


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

IBM Employees should enter Ideas at https://ideas.ibm.com


Status Not under consideration
Workspace Watson Discovery
Created by Guest
Created on Oct 24, 2019

Highlights should include JSON object for Entity and relations

During Projcet Daisey development, we are experiencing very long overall query times and one of the issues is the amount of processing we have to perform on the results to be able to display evidence to the user.

 

When a user searches in WDS for an Entity or Relation, the highlight returns  a highlights section like this :

 

 "highlight": {
"enriched_text.entities.type.raw": [
"<em>CASH</em>"
],
"enriched_text.relations.type.raw": [
"<em>Inventory_for</em>"
]
}

 

This cannot be displayed to a user as a search result, so the application is required to search through the entire enrichment section of the results to find which enrichment was the one that matched.

This can take quite a bit of time if there are a lot of enrichments, as the enriched text also includes every Entity and Relation

 

If the JSON object for the Entity or Relation was included in the Highlights section it would significantly improve overall query time

 

 

 

  • Guest
    Reply
    |
    Oct 25, 2019

    My query could be for a specific relation - entity combo from my custom model:

    (enriched_text.relations:(type::"Related_Costs",arguments.entities.type:"FIXED_ASSETS"))

     

    My query currently returns these highlights,

        "highlight": {
                    "enriched_text.relations.arguments.entities.type": [
                        "<em>FIXED_ASSETS</em>"
                    ],
                    "enriched_text.relations.type.raw": [
                        "<em>Related_Costs</em>"
                    ],
                    "enriched_text.entities.type": [
                        "<em>FIXED_ASSETS</em>"
                    ]
                }

     

    (which actually is wrong because i did not ask to match on enriched_text.entities.type::"FIXED_ASSETS" but only

    enriched_text.relations.arguments.entities.type::"FIXED_ASSETS")

     

    However,  I should be getting back the evidence where it matched, I should get back something like this which would be somewhat the equivalent of getting back the sentence for a keyword match with the term highlighted.:

     

           "highlight": {

     "enriched_text.relations.type.raw": [
                 {
                            "type": "Related_Costs",
                            "sentence": "The recoverable Extra Expense loss is the reasonable and necessary extra costs incurred by the Insured for the following during the PERIOD OF LIABILITY: (1) Extra expenses to temporarily continue as nearly normal as practicable the conduct of the Insured's business; and (2) Extra costs of temporarily using property or facilities of the Insured or others; less any value remaining at the end of the PERIOD OF LIABILITY of property obtained in connection with sub -paragraph (1) or (2) above.",
                            "score": 0.5205,
                            "arguments": [
                                {
                                    "text": "property",
                                    "location": [
                                        34941,
                                        34949
                                    ],
                                    "entities": [
                                        {
                                            "type": "FIXED_ASSETS",
                                            "text": "property",
                                            "disambiguation": {
                                                "subtype": [
                                                    "NONE"
                                                ]
                                            }
                                        }
                                    ]
                                },
                                {
                                    "text": "costs",
                                    "location": [
                                        34914,
                                        34919
                                    ],
                                    "entities": [
                                        {
                                            "type": "COSTS",
                                            "text": "costs",
                                            "disambiguation": {
                                                "subtype": [
                                                    "NONE"
                                                ]
                                            }
                                        }
                                    ]
                                }
                            ]
                        }

     }

        ]

     

    Or at least I should get back relation.sentence for a relation match:

     

    "highlight": {

     "enriched_text.relations.type.raw": [
                 {
                            "type": "Related_Costs",
                            "sentence": "The recoverable Extra Expense loss is the reasonable and necessary extra costs incurred by the Insured for the following during the PERIOD OF LIABILITY: (1) Extra expenses to temporarily continue as nearly normal as practicable the conduct of the Insured's business; and (2) Extra costs of temporarily using property or facilities of the Insured or others; less any value remaining at the end of the PERIOD OF LIABILITY of property obtained in connection with sub -paragraph (1) or (2) above.",

     

     

    and for an entity match I would expect

    {
                            "count": 1,
                            "text": "Equipment",
                            "confidence": 0.768784,
                            "mentions": [
                                {
                                    "sentence_location": [
                                        0,
                                        299
                                    ],
                                    "location": [
                                        130,
                                        139
                                    ],
                                    "text": "Equipment",
                                    "mention_type": "none",
                                    "confidence": 0.768784
                                }
                            ],
                            "type": "FIXED_ASSETS",
                            "disambiguation": {
                                "subtype": [
                                    "NONE"
                                ]
                            }

     

    or at least matching entity text

     

    {
                            "count": 1,
                            "text": "Equipment",
                            "confidence": 0.768784,
                            "type": "FIXED_ASSETS",
                            "disambiguation": {
                                "subtype": [
                                    "NONE"
                                ]
                            }

     

    Let me know if this example makes sense Phil.

  • Guest
    Reply
    |
    Oct 24, 2019

    Can you provide an example of the query that is returning this? It seems like it would mainly be an issue if a bunch of different queries were ORed, or a substring search was performed (otherwise you'd know which enrichment matched as you searched for it)

  • Guest
    Reply
    |
    Oct 24, 2019

    This is critical for clients who develop their own custom model to annotate domain knowledge and then retrieve documents for domain insights.  The current highlight returns only entity and relation type but to show the EVIDENCE of why a particular document matched, i.e. show the actual relation.sentence or entity.text requires returning all enrichments and searching for the exact enrichment that matched. This is  a huge gap for Project Daisey where we need to cache enrichments in an external store for faster retrieval.