Improve Search Functionality

Business Requirement

We request an enhancement to the search capabilities within the IBM watsonx.data Intelligence Catalogue to improve data discovery through business-oriented metadata. Currently, catalogue searches primarily rely on physical or technical asset names, with limited support for business metadata. Consequently, users are often unable to locate relevant assets using the business terminology commonly used across the organisation. To improve search effectiveness and user experience, business metadata should be treated as a first-class search criterion and included within search indexing, matching, and ranking algorithms.

The search engine should index and search against:

• Business Terms
• Logical Names
• Business Descriptions
• Glossary Terms and Associations
• Column-Level Business Metadata
• Related Metadata Relationships
• Synonyms and Business Vocabulary

Search results should prioritise relevant business metadata matches alongside physical asset names.

Current Behaviour

When searching within the catalogue, assets are generally returned when the search term matches a physical table, view, or column name. Searches based on associated business terms, logical names, or glossary metadata do not consistently return the expected assets.

Example 1: Business Term Search

Multiple data assets contain columns mapped to the business term "Architecture Model".

When searching for:

• Architecture Model
• Architecture
• Model

the catalogue primarily returns assets where these values appear within physical column names.

Assets associated with the business term "Architecture Model" are not consistently returned, despite having an explicit glossary relationship.

Example 2: Column Search Within Large Tables

When viewing a table containing hundreds of columns, searching for:

• Architecture Model
• Architecture
• Architecture M

does not return columns associated with the corresponding business term unless the technical column name contains the searched text. This suggests that the column search functionality evaluates only physical column names rather than associated business metadata.

Example 3: Logical Name Search

A view may have:

Physical Name: emp_sum_vw

Logical Name: Employee Summary View

Searching for "Employee Summary View" does not return the asset. Users can only locate it if they know the underlying technical object name.

Business Impact

This limitation significantly reduces the effectiveness of the catalogue as a self-service data discovery platform.

Impact on Users

• Business users search using business terminology rather than technical database object names.
• Data stewards invest significant effort in creating and maintaining glossary terms, logical names, and business metadata.
• Data consumers struggle to locate relevant assets without prior technical knowledge.
• Metadata investments deliver limited value when business context is not fully leveraged during search.
• Catalogue adoption is reduced because users cannot easily discover assets using familiar business language.
• Dependence on technical SMEs increases as users require assistance locating data assets.