Model Repository and Custom Repository Integration

The current state of watsonx.ai in terms of installing models is far away from our expectatins. You told us, that watsonx.ai should be a solution for business users, but if you want to load a model this is very complex, intransparent, time consumeing and frustrating. And loading models is the basic of every ai use case....

What are the pain points:

-> Model loading has to be triggerd with the oc cli, this is not the expecatin for a tool to be used from business users

-> The loading itself is frustrating. After editing the config map you see a % progress bar and if it gets stuck you don't know where and why (maybe everything works fine)

-> It is not possible to integrat an third party tool for model hosting as nexus. In our high regulated banking environment we need to scan all models downloaded, so we want to integrate nexus

-> The model documentation is really very bad (content, expectatins....)

here between huggingface and the runtime.

What we would like to have:

-> An Model Repository UI where wecould set up our model repositorys for example huggingface, nexus (inhouse), TensorFlow Hub, PyTorch Hub, ONNX Model Zoo, Open Model Zoo, GitHub / Git repos models

-> An Model UI with the following areas

1) Foundation Models - IBM Provided Models / Red HAT Optimized Models

-> A List of all Available models

-> Model Requirements (gpus vram)

-> Model Description

-> Model Runtime information hosting the model

-> Download status

-> available - model is ready to be installed

-> downloading - model is currently downloading (speed, % finished from GBs)

-> downloaded - model is in the local repository (pvc) and ready to be installed, not all local models should be used immediately

-> installing - model runtime is starting

-> ready - model is ready to use -> link to model endpoint with api description

-> Action Buttons

-> Download

-> Install

-> Uninstall

-> Delete

2) Custom Models - Non Standart Models from a model repository Provided

-> A List of all Available models

-> Action Buttons

-> Add Model

-> Model Check Button - should check, if the model could be loaded (if all files needed are there)

-> Choose Runtime for the model (KServe, etc)

-> Download

-> Install

-> Uninstall

-> Delete

We are currently testing watsonX.ai and I really have to say I expected this to be in the product, what is currently in there is "not useable" and I don't say the s word now.

I see no reason to buy watsonX.ai withou this functionality, because then it is better and cheaper to just use openshift ai. there at least you can controll what you are doing.