Description
watsonx Orchestrate on-premises already supports connecting agents to an external Milvus instance as a knowledge source.
However, the external Milvus collection must be created and populated before it can be connected to watsonx Orchestrate. Customers must therefore build and operate their own ingestion pipeline to:
Extract text from documents.
Split documents into chunks.
Generate embeddings.
Create the required Milvus schema and collections.
Insert vectors and metadata.
Manage document updates and deletions.
The native document upload experience available through Agent Builder and the ADK performs these operations automatically, but relies on the embedded Tiny Milvus instance.
Tiny Milvus is intended for limited or non-production use and cannot provide the scalability and operational resilience required for enterprise production workloads.
We request the ability to use a supported external Milvus instance as the managed destination for the native watsonx Orchestrate document ingestion pipeline.
Current behavior
Users have two separate options:
Use the native document upload capability, with automatic parsing, chunking and embedding generation, but store the resulting vectors in Tiny Milvus.
Use an external Milvus instance, but manually create and populate the collections before connecting them to watsonx Orchestrate.
As a result, customers cannot combine the native low-code document upload experience with a production-grade external Milvus deployment.
Expected behavior
Administrators should be able to configure a supported external Milvus instance as the default managed knowledge backend for watsonx Orchestrate.
After this configuration, users should be able to upload documents through Agent Builder, the ADK or supported APIs, while watsonx Orchestrate automatically manages:
Document parsing and text extraction.
Document chunking.
Embedding generation using the selected embedding model.
Collection and schema creation.
Vector and metadata ingestion.
Document updates and deletion.
Association between agents, knowledge bases, documents and Milvus collections.
Collection lifecycle management.
External Milvus configuration should support:
TLS connections and custom certificate authorities.
Supported authentication mechanisms.
Milvus database and collection configuration.
Supported external Milvus deployments, including Milvus provided by IBM watsonx.data.
Production-grade and highly available Milvus architectures.
Connected and air-gapped IBM Software Hub environments.
This request primarily targets Agent Knowledge ingestion through Agent Builder, the ADK and supported APIs.
Business need
Organizations deploying watsonx Orchestrate on-premises require a scalable and production-grade vector database for enterprise knowledge workloads.
Requiring each customer to build a separate ingestion pipeline significantly increases:
Implementation time.
Technical complexity.
Operational and maintenance costs.
Dependency on custom scripts or third-party ingestion tools.
Differences between development and production environments.
Supporting native ingestion into an external Milvus instance would:
Make Agent Knowledge suitable for production workloads.
Preserve the existing low-code user experience.
Eliminate the need for customer-developed ingestion pipelines.
Improve consistency between development, test and production environments.
Support larger document volumes and enterprise-scale agent deployments.
Simplify adoption in sovereign, private-cloud and air-gapped environments.
Example customer scenario
Our watsonx Orchestrate environment is deployed on-premises on IBM Software Hub.
We use an external production-grade Milvus deployment to provide the required scalability, persistence and operational resilience.
watsonx Orchestrate can query this Milvus instance, but users cannot upload documents directly into it through Agent Builder or the ADK.
We must therefore implement and maintain a custom solution for document extraction, chunking, embedding generation, schema creation and vector ingestion.
We would like watsonx Orchestrate to manage these operations automatically while using the external Milvus instance as the storage backend.
Suggested acceptance criteria
An administrator can register a supported external Milvus instance as a managed knowledge ingestion backend.
Agent Builder can upload and ingest documents directly into the configured external Milvus instance.
ADK knowledge bases containing document files can use the configured external Milvus backend.
Supported APIs can trigger document ingestion into the external Milvus instance.
watsonx Orchestrate automatically performs parsing, chunking, embedding generation and vector ingestion.
Updating or deleting a document updates or removes the corresponding vectors and metadata.
Multiple agents and knowledge bases can use the same Milvus service with appropriate database or collection isolation.
TLS, custom certificate authorities and supported authentication mechanisms are supported.
The capability is available in connected and air-gapped IBM Software Hub environments.
IBM documents the supported Milvus versions, deployment architectures, schemas, authentication methods and operational responsibilities.
IBM Support accepts cases involving the managed ingestion pipeline when a validated external Milvus configuration is used.
