Regarding Mistral Large, I would like the context size to be able to use up to the model size limit (128K) in order to implement RAG & Agentic applications.

There are the following limitations when using Mistral Large via watosonx.ai.

Mistral Large model limit: 128K watsonx.ai limit: 32K

https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx#mistral-large

Currently, in the Japanese market, Mistral Large has good accuracy for Japanese, and this model is often used for RAG applications.

If you implement a RAG application with a 32K limit, you will have to reduce the number of documents included in the context by 1/4, making it difficult to maintain RAG accuracy.

This is not a problem for sample applications, but if you are trying to implement a RAG application or AI agent used in production, a context size of 128K is desirable.

Needed By

Month

Post comment

By clicking the "Post Comment" or "Submit Idea" button, you are agreeing to the IBM Ideas Portal Terms of Use.
Do not place IBM confidential, company confidential, or personal information into any field.

Please enter your email address

RELATED IDEAS

Regarding Mistral Large, I would like the context size to be able to use up to the model size limit (128K) in order to implement RAG & Agentic applications.