hermes - 💡(How to fix) Fix Mistral models fail with 'Unexpected role user after role tool' in multi-turn tool conversations [1 participants]

When using Mistral models (e.g. mistralai/mistral-small-4-119b-2603) via NIM or any OpenAI-compatible endpoint, conversations break after tool use. The error is:

TL;DR

Insert a synthetic empty assistant message before appending the next user message when using Mistral models to resolve the HTTP 400 error.

Guidance

• Verify that the issue occurs only with Mistral models and not with other providers like OpenAI or Anthropic.
• Check the conversation flow to ensure that an assistant message is sent after tool results and before the next user message.
• Implement the suggested fix by inserting a synthetic empty assistant message or merging tool results into the preceding assistant turn.
• Test the fix by reproducing the issue using the provided reproduction steps and verifying that the HTTP 400 error is resolved.

Example

# Example of inserting a synthetic empty assistant message
def send_user_message(message):
    # ... (existing code)
    if using_mistral_model:
        send_assistant_message("")  # Send an empty assistant message
    send_user_message(message)

Notes

This fix may not apply to other models or providers that do not strictly enforce the conversation format. Additionally, the implementation details of inserting a synthetic empty assistant message may vary depending on the specific use case and codebase.

Recommendation

Apply the workaround by inserting a synthetic empty assistant message, as it is a targeted fix for the specific issue with Mistral models.