Fix missing content in OpenAI assistant message

[slmnsh]

When LLM responds with ToolCall directly without any TextPart it's missing content key in Assistant Message. It usually doesn't cause any error but if you try to pass PDF afterwards OpenAI API responds with 500

[
  {
    "content": "...",
    "role": "system"
  },
  { "role": "user", "content": "yash resume search" },
// --------- Assistant message without content key ----------
  {
    "role": "assistant",
    "tool_calls": [
      {
        "id": "call_EPt7QObN5htQwx6WBQaWS8GX",
        "type": "function",
        "function": {
          "name": "search_tool",
          "arguments": "{\"query_text\":\"I am searching for resumes or CVs related to a person named Yash. The document should be a resume or curriculum vitae. yash, resume, CV, curriculum vitae\",\"max_num_documents\":8}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "tool_call_id": "call_EPt7QObN5htQwx6WBQaWS8GX",
    "content": "{\"documents\":[{\"file_path\":\"Yash_resume.pdf\",\"id\":\"e1d7eb5a-e151-450a-b23b-ac446cde4179\",\"score\":0.01639344262295082,\"file_name\":\"Yash_resume.pdf\",\"content\":\"\"}]}"
  },
  {
    "role": "assistant",
    "content": "I found a document titled \"Yash_resume.pdf\" that matches your search for a resume related to a person named Yash. \n\nWould you like me to extract the content, summarize, or perform any specific action with this resume? Please let me know how you'd like to proceed."
  },
// --------- Now this message returns 500 error ------------
  {
    "role": "user",
    "content": [
      { "text": "who is yash?", "type": "text" },
      {
        "file": {
          "file_data": "data:application/pdf;base64,...",
          "filename": "filename.pdf"
        },
        "type": "file"
      }
    ]
  }
]

OpenAI Error:

pydantic_ai.exceptions.ModelHTTPError: status_code: 500, model_name: gpt-4.1, body: {'message': 'The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID df449f7f-6810-4aa8-b06a-c08eef9f09b2 in your email.)', 'type': 'server_error', 'param': None, 'code': None}

[DouweM]

@slmnsh Thanks Salman! Can you please share the exact error you received from OpenAI? And please add a test to ensure we don't accidentally reintroduce this issue in the future.

[slmnsh]

@DouweM Added exact error in description and tests

[DouweM]

@slmnsh Thanks. I'm a bit hesitant to merge this because:

• it's clearly an OpenAI server side bug, as indicated by error 500, it working without the PDF, and their docs explicitly stating that content can be skipped when there are tool_calls
• it could actually break other providers/models whose "OpenAI-compatible" APIs are used through the OpenAIModel, as we've seen other models error on empty contents: Prevent Anthropic API errors from empty message content #1934

So I don't know if this workaround would break more than it fixes. Since this is a real internal error on OpenAI's side, I'd prefer seeing if we can get them to fix it first. Can you please provide me with a minimal reproducible example, ideally directly using their SDK instead of Pydantic AI, that I can try to get to their attention?

Edit: Also, for the test, I'd expect something that actually shows the OpenAI error with the current code, and works with the new code -- as is it's basically just restating the fix instead of actually verifying that the fix works against the API!

[slmnsh]

I thought you guys missed this but seems like OpenAI issue here.

Minimal Example:

import base64
import httpx
from openai import OpenAI
from openai.types.chat import ChatCompletionMessageParam

client = OpenAI()

messages: list[ChatCompletionMessageParam] = [
    {"content": "This is system prompt", "role": "system"},
    {"role": "user", "content": "search abc data"},
    {
        "role": "assistant",
        "tool_calls": [
            {
                "id": "call_EPt7QoBn5HTqWx6wbqAws8gX",
                "type": "function",
                "function": {
                    "name": "search_tool",
                    "arguments": '{"query_text":"ABC Data"}',
                },
            }
        ],
    },
    {
        "role": "tool",
        "tool_call_id": "call_EPt7QoBn5HTqWx6wbqAws8gX",
        "content": "ABC is a company.",
    },
    {"role": "assistant", "content": "I found that ABC is a company"},
]

def main():
    pdf_url = "https://assets.ctfassets.net/l3l0sjr15nav/29D2yYGKlHNm0fB2YM1uW4/8e638080a0603252b1a50f35ae8762fd/Get_Started_With_Smallpdf.pdf"
    pdf = httpx.get(pdf_url)
    base64_encoded = base64.b64encode(pdf.content).decode('utf-8')
    messages.append({
        "role": "user",
        "content": [
            {"text": "Can you summarize this", "type": "text"},
            {
                "file": {
                    "file_data": f'data:application/pdf;base64,{base64_encoded}',
                    "filename": "filename.pdf",
                },
                "type": "file",
            },
        ],
    })
    completion = client.chat.completions.create(model="gpt-4.1", messages=messages)
    print(completion.choices[0].message.content)

if __name__ == "__main__":
    main()

Sorry about tests. I am kinda new to testing 😅

[DouweM]

@slmnsh Thanks, I've reported it, let's see what they say.