Support Anthropic Prompt Caching

[serialx]

Unlike OpenAI APIs, Anthropic models require explicit cache control blocks for the caching to work. And the way they achieve this is by adding an additional key to the ChatCompletionMessageParam like:

    "system": [
      {
        "type": "text",
        "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
      },
      {
        "type": "text",
        "text": "<the entire contents of Pride and Prejudice>",
        "cache_control": {"type": "ephemeral"}
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "Analyze the major themes in Pride and Prejudice."
      }
    ]

ref: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching

It seems it is not trivial to add this feature in the codebase since it does not comply with the type system.

[hanil-jihun]

I need it, too