Getting ValueError When Using RunnableLambda with LLM in LangChain

[quartermaine]

I'm trying to dynamically initialize an LLM using RunnableLambda and pass it into a chain. I want to be able to get the deployment credentials for llm either by the user of from the environment file. But I am getting the below error:

ValueError: Invalid input type <class 'dict'>. Must be a PromptValue, str, or list of BaseMessages.

Minimal Reproducible Code

from langchain_core.runnables import RunnableLambda, RunnablePassthrough
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Dummy prompt for SQL generation
final_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant"),
    ("human", "{question}")
])

llm_az = RunnableLambda(lambda x: None)  # Placeholder for LLM

def get_llm_from_state(state):
    global llm
    if llm is None:
        llm = AzureChatOpenAI(
            model="gpt-35-turbo",
            deployment_name="gpt-35-turbo",
            openai_api_version="2023-05-15",
            azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
            openai_api_key=os.getenv("AZURE_OPENAI_KEY")
        )
    return llm

sql_chain = (
    RunnablePassthrough.assign(
        llm=RunnableLambda(lambda x: get_llm_from_state(x))
    )
    | final_prompt
    | llm_az
)

# Sample input
result = sql_chain.invoke({"question": "What is 2+2?"})
print(result)

Why am I getting this error?

I believe it's happening because the prompt isn't formatted correctly before being passed to .invoke(), but I'm not sure how to fix it.

Any idea what I'm doing wrong? How can I safely pass a dynamically initialized LLM into a LangChain runnable chain?

[dosubot[bot]]

The ValueError you're encountering is due to the input type being a dictionary, which is not directly supported by the .invoke() method in LangChain. To resolve this, ensure that the input to .invoke() is formatted as a PromptValue, str, or a list of BaseMessages. Here's how you can fix the issue:

1. Format the Input Correctly: You need to ensure that the input passed to the final_prompt is correctly formatted. You can use the ChatPromptTemplate to format the input as a string or a list of messages.

2. Modify the Chain: Adjust the chain to ensure the input is correctly formatted before being passed to the LLM. Here's an updated version of your code:

from langchain_openai import AzureChatOpenAI
from langchain_core.runnables import RunnableLambda, RunnablePassthrough
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
import os

# Dummy prompt for SQL generation
final_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant"),
    ("human", "{question}")
])

llm_az = RunnableLambda(lambda x: None)  # Placeholder for LLM

def get_llm_from_state(state):
    global llm
    if llm is None:
        llm = AzureChatOpenAI(
            model="gpt-35-turbo",
            deployment_name="gpt-35-turbo",
            openai_api_version="2023-05-15",
            azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
            openai_api_key=os.getenv("AZURE_OPENAI_KEY")
        )
    return llm

sql_chain = (
    RunnablePassthrough.assign(
        llm=RunnableLambda(lambda x: get_llm_from_state(x))
    )
    | final_prompt
    | llm_az
)

# Sample input
formatted_input = final_prompt.invoke({"question": "What is 2+2?"})
result = llm_az.invoke(formatted_input)
print(result)

In this updated code, the final_prompt.invoke() method formats the input correctly before passing it to the LLM. This should prevent the ValueError by ensuring the input is in an acceptable format ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[quartermaine]

@dosu

I want to use the sql as an input to another chain latter, how to format the sql_chain?

[dosubot[bot]]

To use the output of sql_chain as input for another chain, you need to ensure that the output is in a format that the next chain can accept. You can achieve this by using the AgentExecutor to manage the sequence of operations. Here's an example of how you can format and chain the outputs:

from langchain_openai import AzureChatOpenAI
from langchain_core.runnables import RunnableLambda, RunnablePassthrough
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.agents import AgentExecutor, create_tool_calling_agent
import os

# Dummy prompt for SQL generation
final_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant"),
    ("human", "{question}")
])

llm_az = RunnableLambda(lambda x: None)  # Placeholder for LLM

def get_llm_from_state(state):
    global llm
    if llm is None:
        llm = AzureChatOpenAI(
            model="gpt-35-turbo",
            deployment_name="gpt-35-turbo",
            openai_api_version="2023-05-15",
            azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
            openai_api_key=os.getenv("AZURE_OPENAI_KEY")
        )
    return llm

# Define the sql_chain
sql_chain = (
    RunnablePassthrough.assign(
        llm=RunnableLambda(lambda x: get_llm_from_state(x))
    )
    | final_prompt
    | llm_az
)

# Sample input
formatted_input = final_prompt.invoke({"question": "What is 2+2?"})
sql_result = llm_az.invoke(formatted_input)

# Use the output as input for another chain
# Define another chain or tool that takes the SQL result as input
def process_sql_result(sql_result):
    # Example processing function
    return f"Processed SQL result: {sql_result}"

# Create a tool for the next chain
@tool
def next_chain_tool(sql_result: str) -> str:
    return process_sql_result(sql_result)

# Create an agent executor to manage the sequence
tools = [next_chain_tool]
agent = create_tool_calling_agent(llm_az, tools, final_prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Invoke the next chain with the SQL result
next_chain_result = agent_executor.invoke({"sql_result": sql_result})
print(next_chain_result)

In this setup, the sql_chain produces an output that is then used as input for next_chain_tool, which processes the SQL result. The AgentExecutor manages the sequence, ensuring that the output from one chain is correctly formatted and passed to the next ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[quartermaine]

@dosu

I want to use the sql_chain in the following chain:

chain = ( sql_chain | RunnableLambda(lambda x: add_attribute(x, 'command', get_query_memory())) # Add query memory to the result | RunnableLambda(lambda x: add_attribute(x, 'response_metadata', filter_token_usage(x))) # Filter and add token usage metadata ).with_types(input_type=InputType, output_type=OutputType)

[dosubot[bot]]

To integrate sql_chain into another chain and add attributes like 'command' and 'response_metadata' using RunnableLambda, you can structure your chain as follows:

chain = (
    sql_chain
    | RunnableLambda(lambda x: add_attribute(x, 'command', get_query_memory()))  # Add query memory to the result
    | RunnableLambda(lambda x: add_attribute(x, 'response_metadata', filter_token_usage(x)))  # Filter and add token usage metadata
).with_types(input_type=InputType, output_type=OutputType)

Ensure that add_attribute is a function that correctly modifies the input dictionary by adding the specified attribute. The with_types method is used to specify the input and output types, which helps maintain the integrity of the chain by ensuring the data types are correctly handled ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[quartermaine]

@dosu

I am getting the following error in the sql_chain

raise ValueError(msg) ValueError: Invalid input type <class 'dict'>. Must be a PromptValue, str, or list of BaseMessages.

[dosubot[bot]]

To resolve the ValueError in sql_chain that states "Invalid input type <class 'dict'>. Must be a PromptValue, str, or list of BaseMessages", you need to ensure that the input you are passing to the function is of the correct type. According to the code in llms.py, the valid input types are PromptValue, str, or a list of BaseMessages ^[1].

If you are currently passing a dict, you should convert it to one of the accepted types. For example, if your dict represents a prompt, you might convert it to a str or wrap it in a PromptValue object if such a class is defined elsewhere in your codebase. If the dict represents a sequence of messages, you should convert it to a list of BaseMessages.

Here's a hypothetical example of how you might convert a dict to a str:

# Assuming your dict is something like this
input_dict = {"prompt": "What is the weather today?"}

# Convert it to a string if the dict contains a prompt
input_str = input_dict.get("prompt", "")

# Now pass input_str to the function
result = your_function(input_str)

If you have a list of messages, ensure they are instances of BaseMessages:

# Assuming your dict is something like this
input_dict = {"messages": ["Hello", "How can I help you?"]}

# Convert it to a list of BaseMessages
input_messages = [BaseMessage(text=msg) for msg in input_dict.get("messages", [])]

# Now pass input_messages to the function
result = your_function(input_messages)

Adjust the conversion based on the actual structure and content of your dict ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[quartermaine]

@dosu

my final_prompt looks like:

final_prompt = ChatPromptTemplate.from_messages( [("system", os.environ.get("ANSWER_TEMPLATE", None)), ("human", "{question}")] )

[dosubot[bot]]

To ensure that the final_prompt created using ChatPromptTemplate.from_messages is correctly formatted and avoids the ValueError when used in sql_chain, you need to override the format_messages method in the ChatPromptTemplate class to handle f-strings. Here's how you can do it:

class CustomChatPromptTemplate(ChatPromptTemplate):
    def format_messages(self, **kwargs: Any) -> list[BaseMessage]:
        kwargs = self._merge_partial_and_user_variables(**kwargs)
        result = []
        for message_template in self.messages:
            if isinstance(message_template, BaseMessage):
                formatted_content = message_template.content.format(**kwargs)
                message_template.content = formatted_content
                result.extend([message_template])
            elif isinstance(message_template, (BaseMessagePromptTemplate, BaseChatPromptTemplate)):
                message = message_template.format_messages(**kwargs)
                result.extend(message)
            else:
                msg = f"Unexpected input: {message_template}"
                raise ValueError(msg)
        return result

generate_final_prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            os.environ.get("ANSWER_TEMPLATE", "You are a helpful assistant"),
        ),
        (
            "human",
            (
                """Use the following results retrieved from a database to provide
a succinct, definitive answer to the user's question.

Respond as if you are answering the question directly.

Results: {results}
Question: {question}"""
            ),
        ),
    ]
)

generate_final_chain = generate_final_prompt | llm | StrOutputParser()

def generate_final_answer(state: OverallState) -> OutputState:
    final_answer = generate_final_chain.invoke(
        {"question": state.get("question"), "results": state.get("database_records")}
    )
    return {"answer": final_answer, "steps": ["generate_final_answer"]}

This setup ensures that the final_prompt is formatted correctly with the necessary variables, preventing any ValueError when used in sql_chain ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}