How To Speak LLM - An Adventure from Tokens to Agents
0. Settings
OpenAI API Key
1. Next token
Input
Hint
2. Next token in a loop
Input
Hint
3. Chat
Thread
User message
Hint
4. Chain of Thought
Thread
Below is a transcript of user interacting with an Assistant. The assistant responded with its internal monologue thinking and final response in one turn. In this format """ [User] ... [Assistant] thought: ... thought: ... ... final response: ... """
User message
Hint
5. Tools
Thread
Below is a transcript of user interacting with an Assistant. The Assistant thinks step by step. In each turn, the assistant responds in one of the following formats: Format 1: think and respond Use this format when you need to ask user a question or chat with user without using tools. """ [Assistant] thought: <make plans on how to respond to the user> response: <the actual response to the user> """ Format 2: think, use tool, observe, and respond Make sure you have obtained the correct info from the user with Format 1. """ [Assistant] thought: <make plans on how to respond to the user> tool_use: <example_tool("example_args")> tool_output: <{ result: "example_output" }> observation: <observation based the tool output> response: <the actual response to the user> """ In format 2, The assistant only has access to the following tools: lookup_product_info("<keywords>") - returns product info. lookup_shipping_info("<order number obtained from the user>") - returns the info about shipping. submit_order("<product_id>") - submits an order for the product with the given id. The full transcript will alternate between [User] and [Assistant] like this: """ [User] ... [Assistant] ... [User] ... [Assistant] ... """ Here is the full step-by-step transcript: """
User message
Hint
Tool name
Tool input
Tool output
6. Tools automated
Thread
Below is a transcript of user interacting with an Assistant. The Assistant thinks step by step. In each turn, the assistant responds in one of the following formats: Format 1: think and respond Use this format when you need to ask user a question or chat with user without using tools. """ [Assistant] thought: <make plans on how to respond to the user> response: <the actual response to the user> """ Format 2: think, use tool, observe, and respond Make sure you have obtained the correct info from the user with Format 1. """ [Assistant] thought: <make plans on how to respond to the user> tool_use: <example_tool("example_args")> tool_output: <{ result: "example_output" }> observation: <observation based the tool output> response: <the actual response to the user> """ In format 2, The assistant only has access to the following tools: lookup_product_info("<keywords>") - returns product info. lookup_shipping_info("<order number obtained from the user>") - returns the info about shipping. submit_order("<product_id>") - submits an order for the product with the given id. The full transcript will alternate between [User] and [Assistant] like this: """ [User] ... [Assistant] ... [User] ... [Assistant] ... """ Here is the full step-by-step transcript: """
User message
Hint
Tool name
Tool input
Tool output