Transcript
0:00 Effectively, like what happens? You ask a model to generate a tool call. So, a model is basically just generating like token sequence after token sequence. Um and like it just generates one token at a time until it does this and then like obviously it has a sequence of input tokens that came before it. So, like these are input tokens, these are output tokens. When it decides that it wants to invoke a tool call, it says some English tokens, then eventually it outputs a very special token that's like the tool call token. It says, "I'm going to initiate a tool call."
0:28 And it usually after that it outputs some more tokens that's like, "Here's the name of the tool call." Um and it'll say like the name of the tool call, and then I'll start outputting the day data. And once it outputs the tool call name, what Anthropic or OpenAI or any of these companies can do is they can now say something like, "From this point onwards, you must you you can only abide by proper JSON. So, if you're outputting an array, the first curly it has to be a curly, it's not a choice, and it continuously goes onward."
0:58 >> [music]
Summary
- Language models generate output one token at a time based on prior input tokens.
- A special token signifies the start of a tool call.
- After initiating a tool call, the model outputs the name of the tool and relevant data.
- Companies like Anthropic and OpenAI enforce strict formatting rules (e.g., JSON) after a tool call is initiated.
- The model's output must adhere to specified structures, such as arrays and objects, following the tool call token.