Add support for calling tools in conversations #4
Conversation
|
Sorry for late reply @jhrozek , do you think it would be possible to make this a generic http backend, without the trusty'isms (I guess mainly the struct around trusty's payload and the URL etc). The thinking is over time this might become a generic go to library used by folks outside of stacklok. |
I can make the example more generic, but the parameters of the tool will always be specific to the tool. But at the same time, the back end only needs to know the tool name (to return it back), the tool description (to match the tool in prompts) and the parameters which are a generic |
Adds two helper utilities for converting a structure to map[string]any as well as printing as JSON.
Both OpenAI Ollama have the same API structure for executing tools. This commit adds a tool registry that the library user can use to register a new tool that the model will be able to call.
Adds a Converse method for Ollama. Due to how Ollama models are implemented (ollama/ollama#6127) we try to detect if a non-existing tool was called and just route the prompt to the model again with tools disabled.
Implements conversationw with tool support for OpenAI. The OpenAI models expect an assistant message that includes the details about the function call and the function call ID in the response.
Adds Converse as a back end method, making it usable by library clients.
Adds an example of tool function with pretend weather reports. Example invocations: ``` BACKEND=ollama go run ./examples/tools/main.go "What is the weather in London?" 2024/10/17 22:42:11 No model selected with the MODEL env variable. Defaulting to qwen2.5 2024/10/17 22:42:11 Using Ollama backend: qwen2.5 2024/10/17 22:42:14 Tool called 2024/10/17 22:42:14 Response: 2024/10/17 22:42:14 The current temperature in London is 15°C and the conditions are rainy. ```
|
Sorry it took me so long to get back! I rebased the patch atop the current master and took the new structures like or with OpenAI: |
|
btw there are no tests now. it's something I'd like to fix if we agree about the code direction |
This PR adds support for calling tools in conversations along with a couple of
utilities to register and drive the tools and an example calling out to
trusty.
The code is not super polished, as an example it always only evaluates the first function
call or doesn't really check the stop reasons well. But it's a working PoC of what we could do.
What is this good for?
Calling a function enables the model to compute realtime information.
How does calling out to tools differ from RAG?
RAG is more useful for data that doesn't change that much and that we can prepare beforehand.
In the context of Minder, we might want to store ruletypes and profiles into a vector database
to provide them to the model as RAG, but we'd want to call a function to get up-to-date information
about an entity.
CLI examples
There is a small CLI app that calls to trusty and gives the user summary of information
about a package the user queries.
To run the app with OpenAI:
or with Ollama: