Support References in Tool Calling Schemas

[GICodeWarrior]

Feature request

Currently, refs are not supported in tool calling schemas.

It appears that text-generation-inference is dropping reference definitions before passing the schema to outlines-core to generate a regex.

text-generation-inference/router/src/infer/tool_grammar.rs

Lines 63 to 107 in 23bc38b

	let functions: HashMap<String, serde_json::Value> = tools_to_use
	.iter()
	.map(|tool| {
	let func = tool.function.clone();
	let mut params = Map::new();
	params.insert(
	"description".to_string(),
	Value::String(func.description.unwrap_or_default()),
	);
	let mut properties = Map::new();
	let mut required = vec![Value::String("_name".to_string())];
	properties.insert(
	"_name".to_string(),
	json!({
	"type": "string",
	"const": func.name.clone(),
	}),
	);
	if let Value::Object(args) = func.arguments {
	if let Some(Value::Object(props)) = args.get("properties") {
	properties.extend(props.clone());
	}
	if let Some(Value::Array(reqs)) = args.get("required") {
	required.extend(reqs.clone());
	}
	params.insert(
	"additionalProperties".to_string(),
	Value::Bool(
	args.get("additionalProperties").and_then(|v| v.as_str())
	== Some("true"),
	),
	);
	}
	params.insert("properties".to_string(), Value::Object(properties));
	params.insert("required".to_string(), Value::Array(required));
	(func.name, Value::Object(params))
	})
	.collect();

outlines-core does support JSON schemas with references.

Example API call with references:

% curl -X POST \
  https://api-inference.huggingface.co/models/meta-llama/Llama-3.3-70B-Instruct/v1/chat/completions \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{
        "messages": [{"role": "user", "content": "Call the function Response with some example inputs."}],
        "tools": [
          {
            "type": "function",
            "function": {
              "name": "Response",
              "parameters": {
                "$defs": {
                  "Source": {
                    "properties": {
                      "url": {
                        "type": "string"
                      }
                    },
                    "type": "object"
                  }
                },
                "properties": {
                  "sources": {
                    "items": {
                      "$ref": "#/$defs/Source"
                    },
                    "type": "array"
                  }
                },
                "type": "object"
              }
            }
          }
        ]
      }'
{"error":"Input validation error: cannot compile regex from schema: Invalid reference path: $defs","error_type":"validation"}

Motivation

This feature is useful for calling complex functions.

Some other providers support schemas like this (e.g. OpenAI, Anthropic, MistralAI, etc.).

Your contribution

I can provide more examples and testing if helpful.