[Obs AI Assistant] More detailed request & token tracking

[dgieselaar]

Currently, we have the following events:

• observability_ai_assistant_user_sent_prompt_in_chat
• observability_ai_assistant_insight_response

That indicate that a user "does something". We track this with EBT. However, for capacity planning, we need to be able to predict usage at a lower level: LLM requests and token counts.

We can add the following metrics to these events:

• input_tokens
• output_tokens
• llm_requests

We can also separately track LLM requests (in order to have a baseline, regardless of which event they belong to), with the following metrics:

• input_tokens
• output_tokens
• response_time

Additionally, we can also track tool calls:

• tool_name
• input_tokens
• output_tokens
• response_time
• is_error

[elasticmachine]

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)