feat(context-agent): move Deep Cody out of model dropdown

[abeatrix]

Adds on #6513
CLOSE https://linear.app/sourcegraph/issue/CODY-4535 https://linear.app/sourcegraph/issue/CODY-4563 https://linear.app/sourcegraph/issue/CODY-4539

Building blocks of https://www.loom.com/share/16b55f92368844b4b766083ed4d30091

This PR introduces a new UI-based approach for managing Cody's agent capabilities, replacing the previous VS Code settings configuration. It also moves Deep Cody out of the model dropdown into its own Agent Settings component

Key Changes

1. New Agent UI Controls:

   • Added ToolboxButton component in chat input area with:
   • Bot icon indicating agent status (active/inactive)
   • Green pulse animation for active state
   • Popover menu for settings
   • Dedicated switches for agent and terminal features

2. Deep Cody Architecture Improvements:

   • Simplified agent ID for Deep Cody to 'deep-cody' from previous ModelRef format
   • Added support for model selection between default chat model and Claude 3.5 Haiku
   • Moved shell context permission to UI toggle with warning message
   • Enhanced context handling with deduplication for repeated queries

3. API & Type Updates:

   • Added toolboxSettings and updateToolboxSettings to WebView API
   • Updated AgentToolboxSettings type to use string-based agent ID
   • Enhanced message types to support agent attribution
   • Added feature flag for chat model selection

4. Code Organization:

   • Refactored CodyToolProvider into namespace pattern
   • Improved tool factory initialization and management
   • Enhanced test coverage for new UI components
   • Removed deprecated VS Code settings

5. Default Model

   • Defaulted to use 3.5 Haiku as the reflection model
   • When the ContextAgentDefaultChatModel feature flag is enabled on the instance, reflection model will be set to Sonnet instead

6. Merge CodyChatAgent with DeepCodyAgent.

7. Introduce ToolboxManager and AgentToolboxSettings

   • Added a ToolboxManager class to centralize the management of agent toolbox settings, including the terminal/shell context.
   • Introduced the AgentToolboxSettings interface to represent the user's preferences for the agent and terminal context.
   • Integrated the ToolboxManager into the ChatController to update the user's terminal context setting based on the configuration.
   • Persisted the agent and terminal context settings in the local storage for user-specific preferences.
   • ToolboxManager settings for terminal context gets updated on user config changes

8. Improve prompt and response format for context review

   • Enhance the review prompt to provide clearer instructions and the expected response format.
   • Add new tags for context and answers to standardize the response structure.
   • Provide examples of valid and invalid responses to guide the user.
   • Update the prompt to emphasize the importance of only including the expected tags in the response.
   • Integrate the ToolboxManager into the ChatController to update the user's terminal context setting based on the configuration.
   • Persist the agent and terminal context settings in the local storage for user-specific preferences.

9. Improve context retrieval for deep-cody / context agent

   • send context to webview for display before sending the request
   • update context retrieval to only include validated context items that match the end of the path
   • post empty message in progress when processing steps are updated

TO-DO

Will be continued in follow-ups:

• Update Deep Cody Notice / upsell with the new changes
• Remove Deep Cody rate limit
• Update Agentic context UI
• Update New toolbox setting UI

Test plan

Updated all existing unit tests and added new unit tests to cover all the Deep Cody components.

Deep Cody should work as it does currently:

Example Question: how many files are there in the root of my codebase? Find where PromptSelectField is defined in the codebase

In UI, enabling the Terminal Context Agent would enable terminal context

In UI, disabling the Terminal Context Agent would disable terminal context

1. Agent Controls:

   • Verify bot icon appears in chat input
   • Check icon states (active/inactive)
   • Confirm green pulse animation when active
   • Test settings popover interaction

2. Settings Management:

   • Enable/disable agent via toggle
   • Toggle terminal access (when available)
   • Verify warning message for terminal access
   • Confirm settings persist across sessions

3. Chat Experience:

   • Test chat with agent enabled/disabled
   • Verify model selection works correctly
   • Check context gathering improvements
   • Confirm terminal integration (when enabled)

4. Settings Transition:

   • Verify existing setting is not respected and is marked as deprecated
   • The Terminal context is disabled by default in the new UI component
   • Test default states as PLG and Enterprise users
   • Check settings sync behavior

Changelog

[github-actions]

‼️ Hey @sourcegraph/cody-security, please review this PR carefully as it introduces the usage of an unsafe_ function or abuses PromptString.

[valerybugakov]

Went through half of it. Will complete my review tomorrow morning!

[valerybugakov]

A few notes from the interaction I tested:

1. The context row in the UI shows "none fetched," but there are two items with green check marks under the collapsible context section.
2. The green check marks suggest that context was successfully retrieved, but the output channel logs show errors when attempting to use the SearchTool.
3. And the obvious: the output doesn’t match what’s shown in the screenshot from the PR description. Cody couldn’t use the tools, and as a user, I don’t understand why that happened.

outputChannelId:"16472d3f-0a82-4a46-9f8f-dbbdd5bd2e34": {
  "completion": "<TOOLFILE><name>.</name></TOOLFILE><TOOLSEARCH><query>promptselectfield</query></TOOLSEARCH>"
}
█ CodyTool requesting 1 files
█ SearchTool searching codebase for promptselectfield
█ getContextFileFromUri failed: {
  "name": "CodeExpectedError",
  "message": "cannot open file:///Users/val/Desktop/sourcegraph-root/cody-chat-eval. Detail: Unable to read file '/Users/val/Desktop/sourcegraph-root/cody-chat-eval' (Error: Unable to read file '/Users/val/Desktop/sourcegraph-root/cody-chat-eval' that is actually a directory)"
}

[valerybugakov]

Checkmarks on the context items in the light theme have an odd shadow that seems out of place:

[valerybugakov]

Is it intentional that the agent icon is noticeably lighter than the "Open in Editor" icon?

[abeatrix]

Thanks for testing this out!

@valerybugakov are you in the cody report when you run the question? Do you have any file opened in your editor? (Our search only works if you have file from the codebase opened in your editor, not an issue with deep cody since deep cody is just reusing our search).

For the second issue where file fetch failed - Since you don't have terminal setting enabled it fallback to use file search and search for all (".") so that's why it failed (will work on integrating the error next). Right now, the icon on the left only show you that it run the tools but it doesn't tell you how many files it fetch, which is what the none fetch is for (we will move from this UI soon).

[valerybugakov]

This PR is pretty big, with many independent parts being updated simultaneously, which makes it much harder to review, especially for engineers who aren't working full-time on Deep Cody. Based on the sections in the PR description, it could be split into around nine smaller, independent PRs that we can merge and roll back separately if needed.

Sharing one of my earlier comments on this topic for more context on why this approach is helpful: #6066 (review).

It might seem like a lot of extra work, but by stacking PRs and iterating quickly on reviews, we can merge small chunks of functionality much faster while reducing the risk of breaking the release candidate. Let's give that a try for the upcoming changes!

[valerybugakov]

Thank you for addressing all the comments, @abeatrix! You rock! 🤘

[valerybugakov]

Yes, that's perfect! We have a couple of helpers in the codebase to do that. Search for dependentAbortController or forkSignal.

[abeatrix]

replaced it with forkSignal. Thank you so much!