Update embeddings docs to support 5.3 vs 5.2 (and older) versions

[taylorsperry]

I've created a document for us to collaborate on the necessary changes to update our documentation around embeddings: https://docs.google.com/document/d/1mrKd8Oh1pd-sI0oJi3jh90yNhklkRYdasVoDjigNaFk/edit

[taylorsperry]

@toolmantim and @dominiccooney would love your input here as well, please!

[dominiccooney]

High-level product, feature or concept breakdown

Context—the way Cody fetches codebase snippets to answer questions in chat—has changed/is changing. Here's a summary of the changes:

• Enterprises no longer generate embeddings in 5.3. Embeddings will be disabled, eventually, in 5.2.
• Enterprises can now select multiple repos to query, like Cody Web could do.
• Consumers continue to have locally stored embeddings. However sourcegraph.com no longer provides a remote embeddings index (referred to here). As a result, if they are working in an open source repository, they can index it themselves.

Note, we continue to experiment with/invest in embeddings in different parts of Cody. So we should not give the impression that we don't use, or won't use, embeddings. We just need to remove references that are out of date to avoid confusing people.

Docs expectations

• Remove mentions of embeddings in the Enterprise context
• Document the ability to add multiple repositories in chats, using the Enhanced Context Selector in chat. (Enterprise only, for now.)

Docs placement

Links which need content updates:

• https://docs.sourcegraph.com/cody/overview/install-vscode (no Enterprise embeddings; choose repos for Enterprise users)
• https://docs.sourcegraph.com/cody/overview/install-jetbrains (context selection UI has changed)
• https://docs.sourcegraph.com/cody/core-concepts/context/context-selection (don't refer to keyword search as a fallback)
• https://docs.sourcegraph.com/cody/core-concepts/keyword-search (remove Enterprise embeddings mention; don't refer to keyword search as a fallback)
• https://docs.sourcegraph.com/cody/core-concepts/embeddings/embedding-index (remove Enterprise-specific content: admin tasks; incremental embeddings; time interval; third-party embeddings providers; etc.)
• https://docs.sourcegraph.com/cody/usage-and-pricing (remove "embeddings on open-source code via Sourcegraph.com" which is going away; Enterprise plans should lose Embeddings row)
• https://docs.sourcegraph.com/cody/troubleshooting#codebase-is-not-indexed (Codebase indexing would no longer be an "embedding job" etc.)
• https://docs.sourcegraph.com/cody/faq#embeddings (Embeddings section can be simplified; we should include the possibility that the product uses embeddings (consumer does; Enterprise probably will again at some point) but the details like running and stopping jobs can go.)

Audience

• Enterprise users, consumer users
• Enterprise admins need updated instructions to remove embeddings instructions, and add instructions for indexing repositories for getCodyContext when a repository is not indexed. (Which would show up as, a user opens the Choose Repositories box and the repository is not there.)

Brain dump (Initial docs draft)

See this issue for PRD-level details and this issue

FAQs

💡Following are the frequently asked questions that we would appreciate as a part of this information-gathering phase. Please fill out the applicable details or any additional information we should know.

• Access to the feature for testing: See this doc.
• Any type of retention period (if applies): NA
• Feature availability: Mostly applies to Enterprise
• Impact on billing (if applies): NA
• Release stage (public beta, GA, private beta, etc.): 5.3/Feb Enterprise GA