Continue Fix this Code / Optimize this Code does not work with vLLM

[lukehinds]

Describe the issue

I believe this is an upstream issue and not much we can do, but tracking anyhow.

continuedev/continue#3021

When using vLLM, none of features mentioned in the title work, instead a failure is seen.

Steps to Reproduce

Use a vLLM provider in codegate and attempt to use Fix / Optimize in continue

Operating System

MacOS (Arm)

IDE and Version

Version: 1.96.2 (Universal)

Extension and Version

0.9.245

Provider

vLLM

Model

Qwen/Qwen2.5-Coder-14B-Instruct

Logs

No response

Additional Context

No response

[aponcedeleonch]

Just FYI, the workaround mentioned in this issue continuedev/continue#2388 worked the last time I checked. Basically is changing the provider to openai instead of vllm.

{
  "title": "Qwen2.5-Coder-7b-Instruct",
  "provider": "openai",
  "model": "Qwen2.5-Coder-7B-Instruct",
  "apiBase": "<some_api>"
}