Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

qwen2-vl的视觉模型支持sglang框架启动 #2718

Open
948024326 opened this issue Dec 29, 2024 · 3 comments
Open

qwen2-vl的视觉模型支持sglang框架启动 #2718

948024326 opened this issue Dec 29, 2024 · 3 comments
Assignees
@amumu96
Labels
feature
Milestone
v1.x

Comments

@948024326
Copy link

Feature request / 功能建议

qwen2-vl的视觉模型支持sglang框架启动

Motivation / 动机

sglang已经支持了qwen2-vl系列模型，希望xinfer也可以支持，现在qwen2-vl没有sglang的选项，然后我自己注册一个模型选的qwen2-vl的family注册后，也是只有trans和vllm两种框架
f898463c04917537bbb172ec2629103

Your contribution / 您的贡献

谢谢
@XprobeBot XprobeBot added this to the v1.x milestone Dec 29, 2024
@948024326
Copy link
Author

sglang官方表示支持

https://sgl-project.github.io/references/supported_models.html

fd9f9a6cb759e8531ba2051b83aeded

@qinxuye
Copy link
Contributor

qinxuye commented Jan 3, 2025

@amumu96 帮忙看下对 sglang 增加 vl 模型支持。

@948024326
Copy link
Author

@amumu96 帮忙看下对 sglang 增加 vl 模型支持。

我看了下，现在是可以自己注册然后用sglang启动。但可以把这个sglang加到内置启动的选项框中

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amumu96 amumu96

Labels
feature
Projects
None yet
Milestone
v1.x
Development

No branches or pull requests
4 participants
@qinxuye @948024326 @XprobeBot @amumu96