cpu服务器，通过curl调用接口推理，报"NoneType' object has no attribute 'size'

[lizhao-8202]

System Info / 系統信息

cuda:无，CPU版本
python:3.10.4
操作系统:Centos7.9
transformers:4.47.1,xinference 1.1.0

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

1.1.0

The command used to start Xinference / 用以启动 xinference 的命令

xinference xinference-local --host 0.0.0.0 --port 8080

Reproduction / 复现过程

1部署模型

xinference launch --model-engine Transformers -u my-qwen-chat-pytorch-7b -n qwen-chat -s 7 -f pytorch -e ``` http://127.0.0.1:8080 
2推理
```curl -X 'POST'   'http://127.0.0.1:8080/v1/chat/completions'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -d '{
    "model": "my-qwen-chat-pytorch-7b",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is the largest animal?",
"size": 1
        }
    ]
  }'

报错

2024-12-23 12:48:42,184 xinference.model.llm.transformers.utils 17241 ERROR    Internal error for batch inference: 'NoneType' object has no attribute 'size'.
Traceback (most recent call last):
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/model/llm/transformers/utils.py", line 491, in batch_inference_one_step
    _batch_inference_one_step_internal(
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/model/llm/transformers/utils.py", line 318, in _batch_inference_one_step_internal
    out = model(**inf_kws, use_cache=True, past_key_values=past_key_values)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/models/XINFERENCE_MODEL/openmind_hub/huggingface/modules/transformers_modules/qwen-chat-pytorch-7b/modeling_qwen.py", line 1045, in forward
    transformer_outputs = self.transformer(
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/models/XINFERENCE_MODEL/openmind_hub/huggingface/modules/transformers_modules/qwen-chat-pytorch-7b/modeling_qwen.py", line 893, in forward
    outputs = block(
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/models/XINFERENCE_MODEL/openmind_hub/huggingface/modules/transformers_modules/qwen-chat-pytorch-7b/modeling_qwen.py", line 612, in forward
    attn_outputs = self.attn(
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/models/XINFERENCE_MODEL/openmind_hub/huggingface/modules/transformers_modules/qwen-chat-pytorch-7b/modeling_qwen.py", line 524, in forward
    -1, -1, causal_mask.size(2), -1
AttributeError: 'NoneType' object has no attribute 'size'
2024-12-23 12:48:42,188 xinference.core.model 17241 ERROR    [request 2e59c184-c0e9-11ef-befc-5d5c2c3f1a45] Leave chat, error: 'NoneType' object has no attribute 'size', elapsed time: 1 s
Traceback (most recent call last):
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/utils.py", line 90, in wrapped
    ret = await func(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/model.py", line 740, in chat
    return await self.handle_batching_request(
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/model.py", line 723, in handle_batching_request
    result = await fut
ValueError: 'NoneType' object has no attribute 'size'
2024-12-23 12:48:42,191 xinference.api.restful_api 17053 ERROR    [address=0.0.0.0:39917, pid=17241] 'NoneType' object has no attribute 'size'
Traceback (most recent call last):
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/api/restful_api.py", line 2098, in create_chat_completion
    data = await model.chat(
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/context.py", line 231, in send
    return self._process_result_message(result)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/pool.py", line 667, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
    return await coro
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/model.py", line 103, in wrapped_func
    ret = await fn(self, *args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xoscar/api.py", line 462, in _wrapper
    r = await func(self, *args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/utils.py", line 90, in wrapped
    ret = await func(*args, **kwargs)
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/model.py", line 740, in chat
    return await self.handle_batching_request(
  File "/root/anaconda3/envs/xinference/lib/python3.10/site-packages/xinference/core/model.py", line 723, in handle_batching_request
    result = await fut
ValueError: [address=0.0.0.0:39917, pid=17241] 'NoneType' object has no attribute 'size'

Expected behavior / 期待表现

能得到答案

[qinxuye]

qwen-chat 太老的模型了，建议用 qwen2.5-instruct。

[github-actions]

This issue is stale because it has been open for 7 days with no activity.