Deepseek_lora custom keys in input_data fails

[Zhaoyi-Yan]

System Info

For some models, eg, deepseek, we need to manually make the specified target_modules like : #2128 . It reminds us a link:

peft/src/peft/utils/constants.py

Line 87 in e6f927b

TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING = {

. For Qwen2, which exists in this link, supports custom keys in input_data and qwen2_lora works fine. However, for deepseek, deepseek_lora fails. The error is:

[rank0]: File "/zju_0038/yzy/miniconda3/envs/torch2.3_py3.10/lib/python3.10/site-packages/peft/tuners/tuners_utils.py", line 197, in forward [rank0]: return self.model.forward(*args, **kwargs) [rank0]: TypeError: DeepseekV2ForCausalLM.forward() got an unexpected keyword argument 'data_index'

However, for qwen_lora, it seems that it does not trigger this function, or maybe it is due to the DeepseekV2ForCausalLM does not hold custom keys?

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

Expected behavior

[BenjaminBossan]

I could not fully understand the issue. What I get is that for Qwen, we have default target_modules but we don't have them for Deepseek. This is correct, you need to specify the target modules yourself (you can try target_modules="all-linear" as a start).

Regarding the error you get, you would need to provide some minimal code so that I can try to reproduce the error, otherwise I can't help you.

[TVR28]

Adding to @BenjaminBossan's answer, the typeerror occurs because the DeepseekV2ForCausalLM model's forward method doesn't accept a data_index parameter.

Remove the data_index from your input kwargs when calling the model. If you need to pass custom data, only use the parameters supported by the model's forward method.

[BenjaminBossan]

Thanks for the additional insight @TVR28.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.