TypeError: _flash_attn_forward() missing 1 required positional argument: 'softcap'

[Ziyang412]

Hi, thanks for sharing the awesome work. I am trying to reproduce the codes and when I do the evaluation, I encounter the error below:

The command I am using is:
accelerate launch --num_processes 8 --config_file accelerate_configs/deepspeed_inference.yaml --main_process_port 6000 eval_needle.py \ --model PY007/EasyContext-1M-Llama-2-7B \ --max_context_length 1000000 \ --min_context_length 50000 \ --context_interval 50000 \ --depth_interval 0.1 \ --num_samples 2 \ --rnd_number_digits 7 \ --haystack_dir PaulGrahamEssays

I am wondering whether there are any compatible between some newer version of flash attention and this repo or anything other. I think the only difference I could tell from the environment is that the given version of pytorch is archived, I switched to the version of "2.4.0.dev20240527+cu118".

Thanks a lot!

[sunyoe]

hello, have you fixed this error?
I face the same error with pytorch version of "2.4.0+cu121". Maybe this problem is caused by ring-flash-attn, which installed by the source code of github. And I find out the issue: zhuzilin/ring-flash-attention#44, but no answer either.

[zhuzilin]

I have fixed the bug in ring-flash-attention with zhuzilin/ring-flash-attention#45 and zhuzilin/ring-flash-attention#47. Could you reinstall it and give it another try?