New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

拓展长上下文的技术是？ #41

Open

zzhdbw opened this issue Jul 22, 2024 · 2 comments

zzhdbw commented Jul 22, 2024

实际上本项目的源码比yarn的源码更简洁明了，但我没有在其中看到有关yarn算法的代码，而直观感觉上本项目更像是使用了线性插值，请问本项目使用的是YaRN还是线性插值呢？

期待您的回复，感谢您的解惑。

Owner

jzhang38 commented Jul 22, 2024

Simply increase the rope theta

Author

zzhdbw commented Jul 22, 2024

收到，简单而又有效的方法！感谢您的回复。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment