Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix qwen-vl failed with FSDP #30

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix qwen-vl failed with FSDP #30

wants to merge 4 commits into from

Conversation

Yancey1989
Copy link

@Yancey1989 Yancey1989 commented Nov 5, 2024 •
edited
Loading

this PR search attribute from the original module if the attribute is missing in the FSDP wrapper module.

@Yancey1989 Yancey1989 requested a review from baoleai November 5, 2024 05:52
@anw90 anw90 requested a review from yitongh November 5, 2024 06:18
@Yancey1989 Yancey1989 requested a review from eedalong November 5, 2024 12:58
yitongh
yitongh reviewed Nov 6, 2024
View reviewed changes
torchacc/dist/distributed_parallel.py Outdated Show resolved Hide resolved
torchacc/dist/distributed_parallel.py Outdated Show resolved Hide resolved
yitongh
yitongh reviewed Nov 11, 2024
View reviewed changes
Copy link
Contributor

@yitongh yitongh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also change the self.model to self._module in fsdp.py, spmd_fsdp.py, pp/pipeline.py, dp.py

@Yancey1989
Copy link
Author

Should we also change the self.model to self._module in fsdp.py, spmd_fsdp.py, pp/pipeline.py, dp.py

@yitongh good idea, I will do that in the next PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yitongh yitongh yitongh approved these changes

@baoleai baoleai Awaiting requested review from baoleai

@eedalong eedalong Awaiting requested review from eedalong

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Yancey1989 @yitongh