Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix perplexity logging #622

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

fix perplexity logging #622

wants to merge 7 commits into from

Conversation

sichu2023
Copy link
Collaborator

@sichu2023 sichu2023 commented Jan 17, 2025

Replace callback method in logging perplexity with torchmetrics.
W B Chart 1_17_2025, 1_46_04 PM
W B Chart 1_17_2025, 1_46_09 PM

Loss curve is shifted by a constant when TP=2 but gradient is not affected and the perplexity value match with the loss curve without TP. This is a previously unidentified problem and will be resolved by another PR.
W B Chart 1_17_2025, 1_46_13 PM
W B Chart 1_17_2025, 1_46_17 PM
W B Chart 1_17_2025, 1_46_22 PM

@codecov-commenter
Copy link

codecov-commenter commented Jan 17, 2025

Codecov Report

Attention: Patch coverage is 90.62500% with 3 lines in your changes missing coverage. Please review.

Project coverage is 86.57%. Comparing base (0c990a7) to head (0bd0330).

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...-packages/bionemo-llm/src/bionemo/llm/lightning.py 88.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #622      +/-   ##
==========================================
- Coverage   86.62%   86.57%   -0.05%     
==========================================
  Files         116      116              
  Lines        6961     6929      -32     
==========================================
- Hits         6030     5999      -31     
+ Misses        931      930       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sichu2023 sichu2023 force-pushed the sichu/torchmetric-ppl branch from ea78996 to 0bd0330 Compare January 18, 2025 03:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants