Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fine-tuning documentation #76

Open
johnbradley opened this issue Dec 17, 2024 · 3 comments
Open

Add fine-tuning documentation #76

johnbradley opened this issue Dec 17, 2024 · 3 comments

Comments

@johnbradley
Copy link
Collaborator

Provide documentation on how users could fine-tune BioCLIP.

@johnbradley
Copy link
Collaborator Author

@johnbradley
Copy link
Collaborator Author

johnbradley commented Dec 18, 2024

My understanding of the three example above: These use only the visual part of the BioCLIP model and train SVM/ridge classifier/simpleshot based on the image embeddings. No text embeddings are used.

@samuelstevens
Copy link
Contributor

Yep, that's correct. biobench is a method to evaluate vision-only models like the vision encoder from BioCLIP, or DINOv2, etc.

johnbradley added a commit that referenced this issue Dec 30, 2024
Adds a notebook that trains a SVM that predicts species based on
image embeddings created by BioCLIP. The SVM logic is based on
https://github.com/samuelstevens/biobench by @samuelstevens.

Part of issue #76.
johnbradley added a commit that referenced this issue Dec 30, 2024
Adds a notebook that trains a SVM that predicts species based on
image embeddings created by BioCLIP. The SVM logic in init_svc()
is copied from https://github.com/samuelstevens/biobench by
@samuelstevens.

Part of issue #76.

Co-authored-by: Samuel Stevens <[email protected]>
Co-authored-by: Copilot <[email protected]>
johnbradley added a commit that referenced this issue Jan 7, 2025
Adds a notebook that trains a SVM that predicts species based on
image embeddings created by BioCLIP. The SVM logic in init_svc()
is copied from https://github.com/samuelstevens/biobench by
@samuelstevens.

Part of issue #76.

Co-authored-by: Samuel Stevens <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Hilmar Lapp <[email protected]>
johnbradley added a commit that referenced this issue Jan 10, 2025
Adds a notebook that trains a RidgeClassifier that predicts
species based on image embeddings created by BioCLIP. The
RidgeClassifier logic is simplified from init_svc() in
https://github.com/samuelstevens/biobench by @samuelstevens.

Part of issue #76.

Co-authored-by: Samuel Stevens <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants