Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use 'int64' instead of non-portable 'long' #123

Merged
merged 6 commits into from
Jan 16, 2025

Conversation

k-dominik
Copy link
Contributor

@k-dominik k-dominik commented Jan 13, 2025

Unfortunately 'long' does not mean the same thing in different osses/compilers. On windows long will only be 32 bit (int32). Setting width explicitly is more portable.

The default dtype for in in numpy on windows is int32, but for pytorch int64... anyway. Cross-entropy loss will fail with 'long' on windows.

Bonus: Added CI to test win, macos ❤️

unfortunately 'long' does not mean the same thing in different osses/
compilers. On windows long will only be 32 bit (int32). Setting width
explicitly is more portable.
@wolny
Copy link
Owner

wolny commented Jan 14, 2025

Thx @k-dominik, du bist klasse!

Copy link
Owner

@wolny wolny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

almost there! we just need to fix the macos and windows runners :)

@@ -22,7 +25,7 @@ jobs:
with:
auto-activate-base: true
activate-environment: ""
channels: local,conda-forge,defaults
channels: local,conda-forge
Copy link
Owner

@wolny wolny Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we need to add miniconda-version: "latest" to this action to make it work on macos

@@ -10,8 +10,11 @@ on:

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we need to replace RELEASE_VERSION=$(date +%Y%m%d%H%M%S) by something that runs on windows-latest runner

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch - also - it looks like the versioning scheme has recently been changed to calver, but no release has been done yet. I would strongly argue against date-based versioning. Meaningful pinning of this package for people that use it as a library is basically impossible with calver. Also it does not allow you to reason about versions in any meaningful way.

Of course this is also not automatically given with semver, and I have been fooled by relying on it as well. But most of the time I find it more useful than not. Only for end-user softwares (ubuntu) I can see a benefit in using calver. So... While orthogonal to this, would you be willing to reconsider?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

completely agree! It should be semver everywhere

@k-dominik
Copy link
Contributor Author

I've added a commit reverting back to the original versioning scheme, also reading the metadata from setup.py -> no need for the environment variable shenanigans. As it stands this repo would have produced packages and code with versions out of sync (conda package calver, code would rely on init.py...)

Copy link
Owner

@wolny wolny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great! kudos for making the versioning consistent 🚀

@@ -1,6 +1,8 @@
{% set setup_py_data = load_setup_py_data() %}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice one!

@@ -10,8 +10,11 @@ on:

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

completely agree! It should be semver everywhere

miniconda pulls in defaults channel which most people like to avoid for
licensing reasons. Also sets channel priority to strict and removes
multiple channel source definitions for more reproducible environments.
@k-dominik
Copy link
Contributor Author

k-dominik commented Jan 15, 2025

I can fix the windows test errors on all platforms (some dependency related, and special sauce on win) later - should be easy but need to do something else.

@k-dominik
Copy link
Contributor Author

k-dominik commented Jan 15, 2025

okay weird, the numpy<2 bin should not be necessary (I'm investigating...) - could you trigger the pipeline in any case to run please @wolny

on win there are permission issues if delete_on_close is false
strict solution of env currently not possible on linux
@k-dominik k-dominik force-pushed the fix-win-compat-long branch from 4fdd701 to ddcd2a3 Compare January 15, 2025 15:32
@k-dominik
Copy link
Contributor Author

Okay, removed the numpy<2 pin as I could at least track down the environment solving problem to the nvidia channel. I am pretty sure I had an issue open somewhere in the pytorch org but it seems that the respective repo got privated... Not sure exactly which library is shadowing which, but strict solving only works without the nvidia channel.

@k-dominik
Copy link
Contributor Author

this is good to go imo.

@wolny
Copy link
Owner

wolny commented Jan 16, 2025

amazing that you've made the build work on all 3 OSes! Thanks again for this PR 🚀

@wolny wolny merged commit dcd4dec into wolny:master Jan 16, 2025
3 checks passed
@k-dominik k-dominik deleted the fix-win-compat-long branch January 16, 2025 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants