Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EnzymeCore weakdep and an extension with a custom rule for the Levin transformation #97

Merged
merged 17 commits into from
May 18, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions ext/BesselsEnzymeCoreExt.jl
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,17 @@ module BesselsEnzymeCoreExt
Duplicated(ls, dls)
end

# This is fixing a straight bug in Enzyme.
function EnzymeRules.forward(func::Const{typeof(sinpi)},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any way we can get this fixed in enzyme itself?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this should use sincospi

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I poked over at EnzymeAD/Enzyme.jl#443
I don’t think I know exactly how to solve that though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it turns out this is not the only problem. Something else in the generic power series is broken for Enzyme but not ForwardDiff. Leaving a summary comment below, one moment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you have time could post a specific example where this is broken? I will try to figure out what line is causing the issue even when separating out the sinpi.

These issues though are especially annoying......

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ironically sincospi using Enzyme should be fine. I'm adding a pr for sinpi/cospi now which hopefully will be available in a few days.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, though I'm solving in a different way from this PR (internal to Enzyme proper rather than Enzyme.jl custom rule), rules like this are welcome as PR's to Enzyme.jl

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect! Thanks for looking at this. I'll change it over here once that is available.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It occurs to me to mention here for people looking at this PR in the future that the problem was just the sinpi, but that I didn't understand how to properly write EnzymeRules and just needed to propagate the x.dval in the derivative part of the returned Duplicated object. I didn't include tests for power series accuracy here because that will probably be a little bit of a project to get the last few digits, but once I fixed my custom rule that worked fine.

@wsmoses, would you like me to make a PR with this rule in the meantime? If it is fixed and will be available in the next release, maybe not point unless you would make a more immediate release that has the custom rule. I'd be happy to try and make that PR if you want, but I understand if it isn't the most useful.

Sorry that this thread looks on a skim like Enzyme problems but was actually "Chris doesn't know how to write custom rules" problems.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See EnzymeAD/Enzyme#1216. Hopefully that fixes this issue here and we can remove that part in the future.

P.s. I have the general besselk working now locally so hopefully we can get that merged soon and can test the general derivative cases.

::Type{<:Duplicated},
x::Duplicated)
Duplicated(sinpi(x.val), pi*cospi(x.val))
end

function EnzymeRules.forward(func::Const{typeof(sinpi)},
::Type{<:Const},
x::Const)
sinpi(x.val)
end

end
13 changes: 7 additions & 6 deletions src/BesselFunctions/besselk.jl
Original file line number Diff line number Diff line change
Expand Up @@ -654,16 +654,17 @@ function besselk_power_series_temme_basal(v::V, x::Float64) where{V}
end

function besselk_power_series_int(v, x::Float64)
v < zero(v) && return besselk_power_series_int(-v, x)
flv = Int(floor(v))
_v = v - flv
v = abs(v)
(_v, flv) = modf(v)
if _v > 1/2
(_v, flv) = (_v-one(_v), flv+1)
end
(kv, kvp1) = besselk_power_series_temme_basal(_v, x)
abs(v) < 1/2 && return kv
twodx = 2/x
for _ in 1:(flv-1)
for _ in 1:flv
_v += 1
(kv, kvp1) = (kvp1, muladd(twodx*_v, kvp1, kv))
end
kvp1
kv
end

2 changes: 1 addition & 1 deletion src/Math/Math.jl
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,6 @@ end
end

# TODO (cg 2023/05/16 18:09): dispute this cutoff.
cgeoga marked this conversation as resolved.
Show resolved Hide resolved
isnearint(x) = abs(x-round(x)) < 1e-7
isnearint(x) = abs(x-round(x)) < 1e-5

end
38 changes: 38 additions & 0 deletions test/besselk_enzyme_test.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
using EnzymeCore, Enzyme
import Bessels.BesselFunctions: besselkx_levin
import Bessels.BesselFunctions: besselk_power_series

dbesselkx_dv(v, x) = autodiff(Forward, _v->besselkx_levin(_v, x, Val(30)),
Duplicated, Duplicated(v, 1.0))[2]

dbesselkx_dx(v, x) = autodiff(Forward, _x->besselkx_levin(v, _x, Val(30)),
Duplicated, Duplicated(x, 1.0))[2]

#=
dbesselk_ps_dv(v, x) = autodiff(Forward, _v->besselk_power_series(_v, x),
Duplicated, Duplicated(v, 1.0))[2]

dbesselk_ps_dx(v, x) = autodiff(Forward, _x->besselk_power_series(v, _x),
Duplicated, Duplicated(x, 1.0))[2]
=#


for line in eachline("data/besselk/enzyme/besselkx_levin_enzyme_tests.csv")
(v, x, dv, dx) = parse.(Float64, split(line))
test_dv = dbesselkx_dv(v, x)
test_dx = dbesselkx_dx(v, x)
@test isapprox(dv, test_dv, rtol=5e-14)
@test isapprox(dx, test_dx, rtol=5e-14)
end

#=
for line in eachline("data/besselk/enzyme/besselk_power_series_enzyme_tests.csv")
(v, x, dv, dx) = parse.(Float64, split(line))
test_dv = dbesselk_ps_dv(v, x)
test_dx = dbesselk_ps_dx(v, x)
#@test isapprox(dv, test_dv, rtol=5e-14)
#@test isapprox(dx, test_dx, rtol=5e-14)

end
=#

2 changes: 1 addition & 1 deletion test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ import SpecialFunctions
@time @testset "gamma" begin include("gamma_test.jl") end
@time @testset "airy" begin include("airy_test.jl") end
@time @testset "sphericalbessel" begin include("sphericalbessel_test.jl") end
@time @testset "enzyme autodiff" begin include("enzyme_test.jl") end
@time @testset "besselk enzyme autodiff" begin include("besselk_enzyme_test.jl") end