error CUDADevice not defined

[CarloLucibello]

Hi,
is this error known/expected?

julia> using Tullio, CUDA

julia> CUDA.allowscalar(false)

julia> A, B, C = CUDA.rand(5,321,482), CUDA.rand(5,321,482), CUDA.rand(5,482);

julia> @tullio C[i,j] := A[i,k,j] * B[i,k,j]
ERROR: scalar getindex is disallowed
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] assertscalar(op::String)
   @ GPUArrays ~/.julia/packages/GPUArrays/WV76E/src/host/indexing.jl:41
 [3] getindex
   @ ~/.julia/packages/GPUArrays/WV76E/src/host/indexing.jl:96 [inlined]
 [4] 𝒜𝒸𝓉!
   @ ~/.julia/packages/Tullio/PhxRJ/src/macro.jl:1208 [inlined]
 [5] 𝒜𝒸𝓉!
   @ ~/.julia/packages/Tullio/PhxRJ/src/macro.jl:1038 [inlined]
 [6] threader(fun!::var"#𝒜𝒸𝓉!#5", ::Type{CuArray{Float32, N} where N}, Z::CuArray{Float32, 2}, As::Tuple{CuArray{Float32, 3}, CuArray{Float32, 3}}, Is::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, Js::Tuple{Base.OneTo{Int64}}, redfun::Function, block::Int64, keep::Nothing)
   @ Tullio ~/.julia/packages/Tullio/PhxRJ/src/eval.jl:90
 [7] (::var"#ℳ𝒶𝓀ℯ#8"{var"#𝒜𝒸𝓉!#5"})(A::CuArray{Float32, 3}, B::CuArray{Float32, 3})
   @ Main ~/.julia/packages/Tullio/PhxRJ/src/macro.jl:803
 [8] (::Tullio.Eval{var"#ℳ𝒶𝓀ℯ#8"{var"#𝒜𝒸𝓉!#5"}, var"#13#∇ℳ𝒶𝓀ℯ#7"{var"#∇𝒜𝒸𝓉!#6"}})(::CuArray{Float32, 3}, ::Vararg{CuArray{Float32, 3}, N} where N)
   @ Tullio ~/.julia/packages/Tullio/PhxRJ/src/eval.jl:20
 [9] top-level scope
   @ ~/.julia/packages/Tullio/PhxRJ/src/macro.jl:973

Test on

julia 1.6
Tullio 0.2.13
CUDA 2.6.2
KenelAbstractions 0.6.0

[mcabbott]

I think this should work. CI has KernelAbstractions = "0.5.2", maybe 0.6 needs some change?

Edit -- CI with 0.6 fails, but with "UndefVarError: CUDADevice not defined" rather than scalar indexing: https://buildkite.com/julialang/tullio-dot-jl/builds/87#81ab0814-e2dc-4c38-876b-07ef2afc15d7

[CarloLucibello]

I see "CUDADevice not defined" as well in a longer code from which I extracted the mwe in OP. Don't know why I get these two different behaviours

[CarloLucibello]

I observe the scalar indexing error also on KernelAbstractions@v0.5.2

[mcabbott]

One question is whether you are sure you have loaded KernelAbstractions? Like using Tullio, CUDA, KernelAbstractions, the macro needs to see the module before it generates the right code.

If it does, this is the core of what it generates, which is the next step:

julia> @macroexpand @tullio C[i,j] := A[i,k,j] * B[i,k,j] verbose=2 grad=false
┌ Info: =====KA===== KernelAbstractions kernel 
│   verbosetidy(kex1) =
│    quote
│        KernelAbstractions.@kernel function var"##🇨🇺#257"(ℛ::AbstractArray{𝒯}, A, B, 𝒶𝓍i, 𝒶𝓍j, 𝒶𝓍k, ♻️, 💀) where 𝒯
│                (i, j) = @index(Global, NTuple)
│                begin
│                    𝒜𝒸𝒸 = if ♻️ === nothing
│                            zero(𝒯)
│                        else
│                            ℛ[i, j]
│                        end
│                    for k = 𝒶𝓍k
│                        𝒜𝒸𝒸 = 𝒜𝒸𝒸 + A[i, k, j] * B[i, k, j]
│                    end
│                    ℛ[i, j] = 𝒜𝒸𝒸
│                end
│            end
└    end
┌ Info: =====KA===== KernelAbstractions CUDA actor 
│   verbosetidy(kex2) =
│    quote
│        local @inline(function 𝒜𝒸𝓉!(::Type{<:CuArray}, ℛ::AbstractArray{𝒯}, A, B, 𝒶𝓍i, 𝒶𝓍j, 𝒶𝓍k, ♻️ = nothing, 💀 = true) where 𝒯
│                    @info "running KernelAbstractions + CUDA actor " maxlog = 3 _id = 0x90b345dc80f99c72
│                    cu_kern! = var"##🇨🇺#257"(CUDADevice(), 256)
│                    (first)(𝒶𝓍i) == 1 || (throw)("KernelAbstractions can't handle OffsetArrays here")
│                    (first)(𝒶𝓍j) == 1 || (throw)("KernelAbstractions can't handle OffsetArrays here")
│                    𝒜𝒸𝒸 = cu_kern!(ℛ::AbstractArray{𝒯}, A, B, 𝒶𝓍i, 𝒶𝓍j, 𝒶𝓍k, ♻️, 💀; ndrange = tuple(length(𝒶𝓍i), length(𝒶𝓍j)), dependencies = Event(CUDADevice()))
│                    KernelAbstractions.wait(CUDADevice(), 𝒜𝒸𝒸)
│                end)
└    end

[CarloLucibello]

Ah! I forgot using KernelAbstractions, sorry about that, now I get the CUDAdevice error:

julia> using Tullio, CUDA, KernelAbstractions

julia> A, B, C = CUDA.rand(5,321,482), CUDA.rand(5,321,482), CUDA.rand(5,482);

julia> @tullio C[i,j] = A[i,k,j] * B[i,k,j]
ERROR: UndefVarError: CUDADevice not defined
Stacktrace:
 [1] 𝒜𝒸𝓉!
   @ ~/.julia/packages/Tullio/PhxRJ/src/macro.jl:1167 [inlined]
 [2] 𝒜𝒸𝓉!
   @ ~/.julia/packages/Tullio/PhxRJ/src/macro.jl:1166 [inlined]
 [3] threader(fun!::var"#𝒜𝒸𝓉!#5", ::Type{CuArray{Float32, N} where N}, Z::CuArray{Float32, 2}, As::Tuple{CuArray{Float32, 3}, CuArray{Float32, 3}}, Is::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, Js::Tuple{Base.OneTo{Int64}}, redfun::Function, block::Int64, keep::Nothing)
   @ Tullio ~/.julia/packages/Tullio/PhxRJ/src/eval.jl:90
 [4] top-level scope
   @ ~/.julia/packages/Tullio/PhxRJ/src/macro.jl:1001

[mcabbott]

It's possible you now need using KernelAbstractions, CUDAKernels, CUDA, after this change JuliaGPU/KernelAbstractions.jl@7fab1ea which is part of KA 0.6.

That seems to make CI happy: https://buildkite.com/julialang/tullio-dot-jl/builds/89#bbe05b24-4b59-46ed-b67e-ecca729a9300/194-311

[CarloLucibello]

makes sense, thanks!

[RomeoV]

I'm having this problem again. I think CUDAKernels moved to the CUDA.jl package (?), and now using CUDA.CUDAKernels doesn't solve the issue, and explicitly installing add CUDAKernels forces a downgrade on many packages.

[mcabbott]

I think that's #168 now, but I haven't got to trying to fix it, sorry.