Julia Community 🟣

Daniel Pinyol
Daniel Pinyol

Posted on

Detecting @test @allocated gotchas

#testing #macros #allocations

Hi,
if you care about performance, you probably end up having a lot of these on your unit tests, to ensure that your functions do not allocate when they don't need to:

@test @allocated testedfunction(...) === 0
Enter fullscreen mode Exit fullscreen mode

So far so good. But, if you need assignments or comparisons, you'll need some brackets.

@test (@allocated var=yourexpression) === 0))
Enter fullscreen mode Exit fullscreen mode

This is where you start thinking on DRY: should you create a macro to enable more expressive tests?

@testnoallocations var=yourexpression

macro testnoallocations(expressions...)
    return esc(
        quote
            @test iszero(@allocated $(expressions...))
        end
    )
end
Enter fullscreen mode Exit fullscreen mode

I confess I'm a encapsulation junkie, so I'll try to justify why I think it's a good idea.

1. Global non-consts 

foo = "foo"
@testset "myset" begin
    @test @allocated startswith(foo, "foo")
end
Enter fullscreen mode Exit fullscreen mode

If your tests are complex or you're a Julia noob, it's not difficult to miss that this test will fail because foo is a non-const global. Why not adding the check within your macro? You just need to recursively get all the symbols, and find those which are global (isdefined(__module__, arg)) but !isconst.


macro testnoallocations(expressions...)
    _failIfNonConstGlobalsInExpressions(__module__, expressions...)
    return esc(....)
end

function _failIfNonConstGlobalsInExpressions(mod::Module, expressions...)
    for e in expressions
        nonConstGlobals = (
            arg for arg in expressionsymbols(e) if isdefined(mod, arg) && !isconst(mod, arg)
        )
        if !isempty(nonConstGlobals)
            error(
                "testnoallocations called with expression containing non const global symbols $(collect(
            nonConstGlobals
        ))",
            )
        end
    end
end

" Return all the symbols that make up an expression (or itself if a symbol is passed)"
function expressionsymbols(e::Union{Expr, Symbol, Number})
    !isa(e, Expr) && return (ex for ex in (e,) if isa(e, Symbol))
    topSymbols = (arg for arg in e.args if isa(arg, Symbol))
    subExpressions = (expressionsymbols(arg) for arg in e.args if isa(arg, Expr))
    return (topSymbols..., Iterators.flatten(subExpressions)...)
end
Enter fullscreen mode Exit fullscreen mode

2. @testsets are not functions

I don't have a MWE for this, but a more difficult gotcha is when your function allocates because you're calling it from a function. Indeed, @testset bodies run from the global scope. You can detect this by exploiting the fact that requesting the current function with nameof(var"#self#") will fail when called from the global scope.

macro testnoallocations(expressions...)
    return esc(
        quote
            !@isCalledFromFunction() &&
                @warn "Since not called from a function @allocated could be imprecise"
            @test (@allocated $(expressions...)) === 0
        end,
    )
end

macro isCalledFromFunction()
    expr = esc(:(
        try
            currentFunctionName = nameof(var"#self#")
            true
        catch
            false
        end
    ))
    return expr
end
Enter fullscreen mode Exit fullscreen mode

3. Running with --compile=min

A trick to run short tests very quickly is running them with --compile=min. Unfortunately, the interpreter may then allocate extra memory.

macro testnoallocations(expressions...)
    if !iscompileenabled()
        @warn "Allocations measures are not precise because executed with --compile=min"
    end
    ...
end
iscompileenabled() = Base.JLOptions().compile_enabled == 1
Enter fullscreen mode Exit fullscreen mode

That's all

I hope the post is also useful to show how powerful are Julia reflection capabilities. You comments are more than welcome, specially if you know similar tricks to ease your testing experience. Happy testing!

Oldest comments (0)