Faster than ArgParse, but there still is room for improvement

[davidavdav]

Hello,

Thanks for making this package. It is really a lot better than ArgParse. It is a big frustration for me that I can't sensibly use Julia as a scripting language for small tasks—maybe that was never intended—because of the very slow start up /command line parsing.

If there are very few parameters to be parsed, ArgMacros performs pretty well. But it seems that with more arguments parsing time increases linearly.

I don't really understand why command line parsing has to happen in macros. Is that faster than if it were a function? Or is the problem that you want a type-stable Args struct? Is it not possible to have parameters in a sort-of json object, just supporting a few basic type with which we can cover most parameter styles. Or maybe command line parsing can be done at the same level that the julia startup parameters are parsed, this should be fast, right?

I've learned python after I has started to work in Julia, but I think the python argparse approach leaves very little to wish for.

[zachmatson]

I first wrote ArgMacros to supplement a messy perl script that I was translating to Julia, which involved a lot of arguments coming in as local variables. For this purpose, getting all of the arguments to local variables and having those variables typed was my goal, and I wanted this process to happen faster than what I could get with ArgParse (which has of course caused lots of headaches for anyone writing scripts in Julia). I went with a pretty direct model where the argument macros translate to local variable assignments which depend on fairly simple parsing functions with some setup code and pre-processing in the main arguments block because this worked well for me in that use case, and the rest of the package has pretty much grown from that approach "just working" well enough.

However, as I believed (perhaps naively) that building up the argument data structure was the source of speed issues, and that local variable use which avoided this was the solution, I set up each argument macro to search through all of the arguments to try to find its value, then remove that value from the vector of arguments. This means that performance is mainly linear with regards to the number of arguments you are looking for.

I'm thinking about trying a rewrite of the parsing backend for this package which will modify the approach, but I haven't had time yet. Since it's Julia scripting and we're going for the lowest latency, it's hard to weigh the tradeoffs in compile time/latency vs parsing latency. There are a few other approaches I'm considering, but if they make the simple cases slower I'm not sure whether that's something most users right now want.

As for macros, they are necessary for a type stable struct or for the local typed argument approaches that are available with ArgMacros, I don't know that they're strictly necessary for good performance. As I said earlier in this post, too, they are also used here to make each variable partly responsible for its own parsing.

These days I don't write Julia scripts much either, but I'm still interested in improving this project for people who do. Do you have any test cases with more arguments that you would be interested in providing to help profile newer versions?

[davidavdav]

I was writing a reply with some timing measurements when I found out that the slow start-up time I was experiencing (~10s) was due to the fact that the rest of the code took long compiling (200 lines in total), every single time I did a time script.jl --help.

So, just for myself and future reference, it makes sense in a Julia script to define the arguments before using, importing or includeing any other code, e.g., like this:

#!/usr/bin/env julia

using ArgMacros
@structarguments true Args begin
    @argumentdefault Float64 15.0 snr "--snr"
    @argumentdefault Float64 0.0 snrsd "--snrsd"
    @argumentrequired String scp "--scp"
    @argumentrequired String outdir "--outdir"
    @argumentdefault String "out.scp" outscp "--outscp"
    @positionalrequired String noisefile "Noise file to be added"
end
args = Args()

import WAV
import Random

include("noise.jl")

...