A year of using Quarto with Julia

Earlier this year in July, I gave a short Experience Talk at JuliaCon. In a related blog post I explained how the introduction of Quarto made my transition from R to Julia painless: I would be able to start learning Julia without having to give up on all the benefits associated with R Markdown.

In November, 2022, I am presenting on this topic again at the 2nd JuliaLang Eindhoven meetup. In addition to the slides, I thought I’d share a small companion blog post that highlights some useful tips and tricks for anyone interested in using Quarto with Julia.

General things

We will start in this section with a few general recommendations.

Setup

I continue to recommend using VSCode for any work with Quarto and Julia. The Quarto docs explain how to get started by installing the necessary Quarto and IJulia extensions. Since most Julia users will regularly want to update their Julia version, I would additionally recommend to add IJulia.jl to your ~/.julia/config/startup.jl file:

# Setup OhMyREPL, Revise and Term
import Pkg
let
    pkgs = ["Revise", "OhMyREPL", "Term", "IJulia"]
    for pkg in pkgs
        if Base.find_package(pkg) === nothing
            Pkg.add(pkg)
        end
    end
end

Additionally, you only need to remember that …

… if you install a new Julia binary […], you must update the IJulia installation […] by running Pkg.build("IJulia")

Source: IJulia docs

I guess this step can also be automated in ~/.julia/config/startup.jl, but haven't tried that yet.

Using .ipynb vs .qmd

I also continue to recommend working with Quarto notebooks as opposed to Jupyter notebooks (files ending in .qmd and .ipynb, respectively). This is partially just based on preference (from R Markdown I'm used to working with .Rmd files), but there is also a good reason to consider using .qmd, even if you're used to working with Jupyter: the code chunks in your Quarto notebook automatically link to the Julia REPL in VSCode. In other words, you can run code chunks in your notebook and then access any variable that you may have created in the REPL. I find this quite useful, cause it allows me to quickly test code. Perhaps there's a good way to do this with Jupyter notebooks as well, but when I last used them I would always have to insert new code cells to test stuff.

Either way switching between Jupyter and Quarto notebooks is straight-forward: quarto convert notebook.qmd will convert any Quarto notebook into a Jupyter notebook and vice versa. One potential benefit of Jupyter notebooks is their connection to Google Colab: it is possible to store Jupyter notebooks on Github and make them available on Colab, allowing users to quickly interact with your code without the need to clone anything. If this is important to you, you can still work with .qmd documents and simply specify keep-ipynb: true in the YAML header.

Dynamic Content

The world and the data that describes it is not static 📈. Why should scientific outputs be?

One of the things I have always really loved about R Markdown was the ability to use inline code: the Knitr engine allows you to call and render any object x that you have created in preceding R chunks like this: r x. This is very powerful, because it enables us to bridge the gap between computations and output. In other words, it allows us to easily produce reproducible and dynamic content.

Until recently I had not been aware that this is also possible for Julia. Consider the following example. The code below depends on remote data that is continuously updated:

using MarketData
snp = yahoo("^GSPC")

using Dates
last_trade_day = timestamp(snp[end])[1]
p_close = values(snp[end,:Close])[1]
last_trade_day_formatted = Dates.format(last_trade_day, "U d, yyyy")

It loads the most recent publicly available data on equity prices from Yahoo finance. In an ideal world, we’d like any updates to these inputs to be reflected in our output. That way you can just re-render the Quarto notebook to get an updated report. To render Julia code inline, we use Markdown.jl like so:

using Markdown
Markdown.parse("""
When the S&P 500 last traded, on $(last_trade_day_formatted), it closed at $(p_close). 
""")

When the S&P 500 last traded, on November 18, 2022, it closed at 3965.340088.

In practice, one would of course set #| echo: false in this case. Whatever content you publish, this approach will keep it up-to-date. This practice of simply re-rendering the source notebook also ensures that any other output remains up-to-date (e.g. Figure 1).

Code Execution

Related to the previous point, I typically define the following execution options in my _quarto.yml or _metadata.yml. The freeze: auto option ensures that documents are only rerendered if the source changes. In cases where code should always be re-executed you whould want to set freeze: false, instead. I set output: false because typically I have a lot of code chunks that don't generate any output that is of immediate interest to readers.

Reproducibility

To ensure that your content can be reproduced easily, it may additionally be helpful to explicitly specify the Julia version you used ( jupyter: julia-1.8) and set up a global or local Julia environments. Inserting the following at the beginning of your Quarto notebook

using Pkg; Pkg.activate("<path>")

ensures that the desired environment that lives in is actually activated and used.

Package Documentation

I have also continued to use Quarto in combination with Documenter.jl to document my Julia packages. This essentially boils down to writing up documentation using interactive .qmd notebooks and then rendering those to .md files as inputs for Documenter.jl. There are a few good reasons for this approach, especially if you're used to working with Quarto anyway:

1. Re-rendering any docs with eval: true provides an additional layer of quality assurance: if any of the code chunks throws an error, you know that your documentation is outdated (perhaps due to an API change). It also offers a straight-forward way to test package functions that produce non-testable (e.g. stochastic) output. In such cases, the use of jldoctest is not always straight-forward (see here).
2. You get some stuff for free, e.g. citation management. Unfortunately, as far as I’m aware there is still no support for cross-referencing.
3. You can use Quarto execution options like execute-dir: project and resources: www/ to globally specify the working directory and a directory for external resources like images.

There are also a few peculiarities to be aware of. To avoid any issues with Documenter.jl, I've found it useful to ensure that the rendered .md files do not contain any raw HTML and to preserve text wrapping:

format: 
  commonmark:
    variant: -raw_html
    wrap: preserve

When working with .qmd files you also need to use a slightly different syntax for admonitions. The following syntax inside the .qmd

| !!! note \"An optional title\"
|     Here is something that you should pay attention to.   

will generate the desired output inside the rendered .md:

!!! note "An optional title"
    Here is something that you should pay attention to.   

Any of my package repos — CounterfactualExplanations.jl, LaplaceRedux.jl, ConformalPrediction.jl — should provide additional colour on this topic.

Quarto for Academic Journal Articles

Quarto supports LaTeX templates/classes, which has helped me with paper submissions in the past (e.g. my pending JuliaCon Proceedings submissions). I’ve found that rticles still has an edge here, but the list of out-of-the-box templates for journal articles is growing. Should I find some time in the future, I will try to add a template for JuliaCon Proceedings. The beauty of this is that it should enable publishers to not only use traditional forms of publication (PDF), but also include more dynamic formats with ease (think distill, but more than that.)

Wrapping up

This short post has provided a bit of an update on using Quarto with Julia. From my own experience so far, things have been getting easier and better (thanks to the amazing work of Quarto dev team). I’m exicted to see things improve even further and still think that Quarto is a revolutionary new tool for scientific publishing. Let’s hope publishers eventually recognise this value 👀.

Originally published at https://www.paltmeyer.com on November 21, 2022.