Render Quarto Tutorials in Documenter.jl with GitHub Actions

Introduction

I am a large fan of tutorials to provide users an easy start for example when using a new Julia package. For best of cases, a tutorial contains code that can not only be run by the user, but that is also

• run in the example to illustrate which result to expect
• included in the documentation and matches in style
• consistent – that is does not suddenly break

These points brought me to quarto, since it is easy to write, in markdown + Julia Code, get‘s rendered into Markdown, which can be stored in the documentation folder easily to match the general style. One can even reference other functions in the documentation using just @ref from Documenter.
My favourite feature of Quarto is that, since it is just Julia Code evaluated, it allowed to use local environments for consistency. For a package, I prefer to change one small thing from pure consistency, that is, load always the master branch of the package the documentation / tutorial is for. This way, one can easily check, that also the tutorials are not breaking.

Since this workflow means, that some markdown files of the documentation, it would be great to have the examples automatically run on GitHub CI when generating the documentation.
Of course, since some examples might take a bit to run, caching them would also be great. Luckily, Quarto already has a _freeze mechanism.

Structure of this tutorial

After a short explanation of the setup we use in Manopt.jl, this tutorial explains how to set up you Documentation CI to also include Quarto examples to be run, if they have been changed.

Setup

In the Manopt.jl structure, the tutorials share a common environment in tutorials/, with their own Project.toml.

Each of the tutorials should be developed locally and tested with quarto render (or quarto preview).

This workflow was developed with Julia 1.9 and Quarto 1.3

Handling Python Dependencies

A main challenge of rendering quarto markdown files is, that this is done with Jupyter and hence requires python as a dependency.

So there is a necessity to handle python dependencies when/before running quarto. Since this happens within the documentation, the environment Project.toml for the docs/ folder takes care of this employing CondaPkg.jl, see also the CondaPkg.toml file, which is quite minimal.

Together with IJulia.jl just calling

CondaPkg.withenv() do
    [...]
end

allows to execute command line calls with the specified python packages (here only Jupyter) installed and having a Julia kernel available

Documentation: Modifying the make.jl

In order to make the tutorials only render when specified, the make.jl is executable and has as a first line

#!/usr/bin/env Julia

This way a normal run of building the documentation, for example running include("make.jl") on REPL would not trigger running all tutorials, but on terminal doing

./make.jl --quarto

does. The idea is that either quarto render or the command line call would only be executed from time to time and usually the .md files from the tutorials are already rendered and available in the documentation source tree.

Besides that, the make.jl contains a block that, given --quarto, renders all .qmd files.

To be safe, this block not only runs in the CondaPkg environment, but also resolves/instantiates the tutorial folder for once, but also makes sure, the new Jupyter (if not from cache) is aware of the IJulia kernel by once building that package in the tutorials environment:

CondaPkg.withenv() do
    @info "Rendering Quarto"
    tutorials_folder = (@__DIR__) * "/../tutorials"
    # instantiate the tutorials environment if necessary
    Pkg.activate(tutorials_folder)
    Pkg.resolve()
    Pkg.instantiate()
    Pkg.build("IJulia") # build IJulia to the right version.
    Pkg.activate(@__DIR__) # but return to the docs one before
    run(`quarto render $(tutorials_folder)`)
end

Since the quarto config is set up to store the results in the documentation docs/src directory, they are rendered when running make.jl into html. Here, the quarto command is also run within the environment having the python packages installed, as discussed in the last section.

CI: Modifying the GitHub Action

The second part of this tutorial is concerned with the documenter.yml workflow, which is modified to also allow caching of the results.

We first install Quarto with the corresponding action

- uses: quarto-dev/quarto-actions/setup@v2
  with:
    version: 1.3.353

But the main part is, to cache quite a few folders between CI runs. This is done by Hashing the corresponding folder and allowing to load old caches from this folder as well.
We cache

Julia

Which is possible with its own cache

- name: Julia Cache
  uses: julia-actions/cache@v1

Quarto

Since quarto stores pre-rendered results in the _freeze subfolder, we cache this folder. Note that the restored keys allow to also load previous versions, where the current hash does not hit, that way, if some tutorial is changed, the unchanged ones still obtain their old cached versions

- name: Cache Quarto
  id: cache-quarto
  uses: actions/cache@v3
  env:
    cache-name: cache-quarto
  with:
    path: tutorials/_freeze
    key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('tutorials/*.qmd') }}
    restore-keys: |
      ${{ runner.os }}-${{ env.cache-name }}-

Documenter

Since we also want to cache the old .md files that quarto stores within the documenter tree, we also cache that folder

- name: Cache Documenter
  id: cache-documenter
  uses: actions/cache@v3
  env:
    cache-name: cache-documenter
  with:
    path: docs/src/tutorials
    key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('tutorials/*.qmd') }}
    restore-keys: |
      ${{ runner.os }}-${{ env.cache-name }}-

CondaPkg

It might be useful to also cache the downloaded artefacts from CondaPkg

- name: Cache CondaPkg
  id: cache-condaPkg
  uses: actions/cache@v3
  env:
    cache-name: cache-condapkg
  with:
    path: docs/.CondaPkg
    key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('docs/CondaPkg.toml') }}
    restore-keys: |
      ${{ runner.os }}-${{ env.cache-name }}-

Finally we modify the call to run the documentation rendering such that it always runs the rendering. This will only include changed tutorials, since the others are cached due to all the caches above

- name: "Documenter rendering (including Quarto)"
  run: "docs/make.jl --quarto"
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }}

Summary

While a bit technical with all the caches, this modification of the documentation build run and the documentation GitHub Action allows to automatically build quarto notebooks within the documentation. This way there is no need to commit the (pre-)rendered markdown files in the documentation, which might have led to them being outdated. Since all the caches avoid running the tutorials on every run, the GitHub CI run is still reasonably fast.

The main files to look at are in summary

• the tutorials/_quarto.yml to configure Quarto, mainly to store the result directly in the docs/src/tutorials/ folder
• the docs/make.jl file to render the documentation including running the quarto rendering, but especially also the python dependencies.
• the .github/workflows/documenter.yml especially with a bit of caches.

Open Problems

Two tutorials are currently still rendered locally, see the _quarto.yml, one since it requires Asymptote for rendering, which is not so easy to get on CI, and the other one since benchmarking might be a bit too slow on CI.

Since the .md of the tutorials are not committed to the GitHub repository, but only cached/regenerated on CI before running Documenter.jl, Edit on GitHub link is broken for these. It would be nice to add a link here to the .qmd file in the tutorials/ folder instead.