DearDiary.jl: A lightweight but powerful machine learning experiment tracking tool for Julia

Jose Esparza — Tue, 04 Nov 2025 22:53:25 +0000

After months of planning, and some weeks of development, the package is finally here and ready for use! As a solution for tracking machine learning experiments in Julia, DearDiary.jl aims to be lightweight, easy to use, and flexible enough to adapt to different workflows.

Motivation

After the unpleasant experience trying to maintain an interface for the REST API from Python's MLFlow (MLFlowClient.jl, MLJFlow.jl), after finding out that it is poorly documented, incomplete, and has some abandoned or partially implemented features (and they are still adding new ones...); an idea came to my mind: why not write the same API but well designed and documented but in Julia? This package is that idea.

Core concepts

Architecture-first

Unlike the common monolithic architecture found in many Julia packages, my goal was to implement something that can be easily maintained and extended over time, focusing on developer experience and code readability (inspired by Alan Edelman's TED talk and MLJ.jl "micro-package" architecture).

DearDiary.jl consists of an N-layered architecture, which bring us the possibility of encapsulating different functionalities to allow better collaboration and separation of concerns.
Now it is composed of the following layers:

Repository layer: responsible for data storage and retrieval.
Service layer: handles package logic and data processing.
Route layer: manages RESTful API endpoints and HTTP requests.

And having the idea of implementing a frontend layer in the future.

Simple types

One of the problems I found while working on the integration project was the overuse of complex types. Imagine a type that has a field that is another type with field that is another type that has a field with an integer. Well, that's real and you can find it if you are curious enough.
DearDiary.jl tries to avoid that by keeping types simple and flat, totally immutable, and clear as possible. Never search for complexity when you don't need it.

Flexible by design

DearDiary.jl is flexible enough to adapt to different workflows. You can use it as a standalone package, or integrate it with other tools in your ML pipeline, or call it from the "outside world" via its RESTful API.
In the case something is not implemented in the way you want it, you can always modify or extend it, thanks to its modularity.

Portability

DearDiary.jl is designed to be portable. You can run it locally, on a server, or in the cloud. Thanks to SQLite as the default storage backend, you can easily move your projects between different environments without worrying about compatibility issues.

Note: one of the main goals for next releases is to support more storage backends, coming from SQL and NoSQL databases, or cloud storage solutions.

Getting started

A Tutorial is available in the documentation. It covers installation, and a workflow example with MLJ.jl.

Contributing

Contributions are welcome! If you find a bug or have a feature request, please open an issue on the GitHub repository. Pull requests are also encouraged. Please make sure to follow the existing code style and include tests for any new features.

Adding MLflow tracking to MLJ - GSoC'23

Jose Esparza — Wed, 23 Aug 2023 07:01:07 +0000

Introduction

When I started in the world of Machine Learning, I used other programming languages throughout my learning process and also my professional development. Julia was introduced to me when I was searching for bleeding edge technologies that could be the field's future and contribute to the state of the art. In a short time, this programming language became my first line source of learning and the project I wanted to see it being applied across my country (Peru). So with hands on, I started practicing to implement some old projects but using the awesome features coming from Julia. But that wasn't enough; I knew about Google Summer of Code since I was in my college second year and I always wanted to be part of the program but I didn't considered myself as "eligible" (my imposter syndrome). Well, that was until this year that I decided to apply to an interesting project that I was using: MLJ.

Unravelling the depths

Choosing the project

When I started planning getting into the program, I didn't know exactly which project I will work on. I sent some emails to Anthony Blaom, the lead contributor of MLJ and also my mentor; showing my interest about participating in the program and also be a project contributor. He always answered all my concerns, and sent me a list of projects that I could take. My interest was divided by two projects: a survival analysis model implementation and a MLOps platform integration. My not advanced experience with statistics made me decide to work in the project related with MLFlow, a well-known machine learning model tracking platform. After Anthony suggestion, I sent some emails to the other mentor behind this project, Deyan Dyankov, to delve into the details of MLFlow.

The project

MLJ didn't have a way to log the most important workflow information in a simpler way, just like other programming languages that have special libraries to work with tracking platforms. The main goal was to adapt the MLJ components behind the training process to log all the required information. To achieve this objective, my mentors prepared me to understand the functionality behind each project that will allow the development of our interface: MLJBase, MLJModelInterface, and MLFlowClient.

Making MLFlowClient up to date

Deyan mentioned that MLFlowClient was ensured to work with versions 1.21.0 and 1.22.0 of mlflow. However, some endpoints from the REST API changed their responses over time and this broke certain core functionalities.
My first step in this project was about converting the server timestamps to UTC time. This error was particular because it was not completely related to the library, but with mlflow internal functionality (the UI platform is showing the client side time). It was good to know that with this change Deyan considered my GSoC application and sent a email to Anthony to continue with the selection process.
A lot of work was done implementing new features coming from new versions of mlflow, but I consider that the most important change in this phase was the complete project refactoring. This update changed the entire project architecture structure and improved the test suite in a way that new developers can start collaborating with the project easily.
After that, I revisited this project just to implement utilities that will be used by other projects (like the health check).

Different approaches with MLJBase

Just after finishing with MLFlowClient, I continued learning about MLJBase: the core package behind the entire MLJ ecosystem. Anthony helped me a lot to understand the deep functionality behind everything on this project. I'm completely thankful about his patience, because some concepts were completely new for me (like multiple dispatch).

The package extensions approach

Our first approach was to implement all the changes related to integrating MLFlowClient using package extensions. This bleeding edge feature was introduced in Julia 1.9, inheriting the functionality implemented by a package named Requires. In simple terms, this allows us to include external dependencies without including them... that sounds weird, right? Well, it's very simple.

Figure 1: A package extensions example

With this approach, we can define functions that use heavy dependencies without including them when we first instantiate the project, incrementing the importing speed a lot!
However, even if it seems like a great way to implement the project, MLFlowClient wasn't the heavy project that will make MLJ slower, and also we disagreed with the idea that the user will need always to import that library. So was it completely necessarily? I don't think so.

An interface: a classical approach

After our concerns, Anthony suggested to create a new package that joins all the MLJ components and MLFlowClient... what a great idea! This allowed different things that package extensions make difficult:

A clean and classical way to implement things
Removing the extra step of importing the package to the user (enhancing user experience!)
Sharing utilities across the extended functions (avoiding duplication)

We started to design how this is going to work, documenting our ideas and taking a weekly meeting to discuss about the next project steps.

Figure 2: The project design (well, until now)

With this in mind, the first task was moving the entire implementation made in the previous approach to the new package. This included changes in MLJBase.jl and MLJModelInterface.jl to allow our new logger type to be passed throughout the MLJ workflow. Next, we polished that code to met the project requirements, and improving it.

The project

MLJFlow is a package that extends the MLJ capabilities to use MLFlow as a backend for model tracking and experiment management. Other programming languages have official API implementation, but Julia don't. This project helps it to achieve that. An example with a simple workflow is shown below:

using MLJ

# Instancing the logger
logger = MLFlowLogger(
    "http://127.0.0.1:5000";
    experiment_name="MLJFlow test",
    artifact_location="./mlj-test"
)

# Get the data and choose our favorite model
X, y = make_moons(100) # a table and a vector with 100 rows
DecisionTreeClassifier = @load DecisionTreeClassifier pkg=DecisionTree
model = DecisionTreeClassifier(max_depth=4)

# Train it!
evaluate(model, X, y,
    resampling=CV(nfolds=5),
    measures=[LogLoss(), Accuracy()],
    logger=logger
)

The information must be logged in a new run in your MLFlow instance under the MLJFlow test experiment name. You didn't need to define anything. That's awesome!

You can see more detailed information in this readme file.

What is left to do?

The project is perfectly working with singular models and composite ones. These contain simple non-special implementations in the functions that our project is using. However, the next cases are still in development:

MLJ's TunedModel workflow integration
MLJ's IteratedModel workflow integration

Achievements and acknowledgments

For me, the most important achievement was learning about how to contribute to the open source community. I'm feeling grateful for that because it was a new adventure for my professional development. It's a fact that this opportunity is going to improve the way I think about how to implement a project. As a contributor from now on, I am quite enthusiastic to be able to cooperate with the MLJ project, to the point of being able to apply it within my professional and academic circle not only as a user, but also as an ambassador.

I'm completely grateful with my mentors, Antony and Deyan, who had the patience to help me with my questions; giving me a piece of their time. I'll never forget it.

Julia Community 🟣: Jose Esparza

DearDiary.jl: A lightweight but powerful machine learning experiment tracking tool for Julia

Motivation

Core concepts

Architecture-first

Simple types

Flexible by design

Portability

Getting started

Contributing

Adding MLflow tracking to MLJ - GSoC'23

Introduction

Unravelling the depths

Choosing the project

The project

Making MLFlowClient up to date

Different approaches with MLJBase

The package extensions approach

An interface: a classical approach

The project

What is left to do?

Achievements and acknowledgments

Useful links