Hiya, I'm an artist, and for the past few years I've been following the progress of AI based art tools. Recently I've come across Stable Diffusion, Midjourney, ArtBreeder, GauGan and other online art generation apps powered by AI.
I'm a noob at coding, having only limited experience with c# and Unity. But I've also been hearing amazing things about how simple and fast Julia language is.
So the artist-nerd in me is asking the code muses out there - anyone creating or crafting a Stable Diffusion port, or equivalent, written in Julia?
If not, then I'd like to make a small contribution to the coding and the art side. Just need to get started with training or creating deep learning models for art: Jupyter notebooks, Pluto and Google Collab I hear may be a good start?
Happy to hear from anyone who'd like to give any suggestions, assistance or advice - this would be much appreciated! Thank you! + Happy Coding!
Oldest comments (3)
In my experience, you're in a great position! There is a clear goal and you seem motivated. The way to make it happen is to dig in and make it happen. It's gonna be tough, but you'll learn tons of stuff along the way. Way more than when you could just take an existing project and apply that in 5 minutes.
More specifically, I'm not an expert on this, but my guess is that you probably want to wrap Julia around the original Python implementation of stable-diffusion as a start, see github.com/IQVIA-ML/LightGBM.jl as an example project which also wraps around another project. There are more such wrapper projects in Julia which can all provide as examples. Also, you need a Nvidea GPU with at least 4 GB of VRAM to be able to work on your local system. Luckily, video cards have become much cheaper lately.
Once the wrapper is working, you can start moving logic from the Python model to Julia. For example, maybe loading of images isn't so easy via Python, so then use Julia for that. Repeat this process until Python is gone or until you decide to rewrite the whole project in Julia.
Most excellent suggestion - thank you!
I'm working on a old macmini at moment and am condsidering Google Colab and Lambda Labs as potential options for training my own models in Julia via Jupyter notebooks which come build in I believe.
I guess the AI training code itself, rather then fetching final results, would be a lot more demanding to port as it requires knowledge of the neural networks.
Will give it a go!
Add to the discussion