I always wanted to do this, I think Julia is a good language to develop audio plugins and virtual instruments. But its currently missing real time tooling and binding to audio plugin frameworks such as CLAP JUCE VST3
There is a way to compile Julia to webassembly: https://tshort.github.io/WebAssemblyCompiler.jl/stable/examp... , it might be possible for you to run the julia script clientside. I don't know whether FFTW.jl can run though for the spectrogram plot
Another related demo is https://youtu.be/bujv4mxG7mE?si=LMGri56Gzs7mWTN_ , they solve differential equations of a saxophone
One of the issues with Julia for this kind of thing is that it’s tuned for throughput more than latency, especially the sort of worst-case latency you worry about for realtime systems. You have to be careful to make sure any methods that are called ahead of time so they’re not JIT-compiled in your audio loop. It’s also hard to write zero-allocation code, which is what you need if you don’t want the GC pausing your program at an inopportune time.
These things are all much improved recently, but you are correct not there yet. ‘juliac’ is a new static compilation option which should release around 1.12, and there are others like Cognibotics experimenting with Julia for hard real time applications.
Julia was also recently used by WhatsApp for audio codec R&D. Hopefully these use cases will drive better guarantees and tooling.
That's awesome. Looking forward to the code!
git clone https://lee-phillips.org/pluckit/code
Thank you!
What does Julia have to do with this?
> The page communicates over a websocket with a Julia program running on a server. That program does the calculations, generates the plots and the sound, and sends them back for insertion in the page. No files are created. The part of the Julia program handling the communication amounts to just a few lines, calling functions from the HTTP.jl library. We don’t need any web frameworks or boilerplate. I’ll supply the source shortly; check back here or subscribe to my feed for updates.