(s)tones

tekhnē residency research log of Laura Conant

this research was conducted from the 29-07-2024 to the 06-09-2024 at Q-O2 in Brussels

video is courtesy of Handsans on Youtube.

Disclaimer: this website is not intended to be a thorough documentation nor a tutorial but more an atempt to render my research process visible

This project explores the development of an audio plugin — later on named (s)tones — that reimagines how we engage with sound technologies. Instead of adhering to traditional, often exclusionary approaches in audio software and hardware design, the focus here is on accessibility, playfulness, and creative experimentation.

The plugin serves not just as a sound manipulation tool, but as an interactive environment that invites to explore sound in intuitive and unexpected ways. Prioritizing user experience over technical precision, it encourages a more inclusive and open-ended relationship with sound technology.

Central to the project is the personal journey of learning to code in C++, which informed the development process and helped shape a deeper understanding of Digital Signal Processing (DSP) through the JUCE framework. Sharing insights from this learning experience is part of an ongoing effort to demystify coding and make audio tool creation more approachable.

At its core, this project asks: What happens when we shift the focus of audio technology design toward accessibility and play? By embracing a DIY ethos, it aims to empower users with the knowledge and confidence to engage more critically and creatively with the tools they use.

Play is often seen as something “unserious” or the opposite of being skilled, but that’s exactly what makes it powerful. It breaks down hierarchies and shifts control away from the expert, giving everyone a chance to shape their experience. This idea really resonates with me, especially in music and tech, where there's often pressure to be technically proficient before you’re allowed to participate. (Of course, you can be super skilled and still love to play, those things aren’t mutually exclusive!)

Encouraging play means encouraging users to question norms and create new ways of interacting. It’s about keeping things open, and a bit unpredictable.

As a dedicated pebble collector, the idea of building the user experience around the act of throwing a pebble into water came quite quickly. It’s a playful and satisfying gesture—something simple and familiar. And since sound is made of waves too, the connection just made sense. That moment of interaction became a natural metaphor for how the plugin responds and invites exploration.

Manipulating sound waves with water waves.

Later on, after the residency, I fell on the concept of "small software", in the text Kill the “User”: Musings of a disillusioned technologist on the pastagang blog, which resonated a lot with me.

"in a world delineated by the overreaching ambitions of those building everything apps, and the walled gardens, [...] there is something refreshing about the idea of people making small, meaningful things for one other."

First day

29-07-2024

Creating the Research log

Learning about basic C++ structure which we'll need to use JUCE:

W3School documentation

Starting with JUCE

30-07-2024

Learning more about C++:

Getting started on JUCE:

Beginner JUCE tutorial, making a simple EQ

Managed to build and run a simple GUI with JUCE

Continuing on the EQ

31-07-2024

Slow day

Continuing on the EQ tutorial

Having a peek at:

Composing Interactions, Marije Baalman (book)

Peak filter is working!

Interface

02-08-2024

Thinking about collaborative interfaces

Throwing pebbles into a "pond", the size of the pebbles influence a certain parameter of the sound.

Because of the different sizes of pebbles it will also influence their ripples, also modifying a parameter. The viscosity of the liquid is influencing the damping of the ripples, the higher the density the faster they die out and the slower they move.

The x and y axis are also each a sound parameter.

Several persons can throw pebbles at the same time.

Parameters:

pebble size
pebble weight (size x density)
number of ripples
ripples intersections
ripple wave length
viscosity handling ripples wave length and life

pond schema related rates calculation

What could the vst do?

A big part of my sound pratice is processing and reprocessing sound.

Taking a melody, applying granular synthesis, taking a small chunk of it, aplying an effect etc, recording the outcime etc.

I love also that in the Visco VST you can "morph" two sounds together.

Maybe it explains why I love this technique so much. What I like about sound is the never ending possibilities of morphing sounds, espically in electronic music. I also enjoy the border between the natural and "unatural". I'm very interested in physical modelling synthesis.

I really like VST that have a touch of incertainty to it, that are quite organic.

FM Synthesis is working!

03-08-2024

Declaring the oscillators (carrier and modulator):

juce::dsp::Oscillator<float> osc { [](float x) { return std::sin (x); }};
juce::dsp::Oscillator<float> modulatorOscillator { [](float x) { return std::sin (x); }};

Defining the frequency of the oscillators:

lastFreq = *treeState.getRawParameterValue("frequency");
osc.setFrequency(*treeState.getRawParameterValue("frequency") + fmDepth);
modulatorOscillator.setFrequency(*treeState.getRawParameterValue("modFreq"));

Audio can come in multiple channels (like stereo audio which has two channels: left and right). This part of the code makes sure that the processing happens for each channel separately.

Digital audio is made up of many tiny pieces called samples. This loop looks at each sample one by one within the current channel to apply the effect. (the number of samples is dependent on the sample rate, the higher the sample rate, the higher the sound quality).

for (int ch = 0; ch < audioBlock.getNumChannels(); ++ch)
    {
        for (int s = 0; s < audioBlock.getNumSamples(); ++s)
        {
            // Update the modulation signal
            float fmMod = modulatorOscillator.processSample(audioBlock.getSample(ch, s)) * fmDepth;
            // Apply modulation to the oscillator frequency
            float modulatedFrequency = lastFreq + fmMod;
            osc.setFrequency(modulatedFrequency);
        }
    }

Sample rate examples:

sample rate schema

Sometimes it crashes because the calculation of the modulatedFrequency would give a negative value. To avoid this we need to do what we call "wrapping the value around zero":

 float modulatedFrequency = lastFreq + fmMod;
            if (modulatedFrequency < 0.0f)
             {
                modulatedFrequency = 0.0f;
             }

or:

float baseFrequency = 10.0f;

float modulatedFrequency = lastFreq + fmMod;
    while (modulatedFrequency < 0.0f)
    {
        modulatedFrequency += baseFrequency;
    }

Imagine a circle, and you start at the top. If you go all the way around the circle, you've moved 360 degrees and are back at the top again. If if you keep going, 370 degrees would be the same as 10 degrees because you’ve wrapped around the circle and ended up near the top again.

In FM synthesis, something similar happens with sound waves. Instead of degrees, we’re talking about "phase" which tells us where we are in the wave's cycle. As you keep adding or subtracting to this phase, it can go past the normal limits (like going beyond 360 degrees).

"Wrapping around zero" means that if the phase goes too far in one direction, it wraps back around to the beginning, just like going past 360 degrees on a circle brings you back to the start.

Whether you reach 370 degrees or just 10 degrees, the system treats them the same because it’s all about where you are on that circle (or in the wave cycle). This wrapping ensures that the sound wave stays smooth and continuous, avoiding any sudden jumps or glitches.

Added quantization to the value of the modulated frequency based on the quantization funtion I used prior to this:

    float modulatedFrequency = *treeState.getRawParameterValue("frequency") + fmMod;
    while (modulatedFrequency < 0.0f)
        modulatedFrequency += baseFrequency;

    float quantizedModulatedFreq = quantizeFrequency(modulatedFrequency);
    osc.setFrequency(quantizedModulatedFreq);

Ideas for ways of functionning

05-08-2024

Morning: stress freeze, can't think

How would the vst work:

axis wave length intersections circle overlapping operators and modulators

Wavespeed: Speed(m/s)=Frequency(Hz)×Wavelength(m), or v=fλ

First step towards the GUI

06-08-2024

Morning: Watched a lot of videos explaining Euclidian rythms, Polyrythms, Modular Synthesis to see we're I'm going function wise.

Afternoon: Starting to see how to make a GUI with parameters that interest me: The Main component Animating geometry tutorial

Managed to display circles on click each disppearing after 3s:

Coming from mainly coding with javascript for interactive elements, it was interesting to see how to actually calculate time passing by within the software.

If javascript offers you the very handy setTimeout(), for example:

// Say "hi" after 1s

const myTimeout = setTimeout(sayHi, 1000);

function sayHi(){
  console.log("hi");
}

Small victories

07-08-2024

Bumped my head on using the time to update the radius of each circles independently, based on their creation time:

Finding intersections of circles

08-08-2024

Calculating interection points between circles:

Math course about circles intersections

Another one (scroll to "Intersection of two circles")

Final calculations:

If intersections:

c1 = first circle, with center coordinates (c1x, c1y)
c2 = second circle, with center coordinates (c2x, c2y)
(both circles here have the same radius)

// Calculate the distance between the centers
        float distance = std::sqrt(std::pow(c2x - c1x, 2) + std::pow(c2y - c1y, 2));

// Midpoint between the centers
        float mid_x = (c1x + c2x) / 2.0f + radius;
        float mid_y = (c1y + c2y) / 2.0f + radius;

// The intersection points lie on a line perpendicular to the line segment connecting the centers      

// Distance from the midpoint of the line segment connecting the centers of the two circles to the intersection points of the circles
        float h = std::sqrt(std::pow(radius, 2) - std::pow(distance / 2.0f, 2));

// Normalized direction vector from c1 to c2
        float dx = (c2x - c1x) / distance;
        float dy = (c2y - c1y) / distance;

// Intersection points
        float intersection_x1 = mid_x + h * dy;
        float intersection_y1 = mid_y - h * dx;
        float intersection_x2 = mid_x - h * dy;
        float intersection_y2 = mid_y + h * dx;

        g.drawSingleLineText(juce::String(intersection_x1), 2, 20);
        g.drawSingleLineText(juce::String(intersection_y1), 2, 30);
        g.drawSingleLineText(juce::String(intersection_x2), 2, 40);
        g.drawSingleLineText(juce::String(intersection_y2), 2, 50);

        juce::FontOptions font(20.f); // Set font size to 20.0
        g.setFont(font);
        g.drawSingleLineText(juce::String("INTERSECTING"), 2, 80);

// Draw the intersection points
        g.fillEllipse(intersection_x1 - 5, intersection_y1 - 5, 10, 10);
        g.fillEllipse(intersection_x2 - 5, intersection_y2 - 5, 10, 10);

Combining growing radius and intersections, managing performance

09-08-2024

Made a change in the calculations, adding the variable radius resulted in miscalculations in the coords of the intersections points.

Removed:
float distance = std::sqrt(std::pow(c2.x - c1.x, 2) + std::pow(c2.y - c1.y, 2));
float vertical_dist = std::sqrt(std::pow(c1.radius, 2) - std:(distance / 2.0f, 2));

float mid_x = (c1.x + c2.x) / 2.0f;
float mid_y = (c1.y + c2.y) / 2.0f;

float dx = (c2.x - c1.x) / distance;
float dy = (c2.y - c1.y) / distance;

float intersection_x1 = mid_x + vertical_dist * dy;
float intersection_y1 = mid_y - vertical_dist * dx;
float intersection_x2 = mid_x - vertical_dist * dy;
float intersection_y2 = mid_y + vertical_dist * dx;

Added:
float a = (c1Radius * c1Radius - c2Radius * c2Radius + distance * distance) / (2 * distance);
float h = std::sqrt(c1Radius * c1Radius - a * a);

float dx = (c2.x - c1.x) / distance;
float dy = (c2.y - c1.y) / distance;

// Calculate intersection points
float intersection_x1 = c1.x + a * dx + h * dy;  
float intersection_y1 = c1.y + a * dy - h * dx; 
float intersection_x2 = c1.x + a * dx - h * dy;
float intersection_y2 = c1.y + a * dy + h * dx;

Seeing that the app is growing in memory but never libarating any, looking into optmizations.

Moving calculations from paint() from the public are to the private area and making them functions that are called within paint() seems to limit CPU usage (passed from 80% when there's a lot of circles to 30%).

Memory is increasing less and sometimes deven drops down.

Hot day, slow day

12-08-2024

Looking into more optimization, deciding I'll do it when my project is advanced enough.

Things to do:

Collaborative aspect
Deciding on utility (for now):
- FM synthesis
- Echo
- Reverb / delay
- Weird Euclidean rhythm sequencer
Transforming the x and y value into something I could use
Sliders to handle water density, pebble size (and density ?)
Different circles coming out of the same point at equal interval
Take the time to write a bit about playful, "experimental" interfaces

Transfering the interface into a Audio Plugin project

13-08-2024

Transfered the interface into an Audio Plugin project. Doing this actually relsoved my CPU problem. Turns out you need to define your functions into the cpp file and not everthing inside of the h file. The h file is for declaring functions, classes, and other symbols. While the cpp file contains the implementation of the functions declared in the header file.

Looked into JUCE's GUI components and how to call them:

Example for the toggleThis() function

h file: 
void toggleThis();

cpp file: 
void YourProjectAudioProcessorEditor::toggleThis(){
    // define your function here
}

Instead of a bool I'll need to keep track of new intersections, eaning those who weren't from a present in a past state. (keep track of the state of intersections from the previous cycle and compare it with the current state)

I'll need to create a new vdata structure to store intersections and compare them.

Title

14-08-2024

Storing the intersection pairs into a vector, erasing them as soon as one of the two circles "dies down".

// Vector: 

 struct IntersectionPair
      {
          float x1;
          float y1;
          float x2;
          float y2;
      };

    std::vector<IntersectionPair> intersectionPairs;

// Check circles life time to erase the intersections from intersectionPairs:

intersectionPairs.erase(
            std::remove_if(intersectionPairs.begin(), intersectionPairs.end(),
                [this](const IntersectionPair& intersection)
                {
                    // Get the circles from the stored coordinates
                    auto circle1It = std::find_if(circles.begin(), circles.end(),
                        [&intersection](const Circle& c) {
                            return c.x == intersection.x1 && c.y == intersection.y1;
                        });

                    auto circle2It = std::find_if(circles.begin(), circles.end(),
                        [&intersection](const Circle& c) {
                            return c.x == intersection.x2 && c.y == intersection.y2;
                        });

                    // Check if circles are still present
                    if (circle1It == circles.end() || circle2It == circles.end())
                        return true; // Remove if either circle is missing

                    const Circle& c1 = *circle1It;
                    const Circle& c2 = *circle2It;
                    float r1 = calculateRadius(c1);
                    float r2 = calculateRadius(c2);

                    // Check if the circles still intersect
                    return !doCirclesIntersect(c1.x, c1.y, r1, c2.x, c2.y, r2);
                }),
            intersectionPairs.end());

Transforming the x and y axis into values that are more usable, a few ideas:

values

Tihnking about how to interface better the research log so that it's nice and friendly.

Create a circular buffer for effect like a delay:

JUCE Tutorial - Creating a Circular Buffer

Bux fixing

19-08-2024

Watch some videos from the a Applied DSP series, very clear:

Applied DSP series by Youngmoo Kim

Found a bug, it seems like if I click on the canvas and circles are intersecting directly the app crashes.

Spent 2h debugging only to find out i put a breakpoint by accident in my code.

Now I know how to set breakpoints in Xcode 👍

Set up the slider to change the radius of the circles dynamically.

Now searching how to make an oscillator and trigger it's enveloppe at each newly created intersection.

Trying to connext sound to the inteface

20-08-2024

Tried to implement an ADSR envelop and MIDI inputs, did not work. Worked on the interface of the research log.

Shifting frequencies using the interface

22-08-2024

Shifting frequencies using the interface.

Several circles on one point

23-08-2024

Tried to implement an ADSR envelop and MIDI inputs, did not work. Worked on the interface of the research log.

Beginning with FM Synthesis

26-08-2024

Frequency modulation synthesis (or FM synthesis) is a form of sound synthesis whereby the frequency of a waveform (the operator) is changed by modulating its frequency with a second waveform called a modulator.

Easy explanation of FM synthesis:

FM SYNTHS in under 4 minutes

More complicated explanation:

Sound Design Basics: FM Synthesis

Tried to make work FM synthesis within JUCE with:

Let's Build a Synth with Juce Part 8 - FM Frequency Modulation Synthesis

FM Synthesis is working.... But not

28-08-2024

Noticing that FM sytnhesis does something but not really what it is supposed to?

The sound wobbles faster and faster then slows down then wobbles foaster etc.

Spent the day trying to see what's wrong, coud not find it...

FM Synthesis is working!!

03-09-2024

Sought help from Leonard Steyaert, musician + programmer and developer at Shakmat Modular in Brussels.

Turns out juce::dsp::Oscillator, the oscillator from JUCE, was causing the issue?

Directly declaring the sinewaves with math formulas solved it.

Fine tuning

05-09-2024

Fine tuning the interface for the presentation + preparing the presentation.

Presentation day

06-09-2024

Presentation day!