0:00

Hello, welcome to the course analyzing and processing for music applications.

This week we are focusing on transformations on

describing one application

of the spectral modeling techniques that we are covering in this course.

And in this programming lectures,

we are actually trying to understand the programming aspect of it and

the code that have been put in together on the SMS tool set package.

So in these lectures, I want to talk about the most sophisticated model that

we have covered, which is the harmonic plus stochastic model and

the transformations we can do with that.

So we have been describing this type of block diagram

in which after all this analysis that we do of understanding the harmonics,

extracting the information, and then subtracting them from the original signal.

We obtain the stochastic approximation, and now we are focusing on these two red

1:06

blocks in which we can apply transformations to both

a stochastic representation and harmonic representation.

And of course then we can synthesize back.

One of these transformations is little bit special, which is the morphing.

The morphing requires two analysis of two sounds.

So we have one sound that performs all these analysis and

another one that does the same.

And we are basically interpolating these two set of functions.

The arrays of frequency amplitudes and the envelopes of the stochastic approximation.

1:49

Okay, so let's go to first the code that performs these transformations.

And the one for the harmonics is harmonic transformations.

So we have this file called harmonic transformations in which

we do the transformation just on the harmonic component of this model.

And the ones that we have implemented, the transformations that we have in the SMS

tools, are the idea of scaling the harmonics then,

stretching the harmonics in a non-harmonic way.

And then the idea of preserving or

not, the spectral shape or the timbre of the sound.

Okay, so within this function, harmonic frequency scaling we can do

all this at the same time if we pass these three control parameters.

We're not going to go into the detail of this code but basically we can see

there is here, at the end, the three basic transformation lines.

Which we have the frequencies that these are the input

frequencies of all the harmonics of a particular frame and

a particular kind of set of harmonics, the ones that exist.

And we apply a scaling factor and multiplicity scaling factor.

So these would be the transposition, what we call transposition.

3:26

And then once we can transpose, we can also do the stretching.

Okay, and this stretching is a factor that depends on the harmonic.

This variable invalid is basically the harmonic number if it exists.

So we are elevating this stretching factor to this value.

So it's a way to change the frequencies depending on the harmony.

There's several ways of implementing this idea of stretching.

We can do it in fact the other way around by having

the harmony number to the power of the stretching factor or this.

You can play around with these two implementations and

then there is the idea of timbre preservation.

So in this particular function we are not manipulating the magnitudes but

what we're doing is if we are preserving the timbre, then we are changing

the magnitudes in a way that we give to every harmonic the magnitude

that is closer to the frequency that it originally had.

So basically we have a spectral envelope and

then we recompute the magnitude in a way that it preserves

4:50

the shape that originally had.

Then, we have the transformations that are specific for

the combination of the harmonic and stochastic components.

Which are in this HPS transformation file, and

there is basically two functions, one that performs time scaling.

And the time scaling is done on the three components at the same time,

the frequency and amplitude of the harmonics, and the stochastic envelope.

So it's perform in this core loop.

And basically in interpolates while in fact it just adds or

deletes frames depending on these time scaling parameter.

And then these more complex function, which is the morphing that accepts

the race from the two sounds, so the analysis of the two sounds.

5:53

So the core of the function is just this loop,

that basically iterates over the whole set of frames of the sound.

It finds the harmonics that are present in both sounds,

because of course the interpolation between two sounds can only happen

if there is content in the two.

One has a zero harmonic, that becomes a very complex thing to handle.

So in this code it looks for the intersection, so

the harmonics that are present.

And then it just performs the interpolation of the frequencies,

the magnitudes, and the stochastic envelope when there is some

information existing in both sounds, of the frame of both sounds.

So clearly the number of frames has to be the same and

ideally the number of harmonics and stochastic envelope, the number

of elements over break points and the approximation function has to be the same.

And that's what it does, basically we have this function called

HPS morph function that wraps these functions and allows us to do analysis and

synthesis and is the one that is called from the interface.

So we have basically two functions.

One is analysis that returns the results of the analysis.

And then another is the transformation and

synthesis that performs the transformation and synthesis of the sound.

Okay, and if we run this we can just execute that so

we can just type run and hpsMorph.

7:43

Okay, this will execute both functions.

Of course we can also execute one and the other independently by

importing the file and then just calling the two separately.

So this is the result of the analysis function that

performs analysis of the two.

And this is the result of the transformation and

synthesis that performs the resynthesized version of that.

Since we already have listened to these sounds so

there is no need to listen to them here now.

But please feel free to play around with these functions.

It is quite interesting what you can do.

And of course by programming you can extend these quite a lot.

And that's all.

Basically we have gone through the HPS part of the transformations

using SMS tools, these number of transformations that

are specific for the harmonic plus stochastic model.

And being the most sophisticated one, it can perform some interesting things.

But it's a little difficult to use it, so you have to know to do proper analysis and

to control these representations that they're a little bit more sophisticated.

9:08

Anyway, so hopefully with these you got an idea of the harmonic

stochastic model in terms of each potential for transforming sounds.

Sounds that of course have to be harmonic and

they have to have a meaningful stochastic representation.

If those sounds fulfill that model then

the number of things we can do with it is really great.

So, thank you very much for your attention and I will see you next class, bye bye.