Audio Effects

Shadi Ashnai, WRI

Join the Conversation #WolframTechConf

Why audio effects?

Audio effects can modify the output of a generated or processed audio signal. These are some examples of audio effects.

amplify or normalize — boost the gain so that the mean or maximum gain value is at a desired level

distortions

delays and echos — add decayed repeats of the audio signal

reverberations

equalization — change the frequency response

compression and expansion

pitch shifting and time stretching

lowpass, highpass filtering — remove or keep high frequency content

Tremolo Effect

Add variations to amplitude of sound over time. This can be implemented using a simple amplitude modulation.

drums=ExampleData[{"Audio","Drums"},"Audio"]

Manipulate[Module[{a},a=(.5+m AudioGenerator[{"Sin",f},Duration[drums]]);Row[{drums*a,AudioPlot[a,ImageSize->Medium]}]],{{m,.5},0.1,.5},{f,.5,2}]

Distortion

Apply any function to amplitude samples of an audio and hear effects after distorting and clipping samples.

a=Audio[Sound[Table[SoundNote[i,.2],{i,7}]]]

Plot[Tanh[10x],{x,-1,1}]

Tanh[10a]

Plot[Cos[10x],{x,-1,1}]

Cos[10a]

Delay Effect

Audio Delay

AudioDelay simulates the effect of a decaying echo.

a=AudioNormalize@Import["ExampleData/rule30.wav"]

AudioDelay[a,.7,.8,.4,PaddingSize->4]

Chorus Effect

A chorus effect attempts to simulate the rich audio sound when multiple performers play the same part. It is typically implemented by adding versions of the audio with small variations in pitch and timing. Flanger can also similarly be implemented, using shorter delay times and delay feedback.

AudioNormalize@Mean[Table[AudioPitchShift[AudioDelay[a,Quantity[RandomReal[{20,50}], "Milliseconds"],0,1],Quantity[RandomReal[{-.1,.1}],"Semitones"]],5]]

Reverberation Effect

Audio Reverb

Reverberation is the composition of an audio signal with its reflection from the environment that it is played in, after absorption by different objects in the room.

a=ExampleData[{"Audio","MaleVoice"},"Audio"];

AudioReverb[a]

AudioReverb[a,ExampleData[{"Audio","IRStMarysChurch"}]]

Other arbitrary convolutions

ExampleData[{"Audio","Clapping"}]//AudioPlot

AudioReverb[a,ExampleData[{"Audio","Clapping"}],1]//AudioNormalize

AudioReverb[a,ExampleData[{"Audio","ChurchBell"}],1]//AudioNormalize

AudioReverb[a,ExampleData[{"Audio","Water"}],1]//AudioNormalize

Synthesize the Sound of a Vibrating String

Perform Karplus–Strong synthesis by adding a short delay with a high feedback value to a burst of noise. This will simulate the sound of a vibrating string.

freq=60;
feedback=0.99;
AudioDelay[AudioGenerator["Pink",.01],1/freq,feedback,1,PaddingSize->5,Method->{"LowpassCutoff"->(Quantity[8000, "Hz", hertz, "Hertz"])}]

Simulate the strumming of a chord.

chord ={130.81,164.81,196,261.62};

Table[AudioDelay[AudioPad[AudioGenerator["Pink",.01],{RandomReal[.2],0}],1/chord[[i]],feedback,1,PaddingSize->5,Method->{"LowpassCutoff"->(Quantity[8000, "Hz", hertz, "Hertz"])}],{i,Length[chord]}]//Mean

Time Stretching & Pitch Shifting

Time Stretching

a=ExampleData[{"Audio","MaleVoice"},"Audio"]

AudioTimeStretch[a,2]

AudioTimeStretch[a,.8]

AudioTimeStretch[a,(Quantity[4, "s", seconds, "Seconds"])]

Frequency Shifting

(RowDefault)

AudioFrequencyShift[a,(Quantity[200, "Hz", hertz, "Hertz"])]

Perform a tIme-varying frequency shift by specifying the shift as a function or a time series.

AudioFrequencyShift[a,300+600Sin[Pi#]&]

Create a pseudo-chorus effect by performing a time varying frequency shift.

AudioFrequencyShift[a,2Sin[Pi#]&,.5]//AudioNormalize

Pitch Shifting

(RowDefault)

AudioPitchShift[a,(Quantity[2, "semitones", semitones, IndependentUnit["semitones"]])]

AudioPitchShift[a,(Quantity[-2, "semitones", semitones, IndependentUnit["semitones"]])]

Compression & Expansion

Compression is the process of reducing the dynamic range to create a more even signal level, reducing the level of the loudness parts.

a=AudioNormalize[ExampleData[{"Audio","NoisyTalk"}]]

AudioPlot[a,ImageSize->Medium]

Compute the RMS amplitude of the signal.

amplitude=AudioLocalMeasurements[a,"RMSAmplitude",PartitionGranularity->{.01,.005}];
ListLinePlot[amplitude,AspectRatio->1/8]

Perform a compression effect using a threshold value of 0.05 and the compression ratio of 3.

ratio=3;
b=AudioNormalize[a/AudioChannelMix[AudioGenerator[LowpassFilter[1+(ratio-1)TimeSeriesMap[Boole[#>.05]&,amplitude],(Quantity[15, "Hz", hertz, "Hertz"]),201]],"Stereo"]]

AudioPlot[b]

Perform an expansion.

ratio=3;
b=AudioNormalize[a/AudioChannelMix[AudioGenerator[LowpassFilter[1+(ratio-1)TimeSeriesMap[Boole[#<.05]&,amplitude],(Quantity[15, "Hz", hertz, "Hertz"]),201]],"Stereo"]]

AudioPlot[b]

Create an Equalizer

Prototype an equalizer using a series of analog filters.

loshelf=ToDiscreteTimeModel[BiquadraticFilterModel[{"LowShelf",{{fl,1}},gl}],1/44100.,z,Method->{"BilinearTransform","CriticalFrequency"->fl}];
hishelf=ToDiscreteTimeModel[BiquadraticFilterModel[{"HighShelf",{{fh,1}},gh}],1/44100.,z,Method->{"BilinearTransform","CriticalFrequency"->fh}];
peak=ToDiscreteTimeModel[BiquadraticFilterModel[{"Peaking",{{fp,1}},gp}],1/44100.,z,Method->{"BilinearTransform","CriticalFrequency"->fp}];

eqfilter=SystemsModelSeriesConnect[SystemsModelSeriesConnect[peak,loshelf],hishelf];

equalizer[a_]:=Manipulate[RecurrenceFilter[eqfilter/.{fl->2 Pi ffl,gl->10^(ggl/20.),fh->2 Pi ffh,gh->10^(ggh/20.),fp->2 Pi ffp,gp->10^(ggp/20.)},a]//Row@{AudioNormalize@#,Periodogram[#,10000,ScalingFunctions->{"Log10","dB"},PlotRange->{{10,22050},{-40,20}},ImageSize->Medium]}&,{{ffl,50,"LowShelf freq"},30,500},{{ggl,0,"LowShelf gain"},-20,20},{{ffh,10000,"HighShelf freq"},4000,20000},{{ggh,0,"HighShelf gain"},-20,20},{{ffp,1000,"Peaking freq"},500,10000},{{ggp,0,"Peaking gain"},-20,20},ContinuousAction->False]

a=AudioGenerator["PRN"]

Periodogram[a]

equalizer[a]

Try with a drums sample.

equalizer[drums]

More Effects by Modifying the Short-time Fourier Transform

a=ExampleData[{"Audio","Apollo11ReturnSafely"},"Audio"]

Spectrogram[a]

Denoise

Diminish values in the spectral domain smaller than a threshold.

Audio`AudioSpectralApply[a,#*Abs[#]/(Abs[#]+10)&]

Sound like a robot

Audio`AudioSpectralApply[a,Abs[#]&,(Quantity[40, "ms", milliseconds, "Milliseconds"])]//AudioNormalize

Perform a time and frequency dependent filter

AudioNormalize@Audio`AudioSpectralApply[AudioGenerator["White",5],#1(Tanh[(#2-(2000-#3*1000.))]/2+.5)&,(Quantity[30, "ms", milliseconds, "Milliseconds"])]

Morphing and Transforming the STFT

User defined function f takes as input a list of {freq, time} and performs spatial transformation on the short-time Fourier transform of the audio signal.

Code

ClearAll[AudioTransformation];
AudioTransformation[a_Audio, f_,OptionsPattern[{PartitionGranularity -> Automatic,DataRange->Automatic}]]:=
Block[{audio,channels,samplerate,length,partSpec,windowSize,offset,window,stft,img,transformed,res},
audio=AudioChannelMix[a,1];
channels = AudioChannels[audio];
samplerate =QuantityMagnitude@AudioSampleRate[audio];
length = QuantityMagnitude@AudioLength[audio];
partSpec = OptionValue[PartitionGranularity];
{windowSize,offset,window} = Audio`AudioUtilitiesDump`checkPartitionGranularity[partSpec,samplerate,AudioTimeStretch,length,False];If[ListQ[window],
		window = Developer`ToPackedArray[N@window, Real],
		window = Developer`ToPackedArray[N@Reverse[(window/.None->DirichletWindow)/@(N@Range[windowSize]/N@windowSize-.5)], Real]
];
stft = SpectrogramArray[a,windowSize,offset,window,FourierParameters->{1,-1}];img = Image[{Re[stft],Im[stft]}, Interleaving->False];transformed = Map[Complex@@#&, ImageData@ImageTransformation[img, f,DataRange->OptionValue[DataRange]], {2}];res = ConstantArray[0,offset*Length@transformed+windowSize];
Do[
res[[1+(i-1)*offset;;(i-1)*offset+windowSize]] += Re[InverseFourier[transformed[[i]],FourierParameters->{1,-1}]]*window,{i,Length@transformed}];
Audio[res,SampleRate->samplerate]/(Total[window^2]/offset)
]

Examples

a=ExampleData[{"Audio","NoisyTalk"},"Audio"];
spect=Spectrogram[a,ImageSize->Medium,FrameTicks->None];

sample=RadialGradientImage[{{80,50},{150,150}}->"Rainbow"];

Power.

f=((#)^1.2)&;
ImageCollage[{sample,ImageTransformation[sample,f]},ImagePadding->2,Background->White]
res=AudioTransformation[AudioNormalize@a,f,PartitionGranularity->{.04,.005,HannWindow}]//AudioNormalize;
Row[{spect,Spectrogram[res,ImageSize->Medium,FrameTicks->None]}]
res

Reverse.

f=Reverse;
ImageCollage[{sample,ImageTransformation[sample,f,DataRange->Full]},ImagePadding->2,Background->White]
res=AudioTransformation[AudioNormalize@a,f,PartitionGranularity->{.04,.005,HannWindow},DataRange->Full]//AudioNormalize;
Row[{spect,Spectrogram[res,ImageSize->Medium,FrameTicks->None]}]
res

Mosaic.

f=8Floor[#/8]+1&;
ImageCollage[{sample,ImageTransformation[sample,f,DataRange->Full]},ImagePadding->2,Background->White]
res=AudioTransformation[AudioNormalize@a,f,PartitionGranularity->{.04,.005,HannWindow},DataRange->Full]//AudioNormalize;
Row[{spect,Spectrogram[res,ImageSize->Medium,FrameTicks->None]}]
res

Rain effect.

f=Function[p,With[{C=150.,R=35.},{p[[1]],p[[2]]+(R*Cos[(p[[1]]-C)*360*2/R]/6)}]];
ImageCollage[{sample,ImageTransformation[sample,f,DataRange->Full]},ImagePadding->2,Background->White]
f=Function[p,With[{C=150.,R=400.},{p[[1]],p[[2]]+(R*Cos[(p[[2]]-C)*10000*2/R]/6)}]];
res=AudioTransformation[AudioNormalize@a,f,PartitionGranularity->{.04,.005,HannWindow},DataRange->Full]//AudioNormalize;
Row[{spect,Spectrogram[res,ImageSize->Medium,FrameTicks->None]}]
res

Fisheye effect.

f[pt_]:=With[{s={0.5,0.5}},Module[{r,a},
r=((Norm[pt-s])^2)/Norm[s];a=ArcTan@@(pt-s);
s+r{Cos[a],Sin[a]}]];
ImageCollage[{sample,ImageTransformation[sample,f,DataRange->Automatic]},ImagePadding->2,Background->White]
res=AudioTransformation[AudioNormalize@a,f,PartitionGranularity->{.04,.005,HannWindow}]//AudioNormalize;
Row[{spect,Spectrogram[res,ImageSize->Medium,FrameTicks->None]}]
res

Spiral mirror.

f[pt_]:=With[{s={.5,.5}},Module[{r,a},
r=((Norm[pt-s])^2)/Max[s];a=ArcTan@@(pt-s)/Pi*180;
pt+{Mod[(a/200+r/2.),16/200.]-8/200.,0}]];
ImageCollage[{sample,ImageTransformation[sample,f,DataRange->Automatic]},ImagePadding->2,Background->White]
res=AudioTransformation[AudioNormalize@a,f,PartitionGranularity->{.04,.005,HannWindow}]//AudioNormalize;
Row[{spect,Spectrogram[res,ImageSize->Medium,FrameTicks->None]}]
res

Thanks!

Wait! We have two more great talks.

Audio Synthesis — 10-10:30am

Audio Analysis — 10:30-11am