Time-Frequency Transforms

TFT module

Time-Frequency Transforms

TODO: turn this into something more self-contained (like defining a super class for all the possible time-freq transforms)

class pyfasst.tftransforms.tft.TFTransform(fmin=25, fmax=1000, bins=12, fs=44100, q=1, atomHopFactor=0.25, thresh=0.0005, winFunc=None, perfRast=0, cqtkernel=None, lowPassCoeffs=None, data=None, verbose=0, **kwargs)

TFTransform is the Time-Frequency Transform base class. All the TF representations sub-classing it should implement the following methods:

• TFTransform.computeTransform() to compute the desired transform on data. The transform is then stored in TFTransform.transfo
• TFTransform.invertTransform() to invert the transform from the stored transform in TFTransform.transfo

computeTransform(data)

Computes the transform on the provided data. The sub-classes should re-implement this method, and store the result in the attribute TFTransform.transfo.

invertTransform()

Computes the inverse transform from the stored transform in TFTransform.transfo

transfo = None

TFTransform.transfo receives the transform when computeTransform is called.

pyfasst.tftransforms.tft.tftransforms = {'minqt': <class 'pyfasst.tftransforms.minqt.MinQTransfo'>, 'mqt': <class 'pyfasst.tftransforms.minqt.MinQTransfo'>, 'nsgmqt': <class 'pyfasst.tftransforms.nsgt.NSGMinQT'>, 'stft': <class pyfasst.tftransforms.stft.STFT at 0x102a8f390>, 'stftold': <class 'pyfasst.tftransforms.tft.TFTransform'>, 'cqt': <class 'pyfasst.tftransforms.minqt.CQTransfo'>}

A convenience dictionary, with abbreviated names for the transforms.

STFT module

class pyfasst.tftransforms.stft.STFT(linFTLen=2048, atomHopFactor=0.25, winFunc=<function hanning at 0x1025d58c0>, fs=44100, synthWinFunc=None, **kwargs)

Object that implements the computation of Short-Term Fourier Transforms (STFT) and its inverse.

Inputs:

Parameters:

• linFTLen (integer) – size of the Fourier transform
• atomHopFactor (double) – ratio of delay from frame to frame. 0.25 corresponds to a 25% “hop” ratio, or equivalently to 75% of overlap between succesive frames.
• winFunc (function) – analysis window function.
• fs (integer) – sampling rate of the processed signals
• synthWinFunc –
• kwargs –

filter_conv_stft(data, W, analysisWindow=None, synthWindow=array([ 0. , 0.00153398, 0.00306796, ..., 0.00460193,

0.00306796, 0.00153398]), hopsize=256.0, nfft=2048.0, fs=44100.0, verbose=0)

Sequentially compute Fourier transfo, filter and overlap-add

INPUTS

W

M x F x N (or M x F) filter for the data, which should be single channel

data

T (number of samples, number of channels)

...

filter_stft(data, W, analysisWindow=None, synthWindow=array([ 0. , 0.00153398, 0.00306796, ..., 0.00460193,

0.00306796, 0.00153398]), hopsize=256.0, nfft=2048.0, fs=44100.0)

Sequentially compute Fourier transfo, filter and overlap-add

W is the M x M x F x N filter for the data, which should be T x M data T x M (number of samples, number of channels)

istft(X, window=array([ 0. , 0.00153398, 0.00306796, ..., 0.00460193,

0.00306796, 0.00153398]), analysisWindow=None, hopsize=256.0, nfft=2048.0)

data = istft(X,window=sinebell(2048),hopsize=1024.0,nfft=2048.0,fs=44100)

Computes an inverse of the short time Fourier transform (STFT), here, the overlap-add procedure is implemented.

Inputs:

X :

STFT of the signal, to be “inverted”

window=sinebell(2048) :

synthesis window (should be the “complementary” window for the analysis window)

hopsize=1024.0 :

hopsize for the analysis

nfft=2048.0 :

number of points for the Fourier computation (the user has to provide an even number)

Outputs:

data :

time series corresponding to the given STFT the first half-window is removed, complying with the STFT computation given in the function stft

stft(data, window=array([ 0. , 0.00153398, 0.00306796, ..., 0.00460193,

0.00306796, 0.00153398]), hopsize=256.0, nfft=2048.0, fs=44100.0)

X, F, N = stft(data,window=sinebell(2048),hopsize=1024.0,

nfft=2048.0,fs=44100)

Computes the short time Fourier transform (STFT) of data.

Inputs:

data :

one-dimensional time-series to be analyzed

window=sinebell(2048) :

analysis window

hopsize=1024.0 :

hopsize for the analysis

nfft=2048.0 :

number of points for the Fourier computation (the user has to provide an even number)

fs=44100.0 :

sampling rate of the signal

Outputs:

X :

STFT of data

F :

values of frequencies at each Fourier bins

N :

central time at the middle of each analysis window

MinQTmodule

Constant-Q transform after the work by C. Scholkhuber and A. Klapuri 2010 [SK2010]

Adaptation of the Constant Q transform as presented in

[SK2010]

(1, 2) Schoerkhuber, C. and Klapuri, A., “Constant-Q transform toolbox for music processing,” submitted to the 7th Sound and Music Computing Conference, Barcelona, Spain.

Comments beginning with ‘%’ and ‘%%’ are retained from the original Matlab code.

Python/Numpy/Scipy by Jean-Louis Durrieu, EPFL, 2012 - 2013

class pyfasst.tftransforms.minqt.CQTKernel(fmax, bins, fs, q=1, atomHopFactor=0.25, thresh=0.0005, winFunc=<function sqrt_blackmanharris at 0x102a8ded8>, perfRast=0)

The CQT Kernel contains everything that can be precomputed for Constant-Q transforms. This relies on [SK2010], and therefore computes a Kernel for a single octave. It is then efficiently used to compute the decomposition on the different octaves by downsampling the signal.

Parameters:

fmax

The maximum desired central frequency

bins

The number of bins per octave

fs

Sampling rate of the audio files

q

parameter that controls the quality

atomHopFactor

hopsize rate (0.25 is a hopsize of 25% the size of the windows) between successive analysis windows

thresh

threshold value for sparsifying the kernel (Note: in this implementation, we do not use the sparsity, more efficiency could be achieved by considering it)

winFunc (python function that outputs an array)

the analysis window function

perfRast

whether computing rasterized version or not (if so, the decompositions at all scales will have the same number of frames, otherwise, each lower analysis octave will have half as many frames as the direct upper analysis octave.)

Attributes:

sparKernel weight atomHOP FFTLen fftOLP fftHOP bins winNr Nk_max Q fmin fmax frequencies perfRast first_center fs winFunc thresh q

class pyfasst.tftransforms.minqt.CQTransfo(fmin, fmax, bins, fs, q=1, atomHopFactor=0.25, thresh=0.0005, winFunc=<function sqrt_blackmanharris at 0x102a8ded8>, perfRast=0, cqtkernel=None, lowPassCoeffs=None, data=None, verbose=0, **kwargs)

Constant Q Transform

cellCQT2spCQT()

compute the full cqt from self.cellCQT

computeTransform(data)

Computes the desired transform

freq_stamps

frequency stamps for spCQT

invertFromCellCQT()

inverting the Cell CQT

invertFromSpCQT()

Assuming we have self.spCQT, and not self.cellCQT, we recompute self.cellCQT from self.spCQT, and then invert as usual.

NB: here, self.cellCQT is written over, if it existed.

invertFromSpCQTRast()

this inverts the transform, if perfRast, then this means we can invert each hop of the different octaves.

invertTransform()

Invert the desired transform, here invert CQT from the cell CQT: like the original from [Schorkhuber2010]

qValues

$Q$ values, approximated

spCQT

spCQT: the constant Q transform, in a readable format.

spCQT2CellCQT()

generates self.cellCQT from self.spCQT

NB: after transformation of spCQT (by filtering, for instance), this method only keeps downsampled versions of each CQT representation for each octave. More elaborated computations may be necessary to take into account more precise time variations at low frequency octaves.

time_stamps

time stamps for spCQT

transfo

returns the computed transform

class pyfasst.tftransforms.minqt.HybridCQTKernel(**kwargs)

Hybrid CQT/Linear kernel

computeMissingLinearFreqKernel()

Compute the missing (high) frequency components, and make a similar Kernel for them.

We can use this for the first octave (the highest frequency octave) to extend the high frequencies. Actually, this can be used to compute a hybrid CQT transform on the low frequencies, while keeping linear freqs in the high spectrum, and still benefiting from the invertibility of the CQT transform by Schoerkhuber and Klapuri

class pyfasst.tftransforms.minqt.HybridCQTransfo(**kwargs)

Hybrid Constant Q Transform

cellCQT2spCQT()

the spCQT is computed from self.cellCQT

computeHybrid(data)

calculates a hybrid CQT/FT representation of a sound stored in data

computeLinearPart(data)

Same as computeCQT, except it uses the linear frequency components in cqtkernel.linearSparKernel

NB: since this should be equivalent to computing an FFT after windowing each frame, there may be a faster way of implementing this function. For now, keeping the same rules as the original CQT implementation, for consistency and also for avoiding problems with window synchrony

computeTransform(data)

Computes the desired transform

invertHybridCQT()

Invert the hybrid transform.

Linearity allows to perform the cqt inverse first, and add the inverse of the linear freqs part thereafter (or the other way around).

invertLinearPart()

This inverts the linear part of the hybrid transform

NB: as for the computation of this part in transform, a windowed version of a plain FFT should do the same job, and faster.

invertTransform()

invert the desired transform

class pyfasst.tftransforms.minqt.MinQTKernel(bins, fmax, fs, linFTLen=2048, **kwargs)

Min Q Transform Kernel

computeMissingLinearFreqKernel()

Compute the missing (high) frequency components, and make a similar Kernel for them.

We can use this for the first octave (the highest frequency octave) to extend the high frequencies. Actually, this can be used to compute a hybrid CQT transform on the low frequencies, while keeping linear freqs in the high spectrum, and still benefiting from the invertibility of the CQT transform by Schoerkhuber and Klapuri

class pyfasst.tftransforms.minqt.MinQTransfo(fmax, bins, linFTLen, fs, fmin=70, **kwargs)

Minimum Q Transform

cellCQT2spCQT()

converts the cellCQT into a spCQT (matrix form)

computeLinearPart(data)

Compute the linear frequency part with an STFT, and taking only the desired frequencies.

computeTransform(data)

Computes the desired transform

invertFromSpCQTRast()

invertLinearPart()

This inverts the linear part of the hybrid transform

NB: as for the computation of this part in transform, a windowed version of a plain FFT should do the same job, and faster.

invertTransform()

invert the desired transform

linCellCQT2LinSpCQT()

converts cellCQT[‘linear’] into the corresponding bins in self._spCQT

spCQT2CellCQT()

Reverting spCQT (matrix form of the transform) back into the (original?) cellCQT (one matrix per octave, one for the linear part of the transform).