demix

DEMIX Python/NumPy implementation

Description

DEMIX is an algorithm that counts the number of sources, based on their spatial cues, and returns the estimated parameters, which are related to the relative amplitudes between the channels, as well as the relative time shifts. The full description is given in [Arberet2010]:

Arberet, S.; Gribonval, R. & Bimbot, F. A Robust Method to Count and Locate Audio Sources in a Multichannel Underdetermined Mixture IEEE Transactions on Signal Processing, 2010, 58, 121 - 133

This implementation is based on the MATLAB Toolbox provided by the authors of the above article.

Additionally, this implementation further allows time-frequency representations other than the short-term Fourier transform (STFT).

Copyright

Jean-Louis Durrieu, EPFL-STI-IEL-LTS5

jean DASH louis AT durrieu DOT ch

2012-2013

Reference

class pyfasst.demixTF.DEMIX(audio, nsources=2, wlen=2048, hopsize=1024, neighbors=20, verbose=0, maxclusters=100, tfrepresentation='stft', tffmin=25, tffmax=18000, tfbpo=48, winFunc=<function sqrt_blackmanharris at 0x102b1e9b0>)

DEMIX algorithm, for 2 channels.

adaptive_thresholding_clusters()

compute for each cluster in self.clusters a threshold depending on the other clusters, in order to keep only those points in cluster that are close to the actual centroid, but not close to centroids of other clusters.

The returned clusters are the original clusters thresholded.

comp_clusters(threshold=0.8)

Computes the time-frequency clusters, along with their centroids, which contain the parameters of the mixing process - namely theta, which parameterizes the relative amplitude, and delta, which is homogeneous to a delay in samples between the two channels.

comp_parameters()

comp_pcafeatures()

Compute the PCA features

comp_sig_repr()

Computes the signal representation, stft

compute_temporal(ind_cluster_pts, zoom)

This computes the inverse Fourier transform of the estimated Steering Vectors, weighed by their inverse variance

The result is a detection function that provides peaks at the most likely delta - the delay in samples.

create_exclusive_clusters()

reconfigures the cluster indices in self.clusters such that all the Time-Freq points that appear in more than one cluster are dismissed from all computations

estimDAOBound(confidence, confidenceVal=None)

computes the max distance between centroid and points

getTFPointsNearDelta(centroid)

returns a TF mask which is True if their corresponding value of delta is close enough to the delta from the centroid.

getTFpointsNearTheta_OneScale(centroid_tfpoint)

returns the TF points whose theta is close to that of the centroid, among the points considered in index_pts_to_classify

TODO: make the function for different scales, as in matlab toolbox

get_centroid_distance()

distance between the centroids

identify_deltaT(ind_cluster_pts, centroid, threshold=0.8)

returns the delay maxDelta in samples that corresponds to the largest peak of the cluster defined by the provided cluster index

reestimate_clusterCentroids()

reestimate cluster centroids

considering all the cluster masks, reestimate the centroids, discarding the clusters for which there was no well-defined delta.

refine_clusters()

Refining the clusters in order to verify that they are possible. Additionally, if self.nsources is defined, this method only keeps the required number. Otherwise, it is decided by choosing the most likely centroids.

remove_empty_clusters()

DJL: this did never happen in DEMIX Matlab version, have to contact authors for explanations...

spatial_filtering()

using optimal spatial filters to obtain separated signals

this is a beamformer implementation. MVDR or assuming the sources are normal, independent and with same variance (not sure whether this does not mean that we can’t separate them...)

From:

Maazaoui, M.; Grenier, Y. & Abed-Meraim, K.
``Blind Source Separation for Robot Audition using
Fixed Beamforming with HRTFs'', 
in proc. of INTERSPEECH, 2011.

per channel, the filter steering vector, source p:

\[b(f,p) = \frac{R_{aa,f}^{-1} a(f,p)}{a^{H}(f,p) R_{aa,f}^{-1} a(f,p)}\]

steeringVectorsFromCentroids()

Generates the steering vectors a(p,f,c) for source p, (reduced) freq f and channel c.

\[a[p,f,0] = \cos(\theta_p)\]\[a[p,f,1] = \sin(\theta_p) \exp(- 2 j \pi f \delta_p)\]

pyfasst.demixTF.confidenceFromVar(variance, neighbors)

Computes the confidence, in dB, for a given number of neighbours and a variance.

pyfasst.demixTF.get_indices_peak(sequence, ind_max, threshold=0.8)

returns the indices of the peak around ind_max, with values down to threshold * sequence[ind_max]