Making Vamp Plug-Ins (for Sonic Visualiser)

· development, Research
Author

I have recently been developing some of my research topic into a Vamp Plug-In. On this page, I would like to share this (sometimes painful) experience.

This is still work in progress, but in case the “work in progress” would take too long, I decided to publish it while writing it. That good enough for ya?

Context

Although not exactly the topic of this page, it may be necessary to introduce what kind of plug-in I wanted to develop.

As part of my research, I develop a way of representing the fundamental frequency (F0) salience in an audio file. This might seem trivial to many people in the field, often referring to harmonic sums or derivatives. Here, I proposed something related with Non-negative Matrix Factorisation (NMF): by decomposing the signals power spectrum onto a dictionary of harmonic spectra, one may indeed avoid several “classical” errors, such as octave errors.

The most important to understand here is that for my plug-in, I need several computations for which I rely on libraries freely obtainable. with NMF, matrix products have to be expected: I used the ATLAS library, more specifically functions like cblas_sgemm. To generate the harmonic spectra, my best bet has been to generate a harmonic signal, and then compute its Fourier transform (FT). For this, I used FFTW. In addition, of course, you need the vamp plug-in SDK library (and can’t go around it, I suppose…).

These two libraries are easily installed on Linux, a bit more tricky to use on MacOsX, and nightmarish to use under Windows. Any sensible person would probably decide not to use these libraries and instead go for a direct implementation of matrix product (very slow in our case) and of FTs (slow, but only affects initialisation, so would actually be fine). Well, I thought, at least for ATLAS, to give a try. This is mostly the story of that crazy cross-compiling adventure that I want to related in this page. I hope this will be of some help to some other people. Enjoy!

Some remarks on the plug-in class

Compiling under Linux

That’s the easy one: just use your favorite package manager, install the required libraries (which are all present for Ubuntu, for instance). To make the shared library (‘.so’), with dynamic linking to all the libraries, the provided skeleton will do.

It s a bit more complicated to make a ‘.so’ which is statically linked. So far, I have succeeded in doing so, but for some reason, the produced library often results in segmentation faults. That however does not seem to happened in the other versions I could compile. More testing will be needed, and maybe more to appear here later…

Compiling under MacOsX

Under MacOsX, with Xcode installed, there should not be much trouble. Using the skeleton Makefile, that was easy… Well, at least as long as the necessary libraries are installed as well!

As concerns the use of BLAS and ATLAS, one can use the provided “Accelerate” library, found on my MacOsX 10.6 in the folders:

  • include: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/Headers/
  • lib: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/

For FFTW, it is also important that the library be built for the right architecture (32 bits for the version of Sonic Visualiser provided on the website). The same goes for libsndfile, if you need to compile the vamp plug-in SDK, of course… From my experience, building universal binaries does not work in my case, maybe due to the fact that my libraries were themselves not universal. If someone is interested to try it, I d be glad to hear about that…

Compiling under Windows… well, compiling under Linux for Windows… or something like that, anyway!

That is the big bit, the nasty one. I did not really want to use Microsoft software, although I have nothing against Microsoft really: it s just not my cup of coffee. And I use open source stuff which does not easily go along with MSVC, such as libsndfile or ATLAS. Anyway, since many people under Windows may actually be interested in the plug-in, I had to do something there.

I will mainly stick to cross-compiling the library from Linux Ubuntu 10.04 to Windows (XP), using MingW.

In order to compile the plug-in and make a DLL, as suggested by Chris Cannam on this post, you need to compile the SDK with mingw. Using the provided Makefile. mingw in the build directory of the SDK source directory, that actually went quite smoothly. The only thing I needed in addition to that was the compiled libsndfile library which can be found there.

For ATLAS, well, that’s the hard part. Assuming it’s worth it (not sure, because with all this cross-compiling issue, maybe the “auto-tuning” gets lost and you would have been better off with some sub-optimal but easy direct matrix product implementation. Whatever…), so assuming it’s worth it, what worked for me was:

  1. Compile ATLAS with Cygwin, under Windows.
  2. Get the produced libraries .a and .dll, put them somewhere mingw can find them.

What does not work for ATLAS, obviously, is to cross-compile it under Linux, with MingW. It would seem that the configure step does not really recognize the compiler, even when setting it explicitly, and always falls back onto gcc (instead of the MingW compiler).

Additional stuff

I should at last add that the plug-in is available on our research website, companion website for our JSTSP article (as of 22/05/2011 to appear).

A GIT repository also exists here.

Remarks and help welcome!