Automatic Transcription and Separation of the Main Melody from Polyphonic Music Signals
Introduction
Transcription and Separation
Applications
State of the art
Contributions
Presentation Outline
Signal Models
Signal Models: Short-Time Fourier Transform (STFT)
General Framework
Dependency graph
Instantaneous Mixture
Instantaneous Mixture
Instantaneous Mixture
Instantaneous Mixture
Lead Instrument Gaussian Scaled Mixture Model (GSMM)
Interpretation of the GSMM
Leading instrument, need for variability
Leading instrument, need for variability
Leading instrument, need for variability
Leading instrument, need for variability
Source/Filter for leading voice
GSMM Generative Schema
Graphical Model
Source/Filter GSMM
Accompaniment
GSMM summary
Alternative model: the Instantaneous Mixture Model (IMM)
IMM Generative Schema
Instantaneous Mixture Model (IMM)
IMM decomposition
(better) IMM decomposition
Comparison between IMM and GSMM
Temporal Constraints: Markov model for and
Temporal Constraints: Duration Model for Note Layer E
Parameter and sequence estimation
Issues with Parameter and Sequence Estimation
(1) Criteria for parameter estimation
(1) Multiplicative gradient principle
(2) Viterbi melody tracking
(3) Beam Search Algorithm
(3) Beam search outputs
Systems
Results for the Transcription Tasks
Estimation of the main melody F0
Estimation of F0: Performance measures
MIREX 2008 results
MIREX 2009 results
Estimation of the main melody F0: F-III
Transcription: melody note estimation
Transcription: melody note estimation
Leading Instrument and Accompaniment Separation
Leading Instrument separation
System SEP-I
Parameter estimation knowing the melody
Adaptive Wiener filtering
Results
More examples...
Conclusion
Publications
To go further...
Signal Model: complex Gaussians
Signal Model: Model details
Signal Model: GSMM for mixture
Signal Model: HMM on F0 sequence
Signal Model: Temporal constraints
Signal Model: Temporal constraints
Systems: Viterbi algorithm
Systems: Beam Search, Opera example