Scoring synchronization between music and motion: local Vs global approaches
Résumé
This paper compares methods for scoring the synchronization between music and motion. A wide range of local and global methods such as the Gaussian-based method, relative phase, and dynamic time warping (DTW) have been evaluated.
Firstly, these methods were tested on synthetic data representing various scenarios of motion and music interaction. Secondly, we
introduce a full system able to evaluate real video performances of music and movement synchronization in which participants
were asked to tap their fingers to the beat. The system takes a motion video clip as input and extracts music and motion
beats. From these two vectors, the synchronization is scored with different metrics. Results emphasize the complementary nature
of the scoring approaches. The data and code are available online at https://github.com/baya1235/Scoring-synchronization-
between-motion-and-music.