Résumé
The process of synchronizing movements to musical rhythms involves a complex interplay betweenauditory perception and motor coordination. This study explores the impact of tempo, beat division,movement type, and musical expertise on sensorimotor synchronization.Participants, from France, were tasked with synchronizing two types of movements — haptic (fingertapping) and no haptic (arm swinging) — to an external audio rhythm. The audio stimuli were takenfrom the GTZAN rhythm corpus, featuring a variety of tempo levels (slow, medium, fast) and beatdivisions (binary, swing, ternary).We developed a software that extracts kinematic data using a deep learning-based 2D pose estimatorfrom a live webcam at 30 fps, that is sufficient for capturing detailed information up to 120 bpm. Thissoftware captures skeletal movements, enabling us to track interphalangeal joints of the hand and theposition of arm joints. It provides clear and interpretable data on the timing and spatial coordination ofmotion dynamics. We use the Euclidean distance between fingers to identify motion beats in the tappingtask. We analyze changes in arm joint positions to determine the motion beats in the arm swing task.We computed synchronization scores based on music and movement features from each audio-videoperformance, where the data are represented as discrete time series. In that scope, we considered bothglobal (e.g., synchronization continuity) and local approaches (e.g., Gaussian function) to calculate thescore from these time series of music and movements. Each metric evaluation had its contribution andsignificance, providing a multi-faceted view of synchronization quality.Our results demonstrate clear trends regarding the influence of factors such as tempo, beat division,musical skills, and type of movement on synchronization performance. For instance, synchronizationscores decreased as tempo increased. Additionally, the task of finger tapping enabled better synchronizationscores than arm motion, both among participants with and without musical skills. This could beexplained by the more accurate sensory feedback (haptic and auditory) provided by finger tapping, whichis not present in the arm swing motion.