A research group led by Assistant Professor Eita Nakamura of Kyoto University has developed a technology to automatically generate a score from piano performance audio data, and succeeded in generating a score at a level close to practical use for the first time in the world.
"Transcription", which involves listening to music performances and transcribing them into sheet music, is also called ear copy, and is an ability that only people with special training have."Automatic transcription technology" that reproduces this ability on a computer and can be used by anyone is being researched.Especially when transcribing a piano performance, it is very difficult to recognize a complicated combination of pitch and rhythm.
For piano transcription, "multiple pitch detection" that estimates which pitch is sounding at each time in the voice signal, and recognition of the pronunciation time and note length of each note in beat time units. There is a problem of "rhythm quantization".Research on methods using machine learning is progressing for each, but this time the research group has constructed an automatic piano transcription system that integrates both methods.
In the multiple pitch detection, a deep neural network was used to estimate the pitch and intensity of the sound included in each time and the presence or absence of keystrokes for the input voice data.Rhythm quantization used a model based on a model of temporal fluctuations contained in human performance and a model representing the statistical characteristics of general rhythm patterns appearing in musical scores.In addition, for the recognition of time signatures and bar line positions, we improved the transcription accuracy by using statistics that capture the relationships between notes.As a result, we succeeded in automatically generating a musical score that can be partially used for playing and assisting human transcription.
In the future, it is expected to lead to research and development for practical use, application to musicology research and music education, and scientific understanding of intelligence that supports culture.He also said that there is an urgent need to discuss issues related to copyright law and its impact on musicians.
Paper information:[Information Sciences] Non-Local Musical Statistics as Guides for Audio-to-Score Piano Transcription
