A research team led by Assistant Professor Shinnosuke Takamichi of the Graduate School of the University of Tokyo has developed a technology to artificially reproduce the singing voice of singer Yumi Matsutoya when she made her debut 50 years ago, as part of commissioned research from Toei Co., Ltd.'s Tsukun Research Institute.
Japanese song culture has formed new music and Japan's unique J-POP culture while being influenced in real time by Western song culture, and the interaction between cultures has created a new culture.Similarly, if we can reproduce precious sound materials with modern sound quality, we can contribute to the creation of a new culture.
This time, the research group aimed to artificially reproduce the singing voice of Yumi Matsutoya, who has been active as a front-line musician from the new music era to the present day, 50 years ago.It was difficult with conventional information engineering technology because the existing sound materials at the time of his debut are few and not suitable for reproducing singing voices.
Therefore, the research group has developed a multi-machine learning system that can collectively perform ``text-to-speech synthesis'' that synthesizes spoken voice from text, ``singing voice synthesis'' that synthesizes singing voice from lyrics, and ``singing voice conversion'' that converts a certain singing voice into the singing voice of the time. A stepwise synthetic conversion task mixed learning algorithm” was used.It also adopts a "data editing and pruning method" that semi-automatically deletes unnecessary data for machine learning.As a result, we succeeded in reproducing the voice tone and singing expression of those days on a new song with a sound quality that corresponds to the present age.
The reproduced singing voice will be performed as a duet with Yumi Matsutoya's current singing voice under the artist name "Arai Yumi" at the time of her debut, and will be released to the public on YouTube as a music video (MV) for the song "Call me back" (October 2022). 10 day) was done.
It is expected that the use of the results of this research will lead to the creation of new methods of creating and appreciating songs through the interaction of past singing voice/audio data and current culture.
