Music Analysis AI - Deep12

Deep12 is a music analysis AI researched and developed by the Deep12 team at Sony Computer Science Laboratories (Sony CSL), which enables similar sound search in a large catalog of songs and identifies chord progressions and the beginning of chorus in songs. Deep12 is installed in SONY MUSIC PUBLISHING LIBRARY MUSIC, which is run by Sony Music Publishing Inc. and available as "similar sound source search" and "music analysis" function.

Deep12AGCT - "Similar Sound Source Search Function"

In conventional music search, it was necessary to search from tag information attached to a music piece, such as genre and instrument. However, Deep12AGCT's similar sound source search function uses deep learning to analyze the acoustic impression and atmosphere (a multilayered combination of timbre, rhythm, pitch, and dynamics) of the input sound source to find similar songs from a catalog, and displays them in order of similarity.In addition, since similar sound sources in the database can be searched by the specified phrase, users can specify a chorus section and search for sound sources similar to that section.

Deep12AGCT is based on the concept of A Geometric Clustering Tool (AGCT), a gene clustering technology that has been studied mainly by Sony CSL researcher Dr.Polouliakh, and developed by Dr.Akama by applying Deep Learning. It is implemented in the SONY MUSIC PUBLISHING LIBRARY MUSIC website as "Similar Sound Source Search Function".

Deep12MIR - "Music Analysis Function"

Deep12MIR is a technology that analyzes the input sound source, estimates the following three types of musical information, and displays them on the waveform of the sound source, allowing the user to visually understand the sound source information.

  1. The structure of the song such as the verse and chorus
    By utilizing this function, for example, the beginning of a chorus can be identified so the song can be played from the chorus.
  2. Music theory information such as beat position, BPM, key signature, chord progression, etc.
  3. Music meta information such as the genre and mood
    Deep12MIR is applied to SONY MUSIC PUBLISHING LIBRARY MUSIC as "Music Analysis Function".

About the Deep12 Team

Deep12 is a music analysis AI developed by Sony CSL's Deep12 team based on a joint research project between Sony Group Inc. and Sony CSL, led by Taketo Akama and Natalia Polouliakh, developed by Haruhiko Kishi, and promoted by Ikuko Matsusaka.

*Focal Point (FP) is a framework for promoting research activities at Sony Computer Science Laboratories (Sony CSL), and a team of members who have come together to work on a specific research theme. Each FP is assigned a unique mark and a six-digit character string. The FP marks are designed from elements of the Sony CSL logo.