Deep Learning for Continuous Symbolic Melody Generation Conditioned on Lyrics and Initial melodies

Abstract

Symbolic music generation is an interdisciplinary research area, combining machine learning and music theory. This project focuses on the intersection of two problems within music generation, namely generating continuous music following a given seed (introduction), and rhythmically matching given lyrics. It enables artists to use AI as creative aid, obtaining a complete song having only written the lyrics and an initial melody. We propose a method for targeted training of a recursive Generative Adversarial Network (GAN) for initial melody conditioned generation, and explore the possibilities of using other state-of-the-art deep learning generation techniques, such as Denoising Diffusion Probabilistic Models (DDPMs), Long-short-term-memory networks (LSTMs) and the attention mechanism.