Mitch Dörfler's “Sonic Lifeforms” is an interactive procedural music album featuring infinitely changing, responsive visual elements and controllable…
Mitch Dörfler's “Sonic Lifeforms” is an interactive procedural music album featuring infinitely changing, responsive visual elements and controllable interactions. This marvelous work was created using the power of Unreal Engine, MetaSounds, and Blueprints. Let's explore what impact this project has on music interaction, and the future of game and video content development.
The Creative Process and Challenges of Procedural Music
Procedural music is art in itself. Mitch said he used a combination of "Melody and Pattern-based Systems" and "Pre-produced Musical Building Blocks." The former is a method where algorithms generate individual notes and patterns, while the latter uses musical elements pre-generated by human musicians. He stated:
“Both approaches are appealing, but for now, I find using pre-produced musical building blocks more appealing.”
A Feast of Interaction with Unreal Engine
Unreal Engine has established itself as a powerful tool for creating procedural music. Centered around Master Blueprint Actors and MetaSound Sources, it created music that dynamically changes based on user interaction. Mitch explained the following:
“The randomization logic is controlled in Blueprint, generating various musical combinations by simultaneously triggering multiple sound layers randomly.”
Interaction with the user, implementation on iOS
"Sonic Lifeforms" was released as an iOS app, allowing users to adjust music and visual elements using an XY pad as a central control element. This interaction reflects the user's personal emotions and experiences into the music, providing a truly personalized musical experience. Mitch mentioned the following regarding iOS development:
“Apple’s certification system and audio management mechanism are complex, but we utilized various resources and tools to overcome this complexity.”
Introduction of MetaSound
MetaSound, introduced in Unreal Engine 5, provides audio designers with the opportunity to perfectly control a Digital Signal Processing (DSP) graph for generating sound sources. This system ensures tremendous flexibility in sound design within the editor, as well as user customization, third-party extensibility, and graph reusability.
“MetaSound allows audio designers to synthesize and generate audio at runtime, and freely combine and connect procedurally generated sounds with other audio sources.” – Unreal Engine 5 MetaSound Introduction
Music expression in the metaverse
In the metaverse environment, interaction and user experience are central. MetaSound can be easily integrated with game data and player interactions to create immersive experiences triggered by gameplay events. This functionality enables personalized and dynamic audio experiences within the metaverse.
Deep Features of MetaSound
The MetaSound Editor is a node-based interface that allows even audio designers without programming experience to easily create procedural sound. Real-time preview, various control options, and an intuitive graphical interface provide users with effective control.
Sample-level precise audio control capabilities take audio design in games and real-time applications to the next level. Tight linkage with interactive play events makes the audio experience in the metaverse richer and more diverse.
“Each MetaSound is an audio rendering engine in itself. They render in parallel and have independent rendering formats such as sample rate, buffer size, and number of channels.” – Unreal Engine 5 MetaSound Introduction
Conclusion: The Future of Music Experience
“Sonic Lifeforms” goes beyond just a music album; it explores the possibilities of interactive audio-visual experiences. This approach changes how users experience music and establishes new boundaries for future digital art and entertainment. Mitch added at the end:
“Unreal Engine can be considered a powerful tool for creating music applications, and we tried to present basic ideas, workflows, and possible solutions.”
We now have the tools and technologies to create more than just games and movies. “Sonic Lifeforms” shows how we can combine art and technology, and how we can express infinite creativity through it, going beyond a simple musical experience.
Emotionwave AI Research Institute
The Emotionwave AI Research Institute is researching various algorithms and artificial intelligence models that apply LLMs and generative AI based on media technology.
Reference
– (In-Depth Tutorial | “Sonic Lifeforms”- the never-ending album)
– https://apps.apple.com/us/app/sonic-lifeforms/id6444698992
–
– https://dev.epicgames.com/community/learning/tutorials/G3pj/unreal-engine-a-mighty-music-creation-tool
– http://tripbox.com/portfolio/slf/ #Web3.0 #Metaverse #Algorithm #MusicCreation #MusicAI #ProceduralMusicGeneration Emotionwave Editorial Team