XDMind Brings AI-Powered Music Adaptation To Game Soundtracks
Machines are getting smarter, which means the music industry is taking AI and metaverse more seriously. According to the duo behind XDMind, gaming is a perfect place to unleash AI-powered adaptive music technology.
A game without music is like a car without a stereo system. Not funny. But what about all the gamers muting their games while listening to their own Spotify playlists? Despite the presence of brilliant soundtracks and fantastic musical integrations in games like Grand Theft Autothere seems to be a lot of room for evolution in this area.
Now startup XdMind is trying to fill the “dumb void”. Using adaptive artificial intelligence technology, XdMind’s GameDJ allows a gamer to integrate any favorite music into any game. But instead of a parallel stream, GameDJ intelligently integrates the music directly into the game. ‘stock. That means stepping up the music around more action-heavy sequences, for example, and toning things down during downtime (including non-action modes like character select). The result is a more responsive, action-appropriate music soundtrack powered by AI.
“A large percentage of gamers enjoy listening to their own music,” relayed XDMind co-founder Ara Bernardi, a 20-year Microsoft engineering veteran. “I’m a case study in this area: I exclusively listen to my music while gaming, especially when in-game music gets boring or irritating.”
“There are so many game soundtracks on Spotify for one simple reason: gamers want more than just in-game music.”
XdMind’s ‘your games, your music’ concept has already attracted strong interest from the gaming industry and its investors (currently, XDMind is privately funded).
According to XDMind, game developers have struggled with musically disconnected gamers for years, with mixed success. But the prospect of an integrated, adaptive musical soundtrack offers the possibility of greater musical choice and greater personalization – and therefore, vastly increased engagement.
“Game makers don’t really know what music you like; only you do,” said XdMind co-founder Linda Bernardi, a veteran tech disruptor and innovator. “Music is very personal and games can be enjoyed more dramatically if your favorite music is matched to your favorite game, seamlessly and without any interruption to your gameplay.”
“We enable true music customization and dynamic adaptation of music to video games via AI like never before, realizing true personalization.”
The first iteration, seen by Digital Music News, offers a nice glimpse of the possibilities (DMN became a pre-launch partner of XDMind late last year). The Android-specific concept uses personal collections on mobile games like Subway Surfers and Candy Crush. The first step is to swap the in-game music with a user-selected track. From there, the music action is paired with the game action.
GameDJ is designed to integrate immediately, without extensive configuration or app updates.
Once engaged, GameDJ works alongside the game without forcing interruptions or distractions. According to XDMind, this out-of-the-box “side-by-side” development philosophy will improve early engagement beyond super geek gamers. The company also staunchly avoids ads, tracking, or other pesky intrusions.
Over time, GameDJ adapts to the game to improve fit using its video understanding technology. Ultimately, the goal is to create a harmonized contextual experience that correctly matches certain sections of songs – while subtly altering those songs – to fit the action and enhance the gameplay experience.
The first version of GameDJ is an excellent step towards this vision. Initially, GameDJ relies on fade-ins and fade-outs to transition its matches, though eventually more advanced techniques like looping, stylistic changes, and even mashups will be used in future releases, depending on the company.
This is new to the gaming industry, but GameDJ is also attracting interest from the music industry.
Both industries benefit from skyrocketing growth and untapped expansion opportunities. Accenture recently valued the global gaming industry at over $300 billion, with mobile gaming spawning hundreds of millions of new gamers in the past few years alone. This is happening alongside a massive app boom: according to the latest data from App Annie, a record 230 billion apps were downloaded in 2021, with total app spending jumping 19% in 2021 to reach $170 billion.
The music industry is also seeing a dramatic increase: according to data released by global industry trade group IFPI, global recording revenues jumped 18.5% to $25.9 billion in 2021. But revenue from streaming platforms like Spotify, which fueled the surge, is starting to cool. Suddenly the industry is exploring new platforms for growth, and games and the metaverse are at the top of the list.
In its current iteration, GameDJ only uses locally stored, user-owned music collections and associates those songs with an existing game.
This structure is designed to make proof of concept more flexible without triggering a copyright issue. The next step is to introduce greater music options and licensing catalogs.
On this front, XDMind is already in advanced discussions with major music IP owners. The company noted that the talks involve “very large libraries of music for streaming,” with considerable flexibility in usage settings and creative edits.
At first, the prospect of bend and match AI can seem daunting for major content owners. But within the confines of a gaming add-on, the math changes quickly. “GameDJ allows gamers to expand their music in a totally different way and in a controlled environment,” Linda explained. “I think they see the benefits pretty quickly.”