NieR: Automata's Dynamic Hacking Music Tech Detailed by PlatinumGames' Masami Ueda


#1

Today, we were blessed with this short but fascinating article about a facet of NieR: Automata’s most interesting tech–the incredible dynamic music transitions that are present all over the game. Short of maybe INSIDE’s detailed music loops that act as cues for puzzles and keep the right timing to still be going even when you die and respawn, I think Automata’s transitions are the most amazing music tech happening right now in games. This article convinced me of that even more.

This tech was also a part of the original NieR, but there it was only different versions of a song switched out via smooth volume fade, and it was fairly noticeable, but the fades in Automata are even smoother and I was sure something was being done to make the two fading songs gel better. Some of that was revealed here! I hope the rest will be too, but for now…

This is the image included with the article showing the complicated music workflow of these transitions into hacking segments, all this for something that only lasts a few seconds.

Tone Filter
Over the course of the transition when you press Triangle/Y to hack, the original music is being filtered into a narrower band of frequencies, what they call a 4-octave tone filter here. They do this because very low or very high frequencies would sound real bad with what comes next…

Distortion
Bit-crushing distortion that creates a harder sound associated with the square waves of retro game console PSG sound chips. Chiptune artists would probably call what they’re doing here “fake-bit”.

Depth/Level
But luckily, this distorted version of the original song is not the only thing playing. It is lowered in volume to be a subtle touch, and a fully-produced 8-bit version of the song comes in over it! This is where they determine the depth and volume level of each track, and look at whether any important sounds are getting buried in the mix. Yes, the NieR: Automata composers made 8-bit versions of almost all of the songs just for this. AND the game is mixing individual tracks of the music LIVE. That’s… I’m sure that was the product of a fun meeting.

So, the combination of the 8-bit cover of the track and the quieter, distorted original track make for some very expressive and smooth transitions. I had thought I heard real instruments being distorted during some of these hacking segments–not just chiptune–and this explains that! I can only guess at what kind of stuff they did for the other music transitions, and I hope we learn more.

And if you learn better by watching and listening, Ueda has three video demos at the link that shows him doing the transitions live in the Wwise audio middleware program.


Best discourse.zone Posts of 2017
#2

Thank you so much for sharing this!


#3

Masami Ueda is a genius. I’m not surprised to see that the man who made Resident Evil 2 so memorable had a hand on NieR:A.


#4

The fact that they programmed the game to re-balance all of those 48 tones for what is ultimately only a few seconds of a transition (at least most of the time) is wild. Especially when I imagine that a lot of other studios would have just cross-faded into the 8-bit track and left it at that. That’s an attention to detail that I can really respect.


#5

this is legitimately incredible, thank you so much for sharing this ^^


#6

I saw some of your tweets about this, 2Mello, including the diagram, but couldn’t make heads-or-tails of it until this breakdown, so thank you for sharing. The NieR: Automata soundtrack and music tech is really great, even for an absolute layperson like me. I’m sure it was a total nightmare for them to implement (and a ton of work too), but, as a player, I couldn’t help but appreciate it.

I hope Platinum continues to deliver great sound work in the future. I still remain impressed by Metal Gear Rising: Revengeance’s smooth fade-ins to lyrics, even though I’m sure that was a little simpler than what NieR: Automata is doing.


#7

Fascinating. I’m implementing (or to be more precise the programmer/designer is implementing) a similar crossfade between a more orchestral and a more chiptune sounding mix for a turn based game I’m doing the music for and we’re doing it with simple fades which seems to work fine for us, but it has been an interesting project to work on different parallel versions of the tune to fade between (we actually use 3 versions - 2 for each side’s turn, and 1 light version for when we need the music to be more backgroundy.


#8

Glad I could help :slight_smile:

I’ll have to be honest that even I’m not sure what a tone filter is, sounds like it’s something they custom-built since Ueda did a GDC talk about it–but I understand their end goal with it, so I’m assuming a little bit.


#9

Definitely seems like a custom band-pass (I guess tone-pass, in this case?) filter if I had to guess based on what it does, at least as far as what the article says.


#10

Yeah, I think maybe tone = frequency and it was lost in translation, but then they say “4 octave” and I’m just like, is this a thing, should I know about this??? I’ve never referred to any frequency range as an octave.


#11

They probably just mean a musical octave? My read was “tone = pitch,” which is getting into semantics, but since we’re talking about music might as well make the distinction. The filter seems like it isolates the fundamental frequency of each musical pitch and then removes the overtones, since it’s outputting sine waves.


#12

The “4 octave” is just denoting the maximum range of frequencies - a standard keyboard covers just over 7 octaves, for example, so a 4 octave span is a relatively small range. In more frequency-relevant terms, the ratio of the higher note in an octave to the lower note is 2:1, so if you play the standard “A” note at 440Hz, one octave above this (which is the next “A” note) will be 880Hz, and the “A” note below it will be 220Hz


#13

I have nothing to add to this other than the fact that Keiichi Okabe is underhyped as a composer and deserves to be put on the same level as Shimomura and Uematsu. Just an incredible artist.


#14

Thank you! I wasn’t used to talking about frequencies this way :slight_smile: