PSO2 Music Swap

[PrinceBrightstar]

Forward

Some of you may remember I put out on these forums the stickied PSU Music Replacement Guide a while back. When the PSO2 Alpha got leaked, I jumped on the opportunity grab the music and see what could be similarly done. We’ve known about the Sympathy system for quite some time, but now we have a working swap method and we can get into the deep inner workings of how this system works. A little history first though. (TL;DR scroll to Music Swap)

ADX

As part of the release of Phantasy Star Online SEGA chose to bring a type of game code on that would allow the streaming of music data in real time called ADX by CRIWare. This would also allow the music to transition back and forth without the music missing a beat as well as loop endlessly. This type of system had been created many years prior as the iMUSE system by Michael Land (employed by LucasArts) which appeared in games such as Secret of Monkey Island and Maniac Mansion. However while that system was limited to just midi files, ADX allowed standard audio files (wave or otherwise) to be played in sync with each other through compression into a .adx file type which enabled seamless looping and transitions. This also allowed PSO to load new weapon sounds in real time without having the music stutter.





ADX has been tied to the series (along with many other Sega series) for a long time and one of the benefits of using this format was the low CPU overhead it required. With the release of Phantasy Star Universe, SEGA continued to use ADX, however the music no longer transitioned for battles. Instead tracks would simply play from start to end and have a loop point. The music was also encoded to prevent it from being played outside the game. This was eventually defeated however.

ADX2

Much of the music in PSO2 is using the new ADX2 format. The new format improves on the previous ADX by allowing layered music tracks which play multiple tracks of music at the same time. By doing this and through implementation of the Sympathy system the game is able to add in different rhythm or musical elements in real time. Each track is broken into multiple segments. Forest back in alpha for instance had about 200 segments. Lilipa Quarry has 1971!



So what does this mean to us, the player?
· Faster music transitions
· Music can change dynamically as the situation requires.
· Incredibly hard to extract and put back together for playback because you’re no longer dealing with 1 file that needs to be decoded

In order for the game to read the music data properly, the data needs to be stored in a CPK container which has all of the music in one file. ADX2 and Sympathy will then read the files and play them as needed. For playback outside the game you would need to extract the CPK container into the individual files that make it up, decode the HCA audio into WAV, and reconstruct the files.

This may seem like a hopeless situation, however not every track in the game uses sympathy, and that's where we can do some swaps.

Music Swap

Up until recently there was no way to actually hear the audio in an HCA file. We now have a working music swap method because a way to convert .HCA to .WAV was found. It involves switching files around, very similar to how the first version of the PSU music swap worked. However there are a few limitations.

So far only jukebox tracks have proven to be successful with a swap. Trying to swap in a Non-Jukebox track into a Jukebox Track may cause silence on that track instead. For example when I tried to swap in the CODE: Episode 2 track it did not work. Just keep trying and you'll find one eventually that works. Also you need to close the game before you make each swap or gameguard may throw a fit.

The method is simple. Look at the Non-Packed HCA tab in the spreadsheet in the .zip attached to this post and figure out what track you want to insert onto a jukebox track. From there back up the original file and overwrite it with the new one you want to use.

There are some missing tracks on the list, mostly because I don’t own the discs. If anyone can help me fill out the list I’d appreciate it.

Of course this doesn’t allow you to replace stage and boss music. That’s the ultimate goal here. I have a method of swapping music from stages in development, however it has hit a technical snag and I won’t be able to test it until we have a way of converting from .wav to .hca. Supposedly the SDK for ADX2 provides a way to do this, however I have been unable to locate a copy of it. If you’d like to know more about how this process will work once there is a way to actually do it and I can confirm it works click below.

Spoiler!

STEP 1: File Extraction

· Go to http://oasis.xentax.com/ and download the latest version of Noesis.
· Copy the entire data folder for PSO2 into another folder on your HD (c:\data is easiest for the next step)
· Open a command window and go to the folder you copied the files to.
· Type the following without quotes. “rename *. *.cpk”
· Open Noesis, browse to the folder where you copied the files to (Note: This will take minutes to complete because of how many files make up PSO2, your computer has NOT frozen.) Sort by largest to smallest.
· Double click on the largest file and go smaller and smaller. You should eventually find one that says that nothing could be viewed but it can be extracted. Allow the program to do so. You’ll have a number of files dumped into a new directory. If they’re .adx files we’re in business. You can either move to the next step, or try every file in the PSO2 data folder before proceeding. (Using something like auto hot key will help with the process but don’t look to me for instructions on how to do it.) NOTE – The rename above is important. The files won’t extract otherwise

STEP 2: Convert from HCA to WAV

· Download HCA2WAV from http://www.sendspace.com/file/ikobo5
· Place both files into the folder that you extracted and run the .bat file. This will convert all of the files automatically.

STEP 3: Reconstructing a Track for external playback

NOTE: Much of this data was assembled during the alphas when the encryption was not HCA, but rather ADX which had not changed since Phantasy Star Universe. This change was made as of the original Character Creator release. The method still works from my testing.

So how do we rebuild the files? That’s the main trick of things. You can’t just simply play each track in turn and get what you’re looking for. (There is an exception however, any file that does not need extraction from a CPK will play straight through. However this mostly contains music from discs you find in game and some of the incidental music from the story quests.)
What will throw most people off is the fact that you need to play the next piece of music as the previous file is in the process of ending. Please see the video below to help you understand what that looks like.



As you can see each piece of the track begins slightly before the previous one ends hence why you have to alternate between two tracks. The end of the previous file would get cut off otherwise. This is the basis of what will trip most people up when reconstructing the files. You need a good ear to listen for the proper timing, but once you understand this, you’re pretty much good to go.

As for what tools to use when reconstructing the files, there are many ways to do it, however I’m looking back towards the old Commodore 64 days. If you know what Scream/Impulse/Schism Tracker or Modplug is, or if you’ve ever seen a .mod, .s3m, .xm, or .it file’s inner workings before you’ll be right at home. Back then, in order to create music tracks they needed to be sequenced out of samples. It’s the same concept as a midi file, except using samples you could get much higher quality audio. The basic principal of reconstructing the files is the same, laying samples in a pattern, except you’ll never change the pitch of the sample (C5 will always be what you use since Sega already did that step when composing it in CRI Atom Craft.)

So you’ve got a folder of tracks, but there are a few more things to do. Let’s take a look at the file structure. Each of the files here could be considered a sample.



While the files may not make sense to the untrained eye, the files themselves indicate how the tracks need to be played.
FB_MAIN_M1_8_T145_B06
FB = Track Name
MAIN = Possibly Layer Name
M1 = Pattern #
8 = Sample #
T145 = Tempo
B06 is unknown at this time, could be related to volume control.

So this file represents the track Forest Battle on the first layer in pattern 1 and sample 8. The song should be playing at a tempo of 145. Knowing this let’s open up OpenMPT and set up the song. We’ll set up the Tempo and since we know that one sample starts slightly before the previous we’ll set up a Battle 1 and Battle 2 channel. I’d also recommend setting Sample Volume to 192 or you’ll have to turn your speakers up wherever you play the track.



We need to set up one more thing or we’re going to run into problems down the road. Click on Song Properties… and change the type to OpenMPT MPTM and the Mix Levels to OpenMPT 1.17RC3.



From here you import the samples, place the samples 24 ticks apart from each other (or whatever sounds right, 24 is the example for Forest) building the file up and up until you reach the end.

In its current configuration you should end up with about 21 patterns that make up the music of FB. However something’s off. It feels like something is missing when it’s played back doesn’t it?

Head back to the general tab and add 2 more instruments called RH 1 and RH 2 and follow the same process as before for placing the RH samples in. Once completed you’ll have the full track completed.

Transitions are a little trickier because they don’t all follow the same timing and technically can fit anywhere in the track because of how ADX2 works. Personally what I do is I line up a sample before the transition sample by listening for which one flows the most naturally into it and make those two samples their own pattern. I also give the transition track its own Channel to prevent confusion. Here’s an example. Eventually you’ll be able to work through the data and have the full track together with battle and normal music and transitions, similarly to how the ogg files for PSOBB were laid out and once that’s done you’ll be all set for a track that transitions.

Step 4 Replacing with your own music.

Now that you know how the track is laid out and have a working model to work with you can begin to overwrite the song. However we need something to convert from WAV to HCA and then a way to pack everything together in a cpk. This is a dead stop on replacing music until then.

The basic idea however if this works the way it's supposed to is you'll need to write each individual HCA track over keeping the same timings for each track (it may be possible to make it longer, but if that's stored within the PSO2 exe we'll likely never be able to get around this. and then pack up the cpk with the appropriate file name for the area under the packed HCA tab. When we finally do get a WAV to HCA converter I'll update this guide with an example.

[Coatl]

Oooooh this is..really nice.

[Pdxdash]

That's pretty neat!

[Gama]

does this mean we can replace de CODE : something audio?

i hate the voice.

[Shinamori]

So, what file is that forest theme?

[PrinceBrightstar]

That's a reconstruction I had worked back on during alpha that inverted the placement of the rhythm from the battle theme to the calm theme.

Unfortuantely I don't know where the voice files are located. They don't seem to be within the HCA files. It could be another format.

[Zenobia]

Awww yissssss *Bird Man Hand Rub*

Bout to get down and dirty wit dis.

[Akakomuma]

Does this work with Lobby music? There's only so much PSU event lobby music that I can take before I go crazy!

[Shadowth117]

Does this work with Lobby music? There's only so much PSU event lobby music that I can take before I go crazy!

It should, though it might take an annoying amount of effort.

[Akakomuma]

All right, so I've experimented in the past with swapping the PSO2 Discs with others and that works perfectly, and at the same time, a PSO2 Disc counts as Lobby music so when you swap, let's say...PSO2 Wedding Music with Day Dawns, it does in-fact play that in the Lobby with no issues. It also is the easiest way since you don't have to use any programs to swap with other PSO2 Lobby music.

I'm trying to put custom music into the lobby, however I'm at a big of a snag here. I've noticed HCA2WAV converts HCA files to WAV files, and when I convert the PSO2 Discs/Lobby music into WAV it works, however, I notice that the file extension for these PSO2Discs/Lobby music is non existent. So, it's like, yeah, the HCA2WAV program works, but I'm entirely confused why the folders in the PSO2 folder have no extension at all.

I figured they were HCA files somehow, so I did, in-fact, find a program that does the opposite, convert WAV2HCA, and when it's done it has a .HCA file extension instead of a blank extension. So, just playing it in the win32 folder for swapping out music simply is not enough, I have to entirely remove the .HCA extension to make it a blank extension to replace PSO2 Disc/Lobby music.

So here's where I'm snagged. When I rename a .MP3 file or a .WAV file with the name of a PSO2 Disc/Lobby file's name and remove the extension, no sound happens at all, when I do it with .HCA file, sound does in-fact happen. However, the sound is jarbled and it sounds like the numbers falling down in the Matrix movies. It doesn't sound like a song or anything.

Any help with this?