Jump to content

psy_commando

Innovator
  • Posts

    425
  • Joined

  • Last visited

  • Days Won

    1

Everything posted by psy_commando

  1. Ok, I got a full verbose version out : https://dl.dropboxusercontent.com/u/13343993/my_pmd_utilities/ppmd_kaoutil_0.42_FIXED_VERBOSE.7z Just add -v to the command line, and it will spit out a lot of verbose. And it will take at least 20x as long as it usually does to run. I recommend pipping the output to a file to make it a bit faster! I included the PDB too. Its not intended for release though. At least not for now.
  2. Just like that I can only think of a few things that would crash with an access violation due to the content of the file. 1. The file's pointer table got corrupted. 2. The file isn't the expected length. 3. There's an error in the compressed data of the images. 4. The palette before an image is missing in the file, but the header of the AT4PX is handled as being a palette. Though, the in-game image you've shown is similar to what I got when I corrupted the kaomado file manually while researching it. So, idk, it looks to me like some kind of corruption is going on here.. The game just aborts running the decompressor/image loading code and just displays the empty buffer. So whatever is keeping kaoutil from running is also keeping the game from doing its job.. I could see about generating more verbose output.. But the only place I clearly remember not wrapping in exception check is the decompressor/compressor, because it got incredibly slow with all those security checks, and corruptions is usually detected before it gets there.. In the meantime, here's the pdb file for kaoutil, maybe it'll reveal a bit more details? https://dl.dropboxusercontent.com/u/13343993/my_pmd_utilities/ppmd_kaoutil_pdb.7z
  3. You need to rebuild it, like specified in evandixon's explanations, and play with the rebuilt rom. Because the reason you don't see any changes, is because you're just playing the original unmodified rom..
  4. I ran the test, his kaomado file is identical to mine. And the kaomado I made on my side works perfectly when using a cheat code to get buneary on the team: https://dl.dropboxusercontent.com/u/13343993/Screenshot%202015-11-15%2021.53.36.png @psy_commando: Are you sure you're loading the correct rom in Desmule? And are you sure rebuilding the rom actually worked ?
  5. Nice So, does 0.42 works after you deleted the zip, or is it still crashing? I guess I might not be checking whether a file is actually a directory and not a file.. sigh.. Getting feedback is nice https://www.gimp.org/downloads/ EDIT: Also all the portraits in the zip you linked are 24bpp. I don't know if you intended to make them 4bpp, or just wanted to post them like that though.
  6. Ok now its really fixed This time I trashed the crappy output path checker I wrote in kaoutil: https://dl.dropboxusercontent.com/u/13343993/my_pmd_utilities/ppmd_kaoutil_0.42_FIXED.zip It should be good enough for release too, as the code the application runs is doing its own check later on.
  7. Ok, I found the problem.. Turns out its something really dumb.. I was checking if the target output kaomado file exists before writing the new kaomado file! And there was also an issue when specifying as output path a relative directory with a single level.. I really need to port kaoutil's functionalities to gfxcrunch ASAP. I'm using a much more robust and tested system for checking input/outputs.. Plus it won't need those dumb face names files and etc anymore Here's the temporary link to the fixed 0.42 version ! https://dl.dropboxusercontent.com/u/13343993/my_pmd_utilities/ppmd_kaoutil_0.42.zip I'll make a commit later.. But I currently seriously messed up my branches, so it will take me some more time to push the code and make a proper download link.. ^^; EDIT: Crap.. Now it crashes when the output kaomado file exists.. >_< There's something really strange going on, or I missed something really obvious..
  8. Yep, that's what I do usually. kaoutil shouldn't care whether the images in the folder are bmps or pngs, or a mix of either. So, if you didn't do anything to validate yourself, it should work. And compression doesn't matter, png++ is handling the compression automatically. I tried again, and I still can't see them.. Maybe its because you're a mod, and there's some kind of spambot protection on new users ? Because I get : "Invalid Attachment specified. If you followed a valid link, please notify the administrator"
  9. Apparently your attachments are invalid. I can't see them But, without looking at the images, the correct format for the images is 40x40 pixels, 4bpp (16 colors), png or bmp images. Also, there's a small catch. For the portraits, the first color in the palette is used as the "transparency color". Its usually set to absolute black, if I remember correctly. So that could be the problem too.
  10. I know, but I just want to make sure if its a bug or not. I never get any feedback, so its very hard to improve the tool. Which is why I was asking. But if you don't want to that's fine I guess..
  11. Just wondering, you had trouble unpacking or repacking the portraits ? Could you make a list of the steps you've followed ? Its just that I'm looking to get rid of any possible bugs, and possibly make the instructions a bit clearer in the future.
  12. Good news ! I managed to find a PSMD cia release that was decrypted! (after downloading about 1500 PSMD rom releases.. ) So, I can confirm that, internally everything is mostly the same as GTI! The game has still a lot of files named "paradise", and those are from GTI. Even the rom's "name" is still Paradise apparently. The scripts are organized in a similar fashion, and they're also simply compiled, not encrypted or anything else, I was able to decompile them the same way I did with GTI. Interestingly, there's a file named "shiren_arts_data_info.bin" that was added to the "pokemon" folder, maybe this could have to do with IQs skills possibly? Since they were originally from shiren, and this file looks like it has been ripped from a shiren game judging from the name.. (I'll try looking up EMD to see if that was also in there ) Anyways, I won't be digging too deep into this until I've played the game, but I'll try to get some research going in the meantime. I'd like to at least get a tool for extracting/repacking the faces along with FARC archives going, with source code and everything before PSMD is out in north america. Though with Fallout 4 coming out soon, that might get a bit delayed ^^; Not to mention, I'm making some progress with the tiles format in PMD2, and it would be awkward to drop everything to work on PSMD/GTI instead.
  13. ( I know, "backup games" sounds really cheesy, but I can't bring myself to use the term "piracy" because it makes me facepalm too hard and gives me a headache ) Yeah, but I'm getting really sick of screwing up everytimes some new exploit comes out. This is pretty much the 3-4th time I lock myself out of homebrew. Twice because I updated my firmware too soon. Then something else happened, and now I thought I finally took all precautions, but nope, I locked myself out again.. So something like cubic ninja might be ideal for me. But then, I really don't have $60 dollar to spend on that cubic ninja game without cutting in my budget somewhere else. The most annoying part is that a bunch of people have been artificially raising prices for CN on ebay, buying a lot of them at gamestop and etc for peanuts, and selling them for 50+ dollars on auction sites and amazon. And well, maybe in a week or two someone will find another entry point/fix the old ones, and I'd have spent $60 for nothing. And nope, there isn't anyone I know with an unlocked 3ds living close-by EDIT: Welp, buying Cubic Ninja just isn't an option anymore.. Only the most expensive cartridges are left for sale now. I definitely can't pay $130 for that ^^; I guess I could try to look around local pawn shops.. But I got the feeling that it will probably be a waster of time..
  14. Well, the sky3ds is twice the price of cubic ninja, $120. Its on sale at $88 right now though. But still that's much more expensive, and I don't really play backup game so much, so it would be mostly useless to me..
  15. Yeah I saw it.. But I'm really unlucky/stupid, and now ironhax won't launch at all.. ;_; I don't even know why. And I can't reinstall it because browserhax and youtubehax are dead. Something wrong with the savegame maybe.. I suspect I might have somehow forgotten to update the ironhax install, and made a firmware update. But, I remember it working on this firmware a while ago.. I'm considering skipping on fallout 4 to get enough to buy cubic ninja for $60 at this point..
  16. I've been trying to see if I couldn't find a way to get a more accurate export of the presets, but the soundfont format is too limited to support all cases.. And the big issue I got is that the volume envelope in the soundfont format are all logarithmic. While those in DSE are linear.. That makes the sound go quieter much earlier in the curve, and the tracks were written to compensate for the linear volume ramps.. So, I've been considering "baking" the samples with the full DSE volume envelope, beside the release phase. But I can't bake the LFO effects unfortunately, because it wouldn't work properly while the loop is played.. I'm also considering writing my own player to play/exporter the tracks properly in higher quality than the NDS, with some proper interpolation and etc.. In other news I finally got the courage to setup a testing setup for editing map and script stuff in Explorers of Time (because the debug menu is much easier to access in it). And I'm slowly figuring out what the file formats do. For now, I can say that the BPL files are for storing map tiles palettes for map tilesets, the BMA contain tile placement data and decide the size of the map, the BPC contains the actual tileset images and tile assembly info. I'm still poking at the format, changing values and seeing what happens in the map viewer. But hopefully soon we'll have a better understanding of the map tileset format !
  17. psy_commando

    DSE SWDL Format

    General Information SWDL containers are used to contain sample and programs/presets information for any accompanying SEDL or SMDL files. They can be used in a few ways. To accompany a SMDL, and contain both the samples it uses and the data for the programs it uses. To accompany a SMDL, and contain only the program/preset data while referring to a main sample bank for the samples it uses. As a sample bank. When used in the second manner, it allows the game to only load the samples it actually uses from the main bank, and it keeps redundancy to a minimum. SWDLs also seems to override some of the data they contains. So, if a sample has its rootkey set to 60 in the main bank, and it was set to 80 in another SWDL referring to it, the rootkey actually used after loading that last SWDL would be 80. This works with a lot more parameters however. File Structure The file format is based around chunks, a bit like the RIFF file format. There doesn't seem to be a particular order to the chunks other than the header and the eod chunk. Overview[edit] Offset Length Name Description 0x0 80 SWDLHeader The container's header. - Varies wavi Chunk Contains details on all the samples contained, or referred to by the SWDL container. - Varies prgi Chunk The prgi chunk contains the programs/presets used by the SMDL files. (It may be omitted in SWDL purely for storing sample data.) - Varies kgrp Chunk The kgrp chunk contains information on every keygroups used in the SWDL by the programs/presets. (It may be omitted in SWDL purely for storing sample data.) - Varies pcmd Chunk The pcmd chunk contains the raw sample data for every samples contained in the file. (It may be omitted if the SWDL refers to a main bank for its sample data.) - 16 End of Data Chunk This empty chunk marks the end of the SWDL container. SWDL Header Total length 80 bytes Offset Length Type Name Description 0x00 4 char[4] magicn The 4 characters "swdl" {0x73, 0x77, 0x64, 0x6C} 0x04 4 - unk18 4 bytes of zeroes. 0x08 4 uint32 flen File length in bytes. 0x0C 2 uint16 version? Version number? ( 0x1504 ) 0x0E 1 uint8 unk1 Unknown. 0x0F 1 uint8 unk2 Unknown. 0x10 4 - unk3 4 bytes of zeroes. 0x14 4 - unk4 4 bytes of zeroes. 0x18 2 uint16 year Year the file was last modified. 0x1A 1 uint8 month Month the file was last modified. 0x1B 1 uint8 day Day the file was last modified. 0x1C 1 uint8 hour Hour the file was last modified. 0x1D 1 uint8 minute Minute the file was last modified. 0x1E 1 uint8 second Second the file was last modified. 0x1F 1 uint8 centisecond? Could possibly be the centisecond that the file was last modified. 0x20 16 char[16] fname Filename, ASCII null terminated string. Any extra space after the 0 on the total 16 bytes, is padded with 0xAA. 0x30 4 uint32 unk10 Always 0x00AA AAAA 0x34 4 uint32 unk11 4 bytes of zeroes. 0x38 4 uint32 unk12 4 bytes of zeroes. 0x3C 4 uint32 unk13 So far always 0x10 0x40 4 uint32 pcmdlen Length of "pcmd" chunk if there is one. If not, is null! If set to 0xAAAA0000 (The 0000 may contains something else), the file refers to samples inside an external "pcmd" chunk, inside another SWDL ! 0x44 4 uint32 unk14 4 bytes of zeroes. 0x46 2 uint16 nbwavislots Numbers of sample pointer slots, empty or not, in the "wavi" chunk's "WavTable". 0x48 2 uint16 nbprgislots Numbers of presets pointer slots , empty or not, in the "prgi" chunk's "TablA". 0x4A 2 uint16 unk17 Unknown 0x4C 4 uint32 wavilen Length of "wavi" chunk. wavi Chunk The wavi chunk contains information on all the samples. Its what links the prgi chunk to the sample data within the local or external pcmd chunk. Its made up of two main parts: A pointer table, with a slot for every samples used globally. A table of sample information. Each non-null pointers in the first table points to a single sample information entry in the second table. Null pointers indicate unused, or missing samples, depending on the context. Offset Length Type Name Description 0x00 4 char[4] label "wavi" {0x77, 0x61, 0x76, 0x69} 0x04 2 uint16 unk1 Always 0. 0x06 2 uint16 unk2 Always 0x1504. 0x08 4 uint32 chunkbeg Seems to always be 0x10, possibly the start of the chunk data. 0x0C 4 uint32 chunklen Length of the chunk data. Begins counting right after this field 0x10 Varies - WabTable Array containing 2 bytes offsets from the beginning offset of WavTable to an entry in the SampleInfoTbl table! It may be null. After WavTable 0-15 - Padding Bytes 0xAA padding bytes to align the next part on 16 bytes. After Padding varies - SampleInfoTbl This table contains details on each sample entries in the "WavTable". WavTable The table of pointers to the sample info. Each pointers is 16 bits, and may be null. The nb of entries is set in the SWDL header. SampleInfoTb The table made up of sample info entries. Each entries are 64 bytes, thus no padding is ever needed between entries. Here's the format of a sample info entry: Offset Length Type Name Description 0x00 2 uint16 unk1 Entry marker? Always 0x01AA. 0x02 2 uint16 ID Index number from WavTable. Empty/null entries in WavTable are counted! 0x04 1 int8 ftune The pitch fine tuning in cents(?). 0x05 1 int8 ctune Coarse tuning, possibly in semitones(?). Default is -7. 0x06 1 int8 rootkey The MIDI note associated to the sample. (The note that the instrument sampled is playing) It doesn't seems to have any effect in the game. 0x07 1 int8 ktps Key Transpose. Is basically the difference between rootkey and 60. 0x08 1 int8 volume The volume of the sample. (0-127) 0x09 1 int8 pan The pan of the sample. (0-64-127) 0x0A 1 uint8 unk5 Possibly Keygroup parameter for the sample. Always 0x00. 0x0B 1 uint8 unk58 Unknown. Always 0x02. 0x0C 2 uint16 unk6 Always 0x0000. 0x0E 2 - unk7 0xAA padding. 0x10 2 uint16 unk59 Always 0x1504. 0x12 2 uint16 smplfmt Sample format. 0x0000 : Possibly 8 bits PCM 0x0100 : 16 bits PCM 0x0200 : 4 bits ADPCM 0x0300 : Possibly PSG ? 0x14 1 uint8 unk9 Often 0x09 0x15 1 uint8 smplloop Flag indicating whether the sample should be looped or not ! (1 = looped, 0 = not looped) 0x16 2 uint16 unk10 Often 0x0108 0x18 2 uint16 unk11 Often 0004. 0x1A 2 uint16 unk12 Often 0x0101. 0x1C 4 - unk13 Often 0x0000 0000. 0x20 4 uint32 smplrate Sample rate in hertz. 0x24 4 uint32 smplpos The offset of the sound sample in the "pcmd" chunk when there is one. Otherwise, possibly offset of the exact sample among all the sample data loaded in memory? (The value usually doesn't match the main bank's) 0x28 4 uint32 loopbeg The position in bytes divided by 4, the loop begins at, from smplpos. ( multiply by 4 to get size in bytes ) Adding loopbeg + looplen gives the sample's length ! (For ADPCM samples, the 4 bytes preamble is counted in the loopbeg!) 0x2C 4 uint32 looplen The length of the loop in bytes, divided by 4. ( multiply by 4 to get size in bytes ) Adding loopbeg + looplen gives the sample's length ! 0x30 1 uint8 envon If not == 0, the volume envelope is processed. Otherwise its ignored. 0x31 1 uint8 envmult If not == 0, is used as multiplier for envelope paramters, and the 16bits lookup table is used for parameter durations. If 0, the 32bits duration lookup table is used instead. This value has no effects on volume parameters, like sustain, and atkvol. 0x32 1 uint8 unk19 Unknown. Usually 0x1 0x33 1 uint8 unk20 Unknown. Usually 0x3 0x34 2 uint16 unk21 Unknown. Usually 0x03FF ( Little endian -253) 0x36 2 uint16 unk22 Unknown. Usually 0xFFFF 0x38 1 int8 atkvol Sample volume envelope Attack Level.(0 to 127) Higher values towards 0x7F means the volume at which the attack phase begins at is louder. Doesn't shorten the attack time. 0x39 1 int8 attack Sample volume envelope Attack.(0 to 127) Higher values towards 0x7F means the attack phase takes longer to reach full volume! 126 is ~10 seconds. 0x3A 1 int8 decay Sample volume envelope Decay. (0 to 127) The duration the note has to be held until the volume is smoothly decreased to the value of "Sustain Volume". Higher values towards 0x7F means it takes longer before the held note's volume changes to "Sustain Volume". 0x3B 1 int8 sustain Sample volume envelope Sustain.(0 to 127) The volume at which the held note will stay at. (Default 0x7F) 0x3C 1 int8 hold Sample volume envelope Hold (0 to 127). Higher values towards 0x7F means the note is held at full volume longer after the attack phase. 126 is ~10 seconds. 0x7F, does the same as 0. 0x3D 1 int8 decay2 Sample volume envelope Decay2 (0 to 127). Higher values towards 0x7F means longer fade-out. 0x7F means never fade-out. (Default 0x7F) At 0x7E, it takes ~8 seconds for the volume to reach 0. 0x3E 1 int8 release Sample volume envelope Release parameter(0 to 127). Higher values towards 0x7F means longer release. Negative values mirror positive range. (Default is 0x28(40)) 0x3F 1 int8 unk57 Unknown. Usually 0xFF. prgi Chunk The prgi chunk contains programs/presets that the SMDL music sequences use as instrument presets in their tracks. Its made up of : A table of pointers to all the programs info entries. Some may be null. A table containing program info entries. The pointer table works in the exact same way as it does in the wavi chunk. 16 bits offsets from the beginning of the table to a program info entry. Offset Length Type Name Description 0x00 4 char[4] label "prgi" {0x70, 0x72, 0x67, 0x69} 0x04 2 uint16 unk1 Always 0. 0x06 2 uint16 unk2 Always 0x1504. 0x08 4 uint32 chunkbeg Seems to always be 0x10, possibly the start of the chunk data. 0x0C 4 uint32 chunklen Length of the chunk data. Begins counting right after this field 0x10 (nbprgislots * 2) + padding - ProgramPtrTbl A table of 16 bits pointers to entries in the ProgramInfoTbl. Some may be null. It usually has 128 slots. Like General Midi. If the nb of presets were to change, its possible there would be a need for padding bytes, seeing how the wavi chunk works. After ProgramPtrTbl Varies ProgramInfo[nbprgislots] ProgramInfoTbl A table containing information on all the presets available in the current SWDL. ProgramInfoTbl This table contains entries for every single presets available in the SWDL. Each entry is pointed to by a pointer in the ProgramPtrTbl. It contains ProgramInfo entries: ProgramInfo A ProgramInfo entry is minimum 144 bytes long. Its made of 3 parts: The program info header. The LFO table. The split table. The program info header contains details for identifying the preset, and the size of the LFO and split table. Offset Length Type Name Description 0x00 2 uint16 ID Index of the pointer in "TableA" that points to this entry. Also correspond to the program ID used in the corresponding SMDL file! 0x02 2 uint16 nbsplits Nb of samples mapped to this presets, in the split table. 0x04 1 int8 prgvol Volume of the entire program. 0x05 1 int8 prgpan Pan of the entire program. (0-127, 64 is middle, 127 is full right, 0 is full left ) 0x06 1 uint8 unk3 Most of the time 0x00. 0x07 1 uint8 thatFbyte Most of the time 0x0F. 0x08 2 uint16 unk4 Most of the time is 0x200. 0x0A 1 uint8 unk5 Most of the time is 0x00. 0x0B 1 uint8 nblfos Nb of entries in the LFO Table. 0x0C 1 uint8 PadByte Most of the time is 0xAA, or 0x0. The value here is used as the delimiter and padding ! 0x0D 1 uint8 unk7 Most of the time is 0x0. 0x0E 1 uint8 unk8 Most of the time is 0x0. 0x0F 1 uint8 unk9 Most of the time is 0x0. 0x10 (nblfos * 16) LFOEntry[nblfos] LFOTbl Table that contains details on how to use the 4 LFOs with this preset. After LFOTbl 16 - Delimiter 16 bytes of "PadByte" padding bytes, possibly to delimit the start of the section below. Uses the value of PadByte as padding value! After Delimiter (nbsplits * 48) SplitEntry[nbsplits] SplitsTbl Table of samples splits mapped to this program. LFOEntry These determine how to configure the 4 Low Frquency Oscillators (LFO) linked to this program. It allows to set the shape of the waveform/the function that generate the value. The output of the LFO, its frequency, depth, delay, fade-out. And possibly more. Here's the structure of an entry: Total Length 16 bytes Offset Length Type Name Description 0x00 1 uint8 unk34 Unknown, usually 0x00. It does seems to have an effect with a certain combination of other values in the other parameters. 0x01 1 uint8 unk52 Unknown, usually 0x00. Most of the time, this value is 1 when the LFO is in use. 0x02 1 uint8 dest The destination of the LFO's output. 0x0 : none/disabled 0x1 : pitch 0x2 : volume 0x3 : pan 0x4 : lowpass/cutoff filter? 0x03 1 uint8 wshape The shape/function of the waveform. (When the LFO is disabled, its always 1) Square Triangle? Sinus? ? Saw? Noise? Random 0x04 2 uint16 rate Rate at which the LFO "oscillate". May or may not be in Hertz. 0x06 2? uint16? unk29 Changing the value seems to induce feedback or resonance. (Or perhaps its because it ended up corrupting the sound engine state when messing with the parameter?) 0x08 2 uint16 depth The depth parameter of the LFO. 0x0A 2 uint16 delay Delay in milliseconds before the LFO effect is applied after the sample begins playing. 0x0C 2 uint16 unk32 Unknown, usually 0x0000. Possibly fade-out in milliseconds. 0x0E 2 uint16 unk33 Unknown, usually 0x0000. Possibly an extra parameter? Or a cutoff/lowpass filter's frequency cutoff? SplitEntry This represents a sample mapped to the preset. Those are played depending on certain conditions when a playnote event is received by the sequencer for that particular preset/program. Some samples may be played only for a certain range of keys or velocities for example. Offset Length Type Name Description 0x00 1 - unk10 A leading 0. 0x01 1 uint8 id The Index of the sample in the SplitsTbl! 0x02 1 uint8 unk11 Unknown. Is always the same value as offset 0x1A below ! (Possibly "bend range" according to assumptions made from the DSE screenshots) 0x03 1 uint8 unk25 Unknown. Possibly a boolean. 0x04 1 int8 lowkey Usually 0x00. Lowest MIDI Key this sample can play on. 0x05 1 int8 hikey Usually 0x7F. Highest MIDI Key this sample can play on. 0x06 1 int8 lowkey2 A copy of lowkey, for unknown purpose. 0x07 1 int8 hikey2 A copy of hikey, for unknown purpose. 0x08 1 int8 lovel Lowest note velocity the sample is played on.(0 - 127) 0x09 1 int8 hivel Highest note velocity the sample is played on.(0 - 127) 0x0A 1 int8 lovel2 A copy of lovel, for unknown purpose. Usually 0x00. 0x0B 1 int8 hivel2 A copy of hivel, for unknown purpose. Usually 0x7F. 0x0C 4 - unk16 Usually the same value as "PadByte", or 0. Possibly padding ? 0x10 2 - unk17 Usually the same value as "PadByte", or 0. Possibly padding ? 0x12 2 uint16 SmplID The ID/index of sample in the "wavi" chunk's lookup table. 0x14 1 int8 ftune Fine tune. The fine tune in cents. 0x15 1 int8 ctune Coarse tuning. Default is -7. 0x16 1 int8 rootkey Note at which the sample is sampled at ! 0x17 1 int8 ktps Key Transpose. Is basically the difference between rootkey and 60. 0x18 1 int8 smplvol Volume of the sample. 0x19 1 int8 smplpan Pan of the sample. 0x1A 1 uint8 kgrpid Keygroup ID of the keygroup this split belongs to! 0x1B 1 uint8 unk22 Unknown, possibly a flag. Usually 0x02. 0x1C 2 uint16 unk23 Unknown, usually 0000. 0x1E 2 - unk24 Usually the same value as "PadByte", or 0. Possibly padding ? Those last 16 bytes are for the volume envelope. They override the sample's original volume envelope! 0x20 1 uint8 envon If not == 0, the volume envelope is processed. Otherwise its ignored. 0x21 1 uint8 envmult If not == 0, is used as multiplier for envelope paramters, and the 16bits lookup table is used for parameter durations. If 0, the 32bits duration lookup table is used instead. This value has no effects on volume parameters, like sustain, and atkvol. 0x22 1 uint8 unk37 Unknown. 0x23 1 uint8 unk38 Unknown. 0x24 2 uint16 unk39 Unknown. 0x26 2 uint16 unk40 Unknown. 0x28 1 int8 atkvol Sample volume envelope Attack Level.(0 to 127) Higher values towards 0x7F means the volume at which the attack phase begins at is louder. Doesn't shorten the attack time. 0x29 1 int8 attack Sample volume envelope Attack.(0 to 127) Higher values towards 0x7F means the attack phase takes longer to reach full volume! 126 is ~10 seconds. 0x2A 1 int8 decay Sample volume envelope Decay. (0 to 127) The duration the note has to be held until the volume is smoothly decreased to the value of "Sustain Volume". Higher values towards 0x7F means it takes longer before the held note's volume changes to "Sustain Volume". 0x2B 1 int8 sustain Sample volume envelope Sustain.(0 to 127) The volume at which the held note will stay at. (Default 0x7F) 0x2C 1 int8 hold Sample volume envelope Hold (0 to 127). Higher values towards 0x7F means the note is held at full volume longer after the attack phase. 126 is ~10 seconds. 0x7F, does the same as 0. 0x2D 1 int8 decay2 Sample volume envelope Decay2 (0 to 127). Higher values towards 0x7F means longer fade-out. 0x7F means never fade-out. (Default 0x7F) At 0x7E, it takes ~10 seconds for the volume to reach 0. 0x2E 1 int8 release Sample volume envelope Release parameter(0 to 127). Higher values towards 0x7F means longer release. Negative values mirror positive range. (Default is 0x28(40)) 0x2F 1 int8 unk53 Usually 0xFF. kgrp Chunk The kgrp chunk contains a list of all the keygroups in use in this SWDL. Keygroups are used to allow finer control on how the audio engine allocates the limited amount of voices to the presets. They define a per group priority and a range of voice channel available to each, along with the maximum number of simultaneous voice channels each can used. Offset Length Type Name Description 0x00 4 char[4] label "kgrp" {0x6B, 0x67, 0x72, 0x70} 0x04 2 uint16 unk1 Always 0. 0x06 2 uint16 unk2 Always 0x1504. 0x08 4 uint32 chunkbeg Seems to always be 0x10, possibly the start of the chunk data. 0x0C 4 uint32 chunklen Length of the chunk data. Begins counting right after this field 0x10 Varies Keygroup[] Keygroups A table containing all the keygroups used in the SWDL. The first entry is usually the global Keygroup, of which most splits are part of. After Keygroups 0 or 8 - Padding? When there is an odd number of Keygroup entry, it appears there is some garbage(?) inserted here to make the next chunk start on an offset divisible by 16. Keygroup A single entry in the Keygroups table. This is meant to mitigate issues with the limited polyphony of the NDS. Basically, this is used to tell what sets of instruments gets to play notes over the others. (Wikipedia has an excellent article on synthesizer polyphony here: https://en.wikipedia.org/wiki/Polyphony_and_monophony_in_instruments ) The polyphony parameter allows to set how many simultaneous voices a member of the keygroup may use. The vc high and vc low parameters defines the range out of all the available voice channels that members of the group may use. The priority parameter is meant to tell which keygroups has priority over using a voice channel. For example, lets say that a split "splitA" is a member of a priority 1 keygroup, and another split "splitB" of a priority 15 keygroups, and there is no voice channel available to use as they're all being used. SplitB needs to play a note, but splitA is currently playing one. Then SplitA will be cut abruptly, and its voice channel will be reassigned to splitB so it can play its note! Keygroup 0 is the global keygroup everything uses by default, and its always there. Total Length 8 bytes Offset Length Type Name Description 0x00 2 uint16 ID Index/ID of the keygroup. 0x02 1 uint8 poly Polyphony. AKA max number of simultaneous notes played. 0 to 15. -1 means disabled. 0x03 1 uint8 priority Priority over the assignment of a voice channel for the members of this group. A value from 0 to possibly 99. Default is 8. Lower values means lower priority, while higher means higher priority. 0x04 1 uint8 vclow Lowest voice channel of the range this group may use. Usually between 0 to 15. 0x05 1 uint8 vchigh Highest voice channel this group may use. Usually between 0 to 15. 0x06 1 uint8 unk50 Unknown. 0x07 1 uint8 unk51 Unknown. pcmd Chunk The pcmd chunk contains the sample data for every samples. Each samples is stored one after the other, regardless of the sample type or sample rate, without headers or delimiters of any sort! Each samples is located by using the sample entries in the wavi chunk. Offset Length Type Name Description 0x00 4 char[4] label "pcmd" {0x70, 0x63, 0x6D, 0x64} 0x04 2 uint16 unk1 Always 0. 0x06 2 uint16 unk2 Always 0x1504. 0x08 4 uint32 chunkbeg Seems to always be 0x10, possibly the start of the chunk data. 0x0C 4 uint32 chunklen Length of the chunk data. Begins counting right after this field 0x10 Varies - SampleDataBlob Contains the sample data for all the samples used in the SWDL. The data can be stored in any of the compatible formats: The NDS's 4 bits IMA ADPCM encoding (Same as official IMA ADPCM. Even uses the same tables. only the way sample values are clamped when converting back to PCM16 differ a tiny bit, which might not even be noticeable.) raw PCM16 samples. raw PCM8 samples. Possibly PSG or something else ? Note About ADPCM: Each ADPCM sample begins with the initial value of the "predictor" on 16 bits. Then the "step index", also on 16 bits. Afterwards, comes the actual sample data. The ADPCM preamble is included in the value of the "loopbeg" parameter of the sample! Eod Chunk This chunk marks the the end of the SWDL container/file. There is nothing past the chunk header! Total Length 16 bytes Offset Length Type Name Description 0x00 4 char[4] label "eod\20" {0x65, 0x6F, 0x64, 0x20} 0x04 2 uint16 unk1 Always 0. 0x06 2 uint16 unk2 Always 0x1504. 0x08 4 uint32 chunkbeg Seems to always be 0x10, possibly the start of the chunk data. 0x0C 4 uint32 chunklen Length of the chunk data. Begins counting right after this field. Always 0 for eod chunk! Volume Envelopes Here is some more information on how volume envelopes are parsed. In PMD2 Explorers of Sky, the north american version, here are the routines at runtime in memory that handle parsing the envelopes: 0x02074E0C : Code that picks an envelope's starting phase. 0x02074F54 : Code that handles updating the envelope's state. 0x02074D58 : Code handling lerping the volume values for any of the envelope parameters indicating a duration. The parameters indicating a duration in the envelopes are used as indices in one of 2 lookup tables to get an actual duration. If the "envmult" parameter of the envelope is not 0, the 16 bits lookup table is used, while if its 0, the 32 bits lookup table is used. Here's a heavily commented and re-organized assembly snippet to demonstrate how the duration lookup tables are used, and how a single parameter is parsed: Fun_02074D58( R0(SplitEnvelopeAddress), R1(TargetVolume), R2(EnvelopeParam) ) { 02074D58 E92D4038 push r3-r5,r14 02074D5C E1A05000 mov r5,r0 02074D60 E1A04001 mov r4,r1 02074D64 E352007F cmp r2,7Fh //See if the envelope parameter is == 0x7F 02074D68 1A000004 bne 02074D80 if( EnvelopeParam != 0x7F ) { 02074D80 E5C5401D strb r4,[r5,1Dh] //Put the TargetVolume here 02074D84 E5D53001 ldrb r3,[r5,1h] //Load envmult 02074D88 E3530000 cmp r3,0h 02074D8C 1A000007 bne 02074DB0 if( envmult != 0 ) { 02074DB0 E59F0050 ldr r0,=20B0F50h //16 bits lookup table for durations 02074DB4 E1A01082 mov r1,r2,lsl 1h //R1 = EnvelopeParam << 1 (Basically, multiply by 2 the envelope param, so that we get a byte offset in a 16bits integer array) 02074DB8 E19020B1 ldrh r2,[r0,r1] //Get the duration from the table 02074DBC E59F1040 ldr r1,=22B7330h //That's the static address of DSE driver's memory 02074DC0 E3A00FFA mov r0,3E8h 02074DC4 E0020293 mul r2,r3,r2 //We multiply the duration with envmult (the envelope duration multiplier) 02074DC8 E0000092 mul r0,r2,r0 //We multiply our duration by 1000 02074DCC E1D112F8 ldrsh r1,[r1,28h] //This address always contains the value 10000 02074DD0 EB006C33 bl 0208FEA4 //(Division signed R0 = R0 / R1) //Continues to 02074DD4.. } else { 02074D90 E59F0068 ldr r0,=20B1050h //32 bits lookup table for durations 02074D94 E59F1068 ldr r1,=22B7330h //That's the static address of DSE driver's memory 02074D98 E7902102 ldr r2,[r0,r2,lsl 2h] //Multiply the EnvelopeParam by 4( EnvelopeParam << 2 ) to get the byte offset in a int32 array 02074D9C E3A00FFA mov r0,3E8h 02074DA0 E0000092 mul r0,r2,r0 //We multiply our duration by 1000 02074DA4 E1D112F8 ldrsh r1,[r1,28h] //This address always contains the value 10000 02074DA8 EB006CC0 bl 020900B0 //(Unsigned division R0 = R0 / R1) 02074DAC EA000008 b 02074DD4 //Continues to 02074DD4.. } 02074DD4 E5850018 str r0,[r5,18h] //Put EnvelopePhaseTimeLeft here 02074DD8 E5951018 ldr r1,[r5,18h] //Read it back into R1 02074DDC E3510000 cmp r1,0h if( EnvelopePhaseTimeLeft == 0 ) { 02074DE0 03A00000 moveq r0,0h 02074DE4 05850014 streq r0,[r5,14h] //Set VolumeIncrement to 0 02074DE8 08BD8038 popeq r3-r5,r15 //Return } 02074DEC E5950010 ldr r0,[r5,10h] //Load EnvPhaseCurrentVolume 02074DF0 E0600B84 rsb r0,r0,r4,lsl 17h // (TargetVolume << 0x17) - EnvPhaseCurrentVolume, or (2^23 * TargetVolume) - EnvPhaseCurrentVolume 02074DF4 EB006C2A bl 0208FEA4 //(Division signed R0 = R0 / R1) Divide volume difference between TargetVolume and EnvPhaseCurrentVolume by the EnvelopePhaseTimeLeft 02074DF8 E5850014 str r0,[r5,14h] //Set VolumeIncrement 02074DFC E8BD8038 pop r3-r5,r15 } else { //Envelope param disabled 02074D6C E3A00000 mov r0,0h 02074D70 E5850014 str r0,[r5,14h] //Set the VolumeIncrement to 0 02074D74 E2400106 sub r0,r0,80000001h //This basically puts 0x7FFFFFFF into R0 (Aka the maximum signed, positive value for a signed 16bits integer ) 02074D78 E5850018 str r0,[r5,18h] //Put 0x7FFFFFFF as EnvelopePhaseTimeLeft 02074D7C E8BD8038 pop r3-r5,r15 } } Duration Lookup Tables The durations in the tables are in milliseconds, conveniently! 16 bits table located at 0x020B0F50: const int16_t Lookup_Table_20B0F50 [128] = { 0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007, 0x0008, 0x0009, 0x000A, 0x000B, 0x000C, 0x000D, 0x000E, 0x000F, 0x0010, 0x0011, 0x0012, 0x0013, 0x0014, 0x0015, 0x0016, 0x0017, 0x0018, 0x0019, 0x001A, 0x001B, 0x001C, 0x001D, 0x001E, 0x001F, 0x0020, 0x0023, 0x0028, 0x002D, 0x0033, 0x0039, 0x0040, 0x0048, 0x0050, 0x0058, 0x0062, 0x006D, 0x0078, 0x0083, 0x0090, 0x009E, 0x00AC, 0x00BC, 0x00CC, 0x00DE, 0x00F0, 0x0104, 0x0119, 0x012F, 0x0147, 0x0160, 0x017A, 0x0196, 0x01B3, 0x01D2, 0x01F2, 0x0214, 0x0238, 0x025E, 0x0285, 0x02AE, 0x02D9, 0x0307, 0x0336, 0x0367, 0x039B, 0x03D1, 0x0406, 0x0442, 0x047E, 0x04C4, 0x0500, 0x0546, 0x058C, 0x0622, 0x0672, 0x06CC, 0x071C, 0x0776, 0x07DA, 0x0834, 0x0898, 0x0906, 0x096A, 0x09D8, 0x0A50, 0x0ABE, 0x0B40, 0x0BB8, 0x0C3A, 0x0CBC, 0x0D48, 0x0DDE, 0x0E6A, 0x0F00, 0x0FA0, 0x1040, 0x10EA, 0x1194, 0x123E, 0x12F2, 0x13B0, 0x146E, 0x1536, 0x15FE, 0x16D0, 0x17A2, 0x187E, 0x195A, 0x1A40, 0x1B30, 0x1C20, 0x1D1A, 0x1E1E, 0x1F22, 0x2030, 0x2148, 0x2260, 0x2382, 0x2710, 0x7FFF }; 32 bits table located at 0x20B1050: const int32_t Lookup_Table_20B1050 [128] = { 0x00000000, 0x00000004, 0x00000007, 0x0000000A, 0x0000000F, 0x00000015, 0x0000001C, 0x00000024, 0x0000002E, 0x0000003A, 0x00000048, 0x00000057, 0x00000068, 0x0000007B, 0x00000091, 0x000000A8, 0x00000185, 0x000001BE, 0x000001FC, 0x0000023F, 0x00000288, 0x000002D6, 0x0000032A, 0x00000385, 0x000003E5, 0x0000044C, 0x000004BA, 0x0000052E, 0x000005A9, 0x0000062C, 0x000006B5, 0x00000746, 0x00000BCF, 0x00000CC0, 0x00000DBD, 0x00000EC6, 0x00000FDC, 0x000010FF, 0x0000122F, 0x0000136C, 0x000014B6, 0x0000160F, 0x00001775, 0x000018EA, 0x00001A6D, 0x00001BFF, 0x00001DA0, 0x00001F51, 0x00002C16, 0x00002E80, 0x00003100, 0x00003395, 0x00003641, 0x00003902, 0x00003BDB, 0x00003ECA, 0x000041D0, 0x000044EE, 0x00004824, 0x00004B73, 0x00004ED9, 0x00005259, 0x000055F2, 0x000059A4, 0x000074CC, 0x000079AB, 0x00007EAC, 0x000083CE, 0x00008911, 0x00008E77, 0x000093FF, 0x000099AA, 0x00009F78, 0x0000A56A, 0x0000AB80, 0x0000B1BB, 0x0000B81A, 0x0000BE9E, 0x0000C547, 0x0000CC17, 0x0000FD42, 0x000105CB, 0x00010E82, 0x00011768, 0x0001207E, 0x000129C4, 0x0001333B, 0x00013CE2, 0x000146BB, 0x000150C5, 0x00015B02, 0x00016572, 0x00017015, 0x00017AEB, 0x000185F5, 0x00019133, 0x0001E16D, 0x0001EF07, 0x0001FCE0, 0x00020AF7, 0x0002194F, 0x000227E6, 0x000236BE, 0x000245D7, 0x00025532, 0x000264CF, 0x000274AE, 0x000284D0, 0x00029536, 0x0002A5E0, 0x0002B6CE, 0x0002C802, 0x000341B0, 0x000355F8, 0x00036A90, 0x00037F79, 0x000394B4, 0x0003AA41, 0x0003C021, 0x0003D654, 0x0003ECDA, 0x000403B5, 0x00041AE5, 0x0004326A, 0x00044A45, 0x00046277, 0x00047B00, 0x7FFFFFFF };
  18. Ok, fixed the looping bug : https://github.com/PsyCommando/ppmdu/releases/tag/ppmd_audioutil_0.11 EDIT: Also created a wiki page for the SWDL format finally ! http://projectpokemon.org/wiki/Dse_swdl The SMDL page will feel a bit less lonely now ! EDIT2: Fixed tracks without a loop point being looped: https://github.com/PsyCommando/ppmdu/releases/tag/ppmd_audioutil_0.12
  19. I put together a pre-alpha release of ppmd_audioutil ! Its pretty limited right now. - It can extract the pmd2 tracks to MIDI files + a soundfont. (I recommend using foobar2000 with the midi plugin to automatically play those with the correct soundfont from the same folder, so you don't have to manually change the soundfont each time in bassmidi or whatever you're using : foobar2000 player: http://www.foobar2000.org/ midi plugin : http://www.foobar2000.org/components/view/foo_midi ) - It can export samples from a SWDL file. - It can export single SMDL files as-is. - It can export singel SMDL files according to conversion rules defined in a XML file. (example included!) However right now it cannot: - Export SEDLs. - Convert MIDIs to SMDLs + SWDLs. - Build SWDL sample banks. - Build SEDLs. And the MIDI export + Soundfont export is not 100% accurate yet. - Duration for notes are still guestimates I got through trial and errors. - Volume values are simply a proportion of 20 decibel, which is most likely inacurate. - A lot of events are unsuported, but most of the events used in the PMD2 soundtrack are supported. - The soundfont format doesn't let you tweak its LFOs or redirect them. While DSE support up to 4 LFO effects with a basic delay and fade-out. No phaser, no affecting the volume via LFO, etc.. - The soundfont format doesn't support a second decay from sustain volume to complete silence while the key is held. - Still haven't figured out the pitch correction parameters yet, so some samples sound off. - No resonance applied on the samples yet. Here's the readme: The midis exported use the GS standard, and contains GS SysEx messages to make channel 10 a regular track not dedicated to drums. The soundfont is filled sequentially, so there is no order or labels for each presets, and since each tracks in the game has its own "soundfont" there are a lot of repetition, with small changes. If someone would make a list of what each tracks use, and give them a name, it would be possible to export a proper soundfont. I don't really have time for that honestly.. Individually exported MIDIs by default use the drumkit located in Bank 127, Preset 1. So you'll need a soundfont that has drumkits in bank 127 to listen to those. The XML file contains a list of the presets used in each tracks, and what MIDI preset and bank to use for each in the exported MIDI. It also allows to remap certain keys for drumkits. It contains only the data for 4 tracks from PMD2:EoS right now, and anyone is welcome to contribute to it so we can make it a bit more complete hopefully! (To clear things up, that doesn't affect exporting to MIDI+soundfont, only when exporting to a midi using the XML file for conversion) Here's the download link on my github : https://github.com/PsyCommando/ppmdu/releases/tag/ppmd_audioutil_0.1 (see post below for latest fixed version!) EDIT: For some reasons when export the midis from pmd2, they won't loop properly.. There is a delay between loops. I'll look into it..
  20. Thanks ! I gathered most of that already though, but every little bits helps ! Also, I haven't updated here in a while, but I made some pretty big progress. I don't have much to show for it though.. I added BGP support to GFXCrunch, but I still need to fix a lot of little ongoing issues with the program to continue working on it and adding features. Also, I figured out how the LFO work more or less. Have a 20 mins video of me changing the LFO values : ^^;I made a pretty big breakthrough in exporting the midis from the game. The main issue was that I coded most of that at around 3am, and it really did show.. So I recoded some parts of it, and it fixed issues with presets not being assigned correctly, which in turns mean that volume envelopes worked more like expected. Exporting to the soundfont format is going to be lossier than expected though.. Soundfont doesn't support a lot of features that DSE does. Like LFOs and priority/polyphony groups.. I could fake some of those things using midi, but a lot of advance features like filters or even just setting a frequency or delay+fadeout on a LFO based effect is not possible because its implementation specific.. We might have to write a mini DAW to work with DSE stuff.. Or maybe I could just export XML data.. We'll have to see.. I'm hoping to resume work on statsutil soon, but that huge breakthrough was too much to let go just now Here's a sample of what audioutil can export now: https://dl.dropboxusercontent.com/u/13343993/my_pmd_research_files/converted_with_audioutil/Oct10/21_B_DUN_SHIMETTAI.mp3 Its not perfect yet, but its close. And here's one of the more problematic tracks : https://dl.dropboxusercontent.com/u/13343993/my_pmd_research_files/converted_with_audioutil/Oct10/45_B_DUN_ISONODOUK.mp3 There is a sample that is completely off-pitch and I'm not sure why. I guess its probably linked to the 3 pitch-changing paramters in the SWD file. But I'm not sure what they represent.. The default pitch value is 249 for some reason.. I updated my notes with the details: https://dl.dropboxusercontent.com/u/13343993/my_pmd_research_files/PMD2_MusicAndSoundFormats.txt And also, here are some of my really messy notes listing all the DSE events : https://dl.dropboxusercontent.com/u/13343993/my_pmd_research_files/SMD_FileParsingCode.txt If anyone has the time to look at those and figure out what the unknown ones do, it would be greatly appreciated !
  21. Sorry, I missed your reply.. >_< I sent you 3 PMs, so you should have received at least one of them. That's odd You might want to check in your profile options if you have PMs on, or if you allow receiving PMs from others: http://projectpokemon.org/forums/profile.php?do=editoptions
  22. That's all good. I just mentioned it because I keep forgetting things like that, and I wish people would remind me
  23. Alright, sorry for asking. I just feel like I might have sent the wrong message to people in the past, and I just wanted to make sure it wasn't the case here. ^^; Hmm.. Maybe its not utf-16 then.. From what I've read around the web, 16 bits characters may also contain utf-8. Mainly because people tend to not follow a standard implementation. Have you tried parsing everything as utf-8? Also, you might want to just write escaped characters, for those you can't parse. It might make things easier when parsing it back. I mean, writing it as a number prefixed with a "\u" or "\U" and etc, instead of using your own syntax. https://docs.python.org/2/howto/unicode.html#unicode-literals-in-python-source-code There are more chances that it would be compatible with other software and python itself, as its a pretty common way to escape unicode characters. Also, now that you bring that up, its possible PMD2 is actually encoded in unicode too, which would explain some of the multi-byte symbols.. http://unicode.org/faq/utf_bom.html
  24. That looks like some nice work! I'm always glad to see some more utilities for PMD2 popping up! And I'm also happy that all that info on the wiki got some use! It was a pain in the butt to write And nice find with the Explorers of Time/Darkness thread on gamefaqs. I didn't even think about looking for that one, even though I read through the EoS one.. ^^; I'll probably have to update the wiki and etc now.. And by the way, if you ever want to discuss about PMD2 research or want to share some discoveries, etc, my PMD2 thread has pretty much become a PMD2 research thread Its always nice to get some input from other people working on PMD2 ! http://projectpokemon.org/forums/showthread.php?40199-Pokemon-Mystery-Dungeon-2-Psy_commando-s-Tools-and-research-notes There are a lot of little things I haven't documented on the wiki yet, so feel free to ask if you ever need to know anything! Its possible one of us will have an answer. EDIT: Also, I looked a bit at how your tool worked. And if you don't mind, I had just a comment. Because of the way the move lists are encoded, the size of the waza_p.bin file will increase/decrease in the rom depending on the amount of moves lists containing IDs bigger than 0x7F. That could be a problem if a lot of move IDs are changed that way, it could in theory start overwriting data after the waza_p.bin file. You might want to unpack and repack the rom with ndstool, or use some code to modify the FAT of the nitrofs. Just to avoid any potentially random bugs with modifying stuff directly via file offsets. Plus, that way you'd be able to add/remove moves from the level-up lists randomly without issues. And the randomizer would be mostly compatible with modified roms and other regions roms. Its just a suggestion though.
×
×
  • Create New...