Jump to content


  • Content Count

  • Joined

  • Last visited

  • Days Won


Everything posted by psy_commando

  1. Hi ! I'm fairly new around here, and I'm relatively new to rom hacking and to the NDS scene in general, but I'm working on several little projects for Pokemon Mystery Dungeon Explorers of Sky / Time / Darkness, and I thought it would nice to share it with others ! Especially with others with more technical knowledge than I ! I hope this will be useful ! :biggrin: And be sure to report any bugs, or tidbits of information you have ! Those would prove invaluable ! Feel free to mention whether it worked for you, or not so much ! Any feedback is always great ! You might need 7zip or another 7z compatible program to open the compressed files on this page. Looking for: - Need people to write tutorials for some of the tools ! - Need feedback on the tools! - Need people to discover what some of the "unk" variables in the character sprites do ! - Need info on the NDS's DSP. The Tools: Compression/Decompression: PPMD UnPX Description: A quick little implementation of a "PX" compressed data decompressor. Its based on Zhorken's Python script for decompressing the character portraits, which happened to be stored as compressed "PX" files ! Just feed it an AT4PX or PKDPX file by drag and dropping, or use the command line argument for specifying the input file and output folder. The result will have an extension appended, or not, depending on whether its currently possible to guess the content. Again, many thanks to Zhorken for finding out about the compression scheme ! Donwload: Version 0.41 : ppmd_unpx_0_41.zip on Github PPMD DoPX Description: A compressor to compress files back into PX compressed files, either PKDPX or AT4PX files. Its still pretty new and experimental, so, watch out for bugs ! By default anything compressed is outputed to a PKDPX. To output to a AT4PX, the output filename has to be specified and to end with ".at4px" ! Donwload: Version 0.31 : ppmd_dopx_0_31.zip on Github Packing/Unpacking: PPMD Pack File Utility Description: This tool unpacks and re-packs "pack files". Those are single file that contains many sub-files. Pack files often end with a .bin extension, and they can contain a lot of different types of files at the same time. Mostly for research purpose. GUI Image: Donwload: Version 0.53 + GUI 1.2 : ppmd_packfileutil_gui_1_2-0_53.zip on Github Graphics: PPMD KaoUtil Description: This utility allows to both unpack the kaomado.kao file containing all the pokemon portraits, but also to rebuild the file! Basically, you can add new potraits into the game using this tool ! It just extract everything like a zip file if you want. And you just have to add a properly named 4 bits per pixels/16 colors png or bmp image to the folder matching the pokemon you want, before repacking the parent folder again into a ".kao" fle. The first color in the palette is reserved by the game and is forced to pure black, so you have really 15 colors. Example: For example, here's a crappy little edit of Poochyena's basic portrait. I inserted it into slot 2, which correspond to the emotion named "Grin"! Poochyena by default doesn't have a portrait for this emotion, but by using the tool, I was able to insert this very easily ! GUI Image: Donwload: Version 0.41 + GUI : ppmd_kaoutil_gui_1_0-0_41.zip on github Dowload: Version 0.42 No GUI: ppmd_kaoutil_0.42 on github PPMD GfxCrunch Description : Handles converting various image formats from the game, including WAN sprites, BGP images, Compressed Pokemon sprites "*.pkdpx", and more to come! It exports animated sprites to XML for easier editing. It also can handle all at once the entire Pokemon sprites containers "monster.bin", "m_ground.bin", "m_attack.bin". Currently Supported Graphics Formats: Individual Pokemon sprites "*.wan" file ( Type 1 ). Compressed Pokemon sprites "*.pkdpx" Entire Pokemon sprites container "monster.bin", "m_ground.bin", "m_attack.bin" Currently Partially Supported Graphics Formats: Prop sprites with orphaned images "*.wan" file ( usually Type 0 or 3 ). Can extract, but wrong resolution and tile/pixel ordering, and can't be re-packed properly. WAT sprites "*.wat" file. (basically "*.wan" sprites with a different file extension and with their type byte set to type 3) Not tested. BGP Images. "*.bgp" files. Currently Unsupported Graphics Formats: WTE Image container. "*.wte" file. WTU Color animation data file ? "*.wtu" file. kaomado.kao. (Use ppmd_kaoutil instead for now!) Raw at4px-compressed images. (This will never be perfect, because its raw data.) Raw images. (This will never be perfect, because its raw data.) Everything else.. Donwload: Alpha 0.13 : ppmd_gfxcrunch_0_13_alpha on GitHub Stats/Text/etc..: PPMD StatUtil Description: A tool for exporting game statistics to XML, and importing them back into the game's files. Making it easy for third party tools developer to make more complete and user friendly utilities, can also be edited by humans directly and opened with anyone's favorite XML editor.(Notepad++ highly recommended) So far has limited support. Currently supports editing: Move data, Pokemon Data, Item data, Generic game text, Game scripts + game dialog. Donwload: Alpha 0.23 : ppmd_statsutil_0.23.2 on GitHub Tutorial/Example: here, also include a few files and tools to make things easier! Example Video for the Script Editing : Audio: PPMD AudioUtil Description: A tool for exporting, and eventually importing/modifying, audio to/from games using the DSE audio driver, such as PMD2 ! Can output the samples to wav files, or the samples and music tracks to a soundfont and MIDIs, or to MIDIs only. You can specify your own conversion rules for exporting only MIDIs, such as what instruments the native DSE preset translate to when converted into MIDI ! This program REQUIRE you to read the readme btw. Its a console program only, for now. There is so much stuff going on in it that there won't be a GUI frontend for a while.. Please Note: Its still far from perfect! Currently supports: Exporting Music to MIDI and soundfonts. Exporting MIDIs only. Exporting samples. Donwload: Version 0.37 (2017/12/14) : latest ppmd_audioutil on Github Source Code: All source code for all utilities is available. Its not always pretty, because this codebase is also used for research, and messing around. But its much better than not sharing anything at all, I guess. Main repository for the actual utilities : ppmdu Secondary repository for the GUI frontends : ppmdu_gui_frontends Research Notes: For the most reliable, tested, and up to date info, check the Project Pokemon Wiki: http://projectpokemon.org/wiki/Pok%C3%A9mon_Mystery_Dungeon_Explorers All my current personal research notes files: File Formats : https://www.dropbox.com/sh/8on92uax2mf79gv/AADCmlKOD9oC_NhHnRXVdmMSa?dl=0 General notes : https://www.dropbox.com/s/ucy2t9hqgtfn0y0/PMD2_GeneralNotes.txt?dl=0 PMD2_MusicAndSoundFormats: https://www.dropbox.com/s/bw0aym9rn22z4wu/PMD2_MusicAndSoundFormats.txt?dl=0 PMD2_MusicTracksAndInfo : https://www.dropbox.com/s/rwjq0s0fvtiamsv/PMD2_MusicTracksAndInfo.txt?dl=0 A bunch of jumbled notes on scripts : https://www.dropbox.com/s/1roqlwkhfqb9amz/PMD2_Scripts_And_Resources.txt?dl=0 Misc Unsupported Utilities Here are a few little utilities I made to help with researching the ROMs. They're nothing special, and probably not very stable, but they're really handy: SIR0PtrOffsetsDecoder : Use this to decode a SIR0 pointer offset list, which can help tremendously in finding out more about what a sir0 file contains! Just copy/paste the SIR0's ptr offset list only, and put it into its own separate file, and drag-and-drop that onto the .bat file, it will output the result in "decoded.txt". The program itself only output the result to the console, so use the batch script, or pipe the output! IntegerListDecoder : Use this one to decode a list of encoded integers. Things like the stats growths in "m_level.bin", the move lists in "waza_p.bin". You can just put all the lists one after the other in a new file, and drag-and-drop that onto the .bat file and all the decoded lists will be outputed to "decoded.txt" ! Pages of Interest: Todo
  2. Abilities are just code. You'd have to edit the asm to do it. There might be some special characters or space in the path to your desktop.
  3. Nice! That's pretty impressive! Do you happen to have some documentation on this? Having the source is nice, but it takes a while to dig through.
  4. Please be sure to hide any PSMD spoilers within spoiler tags, until the game is released in all regions ! About This Thread: This thread is mostly for posting on-going research notes/progress and findings for the Pokemon Mystery Dungeon Gates to Infinity and Pokemon Super Mystery Dungeon games. Both games use very similar formats, and work generally the same way, which is why they both share the same thread. Information: Here are links to interesting posts in the thread, and to external websites. How to extract PMD:GTI rom's content How to batch decompile the lua scripts A nice little cheat sheet + Lua primer for those interested! Lua 5.1 experimental sandbox escape exploit Notes: Here are some links to notes, or to the wiki on the various file formats and etc of the games. SIR0 format Megadrifter's notes on the IMG format Utilities: Here are some utilities for dealing with the file formats in PSMD/PMD:GTI : ... Current Problematic Issues: Here are some of the things that are currently holding us back right now: No way to test modifications/investigate on a 3DS/emulator. My 3DS isn't unlocked. So having someone with one willing to try things out would be nice! (EDIT: Actually,thanks to ironhax I can run homebrew on it now But not much else ) This post will fill up as time goes.
  5. SED files are a bit more complicated. I haven't done much with them. But maybe eventually
  6. Not yet. But we do know how most of it works. We still need to write tools to do it. Tilesets are still tricky. Well, you can rip the individual samples inside a SWD file to wavs using my tool.
  7. You could also just go into the routine in the common script that runs the new game start and plug in the scene you made. I think you can skip the personality quiz too if I remember correctly.
  8. Just to add to this. The XML outputed is in a very similar structure the data inside the script files, its just a bit more organized. So the details on the script op codes in my notes and etc work with those as well.
  9. Well, I think fl studio's midi import was meant more for importing stuff from your typical music notation software that doesn't do anything very fancy with the midis. Roland's GS is an extension on General midi, same with Yamaha's XG, that tried to add missing features to it, but they're their own standards afaik. But, yeah, lots of things done via sysex messages aren't GM standard. Although GS, or at least some parts of it like some of their extra instruments, kinda became more or less commonly supported outside the standard from what I know/can see. So its really a huge headache to deal with midi compatibility XD
  10. @xusu I had been planning to release a version of the program that exports one soundfont per track to help with this kind of things. One per track mainly because samples are tuned differently for every single tracks. I never really got around to it, since the whole thing is a broken mess really. Don't use -gm. It won't work well with pmd2 since the tracks have several percussive tracks, and some just don't. GM expects track 10 to be drums always, GS doesn't. The game also never uses more than 16 tracks, so don't worry about it, the default mode will always have 16 tracks max. I think your problem with the chromatic/percussive tracks not switching modes properly comes from how the FL studio sequencer doesn't seems to handle importing/converting sysex midi messages. My tool puts in each midi file some sysex messages to tell the synth to run in GS mode, and set some tracks to chromatic/percussion tracks. A bunch of older synths, like windows' default built-in synth don't support that. So you'll probably have to change those things manually when you import it in FL studio since its midi import is far from perfect. Also, I suggest you dump the raw pmd2 samples using this line : "EoSRomRoot/data/SOUND/BGM/bgm.swd" "out_pmd2samples" -hexnum And then export the list of sample used for each presets for each tracks: -swdlpath "EoSRomRoot/data/SOUND/BGM/" "out_pmd2samples.txt" -listpres -hexnum Then if you compare the cv_info file and the out_pmd2samples.txt files for each tracks you'll see what samples each track/instrument presets uses, and can listen to each individual samples exported from the soundfont. That's how I did to figure out what samples are what. It really helps a ton, since the game's soundtrack messes with the samples a lot with effects and etc. For instance, in treasure town, if you listen to the samples you said are bagpipes, they don't sounds at all like bagpipes. The composer just layered several notes to sound like a bagpipe. But in other tracks, those very same samples are used for something that sounds closer to what sounds more like an English horn or something close to it, maybe a bassoon or some "non-standard" variation that sounds like that. The actual bagpipe samples are used in bgm134 and 137 for example. Also, some of the instrument samples used in the game are instruments not in general midi or gs. Some percussion(like that Indian percussion in track#29), and etc.. So I put what was closest. About your earlier questions: 1. About the cv_info file, to change the midi instrument used for a given in-game instrument/preset just set the number in-between the MIDIPreset tags to the preset number you want. Like: <MIDIPreset>81</MIDIPreset> You don't need to touch the DSE preset. The tags are explained at the top of the cv_info file too. 2. That's because of FL studio's Fruity LSD does not support GS sysex messages, or basically any sysex message afaik. Better use something more compatible like foobar2000 or the bassmidi driver, or anything based on fluidsynth to play the midi with a GS compatible soundfont and you'll notice the drum/chromatic tracks are fine.
  11. Its fine, the whole point of this thread is to be bumped every once in a while! XD I'm not really sure what you're trying to do? The xml file is for remapping PMD2's instruments to GM/GS when doing a midi only export. It has no effect when exporting the tracks along with their soundbank. And when you say the wrong instrument is set, what do you mean? In the midi-only rips or the soundfont, or in the midi files that accompany the soundfont? Adding the -gm option on the command line just tries to make it so old GM only software has a better shot at playing it back by swapping any existing drum channel with the right one. By default the files are all exported for a GS compatible midi driver. GS allows things like disabling the drum channel as needed and etc.. The way DSE works is similar to GS and not GM. Which means some tracks just won't work when exporting as GM since they use the drum channel as a melodic channel.
  12. You can't just use that on the rom file. You need to extract the content of the rom and use it on one of the pack files that's listed in the readme. But even then, this tool is mainly meant for researchers/toolmakers since it doesn't do much besides unpack some bin files into its sub-files.
  13. If I remember correctly, missing assets/not precached assets makes the starter select screen crash. I remember testing some changes at one point and you ended up testing them? Also, all the starter models are all on the select screen and get unhidden depending on the pokemon #. Basically adding starters to GTI is a lot more involved than PSMD.
  14. Nope, GTI works very differently inside. There's stuff in the works for it though! Both PSMD and GTI are very similar, so mod tools for one will most likely work for the other. @evandixon got some tools working for those too!
  15. Well, 0x4 in a pokemon's entry changes its displayed dex id. I don't think it has anything to do with evolution. And that's possible. I know that shaymin is also another special case that's handled in the code. And glad it was useful! ^^
  16. I'm not sure what you mean? The sprites for each pokemon are stored inside "m_ground.bin", "m_attack.bin", and "monster.bin", under the data//MONSTER directory. Each of those 3 files contains one of the 3 parts of each pokemon's animations/sprite. All pokemons have an entry in all 3 files. m_ground contains the sprites on the overworld/ground mode. m_attack the attack and dungeon animations for the pokemons. And monster the enemy sprites, which are just stripped down versions of the m_attack sprites to save memory on enemy pokemons.
  17. Just drag and drop the "m_ground", "m_attack", or "monster" on the gfxcrunch executable. It'll export the content to a folder. Then do your changes to the folder, and drag and drop the folder on the gfxcrunch executable. But I'm working on a sprite editor currently to supersede gfxcrunch.
  18. Use the batch files. Or SHIFT + right click with nothing selected on the background of the folder with the program in, and click on "open command prompt here" then look at the command lines examples in the readme.
  19. Well they're in the rom. You have to extract the rom, then look under data/FONT/kaomado.kao. You can use the batch scripts for doing that automatically too. Then you have to rebuild the rom with the modifications. Think of a ".nds" file as a zip file really. Except you need something else than winzip/7zip to extract/rebuild them. Well, its mainly about finding the UI API functions in the code, then hooking in your modified code. Its not that hard, but it takes time and patience. Especially if you're doing it by hand instead of using something like Radare2 or IDA. And yeah, I plan to release a bunch of QoL asm hacks eventually. I got some done already on my github mainly to load map lists from file and etc.. In order to make modding easier. I'll probably dive back into that eventually, once I can fix my utilities and get to the big map editor I'm planning. You could already fix the sleeping animations though. You'd need to basically copy in the missing sleep anims it looks for from the dungeon sprites into the ground mode sprites for the specified pokemons, and it would work.
  20. Well, it might be possible. But not without some ASM hacking. Right now I don't think we know enough to do it easily. But that could be a possibility.
  21. Well, I mean the unknowns are things I don't know about for the most part. But you seemed to be detailing the overall file structure of the waza_p file, and that's something I already know about and is pretty well documented in my notes and on the wiki. I appreciate the help however.There are still lots of things that weren't investigated yet though. If you wanna play around with script engine commands and figure out what they do, or mess with dungeon data, find reward lists offsets and etc.. There are still a lot of things like that.
  22. @Zerea Yeah, a lot of this I already knew. And I don't recommend removing move xml files when rebuilding them with statsutil right now, since its undefined behavior. I read your posts, but I'm not really sure what you were asking/saying? Sorry. Great! Thanks for the info! I could reproduce the crash, but I'm not really sure what's going on.. I'll have to take a closer look. Seems like it crashes in release, but is completely fine in debug... Looks like its gonna be one of those bugs..
  23. Well, my assumption had been that those were bitfield, aka the game would check for the value of a specific bit to toggle something on or off, but it could also be an enumeration.. I used to modify those bitfield in memory while running the game, and it seemed to me that unk#4-5 values both contributed to the resulting effect of the move. Like unk5 seemed to affect ranged move particularities.. I took some notes from your analysis, but I think it might need some testing to see how it works in practice?
  24. Thanks for the data! I was pretty sure it had to do with critical chance, but I wasn't sure if it was a ratio or something else! And you're not derailing anything, this thread was meant to share research results and etc, so you're welcome to post any findings you wanna share! ^^