research Pokemon Super Mystery Dungeon And PMD:GTI Research And Utilities

[evandixon]

Also, I think evandixon has been working on some pretty nice stuff for handling eventual romhacks. But he'll probably drop by to talk about those himself eventually.

In the ROM Hacking department, I already have a way to make a patcher for a ROM hack. In GTI and PSMD, the script engine consists of compiled lua scrips. Coming soon™, there will be an easy way to decompile and recompile them, possibly even editing them from inside Sky Editor. So far I have everything done, except for actually using the lua compiler and decompiler. For now, I'll probably only research and develop with GTI to avoid being exposed to spoilers.

Back to you, psy_commando.

[Josephvb11]

I was thinking that there could be a way to extract all the passwords for the american version of the game from the rom, right? As it doesn't look like it checks online for them.

[PokemonWorldMaster1]

I was curious if someone could extract an item list? I just want to comfirm if joy seeds or golden seeds exist. Because I've invested 30 hours into the game and still haven't found any.

[psy_commando]

I was thinking that there could be a way to extract all the passwords for the american version of the game from the rom, right? As it doesn't look like it checks online for them.

Passwords aren't stored. They're interpreted.

And in order to make our own passwords to get whatever you want from those, someone needs to decipher the way they're made/interpreted. See the wondermail generator for pmd2 for example. Though, that's not really our priority.

We're pretty much just taking apart file formats at this stage.

And, personally, Idk how I'd go about reversing passwords without a decent debugger+emulator.

I was curious if someone could extract an item list? I just want to comfirm if joy seeds or golden seeds exist. Because I've invested 30 hours into the game and still haven't found any.

You probably want to look up gamefaqs, bulbapedia, or serebii. Because, its possible we'd find mention of those, but it wouldn't mean they'd be in-game. Besides, its going to take a while before most of us are done with the game, and/or get their hands on a decrypted US rom of the game.

[Andibad]

Passwords aren't stored. They're interpreted.

i think is still same, stored on wonder_mail.bin. is containing full of code stored on the game.

I was curious if someone could extract an item list? I just want to comfirm if joy seeds or golden seeds exist. Because I've invested 30 hours into the game and still haven't found any.

yes, i found string for "Joy Seed" but golden seeds is not found, you can found that on common.bin on message_en folder (for Super Mystery Dungeon)

[Josephvb11]

Passwords aren't stored. They're interpreted.

Wow, is that true? Because the passwords in this game seem to be much smaller than in previous games, I can't think how the game would expand some of those strings into full missions without having information about each password.

For example, the CoroCoro dungeon password for the japanese version is JQMJT46S while a password for 20 apples is JQQPK0R6, they are alphanumeric.

[Cstylzzz]

http://pastebin.com/BhaBVj73

All the us wondermail codes if someone needs those

Edit:

Eu version wondermail codes http://pastebin.com/Crk9uBqR

[Lilothestitch]

Do you know how to get the model, texture, and animation files from Pokémon Super Mystery Dungeon?

[psy_commando]

Wow, is that true? Because the passwords in this game seem to be much smaller than in previous games, I can't think how the game would expand some of those strings into full missions without having information about each password.

For example, the CoroCoro dungeon password for the japanese version is JQMJT46S while a password for 20 apples is JQQPK0R6, they are alphanumeric.

Those missions are probably just enabled by that code, they're most likely not contained in it..

As far as I know, wondermail passwords have always had an encoding and a syntax. Like the 20 apple thing, there's probably a part of the password that sets the type of password to give you items, the amount of items, and the actual item.

But it doesn't mean they contain entire level data and etc.

i think is still same, stored on wonder_mail.bin. is containing full of code stored on the game.

By "code", what do you mean exactly ? Like identifiers/translation table ?

Or did the way wondermail work changed since PMD2?

http://pastebin.com/BhaBVj73

All the us wondermail codes if someone needs those

Edit:

Eu version wondermail codes http://pastebin.com/Crk9uBqR

Oh, so you guys were talking about a list of individual passwords to get each items and etc ?

I guess I took this a bit too literally then

Do you know how to get the model, texture, and animation files from Pokémon Super Mystery Dungeon?

Don't quote me on that, but some have claimed the models are in BCH format.

I still need to verify that on my end.

We'll eventually get to extracting/importing animations and textures, and etc.. Though, I think most of us haven't finished the game yet, and probably some of us don't want to get spoiled ^^;

EDIT:

Nevevmind that about models. They're identified as BGRS files. in the pokemon_graphics_database.bin file. I don't know if its any similar to existing formats though.. But from the looks of it, you need to look up model files in the pokemon_graphics_database.bin file, and get them in the pokemon_graphic.bin file. I don't know if existing tools do that. Feel free to correct me if I'm wrong though. I appreciated the information.

[Lilothestitch]

Sorry for quoting you on that. Its just that I want to rip the models from it so I might need the files for them.

[psy_commando]

Sorry for quoting you on that. Its just that I want to rip the models from it so I might need the files for them.

Its fine, "don't quote me on that" is just an expression to say "I might be wrong".

You can quote me if you want

But yeah, I think your best short-term bet might be to ask the people working on the model tools for ORAS and etc first.

Then, if they don't support this model format, it will take us a while to reverse it and be able to actually rip the models.

[Lilothestitch]

I spoke to Random Talking Bush and he told me that the game uses .bcmdl files.

[psy_commando]

Nice. And did he mention anything else about animations or texture formats ?

[Lilothestitch]

He mentioned that the textures are .bctex

[Lilothestitch]

Can you send me the pokemon_graphics_database.bin file?

[FODnow]

PM me, I'll hit you up

[ThatTrueStruggle]

Hey guys, I'm ThatTrueStruggle. You might know me from VG-Resource, but enough of that. I've come in searching of knowledge for finishing a script. Right now, I have a script written to extract CGFX files from those BIN files. However what's giving me grief is a format called the BGRS format. I'm assuming it's a type of subfile for BCH files. Also since BCH files don't have a set file size, I can't use the same method on the CGFX files. But yeah any knowledge you guys have, It'd be amazing. Here's my current script for use with QuickBMS.

idstring "FARC"
get Size asize
callfunction CGFXExtract
goto 0 
callfunction BCHExtract

startfunction CGFXExtract
for i = 0
	get PATH basename
	SavePos Position
	if Position == SIZE
		break
	endif
	FindLoc Offset string "CGFX" 0 "noCGFX"
	if Offset == "noCGFX"
		break
	endif
	goto Offset
	idstring "CGFX"
	get Endianness short
	get Length short
	get Revision long
	get FileLength long
	goto Offset
	goto FileLength 0 SEEK_CUR
	string PATH += i
	string PATH += ".cgfx"
	log PATH Offset fileLength
next i
endfunction

startfunction BCHExtract

endfunction

[psy_commando]

Hey guys, I'm ThatTrueStruggle. You might know me from VG-Resource, but enough of that. I've come in searching of knowledge for finishing a script. Right now, I have a script written to extract CGFX files from those BIN files. However what's giving me grief is a format called the BGRS format. I'm assuming it's a type of subfile for BCH files. Also since BCH files don't have a set file size, I can't use the same method on the CGFX files. But yeah any knowledge you guys have, It'd be amazing. Here's my current script for use with QuickBMS.

You'll have to point me to what bin contains CGFX. I remember seeing that somewhere but I can't find it. Also, is that in GTI or PSMD ?

And, I'm looking for data on regular BCH models files actually to figure out the "bgrs" format. So if you have any to share that would help.

Some people have claimed they could get models out of the game by the Japanese release date, so that wasn't exactly a priority for me personally this far. But then again, the people I heard that from don't really seem all that reliable either..

And if you're looking for file lengths and offsets, they're most likely stored in the sir0 at the beginning, or in the pokemon_graphics_database.bin file.

Chunsoft are known for loving their customized file formats though, so I'm not really surprised they deviated from the usual format.. At least, they pretty much use 0 compression in this game thankfully..

The SIR0 format is a blessing for reverse engineers though. The encoded pointer list at the end lists all file pointers within whatever is inside the SIR0, so it makes it easier to guess what's a pointer to something in the file/memory or not.

EDIT:

Here's a handy little tool for converting the ptr offset list quickly to something human readable :

https://dl.dropboxusercontent.com/u/13343993/my_pmd_utilities/sir0_ptroffsetdecoder.zip

Just dump only the encoded ptr offset list into a file, as binary data, and drag and drop that onto the batch file, and the utility should output a list with all the offsets(from the start of the SIR0 header) the pointers are at within the SIR0.

Its not really meant to be used for anything else than research though.

Also, data on the SIR0 format is on the wiki, and its linked in my first post in this thread.

[Cstylzzz]

some textures from the game

[Cstylzzz]

mg_gimmick_graphic.bin and eventmap_graphic.bin contain CGFX

[psy_commando]

Posting textures is nice, but this is a research thread, so it would be appreciated to actually share notes/info over sharing ripped assets as much as possible. ^^;

And thanks, I'll take a look at those files.

[ThatTrueStruggle]

I had actually written a program that reads a *SIR0 table and extracts the files from it: https://www.dropbox.com/s/0q67hp1um2w4nn5/FARCReader.exe?dl=0

*: It works only on a few SIR0 tables. It only works on the ones that have file names and then the offsets afterwards. This is the code I followed from gdkchan :

0x0 - FARC Signature
0x20 - Number of files
0x24 - SIR0 Offset
0x28 - SIR0 Length
0x2C - Data Offset

SIR0 Table Structure
(All offsets here are relative to the SIR0 offset)
0x0 - SIR0 Signature
0x4 - Offset to Pointers Table
0x8 - Offset to encoded Relocation Table
0xC - Padding only
(String table with file names starts after this, they're UTF16 and Null Terminated)

Pointers Table
0x0 - Offset to the Files Table
0x4 - Total number of files

Files Table
(Each entry have 0x10 bytes)
0x0 - Offset to the Name of the file (relative to the SIR0 offset)
0x4 - Offset to the File Data (relative to Data Offset at 0x2C, see first header)
0x8 - Length of the file in bytes
0xC - Only 0x0 was observed, may be related to compression tho

[psy_commando]

I had actually written a program that reads a *SIR0 table and extracts the files from it: https://www.dropbox.com/s/0q67hp1um2w4nn5/FARCReader.exe?dl=0

*: It works only on a few SIR0 tables. It only works on the ones that have file names and then the offsets afterwards. This is the code I followed from gdkchan :

0x0 - FARC Signature
0x20 - Number of files
0x24 - SIR0 Offset
0x28 - SIR0 Length
0x2C - Data Offset

SIR0 Table Structure
(All offsets here are relative to the SIR0 offset)
0x0 - SIR0 Signature
0x4 - Offset to Pointers Table
0x8 - Offset to encoded Relocation Table
0xC - Padding only
(String table with file names starts after this, they're UTF16 and Null Terminated)

Pointers Table
0x0 - Offset to the Files Table
0x4 - Total number of files

Files Table
(Each entry have 0x10 bytes)
0x0 - Offset to the Name of the file (relative to the SIR0 offset)
0x4 - Offset to the File Data (relative to Data Offset at 0x2C, see first header)
0x8 - Length of the file in bytes
0xC - Only 0x0 was observed, may be related to compression tho

I never heard of gdkchan before.

Also, SIR0 isn't a consistent format. Its just a wrapper for importing data structures as-is into memory and translating the pointers.

So you'll have to go by filenames if you want to differentiate them.

For example, what's labelled as "pointer table" is literally a header with a fourcc and all in some cases. Like in the pokemon_graphics_database.bin file for instance, or several other file formats in PMD:EoS. ( Knowing myself I probably called it pointer table on the wiki as well )

But, what's causing you issues then if you can export some of the files ? Could you tell me a bit more ? Like, what gets extracted correctly and what doesn't ?

I could probably figure something out much faster if I can just look at specific part, instead of having to reverse the whole thing first.

[Andibad]

I had actually written a program that reads a *SIR0 table and extracts the files from it: https://www.dropbox.com/s/0q67hp1um2w4nn5/FARCReader.exe?dl=0

*: It works only on a few SIR0 tables. It only works on the ones that have file names and then the offsets afterwards. This is the code I followed from gdkchan :

0x0 - FARC Signature
0x20 - Number of files
0x24 - SIR0 Offset
0x28 - SIR0 Length
0x2C - Data Offset
0x0 - FARC (Magic Header)
0x20 - Type of FAT SIR0*
0x24 - first SIR0 Offset as FAT
0x28 - first SIR0 Length as FAT
0x2C - first DAT offset
0x30 - first DAT Length (with first SIR0 last padding)
^ 0x20 each volume. up to 3.
well for SIR0 0x4 is not refer to pointer table, but i think is special header for game config format ... , some file is containing magic header (like PKDB, etc) on it too ... (not an offset value). well is same as xbb file on harvest moon (on game configuration file one).
EDIT (*) : for type FAT SIR0 (only for SIR0 used as FAT on FARC)
if is 0x5, it will used 12 byte set.

• without filename : [FileNameOffset*][FileOffset][Filesize] *) might FileNameOffset, is always > sir0 size
  
• with filename : [FileNameOffset][FileOffset][Filesize]
  

with filename:

without filename:

if is 0x4, it will used 16 byte set.

• with filename : [FileNameOffset][FileOffset][Filesize][0x0]
  

Edited December 2, 2015 by Andibad

[psy_commando]

0x0 - FARC (Magic Header)

0x20 - Type of FAT SIR0*

0x24 - first SIR0 Offset as FAT

0x28 - first SIR0 Length as FAT

0x2C - first DAT offset

0x30 - first DAT Length (with first SIR0 last padding)

^ 0x20 each volume. up to 3.

well for SIR0 0x4 is not refer to pointer table, but i think is special header for game config format ... , some file is containing magic header (like PKDB, etc) on it too ... (not an offset value). well is same as xbb file on harvest moon (on game configuration file one).

EDIT (*) : for type FAT SIR0 (only for SIR0 used as FAT on FARC)

if is 0x5, it will used 12 byte set.

• without filename : [FileNameOffset*][FileOffset][Filesize] *) might FileNameOffset, is always > sir0 size
  
• with filename : [FileNameOffset][FileOffset][Filesize]
  

with filename:

without filename:

if is 0x4, it will used 16 byte set.

• with filename : [FileNameOffset][FileOffset][Filesize][0x0]
  

That console output you got with your tool, from what file is it ?

Did you find out to what the offsets in the file allocation table are relative to ? I'd guess they're relative to the data blob that comes afterwards, but it wasn't all that conclusive from taking a very quick look at it.

And sorry if I'm having a bit of difficulty understanding what you mean.

In other news, I'm getting oot3d hax soon. So I'll probably be able to do a lot more with this game once I get HANS and everything setup.