Stars' Pokémon Colosseum and XD Hacking Tutorial

I'm going to be making some video tutorials on hacking Pokémon Colosseum and XD, and here is a video just to show a bit of the stuff I've done. P.S. Haven't made many videos before so it isn't great ? Part 1:  

EDIT: Note this tutorial is outdated and a lot more progress has been made since I first wrote this many years ago. The easiest way to get into modding these games is to use my Hack tools (currently mac only but the windows version is in beta testing in the discord server).  We have a discord server for everything Pokemon Colosseum, XD and Battle Revolution hacking here: https://discord.gg/xCPjjnv ---------------------   I can finally start posting all the data I have so far on hacking Pokemon Colosseum and Pokemon XD. There is a lot of it so it will take a while. Here is the first part. This is very important as it is a prerequisite for almost anything you want to hack in these games. Before I get into the details of the hacking I have some hints and tips that you should read. ===================== Prerequisites ===================== Some useful files for looking up data:  Abilities Common Values Item List Pokemon IDs Pokemon Moves Especially useful is the list of pokemon ids since they aren't always the same as their pokedex order. Those used to hacking gen III may already be aware of this since it is unique to the gen III games. The source code for my editing tools: https://github.com/PekanMmd/Pokemon-Colosseum-Hacking-Resources Some files you'll need: fsys extract and decompress script.txt pokemon lzss recompress script.txt - I. Things you'll need - Download Gamecube rebuilder (or similar program). Download QuickBMS. Any hexadecimal converter (or you can do it all in your head if you prefer). Any free online converter is enough. You may also want a binary converter for XD. [Edit by @evandixon: When set to Programmer mode, the built-in calculator in Windows will work.] A hex editor. (No tools as of the time of writing except my iOS apps which won't be released for the reasons explained in part III) Check out my git hub repository and download the quickBMS scripts. If you want to edit the banner for the ISO download a program called GCBanner. I won't be explaining it here but there are tutorials somewhere online. An emulator will be very useful for testing. Dolphin is really good these days. It plays Colosseum perfectly with the right settings but does have a few glitches with XD. It is convenient having the emulator on the computer you're hacking on because once you build the ISO you can test right then and there. However, computers can rarely produce good emulation speeds. I recently started using Nintendon't on my Wii U and it is brilliant. If you have a Wii or Wii U I recommend finding a tutorial online on installing Nintendon't as the consoles have built in GameCube emulation made by Nintendo themselves and so runs the games perfectly at full speed. - II. Useful knowledge - Learn basic hexadecimal to decimal conversion. As long as you understand what it is a hex converter will do the rest. I will not be covering this myself as there are many tutorials already. A very tiny bit of binary also comes up in XD. If you understand hex you probably get binary too and there are many converters online. Experience hacking other gen III games is very valuable. Hacking other gen games will also be of slight use but gen III does have it's quirks. My git hub repo also has some useful files for looking things up like the ids of each Pokemon so download those files also. ( I haven't put it up yet but will do soon) Less importantly, since the game was originally coded by the Japanese, a lot of the file names are transliterations of the Japanese names. For example, all the Pokemon and NPC models are named by their Japanese names. I happen to speak Japanese and I've found it really useful because it allows me to more easily figure out what is inside a file. Sometimes the name makes it very obvious. Like in field_common.fsys there is a file called "bikkuri". I know this means surprise in Japanese so I was able to make the assumption that this file contains data for when an NPC trainer spots you and the little surprise animation pops up above their head. There are no Japanese characters anywhere so just understanding a bit of the language phonetically helps. Some phrases that might come in handy: Japanese English   snatch ball snag ball   snatch dan team snagem   dark pokemon shadow pokemon   gonza gonzap   mirabo / mirrorbo miror B.   esaba pokespot (literally feeding place) waza move/attack   usohachi bonsly   gonbe munchlax   lugia lugia ( convenient ) runpappa ludicolo   relive purify   labo lab   nakigoe cry (as in a pokemon's 'roar') 10manvolt thunderbolt (literally 100,000 volts) - III. N.B. - The GameCube files are in "big endian". Those used to hacking on other platforms will be used to reversing bytes but there is no need for that at any time on the GameCube. The downside is that this makes opening files on "little endian" computers (as far as I know that's about every computer in common use) a bit trickier. Programs may need to byte-swap when reading and writing values longer than 1 byte in length. The amount of time spent compressing and recompressing files (not to mention the difficulty of emulating GameCube games) means testing can be a long and tedious process so not everything has been tested fully. Also I'll be teaching you 'safe' techniques. I haven't reached the level of repointing to expand the size of data regions. Hopefully when more automated tools are available more possibilities will open up. The preparation section isn't very exciting but is necessary so pay attention. You may find yourself repeating these steps many times before programs are available. In order to minimize repeating tedious steps, try to edit the files as much as possible all in one go before recompressing. Don't forget to be conscious of increasing the file size as I'll mention in the preparation section. All of my tools are iPad apps which I can't release because I doubt it'll be allowed. Even if Apple were to somehow accept them for the App store, Pokemon Company/Genius Sonority would probably complain. The programming was all done in a rush and the UI is pretty poor but I will be releasing the source code for everything so hopefully someone else will make some better tools. Do check out my YouTube videos. They're a bit long but I do go into more depth about many of the concepts and techniques introduced in these tutorials. I especially recommend the fsys and compression video because that is a bit tricky to describe. Since there are few automated processes and it is very easy to make mistakes, make sure you keep backups of your files. Changing just one byte incorrectly can ruin your files. The two most interesting files are the Start.dol and common_rel.fdat (in common.fsys). These two files contain a lot of the important data. They are loaded into the game's ram when the game is loaded which means that anything that can be done in those files can, in theory, also be achieved through Action Replay codes or if you prefer you could do a ram dump, edit the ram directly and then load the edited ram. This includes editing pokemon stats, move data, TMs list, a little bit of text, starter pokemon and gift pokemon and trainer teams(colo only). XD has the physical/special split! It's only used for shadow moves though. I would really like to find a way to incorporate it into all the moves. I'm really hoping someone can crack this. I'm hacking the US versions of the two games. There may be differences if you're using a different region. ================================== Part 1: Extraction,Decompression and Recompression ================================== - I. File extraction - This is how we get individual game files out of the ISO. Run Gamecube Rebuilder. Select: Image >> open. navigate to your gamecube ISO/GCM and open it. In the file browser to the right, right click on 'root' and then click 'extract'. Choose a location to save the folder in on your computer. This will save all the game files in a folder called 'root' wherever you choose to save it. This can take a while but is useful if you want easy access to all the files. Alternatively, if you only want an individual file then you can right click on the specific file you want and extract it in the same way. Edit any of the files, making sure to either keep them in the same folder or put them back later with the same file name. - II. File reimporting and ISO rebuilding - This is how we put the files back into the ISO after we've edited them. Try to make sure the file size is the same as the size of the original file you are replacing. If it is too large then you may mess up our ISO. - importing an individual file- open the ISO image as in part i.  find the file you have edited, right click it, click 'import' and load the edited file. - rebuilding whole ISO- If GameCube Rebuilder is still open with an image loaded, select: image >> close. Run GameCube Rebuilder. Select: root >> open, navigate to the 'root' folder you made in part I. Select: root >> save, choose where you want to save your new ISO/GCM and name it. Select root >> rebuild. A new ISO/GCM will be saved as you specified in step 3. - III. fsys extraction and decompression - Most of the game files are compressed and archived in .fsys files (kind of like a .zip or .rar file). This is how we get them out and decompress them in order to be able to edit them. Some files haven't been archived and don't require decompression but most of the useful and interesting ones do. Run quickBMS. Select the "fsys extract and decompression script" (see attatched files or source code on github) and click 'open'. Select your .fsys file and click 'open'. You can select multiple files at the same time and the program will extract from them one at a time. Choose a folder to save the files in and then click 'save' (warning this will most likely extract multiple files so you may want to create a folder for them). You can stay in the same folder if you wish. This script automatically decompresses and adds a '.fdat' file extension to each file extracted. Open the .fdat files in a hex editor or if opening them in a program then you can read the bytes as binary data. If the fsys archive contains multiple files with the same name (which is quite common actually), then simply type 'r' into the command line and then press the return/enter key on your keyboard quickBMS will automatically rename the files with sequential numbers. - IV. File recompression - Of course, once the files have finished being edited they must be recompressed so we can put them back in. Open the file you want to recompress in your hex editor and record the exact number of bytes in the file. Any decent hex editor should show this value somewhere; if not automatically then highlight all the bytes (ctrl + a on windows) and then it should show you how many bytes are highlighted. Make sure you convert this number to decimal if it is in hexadecimal. Open the "pokemon lzss recompress script" (see attatched files or source code on github) in any text editor. A simple one like notepad is best. At the end of the file you will see 2 identical numbers. Replace them with the file size you recorded in step 1 (again remember to convert it to decimal). You should now have two new identical numbers which represent the file size. Save the modified script. (If you are editing the same file a lot without changing the file size it may be convenient to save a separate script for the specific file with a different name so you can reuse that script and skip the first 2 steps.) Run quickBMS and follow the steps in section III above, however, this time run it with the script from step 2 and then the file you are trying to compress. This script will save the file without a .LZSS extension. - V. fsys reimporting - Once the files have been recompressed they now need to be put back precisely where they were found in the archive so that they game can load our new edited data. Make sure your compressed file size is smaller than the original compressed file size. In each section where I edit .fdat files I will share tips on how to effectively do this. In general, if you can make long rows of 00 bytes, for example by deleting text or removing items, EVs, or a whole Pokémon from a trainer, your compressed size will decrease. If you edit data and you aren't creating long rows of 00bytes then you are most likely going to increase the compressed file size. Such is life. If you want to learn more then research how the LZSS compression algorithm works and about 'entropy' in terms of data compression, which is similar to the concept of entropy in physics. In very basic terms, we are trying to decrease the randomness of the data so it compresses better. Open the .LZSS file you want to reimport in your hex editor. Note the exact number of bytes in the file. If you had changed the uncompressed file size then also note that beforere compressing the .fdat file. Open the .fsys file you originally extracted the .LZSS file from (as a .fdat file) in your hex editor. Note the exact number of bytes in the file (in hexadecimal this time). I recommend checking out my youtube video on fsys and decompression as this step gets complicated. Bear with me, this step is hard to explain by typing. However, it is usually very easy to do. The .fsys file is an archive comprising multiple files (or sometimes one file) within it. the files in the .fsys are all separated by some rows of 00 bytes and each one starts with the magic bytes "LZSS" which you can see in the ASCII version of the data in your hex editor. Your task in this step is to figure out which file in the archive is the one you have edited and are trying to replace. Not far from the start of the .fsys file you have the file names of the files in the archive. Use this to figure out the position of your file in the archive. If the file was sequentially named by quickBMS because of name conflicts then the lower numbered ones were the first ones in the archive. You can also figure out which file you want by looking at the header for each file which starts with the "LZSS" and then has the compressed and uncompressed file sizes. Match these with the sizes you expect for your file. Also the data should start with the same bytes as your file since you probably didn't change much at the very beginning of the file so it decompresses the same way. The files I edit a lot "common_rel.fdat" which is the first file in common.fsys, and the text data files for the maps which is the third file in any of the .fsys files for a map. This is true for both games and makes this step easy to figure out. If you somehow got through all the text and managed to make sense of it then it will soon become obvious how to figure all this out. There is a very easily recognised pattern to the .fsys files. If you scroll down just past the file names then you will find data separated by rows of 0x11 bytes (i.e. 11 11 11 11 11 11 11 11 11 11 11 11) in colosseum or just 0x00 (0x0000000000000000000) in XD. The blocks of data which precede each of these rows describes and individual file in the archive and this data is in the same order as the file names and the same order that the files in the archive are saved in. This block of data contains the offset within the .fsys file that the .LZSS file starts as well as both the compressed and uncompressed file sizes. If we have changed any of these values (usually the compressed file size) then we must update this information here or the game will crash even before we reach the title screen. Before you change the decompressed file size value you must add 16 bytes (0x10) to your value because there is also a 16 byte header which isn't part of the LZSS file we created but is in the .fsys file. Scroll to the start of the file in question. The "LZSS" bytes at the start will let you know you're at the start of the file. The 16 bytes starting with "LZSS" are the file header. Again this has the compressed and decompressed file sizes for that file so if you have changed these then update these values once again (remembering to add 16 bytes to your decompressed file size). Always remember you need to update the decompressed file size in two places. Copy all the bytes from your .LZSS file (ctrl + a to highlight all then ctrl + c to copy). Starting from the byte after the header in the .fsys file, highlight as many bytes as the number of bytes you copied from the .LZSS file. Assuming your .LZSS is smaller there will still be some bytes left over from the previous file. Replace these extra bytes with 00 until the end of the previous file. If your file is too large then you willoverlap with the next file, hence the emphasis on keeping your file as small as possible. We can move the next file if necessary as long as we update the pointers in its block of data before the 111111 row corresponding to the next file. I've never actually tested this myself but it seems obvious enough. There are a few bytes between each of the files for tidiness but we can overwrite these if we want to since this won't increase the size of the .fsys file. Again, I haven't tested this but if you wanted to increase the total size of the .fsys file and repointed everything properly then you would probably need to update the ISOs .toc file with the new file size of the .fsys. I haven't done this before though so good luck if you attempt it. The details of the .toc file have probably been documented somewhere before as it is pretty much the same in every gamecube ISO. That's it ! You've successfully extracted, decompressed, recompressed and reimported a game file from a .fsys file. This process is a lot easier to do than describe and not knowing how to do this was the only reason Pokémon Colosseum and XD hadn't been hacked before. Now we can research and edit the game files with ease. Part 2 -  

** You will need to understand Part 1 of this tutorial before you can decompress the files used in any other parts** Text editing is the easiest thing to do in just about any game that is being hacked. Text editing in colo/xd is also very easy to do but can be very tedious as the text in the game is scattered between many different files. This means that you have to first figure out which file has the text you want, then you may have to unarchive the file if it's in a .fsys file (most of the text is in .fsys files) and if it is in one then you must be careful not to increase the compressed file size for when you recompress it. However, if it isn't in a .fsys file then you can safely write over it without trouble. All the text in-game is in Unicode which takes up 2 bytes per character. so for example the character 'A' is 0x0041. To change this to 'B' you just need to change the second byte from 0x42 to 0x41 and you can ignore the 0x00 between each letter. The easiest place to start is in Start.dol in the "&&SystemData" folder of the ISO file system. This file doesn't get compressed so you can edit the text as much as you like without it increasing the file size, assuming you replace bytes rather than insert new bytes. The dol is really large and most of it isn't text so the text may be hard to find. Try using your hex editor to search for common words. common_rel.fdat in common.fsys also contains a lot of interesting text like the names of all the pokemon and moves. The text is always in a file I call a "string table". It stores each piece of text with an id number. That id number can be used elsewhere in the game to reference that piece of text. I will refer to this later when editing moves and pokemon. You can recognise the start of the string table file because it has the acronym for the language the table is in. My games are the US version so the string tables all start with "US" (0x5553). You can search for this in a file to see if it contains an embedded string table, although it's usually obvious by the presence of unicode text. The table begins with pairs of values. The first 4 bytes is the id I mentioned previously and the second 4bytes is a pointer to the offset from the start of the string table (6bytes before US) to the string referenced by that id. They exist in so many files that I can't list them all but here are some of the important ones. Start.dol has a string table at 0x2CC810 (Colosseum), 0x374FC0 (XD) . common_rel has some string tables, the first one being at 0x59890 (Colosseum), 0x0x4E274 (XD) and the next following consecutively. A few files in fight_common are string tables. The third file in any .fsys containing a map is a string table. This will containing all the NPC dialogue for that map. When changing the text, make sure you only write over one string at a time. You can usually spot the end of a string because it is terminated by 0x00 or 0xFF. Writing passed this will overwrite the next string and mess up the pointers. So either repoint or try to fit you text in the same amount of space. In order to keep the compressed file size small, try to find some other strings which are less important and shorten them. For example, changing the description for the inner focus ability from "prevents flinching" to something like "stops flinches" will decrease the randomness. Also in XD it seems that all the text translations for other languages were included in the US version which is the one I happen to have been hacking. You can probably safely erase all of the text in those files by replacing each character with 0x00 since you'll probably only ever play in one language. I haven't done this myself but I doubt it would cause issues. Part 3:  

** You may want to understand Part 1 of this tutorial before you can extract the files used in any other parts** **I haven't found this data in XD yet so it is currently only information on Colosseum. Pokémon that are given to you have their data loaded by the .dol file. For Duking's Plusle, mt.battle ho-oh, agate Pikachu and agate Celebi you can change the Pokémon and its level and its moves will be taken from the last 4 of its natural level up moves at that level. With Espeon and Umbreon you can also edit their moves as you please. For the ones which you can't change their moves, I recommend just changing their level up moves before that level to the moves you want it to have since those Pokémon can only be caught once and without a move tutor level up moves from levels below the level any Pokémon is caught at are useless (as well as level up moves for Pokémon that are unobtainable) don't count for anything else. The following are the starting offsets for the gift Pokémon in Colosseum: Pokémon Offset Value Espeon 0x12DAC8 388000C4 Umbreon 0x12DBF0 388000C5 Duking's Plusle 0x12D9C8 38800161 Mt. Battle Ho-Oh 0x12D8E4 388000FA Agate Celebi 0x12D6B4 388000FB Agate Pikachu 0x12D7C4 38800019 Counting the first byte (0x38) as offset 0 from the start of the Pokémon's data, the Pokémon's species is at offset 0x2 and is 2 bytes long. The Pokémon's level is at offset 0x7 and is 1 byte long. For the Pokémon with editable moves (Espeon and Umbreon) the moves are at offsets, 0x16, 0x26, 0x36 and 0x46 and each is 2 bytes long. I will add the information for XD once I find it. Part 4:    

** You may need to understand Part 1 of this tutorial before you can extract the files used in any other parts** This is really easy to do. The moves contained in the TMs (and HMs which aren't actually obtainable in game and their items don't have any data) are just listed one by one in the start.dol file. If you change the move number the move contained in the TM will change. You don't even need to worry about compression. They are each 6 bytes of 0x00 followed by the 2 byte value representing the move. The TM data is located at 0x365018 in Colosseum and 0x4023A0 in XD. (or just search for 0x00 00 00 00 00 00 01 08 00 00 00 00 00 00 01 51 which represent focus punch and dragon claw, the first 2 TMs.) The HMs come right after the TMs. You will also notice that the HMs have an 0x01 for the first byte rather than the 0x00. I haven't tested exactly what effect this has but HMs can't be deleted without the use of the move deleter. It is possible that changing this value to 0x00 like the TMs then that may allow you to delete the move. There is also an HM flag in the move data so I'd set that to 0x00 as well. HMs have no other effect in colo/xd so there is no advantage to counting them as HMs. You may also want to change the text for the TM. If you look at the text for the items the TMs have the text that describes the original move they contain. Part 5:  

** You will need to understand Part 1 of this tutorial before you can decompress the files used in any other parts** The stats for the Pokémon follow a similar but slightly different format in each game. Most of the data is self-explanatory and it all works the same way as in Ruby and Sapphire. The data starts with Bulbasaur as you might expect, there are empty entries between Celebi and Treeko just like in Ruby and Sapphire. The order is the same as the Ruby and Sapphire order with the Gen III Pokémon is a slightly different order to their national dex order. Chimecho is at the end in Colosseum and in XD there is an empty entry after Chimecho and then the entry after that is Bonsly. Bonsly's data isn't complete; for example it's missing the level up rate data (since it only appears as a bingo card in-game). If you fill in the rest of Bonsly's data then Bonsly will function like any other Pokémon in the game. Even more interestingly the entry after that has data for Munchlax. The string id for the name of that entry points to the string "Munchlax". However, even less data is filled in for this row. I've focused mainly on Colosseum so I haven't done many tests on XD but Munchlax doesn't have a pkx model (pkx_gonbe.fsys with Gonbe being its japanese name) which I think are the models used in battle. Its possible that filling in its data would allow it to be used but I haven't tried yet. Take a look at my source code on Github and look for the "PokemonStats.swift" object. The data The first stats (bulbasaur) are in common_rel.fdat at offset 0x12336C in colo and offset 0x29ECC in XD. The size of each pokemon's stats is 0x11C in colo and 0x124 in XD. The offsets are how many bytes from the first byte of the pokemon's entry the data for that stat is. Each pokemon has a series of level up moves which are each 4 bytes long. The first byte is the level and the 3rd and 4th bytes are the move with the second byte being apparently unused. Each pokemon has enough space for about 20 level moves. (of course adding more will increase the file size.) Also the TMs and HMs work differently from the GBA games. You find in colo/xd that a lot of data that was represented by a single bit in the GBA games now gets a whole byte. The TMs and HMs each pokemon can learn are assigned one byte each. If that byte is 1 the pokemon can learn the move, if it's 0 then it can't. The HMs come straight after the TMs. Some of the XD data isn't filled in yet because I hadn't recorded the data I wasn't interested in. N.B. In XD the shadow pokemon data in DeckData_DarkPokemon seems to have a byte for a separate catch rate. That would mean that the catch rates in the pokemon stats are probably overriden and only take effect for the pokespot pokemon. I haven't tested this but a lot of the values at that offset are the same as the catch rate of the original pokemon with only a few being slightly different but still matching typical catch rates. Pokemon stats data table Stat Size (bytes) Colosseum Offset XD Offset   Level-up Rate 1 0x00 0x00   Catch Rate 1 0x01 0x01   Gender Ratio 1 0x02 0x02   Base Exp Reward 1 0x07 0x05   Base Happiness 1 0x09 0x07   First Evolution 6 0x9C 0xA6   Base HP 1 0x85 0x8F   Base Attack 1 0x87 0x91   Base Defense 1 0x89 0x93   Base Sp. Attack 1 0x8B 0x95   Base Sp. Defense 1 0x8D 0x97   Base Speed 1 0x8F 0x99   Type 1 1 0x30 0x30   Type 2 1 0x31 0x31   Ability 1 1 0x32 0x32   Ability 2 1 0x33 0x33   First TM 1 0x34 0x34   First Level-up Move 4 0xBA 0xC4   Name String ID 2 0x1A 0x1A   ID of Model Used 2 0x2E ??   ID of Cry Used 2 0x0E ??   ID of Mugshot Used 2 0x10E ??   Egg Group 1 1 0x6E ??   Egg Group 2 1 0x6F ??   Egg Cycles 1 0x17 ??   Height (feet) 1 0x0A ??   Height (inches) 1 0x0B ??   Weight in Pounds x10 2 0x0C ?? (e.g. a value of 0x64 gives a weight of 10.0 pounds) Wild Held Item 1 1 0x70 ??   Wild Held Item 2 2 0x72 ??   ID of Species Name 2 0x1E ?? (e.g. Suicune is the aurora Pokémon) EVs rewarded for defeat are at around 0x91 for Colosseum. My advice for editing the stats without increasing the compressed file size is to remove all the level up moves for the Pokémon that aren't obtainable in the game as that data is never accessed. Each trainer has custom move sets for each Pokémon. You can also remove all wild held items, egg groups and all egg cycles, as well as level up rates and evolution data for unobtainable Pokémon. This is less important in Colosseum because you can erase all the battle mode trainers and that gives you enough space for just about anything you like. If you require battle mode or you're hacking XD (where the trainer data is in different files) then this can be useful. Part 6:  

** You will need to understand Part 1 of this tutorial before you can decompress the files used in any other parts** Editing moves works pretty much like in the GBA games. You can attempt to make new moves by mixing and matching the different move effects available and manipulating accuracy, damage etc. I remember seeing a post on a forum somewhere where someone posted every Gen IV - VI move that was possible to remake using gen III effects so you can try searching for that if you want ideas. The move data is pretty much the same in Colosseum and XD but differs slightly for a few bytes. A significant difference is that XD has the Gen IV style physical/special split on shadow moves. Also of note is that all the shadow moves are simply recorded as normal type moves and the battle routine automatically calculates the move effectiveness as a shadow move (always neutral in colo; super effective on non-shadow pokemon, not very effective on shadow pokemon in XD) when it detects a move that is meant to be a shadow move. There doesn't appear to be any marker for the shadow moves in the move data. The game seems to decide based on the id of the move (i.e. move 0x164 in colo which is shadow rush and moves 0x164-175 in XD which are the shadow moves starting with shadow blitz and ending with shadow half). The full move list is available on Github. There is also a list of each of the move effects (same as in RS/FRLG) in order and an enumeration for each of the possible values for the target. A lot of data that was represented by 1 bit in the gba games gets a whole byte in these games. Each move has a series of flags in its data like whether or not it is blocked by protected or affected by the king's rock. Set the value to 1 for true or 0 for false. I represent these boolean variables by putting a '?' at the end because a full description takes up a lot of room on the table. The first move (pound) is in common_rel.fdat at offset 0x11E048 in colo and offset 0xA2748 in XD. The size of each move is 0x38 (56) in colo and in XD. The offsets are how many bytes from the first byte of the move's entry the data for that variable is. N.B. Setting the physical/special flag on a normal move is not enough to make it adopt the physical/special split. It sucks, I know... Also HMs aren't available in the game and the only effect they have is that they can't be deleted without a move tutor. Removing the HM flag should allow the moves to be deleted. However, this hasn't been tested yet and there is also an HM flag in the TM and HM list in the start.dol which could have an effect. Damage seems to be part of a move effect. This means that no matter how much base power you give a move, if the move's effect is the same as leer's then it will decrease the target's defense and do no damage. You would need to use rock smash's effect which is damage as well as lowering defense and you could change the effect accuracy as you like. Move data table Variable Size (bytes) Colosseum Offset XD Offset   Move Priority 1 0x00 0x00 (Look up a gen III move tutorial to understand how priority works. It isn't obvious for negative priorities like counter but is basically 255 minus the value you want. i.e. a priority of -3 is represented by 252 = 0xFC) Base PP 1 0x01 0x01   Type 1 0x02 0x02   Targets 1 0x03 0x03   Accuracy 1 0x04 0x04   Effect Accuracy 1 0x05 0x05   Makes contact? 1 0x06 0x06   Blocked by protect? 1 0x07 0x07   Magic coat reflects? 1 0x08 0x08   Snatch steals move? 1 0x09 0x09   Mirror move copies? 1 0x0A 0x0A   King's rock affects? 1 0x0B 0x0B   Is sound-based? 1 0x10 0x10   Is an HM? 1 0x12 0x12   Base Power 1 0x17 0x19   Effect 1 0x1B 0x1D   ID for Move's Name 2 0x22 0x22   ID for Animation 2 0x32 0x32   Physical/Special 1 N/A 0x13 XD Shadow Moves Only Description Text ID 2 0x2E 0x2E This points to text in the string table contained in start.dol. Recoil Flat 1 0x13 0x14 Moves like Hyper Beam, Jump Kick, Thrash, Psycho Boost, and Double edge have this flag set. Interestingly there is an entry after shadow half in XD which doesn't appear in the game and has a little bit of data missing. If you navigate to the string its name id points to then you find the text "????". The move is unselectable in battle probably because it doesn't have an animation set in its data and one probably doesn't exist for it. The id for the string of its description points to the text "this move can't be used because the heart's door is shut". Similarly in Colosseum there is an entry after shadow rush with incomplete data which shares a few values with the XD move. Its name id also points to the text "????". Part 7:  

** You will need to understand Part 1 of this tutorial before you can decompress the files used in any other parts** Trainer Offsets: colosseum trainer data.json Shadow Pokémon Original Offsets: shadow trainer data.json For each trainer there are 6 Pokémon entries but not all of the slots have to be used. After 6 slots of Pokémon, there is one unused entry then the first slot of the next trainer is right after that. This means that you can start at the first trainer and to get to the next trainer all you need to do is jump 7 slots. This also means that you can easily give any trainer up to 6 Pokémon without having to repoint or expand the data. (Of course the more Pokémon you add the larger the compressed file size will become). Each Pokémon has a byte which you can set to make it a Shadow Pokémon. That byte is effectively the flag that gets set when you catch that Pokémon and just by setting the flag the game will consider it to be a Shadow Pokémon and your partner will react to it (even if she hasn't actually joined you yet). I don't actually change this because setting this flag to an unused flag just means every time you fight that trainer they will have the Shadow Pokémon no matter how many times you catch it. If you use a flag that is used by the game then you will affect the trainers who's teams rely on that flag. So if I set the flag to the same flag that Evice's Tyranitar uses, then when I reach Evice he won't use the team with Shadow Tyranitar, he'll just use the replacement one. Also each team that has a Shadow Pokémon has an exact copy of the Shadow Pokémon. So if you want to change one of its moves you will have to change that move on every team that it appears on which can sometimes be up to 4 teams. I made a program that takes the changes to one and applies it to all of the other iterations automatically. In the source code there is a .json file containing all the default occurrences of each Shadow Pokémon. The source code also contains a .json file with details for each trainer in the game. Since I don't know where the trainer data is saved I had to fill it in manually. I recorder the name, trainer class, in-game location and file offset for each of the trainers. The first trainer in Colosseum (Folly at the Phenac city entrance) is in the common_rel at offset 658820 (0xA0D84). Each Pokémon is 80 (0x50) bytes long and so each team is 480 bytes long and to go from the start of one team to the start of the next team you jump 560 bytes (because there is an empty Pokémon inbetween). The actual first Pokémon entry is at offset 0x9FE28 but the first few are a bit dodgy, maybe used when they were testing the game. Folly's Pokémon start at the 0x31 (49th) Pokémon entry. N.B. The trainers don't change their teams based on the Shadow Pokémon you've caught but based on the flag set when it was caught. So when you catch a trainer's Shadow Pokémon, the next time you fight them the game loads up a different team so you will find trainers who have two different teams, one for if you've caught their Shadow Pokémon and one for if you haven't even if the "caught" team is just the same team without the Shadow Pokémon. When I was still trying to figure the game out, the trainers were the first thing I researched and the first thing I discovered. This meant that at the time I was documenting it and writing my programs I didn't have everything fully figured out yet. I noticed that each Pokémon has the value (0x0004) in it at a certain point and so I used this to search for Pokémon before I realized they were all next to each other and the distances between them were just unused Pokémon. All of the offsets in this section are relative to this 0x0004 value which is actually 12 bytes into the Pokémon's data. If you want you can subtract 12 bytes from each of the start offsets I use in the trainer data json files and then add 12 bytes to each of the offsets in the Pokemon.swift object from the source code (the same offsets in the table below.) I just stuck with it because it would be tedious to update all my code (I'll do it eventually). The Pokémon's ability can be either 0 or 1, each representing one of the two abilities set in the stats for that Pokémon. You can't give it any ability. For values of other variables like genders, items and natures check out the enumerations in the source code on GitHub. For the values of the Pokémon's ability, nature, gender, EVs and IVs you can set the value to 0xFF (255) and the game will automatically replace those values with randomly generated values. All the Shadow Pokémon had 0xFF for those values by default. The "FF Offsets" detailed below are the bytes which are set to 0xFF for any Pokémon that isn't in use. I'm not sure if it's required by the game but my program will fill them in automatically for me when I delete a Pokémon from a trainer while setting all the other necessary values to 0. However if you do something as simple as putting an invalid species for the Pokémon like Pokémon number 0 or Pokémon number 252 (which is one of the empty slots in gen III) then the game will play as usual but the team will be loaded without that Pokémon. However, if you set proper stats for the Pokémon in slot 252 including choosing one of the models and names already available in the game files then the Pokémon will be playable. You could use this for something like making a duplicate of a Pokémon except it has different abilities and that way you could in essence have a Pokémon with up to 4 different abilities because you could use the two species interchangeably. You could also make something like Meowstic/Gallade/Frosslass where you make a copy of a Pokémon except one of them is all male and the other is all female. You could then give them different level up moves or different evolutions depending on which one it is. In the future, if we crack the model formats we could probably use those slots to add new Pokémon without having to replace old ones. Just to clarify, the offsets for each variable are relative to the "four offset" at 0x12 from the actual start of the Pokémon so the first variable is documented as being at an offset of -12 from the 0x0004 but is really at an offset of 0x00 from the Pokémon start. I will also add the "true offset" which is relative to the actual start of the Pokémon but if you are comparing with my source code on git hub then use the four offset. Trainer Pokémon data table Variable Size (bytes) Offset from 0x0004 True Offset   Ability 1 0x0C 0x00   Gender 1 0x0B 0x01   Nature 1 0x0A 0x02   Shadow ID 1 0x09 0x03   Level 1 0x08 0x04   Pokémon Species 2 0x02 0x0A   "Four offset" 2 0x00 0x0C (almost always 0x0004) Held Item 2 0x06 0x12   HP IV 1 0x10 0x1C   Attack IV 1 0x11 0x1D   Defense IV 1 0x12 0x1E   Sp. Attack IV 1 0x13 0x1F   Sp. Defense IV 1 0x14 0x20   Speed IV 1 0x15 0x21   HP EV 1 0x17 0x23   Attack EV 1 0x19 0x25   Defense EV 1 0x1B 0x27   Sp. Attack EV 1 0x1D 0x29   Sp. Defense EV 1 0x1F 0x2B   Speed EV 1 0x21 0x2D   Move 1 2 0x2A 0x36   Move 2 2 0x32 0x3E   Move 3 2 0x3A 0x46   Move 4 2 0x42 0x4E   ID String of Name 2 0x0A 0x16 (The Pokémon's nickname will be the string this points to. This is why with the current AR codes when you change a Pokémon's species it's name doesn't change. For the most part setting this value to 1000 + the Pokémon's national dex number, i.e. Bulbasaur is 1001 (0x3E9) and Treeko is 1252 (0x4E4), gives you the right string but I think it stops following the trend somewhere half way through the gen III Pokémon. I haven't tested it but it may be possible to give a trainer's Pokémon a nickname by setting this to the id of a string you've changed.) Typically a trainer will send their Pokémon out in the order they are specified in the data. However, sometimes, and for reasons I don't know yet, a Pokémon will be sent out in a different order to that specified. Usually they are Shadow Pokémon. Most are simply added to the team last but some are added first and still get sent out last every time. Some are added first and come out first as expected. Sometimes a trainer will randomly mix up their Pokémon order in different battles (like after a soft reset). More research is required. Editing the trainer teams is really fun and creating lots of interesting teams will likely be the main goal in many hacks. However, the more you change, the more likely you are to be increasing the compression size as you start to make lots of unique Pokémon, while the original has a lot of repetition since it didn't feature that many, especially not from gen I. The battle mode trainers are made to be more competitive, but the story mode trainers never have any IVs or EVs on them (hence they're almost all so pathetically weak) although they sometimes have beneficial natures. They almost never have held items either. Almost no trainers have 6 Pokémon and they don't all know 4 moves. This means that the process of making the teams more fun, varied or competitive will almost definitely result in a much larger compressed file size. By far the best way to decrease the compressed file size is to put 0x00s over all the Pokémon in battle mode if you don't mind not having it in your hack. This will give you enough space to go wild. By doing this my common_rel is 7 KB smaller than the original even though I've changed and added so many Pokémon as well as editing a lot of moves and stats. You can choose to just remove moves, items, evs and/or ivs or even 1 or 2 Pokémon from battle mode teams rather than removing all of their Pokémon completely in order to have a more competitive story for a less competitive battle mode. You can also rely on remove data from other parts of the file like the moves, stats and text since there are all of them in the same file. Finally, you could delete some Pokémon from some of the less important story mode characters. There are Zigzagoon and Hoothoot that were going to get OHKO'd anyway so might as well save the space. I'm sure we'll soon find out how to safely increase file sizes and then we won't have to make these trade-offs but I think it's worth it for now. While writing this documentation I have just finally found where the trainer data is stored in Colosseum! It is in the common_rel file with the first trainer being at offset 0x92F04. The order of the trainer data seems to differ from the order of the trainer teams though. Each entry is 52 (0x34) bytes long. The id of the trainer's name string refers to text in the common_rel string table. There are 3 optional string ids at the end of each trainer's data. The first being what they say before the fight (usually only used in colosseums since the you don't see those battles are initiated automatically), the second is what they say if they win (usually only for colosseums and mount battle because anywhere else you just "black out")and they third is what they say if they lose. These strings are found in the string tables in fight_common.fsys's "fight" and "tool_fight" files. Each trainer also has an index for their in-battle model and their trainer class. I will compile a list of these indexes eventually. The index of the trainer's first Pokémon also gets 2 bytes. The Pokémon at index 1 starts at offset 0x9FE28 (a value of 0x31 means the 49th Pokémon entry including the empty rows between trainer teams). Each trainer can also have up to 8 items like potions and x-attacks. Trainer data table Variable Size (bytes) Offsets Index of Trainer Class Name 1 0x03 Index of Trainer's First Pokémon 2 0x04 Name String ID 2 0x0A Index of Trainer Model 1 0x13 Item 1 2 0x14 Item 2 2 0x16 Item 3 2 0x18 Item 4 2 0x1A Item 5 2 0x1C Item 6 2 0x1E Item 7 2 0x20 Item 8 2 0x22 Post Battle String ID 2 0x26 Win String ID 2 0x2A Lose String ID 2 0x2E Part 8:  

** You will need to understand Part 1 of this tutorial before you can decompress the files used in any other parts** First of all I would like to point out again that I spent most of my time testing Colosseum and only recently began to decode XD as well. I haven't actually tested a lot of the stuff in this section but most of it seems obvious enough for me to assume I was right. A lot of stuff is similar to Colosseum anyway so its easy to extrapolate. I did test a little bit thought so I'm confident it works. Now, in XD, the trainer data is split up across several files which you will find in the deck_archive.fsys file. This contains 2 identical versions of each .bin file within. One just has _EU added to the file name, maybe its loaded in the PAL version instead of the other one. I don't know. However, it looks like you can get away with editing just the regular one and worst case scenario, the files are identical so you could just edit one, duplicate it and replace the _EU version with the duplicate. A big issue with these files is that it is hard to decrease their compressed file sizes. Their data is so much more compact than the data in the Colosseum files and just about every byte of data is actually important. It may be more effective to make a larger .fsys archive and edit the .toc of the iso. However, I speculate that if you are playing the US version of the game, the EU versions of the files are never loaded up. If that is the case then these files could be reduced to 1 byte (or maybe even removed completely), effectively doubling the space available in the .fsys file. The .bin files are as follows: DeckData_Bingo: This contains the data for the Pokémon used on the battle bingo cards. Unfortunately, changing the moves and stats of pokemon will have a butterfly effect on these challenges and may ruin some of them as they were often quite precise. DeckData_DarkPokemon: This contains some information about the Shadow Pokémon. The Pokémon itself is detailed like a normal Pokémon in DeckData_Story. The DDPK file just has things like the shadow moves to replace some of their regular moves with until purification, their purification steps and their adjusted catch rate. In case you haven't read the note in the Pokémon stats section, each shadow Pokémon has a catch rate with it's data which I believe override's their regular catch rate. They're usually the same though (a few are different). The file specifies which Pokémon in DeckData_Story each of the shadow data accompanies. DeckData_Story: This contains the data for all the trainers and their Pokémon in the story line. Unlike Colosseum the trainer data was easy to find as the files in XD have useful magic bytes and simple headers. DeckData_Colosseum: The data for the trainers and Pokémon the story mode Colosseum challenges. DeckData_Hundred: The data for the trainers and Pokémon on mt.battle in story mode DeckData_Imasugu: The data for the trainers and Pokémon if you choose "quick battle" on vs. mode. (imasugu literally means "right now") DeckData_Sample: These are Pokémon teams that are quite random and I don't believe they appear in the game at any point. Especially being called 'sample' it can be assumed that these were teams which genius sonority just used to test the game during development and never took out. DeckData_Virtual: These are the trainers and Pokémon that appear in the battle CD challenges in story mode. Unfortunately, changing the moves and stats of Pokémon will have a butterfly effect on these challenges and may ruin some of them as they were often quite precise. Each CD has 2 teams in this file. One is the opponents team and one is the team that you are given for the CD. I have edited the data for the Salamence you get in the first sim battle right at the start of story mode and tested a few things so my data should be fairly accurate. In all the decks apart from DeckData_DarkPokemon, there are 4 sections; DTNR (Deck trainer), DPKMN (Deck Pokémon), DTAI (maybe something like Deck trainer AI?) and DSTR (Deck string). The first 16 (0x10) bytes are the DECK file's header. The first 4 bytes of this header are the magic bytes "DECK". The next 4 bytes are the size of the file in bytes. The first row of each section is the header. The first 4 bytes are the magic bytes identifying the start of that section. The next 4 bytes is the size of that section in bytes (including the header itself which is 16 (0x10) bytes) and the 4 bytes after that are the number of entries there are in that section. DTNR contains the data for each trainer in that file. Each entry in DTNR is 56 (0x38) bytes. Each trainer has six slots for Pokémon. An id number for each Pokémon is specified and the Pokémon at that index in DPKM (or DDPK as the next note will explain) will be used as that Pokémon. The byte at 0x4 is what I call the "shadow mask" it is a bit mask of the trainer's Pokémon. If you convert the number to binary each of the last 6 bits represents one of the trainer's Pokémon. If the bit corresponding to a Pokémon is 1 then the Pokémon will be treated as a Shadow Pokémon and loaded from DeckData_DarkPokemon's DDPK section instead of the DPKM section and so the id used to reference that Pokémon is actually a reference to its entry number in DDPK. Every other Pokémon is specified by an id which references a Pokémon from DPKM. i.e. an id of 1 will be the Pokémon which is the first entry in DPKM unless the bit for that Pokémon is set in which case it will refer to the first entry in DDPK (which refers to a Pokémon in DPKM, in this case it points to the Teddiursa which is entry 28/0x1C in DPKM). A shadow mask value of 0x01 (0b0001 in binary) means the first Pokémon is a Shadow Pokémon whereas 0x6 (0b0110 in binary) means the 2nd and 3rd are Shadow Pokémon. The first entry is empty so the first trainer in a DTNR section starts at 0x38 bytes after the end of the DTNR header. Since the DTNR section comes first in all the decks that have one, it is always 0x58 bytes from the start of the file. There is an id value for each of the trainer class names and each of the trainer class models. (The name hunter refers to both male and female hunters but the they have separate models). I haven't figured out which numbers correspond to which classes yet though but 0x11 is the cipher peon class name while 0x10 is the model for blusix. It won't be very hard to compile a list since you can tell which trainer each entry is by their team, it's just tedious. DTNR table Variable Size (bytes) Offset   Offset of Trainer Name in DSTR 2 0x00   Shadow Mask 1 0x04   ID of Trainer Class Name 1 0x05   ID of Name String 2 0x06 The ID refers to text in the string table in common_rel ID of Trainer Class Model 1 0x11   Trainer AI? 1 0x13   DPKM Index of 1st Pokémon 2 0x1C   DPKM Index of 2nd Pokémon 2 0x1E   DPKM Index of 3rd Pokémon 2 0x20   DPKM Index of 4th Pokémon 2 0x22   DPKM Index of 5th Pokémon 2 0x24   DPKM Index of 6th Pokémon 2 0x26   The next section of the DECK is the DPKM section. This contains the data for individual Pokémon. There aren't too many gimmicks here. You get the data for that Pokémon like moves, gender and nature. This section specifies nothing about whether or not a Pokémon is a Shadow Pokémon or not. The extra data for the Shadow Pokémon is in the DDPK section of DeckData_DarkPokemon and that data specifies which Pokémon in the DPKM file of the story deck it adds to. Remember that the Pokémon's level will be overridden by the level specified in DDPK if it is a shadow Pokémon. The one tricky little thing is the Pokémon's personality id. Genius sonority did things a little differently from game freak. I tested it out using the value for the Salamence in the sim battle at the start of a new game and comparing to the gender, nature and ability it had since these were the only 3 variables I wasn't able to find yet. To see what is happening you need to convert the personality value to binary. Sim Salamence personality value - 0x6A = 0b01101010 bit 7 6 5 4 3 2 1 0 value 0 1 1 0 1 0 1 0 The Pokémon's ability is determined by bit 0 which in this case is it's first ability (0x00). The Pokémon's gender is determined by bits 2-1 which in this case is female (0x1 = 0b01). Male is 0x00 and genderless is (0x2 = 0b10). The nature is determined by bits 7-3 which in this case is jolly (0xD = 0b01101). To programmatically get each of these values you take the original number 'p'. p % 2 (modular division) gives you the ability. (p / 2) % 4 is the gender. p / 8 is the nature. Each Pokémon is 0x20 (32) bytes long. The first Pokémon (which is an empty entry) is straight after the DPKM header which is 16 bytes long. N.B. I haven't tested the order of the IVs and EVs but I've assumed they're in the same order as Colosseum. This also coincides with the order of the natures so it's a pretty safe assumption. Also the Pokémon's moves will be replaced by shadow moves from DDPK if it is a Shadow Pokemon. DPKM Pokémon data Variable Size (bytes) Offset Pokémon Species 2 0x00 Pokémon Level 1 0x02 Happiness 1 0x03 Item 2 0x04 HP IV 1 0x08 Attack IV 1 0x09 Defense IV 1 0x0A Speed IV 1 0x0B Sp. Attack IV 1 0x0C Sp. Defense IV 1 0x0D HP EV 1 0x0E Attack EV 1 0x0F Defense EV 1 0x10 Speed EV 1 0x11 Sp. Attack EV 1 0x12 Sp. Defense EV 1 0x13 Move 1 2 0x14 Move 2 2 0x15 Move 3 2 0x16 Move 4 2 0x17 Personality ID 1 0x1C The DSTR section doesn't seem very useful so I won't go into much detail. Basically, each trainer has a string which goes with them. Each trainer in DTNR has 2 bytes which say which offset in DSTR their code name starts at. The only use this has is that it can give you a hint as to where the trainer appears in-game. The code names usually start with the id of the map that the trainer is in like s1 (outskirt stand), d1 (shadow lab) or m5 (Pokémon hq lab). You may be able to reduce the compressed size of the DECK files by deleting all the DSTR code names since I don't think they are loaded by the game but I don't know if that will have any side-effects yet. The other deck type is DeckData_DarkPokemon. Each shadow Pokémon has its regular Pokémon data saved in DeckData_Story's DPKM section like any other normal Pokémon. The shadow data in DeckData_DarkPokemon's DDPK section specifies the Pokémon it complements. The trainer that has the Shadow Pokémon sets the shadow mask bit telling the game that the Pokémon in that slot is referencing DDPK rather than DPKM and then that entry in DDPK has a reference back to one of the Pokémon in DPKM which is the Pokémon that will be the Shadow Pokémon. The header is 16 (0x10) bytes long. The first 4 bytes of this are the magic bytes "DECK" followed by 4 bytes which give the file size in bytes. The only section in this file is the DDPK (Dark Pokémon which is Shadow Pokémon in Japanese ). The header is 16 (0x10) bytes long and each entry is 0x18 (24) bytes long. The first entry is empty after which there is an entry for each Shadow Pokémon and a whole bunch left over. You could technically add more shadow Pokémon and I think XD may handle the caught Shadow Pokémon a bit better than Colosseum does so it might actually be as simple as adding another entry to this file. However, it is already extremely hard to decrease the file size of the decks so that must be considered. The catch rate overrides the catch rate of the Pokémon in it's data in common_rel and it's level overrides the level specified in DPKM in DeckData_Story. Like with just about everything else in XD, i haven't tested this but I think each Shadow Pokémon gets a purification counter, kind of like the steps an egg needs to hatch in other games in the series. The higher this value, the harder the Pokémon is to purify. The Shadow Pokémon can be given between 1 and 4 shadow moves. DDPK data table Variable Size (bytes) Offset Catch Rate 1 0x01 Level 1 0x02 Pokémon Index in DPKM (Story) 2 0x06 Purification Counter 2 0x08 Shadow Move 1 2 0x0C Shadow Move 2 2 0x0E Shadow Move 3 2 0x10 Shadow Move 4 2 0x12