Gen I/II to Gen III/IV/V Conversion.

[er1c1996]

I know this has been talked about time and time again... but I need to clear this up for myself.

If someone was motivated to make a program and deal with all the "approximations" with IVs, nature, and the like, would it be possible to make a flat out converter from Gen I/II .pkm's to Gen III/IV/V?

I have the motivation to deal with such issues myself, but not the knowledge of programming the program. Plus it seems like every time this is brought up it's shot down as if it's not even possible.

[Guest]

It's easily possible if it's just about importing from a Gen I/II savegame file and exporting the converted Pokémon data to a Gen III/IV/V savegame file.

But getting the savegame from your I/II cart might be the main problem here.

[codemonkey85]

I know a guy who is working on an open source library dealing with save files across every generation. I'm sure his implementation would have no problem handling your use case.

https://github.com/ncorgan/Pokemon-sim

If you were a programmer, this would be a great place to start.

[POKEMONMASTER260]

If this is actually gonna become a thing, (miraculously,) please, please, please consider incorporating Rental & Registered Pokemon from the Stadium games at some point.

Please.

(Although the problem is there's no real way to dump FlashRAM saves from N64 cartridges. I think there are ways to convert FlashRAM saves to mempak saves & then use a DexDrive to dump mempak saves, though, if that helps in the slightest...)

[Metropolis]

Please see the new thread for full Mac OS X download links.

Hi - I wrote some Java code to do this recently, see the attachment (it will need unzipping first). I'll work on Stadium 1 and 2 compatibility in the next version, indeed that would be cool! For the moment it converts between any of the first three generations of games. Let me know what you guys think. There seems to be a bug regarding the PID and ability flag, where Encyclopedia Pokemonica reports an error even though the parities match. Should be just a case of comparing the PID and Ability flag parts of the file before and after Encyclo. fixes it to see what the issue is. The checksum should be OK, but haven't tested it since I rewrote the code. Enjoy!

Pokemon Time Machine by Metropolis

- version 0.1 beta

Instructions:

Run the TimeMachine.jar file in command prompt by first placing it in your root user folder then entering: java -jar TimeMachine.jar

You should be prompted to enter the filename of the PKM or 3GPKM file to be converted. The easiest thing to do here is to drag the file into the same folder, then type e.g. bulbasaur.pkm into the command prompt. The destination game can be a single word such as ruby or the number of the generation. This determines the format the Pokemon will be converted to. Finally, enter party or pc as appropriate and the new file will be saved as newpokemon.pkm or newpokemon.3gpkm in the same folder.

Due to the changes in code from GSC to RSE, a perfect backwards conversion is not always possible. For example, the only shiny Unown in GSC are of forms I or V, and all shiny Charizard are male. This creates a problem when converting the Pokemon, so the following priority list is used to determine which attributes should be preserved:

Unown Shape > Shiny > Gender > IVs.

This means that Shiny Unown A become regular Unown A, Shiny Female Charizard become Shiny Male Charizard and the Attack IV of any Pokemon is lost in favour of its gender value.

Since natures did not exist in GSC and the HP stat runs off the four main IVs, minor changes to stats are inevitable whatever priorities are applied. IVs are halved and Effort values are squared to match the changes in range and stat formula.

The following information will be lost when a Pokemon travels backwards - HP IV, Attack IV, Special Defense IV and EV, Personality value, the secret ID, language and version of the Original Trainer, Marks, Pokeball type, Contest stats, Ribbons and any move or held item that did not exist in GSC. If the Pokemon was caught at a level exceeding 63 or at a location outside of Kanto and Johto, then this information will be lost also. Unown ? and ! become Unown F, the default Unown used in Sugimori artwork.

Status ailments and Pokerus are currently not supported, but will be included in a future update. Held items are converted according to the following table:

PSNCureBerry = Pecha Berry

PRZCureBerry = Cheri Berry

Burnt Berry = Aspear Berry

Ice Berry = Rawst Perry

Bitter Berry = Persim Berry

Mint Berry = Chesto Berry

Pink Bow = Silk Scarf

Miracleberry = Lum Berry

Mysteryberry = Leppa Berry

Gold Berry = Sitrus Berry

The code hasn't been tested thoroughly, so there may be well be a few bugs. Feedback appreciated!

Please see the new thread for full Mac OS X download links.

Edited November 21, 2014 by Metropolis

[1quacka1]

You'd have to artificially change/introduce so many aspects of the Pokémon is there really any point? You might as well hack the Pokémon into RSE with the moves/shinyness/roughIVs/etc, rather than create an elaborate "transferral" system.

[Metropolis]

it seems like every time this is brought up it's shot down as if it's not even possible.

You'd have to artificially change/introduce so many aspects of the Pokémon is there really any point? You might as well hack the Pokémon into RSE with the moves/shinyness/roughIVs/etc, rather than create an elaborate "transferral" system.

Yes there's very much a point: firstly it's something users would like to see, so I'm only trying to help secondly the code lends itself to automation. I hope to introduce a batch converter in the next update that would allow mass conversion from one .sav to another. That's just a case of getting the memory locations right, I've done all the fiddly stuff already. Don't lose the wood for the trees - only a very small part of the Pokemon has to be artificially changed and this is totally unnoticeable in regular gameplay. Pokemons' stats change slightly upon trading anyway - look at the recent change to base stats in X and Y from Black and white for example. The important thing is that is is very much the same Pokemon, and will always migrate in the same way, nothing is randomised.

Rather than knocking the concept, could we run with the concept so that those who would like the ability to convert Pokemon between versions can do so as best as possible? Did you try running the code itself?

[codemonkey85]

Rather than knocking the concept, could we run with the concept so that those who would like the ability to convert Pokemon between versions can do so as best as possible?

Agreed. Any more posts about "what's the point" will be deleted - this is now a contribution thread.

[POKEMONMASTER260]

Metropolis, you, sir, are my hero.

The little kid in me who got Pokemon Ruby & hated how he couldn't trade from his old Crystal version now loves you forever.

THANK YOU!!!!!!!!!!!!!

EDIT: BTW, I believe the GameShark for N64's Memory Card Manager is what's used to convert FlashRAM saves to mempak saves.

EDIT 2: I just heard of something called a Retrode. If I understand it correctly, it is a device that you can plug cartridges from various consoles into, (including N64,) & then you plug it into a USB port and the cartridge will appear as a regular USB drive with a ROM on it. Hopefully, the save will be included too, so you could look into using Retrode as a way to get ROMs to extract Rental Pokemon from & hopefully saves to extract Registered Pokemon from.

Also, apparently a 64Drive is much better than a DexDrive for the purpose mentioned in my previous post.

EDIT 3: According to this page, FlashRAM save support might be in a future firmware update for the Retrode.

Edited November 19, 2013 by POKEMONMASTER260

[Metropolis]

Glad you're happy with what I've done so far, thanks for the feedback! I'll have a play around with my Pokemon Stadium and Stadium 2 save files (the .fla ones around the 128KB size). In theory it should just be a case of searching for the hex matching the pokemon in storage, then using that to deduce the memory locations of the stored and registered Pokemon. I suspect the storage mode will use the same formatting as the gameboy games, in which case it will be no harder than writing to game boy save files.

Plan is for the next version to read/write party and PC pokemon from one save file to another. Turning this into a full editor wouldn't be too difficult, but there's editors out there already, so I'd rather focus on something new. Working towards a "complete migration" tool that would transfer all Pokemon from one save onto the other without overwriting the data. This creates a problem with regards to synchronising process - suppose the user plays one version a bit more and wants to copy only the Pokemon they've caught since the last migration? My proposed solution is to create a hash code for each pokemon on each game that encodes the IVs, OTID and Species Number. This means that attacks, effort values, experience, etc. can change and the Pokemon will still be recognised as "the same one" that got transferred before. This comes with the disadvantage that if a hacker were to max out all IVs they wouldn't be able to transfer more than one of the same species. Keep in mind that Gold/Silver does something like this itself when checking if the Defense and Special IVs match when breeding. Without this check you could easily end up with lots of duplicate Pokemon and have to release them all, which would be annoying.

Eventually this will probably end up as a whole suite of conversion and save editing tools, because the abstract representation of the Pokemon is the same for each tool. It would be cool to be able to transfer items from RBY to GSC and RSE - even the Stadium and Stadium 2 games won't let you do this as the items are separated into color and metal boxes! As soon as I get the free time to code the remaining sections I'll develop this into a save editor not just PKM converter. All the offsets/info needed are on Bulbapedia, I know how I'll code it out.

[POKEMONMASTER260]

Glad you're happy with what I've done so far, thanks for the feedback! I'll have a play around with my Pokemon Stadium and Stadium 2 save files (the .fla ones around the 128KB size). In theory it should just be a case of searching for the hex matching the pokemon in storage, then using that to deduce the memory locations of the stored and registered Pokemon. I suspect the storage mode will use the same formatting as the gameboy games, in which case it will be no harder than writing to game boy save files.

Plan is for the next version to read/write party and PC pokemon from one save file to another. Turning this into a full editor wouldn't be too difficult, but there's editors out there already, so I'd rather focus on something new. Working towards a "complete migration" tool that would transfer all Pokemon from one save onto the other without overwriting the data. This creates a problem with regards to synchronising process - suppose the user plays one version a bit more and wants to copy only the Pokemon they've caught since the last migration? My proposed solution is to create a hash code for each pokemon on each game that encodes the IVs, OTID and Species Number. This means that attacks, effort values, experience, etc. can change and the Pokemon will still be recognised as "the same one" that got transferred before. This comes with the disadvantage that if a hacker were to max out all IVs they wouldn't be able to transfer more than one of the same species. Keep in mind that Gold/Silver does something like this itself when checking if the Defense and Special IVs match when breeding. Without this check you could easily end up with lots of duplicate Pokemon and have to release them all, which would be annoying.

Eventually this will probably end up as a whole suite of conversion and save editing tools, because the abstract representation of the Pokemon is the same for each tool. It would be cool to be able to transfer items from RBY to GSC and RSE - even the Stadium and Stadium 2 games won't let you do this as the items are separated into color and metal boxes! As soon as I get the free time to code the remaining sections I'll develop this into a save editor not just PKM converter. All the offsets/info needed are on Bulbapedia, I know how I'll code it out.

This is all very, very interesting. I like what the future holds.

However, by any chance, if you do find the memory locations of the Registered Pokémon, would it be okay if you could publish them?

But, anyways, thanks for all your work. I really like what you're doing with all this.

[Metropolis]

Thanks again. An update regarding the Pokemon Stadium file: Bad news is a simple hex search for matching data reveals nothing, but there is hope. Viewing the .fla and .pj save files in a hex editor, I can see the bytes 4F 50 00 00 7F 01 45 4B appearing more than once. Translated to ASCII this gives OP?? ??EK which means the data is being stored little-endian instead of big endian. This is the case with Ruby/Sapphire too and makes searching for text a little more complicated as each byte has to be flipped around in a special way. In layman's terms "it writes each bit of data backwards". I'll translate a PC Pokemon's data this way and query it, hopefully will reveal the offset of the data and I can adjust my algorithms to write/read the code from Pokemon Stadium and Pokemon Stadium 2!

Current stage of the program: Full read/write functionality from any generation 1 game save to another. The Pokemon are loaded in a way that the conversion routines in the test program above will run on them, so it's very close to completion. Tested a load of all Party and PC Pokemon from a Red save and write of them all onto a Blue save, and it works perfectly. I still need to implement pokedex flags and items etc, but quite a cool tool in itself for a player who felt like starting the story afresh without losing their Pokemon!

What features would people like to see? I think I'll go with the "new story" feature that would reset all but the "core" pokemon/items/pokedex/name/etc.

[Metropolis]

Update: New Game+ is finished and tested.

If anyone's interested in trying it out, let me know and I'll compile it, otherwise will include the feature in the first main release. There's one issue that I still need to iron out - if any key items are deposited in the PC, they don't get deleted when the key items pocket is empty which would lead to 2 Super Rods etc. I'll add a routine to search the PC for any deposited key items and delete those, otherwise it's working perfectly for Red/Blue/Yellow/Gold/Silver/Crystal. Something I've always wanted to see - the ability to go through the story again to collect the other fossil, eevee, hitmonlee etc without losing your Pokemon, items, money and Pokedex entries!

Edit: Bug fixed, now working on 3rd gen support and user interface.

Edited November 26, 2013 by Metropolis

[Metropolis]

Latest news:

Project approaching completion - full

support for loading/saving all gen 1 and 2 games, with a range of mass edit functionality e.g "everything shiny".

Gen 3's checksum has been giving me issues - got the item encryption and block shuffling to work as well as the Pokemon data shuffling. No longer getting full save corruption but still need to sort out the Pokemon checksums - it's all bad eggs at the moment! It's only eight lines or so of code amongst thousands of lines, so can flesh out the GUI and release a fully functioning .exe version as soon as that's sorted.

If anyone has any more details or standard code to perform the checksum as described in Bulbapedia would be much appreciated.

[POKEMONMASTER260]

Latest news:

Project approaching completion - full

support for loading/saving all gen 1 and 2 games, with a range of mass edit functionality e.g "everything shiny".

Gen 3's checksum has been giving me issues - got the item encryption and block shuffling to work as well as the Pokemon data shuffling. No longer getting full save corruption but still need to sort out the Pokemon checksums - it's all bad eggs at the moment! It's only eight lines or so of code amongst thousands of lines, so can flesh out the GUI and release a fully functioning .exe version as soon as that's sorted.

If anyone has any more details or standard code to perform the checksum as described in Bulbapedia would be much appreciated.

Any news regarding the Stadium games? Also, congrats!

Sorry if I'm bothering you too much about it...

[Metropolis]

Nah it's fine, ask away. It's lower on my priorities than getting a release-able working version, but will definitely be implementing Stadium and Stadium 2 as the next step.

I'm exactly the same - would love to see a Stadium editor to directly load in registered pokemon and edit the N64 boxes. Also possible would be a Pokemon Box RS editor, but it's such a niche game there isn't much of a point. I mean it only exists to store Pokemon from RS, which a save editor kinda does already lol.

Whereas editing Stadium allows bypassing the transfer pak system, which would be cool. I'll keep you up-to-date. In the meantime, any help at all with the Pokemon Data Checksum would be appreciated - I'm at a standstill with it. There's other things I can test/implement without it (such as reading in Pokemon and editing them, converting Pokemon from gen1/2 and back), but writing Pokemon to Gen3 isn't possible at the moment.

Worst case scenario I'll release a temporary version which offers Gen 3 to Gen 2 only, in addition to full Gen1/2 functionality. You'd be able to transport Pokemon back to GBC games, just not forwards until I sort out the checksum!

[POKEMONMASTER260]

Nah it's fine, ask away. It's lower on my priorities than getting a release-able working version, but will definitely be implementing Stadium and Stadium 2 as the next step.

I'm exactly the same - would love to see a Stadium editor to directly load in registered pokemon and edit the N64 boxes. Also possible would be a Pokemon Box RS editor, but it's such a niche game there isn't much of a point. I mean it only exists to store Pokemon from RS, which a save editor kinda does already lol.

Whereas editing Stadium allows bypassing the transfer pak system, which would be cool. I'll keep you up-to-date. In the meantime, any help at all with the Pokemon Data Checksum would be appreciated - I'm at a standstill with it. There's other things I can test/implement without it (such as reading in Pokemon and editing them, converting Pokemon from gen1/2 and back), but writing Pokemon to Gen3 isn't possible at the moment.

Worst case scenario I'll release a temporary version which offers Gen 3 to Gen 2 only, in addition to full Gen1/2 functionality. You'd be able to transport Pokemon back to GBC games, just not forwards until I sort out the checksum!

It's fine, I can wait, thank you!

BTW, when you release the next version of this program, can you still keep the original version online?

[Metropolis]

The original version will be entirely superseded by the GUI implementation that loads savs, but I won't be deleting the command line version I posted originally.

If you'd like a command line PKM to PKM converter like the original but without the bugs (there's been a few glitches ironed out) I can do that.

[codemonkey85]

In the meantime, any help at all with the Pokemon Data Checksum would be appreciated - I'm at a standstill with it.

Well according to Bulbapedia, the Pokemon data checksum "is computed by simply adding all the unencrypted values one word at a time". Are you having trouble doing that?

[Metropolis]

Yes! I've done much more complicated things including the full save block rotation and checksum, but whatever I try it's bad egg after bad egg! I'll keep trying variations.

Probably doing something wrong with the endianness - got code to flip the bytes right elsewhere but it's trickier bit by bit to set the egg flag and things.

[codemonkey85]

Hey Metropolis - could you provide me with a test Pokemon from Gen III? Or better yet, maybe a quick app that just dumps a bunch from a Gen III save file? I'd like to try doing the checksum myself, and I need some test cases to test it out.

If I find any success I'd be happy to share my code with you.

[POKEMONMASTER260]

Hey Metropolis - could you provide me with a test Pokemon from Gen III? Or better yet, maybe a quick app that just dumps a bunch from a Gen III save file? I'd like to try doing the checksum myself, and I need some test cases to test it out.

If I find any success I'd be happy to share my code with you.

Speaking of sharing code, I was just about to ask if Time Machine could be open source. Maybe share it on GitHub?

[codemonkey85]

Bam.

void calcchecksum(gbapokemon * pkm)
{
uint32 sum = 0;
uint16 * word = new uint16();
word = reinterpret_cast<uint16*>(&(pkm->encrypted));
for(int i = 0; i < 24; i++)
{
sum += word[i];
}
pkm->checksum = sum;
}

This is in C++. I used some structs and typedefs but you should get the idea.

If you have any questions ask away. You can find me on the IRC as well.

Edited February 23, 2014 by codemonkey85

[Metropolis]

Speaking of sharing code, I was just about to ask if Time Machine could be open source. Maybe share it on GitHub?

I'm happy to share any details about the program and anyone is welcome to use any code from it they'd like in their own project just send me a PM and I'll share the code, but would rather not make the whole source code public and allow someone to pass off the whole program as their own.

Thanks codemonkey85, I've finally got the checksum working but the C++ doesn't translate into Java well. There's no unsigned primitives here, it's a major pain!

Currently I only have read/write support for party pokemon, just need to figure out the way stored pokemon span the section boundaries, it's fiddly.

The migration itself works, but a few issues to iron out before I'll release the fully working version:

Ability flag: According to Bulbapedia it's PID mod 2, but neither that nor (PID mod 2) flipped works? In each case I get at least one party member with a glitched ability. Does this depend on the species itself as well as the PID? I'm aware of the subtleties with abilities changing upon evolution but can't see what's going wrong here at all

Pokerus: Bulbapedia isn't clear here, so the program currently erases Pokerus. I'll do a bit of digging through the hex codes to see how to support this, but any advice would be helpful.

Japanese characters: I haven't supported Japanese characters yet, so the program always leaves spaces where the Pokemon had a japanese symbol. Not an issue for Gen1 or 2, but since it's possible for english Gen3 games to have japanese pokemon on, I should support this.

[Metropolis]

Demo Version 0.2 - Gen 1 PKM only

Attached is a working demo with full GUI interface. If your computer has Java installed, double-clicking the .JAR file should open the program. This is mainly to test compatibility - it would be really helpful to know whether you are able to run this and if an update of Java is required :wink:

Don't get too excited - currently only Generation I PKM files are supported, but the demo will give you an idea of what to expect from the full program. I've coded lots more behind the scenes, including full read/write support for Gen 2, Gen 3 and conversion scripts. The full version will support batch migration as opposed to opening up an editor for individual PKM files, again this is all coded but I need some time to design the user interface.

Included is a sample PKM file, but others from existing Generation I editors should load in too. Please try and break it and let me know what you did!! Try setting the move PP and experience to 999999 and see what happens - the validation is designed to be foolproof and prevent any glitches whatsoever from being introduced. Note that Struggle is unavailable on the dropdown and Pokemon with duplicate moves will have them fixed. Try deleting all of your Pokemon's moves etc.

In the next version, Gen 2 will be supported, though again PKM only at this stage and the "Convert to Gen 2" button will work. The program will detect which sort of PKM file was selected and open up the appropriate editor window.

Thanks for the help and feedback everyone.

Metropolis

Edited February 18, 2014 by Metropolis