Zaz Posted December 16, 2013 Share Posted December 16, 2013 I'm currently trying to figure out Gen 3 game saves, and it is going quite well, however when reading discussions on this forum I find one thing confusing. While discussing PKM files, the consensus seem to be that storing the data in decrypted form is much better than the alternative. I'd like to argue that the opposite is true. The main argument for the decrypted storage is, as far as I can tell, that parsing the files is faster. My counter to that is: Is that really relevant? My computer can parse around 50000 encrypted gen 3 files in a second, so unless gen 6 adds a considerable load, aren't we good performance-wise? The argument I wish to present _against_ decrypted storage is that the file format feels very unintuitive from an implementation standpoint. Basically, someone has to define exactly what 'decrypted' means. While the answer to questions like "Which is the correct subsection order?" or "Big endian or little endian?" may feel intuitive, someone that is not-official has to make that decision. While the encrypted format is pre-defined as valid when the game can parse the file, the decrypted files are valid when unofficial software considers them valid. This just seems strange to me. I would guess that the praxis grew out of the fact that one very popular piece of software started doing it and all others just conformed to that behavior. I figure every piece of software that handles pkm files already has parts that take the raw byte array (or stream) from the file save and creates some sort of internal storage representation to use internally. All software can also export the internal version back to the encrypted form. So why add another conversion to the _other_ external form? I haven't been coding save files for very long, so I expect that there are many things I don't know here, so feel free to educate me. : D And yes, I only made this topic because exporting encrypted storage files is much easier for me right now. Anyone curious about why can ask, but I guess most people wont be. ^^ Link to comment Share on other sites More sharing options...
evandixon Posted December 16, 2013 Share Posted December 16, 2013 If pkm files must be decrypted to be read by programs, encrypting them when saving them as pkms is an extra step. Especially when the only thing pkm files are used by are editing programs. If you want to store them encrypted, the typical extention is .bin. The real question is why have them stored encrypted? Link to comment Share on other sites More sharing options...
theSLAYER Posted December 16, 2013 Share Posted December 16, 2013 I agree with Evandixon entirely. To add on, I believe that GF had the encryption implemented in Pokemon to prevent data extraction and manipulation, however hackers triumphantly rendered it useless by cracking the encryption. Since the encryption is cracked, there isn't any point to save in encrypted form, as being able to extract the data, kind of defeats the purpose of having it encrypted. Link to comment Share on other sites More sharing options...
Zaz Posted December 16, 2013 Author Share Posted December 16, 2013 Evandixon, one thing I think you're skipping over without mentioning is my assertion that there isn't one single version of the 'decrypted' data. (But I will admit that it seems that everyone but me seem to use the same variant.) My argument is basically 'one encrypted version, many decrypted versions - hierarchically, the encrypted one is the only arbiter of truth'. Also, bin files - binary files - would be a fitting extension to both these kinds of files, since they both contain binary data. But that extension also carries no association with pokemon, so maybe .epkm or something would be better? ^^ Now the reason to store them encrypted I'd think be self evident. If i wish to copy one pokemon from one save file to another, I'd just copy the correct 80 bytes from one file, and write them to the other. No calculation, encrypting, decrypting, moving bytes around needed at all. Also it seems unneccesary to decrypt the entire file if maybe I'm just interested in some specific part of the file. Say I wish to just list which unique monster types are stored in a save. I don't actually need to calculate any sub sections for that, so 'decrypting' the data would be wasted effort And I admit that it is an extra step, but it is a step that every application that parses save files already has code for. theSLAYER, I like your argument, it seems almost philosofical in a 'stickin it to the man' kind of way. : ) You basically mean that the developers of the game have one version of the data, the 'truth' so to speak, and then add encryption and obfuscation to hide it from us. And we are smart enough to see through their deception and display the truth for all to see? Anyway, thanks for replying you guys, I really appreciate it. : ) Link to comment Share on other sites More sharing options...
evandixon Posted December 16, 2013 Share Posted December 16, 2013 The real reason they're stored decrypted is because Pokesav saved them decrypted, and everyone else did in their tools for the sake of consistency and compatibility. If you as a developer want to make your own tool that saves encrypted Pokémon as .epkm files, you can do that; however, users will be a bit annoyed if it doesn't support the standard .pkm everyone's used to. If you want to list unique monster types, you don't have any standard to follow. Users don't care whether or not the program internally decrypts the Pokémon. All they care about is that it supports their saves/pokemon and it works. Link to comment Share on other sites More sharing options...
Zaz Posted December 16, 2013 Author Share Posted December 16, 2013 Yeah, I know.. (still makes me a sad panda ^^ ) Do you know if the exact specs for a third gen pkm file are published somewhere? I guess its just XOR the stuff that should be xor'ed, and store the subsections in the order {species, moves, EVs, misc} since that is the order every example seems to give? Also, I just realized I lied about the type example thing. It is too found in the subsections, so that was a bad example. >_< Link to comment Share on other sites More sharing options...
theSLAYER Posted December 16, 2013 Share Posted December 16, 2013 Evandixon, one thing I think you're skipping over without mentioning is my assertion that there isn't one single version of the 'decrypted' data. (But I will admit that it seems that everyone but me seem to use the same variant.) ... You basically mean that the developers of the game have one version of the data, the 'truth' so to speak, and then add encryption and obfuscation to hide it from us. And we are smart enough to see through their deception and display the truth for all to see? Yup precisely! Also, I'm quite sure that everyone uses the same decrypted format, cause I don't think you will get a reliable result if you decrypted it using another method. (I once attempted decryption for Gen V, before I published my program. During the beta tests, messed up the rng Seed and block shuffling on two different accounts. that led to me never decrypting it, hence no program was able to read my decrypted files.) [Afaik] Think of it as the master key (our sharp eyes detecting the deception) to that one lock (the one truth). Using any other key (any other argument) will not decrypt the file correctly (instead corrupts the truth). Do you know if the exact specs for a third gen pkm file are published somewhere? I believe this is what you're looking for Gen III Pkm Data Structure Gen III Save Structure For Comparison: Gen IV Pkm Data Structure As a note, .pkm structure changed in every generation. Link to comment Share on other sites More sharing options...
Zaz Posted December 17, 2013 Author Share Posted December 17, 2013 I have used those bulbapedia articles extensively to parse the save files, but the focus of the articles are where to find the data in the encrypted file, not how to combine the result into a pkm file. It is only through subtext that the sub section order is understood to be what it is. Anyway, I know no one asked, but I will say it anyway. :b I simply load the encrypted data into a buffer and lazy load the encryption/decryption work until it is actually needed. It made for a very sleek and intuitive solution that was really easy to extend to cover the entire save file. At least it seemed like a cleaner solution compared to most other tools i've checked out, but then I couldn't find any sources for most any tools? Don't you guys share code in any way? I know codemonkey is on github and evandixion has code on codeplex, which is great, but those is the only one i have found? The other tools I had to reflect into to figure out how they worked. ^^ Link to comment Share on other sites More sharing options...
theSLAYER Posted December 17, 2013 Share Posted December 17, 2013 I have used those bulbapedia articles extensively to parse the save files, but the focus of the articles are where to find the data in the encrypted file, not how to combine the result into a pkm file. It is only through subtext that the sub section order is understood to be what it is. Anyway, I know no one asked, but I will say it anyway. :b I simply load the encrypted data into a buffer and lazy load the encryption/decryption work until it is actually needed. It made for a very sleek and intuitive solution that was really easy to extend to cover the entire save file. At least it seemed like a cleaner solution compared to most other tools i've checked out, but then I couldn't find any sources for most any tools? Don't you guys share code in any way? I know codemonkey is on github and evandixion has code on codeplex, which is great, but those is the only one i have found? The other tools I had to reflect into to figure out how they worked. ^^ To my understanding, you're doing Gen 3 .pkm files right? I'm sorry that I can't help you there; to my recollection, Pokemon Save Editing only kicked in full blow at the introduction of Gen IV. Back then there was Pokesav, then came in Jiggly's PPSE, some hints of Pokestock, and eventually Pokegen. Finally around the time of B2W2 release, appeared PikaEdit. It's only around the introduction of HGSS, then I noticed the surfacing of Gen III editing tools, Enciclopedia (spanish I think?) and Pokestock's "PokeGBA" or "Pokebox" w/ Gen III functionability. Gen III lasted between 2002 (JP RS) to 2005 (US E), with DP first released in Japan in 2006. It came close to decade (actual figure is 8 years +) for the most stable version (IMO) of Advanced Generation Editing to come to light, released only in the third quarter of this year: KazoWAR's A-save editor. Anything Gen III related before that, to me, was all RAM editing. My point to the long, long history lesson, is that I haven't seen any source codes for Gen III, Probably because some programs and sources might have been lost in time (File hosting sites died, data corruption etc), and it have might been deemed obsolete by some, especially due to "deemed complexity" of the nature of the file structure; the PID-IV relations, and whatnot. (also including the fact that Gen IV and V data can be edited easily with existing programs. only problem for newbs is they might fail to create gen III data on pokegen, due to palpark trash bytes) :'( Link to comment Share on other sites More sharing options...
Zaz Posted December 17, 2013 Author Share Posted December 17, 2013 Thanks for the info, I really appreciate that you guys take the time to explain these things. : ) Link to comment Share on other sites More sharing options...
codemonkey85 Posted December 18, 2013 Share Posted December 18, 2013 It is true - the whole reason we use decrypted PKM files is because COM (creator of Pokesav) did it. He didn't do many things correctly, but we learned some of his bad habits all the same. At least it seemed like a cleaner solution compared to most other tools i've checked out Really? Cleaner than just defining the binary layout in a multi-tier structure then creating a pointer at the memory location of the save file? I find my way so easy to work with in code! struct pokemon_obj : pkmunencryptblock,pkmblocka,pkmblockb,pkmblockc,pkmblockd {}; //... struct box_obj { // size: 0x1000 public: std::array<pokemon_obj,30> pokemon; // uint16 : 16; uint16 checksum; byte buf[0x0c]; box_obj() { memset(this,0,sizeof(box_obj)); } }; //... struct bw2savblock_obj { // //... std::array<box_obj,24> boxes; // size: 0x18000 //... }; Then I get to do nifty stuff like: std::cout << "The Attack IV of the Pokemon stored in Box 1, slot 1 is " << sav->cur.boxes[0].pokemon[0].ivs->attack << "\n"; Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now