How hard is programming with the .sav file?

[greentea]

I've done extensive programming with the pkm file, I've got almost all the details from the pkm file completed. Now I'm considering making a project with the sav file. A suggestion was a search function to search through your pkm files in the save to sort out files. But I looked through and it doesn't seem as linear as pkm files.

Is it hard to do? I would only plan on displaying a couple of basic things like the OT and stuff which are simple, and then the PC Pokemon stuff. I just know that it's not as simple cause I looked at it. Looks like it's encrypted or something. If it's not too hard, would someone mind telling me how to do the box stuff?

Thanks.

[codemonkey85]

The only data in the save file that is confirmed to be encrypted is the PKM data; we do suspect the Battle Videos too, but as far as I know no one has actually cracked that structure yet.

Decrypting the PKM data is not hard. It is explained in a straightforward manner right here. The PKM data is located in various places in the save file; boxed Pokémon are located in the "big block" or "storage block", and the rest of the Pokémon (party, Day-Care Center, roaming, etc.) are located in the "small block" or "general block".

The general outline of save blocks are discussed right here, and what we know about loading and / or saving a save file is on that same page.

If you have any specific questions, feel free to post 'em!

[SCV]

Well, the problem with the sav is that its MUCH bigger than a pkm file. As such it is much harder to find where things are. Once locations for things are know its usually not that hard to make a program to edit it. Of course there are things like Battle Videos, that are harder.

[codemonkey85]

Very true, although it sounds like greentea is only (currently) interested in the Pokémon themselves. The save file in its entirety is, as we all know, quite a grand undertaking.

Heck, Diamond and Pearl have been out in Japan since September '06 and we still don't have their save file mapped out entirely yet.

[greentea]

It first started that I was only interested in programming certain parts of the pkm file, but it eventually grew into programming everything. I've got what's essentially a pokesav edit screen that I've made entirely myself (less the stuff you helped me out with).

I'll take a look at what you posted and see what I get. Right now, I'm only concerned with the Pokemon in the sav file and particularly only the PC Storage ones.

What I plan to do is some sort of program that can save pkm files out of the sav file like Pokesav. But what I also plan to do is have the user be able to define some sort of search parameters and then display the pokemon that meet requirements and display them. So if you say bred 50 pokemon and had the eggs, you could search which ones had good IVs or if you wanted to find all the EV trained Pokemon in Attack, or even all the, say 10 ANIV events.

[codemonkey85]

If that's your goal, then I suggest you build a Pokémon class structure to hold the properties for the stats you are interested in. Then you could build a fairly simple search algorithm to seek out and / or sort those properties by their numeric values.

In my software, it looks something like this:

Dim PKM As New Pokemon = SaveFile.StorageBlock.Boxes(0).StoredPokemon(0)

If PKM.IVs.Attack = 31 Then MessageBox.Show(PKM.Nickname & " has a perfect Attack IV!")

[greentea]

I plan on doing something like that, but filtering for all pokemon on the save that meet the criteria.

Other thing is I have limited programming knowledge. I've got a book I'm working through but I still don't know too much. I have one crappy course worth, and the last project made me google/ask experts for help with. My idea of a database of information right now is a record of arrays.

[codemonkey85]

Personally, I don't think a record of arrays hardcoded into the software is a good idea. I think you should look at doing a binary database, which can be small, fast, and easily updateable / extendable... not to mention I believe using arrays would cause your program's footprint to be much larger than necessary.

[greentea]

I hope this book has that, because I do want to learn. I think it has a section on databases so I'll look that up soon.

For now, using records of arrays works for me. It's obviously not the best method, but at the end of the day, if it can do what I want, then I have no complaints really. It does what I need it to do, and works fast enough not to care. I try to use whatever knowledge I can first before having to search new stuff.

I use text files to store the information and read them in as variables into the arrays, I think it's better than hard coding, especially when there's a large upper bound, but obviously not as good as your method.

But again, it works fine right, program doesn't take forever to load so it's OK.

[greentea]

OK, let's see if I've got this correct.

There are two pairs of blocks and each block has three parts, the general, storage and hall of fame. For my purposes though, I will only concern myself with the storage block. I may display a few things in a potential program from the general block such as trainer OT, ID and SID, but I won't concern myself with that for now.

The first storage block starts at C100 and the second is at 4C100, but one is just a copy of the other, the backup. I think each 128 values is a new pkm file in the save?

The PKM info of the .sav is encrypted so I have to decrypt it:

using X[n+1] = (0x41C64E6D * X[n] + 0x6073)

1) sub X[n] as the checksum value where the checksum value is:

the sum of the encrypted hex from 0x08 to 0x87 truncated to 16 digits.

2) Y xor rand() where:

Y is the is the 2-bit word (hex value) from 0x08 to 0x87 and

rand() is X[n+1]

This value becomes the unencrypted byte (of what?).

3) Unshuffle the blocks using the algorithm and equation:

((pv >> 0xD) & 0x1F) % 24

where pv is 0x00 to 0x03 and

the rest is straight forward enough, it's just math stuff

(Is the pv there the encrypted pv of the pkm file in the .sav file?)

then I rearrange the values of the pkm file in the .sav file according to the decimal value. I essentially assign A as the first 32 hex values of the file and continue through to D as the last 32 hex values.

(how does it work if it's 128 hex values when I use values over a decimal of 128 from the pkm file?)

As you can see I'm a little confused, but I think I get a majority of it. Hopefully I can get some understanding of it soon so I can actually begin to program something later this week.

Now I should have the information for the pkm file as if it were a pkm file right?

[codemonkey85]

There are two pairs of blocks and each block has three parts, the general, storage and hall of fame.

That's not quite right... the save file is split into two equal halves. One half is a backup of the other half. Each half consists of three blocks; one "general", one "storage", and one "hall of fame". (It sounds like you had the right idea though.)

For my purposes though, I will only concern myself with the storage block. I may display a few things in a potential program from the general block such as trainer OT, ID and SID, but I won't concern myself with that for now.

I think you need to at least check the footers on those blocks to make sure you're reading the correct information though, as they are all linked together in one half of the save file.

The first storage block starts at C100 and the second is at 4C100, but one is just a copy of the other, the backup. I think each 128 values is a new pkm file in the save?

Don't forget that one of those storage blocks is not "current", and is only a copy of the previous save. The two blocks should not be expected to be identical. Also, each storage Pokémon is 136 bytes long; the first eight bytes are unencrypted data (the PID and checksum basically), and the last 128 contain the encrypted data, which is everything else.

Y is the is the 2-bit word (hex value) from 0x08 to 0x87 and

rand() is X[n+1]

This value becomes the unencrypted byte (of what?).

Once you apply the RNG to each 2-byte (not bit) word, you can split those words into the unencrypted bytes. The data should be the same length, just now it is unencrypted.

(Is the pv there the encrypted pv of the pkm file in the .sav file?)

The "pv" is the personality value, or the PID, of the Pokémon in question.

then I rearrange the values of the pkm file in the .sav file according to the decimal value. I essentially assign A as the first 32 hex values of the file and continue through to D as the last 32 hex values.

(how does it work if it's 128 hex values when I use values over a decimal of 128 from the pkm file?)

I'm not entirely sure what this question means, but you apply the shuffling algorithm to the encrypted data only, so you skip past the first eight bytes and unshuffle the remaining 128 bytes in four groups of 32 bytes each.

Now I should have the information for the pkm file as if it were a pkm file right?

That's the idea.

[greentea]

So two separate parts, one is a backup of the other. I guess it shouldn't always be an exact copy since it's an archival backup in-case of a save fail, but I'm only going to access one so it works the same if I think of it that way.

OK, so 136 bytes for the entire pkm file with the first 8 being the unencrypted values of the pv and checksum.

And I guess I understood the rest of it, so hopefully I'll get a chance to crack this tomorrow and see how it goes.

[codemonkey85]

...but I'm only going to access one so it works the same if I think of it that way.

But like I said before, you really need to look at both halves in order to determine which one is more current. After you figure that part out, then yeah, for your purposes you will only work with the current half.

[greentea]

Oh, I get what you mean now. I should check the date on both to see which one was the last saved one. Makes sense. I guess I'm going to display that general block info first then, might as well if I'm grabbing the date anyways.

[codemonkey85]

greentea, I think you should go back to those links from my earlier post and read the information on the wiki carefully. The last one I posted was a link to the exact functions used to determine which blocks are more current, and there is nothing to do with a date. To wit:

When one starts a new game for the very first time, every byte of the save file is set to 0xFF. The first time the game is saved, the "current" info is written to the second half of the save file. This is important to note when loading a save.

When the game loads a save file, it first compares the small block save count of both general blocks. If the general block with the highest save count has a correct checksum, then the game moves on to check the storage blocks. The storage block with the highest storage block save count is chosen first, and if its general block save count matches the currently chosen general block (and has a correct checksum), the file is successfully loaded. If both storage blocks have the same storage block save count, then the storage block with a higher general block save count is chosen instead, and the if the checksum is correct, the file is successfully loaded. If the checksum was incorrect for either block, the game falls on the other pair of general and storage blocks; if the checksums are correct in both, the game reports a save file corruption error, and that is the data loaded. If the checksum in either block is incorrect, the game reports a save file corruption error, and the save file is not loaded. A new game must be started at that point.

Bear in mind that the Hall of Fame block had not been thoroughly examined when I wrote this, and for all we know the footer is checked for that block as well. However, the information there is only written to when the Hall of Fame is being updated right after a victory against the Champion, so it tends not to matter much.

[greentea]

Ya OK, I have to check the save count. I remember reading it at like 11 or something when I was really tired and supposed to be reading for English.

[codemonkey85]

Well, don't let your project get in the way of your homework. And FYI, I'm generall most productive during the day, when I can think more clearly because I'm not falling asleep at the computer.

Let me know if you have any trouble writing out the RNG and stuff.

[greentea]

I don't do my homework most of the time anyways, I'm considerably lazy, and yet I'm still doing fairly well. I've got a long weekend that I'm going to use to study Chemistry, read English and do some Calculus.

I prefer to program in the morning shortly after I get up. Even before I eat or anything, don't know why, but that's when I always programmed when I was loafing during summer. However I rarely have time when I wake up, I still wake up early enough to do minor things but I wish I had weekends (I hate work and I honestly don't need to work). Typically I'd work after dinner or something like that, it depends where it fits into my schedule.

And as always, I'll have a dozen questions and this topic will be 10 pages by the time I'm done