Pokemon Ranger Save FIle Encryption?

[BlackShark]

Does anyone know how the save files of the first Pokemon Ranger game are encrypted? Or at least if there is any kind of encryption that has been commonly used for other NDS games which might have been used for Pokemon Ranger as well?
Shadows of Almia and Guardian Signs are not encrypted and I don't know any other NDS Pokemon game that has a fully encrypted save file.

Any idea about this matter would be appreciated, thanks.

[BlackShark]

Nobody got an idea?

[theSLAYER]

12 minutes ago, BlackShark said:

Nobody got an idea?

I've got no clue. Are they even encrypted?

(I assume that you determined that with before and after saves? )

[Guest]

The thing is we have a Japanese save file with all 3 DLC packages WMission (Deoxys Mission & Manaphy Mission), Celebi Mission & Mew Mission. Thanks to this website just btw: https://ux.getuploader.com/savedate/download/226.

The Mew Mission was extreme rare and I have been looking for it for quite a while. (Years!)
It was distributed in partnership with ANA and you were only able to get it on respective airport terminals for the most part. Now the save file with all missions got finally leaked this year so we are damn lucky! The problem is the missions are already played and we would like to have a blank save file with the untouched special missions.

One idea from my side would be to maybe edit everything from memory and then save the game. It's complicated too though and regardless of which way we go some research might be required, to learn what activated the missions ect... since there was no option to delete the save file but a new game option, I kinda wonder how fresh a newly started game is? At least the missions won't be removed from what I remember?

Edited September 28, 2018 by ajxpk

[Guest]

Meanwhile I did some reverse engineering and I found the encryption/decryption algorithm, it's this function...

 unsigned int __fastcall PokemonRangerCrypt(int a1, unsigned int s_rand, int size, _DWORD *dest)
{
  unsigned int i; // r5
  unsigned int result; // r0
  unsigned int rand; // ST04_4
  unsigned int h_rand; // r0

  for(i = 0; i < size/4;)
  {
    rand = 1566083941 * s_rand;                 // the initial seed is located before the decrypted/encrypted data
    h_rand = rand;
    s_rand = 1566083941 * rand;
    *dest ^= h_rand & 0xFFFF0000 | (s_rand >> 16);
    ++dest;                                     // this increments by 4 (u32 data)
    ++i;
  }
  return result;
}

The initial seed for this comes right before the encrypted data itself... for example, the first two chunks include a header with the string "PKR-059 a00 ".
Right after that (loc 0xC) there is the initial seed which becomes the first "s_rand" value, the encrypted data follows (at loc 0x10). The encrypted chunk size for the first chunk is 496 bytes, the 2nd chunk starts at location 0x200, size of this chunk is still unknown. 

We can't just loop through this thing from start to end and have to figure out where each chunk starts and hopefully their seeds are stored there as well, but I think this should be easy from here now and tests should reveal that... It's still unknown how the initial seed was originally calculated... (could be a checksum?). At least the two first chunks are sharing the same seed. It looks like there could be different initial seeds for other chunks, so if something doesn't decrypt properly watch out.

Edited March 13, 2021 by ajxpk

[BlackShark]

@ajxpk nice work! Thank you very much!

Spoiler

I was able to figure out the offsets of the other chunks/seeds.

Seed	Start	Length
0x00C	0x010	0x1F0
0x20C	0x210	0x1F0
0x400	0x404	0x2620
0x2B00	0x2B04	0x2620
0x5200	0x5204	0x30E8
0x8300	0x8304	0x30E8
0xB400	0xB404	0x30E8
0xE500	0xE504	0x30E8
0x11600	0x11604	0xE7FC
0x1FE00	0x1FE04	0x67FC
0x26600 0x26604	0x4FFC
0x2B600	0x2B604	0x57FC

The checksum of a chunk is a simple CRC16 right after the seed (first 4 bytes of the encrypted data).

Ok, forget what I wrote previously.

Only the first two chunks are static they contain the information about the other chunks.

Save Info Chunk (512 bytes)
0x000-0x00B	PKR-059 a00 
0x00C		seed
0x010		CRC16 over the next 492 bytes
0x014		time stamp
0x018		time stamp (see https://projectpokemon.org/home/forums/topic/45846-pokemon-ranger-save-file-encryption/?tab=comments#comment-265265)
0x01C		always 0x000000A0
0x020-0x1FF	chunk info entries

Chunk Info Entry (48 bytes)
0x00 file name
0x1E number of copies
0x20 offset
0x24 data size
0x28 reserved space

Each of the other chunks has a 24 byte header.

Chunk
0x00 seed
0x04 CRC16 over the chunk data starting at 0x18
0x08 unknown
0x0C unknown
0x10 save counter
0x14 always 0x00000001
0x18-0x?? chunk data

 

A small tool to decrypt/encrypt and fix checksums for Ranger saves (use with caution!): https://github.com/Bl4ckSh4rk/rangercrypt/releases/latest

[Guest]

Good work!

Now I finally understand why there were 3 functions for decryption. One was for the first two chunks with the save info, the second is a hardcoded decryption function with a * 5 loop to decrypt the first 20 bytes of a chunk, (I guess just for save file checks) and the third one for the rest of the non-static chunks.

This is the directory list struct based on my understanding.
(Updated! @BlackShark Sorry for the confusion, I think I mixed the numbers up a little, the struct with the save info is complete now...)

Savedata List
0x00 File name (28 chars)
0x1C Unknown (u16)
0x1E Mirror (u16)
0x20 Address (u32)
0x24 Data Size (u32)
0x28 Sector Size (u32)
0x2C Unknown (u32)

 

file name		type	data size	sector size
siskin.tsd		2	0x260C		0x4E00
wren.tsd		2	0x30D4		0x6200
hazel.tsd		2	0x30D4		0x6200
set_delivery001.dat	1	0xE7E8		0xE800
set_delivery003.dat	1	0x67E8		0x6800
set_delivery002.dat	1	0x57E8		0x5800
set_delivery004.dat	1	0x4FE8		0x5000

The most interesting one is siskin.tsd, which starts with the setting data, it also includes the ranger id (offset 0x10) and data related to the special missions, like the "new mission" flag (offset 0xA) and manaphy egg information at offset 0x8 (not received yet/received/sent) and the time info when it was obtained year/month/day (starts at offset 0xC). There's also the title and desciption strings, wren.tsd apparently includes player info... the name is at offset 0xE for example. Last but not least, hazel.tsd is used for quick save/save states... 

And then we have the Special Mission .dat files... the international versions have those localized included in the directory data\mission.

Edit:
I extracted the special mission .dat files and attached them to this post
In addition I attached a thingy table which I made in case you want to look for strings.

Pokemon Ranger Special Missions (JP).zip Pokemon Ranger Character Encoding (JP).tbl

I still don't know how we can utilize the .dat files at this point... they seem to include all the data that is needed for the missions, including the title and description, but just using them with a ZP3J cartridge doesn't seem to work, I still get this screen when I boot the game with it...



I will have to find out in what kind of form it wants to receive the data. I already located the functions (in version 1.1), the Ranger Net jump table function for Slot-2 is in overlay_0009.bin (loc 0x020FC628), there's the call to a familiar function that gets the cartridge data at 0x0209A558 (it's almost identical to the one in Diamond & Pearl ect.). It seems everything is ok, At least it gets the size information from 0x8100000 and the data starting from 0x8100010. Just for some reasons it returns false when the data is checked. Maybe because of failed validation checks. Now if I force it to return true I get this btw...




I found some pictures online related to the download process...



Source: https://www.inside-games.jp/article/2006/08/12/18877.html

Edited September 4, 2022 by ajxpk

[BlackShark]

2 hours ago, ajxpk said:

I attached the extracted .dat files. Still don't know how we can utilize them...
In addition I attached a thingy table I made in case you want to look for strings.

With a modified siskin.tsd the missions will show up ingame. Idk where the data comes from, it probably gets written there during the download process.

Now I need to find a save file that doesn't have any missions yet to test if the injected misiions are properly playable.

Pokemon Ranger (JP).sav

[Guest]

Recently I found out that when you do something like writing on these encrypted save file chunks the game will delete them.
When you for example overwrite the encrypted "PKR-059 a00 " save block (or its seed) it erases the whole save file. And that's for example how you can delete the save game but still keep the special missions... I think the same will also work vice versa, so you can just destroy the data/do anything that the sum check fails. 

The field at offset 0x1C is a u32 value and always 0xA, it's set when the game creates the header. Can be used to check whether a save file is already decrypted/encrypted. Also... Offset 0x10 of these other save file headers could be some kind of save count. Looks like the 2nd is really a backup save, its chunk is created when you save for the 2nd time (before that it only has these blocks only have the save file header).

Edit:
Meanwhile I located the functions for checking the agb ("beacon") & mission data. The first 128 bytes starting from offset 0x08100010 is an array which is swapped for some kind of cryptostuff which reminds me on RC4, but tbh I don't know much about this and it looks tough to reverse engineer so I think I will leave it there. If I understand it correctly it's in fact very similar to what I've seen in the localized versions of Diamond & Pearl etc. where it does a signature verification with 128 bytes from 0x08020000, it looks almost identical (just the key is different from what I've seen). In addition there are SHA1 checks for the mission data starting at offset 0x08100090, apparently the total size checked is the size of the full data -191 bytes. I didn't even go further (it's possible that the data is compressed), this whole thing is more complicated than I thought, a lot research would be necessary to make it work...

If someone is interested in it... the initiate BL to the subroutines for the SHA1 and the cryptostuff is at offset 0x020B0284. The function that makes the .dat save file block and updates the main save (PKR) is at 0x02001D80. I think the best approach would be to figure out how saving works in detail, which would have to be done either way as we need to know more about these other values from the header...

Edit 2:
Btw. Meanwhile I know where the Special Mission data struct from siskin.tsd is coming from, they were installed from the .dat files...
This also exists in the localized versions, the data is for accessing the missions via Ranger Net.  The D00X struct is always 100 bytes copied straight from the .dat or maybe it's coming from another payload, we might never know unless someone wants to give it a try and reserve engineer it. Missions are sorted by their IDs, regardless in which order they were downloaded. The data includes information like mission ID, lengths (u16) for description and client name and strings for title, description and the name of the client.
 

set_delivery001.dat

Mission: D001 (0x50D0)
Description offset: 16 chars
Client name offset: 97 chars
Title: "だいじなタマゴを　とりもどせ！" (0xD490)
Description:
"シンバラきょうじゅが　てにいれた
マナフィの　タマゴを
ゴーゴーだんに　うばわれてしまった！
かれらに　あくようされる　まえに
だいじな　タマゴを　とりもどせ！" (0xD4B0)
Client: "シンバラ" (0xD554)

set_delivery003.dat

Mission: D002 (0x1D10)
Description offset: 16 chars
Client name offset: 111 chars
Title: "デオキシスと　わかりあえるか?" (0x5874)
Description:
"なんらかの　りゆうで　こうげきてきに
なった　デオキシスが　はっけんされた。
まわりの　ポケモンや　にんげんに
ひがいが　でるまえに　デオキシスを
せいじょうな　じょうたいに　もどすのだ！" (0x5894)
Client: "ハヤテ" (0x5954)

set_delivery002.dat

Mission: D003 (0x1BC0)
Description offset: 13 chars
Client name offset: 107 chars
Title: "セレビィを　すくいだせ！" (0x43A4)
Description:
"ライラのもりに　あらわれた　セレビィを
ゴーゴーだんの　れんちゅうが
ねらっているという　じょうほうが　ある。
セレビィを　ぶじに　ほごするために
かれらよりも　さきに　キャプチャせよ！" (0x43C0)
Client name: "ハヤテ" (0x447C)

set_delivery004.dat

Mission: D004 (0x1520)
Description offset: 15 chars
Client name offset: 115 chars
Title: "まぼろしのミュウを　さがせ！" (0x397C)
Description:
"まぼろしのポケモンと　いわれている
ミュウが　ジャングルで　もくげきされた。
シンバラきょうじゅの　けんきゅうの　ため
ひとりの　レンジャーの　めいよの　ため
ぜひ　ミュウを　キャプチャしてほしい！" (0x399C)
Client name: "シンバラ" (0x3A64)

Edited August 8, 2022 by ajxpk

[Guest]

It turns out that the PKR-059 a00's header fields at offset 0x14 and 0x18 are time stamps representing minute, second, hour (u32) and year, month, day (u16).
The .tsd and .dat save headers have their time stamps too at field offset 0x8 (minute, second, hour) & 0xC (year, month, day). 

date(u16) format = year, month, day
time(u32) format = minute, second, hour

PKR-059 a00
date = Previous date from header offset 0x18 (default value = 0xFFFF)
time = Previous time from header offset 0x14 (default value = 0xFFFFFFFF)

.tsd & .dat
Whatever is currently on the stack (gets XORed all the time from what I've seen so far)

*date = *date & 0xFF80 | year & 0x7F;
*date = *date & 0xF87F | ((month & 0xF) << 7);
*date = *date & 0x7FF | ((day & 0x1F) << 11);
*time = *time & 0xFFE0FFFF | ((hour & 0x1F) << 16);
*time = *time & 0xFFFFFFC0 | minute & 0x3F;
*time = *time & 0xFFFFF03F | ((second & 0x3F) << 6);

Field at 0x1A (u16) (PKR-059 a00 ) is a count. It's defaults at 0 in increments when the file is created and each time the file is overwritten until it maxes out at 0xFFFF (it doesn't overflow). With siskin.tsd, wren.tsd, hazel.tsd, set_delivery001.dat, set_delivery003.dat, set_delivery002.dat & set_delivery004.dat set it reaches 8 and will never be updated from here. Field 0x1C is always 10, the value is used as the maximum of files that can be saved and to calculate the reserved space (0xA * 0x30 + 0x20 = 0x200).

.tsd & .dat files have a save count too at 0x10 (u32), which serves to select the current mirror, the top .tsd file being saved first takes the odd and the 2nd takes the even numbers. It maxes out at 0xFFFFFFFF without overflowing, there is an additional count for the block at 0xE (u32), which increments with each time the block is saved. It maxes out at 0xFFFF (again, no overflow).

Field 0x14 in .tsd & .dat is used for enabling/disabling a save block, mainly used for the backup where the first time these save files are created an empty disabled backup block is created. Hazel.tsd (used for save states) makes use of this commonly as save states are deleted when they're used.

PKR Main Save File header
Adrs	Description		Type
0x00	"PKR-059 "		str
0x07	"a00 "			str
0x0C	Seed			u32
0x10	CRC16			u32
0x14	Time			u32
0x18	Date	 		u16
0x1A	Count			u16
0x1C	Save Data Table Max	u16
0x1E	Unknown			u16

 

Save Data header (.tsd & .dat)
Adrs	Description		Type
0x00	Seed			u32
0x04	CRC16			u32
0x08	Time			u32	
0x0C	Date			u16
0x0E	Block Count		u16
0x10	Global Count		u32
0x14	Enable Flag		u16
0x16	Unknown			u16

Btw. the seeds for the encryption is created by via timer 0 (REG_TM0D) + a counter of its overflow (REG_TM0CNT = TIMER_ON | TM_IRQ | TM_FREQ_64).

(REG_TM0D | (TM_IRQ_count << 16)) + 1

Edited February 8, 2022 by ajxpk

[NickPlayeZ]

On 3/8/2021 at 10:15 PM, BlackShark said:

A small tool to decrypt/encrypt and fix checksums for Ranger saves (use with caution!): https://github.com/Bl4ckSh4rk/rangercrypt/releases/latest

Hey there so I was just trying to use this but when I drag my (german) ranger save onto it (open the save with that) it just quickly opens the command thingy and immediately closes it again without having any effect from what I can tell... I'm not really familiar with programming or anything like that; do you know what I might be doing wrong?

 

1 hour ago, NickPlayeZ said:

Hey there so I was just trying to use this but when I drag my (german) ranger save onto it (open the save with that) it just quickly opens the command thingy and immediately closes it again without having any effect from what I can tell... I'm not really familiar with programming or anything like that; do you know what I might be doing wrong?

all good now for some reason worked after trying it 3 more times? Weird but alright

 

Another Question though:
whenever I decrypt, then change some stuff and try to encrypt it again it wont work (even if I revert any changes I made back). Why could that be?