[Gen 4] Pal Park Research

[ZZAZZ]

If I recall correctly, the Korean DS titles are compatible with all other language GBA carts, due to not having gen 3 releasing in Korean. (Maybe as a way to make up for the fact?) So if anyone has a DS flash cart with Korean roms (or the real Korean DS carts), and any of the non US language GBA carts, feel free to contribute migrated Pokemon from them, as I can't. I only have the US games at my disposal, and as mentioned before my Everdrive X5 won't get recognized by any of the DS games. So testing other gba languages isn't possible for me. 

 

I'll keep testing with these current "A" nickname saves for now, then maybe I'll submit a few more after you update them with blank names @ajxpk

Edited September 14, 2017 by ZZAZZ

[Guest]

I got some bad news. This is getting more inconsistent...
@ZZAZZ first of all, I can't thank you enough for these files. I just realized that what you did was in fact necessary.
Actually it looks like we're facing a huge problem...
It appears that the first Offset in these Trash Bytes slightly differs from migration session to migration session. 
I fear some research needs to be done to learn what these Offsets really mean. 
Also the 2nd Offset in the Trash Bytes differs as well, so the wikia article was right.
It's fix however in the Pokemon of Slots 2-5 of one session.  

For now there is nothing we can do about that and we will continue migrating Pokemon just as before to get more examples.
Hopefully over the time I can figure out about everything.

[Guest]

I also have some good news.

I did more migrations on Emulator, this time with the English Version of Pokemon Diamond (Rev 5).
The Trash Bytes look perfectly fine actually.

 1:

2B 01 FF FF 18 20 0D 02 42 00 00 00 00 00 00 00 48 A1 0C 02 E0 FF

 
2 - 6:

2B 01 FF FF 2D 00 00 00 B0 8E 27 02 E4 D1 26 02 E9 9A 06 02 00 00

2B 01 FF FF 2D 00 00 00 9C 8F 27 02 E4 D1 26 02 E9 9A 06 02 00 00

2B 01 FF FF 2D 00 00 00 88 90 27 02 E4 D1 26 02 E9 9A 06 02 00 00

2B 01 FF FF 2D 00 00 00 74 91 27 02 E4 D1 26 02 E9 9A 06 02 00 00

2B 01 FF FF 2D 00 00 00 60 92 27 02 E4 D1 26 02 E9 9A 06 02 00 00

I need to investigate it further, but this makes me think that Pokemon migrated on DeSmuME having wrong Trash Bytes is just a rumor.
I will have to do a lot more tests to confirm it. The build I used btw. is 0.9.11 x64. Slot 2 Emulation is working pretty stable and I can highly recommend it.
If you want you can migrate some Pokemon on Emulator and send them to me, so that I can check them.

Edit:
@Sabresite gave me a clue why the RAM Offset might differ. Shuffled Memory. Dynamic Allocation. I completely forgot about that. So I think to solve the the problem of before we will have to determine the Offset range.

Considering the Emulation is fine my idea of having an Pal Park Editor becomes kinda obsolete, but that's a good thing.
Even if the Emulation works well I still think that this is very important to know for legality analysis, so I will continue this research.
This should be also an interesting Topic for Event Collectors, because this affects migrated Event Pokemon from Gen 3 as well.

Edit2:
I updated the 3rd Gen Research save files. 
 

Edited September 17, 2017 by ajxpk

[ZZAZZ]

Here's some extra files I had started to transfer from Emerald and LeafGreen to several of the Europe region DS games.

US Emerald to Europe.rar

US LeafGreen to HGSS Europe.rar

I started on the new gen 3 saves importing them into the Europe DS games, but it will take me a few days to get them all into Pt and D+P.

[fabio00]

I just discovered this today.

I have used Desmume to Migrate some Pokemon a few years ago and I must be honest, I don´t really understand much of the things that you say here XD.

But my question is: Does Desmume do the migration like on real hardware? Thanks in advance

[Guest]

Thanks a lot again @ZZAZZ. I really appreciate your help so much. At the moment I am very busy with other projects so I wasn't able to migrate many Pokemon.
I will check your files very soon.

@fabio00 Yes. It looks like pal parked Pokemon on DeSmuME are totally fine. 
The thing is people keep saying that it gives wrong Trash Bytes, but what they mean with "wrong" has never been exactly explained.
My assumption is that they might have used the legality checker of the old days. But the thing is this research was never completed...
Especially when it comes to Platinum, HeartGold&SoulSilver there is still a lot of work to do.

So far I haven't been able to reproduce whatever these false Trash Bytes might be. 
It would help to migrate more Pokemon on a real DS/DS Lite with real cartridges.
I also hope that @HaxAras will pal park some Pokemon on the Japanese Pt and HGSS versions.
I already did that on Emulator just recently but I would like to compare and double check. 

From now on you can also contribute files migrated on Emulator but it would be good if you note wether it was migrated on Emulator...
Real Hardware with real cartridges or flash carts.

As I stated earlier I have some issues with mapping the Trash Bytes because the DS Games using shuffled memory for cheat code protection.
That's why some of these values appear to be random. What I have to do is, I will have to note the minimum and maximum possibilities of these numbers.
This will take a while and I need to find an efficient way to do it. At the moment I stuck because of lack of ideas... so this research might take a while.
If I would do it now, I would have to check the Memory right in the middle of the migration process... on an Emulator that's no problem to do...
But I would prefer if I can find a way to force the allocation into my favor, maybe with save states or something at least.

Thanks to ZZAZZ things are speeding up quite a bit though. The more files we can get the better it is.

Edited September 17, 2017 by ajxpk

[HaxAras]

I'm in the middle of getting my DS repaired. The L + R buttons don't work. 

Spoiler

The DS I've been using to migrate barely works and it makes migrating Pokemon even more terrible than it already is. It's got mushy buttons, a broken hinge, broken R button and missing L and the top screen doesn't work. So I decided to work on my Colosseum project instead. I was also tired of putting that project off. 

Spoiler

 

[Guest]

Btw. nothing too important but I just want to say there's no difference between English Versions and Revisions. There's no difference wether an English Pokemon Game was released in America or Europe, English Diamond for example is always English Diamond... and there's also no difference wether it's for example Pokemon Diamond Revision 5 or Rev 13. I just want to say this so that you know that we don't have to do more than necessary.

Edited September 17, 2017 by ajxpk

[fabio00]

1 hour ago, ajxpk said:

From now on you can also contribute files migrated on Emulator but it would be good if you note wether it was migrated on Emulator...
Real Hardware with real cartridges or flash carts.

I already have some projects of my own at the time and I don´t have a DS/DS Lite

But when I manage to get some time i´ll try to help 

[Sabresite]

To clarify, I made a mistake about the shuffled memory. Gen 4/5 use dynamic allocation which means it is variable. Depending on circumstance, the pointers would be slightly different. We are thinking by as much as 0x400.

[Guest]

Of course I knew that from our conversation and I even saw it with my own eyes. My mistake. I don't know why I wrote shuffled memory, this kinda stuck in my head.  

Edit: Fixed.

Edited September 17, 2017 by ajxpk

[Brittany]

@ajxpk

Not sure if it helps, but I know Pokecheck was one of the legality checkers that flagged this. While I personally haven't transferred through the emulator, I know my friend use to do it in Gen 5. The invalid trash bytes were something that showed up on Pokecheck. I don't think she saved any of the invalid checks though, since she would just clone the pokemon and send them up in bulk until she wouldn't get the invalid trash bytes flag. When inquiring with her earlier about it, she said it didn't happen all the time. Just stuff that occasionally occurred. 

I'm still worried about it though, which is why I'm considering getting something that can rip and restore .savs on the cartridges if there was something wrong with them.

I may be able to contribute to the Japanese games. I have a (very bad fake...don't buy from ebay without boxes, kids) JP Fire Red. The battery doesn't hold a save for more than a few minutes (or that's how it was when I did it back when JP Diamond *originally* got released 11 years ago). I might have some of the things I originally shifted sitting around somewhere. Just not sure if it's still on Gen 4 or if I moved it to Gen 5 before I started backing up my DS games. I'll poke around my saves later. 

[Guest]

DS Emulators couldn't link back in the days. So it's highly unlikely that Gen 5 migration/trades were possible. From what I know DeSmuME could do something in an earlier version, at least I have read about it, but the feature was shortly removed after. melonDS which is still a very young Emulator (just 1 year old) is the first DS Emulator that makes an serious attempt to emulate the WiFi functionalities and If the development goes well it will even be able to link with an real Nintendo DS. 

Thanks! Any help is appreciated. 

Edited September 18, 2017 by ajxpk

[Brittany]

Alright, spoke to her to clarify what she did.

She went: DeSmuME on her Soul Silver to shift from the Gen 3 (Emerald in particular) rom -> Move the SS emulator save using NDS Adapater+ to her cartridge Soul Silver -> Gen 5. She said the invalid trash bytes appeared in Gen 4, so she'd check it via pokecheck. If fine, send forward again. If not, do another batch until there were no invalid trash bytes. Based on old chat logs, it was in July '12. I don't think she kept the .pkm that got flagged, as it was so long ago.

I'll need to do that 24 wait cool down to send things to JP Diamond. I never completed JP SS so I can't assist with that. Still not sure if I'll be able to shift with how finicky the bootleg FR is, but I could open the cartridge up and see if maybe something is loose if it doesn't work.

 

Here's some random shifts I saw when digging through my own saves though.
Veral, Squir, and Char were from my fake fire red that I shifted back in the day to JP Diamond. They're in Gen  5 only it seems.
Tifa was from Sapphire. Regly was from Sapphire. Both of the were shifted to Eng Pearl, IIRC. (Since I wasn't risking my favorite when the fake FR couldn't hold a save long)

Jule was from my hack containment game in Gen 3. No idea if she got caught up in the cross fire of me screwing around.

For things in Gen 4:
The houndour and houndoom were probably from the GTS, but I had them sitting on my JP Diamond. Lionne is just random pokemon from my Emerald. Must've used it as an HM slave

003 - Ｖｅｒａｌ - EA86AF65D844.pk5

006 - ＣＨＡＲ - 7C3E6BC422EE.pk5

009 - Ｓｑｕｉｒ - 866BD7E1D764.pk5

228 - HOUNDOUR - 5E38A738497F.pk4

229 - ヘルガー - 0C251EE072DD.pk4

257 - Tifa - 779E7492CF13.pk5

260 - Jule - 0D5AAAC2DC1E.pk5

264 - LINOONE - 25B67A2F0959.pk4

282 - Regly - 3542A18466F7.pk5

Edited September 18, 2017 by Britty

[ZZAZZ]

While waiting for the next batch of Pokemon from the European DS games, (and reading I probably won't be needing to do the US ones) I decided to throw together this strange test batch just for curiosity's sake. I started a fresh Pokemon red save on my actual cart, and caught a pidgey. Then I resumed the flashed twitch plays Pokemon crystal save on my real crystal cart, caught a hoothoot, and traded the pidgey over to join it. Then I dumped the save to my gb everdrive with a mega memory card, and used pk2pk to hack convert them to gen 3. Using Pkhex I fiddled around with data and cloned each 5 times to get several variants. Then I saved them into the gen 3 research save, and flashed it to my sapphire rev 0 cart, and started transferring them into my real copy of diamond. Then for the heck of it I transferred the rest into soulsilver to save time. 

I don't know if these will yield any interesting data about hacked gen 1+2 mon being transferred through Pal Park, or anything different with the trash bytes, but I figured it'd be a fun experiment to try while waiting to transfer the rest of the blank named bulbasaurs. 

Sapphire-DiamondUS.rar

Sapphire-SoulSilverUS.rar

[Guest]

I don't even need to check them, I think it's no secret but I'm not really a fan of fake conversions like Gen 2 - Gen 3. Because the results of this fanmade idea are comparable with bad hacks. There is nothing special about it and nothing interesting IMO and the reason for this project is to define what's legal so we could say what we do is the complete opposite. The DS migration itself doesn't even know and doesn't even care if the migrated Pokémon is a hack... as long as it's not a bad egg... Even in terms of Trash Bytes it's also not really interesting because without checking I already know what happens. String gets copied until the Terminator. Because that's what the DS Game does...

The Trash Bytes are actually just stored memory from the process  during the migration. Imagine it like this... the DS Game looks for the Nickname String and checks where the Terminator is. Then it copies the String from the beginning to the Terminator.

The Nickname String then gets converted from Gen 3 to Gen 4 format and stored on top of other data. And then later the Game copies the complete String of 22 Bytes, which of course are too many Bytes in the case of Pokémon with shorter Names...

Keep in mind that I haven't disassembled it. So my explanation might differ in some points to what actually happens. This is just based on what I know from similar Trash Bytes this.

Anyway, thanks to you two. Like I said, the more migrated Pokémon we gather the better it is for the Research. Also to see if there are some still unknown inconsistencies. But so far it looks fine...

Edited September 18, 2017 by ajxpk

[JHorbach]

Good Job ajxpk, do you know how PKHex handle the Pal Park and Trash Bytes when you drag and drop a Pokemon from Gen 3 to 4?

[Sabresite]

2 hours ago, ajxpk said:

I don't even need to check them, I think it's no secret but I'm not really a fan of fake conversions like Gen 2 - Gen 3. Because the results of this fanmade idea are comparable with bad hacks. There is nothing special about it and nothing interesting IMO and the reason for this project is to define what's legal so we could say what we do is the complete opposite. The DS migration itself doesn't even know and doesn't even care if the migrated Pokémon is a hack... as long as it's not a bad egg... Even in terms of Trash Bytes it's also not really interesting because without checking I already know what happens. String gets copied until the Terminator. Because that's what the DS Game does...

The Trash Bytes are actually just stored memory from the process  during the migration. Imagine it like this... the DS Game looks for the Nickname String and checks where the Terminator is. Then it copies the String from the beginning to the Terminator.

The Nickname String then gets converted from Gen 3 to Gen 4 format and stored on top of other data. And then later the Game copies the complete String of 22 Bytes, which of course are too many Bytes in the case of Pokémon with shorter Names...

Keep in mind that I haven't disassembled it. So my explanation might differ in some points to what actually happens. This is just based on what I know from similar Trash Bytes this.

Anyway, thanks to you two. Like I said, the more migrated Pokémon we gather the better it is for the Research. Also to see if there are some still unknown inconsistencies. But so far it looks fine...

Very similar to what you said.  IIRC, the 22byte array is recycled memory.  It gets filled to the terminator, then the array is copied whole to the pokemon structure.

[ZZAZZ]

Okay, it's good to know hacked ones don't seem to matter to the transfer, so I don't have to do crazy conversions like that again. Here's some random pokemon pulled from a vba emerald save, copied over to my sapphire cart to soulsilver. Figured I'd send them through my DS for the heck of it.

 

084 - lrprfyfqy - 84C7D5A91EEE.pk4

384 ★ - Rayquaza - F772CD0AC119.pk4

261 - Fang - 2752E3CDD62C.pk4

150 - Mewtwo1N - AE9DB5052D23.pk4

117 - EXXQXCCA - 52CCE3B3D034.pk4

025 - ZAPPY - A45666F8B8B5.pk4

 

EDIT: Here's all of the Pal Park Research bulbasaurs transferred between every combo of US GBA and DS Europe carts on my real DS.

GBA US to Europe DS .rar

 

Edited September 20, 2017 by ZZAZZ
Added bulbasaurs

[HaxAras]

On 9/18/2017 at 5:48 PM, ajxpk said:

I don't even need to check them, I think it's no secret but I'm not really a fan of fake conversions like Gen 2 - Gen 3. Because the results of this fanmade idea are comparable with bad hacks. There is nothing special about it and nothing interesting IMO and the reason for this project is to define what's legal so we could say what we do is the complete opposite. The DS migration itself doesn't even know and doesn't even care if the migrated Pokémon is a hack... as long as it's not a bad egg... Even in terms of Trash Bytes it's also not really interesting because without checking I already know what happens. String gets copied until the Terminator. Because that's what the DS Game does...

I FINALLY got my DS 'Pavement Orca' model back. So I can use the GBA backup tool and start migrating more Pokemon for you. I'd be happy to beat Platinum JP. I just need a Pokemon good enough to beat the Elite 4 and remainder of the game. I didn't get to the point it's at so I don't know what's left to do. (Not as familiar with gen 4 as I am with gen 3. 

I can do more migrations to the English and Japanese Diamond/Pearl and English Platinum and HG. I still haven't messed with my JP SS save. Do you have a complete file I can use? I just need to make sure I backup my current save with my Pokewalker mons on it. I just recall reading something about you needing more specimens of Pokemon from the games you already have. (Like multiple Pokemon from English Diamond, etc.)

I kinda want to see what happens if I use a bootleg gen 3 cart to migrate from. If it will work and what it would do with the trash bytes.

[Guest]

Use the files I uploaded in the first Post. You can use them and mess with these files as much as you want. Pal Park is unlocked and everything. 
At the moment I'm very busy and can't check much, but I will do as soon as I have more time. Gotta finish some other projects for now. 

[HaxAras]

3 minutes ago, ajxpk said:

Use the files I uploaded in the first Post. You can use them and mess with these files as much as you want. Pal Park is unlocked and everything. 
At the moment I'm very busy and can't check much, but I will do as soon as I have more time. Gotta finish some other projects for now. 

Alright, thanks. I couldn't remember which files were in the first post. 

I was running out of money in my Emerald cart. And running out of Pokeballs. Now that I can restore saves, I don't have to shuffle between 15 Japanese carts to figure out which ones have Pokeballs either. 

[HaxAras]

I've been putting this off for longer than intended. Sorry 'bout that. 

You said you wanted more examples of Pokemon migrated to the same games and I've forgotten what I've already done. I was going to do Diamond instead of HG, but one was already on my desk and the other wasn't. I can also just do Diamond later and hopefully the JP Soul Silver (finally)

Emerald to Platinum. 

261 - A - 9F7BC8BF0AE0.pk4

261 - A - 176AB5ACB7D8.pk4

261 - A - EA42A04AF237.pk4

265 - A - 967ABD4E6FD2.pk4

265 - A - A91692D6C2B8.pk4

309 - A - F401D757A8BC.pk4

Emerald to Heart Gold. 

265 - A - 93A406B366CD.pk4

261 - A - DE4F5597136F.pk4

261 - A - AD227E6704A1.pk4

261 - A - 19E688FEA46B.pk4

261 - A - 4BB057324BA1.pk4

043 - A - CDD553A68315.pk4

Edited October 7, 2017 by HaxAras

[Brittany]

Opps, forgot about this.

First part is... bad news. The badly fake JP FR I have does not read at all anymore, so I definitely cannot do tests from JP Diamond on retail.

But, for some... mildly interesting news, I found a backup of my emerald on my computer. Because of this, I was able to test a batch of the same pokemon being sent up on US Pearl retail vs being sent up on the emulator. Unfortunately, I don't know if I brought my gameshark (or whatever) with me when I moved since I could only make my ancient laptop run the program, so once I run out of pokemon that originated on that cart. That'll be it. However, I can test a variety of things that are still on the cartridge if requested.  (Like specific nicknames or whatever)

This batch is all not-nicknamed, but I can send up a nicknamed batch tomorrow. 

One thing of note though: The Pineco and one of the miltank were holding a dive ball when I sent them up on retail. I dunno if this affects the trash bytes or not, but I didn't have them hold it on the emu transfer.

 

Aipom - Emu.pk4

Aipom - Retail.pk4

Docile Miltank - Emu.pk4

Docile Miltank - Retail.pk4

Mareep - Emu.pk4

Mareep - Retail.pk4

Pineco - Emu.pk4

Pineco- Retail.pk4

Quiet Miltank - Emu.pk4

Quiet Miltank - Retail.pk4

Spinark - Emu.pk4

Spinark - Retail.pk4

Edited October 15, 2017 by Britty

[Guest]

I haven’t worked on it for a while. But just in case someone would like to support this project, I want to give information about what has to be done next...

We need migrated Pokémon on all different Game and language versions which means DP/Pl/HGSS in all different languages J/E/F/G/I/S/K. Can be done on Emulator...

And most importantly we need to determine and document the lowest possible Offset/Base Offsets of the pointer inside the Trash Bytes of migrate Pokémon #2. For this we have to find a way to stop the Memory Allocation. Maybe there are cheat codes for that?

Edited October 31, 2018 by ajxpk