How Pokémon save data is stored, exactly? And transfers

[N1zle]

Hey everyone! Hope you are all doing fine. I'm a new member to the forum but not first time PKHeX user, so excuse me if this isn't the ideal post for the forum.

This is going to be a long, technical questions thread, that I would like to get cleared, and think this would serve as a great reference for future genners (and me after I forget everything I ressearched for the third time). Hopefully, you can help me understand the intricacies of the Pokemon franchise's data structure, how they save pokemons, etc.

Here's my goal: I want to transfer gen 1, 2, 3 and 4 (let's stop here for now lol), all the way to my switch. I have messed around with genning mons on gen1 before, but the save files/pk files are so small I mapped their hex values and edited what I wanted with HxD. So, here's my first question:
    1. Regarding the games' structure and the .pk1(/2/or other gens' equivalent). If a pokemon is, let's say, a 20 byte long data string, are those 20 bytes the same, whether in the .sav or the .pk1 of that specific mon? Or is there a "preferred" way to read, extract and save that pokemon for transfer? And is this the same for all generations?

I have cartridge dumps/save files from emulators/save files from VC/pk1 mons generated directly from PKHeX. So here goes the next question:
    2. Let's say I have an event pokemon on Pokemon Blue on a physical cartridge that I can dump, another one that I "generated" in game on an emulator and on VC, and another one generated directly from PKHeX. If I manage to input exactly the same trainer ID, level/moves/IVs (basically everything that's visible/checkable), is there a way to differentiate the one from the physical quando cartridge from the virtual/generated ones? I'm basically asking, if it's possible to generate a perfect mon on PKHeX, what other things do I need to care for to make sure it's a 100% perfect match? Same as Q1, does this hold true for all generations?

Regarding specifically gens 1 and 2 now, and their VC counterpart (since it's the only way to legitimately transfer mons from gens 1/2 to Home/Switch). I know that transferring mons from one game to another can add/edit some data, because the newer games usually have a different data structure.
    3.1. What gets edited if I transfer from a Blue to Gold cartridge?
    3.2. What gets edited if I transfer from Blue VC to Home?
    3.3. What happens if, instead, I inject that pokemon directly into a newer gen game (7/8), that can then transfer to Home?
    3.4. If I extract the mons out of Gold from case 3.1 and Blue from 3.2 and inject them into Sun (3.3), will they have the same data attached? Or is something going on behind the scenes that can make their hashes differ?

Questions 3 came to my mind specifically when talking about gens 3/4, but I wanted to ask about gens 1/2 as well since they have the "physical cart/digital VC" debate. But now about gen 4.
    4. If I extract a mon directly from Emerald, or transfer from Emerald to Platinum and then extract the PKHeX file for that mon, or go the full route (Plat > Black > Bank > Home), will I have 3 different mons in the end? Or do they match perfectly?

I came to this quest after stumbling online on different Event Mon collections, and after checking, even the ones with fixed trainer id/OT/iv/moveset/name/region, they still didn't match hashes perfectly. Then my perfectionist insticts surfaced and I realized I can't have them like that, and starting genning my own event mons, based on all info I could find online about them. I learned that if I name myself the same as an OT for event pokemon, for example, it doesn't match (I think there are trash bits representing the unused characters, that don't exist on the actual event pokemon), and other nitpicks like that that I would like cleared.

I'm deeply grateful for anyone that can chime in any information regarding this topic, and hopefully we can compact all this data into a single thread for future reference.

Excuse any mistakes as English is not my first language, and feel free to correct me, or suggest more questions I can add to this post. This is not meant to be a one and done question, as I will need to come back to it in the future due to my failing memory.

[Kaphotics]

The program is open source and you can test everything you want with the program and referencing a hex editor and the source code. Nobody is going to write you an essay describing how everything works in excruciating detail; just refer to the program's source code and various threads on the internet.

PKHeX edits the save file and models its transfer mechanisms to the best of the community's understanding of how the official transfer mechanics work.

[N1zle]

10 hours ago, Kaphotics said:

The program is open source and you can test everything you want with the program and referencing a hex editor and the source code. Nobody is going to write you an essay describing how everything works in excruciating detail; just refer to the program's source code and various threads on the internet.

PKHeX edits the save file and models its transfer mechanisms to the best of the community's understanding of how the official transfer mechanics work.

I understand nobody is here to babysit me or anyone else on how the program works. Can you at least link me to sources to those questions or am I going to have to go scraping every page of the wiki? I thought this was a help forum, not one to be told "go look for it somewhere else".

[theSLAYER]

If you want to perform transfers without actual hardware, PKHeX can transfer up stuff for you. Open 2 saves on 2 PKHeX windows, and drag the mons across.

 

If you want to know what gets changed, well, ask the one question (which the scope isn’t at all big) that is most important to you (relating to this). Perhaps we can move forward, bit by bit. As mentioned above, an essay probably shouldn’t be expected. Yes, this is a help forum, and one may feel entitled to get help, but you gotta remember that no one here is paid nor obliged to help. If your question appears too big/too demanding, it may in fact shun away those that otherwise would have been willing to help. Please understand.

[N1zle]

3 hours ago, theSLAYER said:

If you want to perform transfers without actual hardware, PKHeX can transfer up stuff for you. Open 2 saves on 2 PKHeX windows, and drag the mons across.

 

If you want to know what gets changed, well, ask the one question (which the scope isn’t at all big) that is most important to you (relating to this). Perhaps we can move forward, bit by bit. As mentioned above, an essay probably shouldn’t be expected. Yes, this is a help forum, and one may feel entitled to get help, but you gotta remember that no one here is paid nor obliged to help. If your question appears too big/too demanding, it may in fact shun away those that otherwise would have been willing to help. Please understand.

Understood, and I do comprehend I'm not entitled any help. Just wanted to get a discussion started on those topics, I didn't quite grasp the size of the community at first, and now I realize it's a lot more focused than people "broadly creating pokemon out of thin air".

As for my main question, I guess the others could be solved by trial and error, but I still have not found confirmation on my first one.

"1. Regarding the games' structure and the .pk1(/2/or other gens' equivalent). If a pokemon is, let's say, a 20 byte long data string, are those 20 bytes the same, whether in the .sav or the .pk1 of that specific mon? Or is there a "preferred" way to read, extract and save that pokemon for transfer? And is this the same for all generations?"

I got mGBA with the memory viewer open, HxD on the side with the .sav file, and 3 wiki pages telling me "where" the Pokemon's data is saved, and still couldn't find my Bulbasaur. Then, my new, single question:
On generation 1 games, what is the best way to locate Pokemons in my party on the .sav file? All I want to do it cross-check it with mons generated in PKHeX (since that one is easy to read on HxD).

Sorry if I seemed hasty/jumpy on my first reply. If answers are too much, I'd happily take links to where I could gain more knowledge on the matter, as to come back and reply with answers to all those questions eventually.

[Kaphotics]

Dump your pk1 data and search the save file for the same sequence of bytes.

https://bulbapedia.bulbagarden.net/wiki/Save_data_structure_(Generation_I)

Structure is not entirely sequential due to how they store strings separately from data.