research Gen 3 Event Generation Algorithm Research (10ANNIV, etc)

[Guest]

@Invader TAK It has been said and already stated before but maybe I should say it more clear.
A hacked and customized 10th Anniversary Rom would NOT be equal to a Official Distribution Rom of any respective Event you want it to be.
Besides this is not something anyone could do, it needs someone who has some decent IT knowledge and experience.
I personally don't know anyone here who could do it except the likes of @Bond697, @SciresM and @Kaphotics.
It also must be said that a generated Pokemon from a hacked Rom still wouldn't be legit and at the same time wouldn't be more legal than any other perfectly hacked Event Pokemon. Therefore it is questionable how it would be beneficial considering the time and effort that is needed to make it even possible, which is why I personally don't go deeper into this Rom other than for needed Research purposes.
I'm not saying that it wouldn't be interesting to see this from a hacking standpoint, I just don't see it happening.

[Invader TAK]

40 minutes ago, ajxpk said:

@Invader TAK It has been said and already stated before but maybe I should say it more clear.
A hacked and customized 10th Anniversary Rom would NOT be equal to a Official Distribution Rom of any respective Event you want it to be.
Besides this is not something anyone could do, it needs someone who has some decent IT knowledge and experience.
I personally don't know anyone here who could do it except the likes of @Bond697, @SciresM and @Kaphotics.
It also must be said that a generated Pokemon from a hacked Rom still wouldn't be legit and at the same time wouldn't be more legal than any other perfectly hacked Event Pokemon. Therefore it is questionable how it would be beneficial considering the time and effort that is needed to make it even possible, which is why I personally don't go deeper into this Rom other than for needed Research purposes.
I'm not saying that it wouldn't be interesting to see this from a hacking standpoint, I just don't see it happening.

I'm not saying hack the 10ANNIV ROM, I'm talking about extracting the generation algorithm from the 10ANNIV ROM and using it to make an entirely new ROM. But considering no one's bothered at this point, I'll try to make this is the last time I bring it up. Only reason I did was because a custom ROM that uses the actual event generation method could be an "as legit as possible" replacement for the distros that are lost forever.

Edited March 19, 2017 by Invader TAK

[Guest]

Yeah, I agree with this. Making a Rom from the scratch IMO would also be more efficient. Because you would have a prototype and can change its algorithm so that it covers other algorithms as well. But it's just as complicated to do and the problem basically remains the same. I think if there is someone who wants to do it, it will happen. The thing is just, it's not happening right now. Maybe @suloku is interested in it, but he's very busy at the moment. 

[Invader TAK]

17 hours ago, ajxpk said:

Yeah, I agree with this. Making a Rom from the scratch IMO would also be more efficient. Because you would have a prototype and can change its algorithm so that it covers other algorithms as well. But it's just as complicated to do and the problem basically remains the same. I think if there is someone who wants to do it, it will happen. The thing is just, it's not happening right now. Maybe @suloku is interested in it, but he's very busy at the moment. 

That's a good thought on being to cover other generation algorithms with a custom ROM. It'd make distributing the likes of Ageto Celebi And WISHMAKR/CHANNEL Jirachi a lot easier (and portable). Plus you could remove the "one Pokemon per file" limit if you wanted.

[suloku]

On 19/3/2017 at 6:15 PM, ajxpk said:

Maybe @suloku is interested in it, but he's very busy at the moment. 

I'd change that to "I don't have the skills", but it seems someone has already covered the difficult part!: http://forums.glitchcity.info/index.php?topic=7861.0

Basically, it's a custom rom that gets sent over to GBA and does stuff to the savefile. The example changes the player name's first letter to "z", but adding a pokémon to the party/first slot would be easy. The only thing really missing would be pokemon generation algorythm, the downside is that this uses gamecube/wii connection, but maybe doing it over two GBA wouldn't be that difficult for someone familiar with GBA development.
Edit: I think it does alter the savefile in RAM, so changes only apply if the player runs the game and saves afterwards.

[jojo12100]

2 hours ago, suloku said:

I'd change that to "I don't have the skills", but it seems someone has already covered the difficult part!: http://forums.glitchcity.info/index.php?topic=7861.0

Basically, it's a custom rom that gets sent over to GBA and does stuff to the savefile. The example changes the player name's first letter to "z", but adding a pokémon to the party/first slot would be easy. The only thing really missing would be pokemon generation algorythm, the downside is that this uses gamecube/wii connection, but maybe doing it over two GBA wouldn't be that difficult for someone familiar with GBA development.
Edit: I think it does alter the savefile in RAM, so changes only apply if the player runs the game and saves afterwards.

Really interesting, I think we have first to try just to add a pokemon to the party before trying to emulate the pokemon generation algorythm.

In fact what interest me the most is how they bypass the language/version/region check.

[Guest]

It's very easy if you have access to the GBA SDK. It comes with a C sample called "multi_boot":

Looks familiar, doesn't it?
The sample is complete with server and client application.

For the most basic distribution, just rewrite the client to do the following:
1. Check if the correct GBA cartrdige is inserted by looking at the header ("POKEMON EMERBPEE" etc.), show an error if not
2. Read the savegame of the inserted GBA cartridge
3. Check if the party has a free slot, show an error if not
4. Generate your PK3 data, write it to the party (or anything else you want, such as wondercards, Old Sea Map etc., anything is possible)
5. Fix checksums
6. Write the savegame back to the inserted GBA cartridge
7. Show success message

[jojo12100]

52 minutes ago, Purin said:

It's very easy if you have access to the GBA SDK. It comes with a C sample called "multi_boot":

Looks familiar, doesn't it?
The sample is complete with server and client application.

For the most basic distribution, just rewrite the client to do the following:
1. Check if the correct GBA cartrdige is inserted by looking at the header ("POKEMON EMERBPEE" etc.), show an error if not
2. Read the savegame of the inserted GBA cartridge
3. Check if the party has a free slot, show an error if not
4. Generate your PK3 data, write it to the party (or anything else you want, such as wondercards, Old Sea Map etc., anything is possible)
5. Fix checksums
6. Write the savegame back to the inserted GBA cartridge
7. Show success message

 

Great, I'll try to watch if I can do something, hope that other people will try too.

Theorically it's not difficult.

Maybe we could do our 10ANNIV universal distrib rom like that or better the Bonus Discs (Celebi, Jirachi ...) distrib?

[Guest]

On 3/22/2017 at 8:24 AM, jojo12100 said:

Great, I'll try to watch if I can do something, hope that other people will try too.

Theorically it's not difficult.

Maybe we could do our 10ANNIV universal distrib rom like that or better the Bonus Discs (Celebi, Jirachi ...) distrib?

10ANNIV actually uses 10 different client applications, one for each Pokémon. You can either do it like this and compile one client for each Pokémon, or you could do it a bit more sophisticated by sending a few parameters from the server to the client after the client application has successfully booted. That way only one client is needed for everything, while still being able to choose what to distribute from the server.

[jojo12100]

1 minute ago, Purin said:

10ANNIV actually uses 10 different client applications, one for each Pokémon. You can either do it like this and compile one client for each Pokémon, or you could do it a bit more sophisticated by sending a few parameters from the server to the client after the client application has successfully booted. That way only one client is needed for everything, while still being able to choose what to distribute from the server.

I don't have the skills to do more sophisticated. But I'm pretty sure amazing members will have.

[Guest]

Does anyone have 1) the PK3 generation algorithm and 2) code for fixing checksums of all Pokémon GBA saves, in C language?

After that we only need to figure out how to dump saves into RAM, inject the distribution and put it back to the cartrdige.

[suloku]

13 hours ago, Purin said:

1) the PK3 generation algorithm

That would be the most troublesome to get, not sure how the RE efforts on that have gone.

13 hours ago, Purin said:

2) code for fixing checksums of all Pokémon GBA saves, in C language

BlackShark's Mirage Island appearance program was what I used as basis, which I think in turn is based on kaphotics checksum verification tool. Thinking back, if it weren't for his program I probably wouldn't have dared to play with gen 3 savegame editing.

Also, I think the client from glitchcity I posted previously has been updated to handle checksums and even party/pc pokémon with simple functions.

I have another alternative, for now we have:
1) A client for each distro
2) Client-host communication

As an (easiest?) alternative, I suggest that the final client binary has a dummy pk3 file embeded. The host generates the pk3 once selected using the needed algorythm, then it changes the dummy pk3 file to the actual pk3 on the client and sends it. The client would only need to be a really simple app to check if there's party/pc space and add the pk3.

[Sabresite]

17 hours ago, Purin said:

Does anyone have 1) the PK3 generation algorithm and 2) code for fixing checksums of all Pokémon GBA saves, in C language?

If someone provides this, I will see what I can do.

The Gen 3 generation algorithms are well known.  Past RNG Research thread on Smogon details just about every generation method known.
I'll give a quick overview of how it works.  You will have to do the research to fill in the gaps.  Also this may not be 100% accurate.

1) battle should happen?
2) which encounter slot? (Determines species & level based on location)
3) Choose nature or Sync is determined
3b) Loop through creating PIDs until you find one that matches above nature.
4) PIDL
5) PIDH
6) IV1
7) IV2

Here is the shit part.  There are usually between 1 and 2 vBlanks.  Where the vBlanks occur depends on the situation. For example, fishing, caves, grass, etc.
Lastly, static encounters, gifts do not have #1-3, and this doesn't apply to breeding.

[Guest]

So I was working on this for the past 15 hours and it was coming along quite well with custom font code and all working, but then Notepad++ crashed and left me with a 0 byte file. I'm so angry right now... I'm not going to try again, but I hope someone else will.

All the relevant code for reading/writing a savegame on the GBA is here. It also works fine from a multiboot client as long as you keep memory usage down. Good luck.

[Invader TAK]

Thinking about it, I'd be fine with a GC/Wii homebrew program that can handle every Gen 3 event we have the algorithms for (or can figure out). I'd prefer a GBA or DS one for the portability, but I'd be fine with anything. Like I said, I'm only suggesting this so we can replace distros that are lost forever (like the US 10 ANNIV). I personally feel as long as the event Pokemon is generated the EXACT same way it was originally, it'd still be legit even if it was from a homebrew distro.

[Guest]

The exact same way would not be possible, because no one would ever be able to recreate the whole generation process in a 100% accurate way.
Legal is as far as you would be able to get, no matter which way you choose to go.

[suloku]

24 minutes ago, ajxpk said:

no one would ever be able to recreate the whole generation process in a 100% accurate way.

Are we talking about trash bytes here? Just mentioning due to colosseum/xd getting rid of those, thus making them virtually the same.

A generator for all the events for me would just be something nice and insteresting to have both for users and documentation purposes on the algorythms, even though I'd rather have compilations of the real deal (which has been in the works for a while) or distro rom dumps.

The only real use would be for those pokémon that came with exclusive moves on the events, as you can get anything else trough legit means and RNG manipulation (probably?).

[Guest]

I think what would be good to have is an open source distro rom which does the same as 10ANNIV etc. I was trying, but then that accident happened.

[Guest]

@suloku Yes, not just the trash bytes itself so much actually but the fact that they're originating from thumb code, which is something that is impossible to reproduce the same way it originally was. And as a researcher all I can do is documenting the trash bytes that are there in the pk3 files I get, but nothing more than that. 

Speaking of Trash Bytes... I'm not so much a fan of saying "We can just wipe the trash, which is achievable in a legal way and then it's fine.".
I mean you're correct when you say a 3rd gen pokemon with 0 trash bytes is still legal because it's achievable with the use of COL/XD. It is absolutely. But at the same time as it is legal this will also mean that it's touched. And from my own experience of collecting 3rd Gen Events this is also extremely rare, I mean so far I have only witnessed one Event Pokemon with 0 trash, which I'm not even sure about if it's legit...
You can imagine if you have a collection of hundreds of Event Pokemon with 0 trash that would be highly suspicious.
That's why I do think that they play an important role in legality analysis at least when it comes to Gen 3.
And in a legality checker one should be at least informed about the absence of Trash Bytes.
(It doesn't necessarily has to affect the legality, but is still an indicator.)

And don't worry trash isn't really the problem when we talk about the results, except for the events which trash bytes hasn't been documented so far.
What I was talking about is the generation process as a whole and there will never be a 100% legit recreation of it.
I mean if we take the word "legit" seriously we would have to recreate the whole generation process 1:1.
And even this would be already a stretch of the word legit. But let's say if the algorithm would be enough for it being legit.
That's not gonna happen. So we can talk about legal but not legit.



I think I also have to distant myself a little, because some of my collector friends fear that my motive isn't clear.
I personally want to support the legality analysis in the first place. 
So that's on my priority list right now.

But I also have to admit that an open source multiboot rom would be interesting to see indeed.
So I can only wish good luck on that and I'm curious to see the progress.  

[Invader TAK]

Wouldn't the algorithm generate the trash bytes if it's generating via GBA multiboot? Or are the trash bytes a product of the official distro cart/ROM (or disc/ISO in the case of Berry Glitch Fix Zigzagoon, Ageto Celebi, WISHMAKR Jirachi and CHANNEL Jirachi)?

Ignore that, I didn't see the post above this one.

Ugh, these damn trash bytes are starting to make me think an open source multiboot ROM is only worth it for people who don't care about them. And at that point, just gen.

Ok, I apologize for that. I've had time to calm down. I was just frustrated at seeing "yeah, but the trash bytes" for the millionth time. I get why they're important, but they're still frustrating. Makes me glad GameFreak fully embraced Wonder Cards in Gen 4 and have kept them every since. They make life so much easier.

Besides, the people hoarding events just because they're rare frustrate me even more. But that's a rant for another time.

Edited March 25, 2017 by Invader TAK

[suloku]

The trash bytes are unique to the device/rom that generates the pokémon, since they are just what lied in memory when the pokémon was generated (from my point of view I don't see a reason why they didn't clear those bytes, but at the same time, I also don't see a reason why they should have cared).

What I said about Colo/XD removing them is from this point: OK, we know we can't never do 1:1 replication (since that needs the actual distro device, or a copy of it), but if we throw trash aside and just concentrate on what makes the pokémon unique to that event (PID, TID, OT, OTgender, IV, level, met place...etc) and we can replicate that 1:1 that would still be awesome. If you are using a homebrew app to get them I'd personally still prefer single pk3 files from people who actually got the event.

But for personal use? Fun? Gen 4 onwards (since that clears the trash too)? I'd really like to have something like that, albeit I understand the concerns collectors may have, but faking some gen 3 event on gen 4 onwards is really difficult to discern (and many are available anyways).

That's just my personal opinion, actually I think the fact that we can't replicate trash is good, because A) Doesn't matter for gameplay/gen 4 onwards B) It's a good way to quickly know if the pk3 file was just generated C) There's a legit way to get rid of the trash bytes so even generated pokémon can be claimed to be fully legal without having trash bytes.

ps: I think I finally found a way to tell legal from legit appart (I always confuse the correct word), hope I didn't mix them up.
ps2: yes, GF switching to an in-game generation for most events is really nice, gen 1-3 are a nightmare for events....and gen 3 is actually better than those.

[St. GIGA]

Does anybody know how to check trash bytes on Japanese Pokemon Emerald Method 1 Pokemon, and what might affect the trash bytes in both the OT and Nickname? I would like to know how to check my Mew that Suloku RNG'd.

[Guest]

@St. GIGA I have no idea about Ingame generated Pokemon. Not even sure if this was ever fully analyzed?

@suloku Yeah this legal/legit part is always confusing.
Especially what's legitimate sometimes isn't clear and needs to be defined specifically...
For my example I already stretched the definition by saying a perfectly replicated algorithm would be enough.
And I know many people would disagree with that... because strictly viewed, only what comes from Nintendo/GF is legit.
But technically viewed I think it would say it's close to be legit. Especially if it comes from a multiboot rom which is even more close.
So I really think that with a multiboot rom, you would come as close as you can get and that's why it's so interesting, right?

Btw. like I already said the Trash Bytes wouldn't even be the issue if we are just talking about legality.
With the exception of Events with variating/dynamic Trash Bytes...
This includes...

Negaiboshi Jirachi
PCJP03 Egg
Mystry Mew (2nd to 5th Mew of a batch...)

And Nintendo DS Download play Events like:
PokePark Meowth
Hadou Titants

[St. GIGA]

Yeah, I meant ingame events, as those I cannot figure out what changes trash bytes. For instance, the attached copy of my Faraway Island Mew has a static PID of 7942ef72 which is needed to make it Flawless, but it also has a static SID of 31337 and TID of 60510, plus a Static OT gender of Female, the same moves, no nickname, a Lum Berry,  and all that. I would like to know if the trash bytes on the attached Mew from RoC's are valid. Suloku says they are, but I want to double check that. Just to inform those wondering, this Mew is a test Mew to see how faithful I can get it, as Suloku's Save that he hat had caught Mew on for me with the ID in the PKM did not get the Trainer Gender right, so I fixed that and added a Lum Berry ingame after harvesting one. I also wish to inform you that the Mew was caught by suloku on an emulator without the vblank patch, and that only the OT gender was edited on THIS particular 3gpkm to make its OT Female to match the Real Pokepark Mew event. In game, I enabled the clock and obtained one lum berry for Mew. I just need to know if the one OT gender bit change affects any trash bytes. This is only for research, but I do not know if I accidentally submitted this one to RoC. If I did, and trash bytes check out, consider this a happy accident turned bugfix from suloku's original Mew. It is possible that glitzer or pomeg bugs could do this without A-Save (which I used on a duplicated copy of suloku's mew Save file that he gave me. He also said when he released it that he would provide offsets for editing the trainer info so anyone could customize the mew without re-waiting 34.25 days)

151 - 60510 - Mew - (Timid) - {S} {F} {St. GIGA} {Suloku}.3gpkm

[St. GIGA]

Can someone answer

4 hours ago, St. GIGA said:

Yeah, I meant ingame events, as those I cannot figure out what changes trash bytes. For instance, the attached copy of my Faraway Island Mew has a static PID of 7942ef72 which is needed to make it Flawless, but it also has a static SID of 31337 and TID of 60510, plus a Static OT gender of Female, the same moves, no nickname, a Lum Berry,  and all that. I would like to know if the trash bytes on the attached Mew from RoC's are valid. Suloku says they are, but I want to double check that. Just to inform those wondering, this Mew is a test Mew to see how faithful I can get it, as Suloku's Save that he hat had caught Mew on for me with the ID in the PKM did not get the Trainer Gender right, so I fixed that and added a Lum Berry ingame after harvesting one. I also wish to inform you that the Mew was caught by suloku on an emulator without the vblank patch, and that only the OT gender was edited on THIS particular 3gpkm to make its OT Female to match the Real Pokepark Mew event. In game, I enabled the clock and obtained one lum berry for Mew. I just need to know if the one OT gender bit change affects any trash bytes. This is only for research, but I do not know if I accidentally submitted this one to RoC. If I did, and trash bytes check out, consider this a happy accident turned bugfix from suloku's original Mew. It is possible that glitzer or pomeg bugs could do this without A-Save (which I used on a duplicated copy of suloku's mew Save file that he gave me. He also said when he released it that he would provide offsets for editing the trainer info so anyone could customize the mew without re-waiting 34.25 days)

151 - 60510 - Mew - (Timid) - {S} {F} {St. GIGA} {Suloku}.3gpkm

Could someone answer this?