research Gen 3 Event Generation Algorithm Research (10ANNIV, etc)

[Guest]

I agree with the first sentences from @InsaneNutter.
I think it is in fact interesting to know how and where we got this Distribution Cartridges from. (Even for this Research...)
But considering that many facts are kept in the dark, it's really hard to tell what actually happened.

Also I feel that discussing this will only spread more myths and rumors.
Btw. I'm not even convinced in terms of how effective these "digital fingerprint" are as long as I haven't seen it with my own eyes.
The word "digital fingerprint" isn't even a technical word and hides informations in itself. 
What if for example... if these cartridges aren't as individual as you might think?
There are just too many possibilities of what might have happened.

One last thing... with all due to respect...

On 3/12/2017 at 12:39 PM, Purin said:

All this may have been beneficial for the community, but it was still a terrible action and morally very, very wrong.

Talking about double standards...
I said this earlier and I say this again. This isn't really the right topic to talk about morals.
I mean we're talking about stolen items, so talking about what is immoral is kinda ironic in this case.   

[theSLAYER]

Digital fingerprint could be something as simple as all the roms being the same, but are signed differently somewhere.
Except if multiple copies of said rom was never dumped for public, not everyone would know.

Absence of evidence in this case simply means the public doesn't have large enough a sample to rule otherwise.


Anyway, this is getting out of hand.
I've understood the opinions being bounced around, more or less,
however we are going way off topic, and any further discussion should go back to 3rd event research related material.

I'm not sure how open @Purin is to receiving PMs,
but if you purchased Gen 3 distribution devices, and want to test it's legitimacy or whether you're scammed (digital fingerprint wise),
I'll say ask his help. (it doesn't mean he'll tell you what the fingerprints data are, he could either check em through or point you in the right direction)
Of course, if Purin has reservations helping, whether is it contract based or personal reasons, I'll remove this portion of my response.
[side note: if this conversation does occur, I wouldn't mind joining in and learning about the details, if all parties are open to it]

In any case, let's not derail this topic any further

[Invader TAK]

Right now, I'm just wondering what pre-2006 events used the 10ANNIV/10 ANIV generation method (if any) and how hard it'd be to make a custom GBA or DS distro ROM using said generation method that allows you to select ANY of the Pokemon using that method (including the 10ANNIV/10 ANIV ones) and which language you're sending to.

[Guest]

@Invader TAK It has been said and already stated before but maybe I should say it more clear.
A hacked and customized 10th Anniversary Rom would NOT be equal to a Official Distribution Rom of any respective Event you want it to be.
Besides this is not something anyone could do, it needs someone who has some decent IT knowledge and experience.
I personally don't know anyone here who could do it except the likes of @Bond697, @SciresM and @Kaphotics.
It also must be said that a generated Pokemon from a hacked Rom still wouldn't be legit and at the same time wouldn't be more legal than any other perfectly hacked Event Pokemon. Therefore it is questionable how it would be beneficial considering the time and effort that is needed to make it even possible, which is why I personally don't go deeper into this Rom other than for needed Research purposes.
I'm not saying that it wouldn't be interesting to see this from a hacking standpoint, I just don't see it happening.

[Invader TAK]

40 minutes ago, ajxpk said:

@Invader TAK It has been said and already stated before but maybe I should say it more clear.
A hacked and customized 10th Anniversary Rom would NOT be equal to a Official Distribution Rom of any respective Event you want it to be.
Besides this is not something anyone could do, it needs someone who has some decent IT knowledge and experience.
I personally don't know anyone here who could do it except the likes of @Bond697, @SciresM and @Kaphotics.
It also must be said that a generated Pokemon from a hacked Rom still wouldn't be legit and at the same time wouldn't be more legal than any other perfectly hacked Event Pokemon. Therefore it is questionable how it would be beneficial considering the time and effort that is needed to make it even possible, which is why I personally don't go deeper into this Rom other than for needed Research purposes.
I'm not saying that it wouldn't be interesting to see this from a hacking standpoint, I just don't see it happening.

I'm not saying hack the 10ANNIV ROM, I'm talking about extracting the generation algorithm from the 10ANNIV ROM and using it to make an entirely new ROM. But considering no one's bothered at this point, I'll try to make this is the last time I bring it up. Only reason I did was because a custom ROM that uses the actual event generation method could be an "as legit as possible" replacement for the distros that are lost forever.

Edited March 19, 2017 by Invader TAK

[Guest]

Yeah, I agree with this. Making a Rom from the scratch IMO would also be more efficient. Because you would have a prototype and can change its algorithm so that it covers other algorithms as well. But it's just as complicated to do and the problem basically remains the same. I think if there is someone who wants to do it, it will happen. The thing is just, it's not happening right now. Maybe @suloku is interested in it, but he's very busy at the moment. 

[Invader TAK]

17 hours ago, ajxpk said:

Yeah, I agree with this. Making a Rom from the scratch IMO would also be more efficient. Because you would have a prototype and can change its algorithm so that it covers other algorithms as well. But it's just as complicated to do and the problem basically remains the same. I think if there is someone who wants to do it, it will happen. The thing is just, it's not happening right now. Maybe @suloku is interested in it, but he's very busy at the moment. 

That's a good thought on being to cover other generation algorithms with a custom ROM. It'd make distributing the likes of Ageto Celebi And WISHMAKR/CHANNEL Jirachi a lot easier (and portable). Plus you could remove the "one Pokemon per file" limit if you wanted.

[suloku]

On 19/3/2017 at 6:15 PM, ajxpk said:

Maybe @suloku is interested in it, but he's very busy at the moment. 

I'd change that to "I don't have the skills", but it seems someone has already covered the difficult part!: http://forums.glitchcity.info/index.php?topic=7861.0

Basically, it's a custom rom that gets sent over to GBA and does stuff to the savefile. The example changes the player name's first letter to "z", but adding a pokémon to the party/first slot would be easy. The only thing really missing would be pokemon generation algorythm, the downside is that this uses gamecube/wii connection, but maybe doing it over two GBA wouldn't be that difficult for someone familiar with GBA development.
Edit: I think it does alter the savefile in RAM, so changes only apply if the player runs the game and saves afterwards.

[jojo12100]

2 hours ago, suloku said:

I'd change that to "I don't have the skills", but it seems someone has already covered the difficult part!: http://forums.glitchcity.info/index.php?topic=7861.0

Basically, it's a custom rom that gets sent over to GBA and does stuff to the savefile. The example changes the player name's first letter to "z", but adding a pokémon to the party/first slot would be easy. The only thing really missing would be pokemon generation algorythm, the downside is that this uses gamecube/wii connection, but maybe doing it over two GBA wouldn't be that difficult for someone familiar with GBA development.
Edit: I think it does alter the savefile in RAM, so changes only apply if the player runs the game and saves afterwards.

Really interesting, I think we have first to try just to add a pokemon to the party before trying to emulate the pokemon generation algorythm.

In fact what interest me the most is how they bypass the language/version/region check.

[Guest]

It's very easy if you have access to the GBA SDK. It comes with a C sample called "multi_boot":

Looks familiar, doesn't it?
The sample is complete with server and client application.

For the most basic distribution, just rewrite the client to do the following:
1. Check if the correct GBA cartrdige is inserted by looking at the header ("POKEMON EMERBPEE" etc.), show an error if not
2. Read the savegame of the inserted GBA cartridge
3. Check if the party has a free slot, show an error if not
4. Generate your PK3 data, write it to the party (or anything else you want, such as wondercards, Old Sea Map etc., anything is possible)
5. Fix checksums
6. Write the savegame back to the inserted GBA cartridge
7. Show success message

[jojo12100]

52 minutes ago, Purin said:

It's very easy if you have access to the GBA SDK. It comes with a C sample called "multi_boot":

Looks familiar, doesn't it?
The sample is complete with server and client application.

For the most basic distribution, just rewrite the client to do the following:
1. Check if the correct GBA cartrdige is inserted by looking at the header ("POKEMON EMERBPEE" etc.), show an error if not
2. Read the savegame of the inserted GBA cartridge
3. Check if the party has a free slot, show an error if not
4. Generate your PK3 data, write it to the party (or anything else you want, such as wondercards, Old Sea Map etc., anything is possible)
5. Fix checksums
6. Write the savegame back to the inserted GBA cartridge
7. Show success message

 

Great, I'll try to watch if I can do something, hope that other people will try too.

Theorically it's not difficult.

Maybe we could do our 10ANNIV universal distrib rom like that or better the Bonus Discs (Celebi, Jirachi ...) distrib?

[Guest]

On 3/22/2017 at 8:24 AM, jojo12100 said:

Great, I'll try to watch if I can do something, hope that other people will try too.

Theorically it's not difficult.

Maybe we could do our 10ANNIV universal distrib rom like that or better the Bonus Discs (Celebi, Jirachi ...) distrib?

10ANNIV actually uses 10 different client applications, one for each Pokémon. You can either do it like this and compile one client for each Pokémon, or you could do it a bit more sophisticated by sending a few parameters from the server to the client after the client application has successfully booted. That way only one client is needed for everything, while still being able to choose what to distribute from the server.

[jojo12100]

1 minute ago, Purin said:

10ANNIV actually uses 10 different client applications, one for each Pokémon. You can either do it like this and compile one client for each Pokémon, or you could do it a bit more sophisticated by sending a few parameters from the server to the client after the client application has successfully booted. That way only one client is needed for everything, while still being able to choose what to distribute from the server.

I don't have the skills to do more sophisticated. But I'm pretty sure amazing members will have.

[Guest]

Does anyone have 1) the PK3 generation algorithm and 2) code for fixing checksums of all Pokémon GBA saves, in C language?

After that we only need to figure out how to dump saves into RAM, inject the distribution and put it back to the cartrdige.

[suloku]

13 hours ago, Purin said:

1) the PK3 generation algorithm

That would be the most troublesome to get, not sure how the RE efforts on that have gone.

13 hours ago, Purin said:

2) code for fixing checksums of all Pokémon GBA saves, in C language

BlackShark's Mirage Island appearance program was what I used as basis, which I think in turn is based on kaphotics checksum verification tool. Thinking back, if it weren't for his program I probably wouldn't have dared to play with gen 3 savegame editing.

Also, I think the client from glitchcity I posted previously has been updated to handle checksums and even party/pc pokémon with simple functions.

I have another alternative, for now we have:
1) A client for each distro
2) Client-host communication

As an (easiest?) alternative, I suggest that the final client binary has a dummy pk3 file embeded. The host generates the pk3 once selected using the needed algorythm, then it changes the dummy pk3 file to the actual pk3 on the client and sends it. The client would only need to be a really simple app to check if there's party/pc space and add the pk3.

[Sabresite]

17 hours ago, Purin said:

Does anyone have 1) the PK3 generation algorithm and 2) code for fixing checksums of all Pokémon GBA saves, in C language?

If someone provides this, I will see what I can do.

The Gen 3 generation algorithms are well known.  Past RNG Research thread on Smogon details just about every generation method known.
I'll give a quick overview of how it works.  You will have to do the research to fill in the gaps.  Also this may not be 100% accurate.

1) battle should happen?
2) which encounter slot? (Determines species & level based on location)
3) Choose nature or Sync is determined
3b) Loop through creating PIDs until you find one that matches above nature.
4) PIDL
5) PIDH
6) IV1
7) IV2

Here is the shit part.  There are usually between 1 and 2 vBlanks.  Where the vBlanks occur depends on the situation. For example, fishing, caves, grass, etc.
Lastly, static encounters, gifts do not have #1-3, and this doesn't apply to breeding.

[Guest]

So I was working on this for the past 15 hours and it was coming along quite well with custom font code and all working, but then Notepad++ crashed and left me with a 0 byte file. I'm so angry right now... I'm not going to try again, but I hope someone else will.

All the relevant code for reading/writing a savegame on the GBA is here. It also works fine from a multiboot client as long as you keep memory usage down. Good luck.

[Invader TAK]

Thinking about it, I'd be fine with a GC/Wii homebrew program that can handle every Gen 3 event we have the algorithms for (or can figure out). I'd prefer a GBA or DS one for the portability, but I'd be fine with anything. Like I said, I'm only suggesting this so we can replace distros that are lost forever (like the US 10 ANNIV). I personally feel as long as the event Pokemon is generated the EXACT same way it was originally, it'd still be legit even if it was from a homebrew distro.

[Guest]

The exact same way would not be possible, because no one would ever be able to recreate the whole generation process in a 100% accurate way.
Legal is as far as you would be able to get, no matter which way you choose to go.

[suloku]

24 minutes ago, ajxpk said:

no one would ever be able to recreate the whole generation process in a 100% accurate way.

Are we talking about trash bytes here? Just mentioning due to colosseum/xd getting rid of those, thus making them virtually the same.

A generator for all the events for me would just be something nice and insteresting to have both for users and documentation purposes on the algorythms, even though I'd rather have compilations of the real deal (which has been in the works for a while) or distro rom dumps.

The only real use would be for those pokémon that came with exclusive moves on the events, as you can get anything else trough legit means and RNG manipulation (probably?).

[Guest]

I think what would be good to have is an open source distro rom which does the same as 10ANNIV etc. I was trying, but then that accident happened.

[Guest]

@suloku Yes, not just the trash bytes itself so much actually but the fact that they're originating from thumb code, which is something that is impossible to reproduce the same way it originally was. And as a researcher all I can do is documenting the trash bytes that are there in the pk3 files I get, but nothing more than that. 

Speaking of Trash Bytes... I'm not so much a fan of saying "We can just wipe the trash, which is achievable in a legal way and then it's fine.".
I mean you're correct when you say a 3rd gen pokemon with 0 trash bytes is still legal because it's achievable with the use of COL/XD. It is absolutely. But at the same time as it is legal this will also mean that it's touched. And from my own experience of collecting 3rd Gen Events this is also extremely rare, I mean so far I have only witnessed one Event Pokemon with 0 trash, which I'm not even sure about if it's legit...
You can imagine if you have a collection of hundreds of Event Pokemon with 0 trash that would be highly suspicious.
That's why I do think that they play an important role in legality analysis at least when it comes to Gen 3.
And in a legality checker one should be at least informed about the absence of Trash Bytes.
(It doesn't necessarily has to affect the legality, but is still an indicator.)

And don't worry trash isn't really the problem when we talk about the results, except for the events which trash bytes hasn't been documented so far.
What I was talking about is the generation process as a whole and there will never be a 100% legit recreation of it.
I mean if we take the word "legit" seriously we would have to recreate the whole generation process 1:1.
And even this would be already a stretch of the word legit. But let's say if the algorithm would be enough for it being legit.
That's not gonna happen. So we can talk about legal but not legit.



I think I also have to distant myself a little, because some of my collector friends fear that my motive isn't clear.
I personally want to support the legality analysis in the first place. 
So that's on my priority list right now.

But I also have to admit that an open source multiboot rom would be interesting to see indeed.
So I can only wish good luck on that and I'm curious to see the progress.  

[Invader TAK]

Wouldn't the algorithm generate the trash bytes if it's generating via GBA multiboot? Or are the trash bytes a product of the official distro cart/ROM (or disc/ISO in the case of Berry Glitch Fix Zigzagoon, Ageto Celebi, WISHMAKR Jirachi and CHANNEL Jirachi)?

Ignore that, I didn't see the post above this one.

Ugh, these damn trash bytes are starting to make me think an open source multiboot ROM is only worth it for people who don't care about them. And at that point, just gen.

Ok, I apologize for that. I've had time to calm down. I was just frustrated at seeing "yeah, but the trash bytes" for the millionth time. I get why they're important, but they're still frustrating. Makes me glad GameFreak fully embraced Wonder Cards in Gen 4 and have kept them every since. They make life so much easier.

Besides, the people hoarding events just because they're rare frustrate me even more. But that's a rant for another time.

Edited March 25, 2017 by Invader TAK

[suloku]

The trash bytes are unique to the device/rom that generates the pokémon, since they are just what lied in memory when the pokémon was generated (from my point of view I don't see a reason why they didn't clear those bytes, but at the same time, I also don't see a reason why they should have cared).

What I said about Colo/XD removing them is from this point: OK, we know we can't never do 1:1 replication (since that needs the actual distro device, or a copy of it), but if we throw trash aside and just concentrate on what makes the pokémon unique to that event (PID, TID, OT, OTgender, IV, level, met place...etc) and we can replicate that 1:1 that would still be awesome. If you are using a homebrew app to get them I'd personally still prefer single pk3 files from people who actually got the event.

But for personal use? Fun? Gen 4 onwards (since that clears the trash too)? I'd really like to have something like that, albeit I understand the concerns collectors may have, but faking some gen 3 event on gen 4 onwards is really difficult to discern (and many are available anyways).

That's just my personal opinion, actually I think the fact that we can't replicate trash is good, because A) Doesn't matter for gameplay/gen 4 onwards B) It's a good way to quickly know if the pk3 file was just generated C) There's a legit way to get rid of the trash bytes so even generated pokémon can be claimed to be fully legal without having trash bytes.

ps: I think I finally found a way to tell legal from legit appart (I always confuse the correct word), hope I didn't mix them up.
ps2: yes, GF switching to an in-game generation for most events is really nice, gen 1-3 are a nightmare for events....and gen 3 is actually better than those.

[St. GIGA]

Does anybody know how to check trash bytes on Japanese Pokemon Emerald Method 1 Pokemon, and what might affect the trash bytes in both the OT and Nickname? I would like to know how to check my Mew that Suloku RNG'd.