[GEN 3] Mystery Event/Gift Research

[superguideguy]

I've been lurking for a while now, but I think I'm finally ready to add what information I know about the distribution ROM. I apologize if I repeat something that was already known, but I checked the last 5 or so pages, and I didn't find the information I have.

1. The actual script data for the Aurora Ticket distribution starts at 0x12728. It is the raw data - unencrypted and uncompressed. Noteworthy is the fact that the checksum and header are not stored here. I edited the script to give me both the Aurora Ticket and Mystic Ticket, and it worked! This means the checksum is calculated dynamically, rather than stored elsewhere on the ROM.
2. The Wonder Card starts at 0x14FB8. At least, that is where the checksum would be. I haven't confirmed if the checksum is calculated dynamically, but I can't find it stored anywhere in the ROM, so I'm assuming it's dynamic.
3. I tried my hand at disassembly. Here's what I've learned:
   • The disassemblers designed for generic ARM7-TDMI code don't like disassembling THUMB code.
   • The disassemblers designed specifically for the GBA (e.g. luvdis) don't like disassembling ARM code.
   • Overall, I'm bad at reading assembly code.
   • Something (maybe) useful: the Wireless Adapter uses the 32-bit "normal" serial communication mode. (It sends data by writing to register 0x04000120.)
4. Not strictly about the distribution ROM, but the "virtual" script functions (opcodes B8 through BF) can access data in the Wonder Card and Wonder News sections. Thus, a properly crafted Wonder Card could add an additional 320 bytes of script data. Finding a way to distribute Wonder News would add another 440 bytes of usable script data.
5. I've been doing some digging into the decompilation projects, and I've found two functions that might be responsible for Emerald rejecting the distribution:
   • "ValidateMEventClientHeader" / "sub_801B6A0"
     • FireRed: https://github.com/pret/pokefirered/blob/2880cf2a51ea36fa36f00d9ecf07177e5955c882/src/mevent.c#L788
     • Emerald: https://github.com/pret/pokeemerald/blob/4dd3cc7b7691afb8ecd2e8af8b2ebfa717dd2414/src/mevent2.c#L380
   • "CheckCompatibility"
     • Ruby: https://github.com/pret/pokeruby/blob/f839afb24aa2c7b70e9c28a5c069aacc46993099/src/mystery_event_script.c#L38
     • FireRed: https://github.com/pret/pokefirered/blob/2880cf2a51ea36fa36f00d9ecf07177e5955c882/src/mystery_event_script.c#L30
     • Emerald: https://github.com/pret/pokeemerald/blob/4dd3cc7b7691afb8ecd2e8af8b2ebfa717dd2414/src/mystery_event_script.c#L28
6. Given that "ValidateMEventClientHeader" appears in FRLG and Emerald, but not RS, I suspect it is this function that is used for the Wireless Adapter, while "CheckCompatibility" is used for the eReader. Regardless, as long as it is one of these two functions, the remedy is the same. Replace a 0x1 somewhere with 0x5, and replace a 0xF with 0x20F or 0x38F. Unfortunately, like the checksums, these values do not appear to be stored directly on the ROM. They are likely caused by MOVS or ADDS instructions, instead.

I hope this helps, and let me know if there's anything else I can do to help.

[Sabresite]

Great job, thank you!

[superguideguy]

Long-winded story in the spoiler tag.

Spoiler

For the past week I've been working on finding out how the distribution ROM was sending the compatibility header to the game. To cut a very long story very short, that adventure was fruitless.

First, I tried making a custom disassembler. My issue with reading assembly was that it looked weird, so I thought that if I translated every THUMB function to a "C" look-a-like, I would understand it better. I got pretty far, but then I ran into the horror that are function pointers. This also caused my trials with luvdis's configuration mode to end as well. Simply put, without an extremely good guessing algorithm, function pointers make it near-impossible to analyze a ROM.

I turned back to the internet, and researched if there were any other GBA disassemblers other than luvdis. I found a Medium article that recommended ISA Pro, but at ~$2600, I said no way. In desperation, I stumbled across a blog post (https://cturt.github.io/pinball.html) that recommended a program called Ghidra. Ghidra is a multi-platform, free and open source decompiler, that is compatible with ARMv4T. The one major issue that I have with it is that it is developed by the NSA. Yes, that NSA.

The tool took some getting used to, but I found that it turned the ROM from a messy mix of THUMB and ARM functions into a nice C-style program (save for the absolutely useless variable names like "bStack11"). Even with this, though, I failed to find how the program was writing the script to the SIODATA32 register. (I still have not found how it does this. My best guess is that it actually sends a command to the Wireless Adapter to DMA from ROM, or from WRAM around 0x03002500.)

What Ghidra did help me find was similarities between the distribution ROM and FireRed/Emerald. In particular, I found some low-level functions in the FireRed/Emerald disassemblies (called STWI), that were also present in the distribution ROM. This reminded me that FireRed & Emerald can send Wonder Cards, and thus act as a server as well as a client.

Even this was a wild goose chase. I was unable to find out how the server commands linked to the low-level STWI commands. Thus, I turned back to my original strategy of looking for the hex values associated with the compatibility header, in particularly 0x101. To my surprise, Ghidra found a function that relied upon it. A very familiar function...

It was a copy of FireRed's ValidateMEventClientHeader function! Nothing called it, of course; stupid function pointers ruining everything. But it made me wonder, why was it here? As I wondered, I returned to common_mainseq_4 (https://github.com/pret/pokefirered/blob/2880cf2a51ea36fa36f00d9ecf07177e5955c882/src/mevent_server.c#L118), which had been a common thread in my investigation. Indeed, I had been puzzling over this function for weeks. But, I had been looking at Emerald's version. The one thing the FireRed disassembly has over Emerald's is that more functions and constants have names. This ended up being the key factor in the solution.

In a fit, I looked at FireRed's gMEventSrvScript_SendCard. Emerald's is an absolutely atrocious mess (https://github.com/pret/pokeemerald/blob/5183cdb35722549d6b465ccaf8c4a21422ecb254/src/mevent_scripts.c#L178). But FireRed's is much cleaner (https://github.com/pret/pokefirered/blob/a0b2fa5d33c43bc4f42fc4f132f73e960d771d5b/src/mevent_scripts.c#L179). In particular, it has the line "SRV_SEND(0x20, gMEventClientScript_Send1442CC),". I knew that 1442CC was associated with the compatibility header, so I went down the rabbit hole. The first instruction sent was by gMEventClientScript_Send1442CC was "CLI_SNDHEAD".

I realized that I had everything all wrong. The distribution ROM doesn't send the compatibility header to the games, the games send the compatibility header to the distribution ROM! The only reason the games have the ValidateMEventClientHeader is because they can act as the server. I checked the definition of CLI_SNDHEAD, and saw it was client instruction 8. I looked at the client's version of common_mainseq_4, case 8, and found a function called "BuildMEventClientHeader" (https://github.com/pret/pokefirered/blob/2880cf2a51ea36fa36f00d9ecf07177e5955c882/src/mevent.c#L753). The function, as expected, builds the header in question.

Emerald's counterpart is sub_801B580 (https://github.com/pret/pokeemerald/blob/ebade7affb31d5bcdc17cdcd3895758010ee6f66/src/mevent2.c#L338), which also build the header as expected. At least, if "a1" is 0. If a1 is not 0, then it should already be compatible. One more trace later, and I discovered that a1 is 1 for Wonder News, and 0 for Wonder Cards. Finally, the mystery had been solved.

TL;DR: Rather than the games checking the compatibility header, the distribution ROM does. Patch instructions below. (If anyone wants to make an IPS/UPS/BPS patch, go ahead! I just don't feel like downloading one more program for this project to make one.)

Due to space constraints, it is impossible to patch a fix in to the same location. However, it is possible to patch out the check completely. Replacing 0x714C through 0x7173 (inclusive) with 0x00 patches out the region, game, and language checks. It leaves the sanity check, however. (Also, see below regarding the language check.)

Why does this matter? Using direct injection, I already knew that it is possible to use "comparefarbytetobyte 0x80000AE 'E'" (ASCII 'E', not the proprietary encoding 'E') to check for whether a game is Emerald or not. "comparefarbytetobyte 0x80000AF" can also be used to check for language. Thus, while scripts for both versions - and texts for all languages - would need to fit into 1000 bytes, it is now possible to make a multi-version multi-language distribution ROM.

One last thing. I found a glitchcity forum post (Reply #35 at https://forums.glitchcity.info/index.php?topic=7114.30) that confirmed my suspicions regarding the CheckCompatibility function. FireRed, LeafGreen, and Emerald act as Japanese cartridges, regardless of their actual language. This carries over to ValidateMEventClientHeader. Thus, all games can receive the Mystery Gift, regardless of the language of the distribution. This does leave a question as to how the European distribution ROM knew which version to send. It may be the case that only English carts act as Japanese carts. However, without a European decompilation, and without a European cart of my own, I'm left without means to test this hypothesis.

[skyparrillo]

I'm curious as to how possible it would be to edit these event files, or other ROMs that have been released public like Aura Mew, to distribute custom Pokemon. Say I wanted to create an event cartridge using a flash card on my GBA to share a custom shiny Bulbasaur with my friends, instead of just trading it over. To try and recreate the feeling of getting a cool new event Pokemon through an "actual" event. How possible is this currently?

[superguideguy]

On the event side, it’s just a flag. (See first post in thread for which flag.) If the flag is set, the event can be redistributed to the same region. (FRLG can distribute to FRLG, and E can distribute to E.)

As for if you want to distribute to both FRLG and E at the same time, you need a special distribution ROM. Luckily, any FRLG/E version can be turned into a distribution ROM with a simple patch. That is, if you know where the function is.

I’ve been a bit busy with university, so I haven’t updated it in a while, but I actually have a patcher on my GitHub. As long as you have an English FireRed/LeafGreen v1.1 or English Emerald, you can make a distribution ROM from it.

GitHub is here: https://github.com/superguideguy/gen-iii-event-patcher

[skyparrillo]

4 hours ago, superguideguy said:

On the event side, it’s just a flag. (See first post in thread for which flag.) If the flag is set, the event can be redistributed to the same region. (FRLG can distribute to FRLG, and E can distribute to E.)

As for if you want to distribute to both FRLG and E at the same time, you need a special distribution ROM. Luckily, any FRLG/E version can be turned into a distribution ROM with a simple patch. That is, if you know where the function is.

I’ve been a bit busy with university, so I haven’t updated it in a while, but I actually have a patcher on my GitHub. As long as you have an English FireRed/LeafGreen v1.1 or English Emerald, you can make a distribution ROM from it.

GitHub is here: https://github.com/superguideguy/gen-iii-event-patcher

I found this when I was searching, but couldn't get the latest version to work. I've tried running it on two computers but it just errors out and doesn't launch. Using CMD or Powershell didn't provide any different results. I got the last release from September to launch though! I'll have fun messing with this 

[superguideguy]

I’ll look into it when I get home. It should just be able to be executed like any other jar file. What was the error, if you don’t mind me asking? Alternatively, you can report it on the GitHub issues tab.

[skyparrillo]

Double clicking it gives the error "Error: A JNI error has occured, please check your installation and try again" Hitting Ok on that error brings up another window saying "A Java Exception has occurred."

Running it with the java command in CMD gives the error "Error: Could not find or load main class g3ep.jar"

I am curious about the "ROM Patcher" and "Event BIN Creator" tabs. I assume Event BIN Creator isn't available yet as it's grayed out, but the ROM Patcher isn't, does this mean it's working? I'm having trouble understanding how some of this works. I was able to use the SAV Patcher and test it, which is very cool, but I am hoping you can point me in the right direction on using the ROM Patcher. I'm reading through all the info on your github but can't find anything specifically referring to it

EDIT: I didn't realize the tabs weren't seperate, as in I need to use the SAV Patcher AND the ROM Patcher at the same time. DOH! I think I'm making progress, just slowly

Edited February 12, 2021 by skyparrillo
extra information to avoid double posting

[superguideguy]

As for why v0.8.0 is causing issues, it appears that the "A JNI error has occurred" is Java's unhelpful way of saying your Java is outdated. There was a function that relied on Java 11, but I can revert it to a Java 8 compliant counterpart. Should be done a few minutes after posting this.

As for how to use v0.7.2 (or earlier), there was some documentation, but I got rid of it, since it didn't work for 0.8.0. Yes, the Event Creator wasn't developed then. Technically, it still isn't released; there's some files in the source code, but I wouldn't trust them too much. As for ROM Patcher, the patches are hardcoded into the program. I forget if that version is Emerald only or LeafGreen and Emerald; I know for a fact FireRed doesn't work. Version 0.8.0 fixed that however.

I'll hopefully have v0.8.1 out in a few minutes. If there's anything else, let's keep it to GitHub or DMs - no need to clutter this thread anymore.

[CitrusBolt]

Are the PCNY Egg scripts available?  I'm trying to figure out why/how they're Method 2.  Trying to reconstruct them from the prototype Surfing Pichu script gives me Method 1 and I can't figure out how to it's possible to consistently end up with a Method 2 egg without modifying the game code.  I must be overlooking something, but I can't make sense of it.