5th Gen GTS Research

[Icehawk78]

This is a new thread that I'm starting to keep track of research into the new B/W GTS.

Infinite Recursion has released an update to IR-GTS to work with 5G. Please take all questions regarding his program to this thread: http://projectpokemon.org/forums/showthread.php?14266-IR-GTS-BW-Release&p=118574

I've done some research including stuff based off of what GrovyleGibberish found.

(Updated 3/10/11 as per magical's discovery of response hash)

The process is roughly the same as that of Platinum and HG/SS. Challenge/response is handled as follows:

SALT = 'HZEdGCzcGGLvguqUEKQN'
request: [url]http://gamestats2.gs.nintendowifi.net/syachi2ds/web/[/url][worldexchange or common]/[action]?pid=#######
response: [challenge token]
request: [url]http://gamestats2.gs.nintendowifi.net/syachi2ds/web/[/url][worldexchange or common]/[action]?pid=#######&hash=[sHA1(SALT+token)]&data=[base64 encoded request data]
response: [response]+SHA1(salt + [base 64 encoded response] + salt);

Differences:

• Salt for the request hash ('HZEdGCzcGGLvguqUEKQN' instead of 'sAdeqWo3voLeC5r16DYv')
  
• Request URL (syachi2ds/web... instead of whatever the old one was)
  
• Response hash (The game checks this, to verify that it's a real server and not a fake one.)
  

Action Items:

• Determine response salt. (Rom hackers/peoples who found the request salt? I never found the original, just worked with it after someone else figured it out for me.) (3/10/11 - magical)
  
• Map out fill list of expansion of "data" part of the request Update - currently in progress. Doesn't quite match Grovyle's mode, but is close.
  
• Map out structure of "GTS Info" with Pokemon response data (searching and traded pokemon each)
  

If you want to help:

• If you know how to do rom hacking and research without someone holding your hand, the response salt is the biggest thing stopping me from going further. Find that and we'll have a working sendpkm in a few days, basically.
  
• New: Work on mapping out GTS info either sent to/from the server. Look at the data=[stuff] portion of a request, run it through a Base64 decoder, and figure out which bytes mean what.
  
• If you're not sure how else you can help, come visit us in the IRC (irc.pokestation.net, #projectpokemon). I'm 'nicholas' in the IRC and occasionally will ask for help with testing code when I get new ideas. Please don't be stupid, and please don't answer questions if you're just guessing.
  

=====

The following other details haven't actually been verified by me personally in a transaction but are presumably accurate. As far as I can tell, this seems to be primarily asking specifically about the search requests/responses.

Regarding the new Black & White GTS...

I've managed to get the details how the games and the server are communicating

Which are the following:

Checksum is XORed with 0x2db842b2 instead of 0x4a3b2c1d

Hash is calculated from SHA1("HZEdGCzcGGLvguqUEKQN" + token) instead of "sAdeqWo3voLeC5r16DYv" + token

The request from the DS to the BW server is not encrypted (unlike the GRNG with the checksum as seed in DPPt)

Length of the request is 0x0E or 0x0F:

0x00 - 0x03: PID Trainer

0x04 - 0x07: Total length of the following statements

0x08 - 0x09: Pokémon ID

0x0A: Gender

0x0B: Min. Level

0x0C: Max. Level

0x0D: Unknown

0x0E: Total results

0x0F: Country

Host for BW is the same as DPPt: http://gamestats2.gs.nintendowifi.net/

Root directory is different: /syachi2ds/web/worldexchange/

Game ID of Black is 0x14, White is 0x15.

GTS return data is 296 bytes:

0x000 - 0x001: unknown (2 bytes)

0x002 - 0x0DD: Pokémon data (220 bytes)

0x0DE - 0x0ED: unknown (always zero?) (16 bytes)

0x0EE - 0x127: GTS specific data (58 bytes)

The only difference is:

0x20 - 0x21: Trainer ID

0x22 - 0x23: Secret ID

0x24 - 0x33: Trainer Name

For everything behind this point, add 0x02 to the DPPt server

Have fun with it!

Oh... if someone's interested, I've created a program which can search the GTS for a Pokémon like the game itself does. It's B&W compatible as well.

Grtzz!!

Grovyle91

Edited March 23, 2011 by Icehawk78
MORE Updates

[Icehawk78]

I've also created a simple Ruby script for making requests to the real GTS server on the fly, that anyone else can feel free to use. This requires the Mechanize rubygem, which is fairly simple to install.

require 'mechanize'
require 'digest'
agent = WWW::Mechanize.new
url = "http://gamestats2.gs.nintendowifi.net/syachi2ds/web/worldexchange/result.asp?pid=289086352"
data = "&data=LbhNapAbOxGwAQAAFQIAAGR0NfQr_1X_Uv9U____AAAAAP__AAAAAAAk83_MmAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
secret = "HZEdGCzcGGLvguqUEKQN"
page = agent.get(url + '&hash=' + Digest::SHA1.hexdigest(secret + agent.get(url).content) + data).content

-----

The requested URL and requested data can be altered (I've just been using requests that my own game specifically made so that I knew everything would be properly formatted), and the last line will always get the correct response from Ninty's GTS.

[Seven Fates]

This thread is definitely something I'll be keeping an eye on.

[Icehawk78]

The following string was found in the game's code:

HZEd GCzc GGLv guqU EKQN

0001 d935 0000 2dd5 0000 0008

2db8 42b2

All together in one long thing. The first line is the salt used for request hashes. The last line is "what the checksum is XORed with" (though I'm unsure exactly of what that means, since the GRNG is no longer used).

I'm not certain what, exactly, the characters in the middle line are used for, nor am I certain now that the appended junk in the response is necessarily a new hash (though it appears to be - same length and character set used in SHA1).

[ceolceol]

edit: missed the first part.

[Icehawk78]

Apparently the old default return values don't work. 0x0001 for webexchange/info.asp issues an error 13247.

Right, that was what I mentioned above, with the bit about the "response hash".

For example, instead of 0x0001, the response needs to be "\x01\x00729abc61b2d89c9b4ed122265c827524a032cb42' (Note: that's a hex value of 01 00 32 27 29 ...)

Ceolceol: If you're doing research or want to help, feel free to join the IRC, I'm much more active there.

[Icehawk78]

Further updates:

The response hash is generated entirely by the response code itself. Two separate pages (info.asp and post.asp) that were requested and previously responded with response code of 0x0001 both had a consistent response of "\x01\x00729abc61b2d89c9b4ed122265c827524a032cb42"

[ceolceol]

This is the data sent when a user deposits a Pokémon.

Created from this base64 urlsafe string:

(deleted)

I had to remove the last C to decode it properly; nicholas, I think the string you posted was cut off in IRC, so I didn't get the full thing. Could you post the string in this thread?

It's little endian like everything else. It looks like the GTS Pokémon data starts at 0xF8 (National Dex ID, gender, level, requested Pokémon data; pretty much the same as what's specified here with Grovyle's offsets). I believe the actual Pokémon data is encrypted, though. What's odd is that it's ~345 bytes, which is much larger than the data sent in gen 4.

Edited March 8, 2011 by Guested

[Icehawk78]

Lbjy2R9RdBOwAQAAisQmBgAAW6Yb-A6J74cJBJEaKxOPb5bV2c-PWBJQNyaVt-EjdUeHRgv0ukl6kUOwCasux1dvAr_-oVFBhCXTi8jv_yVK2dZ3XlRkWlKZjrC0PPAG1HjSJnmhfXsto3qJl29_XirXGvBDEVKbYvJmbd8prcTo_6ZwrA0bwRbxyvfg0sa3GCDA8CB79LSSVhC9nzXsr9SeNjl0linLJBKJNJQ2mBhzcyqwXaRTHpkLn9mTbKYEud2TdkswI00t8qe9IOVgU_C93HwCq-tsBFhebBguK--6YR1rgve6QQAAAAAAAAAAAAAAAAAAAAD4AQEDEwIDAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAJDVvB1DAGUAbwBsAP__AAAAAP__3BQBABQCCAAAAAAAdFB3VUbTZZlJgHUICsST66dZInwPBhTE7dtp14CDiHd6CaccqxncUrZSZrNVZjvyKymDTWEAthBr8KVSaZMOzBvSy6ohlk8dnN6wBWo-rwrss2yR6gwEkNOXscZtmtkqEIjbFS2Qlu57K1d3XOQ0C5yVkhE_Gd1Oz7bwGb0GDnaAAAAA

[Icehawk78]

Wow, that's a lot bigger than I expected. 444 bytes.

Attached is the updated deposit data.

What. No. Why...?

Extract the .pkm from your save file using Pokegen or somesuch, and then post that.

[ceolceol]

What. No. Why...?

Extract the .pkm from your save file using Pokegen or somesuch, and then post that.

It's a retail cart, I can't. That's why I assumed you wanted the deposit data. My apologies.

[ceolceol]

Some additional hashes.

00 00 00 00 00 00 00 00

d9554885d70a779893c91e8c577c829ca3a110fd

05 00

4cdfc0f9a1d6b3eceac275448c672b0a6f5ef46e

04 00

6d7055ad1456823e9f714b5f9c9c9e79ab3ac363

(retrieved a deposited lv10 Zorua looking for a lv1-10 Entei)

3E 54 8D B3 00 00 FE E9 29 C0 76 05 17 3F F0 44 24 3D 69 B7 3D 37 14 96 C7 D8 91 D7 28 B5 6C 04 14 E8 03 EC 1B 05 CD 8B CC 5E 64 30 B5 C3 0B B5 7D D5 D6 D3 1B B8 A2 E7 6D C1 94 F9 5B A6 62 A1 32 4F C2 FC EC 09 8B 25 57 05 83 3D D6 50 7B 47 AA 21 F6 01 3C 5E 86 E1 16 55 E0 62 1C AA 45 B2 F1 9D 88 38 EA C4 13 9E 78 C5 8D 60 D7 89 24 BB 65 DA 10 F0 5A E1 B8 7C 01 7B 63 C2 7F 87 A0 F4 8C D3 87 B6 04 B0 7F 2E CC 68 5F DE 8B 31 4E D8 82 FB 38 0E A0 FE EF 84 CA 48 8A 03 F3 5F F8 E0 9C 9B E1 1E 14 DF 96 AA 7D 9D 2E BB 31 D2 1F 68 CF 6E 63 6E 64 20 0B B3 9F 3C 53 F3 FC FF 06 B9 1C 51 F3 6B B3 AF AD 10 41 0B 5B DB 2A 4A C0 A6 4C 8C 68 72 96 D1 93 A2 ED 2D EB F5 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3A 02 01 0A F4 00 03 00 09 00 00 01 DB 07 03 08 0E 22 33 00 00 00 00 00 00 00 00 00 1F 51 74 13 90 D5 BC 1D 43 00 65 00 6F 00 6C 00 FF FF 00 00 00 00 FF FF DC 14 01 00 14 02 08 00

971978ca1e79ddb66ae553cd7ae5f17b3f85e16f

Additionally, all 01 00 responses were signed with the same hash that nicholas got, and worldexchange/search.asp returns a 01 00 + hash response for no results (as opposed to nothing in gen 4).

Edited March 9, 2011 by ceolceol

[ceolceol]

I modded my GTS so that any B/W request will get hard-coded hashes and a Zorua sent to it. You can try it out by setting your DS's DNS to 205.185.126.109 and connecting to the GTS. You should get a Zorua, but it will say that you got the last Pokémon in your party (or an egg if that slot is empty). Go to your party/box and it will be Zorua.

[Seven Fates]

I can verify that this does indeed retrieve a Zorua without a hitch, though in my experience, it initially claimed to be an egg, and not the purrloin in my end spot. I suppose this is both a good job as well as a thank-you.

[ceolceol]

I can verify that this does indeed retrieve a Zorua without a hitch, though in my experience, it initially claimed to be an egg, and not the purrloin in my end spot. I suppose this is both a good job as well as a thank-you.

Thanks for testing it out. It's nice because it doesn't kick you back off afterwards like the gen 4 one used to do for some games.

[magical]

Response hash is calculated as:

sha1(salt + base64(data) + salt)

Where base64 is the url-safe variant, and salt is still HZEdGCzcGGLvguqUEKQN.

magical: 2

Game Freak: 0

[Seven Fates]

Thanks for testing it out. It's nice because it doesn't kick you back off afterwards like the gen 4 one used to do for some games.

Not a problem. I can never pass up the opportunity for a rare pokemon.

I guess this means it's only a matter of time until we have a fully functional GTS emu for Gen V.

[ceolceol]

Response hash is calculated as:

sha1(salt + base64(data) + salt)

Where base64 is the url-safe variant, and salt is still HZEdGCzcGGLvguqUEKQN.

magical: 2

Game Freak: 0

I'm speechless. Excellent work!

[Highlighter]

I registered for the sole purpose of saying Thanks. I really appreciate the effort you guys are doing to develop the Gen 5 GTS. The Gen 4 GTS made Heart-Gold so much more enjoyable (having an HM Mule makes the game a lot better, in my opinion.) So, thank you again and keep up the good work!

(I wish I could help, but I have no programming experience or expertise. )

[azurfire]

Response hash is calculated as:

sha1(salt + base64(data) + salt)

Where base64 is the url-safe variant, and salt is still HZEdGCzcGGLvguqUEKQN.

magical: 2

Game Freak: 0

Knowing how long those dudes in the IRC worked yesterday to find this darn salt, i'm speechless too. Good job, lol.

[abrammazal]

Great job to everyone that contributed to this, I've been following GTS development for gen 4 very closely and look forward to seeing this project advance.

[Icehawk78]

First post updated with new info.

[Nitronic]

is that modded GTS still up

Edit: Works like a charm looking forward to seeing a fully working GTS in the next few days

[Icehawk78]

Keep dreaming. I've been working on a fully working 4th gen GTS for over a year and it still doesn't work.

[Highlighter]

Keep dreaming. I've been working on a fully working 4th gen GTS for over a year and it still doesn't work.

Well, what about just something to connect the PC with a retail cart, ala IR-GTS?

Edit

IR-GTS info