Difference between revisions of "GTS protocol"

From ProjectPokemon Wiki
Jump to: navigation, search
(Protocol)
Line 1: Line 1:
The following is '''wild conjecture''' based on LordLandon's [http://projectpokemon.org/forums/showthread.php?780-GTC-website-research&p=67086&viewfull=1#post67086 sendpkm.py].
+
=[http://uwujojedeh.co.cc Under Construction! Please Visit Reserve Page. Page Will Be Available Shortly]=
 +
The following is '''wild conjecture''' based on LordLandon's [http://projectpokemon.org/forums/showthread.php?780-GTC-website-research&p=67086&viewfull=1#post67086 sendpkm.py].
  
 
Communication with the GTS is done over regular HTTP with http://gamestats2.gs.nintendowifi.net/.  The same protocol is used for all five Gen IV games.
 
Communication with the GTS is done over regular HTTP with http://gamestats2.gs.nintendowifi.net/.  The same protocol is used for all five Gen IV games.
Line 9: Line 10:
 
==Protocol==
 
==Protocol==
  
All requests to the server are GET requests of the form <code><var>page</var>.asp?pid=<var>pid</var>&amp;hash=<var>hash</var>&amp;data=<var>data</var></code>.
+
All requests to the server are GET requests of the form &lt;code&gt;&lt;var&gt;page&lt;/var&gt;.asp?pid=&lt;var&gt;pid&lt;/var&gt;&amp;amp;hash=&lt;var&gt;hash&lt;/var&gt;&amp;amp;data=&lt;var&gt;data&lt;/var&gt;&lt;/code&gt;.
 
However devices prior to the DSi also send requests without the hash (followed by a requests with hash) to which you should also return proper values or else the DS will assume communication broke off.
 
However devices prior to the DSi also send requests without the hash (followed by a requests with hash) to which you should also return proper values or else the DS will assume communication broke off.
  
Line 15: Line 16:
  
 
The pid is an unsigned 32-bit integer that appears to uniquely identify a game cartridge. Although this has not been confirmed entirely:
 
The pid is an unsigned 32-bit integer that appears to uniquely identify a game cartridge. Although this has not been confirmed entirely:
Your PID is generated when you get your friendcode for the first time and is set to: friendcode & 0x7fffffff
+
Your PID is generated when you get your friendcode for the first time and is set to: friendcode &amp; 0x7fffffff
When you change the device and are forced to change your friendcode with it, your PID doesn't change but you get a new friendcode (at this point there's no friendcode->pid mapping anymore).
+
When you change the device and are forced to change your friendcode with it, your PID doesn't change but you get a new friendcode (at this point there's no friendcode-&gt;pid mapping anymore).
  
 
For the mathematically inclined: Eevee's Platinum pid is 192615460 (0x0b7b1424) and his Pal Pad code is 0904 2026 4621.
 
For the mathematically inclined: Eevee's Platinum pid is 192615460 (0x0b7b1424) and his Pal Pad code is 0904 2026 4621.
Line 24: Line 25:
 
(4th Gen)
 
(4th Gen)
  
Before each "real" request, the game sends a request of the form <code><var>page</var>.asp?pid=<var>pid</var></code> and the server responds with a 32-byte challenge token. The game computes <code>sha1("sAdeqWo3voLeC5r16DYv" + token)</code> and uses that as the <var>hash</var> value which it sends to the server. The <var>data</var> parameter is encrypted, as explained further down.
+
Before each &quot;real&quot; request, the game sends a request of the form &lt;code&gt;&lt;var&gt;page&lt;/var&gt;.asp?pid=&lt;var&gt;pid&lt;/var&gt;&lt;/code&gt; and the server responds with a 32-byte challenge token. The game computes &lt;code&gt;sha1(&quot;sAdeqWo3voLeC5r16DYv&quot; + token)&lt;/code&gt; and uses that as the &lt;var&gt;hash&lt;/var&gt; value which it sends to the server. The &lt;var&gt;data&lt;/var&gt; parameter is encrypted, as explained further down.
  
 
(5th gen)
 
(5th gen)
  
The process is mostly the same, except that the response hash is <code>sha1("HZEdGCzcGGLvguqUEKQN" + token)</code>
+
The process is mostly the same, except that the response hash is &lt;code&gt;sha1(&quot;HZEdGCzcGGLvguqUEKQN&quot; + token)&lt;/code&gt;
  
  
Line 34: Line 35:
 
That is, each request looks like the following:
 
That is, each request looks like the following:
  
# Game requests <code>GET /pokemondpds/<var>page</var>.asp?pid=<var>pid</var></code>
+
# Game requests &lt;code&gt;GET /pokemondpds/&lt;var&gt;page&lt;/var&gt;.asp?pid=&lt;var&gt;pid&lt;/var&gt;&lt;/code&gt;
# Server responds with <var>token</var>
+
# Server responds with &lt;var&gt;token&lt;/var&gt;
# Game requests <code>GET /pokemondpds/<var>page</var>.asp?pid=<var>pid</var>&amp;hash=<var>sha1(...)</var>&amp;data=<var>data</var></code>
+
# Game requests &lt;code&gt;GET /pokemondpds/&lt;var&gt;page&lt;/var&gt;.asp?pid=&lt;var&gt;pid&lt;/var&gt;&amp;amp;hash=&lt;var&gt;sha1(...)&lt;/var&gt;&amp;amp;data=&lt;var&gt;data&lt;/var&gt;&lt;/code&gt;
 
# Server responds with payload
 
# Server responds with payload
  
Line 45: Line 46:
 
A 32-bit checksum of the data is used as the key for the encryption. The checksum is simply the sum of every byte of the data.
 
A 32-bit checksum of the data is used as the key for the encryption. The checksum is simply the sum of every byte of the data.
  
The first 4 bytes of the sent data are the checksum, <code>xor</code>ed with <code>0x4a3b2c1d</code>, and sent in network byte order (big-endian).
+
The first 4 bytes of the sent data are the checksum, &lt;code&gt;xor&lt;/code&gt;ed with &lt;code&gt;0x4a3b2c1d&lt;/code&gt;, and sent in network byte order (big-endian).
  
 
===Encryption===
 
===Encryption===
Line 51: Line 52:
 
The data sent to the server is encrypted with a [http://en.wikipedia.org/wiki/Stream_cipher stream cipher], and [http://en.wikipedia.org/wiki/Base64#URL_applications urlsafe-base64]-encoded.
 
The data sent to the server is encrypted with a [http://en.wikipedia.org/wiki/Stream_cipher stream cipher], and [http://en.wikipedia.org/wiki/Base64#URL_applications urlsafe-base64]-encoded.
  
This encryption algorithm, like others used in the game, uses a [http://en.wikipedia.org/wiki/Linear_congruential_generator Linear Congruential Generator] (not a very strong choice). The multiplicative constant is <code>0x45</code>, and the additive constant is <code>0x1111</code>. It appears that the game uses a signed dword to store the seed, which doesn't really matter; it shouldn't affect anything. (It means that the modulus is effectively <code>0x80000000</code>.)
+
This encryption algorithm, like others used in the game, uses a [http://en.wikipedia.org/wiki/Linear_congruential_generator Linear Congruential Generator] (not a very strong choice). The multiplicative constant is &lt;code&gt;0x45&lt;/code&gt;, and the additive constant is &lt;code&gt;0x1111&lt;/code&gt;. It appears that the game uses a signed dword to store the seed, which doesn't really matter; it shouldn't affect anything. (It means that the modulus is effectively &lt;code&gt;0x80000000&lt;/code&gt;.)
  
  GRNG[n+1] = (GRNG[n] * 0x45 + 0x1111) & 0x7fffffff
+
  GRNG[n+1] = (GRNG[n] * 0x45 + 0x1111) &amp; 0x7fffffff
  
The GRNG is seeded with the <var>checksum</var>, like so:
+
The GRNG is seeded with the &lt;var&gt;checksum&lt;/var&gt;, like so:
  
  GRNG[0] = <var>checksum</var> | (<var>checksum</var> << 16)
+
  GRNG[0] = &lt;var&gt;checksum&lt;/var&gt; | (&lt;var&gt;checksum&lt;/var&gt; &lt;&lt; 16)
  
 
The keystream is composed of the lower byte of the high word of successive GRNG values.
 
The keystream is composed of the lower byte of the high word of successive GRNG values.
  
  keystream[n] = (GRNG[n] >> 16) & 0xff
+
  keystream[n] = (GRNG[n] &gt;&gt; 16) &amp; 0xff
  
<code>Xor</code> the keystream with the plaintext to get the ciphertext. <code>Xor</code> the keystream with the ciphertext to get the plaintext.
+
&lt;code&gt;Xor&lt;/code&gt; the keystream with the plaintext to get the ciphertext. &lt;code&gt;Xor&lt;/code&gt; the keystream with the ciphertext to get the plaintext.
  
 
==Conversation==
 
==Conversation==
  
The first request the game makes is to <code>/pokemondpds/worldexchange/info.asp</code>. The server responds with 0x0001.
+
The first request the game makes is to &lt;code&gt;/pokemondpds/worldexchange/info.asp&lt;/code&gt;. The server responds with 0x0001.
  
Platinum, Heart Gold, and Soul Silver will then make a request to <code>/pokemondpds/common/setProfile.asp</code>.  The server responds with eight NULs (0x00000000 0x00000000).
+
Platinum, Heart Gold, and Soul Silver will then make a request to &lt;code&gt;/pokemondpds/common/setProfile.asp&lt;/code&gt;.  The server responds with eight NULs (0x00000000 0x00000000).
  
After the above step(s) or '''performing any of the tasks below''' other than searching, the game makes a request to <code>/pokemondpds/worldexchange/result.asp</code>.  If the game has had a Pokémon sent to it via a trade, the server responds with the entire encrypted [[Pokemon NDS Structure|Pokémon save struct]].  Otherwise, if there is a Pokémon deposited in the GTS, it responds with 0x0004; if not, it responds with 0x0005.
+
After the above step(s) or '''performing any of the tasks below''' other than searching, the game makes a request to &lt;code&gt;/pokemondpds/worldexchange/result.asp&lt;/code&gt;.  If the game has had a Pokémon sent to it via a trade, the server responds with the entire encrypted [[Pokemon NDS Structure|Pokémon save struct]].  Otherwise, if there is a Pokémon deposited in the GTS, it responds with 0x0004; if not, it responds with 0x0005.
  
 
===Receiving a traded Pokémon===
 
===Receiving a traded Pokémon===
  
If the game receives a Pokémon from a successful trade as a response from <code>result.asp</code>, it next requests <code>/pokemondpds/worldexchange/delete.asp</code>.  The server responds with 0x0001.
+
If the game receives a Pokémon from a successful trade as a response from &lt;code&gt;result.asp&lt;/code&gt;, it next requests &lt;code&gt;/pokemondpds/worldexchange/delete.asp&lt;/code&gt;.  The server responds with 0x0001.
  
 
====A note on sendpkm.py====
 
====A note on sendpkm.py====
Line 83: Line 84:
 
===Depositing a Pokémon===
 
===Depositing a Pokémon===
  
Pokémon are offered on the GTS by requesting <code>/pokemondpds/worldexchange/post.asp</code>.  The sent data is 300 bytes long, and includes the Pokémon struct. If the Pokémon is rejected by the server, the response is 0x000c; otherwise, if the deposit is successful, 0x0001.
+
Pokémon are offered on the GTS by requesting &lt;code&gt;/pokemondpds/worldexchange/post.asp&lt;/code&gt;.  The sent data is 300 bytes long, and includes the Pokémon struct. If the Pokémon is rejected by the server, the response is 0x000c; otherwise, if the deposit is successful, 0x0001.
  
The game then saves. After the save is complete, it issues a request to <code>/pokemondpds/worldexchange/post_finish.asp</code>.
+
The game then saves. After the save is complete, it issues a request to &lt;code&gt;/pokemondpds/worldexchange/post_finish.asp&lt;/code&gt;.
  
 
===Retrieving the deposited Pokémon===
 
===Retrieving the deposited Pokémon===
  
Checking on the deposited Pokémon is apparently done by <code>/pokemondpds/worldexchange/get.asp</code>.  The response appears to be a [[Pokemon NDS Structure|Pokémon save struct]].
+
Checking on the deposited Pokémon is apparently done by &lt;code&gt;/pokemondpds/worldexchange/get.asp&lt;/code&gt;.  The response appears to be a [[Pokemon NDS Structure|Pokémon save struct]].
  
Retrieving the deposited Pokémon is done by <code>/pokemondpds/worldexchange/return.asp</code>.  The response is merely 0x0001; the actual Pokémon data is taken from the <code>get.asp</code> request.
+
Retrieving the deposited Pokémon is done by &lt;code&gt;/pokemondpds/worldexchange/return.asp&lt;/code&gt;.  The response is merely 0x0001; the actual Pokémon data is taken from the &lt;code&gt;get.asp&lt;/code&gt; request.
  
 
===Searching===
 
===Searching===
  
Searching is done through <code>/pokemondpds/worldexchange/search.asp</code>.  The sent data is either 15 or 16 bytes long.
+
Searching is done through &lt;code&gt;/pokemondpds/worldexchange/search.asp&lt;/code&gt;.  The sent data is either 15 or 16 bytes long.
  
The server responds with a full 292-byte Pokémon struct for each result.  If there are <var>n</var> results, the response will be 292 * <var>n</var> bytes long.  If there are no results, the server will give an empty response (0 bytes).
+
The server responds with a full 292-byte Pokémon struct for each result.  If there are &lt;var&gt;n&lt;/var&gt; results, the response will be 292 * &lt;var&gt;n&lt;/var&gt; bytes long.  If there are no results, the server will give an empty response (0 bytes).
  
 
==Pokémon struct==
 
==Pokémon struct==
  
The Pokémon data for the GTS is 292 bytes—56 bytes larger than a party Pokémon struct. The extra 56 bytes are GTS-specific data, such as the player's name & country, and what Pokémon they are requesting.
+
The Pokémon data for the GTS is 292 bytes—56 bytes larger than a party Pokémon struct. The extra 56 bytes are GTS-specific data, such as the player's name &amp; country, and what Pokémon they are requesting.
  
 
They are as follows:
 
They are as follows:
  
<pre>
+
&lt;pre&gt;
 
0x00-0x01 Current Pokémon
 
0x00-0x01 Current Pokémon
 
0x02      Gender          1=male; 2=female; 3=either/neither
 
0x02      Gender          1=male; 2=female; 3=either/neither
Line 127: Line 128:
 
0x36      Version        0x0A=Diamond; 0x0B=Pearl; ...
 
0x36      Version        0x0A=Diamond; 0x0B=Pearl; ...
 
0x37      Language        1=jp; 2=en; 3=fr; ...
 
0x37      Language        1=jp; 2=en; 3=fr; ...
</pre>
+
&lt;/pre&gt;
  
 
The timestamps are set by the server, and are always PST (UTC-8). Timestamp format:
 
The timestamps are set by the server, and are always PST (UTC-8). Timestamp format:
<pre>
+
&lt;pre&gt;
 
0x00-0x01 Year
 
0x00-0x01 Year
 
0x02      Month
 
0x02      Month
Line 138: Line 139:
 
0x06      Second
 
0x06      Second
 
0x07      always 0?
 
0x07      always 0?
</pre>
+
&lt;/pre&gt;

Revision as of 00:45, 24 November 2010

Under Construction! Please Visit Reserve Page. Page Will Be Available Shortly

The following is wild conjecture based on LordLandon's sendpkm.py.

Communication with the GTS is done over regular HTTP with http://gamestats2.gs.nintendowifi.net/. The same protocol is used for all five Gen IV games.

HTTP headers

The games don't seem to care about these at all. The GTS sends back a bunch of boilerplate response headers, but the game happily accepts a response with only a Content-Length.

Protocol

All requests to the server are GET requests of the form <code><var>page</var>.asp?pid=<var>pid</var>&amp;hash=<var>hash</var>&amp;data=<var>data</var></code>. However devices prior to the DSi also send requests without the hash (followed by a requests with hash) to which you should also return proper values or else the DS will assume communication broke off.

pid

The pid is an unsigned 32-bit integer that appears to uniquely identify a game cartridge. Although this has not been confirmed entirely: Your PID is generated when you get your friendcode for the first time and is set to: friendcode & 0x7fffffff When you change the device and are forced to change your friendcode with it, your PID doesn't change but you get a new friendcode (at this point there's no friendcode->pid mapping anymore).

For the mathematically inclined: Eevee's Platinum pid is 192615460 (0x0b7b1424) and his Pal Pad code is 0904 2026 4621.

Challenge/response

(4th Gen)

Before each "real" request, the game sends a request of the form <code><var>page</var>.asp?pid=<var>pid</var></code> and the server responds with a 32-byte challenge token. The game computes <code>sha1("sAdeqWo3voLeC5r16DYv" + token)</code> and uses that as the <var>hash</var> value which it sends to the server. The <var>data</var> parameter is encrypted, as explained further down.

(5th gen)

The process is mostly the same, except that the response hash is <code>sha1("HZEdGCzcGGLvguqUEKQN" + token)</code>


That is, each request looks like the following:

  1. Game requests <code>GET /pokemondpds/<var>page</var>.asp?pid=<var>pid</var></code>
  2. Server responds with <var>token</var>
  3. Game requests <code>GET /pokemondpds/<var>page</var>.asp?pid=<var>pid</var>&amp;hash=<var>sha1(...)</var>&amp;data=<var>data</var></code>
  4. Server responds with payload

The exact details of the game's requests are not currently known, but we are working on it!

Checksum

A 32-bit checksum of the data is used as the key for the encryption. The checksum is simply the sum of every byte of the data.

The first 4 bytes of the sent data are the checksum, <code>xor</code>ed with <code>0x4a3b2c1d</code>, and sent in network byte order (big-endian).

Encryption

The data sent to the server is encrypted with a stream cipher, and urlsafe-base64-encoded.

This encryption algorithm, like others used in the game, uses a Linear Congruential Generator (not a very strong choice). The multiplicative constant is <code>0x45</code>, and the additive constant is <code>0x1111</code>. It appears that the game uses a signed dword to store the seed, which doesn't really matter; it shouldn't affect anything. (It means that the modulus is effectively <code>0x80000000</code>.)

GRNG[n+1] = (GRNG[n] * 0x45 + 0x1111) & 0x7fffffff

The GRNG is seeded with the <var>checksum</var>, like so:

GRNG[0] = <var>checksum</var> | (<var>checksum</var> << 16)

The keystream is composed of the lower byte of the high word of successive GRNG values.

keystream[n] = (GRNG[n] >> 16) & 0xff

<code>Xor</code> the keystream with the plaintext to get the ciphertext. <code>Xor</code> the keystream with the ciphertext to get the plaintext.

Conversation

The first request the game makes is to <code>/pokemondpds/worldexchange/info.asp</code>. The server responds with 0x0001.

Platinum, Heart Gold, and Soul Silver will then make a request to <code>/pokemondpds/common/setProfile.asp</code>. The server responds with eight NULs (0x00000000 0x00000000).

After the above step(s) or performing any of the tasks below other than searching, the game makes a request to <code>/pokemondpds/worldexchange/result.asp</code>. If the game has had a Pokémon sent to it via a trade, the server responds with the entire encrypted Pokémon save struct. Otherwise, if there is a Pokémon deposited in the GTS, it responds with 0x0004; if not, it responds with 0x0005.

Receiving a traded Pokémon

If the game receives a Pokémon from a successful trade as a response from <code>result.asp</code>, it next requests <code>/pokemondpds/worldexchange/delete.asp</code>. The server responds with 0x0001.

A note on sendpkm.py

After doing the above, some Platinum, Heart Gold, and Soul Silver games will report a communication error and dump the player back to the title screen. The Pokémon is still successfully received. At least one person with HG/SS has received a Pokémon from a fake server without getting the error, and Diamond/Pearl have never been reported to have the problem.

Depositing a Pokémon

Pokémon are offered on the GTS by requesting <code>/pokemondpds/worldexchange/post.asp</code>. The sent data is 300 bytes long, and includes the Pokémon struct. If the Pokémon is rejected by the server, the response is 0x000c; otherwise, if the deposit is successful, 0x0001.

The game then saves. After the save is complete, it issues a request to <code>/pokemondpds/worldexchange/post_finish.asp</code>.

Retrieving the deposited Pokémon

Checking on the deposited Pokémon is apparently done by <code>/pokemondpds/worldexchange/get.asp</code>. The response appears to be a Pokémon save struct.

Retrieving the deposited Pokémon is done by <code>/pokemondpds/worldexchange/return.asp</code>. The response is merely 0x0001; the actual Pokémon data is taken from the <code>get.asp</code> request.

Searching

Searching is done through <code>/pokemondpds/worldexchange/search.asp</code>. The sent data is either 15 or 16 bytes long.

The server responds with a full 292-byte Pokémon struct for each result. If there are <var>n</var> results, the response will be 292 * <var>n</var> bytes long. If there are no results, the server will give an empty response (0 bytes).

Pokémon struct

The Pokémon data for the GTS is 292 bytes—56 bytes larger than a party Pokémon struct. The extra 56 bytes are GTS-specific data, such as the player's name & country, and what Pokémon they are requesting.

They are as follows:

<pre> 0x00-0x01 Current Pokémon 0x02 Gender 1=male; 2=female; 3=either/neither 0x03 Level 0x04-0x05 Requested Pokémon 0x06 Gender 0x07 Min level 0x08 Max level 0x09 always 0? 0x0A Trainer Gender 0=male; 1=female 0x0B always 0? 0x0C-0x13 Timestamp - Deposited time 0x14-0x1B Timestamp - Time traded away? 0x1c-0x1F pid - also 0x4c in the sav 0x20-0x2F Trainer Name 0x30-0x31 Trainer ID 0x32 Country 0x33 City 0x34 Trainer Sprite 0x35 Exchanged flag 0x36 Version 0x0A=Diamond; 0x0B=Pearl; ... 0x37 Language 1=jp; 2=en; 3=fr; ... </pre>

The timestamps are set by the server, and are always PST (UTC-8). Timestamp format: <pre> 0x00-0x01 Year 0x02 Month 0x03 Day 0x04 Hour 0x05 Minute 0x06 Second 0x07 always 0? </pre>