Post additions (pinned replies):
[Apr 24th, 2009] AngelSl (pcap log of GTS communication): post#12
[Mar 1st, 2010] LordLandon (logged GTS communication in more close details): post#39
[Mar 2st, 2010] LordLandon (information on actually trading with a fake GTS server): post#41
[Mar 2st, 2010] LordLandon (python script to act as GTS server): post#43
[Mar 22nd, 2010] LordLandon (python script, IP fix): post#100
[Mar 23rd, 2010] magical (clarification how data prefix is encoded on website): post#104
[Mar 23rd, 2010] イーブイ (started Wiki article, check next page after clicking this link for more test results): post#105
[Apr 8th, 2010] AngelSl (dnsserver.py modification hint -to support CNAMEs and not only IPs): post#193
[Apr 10th, 2010] Vlad (port check tool in AutoIT3): post#224
[Apr 11th, 2010] Vlad (dns and http server package): post#129
Note about LordLandon's Python application:
You get "An error occurred while attempting to communicate (00000) Please turn off the power, restart this DS system, and try again.
If this message keeps appearing, please contact Nintendo Consumer Service." when the .pkm file is a storage file. It must be a party file in order to work properly (issue is that storage files are 136bytes while party once are 236bytes).
Original post text:
Odd that no one is curious, but I am so here I go!
Basically the game talks to Nintendo and the global trade center, but I wonder if we can get some of that data from external tools. I am pretty sure the official GTC flash site also reads from here, anyway I found out the URL addresses by hex searching in the rom file (platinum), here are the results:Obviously one is the battletower and other is the trade. Now it seems to operate on GET and POST, since I found a lot of HTTP headers. They also set a User-Agent that I think is called "GameSpyHTTP" (but might be a variable name or refrence, as I also saw "User-Agent: HTTP_X_GAMECD" a line or two above.Code:http://gamestats2.gs.nintendowifi.net/pokemondpds/common/setProfile.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/post.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/post_finish.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/get.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/result.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/delete.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/return.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/search.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/exchange.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/exchange_finish.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/worldexchange/info.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/battletower/roomnum.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/battletower/download.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/battletower/upload.asp http://gamestats2.gs.nintendowifi.net/pokemondpds/battletower/info.asp
Anyway it is possible to get information of trades and pokemons available using external tools, thus you may create a "alert me when X is available" tool if you like, or filter out those "I want a Mew for a Mew" people that are just annoying.
Any thoughts or information you have learned?
Yeah you are correct on your information. We have done research into this already and currently are figuring out the encryption. However it seems very difficult without the ability to debug the game as it is running.
Good luck on your research though
On and regarding making an external tool. If you use that website too much from a specific IP, especially a static one associated with a website server, I would take the assumption that they may figure it out
I just share what I come across as I see almost none other results on Google of what others have concluded -I just try to contribute.
Another thing is that it seems that it operates on arrays, at least the game. That might mean it might send or receive a array with data at some point.
Another thing is that you can (on each URL) access by sending a GET "pid" with a value of maximum 2147483647 (decimal, maximum 32-bit integer value). It returns a random string that looks like a session id as it changes each time you request the page. It is always 32 chars and contains letters and characters.
Additionally you have a GET "hash" that can be a random string containing numbers and letters. It's lenght is not specified as you may go up and up as much as you like, only you will get "error: token expired" all the time. To call "hash" you need to specify a "pid", and since pid alone returns a random string that looks like a hash.
If you do a GET pid with a random decimal (let's say "1"), you get a session key (let's call it that). Now if you add the GET hash and add the key you got from the earlier call, you get "error: incorrect hash". Now if you do a new request but only change the GET pid value (keeping GET hash the same) you get "error: token expired" so that must mean it expects a valid reply based on the session key you got.
I assume the next step would be allowing you to enter if you "enter the right password" based on the key you get, this is something the game does and this way they keep "us" out from their servers.
Also having user-agent set to "GameSpyHTTP" does not seem to do anything, the results remain the same.
I guess the game generates a pid based on time, perhaps ds id but I would think that the pid would be something it gets by requesting the server too, otherwise you would have to blindly try random numbers between 1-2147483647 until the DS gets a key that is not in use, I mean I guess the most logical way the game communicates with the server is like this:
1. Request a PID from the server, get a PID or a "maxusers" error
2. Use PID to request a session key
3. Use PID and send in decoded session key
At this point, the PID is flagged valid and you are allowed to browse and request more data, so the PID will be registered as "in use" and no one else can use it as it will not be send out by the server to new connecting clients. When there is a timeout, the session data is cleared and the PID is now publicly available for new connecting clients.
If this is how it really works, this means there is a maximum connection of 2 147 483 647 (2 billion 147 million 483 thousand 647 users) witch is plenty imo... :P Anyway, I assume you are also stuck on this part, or there is something I have missed.
I have also tried to use a network sniffer tool to monitor the connection between me using the flash based GTC by forcing it to only show HTTP based requests and connections, but I didn't really find much of useful info, didn't find the right data at all.
I will attempt by looking for connections done to the domains holding the urls posted above and see if there is any results when I access the flash site.
Alternatively, you may harvest GTC data from a mirror, thus utilizing the Pokémon GTS website. It is flash based but accesses data from the server, and it is conveniently in plain text format, hehe.
Example, http://www.pokemon-gts.net/data/pokemon/1.txt would bring up information on Bulbasaur. I have not decoded what is what, but there are 2 lines on top that are headers most likely, and the rest of the data is just information of what is happening.
Headers consist of 10 columns, each value is separated by comma (,).
The body is made of 14 columns but no comma separations this time.
The site also uses
http://www.pokemon-gts.net/data/entrusted.txt - available for trade (trading away)
http://www.pokemon-gts.net/data/requested.txt - requested by others (offers)
http://www.pokemon-gts.net/data/exchanged.txt - exchanged, more like a "log" file but I can't make out if it just yet, perhaps it counts how many that have been traded over I guess
I am looking into the flash files of the site, since the main flash loads sub-flashes it would be possible to get more information where to find trainer information, level, e.g. by disassembling the files.
Trying that, the sub flash files returns only images and sounds. The core files might contain scripts, but they all seem encrypted so that's lame. ;P Anyway I'll see what I figure out by looking on the text files, hopefully something useful.
Hi. I have access to a softAP which my DS uses to connect to Nintendo Wifi (see my other thread in the same section). Nintendo Wifi club uses SSL (the one on the PC bottom floor), so I would'nt be surprised if this uses SSL too (I saw SSL handshakes when I was logging GTS, btw)
If you happen to need my help, just throw me a PM.
Thanks AngelSl, it's nice of you.
Right now I just made a small GTS site, nothing special (works fine in Firefox but not IE6) that let's you basically access the pokemon data on the gts site without having to load all the flash and all, and shows who is trading X for X (same pokemon) and who wants a pokemon that is banned from GTS (mew, celebi, darkraid, e.g.) so it is easier to see the valid results.
Only issue is that the moves they know are ID numbers apparently but the values do not correspond to my database, like 33 for me is tackle while on the file it can be 210 or what ever. Also locations where the pokemon come from seem to get the values from a table of some sort, takes ages to figure out all the valid ids from 1-250 to get the location names lol.
Pretty cool stuff going on here keep up the good work.
By the way.
Copyright © Driven by Boredom
I noticed that the data on the site is basically logs 1 day behind, thus what is available right now does not seem to be there. For that I think I really need to use the other domain address, think it's time to contact AngelSl and see what he got to say on the matter.
Anyway the exchanged.txt is a log of the most traded pokemon. Left side is the national id and the right side is the "rank". requested.txt and entrusted.txt work the same way, only it shows the top wanted and deposited.
The /pokemon/#.txt files contain 2 header lines and 100 entries showing what were traded for what, level, move abilities, e.g. but it is all a log from yesterday.
I haven't had time to log GTS yet (the last time I didn't save the pcap file), but I'll do it today after my homework's done.
I don't think pokemon-gts.net contains up-to-date data. gamestats2.gs.nintendowifi.net on the other hand does. Checked data on the gts site, was 1 day old.
Btw, while checking my router for connection info when I connected to Nintendo with my NDS, I got a interesting IP that leads me to a router login (haha): 18.104.22.168 - not so secure is it? Having it exposed like that.
The information is always a few hours off and I am not so sure how they cycle through there updates like what times.
I've attached a pcap file of my packet logging.
Here's what I did:
> Search for a Bulbasaur: Either gender, Any level
> Search for a Darkrai: Either gender, Any level
And I incorrectly said it was SSL. Sorry.
Thanks AngelSl! I'll take a look into the file and see what I can do.
It looks like it communicates using SSLv3, at least requests a certificate each time you get on the GTS (the certificate has an expiration date in 2015) that the NDS reads and gets it's public key, then using that to establish a connection with the server.
I am trying to make my PHP work with OpenSSL but something bugs -argh. If someone that develop would like to take a peak in the certificate and attempt to open a connection would the GTS -it would be lovely.
The extracted certificate from the logged packet:
SSL (inc. certificates) is not my strong side either. Need time to learn about how SSL operates and if I can use PHP to fetch data. Never looked into it before now so I am kinda blank on experience too.
Anyway it's nice that we do have some progress at least. I'll drop off messages as I make new discoveries on this matter in case someone is interested.