AngelSL

It's good to get some form of confirmation.

I've gotten my cards, but have yet to collect them cause I'm still out ??

It's the same event; here's a screenshot of WC6 editor after I collected it:

The card: (Code is used)

I wonder if we might've seen each other at Waterway Point, heh.

Exactly. =D

nicholas on IRC has said he might give the first method a shot. I'd like to try myself regardless, and I've been putting off solidly learning a programming language for too long. Not that writing something stupid like this would make it "solid" but it's a step in the right direction. Hopefully somebody can step in to help with the RNG because even if I were to start now I doubt I could figure that out in the foreseeable future. =/

I'm sure magical or someone else can figure out the RNG.

If not we can find someone who knows ARM assembly.. if not I'll see what I can do.

Method 1 should be easy: Packetlog the DS, then recreate that packetlog except screwing with the selected cipher. We'll need to see if Apache (I'm assuming the SSL part is HTTPS) works with it; if not we can try to write our own.

As for the first one, I thought it was a given that we have to redirect traffic to the local machine (or maybe a remote one eventually) for ANY solution. It's not hard to do. I do have my doubts that the DS won't crash or something when it is told by the server to use no encryption though.

And for the second one, I don't think we do. Correct me if I'm wrong but we should be able to send the certificate without any modifications. Because we already know what the decrypted pre-master secret is (via RNG prediction) there is no need for the server's private key to figure it out. After that the entire connection is ours because everything stems from the pre-master secret.

After doing more reading, I see what you mean - we figure out the premaster secret, and then just give the cert Nintendo uses.

Alright, that's what I thought.

Trying to bruteforce a key seems to be a fools errand. What we should go after isn't SSL itself but GAMEFREAK's implementation of it. I have two ideas:

This first one is more just wishful thinking than anything, because it shouldn't work, but since its a DS game we're talking about and not a web browser I'll allow myself the glimmer of hope. The DS tells the server that it supports the cipher suites RSA_WITH_RC4_128_MD5 and RSA_WITH_RC4_128_SHA, and the genuine Nintendo server selects MD5. I wonder what would happen if a fake server sent back the default/non-encrypted cipher suite NULL_WITH_NULL_NULL as its selection. Any SSL implementation worth using would terminate the connection at this point, but there's not any guarantee that one of the pokemon games would. On the incredibly low chance that that works, we could send the DS an unmodified version of the real server's certificate and it would start sending data assuming that it's secure from everyone but the real server. Of course it wouldn't be though, and we could send replies back without worrying about keys.

The second one I believe is a very real possibility, but not without someone who can do some dissembler work to lay the foundation (not me lol). We know that the pokemon games use RNGs that aren't really all that random. In fact they're so predictable people abuse them all the time to get the PIDs and IVs they want. A "secure" RNG is critical to the effective use of SSL because the pre-master secret is nothing but a random number encrypted with the server's public key. We need that number decrypted on the server side, which should only be possible with the server's private key, but since we can probably predict what the number is via a program like RNG Reporter (but set up to figure out this new RNG of course, assuming it is even a different RNG) we have the rest of the connection at our fingertips.

Thoughts anyone?

The first way will never work (unless you use the hosts file to spoof the domain AND manage to get the DS to transmit unencrypted).

The second way is more plausible but we still need to get past the problem of certificate.

Are you sure the public key is hardcoded into the ROM? That would prevent them from ever changing the server's certificate right?

EDIT: You can probably disregard that. I'm still trying to get educated on SSL and I assumed you were talking about the server's public key. Now I see that the certificate is signed and you must've meant the signer's (Nintendo CA's) public key. So what the DS should/probably does check is that the certificate is signed by Ninty? Just wanna make sure I've got that right.

My guess is that they do this: (as quoted from wikipedia)

# The client may use the certificate authority's (CA's) public key to validate the CA's digital signature of the server certificate. If the digital signature can be verified, the client accepts the server certificate as a valid certificate issued by a trusted CA.

# The client verifies that the issuing CA is on its list of trusted CAs.

Nintendo or GameFreak would thus be the 'trusted CA'.

They're not as similar as you'd think. The games use SSL to verify that they are connecting with the real Nintendo servers for mystery gift downloads (as well as most other online functions). For whatever reason, they don't use SSL with the GTS, and faking the server is as easy as redirecting traffic and running the right software.

In order to make a fake mystery gift server we would have to get a certificate that can pass whatever checks the games perform before finalizing the connection. Certificates can be made easily enough, but I'm not sure if it's possible to make one that can actually pass the tests.

That would be pretty hard. We'd need to get a private key that matches the public key the DS has (either that or we figure out how to change the certificate in the ROM, but that would be pointless, since if you can load a hacked ROM, ...). (and no it's not possible to get a private key from a public key). Best chance we have is brute forcing the key.

http://en.wikipedia.org/wiki/Transport_Layer_Security#Security. The DS<->GTS would fall under the first few applications there.

Oh, I was looking at a different post. That's likely how it would work. Either mimicking PBR or Ranch.

I think they communicate using the same protocol.

Yeah he was but maybe you could make the homebrew so it lets you check other hidden stuff that you can't see in-game, such as Secret ID for the actual game, not a ROM on a flashcart, you can use Pokesav for that.

But if you have a flashcart then why not just dump the SAV file? And if it's on HGSS then you could trade a Pokemon over using a friend's DS and check it out.

Okay, I need someone with really good DNS experience to PM me. I apparently need to make a name server on a Linux (Ubuntu) server, and have it so that Port 80 and Port 53 are open. Or just enable Port 53 on my current server setup.

If you have shared hosting, port 53 would be used as they need to host their own DNS server for everyone on that server there.

Unless you have a dedicated IP, that's a different story.

Capitalise the last 'l' in my username.

(i.e 'AngelSL'). Thanks.

Thx for your replies, all suggestions how to make HyperGTS better are welcome!

The IP has to be the one of the computer on which HyperGTS is running.

I have made the window fixed size because I havn't configured everything to expand like it should and it wasn't my highest priority to implement it. I hope this is ok for you

The GTS isn't stoppable because I havn't managed to stop the backgroundworker when it is waiting for a connection... I hope I can fix this soon, until then you will have to restart the whole app, sorry.

Hi, fellow C# developer.

It'd be easier if I could talk to you on MSN or IRC (something live) so you can respond instantly.. but..

Here, try this as a solution to 'havn't managed to stop the backgroundworker when it is waiting for a connection'.

private void BGW_GTS_DoWork(object sender, DoWorkEventArgs e)
       {
           BGW_GTS.ReportProgress(1, "GTS started...");
           Socket serv = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
           serv.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, 1);
           try
           {
               serv.Bind(new IPEndPoint(IPAddress.Any, 80));
           }
           catch (System.Net.Sockets.SocketException)
           {
               BGW_GTS.ReportProgress(1, "Server could not be started (Port 80 already used)");
               return;
           }

           serv.Listen(50);
           List<Socket> clients = new List<Socket>();

           SocketAsyncEventArgs saea = new SocketAsyncEventArgs();
           bool readyToContinueAccept = true;
           EventHandler<SocketAsyncEventArgs> clientAccepted = (sendr, eventargs) => // True Async request handling.
                                                                   {
                                                                       readyToContinueAccept = true; // AcceptAsync finished.
                                                                       if (eventargs.SocketError != SocketError.Success)
                                                                           return; // did the AcceptAsync succeed? if not, quit.
                                                                       Socket client = eventargs.AcceptSocket; // get the new Socket (duh)

                                                                       ThreadPool.QueueUserWorkItem(si =>
                                                                                                        {
                                                                                                            try
                                                                                                            {
                                                                                                                clients.Add(client);
                                                                                                                asyncReq(client);

                                                                                                            } catch
                                                                                                            {

                                                                                                            } finally
                                                                                                            {
                                                                                                                if (clients.Contains(client))
                                                                                                                    clients.Remove(client);
                                                                                                                client.Close();
                                                                                                            }
                                                                                                        });

                                                                   };
           saea.Completed += clientAccepted;
           Action continueAccept = () =>
                                       {
                                           saea.AcceptSocket = null;
                                           if (!serv.AcceptAsync(saea))
                                           {   // AcceptAsync completed synchronously, call ClientAccepted
                                               clientAccepted(null, saea);
                                           }
                                       };
           while(true)
           {
               if(!BGW_GTS.CancellationPending)
               {

                   if(readyToContinueAccept) // has the previous AcceptAsync finished?
                   {
                       // if yes, launch another
                       readyToContinueAccept = false;
                       continueAccept();
                   } // if not, wait 30 ms and check again

               } else
               {
                   // cancellation pending
                   break; // let's quit
               }

               Thread.Sleep(30);
           }
           foreach(Socket client in clients)
           {
               if(client.Connected) client.Close();
           }
           serv.Close(); // this will call clientAccepted with saea.SocketError != SocketError.Success. It will do nothing.

           /*while (!BGW_GTS.CancellationPending)
           {
               SocketAsyncEventArgs saea = new SocketAsyncEventArgs();
               saea.
               serv.AcceptAsync(new SocketAsyncEventArgs {})
               Socket client = serv.Accept();

               try
               {

                   clients.Add(client);
                   asyncReq(client);
               }
               catch
               {

               } finally
               {
                   if(clients.Contains(client)) clients.Remove(client);
                   client.Close(); // Because asyncReq just sends the response, and then returns, so might as well close the socket?
               }
           }*/


       }

Following the previous 2 generations (III and IV), where I and II's first paired games (respectively) were remade, what is the possibility of Ruby/Sapphire being remade for Gen V?

This may be the first generation where Kanto does not appear in the main series for that generation, should Black/White/<improved version> not include Kanto.

It is also possible that the remakes of Gen I are remade again. If they are, then B/W/<improved version> would not need to include Kanto. However, I think this is extremely unlikely, either Kanto will appear in B/W/<improved version>, or Kanto will not appear at all.

GF has already broken the pattern by releasing Gen V for the DS. It's also possible they break the pattern of Kanto appearing at least once in every generation.

Comments?

Very nice, Vlad. Also, my host is crap. They don't respond to any of my emails, and they keep their sites on one IP. Also, it won't let me configure an A or CNAME on my cpanel. Even if I ask for a new IP or CNAME to be activated, they won't respond to emails.

I'm thinking about switching to Linode, unless anyone else has any other better suggestions on VPS servers. =/

I don't think BIND (at least I think that's what your host is using) allows you to specify CNAMEs for domains not in your domain's zone.

I seem to be having a problem with my dnsserver.py. It gives me an error (which I had to screenshot very quickly, as it disappeared upon opening), and shows that this is incorrect:

      packet+="\x03gts\x0Challoforigin\x03com\x00"str.join("",map(lambda x: chr(int(x)), "gts.halloforigin.com".split(".")))

Any help on this one?

str.join was supposed to be commented out.

Comment it out by appending a #

Alright, so if I upload the files to my site's front page, it should have a better chance at working, right?

No, problem is the DS doesn't support CNAMEs probably. Did you do the fix I told you to try?

If this doesn't work, putting it at your frontpage won't either as you don't have a dedicated IP.. (shared IP using virtual hosts)

@Vlad I modified his script to return a CNAME instead of an IP. Theoretically it should work.

What we can do is distribute key items via Pokemon (as "held" items I suppose), hehe. Have not tried but perhaps it could work, if not meh then isn't a custom GTS server enough? D:

Won't work, you also need the event flag set, not just the Key Item. (which is why there is a special wonder card type for each of the event items in DPPHGSS, and not just a 'Item')

I tried doing this on my site, and uploaded all the files I *think* I was supposed to upload, but when I opened dnsserver.py from my computer, it got to here:

And then disconnected me from Wi-Fi. Any help on this issue?

Also, here are the files I uploaded to the server:

In case it helps.

EDIT: I didn't upload cgi-bin, that was there when I made the subdomain. Also, indecks.php is the old index.php, which serves virtually no purpose besides being a backup.

I think I didn't do the CNAME properly or something.

Or the DS just doesn't support CNAMEs.

Try this: open dnsserver.py in WORDPAD.

Find these lines: (line 57 & 58)

     packet+="\x00\x05\x00\x01\x00\x00\x00\x3c\x00\x16"
     packet+="\x03gts\x0Challoforigin\x03com\x00"

Replace with

     packet+="\x00\x05\x00\x01\x00\x00\x00\x3c\x00\x18"
     packet+="\x03gts\x0Challoforigin\x03com\x01.\x00"

Do NOT mess up the spaces. There should be 6 spaces behind the lines. Or else python will complain.

I'm having this same problem. All ports are open, my firewall on the router is completely off, Windows Firewall is completely off, and he has no hacked Pokemon with him. Any help?

(btw, I solved my other problem stated in reply #155)

Nevermind, my ISP sucks. It's blocking 53 and 80. Thanks anyway. [noparse][/noparse]

Vlad posted a PHP script you could use.

http://projectpokemon.org/forums/attachment.php?attachmentid=3930&d=1269817603

Your ISP isn't blocking 53 by the way, your forwarding has some problems.

You could use that script on your webhost (on a subdomain), and have the simple DNS CNAME to that subdomain. Then have your friend set the DNS to you, and he'll get redirected to that script.

I'm having this same problem. All ports are open, my firewall on the router is completely off, Windows Firewall is completely off, and he has no hacked Pokemon with him. Any help?

(btw, I solved my other problem stated in reply #155)

Firewall doesn't matter, you need to portforward. (unless you DMZ'd your computer)

Portforward TCP&UDP 53 and TCP&UDP 80 to your LAN IP (I suggest setting your lan IP to static (in the DHCP reservation table))

That should fix it

Well thing is that i dont get an error message

I try and connect, but then it just says it cant conenct and that im gonna get send to the counter, but theres not a message with an error #

I cant disable the router firewall for th whole router or the one PC.

Windows firewall is turned off.

heres a screenie with the sendpkm.exe log

That isn't much help. Check if your IP is correct, try using your LAN IP, etc..

-end reply to quote-

I might try to convert this into a PHP script ASP.NET and then PHP (so that you can use it on your webhost). You'll need to figure out how to redirect DNS, though.

Python script won't work there.

Just a quick question, but how'd you set this up?

I'm having issues trying to figure out how to set it up so that I can publish my DNS to allow people outside my network to connect. I was under the original assumption that your script is made so it can go up on a webhost, but when I put it up on mine it doesn't do anything (probably due to shared hosting), or maybe I'm missing on how to do this altogether, working on like 6 hours of sleep from 2 days ago.

Forgot to mention, I'm really only interested in serving up pokemon, preferably many at a time to different people (if possible) and being able to just have the script run at different times of day (easy enough to setup).

I tried no-ip (what you're using), but was still a bit lost despite having used it before.

You guys, why not install a full-fledged DNS server and redirect your DS there?

Have the DNS server lookup the real DNS server (your ISP provides). So you can just replace <insert GTS domain here> while still being able to trade (normal, not GTS) without changing wifi settings every time?

And for datoneguy, use the simple DNS script, and portforward TCP&UDP 53. Direct people to set their DNS server to your WAN ip.

I got that map_matrix stuff done.

The extractor is here (exports to MediaWikicode.). Yes, the coding is horrendous, I didn't put much effort to it, it works.

/* By AngelSL, remove credits, whatever, as long as it complies with the below */
/* This program is free software: you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
   the Free Software Foundation, either version 3 of the License, or
   (at your option) any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program.  If not, see <http://www.gnu.org/licenses/>.
*/
/* DISCLAIMER: THIS FILE IS FOR EDUCATIONAL PURPOSES ONLY, blah blah blah  */
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;

namespace MapMatrix2Table
{
   class Program
   {
       static void Main(string[] args)
       {
           if(args != null && args[0] == "/all")
           {
               args = Directory.GetFiles(Environment.CurrentDirectory, "map_matrix_???");
           }
           foreach (string s in args)
           {
               if(!s.EndsWith(".bin") && !s.EndsWith(".hex") && s.Contains(".")) {Console.WriteLine("Skipping " + s);continue;}
               FileStream fs = File.OpenRead(s);
               BinaryReader br = new BinaryReader(fs);
               byte columns = br.ReadByte();
               byte rows = br.ReadByte();

               int entries = rows*columns;
               ushort[,] matrix = new ushort[columns,rows];
               br.ReadInt16();
               string prefx = new string(br.ReadChars(br.ReadByte()));
               int row = 0;
               int column = 0;
               for(int x = 0; x < entries; x++)
               {
                   if(column == columns)
                   {
                       column = 0;
                       ++row;
                   }

                   matrix[column,row] = br.ReadUInt16();
                   ++column;
               }
              File.WriteAllLines(s + ".txt", ProcessAndDestroyMatrix(matrix, columns, rows, prefx));

           }


       }

       static string[] ProcessAndDestroyMatrix(ushort[,] matrix, byte columns, byte rows, string prfx)
       {
           List<String> ret = new List<string>();
           ret.Add("{| class=\"wikitable\" style=\"text-align:center; width:auto; height:auto;\" border=\"1\" ");

           // Table headers
           {
               StringBuilder headers = new StringBuilder();
               headers.Append("! x");
               for (byte x = 0; x < columns; x++)
               {
                   headers.Append(" !! ");
                   headers.Append(x);
               }
               ret.Add(headers.ToString());
           }


           for(int row = 0; row < rows; row++)
           {

               ret.Add("|-");

               ret.Add("! " + row);
               for(int column = 0; column < columns; column++)
               {
                   StringBuilder line = new StringBuilder();

                   line.Append(matrix[column, row] != 0 ? "| style=\"background-color:yellow;\" | " : "| ");
                   line.Append(prfx);

                   line.Append(column.ToString("00"));
                   line.Append("_");
                   line.Append(row.ToString("00"));
                   line.Append("c");
                   line.Append("<br />");
                   line.Append(matrix[column, row]);

                   ret.Add(line.ToString());
               }

           }
           ret.Add("|}");

           return ret.ToArray();

       }
   }
}

If anyone would like to make their own parser, then look in the wiki for the article regarding the format. However! I'm not certain whether the file is supposed to be read by column or row, try both. Or derive it from my source ^

Anyway, we can see that the game uses their own internal name. An example is map03_27c which is Twinleaf town. If you extract land_data_release.narc and use "grep map03_27c *" to grep files containing that, you would see that only 1 file contains that aka the file containing Twinleaf's data.

What we need to do now is to find where the game stores the map ID/intername/etc to real displayed name stringtable. The game could identify this by map ID, intername, or even index of the entry in land_data_release, or something else.

Deciphering data/fielddata/mapmatrix/map_matrix.narc in Diamond/Pearl's contents.

I would appreciate it if anyone that has looked into it before provide their insight.

And if you figured out the structure of the contents, I'd appreciate it if you could post it here.

I do know the structure of a NARC file, I'm looking into the contents of the map_matrix.narc file.

If Alpha reads this, SCV said that you might know more about map_matrix.narc's contents.

Shoutout.

Well, doesn't seem anyone has made much progress..

I has a quick look into this today, using a slightly different method. I was still using Wireshark for the packet-logging, but I was using a APR spoofer to intercept communications between the DS and router.

With this in place I went into the Global Trade Station in Jubilife City, connected to the GTS, deposited a Pokemon for trade and then searched for a couple of other Pokemon.

This fired off a load of connections to various servers owned by Akamai Technologies (a company that, amongst other things, provides network services for MMO games and such).

I haven't done any analysis on this yet as I'm having trouble getting Wireshark to give me any reasonable data beyond the packet headers?!

But, I didn't notice any UDP data flying around, which is different to AngelSI's findings. AngelSI: did we follow roughly the same procedure or were you trying to trade using the normal wireless communications (i.e. a non-GTS trade)?

If not, are you treating lower-level protocols such as ARP and DHCP as UDP. Anything relating to ARP, DHCP or ICMP can be disregarded - it's all standard connection and address negotiation stuff.

Andy

I'm not too sure, this was a long time ago. But yes I still have the equipment to sniff the data going through my PC (my PC acts as a wireless access point for my DS), so if you still need it, I can do it.

I'll check the pcaps.

EDIT: Yes, there were UDP stuff with Wireshark's description "Source port: xxx Destination port: xxx", and no I was NOT using the GTS

And down again. :

I'm sensing a pattern here

1st Gen RBY

2nd Gen GSC

3rd Gen remake of RBY, RSE

4th Gen remake of GSC, DPP

5th Gen potential remake of FRLG/RSE, and 5th gen stuff

Thus, you can say that 3rd gen and onwards will have remakes of (gen no. - 2)th gen.

If my speculation is correct, in a few years Nintendo will release their newest gaming device, then announce 5th generation, then 2-3 years after that, remakes of 3rd Gen.

Your comments?