threads

TCP C++ server upgrades

A quick snippet from tonight’s diary entry:

20:30 – I’m still reading up on some stuff about non-blocking TCP server programming and so far the best examples come from http://developerweb.net/viewtopic.php?id=2933. Ideally what I want is NOT to have a shed load of threads running, which I’ll be using a thread manager for, but a way of quickly determining if a connection is valid before shutting it down.

By the looks of the examples,  a continuous loop could be checking upto 1024 connections at any time before sending them off to another thread. This method has caught my attention big time because I should be able to setup some kind of timeout very quickly to close the connection if it doesn’t respond to the initial connection request. And I can still serve 100 to 1000 devices without any problems at all.

The same single thread can be constantly checking for connections and readability. It’s the readability part that matters the most here because it will be timed very precisely. Once a connection passes the readability test by passing the initial CSPRNG test, then it can go to a thread handler. Otherwise it gets closed. All of this a single core/thread can easily handle.

I will be studying this a lot further as this looks to be the way forward for a tcp server.

And this is just from my crappy laptop at home.

Basically, the new software upgrades to the server will be 1000 times more efficient than the current Java version. Although once in the cloud, the Java version can easily serve over 500 devices because of the bandwidth increase.

The updated C++ version will increase that significantly as well as handle dodgy connections better. Once a connection has passed the CSPRNG test then the TIMEOUT increases a smidgen.

I’ve already got the CSPRNG functions running and tested and written in assembler code to reduce that bottleneck too.

Until next time… WLGfx…

Java vs C++ for 3D mesh building

A few days ago, playing around with LibGDX again I was getting back into 3D programming and created a very simple demo of a load of spheres and the camera rotating around the scene as below:

Which on my laptop ran just fine. It took about a second to generate all the meshes for each of the spheres.

Took bloody ages on my phone though, even though it’s a quad core 64 bit android phone. Approximately 5 to 8 seconds. Ouch.

None of this nonsense in C++ for starters. I know this from a very old project I was working back in the day. Procedural map generator. More about that here: PROCMAP DBP LINK

So for the last couple of days (actually on the evenings because of work), I’ve looked into creating a single sphere mesh and then duplicating it. In LibGdx

A minor problem to start off with is that a lot of LibGdx’s mesh generators have been deprecated and I don’t know how long for now, and I can’t find examples on how to use the new stuff. This would have made things easier, but that minor problem got bigger.

It seems that ModelBuilder only runs on the GL thread, so I tried the MeshBuilder. No luck there after trying to hunt info down on google, (Maybe google is hiding it from me :P) how to use the thing.

So I gave up, closed the project, might go back to it if the next step fails.

I’m waiting on a huge update from Qt. Oh! At this time it has finished downloading and updated.

The previous versions of Qt didn’t work with the latest Android SDK/NDK. Fingers crossed with this one…

Next…

Cryptography Secure TCP Sockets in Java

Well, I finally got round to testing out a basic encryption over TCP sockets.

First off, the main() entry looks very familiar and is easy to understand. I simply create the listening server (doesn’t spawn thread for simplicity), with the ability to safely shut it down.

Then I use the DataInputStream and DataOutputStream to send data back and forth. In this test case I sent a string which both the client and server will print to the console.

Once that was working, I extended the OutputStream and InputStream and override the read() and write() methods (shown in the code examples below). Notice that I manipulate the byte in two different ways, one by adding and subtracting and the other by XOR’ing (commented out but works). If these manipulations match up whilst data is being sent back and forth then there’s no errors.

Main class

The main class and entry point to the test which spawns the server and then runs a couple of tests:

package com.wlgfx.cryptotest;

public class CryptoServer {

    public static void main(String[] args) {
        WLCryptoServer server_run = new WLCryptoServer();
        Thread server = new Thread(server_run);
        server.start();
        
        System.out.println("Server thread started");
        
        new Thread(new WLCryptoClient("Hello world!")).start();
        
        try { Thread.sleep(500); } catch (InterruptedException ex) {}
        
        new Thread(new WLCryptoClient("Jump around and stomp your feet!")).start();
        
        try { Thread.sleep(500); } catch (InterruptedException ex) {}
        
        server_run.quitServer();
    }
}

Server class

The server code which take the input and output streams and creates new subclasses of the encryption layer classes and then creates the usual DataInputStream and DataOutputStream for the rest of the normal IO. This matched in the following code for the client.

package com.wlgfx.cryptotest;

import com.wlgfx.cryptotest.WLC.WLInputStream;
import com.wlgfx.cryptotest.WLC.WLOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;

public class WLCryptoServer implements Runnable {
    
    private final int port = 1999;
    private boolean quit = false;

    @Override
    public void run() {
        try {
            ServerSocket socket =  new ServerSocket(port);
            socket.setSoTimeout(1000);
            
            while (!quit) {
                Socket client = socket.accept();
                
                WLOutputStream wlout = new WLOutputStream(client.getOutputStream());
                DataOutputStream out = new DataOutputStream(wlout);
            
                WLInputStream wlin = new WLInputStream(client.getInputStream());
                DataInputStream in = new DataInputStream(wlin);
                
                String text = in.readUTF();
                System.out.println("SERVER: " + text);
                out.writeUTF(text);
                
                client.close();
            }
        } catch (IOException ex) {
            System.out.println("SERVER: " + ex.getLocalizedMessage());
        }
    }
    
    public void quitServer() {
        quit = true;
    }
}

Client class

The client code is just the same as the server class code.

package com.wlgfx.cryptotest;

import com.wlgfx.cryptotest.WLC.WLInputStream;
import com.wlgfx.cryptotest.WLC.WLOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.Socket;

public class WLCryptoClient implements Runnable {
    
    private String text;
    private Socket socket;
    
    private final int port = 1999;
    
    public WLCryptoClient(String test) {
        text = test;
    }
    
    @Override
    public void run() {
        try {
            socket = new Socket("localhost", port);
            
            WLOutputStream wlout = new WLOutputStream(socket.getOutputStream());
            DataOutputStream out = new DataOutputStream(wlout);
            
            WLInputStream wlin = new WLInputStream(socket.getInputStream());
            DataInputStream in = new DataInputStream(wlin);
            
            out.writeUTF(text);
            text = in.readUTF();
            System.out.println("CLIENT: " + text);
            
            socket.close();
        } catch (IOException ex) {
            System.out.println("CLIENT: " + ex.getLocalizedMessage());
        }
    }
    
}

The encryption class

This is where you can see exactly how easy it is to manipulate the data being sent back and forth.  Here I am showing you two methods, XOR and add and subtract. It’s very simple, but for heavier encryption you could look further at CSPRNG, or Crypto Secure Pseudo Random Number Generator. PRNG’s are ten a penny, from simple ones to hefty things. Take your pick. The Crypto and Secure layers are the ones you really need for truly undecipherable data communication.

package com.wlgfx.cryptotest;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

public class WLC {

    // NB: short inner classes are static to avoid errors
    
    public static class WLInputStream extends InputStream {

        private InputStream in;

        public WLInputStream(InputStream input) {
            in = input;
        }

        @Override
        public int read() throws IOException {
            byte a = (byte) in.read();
            a--;
            return (int) a;

            //return in.read() ^ 255; // WORKS TOO
        }
    }

    public static class WLOutputStream extends OutputStream {

        private OutputStream out;

        public WLOutputStream(OutputStream output) {
            out = output;
        }

        @Override
        public void write(int b) throws IOException {
            byte a = (byte) b;
            a++;
            out.write((int) a);

            //out.write(b ^ 255); // WORKS TOO
        }
    }
}
Console output:
run:
Server thread started
SERVER: Hello world!
CLIENT: Hello world!
SERVER: Jump around and stomp your feet!
CLIENT: Jump around and stomp your feet!
SERVER: Accept timed out
BUILD SUCCESSFUL (total time: 1 second)

Until next time

😛

Tonight I’ve sussed out the thread pooling in Java. I wanted a way of being able to queue up threads to be run, but I also wanted to allow the execution of so many threads at a time in this pool. I’ve heard about these thread pools but as of yet, not used them. I need something like this for my own project for both the server and the client.

package threadpooltest;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;

public class ThreadPoolTest {

    public static void main(String[] args) {
        ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
        
        for (int i = 0; i <= 8; i++)
        {
            WorkerTest task = new WorkerTest("Task " + i);
            System.out.println("A new task has been added : " + task.getName());
            executor.execute(task);
        }
        
        System.out.println("Main thread sleeping");
        try { Thread.sleep(8000); } 
        catch (InterruptedException ex) {}
        System.out.println("Finished sleeping");
        
        for (int i = 0; i <= 10; i++)
        {
            WorkerTest task = new WorkerTest("Task " + (i + 5));
            System.out.println("A new task has been added : " + task.getName());
            executor.execute(task);
        }
        
        // THIS DOESN'T WORK
        //System.out.println("WAITING FOR TASKS TO FINISH");
        //try { executor.awaitTermination(1, TimeUnit.DAYS); } 
        //catch (InterruptedException ex) {}
        //System.out.println("FINISHED");
        
        executor.shutdown();
    }
    
}

No proper code snippet plugin yet! Grrr!

Anyway, the above code creates the thread pool which will run a max of 5 threads and queue the rest up. Just what I wanted.

package threadpooltest;

import java.util.concurrent.TimeUnit;

public class WorkerTest implements Runnable {

    private String name;
    
    public WorkerTest(String name) {
        this.name = name;
    }
    
    public String getName() {
        return name;
    }
    
    @Override
    public void run() {
        try
        {
            Long duration = (long) (Math.random() * 5);
            System.out.println("Start Task: " + name);
            TimeUnit.SECONDS.sleep(duration);
            System.out.println("Finished Task: " + name);
        }
        catch (InterruptedException e) {}
    }
}

The above is the example thread I was testing this with.

All I need now is to do the same with C++, but I’ve already found a header only project on github that does just that.

More to come…

EDIT:

I’ve also noted that the above works on Android too.

At the moment, I’m not holding my breath to see it this works with GWT.