Elsewhere there are things that we all miss, yet it takes just one to notice...

TCP C++ server upgrades

A quick snippet from tonight’s diary entry:

20:30 – I’m still reading up on some stuff about non-blocking TCP server programming and so far the best examples come from http://developerweb.net/viewtopic.php?id=2933. Ideally what I want is NOT to have a shed load of threads running, which I’ll be using a thread manager for, but a way of quickly determining if a connection is valid before shutting it down.

By the looks of the examples,  a continuous loop could be checking upto 1024 connections at any time before sending them off to another thread. This method has caught my attention big time because I should be able to setup some kind of timeout very quickly to close the connection if it doesn’t respond to the initial connection request. And I can still serve 100 to 1000 devices without any problems at all.

The same single thread can be constantly checking for connections and readability. It’s the readability part that matters the most here because it will be timed very precisely. Once a connection passes the readability test by passing the initial CSPRNG test, then it can go to a thread handler. Otherwise it gets closed. All of this a single core/thread can easily handle.

I will be studying this a lot further as this looks to be the way forward for a tcp server.

And this is just from my crappy laptop at home.

Basically, the new software upgrades to the server will be 1000 times more efficient than the current Java version. Although once in the cloud, the Java version can easily serve over 500 devices because of the bandwidth increase.

The updated C++ version will increase that significantly as well as handle dodgy connections better. Once a connection has passed the CSPRNG test then the TIMEOUT increases a smidgen.

I’ve already got the CSPRNG functions running and tested and written in assembler code to reduce that bottleneck too.

Until next time… WLGfx…

Devices unique identifier

Now I could have just got the MAC address and used that to communicate with the server, but I wanted to take it to the extremes. Mainly because I wanted to obscure the communication with the previous methods of encryption so the server can identify which device is making a request.

For this I create a string using the devices host name and all the network interfaces connected to it with their MAC addresses. Although I might change this in the near future, at the moment, this gives a good length string for the server to identify the device.

Below is the code used to create this identity string.

    public String host;
    
    private void testMACAddress() {
        try {
            InetAddress ip = InetAddress.getLocalHost();
            
            host = ip.getHostName();

            Enumeration&ltNetworkInterface&gt networks = NetworkInterface.getNetworkInterfaces();
            while (networks.hasMoreElements()) {
                NetworkInterface network = networks.nextElement();
                byte[] mac = network.getHardwareAddress();

                if (mac != null) {
                    StringBuilder sb = new StringBuilder();
                    
                    for (int i = 0; i &lt mac.length; i++) {
                        sb.append(String.format("%02X%s", mac[i], (i &lt mac.length - 1) ? "-" : ""));
                    }
                    
                    host += " " + network.getDisplayName() + " " +  sb.toString();
                }
            }
        } catch (UnknownHostException | SocketException e) {
            System.out.println(e.getMessage());
        }
    }

As mentioned above, this identity string will then be encrypted and sent to the server. Further communication will commence if the server accepts it.

Cryptography Secure TCP Sockets in Java

Well, I finally got round to testing out a basic encryption over TCP sockets.

First off, the main() entry looks very familiar and is easy to understand. I simply create the listening server (doesn’t spawn thread for simplicity), with the ability to safely shut it down.

Then I use the DataInputStream and DataOutputStream to send data back and forth. In this test case I sent a string which both the client and server will print to the console.

Once that was working, I extended the OutputStream and InputStream and override the read() and write() methods (shown in the code examples below). Notice that I manipulate the byte in two different ways, one by adding and subtracting and the other by XOR’ing (commented out but works). If these manipulations match up whilst data is being sent back and forth then there’s no errors.

Main class

The main class and entry point to the test which spawns the server and then runs a couple of tests:

package com.wlgfx.cryptotest;

public class CryptoServer {

    public static void main(String[] args) {
        WLCryptoServer server_run = new WLCryptoServer();
        Thread server = new Thread(server_run);
        server.start();
        
        System.out.println("Server thread started");
        
        new Thread(new WLCryptoClient("Hello world!")).start();
        
        try { Thread.sleep(500); } catch (InterruptedException ex) {}
        
        new Thread(new WLCryptoClient("Jump around and stomp your feet!")).start();
        
        try { Thread.sleep(500); } catch (InterruptedException ex) {}
        
        server_run.quitServer();
    }
}

Server class

The server code which take the input and output streams and creates new subclasses of the encryption layer classes and then creates the usual DataInputStream and DataOutputStream for the rest of the normal IO. This matched in the following code for the client.

package com.wlgfx.cryptotest;

import com.wlgfx.cryptotest.WLC.WLInputStream;
import com.wlgfx.cryptotest.WLC.WLOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;

public class WLCryptoServer implements Runnable {
    
    private final int port = 1999;
    private boolean quit = false;

    @Override
    public void run() {
        try {
            ServerSocket socket =  new ServerSocket(port);
            socket.setSoTimeout(1000);
            
            while (!quit) {
                Socket client = socket.accept();
                
                WLOutputStream wlout = new WLOutputStream(client.getOutputStream());
                DataOutputStream out = new DataOutputStream(wlout);
            
                WLInputStream wlin = new WLInputStream(client.getInputStream());
                DataInputStream in = new DataInputStream(wlin);
                
                String text = in.readUTF();
                System.out.println("SERVER: " + text);
                out.writeUTF(text);
                
                client.close();
            }
        } catch (IOException ex) {
            System.out.println("SERVER: " + ex.getLocalizedMessage());
        }
    }
    
    public void quitServer() {
        quit = true;
    }
}

Client class

The client code is just the same as the server class code.

package com.wlgfx.cryptotest;

import com.wlgfx.cryptotest.WLC.WLInputStream;
import com.wlgfx.cryptotest.WLC.WLOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.Socket;

public class WLCryptoClient implements Runnable {
    
    private String text;
    private Socket socket;
    
    private final int port = 1999;
    
    public WLCryptoClient(String test) {
        text = test;
    }
    
    @Override
    public void run() {
        try {
            socket = new Socket("localhost", port);
            
            WLOutputStream wlout = new WLOutputStream(socket.getOutputStream());
            DataOutputStream out = new DataOutputStream(wlout);
            
            WLInputStream wlin = new WLInputStream(socket.getInputStream());
            DataInputStream in = new DataInputStream(wlin);
            
            out.writeUTF(text);
            text = in.readUTF();
            System.out.println("CLIENT: " + text);
            
            socket.close();
        } catch (IOException ex) {
            System.out.println("CLIENT: " + ex.getLocalizedMessage());
        }
    }
    
}

The encryption class

This is where you can see exactly how easy it is to manipulate the data being sent back and forth.  Here I am showing you two methods, XOR and add and subtract. It’s very simple, but for heavier encryption you could look further at CSPRNG, or Crypto Secure Pseudo Random Number Generator. PRNG’s are ten a penny, from simple ones to hefty things. Take your pick. The Crypto and Secure layers are the ones you really need for truly undecipherable data communication.

package com.wlgfx.cryptotest;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

public class WLC {

    // NB: short inner classes are static to avoid errors
    
    public static class WLInputStream extends InputStream {

        private InputStream in;

        public WLInputStream(InputStream input) {
            in = input;
        }

        @Override
        public int read() throws IOException {
            byte a = (byte) in.read();
            a--;
            return (int) a;

            //return in.read() ^ 255; // WORKS TOO
        }
    }

    public static class WLOutputStream extends OutputStream {

        private OutputStream out;

        public WLOutputStream(OutputStream output) {
            out = output;
        }

        @Override
        public void write(int b) throws IOException {
            byte a = (byte) b;
            a++;
            out.write((int) a);

            //out.write(b ^ 255); // WORKS TOO
        }
    }
}
Console output:
run:
Server thread started
SERVER: Hello world!
CLIENT: Hello world!
SERVER: Jump around and stomp your feet!
CLIENT: Jump around and stomp your feet!
SERVER: Accept timed out
BUILD SUCCESSFUL (total time: 1 second)

Until next time

😛

Blender progress

I’ve got the mother-in-law down for a few days so programming after work is out because I can’t ‘zone’ with all the conversations going on.

This hasn’t stopped me from working with Blender to increase my skills with it though.

I can now easily import 3D models and animate them. Slamming text and jiggling them around is very easy too. All in all, another good demonstration video produced in a little over an hour. Fast moving 3D animated text and models with a video textured background. I’ve impressed myself this time. No stopping me now.

The server software and the Android app development will continue within the next two days when I can ‘zone’ again. There’s not much left to do on the media management software. Then there’s the device management through an android app. I’m also looking into SSL and other cryptography methods for secure communication all round.

Not been that busy lately this time

I had 2.5 days carried over for holidays so I took a long weekend off beginning half day on Friday. All to go for long rides with the missus on our motorbikes. And she was very happy because she has only been riding for almost a month now.

Anyway, documents have been written up towards this project and a very important document that I created regarding it from about six years ago needs altering. No database plan has been setup yet until the spreadsheets are finalised. Number crunching has been pretty good.

At the moment I have four sample videos created, although I need a lot more, to which a demonstration setup can be made. When all the sample videos have been produced, they will range through all categories of businesses.

My Intel tiny PC turned up and I am not disappointed. I may as well stick to 30 frames per second instead of 60 as the current videos play perfect. I’ve got it to boot up in Linux and auto run my software to play the videos on a loop. No screen tearing and running all over these last four days.

Over the next one to three weeks I should have the server infrastructure in place where I can live test my new system. As mentioned above, the documentation has been more of an important matter over this weekend off when I’ve had the time. The infrastructure will include storage, database solutions, communications, security protocols and a users interface to the server.

Throughout this entire project I’ve kept to a strict diet of royalty free and open source. In the future, the software I have used will get donations.

PS…

The initial test was a single ‘cpp’ program which links to ‘libvlc’ and ‘X11’ which opens a full screen window and attaches each instance being played to the full screen window. Now it gets complex. (not really)…

Notes on encryption and security

An important part of communicating over the internet is having some security in place. For my own project this will have the server setup so that it only accepts connections from known devices. It will also encrypt data before it is sent and decrypted when the server receives it.

Encryption:

There are many ways to encrypt data:

  1. Inverting all bits in the data being sent. This was handy in the very old days of 8-bit cpu’s to hide data in memory to stop people spying on code and data in memory.
  2. Apply a key to the data. A good method of securing data which could be a password protected to unlock on the receiving side. However, it doesn’t take long for key to be found.
  3. Apply a key and add salt. Skewing the data like in method 2 is good, but when adding salt, it makes it almost impossible to decrypt without having access to the code.

I’ll be using a system which adds a key and pours in some salt. Now how this is done is using a key to skew the data, and then by splitting the bits of the data up randomly and adding random bits in between, the bitstream will be larger but impossible to decrypt. 2 things will be needed to decrypt the data, the code itself and the key.

Security:

Most of the time, security means logging in with a username and password. The final project will require the device to first log in which will have a timeout feature on it. This is just like session handling in GWT for the web. The login data will still be sent encrypted.

Fortunately, I’ve worked with encryption since 8-bit cpu’s, so this shouldn’t pose a problem for me at all. I’ll have some generic classes written that can be used on both client and server for de/encryption. Apart from the usual login credentials, the client and server will have some hardcoded (still encrypted) transport keys, which can be updated when software is updated. The server can also handle software updates which includes data security updates. This will mean that software that isn’t updated will no-longer work unless it gets an update.

Obviously, this means a lot of boilerplate code, but it will be worth it. And hiding it from a web interface by using TCP, it will be more secure.

One last thing to note about security is server/port attacks. On each socket connection, the server will log the number of requests from an IP address. It it gets too many then it will ignore that IP address for 30 minutes or another set time. This will reduce DoS attacks on the servers port/s.

When the application is transferring data back and forth to the server, most of the communication will be done in one connection. Too many connection attempts will simply be denied.

Scroll to top