Java – Page 3

Good and bad for embedded and Android

Embedded Linux is the bees knees for doing a lot of things, especially when you really don’t want the Android overhead on top of it. For a device such as the Odroid range which can be used as a headless server providing and IPTV gateway and a web interface for controls, linux is the better option.

When it comes to graphical applications, for example decoding video streams and openGL rendering, it always becomes a toss up between Linux and Android. A lot of people who are familiar with Java will automatically pick Java and I have to say, I do actually like the Java Virtual Machine. The JVM is handy whilst developing because a crashed application just gets cleared out.

Qt can also be used for developing mobile applications and has a very extensive framework and because it’s C++, time critical code isn’t a problem.

The only issue I have with Qt is that when it comes the day when I do want to release and application for whatever it is I finally do, packaging it up for all OS’s becomes a pain.

Qt for Android is easy, just build the project and get the APK. Just one file and it can be copied to any Adroid device.

Qt for Windows is a nuisance because you have to faff about with the terminal (which is awful in Windows) to package all the required libraries, and then it’s not guaranteed to run straight out of the box. For Windows you have to use a dependency walker to find the missing libraries.

Qt for Linux isn’t too bad unless you want the latest libraries. At this moment you can install Qt5.5 from the Linux repositories and away you go. Unfortunately, the latest version of Qt is 5.7 with some very nice additions so it’s back to packaging stuff up.

For embedded Linux it would have to be Qt from the Linux repositories.

For Android it is still a toss up between Qt and Android Studio + NDK. Although if I could get away with it I would use Qt and Linux unless it was going to be an application for the Android market.

GWT + 3D

When my home server Apache is running (basically when my PC at home is on), then the very first test of 3D inside a web page is working…

WLGfx@home

However, the RPC calls are not working. I’ll get onto that straight away.

Eclipse update and GWT

After upgrading from Linux Mint 17.3 to 18, I found that the minimum Java available was 8. For GWT projects, up to 7 is allowed because there are issues running with Java 8.

So, I deleted Eclipse ready to install the latest. It installed fine. Still some issues along the way with Java 7 and 8 with GWT.

Eventually I installed Oracle Java 7. So far so good.

After finding that there is no longer support for Android in eclipse and installing all the Android stuff I cleared eclipse again. Actually I removed Eclipse a few times until I got Java 7 and 8 and all the GWT web stuff. Including Java EE version of eclipse.

Now I’ve figured out how to run a GWT project straight out of Eclipse without needing to upload onto the Apache server too. This makes for much faster build times. However, one cavaet is that I will have to create a linux group and add my user to the group next so that Tomcat7 and my user can read/write to the servers home directory.

Having sped up the build times, development of GWT projects will be so much easier and less time consuming. I will only need to deploy to the Apache main server when it has all been done. All I need to do is to save all files that I’ve been editing, click refresh in the server pane in Eclipse and go to the page in the browser. The browser will then tell the project to build itself the changes made.

Good stuff for a few hours of headaches.

LibGdx TextureAtlas (non .pack file)

Okay, I hit a block when I was starting to think about writing a quick networked multiplayer shoot em up game in LibGdx.

LibGdx texture atlas’s are very handy things because you can store a load of smaller images in one image. This also optimises the GPU draw routines because it doesn’t have to keep swapping textures for each sprite drawn.

I found pre-made free to use sprites from Kenney.nl, below is a section of what they look like:

spritesheet

Unfortunately though, LibGdx uses .pack files for it’s texture atlas’s and this one come with one that looks like this:

<TextureAtlas imagePath="sprites.png">
 <SubTexture name="spaceAstronauts_001.png" x="998" y="847" width="34" height="44"/>
 <SubTexture name="spaceAstronauts_002.png" x="919" y="142" width="37" height="44"/>
 <SubTexture name="spaceAstronauts_003.png" x="824" y="0" width="50" height="44"/>

...
</TextureAtlas>

Which is completely different to a .pack file.

So without further ado I decided to test out an xml parser to retrieve all the sub-regions from the xml file.

Test code as follows:

[snippet id=”19″]

So, armed with this now, all thanks mainly to working with GWT, I can quickly move on to creating my own class to generate the texture atlas in LibGdx.

Android client TCP server image request

Well, I finally finished off the initial test where I can run an app on my phone or tablet anywhere, click something and it fetches a random image from my PC at home.

The updated sources can be found HERE.

android-tcp-client-server-app

Android load and scale a bitmap

Before I move onto the application itself, I was testing out loading in and scaling a bitmap, when I came across a very annoying Out Of Memory error. The image I was testing was ~4000 x ~3500 pixels and the test was running in the android emulator.

Initially, after using the android logging I found the image wasn’t being scaled and I couldn’t load it in and then scale it. And I didn’t want to up the memory on the emulator.

I know I needed an AsyncTask to do this, plus I needed to figure out how to implement them anyway as I’ve used Async’s in GWT extensively.

Eventually after a bit of digging around I eventually found out how to get everything done and it doesn’t waste any memory.

After this has been done, I want to figure out how to display a large image with a zoom option.

See below for the code…

[snippet id=”15″]

TCP home server test client

With the previous post setup I’ve written a very basic example of how to test the server. At the moment, there are no back and forth communications, just a simple send of a 4 byte header.

Run the server program first, and then run the test. These can actually be run in the same order from Eclipse.

The good news is, this also works over the internet. The ‘Global’ in the code hold a string of my web server address and also the port that my router has open.

So, all working as expected.

[snippet id=”11″]

The output from the server will be:

WLGfx Home Server - Starting up
Shutdown hook attached
TCP Server thread started...
TCP Incoming socket handler
Got header: HEAD

Java server test 1 [UPDATED]

My first test for Java as a console daemon. There’s not much here at the moment and the project is in Eclipse IDE.

From Eclipse, right click on the project and select export and then export as a runnable jar. Then in the terminal just type:

java -jar myexportedjar.jar

The JVM will continue to run as long as there are non daemon threads still running as in the below code. Also note that the hook handler allows catching of CTRL+C and other errors.

UPDATE:

I’ve now added the actually tcp server code which has a timeout on the server socket to allow for the thread to shut down safely. And a thread to handle each incoming connection.

[snippet id=”9″]

(I’m currently not well at the moment but I’m plodding on with some studies.)

1 2 3

‘Project’ updates

Maybe things have plodded along at a gentle pace recently, but that’s also a good thing because it give me a chance to reflect back and get things back into perspective. Which I have done.

A few weeks ago I was ready to plough ahead with this. And then as I get stopped in my tracks, or more so, slowed down, I start to find a few more faults, cracks, bugs, etc. All fixed now.

Anyway, the server has now moved onto a VPS somewhere in the cloud which means better bandwidth speed than running from home. I’m still testing out the VPS and getting myself familiar with using it. Everything is via terminal so my previous Linux knowledge is becoming extremely useful here. Commands such as ‘rsync’ to copy files from my PC to the server for example. Now I need to create a way of restarting the server on demand.

The Java TCP server still has its minor issues, but it just works. Any problems I face I can just restart it and everything just carries on. Once it is securely setup on the VPS then I can leave it alone and work on the C++ server upgrade coding.

I won’t talk about the many other things that have happened around this project in the recent weeks apart from to say that it will be big. Fair enough, now I’m starting to feel the pressure is on me to come up with the C++ server code. Which I might actually continue with tomorrow after I figure out how to get my server software to automatically start on the VPS.

Everything will come to light, … in time.

“Follow your passion, not your paycheck. The money will come eventually.”

Just some random updates

The ‘project’ has slowed a bit because of content and artwork which I will be jumping on over these next few days.

Server updates are now being reconsidered because I read last night that non-blocking IO can also be done in Java. I can quickly setup stuff in Java than I can in C++ which means Java has that slight advantage being a language. It may end up being 10-15% slower overall, but it just does stuff. And development is much faster.

However, saying that, I’ve still to test out the non-blocking IO in Java. So time will tell on that score.

If it works then the previous devices commands will be ported over so they will carry on running. The current server is running Java so why not.

If it doesn’t, then it will be back to the fd_set and select() in C which was coming on very nicely until the distraction.

Video editing has been done mainly with Blender. Recently though, I tried Adobe Premiere CS2 and at first glance it looked like it could really do the job. First off, it doesn’t import 99% of video clips and other file formats. I think the only two things that it has going for it is the titling and the rendering of the work area for faster previewing.

Now, Openshot, may not have the best interface, but it just works. Titling is a pain using an external editor. And it also renders the video in memory as you’re going along for faster playback.

With using Openshot, I can create 95% of what I need it for. All titling can be done externally which can be a pain, but more than doable. I just drag n drop the exported PNG’s. I only need to figure out a faster way of setting key frames for animations.

My head is exploding at the moment. I’ve got the ‘project’ to think about, the ‘projects’ server updates, learning new API’s, protocols, etc, blah, as well as trying to do this all in ‘my’ time.

My time now is extremely valuable to me. I’m spending most of it on working on my ‘project’. There are people that know about it, and others I won’t tell because those that know would gladly give up their ‘time’ for me.

This project is actually going to be a huge thing for me. There are only a few involved in it. I was going to involve another but fate stepped in perfectly because of this ‘time’ issue. So it’s carrying on as it was meant to. It is a lot of mental work but it’s will be worth it.

IPv6 server upgrades

A few weeks back I started on the upgrade from the Java server to a C++ server.

I scrapped the first project and started again when I learnt about the handling of fd_set which made things so much better. First tests were great.

Then I scrapped it and decided that if it going worldwide with this secret project, then it needs and uprade. To IPv6.

Which works on my initial tests.

Now all I need it to finish off a few more DNS tests and then I’m ready to make the server ultra fast.

TCP C++ server upgrades

A quick snippet from tonight’s diary entry:

20:30 – I’m still reading up on some stuff about non-blocking TCP server programming and so far the best examples come from http://developerweb.net/viewtopic.php?id=2933. Ideally what I want is NOT to have a shed load of threads running, which I’ll be using a thread manager for, but a way of quickly determining if a connection is valid before shutting it down.

By the looks of the examples, a continuous loop could be checking upto 1024 connections at any time before sending them off to another thread. This method has caught my attention big time because I should be able to setup some kind of timeout very quickly to close the connection if it doesn’t respond to the initial connection request. And I can still serve 100 to 1000 devices without any problems at all.

The same single thread can be constantly checking for connections and readability. It’s the readability part that matters the most here because it will be timed very precisely. Once a connection passes the readability test by passing the initial CSPRNG test, then it can go to a thread handler. Otherwise it gets closed. All of this a single core/thread can easily handle.

I will be studying this a lot further as this looks to be the way forward for a tcp server.

And this is just from my crappy laptop at home.

Basically, the new software upgrades to the server will be 1000 times more efficient than the current Java version. Although once in the cloud, the Java version can easily serve over 500 devices because of the bandwidth increase.

The updated C++ version will increase that significantly as well as handle dodgy connections better. Once a connection has passed the CSPRNG test then the TIMEOUT increases a smidgen.

I’ve already got the CSPRNG functions running and tested and written in assembler code to reduce that bottleneck too.

Until next time… WLGfx…

MS Access Databases

They’re really just as simple as using */UNIX…

Huh…

I’ve been faffing about with my database for most of the night and after relearning a few things, I deleted all my queries and forms and started again.

The relationships screen is where I now layout everything and then I decide to add tables from there well before I start designing the forms.

What I had before would have worked, but I wanted this to be perfect. And that’s why I deleted all my previous work. Using the relationships between Tables and Queries really does make life easier when building up forms and reports.

So far though, with my own project, I’ve not had the need for any VBA unlike at work. The experiences from my own project and the works one too will benefit both projects. I’m hoping that when I decide on changes for the works project, it won’t be too big. Luckily enough, both of them are at the moment small databases.

Just with tonight’s ‘messing’ around, I’ve actually got back to fully understanding the relational side of linking tables and queries. And how it makes creating forms much easier.

It’s only been about two weeks now I think getting back into MS Access, there about.

Ah well… My server and test devices are still running. 🙂

Server bully test

Well, not exactly crashed the server, but I stopped it from running which is a proof of concept.

With a simple Java program which sends legitimate requests from a legitimate device, the server failed after a few minutes. A quick manual restart over ssh got it up and running immediately.

public class BullyServerTest {
    
    public static BullyServerTest bst;

    public static void main(String[] args) {
        bst = new BullyServerTest();
    }
    
    boolean quit = false;
    
    private final String server_host = "SOMEWEBADDRESS"; // LOL
    private final int server_port = SOMESECRETNUMBER;
    
    private final String home = System.getProperty("user.home") + "/";
    
    public BullyServerTest() {
        Runtime.getRuntime().addShutdownHook(new Thread() {
            public void run() {
                quit = true;
                try { Thread.sleep(1000); } catch (InterruptedException ex) {}
            }
        });
        
        while (!quit) {
            try { Thread.sleep(randInt(10, 250)); } catch (InterruptedException ex) {}
            do_hello_world_bully();
        }
    }
    
    Random rand = new Random();
    
    private int randInt(int min, int max) {
        return rand.nextInt((max - min) + 1) + min;
    }
    
    private static final int COMMAND_HELLO = 1234;
    
    private void do_hello_world_bully() {
        int count = randInt(10, 100);
        System.out.println("HELLO WORLD BULLY at " + count + " times");
        
        while (count-- != 0) { // a shed load of threads at once
            new Thread(runnable_HW).start();
        }
        
        System.out.println("Completed...");
    }
    
    Runnable runnable_HW = new Runnable() {
        @Override
        public void run() {
            try {
                WLSocket socket = new WLSocket(server_host, server_port);

                socket.out.writeInt(COMMAND_HELLO);
                String hello = socket.in.readUTF();
                
            } catch (IOException ex) {
                System.out.println("Server failed with HELLO WORLD response");
            }
        }
    };
}

So the next step is to implement a thread pool so that the server will not hang on too many requests. It ran for less than a minute. The above program kept running until my 12Gb RAM ran out which was a bit longer. The current server does only have 4Gb RAM and couldn’t handle the amount of connections at once. I’ve also had to keep the sudden bandwidth usage down as I was hitting my upload and download limits.

Eventually the server and client code will all be rewritten in C++ and handle such things as low latency timeouts.

This has been a very interesting test. I’ll be leaving the hello world server command in for future testing to simulate fake requests.

Project updates

I’ve had the server running for over 3 weeks so far, and although it’s only a few testing devices, it’s looking very promising. The server hasn’t failed once and I’ve even ssh’d into it and run a couple of checks.

My next step, whilst I’m taking the time to build the database software, is to setup another server. More servers means backups and 100% running and up time. The servers can also update each other depending on the regions requirements. That means if I upload content that belongs on other servers, then the servers will organise themselves automatically. A bit like a decentralised distribution of data.

Devices will also receive updates when new servers have been added as periodically they will question what servers are available to them. This will help with manual load balancing as more devices are added. Although were I want to go with this will not need manual load balancing, it is something I want to test out on test servers anyway.

Although all of this is stuck in my head at the moment, I’ve had a few quick browses through my code base and it this seems pretty simply to implement and upgrade without any interruptions to the servers current up time. This is definitely something I want to fully test out. Even down to the servers possibly updating themselves when one has been updated. I will look into this as I’m more leaning towards manually accepting server updates for security.

After long thinking over the weekend I was building myself up to putting this project forward to someone as a partnership proposal. I got home from work today and decided that’s a no. For the time being, this is staying mine.

Updates 2018-Feb-21

My project:

I’ve been taking my time on this project for a damn good reason, I don’t want to rush things and end up making some serious mistakes with it.

There are a few, quite a few bits of code, that will need a rewrite, mainly in C++. This is mainly for future proofing.

I’m currently on stage 2 of this which involves 3 main sections.

The first section involves the database which I am currently working on in MS Access. I used to be an Access master, so I’m getting my hands dirty and learning it all over again. For the past few months I’ve looked for anything that came close to what MS Access can actually do and I’ve found nothing. MS Access is everything for creating a database application from scratch. VBA might be a tad awkward, but it is doing everything I need it for so far. I hope in the future it can also do TCP communications.

The second and third sections of this stage 2 are a lot more involved in prepping for the future. The second being done this next week if all goes well.

The final third of stage 2 will involve the website. I’m hoping that it will be handled by someone I know who is fully capable. This will include handling accounts and payments too. And I’ve been told that it can also handle invoicing.

After that, it’s is just going with the flow. I’m being very weary of contacts because I’ve been bitten before.

Me:

I’ve still got a good few months before the above starts to take shape, although some early testing will be done sooner than expected. Apart from that I’m doing okay with myself apart from slacking with my exercising.

This month has been hard for moving things forward because I lost a lost of money this month. I still get annoyed over that. I could do with a few more devices to put out into the field. And with the lack of testing, I’ve also held back on setting up the cloud server.

Programming has been ‘nicely’ minimal. I’ve mainly been running all my software and fixing bugs. I have been able to chill. Well ‘ish’…

And I’ll get back to a regular weight lifting routine again.

Just a few updates

Well, that’s January over. And it seemed to have shot by really quick too.

The first stage of my personal project is complete and has been running 100% for quite a while now. It’s now just testing out in the field that needs doing.

I’ve got everything running through a DNS server that routes back to my home PC, so no matter where devices are running they can easily connect to my home PC for communications over TCP. Eventually this will be running on a cloud service which I’ve been looking into lately, whether that be Google or AWS or something else, I don’t know yet.

All network traffic runs on two way encryption which was actually very easy to implement. At the moment it is not a very complex encryption because I do want to test for speed overheads with this. If I’m handling 1000 plus devices, ‘at the same time’, then I really don’t want the CPU burning the house down. I’ve a few more cryptographic algorithms I want to test out too, but they can wait for now.

Video creation has been non-existent since before Christmas because I’ve been focused on the programming of this project. Over the last two days I’ve loaded up Blender and although I can move things around easy enough, the old knack has gone again. I really like Blender, but when it comes to finishing something off that requires programming, then Blender has to wait just like most other things.

One thing I do need to start focusing on, and I started looking into this before Christmas, was a full accounting database. The only one from the past that I’m familiar with is MS Access, and trying to find one that works on Linux is still a pain. I’m planning on using Linux so MS will not do. So, when it comes to storing all financial records, invoicing, etc, it looks like I’ll have to write my own software. Shame really, I do like the awesomeness of MS Access. (Which I started using at work earlier today)

So, the next stage is to finalise the tests out in the field. To do this, I will take my demo devices to various places and see it running just nicely. The management software will also be tested out in the field too.

Once that stage has been proven successful over time, then it’s the setting up of the rest of the management of the project. This will include accounts, security, moderation, sustainability tests, reviews on scale-ability, moving to an initial cloud server, and more.

Up until this is a showcase-able project then for the most parts, apart from what I write about, this will be under wraps because I want this to be mine with the help of very few and limited resources. Trusting is hard to do when you lived through enough of life to have experienced people take away your ideas in a wink.

Until next time.

A weekend is over. Updates.

Where is the day between Sunday and Monday?

I needed a little more time this weekend to continue building my project up. The manager is nearing completion with only the transfers of media to devices needing doing now. Then it is getting the devices to simply run on their own.

Distractions at home is what mainly slows my pace down. Noise and stuff.

Ah well.

I only get at most an hour and half after work if that during the week so the first thing I will be doing is adding a shed load of comments to my code.

This last section, the media manager was quite easy to get up and running because the device manager had already been done.

I know that in the future it will eventually get a complete rewrite yet again, but at least this version will handle up to 1000 devices. That’s a low estimate. Once it goes live, I’ll update everything so that it can scale up to anything.

The devices themselves will get a full run by the end of the week. That I am looking forward to seeing. All this work finally being showcased.

1 2 3 4 5 6 … 10 11 12

Posts navigation