Uncategorized – Page 3

A weekend is over. Updates.

Where is the day between Sunday and Monday?

I needed a little more time this weekend to continue building my project up. The manager is nearing completion with only the transfers of media to devices needing doing now. Then it is getting the devices to simply run on their own.

Distractions at home is what mainly slows my pace down. Noise and stuff.

Ah well.

I only get at most an hour and half after work if that during the week so the first thing I will be doing is adding a shed load of comments to my code.

This last section, the media manager was quite easy to get up and running because the device manager had already been done.

I know that in the future it will eventually get a complete rewrite yet again, but at least this version will handle up to 1000 devices. That’s a low estimate. Once it goes live, I’ll update everything so that it can scale up to anything.

The devices themselves will get a full run by the end of the week. That I am looking forward to seeing. All this work finally being showcased.

BASH script VLC player

In just 15 lines of script I’ve got a fully working test player running from BASH and using the VLC player. All I need now is just a simple Java program to fetch the data from the server and if there are changes then restart the player. Ah, scripting makes life easy. From Java I can check the exit status of System.exit(status) in the bash script and act accordingly. This will speed development up a hell of a lot.

Currently the Java server is rock solid. And I do need the much needed device editor so that I can assign playlists to devices. Now that I have the player I will write a simple device manager. Probably in Qt first then I will transfer it over to mobile.

EDIT: Change that to 10 lines of script. Now to get on to a simple device manager.

EDIT 2: Initially I setup a UI in Qt, but then decided to setup the same UI in Netbeans using Swing and Java. This is mainly because the server is written in Java.

Problems with Overlap2D and LibGdx

Linux… Everything so far I’ve done can be done much faster on Linux when it comes to development.

Then came Overlap2D…

It runs on Linux fine for a short while, but when you really start getting into editing a level, all the graphics start to vanish on the display and all it leaves is the white bounding boxes of the level graphics. So far I cannot find a solution.

I’ve installed IntelliJ and a few other bits on Windows 10 and everything loads up fine, but it’s getting a tad late now.

I’ll report back tomorrow when I finally get something working.

I’m wondering if it is something to do with Linux’s Java and a rendering issue. If I get round to it then I might just look into it sometime over the next few days.

In the meantime, I’m going to get Overlap2D working for both Desktop and Android finally.

EDIT: Overlap2D seems to be working on Windows. Although I really do not like developing on Windows, I think it is about time I tried.

LibGdx and Overlap2D download

I had found what looked to be an awesome utility for LibGdx, Overlap2D. The problem is the website is down and for some time all I could find was the github repo for it here. Now that would be alright if I knew how to build it, only a couple of years into Java. I will get round to getting it to build and I will do another post on how to do that.

There are also plenty of other youtube videos on how to use it which without a doubt make it ‘awesome’. Such as youtube link.

However, to get overlap2d running out of the box, and as I’ve said, the main website is down. I did find buried within a forum post a dropbox link.

Ah crud, I’ve closed the link to the forum post. Instead, just for the time being, the dropbox link:

Overlap2D Java executable

Now I can get on with making use of this excellent piece of kit.

EDIT: The github forum link here.

Just minor updates

It’s been nuts today and yesterday because the missus had an emergency. Hospital and all.

Apart from more research into various areas of coding and getting my own project back up and running, I’ve been researching. Yep, it’s been chaotic which means no programming zone. I’ve learnt a few things which is always a good thing.

Hopefully over the next couple of weeks I should have reached the 100% completion mark on my project and ready for a thorough test run. The server runs solidly, the player runs solidly but needs finishing. It’s only really now the mobile app and the media manager software that needs finishing. The cryptographic communication works just great which is something I’m happy with. Various parts of this project I will eventually release the source too, but for now it’s under wraps.

The next step is to get some bash scripts done to automate some of the management things I keep doing manually at the moment. Especially when it comes to making backups. One little script could do the lot.

Until next time.

Random update Dec 11th 2017

Well, the update of the ancient Android Studio project worked perfectly and it builds much faster and optimises better. Well made up with that. Even with a few problems with the network in work (someone had switched the router off and not told me), I finally got everything back up and running.

Tonight I have had running umpteen programs including Chrome and Android Studio (twice), as well as a 2 Gb Android emulator with ram to spare. Chuffed with this 12 Gb in my laptop now.

I’ve started an openGL ES project to test out some stuff. I will keep coming back to it because I want to not just play around with shaders, but also bench marking C++ and assembler. I know I’ll be doing this test with both ARM and x86_64. If I can run some math over bitmaps in assembler the old school way then I should be able to produce some decent effects. Especially some of the stuff I used to write on the Amiga in assembler.

Plans, plans… Fingers crossed I can stick to a night time routine and start getting some stuff done to showcase. At the moment I get home from work and get my weights and exercises done. Then I have my tea and watch an hour worth of TV. And then I try and chill out on my laptop. This chill out time trying to get some coding done can be difficult at times with the background noise of conversations that completely throw me off. I might look into getting some noise cancelling headphones.

Converting an ancient Android Studio project

Well, it’s unlikely this is going to be appreciated, but these last three days I took it up on myself, as a big challenge to convert a very old project in Android Studio and get it working with the latest gradle and CMake things. The project involves the inclusion of the ffmpeg API.

I actually tried this out in work a couple of weeks ago and failed. This time though, I created a new project from the start. The first day was getting all the source files included and getting the directories working in Android Studio. A little bit of googling and I got it.

The next two days were the big headache days. And I’m still unsure whether it is done, but at least it runs on my phone and my tablet.

Two days worth of googling with little to no help and a lot of trial and error, I managed to get the bloody thing to build. This project I had worked on in work for the last year or so. And Android Studio had gone through it’s multitude of changes and upgrades I couldn’t keep up at the time.

Anyway… It’s bed time. I’m up for work in the morning at 6am. Goosed…

The need to scrap code

Today I found the need to scrap over a weeks worth of coding to start again. These days it’s nothing to me to have to do that because I’ve faced issues with programming many times before. You only have to look at my Previous Projects (especially the procedural level generator) to understand that sometimes when something just isn’t right, you need to scrap it and start again. In the 80’s when I was learning, it was disheartening to have to give up. I was young then. Programming is a massive learning curve for anyone who takes the journey. And what I’ve learned is that sometimes it just doesn’t work. You can’t do anything to fix it. At that point, don’t dwell on it, just take what you need from what has already been done and move on.

As I’ve already mentioned, I’ve had to do that today. Everything was working great, apart from being a tad jumpy, it was working. Almost on to the next stage of the project. Then it would crash. This crash would happen within a certain time frame too, every time, and with the same error. After a few hours of continuous testing and fruitless research into the issue I decided I had to admit defeat. Fortunately though, because of a ‘thread pool’ class which I was using extremely heavily which was causing the problem, the rest of the code is sound and plays nice without bad behaviour. (so far)

So having to ditch, dump, throw away, code, happens… Sometime it’s damn painful…

This all reminds me of the ProcMap project.

It started with a random maze generator which worked great but anyone could do that and I wanted to have corridors and rooms. So came version 2 and 3 which each although the math didn’t work, the ideas where there. The maps didn’t turn out as they should even though they looked genuine. The last version, but not the one I actually wanted to finish on had all the corridors and rooms and sloping dungeons and everything. Including the ability to get coordinates to place other items, including rotations and offset. The math worked perfect in V4. Unfortunately I didn’t get started on V5 because it was going to be a gigantic project that was to create a pseudo random generated planet, complete with landscape, towns, cities, dungeons, life, etc. Yep a big project. I’ve still got a few pieces of paper with plans on upstairs somewhere I hope still.

The weekends plans

Yay! Friday finally came around and it’s the evening. I’m currently on a wind down, my body is aching and I’m mentally and physically zapped out. And that’s why I’m writing an update.

Over this last two weeks my website has had a few changes to it, such as https and a theme upgrade, oh and a few speedups which really did need to happen. As mentioned in an earlier blog post, there’s still a couple of minor changes that need doing, but they’re not important at the moment.

My big project I want to get back to work on over this weekend because I’ve kind of left it behind for a while. Most times when I get home from work, I just sit on my backside watching TV with my laptop. So to fix this problem, I transferred all my projects to my laptop. Win win. I already had the router set to divert a test port to this laptop anyway, so with that change in the sources, last night, everything was back up and running, 100% perfect. The mobile app has just two things to finish off, and the media manager will get a rewrite.

I have noticed that the WebGl demos only work on full screen now and not embedded in a frame since the https thingy update. If I get round to it, then I’ll look into it. Web stuff really ain’t my thing, hardcore programming is.

If I get bored over the weekend I might also create a new home hosted website using GWT or CopperLicht even. We’ll see.

Early night for me tonight because it’s been three nights on the trot of hitting the weights bench. I think the peas on my arms are growing.

Until next time.

WLGfx

This website

I’ve still got a few formatting issues with this theme which will be ironed out over time.

I’ve noticed two so far. One with images not stretching to 100% of the posts width. And the other with using the <pre> tags and the line spacing.

Apart from that, I’ll be re-organising the menus so they look a lot better and categories stuff.

I’ve added a new java page about thread pooling which I want to work on a C++ version of it over the next day or two for my personal project.

Catch y’all soon.

1 2 3 4 5 6 7

Amazing feeling

“I feel almost like a god!”

To have my own software running in the cloud and being able to serve its purpose, ‘feels good’.

Yes, I’ve had one of those moments of personal successful achievements accomplished.

Or something like that.

‘Project’ updates

Maybe things have plodded along at a gentle pace recently, but that’s also a good thing because it give me a chance to reflect back and get things back into perspective. Which I have done.

A few weeks ago I was ready to plough ahead with this. And then as I get stopped in my tracks, or more so, slowed down, I start to find a few more faults, cracks, bugs, etc. All fixed now.

Anyway, the server has now moved onto a VPS somewhere in the cloud which means better bandwidth speed than running from home. I’m still testing out the VPS and getting myself familiar with using it. Everything is via terminal so my previous Linux knowledge is becoming extremely useful here. Commands such as ‘rsync’ to copy files from my PC to the server for example. Now I need to create a way of restarting the server on demand.

The Java TCP server still has its minor issues, but it just works. Any problems I face I can just restart it and everything just carries on. Once it is securely setup on the VPS then I can leave it alone and work on the C++ server upgrade coding.

I won’t talk about the many other things that have happened around this project in the recent weeks apart from to say that it will be big. Fair enough, now I’m starting to feel the pressure is on me to come up with the C++ server code. Which I might actually continue with tomorrow after I figure out how to get my server software to automatically start on the VPS.

Everything will come to light, … in time.

“Follow your passion, not your paycheck. The money will come eventually.”

Just some random updates

The ‘project’ has slowed a bit because of content and artwork which I will be jumping on over these next few days.

Server updates are now being reconsidered because I read last night that non-blocking IO can also be done in Java. I can quickly setup stuff in Java than I can in C++ which means Java has that slight advantage being a language. It may end up being 10-15% slower overall, but it just does stuff. And development is much faster.

However, saying that, I’ve still to test out the non-blocking IO in Java. So time will tell on that score.

If it works then the previous devices commands will be ported over so they will carry on running. The current server is running Java so why not.

If it doesn’t, then it will be back to the fd_set and select() in C which was coming on very nicely until the distraction.

Video editing has been done mainly with Blender. Recently though, I tried Adobe Premiere CS2 and at first glance it looked like it could really do the job. First off, it doesn’t import 99% of video clips and other file formats. I think the only two things that it has going for it is the titling and the rendering of the work area for faster previewing.

Now, Openshot, may not have the best interface, but it just works. Titling is a pain using an external editor. And it also renders the video in memory as you’re going along for faster playback.

With using Openshot, I can create 95% of what I need it for. All titling can be done externally which can be a pain, but more than doable. I just drag n drop the exported PNG’s. I only need to figure out a faster way of setting key frames for animations.

My head is exploding at the moment. I’ve got the ‘project’ to think about, the ‘projects’ server updates, learning new API’s, protocols, etc, blah, as well as trying to do this all in ‘my’ time.

My time now is extremely valuable to me. I’m spending most of it on working on my ‘project’. There are people that know about it, and others I won’t tell because those that know would gladly give up their ‘time’ for me.

This project is actually going to be a huge thing for me. There are only a few involved in it. I was going to involve another but fate stepped in perfectly because of this ‘time’ issue. So it’s carrying on as it was meant to. It is a lot of mental work but it’s will be worth it.

IPv6 server upgrades

A few weeks back I started on the upgrade from the Java server to a C++ server.

I scrapped the first project and started again when I learnt about the handling of fd_set which made things so much better. First tests were great.

Then I scrapped it and decided that if it going worldwide with this secret project, then it needs and uprade. To IPv6.

Which works on my initial tests.

Now all I need it to finish off a few more DNS tests and then I’m ready to make the server ultra fast.

TCP C++ server upgrades

A quick snippet from tonight’s diary entry:

20:30 – I’m still reading up on some stuff about non-blocking TCP server programming and so far the best examples come from http://developerweb.net/viewtopic.php?id=2933. Ideally what I want is NOT to have a shed load of threads running, which I’ll be using a thread manager for, but a way of quickly determining if a connection is valid before shutting it down.

By the looks of the examples, a continuous loop could be checking upto 1024 connections at any time before sending them off to another thread. This method has caught my attention big time because I should be able to setup some kind of timeout very quickly to close the connection if it doesn’t respond to the initial connection request. And I can still serve 100 to 1000 devices without any problems at all.

The same single thread can be constantly checking for connections and readability. It’s the readability part that matters the most here because it will be timed very precisely. Once a connection passes the readability test by passing the initial CSPRNG test, then it can go to a thread handler. Otherwise it gets closed. All of this a single core/thread can easily handle.

I will be studying this a lot further as this looks to be the way forward for a tcp server.

And this is just from my crappy laptop at home.

Basically, the new software upgrades to the server will be 1000 times more efficient than the current Java version. Although once in the cloud, the Java version can easily serve over 500 devices because of the bandwidth increase.

The updated C++ version will increase that significantly as well as handle dodgy connections better. Once a connection has passed the CSPRNG test then the TIMEOUT increases a smidgen.

I’ve already got the CSPRNG functions running and tested and written in assembler code to reduce that bottleneck too.

Until next time… WLGfx…

MS Access Databases

They’re really just as simple as using */UNIX…

Huh…

I’ve been faffing about with my database for most of the night and after relearning a few things, I deleted all my queries and forms and started again.

The relationships screen is where I now layout everything and then I decide to add tables from there well before I start designing the forms.

What I had before would have worked, but I wanted this to be perfect. And that’s why I deleted all my previous work. Using the relationships between Tables and Queries really does make life easier when building up forms and reports.

So far though, with my own project, I’ve not had the need for any VBA unlike at work. The experiences from my own project and the works one too will benefit both projects. I’m hoping that when I decide on changes for the works project, it won’t be too big. Luckily enough, both of them are at the moment small databases.

Just with tonight’s ‘messing’ around, I’ve actually got back to fully understanding the relational side of linking tables and queries. And how it makes creating forms much easier.

It’s only been about two weeks now I think getting back into MS Access, there about.

Ah well… My server and test devices are still running. 🙂

Server bully test

Well, not exactly crashed the server, but I stopped it from running which is a proof of concept.

With a simple Java program which sends legitimate requests from a legitimate device, the server failed after a few minutes. A quick manual restart over ssh got it up and running immediately.

public class BullyServerTest {
    
    public static BullyServerTest bst;

    public static void main(String[] args) {
        bst = new BullyServerTest();
    }
    
    boolean quit = false;
    
    private final String server_host = "SOMEWEBADDRESS"; // LOL
    private final int server_port = SOMESECRETNUMBER;
    
    private final String home = System.getProperty("user.home") + "/";
    
    public BullyServerTest() {
        Runtime.getRuntime().addShutdownHook(new Thread() {
            public void run() {
                quit = true;
                try { Thread.sleep(1000); } catch (InterruptedException ex) {}
            }
        });
        
        while (!quit) {
            try { Thread.sleep(randInt(10, 250)); } catch (InterruptedException ex) {}
            do_hello_world_bully();
        }
    }
    
    Random rand = new Random();
    
    private int randInt(int min, int max) {
        return rand.nextInt((max - min) + 1) + min;
    }
    
    private static final int COMMAND_HELLO = 1234;
    
    private void do_hello_world_bully() {
        int count = randInt(10, 100);
        System.out.println("HELLO WORLD BULLY at " + count + " times");
        
        while (count-- != 0) { // a shed load of threads at once
            new Thread(runnable_HW).start();
        }
        
        System.out.println("Completed...");
    }
    
    Runnable runnable_HW = new Runnable() {
        @Override
        public void run() {
            try {
                WLSocket socket = new WLSocket(server_host, server_port);

                socket.out.writeInt(COMMAND_HELLO);
                String hello = socket.in.readUTF();
                
            } catch (IOException ex) {
                System.out.println("Server failed with HELLO WORLD response");
            }
        }
    };
}

So the next step is to implement a thread pool so that the server will not hang on too many requests. It ran for less than a minute. The above program kept running until my 12Gb RAM ran out which was a bit longer. The current server does only have 4Gb RAM and couldn’t handle the amount of connections at once. I’ve also had to keep the sudden bandwidth usage down as I was hitting my upload and download limits.

Eventually the server and client code will all be rewritten in C++ and handle such things as low latency timeouts.

This has been a very interesting test. I’ll be leaving the hello world server command in for future testing to simulate fake requests.

Project updates

I’ve had the server running for over 3 weeks so far, and although it’s only a few testing devices, it’s looking very promising. The server hasn’t failed once and I’ve even ssh’d into it and run a couple of checks.

My next step, whilst I’m taking the time to build the database software, is to setup another server. More servers means backups and 100% running and up time. The servers can also update each other depending on the regions requirements. That means if I upload content that belongs on other servers, then the servers will organise themselves automatically. A bit like a decentralised distribution of data.

Devices will also receive updates when new servers have been added as periodically they will question what servers are available to them. This will help with manual load balancing as more devices are added. Although were I want to go with this will not need manual load balancing, it is something I want to test out on test servers anyway.

Although all of this is stuck in my head at the moment, I’ve had a few quick browses through my code base and it this seems pretty simply to implement and upgrade without any interruptions to the servers current up time. This is definitely something I want to fully test out. Even down to the servers possibly updating themselves when one has been updated. I will look into this as I’m more leaning towards manually accepting server updates for security.

After long thinking over the weekend I was building myself up to putting this project forward to someone as a partnership proposal. I got home from work today and decided that’s a no. For the time being, this is staying mine.

Updates 2018-Feb-21

My project:

I’ve been taking my time on this project for a damn good reason, I don’t want to rush things and end up making some serious mistakes with it.

There are a few, quite a few bits of code, that will need a rewrite, mainly in C++. This is mainly for future proofing.

I’m currently on stage 2 of this which involves 3 main sections.

The first section involves the database which I am currently working on in MS Access. I used to be an Access master, so I’m getting my hands dirty and learning it all over again. For the past few months I’ve looked for anything that came close to what MS Access can actually do and I’ve found nothing. MS Access is everything for creating a database application from scratch. VBA might be a tad awkward, but it is doing everything I need it for so far. I hope in the future it can also do TCP communications.

The second and third sections of this stage 2 are a lot more involved in prepping for the future. The second being done this next week if all goes well.

The final third of stage 2 will involve the website. I’m hoping that it will be handled by someone I know who is fully capable. This will include handling accounts and payments too. And I’ve been told that it can also handle invoicing.

After that, it’s is just going with the flow. I’m being very weary of contacts because I’ve been bitten before.

Me:

I’ve still got a good few months before the above starts to take shape, although some early testing will be done sooner than expected. Apart from that I’m doing okay with myself apart from slacking with my exercising.

This month has been hard for moving things forward because I lost a lost of money this month. I still get annoyed over that. I could do with a few more devices to put out into the field. And with the lack of testing, I’ve also held back on setting up the cloud server.

Programming has been ‘nicely’ minimal. I’ve mainly been running all my software and fixing bugs. I have been able to chill. Well ‘ish’…

And I’ll get back to a regular weight lifting routine again.

Just a few updates

Well, that’s January over. And it seemed to have shot by really quick too.

The first stage of my personal project is complete and has been running 100% for quite a while now. It’s now just testing out in the field that needs doing.

I’ve got everything running through a DNS server that routes back to my home PC, so no matter where devices are running they can easily connect to my home PC for communications over TCP. Eventually this will be running on a cloud service which I’ve been looking into lately, whether that be Google or AWS or something else, I don’t know yet.

All network traffic runs on two way encryption which was actually very easy to implement. At the moment it is not a very complex encryption because I do want to test for speed overheads with this. If I’m handling 1000 plus devices, ‘at the same time’, then I really don’t want the CPU burning the house down. I’ve a few more cryptographic algorithms I want to test out too, but they can wait for now.

Video creation has been non-existent since before Christmas because I’ve been focused on the programming of this project. Over the last two days I’ve loaded up Blender and although I can move things around easy enough, the old knack has gone again. I really like Blender, but when it comes to finishing something off that requires programming, then Blender has to wait just like most other things.

One thing I do need to start focusing on, and I started looking into this before Christmas, was a full accounting database. The only one from the past that I’m familiar with is MS Access, and trying to find one that works on Linux is still a pain. I’m planning on using Linux so MS will not do. So, when it comes to storing all financial records, invoicing, etc, it looks like I’ll have to write my own software. Shame really, I do like the awesomeness of MS Access. (Which I started using at work earlier today)

So, the next stage is to finalise the tests out in the field. To do this, I will take my demo devices to various places and see it running just nicely. The management software will also be tested out in the field too.

Once that stage has been proven successful over time, then it’s the setting up of the rest of the management of the project. This will include accounts, security, moderation, sustainability tests, reviews on scale-ability, moving to an initial cloud server, and more.

Up until this is a showcase-able project then for the most parts, apart from what I write about, this will be under wraps because I want this to be mine with the help of very few and limited resources. Trusting is hard to do when you lived through enough of life to have experienced people take away your ideas in a wink.

Until next time.

1 2 3 4 5 6 … 11 12 13

Posts navigation