For the last two weeks I’ve had an intermittent problem with my C++ version of the crypto socket client, and tonight I finally cracked the issue.
The Java version is using ‘long’ for doing it’s calculations and the C++ version was using a 64 bit integer, which should have been fine until tonight I realised that Java was using its ‘long’ values as signed. Whereas I had been using ‘uint64_t’ in the C++ version.
This caused the C++ version to work once in every 5 to 10 tries. Just by removing the unsigned, it now works flawlessly.
Phew! Now I can blitz the rest…
(Yep, it has been a while since the last post)
Been busy working on my secret project.
It was tough to accept the change from my old website (wlgfx) to this new one (csnorwood), but I couldn’t ignore the benefits.
Yes, this is going to affect the search engines for some time but I’m used to that. I can still slowly build stuff in the meantime while the web crawlers do there stuff.
It’s highly unlikely that the the old web address is even going to link back to this new one, but what the hell. Time to move onward and upwards.
Okay, so my server is currently running on Java and makes use of DataOutputStream and DataInputStream. And whilst I’m on a rewrite, I need to wrap those into C++.
Which I thought might be a tad difficult…
It turns out that Linux automatically handles UTF-8 strings out of the box, so sending and receiving string is of no consequence. Which was one of my main worries before I looked into everything else which slowed down the transition. Java’s DataOutputStream writes it UTF-16 strings as UTF-8 strings over the wire. That’s that problem solved.
The other problem was wrapping the socket connection. Sorted. Easily done by returning the error status on every call and handling the active status of the socket.
The last problem was wrapping the Java functions for sending a Long, Int, Byte, String, etc… These need to be in network byte order, which is not little endian. I’m just going through that now.
Oh yeah, encryption using a Cryptographic Secure Pseudo Random Number Generator. Done and implemented to match perfectly the Java version… For now.
So hopefully soon, I’ll be able to test run this wrapper ready for a production run.
“I feel almost like a god!”
To have my own software running in the cloud and being able to serve its purpose, ‘feels good’.
Yes, I’ve had one of those moments of personal successful achievements accomplished.
Or something like that.
Maybe things have plodded along at a gentle pace recently, but that’s also a good thing because it give me a chance to reflect back and get things back into perspective. Which I have done.
A few weeks ago I was ready to plough ahead with this. And then as I get stopped in my tracks, or more so, slowed down, I start to find a few more faults, cracks, bugs, etc. All fixed now.
Anyway, the server has now moved onto a VPS somewhere in the cloud which means better bandwidth speed than running from home. I’m still testing out the VPS and getting myself familiar with using it. Everything is via terminal so my previous Linux knowledge is becoming extremely useful here. Commands such as ‘rsync’ to copy files from my PC to the server for example. Now I need to create a way of restarting the server on demand.
The Java TCP server still has its minor issues, but it just works. Any problems I face I can just restart it and everything just carries on. Once it is securely setup on the VPS then I can leave it alone and work on the C++ server upgrade coding.
I won’t talk about the many other things that have happened around this project in the recent weeks apart from to say that it will be big. Fair enough, now I’m starting to feel the pressure is on me to come up with the C++ server code. Which I might actually continue with tomorrow after I figure out how to get my server software to automatically start on the VPS.
Everything will come to light, … in time.
“Follow your passion, not your paycheck. The money will come eventually.”
The ‘project’ has slowed a bit because of content and artwork which I will be jumping on over these next few days.
Server updates are now being reconsidered because I read last night that non-blocking IO can also be done in Java. I can quickly setup stuff in Java than I can in C++ which means Java has that slight advantage being a language. It may end up being 10-15% slower overall, but it just does stuff. And development is much faster.
However, saying that, I’ve still to test out the non-blocking IO in Java. So time will tell on that score.
If it works then the previous devices commands will be ported over so they will carry on running. The current server is running Java so why not.
If it doesn’t, then it will be back to the fd_set and select() in C which was coming on very nicely until the distraction.
Video editing has been done mainly with Blender. Recently though, I tried Adobe Premiere CS2 and at first glance it looked like it could really do the job. First off, it doesn’t import 99% of video clips and other file formats. I think the only two things that it has going for it is the titling and the rendering of the work area for faster previewing.
Now, Openshot, may not have the best interface, but it just works. Titling is a pain using an external editor. And it also renders the video in memory as you’re going along for faster playback.
With using Openshot, I can create 95% of what I need it for. All titling can be done externally which can be a pain, but more than doable. I just drag n drop the exported PNG’s. I only need to figure out a faster way of setting key frames for animations.
My head is exploding at the moment. I’ve got the ‘project’ to think about, the ‘projects’ server updates, learning new API’s, protocols, etc, blah, as well as trying to do this all in ‘my’ time.
My time now is extremely valuable to me. I’m spending most of it on working on my ‘project’. There are people that know about it, and others I won’t tell because those that know would gladly give up their ‘time’ for me.
This project is actually going to be a huge thing for me. There are only a few involved in it. I was going to involve another but fate stepped in perfectly because of this ‘time’ issue. So it’s carrying on as it was meant to. It is a lot of mental work but it’s will be worth it.
A few weeks back I started on the upgrade from the Java server to a C++ server.
I scrapped the first project and started again when I learnt about the handling of fd_set which made things so much better. First tests were great.
Then I scrapped it and decided that if it going worldwide with this secret project, then it needs and uprade. To IPv6.
Which works on my initial tests.
Now all I need it to finish off a few more DNS tests and then I’m ready to make the server ultra fast.
A quick snippet from tonight’s diary entry:
20:30 – I’m still reading up on some stuff about non-blocking TCP server programming and so far the best examples come from http://developerweb.net/viewtopic.php?id=2933. Ideally what I want is NOT to have a shed load of threads running, which I’ll be using a thread manager for, but a way of quickly determining if a connection is valid before shutting it down.
By the looks of the examples, a continuous loop could be checking upto 1024 connections at any time before sending them off to another thread. This method has caught my attention big time because I should be able to setup some kind of timeout very quickly to close the connection if it doesn’t respond to the initial connection request. And I can still serve 100 to 1000 devices without any problems at all.
The same single thread can be constantly checking for connections and readability. It’s the readability part that matters the most here because it will be timed very precisely. Once a connection passes the readability test by passing the initial CSPRNG test, then it can go to a thread handler. Otherwise it gets closed. All of this a single core/thread can easily handle.
I will be studying this a lot further as this looks to be the way forward for a tcp server.
And this is just from my crappy laptop at home.
Basically, the new software upgrades to the server will be 1000 times more efficient than the current Java version. Although once in the cloud, the Java version can easily serve over 500 devices because of the bandwidth increase.
The updated C++ version will increase that significantly as well as handle dodgy connections better. Once a connection has passed the CSPRNG test then the TIMEOUT increases a smidgen.
I’ve already got the CSPRNG functions running and tested and written in assembler code to reduce that bottleneck too.
Until next time… WLGfx…
They’re really just as simple as using */UNIX…
I’ve been faffing about with my database for most of the night and after relearning a few things, I deleted all my queries and forms and started again.
The relationships screen is where I now layout everything and then I decide to add tables from there well before I start designing the forms.
What I had before would have worked, but I wanted this to be perfect. And that’s why I deleted all my previous work. Using the relationships between Tables and Queries really does make life easier when building up forms and reports.
So far though, with my own project, I’ve not had the need for any VBA unlike at work. The experiences from my own project and the works one too will benefit both projects. I’m hoping that when I decide on changes for the works project, it won’t be too big. Luckily enough, both of them are at the moment small databases.
Just with tonight’s ‘messing’ around, I’ve actually got back to fully understanding the relational side of linking tables and queries. And how it makes creating forms much easier.
It’s only been about two weeks now I think getting back into MS Access, there about.
Ah well… My server and test devices are still running. 🙂