Programming

Java Thread Pool – A thread pool example which shows how to set up and use. Thread pools are very useful for queuing threads up to run after one another. It also allows the setting up of the number of consecutive threads that are running in parallel. An example of a good usage is the generation of 3d mesh data where the mesh is split into separate sections, or image processing.

Java OO class inheritance – When you have lots of objects that make up a scene it is handy to use sub-classing so that you only fill out an array with one class type. Each object class is sub-classed from the root class. This type of coding is what OO is all about.

Previous projects (my history) – These are links to other projects over the years that I have worked on. These go from crappy games, to pseudo random level generators, to 3d level editors and more.

Programming languages I know – I pick up stuff over the years. I started in 1982 so obviously programming languages just happen to be learnt. Some more than others.

Static Memory Manager – A C++ class that handles a memory block as a memory pool to be used for allocations. This is designed to be quick and also when not needed, can be cleared faster than normal memory allocations. I’ve used this quite a lot lately.

Android TCP Client/Server – An example of using TCP communications over the internet to send images to an Android device.

UDP server echo – I needed a way of having devices locate a server and connect to it. UDP broadcasting from a server was a perfect solution for this. The device would just listen for the broadcast data and then within that data was the location of the server.

Psuedo Random Number Generator in Assembler – A new journey has begun again with assembler programming. I’ve always been a fan of using assembler for speed and this is a simple example of using assembler to encrypt data.

ARM assembler 32 bit with 64 bit variables

Although I won’t be using ARM CPU’s for my main project I started to think about my x86_64 random number generator I wrote in a previous blog post and also included in a page for adding assembler code to Eclipse Neon.

There is now 32 bit and 64 bit ARM CPU’s out there in the mobile world, but mostly they consist of 32 bit ARM’s. What I really am better off doing in that case is to use a C function and grab the disassembly and work from that.

Other uses for assembler on mobile devices can easily be upgraded using assembler code, such as image manipulation. Faster math routines, such as calculating 3D meshes and vertex and face normals. I won’t be studying much of ARM assembler because I’m working with portable x86_64 boards.

And, when I was doing 24/7 assembler on the Atari ST and the Amiga 1200, there was 0.01% need for 64 bit values then.

The only reason at the moment for needing 64 bit variables is for the random number generation so that cryptic code can be written. I’m not using ARM CPU’s so carry on with x86_64.

I was hoping to try out some other assembler stuff tonight, especially using Qt because I can access bitmap data directly. I’ll jump onto that tomorrow night instead. I’ve got the assembler bug again.

Eclipse Oxygen and NASM

Tonight I set myself a mission of getting any IDE to build assembler files alongside my C/C++ code.

Find it here: Eclipse Oxygen and NASM

I tried NetBeans at first but the IDE isn’t good at all, actually quite useless for setting up NASM. I spent about half an hour trying this out and eventually gave up. Apparently there are plugins out there, but I wasn’t going to try them out as they were not part of the NetBeans official plugins.

So I look into Eclipse and initially I was put off because there was nothing in the official plugins. A few searches later and I got it. It was already built in to the CDT plugin for eclipse. I only had to make a minor alteration to get it working.

Now I can have a C/C++ project that will also automatically compile and link my assembler source files.

Here’s me thinking I was going to be stuck over the weekend finding this out. Took an hour. What’s next to move on to? Oh yeah, my project. he he…

SSE SIMD and ARM NEON research

I started off wanting to know the CPU cycles and possible cache misses from SSE SIMD instructions, but was kind of mind blown at what SIMD can actually do. There’s also a hell of a lot that the best compilers cannot do with C or C++ code with SIMD stuff. An example is getting the sign bits from each value in a SIMD register in EAX, which is damn handy for some math.

The SSE SIMD stuff is only the base line because there’s now SSE4 I believe. (just checked and it is. behind times I am) Being able to multiple math operations on a single register has tickled my interest for a long time. Tonight I decided to put some proper research into it. Until I got sidetracked with ARM NEON.

Anyway, besides the point and before I go onto the ARM stuff. Over the next few nights at least, I’m going to be testing out more assembler programming. This time using SIMD instructions and possibly being able to use maybe a noise algorithm. Later on in time, not over the next few nights, I will look at using this experience for 3d matrix calcs.

But… Then I looked deeper into the ARM NEON…

What I found with the NEON is that it is kind of like a hyper-threaded architecture. The cpu will run 2 NEON instructions per cycle but during CPU down things (stalls, waits, etc). I need to get deeper to undertsand that but it does sound very much like the way the hyper-threading works on the intel core processors. Still good.

Another thing I did like about the ARM assembler language is it just just so awesome. In standard assembler, you load registers, multiply one register by 2/4/8, add them and the last instruction grabs the result. In ARM, you load the basic registers, and in one instruction you can offsets and bit manipulate to get the address and store the result. Crazy.

I’ll come back to ARM stuff later one. For now, I’ll be focusing on x86_64 stuff and all the SIMD stuff. Over the next few days I’ll run some test and hopefully post some test code. That is if I get something running.

The need to scrap code

Today I found the need to scrap over a weeks worth of coding to start again. These days it’s nothing to me to have to do that because I’ve faced issues with programming many times before. You only have to look at my Previous Projects (especially the procedural level generator) to understand that sometimes when something just isn’t right, you need to scrap it and start again. In the 80’s when I was learning, it was disheartening to have to give up. I was young then. Programming is a massive learning curve for anyone who takes the journey. And what I’ve learned is that sometimes it just doesn’t work. You can’t do anything to fix it. At that point, don’t dwell on it, just take what you need from what has already been done and move on.

As I’ve already mentioned, I’ve had to do that today. Everything was working great, apart from being a tad jumpy, it was working. Almost on to the next stage of the project. Then it would crash. This crash would happen within a certain time frame too, every time, and with the same error. After a few hours of continuous testing and fruitless research into the issue I decided I had to admit defeat. Fortunately though, because of a ‘thread pool’ class which I was using extremely heavily which was causing the problem, the rest of the code is sound and plays nice without bad behaviour. (so far)

So having to ditch, dump, throw away, code, happens… Sometime it’s damn painful…

This all reminds me of the ProcMap project.

It started with a random maze generator which worked great but anyone could do that and I wanted to have corridors and rooms. So came version 2 and 3 which each although the math didn’t work, the ideas where there. The maps didn’t turn out as they should even though they looked genuine. The last version, but not the one I actually wanted to finish on had all the corridors and rooms and sloping dungeons and everything. Including the ability to get coordinates to place other items, including rotations and offset. The math worked perfect in V4. Unfortunately I didn’t get started on V5 because it was going to be a gigantic project that was to create a pseudo random generated planet, complete with landscape, towns, cities, dungeons, life, etc. Yep a big project. I’ve still got a few pieces of paper with plans on upstairs somewhere I hope still.

Assembler PRNG (from Java to Asm)

So, I had a perfectly working PRNG in Java like this:

public class WLPRNG {
    
    long seed;
    
    public WLPRNG(long seed) { this.seed = seed; }
    
    public int nextInt() {
        long result = seed + 0x123defca;
        result = Long.rotateLeft(result, 19);
        result += 0xbead6789;
        result *= 0x1234567c;
        
        int temp = (int)result;
        
        result ^= 0x5ecdab73;
        result = Long.rotateLeft(result, 48);
        
        if (temp % 4 == 0) result *= 0x87650027;
        
        result += 13;
        seed = result;
        
        return (int)result;
    }

    public byte nextByte() {
        return (byte)nextInt();
    }
}

And I thought I’d test out the Assembler version like this:

; random number generator to be used in crypto transmission
; of sensitive data over the internet

; WLGfx 2017-Nov-19

        section .text

global  main

extern  printf

srand:  mov     [seed],rax                      ; set random seed
        ret

arand:  mov     rax,[seed]                      ; get seed
        mov     rbx,qword 0x023defca321acfed
        add     rax,rbx                         ; add 64 bit value
        rol     rax,19                          ; rotate bits
        mov     rbx,qword 0xbead6789
        add     rax,rbx                         ; another add
        imul    rax,qword 0x1234567c            ; a multiple this time
        mov     rbx,rax                         ; copy into rbx
        xor     rax,qword 0x5ecdab73            ; flip some bits
        rol     rax,48                          ; rotate bits again
        mov     rcx,rax                         ; copy to rcx
        and     rax,0x3                         ; mask and test with 0
        jnz     .notz                           ; 25% chance of other ops
        mov     rax,rbx
        add     rcx,rax
        mov     rbx,qword 0x87650027
        imul    rax,rbx
        jmp     .cont
.notz   mov     rax,rcx                         ; back into rax
.cont   mov     [seed],rax                      ; store into seed
        and     rax,0xff                        ; return byte value only
        ret

main:   mov     rax,9                           ; set seed
        call    srand

        mov     dword[lc],10                    ; set loop counter

.loop   call    arand                           ; get random byte

        push    rbp                             ; stack frame
        mov     rsi,rax                         ; random number
        mov     rdi,pf_msg                      ; format string
        xor     rax,rax                         ; 0
        call    printf                          ; call printf
        pop     rbp                             ; stack frame

        sub     dword[lc],1                     ; dec loop counter
        jnz     .loop
        
        ret

        section .data

seed    dq      0,0,0,0                         ; random seed value 64 bit
lc      dd      10                              ; loop counter

pf_msg  db      "Number: 0x%02x",10,0

Using the build script:

#!/bin/bash
nasm -f elf64 random.asm
gcc -o random random.o

Gives a sample output of:

 ~/dev/asm/tests $ ./buildrand.sh 
 ~/dev/asm/tests $ ./random 
Number: 0xe0
Number: 0x5b
Number: 0xca
Number: 0x7c
Number: 0xfc
Number: 0x2d
Number: 0x79
Number: 0xa5
Number: 0x62
Number: 0x7f

All I need to do now is to be able to link directly to C and C++ code. I’m currently reading up on threading in assembler, but it looks like the standard pthreads are just the same really.

There’s lot’s of potential for using assembler.

X86_64 printf 64 bit assembler test

So, I wanted to start playing about with assembler again. Mainly so I could use it for data encryption over the internet. Here’s a simple sample of printing 64 bit numbers as hex.

section .text
global  main

extern  printf

; use printf to print 64 bit hex string

_test:  push    rbp

        mov     rsi,0x1234567890abcdef
        mov     rdi,pf_msg
        xor     rax,rax
        call    printf

        pop     rbp
        
        ret

main:   call    _test
        
        mov     edx,len
        mov     ecx,msg
        mov     eax,4
        int     128
        
        ;mov     eax,1
        ;int     128
        
        xor     rax,rax
        ret

section .data

msg     db      "Hello world!",10
len     equ     $ - msg

; some testing stuff

pf_msg  db      "Register = %016llx", 10, 0

I set up a simple script to build the executable.

#!/bin/bash
nasm -f elf64 test.asm
gcc -o test test.o

And the output is just…

Register = 1234567890abcdef
Hello world!

The weekends plans

Yay! Friday finally came around and it’s the evening. I’m currently on a wind down, my body is aching and I’m mentally and physically zapped out. And that’s why I’m writing an update.

Over this last two weeks my website has had a few changes to it, such as https and a theme upgrade, oh and a few speedups which really did need to happen. As mentioned in an earlier blog post, there’s still a couple of minor changes that need doing, but they’re not important at the moment.

My big project I want to get back to work on over this weekend because I’ve kind of left it behind for a while. Most times when I get home from work, I just sit on my backside watching TV with my laptop. So to fix this problem, I transferred all my projects to my laptop. Win win. I already had the router set to divert a test port to this laptop anyway, so with that change in the sources, last night, everything was back up and running, 100% perfect. The mobile app has just two things to finish off, and the media manager will get a rewrite.

I have noticed that the WebGl demos only work on full screen now and not embedded in a frame since the https thingy update. If I get round to it, then I’ll look into it. Web stuff really ain’t my thing, hardcore programming is.

If I get bored over the weekend I might also create a new home hosted website using GWT or CopperLicht even. We’ll see.

Early night for me tonight because it’s been three nights on the trot of hitting the weights bench. I think the peas on my arms are growing.

Until next time.

WLGfx

This website

I’ve still got a few formatting issues with this theme which will be ironed out over time.

I’ve noticed two so far. One with images not stretching to 100% of the posts width. And the other with using the <pre> tags and the line spacing.

Apart from that, I’ll be re-organising the menus so they look a lot better and categories stuff.

I’ve added a new java page about thread pooling which I want to work on a C++ version of it over the next day or two for my personal project.

Catch y’all soon.

Not much of an update but here goes

Got many plans, and little time to get everything done. The story of my life. he he

Anyways… My journey with the ffmpeg API has now taken another turn. Now I’m travelling the realm of using the AVIOContext. This is because I need to temporarily store a UDP transport stream locally when a pause of playback is required. Before the weekend I was non the wiser of how to do this, but after a good think about it, sorted.

All I need to do is to join the multicast stream manually, grab the packets, buffer them up and when ffmpeg, via the AVIOContext requests data, just send it. Simple. One problem I’ve yet to face is that after a pause, I want to be able to seek through the saved stream. I’ll come to that.

As for my own stuff. I’m kind of halfheartedly putting my feet up. I really need to put my feet up properly actually but I’ll take what I can get.

WebGL interactive

Nothing to show yet because it would be quite dull, but I’ve got mouse movement and click interaction working in the 3D scene. Basically, if the mouse clicks an object (when it’s not occluded) it will do stuff. And when the mouse hovers over an object. Old school really, but for WebGL, this can demonstrate a lot.

And at this time of night, I’m goosed.

1 2 3 4 5 6 7 8 … 11 12 13