Category Archives: Uncategorized

CryptLib 2.0.0

After a four year gap, I have updated CryptLib at GitHub. I have now released version 2.0.0.

Changes:

  • Added AES and AES-CTR modules. AES-CTR conforms to the same counter mode used with AES in OpenSSL.
  • All algorithms now work on Big-Endian architectures.
  • Now uses CMake for building rather than make files and Visual Studio projects. CMake will generate whatever system is required.
  • Input function parameters are now marked const
  • File names have been changed to have the prefix CryptLib_ rather than Lib
  • Various formatting changes to the files.

This is free and unencumbered software released into the public domain.

Advertisements

CMake

When I was writing this blog four years ago I wanted to write cross-platform C code that was simple to build. At the time I settled on a dual approach of using a make file for non-windows and a set of Visual Studio 2010 projects and solution files for Windows building. I’ve never liked the dual approach but at the time I did not have anything better.

These days I use CMake as my build system. To be technically correct CMake is not a build system, it is a builder of build systems. This is what makes it great. Rather than being locked into a particular system, you write all your build files and dependencies in a language separate from the build system. CMake then will generate the build files for your chosen build system.

Something that CMake is particularly good at is building “out-of-source” builds. Something that Visual Studio is awful at by default. The standard method of Visual Studio is to scatter all the build artefacts throughout your source tree. This is particularly annoying for source control (you end up with tons of rules in .gitignore to avoid committing binaries and temporary files), and also makes it hard to have a clean build as you never know for sure what left overs may be lurking around affecting the build. With CMake you can just delete your build tree at anytime and generate a new one. You can also have many build trees in parallel. For example you may wish to build using Visual Studio and MinGW.

By default CMake will generate files with the most obvious builder it can find on your system. So if you are running Windows and have Visual Studio it will use that. If you are on Linux it will probably choose gcc and make. However you can tell it to generate a specific build system, and there is a large list of supported ones. In particular for Linux and MacOS I typically like it to build using Ninja which is lightning fast compared to make. Occasionally I get it to generate Xcode projects for MacOS.

Something else I particularly like about CMake is the concept of Toolchain files that can be specified on the command line using the (not especially elegant) syntax of:

 -DCMAKE_TOOLCHAIN_FILE=

It will this run this file first before attempting to locate the compilers for your system. With this you can setup cross-compilers or other custom settings you want without cluttering up your in source CMake files. Also you don’t want to put hard coded paths and settings into your actual project that you are planning to share.

So for example I like to keep my CMakeLists.txt files as simple and clean as possible and put all my custom stuff in my dev environment outside of the source trees. So if someone builds one of my projects it should always work with their system as it will just use the defaults. For example if they are using Visual Studio it will build it using fairly standard Visual Studio settings. I don’t actually like Visual Studio default builds, in particular I don’t like its dependent on its own C runtimes that you then have to distribute. So I use a toolchain file that builds using WinDDK 7.1 which links to MSVCRT.DLL. I also have other toolchain files for building on all the other platforms I want to do, including MinGW, Linux, MacOS, and even a Big-Endian mips Linux system!

I have made a sample “Hello World” project that can be built with CMake. A zip file containing the source is here

Assuming you have CMake installed and in the path then simply open a terminal at the root of the sample directory and run the following

cmake -H. -Bbuild -DCMAKE_INSTALL_PREFIX=bin
cmake --build build --target install

This will create the build system and build the binaries and place them in the subdirectory bin

The first command generates the build system for your platform. In Windows this will probably be Visual Studio, in Linux this will probably be gcc and make. The parameter -H. says “source file starts here”. -Bbuild says “put the build system into the subfolder build“. The -CMAKE_INSTALL_PREFIX=bin says “When building with --target install, make the directory the subfolder bin

The second command says “build using the build system in subdirectory build and place the executables in the install location specified before (bin)”

It is highly recommended to always perform an “out-of-source” build. You can however build “in-source” with the following simpler commands:

cmake .
cmake --build .

This will however places the builds files all over the place in the directory and is generally considered messy. However if you are just wanting to get the binaries build with the least fuss, and are not planning to actually to use the source files for coding then this is the quickest way. In this example we did not specify the install location. The binaries will be built in the source tree and not copied to the install location.

I have since converted all my previous projects to using CMake. I will soon be publishing a newer version of CryptLib that uses CMake (and also has the new AES and AES-CTR cipher in it).

 

AES as a stream cipher in counter mode (AES-CTR)

In the previous article I introduced the AES block cipher. This, however, is not too much use on its own. AES encrypts a single 128 bit (16 byte) block. In almost every situation there will be a requirement to encrypt a much larger amount of data. This is solved by using AES in a “mode of operation”.

Several modes exist for using AES block cipher on larger data sets. Several of these have been specified by NIST as standards. The first and most obvious one is known as AES-ECB (Electronic code book). This should really not exist as a standard as it is a terrible idea to use in every situation. Basically it just divides the larger data set into 16 bytes chunks and encrypts each block. The problem with this is that any time the same block of input appears, exactly the same output will be produced. For example if you had a file with large runs of all zeros in it, then you would see a repeating pattern on 16 bytes recurring in the output. So never use ECB, there are plenty of good modes to choose from.

Today I am going to talk about AES-CTR (AES in “counter” mode). This is actually my favourite mode. It turns AES into a stream cipher which is very easy to work with, it can be used on any sized data. It is also more convent to use than RC4 because you can jump to any offset within the output stream and start from there. With RC4 you would have to “spin” through all the previous stream to get to a specific point.

AES CTR is very simple mode. Basically you have a counter which you store in a 128 bit block and you then encrypt that block with AES, that is your first 16 bytes of stream output. You then increment the counter and encrypt to produce the second block, and so on. To add another level of  convenience and speed the counter block is also used as an “Initialisation vector” (iv). This allows someone to create an entirely new stream without having to reinitialise the AES key (this is a somewhat expensive procedure).

NIST don’t actually specify how you should combine an IV and the counter, but suggest a few ways. The method I like best is to split the block into two halves. The first half is the 64bit IV. The second half is the 64bit block counter (store in big endian form). This is the method that openssl uses, and the method I chose to use. The advantage of this method over xoring the IV and counter together is that it is guaranteed that you will never “overlap” by having particular chosen IVs and counters. Remember that this is a stream cipher and just like RC4 if you ever reuse the stream you have removed all security. However with AES CTR it is only necessary to have the a unique set of BOTH Key and IV. Therefore the same Key can be reused with unique IVs. Similarly you may not want to bother with the IV at all and only use unique keys. Personally if I am in a situation where I will be generating new keys for each use, I will generate a key 64 bits larger than AES needs and use the last 64 bits as the IV.

Public Domain source code for AES-CTR

These depend on the AES block cipher files:

I have made a simple program called AesCtrOutput that takes a key and and IV in hex form and produces the specified number of bytes out in hex on stdout.

AesCtrOutput.c

The syntax for this program is pretty simple

Syntax
 AesCtrOutput <Key> <IV> <NumBytes>
 <Key> - 128, 192, or 256 bit written as hex
 <IV> - 64 bit written as hex
 <NumBytes> - Number of bytes of stream to output

Example

>AesCtrOutput.exe 0102030405060708a1a2a3a4a5a6a7a8 b1b2b3b4b5b6b7b8 48
7f1e34c4f33ee8dc162af7fbed6f317aa5806d244dd86557268be2296708ef7327aa4e5ed5780a3c070209ea2db04d79

This result can be confirmed as being the same as openssl by using the following command:

> openssl enc -aes-128-ctr -K 0102030405060708a1a2a3a4a5a6a7a8 -iv b1b2b3b4b5b6b7b8 -in zero.bin -out output.bin

If zero.bin is a file contain 48 zero bytes then output.bin will be created as 48 bytes with the same bytes as in the AesCtrOutput.exe example.

I have made a .zip file containing all the source for AES and AESCTR along with those two sample programs and a unit test set. The root of the folder contains a CMakeLists.txt file for building with CMake.

AesCtr.zip

This is free and unencumbered software released into the public domain.

 

AES Block Cipher

One of the things I meant to do when I started the WaterJuice blog 4 years ago was to build up library of Cryptographic functions. I got as far as putting in some of the common hashes of MD5, SHA1, and SHA2. And I also had RC4 encryption in there. But then got no further. I always meant to put in block ciphers. So today finally I add AES.

AES is a block cipher that operates on 128 bit blocks. A block cipher is a keyed permutation on a block. Every possible value of a 128 block is mapped with a 1-1 correspondence to another block for any given key. AES takes 3 sizes of keys: 128 bit, 192 bit, and 256 bit.

AES was announced by NIST at the end of 2001, this was after a five year process in selecting the new cipher to replace DES which was woefully inadequate by this time. AES is a subset of the Rijndael cipher which was of several algorithms that competed to become the new standard. NIST did not make any changes to Rijndael except to restrict the possible set of key sizes and block sizes. Rijndael works on block sizes and key sizes between 128 and 256 bit that are multiples of 32bit. AES formalises a single block size and 3 key sizes.

In this article I will introduce public domain C code for encrypting and decrypting a block using the AES cipher. This is a basic build block for many other operations. In general AES on its own is not terribly useful as the chances are you’ll want to encrypt something far larger than a single block, however I will talk about modes of operation in a subsequent article. For now I will just present the block cipher primitive.

Before proceeding I’d like to answer the question “why use AES?”. The best answer for that is because it is a safe bet. As the saying went “no one got fired for choosing IBM”, the same can be said about choosing AES. Because it is the standard, it is also the most scrutinised algorithm, and it is is still standing strong 16 years later.

Public domain C source code for AES

The following is a sample program that uses that code to encrypt or decrypt a single block specified by hex on a command line.

AesBlock.c

This has a simple syntax of:

AesBlock [-D]

HexKey must be 128, 192, or 256 bits long, and HexBlock must be 128 bits long. If the -D flag is specified then decryption is peformed

Example:

waterjuice$ AesBlock.exe 2b7e151628aed2a6abf7158809cf4f3c 6bc1bee22e409f96e93d7e117393172a

3ad77bb40d7a3660a89ecaf32466ef97

 

Four years later…

In 2013 I created a blog site called WaterJuice, hosted on a domain “waterjuice.org”. For a  couple of months I wrote various articles on it relating to cryptography and various other C project things that interested me. Then I got busy with other things and left the site be. Eventually the domain expired and at some point I changed my hosting provider and did not even have a copy of the original site.

Recently I have been wanting to recreate past sites I’ve made in a more permanent manner. Preferably one that does not involve me paying indefinitely to keep its presence. For my other newer blog projects (most drawing related) I have been using WordPress.com. Previously I have not wanted to use free hosting and have opted to using wordpress software on my own paid domains. However the problem is after some years of not using a site, the motivation to keep paying the renewal fees wanes. So I wanted something that would stick around after I had got bored with a site. So WordPress.com has been brilliant for that. I have a few new sites hosted on that, happily knowing that if I never update them again they won’t just vanish from the Internet.

I have also been trying to recreate my old sites. I have had various ones that I have hosted myself previously, all go which have eventually faded away and expired. Sadly I never kept a local copy of these sites, however the “Way back machine” has been diligently saving websites over the years. I have been patiently copy and pasting content from my old sites into new wordpress.com ones. I still have not finished my cartoon ones as there is a lot of content and it takes quite a lot of time. My WaterJuice site had a lot fewer articles and has not taken me as long to recreate. I have almost every article I wrote except a few right at the start (including the introduction) resurrected in this new site waterjuiceweb.wordpress.com.

I also plan to start writing some more. A fair bit has changed in the past 4 years. Back in 2013 I was using Visual Studio 2010 (I never got into 2012 or 2013). Now I use 2017 having previously used 2015. I also now use Cmake which I find amazing and brilliant and will write an article about.

Also since 2013, SHA-3 has become a ratified thing, so I will definitely talk about that soon. Additionally Creative Commons CC0 became something that could be applied to software (back in 2013 the recommendation was against it, hence I used UNLICENSE to put my code into public domain).

So a fair bit has changed over the years, and I am quite keen to write some more articles. For how long? I don’t know, but I at least now they should remain available on the internet indefinitely.

Stay tuned for updates 🙂

 

 

 

 

Cellular Automaton

Early on when I started programming I came across “Conway’s life” which is the classic example of cellular automaton. I was programming in Modula-2 on DOS at the time, and wrote an implementation of Conway’s Life. I was fascinated at the little “life forms” that it created, and would watch it for ages. Since then, whenever I have started using a new language or system I tend to write a version of Life. I have programmed dozens now. I have varied it along the way. Sometimes the universe has an edge, sometimes it wraps. Sometimes there are two colours of cells instead of one.

While programming C on Windows is by no means new to me now, the OpenGL framework is. So I decided I should write another implementation of Conway’s life using OpenGL and make it cross platform for Windows, OSX, and Linux.

The following (reduced) screen shots are of a typical run. The first image is shortly after it has started. There are many cells changing with every frame.

ConwaysLife_Early

Later on the cells become less dense and a lot of areas have stabilised.

ConwaysLife_Mid

And finally the entire universe stabilises. Every pattern is either static or is oscillating between a small number of states.ConwaysLife_End

 

Having finished this I wanted to experiment with a different cellular automaton that I had seen years ago. I’ve titled it “racist cells”. Every position in the “universe” is filled with a cell from one of five colours. At each iteration a cell can decide whether it wants to move or stay. If it moves it swaps position with another cell in the universe that also wishes to move. The probability of moving is higher when the cell is surrounded by cells of other colours. The universe is initially seeded by a random distribution of the cells. There is an equal number of each colour. Unlike Conway’s life cells are not destroyed or created. There is always the same number of each colour at each iteration.

The early stage looks like TV static, as all the cells are random and almost all the cells are wanting to move at each iteration.

RacistCells_Early

After a few iterations the cells have started coming closer to other cells of their same colour and happier to stay. So you see it forming into small coloured blobs. There is a lot of movement though because there are a lot of places where the colours are still mixed.

RacistCells_Mid

Over time the colonies become larger as the cells in the centre of each colony are happy to stay put. The boundaries are fought over and the colonies become larger pushing other ones away.
The trend continues but slows down. There are less borders and less movement.
Eventually there will only be one blob per colour (remember the universe wraps). Each blob will be 20% of universe. The borders will then slowly move, but the overall pattern will look the same.

RacistCells_Late

Unlike Conway’s life, this has a random element to it, so each frame is not predictable from the previous one. The probabilities for moving can be modified in the program giving quite different results.

For the examples above the probabilities are given as:
{ 20, 50, 60, 200, 210, 220, 250, 253, 255 }
For each cell a random value between 0 and 255 is picked. The threshold for moving is taken from the array above. The position in the array is based on number of cells of same colour surrounding.

So if a cell is has no neighbouring cells of same colour as it, then it will move if the random number is over 20. In other words, a 92% chance of moving. If a cell has 7 out of 8 neighbours the same colour then its chance of moving is about 0.8%. And if all 8 neighbours are the same colour then it will not move at all.

Its quite a lot of fun to play around with these values in the array and see the different outcomes.

CellularAutomaton.zip – This contains the C source code and executables for Windows, OSX, and Linux. This is free and unencumbered software released into the public domain.

Animated surface ripple with OpenGL

In the previous article I wrote an example program that drew a 3D surface. Starting with a wireframe, and then progressing to a solid surface. This was a demonstration of my simple graphics library in C. I implemented a very basic drawing library using GDI for Windows and OpenGL for OSX and Linux. I then built the 3D surface using those primitives. In the course of writing the library I got to learn a little about OpenGL. My reason for choosing it in the first place was that it was available on OSX and has a C interface. Having seen some of the OpenGL interface I was keen to do a little bit more with it. In particular I wanted to animate the surface wave I had made, and wanted to render it with lighting effects. OpenGL is perfect for this task.

Windows also has OpenGL, it does not have GLUT though which comes with OSX. So I wrote a small wrapper library that provides a very basic windows creating API that works cross platform. On Windows it creates the Window and then attaches OpenGL to it and provides a message loop. For OSX and Linux it uses GLUT.

The library has one function called OpenGLRun

//////////////////////////////////////////////////////////////////////////////////
//  OpenGLRun
//
//  Starts the OpenGL module. This will create a window of the specified size 
//  which will be used for OpenGL operations. The DrawFunction will be called when//  it is time for the window to be drawn as well as each tick of the timer
//////////////////////////////////////////////////////////////////////////////////
bool
    OpenGLRun
    (
        uint16_t                ResolutionX,            // [in]
        uint16_t                ResolutionY,            // [in]
        char*                   Title,                  // [in]
        LibOpenGlFunction       DrawFunction,           // [in]
        int                     TimerElapseMilliseconds // [in]
    );

The prototype for LibOpenGlFunction is a functiont hat takes no parameters and returns no return code. This function is called every TimerElapseMilliseconds (and also other times it needs to redraw). This function can then use OpenGL drawing commands to draw the frame. When the function returns the library will call “SwapBuffers” to make the instant transition from the previous frame.

In my DrawFunction I drew the surface from the plot function (that made the ripple) out of triangles. OpenGL likes triangles in a 3D space.

//////////////////////////////////////////////////////////////////////////////////
//  DrawMesh
//
//  Draws the mesh using OpenGL functions
//////////////////////////////////////////////////////////////////////////////////
void
    DrawMesh
    (
        MeshPoints*     Mesh
    )
{
    int ix;
    int iy;
    double normalX;
    double normalY;
    double normalZ;

    // Draw two triangles between sets of 4 points
    for( iy=gNumMeshPoints; iy>0; iy-- )
    {
        for( ix=1; ix<=gNumMeshPoints; ix++ )
        {
            CalcSurfaceNormal(
        Mesh->meshX[ix-1][iy-1], Mesh->meshY[ix-1][iy-1], Mesh->meshZ[ix-1][iy-1],
        Mesh->meshX[ix-1][iy], Mesh->meshY[ix-1][iy], Mesh->meshZ[ix-1][iy],
        Mesh->meshX[ix][iy-1], Mesh->meshY[ix][iy-1], Mesh->meshZ[ix][iy-1],
        &normalX, &normalY, &normalZ );

            glBegin( GL_TRIANGLES );

            glNormal3d( normalX, normalY, normalZ );
            glVertex3d(
                Mesh->meshX[ix-1][iy-1], 
                Mesh->meshY[ix-1][iy-1],
                Mesh->meshZ[ix-1][iy-1] );
            glVertex3d(
                Mesh->meshX[ix-1][iy],
                Mesh->meshY[ix-1][iy],
                Mesh->meshZ[ix-1][iy] );
            glVertex3d(
                Mesh->meshX[ix][iy-1], 
                Mesh->meshY[ix][iy-1], 
                Mesh->meshZ[ix][iy-1] );

            glNormal3d( normalX, normalY, normalZ );
            glVertex3d(
                Mesh->meshX[ix][iy-1],
                Mesh->meshY[ix][iy-1],
                Mesh->meshZ[ix][iy-1] );
            glVertex3d(
                Mesh->meshX[ix-1][iy], 
                Mesh->meshY[ix-1][iy], 
                Mesh->meshZ[ix-1][iy] );
            glVertex3d( 
                Mesh->meshX[ix][iy],
                Mesh->meshY[ix][iy],
                Mesh->meshZ[ix][iy] );

            glEnd( );
        }
    }
}

I wanted to put some lighting effects in. I used the following

glClearColor( 0.0f, 0.0f, 0.0f, 0.0f );
glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );
glEnable(GL_DEPTH_TEST); 
glEnable(GL_POLYGON_SMOOTH);
glEnable( GL_LIGHTING );
glEnable( GL_LIGHT0 );
glShadeModel( GL_SMOOTH );

I also rotated the view point using

glRotated( angle, 1.0, 0.5, 0.0 );
glRotated( angle/2, 0.0, 0.0, 0.1 );

Where angle is 240 degrees.

One thing I found disappointing in OpenGL is that although you are providing it triangles which are therefore flat planes, it does not calculate the reflection angle of the plane itself. This probably makes sense for efficiency, but it seemed annoying that I had to provide the normals for each triangle where in my opinion it already had all the information it needed.

The following function calculates a normal vector for a triangle in 3D space

//////////////////////////////////////////////////////////////////////////////////
//  CalcSurfaceNormal
//
//  Calculates the surface normal of the triangle
//////////////////////////////////////////////////////////////////////////////////
void
    CalcSurfaceNormal
    (
        double      x1,
        double      y1,
        double      z1,
        double      x2,
        double      y2,
        double      z2,
        double      x3,
        double      y3,
        double      z3,
        double*     pNormalX,
        double*     pNormalY,
        double*     pNormalZ
    )
{
    double  d1x;
    double  d1y;
    double  d1z;
    double  d2x;
    double  d2y;
    double  d2z;
    double  crossx;
    double  crossy;
    double  crossz;
    double  dist;

    d1x = x2 - x1;
    d1y = y2 - y1;
    d1z = z2 - z1;

    d2x = x3 - x2;
    d2y = y3 - y2;
    d2z = z3 - z3;

    crossx = d1y*d2z - d1z*d2y;
    crossy = d1y*d2x - d1x*d2z;
    crossz = d1x*d2y - d1y*d2x;

    dist = sqrt( crossx*crossx + crossy*crossy + crossz*crossz );

    *pNormalX = crossx / dist;
    *pNormalY = crossy / dist;
    *pNormalZ = crossz / dist;
}

The end result looked like this
OpenGlExample_Frame_04Now I wanted to animate it. I did this by changing the plot function to name take a 3rd parameter which is time (in seconds). So it now takes an X and Y coordinate and a time value and returns a Z (height) value.

//////////////////////////////////////////////////////////////////////////////////
//  RippleFunction
//
//  This function is a 3d plot function. It provides a Z value for given X and Y.
//  All values are between 0 and 1
//////////////////////////////////////////////////////////////////////////////////
double
    RippleFunction
    (
        double  x,
        double  y,
        double  seconds
    )
{
    double pi = 3.14159265358979;
    double z;
    double dist;
    double wave;
    double posX;
    double posY;
    double factor;

    // Get distance of point from centre (0.5, 0.5)
    posX = 0.5;
    posY = 0.5;
    dist = sqrt( (posX-x)*(posX-x) + (posY-y)*(posY-y) );

    wave = sin( 4*(2*pi) * dist - (seconds/1.0) ) * 0.4;
    z = (wave / 2.0) + 0.5;
    z *= (1.0-dist*1.2);

    // Now calculate a second wave with a moving centre and a low amplitude and
    // low frequency wave.
    // Get Distance of point from moving centre
    posX = 0.8 * (sin( (2*pi)*seconds))*0.01;
    posY = 0.8 * (sin( (4*pi)*seconds/2))*0.01;
    dist = sqrt( (posX-x)*(posX-x) + (posY-y)*(posY-y) );
    wave = sin( 2*(2*pi) * dist - (seconds/2.0) ) * 0.05;

    // Apply the second wave to first as a factor that varies over a sine wave
    factor = 1.0 - (cos( ((2*pi) * seconds/40) ) + 1.0) / 2.0;
    z += (wave * factor);

    return z;
}

This has two ripples with different frequencies, and one which moves the centre of its origin.  Additionally the second ripple is applied to the first one using a scale factor that is attached to a sine wave based on time. When the factor is 0 only the first wave is seen, which is a nice uniform ripple. As time progresses it becomes more distorted with the second wave and then eventually that fades off again.

An example of it with the second wave distorting it:

OpenGlExample_Frame_21

The end result as a QuickTime .mov file is available here

I had quite a lot of fun playing with this. Small changes in the plot function can create interesting results. I have put the program that generates the above wave online for download. It contains the LibOpenGL library which is the small wrapper library I wrote. It should be noted that this is just quick and simple experimental code. My expertise is not in OpenGL or graphics programming. I did this just for fun, and if other people find it handy that is good. However it is most unlikely that this is the “proper” way of doing this sort of graphics.

OpenGLExample.zip – C Source code using OpenGL to draw an animated ripple on a surface. Includes binaries for Windows, OSX, and Linux. This is free and unencumbered software released into the public domain.