CMake

When I was writing this blog four years ago I wanted to write cross-platform C code that was simple to build. At the time I settled on a dual approach of using a make file for non-windows and a set of Visual Studio 2010 projects and solution files for Windows building. I’ve never liked the dual approach but at the time I did not have anything better.

These days I use CMake as my build system. To be technically correct CMake is not a build system, it is a builder of build systems. This is what makes it great. Rather than being locked into a particular system, you write all your build files and dependencies in a language separate from the build system. CMake then will generate the build files for your chosen build system.

Something that CMake is particularly good at is building “out-of-source” builds. Something that Visual Studio is awful at by default. The standard method of Visual Studio is to scatter all the build artefacts throughout your source tree. This is particularly annoying for source control (you end up with tons of rules in .gitignore to avoid committing binaries and temporary files), and also makes it hard to have a clean build as you never know for sure what left overs may be lurking around affecting the build. With CMake you can just delete your build tree at anytime and generate a new one. You can also have many build trees in parallel. For example you may wish to build using Visual Studio and MinGW.

By default CMake will generate files with the most obvious builder it can find on your system. So if you are running Windows and have Visual Studio it will use that. If you are on Linux it will probably choose gcc and make. However you can tell it to generate a specific build system, and there is a large list of supported ones. In particular for Linux and MacOS I typically like it to build using Ninja which is lightning fast compared to make. Occasionally I get it to generate Xcode projects for MacOS.

Something else I particularly like about CMake is the concept of Toolchain files that can be specified on the command line using the (not especially elegant) syntax of:

 -DCMAKE_TOOLCHAIN_FILE=

It will this run this file first before attempting to locate the compilers for your system. With this you can setup cross-compilers or other custom settings you want without cluttering up your in source CMake files. Also you don’t want to put hard coded paths and settings into your actual project that you are planning to share.

So for example I like to keep my CMakeLists.txt files as simple and clean as possible and put all my custom stuff in my dev environment outside of the source trees. So if someone builds one of my projects it should always work with their system as it will just use the defaults. For example if they are using Visual Studio it will build it using fairly standard Visual Studio settings. I don’t actually like Visual Studio default builds, in particular I don’t like its dependent on its own C runtimes that you then have to distribute. So I use a toolchain file that builds using WinDDK 7.1 which links to MSVCRT.DLL. I also have other toolchain files for building on all the other platforms I want to do, including MinGW, Linux, MacOS, and even a Big-Endian mips Linux system!

I have made a sample “Hello World” project that can be built with CMake. A zip file containing the source is here

Assuming you have CMake installed and in the path then simply open a terminal at the root of the sample directory and run the following

cmake -H. -Bbuild -DCMAKE_INSTALL_PREFIX=bin
cmake --build build --target install

This will create the build system and build the binaries and place them in the subdirectory bin

The first command generates the build system for your platform. In Windows this will probably be Visual Studio, in Linux this will probably be gcc and make. The parameter -H. says “source file starts here”. -Bbuild says “put the build system into the subfolder build“. The -CMAKE_INSTALL_PREFIX=bin says “When building with --target install, make the directory the subfolder bin”

The second command says “build using the build system in subdirectory build and place the executables in the install location specified before (bin)”

It is highly recommended to always perform an “out-of-source” build. You can however build “in-source” with the following simpler commands:

cmake .
cmake --build .

This will however places the builds files all over the place in the directory and is generally considered messy. However if you are just wanting to get the binaries build with the least fuss, and are not planning to actually to use the source files for coding then this is the quickest way. In this example we did not specify the install location. The binaries will be built in the source tree and not copied to the install location.

I have since converted all my previous projects to using CMake. I will soon be publishing a newer version of CryptLib that uses CMake (and also has the new AES and AES-CTR cipher in it).

 

Advertisement

AES as a stream cipher in counter mode (AES-CTR)

In the previous article I introduced the AES block cipher. This, however, is not too much use on its own. AES encrypts a single 128 bit (16 byte) block. In almost every situation there will be a requirement to encrypt a much larger amount of data. This is solved by using AES in a “mode of operation”.

Several modes exist for using AES block cipher on larger data sets. Several of these have been specified by NIST as standards. The first and most obvious one is known as AES-ECB (Electronic code book). This should really not exist as a standard as it is a terrible idea to use in every situation. Basically it just divides the larger data set into 16 bytes chunks and encrypts each block. The problem with this is that any time the same block of input appears, exactly the same output will be produced. For example if you had a file with large runs of all zeros in it, then you would see a repeating pattern on 16 bytes recurring in the output. So never use ECB, there are plenty of good modes to choose from.

Today I am going to talk about AES-CTR (AES in “counter” mode). This is actually my favourite mode. It turns AES into a stream cipher which is very easy to work with, it can be used on any sized data. It is also more convent to use than RC4 because you can jump to any offset within the output stream and start from there. With RC4 you would have to “spin” through all the previous stream to get to a specific point.

AES CTR is very simple mode. Basically you have a counter which you store in a 128 bit block and you then encrypt that block with AES, that is your first 16 bytes of stream output. You then increment the counter and encrypt to produce the second block, and so on. To add another level of  convenience and speed the counter block is also used as an “Initialisation vector” (iv). This allows someone to create an entirely new stream without having to reinitialise the AES key (this is a somewhat expensive procedure).

NIST don’t actually specify how you should combine an IV and the counter, but suggest a few ways. The method I like best is to split the block into two halves. The first half is the 64bit IV. The second half is the 64bit block counter (store in big endian form). This is the method that openssl uses, and the method I chose to use. The advantage of this method over xoring the IV and counter together is that it is guaranteed that you will never “overlap” by having particular chosen IVs and counters. Remember that this is a stream cipher and just like RC4 if you ever reuse the stream you have removed all security. However with AES CTR it is only necessary to have the a unique set of BOTH Key and IV. Therefore the same Key can be reused with unique IVs. Similarly you may not want to bother with the IV at all and only use unique keys. Personally if I am in a situation where I will be generating new keys for each use, I will generate a key 64 bits larger than AES needs and use the last 64 bits as the IV.

Public Domain source code for AES-CTR

• CryptLib_AesCtr.h
• CryptLib_AesCtr.c

These depend on the AES block cipher files:

• CryptLib_Aes.h
• CryptLib_Aes.c

I have made a simple program called AesCtrOutput that takes a key and and IV in hex form and produces the specified number of bytes out in hex on stdout.

AesCtrOutput.c

The syntax for this program is pretty simple

Syntax
 AesCtrOutput <Key> <IV> <NumBytes>
 <Key> - 128, 192, or 256 bit written as hex
 <IV> - 64 bit written as hex
 <NumBytes> - Number of bytes of stream to output

Example

>AesCtrOutput.exe 0102030405060708a1a2a3a4a5a6a7a8 b1b2b3b4b5b6b7b8 48
7f1e34c4f33ee8dc162af7fbed6f317aa5806d244dd86557268be2296708ef7327aa4e5ed5780a3c070209ea2db04d79

This result can be confirmed as being the same as openssl by using the following command:

> openssl enc -aes-128-ctr -K 0102030405060708a1a2a3a4a5a6a7a8 -iv b1b2b3b4b5b6b7b8 -in zero.bin -out output.bin

If zero.bin is a file contain 48 zero bytes then output.bin will be created as 48 bytes with the same bytes as in the AesCtrOutput.exe example.

I have made a .zip file containing all the source for AES and AESCTR along with those two sample programs and a unit test set. The root of the folder contains a CMakeLists.txt file for building with CMake.

AesCtr.zip

This is free and unencumbered software released into the public domain.

 

AES Block Cipher

One of the things I meant to do when I started the WaterJuice blog 4 years ago was to build up library of Cryptographic functions. I got as far as putting in some of the common hashes of MD5, SHA1, and SHA2. And I also had RC4 encryption in there. But then got no further. I always meant to put in block ciphers. So today finally I add AES.

AES is a block cipher that operates on 128 bit blocks. A block cipher is a keyed permutation on a block. Every possible value of a 128 block is mapped with a 1-1 correspondence to another block for any given key. AES takes 3 sizes of keys: 128 bit, 192 bit, and 256 bit.

AES was announced by NIST at the end of 2001, this was after a five year process in selecting the new cipher to replace DES which was woefully inadequate by this time. AES is a subset of the Rijndael cipher which was of several algorithms that competed to become the new standard. NIST did not make any changes to Rijndael except to restrict the possible set of key sizes and block sizes. Rijndael works on block sizes and key sizes between 128 and 256 bit that are multiples of 32bit. AES formalises a single block size and 3 key sizes.

In this article I will introduce public domain C code for encrypting and decrypting a block using the AES cipher. This is a basic build block for many other operations. In general AES on its own is not terribly useful as the chances are you’ll want to encrypt something far larger than a single block, however I will talk about modes of operation in a subsequent article. For now I will just present the block cipher primitive.

Before proceeding I’d like to answer the question “why use AES?”. The best answer for that is because it is a safe bet. As the saying went “no one got fired for choosing IBM”, the same can be said about choosing AES. Because it is the standard, it is also the most scrutinised algorithm, and it is is still standing strong 16 years later.

Public domain C source code for AES

• CryptLib_Aes.h
• CryptLib_Aes.c

The following is a sample program that uses that code to encrypt or decrypt a single block specified by hex on a command line.

AesBlock.c

This has a simple syntax of:

AesBlock [-D]

HexKey must be 128, 192, or 256 bits long, and HexBlock must be 128 bits long. If the -D flag is specified then decryption is peformed

Example:

waterjuice$ AesBlock.exe 2b7e151628aed2a6abf7158809cf4f3c 6bc1bee22e409f96e93d7e117393172a

3ad77bb40d7a3660a89ecaf32466ef97

 

Four years later…

In 2013 I created a blog site called WaterJuice, hosted on a domain “waterjuice.org”. For a  couple of months I wrote various articles on it relating to cryptography and various other C project things that interested me. Then I got busy with other things and left the site be. Eventually the domain expired and at some point I changed my hosting provider and did not even have a copy of the original site.

Recently I have been wanting to recreate past sites I’ve made in a more permanent manner. Preferably one that does not involve me paying indefinitely to keep its presence. For my other newer blog projects (most drawing related) I have been using WordPress.com. Previously I have not wanted to use free hosting and have opted to using wordpress software on my own paid domains. However the problem is after some years of not using a site, the motivation to keep paying the renewal fees wanes. So I wanted something that would stick around after I had got bored with a site. So WordPress.com has been brilliant for that. I have a few new sites hosted on that, happily knowing that if I never update them again they won’t just vanish from the Internet.

I have also been trying to recreate my old sites. I have had various ones that I have hosted myself previously, all go which have eventually faded away and expired. Sadly I never kept a local copy of these sites, however the “Way back machine” has been diligently saving websites over the years. I have been patiently copy and pasting content from my old sites into new wordpress.com ones. I still have not finished my cartoon ones as there is a lot of content and it takes quite a lot of time. My WaterJuice site had a lot fewer articles and has not taken me as long to recreate. I have almost every article I wrote except a few right at the start (including the introduction) resurrected in this new site waterjuiceweb.wordpress.com.

I also plan to start writing some more. A fair bit has changed in the past 4 years. Back in 2013 I was using Visual Studio 2010 (I never got into 2012 or 2013). Now I use 2017 having previously used 2015. I also now use Cmake which I find amazing and brilliant and will write an article about.

Also since 2013, SHA-3 has become a ratified thing, so I will definitely talk about that soon. Additionally Creative Commons CC0 became something that could be applied to software (back in 2013 the recommendation was against it, hence I used UNLICENSE to put my code into public domain).

So a fair bit has changed over the years, and I am quite keen to write some more articles. For how long? I don’t know, but I at least now they should remain available on the internet indefinitely.

Stay tuned for updates 🙂