Author: wyncliffe

Quantum GUIDs!

I recently discovered the following web site operated by the Australian National University (ANU): https://qrng.anu.edu.au

This is a random number generator using quantum physics. This should produce the purest of random numbers. From their web site you can get random numbers in various formats and they also provide a simple API that returns the values in JSON format.

I wrote a python script to collect random values from the site and produce Type 4 GUIDs from it. These GUIDs will have 122 bits of pure random. Quantum random!

The python file will generate between 1 and 1000 GUIDs specified on the command line.

python3 QuantumGuid.py 10
{9b747ca1-f809-44e6-af35-7041b790ddf4}
{6b5c5703-cb16-41c9-9101-c889fb31a93e}
{3f4f6eb7-0ca1-4ffb-a685-1b1afc824235}
{36e75e7c-3317-4a10-b781-d4376decf937}
{e2e85ad6-8a6d-4de1-b145-455c2bead457}
{5a0784aa-a302-40e9-87f3-d151abec44f5}
{7596d675-2e8c-4dbd-8e5e-ca9282222421}
{cdc57458-de8f-49af-811a-29c80ae76107}
{165565bb-1038-4381-b6ba-b64c3e64fe98}
{77d631dc-ef63-465a-acb7-b22b4e8e92c4}

These GUIDs feel so much more random than a regular type 4 version that I really feel they should be in their own type space so as to not be contaminated with regular non quantum GUIDs ;-). Perhaps they could be type 8 GUIDs? (type 6 is already adhoc used, and I already have an idea for type 7!)

If you want to get your own, then download the script

This is free and unencumbered software released into the public domain.

Advertisements

Personal GUIDs – Fixed

Six years ago I wrote the article Personal GUIDs. This introduced a technique for assigning a unique unchanging type 5 GUID to each person on the planet. It made a string of the format:

"Surname;GivenNames;Sex;CountryOfBirth;PlaceOfBirth;DateOfBirth,"

which in theory should be unique for everyone, and turned it into a type 5 GUID using the Namespace GUID:

 {5b390b3f-9a62-508a-b235-6e6e8d270720}

(Read https://waterjuiceweb.wordpress.com/2013/06/16/type-3-and-5-guids/ for a description on how a type 5 GUID is made). I wrote a C program to generate the GUIDs.

Unfortunately, I made a mistake!

The C code did not use the DateOfBirth value in creating the string to be hashed. This meant only the first 5 fields were used. As a result the GUIDs produced did not match the description. Also it would mean two people born in the same place with same name but at different times could generate the same GUID.

I only just noticed the error while I was including the code into a new library I’m producing (Stay tuned for WjGuid coming soon!) and I was testing the results against an online type 5 GUID generator.

I have fixed the code and released PersonalGuid 1.0.1. The latest version is available here:

 >PersonalGuid.exe Doe John m Australia Sydney 19700101
ID: DOE;JOHN;M;AUSTRALIA;SYDNEY;19700101
{897469e0-c10b-5829-bd15-86309b1523cf}

To manually produce the personal guid without this tool you can use a site such as https://www.toolswow.com/generator/uuid-v5

For NameSpace mode use UUID and use the value ” {{5b390b3f-9a62-508a-b235-6e6e8d270720}”.

ScreenShot.png
Screenshot from https://www.toolswow.com/generator/uuid-v5

I additionally made a Python version that will work in Python 2 or 3. The python file is inside the .zip, but I have included it here as well.

 

 

 

Making a basic web site look better

TL;DR: Here is a repo https://github.com/WaterJuice/BottleMaterializeTemplate which contains all you need to use Materialize and Bottle in an self contained host using only Python.

Recently I wrote about OIDs and made a basic website that would issue out a new OID to anyone who wanted one without any registration required. (Here)

I don’t pretend to be a web designer but I think it is safe to say the original site was pretty ugly:

Screen Shot 2019-08-23 at 6.08.10 pm.png

I wanted something simple to “make it look good” without requiring me to go to any particular great effort or learn a bunch of CSS. Materialize was recommended to me, so I had a look. Using one of the samples provided I was quickly able to make my site look considerably better. The end result:

Screen Shot 2019-08-24 at 2.51.30 pm.png

I think this was considerably better. Additionally it works well on mobile devices.

While making the site I decided to make a template that I could easily use to start any future projects. In particular I wanted to make have all the files local so it could be self hosted without Internet access (for example on an internal intranet). I used bottle which is a web server framework written in python. This can be installed with pip, or simply copy the single file (bottle.py). Materialize just required two files (materialize.min.css and materialize.min.js), plus the additional Material fonts which are normally hosted on google fonts. If you want to host them yourself you just need a few file MaterialIcons-Regular.ttf, MaterialIcons-Regular.woff, and MaterialIcons-Regular.woff2. Plus an additional css file that maps them.

I put all the necessary files together and made a very simple starting example called “bottle_app.py”

All of the files are in a github repo: https://github.com/WaterJuice/BottleMaterializeTemplate

All that is required is python (2 or 3). Simply run

python bottle_app.py

This will start a simple web server running on port 8080. Then browse to http://localhost:8080 (or ip address of your computer) to see the sample page. This is what it looks like on an iPhone.

IMG_8601

The template also contains error pages for common HTTP errors. Eg try browsing to an invalid page and you’ll get a 404 error.

Perhaps this will be useful to you.

Materialize and Bottle are both released using the MIT License, so I have retained the same for my template.

Get the template at https://github.com/WaterJuice/BottleMaterializeTemplate

 

 

 

Oh my OID!

TL;DR: Get a free OID online instantly at https://freeoid.pythonanywhere.com

I have long been interested in GUIDs (UUIDs) which provide a mechanism for unique IDs across multiple domains without any central authority required. The version 1 GUID partitioned a huge 128 bit number space up into unique computer, and also time domains. This meant GUIDs could be generated anywhere anytime at  extremely fast rates without any worry of collision. The downside to version 1 GUIDs are that the computer network address (MAC address) is generally encoded into GUID which leads to privacy issues. Version 4 GUIDs take the approach that 128 bits (122 actually) is so huge that if everyone just picks random numbers the chance of collision is close to 0 anyway. Version 4 GUIDs are now the most common form and they work great as unique IDs. It is not, however, the only approach provide globally unique IDs. I recently stumbled across a different scheme called the OID.

I had been revisiting information on version 3 and 5 GUIDs and looking to see what pre-made namespaces were provided. Disappointingly there are only 4 default namespaces: NameSpace_DNS, NameSpace_URL,NameSpace_OID, and NameSpace_X500.I was curious as to what the NameSpace_OID was for and that led me to discover the Object Identifier (OID).

What is an Object Identifier (OID)

The OID is a globally unique identifier that is guaranteed to be unique by using a tree structure where each part of the tree is responsible for assigning the elements directly under it. This provides a controlled, yet distributed mechanism for assigning OIDs so that they will never collide.

The most common way to display an OID is in “dotted decimal” form. Such as:

1.3.6.1.4.1.37476.9000.79 

This OID is registered to this website. I can subdelegate this anyway I wish by appending a futher dot and number. I am responsible for assigning any futher oids from this one. I can also give one of my sub OIDs or range of sub OIDs to someone else to manage.

So how did this ID get formed?

The first level is controlled by ITU-T and ISO organisations. They have assigned only 3 values. 0 for ITU-T, 1 for ISO, and 2 for joint ITU-T and ISO things. The number follows a path down a tree, which in the case of our example is

  • 1 – ISO
  • 1.3 – Identified organization
  • 1.3.6 – DoD (Department of Defence)
  • 1.3.6.1 – Internet
  • 1.3.6.1.4 – Private
  • 1.3.6.1.4.1 – IANA enterprise numbers
  • 1.3.6.1.4.1.37476 – ViaThinkSoft (IANA number 37476)
  • 1.3.6.1.4.1.37476.9000 – FreeOid
  • 1.3.6.1.4.1.37476.9000.79 – WaterJuice

Because any OIDs I create will be appended on as 1.3.6.1.4.1.37476.9000.79.* there is no chance that someone else will create the same IDs as they will be starting theirs from somewhere else in the tree.

The majority of all OIDs are hanging off the 1.3.6.1.4.1 branch. Anyone can (with a bit of effort) get an IANA “Enterprise Number” for free. You have to register for it and provide details and it takes about a week. Once you have your number you can assign sub OIDs however you want. Obviously no one is supposed to just arbitrarily use someone else’s branch and assume they can just add some branches that they don’t think are being used. The whole scheme breaks down if people do that.

There are other OID branches further up the tree but there are much harder to get attached to one of those. They tend to be used for ISO things, or reserved for countries etc. There are some places that will charge you a fee to get an OID. So generally most organisations get themselves a free IANA number. It is disappointing that the IANA numbers can’t start much higher up, ideally even at the top level. It could have been 3.* for the IANA enterprise numbers. However the biggest problem with the OIDs are how precious people who control the early ones tend to be. ViaThinkSoft provide free ones attached to a sub OID from their IANA number. You just need an email address to get one. One will be assigned and sent in a reasonably short amount of time.

There are also two methods to get an OID instantly by converting a GUID into an OID. This requires no registration or server interaction and can be done at anytime by anyone. However neither of the techniques are particularly satisfactory (I’ll discuss why in a later article) and you have a rather ugly looking OID such as

2.25.324293003071620890871350535745399440863

or

1.2.840.113556.1.8000.2554.62456.36735.23508.16633.39838.4613307.1590751

(These incidentally both represent the GUID {f3f88f7f-5bd4-40f9-9b9e-4664bb1845df} )

These might be okay for automatically generated OIDs that stand alone, but they don’t make a particularly good branch to add a whole sub tree to. I could sub-delegate out that tree and give, for example, 1.2.840.113556.1.8000.2554.62456.36735.23508.16633.39838.4613307.1590751.2 to someone to start their own tree, but its not a great starting point, they would generally prefer to be much higher up the tree.

So the methods available to get an OID seemed to be either generate an obnoxiously large one of your own instantly from a GUID, or wait around for someone to give you a nicer one from their tree. The IANA branch is the best one you can realistically get, but it can be an annoying process getting one from them (and you will start getting spam immediately after you get one as they publish your email address!).

Given that OIDs are just numbers and not precious gems, it seems ridiculous that its not possible just to easily get one without any hassle. The only point of the scheme is to avoid collisions, the whole registering for one and providing identification etc is unnecessary. So I decided I would provide a free service to give a decent OID to anyone and everyone who wants one!

Get a free OID

https://freeoid.pythonanywhere.com

This is a very simple website I setup to issue unique OIDs at the press of a button. There is no registration, email address, or anything required. Simply press the button and a new OID will be generated. I was originally going to attach it to a branch off my 1.3.6.1.4.1.37476.9000.79 OID, but I felt that it would be too far down the tree and there is no reason why anyone shouldn’t be able to easily have a better one. So I registered an IANA number for the purpose and have assigned the following OID to be the branch for these free OIDs

1.3.6.1.4.1.54392.5

If you go to the site you can get an instant OID that will be a sub OID from this one. You might be wondering what the .5 part is for. I have already assigned the earlier ones for a different use that I will detail in a future article).

There are no limits on generating OIDs with this site. It will simply increment an internal counter and give you the next one available. There is no practical limit as to how large the number can grow. I don’t imagine the service will be particularly busy so the numbers aren’t likely to get to enormous.

So feel free to get yourself your very own OID for free right now, or get several if you want. I don’t mind how many you want or what you want to do with them!

 

AES-CBC

The previous two cipher modes of AES I wrote into WjCryptLib were AES-CTR and AES-OFB. Both of these turn AES into stream ciphers. In both cases only the AES block encrypt function is used. So today I add AES-CBC (Cipher Block Chaining) mode to the library. I don’t particularly like CBC as a mode personally, however it is one of the most common modes used so I wanted to include it in the library.

Cipher Block Chaining mode works by XORing the previous cipher block onto the plaintext before performing the block encrypt. An IV is used as the first “previous cipher block”. A change in a byte of plaintext will cause all the following cipher text to be different. A disadvantage of the mode is that it has to work with whole number of blocks (16 bytes in the case of AES). This limitation is usually overcome by padding the last block and keeping a count value of the actual data. There is also a fancier technique called cipher text stealing which reduces the limitation to only requiring a minimum of a a single block. I have not included this technique I my implementation.

CBC is not a stream cipher mode, as in it does not generate a parallel stream of bytes that are then applied (usually with XOR) onto the input stream. CBC uses the block encrypt and decrypt block functions on the input data.

I have released WjCryptLib 2.3.0 which contains AES-CBC.

The relevant source files needed are:

This is free and unencumbered software released into the public domain.

 

Pushing old data off a disc

I often like to clean out deleted data from discs. Especially ones that are going to be recycled and used by other people. The problem I find is that secure wiping programs are just too slow. I don’t need that level of protection, I just want to quickly write over every block on the disc.

Recently I was trying to delete a large (1TB) drive. It had been formatted and I just wanted to fill up all the blocks on it with a huge file and then delete the file. This way I would be fairly confident that every block on the disc had been overwritten. The fastest way is to copy /dev/zero onto a file on disc. However I never feel confident that writing zeros actually overwrites anything. It would be very easy for the underlying device to simply mark the block as all zero rather than physically writing it. I believe the old ZIPDRIVE discs did something like this.

Instead of using /dev/zero the obvious solution is to use /dev/random or /dev/urandom. However these are far slower due to generating cryptographically secure random numbers. This is overkill for what I was trying to do. I just wanted to ensure that something was written. In the end I opted for making some big files withs /dev/random and then appending the file over and over until the disc was full.

It seemed unlikely that the device would be able to detect repeated blocks being written but it still niggled at me. Also it was not a particularly convenient method. I just wanted to run something and leave it until the disc was full. So I wrote a small tool called PushFill. This will keep writing data to a file until it runs out of space.

This uses RC4 to create a random 10Mbyte block of data which it writes to the file. It then writes the same 10Mbytes another 255 times, each time with every byte incremented by 1. After 256 writes (2.5G) it starts again with another 10Mbyte block from the RC4 stream. This way the RC4 generator is only used for a small percentage of the time and therefore does not slow down the writing. The step of incrementing each byte in the block by 1 is barely noticeable.

The advantage of this method is it is very fast, while still making every single block written different. Therefore the underlying system can not do any smart cheats such as noticing repeated blocks (think of how DropBox works, where each unique block is hashed and only physically stored once ever). Additionally the output of RC4 prevents any disc compression being able to use less physical blocks to store the data.

The syntax is simple:

PushFill <filename>

This will create or append to the specified filename. It will keep on writing until the disc is full, or program is aborted (ctrl-c).

Every two seconds the program will display how much it wrote in that time along with its rate. It will also display the total amount written so far and the average rate.

A sample output:

Block:    1.9 GB  Rate:  948.1 MBps  |  Total:    1.9 GB  AvgRate:  948.1 MBps
Block:    2.1 GB  Rate:    1.1 GBps  |  Total:    4.0 GB  AvgRate: 1017.2 MBps
Block:    2.3 GB  Rate:    1.1 GBps  |  Total:    6.3 GB  AvgRate:    1.0 GBps

 

Some systems will cache writes so the first few seconds will show a much higher rate than its actually writing to the disc.

Compiled binaries for the program are available here. The package contains binaries for Windows x64, MacOS x64, Linux x64, and Linux Arm (eg a Raspberry Pi).

Full source code available on GitHub here.

This is free and unencumbered software released into the public domain.

 

AES-OFB

I previously added AES-CTR to my library WjCryptLib. AES-CTR is by far the best way to use AES as stream cipher. However it was not the first mode of operation devised for using a block cipher as a stream cipher. Output-Feedback-Mode (OFB) was one of the original modes of operation specified for the original block ciphers like DES. The way OFB works is to start with an IV the size of a block and repeatedly encrypt it. Each encryption produces another block worth of stream bytes.

When running as a single thread AES-OFB is exactly the same speed as AES-CTR. However it can not be parallelised, nor can the stream be synced to an arbitrary location. So in pretty much every situation AES-CTR is a better choice than AES-OFB. However if you are required to use AES-OFB due to a pre specified protocol then there are times you may need it. I have added AES-OFB to WjCryptLib.

Public domain C source code for AES-OFB:

These depend on the AES block cipher files: