Tag: shell

📟

I wanted to learn more about how TOTP works – those six digit codes which are often provided by «Authenticator apps» on mobile phones and used as a second factor for online authentication. My goal was to make a simple command line client which could provide me with such codes for a given credential id. I wanted to program most of the details myself, so that in the end, I would understand it better.

There are many like it, but this one is mine.

The essence of TOTP

TOTP, or Time-based One Time Passwords, is a method to securely supply proof of possession of a shared secret between two parties. A valid proof will increase confidence in the client’s identity claim when used for authentication.

The proof, typically in the form of a passcode, is only valid for a short amount of time, and the code itself does not reveal anything about the shared secret. This significantly reduces the impact of stolen or leaked passcodes and makes it harder to exploit by a malicious third party.

The basic building blocks of TOTP are:

1. A shared secret between the client and the server, referred to as the key.
2. A time-based factor, referred to as the time counter (a 64 bit ever increasing integer).
3. The application of a cryptographic hashing function on a specific combination of the key and the time counter.
4. The extraction of a user friendly passcode based upon the output of the hashing function.

The key is provided as part of the QR code when setting up an authenticator app. The phone app stores the secret key and will use it as a basis to generate passcodes. The server also stores this secret, associated with the identity, so that it can validate provided codes upon authentication.

Relevant standards

TOTP, itself described in RFC 6238, is based upon:

1. HOTP: An HMAC-Based One-Time Password Algorithm, described in RFC 4226. TOTP modifies the HOTP algorithm by using the UNIX epoch time as basis for the moving factor or counter.
2. HOTP again is based upon HMAC: Keyed-Hashing for Message Authentication, described in RFC 2104. HMAC is a well known message authentication code algorithm.

Implementation

I chose to write this as shell script code. But there are some good reasons why that is not the most suitable language for this kind of tool:

1. You need to deal with binary data, and shell scripts cannot safely store and manipulate arbitrary byte arrays in variables without some sort of encoding. This is mainly due to null bytes being treated as the string terminator character.
2. Unless implemented purely in bash (only using bash built-ins), which would be an arduous task, you’ll need to call out to external commands, and there is risk of exposing secrets to other processes running on the same host, as command line arguments.

Getting started with shell code requires basically zero effort and time, which meant I could spend most of this evening project actually thinking about TOTP. (Also, I enjoy doing things in weird ways, just for the fun of it.)

Requirements

The shell script code relies on the following commands:

1. openssl – the famous swiss army knife of command line cryptography.
2. base32 – for working with base32-encoded data, which is the typical carrier format for TOTP secret keys.
3. xxd – work with binary data, encode as hexadecimal digits and decode to raw bytes.

(They are installed by default in a typical Ubuntu/Debian Linux distribution.)

Functional building blocks

1. Decoding base32-data

A TOTP secret key is a random string of bytes that is typically encoded in base32, so we’ll need some way to decode this representation, so that we can work with the raw bytes. Base32 encodes 5 bits of data per digit (2^5 = 32), and the most common alphabet uses the characters A-Z2-7 to represent the numbers 0 through 31. This encoding scheme has interesting properties that you can read more about on Wikipedia.

# Arg 1: base32-encoded value, does not need to be padded.
# Outputs to stdout the raw decoded value.
function base32_decode() {
    local value=$(echo -n "$1"|tr 018a-z OIBA-Z)
    if [ $((${#value} % 8 )) -gt 0 ]; then
        local i pad
        pad=$((8 - ${#value} % 8))
        for ((i=0; i<$pad; i++)); do value+='='; done
    fi
    echo -n "$value"|base32 -d
}

This function will decode a base32-encoded value using the base32 command line tool. That command requires input to be padded to a length that is a multiple of 8, so we make sure to add that if necessary. Padding is often left out for TOTP keys, as it is not strictly required for decoding. The function also normalizes the base32 alphabet using the tr command, making sure that all characters are upper-case and swaps the characters [018]¹ with [OIB].

The result of the decoding operation is raw binary data that is printed to stdout.

2. The HMAC-SHA1 calculation

The basis for a TOTP passcode is the result of applying an HMAC function with the secret key and a time based counter as the message. Specifically, the HMAC-SHA-1 calculation is used for TOTP².

# Arg 1: hash function identifier
# Arg 2: base32-encoded message string
# Arg 3: base32-encoded key string
# Outputs result as hex characters, which is the HMAC code.
# Output length depends on chosen hash function.
function hmac() {
    local hexkey=$(base32_decode "$3"|xxd -p|tr -d \\n)
    base32_decode "$2" | \
        openssl $1 -hex \
                   -mac HMAC \
                   -macopt hexkey:$hexkey | \
        awk '{printf("%s",$2)}'
}

This function can accept any hash function, key and message. Binary input arguments $2 and $3 must be encoded as base32. It starts by decoding the secret key using our previously defined base32_decode() function. This binary key is transformed to hexadecimal digits before we store it in a shell variable, accomplished with the xxd tool.

Then the message is decoded and piped directly to openssl, which does most of the hard work. The key is supplied as an argument for the -macopt option. The -hex option is specified, so that the binary result is output as hexadecimal digits, which are safe to pass around in a shell script.

The result of the HMAC calculation³ when using SHA-1 as the hash function is 160 bits of data, which is printed to stdout as 40 hexadecimal characters.

Notice that whenever we deal with binary data, it’s either encoded as string safe characters or piped as a stream of bytes. So we avoid the shell ever interpreting or handling raw bytes as strings, which would not be safe.

3. Calculating the TOTP passcode

The third and final function uses the previously defined hmac() shell function to perform the hash calculation, then it reduces the result to a six digit decimal passcode, according to the HOTP specification. The function accepts as its only input a base32-encoded TOTP secret key. Together with a properly set system clock, this is all we need to generate valid passcodes !

# Arg 1: base32-encoded TOTP secret key
# Outputs a 6 digit decimal TOTP code to stdout.
# SHA1 is used as hashing algorithm.
function totp() {
    declare -r totp_period_seconds=30
    local key_base32=$1 time_counter t_base32
    time_counter=$(($(date +%s) / $totp_period_seconds))
    t_base32=$(
        printf %016x $time_counter | xxd -r -p | base32
    )

    local hmac_hex hotp_offset hotp_code

    # 40 hexchars (160 bits or 20 bytes):
    hmac_hex=$(hmac sha1 $t_base32 $key_base32)

    # extraction offset is lowest/rightmost 4 bits
    # of 160 bit sha1 (multiplied by 2 for hex offset):
    hotp_offset=$(( 0x${hmac_hex:39:1} * 2 ))

    # Extract 32 bit unsigned int at offset:
    hotp_code=$((
       0x${hmac_hex:$hotp_offset:8} & 0x7fffffff
    ))

    printf %06d $(($hotp_code % 1000000))
}

The time based part of TOTP is just the UNIX epoch time in seconds divided by 30 (integer division). So the time counter increases by 1 every 30 seconds. This is what we use as the HMAC message and is how passcodes vary over time. However, the number must be represented in binary form, as a 64 bit integer. To accomplish this, we use printf to convert the number to 16 hexadecimal characters, then xxd to decode those into 8 raw bytes, before finally base32-encoding the result. It is stored in the variable $t_base32.

Next, the hmac() function is invoked, providing it with the message and key, and its output is stored in the variable $hmac_hex.

Finally, we must reduce the 160 bits (20 bytes) of data into a user friendly six digit decimal passcode according to the HOTP specification. This is accomplished in three steps:

1. Look at the 4 least significant bits of the hash result and interpret those as an integer byte offset. It will be a number between 0 and 15.
2. Indexing into the result as a byte array with the offset extracted in step 1, extract 32 bits (4 bytes) and interpret those as an unsigned integer (big endian).
3. Perform modulus 1 000 000 on the number obtained in step 2, which gives us a number between 0 and 999 999. Zero-pad if necessary up to six digits. This is your passcode.

We have the hash result available as hexadecimal characters in $hmac_hex, and the offset will be the last hex digit (4 bits). Since each byte is represented by two hexadecimal digits, we simply multiply the extracted offset by 2. We use shell substring syntax and arithmetic evaluation to perform decoding of hexcharacters (notice the 0x-prefix added to numbers). For the most significant byte, we mask out the left-most bit, which is done to make the number positive (unsigned).

The function prints the passcode to stdout using printf, performing the modulus operation using shell arithmetic evaluation.

Testing the implementation

Simply pasting all three previous function definitions into a terminal with a bash shell will allow us to test the implementation.

# Generate a 10 byte random TOTP secret key
$ dd if=/dev/random bs=1 count=10|base32
XCGVLAD4EMLOV72B
10+0 records in
10+0 records out
10 bytes copied, 6.3274e-05 s, 158 kB/s

# test the totp() function: 
$ totp XCGVLAD4EMLOV72B
532405

I have successfully used this TOTP implementation for authentication at the following service providers:

• Microsoft services (authenticator app)
• Github
• ForgeRock Identity Platform (authenticator app)
• https://www.verifyr.com/en/otp/check

The TOTP secret key is provided as part of a special otpauth://totp/.. URI scheme in a QR code. There is usually an option to show the secret string separately, which you can use to capture it. Otherwise, just decoding the QR code with a photo app will show the data.

This TOTP-implementation is packaged as a single shell script on Github, where it has been adapted to read keys from stdin for security reasons.

Security considerations

You should in general avoid providing TOTP secrets as command arguments. There is both a risk of it being stored insecurely in your shell history and being exposed to other processes on the system while the command runs. (Calling a shell function natively does not have this problem, because there is no external process involved.)

The simplest way to avoid this is to make the command read the TOTP secret from stdin or a file. Make sure to store TOTP secrets securely, using for instance gpg or a keyring manager that encrypts it. Only decrypt temporarily when you need to generate a passcode for an identity. This can of course all be automated by writing some suitable shell script code, which then uses the totp() function internally.

Please also note that the secret is exposed briefly as a command argument in this implementation, because I found no other way to provide the key to openssl for the HMAC calculation. This is not a big deal, unless you run it on a shared system where other users are present.

Footnotes

1. This is all about being lenient in what you accept. You’ll likely never see a TOTP encoded key with the characters [018] in them, because it’s non-standard. ↩︎
2. The specification allows varying some parameters, depending on security requirements, like the hashing function. But SHA-1 is currently the most common. ↩︎
3. You can read more about how HMAC is calculated on Wikipedia:
   https://en.wikipedia.org/wiki/HMAC#Definition. ↩︎