Analysis of the OpenPGP and OTR protocols

The OTR Protocol

There are various problems with the OpenPGP protocol. For one, after the private key of a user is stolen, a perpetrator is able to decrypt all previously sent messages. Another problem is more subtle: every message sent using the protocol is signed by the sender with his private key. This means that, when given a plain text message and its signature, anyone is able to verify the message was sent by sender. This property, called reputability, is a nice property when dealing with contracts and other formal messages.

When trying to keep a secret conversation, however, we generally don't want to be accredited for our messages. In fact, we would like to be able to deny any contact with another contact. This property, called plausible deniability, or non-reputability, requires a different use of message signing, one that does not use signatures using a private key, but a short-lived key that expires and is forgotten after a session ended.

These shortcomings are addressed in the Off-The-Record messaging protocol. This protocol, making use of chat sessions, in which temporary keys are used, does not use the private and public keys for signing, but rather uses symmetric keys generated at the start of the session. Using symmetric keys makes it unclear who actually sent a message, as we will see later.

In short, OTR has the following properties:

Perfect Forward Secrecy

One of the key features of the OTR protocol is perfect forward secrecy (also: security). This means that, even though the private key of one of the users is compromised, it is impossible for a malicious attacker to gain information of previous conversations -- even if the attacker has the encrypted messages. We created a short example to demonstrate the forward secrecy problem. The applet can be viewed in a pop-up by clicking on the image:

openpgp.

Diffie - Hellman key exchange

Even though they are not used for actual encryption of messages, each user in the conversation must have a public and private key. The OTR-protocol uses the RSA scheme described earlier. Using the keys, chatters can verify each others' identity before the actual encrypted transcript is started. The actual conversation takes place not with these keys, but with a fresh, symmetric, key, created just for this session. Though we said earlier it is difficult to establish a shared secret between two parties, this can in fact be done quite easily. The difficult part lies in the verification of the user.

In 1976, Whitfield Diffie and Martin Hellman created a simple algorithm that is able to supply both end users with a shared secret, even though this secret is never sent across the communications channel, and there is no way to calculate the secret using the data sent. This is done by using the simple mathematical fact:

gxy = gyx

Using this property, Alice can send her g^x to Bob, and Bob can send his g^y to Alice. This way, using their own x and y, they both can compute g^xy.

However, y is very easily computable from g^y (indeed, the correct answer is g log g^y), thus the computation of g^xy would be easy for someone knowing g, g^y and g^x. This problem is solved by not using gy, but rather gy mod n, where n is some large integer number, preferably prime. Calculating x from gx mod n is a very difficult computation, also called the discrete logarithm problem, making it unlikely a perpetrator is able to calculate the value of g xy. Thus, after the protocol run is done, both Alice and Bob have a shared secret key. The Diffie-Hellman is not without its own problems though. We will show a straightforward problem now, and a more intricate example later on.

Using the symmetric key established by the Diffie-Hellman protocol, it is possible to establish perfect forward secrecy, since an eavesdropper will not be able to gain access to the symmetric key used even if he can decrypt all messages encrypted with public keys (in fact, there are no such messages in the OTR-protocol).However, next to using Diffie-Hellman, we need to establish the identity of the chatters, since otherwise gaining control over the conversation is very easy for attacker:

A -> B : gxA
// M intercepts the message
M -> B : gxM
B -> M : gyB
M -> A : gyM

Now, since attacker M has chosen his own gxM with B, and his own gyM with A, he can communicate with A and B even though they might think they are talking directly to each other. Thus, establishing a secret is worth nothing without actually verifying that the person choosing the other value is actually the person you wish to talk to.

This is where the public / private key pairs do their work. OTR uses the following protocol:

Here, {X}K-A function stands for the encryption of X with the private key of A. Since the values are encrypted using the private keys of the two users, each one can verify that the value sent is actually created by the other user. Now, attacker M cannot simply intercept the messages and substitute his own, since he cannot create {gxM}K-A. However, he might still do some other damage, as we will see later on.

After a certain amount, A and B discard their keys and another key is created. When their keys are discarded, it's not even possible for themselves to decrypt messages sent with the key. Thus, when an attacker compromises their system, he will be unable to decrypt past messages.

Message authenticity

One of the other points demonstrated before is the necessity of verifying that a message actually came from a said source. Since signing a message with a private key makes it undeniable that a message came from that source, we need to find another way to sign the message.

An alternative way to do this is with a Message Authentication Code. Using a shared code, Alice and Bob can verify the message was either sent by themselves or the other party. OTR does this by using the hash of the shared key as the shared message key. This is done for practical purposes; since they both have a shared session key, taking the hash of it will create another key. Furthermore, the session key cannot be derived from the message key (since hashing is a one-way function). Since the MAC is also calculated based on the message contents, the receiving party knows the message has not been tainted, since otherwise the MACs wouldn't match. Most MAC's are implemented by taking the message that should be authenticated, concatenating the shared secret to it, and then calculating the hash value over the whole data.

Bob can thus verify a message came from Alice by taking the messsage and the key, and checking whether the hash values match. Since Bob presumably knows he didn't send it himself, it must have come from Alice (since she is the only other person knowing the key). Eve, on the other hand, does not know the shared message key, and thus cannot even verify that the message was sent by either Bob or Alice. To go further, even Bob cannot prove to others that Alice sent a message, since he could just as well have made the MAC himself.

In fact, OTR describes the releasing of the MAC key as part of the protocol. After a new session key has been established, the old message authentication key is published in plain text over the channel. This means that anyone is able to verify the messages were being signed by this key. However, this also means anyone can create a message and sign it himself. Thus, after the release of the key, messages can't be authenticated securely with that key. We will demonstrate this plausible deniability later on.

Example of chat session

To make things more clear, it might be educational to show an example of a chat session. For this reason, we have created an applet demonstrating the use of OTR. It can be found by clicking on the image:

otr