Thursday, October 29, 2009

Memory BIOS and OpenSSL

I spent the past 12 hours or so learning how to "use" OpenSSL with the goal of potentially using it to replace our existing security system. Why? Because security is hard, there are so many ridiculous attacks that can be made that probably have no real effect on anything, but it makes people nervous, like timing attacks that have the ability to decrypt the first few messages in a symmetric exchange. Research is ongoing to discover ways to break security and at the same time to improve protocols and algorithms. I think its all great, but I don't have time to worry about it. Its hard argument to make that creating a new system that takes the best parts of DTLS and IPsec is really the best pathway, if you can mess around with a well supported DTLS implementation to create an IPsec like system. While I didn't find something identical to that, I found that guys over in France have implemented a "P2P" VPN that uses DTLS in a non-standard way. So I suppose it is possible. With the remaining text in this entry, I'll discuss my recent experiment with OpenSSL and DTLS, which was rather quite interesting since I haven't coded in C/C++ in 4.5 years now :).

First off, OpenSSL documentation is horrible, it is spare and spread across the web. Most of the error messages don't really make sense. The requirements and order of operations for creating secure sessions aren't very clear. I used a mixture of sample code and their man pages to figure out what to do. Though there is really little text on DTLS. It turns out for our local testing purpose, it really doesn't matter which security protocol you use.

So onwards to how this whole thing works, we'll assume that you know how to make a self-signed certificate PEM file with an RSA PEM key file, which can be located in "server.pem". Now we can get on to the code, here's where ordering matters. Lesson 1) complete all operations with structures before moving on to others that are created from them.

To keep everything organized, and because I don't know SSL well enough, I created a structure so I can access everything I create:

typedef struct connection {
BIO* read;
BIO* write;
SSL* ssl;
SSL_CTX* ctx;
} connection;

The bios are the memory bios. The names for read and write are from the perspective of the SSL structure, i.e., read is used to write network traffic from, while write is used to read network traffic. The rest will become clear in the later parts of this post.

So here's how to initialize a server (minimized version):

connection *con = malloc(sizeof(connection));
con->ctx = SSL_CTX_new(DTLSv1_method());
SSL_CTX_use_certificate_chain_file(con->ctx, "server.pem");
SSL_CTX_use_PrivateKey_file(con->ctx, "server.pem", SSL_FILETYPE_PEM);
SSL_CTX_use_certificate_file(con->ctx, "server.pem", SSL_FILETYPE_PEM);
SSL_CTX_set_options(con->ctx, SSL_OP_SINGLE_DH_USE);
SSL_CTX_set_session_id_context(con->ctx, sid, 4);

con->ssl = SSL_new(con->ctx);
SSL_set_accept_state(con->ssl);
SSL_set_verify(con->ssl, SSL_VERIFY_NONE, NULL);
SSL_set_session_id_context(con->ssl, sid, 4);

con->read = BIO_new(BIO_s_mem());
BIO_set_nbio(con->read, 1);
con->write = BIO_new(BIO_s_mem());
BIO_set_nbio(con->write, 1);
SSL_set_bio(con->ssl, con->read, con->write);

So what's interesting here? The most annoying issue was the ordering. You need to initialize the CTX with all those options before creating SSL or the SSL won't have the CTX options. That vexed me for an entire night, while I slept for 2 hours. The first two CTX options are clear, we need a certificate and a private key. The following option, SSL_CTX_set_session_id_context, needs to be set to some unique value, apparently not setting could cause a certificate exchange to fail, though I haven't experienced that issue. My favorite was the use of Diffie-Hellman, where I come from, the Diffie-Hellman exchange was handled in the background unless you really want to get in and dirty, not with OpenSSL, you either need to specify a Diffie-Hellman key file, a callback, or use the single use flag. Now we can create the server and since it is a server, we set it up for an accept state. The final components are the creation of memory bios, so that we can abstract the sending mechanism. Memory bios might be non-blocking, but it was explicit anywhere, it works if you don't make them non-blocking, but why risk it? Wow, looking back, that was kind of easy!

The client is even easier:

connection *con = malloc(sizeof(connection));
con->ctx = SSL_CTX_new(DTLSv1_method());
SSL_CTX_use_certificate_chain_file(con->ctx, "server.pem");
con->ssl = SSL_new(con->ctx);
SSL_set_connect_state(con->ssl);
SSL_set_verify(con->ssl, SSL_VERIFY_NONE, NULL);

con->read = BIO_new(BIO_s_mem());
BIO_set_nbio(con->read, 1);
con->write = BIO_new(BIO_s_mem());
BIO_set_nbio(con->write, 1);
SSL_set_bio(con->ssl, con->read, con->write);

We start with the same initialization of the CTX block and then for the SSL structure we set it to connect state. The rest is the same as the server.

So now we have usable client and server ssl structure, we need to do some sending between the two, that looks like this:

SSL_do_handshake(client->ssl);
handle_send(client, server);
handle_send(server, client);
handle_send(client, server);
handle_send(server, client);

The system needs to be started, so we start with a SSL_do_handshake on the client ssl, this causes the client to attempt to connect with the server. We use handle_send to facilitate this and abstract it from this component. After 2 round trips, both the client and server have authenticated.

Here's how handle_send works (minimized):

void handle_send(connection* sender, connection* receiver)
{
char buffer[1024];
int read = BIO_read(sender->write, buffer, 1024);
int written = read > 0 ? BIO_write(receiver->read, buffer, read) : -1;

if(written > 0) {
if(!SSL_is_init_finished(receiver->ssl)) {
SSL_do_handshake(receiver->ssl);
} else {
read = SSL_read(receiver->ssl, buffer, 1024);
printf("Incoming message: %s", buffer);
}
}

As mentioned above, the write bio contains encrypted data, which can be injected into another read bio to insert data into the SSL structure. If the init is finished and successful and we have data waiting, it most likely is a data message. We can use SSL_read to retrieve the decrypted data.

The last part is the fun part, where we can actually see a message go across:

strcpy(buffer, "HelloWorld!");

SSL_write(client->ssl, buffer, strlen(buffer));
handle_send(client, server);

SSL_write(server->ssl, buffer, strlen(buffer));
handle_send(server, client);

This causes both the client and server to send to each other the message "HelloWorld!", which will be printed to the screen. SSL_write is SSL_read's counterpart. It is used to insert unencrypted data into the SSL structure so that it can be read from the write bio as encrypted data.

Now we just need to clean up:

SSL_free(con->ssl);
SSL_CTX_free(con->ctx);
free(con);

There you have it, a clear and concise way to use memory bios! The full source code can be downloaded from here. Due to the girth of the code, the above code is signficantly minimized and not meant to be used as is. For example, functions were squashed and error handling was removed.

So I guess the answer is, yes SSL can be used as a part of an IPsec like system, if you're willing to implement the protocol handling for multiple and parallel sessions. The only other issue that may need to be addressed is the loss of fragmentation and resends due to not using the DGRAM bio, but that's furture work.

10 comments:

  1. Hi. I've been having trouble with ssl for weeks
    and wondering as well if I should bother to use it. Thanks for writing about your experiences.
    I used bzip2 for compression and worked on that first. I wasn't sure how it would go, but it has proven to be a breeze to use and understand compared to open ssl.

    Brian Wood
    http://webEbenezer.net

    ReplyDelete
  2. Just saw this post, sorry for the delay. I feel mixed about OpenSsl. I'm about to start writing my latest experiences with it and maybe that will give you some insight. In general, it is easier than writing your own stack, but you still have to write a LOT to get it functioning.

    ReplyDelete
  3. Thanks for the article, it really helped me to get the memory bios wokring.

    However, do you know, why does the server receive a garbage message if I send it a string longer than "HelloWorld!". The server seems to receive the string but also a lot of garbade data after that.

    ReplyDelete
  4. Thanks :).

    There are a few "bugs" in the code:

    1) Strings are not null terminated, they should be, which is why you are seeing garbage after your string (that's other data that's on the character array)
    2) We should probably copy the buffer array to another char array using strcp and limit it from 0 to read (the amount of unencrypted data pulled from the SSL "object")

    Let me know if that helps.

    Cheers,
    David

    ReplyDelete
  5. the hostname in the url for downloading is missing .org

    Thanks for the info on DTLS. Documentation is definitely lacking.

    ReplyDelete
  6. How to create DTLS sessions for two channel.This two session should be run parallel if possible let me Know the code to implement

    I am implementing CAPWAP split MAC stack , for doing this one i have taken CAPWAP-0.93.3 opensource as a reference.In opensource they have created DTLS session for controll chennal .Same way i am trying to do for my data channal creation , but i am facing some problem related to TLS session resumption .Let me know how to use this TLS session resumption in my data channel code.Here i want to know whether i should use previous session( DTLS session of controll channel ) ,whatever they have created for controll chennal OR i have to shutdown the previous session and need to create new session for data channel ?

    ReplyDelete
  7. Unfortunately my use of DTLS was pretty limited to what I described. I think I found a few issues as time went on, but nothing I was not able to work around :). In my opinion, session resumption seems a little bit insecure, I would opt for constructing a new session and reauthenticate all parties as necessary.

    David

    ReplyDelete
  8. If you want to protect your data you should use some secure file share for secured data exchange. This is really working thing. I hope it will help you!

    ReplyDelete