After my post yesterday, my next task was to do the same in .Net. Boy was I rewarded when I used Google to search for "OpenSSL Net". I was presented with "OpenSSL-NET", a managed wrapper for OpenSSL! Interestingly enough, this wasn't the first time I have looked for such a project. In the past, the most I found was people looking for the same. What a reward, I don't have to implement a C# wrapper for OpenSSL :). Now on to the good stuff, I took the work I did yesterday and converted into a C# object oriented sample. To reproduce my efforts, I have packaged all my work here. For simplicity, I've included an OpenSSL-NET dll. One caveat, the current distribution of OpenSSL.NET won't work with the binary, I had to make a few tweaks, such as making the Ssl, SslContext, and some Enums linkable (i.e. internal to public) and then I added a couple of additional Ssl functions to the library. I've sent a patch to the manager and hopefully the changes get imported into the official library.
What does that leave next? Figuring out how to distribute OpenSSL with our libraries for Windows support. Windows presents an interesting problem as it apparently lacks an up-to-date OpenSSL distribution. So with that in mind, I'll probably attempt to integrate our current security stack and have OpenSSL DTLS as an option instead. That way, users who just want a simple to install system don't have to worry about installing OpenSSL and configuring our application to properly link to it.
Also another booger about OpenSSL is that the library names differ between Windows and Linux and there are a TON of functions called by the library. As such, the author has a library for Linux and Windows. One option to resolve this is to use a Windows version and created symbolic links in the local directory from the Linux ssl libraries to Windows ssl names.
I'm really looking forward to removing dependency of our system from our own security stack. It means that we don't have to worry (as much) about something so critical. I don't regret writing the original security stack, because I learned SO much and much of the framework or at least the ideas will be kept from the transition of the old stack to the new. Also I suspect the transition from managed security code to unmanaged will create a much faster system.
To assist in keeping track of my changes to the OpenSSL.NET code, I've put up a git repository at github.
Friday, October 30, 2009
Thursday, October 29, 2009
Memory BIOS and OpenSSL
I spent the past 12 hours or so learning how to "use" OpenSSL with the goal of potentially using it to replace our existing security system. Why? Because security is hard, there are so many ridiculous attacks that can be made that probably have no real effect on anything, but it makes people nervous, like timing attacks that have the ability to decrypt the first few messages in a symmetric exchange. Research is ongoing to discover ways to break security and at the same time to improve protocols and algorithms. I think its all great, but I don't have time to worry about it. Its hard argument to make that creating a new system that takes the best parts of DTLS and IPsec is really the best pathway, if you can mess around with a well supported DTLS implementation to create an IPsec like system. While I didn't find something identical to that, I found that guys over in France have implemented a "P2P" VPN that uses DTLS in a non-standard way. So I suppose it is possible. With the remaining text in this entry, I'll discuss my recent experiment with OpenSSL and DTLS, which was rather quite interesting since I haven't coded in C/C++ in 4.5 years now :).
First off, OpenSSL documentation is horrible, it is spare and spread across the web. Most of the error messages don't really make sense. The requirements and order of operations for creating secure sessions aren't very clear. I used a mixture of sample code and their man pages to figure out what to do. Though there is really little text on DTLS. It turns out for our local testing purpose, it really doesn't matter which security protocol you use.
So onwards to how this whole thing works, we'll assume that you know how to make a self-signed certificate PEM file with an RSA PEM key file, which can be located in "server.pem". Now we can get on to the code, here's where ordering matters. Lesson 1) complete all operations with structures before moving on to others that are created from them.
To keep everything organized, and because I don't know SSL well enough, I created a structure so I can access everything I create:
The bios are the memory bios. The names for read and write are from the perspective of the SSL structure, i.e., read is used to write network traffic from, while write is used to read network traffic. The rest will become clear in the later parts of this post.
So here's how to initialize a server (minimized version):
So what's interesting here? The most annoying issue was the ordering. You need to initialize the CTX with all those options before creating SSL or the SSL won't have the CTX options. That vexed me for an entire night, while I slept for 2 hours. The first two CTX options are clear, we need a certificate and a private key. The following option, SSL_CTX_set_session_id_context, needs to be set to some unique value, apparently not setting could cause a certificate exchange to fail, though I haven't experienced that issue. My favorite was the use of Diffie-Hellman, where I come from, the Diffie-Hellman exchange was handled in the background unless you really want to get in and dirty, not with OpenSSL, you either need to specify a Diffie-Hellman key file, a callback, or use the single use flag. Now we can create the server and since it is a server, we set it up for an accept state. The final components are the creation of memory bios, so that we can abstract the sending mechanism. Memory bios might be non-blocking, but it was explicit anywhere, it works if you don't make them non-blocking, but why risk it? Wow, looking back, that was kind of easy!
The client is even easier:
We start with the same initialization of the CTX block and then for the SSL structure we set it to connect state. The rest is the same as the server.
So now we have usable client and server ssl structure, we need to do some sending between the two, that looks like this:
The system needs to be started, so we start with a SSL_do_handshake on the client ssl, this causes the client to attempt to connect with the server. We use handle_send to facilitate this and abstract it from this component. After 2 round trips, both the client and server have authenticated.
Here's how handle_send works (minimized):
As mentioned above, the write bio contains encrypted data, which can be injected into another read bio to insert data into the SSL structure. If the init is finished and successful and we have data waiting, it most likely is a data message. We can use SSL_read to retrieve the decrypted data.
The last part is the fun part, where we can actually see a message go across:
This causes both the client and server to send to each other the message "HelloWorld!", which will be printed to the screen. SSL_write is SSL_read's counterpart. It is used to insert unencrypted data into the SSL structure so that it can be read from the write bio as encrypted data.
Now we just need to clean up:
There you have it, a clear and concise way to use memory bios! The full source code can be downloaded from here. Due to the girth of the code, the above code is signficantly minimized and not meant to be used as is. For example, functions were squashed and error handling was removed.
So I guess the answer is, yes SSL can be used as a part of an IPsec like system, if you're willing to implement the protocol handling for multiple and parallel sessions. The only other issue that may need to be addressed is the loss of fragmentation and resends due to not using the DGRAM bio, but that's furture work.
First off, OpenSSL documentation is horrible, it is spare and spread across the web. Most of the error messages don't really make sense. The requirements and order of operations for creating secure sessions aren't very clear. I used a mixture of sample code and their man pages to figure out what to do. Though there is really little text on DTLS. It turns out for our local testing purpose, it really doesn't matter which security protocol you use.
So onwards to how this whole thing works, we'll assume that you know how to make a self-signed certificate PEM file with an RSA PEM key file, which can be located in "server.pem". Now we can get on to the code, here's where ordering matters. Lesson 1) complete all operations with structures before moving on to others that are created from them.
To keep everything organized, and because I don't know SSL well enough, I created a structure so I can access everything I create:
typedef struct connection {
BIO* read;
BIO* write;
SSL* ssl;
SSL_CTX* ctx;
} connection;
The bios are the memory bios. The names for read and write are from the perspective of the SSL structure, i.e., read is used to write network traffic from, while write is used to read network traffic. The rest will become clear in the later parts of this post.
So here's how to initialize a server (minimized version):
connection *con = malloc(sizeof(connection));
con->ctx = SSL_CTX_new(DTLSv1_method());
SSL_CTX_use_certificate_chain_file(con->ctx, "server.pem");
SSL_CTX_use_PrivateKey_file(con->ctx, "server.pem", SSL_FILETYPE_PEM);
SSL_CTX_use_certificate_file(con->ctx, "server.pem", SSL_FILETYPE_PEM);
SSL_CTX_set_options(con->ctx, SSL_OP_SINGLE_DH_USE);
SSL_CTX_set_session_id_context(con->ctx, sid, 4);
con->ssl = SSL_new(con->ctx);
SSL_set_accept_state(con->ssl);
SSL_set_verify(con->ssl, SSL_VERIFY_NONE, NULL);
SSL_set_session_id_context(con->ssl, sid, 4);
con->read = BIO_new(BIO_s_mem());
BIO_set_nbio(con->read, 1);
con->write = BIO_new(BIO_s_mem());
BIO_set_nbio(con->write, 1);
SSL_set_bio(con->ssl, con->read, con->write);
So what's interesting here? The most annoying issue was the ordering. You need to initialize the CTX with all those options before creating SSL or the SSL won't have the CTX options. That vexed me for an entire night, while I slept for 2 hours. The first two CTX options are clear, we need a certificate and a private key. The following option, SSL_CTX_set_session_id_context, needs to be set to some unique value, apparently not setting could cause a certificate exchange to fail, though I haven't experienced that issue. My favorite was the use of Diffie-Hellman, where I come from, the Diffie-Hellman exchange was handled in the background unless you really want to get in and dirty, not with OpenSSL, you either need to specify a Diffie-Hellman key file, a callback, or use the single use flag. Now we can create the server and since it is a server, we set it up for an accept state. The final components are the creation of memory bios, so that we can abstract the sending mechanism. Memory bios might be non-blocking, but it was explicit anywhere, it works if you don't make them non-blocking, but why risk it? Wow, looking back, that was kind of easy!
The client is even easier:
connection *con = malloc(sizeof(connection));
con->ctx = SSL_CTX_new(DTLSv1_method());
SSL_CTX_use_certificate_chain_file(con->ctx, "server.pem");
con->ssl = SSL_new(con->ctx);
SSL_set_connect_state(con->ssl);
SSL_set_verify(con->ssl, SSL_VERIFY_NONE, NULL);
con->read = BIO_new(BIO_s_mem());
BIO_set_nbio(con->read, 1);
con->write = BIO_new(BIO_s_mem());
BIO_set_nbio(con->write, 1);
SSL_set_bio(con->ssl, con->read, con->write);
We start with the same initialization of the CTX block and then for the SSL structure we set it to connect state. The rest is the same as the server.
So now we have usable client and server ssl structure, we need to do some sending between the two, that looks like this:
SSL_do_handshake(client->ssl);
handle_send(client, server);
handle_send(server, client);
handle_send(client, server);
handle_send(server, client);
The system needs to be started, so we start with a SSL_do_handshake on the client ssl, this causes the client to attempt to connect with the server. We use handle_send to facilitate this and abstract it from this component. After 2 round trips, both the client and server have authenticated.
Here's how handle_send works (minimized):
void handle_send(connection* sender, connection* receiver)
{
char buffer[1024];
int read = BIO_read(sender->write, buffer, 1024);
int written = read > 0 ? BIO_write(receiver->read, buffer, read) : -1;
if(written > 0) {
if(!SSL_is_init_finished(receiver->ssl)) {
SSL_do_handshake(receiver->ssl);
} else {
read = SSL_read(receiver->ssl, buffer, 1024);
printf("Incoming message: %s", buffer);
}
}
As mentioned above, the write bio contains encrypted data, which can be injected into another read bio to insert data into the SSL structure. If the init is finished and successful and we have data waiting, it most likely is a data message. We can use SSL_read to retrieve the decrypted data.
The last part is the fun part, where we can actually see a message go across:
strcpy(buffer, "HelloWorld!");
SSL_write(client->ssl, buffer, strlen(buffer));
handle_send(client, server);
SSL_write(server->ssl, buffer, strlen(buffer));
handle_send(server, client);
This causes both the client and server to send to each other the message "HelloWorld!", which will be printed to the screen. SSL_write is SSL_read's counterpart. It is used to insert unencrypted data into the SSL structure so that it can be read from the write bio as encrypted data.
Now we just need to clean up:
SSL_free(con->ssl);
SSL_CTX_free(con->ctx);
free(con);
There you have it, a clear and concise way to use memory bios! The full source code can be downloaded from here. Due to the girth of the code, the above code is signficantly minimized and not meant to be used as is. For example, functions were squashed and error handling was removed.
So I guess the answer is, yes SSL can be used as a part of an IPsec like system, if you're willing to implement the protocol handling for multiple and parallel sessions. The only other issue that may need to be addressed is the loss of fragmentation and resends due to not using the DGRAM bio, but that's furture work.
Monday, August 3, 2009
Reassembling the PDU
While I haven't written much lately, it has nothing to do with lack of productivity but rather too much on my plate. Since I've last updated I have:
The biggest thing I've contemplated as of late is the need of allies or at least peers and elders that will speak well of you. One person that I can look to that has some interesting level of respect is my advisor. He rarely speaks, yet people seem to go out of their way to talk with him. Clearly he is at an advantage, since he gets to remain his quiet self and does not have to become overly talkative to obtain respect. There is probably a lot more to the equation than what I can see, but I feel that it largely falls under respecting the other persons work and their knowledge.
In terms of research, I am in the heat of the NSDI work and have hit a few milestones, but due to events outside of my control, I will need to redouble my effort. The individuals that I am working with are steadfast, and I am sure that their contributions will make the breadeth of the paper amazing. I think it will be a very good read and informative for anyone that is interested in VPN technologies. The only tricky part about writing such a work is that there are many fields that are interested in similar virtual networks (communication) and thus there is no doubt going to be overlap that will not easily be identifiable. I think the only way to really deal with this issue is to make sure that the committee members who may potentially review the paper are appropriately commended for their previous work as it relates to the paper. I will admit though, that I am scared with a recent added burden that I may have too much on my hands to satisfactorily complete the paper. Perhaps my advisor or a fellow student will go above and beyond what previous coauthors have.
- Manuevered through the craziness of shepherded papers
- Put together an elite team of peers to develop a NSDI 10 submission
- Began a P2P student discussion group
- Mentored a high school student
- Organized an Archer trip that will have me visiting 3 schools in September
- Started to learn how to hold my tongue when using it would bring no lasting benefit
- Learning to deal with significant disappointment
The biggest thing I've contemplated as of late is the need of allies or at least peers and elders that will speak well of you. One person that I can look to that has some interesting level of respect is my advisor. He rarely speaks, yet people seem to go out of their way to talk with him. Clearly he is at an advantage, since he gets to remain his quiet self and does not have to become overly talkative to obtain respect. There is probably a lot more to the equation than what I can see, but I feel that it largely falls under respecting the other persons work and their knowledge.
In terms of research, I am in the heat of the NSDI work and have hit a few milestones, but due to events outside of my control, I will need to redouble my effort. The individuals that I am working with are steadfast, and I am sure that their contributions will make the breadeth of the paper amazing. I think it will be a very good read and informative for anyone that is interested in VPN technologies. The only tricky part about writing such a work is that there are many fields that are interested in similar virtual networks (communication) and thus there is no doubt going to be overlap that will not easily be identifiable. I think the only way to really deal with this issue is to make sure that the committee members who may potentially review the paper are appropriately commended for their previous work as it relates to the paper. I will admit though, that I am scared with a recent added burden that I may have too much on my hands to satisfactorily complete the paper. Perhaps my advisor or a fellow student will go above and beyond what previous coauthors have.
Wednesday, June 17, 2009
First conference level publication
Just the other day, I got some excellent news: My publication entitled "On the Design of Scalable, Self-Configuring Virtual Networks" was accepted into SuperComputing 09 with an acceptance rate of ~22.6%. Sadly it was shepherded, so I have to do some tweaks to it, but hopefully it won't be too painful. The only thing that may make it painful, is that all I have left is a tex file and none of the original images from the submission. The paper can be found on the sc09 web page:
http://scyourway.nacse.org/conference/view/pap152
http://scyourway.nacse.org/conference/view/pap152
Tuesday, June 9, 2009
Writing Application Entry Points
A long time ago, I spent a few days working on an option parser for C# tailored around my application, SimpleNode. The name was quite misleading as SimpleNode allowed users to make many nodes, act as user interface into the system, and probably a handful of other behaviors that have long since escaped me. It was largely unwieldly and probably not horribly useful as a learning tool nor very reusable. So I chopped everything out and went with thin applications that were tailored around objects that had simple, specific tasks. This allowed better integration at the cost of dumbing down the command-line interface and having multiple applications that had very similar behavior.
I typically desire to do things right the first time and to not come back later. When I don't get things quite right, I acknowledge the problem but add it to my priority queue, so I may or may not ever get back to it. Recently, we've had an increase of interest in our system along with the a stage 1 completion of my groupvpn, so I felt it fitting to improve the usability of our project. Lucky for me, one my peers in the lab is very interested in technologies and was able to point out NDesk.Options. I think the results are pretty good. What displeases me is that this sort of interface is probably pretty generic and so if a user wants a similar frontend, they'd have to copy and paste 100 lines of code to get the same thing, which may be ridiculous if what they do only requires a few lines, like this.
Originally, the posts title was going to be "Polymorphic Entry Points." I'd love to have something like that. I guess this can be achieved by adding helper methods to each class we instantiate that requires more than 5 lines, but I feel there needs to be a clear distinction between application entry point and classes as I think it makes it easier for users to understand how everything fits together. Maybe I'm wrong and I'm probably over thinking this :).
I typically desire to do things right the first time and to not come back later. When I don't get things quite right, I acknowledge the problem but add it to my priority queue, so I may or may not ever get back to it. Recently, we've had an increase of interest in our system along with the a stage 1 completion of my groupvpn, so I felt it fitting to improve the usability of our project. Lucky for me, one my peers in the lab is very interested in technologies and was able to point out NDesk.Options. I think the results are pretty good. What displeases me is that this sort of interface is probably pretty generic and so if a user wants a similar frontend, they'd have to copy and paste 100 lines of code to get the same thing, which may be ridiculous if what they do only requires a few lines, like this.
Originally, the posts title was going to be "Polymorphic Entry Points." I'd love to have something like that. I guess this can be achieved by adding helper methods to each class we instantiate that requires more than 5 lines, but I feel there needs to be a clear distinction between application entry point and classes as I think it makes it easier for users to understand how everything fits together. Maybe I'm wrong and I'm probably over thinking this :).
Thursday, June 4, 2009
Dissertation Step 1
Main focus point: P2P VPN architecture with emphasis on scalablity, user-friendliness, and performance.
Scalability:
Scalability:
- Effects of latency on VPN connectivity
- Optimal peer count, cost per peer connection
- Trusting a P2P infrastructure
- VPN Router model
- Ability to deploy and manage large, distributed infrastructures
- GroupVPN - ability to use groups to manage VPN connectivity
- Userspace based, non-root VPN solution
- Use of proximity neighbor and shortcut selection
- Hybrid-based VPN model
- Inverted P2P VPN model
Wednesday, June 3, 2009
Out with the old, in with the new.
On Friday at 6 PM EDT, my computer with components of varied ages all over 2.5 years did the unthinkable, died. I'm not really sure the entire process, the motherboards capacitors near the PSU plug were all blown and may have been leaking for a while and the PSU made a loud bang as I tried to see if it was broken. Using another PSU, I was able to boot the computer, but I had no USB nor fans working, I even replaced the broken CAPs to no avail. The PSU of course was dead, but it so happened to destroy two hard drives as well, both of the western digital first generation SATA 3.0. Luckily, I may be able to recover the data if it happens that only the boards are dead and the drives internals are fine.
Around 10 PM, I e-mailed my advisor with the situation and 1 hour later, he had a one line e-mail telling me to price out a machine and aim for less than 1K. Monday morning we ordered the machine, and Wednesday evening, I had a fully functioning system.
I will admit that it really sucks that I lost a lot of time, potentially lot's of data, and my own computer, but on the flip side, I am very happy to be blessed to have a considerate professor who recognizes my need for a high performance desktop computer. The machine is amazing, a new Core i7 920 with 6 GB of RAM. Even typing in commands at the terminal seemed faster than ever before.
As a lesson to this all, I have decided to employ a mechanism of maintaining the same (or nearly the same) home directory on all machines via rsync. When I work on a remote machine, I will begin my session with a pull from the desktop and when I finish, a push. To perform this, I've written to wrapper scripts to rsync, one called push and the other pull.
Push:
rsync -Cvauz --force --delete -e "ssh -i /home/name/.ssh/key" /home/name/ name@desktop:/home/name/
Pull:
rsync -Cvauz --force --delete -e "ssh -i /home/name/.ssh/key" name@desktop:/home/name/ /home/name/
C - uses a ~/.cvsignore, though I could just use an ignore file. By doing this, I can have files that aren't synced, like my mozilla firefox plugins and other cache files.
v - verbose, probably unnecessary
a - archive, keep owner, user, group, modification times, permissions, symbolic lnks, and devices, as well as recurse directories
u - update, skip files that are only newer on the receiver
z - compress files prior to transferring
force - overwrite files with directories and vice-versa
e ssh - how to execute the command
Notice the ending forward slash ("/"), that is necessary or it will actually copy the directory and place it at the destination (so you'd get /home/name/name)
Beyond backing up my data, changes to my configuration files will be moved around with little effort from myself.
Around 10 PM, I e-mailed my advisor with the situation and 1 hour later, he had a one line e-mail telling me to price out a machine and aim for less than 1K. Monday morning we ordered the machine, and Wednesday evening, I had a fully functioning system.
I will admit that it really sucks that I lost a lot of time, potentially lot's of data, and my own computer, but on the flip side, I am very happy to be blessed to have a considerate professor who recognizes my need for a high performance desktop computer. The machine is amazing, a new Core i7 920 with 6 GB of RAM. Even typing in commands at the terminal seemed faster than ever before.
As a lesson to this all, I have decided to employ a mechanism of maintaining the same (or nearly the same) home directory on all machines via rsync. When I work on a remote machine, I will begin my session with a pull from the desktop and when I finish, a push. To perform this, I've written to wrapper scripts to rsync, one called push and the other pull.
Push:
rsync -Cvauz --force --delete -e "ssh -i /home/name/.ssh/key" /home/name/ name@desktop:/home/name/
Pull:
rsync -Cvauz --force --delete -e "ssh -i /home/name/.ssh/key" name@desktop:/home/name/ /home/name/
C - uses a ~/.cvsignore, though I could just use an ignore file. By doing this, I can have files that aren't synced, like my mozilla firefox plugins and other cache files.
v - verbose, probably unnecessary
a - archive, keep owner, user, group, modification times, permissions, symbolic lnks, and devices, as well as recurse directories
u - update, skip files that are only newer on the receiver
z - compress files prior to transferring
force - overwrite files with directories and vice-versa
e ssh - how to execute the command
Notice the ending forward slash ("/"), that is necessary or it will actually copy the directory and place it at the destination (so you'd get /home/name/name)
Beyond backing up my data, changes to my configuration files will be moved around with little effort from myself.
Subscribe to:
Posts (Atom)