OpenSSL "Heartbleed" bug: what's at risk on the server and what is not
A few days ago, a critical bug was found in the common OpenSSL library. OpenSSL is the library that implements the common SSL and TLS security protocols. These protocols facilitate the encrypted tunnel feature that secure services – over the web and otherwise – utilize to encrypt the traffic between the client (user) and the server.
The discovery of such a security bug is a big deal. Not only that OpenSSL is very common, but the bug that was found is one that can be readily exploited remotely without any privilege on the attacker’s side. Also, the outcome of the attack that is made possible is devastating. Exploiting the bug allows an attacker to obtain internal information, in the form of memory contents, from the attacked server or client. This memory space that the attacker can obtain a copy of can contain just about everything. Almost.
There are many essays and posts about the “everything” that could be lost, so I will take the optimistic side and dedicate this post to the “almost". As opposed to with other serious attacks, at least the leak is not complete and can be quantified, and the attack is not persistent.
I will focus on the server as the target of the attack.
Say an attacker exploits the newly discovered bug, and starts dumping out contents of memory addresses from your server. The bug allows to exfiltrate 64K at a time, but multiple iterations are possible to exfiltrate as much data as needed. This dump can contain anything that is stored in memory. The memory involved is the process space of the application or web server that happened to call OpenSSL.
What is there as loot for the attacker? In essence, there is all the state information of the application, including that of the web server process, if the application is web-based. The actual state information depends on what the application or web server is doing, but at a minimum it contains:
All data (including keys) pertaining to the secure connection itself, such as the permanent private key of the server.
All communicated data: the data that is being exchanged between the client and the server.
Potentially any other data that the attacked application processes in the current session, including inputs, outputs, and passwords.
This is not to be taken lightly. This is a lot. Notwithstanding, let us see what was not put at risk. There are three resources that are clearly left out of scope for the attacker:
The contents of disks and storage, which were not loaded into memory during the session of the attack. The bug allows to map memory contents, but not to access the file-system, and not to introduce commands that can load parts of the file-system into memory from where they can be exfiltrated.
Memory contents of other applications and processes on the system, including that of processes that happen to run at the same time as the attacked application. The bug fools the OpenSSL implementation to leak data that it has access to, but it does not fool Linux into allowing OpenSSL to reach beyond its process memory space.
The integrity of the system. This is probably the most important point. The attack is passive in the sense that it gets data out, but cannot change anything in the system. An obvious exception would be if a password that was captured happens to open the door to other attack venues. However, as long as the stolen credentials do not allow their holder to cause persistent damage, no persistent damage could have been made. This implies that by following a few instructions (below), you return to the presumably secure state you were in before the attack was brought to your attention.
What to do next
First and foremost, install the necessary updates so to close the tap.
Second, comprehend the scope of the leakage that might have occurred. Unfortunately, there is no way to tell if your server was hacked and to what extent, so assume it was and enumerate the data that might have been compromised. This data, which we refer to as “session data", consists of all data that is served, processed or obtained by the web (or other application) process that calls OpenSSL. This includes all its inputs that come over the web (or other bearer), all data that goes out, and all data that may be processed in between by the same process that calls OpenSSL (e.g., the web server).
What is not in the scope of leaked data is all data that may be processed but is not served, or otherwise made available to the process that uses OpenSSL. For example, if the application is a web-application, then data that is neither sent nor received over the web, and which is not processed by the web server, would never find itself in the process memory space of the web server, and is thus safe. Also, other data on the server, such as files in home directories, is safe.
The private key of the web server is also at immediate risk, but in most cases it accounts for a change in quantity, not in quality, of the leaked data. In other words, this key will allow an attacker to decipher more sessions, so the attacker can leak not only session data during the attack, but other session data as well, yet it is still session data by the definition above, which we already considered to be entirely lost.
Passwords may be another issue. In most cases, however, stolen passwords only account for yet more session data that can be accessed by impersonating the user, so it is still in the sense of “more of the same". Obviously, if the passwords that your application uses are also used for granting access to other assets – those may be at risk as well.
Private keys of users, if used by the application, are not at risk, because they are never made available to the server in the first place.
To summarize, in the usual case, the maximum leakage that could have occurred consists of all data served or processed by the process calling OpenSSL. Data of other applications and back-end data are safe.
Third, return to secure state. We got lucky with the “heartbleed bug” in that it is passive and cannot cause your system to be “owned", or to be contaminated in a way that calls for a complete re-install or serious scrubbing. After installing the patch to OpenSSL, you need to generate a new key-pair for the OpenSSL deployment, get it certified if your previous key was, revoke the previous key, and change application passwords that might have been leaked. Once this is done, aside of the data that might have been leaked forever, you can consider the incident to be behind you.
If you run an application server utilizing OpenSSL which was subject to attack, a lot of data might have been stolen, both in terms of application data and in terms of credentials. However, the only bright side is that, as opposed to with other serious attacks:
it is relatively easy to quantify the data that could have been stolen, and
(unless credentials that allow further access were leaked) compromise is not persistent: once you patch OpenSSL and replace keys and credentials, you are safe hereafter.
Display comments as Linear | Threaded
Dan Feekes on :
I was talking to a colleague last week who said that their IT security co-workers were examining their logs and traffic over the last couple weeks and have not seen any attempts to exploit the Open SSL issue. Is the bug too new and black hats have not yet had the time to write the code to exploit it yet. If what I've been told is true, does anyone have suggestions why we're not hearing about open SSL attacks yet.
Hagai Bar-El on :
The bug is not new. It is there for more than two years. The only question is whether it was known to black-hats before it was “officially” discovered or not. It is not trivial to determine if it was exploited or not on a given system, because the typical HTTP logs Apache keeps do not show heartbeat packets.
Here you can find possible evidence to past exploitation: https://www.eff.org/deeplinks/2014/04/wild-heart-were-intelligence-agencies-using-heartbleed-november-2013
Ria on :
Hello, thank you for this article. I would like to draw your attention to something else. The sad fact is, that I understand some of this and still would not know how to change keys. Worse is, that in my neighborhood I am the Nurde .. So I would like to ask if anyone would be willing and able to give a “How to” instruction to all the oblivious users that really do not know how to help themselves with this?
Hagai Bar-El on :
Replacing the SSL keys (yes, you need to replace the keys and the certificates, not just the certificate) is done by repeating the same process of installing SSL in the first place.
The technicality depends only on your web server and OS. Search for “set up ssl apache” or “set up ssl iis” to get hundreds of useful guides.
A good one for SSL on Linux/Apache is at: http://www.htmlgoodies.com/beyond/security/article.php/3774876
Joe on :
“The integrity of the system. This is probably the most important point. The attack is passive in the sense that it gets data out, but cannot change anything in the system.”
What if the exfiltrated data allows a hacker to then log in to the server and install a rootkit, or exploit some other installed software to do this?
Hagai Bar-El on :
@Joe In this case, all bets are off… The sentence that follows the one you quoted reads: “An obvious exception would be if a password that was captured happens to open the door to other attack venues.”