An old IT expression goes, “What sounds like a really good idea at 5 p.m. becomes a really bad one by midnight.” This usually refers to making a quick change to a system before you go home on a Friday evening that inevitably goes wrong, forcing you to pull an all-nighter getting the system back to a stable state. This appears to have been the case with an OpenSSL code check-in to include TLS/DTLS heartbeats at nearly 11 p.m. on New Year’s Eve 2011, which eventually led to the vulnerability we’re now calling Heartbleed.

More than half of web servers on the internet use OpenSSL to safeguard data and user accounts. Versions 1.0.1 through 1.0.1f are vulnerable to an exploit that may expose user credentials, credit card data, sensitive documents and the server’s certificate itself. Below is an explanation of what the vulnerability is, how it manifests itself and how you can protect yourself from being compromised.

Roughly 53% of websites on the internet use OpenSSL to safeguard data and user accounts (sites with “https” in the URL). Nearly half of those sites were vulnerable to the bug.

What Is TLS Heartbeat?

TLS is a form of encryption generally used by Web servers to secure transactions such as credit card payments. It also protects login credentials — your username and password — from being exposed across the Internet. TLS, also known as SSL, can be identified in your browser by “https://” (the “s” is for secure) versus “http://” in the website address bar. Most browsers will also show some form of lock icon in the address bar as well to show that the connection is secure.

To set up the encrypted session, TLS must agree upon an encryption method that’s supported by both the Web server (e.g., www.securityintelligence.com) and the client (usually your web browser) as well as exchange encryption keys to secure the session. This is known as the TLS handshake, Generating and validating encryption keys can delay access to the website and consume computing power. TLS heartbeats are sent when there’s no activity — when a user is filling in a Web form, for example — to keep the session from timing out and having to renegotiate the session.

TLS heartbeats are like a network ping: The sender transmits a request with an arbitrary payload and expects the receiver to respond, including the original payload in the response. There’s no need to include anything in the payload — a zero byte payload is acceptable — but there’s nothing to prevent one from being included. The payload can be up to 64 KB in size, or 65,535 bytes, the size of which is specified by the sender in in a field called “length,” which preceded the payload itself.

The Heartbleed Vulnerability

The problem is that OpenSSL blindly trusts the length field set by the sender when it creates a response packet. First the server receiving the request stores a copy of the request on the memory heap, including the original payload. Then it creates a response packet and copies the payload from the original request, starting at the location it stored it on the heap and continuing for the specified length. In our example diagram below, the sender sent 3 bytes of the original payload data, the string “abc,” but claimed it sent 30,000 bytes, which extends past the original payload and deep into the heap.

The heap may contain anything from random data to unencrypted data processed by OpenSSL. The latter generally includes the server’s SSL certificate private key and, in many cases, plain text usernames and passwords for users of Web services. The irony is that the system that gives users confidence that their Web session is secure is the very mechanism that betrays their account credentials.

Affected Devices and Sites

It’s not just typical Web services on the Internet that are affected: It could be any number of software packages or hardware devices that are affected. OpenSSL is the most widely used TLS library. While enterprises should perform a comprehensive assessment of their assets, here are some pointers and notes:

• Definitely check all Web servers, particularly Apache/Tomcat and nginx, which together comprise 66 percent of Web servers on the Internet. Microsoft’s Internet Information Services (IIS) does not make use of OpenSSL. Make sure to take into account load balancers and proxy servers, which may terminate encrypted sessions. If possible, remove load balancers from the equation by testing Web servers from the network they’re directly connected to.
• Check mail servers, database servers and home-grown applications to determine if they use vulnerable OpenSSL versions.
• Get a statement from your cloud providers on whether they’re evaluating and mitigating their vulnerability status. Don’t forget that you may be both an enterprise user — IaaS, PaaS and enterprise SaaS — as well as a user of cloud services such as Box.com, Dropbox, and even social media sites like Facebook and LinkedIn. (See this list of the top 10,000 sites tested for the Heartbleed vulnerability at 16:00 UTC on April 8.)
• Evaluate embedded devices such as VoIP phones and teleconferencing systems, point-of-sale (POS) terminals, industrial control and SCADA systems and medical equipment. You may have to seek help from the manufacturer or vendor.
• Evaluate Internet of Things (IoT) devices such as building control systems (HVAC, lighting, elevators and escalators, etc.), smart meters and safety and security systems.

Mitigations

• Get a patch from your vendor if available. Most Linux distribution vendors have been publishing updated packages (Fedora, RedHat, CentOS, Debian, etc); check with your application or operating system vendor.
• Upgrade to OpenSSL v1.0.1g, released on April 7, if possible.
• If you have obtained OpenSSL separate from a product and have access to the source code, turn off TLS heartbeats.
• Update your IPS and firewall signatures and enable Heartbleed signatures. If your vendor doesn’t provide signatures for Heartbleed, contact them.

Impact Analysis

In order to test if you’re vulnerable and assess the potential damage, here are some steps to take:

• Test your systems for OpenSSL v1.0.1–1.0.1f. See “Affected Devices and Sites” above.
• If vulnerable, assess what information may have been compromised. Most organizations should assume that any information present on the system and accessed by the application using OpenSSL is suspect.
• Revoke existing SSL certificates and reissue new certificates. Attackers may have been able to retrieve SSL certificate private keys, which could allow them to impersonate the service and/or decrypt any future traffic. Don’t generate the new certificates using the old private key. Make sure to create a new private key (e.g., using “openssl genrsa”) and use that new private key to create the new certificate signing request (CSR).
• Perform clean-up activities: For example, if the system provides secure logons, reset user passwords and notify them; if sensitive documents were available to the service, involve your legal counsel; assess your regulatory and contractual compliance obligations; etc.

This is a widely accessible vulnerability since OpenSSL is used to secure Web-facing systems — and it’s easy to exploit.