Bind Certificates to Domain Names for Enhanced Security With DANE and DNSSEC

Phishing and malware delivered via email remain some of the greatest threats to organizations today. According to the “IBM X-Force Threat Intelligence Index 2019,” nearly one-third of all analyzed attacks involved compromises via phishing emails.

Although business operations are increasingly reliant on email, the core of the technology hasn’t changed much over the years, and many of the current server-based email security mechanisms are still vulnerable to attacks. What if we could protect our email communications in the same way we already protect our browsing activities via HTTPS?

Some Background on HTTPS

From a simplified standpoint, we can state that HTTPS is HTTP with a TLS/SSL encryption layer on top. This encryption layer serves three main purposes: confidentiality, integrity and authentication. Simply put, it allows the user to have some confidence that the information being transmitted cannot be eavesdropped on and cannot be altered by malicious actors — and most importantly, that it is being sent to the intended entity. After all, you do not want to send personal or medical information to a website run by criminals.

With HTTPS, it is important to recognize that secure does not necessarily mean safe. A secure website can still cause harm to your computer. Ensuring that information is transmitted solely to the intended entity is typically done by the browser, which verifies the identity of the server to which it connects. This identity is provided as a property of the TLS certificate and describes which domain the certificate belongs to. The client verifies that it can trust the certificate or that it is verified and trusted by a certificate authority (CA) that is trusted by the client. CAs play a crucial role in the whole process and are the entities that issue digital certificates.

Flaws in the Certificate Validation Process

It goes without saying that a CA is not going to issue a TLS certificate randomly. The requester needs to prove ownership of their domain. This can be done via the Domain Name System (DNS), by receiving an email, or by putting a text file up on the website. Unfortunately, this validation process isn’t flawless.

Because verification relies on DNS, there’s much to gain for an attacker in modifying the DNS infrastructure, as Cisco Talos discovered amid the Sea Turtle and DNSpionage campaigns, and FireEye observed earlier this year. In response to these attacks, the U.S. Computer Emergency Readiness Team (CERT) issued an alert, AA19-024A, on a DNS infrastructure hijacking campaign.

A different way of tampering with the validation process is to conduct a local border gateway protocol (BGP) hijack close to the CA, which makes the CA believe the attackers are the owners of a targeted domain. These attacks can be prevented by a conscientious CA with different validation points scattered around the internet.

Once an attacker is able to obtain a certificate for a domain, they can start a man-in-the-middle (MitM) attack, allowing them to capture, for example, login credentials. For this type of attack to be successful, the attacker also needs to divert the victim’s traffic. In general, this can be done by poisoning the DNS cache, placing malware on the victim’s machine, setting up rogue access points or targeting the internet service provider (ISP) directly.

Another approach to obtaining certificates wrongfully is to target the CA directly. The most notorious case of this was the DigiNotar hack in 2011, when false certificates were created for hundreds of websites, including Google and Skype after a breach of the CA.

Reducing Dwell Time

During incident response, it’s crucial to detect the intrusion as soon as possible. It’s reasonable to assume that most CAs enforce security best practices and have sufficient security controls in place.

However, given that there are flaws in the system, browsers have a fairly long CA certificate list included, and certificates are not linked to one CA, it’s not unlikely that your organization will have to face an incident that involves malicious TLS certificates in the future. Despite the fact that certificates can be easily revoked in case of an incident and you can monitor the certificate transparency feed, there still remains a window of opportunity for an attacker.

What Is DNSSEC?

Similar to email, DNS has been around for a very long time. Unfortunately, when it was being designed security was not the top priority. When a computer makes a DNS request, it simply trusts that the information it receives is from a valid and legitimate source, and attackers can abuse this implicit trust to cause mayhem.

This problem can be solved by introducing an actual verified level of trust into DNS, which can be done through Domain Name System Security Extensions (DNSSEC). DNSSEC grants additional resource records, which exist next to the traditional DNS records. These extra records provide a digital signature of the DNS records, allowing a client to verify the source and legitimacy of an answer.

Despite its use of the word “security,” DNSSEC does not provide encryption of DNS; it merely signs responses so that forgeries are detectable.

Introducing DANE

DNS-based Authentication of Named Entities (DANE) goes one step further. DANE is a protocol that uses DNS for storing certificates and DNSSEC to verify the validity of the requests. In other words, instead of relying on an external party to issue certificates, you can now control this yourself by putting the certificates within DNS.

Should this then replace the CA system altogether to solve the problem of having too many CAs? There are arguments for and against DANE replacing the CA system. Transferring the control and responsibility to the DNS system’s owners — top-level domains (TLDs), etc. — might not always be desired, but it can empower organizations and DNS administrators. Nevertheless, DANE can co-exist next to the current CA system.

Because DANE relies on DNS, you definitely need DNSSEC for it to work. Without DNSSEC, it’s impossible to guarantee the required strong integrity protection of the resource record.

How Does DANE Work?

The core of DANE is within the resource record type called Transport Layer Security Authentication (TLSA). These records are used to specify the keys used by the servers within the same domain. An example of a TLSA record name is “_443._tcp.www.myorg.com.” This record name is composed of three values:

  1. The port number of the server, preceded by an underscore
  2. The protocol, preceded by an underscore
  3. The host name

The record returned then consists of four elements:

  1. A value describing the certificate usage. These values include:
    • 0: You specify the CA who provides the certificate. This setting allows you to limit issuing of certificates to your chosen CA.
    • 1: You define the exact certificate that should be used for the domain. The certificate should be issued by a valid CA.
    • 2: This setting indicates the trust anchor to be used in validating the certificate. This is the option to choose if the CA that issued your certificates is not installed by default in the client list of trusted CAs. This can be a solution for internal CAs.
    • 3: With this setting, you specify the exact certificate that should be used for the domain, but contrary to 1, the certificate does not need to be signed by a valid CA.
  2. The selector field. Either verify the entire certificate (0) or just the public key (1). Most often a setting of 1 will be used.
  3. A matching type, if the client needs to look at all the information (0) or a hashed value (1, 2) of the selected data. Most likely, you’ll choose one of the hashed options.
  4. The actual data that needs to be matched.

Setting the certificate usage value to 3 enables you to provide your own certificates, including a self-signed certificate. This can allow you to greatly improve user experience by presenting a valid self-signed certificate. It is important to note that DANE works perfectly fine with existing certificates issued by CAs — as demonstrated, for example, via certificate usage 0 and 1. Because of the flexibility of the protocol, it is also possible to have different certificate usage modes within the same domain.

DANE for Enhanced Security of Email

Encryption of email transport can already be done using STARTTLS. The sending server starts an unencrypted channel to the receiver requesting to switch to an encrypted channel. Unfortunately, with this opportunistic TLS the receiving server can still refuse encryption, and the servers then continue in plain text. This is great for MitM attacks but not so great for organizations.

End-to-end encryption to ensure message integrity and sender authentication can be achieved via Secure/Multipurpose Internet Mail Extensions (S/MIME) and Pretty Good Privacy (PGP). But neither of these are widespread, and both have drawbacks concerning scalability or lack of user-friendliness.

DANE can do little for end user authentication or digital signatures, but it does protect the email transport, from both an integrity and confidentiality point of view. Before transmitting emails, the sending server will look up the TLSA record for Simple Mail Transfer Protocol (SMTP) communication. If this exists, it will immediately start with an encrypted channel toward the receiving server. This limits the opportunities for an intermediate attacker to request a “downgrade” to an unencrypted communication channel.

However, DANE is merely one piece of the puzzle. If the last chain in the email routing is a mail transfer agent (MTA) or a client talking plain text, then you’re back to where you started.

How Do You Get Started With DANE?

By far the most important consideration with DANE is having a strong and well-protected DNS infrastructure. If you’re already at the point of supporting DNSSEC, you hopefully understand the need for strong security controls. Also critical to the success of DANE is a solid DNS provisioning system. Because certificates are now managed as DNS records, having some form of automation, monitoring and change management becomes much more important.

Setting up TLSA records is easy with the use of these toolkits:

Using the tlsa command to create a TLSA record is straightforward:

tlsa –usage 3 –selector 1 –mtype 1 –certificate /path/to/certificate.crt www.myorg.com.

Once the record is created, the next steps are adding this new DNS record to the zonefile, updating the revision number and publishing the zone. On the server side — for example for the mail server Postfix — enabling DANE is as simple as setting the TLS level to DANE and making sure there is support for DNSSEC:

smtp_dns_support_level=dnssec

smtp_host_lookup=dns

smtp_tls_security_level = dane

smtp_tls_loglevel=1

Reap the Benefits of Increased DNS Security

The biggest hurdle for implementing DANE lies with DNSSEC. Granted, setting up DNSSEC can be daunting, but the reward of a much higher level of DNS security can make it worth the effort.

After you implement DNSSEC and set up DANE, you can start monitoring your environment for malicious SSL certificates with Zeek. Zeek extracts certificate metadata from traffic flows and then matches these against an SSL blacklist, for example SSLBL. It can also verify the certificates against a list of fingerprints (JA3 and JA3S). This verification works by looking at the TLS Client Hello (or Server Hello) packet following the TCP 3-way handshake. This packet and the way it is generated is fairly unique for every client (or server), and it is sent before actual encryption takes place.

To detect connections with malicious servers, you need to track both the client and server messages. Why? Because servers respond to different clients differently, but they will always respond to the same client in the same way. This means that even if a bad certificate (or tooling) is moved to a different domain, you can still track it because the fingerprint remains the same.

Contributor'photo

Koen Van Impe

Security Analyst

Koen Van Impe is a security analyst who worked at the Belgian national CSIRT and is now an independent security...