Security researchers have found inconsistencies in how some Simple Mail Transfer Protocol (SMTP) servers handle end-of-data sequences. This allows the injection of SMTP commands into email messages in a way that causes receiving servers to treat them as two separate messages with one of them having its “From” headers spoofed. Furthermore, because the main message’s envelope successfully passes security checks like SPF, DKIM, and DMARC, the spoofed message is delivered to inboxes with no warnings.
The new attack, dubbed SMTP smuggling, was devised by Timo Longin, a senior security consultant at SEC Consult, and it affected millions of domains that used Microsoft Exchange Online or Ionos, a large German web and cloud hosting company, for email services. While both Microsoft and Ionos fixed the issue, domains that use Cisco Secure Email Gateway in default configuration might still be affected.
Longin borrowed the main concept for SMTP smuggling from another class of attacks known as HTTP request smuggling where attackers trick a front-facing load balancer or reverse-proxy to forward specifically crafted requests to a back-end application server in a way where the back-end server processes it as two separate requests instead of one. This is achieved by modifying request headers in a way that gives conflicting information to servers about where the request ends. If the front-end and back-end servers have different interpretations of the header values and therefore the end of the request, rogue requests can be smuggled past the front-end server without subjecting them to the normal security checks.
Similarly, at least two servers are involved when sending email across the internet: the SMTP server used by the sender (outbound) and the SMTP server of the recipient (inbound). If they have different interpretations about where a message ends, an attacker can slip rogue messages past security checks.
How SMTP works
SMTP is the communications protocol used to relay email messages across the internet from one domain to another. When you want to send messages from an email client, also known as a mail user agent (MUA), the application connects to your domain’s SMTP server, also known as a mail transfer agent (MTA), and passes the message to it. The server then looks up the SMTP server for the recipient’s domain name and relays the message to it. In other words, an SMTP server will deal with both outbound and inbound connections depending on whether it’s sending or receiving email for the domain names they’re configured to serve. One SMTP server can handle email for multiple domain names.
A connection from an MUA to an MTA will begin with an extended hello (EHLO) command that communicates the user’s email domain to the server. The server checks if it’s supposed to handle email for that domain and responds with its SMTP capabilities, or supported features, which can differ depending on how it was configured. The client then sends the STARTTLS command to enable encryption if the server supports it, followed by AUTH LOGIN to send the user’s credentials for authentication. The next commands is “mail FROM” and “rcpt TO” to define the sending email address and the receiving address, followed by the DATA command, which indicates to the server the beginning of the actual email message.
The email message contains a few headers — “To: recipient@domain.com”, “From: sender@domain.com”, “Subject: Example subject” — which are followed by the actual body of the message. The “To” and “From” headers specified in this DATA section are the fields that appear by default in most email interfaces when reading a message and they can technically be different from the addresses specified in the “mail FROM” and “rcpt TO” SMTP commands.
You can think of it in terms of a paper letter inside an envelope: The delivery and return addresses written on the envelope are used as directions for the post office — the SMTP protocol in this case — while those written on the letter itself as a header are intended for the reader — the email client.
How SPF, DKIM, and DMARC help prevent email spoofing
However, this is why email spoofing used to be a big problem in the past before security mechanisms such as SPF, DKIM, and DMARC were put in place. Someone could connect to an SMTP server and send a message from attacker@domain.com, but put a spoofed address in the “From” header in the DATA portion to trick the recipient into thinking the message came from someone else. Nowadays, such a message would most likely be rejected by a well-configured server or at the very least would be marked as spam.
Another issue in the past was that an attacker could connect to a recipient’s SMTP server and claim they’re sending a message on behalf of a domain they didn’t actually own, and there would be no way for the SMTP server to verify the authenticity of that claim. This is what the SPF (Sender Policy Framework) was created for. It’s a DNS record that can be set for a domain and will contain a list of IP addresses of the servers that are allowed to send mail on its behalf.
With SPF in place, the inbound SMTP server takes the domain from the address specified in the “mail FROM” command by the outbound server, pulls up the domain’s DNS records, and checks if the request came from an IP address that’s listed in the SPF record. However, SPF does not perform any validation on the “From” header from the DATA portion.
Another standard, DomainKeys Identified Mail (DKIM), allows email messages, including “From” headers, to be cryptographically signed. The signature can be validated by the inbound SMTP server with a public key that’s again stored in the sending domain’s DNS records. Again, the domain whose DNS records will be checked is the one extracted from the “mail FROM” SMTP command, not the one in the “From” header of the DATA command. So, the message can be signed but can still contain a rogue “From” header.
Finally, we have Domain-based Message Authentication, Reporting & Conformance (DMARC), a standard through which domain admins can set policies for accepting email. These policies can include mandatory SPF or DKIM validation as well as alignment of the “From” DATA header field with those policies. When DMARC is enforced properly, the addresses in the “mail FROM” command and the “From” header have to match for an email to be accepted.
End-of-data confusion with unusual line breaks
With SMTP smuggling, a client application sends a message to an outbound SMTP server which then forwards it to an inbound SMTP server. The goal is to trick these servers to interpret the end of the message differently. More specifically, the sender’s server (outbound) should treat the DATA section as one message and pass it along, but the recipient’s server (inbound) should treat the same DATA section as two separate messages because there are SMTP commands smuggled inside.
How does a SMTP server know where the DATA section ends? When it encounters a “.” (dot) on a line by itself. This is known as the end-of-data sequence and is normally encoded as <CR><LF>.<CR><LF> (where CR stands for Carriage-Return and LF for Line-Feed) or rn.rn.
SEC Consult
In theory SMTP server software should ignore attempts to represent the “single dot on a line by itself” in a different manner, but Longin was not convinced that’s happening and set out to test it. First, he created email accounts with a dozen popular email providers so he could send emails through their SMTP servers to an address that used an SMTP server he controlled and could be used to analyze incoming messages. He then started sending emails with unusual line break sequences to see if outbound servers filter them out. “A ‘.’ on a line by itself on Windows would be separated via two carriage return line feeds (<CR><LF>.<CR><LF> or rn.rn), while a ‘.’ on a line by itself on Linux would be separated with two line feeds (<LF>.<LF> or n.n),” the researcher explained.
He therefore crafted an email message that contained a chunk of text, followed by a <LF>.<LF> line, then another chunk of text, followed by <CR><LF>.<CR><LF>. If the outbound server interpreted <LF>.<LF> as a valid end-of-data sequence, it should have cut the message after it. Otherwise, the message relayed to his SMTP server should have included the second chunk of text as well — if not rejected by the outbound server in the first place.
It turned out that SMTP servers had different filtering strategies to deal with the <LF>.<LF> sequences: adding another dot to escape it and making it <LF>..<LF>; replacing it with a <CR><LF>; encoding it as =0A.=0A; removing the sequence completely or simply rejecting the message.
How about more exotic variations of line breaks such as <LF>.<CR><LF>, <CR>.<CR>, <CR>.<LF>, <CR><LF>.<CR> or <CR><LF>x00.<CR><LF> (where x00 represents a nullbyte)? Do outgoing servers have filtering strategies for all these, too? According to Longin’s findings, some of them don’t.
The first victim was GMX Mail, a popular webmail provider that dates to 1997 and has around 20 million active users. GMX’s outbound SMTP server allowed emails with <LF>.<CR><LF> sequences to pass through. This meant that if the researcher also found incoming SMTP servers that treated <LF>.<CR><LF> as an end-of-data sequence, he could potentially smuggle SMTP commands inside a request to spoof messages from any @gmx.net or @gmx.com addresses, including admin, to users of those SMTP servers.
SEC Consult
Longin identified two big email providers whose SMTP servers interpreted <LF>.<CR><LF> as the end of data: Fastmail and Runbox. However, he also found that popular SMTP server software like Postfix and Sendmail were also accepting this end-of-data sequence in their default configurations. According to Shodan scans, more than 1.5 million publicly accessible SMTP servers use Postfix and Sendmail.
The researcher now had the ability to spoof any GMX identities to users of any of these vulnerable SMTP servers in a way where the messages would pass SPF, DKIM and DMARC validation because they were delivered through the real GMX SMTP server without being blocked.
The issue was worse, because GMX also runs the web.de domain and is also a subsidiary of Ionos, a large web hosting company. It turns out Ionos’s SMTP servers ran the same custom software as GMX’s and were therefore also allowing outbound email messages with <LF>.<CR><LF> sequences. Furthermore, the default SPF records for Ionos-hosted domains and GMX had overlapping IP addresses, meaning that attackers could use their GMX account to spoof messages from any of the 1.35 million domain names that used Ionos’ email servers, while still passing security checks.
Like GMX and Ionos, another SMTP provider that allowed outbound emails with <LF>.<CR><LF> was Outlook and Microsoft Exchange Online. This meant that attackers could spoof valid messages from any of the millions of domains that listed Exchange Online’s SMTP servers in their SPF records.
However, the impact was more limited because Outlook and Exchange Online use the BDAT (or chunking) command to send messages by default. This is an SMTP feature that specifies the exact message length in bytes instead of relying on end-of-data sequences and it makes SMTP smuggling impossible. However, there’s a fallback mechanism because not all receiving SMTP servers support BDAT. For those that don’t, the Exchange servers will fall back to using the regular DATA command to send messages.
To be vulnerable to spoofing via Exchange Online messages, an incoming SMTP server needs to meet two conditions instead of one: Not support BDAT and interpret <LF>.<CR><LF> as an end-of-data sequence. This was the case for Fastmail and remains the case for hundreds of thousands of Postfix and Sendmail deployments. Microsoft has since addressed the problem and messages with <LF>.<CR><LF> sequences are no longer allowed via Outlook and Exchange Online.
Cisco Secure Email settings could allow SMTP smuggling
While testing other exotic end-of-data sequences against inbound SMTP servers of the former Alexa top 1,000 domains, Longin found several high-profile domains that accepted <CR>.<CR> as an end-of-data sequence. The domains included Amazon, PayPal, eBay, Cisco, the IRS, IMDb, and Audible.
All those domains were using Cisco’s Secure Email service with on-premises deployments of Cisco Secure Email Gateway or the cloud-based Cisco Secure Email Cloud Gateway. The Cisco Secure Email Gateway can be thought of as a proxy server that checks emails for malicious content before passing them to the user’s real SMTP email server. The software has a configuration option for how to handle messages that contain bare carriage return (CR) or line feed (LF) characters with three settings: Clean, Reject, or Allow.
The behavior of the “clean” setting, which is the default one, consists of converting bare CR or LF characters into CRLF characters meaning that <CR>.<CR> will be converted into <CRLF>.<CRLF> and this is a valid end-of-data sequence for all SMTP servers because it’s the equivalent of <CR><LF>.<CR><LF>. So, if you run an SMTP server that only accepts <CR><LF>.<CR><LF> as end-of-data sequence, as it should, and you put Cisco Secure Email Gateway with default settings in front of it, you just made it vulnerable to SMTP smuggling.
SEC Consult advises Cisco Secure Email Gateway users to change this setting from “Clean” to “Allow” so that messages with <CR>.<CR> are forwarded without modification to their SMTP servers, which should then reject them. Outbound SMTP servers that don’t filter <CR>.<CR> and will allow outbound emails with this sequence inside include Outlook/Exchange Online, iCloud, on-premises Microsoft Exchange servers, Postfix, Sendmail, Startmail, Fastmail, and Zohomail.