SPF - Sender Policy Framework

2023-04-12

Networking Security

Email Message Structure

To understand how SPF (Sender Policy Framework) and DKIM works is to know the difference between email envelope and email header .

Short Definition:

Sender Addresses in E-Mails

Like paper mail letters, e-mail messages have at least two kinds of sender addresses: one on the envelope and one in the letterhead.

The envelope sender address (sometimes also called the return-path) is used during the transport of the message from mail server to mail server, e.g. to return the message to the sender in the case of a delivery failure. It is usually not displayed to the user by mail programs.

The header sender address of an e-mail message is contained in the "From" header and is what is displayed to the user by mail programs. Generally, mail servers do not care about the header sender address when delivering a message.

email message structure

What is SPF (Sender Policy Framework)

Sender Policy Framework (SPF) is an email authentication protocol that allows you to specify the servers authorized to send mail from your domain, making it harder for fraudsters to spoof (forge) your address.

If you send email from a custom domain (for example, @yourdomain.com), you can list the IP addresses of the mail servers authorized to send your mail. By checking your list of IP addresses, email services and ISPs can identify whether the message came from a trusted server.

More precisely: SPFv1 allows the owner of a domain to specify their mail sending policy, e.g. which mail servers they use to send mail from their domain. The technology requires two sides to play together:

the domain owner publishes this information in an SPF record in the domain's DNS zone , and when someone else's mail server receives a message claiming to come from that domain, then
the receiving server can check whether the message complies with the domain's stated policy. If, e.g., the message comes from an unknown server, it can be considered a fake.

how SPF work

How does SPF work

SPF checks whether an incoming email has been sent from a mail server authorized to send mail from that domain.

First, you must create an SPF record that lists the IP addresses of the mail servers authorized to send mail from your domain. Then you're ready to send messages with SPF as follows:

You send an email, for example, from the address you@yourdomain.com. Suppose the message is sent from one of your authorized email servers with the IP address 185.70.40.111.
When the email arrives, the receiving mail server checks the sender's address in the **Return-Path** (also known as **MAIL FROM**) field of the email header and looks up the SPF record for your domain (in this example, yourdomain.com).
The receiving mail server checks whether the IP address from which the mail was sent (185.70.40.111) is listed in your SPF record.
If the IP address is listed, the message passes SPF; if it isn't, it fails.

example: How SPF (Sender Policy Framework) works

Note that in step 2. the receiving mail server looks up the SPF record for the domain in the Return-Path field, which doesn't need to be the same as the domain in the From field. Even if the domain in the From address is spoofed and differs from that in the Return-Path, a message can pass SPF.

That's why you need DMARC to verify that the domain in the SPF and DKIM checks matches the sender's address in the From field.

What is an SPF record

An SPF record is a type of DNS text record that lists the IP addresses of the servers authorized to send email from a particular domain. SPF records are published on the domain's DNS server, so any receiving mail server can look up the list to check whether an email has been sent from a trusted server.

Let's look at an example to give you an idea of how SPF works. Bob owns the domain example.net. He also sometimes sends mail through his Gmail account and contacted GMail's support to identify the correct SPF record for GMail . Since he often receives bounces about messages he didn't send, he decides to publish an SPF record in order to reduce the abuse of his domain in e-mail envelopes:

example.net.  TXT  "v=spf1 mx ip4:185.70.40.111 a:pluto.example.net include:_spf.google.com -all"

The parts of the SPF record mean the following:

SPF DNS key	Description
v=spf1	SPF version 1SPF version 1
mx	the incoming mail servers (MXes) of the domain are authorized to also send mail for example.net
ip4	The IP address(es) of the servers authorized to send email from your domain. They can be IPv4 or IPv6 addresses; IPv6 addresses use the tag ip6.
a:pluto.example.net	the machine pluto.example.net is authorized, too
include:_spf.google.com	everything considered legitimate by gmail.com is legitimate for example.net, too
-all	all other machines are NOT authorized.

SPF Record: Advanced Syntax

SPF records for all syntax are usually defined using a DNS TXT record type with a single string of text. As mentioned, it always starts with the v= element that denotes the SPF version used. There's only one version currently, so SPF1 is used.

Each SPF record has specific terms that work as rules for which hosts can send emails from the domain or display additional information.

SPF record syntax comprises three major elements: SPF Mechanisms, SPF Qualifiers, and SPF Modifiers.

SPF Mechanisms

Mechanisms are SPF tags used in the SPF record structure to tell receiving servers what to match and how to deal with emails.

ALL: It always matches and must be the last mechanism listed at the end of the SPF record. Any mechanisms after this are ignored. It also shows default results like -all for unmatching IPs.
A: Defines a domain name with an A or AAAA dns address record as a match as it resolves to the sender's address. The current domain is used if this DNS SPF record syntax is unspecified. It's typically applied when queries for A or AAAA records are generated in a domain with a sender's IP address.
ip4: A match is successful when the sender is linked to the given ipv4 address range in the SPF record. It's added with a prefix that signifies a range's length. In case of no prefix, /32 is used by default.
ip6: A match is successful if the sender belongs to the given ipv6 address range. It's added with ip4 directive along with a prefix specifying range length. In case of no prefix, /128 is used by default.
MX: This mechanism authorizes senders with an IP address matching the one contained in the dns MX record specified. MX records consist of an IP address and a priority value for each server meant to accept messages. If an MX record of a domain contains an IP address matching the sender's IP address, the sender has permission to send emails using the domain.
PTR: Defines the authorized domain using dns PTR records which resolve IP addresses to subdomains or domains (the opposite of DNS A records). The receiving server reverse-maps the sending IP address to find associated domain names.

For each exact domain match or subdomain, a forward lookup is then performed to find the IP address. If the sending IP address is the same as any IP addresses found during the lookup, then it's a match and the email is validated.

The PTR mechanism is slow and unreliable as it requires multiple lookups. As such, it is not recommended according to the RFC 7208 guidelines . Some receiving servers ignore the PTR mechanism or even the entire SPF record with the PTR mechanism.
EXISTS: This SPF mechanism executes a DNS A record search for the domain provided. A match happens when a valid A record is found, regardless of the actual lookup result. Using macros, you can set up per-user exceptions with this mechanism.
INCLUDE: This mechanism is used to authorize third-party email senders by specifying their domains. A sender is authorized when its IP address matches the IP addresses or domains contained in the SPF record of the listed domain. A permanent error result is returned when no SPF record is found for the listed third-party domain.

The exists mechanism is used to construct an arbitrary domain name that is used for a DNS A record query. If any A record is returned, this mechanism matches. Domains can use this mechanism to specify arbitrarily complex queries.

SPF Qualifiers

An SPF Qualifier is an optional prefix to a mechanism. Adding an SPF record to your DNS with SPF Qualifiers tells receiving mail servers how to treat an email when there's a match with a Mechanism value.

Mechanisms are verified in the order of their occurrence in an SPF record. If a Mechanism lacks a Qualifier and there's a match, SPF authentication passes. However, when there's no match, the default action is neutral; the email message neither passes nor fails authentication.

Each mechanism can be combined with one of four qualifiers.

Qualifier	Result	Action Taken by Receiving Server With a Match
+	Pass	Email message passes authentication and the server is permitted to send emails. Messages are authenticated and this is the default action taken in case of no Qualifier.
-	Fail	Email message fails authentication as the sending server isn't authorized to send emails for the domain. The SPF record instructs the receiving mail server to reject the email.
~	SoftFail	The receiver's mailbox accepts the message, however it is marked as suspicious and lands in the spam folder.
?	Neutral	Email message neither passes nor fails authentication as the SPF record doesn't explicitly state whether an IP address is authorized. Indicates that no match was found for the sender when checked against your authorized IP addresses and domains.

Other returned results include:

None - No SPF record was found for the domain, or the SPF record didn't return a result.
TempError - A transient error usually due to DNS configuration issues.
PermError - A permanent error usually due to SPF record syntax or formatting errors.

SPF macros are an effective and important Sender Policy Framework feature that is used when domain owners demand a more dynamic and scalable SPF record for authenticating their email domains.

SPF Modifiers

SPF Modifiers decide the DNS SPF record syntax's working parameters. They consist of name or value pairs separated by the = symbol, pointing out additional information, specifying exceptions to rules, or changing certain defaults.

The 'redirect' Modifier directs to other SPF records for authentication. Experts use them when they want more than one domain to have the same SPF record content. Only use this SPF Modifier if you control all the domains. The include SPF Mechanism should be used for authorized domains outside of your control.

If the 'all' Mechanism appears in the SPF record, then the 'redirect' Modifier is ignored.
The 'exp' Modifier explains why the receiving server returned a Fail SPF Qualifier when a mechanism matches.

SPF Modifiers can only appear once and only at the end of the SPF record. Unknown modifiers are ignored.

Tips for SPF Records

Before you check your SPF record , keep the following tips in mind:

A domain can't have more than one SPF record.
An SPF record can't contain any uppercase characters.
A character limit of 255 characters applies to SPF records. Any string exceeding this number will result in failed authentication.
Remove any SPF Mechanisms resolving to the same domain.
Remove any 'ip4' and 'ip6' SPF Mechanisms not in use and see whether any address ranges can be merged.
You can create a subdomain to store SPF information by using _spf.domain.com - this is ideal for large organizations with too many IP addresses to include in a single SPF record.

SPF Exceeds Maximum Character Limit - If you encounter this message, it means you are using a single string within your SPF record that exceeds 255 characters. Note: According to RFC 7208 Section 3.3 , a single SPF record can exceed 255 characters, but a single string cannot.

How SPF differs from DKIM and DMARC

SPF, DKIM , and DMARC are email authentication protocols that are used to verify that emails are sent from a legitimate sender, but they all work differently. Here's how they work together:

SPF (Sender Policy Framework) verifies that an email has been sent from an IP address authorized to send emails from the sender's domain.

DKIM (DomainKeys Identified Mail) cryptographically verifies that the sender's address and message contents haven't been changed in transit.

DMARC (Domain-based Message Authentication, Reporting, and Conformance) ensures that the domain in DKIM and SPF checks matches the sender's domain in the From field. It also specifies how email service providers should deliver a message that fails both DKIM and SPF — accept, reject, or mark it as spam.

SPF and mail forwarding

One limitation of SPF is that it doesn't work reliably with forwarded emails.

When forwarding a message, if the sending mail server replaces the original sender's email address in the Return-Path (MAIL FROM) field with the forwarder's address, the receiving server will check the SPF record of the forwarder's domain. In this case, the message should pass SPF.

But not all email platforms rewrite the Return-Path field when forwarding. If the mail server preserves the original sender's address in the Return-Path field, the receiving mail server will check the original sender's SPF record. As this doesn't include the IP address of the forwarding server, the message fails.

Fortunately, if an email fails SPF, it may still pass the DKIM check because DKIM verifies a message's content, not its sending IP address. As long as you don't change the content and structure of the original message, it should pass DKIM.

Why use SPF

Along with DKIM and DMARC , SPF is one of the main ways you can protect your domain and ensure your emails get delivered. Here's why you should use it: Beat spammers and scammers

SPF makes it harder for cybercriminals to send messages pretending to be from your domain (email spoofing ). Specifying trusted mail servers with SPF helps to prevent fraudsters from using your domain for spam and phishing attacks.

SPF helps ISPs and email platforms identify legitimate emails. So using SPF increases the chances of your messages being delivered instead of ending up in the spam folder. The more your emails pass authentication checks like SPF, the more you improve your domain's reputation with email platforms. So SPF can help improve your emails deliverability in the long run.