Monday 11th May 2020
Sender Policy Framework (SPF) provides a way to restrict the mail servers that are permitted to send as your domain, and is particularly effective when used with DMARC.
However, maintaining an SPF policy for a large or complex infrastructure with numerous distinct mail servers can pose a significant operational challenge. Some of the most common issues include:
SPF macros, a seldom used yet widely supported feature of the SPF specification, provide a potential solution to some of these challenges.
This article includes an introduction to SPF macros, as well as several examples of how they can be used to solve the various operational complications that SPF so often poses.
Skip to Section:
Using SPF Macros to Solve the Operational Challenges of SPF ┣━━ What are SPF macros? ┣━━ Macro Syntax ┣━━ Example #1: Permit individual IP addresses by adding a single DNS record for each ┣━━ Example #2: Permit ranges of IP addresses using wildcard DNS records ┣━━ Example #3: Restrict a third-party service to sending from a specific address ┣━━ Example #4: Keep track of what the IP addresses within your SPF record are for ┗━━ Conclusion
SPF macros are a feature of the SPF specification which allow for the creation of dynamic SPF policies. They allow for variables to be included within a policy, which are then evaluated by the receiving MTA and 'filled in' using data from the email being processed, such as the sender address or source IP address.
This enables various advanced SPF processing routines such as conditional lookups, and allows for additional email metadata to influence the decision.
SPF macros are present in the original SPF specification (RFC4408), as well as the revised specification (RFC7208), and are widely supported by MTAs.
SPF macros are represented by different single characters, surrounded by curly braces ({ }) and prepended by a percent (%) sign, e.g. %{i}. There are currently 8 'core' macros that are supported, as defined in section 7.2 of the RFC. These will be evaluated and expanded by receiving MTAs in a way very similar to templating engines such as Jinja.
%{s}: The sender of the received email, e.g. joe@example.com.%{l}: The local part of the sender, e.g. joe.%{o}: The domain of the sender, e.g. example.com.%{d}: The authoritative domain of the current SPF policy --- usually the sender domain (%{o}), but may also represent other domains when evaluating include: statements.%{i}: The source IP address of the message, e.g. 203.0.113.1.%{p}: The validated reverse-DNS domain of the source IP, e.g. if example.com IN A is 203.0.113.1 and 1.113.0.203.in-addr.arpa IN PTR is example.com, the validated domain will be example.com.%{v}: The static string in-addr or ip6 depending on the protocol version of the source IP --- used to construct macros utilising reverse DNS/PTR record syntax as seen in the .arpa zone.%{h}: The domain used in the most recent SMTP HELO or EHLO command, e.g. mail.example.com.Multiple macros can be used within an SPF record. You can also use modifiers such as r in order to reverse the order of the elements within an expanded macro variable, for example to convert a normal IP address such as 203.0.113.1 into a reverse lookup zone (1.113.0.203 in this case).
If your SPF record is getting too long, you may be tempted to begin using multiple include: statements, which can easily get messy, and doesn't actually solve the problem - it just delays it.
Instead of having to specify each individual IP address, you can use an SPF macro within your global policy, and then add a separate DNS record for each IP address that you want to add to your policy.
This keeps your policy short, whilst allowing an essentially unlimited number of IP addresses to be whitelisted.
For example, you could use the following as your global policy:
example.com IN TXT "v=spf1 exists:%{i}._spf.example.com -all"
Receiving MTAs will evaluate this and replace %{i} with the source IP address of the email.
Then, to permit 203.0.113.1 and 203.0.113.2 to send as your domain, add the following DNS records:
203.0.113.1._spf.example.com IN A 127.0.0.2 203.0.113.2._spf.example.com IN A 127.0.0.2
The exists: mechanism within the global policy is used to permit the sender if a DNS A record exists at the queried address. You can use any value for the A record - all that matters is that it contains some value, although it is recommended to use an address that is not publicly routable, such as 127.0.0.2.
The _spf subdomain is known as a semantic scope, as defined in section 4.1.2 of RFC8552.
To permit an IPv6 address, use the full address in dot format rather than using colons:
2.0.0.1.0.d.b.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1._spf.example.com IN A 127.0.0.2
Note that when using IPv6 addresses, a DNS A lookup is still performed, not an AAAA.
Similar to the first example, SPF macros can also be used to permit IP address ranges without having to specify every single one within your global policy.
This is achieved using wildcard DNS records combined with the SPF macro r modifier in order to reverse the parts of the IP address to create a reverse lookup. Unfortunately, with IPv4 addresses, this is limited to /24, /16 and /8 ranges.
For example:
example.com IN TXT "v=spf1 exists:%{ir}.%{v}.arpa._spf.example.com -all"
The %{ir} macro will be replaced with the source IP address of the email, but with the parts reversed (due to the r modifier). For example, 203.0.113.1 will become 1.113.0.203.
The %{v} macro will be replaced with in-addr if the source address is IPv4, and ip6 if the source address is IPv6. This isn't technically required, but it creates a semantically-correct reverse lookup using the .arpa top-level domain.
To permit 203.0.113.0/24 to send as your domain, add the following wildcard record:
*.113.0.203.in-addr.arpa._spf.example.com IN A 127.0.0.2
In many cases, your SPF record will be mainly populated by third-party SaaS systems that each serve a very specific purpose. For example, a customer service system which exclusively sends messages as support@example.com, or a finance system which uses billing@example.com.
Usually, the SPF configuration required for these third-party services means that you are granting them the ability to send as absolutely any address on your domain. This isn't good for security, as in the event that the third-party is compromised, makes a configuration error, or simply turns malicious, the reputation of your entire domain could be at stake.
It is also quite inefficient to globally whitelist a third-party provider when it only needs to send-as one single address.
Using SPF macros, it is possible to restrict third-party services who send on your behalf to a single or small number of addresses. This is great hardening and defence-in-depth, and also helps to keep a tight grip on the list of people authorised to send as your domain.
The following example will allow a third-party service to send email as your domain, but only using noreply@example.com. Other SMTP FROM addresses will be treated as an SPF fail:
example.com IN TXT "v=spf1 include:%{l}._spf.example.com -all"
The %{l} macro will be replaced with the local part of the sender address. For example, if the sender is steve@example.com, the 'local' part is steve.
Then add the third-party SPF records (as generated by the provider, e.g. Mailgun, Sendgrid, etc) to your zone under the noreply name:
noreply._spf.example.com IN TXT "v=spf1 include:spf.example.org"
When the third-party sends an email using the noreply address, the email will be permitted. However, if the third-party attempts to send using a different address, it will not match the SPF policy and will be treated as an SPF fail (as defined by the -all or ~all within your global policy).
Note that the automated account setup within some email services will fail to validate/accept your domain if you set it up this way, as they usually just do a basic Grep against your policy without actually evaluating macros. This can usually be solved by either adding the SPF macro after your domain has been verified, or by politely asking customer support to manually verify it for you.
Within large organisations, there may be multiple people maintaining an SPF record, and potentially lots of different third-party systems sending email. This can sometimes make it challenging to maintain an accurate record of what each whitelisted IP address is for.
By utilising SPF macros to split each permitted IP address into a separate DNS record, you then have the ability to add a DNS TXT record 'next to' the A record to note what it's for. This helps to keep a live audit log of what each IP address is for, who put it there, and when it can be removed.
For example:
203.0.113.1._spf.example.com IN A 127.0.0.2 203.0.113.1._spf.example.com IN TXT "Customer service email system, see internal change notice #6511"
Keep in mind that the content of these TXT records is public, and may allow for the enumeration of systems, etc. However, the reality is that the existence and purpose of these systems is probably already fairly easy to identify, so in most cases I wouldn't consider this to be a significant risk.
If you're still using legacy BIND zone files, you could also add the note using a comment (line starting with a semicolon).
Unfortunately the real-world usage of SPF macros is fairly low, despite them being widely supported by MTAs. They unlock various advanced features, both from a usability and security point of view, so I really hope to see their usage increase in the future.
Whilst researching for this article, I came across this useful guidance from the M3AAWG (Messaging, Malware and Mobile Anti-Abuse Working Group), which while not specifically about SPF macros, still provides a wealth of best-practise and recommendations for managing an SPF policy.