DANE vs MTA-STS for SMTP: which inbound TLS policy fits your domain?

When another mail server delivers a message to your domain, STARTTLS can encrypt the connection. Without an additional policy, that encryption is usually opportunistic: the sender tries TLS, but a network attacker may be able to remove the STARTTLS capability or redirect the SMTP connection to another host.

Two standards address that gap:

• DANE for SMTP publishes certificate associations in DNSSEC-protected TLSA records.
• MTA-STS publishes an HTTPS-served policy that names valid MX hosts and requires publicly trusted certificates.

Both can make inbound SMTP delivery resistant to downgrade and interception attacks for senders that support them. They do it through different trust systems, and neither one magically changes the behavior of every sending MTA on the Internet.

That last point is important. A receiving domain can publish a policy, but the sending server has to implement the policy, discover it, and apply it. The practical choice is therefore not only about which standard is stronger on paper. It is also about which dependencies your team can operate reliably and which senders your important correspondents use.

The short answer

Choose DANE when your DNS is DNSSEC-signed, your inbound mail platform supports DANE, and your team is comfortable operating DNSSEC and TLSA records as part of certificate changes.

Choose MTA-STS when DNSSEC is unavailable or impractical, but you can serve a stable HTTPS policy and use publicly trusted certificates on every MX host. MTA-STS is often the easier first deployment for a domain hosted on ordinary web and mail infrastructure.

Choose both when inbound mail is valuable enough to justify two policy paths. A sender that supports both can apply DANE's DNSSEC-backed association and MTA-STS's policy checks. RFC 8461 specifically says that MTA-STS validation must not override a failing DANE validation.

In every case, publish SMTP TLS Reporting if you can receive and review the reports. TLS-RPT is visibility, not an enforcement policy, but it is the feedback loop that makes certificate, DNS, and policy failures visible.

What these policies protect

SMTP is a store-and-forward protocol. A sender resolves the recipient domain's MX records, opens a connection to one of the resulting hosts, and may negotiate TLS with STARTTLS.

Ordinary opportunistic TLS protects against passive observation when the connection and negotiation are left alone. It does not reliably protect against an active attacker who can:

• remove 250 STARTTLS from the SMTP capabilities
• alter or replace the MX response
• send the connection to an impostor host
• present a certificate that the sender accepts for the wrong reason

The DANE SMTP specification and MTA-STS specification both define ways for a sender to know what secure delivery should look like. If the sender cannot meet the requirement, a conforming sender does not quietly treat the failed secure connection as an ordinary cleartext delivery when the policy is enforcing.

Neither policy provides end-to-end message encryption. TLS protects one SMTP hop at a time, and the receiving MTA can read the message. Neither policy authenticates the From: address or replaces SPF, DKIM, or DMARC. For the identity side of mail, see Beyond DMARC, DKIM, and SPF: MTA-STS + TLS-RPT for inbound SMTP.

How DANE works for inbound SMTP

DANE, or DNS-Based Authentication of Named Entities, uses DNSSEC to authenticate a TLSA record. The record binds a certificate or public key to a particular service name, protocol, and port.

For an MX host named mx1.example.com, the SMTP TLSA name is:

_25._tcp.mx1.example.com.

An illustrative TLSA record might look like this:

_25._tcp.mx1.example.com. IN TLSA 3 1 1 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f

The three numbers are the certificate usage, selector, and matching type. 3 1 1 means a DANE-EE association using the certificate's SubjectPublicKeyInfo and a SHA-256 hash. The hexadecimal value above is intentionally illustrative and will not match a real certificate. Generate the record from the certificate or key that the live SMTP service will present.

DANE prerequisites

An inbound DANE deployment needs all of these pieces:

1. A DNSSEC-signed zone with a valid chain of trust from the parent delegation.
2. DNSSEC-aware sending MTAs that can validate the MX and TLSA responses.
3. An MX record for every receiving path.
4. A TLSA record at the correct _25._tcp name for every MX host that should offer DANE.
5. An SMTP service that supports STARTTLS and presents the certificate or key described by TLSA.
6. An operational process for DNSSEC signing, DNS changes, and certificate or key rollover.

The MX hostname matters. TLSA is not published at _25._tcp.example.com merely because example.com is the recipient domain. It is normally published below each actual MX hostname. If the domain has a primary and backup MX, forgetting the backup creates a split security posture.

DANE's trust model

The trust anchor for the TLSA association is the DNSSEC chain, not the general-purpose public CA store. That lets a domain use a certificate that is not issued by a public CA when the TLSA usage permits it. It also limits the assertion to the named service and port.

This is DANE's main security advantage. A public CA that can issue certificates for arbitrary domains is not needed to authenticate the SMTP service. An attacker who can obtain an unrelated public certificate cannot satisfy a correctly published DANE-EE association.

The tradeoff is that DNSSEC becomes part of the mail delivery critical path. The sender needs a validated answer for the relevant DNS data. A DNSSEC bogus result, a broken signature, or a failed TLSA lookup is not an invitation to try the same host without authentication. The sender skips that host or delays delivery, depending on the situation, to avoid turning a DNS failure into a downgrade.

DANE for SMTP is normally opportunistic DANE. A sender that finds a valid TLSA association uses authenticated TLS. A sender that can securely determine that no TLSA association exists can use ordinary opportunistic TLS. A sender that does not implement DANE may still deliver with ordinary STARTTLS or, if that is unavailable, cleartext. A TLSA record cannot force a non-DANE sender to use it.

How MTA-STS works for inbound SMTP

MTA-STS uses two publication points. The DNS record announces a policy version, and an HTTPS endpoint serves the policy body.

For example.com, the DNS record is:

_mta-sts.example.com. IN TXT "v=STSv1; id=2026090401"

The policy is fetched from:

https://mta-sts.example.com/.well-known/mta-sts.txt

An enforcing policy could contain:

version: STSv1
mode: enforce
mx: mx1.example.com
mx: mx2.example.com
max_age: 604800

The sending MTA validates that the selected MX matches a listed mx pattern, supports STARTTLS, and presents a non-expired certificate that matches the MX hostname and chains to a CA trusted by the sender. The policy host itself also needs a valid certificate for mta-sts.example.com.

MTA-STS has three modes:

• testing lets delivery continue as if the validation failure were not enforcing, while compatible senders can report the failure through TLS-RPT.
• enforce tells a compatible sender not to deliver to an MX that fails the policy checks. The message is normally deferred and retried rather than immediately treated as a permanent failure.
• none is used to remove an existing policy cleanly after cached policies have expired.

MTA-STS prerequisites

An inbound MTA-STS deployment needs:

1. Control of the _mta-sts TXT record.
2. An HTTPS service at mta-sts.example.com for the policy domain.
3. A publicly trusted certificate valid for the policy host.
4. A policy file with every live MX hostname or an intentionally scoped wildcard.
5. Publicly trusted, correctly named certificates on every MX host in the policy.
6. Monitoring for HTTPS availability, TXT and policy consistency, certificate expiry, and policy cache age.

The HTTPS endpoint is not a replacement for the DNS record. Senders use the TXT id to notice that the policy may have changed, then fetch the policy body. Update the HTTPS policy first and change the TXT id second. Reversing that order can cause a sender to see the new ID and retrieve an old or incomplete policy.

MTA-STS policies do not automatically inherit from a parent domain. A sender delivering to mail.example.com does not use the MTA-STS policy for example.com. Publish and operate policies for subdomains that have their own inbound SMTP identity.

DANE and MTA-STS side by side

The table hides an important nuance: the standards do not make exactly the same promise to every sender. DANE's DNSSEC signal is designed to resist downgrade when the sender validates it, including authenticated denial when TLSA is absent. MTA-STS has a bootstrap and refresh dependency on unauthenticated DNS discovery. A sender that has not already cached the MTA-STS policy may fail to discover it if an attacker blocks the TXT query or HTTPS fetch.

MTA-STS reduces that window by having senders cache policies for max_age and refresh them before expiry. It is still a real difference. DANE is generally the cleaner fit when the domain already operates a healthy DNSSEC chain and wants DNS to be the source of truth for the SMTP key association.

Security and operational tradeoffs

DANE: stronger DNS binding, higher DNSSEC responsibility

DANE makes the domain's DNSSEC chain the authority for the SMTP key association. That reduces dependence on a large public CA set and gives a sender a cryptographically authenticated answer about the TLSA record and its nonexistence.

The cost is not just checking a DNSSEC box. The team must keep the DS delegation, DNSKEYs, RRSIGs, authoritative name servers, recursive validation path, and TLSA data healthy. A DNS provider that publishes DNSSEC but mishandles a key rollover can make a zone validate as bogus. For a DANE-aware sender, that can delay mail to the affected MX rather than merely produce a warning.

DANE also maps policy to individual MX hosts. That is precise, but it increases the chance that a forgotten backup or regional hostname will not be covered. It can be an advantage for a domain that wants tight key control and has a small, well-understood MX set.

MTA-STS: simpler trust deployment, larger Web PKI surface

MTA-STS avoids requiring the recipient zone to be DNSSEC-signed. It fits environments where the mail provider already supplies publicly trusted MX certificates and an HTTPS endpoint can be deployed or delegated. Its testing mode gives teams a useful way to find stale MX records and certificate problems before asking participating senders to enforce the policy.

The cost is a larger set of moving parts: DNS TXT discovery, the policy host's DNS and certificate, HTTPS availability, the policy file, TXT ID updates, and every MX certificate. A publicly trusted CA is also part of the authentication model. A CA compromise or an attacker who obtains a valid certificate for an allowed MX name is a risk that DNSSEC-backed DANE-EE does not share in the same way.

MTA-STS also has a cache bootstrap limitation. A sender must first learn that a policy exists and fetch it successfully. An attacker who can interfere with that first discovery can sometimes cause that sender to proceed without MTA-STS. Long, practical max_age values and refresh monitoring reduce the exposure, but they do not make MTA-STS identical to DNSSEC-authenticated policy discovery.

Failure modes that create delivery incidents

The most dangerous failures are the ones that occur during an otherwise routine infrastructure change.

DANE failure modes

• Broken DNSSEC chain: An expired RRSIG, missing DS, bad DNSKEY rollover, or signing error causes a bogus response. A DANE-aware sender should not use that response as a reason to fall back to an unauthenticated connection.
• Wrong TLSA owner name: The record is published for the recipient domain instead of the actual MX hostname, or the port and protocol labels are wrong.
• Wrong digest: The SMTP server presents a new key or certificate that does not match the TLSA association. The TLS handshake fails for DANE-aware senders.
• Incomplete MX coverage: The primary MX has a valid association but a backup MX does not, or the backup's certificate and TLSA record disagree.
• Unsafe key rotation: A key is replaced on the server before the new TLSA record is visible to senders, or the old association is removed before cached DNS data has aged out. Publish overlapping associations and coordinate DNS and service changes rather than changing both abruptly.
• Provider limitations: A DNS host or mail provider may support ordinary TXT records and TLS but not the DNSSEC or TLSA workflow required for a reliable deployment.

MTA-STS failure modes

• Policy and MX mismatch: DNS advertises mx2.example.com, but the policy lists only mx1.example.com, or a wildcard is too narrow for the real hostname.
• Certificate mismatch or expiry: The MX certificate lacks the MX hostname in its SAN, is expired, does not chain to a trusted CA, or is not the certificate actually presented on one regional node.
• Broken policy host: The HTTPS host has a DNS, TLS, HTTP status, content-type, or path problem. A redirect from the well-known URL is not a valid substitute for serving the policy at the required endpoint.
• TXT ID race: The TXT ID changes before the new policy is available, so senders cache a version they cannot successfully apply.
• Stale policy caches: A sender may continue using a previous policy for up to its max_age. Removing an MX or changing policy constraints requires keeping the old arrangement working during the cache window.
• Uncovered subdomain: A policy on example.com does not cover mail delivered to mail.example.com.

In both models, the right response to a failure is not to delete the policy immediately. First determine whether the report indicates a real attack, a stale sender, or your own configuration change. Removing a protection during an incident can turn a visible delivery failure into silent plaintext delivery.

TLS-RPT is the common visibility layer

TLS-RPT lets receiving domains ask compatible sending MTAs for aggregate counts of successful and failed TLS sessions. It can report both DANE and MTA-STS policy outcomes, including certificate mismatches, expired certificates, DNSSEC failures, invalid TLSA records, and MTA-STS fetch errors.

The record is published at _smtp._tls:

_smtp._tls.example.com. IN TXT "v=TLSRPTv1; rua=mailto:tls-reports@example.com"

Reports are aggregate and are not a per-message forensic log. A successful session count is useful as a heartbeat, while a sudden group of certificate-host-mismatch, tlsa-invalid, dnssec-invalid, or sts-policy-fetch-error results points toward a specific operational problem.

TLS-RPT is not universal telemetry. It depends on the sending MTA implementing the reporting standard and choosing to send reports. Keep local SMTP connection logs, DNSSEC validation alerts, certificate expiry monitoring, and HTTPS endpoint checks as well.

A practical decision guide

Use these questions in order:

1. Does the domain have a complete, monitored DNSSEC chain? If no, MTA-STS is the immediate candidate. If yes, DANE is viable, but confirm that your mail host and important sending partners support it.
2. Can every MX present a certificate or key that can be represented in TLSA? If yes, DANE can provide a precise DNS-backed binding. If the mail provider only supports publicly trusted certificates and MTA-STS automation, MTA-STS may be simpler.
3. Can the team operate an HTTPS policy host and public certificate renewals? If yes, MTA-STS is practical even without DNSSEC. If no, do not publish an MTA-STS policy you cannot keep available.
4. Is the domain high value or subject to a strong transport requirement? If yes, deploy both where the infrastructure supports both, and separately configure mandatory DANE for partners that require it. Do not assume public Internet senders will all enforce either policy.
5. Are there many providers, aliases, or changing MX paths? MTA-STS may be easier to express as one policy, but only if the complete MX set and certificate lifecycle are managed. DANE may be cleaner when each host is stable and DNSSEC automation is mature.

The resulting choices usually look like this:

Do not choose DANE merely because it sounds more cryptographically pure, and do not choose MTA-STS merely because it has a testing mode. The best policy is the one whose DNS, certificates, MX inventory, caches, and monitoring can stay correct during a bad day.

Roll out without creating a mail black hole

Start with an inventory, not a DNS record. Record every MX hostname, IP family, certificate chain, TLS version, hosting owner, and planned failover path. Test each host from an external DANE-aware and MTA-STS-aware sender where possible.

For DANE:

1. Enable and validate DNSSEC for the zone and its delegation.
2. Confirm the live certificate and key on every MX.
3. Generate TLSA records from the exact service certificate or key.
4. Publish TLSA and verify the DNSSEC status from independent validating resolvers.
5. Test normal delivery, certificate renewal, backup-MX delivery, and DNS failure handling.
6. Keep an overlapping TLSA association during planned key changes.

For MTA-STS:

1. Serve the policy host with a valid certificate and the exact well-known path.
2. Include every live MX and test the policy syntax externally.
3. Publish mode: testing with a conservative max_age while validating the real mail paths.
4. Publish TLS-RPT and correlate reports with MX and certificate logs.
5. Fix stale, missing, and incorrectly named MX paths.
6. Update the HTTPS body before changing the TXT ID, then allow caches to refresh.
7. Move to mode: enforce only after a complete certificate renewal and failover test.

For either standard, leave the policy in place during ordinary certificate renewals and DNS changes. A short maintenance window that breaks a strict sender's validation can become a queue of delayed messages, followed by non-delivery reports if the underlying defect is not fixed before retry limits expire.

Main point

DANE and MTA-STS are not competing versions of the same DNS record. DANE uses DNSSEC to bind each SMTP service to a certificate or key. MTA-STS uses cached HTTPS policy and public CA validation to tell senders which MX hosts and TLS certificates are acceptable.

DANE is the stronger fit for teams that already operate DNSSEC and want DNS-backed key control. MTA-STS is the more accessible fit for teams that can manage HTTPS and public certificates but cannot make DNSSEC part of their mail platform. High-value domains can use both, with TLS-RPT and independent monitoring around them.

The deciding question is not which acronym wins. It is which trust model and operational process your domain can keep correct when MX hosts fail, certificates rotate, DNS caches disagree, and senders retry delivery from systems you do not control.