Server hardening basics for email workloads

Why email servers need above-average hardening

An email server is a particular kind of target. It sits on the public internet with multiple always-listening ports. It holds authentication credentials for the systems that submit mail. It has network egress to every MX record on earth, which is useful for delivery and useful for exfiltration. It typically runs with elevated privileges to bind low-numbered ports and manage mail queues. And it produces logs that, if not protected, reveal the internal sending patterns of the organization.

The compromise of an email server is also unusually expensive. Reputation damage from spam sent through a compromised MTA takes months to recover from. Credential exposure cascades into every downstream system that authenticates to it. And the lateral movement potential — into the corporate network, into customer data, into the billing stack — is often significant because email servers tend to be trusted by peers. The reputation layer in particular is worth understanding in its own right; sender reputation fundamentals covers what inbox providers are actually measuring when your compromised server starts emitting spam.

This is not a theoretical risk profile. The breaches we've seen over the past eighteen months — Equifax in September, WannaCry and NotPetya in the spring and early summer last year — each had email and adjacent infrastructure components in their kill chains. Systems that were patched, segmented, and monitored survived better than systems that were not, and the surviving systems were rarely exotic; they were ordinary servers with the standard hardening controls actually in place. Our own earliest production deployment, covered in building a dedicated MTA stack, reflects the same pattern applied in practice.

The baseline before anything else

Before applying any specific hardening control, four preconditions need to be in place. Skipping these is how hardening efforts become cargo-cult exercises that don't actually improve security posture.

These are boring. They are also the preconditions that most "I applied CIS Level 2" deployments quietly failed on because they skipped the unglamorous infrastructure work before turning to the exciting hardening checklist.

SSH: the entry point most commonly compromised

SSH is the control plane for your servers. It is also the single most attacked service on the public internet — automated brute-force against SSH on port 22 runs continuously across the entire IPv4 address space. A server with default SSH configuration and a reasonably-guessable password will be compromised within hours of being exposed. A server with proper SSH hardening will survive indefinitely under the same conditions.

The SSH hardening checklist that actually matters

/etc/ssh/sshd_config# Authentication — keys only, never passwords
PasswordAuthentication       no
ChallengeResponseAuthentication no
PubkeyAuthentication         yes

# Root access — disabled entirely
PermitRootLogin              no

# Protocol — explicit version pin (SSH-2 only)
Protocol                     2

# Restrict ciphers to modern, strong algorithms
KexAlgorithms                curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
Ciphers                      chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs                         hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Limit concurrent auth attempts; deter brute-force
MaxAuthTries                 3
MaxSessions                  5
LoginGraceTime               30s

# Optional: restrict which user accounts can SSH
AllowUsers                   deploy ops

# Idle timeout — disconnect abandoned sessions
ClientAliveInterval          300
ClientAliveCountMax          2

# Disable unused features
AllowAgentForwarding         no
AllowTcpForwarding           no
X11Forwarding                no
PermitUserEnvironment        no
PermitEmptyPasswords         no

The three decisions most teams get wrong

1. Leaving password auth enabled "just in case." This is the single most common mistake. If passwords are enabled for any user, the entire SSH surface is vulnerable to brute-force, regardless of how strong individual passwords might be. Disable passwords entirely; use keys.
2. Changing the SSH port as a primary defense. Port 2222 instead of 22 reduces automated brute-force noise by about 99%, but it does not stop a targeted attacker. Treat port change as supplemental hygiene, not as a security control. The real control is key-only auth.
3. Skipping 2FA for admin accounts. SSH key auth plus a TOTP second factor (Google Authenticator, FreeOTP, or hardware tokens like YubiKey) is dramatically stronger than keys alone. The compromise of a developer's laptop no longer means compromise of production. For email servers specifically, where the blast radius of a compromise is outsized, this extra layer is justified.

Firewall and network exposure

An email server has a specific set of ports that need to be open to the internet: 25 (inbound SMTP from other MTAs), 587 (SMTP submission from authenticated clients), 465 (SMTPS submission, less common but still used). Everything else should be closed. Management interfaces, monitoring endpoints, and internal services should be behind a firewall or a VPN, not exposed to the open internet.

nftables — minimal email server ruleset# /etc/nftables.conf
flush ruleset

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # Accept loopback and established connections
        iif "lo" accept
        ct state established,related accept
        ct state invalid drop

        # ICMP — allow basic signaling
        ip protocol icmp limit rate 10/second accept
        ip6 nexthdr icmpv6 accept

        # SSH — from specific management networks only
        tcp dport 22 ip saddr { 203.0.113.0/24, 198.51.100.0/24 } accept

        # Email service ports
        tcp dport { 25, 465, 587 } accept

        # Drop everything else silently
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
    }

    chain output {
        type filter hook output priority 0; policy accept;
    }
}

The pattern above represents a useful default: deny by default, allow specific services, limit SSH to management networks. For a server behind a cloud provider's own firewall (AWS Security Groups, DigitalOcean Cloud Firewall), run both layers. The host firewall is your defense if the cloud firewall misconfigures; the cloud firewall is your defense if the host is compromised and its iptables get rewritten.

IPv6: a frequently-forgotten attack surface

Many teams carefully configure iptables rules for IPv4 and forget that ip6tables is a completely separate table. The result is a server with strict IPv4 rules and fully-open IPv6. Attackers know this pattern and specifically scan via IPv6 looking for it. The nftables example above uses the inet family which covers both v4 and v6 in one table, which is one reason to prefer nftables over the split iptables/ip6tables model if you're building new.

Fail2ban and brute-force mitigation

Even with key-only SSH and a hardened firewall, attackers will continuously probe every exposed service. Fail2ban is the standard tool for turning those probes into temporary IP bans, reducing log noise and eliminating the tail risk of a slow brute-force that eventually gets lucky against some service you've forgotten about.

A sensible Fail2ban configuration for email servers

/etc/fail2ban/jail.local[DEFAULT]
# Ban timing — increment for repeat offenders
bantime          = 1h
findtime         = 10m
maxretry         = 5
bantime.increment = true
bantime.factor    = 2
bantime.maxtime   = 1w

# Avoid locking yourself out
ignoreip         = 127.0.0.1/8 ::1 203.0.113.0/24

# Use nftables for ban actions (matches modern kernel stack)
banaction        = nftables-multiport
backend          = systemd

# Email alerting
destemail        = security@example.com
sender           = fail2ban@mail01.example.com
mta              = sendmail
action           = %(action_mwl)s

[sshd]
enabled          = true
port             = ssh
maxretry         = 3
findtime         = 10m
bantime          = 24h

[postfix]
enabled          = true
port             = smtp,submission,submissions
filter           = postfix
logpath          = /var/log/mail.log
maxretry         = 5

[postfix-sasl]
enabled          = true
port             = smtp,submission,submissions
filter           = postfix-sasl
logpath          = /var/log/mail.log
maxretry         = 5

[recidive]
enabled          = true
filter           = recidive
logpath          = /var/log/fail2ban.log
bantime          = 7d
findtime         = 1d
maxretry         = 5

Three details worth calling out. First, the recidive jail catches IPs that keep getting banned on other jails; after a few cycles, they earn a much longer ban. Second, the postfix-sasl jail specifically catches failed SMTP authentication, which is the email-server equivalent of failed SSH login and just as worth protecting. Third, using nftables-multiport matches the firewall stack modern Linux uses natively.

Mandatory access control: SELinux and AppArmor

Traditional Unix file permissions (chmod, chown) control who can do what. Mandatory access control systems — SELinux on RHEL/CentOS, AppArmor on Ubuntu/Debian — add a second layer: what specific programs can do, regardless of the user's privileges. If an attacker gains root via a vulnerability in Postfix or PowerMTA, MAC systems still constrain what that exploited process is allowed to touch.

The operational rule

Whichever MAC system your distribution defaults to, leave it enabled. The reflexive "SELinux is broken, let me disable it" response to a configuration issue is exactly the wrong instinct. The correct response is to read the audit log (/var/log/audit/audit.log for SELinux, dmesg and /var/log/kern.log for AppArmor), understand why the policy is denying the action, and either adjust the policy or adjust the configuration.

SELinux troubleshooting for Postfix# Check current mode — should be "Enforcing"
getenforce

# See which SELinux booleans affect Postfix
getsebool -a | grep postfix
# postfix_local_write_mail_spool --> on

# Check for recent denials
sudo ausearch -m avc -ts recent | grep postfix

# See human-readable troubleshooting suggestions
sudo sealert -a /var/log/audit/audit.log

# If a specific port needs a non-standard label, add it:
# (example: allow Postfix to listen on port 2525)
sudo semanage port -a -t smtp_port_t -p tcp 2525

For PowerMTA and similar commercial MTAs, vendor documentation usually includes the SELinux or AppArmor profile adjustments needed. For Postfix and Exim, the distribution-supplied policies work out of the box for standard configurations; custom port numbers, chroot setups, or unusual directory layouts are what require policy tweaks.

Keeping SMTP from becoming an open relay

An open relay — an SMTP server that accepts mail from anyone and forwards it anywhere — is both the most dangerous and the most common email-server misconfiguration. Any misconfigured MTA is discovered by scanners within hours of going online. Within a day of being an open relay, your IP is on Spamhaus SBL. Within a week, the domain's reputation is irreparably damaged. Correct relay configuration, combined with proper SPF/DKIM/DMARC setup, is what prevents both accidental relay and unauthorized use of your domain — the multi-domain dimension of this is covered in authentication alignment and policy review for multi-domain senders.

How to not be an open relay (Postfix example)

postfix main.cf — anti-relay configuration# Define your own networks — only these can relay without auth
mynetworks               = 127.0.0.0/8 [::1]/128 10.0.0.0/8
mynetworks_style         = host

# Define who you ACCEPT mail FOR (inbound)
mydestination            = $myhostname, localhost.$mydomain, localhost
# Accept nothing else as final delivery

# Relay restrictions — the critical chain
smtpd_relay_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    defer_unauth_destination

# Recipient restrictions — deny unknown recipients, RBL checks
smtpd_recipient_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_unauth_destination,
    reject_unknown_recipient_domain,
    reject_non_fqdn_recipient,
    reject_rbl_client zen.spamhaus.org,
    permit

# HELO restrictions
smtpd_helo_required        = yes
smtpd_helo_restrictions    =
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_invalid_helo_hostname,
    reject_non_fqdn_helo_hostname

# Sender restrictions
smtpd_sender_restrictions  =
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_unknown_sender_domain,
    reject_non_fqdn_sender

Verify before exposing to the internet

Every time you deploy a new email server or change relay configuration, test it before the MX record goes live. The standard test:

relay test# From OUTSIDE your trusted networks, attempt to relay
telnet mail01.example.com 25
# EHLO test.example.net
# MAIL FROM: <attacker@evil.example>
# RCPT TO: <victim@somewhere-else.example>
# If the server responds 250 without auth: YOU ARE AN OPEN RELAY
# Correct response: 554 5.7.1 Relay access denied

# Automated test alternatives:
# - abuse.net's relay tester (public web form, cautious — rate limited)
# - Test from any machine you know is NOT in your mynetworks

TLS configuration for SMTP

SMTP TLS is opportunistic by design — a sending MTA will fall back to cleartext if TLS negotiation fails. This means broken TLS is worse than no TLS: it silently downgrades to plaintext. A correct TLS configuration prevents the downgrade while still allowing interop with MTAs that don't support it. The broader TLS-for-SMTP story, including the operational history of transport encryption in outbound mail, is covered in TLS for SMTP.

postfix main.cf — TLS configuration# Server certificate and key (use Let's Encrypt or a commercial cert)
smtpd_tls_cert_file        = /etc/letsencrypt/live/mail01.example.com/fullchain.pem
smtpd_tls_key_file         = /etc/letsencrypt/live/mail01.example.com/privkey.pem

# Opportunistic TLS for inbound
smtpd_tls_security_level   = may
smtpd_tls_loglevel         = 1
smtpd_tls_received_header  = yes

# Strong protocols and ciphers only
smtpd_tls_protocols        = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_mandatory_ciphers = high
smtpd_tls_exclude_ciphers   = aNULL, eNULL, EXPORT, DES, 3DES, MD5, RC4

# TLS session cache for performance
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache

# Outbound — opportunistic TLS with verification where possible
smtp_tls_security_level    = may
smtp_tls_loglevel          = 1
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# DANE support — opportunistic DNSSEC-validated TLS
smtp_tls_security_level    = dane
# Requires DNSSEC-validating resolver

Certificate management in 2018

Let's Encrypt certificates are now production-quality, renewed automatically by certbot or acme.sh, and trusted universally. For inbound SMTP, a Let's Encrypt certificate is the default choice and there is rarely a reason to pay for a commercial certificate. The ACME v2 protocol (announced last week) also supports wildcard certificates, which simplifies deployments with many subdomains.

The one exception where commercial certificates remain relevant: deployments with extended-validation or organization-validation requirements, typically driven by compliance audits that have not yet updated their assumptions about Let's Encrypt. Those are cases where the cert type is a compliance artifact, not a security requirement.

Patch management and the Meltdown-class question

January's disclosure of Meltdown and Spectre changed the patch-management conversation. Prior to 2018, patching was mostly about responding to specific application CVEs as they appeared. Meltdown forced everyone to also think about microcode updates, kernel-level mitigations, and the performance implications of mitigation strategies.

Unattended upgrades — the baseline automation

/etc/apt/apt.conf.d/50unattended-upgrades (Debian/Ubuntu)Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
    "${distro_id}ESMApps:${distro_codename}-apps-security";
    "${distro_id}ESM:${distro_codename}-infra-security";
};

// Auto-reboot at 03:00 if a kernel update requires it
Unattended-Upgrade::Automatic-Reboot            "true";
Unattended-Upgrade::Automatic-Reboot-Time       "03:00";

// Email notifications on actions taken
Unattended-Upgrade::Mail                        "ops@example.com";
Unattended-Upgrade::MailReport                  "on-change";

// Remove unused kernels to avoid /boot filling up
Unattended-Upgrade::Remove-Unused-Kernel-Packages  "true";
Unattended-Upgrade::Remove-Unused-Dependencies     "true";

The Meltdown/Spectre era underscores that patch discipline cannot rely on humans remembering to run apt upgrade. Either automatic updates are enabled for security-tagged packages or the server has a scheduled, documented patching window. "We'll patch when we have time" is not a discipline; it is the absence of one.

Audit logging and file integrity

Hardening without auditing is hoping you got everything right. Audit logging and file integrity monitoring are the capabilities that tell you when something has changed, when an action occurred, and whether your baseline still holds.

auditd — the built-in Linux audit subsystem

auditd is part of every serious Linux distribution and captures security-relevant events at the kernel level: process execution, file access, configuration changes, authentication events. A minimal ruleset for email servers:

/etc/audit/rules.d/email-server.rules# Monitor changes to SSH configuration
-w /etc/ssh/sshd_config -p wa -k sshd_config

# Monitor changes to firewall rules
-w /etc/nftables.conf -p wa -k firewall
-w /etc/iptables/ -p wa -k firewall

# Monitor email configuration changes
-w /etc/postfix/main.cf -p wa -k postfix_config
-w /etc/postfix/master.cf -p wa -k postfix_config
-w /etc/aliases -p wa -k mail_aliases

# Monitor user and group changes
-w /etc/passwd -p wa -k identity
-w /etc/group  -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/sudoers -p wa -k privilege

# Track sudo usage
-a exit,always -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=4294967295 -k sudo_exec

# Track loading of kernel modules
-a always,exit -F arch=b64 -S init_module -S delete_module -k modules

AIDE — file integrity monitoring

AIDE (Advanced Intrusion Detection Environment) takes a cryptographic snapshot of the filesystem and alerts when files change outside of expected patterns. The baseline snapshot is stored off-server; periodic checks compare the current state against the baseline. A file modification that shouldn't have happened — a binary replaced, a configuration file edited, a rootkit installed — shows up immediately.

The operational complexity of AIDE is real. Legitimate system activity (package updates, log rotation, routine configuration changes) produces noise that must be tuned out, or the signal becomes useless. A common pattern: run AIDE nightly, compare against baseline, send a daily digest that someone actually reads. Teams that set this up and never look at the output are worse off than teams that didn't set it up at all, because they've built false confidence.

What CIS Benchmarks give you

The Center for Internet Security publishes free benchmarks for most major operating systems, applications, and services. For Linux, the CIS Benchmarks define hundreds of specific hardening controls with rationale, remediation steps, and audit commands. They are not the only baseline, but they are the most widely used, the most regularly updated, and the most cost-effective (free) starting point for a formal hardening effort.

Levels 1 vs 2 — what to aim for

CIS Benchmarks divide controls into two levels. Level 1 is the practical baseline for most production systems — controls that don't significantly impact functionality or administrator convenience. Level 2 adds stricter controls aimed at environments with higher security requirements; some Level 2 controls are genuinely inconvenient for operators.

For most email servers, Level 1 is the right target. Level 2 is appropriate when compliance frameworks (PCI DSS for payment-adjacent environments, HIPAA for healthcare, sector-specific regulations for financial services) explicitly require it. Don't reach for Level 2 as a default just because it sounds more thorough; the operational friction usually isn't worth it unless the compliance requirement is real.

Frequently asked questions

Do we need all of this on a cloud-hosted server where the provider already hardens the host?

Yes. The cloud provider hardens the hypervisor and the physical infrastructure; the guest OS running inside the VM is your responsibility. Misconfiguring an AWS EC2 instance or a DigitalOcean Droplet is not mitigated by the provider's underlying security. The controls above apply the same way.

What's the right cadence for reviewing hardening?

Quarterly formal review, with automated scanning in between. The formal review checks for configuration drift, new CVEs affecting your stack, and any controls that have become outdated. The automated scanning (CIS-CAT, Lynis, or commercial equivalents) catches drift between formal reviews.

How do we balance security against operational convenience?

Most tension resolves through clearer abstractions. Ops teams that need shell access should have named accounts, not shared logins. Developers that need to read logs should have read-only audit log access, not root on production. Automation that needs to deploy should have scoped service accounts with specific capabilities, not administrator credentials. The friction usually comes from shortcuts that were never actually secure.

Is SELinux worth the trouble?

Yes. The learning curve is real but the protection is meaningful. When Postfix or PowerMTA is compromised via some unexpected vulnerability, SELinux is often what contains the damage to the immediate process rather than allowing lateral movement. Disabling SELinux because it's "annoying" is trading a real control for a momentary convenience; the correct move is to learn how to read the audit log and adjust policy when needed.

What about AppArmor for mail servers specifically?

AppArmor on Ubuntu/Debian ships with profiles for Postfix and Dovecot in enforcing mode. For Exim and PowerMTA, profile creation may require additional work. The principle is the same as SELinux: leave it enabled, learn the logs, adjust when needed.

Should the email server be in its own network segment?

Yes, for any production deployment. A VLAN or a VPC subnet dedicated to email infrastructure, with strict firewall rules between that segment and the rest of the network, limits the blast radius of any single-host compromise. Expensive? No — it's mostly configuration. Effective? Very — it's one of the highest-value controls available.

How do we handle forgotten credentials on hardened servers?

Out-of-band recovery: cloud provider console access, or a separate bastion host with its own credential path, or a break-glass procedure documented in advance. Every hardened server needs a recovery path, but that path should not be an attack surface during normal operation.

Closing perspective

Server hardening sounds like a one-time project and behaves like an ongoing discipline. The initial pass — applying CIS controls, configuring SSH and firewall, enabling Fail2ban, turning on SELinux or AppArmor — is work, but it ends. The ongoing discipline — keeping controls applied after configuration changes, reviewing audit logs regularly, patching promptly, scanning for drift — is what separates a hardened server a year later from a server that used to be hardened.

For email workloads specifically, the stakes are higher than for many other server types. A compromised email server produces reputation damage on top of the usual compromise costs, and the reputation damage outlasts the remediation work. The incremental effort to harden an email server above whatever baseline applies to your other infrastructure is usually modest, and the incremental risk reduction is substantial. The asymmetry favors doing the work.

The year we are now in — Meltdown and Spectre disclosed in January, GDPR coming into force in May, the general trend of regulatory and threat pressure increasing — has made the operational case for hardening easier to justify to leadership than it has been in previous years. Teams that weren't getting hardening budget in 2017 are often getting it in 2018. Use the window. The controls that are cheap now become expensive to retrofit later, and the attack surfaces that are tolerable now will not be tolerable for the rest of the year.

For teams just starting this work: the order of operations matters. Get the preconditions in place first — inventory, logging, time sync, separated admin identity. Then SSH hardening, firewall, Fail2ban. Then SELinux or AppArmor. Then MTA configuration and TLS. Then audit logging and file integrity. Then scan against CIS. The list is long; the steps are well-defined; the tooling is mature. Nothing in the list is novel; all of it is boring and applying it is what separates servers that survive the next incident from servers that don't. Boring is the goal.