Protect HestiaCP servers with an ipset list

HestiaCP, in combination with the Fail2ban and Iptables daemons, provides a robust security baseline for Linux servers. However, managing thousands of individual firewall rules in Iptables degrades CPU performance and introduces network latency. IPset addresses this issue by storing dynamic list sets directly in kernel memory using high-speed hash structures. This allows for low-overhead, volumetric blocks at scale.

In this guide, I structure the verification of system prerequisites, mandatory firewall rule backups, whitelist creation to prevent accidental lockouts, script syntax audits, and systematic monitoring and rollback procedures.

1) System prerequisites verification#

Before deploying any automated scripts or IPset firewalls on HestiaCP, ensure all backend dependencies are present and that the control panel service is running:

# 1. Verify if the IPset utility is installed and output its version
which ipset
ipset --version

# 2. Check Fail2ban installation and command line response
which fail2ban-client
fail2ban-client --version

# 3. Check if HestiaCP daemon is active and running (service names may vary: hestia, hestia-web, or hestiacp)
systemctl list-units --type=service | grep -i hestia
# And query the specific service status (e.g., hestia)
systemctl status hestia

# 4. Verify the path and permissions of the native blacklist script
ls -la /usr/local/hestia/install/common/firewall/ipset/blacklist.sh

2) Mandatory preventive firewall backup#

Editing network routing rules or iptables configurations carries an inherent risk of administrative lockouts. Always perform a timestamped backup of the active Iptables ruleset and IPset tables before editing:

# 1. Back up all active Iptables rules to a backup file
iptables-save > /root/iptables-backup-$(date +%Y%m%d).rules

# 2. Back up the active IPset tables structure
ipset list > /root/ipset-backup-$(date +%Y%m%d).txt

# 3. Optional: Back up CSF configuration directories if running in parallel
tar czf /root/csf-backup-$(date +%Y%m%d).tar.gz /etc/csf/

3) IP whitelisting (lockout prevention)#

To prevent administrators, external monitors, or development tools from getting blocked by public spam/abuse feeds, provision a dedicated whitelist set and apply a priority input rule before the block/reject statements:

# 1. Create a whitelist set structure in kernel memory (hash:net to support both single IPs and CIDR subnets)
ipset create WHITELIST hash:net

# 2. Append trusted IP addresses or CIDR blocks to the set
ipset add WHITELIST 10.0.0.1
ipset add WHITELIST 192.168.1.0/24
# Optional: Add your own static administrative public IP
# ipset add WHITELIST 203.0.113.50

LOCKOUT WARNING: Ensure you add your own current public IP address (or VPN/Monitoring IPs) to the WHITELIST before enabling blocking rules. Keep a separate, active SSH session open as a backup while executing firewall updates to avoid lockouts.

# 3. Create a priority ACCEPT rule in Iptables (inserted at the top of the chain)
iptables -I INPUT -m set --match-set WHITELIST src -j ACCEPT

PERSISTENCE NOTE: HestiaCP reconstructs Iptables rules when applying panel changes or after firewall service reboots. Therefore, register the whitelist ACCEPT rule formally in the interface: navigate to Firewall ➔ Add Rule, select Action: ACCEPT, Protocol: ALL, Port: 0, and Address: ipset:WHITELIST. This guarantees the rule is persisted and prioritized over drop rules.

Technical Comparison: Linear Iptables Rules vs IPset Hash Tables#

4) Blacklist script validation#

Ensure the HestiaCP blacklist script is properly formatted, executable, and runs cleanly:

# 1. Grant execution rights to the root user
chmod +x /usr/local/hestia/install/common/firewall/ipset/blacklist.sh

# 2. Perform a dry run syntax check on the bash script
bash -n /usr/local/hestia/install/common/firewall/ipset/blacklist.sh

# 3. Execute in verbose mode to diagnose connection issues or processing errors
sudo bash -x /usr/local/hestia/install/common/firewall/ipset/blacklist.sh 2>&1 | head -n 50

# 4. Manually run the script in the foreground to fetch the initial block list
sudo /usr/local/hestia/install/common/firewall/ipset/blacklist.sh

The script consolidates multiple verified public security feeds, without placeholders:

• ipset-blacklist: conf
• Project Honey Pot: Threat List
• TOR Exit Nodes: Exit List
• MaxMind GeoIP Anonymous Proxies: Sample Feed
• BruteForceBlocker: List Feed
• Spamhaus DROP: List
• C.I. Army Malicious IPs: List
• blocklist.de: List
• GreenSnow: List Feed
• Firehol Level 1: Feed
• Stopforumspam: Feed

5) Register the list in HestiaCP#

Add the IPset feed to the HestiaCP system visually:

1. Open the HestiaCP dashboard and click the gear icon (Server Settings).
2. Click the Firewall tab.
3. Open IP Set Lists.
4. Click Add IP list.

Configure fields as follows:

• Name: BLOCK-LIST
• Data source: Select the malicious IP list or confirm the selector for your HestiaCP version (recent versions use url: for remote feeds; if script: is not available, register the list via direct feed URLs from section 4). Verify the canonical ipset update binary with: ls -la /usr/local/hestia/bin/v-update-firewall-ipset.
• IP version: IPv4
• Auto update: Yes

6) Create firewall rules#

Ipset + HestiaCP blocking flow#

┌────────────────────────── KERNEL ──────────────────────────┐
│                                                            │
│   IPSET BLOCK-LIST ──match──▶ iptables INPUT               │
│     (hash:ip)                      │  DROP / REJECT        │
│                                    ▼                       │
│   Malicious packet ◀── discarded ◀─┘                       │
└────────────────────────────────────────────────────────────┘
    ▲ feeds              ▲ dynamic ban           ▲ DROP/REJECT rules
    │                    │                      │
  blacklist.sh        Fail2ban          HestiaCP Firewall UI
    ▲
    │ trigger — daily cron 03:00

Map the registered ipset:BLOCK-LIST to the corresponding blocking rules in the panel:

Rule 1: Deny inbound TCP#

• Action: REJECT
• Protocol: TCP
• Port: 0 (denies all ports)
• Address: ipset:BLOCK-LIST
• Comment: DROP_BAD_TCP

Rule 2: Deny inbound/outbound UDP#

• Action: DROP
• Protocolo: UDP
• Porta: 0
• Address: ipset:BLOCK-LIST
• Comment: DROP_BAD_UDP

(Note: Using DROP instead of REJECT is recommended for UDP and ICMP protocols. REJECT sends ICMP Port Unreachable messages back to the sender, which confirms that the host is active and listening).

Rule 3: Deny ICMP packets (ping)#

• Action: DROP
• Protocolo: ICMP
• Porta: 0
• Address: ipset:BLOCK-LIST
• Comment: DROP_BAD_ICMP

7) Post-configuration and verification#

Run operational checks to confirm the IPset rules are loaded correctly:

# 1. Verify that the BLOCK-LIST set is active in the kernel
ipset list BLOCK-LIST | head -n 10

# 2. Count the active IP count inside the set (skipping header lines)
ipset save BLOCK-LIST | grep -c "^add "
# Or
ipset list BLOCK-LIST -o save | grep "^add" | wc -l

# 3. Verify that the IPset table is bound to active Iptables rules
iptables-save | grep -- "-m set --match-set"
# Or, in human-readable output
iptables -L -n | grep -E "set (BLOCK-LIST|WHITELIST)"

# 4. Inject a test IP to verify the set accepts manual entry
ipset add BLOCK-LIST 192.0.2.1

# 5. Query the test IP in the active list
ipset test BLOCK-LIST 192.0.2.1

8) Automating ipset updates via cron#

To keep the blocklist current, schedule automatic script runs via cron:

# Test manual update script execution
sudo /usr/local/hestia/install/common/firewall/ipset/blacklist.sh

# Edit root crontab to update automatically every day at 3:00 AM
# Use 'flock' to prevent concurrent executions if list downloads hang or lag
# 0 3 * * * /usr/bin/flock -n /var/run/blacklist-ipset.lock /usr/local/hestia/install/common/firewall/ipset/blacklist.sh > /dev/null 2>&1

### 8-b) ipset persistence across reboots

By default, IPsets are stored in volatile kernel memory and are cleared on reboots. To make them persistent, dump active sets to a rules file and restore them on boot using a systemd service:

Dump active sets to rules file#

ipset save > /etc/ipset.rules

Create `/etc/systemd/system/ipset-restore.service`:

[Unit] Description=Restore IPset Rules Before=network.target

[Service] Type=oneshot ExecStart=/sbin/ipset restore -f /etc/ipset.rules ExecStop=/sbin/ipset save -f /etc/ipset.rules RemainAfterExit=yes

[Install] WantedBy=multi-user.target

Enable the service:

systemctl daemon-reload systemctl enable ipset-restore.service

9) Monitoring ipset and Fail2ban#

Implement health monitoring to ensure the threat list is populated. System network faults or drive failures may interrupt updates:

# 1. Check general Fail2ban jail status
fail2ban-client status
fail2ban-client status sshd

# 2. List currently banned IPs on the sshd jail
fail2ban-client status sshd | grep -i "banned"

# 3. Monitor Fail2ban logs in real-time
tail -n 100 /var/log/fail2ban.log

Health audit script (monitor-ipset.sh)#

Save this monitoring script to /usr/local/sbin/monitor-ipset.sh to audit your list size periodically, confirming that the set exists and counting member IPs accurately:

#!/bin/bash
# monitor-ipset.sh - Verifies minimum threat list size
SET_NAME="${1:-BLOCK-LIST}"
MIN_IPS="${2:-100}"

if ! ipset list "$SET_NAME" >/dev/null 2>&1; then
    MSG="CRITICAL: IPset $SET_NAME does not exist!"
    logger -t monitor-ipset "$MSG"
    which mail >/dev/null 2>&1 && echo "$MSG" | mail -s "Security Alert: IPset Size" [email protected]
    exit 2
fi

# Count only 'add' entries (skipping headers)
COUNT=$(ipset save "$SET_NAME" | grep -c "^add ")

if [ "$COUNT" -lt "$MIN_IPS" ]; then
    MSG="ALERT: The IPset list $SET_NAME is critically small: $COUNT IPs loaded."
    logger -t monitor-ipset "$MSG"
    which mail >/dev/null 2>&1 && echo "$MSG" | mail -s "Security Alert: IPset Size" [email protected]
    exit 1
else
    echo "OK: IPset $SET_NAME running with $COUNT IPs."
    exit 0
fi

(Note: Sending terminal email alerts requires the mailutils package: sudo apt install mailutils).

10) Rollback and contingency plan#

If legitimate access is blocked, execute these rollback commands to flush IPset rules and restore your iptables state:

# 1. Delete list references from active iptables chains
iptables -D INPUT -m set --match-set BLOCK-LIST src -j DROP 2>/dev/null
iptables -D INPUT -m set --match-set BLOCK-LIST src -j REJECT 2>/dev/null

# 2. Flush and destroy kernel IPset sets
ipset destroy BLOCK-LIST
ipset destroy WHITELIST

# 3. Restore original iptables state from the latest preventive backup
RESTORE_FILE=$(ls -t /root/iptables-backup-*.rules | head -1)
echo "Restoring from: $RESTORE_FILE"
iptables-restore < "$RESTORE_FILE"

11) Diagnostic logs#

Inspect these logs to debug firewall problems or panel update failures:

• IPset/Systemd Events:

journalctl --since "1 hour ago" | grep -i "ipset"

• Iptables Block Logs (Kernel messages):

journalctl -k --since "1 hour ago" | grep -i -E "DROP|REJECT" (or tail -100 /var/log/messages | grep -i -E "DROP|REJECT" on legacy syslog setups)

• HestiaCP System logs:

tail -100 /usr/local/hestia/log/system.log

Checklist: HestiaCP ipset firewall validation#

Use this audit checklist when provisioning or validating IPsets on HestiaCP:

• [ ] Prerequisites: Validated IPset, Fail2ban, and blacklist.sh presence.
• [ ] Backups: Completed timestamped iptables rule backup and IPset structure backups.
• [ ] Whitelisting: Provisioned a hash:net WHITELIST set and registered the rule in the panel.
• [ ] Execution: Ran syntax test (bash -n) and verified manual blacklist execution.
• [ ] Registration: Registered BLOCK-LIST in HestiaCP with auto-updates enabled.
• [ ] Rules: Enforced firewall rules with REJECT (TCP) and DROP (UDP/ICMP) actions in the panel.
• [ ] Automation: Enabled cron updates at 3:00 AM.
• [ ] Monitoring: Configured tail checks and deployed the size audit script (monitor-ipset.sh).
• [ ] Contingency: Verified rollback plan is documented and working.

Troubleshooting FAQ#

What should I do if I am accidentally locked out by IPset in HestiaCP?#

Access your server using your VPS/cloud emergency console (VNC or serial terminal). Run ipset flush BLOCK-LIST to immediately flush the active in-memory set, or temporarily stop the firewall using v-stop-firewall so you can add your IP to the administrative whitelist.

Why is iptables -L extremely slow after configuring IPset?#

By default, iptables -L attempts reverse DNS lookups on every single IP address or subnet. Always append the numeric flag: iptables -L -n (or use iptables -S) to disable DNS resolution and get instant terminal output.

How do I ensure IPset lists persist after a server reboot?#

HestiaCP has built-in boot routines to restore panel-managed firewall rules. For native OS-level persistence on Debian or Ubuntu, install ipset-persistent or netfilter-persistent (netfilter-persistent save).

Conclusion#

Protecting HestiaCP servers with IPset combines kernel-level hash structures with straightforward web panel management. In this guide, I structured lockout prevention with a hash:net WHITELIST, automated public threat feeds without CPU exhaustion, and established a reliable rollback runbook. With systemd persistence enabled and regular monitoring active, the server stays protected against tens of thousands of daily threats completely autonomously.

See also

Cybersecurity How to securely change the default SSH port on Linux

Infrastructure How to switch FTP service in HestiaCP: VSFTPD vs proftpd

Infrastructure Why opening ports in Oracle cloud fails: security lists vs Linux iptables

This post is licensed under CC BY-NC.