I’ve spent years building and securing cloud-native apps, and one lesson keeps coming back: even a single small virtual machine (VM) exposed to the internet can become your weakest link if you don’t harden it. This guide walks you through a pragmatic, step-by-step approach to hardening a small cloud VM against the most common attacks. I’ll share what I actually do when I provision a VM for a test workload or a small production service, including specific tools, configurations, and checks you can reproduce yourself.
Start with a minimal, well-known image
My first rule is to pick a small, supported OS image and keep it minimal. I lean toward Ubuntu LTS, Debian stable, or a minimal CentOS/Rocky image depending on the environment. Avoid images with bundled control panels or unnecessary services (cPanel, Plesk, etc.) unless you need them.
Why minimal? Fewer packages = fewer potential vulnerabilities. Before you flip the VM power switch, consider:
Choose an official image from your cloud provider (AWS, GCP, Azure, DigitalOcean, Hetzner) rather than a community build.Pick an LTS or long-term support image to reduce frequent upgrade churn.Use filesystem encryption if you’re dealing with sensitive data at rest and your provider supports it.Secure access and authentication
Remote access is the most common attack vector, so lock it down first.
Disable password authentication for SSH. Use SSH keys and ensure passphrases on private keys. In /etc/ssh/sshd_config set PasswordAuthentication no and PermitRootLogin no.Create a non-root user and add them to sudoers with limited permissions. I often create a user “deploy” with sudo privileges for administration and deploy keys for automation.Use SSH agent forwarding carefully. It’s convenient but can be risky; prefer deploying keys to the server via secure CI/CD if possible.Consider Multi-Factor Authentication (MFA) for cloud console access and for SSH using solutions like Duo, Google Authenticator, or WebAuthn (security keys).Harden the SSH configuration
In addition to disabling password auth and root login, I apply these tweaks in /etc/ssh/sshd_config:
Change the SSH port from 22 to a non-standard port (security through obscurity — not sufficient alone but reduces noisy scans).Use AllowUsers to restrict which accounts can log in via SSH.Enable LoginGraceTime 30 and MaxAuthTries 3 to limit brute-force opportunities.Set strong KexAlgorithms and Ciphers if your distro uses weak defaults — modern OpenSSH typically has safe defaults, but it’s worth verifying.Network-level hardening
Limit exposure with network rules and host-based firewalls.
Use cloud security groups/network ACLs to allow only required inbound ports (typically 22/SSH, 443/HTTPS, and maybe 80/HTTP). Block everything else by default.On the host, enable a host firewall (ufw on Ubuntu, firewalld on CentOS, or iptables/nftables). I usually allow SSH + app ports and deny the rest:<pre>ufw default deny incomingufw allow proto tcp from 203.0.113.0/24 to any port 22 # if you can restrict SSH to your office IPufw allow 443/tcpufw enable</pre>Consider using a jump host (bastion) in a private subnet to access VMs rather than exposing SSH on every machine.Keep the system updated and reduce what’s installed
Patch management is boring but decisive. I automate updates for security patches where acceptable, or use configuration management systems (Ansible, Salt, Puppet) to regularly apply updates and track package state.
Enable unattended-upgrades on Debian/Ubuntu for security packages, or use your provider’s image lifecycle tooling.Remove unnecessary packages: mail servers, ftp, telnet, old interpreters you don’t use. Every package is an attack surface.Install only required runtime components. If you need Python, install only the system Python and use virtualenvs rather than globally installing pip packages that could conflict with OS-managed packages.Limit permissions and use least privilege
Follow least privilege for services and processes.
Run services under dedicated users with no shell when possible (e.g., nginx, postgres users).Use file permissions and ACLs to restrict access to keys, config files, and secrets. Private keys should be 600 and owned by root or the service user.Where available, leverage Linux capabilities or systemd sandboxing: PrivateTmp=yes, NoNewPrivileges=yes, ProtectSystem=strict in your systemd service units.Protect secrets and credentials
Secrets leaking from a VM is catastrophic. I avoid storing long-lived credentials on VMs where possible.
Use cloud provider IAM roles (AWS IAM role, GCP service account) so the VM can access only the resources it needs without embedding static keys.For application secrets, use a secrets store: HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or encrypted files injected at runtime by your orchestration system.Audit environment variables and dotfiles. Developers sometimes leave API keys or tokens in shell history or personal config files.Logging, monitoring, and intrusion detection
Hardening doesn’t stop at configuration — you need visibility.
Send logs to a central log collector (ELK/Opensearch, Splunk, or cloud logging like AWS CloudWatch/GCP Logging). Don’t keep all logs on the VM only.Enable OS-level auditd rules for sensitive files and login events.Run a lightweight IDS/IPS like OSSEC, Wazuh, or defensive tools like fail2ban to react to brute-force patterns. fail2ban is great for blocking repeated SSH attempts.Set up system metrics and alerts (Prometheus + node_exporter or cloud metrics) for CPU, memory, disk, and unusual spikes that might indicate compromise.Harden services and application stack
Each service you run needs its own hardening checklist.
Web servers: enable HTTPS (Let’s Encrypt is easy), disable weak TLS ciphers, use HSTS, and tune your TLS config using tools like Mozilla SSL Configuration Generator.Databases: bind to localhost or private subnet, enforce strong passwords, and restrict which users can access which databases. Use encrypted storage for sensitive databases.Containers: if you run Docker, avoid running containers privileged, drop capabilities, and use user namespaces. Consider using Podman or containerd with runtime restrictions.Backup and recovery
Assume compromise — can you recover? Backups are part of hardening.
Automate regular backups and test restores. An untested backup is a false promise.Encrypt backups in transit and at rest, and keep at least one backup offsite or in a different region.Automate checks and create a checklist
Hardening is a process. I codify checks to avoid manual drift. Here’s a compact checklist I run on each new VM:
| Item | Expected |
| OS image | Official minimal LTS |
| SSH | Key auth, root disabled, non-standard port allowed |
| Firewall | Cloud rules + host firewall deny all but required |
| Updates | Unattended security updates or scheduled patching |
| Secrets | No static keys in repo; use IAM/secrets manager |
| Monitoring | Centralized logs + alerts in place |
| Backups | Automated + tested restores |
Finally, treat hardening as continuous: review configurations after significant changes, rotate credentials periodically, and keep a small, automated test that verifies the most critical hardening items. If you want, I can share example Ansible playbooks or a hardened systemd unit template in a follow-up post — just tell me which distro or cloud provider you use.
