How to Set Up SSH Keys on VPS (2025) — Security Guide

Why SSH Keys Are Essential for VPS Security

SSH keys are the foundation of secure server access. For VPN setups that also need security, see Install WireGuard on VPS. Password authentication is inherently vulnerable to brute-force attacks, dictionary attacks, and credential stuffing. Automated bots scan the entire IPv4 address space and attempt to log in to any server with SSH exposed on port 22. A VPS with default SSH configuration will receive thousands of login attempts per day. For self-hosting security, see VPS for Self-Hosting and Docker on Ubuntu VPS.

SSH keys eliminate this threat entirely. Instead of a password that can be guessed, SSH keys use public-key cryptography. You generate a key pair: a private key (kept secret on your machine) and a public key (placed on the server). Authentication happens through a cryptographic challenge-response that proves you possess the private key without ever transmitting it. For European hosting locations, see Germany VPS and VPS for VPN.

The ed25519 elliptic curve algorithm is the current gold standard for SSH keys. It offers equivalent security to RSA-4096 with a much smaller key size (just 64 bytes compared to 512 bytes for RSA-4096), faster computation, and better resistance to side-channel attacks. Ed25519 has been the default in OpenSSH since version 6.5 (released in 2014) and is supported by all modern SSH clients and servers.

Prerequisites

Step 1: Generate an SSH Key Pair (ed25519)

Run the following command on your local machine (not on the VPS) to generate a new ed25519 SSH key pair.

On Linux and macOS

ssh-keygen -t ed25519 -C "your-email@example.com" -f ~/.ssh/id_ed25519_vps

On Windows (PowerShell)

ssh-keygen -t ed25519 -C "your-email@example.com" -f $env:USERPROFILE\.ssh\id_ed25519_vps

The parameters explained:

• -t ed25519: Specifies the ed25519 elliptic curve algorithm
• -C "your-email@example.com": A comment/label to identify the key (usually your email)
• -f ~/.ssh/id_ed25519_vps: The file path for the key pair. Using a descriptive filename helps when you have multiple keys

You will be prompted to set a passphrase. This passphrase encrypts your private key on disk. If someone steals your private key file, they still need the passphrase to use it.

Enter passphrase (empty for no passphrase): [type your passphrase]
Enter same passphrase again: [type it again]

Two files are created:

• ~/.ssh/id_ed25519_vps — Your private key (NEVER share this)
• ~/.ssh/id_ed25519_vps.pub — Your public key (this goes on the VPS)

Alternative: RSA Keys (for Legacy Compatibility)

If your server runs an extremely old SSH version that does not support ed25519, generate an RSA key with at least 4096 bits.

ssh-keygen -t rsa -b 4096 -C "your-email@example.com" -f ~/.ssh/id_rsa_vps

Step 2: Copy the Public Key to Your VPS

There are several methods to transfer your public key to the VPS. Choose the one that works best for your situation.

Method 1: ssh-copy-id (Recommended)

The easiest method on Linux and macOS. This command copies your public key to the VPS and sets the correct permissions automatically.

ssh-copy-id -i ~/.ssh/id_ed25519_vps.pub root@your-vps-ip

You will be prompted for your VPS password (this is the last time you need it).

Method 2: Manual Copy (Works Everywhere)

If ssh-copy-id is not available, you can copy the key manually using SSH.

# Display your public key
cat ~/.ssh/id_ed25519_vps.pub

Copy the entire output (it starts with "ssh-ed25519" and ends with your comment). Then connect to your VPS and add it to the authorized_keys file.

ssh root@your-vps-ip

# On the VPS:
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... your-email@example.com" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Method 3: Copy via VPS Control Panel

Many VPS providers (including Inferno) allow you to paste your public key during server creation or through the control panel. This is the most secure method because the key is injected during the provisioning process and never transmitted over the network.

Display your public key and paste it into the control panel:

cat ~/.ssh/id_ed25519_vps.pub

Step 3: Test Your SSH Key Login

Before making any changes that could lock you out, verify that key-based authentication works. Open a new terminal window and connect to your VPS.

ssh -i ~/.ssh/id_ed25519_vps root@your-vps-ip

You should be logged in without being prompted for a password (only for your key passphrase if you set one). If this works, your key is correctly configured.

If you are prompted for a password instead, the key authentication is not working. Do not proceed to Step 4 until this is resolved. Common causes include incorrect file permissions on the VPS (see troubleshooting at the end of this guide).

Step 4: Disable Password Authentication

Once you have confirmed that key-based login works, disable password authentication to eliminate the brute-force attack vector entirely.

Edit the SSH Daemon Configuration

# On the VPS, edit the SSH config
nano /etc/ssh/sshd_config

Find and modify the following lines (some may be commented out with # — uncomment them):

# Disable password authentication
PasswordAuthentication no

# Disable empty passwords (should already be no)
PermitEmptyPasswords no

# Disable keyboard-interactive authentication
KbdInteractiveAuthentication no

# Disable challenge-response authentication
ChallengeResponseAuthentication no

# Enable public key authentication (should already be yes)
PubkeyAuthentication yes

# Only allow the root user to log in with keys (optional)
PermitRootLogin prohibit-password

Save the file and restart the SSH daemon to apply the changes.

sshd -t  # Test the configuration for syntax errors
systemctl restart sshd

Step 5: Change the Default SSH Port

Port 22 is the default SSH port, which makes it the primary target for automated scanning and brute-force attacks. Changing the port to a non-standard value eliminates 99% of automated login attempts because bots typically only scan port 22.

Choose a Port

Select a port number between 1024 and 65535. Avoid well-known service ports. You can use a memorable number or generate a random high port.

# Generate a random high port
shuf -i 20000-65000 -n 1

For this example, we will use port 48291. Replace this with your chosen port.

Update the SSH Configuration

nano /etc/ssh/sshd_config

Find the Port directive (near the top of the file) and change it:

Port 48291

Update the Firewall

Before restarting SSH, allow the new port through the firewall. If you use UFW:

ufw allow 48291/tcp comment "Custom SSH port"
ufw reload

Restart SSH.

sshd -t
systemctl restart sshd

Connect on the New Port

From your local machine, specify the port when connecting.

ssh -i ~/.ssh/id_ed25519_vps -p 48291 root@your-vps-ip

To make this permanent, add an SSH config entry on your local machine (see Step 9).

Step 6: Set Up Fail2Ban

Fail2Ban monitors log files for suspicious activity and automatically bans IP addresses that show malicious signs (too many failed login attempts, port scanning, etc.). Even with SSH keys and a non-standard port, Fail2Ban provides an additional layer of protection.

Install Fail2Ban

apt install fail2ban -y

Configure Fail2Ban for SSH

Create a local configuration file that overrides the defaults.

cat > /etc/fail2ban/jail.local << 'EOF'
[DEFAULT]
# Ban IP for 1 hour
bantime = 3600
# Time window for counting failures
findtime = 600
# Max failures before banning
maxretry = 3
# Use iptables for banning
banaction = iptables-multiport

[sshd]
enabled = true
port = 48291
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 86400
findtime = 600
EOF

Note: Update the port value to match the SSH port you configured in Step 5. If you kept port 22, use port = ssh.

Enable and start Fail2Ban.

systemctl enable fail2ban
systemctl start fail2ban
systemctl status fail2ban

Verify Fail2Ban is Working

# Check the status of the SSH jail
fail2ban-client status sshd

The output should show the number of currently banned IPs and the total failures. Check the Fail2Ban log for ban events.

tail -f /var/log/fail2ban.log

Useful Fail2Ban Commands

# List all active jails
fail2ban-client status

# Unban a specific IP address
fail2ban-client set sshd unbanip 1.2.3.4

# Ban a specific IP address
fail2ban-client set sshd banip 1.2.3.4

# Check logs
tail -20 /var/log/fail2ban.log

Step 7: Configure SSH Hardening (sshd_config)

Apply additional security settings to make your SSH configuration as robust as possible. Open the SSH daemon configuration file.

cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup
nano /etc/ssh/sshd_config

Here is a hardened sshd_config with all recommended settings. Apply these changes one at a time, verifying that SSH still works after each modification.

# /etc/ssh/sshd_config — Hardened configuration

# Network
Port 48291
AddressFamily inet
ListenAddress 0.0.0.0

# Authentication
PermitRootLogin prohibit-password
PubkeyAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no
UsePAM yes

# Session limits
MaxAuthTries 3
MaxSessions 5
LoginGraceTime 30

# Connection limits
MaxStartups 3:50:10

# Disable unused features
X11Forwarding no
AllowAgentForwarding yes
AllowTcpForwarding yes
PermitTunnel no
PermitUserEnvironment no

# Logging
SyslogFacility AUTH
LogLevel VERBOSE

# Cryptographic policy
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com
KexAlgorithms curve25519-sha256

# Keep-alive
ClientAliveInterval 300
ClientAliveCountMax 2

# Disable .rhosts files
HostbasedAuthentication no
IgnoreRhosts yes

# Banner (optional)
# Banner /etc/issue.net

Key settings explained:

• MaxAuthTries 3: Limits authentication attempts per connection to 3. Failed attempts trigger Fail2Ban
• MaxSessions 5: Limits the number of simultaneous sessions per network connection
• LoginGraceTime 30: Disconnects if authentication is not completed within 30 seconds
• MaxStartups 3:50:10: Rate-limits new connections. Starts rejecting connections when 3 unauthenticated connections exist (with 50% probability, linearly increasing to 100% at 10 connections)
• X11Forwarding no: Disables X11 forwarding unless you specifically need it (prevents potential security issues)
• LogLevel VERBOSE: Logs more information for security auditing and Fail2Ban detection
• Ciphers/MACs/KexAlgorithms: Restricts to only the most secure cryptographic algorithms, disabling older ones like 3DES, RC4, and SHA-1
• ClientAliveInterval 300 / ClientAliveCountMax 2: Sends keepalive every 5 minutes and disconnects after 2 missed keepalives (10 minutes total). Prevents zombie sessions from consuming resources

Test and restart SSH.

sshd -t
systemctl restart sshd

Step 8: Set Up SSH Agent Forwarding

SSH agent forwarding allows you to use your local SSH keys to authenticate with other servers through a jump host (bastion server). This is useful when you have a VPS that serves as a gateway to other internal servers.

Start the SSH Agent

# Start the agent
eval "$(ssh-agent -s)"

# Add your private key to the agent
ssh-add ~/.ssh/id_ed25519_vps

The agent keeps your decrypted private key in memory. You only need to enter your passphrase once when adding the key.

Enable Agent Forwarding

When connecting to the jump host, use the -A flag to enable agent forwarding.

ssh -A -i ~/.ssh/id_ed25519_vps -p 48291 root@your-vps-ip

From the jump host, you can now SSH to any server that accepts your public key without copying your private key to the jump host.

# From the jump host, connect to an internal server
ssh root@internal-server-ip

Step 9: Manage Multiple SSH Keys

If you have multiple VPS instances or services (GitHub, GitLab, etc.) that require different SSH keys, the SSH config file lets you manage them easily.

Create an SSH Config File

On your local machine, create or edit ~/.ssh/config.

cat > ~/.ssh/config << 'EOF'
# Global defaults
Host *
    AddKeysToAgent yes
    IdentityFile ~/.ssh/id_ed25519_vps
    ServerAliveInterval 60
    ServerAliveCountMax 3

# Inferno VPS
Host inferno
    HostName your-vps-ip
    Port 48291
    User root
    IdentityFile ~/.ssh/id_ed25519_vps
    ForwardAgent yes

# GitHub
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_github

# Hetzner VPS
Host hetzner
    HostName your-hetzner-ip
    Port 22
    User root
    IdentityFile ~/.ssh/id_ed25519_hetzner
EOF

Set correct permissions on the config file.

chmod 600 ~/.ssh/config

Now you can connect to any host using a short alias:

ssh inferno        # Connects to Inferno VPS
ssh hetzner        # Connects to Hetzner VPS
ssh github.com     # Uses your GitHub key

Generate Separate Keys for Each Service

# Key for GitHub
ssh-keygen -t ed25519 -C "github" -f ~/.ssh/id_ed25519_github

# Key for Hetzner
ssh-keygen -t ed25519 -C "hetzner" -f ~/.ssh/id_ed25519_hetzner

# Key for a work server
ssh-keygen -t ed25519 -C "work" -f ~/.ssh/id_ed25519_work

Copy each public key to its respective server or service. Having separate keys means you can revoke access to one service without affecting the others.

Step 10: Troubleshooting Common SSH Key Issues

1. "Permission denied (publickey)"

Cause: The server cannot find or read your public key, or your private key does not match.

Solution: Check that the public key is in ~/.ssh/authorized_keys on the server. Verify permissions: chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys on the server. Ensure you are using the correct private key with ssh -i /path/to/key. Check the server's auth log at /var/log/auth.log for details.

2. Connection Refused After Changing Port

Cause: The firewall did not allow the new port, or sshd failed to restart with the new configuration.

Solution: Use your VPS provider's console (VNC/web console) to access the server directly. Check that the firewall allows the new port with ufw status. Verify the sshd_config syntax with sshd -t. Check if sshd is running with systemctl status sshd.

3. "WARNING: UNPROTECTED PRIVATE KEY FILE"

Cause: The private key file has permissions that are too open. SSH requires strict permissions.

Solution: Fix the permissions on your local machine: chmod 600 ~/.ssh/id_ed25519_vps. If the ~/.ssh directory is too open: chmod 700 ~/.ssh.

4. Agent Has No Identities

Cause: No keys have been added to the SSH agent.

Solution: Add your key: ssh-add ~/.ssh/id_ed25519_vps. Verify with ssh-add -l. If the agent is not running, start it: eval "$(ssh-agent -s)".

5. Locked Out After Disabling Password Auth

Cause: Key authentication was not working when you disabled password auth.

Solution: Use your VPS provider's web console or VNC access to log in directly. Re-enable password authentication temporarily, fix the key setup, test it, and then disable passwords again. Every reputable VPS provider offers console access for exactly this scenario.

Frequently Asked Questions

Are SSH keys more secure than passwords?

Yes, significantly. SSH keys use 256-bit elliptic curve cryptography (ed25519), which is computationally infeasible to crack. Passwords, even long ones, are vulnerable to brute-force attacks, dictionary attacks, and phishing. SSH keys also cannot be reused across servers (unlike passwords that people often reuse).

Can I use the same SSH key on multiple servers?

Technically yes, but it is not recommended for security. If one server is compromised, the attacker gains access to all servers using the same key. Generate a unique key pair for each server or at least for each security domain (work, personal, clients).

What happens if I lose my private key?

You lose access to any server that only accepts that key for authentication. If you still have an active session, you can add a new key. Otherwise, use your VPS provider's web console to log in and add a new public key to ~/.ssh/authorized_keys. This is why keeping a backup of your private keys in a secure location (encrypted storage, password manager) is essential.

How do I add a new SSH key after disabling password auth?

If you have an existing active SSH session, add the new key directly: echo "new-public-key-content" >> ~/.ssh/authorized_keys. If you are locked out, use your VPS provider's web console (VNC) to access the server and add the key.

Should I disable root SSH login?

It is a common security recommendation, but PermitRootLogin prohibit-password (the setting used in this guide) only disables password-based root login while keeping key-based root access. This is a good balance. For maximum security, create a non-root user with sudo access and set PermitRootLogin no. However, this adds complexity without a meaningful security benefit if SSH keys are properly configured.

How often should I rotate SSH keys?

Rotate keys at least once a year, or immediately if you suspect a key has been compromised, an employee with access leaves, or a device storing the key is lost. Rotating keys on a VPS is straightforward: generate a new key pair, add the new public key to authorized_keys, test login, and remove the old public key.

Does Fail2Ban work with SSH keys?

Fail2Ban primarily protects against password-based brute-force attacks. With SSH keys and password auth disabled, Fail2Ban has fewer events to act on. However, it still provides value by detecting and banning IPs that attempt authentication with invalid keys or probe for vulnerabilities.

Can I use an SSH key with a passphrase and still automate deployments?

Yes, using an SSH agent. Run ssh-add to add the key to the agent (entering the passphrase once), then all subsequent SSH commands use the agent for authentication without prompting. In CI/CD pipelines, use key files without passphrases (stored as secrets in the CI/CD platform) or use agent forwarding.