Secure SSH on Linux

Applicable systems: Ubuntu / Debian (service name is ssh)
Example target server IP: 163.7.7.182
Regular user: wynnzen
Core goal: SSH key authentication + TOTP one-time code login, password authentication disabled, and no direct SSH login for root.

1. Overview of Secure Login Approaches ​

The core idea behind securing remote server login is to move from "single-point password defense" to "defense in depth". Common options include:

This document focuses on the actionable configuration of the first five.

2. SSH Key Authentication and Disabling Password Login ​

2.1 Generate a key pair locally ​

ssh-keygen -t ed25519 -C "your_email@example.com"

About the passphrase:

• It is not required by SSH, but it is strongly recommended.
• It protects your local private key file, not the server login password.
• On production machines, laptops, and any device that could be lost, you must set a strong passphrase.
• In automation / CI/CD scenarios, do not use an interactive passphrase; use a dedicated deploy key or a short-lived SSH certificate instead.
• If you already generated a key without a passphrase, you can add one later:

ssh-keygen -p -f ~/.ssh/id_ed25519

• Combine it with ssh-agent so you don't have to type it every time:

eval "$(ssh-agent -s)"
ssh-add -t 8h ~/.ssh/id_ed25519

2.2 Upload the public key to the server ​

ssh-copy-id -i ~/.ssh/id_ed25519.pub wynnzen@163.7.7.182

If password login has already been disabled, configure it manually on the server:

mkdir -p /home/wynnzen/.ssh
echo "ssh-ed25519 AAAA...your_public_key..." >> /home/wynnzen/.ssh/authorized_keys
chown -R wynnzen:wynnzen /home/wynnzen/.ssh
chmod 700 /home/wynnzen/.ssh
chmod 600 /home/wynnzen/.ssh/authorized_keys

Permission requirements:

• ~/.ssh → 700
• ~/.ssh/authorized_keys → 600
• The owner must be the target user

2.3 Modify the SSH configuration ​

It is recommended to use a drop-in file so that editing the main config directly does not get overwritten by cloud-init:

vim /etc/ssh/sshd_config.d/01-custom.conf

Write:

PubkeyAuthentication yes
PasswordAuthentication no

Test and reload:

sshd -t
systemctl reload ssh

Verify:

sshd -T | grep -i passwordauthentication
# should print passwordauthentication no

2.4 Verify that password login is disabled ​

Test from a new local terminal:

ssh -o PreferredAuthentications=password wynnzen@163.7.7.182

It should return Permission denied (publickey).

3. Enabling Multi-Factor Authentication (MFA / TOTP) ​

3.1 Install the Google Authenticator PAM module ​

sudo apt install libpam-google-authenticator -y

3.2 Generate a 2FA key for each user ​

This must be configured separately for every user who needs to log in over SSH.

su - wynnzen
google-authenticator

Key choices:

• Time-based: y
• Update the file: y
• Disallow reuse of the same token: y
• Enable rate limiting: y

Scan the QR code with the Authenticator app on your phone and write down the emergency scratch codes. When finished, log out.

Confirm the file:

ls -l /home/wynnzen/.google_authenticator
# permissions should be 600, owner wynnzen

3.3 Configure PAM ​

Edit /etc/pam.d/sshd:

vim /etc/pam.d/sshd

• Add this at the top of the file:

auth required pam_google_authenticator.so

• Comment out @include common-auth, otherwise login will demand both a verification code and the system password:

# @include common-auth

Note: /etc/pam.d/sshd only affects SSH logins; it does not affect local console login.

3.4 Configure sshd for challenge-response authentication ​

Add the following to /etc/ssh/sshd_config.d/01-custom.conf:

PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

Reload:

sshd -t
systemctl reload ssh

3.5 Test ​

Log in from a new terminal:

ssh wynnzen@163.7.7.182

Expected flow:

1. SSH key verified
2. Prompted with Verification code:
3. Enter the 6-digit code from your phone
4. Login succeeds

You should no longer see a Password: prompt.

4. Other Hardening Measures ​

4.1 Disable direct root login ​

Once you have confirmed that wynnzen can log in normally and sudo works, add the following to /etc/ssh/sshd_config.d/01-custom.conf:

PermitRootLogin no

Reload:

sshd -t
systemctl reload ssh

Test:

ssh root@163.7.7.182
# should be denied

4.2 Restrict the source IP for logins ​

Method one: sshd_config

AllowUsers wynnzen@your_static_ip

Method two: firewall

ufw:

sudo ufw allow from your_static_ip to any port 22 proto tcp
sudo ufw deny 22/tcp
sudo ufw enable

firewalld:

sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="your_static_ip" port protocol="tcp" port="22" accept'
sudo firewall-cmd --permanent --remove-service=ssh
sudo firewall-cmd --reload

4.3 Change the default SSH port ​

Edit /etc/ssh/sshd_config or a drop-in file:

Port 2222

Update the firewall and security groups in step, and only close port 22 after the new port has been tested successfully.

4.4 Keep OpenSSH up to date ​

sudo apt update && sudo apt upgrade -y
ssh -V

Version 9.9p2 or later is recommended (it fixes CVE-2025-26465 and CVE-2025-26466).

5. Key Issues and Troubleshooting Notes ​

5.1 sshd.service not found ​

• Ubuntu / Debian: the service is called ssh; use systemctl restart ssh.
• CentOS / RHEL / Rocky: the service is called sshd; use systemctl restart sshd.

Check whether it is installed:

which sshd
sudo apt install openssh-server

5.2 The current terminal did not drop after a restart ​

This is normal. Restarting SSH only affects new connections; it does not terminate sessions that are already established. This is deliberate so that you keep a "lifeline" if you misconfigure something.

If you need to kick a suspicious session off:

who -a
last -a | head
ss -tnp | grep :22
pkill -KILL -t pts/1
loginctl terminate-user username

5.3 Passwords still work ​

Cause: your configuration was overridden. OpenSSH reads settings in this order:

1. /etc/ssh/sshd_config.d/*.conf (whichever is read first wins)
2. /etc/ssh/sshd_config

A common culprit is /etc/ssh/sshd_config.d/50-cloud-init.conf, which may contain PasswordAuthentication yes.

Diagnosis:

sudo sshd -T | grep -Ei 'passwordauthentication|kbdinteractive|authenticationmethods|permitrootlogin'
sudo grep -rniE 'PasswordAuthentication|KbdInteractiveAuthentication|AuthenticationMethods' /etc/ssh/sshd_config /etc/ssh/sshd_config.d/

Fix: create a higher-priority file at /etc/ssh/sshd_config.d/01-custom.conf and write your policy there.

5.4 Why do providers ship 50-cloud-init.conf? ​

cloud-init automates the first boot of a cloud server. It writes SSH settings through the standard drop-in mechanism to make sure password login still works when no key has been supplied. This is a priority conflict between automation convenience and user customization.

5.5 Both Verification code: and Password: appear at login ​

Cause: keyboard-interactive invokes PAM, and the PAM stack contains both pam_google_authenticator.so and pam_unix.so (via @include common-auth).

Fix: comment out @include common-auth in /etc/pam.d/sshd so that only this remains:

auth required pam_google_authenticator.so

6. Complete Workflow for Creating the Regular User wynnzen ​

# 1. Create the user
adduser wynnzen

# 2. Grant sudo
usermod -aG sudo wynnzen
groups wynnzen

# 3. Configure the SSH public key
mkdir -p /home/wynnzen/.ssh
cp /root/.ssh/authorized_keys /home/wynnzen/.ssh/authorized_keys
chown -R wynnzen:wynnzen /home/wynnzen/.ssh
chmod 700 /home/wynnzen/.ssh
chmod 600 /home/wynnzen/.ssh/authorized_keys

# 4. Configure MFA
su - wynnzen
google-authenticator
exit

# 5. Confirm file permissions
ls -l /home/wynnzen/.google_authenticator

# 6. Test login (new terminal)
ssh wynnzen@163.7.7.182

# 7. Test sudo
sudo whoami

# 8. Once everything checks out, disable direct root login
vim /etc/ssh/sshd_config.d/01-custom.conf
# add PermitRootLogin no
sshd -t
systemctl reload ssh

7. Final Verification Commands ​

sshd -T | grep -Ei 'passwordauthentication|kbdinteractive|authenticationmethods|permitrootlogin|allowusers'
grep -nE 'pam_unix|pam_google_authenticator|common-auth' /etc/pam.d/sshd
ls -l /home/wynnzen/.ssh/authorized_keys
ls -l /home/wynnzen/.google_authenticator
groups wynnzen

Expected results:

• passwordauthentication no
• kbdinteractiveauthentication yes
• authenticationmethods publickey,keyboard-interactive
• permitrootlogin no
• In PAM, common-auth is commented out and pam_google_authenticator.so is present
• authorized_keys permissions 600, owner wynnzen
• .google_authenticator permissions 600, owner wynnzen
• wynnzen is in the sudo group

8. Cautions ​

1. Always keep a logged-in root or sudo session open until the new configuration has been tested successfully in a new terminal.
2. Run sshd -t to test the syntax after every sshd_config change, then use systemctl reload ssh — never restart directly.
3. Do not close your current session: reload does not affect existing connections and is the safe operation.
4. Every user who needs to log in over SSH must run google-authenticator themselves, or their login will fail.
5. The emergency scratch codes must be written down and stored somewhere safe — they are the only way back in if you lose your phone.
6. In /etc/ssh/sshd_config.d/, the filename prefix determines priority; 01- overrides 50-.
7. cloud-init may rewrite the configuration during a system rebuild or update, so check sshd -T regularly.
8. AllowUsers is an allowlist: once enabled you must add wynnzen to it, or the account will be locked out.
9. On cloud servers you must also check inbound security group rules; port changes and IP restrictions have to be mirrored in the security group.
10. After changing the SSH port you must update the firewall and security groups in step, and test the new port before closing the old one.
11. PAM changes only affect SSH logins, not the local console — but a mistake here can lock you out of SSH, so be sure to keep a session open.
12. Enabling fail2ban is recommended so that failed SSH logins are blocked automatically, as an extra line of defense.

9. Configuration Lessons Learned ​

• Roll out in stages: key authentication first → then disable passwords → then MFA → then block root → then restrict IPs / change the port. Verify each step before moving to the next.
• Override with drop-ins instead of editing the main file: keep all custom policy in /etc/ssh/sshd_config.d/01-custom.conf so cloud-init or system updates cannot overwrite it.
• sshd -T is a powerful diagnostic tool: it prints the full configuration actually in effect, which is far more reliable than reading files.
• reload beats restart: reload does not drop existing connections, which makes it suitable for remote work.
• PAM is the source of the password prompt: PasswordAuthentication no does not mean keyboard-interactive won't ask for a password — you must inspect the PAM stack.
• MFA is per user: Google Authenticator is configured per user; setting it up only for root is not enough.
• Permissions are the key to SSH key authentication working: .ssh 700, authorized_keys 600, correct owner.
• Your current session is your lifeline: for every remote security change, test successfully in a new terminal before closing the old session.
• Security is an ongoing process: update OpenSSH regularly, review logs, audit logged-in users, and re-check your configuration.

End result:

The wynnzen user logs in with SSH key + a TOTP code from their phone, and the system password is no longer needed; root is blocked from direct SSH login; password authentication is disabled globally; and the cloud-init override problem has been resolved through drop-in priority.