How to Connect to SSH: A Practical Guide to Accessing Linux Servers

SSH (Secure Shell) is the standard protocol for managing remote Linux servers. If you just provisioned a VPS or received access credentials for a server, the first step is opening an SSH session — everything else (deployments, package installation, service configuration) depends on this working correctly.

This tutorial covers the full path: initial password-based connection, SSH key generation and usage, configuring ~/.ssh/config to simplify repetitive commands, and resolving the most common errors encountered on first access. The focus is practical usage on Linux and macOS environments, with targeted notes for Windows via native OpenSSH or WSL.

Estimated time: 15 to 25 minutes, including key generation and configuration adjustments.

Prerequisites

Before attempting a connection, confirm you have the server’s access credentials and an SSH client available on your local machine.

The typical data you receive when provisioning a VPS looks like this:

Keep these values handy — you will use them in the next steps. If the password arrived by email in plain text, change it on first login.

First Connection via Password

The ssh command follows the format ssh user@host -p port. For a basic connection using the credentials above:

ssh [email protected]

ssh [email protected] -p 2222

The authenticity of host '203.0.113.42 (203.0.113.42)' can't be established.
ED25519 key fingerprint is SHA256:abc123...
Are you sure you want to continue connecting (yes/no/[fingerprint])?

root@server:~#

SSH Key Authentication

SSH keys use asymmetric cryptography: you generate a key pair (private key + public key), store the private key on your local machine, and upload the public key to the server. Authentication happens without ever transmitting secrets over the network.

Generating the Key Pair

ssh-keygen -t ed25519 -C "[email protected]"

ssh-copy-id [email protected]

cat ~/.ssh/id_ed25519.pub | ssh [email protected] "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

ssh [email protected]

Configuring ~/.ssh/config

Typing ssh [email protected] -p 2222 -i ~/.ssh/specific_key every time gets tedious. The ~/.ssh/config file lets you create per-server aliases.

nano ~/.ssh/config

Host production
    HostName 203.0.113.42
    User root
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Host staging
    HostName 198.51.100.10
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_staging

chmod 600 ~/.ssh/config

ssh production

Hardening SSH Access

With key-based auth working, the next step is to disable password login and tighten the SSH service configuration on the server.

sudo nano /etc/ssh/sshd_config

PasswordAuthentication no
PermitRootLogin prohibit-password
PubkeyAuthentication yes

sudo sshd -t

sudo systemctl reload ssh

Verification

Confirm everything works as expected:

ssh -v production

Verbose mode (-v) shows every step: config file reading, algorithm negotiation, key offers. Look for:

debug1: Authentication succeeded (publickey).

This confirms authentication was done by key, not password. If you see password, something has not been applied correctly.

On the server side, check recent connection attempts:

sudo journalctl -u ssh --since "10 minutes ago"

You should see lines like Accepted publickey for root from ... port ... ssh2.

Troubleshooting

Permission denied (publickey)

The public key is not in ~/.ssh/authorized_keys on the server, or the permissions are wrong. On the server:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Also verify that the correct key is being offered by the client — run ssh -v and look for Offering public key.

Connection refused

The SSH service is not running, or the firewall is blocking the port. On the server:

sudo systemctl status ssh
sudo ss -tlnp | grep ssh

If the service is active, check the firewall (sudo ufw status or your provider’s firewall rules).

Connection timed out

Usually a network issue or wrong IP address. Test basic connectivity:

ping 203.0.113.42
nc -zv 203.0.113.42 22

If nc fails but ping works, the port is being blocked somewhere along the path.

Host key verification failed

The server’s fingerprint changed since your last connection (server reinstalled, IP recycled). Remove the old entry:

ssh-keygen -R 203.0.113.42

Then reconnect — the client will prompt you to confirm the new fingerprint.

Next Steps

With SSH working securely, the server is ready for real work. Some directions to explore further:

• Configure fail2ban to ban IPs with excessive authentication attempts.
• Use mosh on unstable connections (mobile, poor Wi-Fi) — it keeps the session alive even through network changes.
• Learn port forwarding (ssh -L and ssh -R) to access internal server services without exposing them to the internet.
• Configure ssh-agent to enter the passphrase once per work session.
• Study Match blocks in sshd_config for per-user or per-network access policies.

If you are deploying a project to production and want to avoid headaches with misconfigured SSH and firewalls, a Hostini VPS comes with SSH enabled, a dedicated IPv4 address, and immediate root access — just follow the steps in this tutorial starting from the provisioning email.