How to Access SSH: Practical Guide to Connect to Linux Servers
Learn how to access SSH on Linux servers step by step: key pairs, config aliases, troubleshooting and security hardening for developers in production.
SSH (Secure Shell) is the standard channel for administering Linux servers remotely. Whether you just provisioned a VPS, ordered a dedicated server, or need to access a staging machine, the first thing you will do is open an SSH session. The protocol encrypts all traffic and supports authentication via password or public key — the latter is the recommended standard for production environments.
This tutorial is aimed at developers who already work with a terminal but want to solidify their SSH workflow: first connection, key generation, host aliases in ~/.ssh/config, basic hardening, and resolution of common errors (Permission denied, Connection refused, Host key verification failed).
Estimated time: 15–20 minutes, including key generation and server-side adjustments.
Prerequisites
Before starting, confirm you have the access credentials provided by your hosting provider and an SSH client installed locally. On Linux and macOS, OpenSSH is available by default. On Windows 10/11, the native OpenSSH client is available via PowerShell — alternatives include WSL2 or PuTTY.
A Linux server reachable over the network (Ubuntu, Debian, Rocky, etc.), initial credentials (IP + user + password or key), and an OpenSSH client on your desktop. Port 22 open in the provider’s firewall.
A typical VPS provisioning sends you something in this format:
203.0.113.42 root 22 sent by email First Connection via Password
The most direct way to connect is with a username and password. It works, but it is not what you want to keep in production — it is a starting point for uploading your public key.
Open a local terminal and run the ssh command with your username and IP:
ssh [email protected]On the first connection, the client will display the server’s public key fingerprint and ask if you trust it. Type yes to continue — this adds the host to ~/.ssh/known_hosts.
Enter the password when prompted. Important: the terminal does not display characters as you type (not even asterisks). Paste or type the password and press Enter.
If authentication succeeds, you land in the server shell. Verify with:
whoami
hostname
uname -aIf the SSH port was customized by the provider (common in hardened setups), use the -p flag:
ssh -p 2222 [email protected]Leaving password as the only authentication method exposes you to brute-force attacks. Right after first access, configure public key authentication and disable PasswordAuthentication on the server.
Generate and Install an SSH Key
SSH keys use asymmetric cryptography: a private key stays on your desktop and never leaves; the public key goes to the server. The recommended algorithm today is Ed25519 — shorter, faster, and as secure as RSA 4096.
Generate the key pair locally:
ssh-keygen -t ed25519 -C "[email protected]"ssh-keygen asks where to save the key (default: ~/.ssh/id_ed25519) and prompts for a passphrase. The passphrase protects the private key if the file is ever leaked — use a long phrase. For automation you can leave it blank, but assess the risk.
Copy the public key to the server using ssh-copy-id:
ssh-copy-id [email protected]The command authenticates via password once and appends your public key to ~/.ssh/authorized_keys on the remote user, with correct permissions (700 on the directory, 600 on the file).
Test the connection. It should now open without prompting for a password:
ssh [email protected]If you configured a passphrase, the client will ask for it the first time — in subsequent sessions ssh-agent keeps it cached.
If ssh-copy-id is not available (common on Windows), do it manually:
cat ~/.ssh/id_ed25519.pub | ssh [email protected] "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"Configure Aliases in ~/.ssh/config
Typing ssh -p 2222 -i ~/.ssh/work_key [email protected] every time is tedious and error-prone. The ~/.ssh/config file lets you create per-host aliases.
Create or edit ~/.ssh/config:
nano ~/.ssh/configAdd one block per server:
Host prod-api
HostName 203.0.113.42
User deploy
Port 2222
IdentityFile ~/.ssh/id_ed25519
ServerAliveInterval 60
Host staging
HostName 198.51.100.10
User ubuntu
Port 22
IdentityFile ~/.ssh/staging_keyFix file permissions (OpenSSH refuses configs with loose permissions):
chmod 600 ~/.ssh/configConnect using the alias:
ssh prod-apiThis is equivalent to ssh -p 2222 -i ~/.ssh/id_ed25519 [email protected]. It also works with scp and rsync: scp file.tar.gz prod-api:/tmp/.
The ServerAliveInterval 60 directive makes the client send a keepalive packet every 60 seconds, preventing NATs and intermediate firewalls from dropping idle sessions. Especially useful for long-running connections inside tmux or screen.
Basic Server Hardening
Once key authentication works, disable password access and direct root login.
Create a non-root user with sudo (if you haven’t already):
adduser deploy
usermod -aG sudo deploy
rsync --archive --chown=deploy:deploy ~/.ssh /home/deployEdit /etc/ssh/sshd_config:
sudo nano /etc/ssh/sshd_configAdjust the following directives:
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication noValidate the syntax before restarting:
sudo sshd -t
sudo systemctl restart sshKeep the current session open and, in a second terminal, test login with the new user. If something is misconfigured and you close the session before validating, you will lose access to the server.
Verification
To confirm everything is working as expected:
ssh -v prod-apiThe -v flag enables verbose mode. Look for these lines:
debug1: Offering public key: ~/.ssh/id_ed25519
debug1: Server accepts key
debug1: Authentication succeeded (publickey)This confirms that key authentication was accepted. If you see Authentications that can continue: publickey and no mention of password, the server is correctly configured to reject password logins.
Troubleshooting
Permission denied (publickey)
The server rejected your key. Common causes: public key not present in ~/.ssh/authorized_keys for the remote user, wrong permissions (.ssh must be 700, authorized_keys 600), or using the wrong private key. Debug with ssh -v and check which key the client offered.
Connection refused
The server did not respond on that port. Check whether the SSH service is running (sudo systemctl status ssh), whether the port is correct, and whether the firewall (UFW, iptables, or the provider’s firewall) allows inbound traffic on the SSH port.
Connection timed out
No response within a reasonable time — usually a firewall block or wrong IP. Test with ping and nc -zv IP 22 to isolate the issue.
Host key verification failed
The server’s public key changed since your last connection (reinstallation, new server on the same IP). Remove the old entry:
ssh-keygen -R 203.0.113.42Then reconnect and accept the new fingerprint.
Next Steps
With SSH access working and hardened, good next moves include:
- Setting up
fail2banto block IPs with repeated failed login attempts - Using
ssh-agentor the1Password CLIto manage passphrases automatically - Configuring
ProxyJumpin~/.ssh/configto access servers through a bastion host - Automating deployments with
rsyncoransible, reusing your config aliases - Enabling multi-factor authentication (TOTP) with
libpam-google-authenticator
If you are provisioning production environments, a Hostini VPS comes with OpenSSH active, a dedicated IP, and an out-of-band console — useful for recovering access if you accidentally lock yourself out of sshd_config.
Frequently asked questions
What is the difference between password and SSH key authentication?
A password is a short secret the server verifies on every login — vulnerable to brute force and credential leaks. SSH keys use asymmetric cryptography: the private key never leaves your desktop and the public key lives on the server. The server sends a cryptographic challenge that only the private key can solve, without ever transmitting the secret over the wire. Key authentication is the recommended method for production.
Can I use the same SSH key on multiple servers?
Yes. The public key can be installed on as many servers as you need — there is no technical limit. Many developers use a single personal key for all hosts. To separate contexts (work vs. personal, prod vs. staging), generate distinct key pairs and reference each one via `IdentityFile` in `~/.ssh/config`.
How do I use SSH on Windows without installing PuTTY?
Windows 10 (build 1809+) and Windows 11 ship with a native OpenSSH client. Open PowerShell and run `ssh user@ip` exactly as you would on Linux or macOS. Key files live in `C:\Users\YOUR_USER\.ssh\`. If you prefer a full Linux environment with `ssh-keygen`, `ssh-agent`, and related tools, install WSL2.
What should I do if I lose my SSH private key?
A private key is irreplaceable — there is no way to reconstruct it from the public key. If you lose access, use your provider's out-of-band console (VNC, serial console, or rescue mode) to log into the server, edit `~/.ssh/authorized_keys`, and add a new public key. This is why it is good practice to pre-authorize a backup key before any crisis occurs.
Why do I get 'Permission denied (publickey)' even with the correct key?
The most frequent causes: wrong permissions on the server (the `~/.ssh` directory must be 700 and `authorized_keys` must be 600, with no group write access), SELinux or AppArmor blocking access, wrong username in the command, or the public key not actually present in the correct user's file. Run `ssh -vvv` on the client and `tail -f /var/log/auth.log` on the server to identify the exact cause.
Is it safe to leave port 22 open to the internet?
Yes, as long as you disable password authentication, disallow direct root login, and configure rate limiting via fail2ban or equivalent. The SSH encryption itself is robust — what gets attacked is the authentication method. Moving SSH to a non-standard high port reduces log noise but is not real security; it is security through obscurity.