SSH: The Case Files - Secure Shell Investigation Manual

CASE #01

Quick Reference

Essential commands every investigator needs on hand. Memorize these like your badge number.

Command	Purpose
`ssh user@host`	Connect to remote host
`ssh -p 2222 user@host`	Connect via non-standard port
`ssh-keygen -t ed25519`	Generate Ed25519 key (recommended)
`ssh-copy-id user@host`	Copy public key to remote
`ssh -L 8080:localhost:80 user@host`	Local port forwarding
`ssh -R 8080:localhost:80 user@host`	Remote port forwarding
`ssh -D 1080 user@host`	Dynamic SOCKS proxy
`ssh -J jump@bastion target@host`	Jump through bastion host
`scp file.txt user@host:/path/`	Copy file to remote
`rsync -avzP dir/ user@host:/path/`	Sync directory efficiently

Detective's Note: These commands are your first line of defense. Keep them close, like a loaded revolver in a dark alley.

CASE #02

Connection Basics

Standard Connection Syntax

Making first contact with the target system.

# Basic connection
ssh user@hostname
ssh user@192.168.1.100

# Specify port
ssh -p 2222 user@host
ssh user@host -p 2222

# Using domain
ssh admin@example.com

Verbose Mode (Debugging)

When the trail goes cold, turn up the lights and see what's happening behind the scenes.

# Increasing levels of verbosity
ssh -v user@host          # Verbose (level 1)
ssh -vv user@host         # More verbose (level 2)
ssh -vvv user@host        # Maximum verbosity (level 3)

Field Report: Use verbose mode to troubleshoot connection issues, authentication problems, or key exchange errors. It's like turning on all the lights in a crime scene.

Jump Hosts & ProxyJump

Sometimes you need to go through the middleman to reach your target. ProxyJump allows connecting through intermediary bastion hosts with end-to-end encryption.

# Single jump host
ssh -J jump@bastion target@destination

# Multiple jump hosts (comma-separated)
ssh -J user1@bastion1,user2@bastion2 user3@target

# With custom ports
ssh -J user@bastion:2222 target@destination

# Alternative ProxyCommand syntax (older method)
ssh -o ProxyCommand="ssh -W %h:%p user@bastion" target@destination

Connection Options

Additional flags for special circumstances in the field.

Option	Purpose
`-i ~/.ssh/custom_key`	Use specific identity file
`-X`	Enable X11 forwarding
`-Y`	Trusted X11 forwarding
`-C`	Enable compression
`-N`	No remote command (tunnels only)
`-f`	Background before execution
`-A`	Enable agent forwarding
`-o ServerAliveInterval=60`	Keep connection alive

# Execute remote command
ssh user@host 'ls -la /var/log'
ssh user@host 'uptime && free -h'

# Keep connection alive through timeouts
ssh -o ServerAliveInterval=60 -o ServerAliveCountMax=3 user@host

Warning: The -o StrictHostKeyChecking=no option disables host key verification. Only use in controlled testing environments - never in production. It's like walking into a dark warehouse without backup.

CASE #03

Key Management

In this business, keys are everything. They open doors, grant access, and leave traces. Handle them with care.

ssh-keygen: Key Generation

Ed25519 (Recommended for 2026)

The sleek, modern key type. Compact, fast, and secure with 256-bit keys.

# Generate Ed25519 key
ssh-keygen -t ed25519 -C "your_email@example.com"

# Specify output file
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_work -C "work key"

# With passphrase (recommended)
ssh-keygen -t ed25519 -C "comment"
# Will prompt for passphrase

# Without passphrase (automation/CI only)
ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519_deploy

RSA Keys

The old reliable. RSA keys must be at least 3072 bits, preferably 4096 bits for 2026 security standards.

# Generate RSA 4096-bit key
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

# Specify custom file
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_legacy

# For legacy systems requiring 2048-bit (not recommended)
ssh-keygen -t rsa -b 2048

Detective's Note: ECDSA has security concerns. Ed25519 is the recommended algorithm for new keys. Think of it as upgrading from a revolver to a modern semi-automatic.

FIDO2/U2F Hardware Keys

OpenSSH 8.2+ supports FIDO2 hardware authenticators (YubiKey, etc.) with enhanced security.

# Generate Ed25519-SK key (requires hardware key)
ssh-keygen -t ed25519-sk -C "yubikey-ed25519"

# Generate ECDSA-SK key
ssh-keygen -t ecdsa-sk -C "yubikey-ecdsa"

# Require PIN verification
ssh-keygen -t ed25519-sk -O verify-required -C "yubikey-with-pin"

# Resident key (stored on device)
ssh-keygen -t ed25519-sk -O resident -C "resident-key"

FIDO2 Benefits

Private keys never leave the hardware device
Physical touch required for each authentication
Optional PIN verification for additional security
Protection against phishing and remote attacks

Key Operations

# Change passphrase of existing key
ssh-keygen -p -f ~/.ssh/id_ed25519

# Show fingerprint
ssh-keygen -lf ~/.ssh/id_ed25519.pub
ssh-keygen -lf ~/.ssh/id_ed25519    # Works with private key too

# Show fingerprint in different formats
ssh-keygen -lf ~/.ssh/id_ed25519.pub -E md5
ssh-keygen -lf ~/.ssh/id_ed25519.pub -E sha256

# Show ASCII art representation
ssh-keygen -lvf ~/.ssh/id_ed25519.pub

# Convert key format (OpenSSH to PEM)
ssh-keygen -p -f ~/.ssh/id_rsa -m pem

# Test key against server
ssh -T git@github.com       # GitHub
ssh -T git@gitlab.com       # GitLab

ssh-agent: Key Agent Management

The agent holds your keys in memory, so you don't have to enter passphrases repeatedly. Like a trusted partner who remembers the safe combination.

# Start ssh-agent
eval "$(ssh-agent -s)"

# Add key to agent
ssh-add ~/.ssh/id_ed25519
ssh-add ~/.ssh/id_rsa

# Add key with timeout (1 hour)
ssh-add -t 3600 ~/.ssh/id_ed25519

# List loaded keys
ssh-add -l              # Short format
ssh-add -L              # Full public keys

# Remove key from agent
ssh-add -d ~/.ssh/id_ed25519

# Remove all keys
ssh-add -D

# Lock agent with passphrase
ssh-add -x

# Unlock agent
ssh-add -X

Distributing Public Keys

# Copy public key to remote server
ssh-copy-id user@host
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@host

# Copy to non-standard port
ssh-copy-id -p 2222 user@host

# Manual method (if ssh-copy-id unavailable)
cat ~/.ssh/id_ed25519.pub | ssh user@host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

# Set correct permissions on remote
ssh user@host "chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys"

Key File Permissions

Critical security requirement. Wrong permissions are like leaving case files on your desk overnight.

chmod 700 ~/.ssh                    # Directory
chmod 600 ~/.ssh/id_ed25519         # Private key
chmod 600 ~/.ssh/config             # Config file
chmod 644 ~/.ssh/id_ed25519.pub     # Public key
chmod 644 ~/.ssh/known_hosts        # Known hosts
chmod 600 ~/.ssh/authorized_keys    # Authorized keys (on server)

Critical: SSH will refuse to use keys with incorrect permissions. Private keys must be readable only by owner (600). This isn't paranoia - it's protocol.

CASE #04

SSH Config File

The SSH config file (~/.ssh/config) is your personal case file system. It simplifies connection management with persistent settings and aliases.

Basic Structure

# ~/.ssh/config

# Default settings for all hosts
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    Compression yes

# Specific host configuration
Host myserver
    HostName 192.168.1.100
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_work

# Using wildcards
Host dev-*
    User developer
    ForwardAgent no

Host prod-*
    User deployer
    StrictHostKeyChecking yes

Case Note: Host aliases let you type ssh myserver instead of ssh -p 2222 admin@192.168.1.100. Less typing, more investigating.

Essential Directives

Directive	Purpose
`Host`	Alias for connection
`HostName`	Real hostname or IP
`User`	Default username
`Port`	Custom port
`IdentityFile`	Private key location
`IdentitiesOnly`	Only use specified keys
`ProxyJump`	Jump through bastion
`ForwardAgent`	Forward ssh-agent
`LocalForward`	Local port forward
`RemoteForward`	Remote port forward

Match Blocks

Conditional configuration based on runtime conditions. Apply different rules based on the situation.

# Match by hostname pattern
Match host "*.prod.example.com"
    ForwardAgent no
    StrictHostKeyChecking yes

# Match by user
Match user developer
    IdentityFile ~/.ssh/id_dev

# Match by local user
Match localuser admin
    ForwardAgent yes

# Match by network (CIDR)
Match host "10.0.0.*"
    ProxyJump bastion

# Multiple conditions (AND logic)
Match host "*.internal" user developer
    ProxyJump bastion.internal

Include Directive

Organize complex configurations across multiple files.

# ~/.ssh/config
Include ~/.ssh/config.d/*.conf
Include ~/.ssh/work_config
Include ~/.ssh/personal_config

Host *
    ServerAliveInterval 60

Then create separate files:

# ~/.ssh/config.d/work.conf
Host work-*
    User employee
    IdentityFile ~/.ssh/id_work

# ~/.ssh/config.d/personal.conf
Host personal-*
    User myusername
    IdentityFile ~/.ssh/id_personal

Practical Examples

# GitHub with multiple accounts
Host github-work
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_work
    IdentitiesOnly yes

Host github-personal
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_personal
    IdentitiesOnly yes

# Jump host setup
Host bastion
    HostName bastion.example.com
    User jumpuser
    Port 22

Host internal-*
    ProxyJump bastion
    User admin

# AWS EC2 instances
Host ec2-*
    User ec2-user
    IdentityFile ~/.ssh/aws-keypair.pem
    StrictHostKeyChecking no
    UserKnownHostsFile /dev/null

# Development environment with multiplexing
Host devbox
    HostName dev.example.com
    User developer
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h:%p
    ControlPersist 10m
    ForwardAgent yes
    LocalForward 3000 localhost:3000
    LocalForward 5432 localhost:5432

CASE #05

Port Forwarding & Tunnels

SSH tunneling creates encrypted connections for secure access to remote services. Think of it as creating secret passages through the network.

Local Port Forwarding (-L)

Forward local port to remote destination. Traffic originates from your machine.

# Syntax: -L [local_ip:]local_port:destination_host:destination_port

# Forward local 8080 to remote's localhost:80
ssh -L 8080:localhost:80 user@remote
# Access via: http://localhost:8080

# Forward to different host through remote
ssh -L 5432:database.internal:5432 user@bastion
# Connect to database through bastion

# Bind to specific interface
ssh -L 127.0.0.1:8080:localhost:80 user@remote
ssh -L 0.0.0.0:8080:localhost:80 user@remote  # All interfaces

# Multiple forwards in one connection
ssh -L 8080:localhost:80 -L 5432:db:5432 user@remote

# Background tunnel (no shell)
ssh -f -N -L 8080:localhost:80 user@remote
# -f: background, -N: no command execution

Use Cases

Access web application on remote server
Connect to remote database securely
Bypass firewall restrictions
Access internal services through bastion

Remote Port Forwarding (-R)

Forward remote port to local destination. Makes local service accessible from remote.

# Syntax: -R [remote_ip:]remote_port:destination_host:destination_port

# Make local service accessible on remote
ssh -R 8080:localhost:80 user@remote
# Remote users access via: remote:8080

# Reverse tunnel to local database
ssh -R 5432:localhost:5432 user@remote

# Bind to specific remote interface
ssh -R 0.0.0.0:8080:localhost:80 user@remote

# Forward to different local host
ssh -R 9090:192.168.1.100:80 user@remote

# Persistent reverse tunnel
ssh -f -N -R 8080:localhost:80 user@remote

Use Cases

Expose local development server to remote testers
Remote access to service behind NAT/firewall
Webhook testing (expose localhost to internet)
Remote support/administration

Dynamic Port Forwarding (-D)

Creates a SOCKS proxy for flexible port forwarding. Route all your traffic through the encrypted tunnel.

# Create SOCKS5 proxy on port 1080
ssh -D 1080 user@remote

# Specify bind address
ssh -D 127.0.0.1:1080 user@remote
ssh -D 0.0.0.0:1080 user@remote    # All interfaces (careful!)

# Background SOCKS proxy
ssh -f -N -D 1080 user@remote

# Configure applications to use proxy:
# Browser: Set SOCKS5 proxy to localhost:1080
# curl: curl --socks5 localhost:1080 https://example.com
# git: git config --global http.proxy socks5://127.0.0.1:1080

Use Cases

Browse internet through remote server
Access geo-restricted content
Bypass network filters/censorship
Route all traffic through encrypted tunnel
Dynamic routing (destination determined at request time)

SSH Config Tunnels

Make tunnels persistent in your config file.

# Local forward
Host webapp-tunnel
    HostName remote.example.com
    User admin
    LocalForward 8080 localhost:80
    LocalForward 5432 db.internal:5432

# Remote forward
Host expose-local
    HostName public.example.com
    User deployer
    RemoteForward 8080 localhost:3000

# Dynamic SOCKS proxy
Host socks-proxy
    HostName proxy.example.com
    User proxyuser
    DynamicForward 1080

Advanced Tunnel Examples

# Jump host with local forward
ssh -J bastion -L 5432:db:5432 app-server

# Persistent autossh tunnel (auto-reconnect)
autossh -M 0 -f -N \
    -o "ServerAliveInterval 30" \
    -o "ServerAliveCountMax 3" \
    -L 8080:localhost:80 user@remote

# Reverse tunnel for SSH access behind NAT
# On NAT'd machine:
ssh -R 2222:localhost:22 user@public-server
# On public server:
ssh -p 2222 localhost  # Access NAT'd machine

Server Configuration: For remote forwarding to be accessible from other hosts, server must enable GatewayPorts yes in /etc/ssh/sshd_config.

CASE #06

File Transfer

Moving evidence between locations. Three tools for different situations.

SCP (Secure Copy)

Simple file copying over SSH. Note: SCP is being deprecated in favor of SFTP/rsync, but still widely used.

# Copy file to remote
scp file.txt user@host:/path/to/destination/
scp file.txt user@host:~/

# Copy from remote to local
scp user@host:/path/to/file.txt .
scp user@host:~/file.txt /local/path/

# Copy directory recursively
scp -r directory/ user@host:/path/

# Custom port
scp -P 2222 file.txt user@host:/path/

# Preserve timestamps and permissions
scp -p file.txt user@host:/path/

# Verbose output
scp -v file.txt user@host:/path/

# Copy between two remote hosts (through local)
scp user1@host1:/path/file user2@host2:/path/

# Custom SSH key
scp -i ~/.ssh/custom_key file.txt user@host:/path/

# Limit bandwidth (in Kbps)
scp -l 1024 file.txt user@host:/path/

# Multiple files
scp file1.txt file2.txt user@host:/path/
scp *.txt user@host:/path/

# Compression
scp -C large_file.dat user@host:/path/

SFTP (SSH File Transfer Protocol)

Interactive file transfer protocol over SSH. More feature-rich than SCP.

# Connect to SFTP server
sftp user@host
sftp -P 2222 user@host        # Custom port

# Custom identity file
sftp -i ~/.ssh/custom_key user@host

# Batch mode (non-interactive)
sftp -b commands.txt user@host

Interactive SFTP Commands

Command	Purpose
`pwd`	Print remote working directory
`lpwd`	Print local working directory
`ls`	List remote directory
`lls`	List local directory
`cd /path`	Change remote directory
`lcd /path`	Change local directory
`get file.txt`	Download file
`get -r dir/`	Download directory
`put file.txt`	Upload file
`put -r dir/`	Upload directory
`mget *.txt`	Download multiple (glob)
`mput *.log`	Upload multiple (glob)
`mkdir dir`	Create remote directory
`rm file.txt`	Delete remote file
`chmod 644 file.txt`	Change permissions

SFTP in Scripts

# Automated SFTP with here-document
sftp user@host << EOF
cd /remote/path
get file.txt
put upload.txt
bye
EOF

rsync over SSH

Efficient file synchronization with incremental transfers. The professional's choice for serious file operations.

# Basic syntax
rsync [options] source destination

# Sync directory to remote
rsync -avz /local/dir/ user@host:/remote/dir/
# -a: archive (recursive, preserve attributes)
# -v: verbose
# -z: compression

# Common options
rsync -avzP /local/dir/ user@host:/remote/dir/
# -P: show progress + keep partial transfers

# Sync from remote to local
rsync -avz user@host:/remote/dir/ /local/dir/

# Custom SSH port
rsync -avz -e "ssh -p 2222" /local/ user@host:/remote/

# Custom SSH key
rsync -avz -e "ssh -i ~/.ssh/custom_key" /local/ user@host:/remote/

# Delete files on destination not in source
rsync -avz --delete /local/ user@host:/remote/

# Exclude patterns
rsync -avz --exclude '*.log' --exclude 'temp/' /local/ user@host:/remote/

# Dry run (preview changes)
rsync -avzn /local/ user@host:/remote/
# or --dry-run

# Limit bandwidth (KB/s)
rsync -avz --bwlimit=1000 /local/ user@host:/remote/

# Show progress for each file
rsync -avz --progress /local/ user@host:/remote/

# Preserve hard links
rsync -avzH /local/ user@host:/remote/

rsync Advanced Examples

# Backup with date
rsync -avz /data/ user@backup:/backups/data-$(date +%Y%m%d)/

# Mirror website
rsync -avz --delete /var/www/ user@webserver:/var/www/

# Sync only specific file types
rsync -avz --include='*.jpg' --include='*/' --exclude='*' /photos/ user@host:/photos/

# Through jump host
rsync -avz -e "ssh -J jumphost" /local/ user@target:/remote/

# Show differences before sync
rsync -avzn --itemize-changes /local/ user@host:/remote/

Performance Comparison

Tool	Speed	Incremental	Resumable	Best For
`scp`	Medium	No	No	Single file transfers, simplicity
`sftp`	Medium	No	No	Interactive transfers, browsing
`rsync`	Fast	Yes	Yes	Directory sync, backups, large data

CASE #07

Authentication

Proving your identity. In the digital world, you are your keys.

Password Authentication

Basic but least secure method. Like using a combination lock instead of a biometric scanner.

# Connect with password prompt
ssh user@host

# Disable password auth (server-side)
# In /etc/ssh/sshd_config:
PasswordAuthentication no
ChallengeResponseAuthentication no

Security Advisory: Password authentication should be disabled on production servers. Use key-based authentication instead.

Key-Based Authentication

Recommended method using cryptographic key pairs. The modern detective's badge.

Setup Process

# 1. Generate key pair
ssh-keygen -t ed25519 -C "your_email@example.com"

# 2. Copy public key to server
ssh-copy-id user@host

# 3. Test connection
ssh user@host

# 4. Disable password authentication on server
# /etc/ssh/sshd_config:
PasswordAuthentication no
PubkeyAuthentication yes

Multiple Keys

# Specify key
ssh -i ~/.ssh/id_ed25519_work user@host

# In config
Host workserver
    IdentityFile ~/.ssh/id_ed25519_work
    IdentitiesOnly yes

Certificate-Based Authentication

SSH certificates provide centralized key management and fine-grained access control.

Benefits

No need to distribute public keys to every server
Centralized revocation
Time-limited access
Host verification
Role-based access

Setup

# 1. Generate CA key
ssh-keygen -t ed25519 -f ca_user_key -C "User CA"

# 2. Sign user's public key
ssh-keygen -s ca_user_key \
    -I user_identifier \
    -n username \
    -V +52w \
    ~/.ssh/id_ed25519.pub
# Creates id_ed25519-cert.pub

# 3. Configure server to trust CA
# /etc/ssh/sshd_config:
TrustedUserCAKeys /etc/ssh/ca_user_key.pub

# 4. Connect (certificate auto-loaded with key)
ssh user@host

Certificate Options

# Time-limited certificate (1 week)
ssh-keygen -s ca_key -I "user_id" -n username -V +1w id_ed25519.pub

# Specific validity period
ssh-keygen -s ca_key -I "user_id" -n username \
    -V 20260201:20260301 id_ed25519.pub

# Multiple principals (usernames)
ssh-keygen -s ca_key -I "user_id" -n "user1,user2,admin" id_ed25519.pub

# Force command
ssh-keygen -s ca_key -I "user_id" -n username \
    -O force-command="/usr/bin/restricted" id_ed25519.pub

# Restrict source address
ssh-keygen -s ca_key -I "user_id" -n username \
    -O source-address="192.168.1.0/24" id_ed25519.pub

# Disable features
ssh-keygen -s ca_key -I "user_id" -n username \
    -O no-port-forwarding \
    -O no-agent-forwarding \
    -O no-x11-forwarding \
    id_ed25519.pub

# View certificate details
ssh-keygen -Lf id_ed25519-cert.pub

Two-Factor Authentication (2FA/MFA)

Add second authentication factor beyond keys. Defense in depth.

PAM-based 2FA

# Install Google Authenticator PAM module
sudo apt install libpam-google-authenticator

# Setup for user
google-authenticator

# Configure SSH PAM
# /etc/pam.d/sshd:
auth required pam_google_authenticator.so

# /etc/ssh/sshd_config:
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

CASE #08

Security Hardening

Lock down your SSH server like Fort Knox. Every open door is an opportunity for the wrong person.

sshd_config Best Practices

Server-side configuration file: /etc/ssh/sshd_config

# === AUTHENTICATION ===
PermitRootLogin no                          # Disable root login
PasswordAuthentication no                   # Disable password auth
ChallengeResponseAuthentication no          # Disable challenge-response
PubkeyAuthentication yes                    # Enable key-based auth
PermitEmptyPasswords no                     # Reject empty passwords
UsePAM yes                                  # Use PAM for auth

# === ACCESS CONTROL ===
AllowUsers alice bob charlie                # Whitelist users
AllowGroups sshusers admins                 # Whitelist groups
DenyUsers guest                             # Blacklist users
DenyGroups restricted                       # Blacklist groups

# === NETWORK ===
Port 2222                                   # Non-standard port
ListenAddress 192.168.1.10                  # Specific interface only
AddressFamily inet                          # IPv4 only

# === PROTOCOL ===
Protocol 2                                  # SSH protocol 2 only
MaxAuthTries 3                              # Limit auth attempts
MaxSessions 10                              # Max sessions per connection
ClientAliveInterval 300                     # 5 min keepalive
ClientAliveCountMax 2                       # Disconnect after 2 missed
LoginGraceTime 60                           # 1 min to authenticate
MaxStartups 10:30:60                        # Connection rate limiting

# === SECURITY OPTIONS ===
StrictModes yes                             # Check file permissions
X11Forwarding no                            # Disable X11 (unless needed)
PermitUserEnvironment no                    # Prevent env manipulation
AllowAgentForwarding no                     # Disable agent forwarding
AllowTcpForwarding no                       # Disable port forwarding (if not needed)
GatewayPorts no                             # Don't allow remote port binding
PermitTunnel no                             # Disable tun device forwarding

# === LOGGING ===
SyslogFacility AUTH                         # Log to auth facility
LogLevel VERBOSE                            # Detailed logging

# === CRYPTOGRAPHY ===
# Modern ciphers only (OpenSSH 7.6+)
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com

# Modern MACs
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Modern key exchange
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512

# Host keys (prefer Ed25519 and RSA)
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key

# === BANNER ===
Banner /etc/ssh/banner.txt                  # Display login banner

Apply Changes

# Test configuration
sudo sshd -t

# Restart SSH service
sudo systemctl restart sshd
# or
sudo service ssh restart

fail2ban Configuration

Automated intrusion prevention - bans IPs with repeated failed attempts. Your automated security guard.

Installation

# Ubuntu/Debian
sudo apt install fail2ban

# CentOS/RHEL
sudo yum install fail2ban

Configuration

# /etc/fail2ban/jail.local
[DEFAULT]
bantime = 3600              # Ban for 1 hour
findtime = 600              # 10 minute window
maxretry = 3                # 3 attempts before ban
destemail = admin@example.com
sendername = Fail2Ban
action = %(action_mwl)s     # Ban and email with logs

[sshd]
enabled = true
port = ssh,2222             # Monitor ports
logpath = /var/log/auth.log # Debian/Ubuntu
# logpath = /var/log/secure # CentOS/RHEL
maxretry = 3
findtime = 600
bantime = 3600

# Whitelist IPs
ignoreip = 127.0.0.1/8 192.168.1.0/24

Management Commands

# Start fail2ban
sudo systemctl start fail2ban
sudo systemctl enable fail2ban

# Status
sudo fail2ban-client status
sudo fail2ban-client status sshd

# Unban IP
sudo fail2ban-client set sshd unbanip 192.168.1.100

# Ban IP manually
sudo fail2ban-client set sshd banip 203.0.113.100

# Reload configuration
sudo fail2ban-client reload

# Check logs
sudo tail -f /var/log/fail2ban.log

Additional Hardening Measures

IP Whitelisting

# iptables firewall
sudo iptables -A INPUT -p tcp -s 192.168.1.0/24 --dport 22 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 22 -j DROP

# Save rules
sudo iptables-save > /etc/iptables/rules.v4

Port Knocking

# Install knockd
sudo apt install knockd

# /etc/knockd.conf
[openSSH]
    sequence    = 7000,8000,9000
    seq_timeout = 10
    command     = /sbin/iptables -I INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
    tcpflags    = syn

[closeSSH]
    sequence    = 9000,8000,7000
    seq_timeout = 10
    command     = /sbin/iptables -D INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
    tcpflags    = syn

Critical Warning: Always maintain an alternative access method (console, IPMI, physical access) before implementing aggressive security measures. Don't lock yourself out.

CASE #09

Multiplexing & Performance

SSH multiplexing allows multiple sessions over a single TCP connection, significantly improving performance. One connection, many possibilities.

ControlMaster Configuration

# ~/.ssh/config

# Enable multiplexing globally
Host *
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h:%p
    ControlPersist 10m

# Or per-host
Host fastserver
    HostName server.example.com
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h:%p
    ControlPersist 600

ControlMaster Options

Option	Behavior
`no`	Disable multiplexing (default)
`yes`	Master mode, listen for connections
`auto`	Try to use master, create if none exists (recommended)
`autoask`	Like auto, but ask before creating master
`ask`	Ask before creating master connection

ControlPath Variables

Variable	Meaning
`%h`	Remote hostname
`%p`	Remote port
`%r`	Remote username
`%L`	Local hostname
`%n`	Original hostname (command line)

Setup Multiplexing

# 1. Create socket directory
mkdir -p ~/.ssh/sockets
chmod 700 ~/.ssh/sockets

# 2. Add to config (shown above)

# 3. Connect - first connection creates master
ssh user@host

# 4. Subsequent connections reuse master (instant!)
ssh user@host               # Nearly instantaneous
scp file.txt user@host:/path/  # Reuses connection

Managing Master Connections

# Check if master exists
ssh -O check user@host

# Stop master connection
ssh -O stop user@host

# Forward port on existing connection
ssh -O forward -L 8080:localhost:80 user@host

# Cancel port forward
ssh -O cancel -L 8080:localhost:80 user@host

# Ask master to exit when no more clients
ssh -O exit user@host

Performance Benefits

Without Multiplexing

Each connection: Full TCP + SSH handshake
Time: ~500-2000ms per connection
Resources: N connections = N TCP sockets

With Multiplexing

First connection: Full handshake
Subsequent: Reuse existing connection
Time: ~10-50ms per additional session
Resources: N sessions = 1 TCP socket

Advanced Multiplexing Patterns

# Long-lived connections (development server)
Host devserver
    HostName dev.example.com
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h:%p
    ControlPersist 8h

# Short-lived connections (deployment)
Host deploy-*
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h:%p
    ControlPersist 5m

# No multiplexing (sensitive operations)
Host prod-*
    ControlMaster no

Complete Performance Config

Host *
    # Multiplexing
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h:%p
    ControlPersist 10m

    # Compression
    Compression yes

    # Keepalive
    ServerAliveInterval 60
    ServerAliveCountMax 3
    TCPKeepAlive yes

    # Speed up initial connection
    GSSAPIAuthentication no

    # Modern ciphers (faster)
    Ciphers chacha20-poly1305@openssh.com,aes128-gcm@openssh.com

Troubleshooting Multiplexing

# Stale socket
rm ~/.ssh/sockets/*

# Permission issues
chmod 700 ~/.ssh/sockets

# Verbose master debug
ssh -vvv -M -S ~/.ssh/sockets/test user@host

# Check master status
ssh -O check -S ~/.ssh/sockets/%r@%h:%p user@host

CASE #10

Pro Tips

Advanced techniques learned in the field. The difference between a rookie and a veteran.

Agent Forwarding (Use Carefully)

Agent forwarding allows remote hosts to use your local SSH keys.

# Enable for connection
ssh -A user@host

# In config
Host trustedhost
    ForwardAgent yes

# Best practice: Only enable for trusted hosts
Host *
    ForwardAgent no

Host bastion.example.com
    ForwardAgent yes

Security Warning: Agent forwarding is risky - root on intermediate host can hijack agent socket. Use ProxyJump instead when possible: ssh -J bastion target

Safer Agent Usage

# Time-limited agent
ssh-add -t 3600 ~/.ssh/id_ed25519  # 1 hour

# Confirm before each use (OpenSSH 8.4+)
ssh-add -c ~/.ssh/id_ed25519

Escape Sequences

Press ~ then another key (after newline):

Sequence	Action
`~.`	Disconnect immediately
`~^Z`	Background SSH session (fg to resume)
`~#`	List forwarded connections
`~&`	Background SSH at logout when waiting
`~?`	Show escape sequence help
`~C`	Open command line (add port forwards)
`~R`	Request connection rekeying

Field Tip: Frozen SSH session? Use ~. to disconnect cleanly. Add port forward mid-session with ~C then -L 8080:localhost:80.

Autossh

Automatically restart SSH connections. Keep your tunnels alive.

# Install
sudo apt install autossh

# Recommended: No monitoring port, use ServerAlive
autossh -M 0 \
    -o "ServerAliveInterval 30" \
    -o "ServerAliveCountMax 3" \
    user@host

# Background persistent tunnel
autossh -M 0 -f -N \
    -o "ServerAliveInterval 30" \
    -o "ServerAliveCountMax 3" \
    -L 8080:localhost:80 \
    user@host

Systemd Service for Autossh

# /etc/systemd/system/autossh-tunnel.service
[Unit]
Description=AutoSSH tunnel
After=network.target

[Service]
Environment="AUTOSSH_GATETIME=0"
ExecStart=/usr/bin/autossh -M 0 -N \
    -o "ServerAliveInterval=30" \
    -o "ServerAliveCountMax=3" \
    -L 8080:localhost:80 \
    user@host
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Quick Connection Tricks

# Pipeline remote command
ssh user@host 'cat /var/log/syslog' | grep ERROR

# Remote compression + local decompression
ssh user@host 'tar czf - /path' | tar xzf -

# Copy directory over network
tar czf - /local/dir | ssh user@host 'tar xzf - -C /remote/'

# Remote screen/tmux session
ssh -t user@host tmux attach
ssh -t user@host screen -rd

# X11 forwarding for single app
ssh -X user@host firefox

# SSH config per-command
ssh -F ~/.ssh/config.alternate user@host

SSH over Hostile Networks

# Use Mosh (mobile shell) for unreliable connections
mosh user@host

# SSH over HTTP proxy
ssh -o ProxyCommand="nc -X connect -x proxy:8080 %h %p" user@host

# SSH over SOCKS proxy
ssh -o ProxyCommand="nc -x socks5://127.0.0.1:1080 %h %p" user@host

# Through HTTP tunnel (corkscrew)
ssh -o ProxyCommand="corkscrew proxy 8080 %h %p" user@host

Debugging Connection Issues

# Verbose connection
ssh -vvv user@host

# Test specific cipher
ssh -c aes256-ctr user@host

# Test specific auth method
ssh -o PreferredAuthentications=publickey user@host

# Check server public key
ssh-keyscan host

# Verify known_hosts entry
ssh-keygen -F host

# Remove known_hosts entry
ssh-keygen -R host

SSH in Scripts

# Non-interactive (no prompts)
ssh -o BatchMode=yes user@host command

# Complete script-friendly command
ssh -o BatchMode=yes \
    -o StrictHostKeyChecking=yes \
    -o ConnectTimeout=10 \
    -o ServerAliveInterval=60 \
    -o ServerAliveCountMax=3 \
    user@host 'command here'

Modern SSH Features (OpenSSH 8.x+)

# FIDO2 hardware keys
ssh-keygen -t ed25519-sk

# Security key resident keys
ssh-keygen -t ed25519-sk -O resident

# Include config files
# ~/.ssh/config:
Include ~/.ssh/config.d/*.conf

# SetEnv (OpenSSH 8.7+)
Host dev
    SetEnv ENV=development DEBUG=true

# SessionType (OpenSSH 9.0+)
ssh -o SessionType=none user@host  # No shell, for port forwarding only

Final Note: SSH is more than a protocol - it's a philosophy. Every connection is secure, every session is authenticated, every tunnel is encrypted. In a world of open networks and unsecured channels, SSH is the private eye's best friend. Use it wisely.