SSHing Into Your Containers on Admiral

SSHing Into Your Containers on Admiral

There are two kinds of people in edge infrastructure: those who’ve tried to debug a misbehaving device in a warehouse two time zones away, and those who will.

Eventually, no matter how good your telemetry is, you’ll want a shell. Not a web terminal, not a remote command runner - a real, interactive SSH session, inside the container that’s actually running your workload, on a specific device in the field.

Admiral is built so you can do this safely, without ever opening an inbound port on the device’s network.

This guide covers the pattern, the security model, and a concrete example drawn from our PS2 emulator demo.

The problem with “just run sshd”

The naive answer is: install openssh-server in your container, expose port 22, and map it to the host. This works, in that packets travel, but it creates three problems in a real deployment:

1. You need an inbound path to every device. That means either public IPs (rare, risky), a VPN per device (operationally painful), or punching holes in customer firewalls (career-limiting).
2. You need to distribute and rotate keys. A static authorized_keys file baked into an image is not a credential management strategy.
3. You lose the audit trail. If the login bypasses your control plane, you have no record of who was on what device when.

Admiral solves all three - but the starting point is still “run sshd in your container.” Let’s do that part right first.

Running sshd in your workload image

Here’s the relevant slice from the Nightfire Dockerfile:

RUN apt-get install -y --no-install-recommends openssh-server

# Pull authorized keys directly from GitHub at build time / chuck mine in, why not!?
RUN mkdir -p /root/.ssh && chmod 700 /root/.ssh
ADD https://github.com/alexanderturner.keys /root/.ssh/authorized_keys_alex
RUN cat /root/.ssh/authorized_keys_alex >> /root/.ssh/authorized_keys && 
    rm /root/.ssh/authorized_keys_alex && 
    chmod 600 /root/.ssh/authorized_keys

RUN sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN mkdir -p /run/sshd
EXPOSE 22

Three things worth calling out:

• Keys come from GitHub. https://github.com/<username>.keys returns every public key a user has uploaded. Pulling it at image build time means you don’t commit keys to source and you don’t have to distribute them yourself. When someone leaves the team, their GitHub keys don’t automatically come with them - rebuild your image and the next rollout locks them out.
• PermitRootLogin yes is fine here because the container has no other users and the root filesystem is immutable. Your threat model might differ; if so, add a non-root user and use that instead.
• EXPOSE 22 is documentation. It doesn’t actually open anything. Exposing port 22 to the public internet is what Admiral’s tunnel architecture is for.

The container’s startup script (start.sh) launches the workload in the background and then execs sshd in the foreground:

/usr/local/bin/run-pcsx2.sh &
mkdir -p /run/sshd
exec /usr/sbin/sshd -D

exec matters here - it makes sshd PID 1, so when Admiral stops the container, SSH shuts down cleanly.

How Admiral gets you in without opening a port

Here’s the part most platforms don’t solve well.

Admiral agents on every device maintain a persistent, outbound, encrypted tunnel to the control plane. It’s a single long-lived connection over TLS 1.3 (with hybrid post-quantum key exchange - see our post on post-quantum edge security for the gory details). No inbound firewall rule is ever required. The device dials out; nobody dials in.

When you run:

admiral ssh edge-prod-aus-0187

…what actually happens is:

1. The Admiral CLI authenticates you against the control plane using your short-lived credentials
2. The control plane verifies you have SSH permission for that specific device
3. A tunnel is opened through the device’s existing outbound connection
4. Your local SSH client is handed a socket on the other end
5. Standard SSH key-based auth happens inside that tunnel, against the authorized_keys in the container

Two layers of authentication: your Admiral credentials get you the tunnel, your SSH key gets you the shell. Either one missing, no access.

Every session is logged in the control plane with the user, device, start time, and duration. Your security team gets an audit trail for free.

Practical tips

Don’t bake sshd into every image. Only the workloads where you actively want interactive debugging. Your production signage player probably doesn’t need it.

Rotate your GitHub keys occasionally. The ADD directive pulls them at build time, not run time. Old images carry old keys until you rebuild.

You can scp too. admiral scp ./patch.tar.gz edge-prod-aus-0187:/tmp/ uses the same tunnel. Useful for pulling logs off a device without writing a whole diagnostic pipeline.

Multi-user? Use a non-root user and PAM. The Nightfire demo uses root because it’s a single-user gaming kiosk. For production workloads with multiple operators, set up a proper user and consider integrating with your identity provider.

When you shouldn’t use SSH

SSH is for debugging, not for operations. If you find yourself SSHing into devices to apply fixes or change config, that’s a signal to push a new image instead. The immutable rootfs is there precisely so production state converges back to what’s in your registry.

Rule of thumb we use internally: if you’re SSHing into the same device twice in a week, stop. Either your telemetry is missing something (fix your telemetry) or there’s a bug you should be shipping a fix for (ship the fix).

SSH is the escape hatch. The goal is to almost never need it.