LXD Configuration Documentation

Overview

LXD server manages all sandbox and exam containers with network isolation and resource limits.

Server Details

Host: 65.109.236.163
Private IP: 10.0.0.3
LXD Port: 8443 (HTTPS API)
SSH Port: 22
Network: 10.35.61.0/24 (lxdbr0 bridge)

LXD Installation

Initial Setup

ssh root@65.109.236.163

# Install LXD
snap install lxd

# Initialize LXD
lxd init

LXD Init Configuration

# Network
Would you like to use LXD clustering? (yes/no): no
Do you want to configure a new storage pool? (yes/no): yes
Name of the new storage pool: default
Name of the storage backend to use (btrfs, dir, lvm, zfs): dir
Would you like to connect to a MAAS server? (yes/no): no

# Network bridge
Would you like to create a new local network bridge? (yes/no): yes
What should the new bridge be called? lxdbr0
What IPv4 address should be used? (CIDR subnet notation): 10.35.61.1/24
What IPv6 address should be used? (CIDR subnet notation): none

# Remote access
Would you like LXD to be available over the network? (yes/no): yes
Address to bind LXD to: 0.0.0.0
Port: 8443
Trust password for new clients: [set secure password]

Network Configuration

Bridge Interface (lxdbr0)

# View network details
lxc network show lxdbr0

# Configuration
ip addr show lxdbr0
# Should show: 10.35.61.1/24

# NAT and forwarding enabled for internet access
iptables -t nat -L POSTROUTING

Container Network

Range: 10.35.61.10 - 10.35.61.250
Gateway: 10.35.61.1
DNS: Automatic (from host)
DHCP: Enabled (may fallback to static assignment)

Container Images

Base Images

Sandbox Images

# Ubuntu with SSH
lxc image list | grep platform-ubuntu-ssh
# Alias: platform-ubuntu-ssh
# Based on: ubuntu/22.04

# CentOS with SSH
lxc image list | grep platform-centos-ssh
# Alias: platform-centos-ssh
# Based on: images:centos/8-Stream

Exam Images

# Linux File System Basics
lxc image list | grep platform-exam-linux-basics
# Alias: platform-exam-linux-basics
# Includes: Basic tools, verification scripts

# Linux Service Management
lxc image list | grep platform-exam-linux-services
# Alias: platform-exam-linux-services
# Includes: nginx, systemd tools, verification scripts

Creating Base Images

Ubuntu SSH Image

# Create base container
lxc launch ubuntu/22.04 ubuntu-ssh-base

# Configure container
lxc exec ubuntu-ssh-base -- bash << 'EOF'
apt update
apt install -y openssh-server sudo curl wget vim nano
systemctl enable ssh
useradd -m -s /bin/bash ubuntu
echo "ubuntu ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/ubuntu
mkdir -p /root/.ssh
chmod 700 /root/.ssh
mkdir -p /home/ubuntu/.ssh
chmod 700 /home/ubuntu/.ssh
chown -R ubuntu:ubuntu /home/ubuntu/.ssh
EOF

# Stop and publish
lxc stop ubuntu-ssh-base
lxc publish ubuntu-ssh-base --alias platform-ubuntu-ssh
lxc delete ubuntu-ssh-base

CentOS SSH Image

lxc launch images:centos/8-Stream centos-ssh-base

lxc exec centos-ssh-base -- bash << 'EOF'
dnf update -y
dnf install -y openssh-server sudo curl wget vim nano
systemctl enable sshd
useradd -m -s /bin/bash centos
echo "centos ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/centos
mkdir -p /root/.ssh
chmod 700 /root/.ssh
EOF

lxc stop centos-ssh-base
lxc publish centos-ssh-base --alias platform-centos-ssh
lxc delete centos-ssh-base

Creating Exam Images

Linux File System Basics

lxc launch ubuntu/22.04 exam-linux-basics-base

lxc exec exam-linux-basics-base -- bash << 'EOF'
apt update
apt install -y openssh-server sudo curl wget vim nano
systemctl enable ssh

# Create ubuntu user
useradd -m -s /bin/bash ubuntu
echo "ubuntu ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/ubuntu
mkdir -p /home/ubuntu
chown -R ubuntu:ubuntu /home/ubuntu

# Setup SSH
mkdir -p /root/.ssh
chmod 700 /root/.ssh
EOF

lxc stop exam-linux-basics-base
lxc publish exam-linux-basics-base --alias platform-exam-linux-basics \
  description="Platform exam image: linux-basics"
lxc delete exam-linux-basics-base

Linux Service Management

lxc launch ubuntu/22.04 exam-linux-services-base

lxc exec exam-linux-services-base -- bash << 'EOF'
apt update
apt install -y openssh-server sudo curl wget vim nano nginx
systemctl enable ssh

# Don't start nginx (students will start it)
systemctl stop nginx
systemctl disable nginx

useradd -m -s /bin/bash ubuntu
echo "ubuntu ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/ubuntu
mkdir -p /root/.ssh
chmod 700 /root/.ssh
EOF

lxc stop exam-linux-services-base
lxc publish exam-linux-services-base --alias platform-exam-linux-services \
  description="Platform exam image: linux-services"
lxc delete exam-linux-services-base

Container Management

List Containers

# All containers
lxc list

# Filter by type
lxc list | grep user-
lxc list | grep exam-

Container Lifecycle

# Create (done by backend API)
lxc launch platform-ubuntu-ssh user-1-sandbox

# Start/Stop
lxc start user-1-sandbox
lxc stop user-1-sandbox

# Execute commands
lxc exec user-1-sandbox -- bash
lxc exec user-1-sandbox -- ls -la /home/ubuntu

# Delete
lxc delete user-1-sandbox --force

Resource Limits

# Set CPU limit
lxc config set user-1-sandbox limits.cpu=2

# Set memory limit
lxc config set user-1-sandbox limits.memory=2GB

# Set disk limit
lxc config device override user-1-sandbox root size=10GB

Remote Access Configuration

TLS Certificates

Generate Client Certificates (on application server)

# Already configured in backend/.lxd/
# client.crt and client.key are used for LXD API access

Trust Client Certificate (on LXD server)

# Add client certificate to LXD
lxc config trust add client.crt

SSH Access

Backend uses SSH to execute commands in containers via lxc exec:

# Example: Execute command in container via SSH
ssh -i /app/.ssh/lxd_key root@65.109.236.163 \
  "lxc exec container-name -- bash -c 'command'"

Security

Firewall

# Allow LXD API (from application server only)
ufw allow from 77.42.82.4 to any port 8443 proto tcp

# Allow SSH
ufw allow 22/tcp

# Enable firewall
ufw enable

Container Isolation

Unprivileged containers (default)
No nested virtualization
Network isolation via bridge
Resource limits enforced

Automatic Cleanup

Backend runs cleanup task every 5 minutes:

Deletes containers older than 1 hour (based on user.expires_at metadata)
Prevents resource exhaustion

Monitoring

Container Status

# List all containers with details
lxc list

# Show container info
lxc info user-1-sandbox

# Resource usage
lxc list --format=yaml | grep -A5 "state:"

Network Status

# Bridge status
ip addr show lxdbr0

# Active connections
lxc list --format=csv -c n,4 | grep -v "10.35.61"

# NAT rules
iptables -t nat -L POSTROUTING -n -v

Storage Usage

# LXD storage pool
lxc storage info default

# Container disk usage
lxc exec container-name -- df -h

Troubleshooting

Container won't start

# Check logs
lxc info user-1-sandbox --show-log

# Check resource limits
lxc config show user-1-sandbox

# Increase limits if needed
lxc config set user-1-sandbox limits.memory=4GB

Network issues

# Container has no IP
lxc exec container-name -- ip addr
lxc exec container-name -- dhclient eth0

# Can't reach internet
lxc exec container-name -- ping -c3 8.8.8.8

# Check NAT rules
iptables -t nat -L POSTROUTING

SSH issues

# SSH not running in container
lxc exec container-name -- systemctl status ssh
lxc exec container-name -- systemctl start ssh

# Check authorized_keys
lxc exec container-name -- cat /root/.ssh/authorized_keys

Image issues

# List images
lxc image list

# Delete image
lxc image delete platform-ubuntu-ssh

# Re-create image (use creation script above)

Maintenance

Update Images

# Update base Ubuntu
lxc image list | grep ubuntu/22.04
lxc image refresh ubuntu/22.04

# Rebuild custom images
# Use creation scripts above

Clean Up Stopped Containers

# List stopped containers
lxc list --format=csv -c ns | grep STOPPED

# Delete stopped containers
for container in $(lxc list --format=csv -c n | grep stopped); do
  lxc delete $container --force
done

Storage Maintenance

# Check storage usage
lxc storage info default

# Optimize storage
lxc storage volume create default cleanup
lxc storage volume delete default cleanup

Best Practices

Resource Limits: Always set CPU and memory limits
Image Updates: Rebuild exam images when updating packages
Monitoring: Check container count regularly
Cleanup: Backend handles automatic cleanup, but verify it's working
Backups: Backup image definitions, not individual containers
Security: Keep SSH keys secure, use TLS for LXD API
Network: Monitor bridge interface for issues

Container Naming Convention

Sandbox: user-{user_id}-sandbox
Example: user-1-sandbox

Exam: exam-{exam_id}-user-{user_id}-{timestamp}
Example: exam-1-user-1-1769968425

Useful Commands Reference

# Quick container count
lxc list | grep -c RUNNING

# Show all container IPs
lxc list --format=csv -c n,4

# Execute command in all running containers
for container in $(lxc list --format=csv -c n | grep user-); do
  lxc exec $container -- command
done

# Check LXD daemon status
systemctl status snap.lxd.daemon

# Restart LXD (careful!)
systemctl restart snap.lxd.daemon

# View LXD daemon logs
journalctl -u snap.lxd.daemon -f

Overview​

Server Details​

LXD Installation​

Initial Setup​

LXD Init Configuration​

Network Configuration​

Bridge Interface (lxdbr0)​

Container Network​

Container Images​

Base Images​

Sandbox Images​

Exam Images​

Creating Base Images​

Ubuntu SSH Image​

CentOS SSH Image​

Creating Exam Images​

Linux File System Basics​

Linux Service Management​

Container Management​

List Containers​

Container Lifecycle​

Resource Limits​

Remote Access Configuration​

TLS Certificates​

Generate Client Certificates (on application server)​

Trust Client Certificate (on LXD server)​

SSH Access​

Security​

Firewall​

Container Isolation​

Automatic Cleanup​

Monitoring​

Container Status​

Network Status​

Storage Usage​

Troubleshooting​

Container won't start​

Network issues​

SSH issues​

Image issues​

Maintenance​

Update Images​

Clean Up Stopped Containers​

Storage Maintenance​

Best Practices​

Container Naming Convention​

Useful Commands Reference​