# Kubernetes-from-Scratch on LXD (Ubuntu 26.04, LXD 5.21 LTS, Kubernetes 1.33.x)

Matches the diagram: `Internet -> VirtualBox NAT (10.0.2.15) -> Ubuntu 26 Host ->
LXD bridge k8sbr0 (192.168.100.1/24, DHCP+NAT) -> k8master (.10) / k8worker1 (.11)`.

## Layout

```
lxd-k8s/
├── lxd-preseed.yaml            # storage pool + bridge + default/k8s-base profiles
├── README.md                   # this file
└── scripts/
    ├── 00-init-lxd.sh          # install LXD snap, apply preseed
    ├── 01-create-containers.sh # launch k8master/k8worker1, pin static IPs
    ├── 02-node-common-setup.sh # run INSIDE each node: containerd + kubeadm/kubelet/kubectl
    ├── 03-init-master.sh       # run INSIDE k8master: kubeadm init + Flannel CNI
    ├── 04-join-worker.sh       # run on HOST: relay join command to k8worker1
    ├── rebuild.sh              # full or container-only rebuild, idempotent
    ├── validate.sh             # health checks for LXD + Kubernetes
    └── uninstall.sh            # staged undo (containers / config / full purge)
```

## One-time setup order

```bash
chmod +x scripts/*.sh

# 1. Install LXD and apply the preseed (storage pool, k8sbr0, profiles)
./scripts/00-init-lxd.sh
newgrp lxd                       # or log out/in so `lxc` works without sudo

# 2. Launch the two containers with static IPs on k8s-base
./scripts/01-create-containers.sh

# 3. Install containerd/kubeadm/kubelet/kubectl on BOTH nodes
lxc file push scripts/02-node-common-setup.sh k8master/root/setup.sh
lxc exec k8master -- bash /root/setup.sh
lxc file push scripts/02-node-common-setup.sh k8worker1/root/setup.sh
lxc exec k8worker1 -- bash /root/setup.sh

# 4. Bootstrap the control plane
lxc file push scripts/03-init-master.sh k8master/root/init-master.sh
lxc exec k8master -- bash /root/init-master.sh

# 5. Join the worker
./scripts/04-join-worker.sh

# 6. Validate
./scripts/validate.sh
```

Or run the whole thing unattended after step 1:

```bash
./scripts/rebuild.sh
```

## Snapshots at each milestone

Take these as you go (also done automatically inside the scripts above):

| Milestone | Command |
|---|---|
| Fresh OS, before k8s packages | `lxc snapshot k8master base-provisioned` |
| containerd/kubeadm installed | `lxc snapshot k8master packages-installed` |
| Control plane initialized | `lxc snapshot k8master control-plane-ready` |
| Worker joined | `lxc snapshot k8worker1 cluster-joined` |

Restore any milestone with:
```bash
lxc restore k8master control-plane-ready
```
This is far faster than re-running kubeadm from zero when you're iterating on
later steps (add-ons, RBAC, storage classes, etc.).

## Static IP + DHCP design note

`k8sbr0` runs DHCP across `192.168.100.100-199` only. `.2-.99` are reserved
for static assignments. `01-create-containers.sh` launches each container
normally (gets a lease), then overrides the NIC with a fixed
`ipv4.address` via `lxc config device override <c> eth0 ipv4.address=<ip>`
and restarts it — this is the supported LXD pattern for "DHCP network,
static host," and avoids editing `dnsmasq` leases by hand.

## Design choices worth knowing

- **`k8s-base` is a separate profile from `default`.** Kubernetes needs
  `security.privileged`, `security.nesting`, extra kernel modules, and an
  unconfined AppArmor profile. Bolting that onto `default` would make every
  future container on this host privileged by accident.
- **containerd, not Docker**, since kubeadm 1.24+ dropped dockershim; this
  matches current upstream guidance for 1.33.x.
- **Flannel** is used for the CNI (simple, matches `--pod-network-cidr
  10.244.0.0/16`). Swap for Cilium/Calico by editing `03-init-master.sh`
  if you want to practice a different CNI.
- **ZFS storage pool** defaults to a loop-backed file (`size: 30GiB`) so it
  works on a single-disk VirtualBox VM out of the box. Swap in a raw second
  disk (`source: /dev/sdc1`) if you attach one — see the comments in
  `lxd-preseed.yaml`.

## Uninstall / undo

```bash
./scripts/uninstall.sh                # interactive, asks at each stage
./scripts/uninstall.sh --containers   # only remove k8master/k8worker1
./scripts/uninstall.sh --lxd-config   # also remove k8sbr0, k8s-base, pool 'default'
./scripts/uninstall.sh --purge        # remove everything, including the LXD snap itself
```

## Troubleshooting quick hits

- `lxc list` says permission denied → you weren't re-added to the `lxd`
  group in this shell; run `newgrp lxd` or log out/in.
- `kubeadm init` preflight fails on `swap` → confirm
  `lxc exec k8master -- swapon --show` is empty; `02-node-common-setup.sh`
  disables it, but a snapshot restore can undo that.
- Worker `NotReady` after join → CNI pods still pulling images; check
  `lxc exec k8master -- kubectl get pods -A` for `flannel` pod status.
- `zpool create` fails during `lxd init` → another zpool already claims the
  loop file/device; run `sudo zpool list` and `sudo zpool destroy <name>`
  from a previous attempt before retrying.

## Reference materials

See `lxd-k8s-reference-guide.pdf` and `lxd-k8s-reference.pptx` (generated
alongside this kit) for curated GitHub repos, official docs, and video
walkthroughs to go deeper.