# Kubernetes Cheatsheet
Tailored to your kubeadm + containerd + LXD lab (k8master / k8worker1)

---

## Cluster & Node Basics

```bash
kubectl cluster-info                        # API server / DNS endpoints
kubectl get nodes -o wide                   # node status, IP, OS, kernel, runtime
kubectl describe node <node>                # full node detail incl. Conditions, taints, capacity
kubectl get nodes --show-labels             # labels on each node
kubectl top nodes                           # CPU/mem usage (needs metrics-server)
```

**Taints & tolerations** (bit you with the registry pod):
```bash
kubectl describe node k8master | grep Taints
kubectl taint nodes k8master node-role.kubernetes.io/control-plane-   # REMOVE a taint (trailing -)
kubectl taint nodes k8master node-role.kubernetes.io/control-plane=:NoSchedule  # ADD it back
```
A pod needs a matching `tolerations:` entry to be schedulable on a tainted node — `nodeName:` alone does **not** bypass this.

---

## Pods

```bash
kubectl get pods                            # current namespace
kubectl get pods -A                         # all namespaces
kubectl get pods -o wide                    # + node, IP
kubectl get pods -l app=hello-api           # filter by label
kubectl get pods -w                         # watch live changes
kubectl describe pod <pod>                  # Events section = root cause of most failures
kubectl logs <pod>                          # current container logs
kubectl logs <pod> -c <container>           # specific container (multi-container / init pods)
kubectl logs <pod> --previous               # logs from a crashed instance
kubectl logs -f <pod>                       # follow/stream
kubectl logs -l app=hello-api               # logs by label (all matching pods)
kubectl exec -it <pod> -- bash              # shell into a pod
kubectl delete pod <pod>                    # force recreation (if managed by a Deployment)
kubectl delete pod <pod> --force --grace-period=0   # kill stuck pod immediately
```

**Diagnosing `Pending`**: check `describe pod` Events — usually taints, insufficient resources, or no matching node.
**Diagnosing `ImagePullBackOff`**: check `describe pod` Events for the *exact* pull error (TLS, DNS, auth, not-found) — `crictl pull <image>` on the node reproduces the same error without the Kubernetes wrapper.
**Diagnosing `CrashLoopBackOff`**: `kubectl logs <pod> --previous` almost always has the real reason.

---

## Deployments & ReplicaSets

```bash
kubectl get deployments
kubectl describe deployment <name>
kubectl scale deployment <name> --replicas=3
kubectl rollout status deployment <name>
kubectl rollout history deployment <name>
kubectl rollout undo deployment <name>              # roll back to previous version
kubectl rollout restart deployment <name>           # force pods to recreate (e.g. after ConfigMap change)
kubectl set image deployment/<name> <container>=<image>:<tag>   # update image in place
kubectl delete deployment <name>
```

---

## Services & Networking

```bash
kubectl get svc
kubectl describe svc <name>
kubectl get endpoints <svc-name>            # empty = selector doesn't match any pod labels (common bug)
```

**Service types quick reference:**
| Type | Reachable from | Use case |
|---|---|---|
| `ClusterIP` (default) | inside cluster only | pod-to-pod, pod-to-service |
| `NodePort` | any node's IP, on a high port (30000-32767) | quick external testing (what you used) |
| `LoadBalancer` | external LB (needs cloud/metallb) | production external access |

```bash
kubectl get svc <name> -o jsonpath='{.spec.ports[0].nodePort}'   # get the actual NodePort
kubectl get svc <name> -o jsonpath='{.spec.clusterIP}'           # get the ClusterIP
```

**Test connectivity, in order of isolation** (cheapest to diagnose first):
```bash
POD_IP=$(kubectl get pod -l app=<label> -o jsonpath='{.items[0].status.podIP}')
curl http://$POD_IP:<containerPort>          # 1. pod directly — proves app + CNI work
curl http://<clusterIP>:<port>               # 2. via Service internally — proves kube-proxy/iptables
curl http://<node-ip>:<nodePort>             # 3. via NodePort externally — proves node-level routing
```

---

## ConfigMaps & Secrets

```bash
kubectl create configmap <name> --from-file=<key>=<path>
kubectl create configmap <name> --from-literal=key=value
kubectl get configmap <name> -o yaml
kubectl get configmap <name> -o jsonpath='{.data.<key>}'    # read one key's raw content
kubectl delete configmap <name>
kubectl edit configmap <name>                               # live-edit (e.g. kube-proxy conntrack fix)
```
> ConfigMaps have a **1MiB size limit** — fat jars/binaries need a hostPath volume or an actual registry, not a ConfigMap mount.

```bash
kubectl create secret generic <name> --from-literal=key=value
kubectl get secrets
kubectl describe secret <name>
```

---

## Namespaces

```bash
kubectl get namespaces
kubectl get pods -n kube-system
kubectl config set-context --current --namespace=<ns>       # switch default namespace for kubectl
```

---

## Jobs (Kaniko-style one-shot builds)

```bash
kubectl apply -f job.yaml
kubectl get pods -l job-name=<job-name> -w
kubectl logs -f job/<job-name>
kubectl delete job <job-name>                # ALWAYS delete before reapplying — Jobs don't self-clean
```

---

## Applying / Editing Resources

```bash
kubectl apply -f file.yaml                   # create or update
kubectl delete -f file.yaml
kubectl get <resource> <name> -o yaml        # dump full current definition
kubectl edit <resource> <name>               # live edit in $EDITOR
kubectl diff -f file.yaml                    # preview what apply would change
kubectl explain <resource>.<field>           # inline docs, e.g. kubectl explain pod.spec.tolerations
```

---

## Debugging & Events

```bash
kubectl get events -A --sort-by='.lastTimestamp'      # cluster-wide event timeline, newest last
kubectl describe <resource> <name>                     # Events section = #1 debugging tool
kubectl get all -A                                      # everything, everywhere
kubectl api-resources                                   # list all resource types kubectl knows about
```

---

## Node-level tools (via `lxc exec`, bypasses Kubernetes entirely)

```bash
lxc exec <node> -- crictl ps                 # running containers on that node (containerd view)
lxc exec <node> -- crictl images             # images cached on that node
lxc exec <node> -- crictl pull <image>       # reproduce a pull error directly
lxc exec <node> -- crictl logs <container>   # container logs at containerd level
lxc exec <node> -- systemctl status kubelet
lxc exec <node> -- systemctl status containerd
lxc exec <node> -- journalctl -u kubelet -f  # live kubelet logs
```

---

## kubeadm-specific

```bash
kubeadm token create --print-join-command    # generate a fresh join command
kubeadm token list                           # active tokens
kubectl get nodes                            # confirm join succeeded
kubeadm reset                                # tear down this node's kubeadm state (careful!)
```

---

## Your lab's known gotchas, quick reference

| Symptom | Cause | Fix |
|---|---|---|
| `SystemVerification` warning re: `configs` module | kubeadm can't inspect kernel config inside a container | Harmless warning on `init`; use `--ignore-preflight-errors=Swap,SystemVerification` on `join` |
| Node stuck `NotReady` | No CNI installed, or CNI pods crashing | Check `kubectl get pods -n kube-flannel`; use `releases/latest/download`, not `master` branch manifest |
| `kube-proxy` `CrashLoopBackOff` | `/proc/sys/net/netfilter/nf_conntrack_max: permission denied` (nested container) | Edit `kube-proxy` ConfigMap, set `conntrack.maxPerCore: 0`, then `kubectl rollout restart daemonset kube-proxy -n kube-system` |
| Flannel init container `no such file or directory` | Manifest/image version mismatch (floating `master` branch) | Use `https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml` |
| `ImagePullBackOff`, "server gave HTTP response to HTTPS client" | containerd assumes HTTPS for your insecure local registry | Add `/etc/containerd/certs.d/<registry>/hosts.toml` with `skip_verify = true` on **every node**, restart containerd |
| Workload pods only ever land on the worker | Control-plane taint (`node-role.kubernetes.io/control-plane:NoSchedule`) | Expected — add a `tolerations:` block if you need something on `k8master`, e.g. the registry |
| `k8sbr0` interface vanishes | LXD daemon lost track of the network | `sudo snap restart lxd` (quick fix; investigate if it recurs) |
| `kubectl create configmap ... --from-file` "already exists" | Reapplying without deleting first | `kubectl delete configmap <name>` before recreating, or use `--dry-run=client -o yaml \| kubectl apply -f -` |

---

## Full teardown / rebuild (from your toolkit)

```bash
./scripts/validate.sh              # health check everything
./scripts/rebuild.sh               # containers-only rebuild, keeps LXD config
./scripts/rebuild.sh --full        # full rebuild incl. LXD reinstall
./scripts/uninstall.sh --containers    # delete just k8master/k8worker1
./scripts/uninstall.sh --lxd-config    # + remove bridge/profile/pool
./scripts/uninstall.sh --purge         # remove LXD entirely
```