Lab Environment
If you want to make an apple pie from scratch, you must first create the universe.
The Hitchhiker’s Guide to the Galaxy, Douglas Adams
Lab environments are vital for the work I’m doing. In most situations fixing problems is “easy” but proving a solution works or that a problem occurs in the first place is the biggest headache I have.
Creating/destroying lab infrastructure takes hours to days and leaving it running in AWS wastes money, so I have my own personal lab for destructive testing.
Setting up and documenting the lab prerequisites has been like pulling out a tree in all honesty. I hope this effort is useful to the reader:
- Podman Quadlet - used to host Nexus for image distribution and also required for LDAP and MSSQL services
- Packer and Nexus - Used to build and host VM images
- Lab CA - Used for self-signed lab PKI infrastructure
- Big ass computer running Windows to run it on 😈
Why Windows when I’m hardcore Linux user? Sometimes I like to power off the lab and do gaming. There’s literally no other reason. If you don’t have this requirement, use Linux hosted VMs on libvirt, proxmox or whatever.
You can probably skip nexus if your in a “hurry” but you now need to manage a bunch of vital vhdx files spread across the system and still need to figure out how to get podman quadlet working if you want to host containerized LDAP and MSSQL. I love using nexus, it means I can trash my windows box and I always know what image my VMs are based on.
Lab Features
- Runs on hyperv
- Powershell script to create all VMs
- Basic SOE (node_exporter)
- Containerized services running in own VMs (for on/off ability):
- ldap
- mssql
- PostgreSQL server (todo)
- Confluent Platform VMs (infra only - no software deployment)
- Lab Kubernetes cluster (K3S)
- Ansible expects your lab CA at ~/lab_ca for ldap server deployment
Setting up a lab
After setting up the above prerequisites, I just follow my own docs in https://github.com/GeoffWilliams/lab and I can create an entire lab environment very quickly and on/off individual servers though hyperv manager:
As soon as a new image is booted, the VM image looks up the VM name from hyperv metadata, sets hostname and reboots. The images are already enabled with my SSH key and my router auto registers hostnames into DNS.
StrictHostKeyChecking accept-new
In ~/.ssh/config means I’m able to fully manage any new VMs with ansible as soon as they are ready.
In summary, having a lab environment is essential but a repeatable process to (re)create it makes work 10x easier.
Of course, the other way to do this would be to run everything in AWS. You can totally do this but you will learn less and pay for the privilege. Running all this stuff locally gives you a lot more things to think about.
All of this lab setup gives us a place to start work(!)
Enjoy.