Foundations
Linux Administration
27 labs · 14h 30mThe layer everything else runs on — files, permissions, users, processes, systemd, logs, disks, and networking.
- Assumes
- Nothing — start here
- Unlocks
- Docker
Almost every problem you will debug in cloud infrastructure is a Linux problem wearing a cloud provider's logo. A container that will not start, a disk that filled overnight, a service that dies on boot, an SSH key that stopped working — these are answered with journalctl and df, not with a console.
How to get a Linux box for these
You need a real Linux system with sudo. Any of these works, cheapest first:
- A VM on your own machine. Multipass (
multipass launch --name lab) or UTM on Apple silicon, running Ubuntu 24.04. Free, and snapshots make mistakes cheap. - WSL2 on Windows. Fine for everything except the
systemdand disk labs, which want a full VM. - A cloud instance. A
t4g.smallor equivalent for a few hours costs less than a coffee. Every lab that provisions anything says so and tells you how to remove it.
macOS is not a substitute. The commands differ, systemd does not exist, and the labs that inspect /proc have nothing to inspect.
The order
Quick start is your first hour: five short labs that get you moving around a machine without guessing. Week 1 and Week 2 are the actual foundation, ten labs paced as two weeks of evenings. Text, scripting, and automation is the half of the job that is not typing commands one at a time — searching a tree, reshaping output with a pipeline, writing a script that fails safely, and scheduling it. For cloud and DevOps work is the set that separates someone who can use Linux from someone who can be trusted with a fleet — service units, log triage, SSH hardening, growing a disk, debugging a network path, and a backup you have actually restored.
Challenge labs give you a goal and no steps. Do them; they are where the earlier labs stop being trivia.
The labs, in order
Quick start1h 15m
- Find your way around a machine you have never seenAnswer the four questions you always ask on a new box — where am I, what is here, what is this thing, and how big is it.15mBeginner
- Read and search files without opening an editorUse less, head, tail and grep to answer questions about a 200MB log you must never load into memory.15mBeginner
- Create, copy, move and delete without regretThe four commands that change a filesystem, and the flags that stop them destroying something you needed.15mBeginner
- Work out who you are and what you are allowed to doRead your own identity, group memberships and sudo rights before you need them in an incident.10mBeginner
- Challenge: the vanished configA service will not start because its config is not where it should be. Find it, work out what changed, and put it back — no steps given.20mChallengeBeginner
Week 1 — files, users, and permissions2h 30m
- Day 01: Read and set file permissionsDecode an ls -l permission string, then set the same thing three ways with chmod and prove which bits changed.30mBeginner
- Day 02: Ownership and groupsChange a file's owner and group, then set up a shared directory two users can both write to without giving anything away.30mBeginner
- Day 03: What a user account actually isBuild an account by hand, read every file that records it, and see why a service account has no shell and no password.35mBeginner
- Day 04: umask, setuid and the sticky bitWork out why new files get the permissions they do, then examine the three special bits and where each one is legitimately used.30mBeginner
- Day 05 challenge: the locked-out teamTwo users cannot use a shared directory that looks correctly configured. Four things are wrong. Find and fix them all.25mChallengeBeginner
Week 2 — processes, services, and the network2h 40m
- Day 06: Processes and signalsFind a process, read what it is doing from /proc, and stop it politely before stopping it rudely.30mBeginner
- Day 07: systemd unitsRead the state of a real service, then write your own unit that survives a reboot and restarts when it crashes.40mBeginner
- Day 08: Reading the journalNarrow a million log lines to the twenty that matter, by unit, by time, by priority, and by boot.30mBeginner
- Day 09: Packages and updatesInstall, inspect and pin packages, and find out which package owns a file you did not expect to be there.25mBeginner
- Day 10 challenge: the service that will not startA unit fails on boot. Three faults, one of which only appears after a reboot. Diagnose from the journal, not by guessing.35mChallengeBeginner
Text, scripting, and automation2h 25m
- Find files by anything, then act on them safelySearch a tree by name, age, size, and permission, then run a command over the results without breaking on a space in a filename.30mBeginner
- Reshape output with a pipelineTurn a raw log into a ranked count using cut, sort, uniq, awk, sed and jq — the six tools that do most of the work in this job.40mIntermediate
- Write a script that fails safelyTake a script that silently corrupts things and fix it with strict mode, quoting, traps, and a real argument check.40mIntermediate
- Schedule work with systemd timers, and with cronRun the same job both ways, then compare what each one tells you when it fails — which is the only reason to prefer one.35mIntermediate
For cloud and DevOps work5h 40m
- Ship your app as a hardened systemd serviceTake a script from "it runs in my terminal" to a unit with its own account, its own state directory, and a sandbox that survives a compromise.45mIntermediate
- SSH keys and hardening a server's front doorSet up key-only access, then turn off the things that let an attacker guess their way in — and verify each change before you rely on it.40mIntermediate
- Grow a filesystem that filled upBuild an LVM volume on loopback files, fill it until writes fail, then extend it online with nothing unmounted and no data lost.50mIntermediate
- Keep logs from filling the diskCap the journal, rotate an application log, and prove both limits hold by generating more data than the cap allows.35mIntermediate
- Debug a network path from one endTake "it times out" apart into six separate questions — DNS, route, listener, firewall, TLS, and the application — and answer each with one command.40mIntermediate
- Triage a machine that is up and too slowDecide whether you are short of CPU, memory, or disk in under five minutes, using load, pressure stall information, and iostat.45mIntermediate
- Challenge: the box that filled up at 3amA disk is full, df and du disagree, and a service is down. Find the cause, restore the space, and stop it recurring — no steps given.40mChallengeIntermediate
- Take a backup and prove the restoreArchive a directory with tar, mirror it with rsync, then restore into an empty tree and verify byte-for-byte that nothing was lost.45mIntermediate
Done in order, the track is about 14h 30m of work.