Skip to content
Linux Administration
Lab 15 of 27·35mChallengeBeginner

Day 10 challenge: the service that will not start

A unit fails on boot. Three faults, one of which only appears after a reboot. Diagnose from the journal, not by guessing.

You need

  • A Linux VM with systemd and sudo access
  • Willingness to reboot the VM

Do first

Everything you need is in Days 06 to 09. The discipline being tested is reading the error before changing anything.

Set up the scenario

Run this exactly as written.

sudo mkdir -p /opt/reportd
sudo tee /opt/reportd/reportd.sh >/dev/null <<'SCRIPT'
#!/bin/bash
OUT=/var/lib/reportd/report.log
while true; do
  echo "report $(date -Is)" >> "$OUT"
  sleep 5
done
SCRIPT
sudo chmod 644 /opt/reportd/reportd.sh
sudo tee /etc/systemd/system/reportd.service >/dev/null <<'UNIT'
[Unit]
Description=Report daemon (broken on purpose)

[Service]
Type=simple
ExecStart=/opt/reportd/reportd.sh
User=reportd
Restart=on-failure
UNIT
sudo systemctl daemon-reload
sudo systemctl start reportd

The situation

systemctl status reportd --no-pager

It is not running. There are three independent faults, plus one thing that is correct now but will break after a reboot.

Your goal

  1. systemctl is-active reportd reports active.
  2. The service is writing lines to /var/lib/reportd/report.log.
  3. It runs as a dedicated non-root account, not as root and not as nobody.
  4. It comes back automatically after sudo reboot.
  5. You can state, in one sentence each, what the three faults were — found from the journal, not from reading the setup script above.

Constraints

  • Do not change User= to root. Running it as root would make two of the faults disappear and is the wrong fix.
  • Do not edit /opt/reportd/reportd.sh. The script is correct.
  • The fix for the reboot fault is one line in one file.

What each fault looks like

They surface in a specific order — fix one and the next appears. That is normal, and the reason to re-read the journal after every change rather than making three changes at once.

Hints, in increasing order of spoiler

  • systemctl status truncates. journalctl -u reportd -n 30 --no-pager does not. Start there and read the actual message.
  • The first error mentions a user. Day 03 covered making a service account, including why it should have --system and nologin.
  • The second error is about executing a file. Day 01 covered which bit that needs, and Day 04 covered why a fresh file does not have it.
  • The third error is about a path that does not exist. The script writes to a directory; something has to create it and something has to own it. systemd has a directive for exactly this — look for StateDirectory in man systemd.exec.
  • The reboot fault is the enabled-versus-active distinction from Day 07. A unit with no [Install] section cannot be enabled.

What success looks like

Verify

systemctl is-active reportd; systemctl is-enabled reportd # active # enabled

Verify

sudo tail -n 2 /var/lib/reportd/report.log; stat -c '%U' /var/lib/reportd # two recent report lines, and the directory owned by your service account

Then reboot and check again. If it does not come back, fault four is still there.

Clean up

sudo systemctl disable --now reportd
sudo rm -f /etc/systemd/system/reportd.service
sudo rm -rf /opt/reportd /var/lib/reportd
sudo userdel reportd 2>/dev/null
sudo systemctl daemon-reload

Where this goes next

That is the foundation. The remaining labs are the set that separates someone who can use a Linux box from someone who can be handed a fleet of them — writing real service units, hardening SSH, growing a disk that filled up at 3am, and debugging a network path that works from one host and not another.