Exercise: Run a container as a service

Run a container as a service

• Write a Quadlet file and start the generated service.
• Diagnose a wrong image name and a missing SELinux label.
• Enable lingering and open the firewall so the service works after a reboot.

Start the exercise

On workstation, start the exercise. It installs podman, pulls the web image for student on servera and creates ~/web/index.html with the line hello from a container.

[student@workstation ~]$ lab start sa-containers-service

Later tasks assume that you are logged in to the server they name: ssh student@servera from workstation, then sudo -i for a root shell.

The Quadlet file

Create ~/.config/containers/systemd/web.container that runs the image, publishes 8080, mounts ~/web read-only with the SELinux option on /var/www/html, restarts always, and is wanted by default.target. Reload systemd, start web.service and fetch the page.

[student@servera ~]$ mkdir -p ~/.config/containers/systemd
[student@servera ~]$ cat > ~/.config/containers/systemd/web.container <<'EOT'
[Unit]
Description=Demo web container

[Container]
Image=registry.access.redhat.com/ubi9/httpd-24:latest
PublishPort=8080:8080
Volume=%h/web:/var/www/html:ro,Z

[Service]
Restart=always

[Install]
WantedBy=default.target
EOT
[student@servera ~]$ systemctl --user daemon-reload
[student@servera ~]$ systemctl --user start web.service
[student@servera ~]$ sleep 3; curl -s localhost:8080
hello from a container
[student@servera ~]$ podman ps --format "{{.Names}} {{.Status}}"
systemd-web Up 3 seconds

Look at what was generated

Show the first lines of the generated unit (systemctl --user cat), then ask whether the service is enabled.

[student@servera ~]$ systemctl --user cat web.service | head -4
# /run/user/1000/systemd/generator/web.service
# Automatically generated by /usr/lib/systemd/user-generators/podman-user-generator
#
[Unit]
[student@servera ~]$ systemctl --user is-enabled web.service
generated

The generated file lives in /run, so editing it is pointless: change the .container file and reload.

Kill it

Kill the container with podman kill systemd-web. A few seconds later, is the service active? How often has systemd restarted it?

[student@servera ~]$ podman kill systemd-web
systemd-web
[student@servera ~]$ sleep 5; systemctl --user is-active web.service
active
[student@servera ~]$ systemctl --user show web.service -p NRestarts
NRestarts=1

Fault 1: a wrong image name

Change httpd-24 to httpd-99 in the file, reload and start the service. What does systemctl --user status show? Find the error line in the journal, then repair the file.

[student@servera ~]$ sed -i 's#httpd-24:latest#httpd-99:latest#' ~/.config/containers/systemd/web.container
[student@servera ~]$ systemctl --user daemon-reload; systemctl --user start web.service
Job for web.service failed because the control process exited with error code.
See "systemctl --user status web.service" and "journalctl --user -xeu web.service" for details.
[student@servera ~]$ systemctl --user is-active web.service
activating
[student@servera ~]$ journalctl --no-pager _SYSTEMD_USER_UNIT=web.service | grep -m1 "Error:" | cut -c1-140
Oct 03 20:23:15 servera.lab.example.com web[2998]: Error: unable to copy from source docker://registry.access.redhat.com/ubi9/httpd-99:latest: …
[student@servera ~]$ sed -i 's#httpd-99:latest#httpd-24:latest#' ~/.config/containers/systemd/web.container
[student@servera ~]$ systemctl --user daemon-reload; systemctl --user restart web.service

activating means systemd is waiting to retry (Restart=always). The cause is in the journal: the image does not exist.

Fault 2: the missing label

Remove ,Z from the Volume= line, reload and restart. Fetch the page: what status do you get and why? Put the option back.

[student@servera ~]$ sed -i 's#,Z$##' ~/.config/containers/systemd/web.container
[student@servera ~]$ systemctl --user daemon-reload; systemctl --user restart web.service
[student@servera ~]$ sleep 3; curl -s -o /dev/null -w "%{http_code}\n" localhost:8080
403
[student@servera ~]$ sed -i 's#:ro$#:ro,Z#' ~/.config/containers/systemd/web.container
[student@servera ~]$ systemctl --user daemon-reload; systemctl --user restart web.service
[student@servera ~]$ sleep 3; curl -s localhost:8080
hello from a container

The service is "active" and the container runs, yet the page is forbidden: always test the useful behaviour, not only the unit state.

Let others in and survive a reboot

From serverb the page is not reachable yet. Open the firewall port permanently, and turn on lingering for student. Check it from serverb, reboot servera, wait for it, and check again from serverb without logging in to servera.

[student@serverb ~]$ curl -sS -m 3 http://servera:8080/
curl: (7) Failed to connect to servera port 8080: No route to host
[student@servera ~]$ sudo firewall-cmd --permanent --add-port=8080/tcp; sudo firewall-cmd --reload
success
success
[student@servera ~]$ sudo loginctl enable-linger student
[student@servera ~]$ loginctl show-user student -p Linger
Linger=yes
[student@serverb ~]$ curl -s -m 3 http://servera:8080/
hello from a container
[student@servera ~]$ sudo systemctl reboot
[student@serverb ~]$ sleep 60; curl -s -m 3 http://servera:8080/
hello from a container

Without lingering the same test after the reboot gives "Connection refused" until student logs in for the first time.

Grade and finish

Check your work with lab grade sa-containers-service, then put the exercise away with lab finish sa-containers-service.