Chapter 22: Review and practice

Diagnosing faults

A broken service is a puzzle with a fixed set of pieces: process, listener, network path, permissions, labels, content. The skill is not knowing every command but using the same method each time, so that the evidence, and not a guess, picks the fix.

The method

One method for every fault: Step through a diagnosis.

State the failure. "serverb gets 403 for http://servera:8090/ since the restore" is a fact you can test again. "The portal is broken" is not.

Gather a narrow baseline before you change anything: the service and its journal, the listener, a request from servera itself, a request from the caller, the zone rules, and the owner, mode, ACL and label of the content. Two requests, one local and one remote, already split the problem in half: if the local one works, the daemon and the content are fine and you look at the network.

Change one thing, then repeat the exact request that failed. Afterwards verify that the other requirements still hold: a fix that opens the firewall to everyone is a new fault.

From symptom to layer

Symptom to layer: Select what the client sees.

A baseline in practice

A request fails with 403. The service is up and the port is open. The baseline points at the content:

The label var_t is wrong for web content; restorecon applied the stored mapping, and the same request now succeeds. If the label had been right, the next suspect would have been the mode or the ACL (getfacl), then the path (namei -l).

A useful table to keep in mind:

SymptomFirst checks
Connection refusedsystemctl status, ss -tlnp, journalctl -u
No route to hostfirewall-cmd --get-active-zones, --list-all --zone=…
Timeoutdrop zone or source, routing, the address in the URL
403ls -lZ, getfacl, namei -l, getenforce
200 but wrong contentthe document root, the backup, timestamps
Will not startthe journal: port label, syntax, port in use

Check your understanding