Service Watchdog and Automatic Restart (systemd WatchdogSec/Restart)
Description
Configures systemd service units with watchdog monitoring (WatchdogSec=) and automatic restart policies (Restart=on-failure or Restart=always, RestartSec=) to recover services that become unresponsive or are killed due to resource exhaustion. When a memory leak causes the OOM killer to terminate a service or the process hangs due to resource starvation, systemd automatically restarts the service from a clean state. The watchdog mechanism requires the service to periodically notify systemd (via sd_notify WATCHDOG=1); failure to notify within the WatchdogSec interval triggers a restart. Combined with cgroup memory limits (CME-704), this provides both containment and recovery — the cgroup bounds the leak, and systemd restarts the service when the bound is hit.
CVSS Vector Impacts
| Metric | Transition | Rationale |
|---|---|---|
| Availability (A) | H → L | Service recovers automatically from resource exhaustion conditions via systemd restart policy; availability impact is bounded by the restart interval (typically seconds) rather than mean-time-to-manual-recovery (minutes to hours). The watchdog timer detects service hangs caused by resource starvation even when the process has not been OOM-killed. |
CWE Relationships
Verification
Check that critical services have restart policies and watchdog timers configured
# Expected: Restart=on-failure (or always); WatchdogSec set to a positive value
# Expected: Critical services have Restart= and WatchdogSec= directives