PowOutMon

PowOutMon (POWer OUTage MONitor) integrates with an existing NUT server to gracefully shut down machines when power is lost and automatically wake them when power is restored. It is intended for use with the Unifi UPS 2U, since it has an integrated NUT server, but incomplete features to control individual outlets.

AI usage notice: This project was written with the help of an LLM, but it was not vibe-coded. All AI-generated changes were scrutinized, tested, and approved by me manually. Still, this is a hobby project and may contain errors. Use at your own risk.

Prerequisites

• Install NUT client (sudo apt install nut-client).
• Install Wake On Lan utility (sudo apt install wakeonlan or sudo dnf install wol).
• Ensure that the battery-powered machine(s) has "restore on AC power loss" disabled in its BIOS.
• Ensure WoL (Wake on LAN) is enabled on target machines.

Canary (UPS Monitor) Setup

canary.sh is the UPS monitoring and shutdown utility. It monitors the battery status and initiates a graceful system shutdown if the battery runtime is critically low or if the UPS becomes unreachable. This script should be run on any machine connected to the UPS.

1. Configuration

It is recommended to place the script and its configuration file in /usr/local/bin. Create a .env.canary file there using .env.canary.example as a template:

sudo cp canary.sh /usr/local/bin/
sudo cp .env.canary.example /usr/local/bin/.env.canary

The following variables can be customized in .env.canary:

• UPS_ID: Your NUT UPS identifier (e.g., ups@localhost).
• RUNTIME_LIMIT: Minimum battery runtime (in seconds) before shutdown (default: 300).
• FAIL_LIMIT: Number of consecutive connection failures before shutdown (default: 15).
• COUNTER_FILE: Path to the file for tracking failures (default: /var/tmp/ups_down_count).
• PUSHOVER_USER: Your Pushover user key.
• PUSHOVER_TOKEN: Your Pushover app token.

2. Cron Job

Create a cron job as root (via sudo crontab -e) to run the script at a regular interval. For example, to run every minute:

* * * * * /usr/local/bin/canary.sh /usr/local/bin/.env.canary

NOTE: The cron job must be run as root. This ensures the script has proper permission to shut down the machine without manual intervention.

Watcher (Recovery) Setup

watcher.sh is the recovery utility. It monitors the UPS battery runtime and automatically wakes target machines using Wake-on-LAN (WoL) once sufficient battery charge is available. It is recommended to set a higher RUNTIME_LIMIT here than in canary.sh to ensure the battery has enough charge to handle another outage immediately.

1. Configuration

It is recommended to place the script and its configuration file in /usr/local/bin. Create a .env.watcher file there using .env.watcher.example as a template:

sudo cp watcher.sh /usr/local/bin/
sudo cp .env.watcher.example /usr/local/bin/.env.watcher

The following variables can be customized in .env.watcher:

• UPS_ID: Your NUT UPS identifier.
• TARGETS: List of target machines (IP and MAC) to wake.
• RUNTIME_LIMIT: Minimum battery runtime (in seconds) before waking machines (default: 360).
• STATE_FILE: Path to the state file for tracking notifications (default: /var/tmp/watcher_ups_state).
• PUSHOVER_USER: Your Pushover user key.
• PUSHOVER_TOKEN: Your Pushover app token.

2. Target Machines Configuration

The target machines are configured directly in the .env.watcher file using the TARGETS variable. This should be a multi-line string where each line contains an IP address and a MAC address separated by a space:

TARGETS="
192.168.1.10 FF:FF:FF:FF:FF:FF
192.168.1.11 EE:EE:EE:EE:EE:EE
"

3. Cron Job

Create a cron job (via crontab -e) to run the script at a regular interval. The following example checks every 5 minutes:

*/5 * * * * /usr/local/bin/watcher.sh /usr/local/bin/.env.watcher

Viewing Logs

Both scripts log their activity to the system log using the tags UPS_MONITOR and UPS_SHUTDOWN. You can follow the output in real-time using journalctl:

journalctl -f -t UPS_SHUTDOWN -t UPS_MONITOR

Roadmap/Ideas

• Add an option to specify a "shutdown script". This could allow for the graceful shutdown of Docker or another sensitive program before forcing a system shutdown to help avoid corrupt data.
• Add a "canary" script to monitor UPS status.
• Add a "watcher" script to send WoL requests when power is restored.