Production operations

Install and start

# Install the compiled binary and a reviewed configuration.
sudo install -m 0755 target/release/omlb /usr/local/bin/omlb
sudo install -d -o omlb -g omlb -m 0750 /etc/omlb /var/lib/omlb
sudo install -o root -g omlb -m 0640 omlb.yaml /etc/omlb/omlb.yaml

# Validate with the same service account and paths used in production.
sudo -u omlb /usr/local/bin/omlb --check --config /etc/omlb/omlb.yaml

# Review and install the repository's unit, then start it.
sudo install -m 0644 examples/systemd/omlb.service /etc/systemd/system/omlb.service
sudo systemctl daemon-reload
sudo systemctl enable --now omlb

The repository ships a systemd unit under examples/systemd/. Review it for your host before installation. Its hardening and capabilities may need adjustment for certificate locations, plugins, or HA. Nodes without HA should not retain HA-specific network capabilities.

Change management

# Hot route/plugin/retry/timeout change: stage a complete candidate
# beside the live file, validate it, then atomically rename it into place.
sudo install -o root -g omlb -m 0640 ./omlb.yaml.next /etc/omlb/omlb.yaml.next
sudo -u omlb /usr/local/bin/omlb --check --config /etc/omlb/omlb.yaml.next
sudo mv -f /etc/omlb/omlb.yaml.next /etc/omlb/omlb.yaml

# Bootstrap-only change (listeners, pools/backends, TLS, L4, HA, endpoints,
# security, server or log settings): use a graceful service restart instead.
sudo systemctl restart omlb

The built-in watcher detects ordinary and atomic file changes. A manual POST /reload to the local admin API is also available, but it cannot bypass the reload boundary. Failed parses, validation failures, and bootstrap-only changes are rejected and leave the running configuration in place.

Keep the prior validated file and the prior binary available. For a TLS change, verify certificate and key ownership on the host. For an HA change, schedule and test it as a coordinated two-node change; do not expect a file watcher to reconfigure VIP behavior live.

Health, alerting, and incidents

# Is OMLB up and is its control plane reachable locally?
systemctl status omlb
curl -fsS http://127.0.0.1:9090/healthz
curl -sS http://127.0.0.1:9090/status
curl -sS http://127.0.0.1:9091/metrics

# Follow structured journal records when log_format: journald is configured.
journalctl -u omlb -f
journalctl -u omlb -p warning..alert

Alert on process availability, listener reachability, upstream error and ejection activity, configuration reload failures, and the rate of 5xx responses. During a backend incident, drain one known backend through the local admin API, observe new traffic move away, then perform maintenance. Do not use a restart to hide a configuration validation failure—read the logged validation errors and correct the candidate file.

Backup and recovery

• Back up the reviewed base YAML, Web UI overlay if used, ACME cache directory, static certificates/keys, token files, plugin artifacts, and the exact binary version.
• Keep secret backups encrypted and restore owner and mode before starting the service.
• Practice restoring a node with the public listener disabled first; validate the config, check control-plane endpoints locally, then admit client traffic.
• Document the client-visible DNS and VIP recovery path separately from OMLB; the proxy cannot repair a host, L2, DNS, or certificate authority outage.