Getting Event Notifications in Pacemaker by Using Alert Agents

This article will help you get started with Pacemaker alert agents.

As an example to get you a little familiar with how the alert agents work, you can use the script below in ‘/usr/local/bin/notification-agent-example.sh’:

#!/bin/bash

### list of variables passed from Pacemaker to all alert agents:
# CRM_notify_recipient the configure recipient
# CRM_notify_node name of affected node
# CRM_notify_desc details about event
# CRM_notify_task requested fencing/resource opertion (fencing and resource notify only)
# CRM_notify_rc return code from operation (fencing and resource notify only)
# CRM_notify_rsc name of the affected resource (resource notify only)
# CRM_notify_target_rc expected return code of an operation (resource notify only)
# CRM_notify_status numerical code used by Pacemaker to represent operation result (resource notify only)

### some variables won't get values; set NA if not populated:
CRM_notify_recipient=${CRM_notify_recipient:-NA}
CRM_notify_node=${CRM_notify_node:-NA}
CRM_notify_desc=${CRM_notify_desc:-NA}
CRM_notify_task=${CRM_notify_task:-NA}
CRM_notify_rc=${CRM_notify_rc:-NA}
CRM_notify_rsc=${CRM_notify_rsc:-NA}
CRM_notify_target_rc=${CRM_notify_target_rc:-NA}
CRM_notify_status=${CRM_notify_status:-NA}

### do something with these values
logger PCMK-NOTIFY: recipient: $CRM_notify_recipient
logger PCMK-NOTIFY: affected node: $CRM_notify_node
logger PCMK-NOTIFY: event details: $CRM_notify_desc
logger PCMK-NOTIFY: requested op: $CRM_notify_task
logger PCMK-NOTIFY: op ret code: $CRM_notify_rc
logger PCMK-NOTIFY: affected res: $CRM_notify_rsc
logger PCMK-NOTIFY: expected rc: $CRM_notify_target_rc
logger PCMK-NOTIFY: pcmk result: $CRM_notify_status

### exit
exit 0

NOTE: You can get some more information regarding external-agents and the variables that are populated here: https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/#_alert_agents

Once your script is in place on both nodes, you can configure the following in your Pacemaker configuration:

primitive cluster-notifications ocf:pacemaker:ClusterMon \
  params user=root update=30 \
  extra_options=--watch-fencing -E /usr/local/bin/notification-agent-example.sh
clone cl-cluster-notifications cluster-notifications

After that, your script will be called for each resource operation that happens in the cluster; this includes all resource starts/stops (so anytime a failover happens many alerts will be generated), when a resource monitor operation is first started, and anytime a monitor operation fails.

If the –watch-fencing flag is in your cluster-notifications Pacemaker resource, Pacemaker will generate an alert when fencing is started and when monitoring is started for that node, which would imply that a member just joined the cluster.

Reviewed 2020/12/03 - DGT