Application Real-Time Monitoring Service:Customize notification templates

Email

Alert name: {{ .commonLabels.alertname }}
{{if .commonLabels.clustername }}
Cluster name: {{ .commonLabels.clustername }}
{{ end }}
{{if eq "app" .commonLabels._aliyun_arms_involvedObject_kind }}
Application name: {{ .commonLabels._aliyun_arms_involvedObject_name }}
{{ end }}
Notification policy: {{ .dispatchRuleName }}
Alert triggered at: {{ .startTime }}
Alert content: {{ if .newIncidentIn }} (New event) {{ end }} {{ for .alerts }} {{.annotations.message}} {{ if .generatorURL }} <a href="{{.generatorURL}}" >Details URL</a> {{end}} {{end}}

The following table explains each section of the template.

Sample output:

Alert name: The memory usage of the container has exceeded 80%.
Cluster name: klyz1688-kubernetes-1
Notification policy: Notification policy name
Alert triggered at: 2023-02-22 07:18:15
Alert content: Namespace: arms-prom / Pod: kube-state-metrics-ccb59dbff-jljg4 / Container: kube-state-metrics. The memory usage of the container has exceeded 80%. Current value: 15.52%.

SMS and phone call

SMS and phone call templates share the same structure as the email template but omit the details URL.

{{ .level }} alert occurred
Alert name: {{ .commonLabels.alertname }}
{{if .commonLabels.clustername }}
Cluster name: {{ .commonLabels.clustername }}
{{ end }}
{{if eq "app" .commonLabels._aliyun_arms_involvedObject_kind }}
Application name: {{ .commonLabels._aliyun_arms_involvedObject_name }}
{{ end }}
Notification policy: {{ .dispatchRuleName }}
Alert triggered at: {{ .startTime }}
Alert content: {{ if .newIncidentIn }} (New event)
 {{end}} {{ for .alerts }} {{ .annotations.message }} {{ end }}

Chatbot (DingTalk, Lark, WeCom)

The chatbot template renders Markdown output. Formatting support varies by platform.

{{ if .newIncidentIn }} (New event) {{end}}
{{if .commonLabels.clustername }}
 >  Cluster name: {{ .commonLabels.clustername }}
 {{ end }}
{{if eq "app" .commonLabels._aliyun_arms_involvedObject_kind }}
 >  Application name: {{ .commonLabels._aliyun_arms_involvedObject_name }}
 {{ end }}
{{ for .alerts }} >  {{ .annotations.message }} {{if .generatorURL }} [Details URL]({{.generatorURL}})  {{end}}{{if .annotations._aliyun_arms_insights_analyze_link }}[<font color='#ff0000'>Root cause</font>]({{ .annotations._aliyun_arms_insights_analyze_link}}){{ end }}{{if  eq "1" .labels._aliyun_arms_denoise_code }} (Important: {{.labels._aliyun_arms_important_reason }}) {{end}}

{{end}}

Chatbot Markdown support:

Webhook

{
  "Alert name":"{{ .commonLabels.alertname }}
{{if .commonLabels.clustername }}", "Cluster name":"{{ .commonLabels.clustername }} {{ end }}
{{if eq "app" .commonLabels._aliyun_arms_involvedObject_kind }}", "Application name":"{{ .commonLabels._aliyun_arms_involvedObject_name }} {{ end }}",
  "Notification Policy":"{{ .dispatchRuleName }}",
  "Alert triggered at":"{{ .startTime }}",
  "Alert content":"{{ for .alerts }} {{ .annotations.message }} {{ end }}"
}

Sample output:

{
  "Alert name": "The memory usage of the container has exceeded 80%.",
  "Cluster name": "klyz1688-kubernetes-1",
  "Notification policy": "Notification policy name",
  "Alert triggered at": "2023-02-22 07:18:15",
  "Alert content": "Namespace: arms-prom / Pod: kube-state-metrics-ccb59dbff-jljg4 / Container: kube-state-metrics. The memory usage of the container has exceeded 80%. Current value: 15.52%."
}

Dot notation

The dot (.) represents the current context scope.

Access the entire context:

{{ . }}

Output: Returns all data in the alert context.

Access a top-level field:

{{ .level }}

Output:

error

Access a nested field:

{{ .commonLabels.alertname }}

Output:

The memory usage of the container has exceeded 80%.

Variables

Store context data in variables for reuse.

Assign a field to a variable:

{{ $alertname := .commonLabels.alertname }} {{$alertname}}

Output:

The memory usage of the container has exceeded 80%.

Access an array element through a variable:

{{ $alert0 := index .alerts 0 }} {{$alert0.labels.alertname}}

Output:

The memory usage of the container has exceeded 80%.

Conditionals: if/else

Check values with if. The condition evaluates to false for null values, 0, nil pointers, empty strings, and zero-length strings.

Simple condition:

{{if eq "warning" .commonLabels.severity }} P3 alert {{ end }}

Output:

P3 alert

if/else:

{{if eq "critical" .commonLabels.severity }} P1 alert {{ else }} P2 alert {{ end }}

Output:

P2 alert

Loops: for

Iterate through a list.

Traverse alerts and get each alert name:

{{ for .alerts}} {{.labels.alertname}} \n {{end}}

Output:

The memory usage of the container has exceeded 80%.

Loops: range

Traverse a sequence of values.

Iterate through integers 0 to 9:

{{ $ran := range 0 10 }} {{ for $ran }} {{.}} {{end}}

Output:

 0 1 2 3 4 5 6 7 8 9

The index function

Retrieve a specific element from an array by index.

Get the first alert event:

{{ $alert0 := index .alerts 0 }} {{$alert0.labels.alertname}}

Output:

 The memory usage of the container has exceeded 80%.

Comparison functions

Example:

{{if eq "warning" .commonLabels.severity }} P3 alert {{ end }}

Output:

  P3 alert

Logical operators: and, or, not

Combine conditions with logical operators.

Example:

{{if eq "warning" .commonLabels.severity or eq "warning" .commonLabels.severity }} P3 alert {{ end }}

Output:

  P3 alert

Built-in functions

{{ $alert0 := index .alerts 0 }} {{ printf "%.2f" $alert0.annotations.value }}

  15.52

{{ "0.95332123123124" | humanizePercentage }}

95.33%

{{ "1671074058234" | humanizeDate }}

2022-12-15 11:14:18

{{ len .alerts }}

1