Skip to content

OpenShift 4: Query Prometheus API for Firing Alert

Avatar photo

https://www.linkedin.com/in/muhammad-aizuddin-zali-4807b552/

Introduction

Prometheus is a robust, efficient and cloud native monitoring tools, graduated from CNCF incubator. It was part of the OpenShift since OpenShift 3.11. In OpenShift 4.x it become part of the platform core component.

In this quick blog, we will show you how to query OpenShift Prometheus API that already being exposed by route. This is very useful for external monitoring integration solution like Nagios active check that can query Prometheus for firing alert.

How to get this done?

First you need an authorization bearer token. In this blog we will use short-lived user token, however for proper integration, we can use ServiceAccount long-lived token with assigned RBAC.

# export TOKEN=$( oc whoami -t)
# export URL=https://prometheus-k8s-openshift-monitoring.apps.ocp4.local.bytewise.my


#curl -s -g -k -X GET -H "Authorization: Bearer $TOKEN" -H 'Accept: application/json' -H 'Content-Type: application/json' "$URL/api/v1/alerts" | jq  -c '.data.alerts[] | select( .state == "firing" and .labels.severity == "critical")'

OUTPUT:
{"labels":{"alertname":"KubeAPIErrorsHigh","resource":"events","severity":"critical","verb":"LIST"},"annotations":{"message":"API server is returning errors for 45.45% of requests for LIST events ."},"state":"firing","activeAt":"2020-11-04T04:12:20.907806203Z","value":"4.545454545454545e-01"}

For full and official Prometheus API documentation, we may refer to https://prometheus.io/docs/prometheus/latest/querying/api/.

Disclaimer:

The views expressed and the content shared in all published articles on this website are solely those of the respective authors, and they do not necessarily reflect the views of the author’s employer or the techbeatly platform. We strive to ensure the accuracy and validity of the content published on our website. However, we cannot guarantee the absolute correctness or completeness of the information provided. It is the responsibility of the readers and users of this website to verify the accuracy and appropriateness of any information or opinions expressed within the articles. If you come across any content that you believe to be incorrect or invalid, please contact us immediately so that we can address the issue promptly.

Tags:

Avatar photo


https://www.linkedin.com/in/muhammad-aizuddin-zali-4807b552/
Red Hat ASEAN Senior Platform Consultant. Kubernetes, OpenShift and DevSecOps evangelist.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.