The Prometheus Operator uses Kubernetes custom resources to simplify the deployment and configuration of Prometheus, Alertmanager, and related monitoring components. The default installation is intended to suit monitoring a Kubernetes cluster the chart is deployed onto.
- 30.2.0
A Default integration will install the following components:
- prometheus-operator.
- prometheus.
- alertmanager.
- node-exporter.
- kube-state-metrics.
- grafana.
- service monitors to scrape internal Kubernetes components:
- kube-apiserver.
- kube-scheduler.
- kube-controller-manager.
- etcd.
- kube-dns/coredns.
- kube-proxy.
This integration also includes dashboards and alerts.
For Alertmanager to work fine, alerting config should be set while deploying the integration. This config also includes sensitive information like SMTP credentials. If you want to skip entering SMTP credentials in plain text, you can do so by following the procedure below
Add Prometheus-Operator integration in the Cluster profile with changes to the following values
alertmanager.alertmanagerSpec.useExistingSecret
- Set to true. This will skip creating the default Alertmanager secretalertmanager.alertmanagerSpec.configSecret
- Provide the name of the custom Kubernetes Secret (should exist in the same namespace as the Alertmanager object) which contains configuration for the Alertmanager instance
Deploy the cluster with Prometheus-Operator integration. At this point, Alertmanager pod will crash (as the custom secret defined is not yet available)
Create the secret manually in the same namespace as that of Alertmanager object
Prepare alertmanager.yaml contents
global:resolve_timeout: 5mreceivers:- email_configs:- auth_identity: noreply@spectrocloud.comauth_password: abcdauth_username: noreply@spectrocloud.comfrom: noreply@spectrocloud.comsend_resolved: truesmarthost: smtp.gmail.com:587to: mark@spectrocloud.comname: prom-alertroute:group_by:- jobgroup_interval: 5mgroup_wait: 30sreceiver: prom-alertrepeat_interval: 4hroutes:- match:alertname: Watchdogreceiver: prom-alertCreate the Secret using the Alertmanager config prepared above
kubectl create secret generic alertmanager-secret -n monitoring --from-file=./alertmanager.yamlAlertmanager pod will recover from the crash in the next reconciliation
Follow below steps to configure Ingress on Grafana
- Change serviceType from "LoadBalancer" to "ClusterIP" (line #427)
- Ingress (line #438)
- Enable Ingress; change enabled from false to "true"
- Set Ingress rules like annotations, path, hosts, etc.
With these changes, you can access Grafana service on the Ingress Controller LoadBalancer hostname / IP