This is an utility to monitor the OSM nodes and pods in the Kubernetes deployment. Metrics are stored in Prometheus and accessible in Grafana. Note that those "Prometheus" instance is not the same in the OSM core, but different one, aimed at the monitoring of the platform itself.
OSM must be/have been deployed using the Kubernetes installer (that is, with the -c k8s option).
For reference, the versions for the external components used are as follows:
Kubernetes cluster metrics (for nodes, pods, deployments, etc.) are stored in the dedicated Prometheus instance and accessible using Grafana.
"Prometheus-operator" (https://github.com/helm/charts/tree/master/stable/prometheus-operator) provides the basic components and the monitoring of the basic Kubernetes resources. Additional "exporters" are used to gather metrics from Kafka, Mysql and Mongodb. It is important to note that Grafana is not installed with this chart because we are using Grafana installed with OSM core.
There are two ways to install the monitoring component based on the OSM global installer (https://osm-download.etsi.org/ftp/osm-9-0-nine/install_osm.sh)
./install_osm.sh -c k8s --k8s_monitor
./install_osm.sh -o k8s_monitor
All the components will be installed in the "monitoring" namespace. In addition, for debugging purposes, there is a standalone script is available in devops/installers/k8s/install_osm_k8s_monitoring.sh. To see the available options, type --help.
usage: ./install_osm_k8s_monitoring.sh [OPTIONS] Install OSM Monitoring OPTIONS -n <namespace> : use specified kubernetes namespace - default: monitoring -s <service_type>: service type (ClusterIP|NodePort|LoadBalancer) - default: NodePort --debug : debug script --dump : dump arguments and versions -h / --help : print this help
The Grafana console can be accessed on the IP address of any node using port 3000, since a NodePort service is used: http://<ip_your_osm_host>:3000
The initial credentials are:
Use the uninstall script
./install_osm.sh -o k8s_monitor --uninstall
In addition, for debugging purposes, there is a standalone script is available in devops/installers/k8s/uninstall_osm_k8s_monitoring.sh. To see the available options type --help.
usage: ./uninstall_osm_k8s_monitoring.sh [OPTIONS] Uninstall OSM Monitoring OPTIONS -n <namespace>: use specified kubernetes namespace - default: monitoring --helm : uninstall tiller --debug : debug script -h / --help : print this help
Dashboard are organized in two folders:
The folder "Kubernetes cluster" contains the dashboards available upstream as part of the standard prometheus operator helm installation:
The folder "Open Source MANO" contains additional dashboards customized for OSM:
New dashboards for OSM components should be included in "Open Source MANO" folder. Once we have the dashboard json file, please follow the instructions below to incorporate it into Grafana.
kubectl -n monitoring create configmap <configmap-name> --from-file=<dashboard-json-file> kubectl -n monitoring patch configmap <configmap-name> --patch '{"metadata": {"labels": {"grafana_dashboard": "1"},{"annotations": {k8s-sidecar-target-directory: "/tmp/dashboards/Open Source MANO"}}}}'
where and needs to be replaced with desired values. A proposal is that begins with "osm-monitoring-osm-"
Once configmap is created and patched, we can download the manifest file for future use with next command:
kubectl -n monitoring get configmap <configmap-name> -o yaml > <confimap-file>
Grafana Sidecar will read the label grafana_dashboard: "1" in the configmap and upload the dashboard information to Grafana.
The current dashboards can also be updated. It is only needed to modify/update the required yaml file available in devops/installers/k8s and apply them via kubectl. As an example kubectl -n monitoring apply -f summary-dashboard.yaml will update the changes made in the summary dashboard.