2024-04-15T01:23:03.083862288Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:07.091579742Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:11.102126527Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:15.102648678Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:19.106831502Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:23.110138605Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:27.114090832Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:31.119768667Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:35.130157743Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:39.134334124Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:43.147387368Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:47.152286995Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:51.158729909Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:55.163637299Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:23:59.168798255Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:03.171884219Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:07.179094654Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:11.182156052Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:15.186198030Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:19.193995203Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:23.197547365Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:27.201169803Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:31.214127734Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:35.218102413Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:39.231783188Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:24:43.236517298Z nc: kafka (10.99.9.24:9092): Operation timed out 2024-04-15T01:25:16.346773642Z kafka (10.99.9.24:9092) open 2024-04-15T01:25:16.350466726Z mongodb-k8s (10.244.0.42:27017) open 2024-04-15T01:26:19.764221114Z 04/15/2024 01:26:19 AM - kafka.conn - INFO - [IPv4 ('10.99.9.24', 9092)]>: connecting to kafka:9092 [('10.99.9.24', 9092) IPv4] 2024-04-15T01:26:19.764457008Z 04/15/2024 01:26:19 AM - kafka.conn - INFO - Probing node bootstrap-0 broker version 2024-04-15T01:26:19.765639203Z 04/15/2024 01:26:19 AM - kafka.conn - INFO - [IPv4 ('10.99.9.24', 9092)]>: Connection complete. 2024-04-15T01:26:19.959382617Z 04/15/2024 01:26:19 AM - kafka.conn - INFO - Broker version identified as 2.5.0 2024-04-15T01:26:19.959569623Z 04/15/2024 01:26:19 AM - kafka.conn - INFO - Set configuration api_version=(2, 5, 0) to skip auto check_version requests on startup 2024-04-15T01:26:20.001869405Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - [IPv4 ('10.99.9.24', 9092)]>: connecting to kafka:9092 [('10.99.9.24', 9092) IPv4] 2024-04-15T01:26:20.008314178Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - Probing node bootstrap-0 broker version 2024-04-15T01:26:20.008324192Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - [IPv4 ('10.99.9.24', 9092)]>: Connection complete. 2024-04-15T01:26:20.113322026Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - Broker version identified as 2.5.0 2024-04-15T01:26:20.120287192Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - Set configuration api_version=(2, 5, 0) to skip auto check_version requests on startup 2024-04-15T01:26:20.301853985Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - [IPv4 ('10.244.0.32', 9092)]>: connecting to kafka-controller-1.kafka-controller-headless.osm.svc.cluster.local:9092 [('10.244.0.32', 9092) IPv4] 2024-04-15T01:26:20.301870952Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - [IPv4 ('10.244.0.32', 9092)]>: Connection complete. 2024-04-15T01:26:20.301875153Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - [IPv4 ('10.99.9.24', 9092)]>: Closing connection. 2024-04-15T01:26:20.677884619Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - [IPv4 ('10.244.0.45', 9092)]>: connecting to kafka-controller-0.kafka-controller-headless.osm.svc.cluster.local:9092 [('10.244.0.45', 9092) IPv4] 2024-04-15T01:26:20.677897577Z 04/15/2024 01:26:20 AM - kafka.conn - INFO - [IPv4 ('10.244.0.45', 9092)]>: Connection complete. 2024-04-15T01:26:20.796954409Z 04/15/2024 01:26:20 AM - kafka.cluster - ERROR - Topic users not found in cluster metadata 2024-04-15T01:26:20.897697417Z 04/15/2024 01:26:20 AM - kafka.cluster - ERROR - Topic users not found in cluster metadata 2024-04-15T01:26:21.005611811Z 04/15/2024 01:26:21 AM - kafka.cluster - ERROR - Topic users not found in cluster metadata 2024-04-15T01:26:21.119523267Z 04/15/2024 01:26:21 AM - kafka.cluster - ERROR - Topic users not found in cluster metadata 2024-04-15T01:26:21.334344906Z 04/15/2024 01:26:21 AM - kafka.cluster - ERROR - Topic project not found in cluster metadata 2024-04-15T01:26:21.440314630Z 04/15/2024 01:26:21 AM - kafka.cluster - ERROR - Topic project not found in cluster metadata 2024-04-15T01:26:21.545991591Z 04/15/2024 01:26:21 AM - kafka.cluster - ERROR - Topic project not found in cluster metadata 2024-04-15T01:26:21.658315736Z 04/15/2024 01:26:21 AM - kafka.conn - INFO - [IPv4 ('10.244.0.34', 9092)]>: connecting to kafka-controller-2.kafka-controller-headless.osm.svc.cluster.local:9092 [('10.244.0.34', 9092) IPv4] 2024-04-15T01:26:21.658346023Z 04/15/2024 01:26:21 AM - kafka.conn - INFO - [IPv4 ('10.244.0.34', 9092)]>: Connection complete. 2024-04-15T01:26:21.765669141Z 04/15/2024 01:26:21 AM - kafka.conn - INFO - [IPv4 ('10.244.0.32', 9092)]>: Closing connection. 2024-04-15T01:26:21.765688538Z 04/15/2024 01:26:21 AM - kafka.conn - INFO - [IPv4 ('10.244.0.45', 9092)]>: Closing connection. 2024-04-15T01:26:21.765700550Z 04/15/2024 01:26:21 AM - kafka.conn - INFO - [IPv4 ('10.244.0.34', 9092)]>: Closing connection. 2024-04-15T01:26:21.767348909Z 04/15/2024 01:26:21 AM - osm_mon.cmd.mon_dashboarder - INFO - Starting MON Dashboarder... 2024-04-15T01:26:21.970795808Z 04/15/2024 01:26:21 AM - db - INFO - Connected to database osm version None 2024-04-15T01:26:21.970807705Z 04/15/2024 01:26:21 AM - osm_mon.cmd.mon_dashboarder - INFO - Starting MON kafka Consumer... 2024-04-15T01:26:21.970810959Z 04/15/2024 01:26:21 AM - osm_mon.cmd.mon_dashboarder - INFO - Starting MON Dashboarder... 2024-04-15T01:26:22.044891127Z 04/15/2024 01:26:22 AM - aiokafka.consumer.subscription_state - INFO - Updating subscribed topics to: frozenset({'project', 'users'}) 2024-04-15T01:26:22.044912202Z 04/15/2024 01:26:22 AM - aiokafka.consumer.consumer - INFO - Subscribed to topic(s): {'project', 'users'} 2024-04-15T01:26:22.066085873Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - ERROR - Group Coordinator Request failed: [Error 15] CoordinatorNotAvailableError 2024-04-15T01:26:22.236329707Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:22.236362776Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - INFO - Revoking previously assigned partitions set() for group mon-consumer 2024-04-15T01:26:22.236386068Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:22.319033131Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:22.554072375Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - ERROR - Group Coordinator Request failed: [Error 15] CoordinatorNotAvailableError 2024-04-15T01:26:22.802613522Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:22.802625624Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:22.804296980Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:23.002372287Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:23.002380090Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:23.002383275Z 04/15/2024 01:26:22 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:23.106437367Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:23.106451398Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:23.143920582Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:23.258600287Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:23.258617649Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:23.260268545Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:23.388320235Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:23.395273126Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:23.403803556Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:23.626741080Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:23.626757924Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:23.646113478Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:23.795904239Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:23.795924638Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:23.799632388Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:23.957667808Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:23.957685262Z 04/15/2024 01:26:23 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:24.018778037Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:24.135516100Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:24.145306809Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:24.150025022Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:24.258349326Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:24.258502445Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:24.373732938Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:24.484438420Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:24.484454024Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:24.484477309Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:24.597612328Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:24.600055354Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:24.600062737Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:24.727589962Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:24.727636407Z 04/15/2024 01:26:24 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:25.044253097Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:25.407740163Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:25.407755760Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:25.416778049Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:25.622229396Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:25.622238263Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:25.713007483Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:25.819992695Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:25.824337563Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:25.825886628Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:25.934145269Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:25.946027683Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:25.953695098Z 04/15/2024 01:26:25 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:26.079842954Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:26.079860515Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:26.085844961Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:26.200209400Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:26.200232703Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:26.209597060Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:26.325194799Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:26.325225703Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:26.346003564Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:26.452119753Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:26.509286060Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:26.509297524Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:26.767102869Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:26.767350282Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:26.786577337Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 2)for group mon-consumer. 2024-04-15T01:26:26.943552341Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 2 for group mon-consumer 2024-04-15T01:26:26.943563667Z 04/15/2024 01:26:26 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-15T01:26:30.777223431Z 04/15/2024 01:26:30 AM - aiokafka.consumer.group_coordinator - INFO - Joined group 'mon-consumer' (generation 1) with member_id aiokafka-0.8.1-588da34c-d7a8-4a50-b49d-63db1982d5d9 2024-04-15T01:26:30.777239894Z 04/15/2024 01:26:30 AM - aiokafka.consumer.group_coordinator - INFO - Elected group leader -- performing partition assignments using roundrobin 2024-04-15T01:26:31.697619892Z 04/15/2024 01:26:31 AM - aiokafka.consumer.group_coordinator - INFO - Successfully synced group mon-consumer with generation 1 2024-04-15T01:26:31.697642556Z 04/15/2024 01:26:31 AM - aiokafka.consumer.group_coordinator - INFO - Setting newly assigned partitions {TopicPartition(topic='project', partition=0), TopicPartition(topic='users', partition=0)} for group mon-consumer 2024-04-15T01:26:52.682262720Z 04/15/2024 01:26:52 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-15T01:32:53.965295357Z 04/15/2024 01:32:53 AM - osm_mon.dashboarder.backends.grafana - INFO - Datasource prom-operator-osm is created in Grafana 2024-04-15T01:32:53.965999558Z 04/15/2024 01:32:53 AM - osm_mon.dashboarder.backends.grafana - INFO - ************* response: {'_content': b'{"datasource":{"id":3,"uid":"e5bf77cc-4e17-4c8e-9761-fec318110cfd","orgId":1,"name":"prom-operator-osm","type":"prometheus","typeLogoUrl":"public/app/plugins/datasource/prometheus/img/prometheus_logo.svg","access":"proxy","url":"http://172.21.248.125:9090","user":"","database":"","basicAuth":false,"basicAuthUser":"","withCredentials":false,"isDefault":false,"jsonData":{},"secureJsonFields":{},"version":1,"readOnly":false},"id":3,"message":"Datasource added","name":"prom-operator-osm"}', '_content_consumed': True, '_next': None, 'status_code': 200, 'headers': {'Cache-Control': 'no-store', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-Xss-Protection': '1; mode=block', 'Date': 'Mon, 15 Apr 2024 01:32:53 GMT', 'Content-Length': '489'}, 'raw': , 'url': 'http://grafana:3000/api/datasources', 'encoding': 'utf-8', 'history': [], 'reason': 'OK', 'cookies': , 'elapsed': datetime.timedelta(microseconds=51287), 'request': , 'connection': } 2024-04-15T01:32:54.146338115Z 04/15/2024 01:32:54 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard osm is created in Grafana 2024-04-15T01:38:55.571927215Z 04/15/2024 01:38:55 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-15T01:40:57.940850182Z 04/15/2024 01:40:57 AM - osm_mon.dashboarder.backends.grafana - INFO - New user basic_15_test_user created in Grafana 2024-04-15T01:41:01.279912045Z 04/15/2024 01:41:01 AM - osm_mon.dashboarder.backends.grafana - INFO - User basic_15_test_user is assigned Admin permission 2024-04-15T01:41:06.070805645Z 04/15/2024 01:41:06 AM - osm_mon.dashboarder.dashboarder - ERROR - Exception processing message: 2024-04-15T01:41:06.070823061Z Traceback (most recent call last): 2024-04-15T01:41:06.070826734Z File "/usr/lib/python3/dist-packages/osm_mon/dashboarder/dashboarder.py", line 116, in _user_msg 2024-04-15T01:41:06.070830374Z project_data = values["changes"]["add_project_role_mappings"] 2024-04-15T01:41:06.070833383Z KeyError: 'add_project_role_mappings' 2024-04-15T01:41:07.331001331Z 04/15/2024 01:41:07 AM - osm_mon.dashboarder.backends.grafana - INFO - New team basic_15_test_project created in Grafana 2024-04-15T01:41:08.990815088Z 04/15/2024 01:41:08 AM - osm_mon.dashboarder.backends.grafana - INFO - Grafana team updated {"message":"Team updated"} 2024-04-15T01:41:17.104718129Z 04/15/2024 01:41:17 AM - osm_mon.dashboarder.backends.grafana - INFO - User basic_15_test_user deleted in Grafana 2024-04-15T01:41:18.911704604Z 04/15/2024 01:41:18 AM - osm_mon.dashboarder.backends.grafana - INFO - Team basic_15_project_test deleted in Grafana 2024-04-15T01:41:26.406433906Z 04/15/2024 01:41:26 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-15T02:52:11.891050770Z 04/15/2024 02:52:11 AM - osm_mon.dashboarder.backends.grafana - INFO - Datasource prom-operator-k8s02 is created in Grafana 2024-04-15T02:52:11.891677182Z 04/15/2024 02:52:11 AM - osm_mon.dashboarder.backends.grafana - INFO - ************* response: {'_content': b'{"datasource":{"id":4,"uid":"b7280caf-8e8c-4325-bbd5-51c5777c2fff","orgId":1,"name":"prom-operator-k8s02","type":"prometheus","typeLogoUrl":"public/app/plugins/datasource/prometheus/img/prometheus_logo.svg","access":"proxy","url":"http://172.21.248.125:9090","user":"","database":"","basicAuth":false,"basicAuthUser":"","withCredentials":false,"isDefault":false,"jsonData":{},"secureJsonFields":{},"version":1,"readOnly":false},"id":4,"message":"Datasource added","name":"prom-operator-k8s02"}', '_content_consumed': True, '_next': None, 'status_code': 200, 'headers': {'Cache-Control': 'no-store', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-Xss-Protection': '1; mode=block', 'Date': 'Mon, 15 Apr 2024 02:52:11 GMT', 'Content-Length': '493'}, 'raw': , 'url': 'http://grafana:3000/api/datasources', 'encoding': 'utf-8', 'history': [], 'reason': 'OK', 'cookies': , 'elapsed': datetime.timedelta(microseconds=45938), 'request': , 'connection': } 2024-04-15T02:52:12.062634527Z 04/15/2024 02:52:12 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard k8s02 is created in Grafana 2024-04-15T02:56:13.047992426Z 04/15/2024 02:56:13 AM - osm_mon.dashboarder.backends.grafana - INFO - Datasource prom-operator-k8s07 is created in Grafana 2024-04-15T02:56:13.048034244Z 04/15/2024 02:56:13 AM - osm_mon.dashboarder.backends.grafana - INFO - ************* response: {'_content': b'{"datasource":{"id":5,"uid":"f9bc2861-98df-4d47-8eec-f54c2c5e8333","orgId":1,"name":"prom-operator-k8s07","type":"prometheus","typeLogoUrl":"public/app/plugins/datasource/prometheus/img/prometheus_logo.svg","access":"proxy","url":"http://172.21.248.125:9090","user":"","database":"","basicAuth":false,"basicAuthUser":"","withCredentials":false,"isDefault":false,"jsonData":{},"secureJsonFields":{},"version":1,"readOnly":false},"id":5,"message":"Datasource added","name":"prom-operator-k8s07"}', '_content_consumed': True, '_next': None, 'status_code': 200, 'headers': {'Cache-Control': 'no-store', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-Xss-Protection': '1; mode=block', 'Date': 'Mon, 15 Apr 2024 02:56:13 GMT', 'Content-Length': '493'}, 'raw': , 'url': 'http://grafana:3000/api/datasources', 'encoding': 'utf-8', 'history': [], 'reason': 'OK', 'cookies': , 'elapsed': datetime.timedelta(microseconds=50327), 'request': , 'connection': } 2024-04-15T02:56:13.244979266Z 04/15/2024 02:56:13 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard k8s07 is created in Grafana 2024-04-15T03:04:06.477049514Z 04/15/2024 03:04:06 AM - osm_mon.dashboarder.backends.grafana - INFO - New team quotas_01_proj_1 created in Grafana 2024-04-15T03:04:12.923830675Z 04/15/2024 03:04:12 AM - osm_mon.dashboarder.backends.grafana - INFO - New team quotas_01_proj_2 created in Grafana 2024-04-15T03:04:15.178641691Z 04/15/2024 03:04:15 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard folder quotas_01_proj_1 created 2024-04-15T03:04:15.383534714Z 04/15/2024 03:04:15 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard quotas_01_proj_1 is created in Grafana 2024-04-15T03:04:15.448761041Z 04/15/2024 03:04:15 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard folder quotas_01_proj_2 created 2024-04-15T03:04:15.673617179Z 04/15/2024 03:04:15 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard quotas_01_proj_2 is created in Grafana 2024-04-15T03:04:19.311594194Z 04/15/2024 03:04:19 AM - osm_mon.dashboarder.backends.grafana - INFO - New user quotas_01_user created in Grafana 2024-04-15T03:04:19.480276575Z 04/15/2024 03:04:19 AM - osm_mon.dashboarder.backends.grafana - INFO - User quotas_01_user is assigned Editor permission 2024-04-15T03:04:19.658947485Z 04/15/2024 03:04:19 AM - osm_mon.dashboarder.backends.grafana - INFO - User quotas_01_user is assigned Editor permission 2024-04-15T03:04:24.009829320Z 04/15/2024 03:04:24 AM - osm_mon.dashboarder.dashboarder - ERROR - Exception processing message: 2024-04-15T03:04:24.009860378Z Traceback (most recent call last): 2024-04-15T03:04:24.009866898Z File "/usr/lib/python3/dist-packages/osm_mon/dashboarder/dashboarder.py", line 116, in _user_msg 2024-04-15T03:04:24.009873491Z project_data = values["changes"]["add_project_role_mappings"] 2024-04-15T03:04:24.009879269Z KeyError: 'add_project_role_mappings' 2024-04-15T03:04:25.772159166Z 04/15/2024 03:04:25 AM - osm_mon.dashboarder.backends.grafana - INFO - Grafana team updated {"message":"Team updated"} 2024-04-15T03:04:35.709254000Z 04/15/2024 03:04:35 AM - osm_mon.dashboarder.dashboarder - ERROR - Exception processing message: 2024-04-15T03:04:35.709273138Z Traceback (most recent call last): 2024-04-15T03:04:35.709276689Z File "/usr/lib/python3/dist-packages/osm_mon/dashboarder/dashboarder.py", line 129, in _user_msg 2024-04-15T03:04:35.709279843Z project_new_name = values["changes"]["name"] 2024-04-15T03:04:35.709282947Z KeyError: 'name' 2024-04-15T03:05:06.031662851Z 04/15/2024 03:05:06 AM - osm_mon.dashboarder.backends.grafana - INFO - Team quotas_01_proj_2 deleted in Grafana 2024-04-15T03:05:07.872222850Z 04/15/2024 03:05:07 AM - osm_mon.dashboarder.backends.grafana - INFO - User quotas_01_user deleted in Grafana 2024-04-15T03:05:09.626699238Z 04/15/2024 03:05:09 AM - osm_mon.dashboarder.backends.grafana - INFO - Team quotas_01_proj_3 deleted in Grafana 2024-04-15T03:05:46.209769326Z 04/15/2024 03:05:46 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-15T03:11:47.711437304Z 04/15/2024 03:11:47 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-15T03:27:21.285643900Z 04/15/2024 03:27:21 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-15T03:52:57.108217235Z 04/15/2024 03:52:57 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana