2024-04-14T01:22:45.850555864Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:22:49.853834005Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:22:53.869207098Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:22:57.874616287Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:01.878010428Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:05.882154212Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:09.887291191Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:13.893227985Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:17.901938425Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:21.902786873Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:25.909217618Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:29.911475282Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:33.915313609Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:37.920919366Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:41.927875257Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:45.931826237Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:49.939236490Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:53.941168475Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:23:57.947238176Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:01.949784885Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:05.953737970Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:09.959193481Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:13.963389544Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:17.966440701Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:21.976373651Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:25.982516255Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:29.988787866Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:33.994157700Z nc: kafka (10.102.64.154:9092): Operation timed out 2024-04-14T01:24:36.999570644Z kafka (10.102.64.154:9092) open 2024-04-14T01:24:37.003432957Z mongodb-k8s (10.244.0.43:27017) open 2024-04-14T01:25:33.719274965Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - [IPv4 ('10.102.64.154', 9092)]>: connecting to kafka:9092 [('10.102.64.154', 9092) IPv4] 2024-04-14T01:25:33.719344061Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - Probing node bootstrap-0 broker version 2024-04-14T01:25:33.733504013Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - [IPv4 ('10.102.64.154', 9092)]>: Connection complete. 2024-04-14T01:25:33.842624119Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - Broker version identified as 2.5.0 2024-04-14T01:25:33.842640074Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - Set configuration api_version=(2, 5, 0) to skip auto check_version requests on startup 2024-04-14T01:25:33.882718373Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - [IPv4 ('10.102.64.154', 9092)]>: connecting to kafka:9092 [('10.102.64.154', 9092) IPv4] 2024-04-14T01:25:33.882736107Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - Probing node bootstrap-0 broker version 2024-04-14T01:25:33.885543504Z 04/14/2024 01:25:33 AM - kafka.conn - INFO - [IPv4 ('10.102.64.154', 9092)]>: Connection complete. 2024-04-14T01:25:34.023448635Z 04/14/2024 01:25:34 AM - kafka.conn - INFO - Broker version identified as 2.5.0 2024-04-14T01:25:34.023466461Z 04/14/2024 01:25:34 AM - kafka.conn - INFO - Set configuration api_version=(2, 5, 0) to skip auto check_version requests on startup 2024-04-14T01:25:34.185520133Z 04/14/2024 01:25:34 AM - kafka.conn - INFO - [IPv4 ('10.244.0.34', 9092)]>: connecting to kafka-controller-1.kafka-controller-headless.osm.svc.cluster.local:9092 [('10.244.0.34', 9092) IPv4] 2024-04-14T01:25:34.185930908Z 04/14/2024 01:25:34 AM - kafka.conn - INFO - [IPv4 ('10.244.0.34', 9092)]>: Connection complete. 2024-04-14T01:25:34.186078413Z 04/14/2024 01:25:34 AM - kafka.conn - INFO - [IPv4 ('10.102.64.154', 9092)]>: Closing connection. 2024-04-14T01:25:34.336625444Z 04/14/2024 01:25:34 AM - kafka.conn - INFO - [IPv4 ('10.244.0.32', 9092)]>: connecting to kafka-controller-2.kafka-controller-headless.osm.svc.cluster.local:9092 [('10.244.0.32', 9092) IPv4] 2024-04-14T01:25:34.336976829Z 04/14/2024 01:25:34 AM - kafka.conn - INFO - [IPv4 ('10.244.0.32', 9092)]>: Connection complete. 2024-04-14T01:25:34.343660923Z 04/14/2024 01:25:34 AM - kafka.producer.record_accumulator - WARNING - Produced messages to topic-partition TopicPartition(topic='alarm_request', partition=0) with base offset -1 log start offset -1 and error None. 2024-04-14T01:25:34.440929926Z 04/14/2024 01:25:34 AM - kafka.cluster - ERROR - Topic users not found in cluster metadata 2024-04-14T01:25:34.567105448Z 04/14/2024 01:25:34 AM - kafka.cluster - ERROR - Topic users not found in cluster metadata 2024-04-14T01:25:34.692787792Z 04/14/2024 01:25:34 AM - kafka.cluster - ERROR - Topic users not found in cluster metadata 2024-04-14T01:25:34.973381198Z 04/14/2024 01:25:34 AM - kafka.cluster - ERROR - Topic project not found in cluster metadata 2024-04-14T01:25:35.198936270Z 04/14/2024 01:25:35 AM - kafka.conn - INFO - [IPv4 ('10.244.0.34', 9092)]>: Closing connection. 2024-04-14T01:25:35.200387766Z 04/14/2024 01:25:35 AM - kafka.conn - INFO - [IPv4 ('10.244.0.32', 9092)]>: Closing connection. 2024-04-14T01:25:35.200904635Z 04/14/2024 01:25:35 AM - osm_mon.cmd.mon_dashboarder - INFO - Starting MON Dashboarder... 2024-04-14T01:25:35.297158113Z 04/14/2024 01:25:35 AM - db - INFO - Connected to database osm version 1.2 2024-04-14T01:25:35.297170134Z 04/14/2024 01:25:35 AM - osm_mon.cmd.mon_dashboarder - INFO - Starting MON kafka Consumer... 2024-04-14T01:25:35.299700219Z 04/14/2024 01:25:35 AM - osm_mon.cmd.mon_dashboarder - INFO - Starting MON Dashboarder... 2024-04-14T01:25:35.407576823Z 04/14/2024 01:25:35 AM - aiokafka.consumer.subscription_state - INFO - Updating subscribed topics to: frozenset({'project', 'users'}) 2024-04-14T01:25:35.407592137Z 04/14/2024 01:25:35 AM - aiokafka.consumer.consumer - INFO - Subscribed to topic(s): {'project', 'users'} 2024-04-14T01:25:35.440828803Z 04/14/2024 01:25:35 AM - aiokafka.consumer.group_coordinator - ERROR - Group Coordinator Request failed: [Error 15] CoordinatorNotAvailableError 2024-04-14T01:25:35.887254791Z 04/14/2024 01:25:35 AM - aiokafka.cluster - ERROR - Topic project not found in cluster metadata 2024-04-14T01:25:36.077310395Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:36.077966587Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - Revoking previously assigned partitions set() for group mon-consumer 2024-04-14T01:25:36.077973495Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:36.139707175Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:36.245551374Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:36.245967400Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:36.288887012Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:36.477813645Z 04/14/2024 01:25:36 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-14T01:25:36.541716427Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:36.541730777Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:36.542392274Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:36.652097848Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:36.652110634Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:36.673780832Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:36.805992140Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:36.806505830Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:36.813636834Z 04/14/2024 01:25:36 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.039099534Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.039169094Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:37.046920180Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.164434466Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.164684649Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:37.184060589Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.315327740Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.315348277Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:37.335791123Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.449200007Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.449214438Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:37.455825968Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.580883414Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.580895097Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:37.589060762Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.711247487Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.711270344Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:37.715997709Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.831114317Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.834099818Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:37.834244804Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:37.971683650Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:37.971701658Z 04/14/2024 01:25:37 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:38.041769428Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:38.210050524Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:38.210061028Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:38.224355155Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:38.358673738Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:38.358690941Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:38.388055629Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:38.499437459Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:38.499453617Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:38.526041565Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:38.713720519Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:38.713732496Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:38.722805446Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:38.835873065Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:38.835976744Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:38.863844152Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:38.982440272Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:38.982456780Z 04/14/2024 01:25:38 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:39.006798088Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:39.121536971Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:39.121576324Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:39.125492440Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:39.271427503Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:39.271464316Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:39.298332882Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:39.402495875Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:39.402512806Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:39.409230429Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - WARNING - Marking the coordinator dead (node 1)for group mon-consumer. 2024-04-14T01:25:39.511658938Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - Discovered coordinator 1 for group mon-consumer 2024-04-14T01:25:39.511694643Z 04/14/2024 01:25:39 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:25:42.615862342Z 04/14/2024 01:25:42 AM - aiokafka.consumer.group_coordinator - INFO - Joined group 'mon-consumer' (generation 1) with member_id aiokafka-0.8.1-c6ff7404-b579-41bb-bdfb-29c082a0f07f 2024-04-14T01:25:42.615879125Z 04/14/2024 01:25:42 AM - aiokafka.consumer.group_coordinator - INFO - Elected group leader -- performing partition assignments using roundrobin 2024-04-14T01:25:42.616073959Z 04/14/2024 01:25:42 AM - kafka.coordinator.assignors.roundrobin - WARNING - No partition metadata for topic project 2024-04-14T01:25:42.825985540Z 04/14/2024 01:25:42 AM - aiokafka.consumer.group_coordinator - INFO - Successfully synced group mon-consumer with generation 1 2024-04-14T01:25:42.826155972Z 04/14/2024 01:25:42 AM - aiokafka.consumer.group_coordinator - INFO - Setting newly assigned partitions {TopicPartition(topic='users', partition=0)} for group mon-consumer 2024-04-14T01:30:35.948096167Z 04/14/2024 01:30:35 AM - aiokafka.consumer.group_coordinator - INFO - Metadata for topic has changed from {'project': 0, 'users': 1} to {'project': 1, 'users': 1}. 2024-04-14T01:30:35.958108248Z 04/14/2024 01:30:35 AM - aiokafka.consumer.group_coordinator - INFO - Revoking previously assigned partitions frozenset({TopicPartition(topic='users', partition=0)}) for group mon-consumer 2024-04-14T01:30:35.958305868Z 04/14/2024 01:30:35 AM - aiokafka.consumer.group_coordinator - INFO - (Re-)joining group mon-consumer 2024-04-14T01:30:35.971520413Z 04/14/2024 01:30:35 AM - aiokafka.consumer.group_coordinator - INFO - Joined group 'mon-consumer' (generation 2) with member_id aiokafka-0.8.1-c6ff7404-b579-41bb-bdfb-29c082a0f07f 2024-04-14T01:30:35.971545398Z 04/14/2024 01:30:35 AM - aiokafka.consumer.group_coordinator - INFO - Elected group leader -- performing partition assignments using roundrobin 2024-04-14T01:30:35.979565782Z 04/14/2024 01:30:35 AM - aiokafka.consumer.group_coordinator - INFO - Successfully synced group mon-consumer with generation 2 2024-04-14T01:30:35.980104751Z 04/14/2024 01:30:35 AM - aiokafka.consumer.group_coordinator - INFO - Setting newly assigned partitions {TopicPartition(topic='users', partition=0), TopicPartition(topic='project', partition=0)} for group mon-consumer 2024-04-14T01:32:38.322247658Z 04/14/2024 01:32:38 AM - osm_mon.dashboarder.backends.grafana - INFO - Datasource prom-operator-osm is created in Grafana 2024-04-14T01:32:38.322400020Z 04/14/2024 01:32:38 AM - osm_mon.dashboarder.backends.grafana - INFO - ************* response: {'_content': b'{"datasource":{"id":3,"uid":"fc9124e4-7bd3-4b3b-8d0a-92be133b7640","orgId":1,"name":"prom-operator-osm","type":"prometheus","typeLogoUrl":"public/app/plugins/datasource/prometheus/img/prometheus_logo.svg","access":"proxy","url":"http://172.21.248.125:9090","user":"","database":"","basicAuth":false,"basicAuthUser":"","withCredentials":false,"isDefault":false,"jsonData":{},"secureJsonFields":{},"version":1,"readOnly":false},"id":3,"message":"Datasource added","name":"prom-operator-osm"}', '_content_consumed': True, '_next': None, 'status_code': 200, 'headers': {'Cache-Control': 'no-store', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-Xss-Protection': '1; mode=block', 'Date': 'Sun, 14 Apr 2024 01:32:38 GMT', 'Content-Length': '489'}, 'raw': , 'url': 'http://grafana:3000/api/datasources', 'encoding': 'utf-8', 'history': [], 'reason': 'OK', 'cookies': , 'elapsed': datetime.timedelta(microseconds=45507), 'request': , 'connection': } 2024-04-14T01:32:38.526360569Z 04/14/2024 01:32:38 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard osm is created in Grafana 2024-04-14T01:39:10.024005283Z 04/14/2024 01:39:10 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-14T01:40:59.909707844Z 04/14/2024 01:40:59 AM - osm_mon.dashboarder.backends.grafana - INFO - New user basic_15_test_user created in Grafana 2024-04-14T01:41:03.109128833Z 04/14/2024 01:41:03 AM - osm_mon.dashboarder.backends.grafana - INFO - User basic_15_test_user is assigned Admin permission 2024-04-14T01:41:07.901773755Z 04/14/2024 01:41:07 AM - osm_mon.dashboarder.dashboarder - ERROR - Exception processing message: 2024-04-14T01:41:07.901793227Z Traceback (most recent call last): 2024-04-14T01:41:07.901796558Z File "/usr/lib/python3/dist-packages/osm_mon/dashboarder/dashboarder.py", line 116, in _user_msg 2024-04-14T01:41:07.901799997Z project_data = values["changes"]["add_project_role_mappings"] 2024-04-14T01:41:07.901803072Z KeyError: 'add_project_role_mappings' 2024-04-14T01:41:09.251241208Z 04/14/2024 01:41:09 AM - osm_mon.dashboarder.backends.grafana - INFO - New team basic_15_test_project created in Grafana 2024-04-14T01:41:10.478781487Z 04/14/2024 01:41:10 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard folder basic_15_test_project created 2024-04-14T01:41:10.691469416Z 04/14/2024 01:41:10 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard basic_15_test_project is created in Grafana 2024-04-14T01:41:10.861026807Z 04/14/2024 01:41:10 AM - osm_mon.dashboarder.backends.grafana - INFO - Grafana team updated {"message":"Team updated"} 2024-04-14T01:41:18.962389007Z 04/14/2024 01:41:18 AM - osm_mon.dashboarder.backends.grafana - INFO - User basic_15_test_user deleted in Grafana 2024-04-14T01:41:20.660258374Z 04/14/2024 01:41:20 AM - osm_mon.dashboarder.backends.grafana - INFO - Team basic_15_project_test deleted in Grafana 2024-04-14T01:41:41.032222967Z 04/14/2024 01:41:41 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-14T02:49:25.277121290Z 04/14/2024 02:49:25 AM - osm_mon.dashboarder.backends.grafana - INFO - Datasource prom-operator-k8s02 is created in Grafana 2024-04-14T02:49:25.277265088Z 04/14/2024 02:49:25 AM - osm_mon.dashboarder.backends.grafana - INFO - ************* response: {'_content': b'{"datasource":{"id":4,"uid":"beb15d61-7a10-46e6-af80-d9d430f70071","orgId":1,"name":"prom-operator-k8s02","type":"prometheus","typeLogoUrl":"public/app/plugins/datasource/prometheus/img/prometheus_logo.svg","access":"proxy","url":"http://172.21.248.125:9090","user":"","database":"","basicAuth":false,"basicAuthUser":"","withCredentials":false,"isDefault":false,"jsonData":{},"secureJsonFields":{},"version":1,"readOnly":false},"id":4,"message":"Datasource added","name":"prom-operator-k8s02"}', '_content_consumed': True, '_next': None, 'status_code': 200, 'headers': {'Cache-Control': 'no-store', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-Xss-Protection': '1; mode=block', 'Date': 'Sun, 14 Apr 2024 02:49:25 GMT', 'Content-Length': '493'}, 'raw': , 'url': 'http://grafana:3000/api/datasources', 'encoding': 'utf-8', 'history': [], 'reason': 'OK', 'cookies': , 'elapsed': datetime.timedelta(microseconds=44059), 'request': , 'connection': } 2024-04-14T02:49:25.430420687Z 04/14/2024 02:49:25 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard k8s02 is created in Grafana 2024-04-14T02:53:56.428963573Z 04/14/2024 02:53:56 AM - osm_mon.dashboarder.backends.grafana - INFO - Datasource prom-operator-k8s07 is created in Grafana 2024-04-14T02:53:56.429218654Z 04/14/2024 02:53:56 AM - osm_mon.dashboarder.backends.grafana - INFO - ************* response: {'_content': b'{"datasource":{"id":5,"uid":"af6a063b-6c46-4292-b718-79b92f512906","orgId":1,"name":"prom-operator-k8s07","type":"prometheus","typeLogoUrl":"public/app/plugins/datasource/prometheus/img/prometheus_logo.svg","access":"proxy","url":"http://172.21.248.125:9090","user":"","database":"","basicAuth":false,"basicAuthUser":"","withCredentials":false,"isDefault":false,"jsonData":{},"secureJsonFields":{},"version":1,"readOnly":false},"id":5,"message":"Datasource added","name":"prom-operator-k8s07"}', '_content_consumed': True, '_next': None, 'status_code': 200, 'headers': {'Cache-Control': 'no-store', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-Xss-Protection': '1; mode=block', 'Date': 'Sun, 14 Apr 2024 02:53:56 GMT', 'Content-Length': '493'}, 'raw': , 'url': 'http://grafana:3000/api/datasources', 'encoding': 'utf-8', 'history': [], 'reason': 'OK', 'cookies': , 'elapsed': datetime.timedelta(microseconds=51137), 'request': , 'connection': } 2024-04-14T02:53:56.590859034Z 04/14/2024 02:53:56 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard k8s07 is created in Grafana 2024-04-14T03:01:42.224306978Z 04/14/2024 03:01:42 AM - osm_mon.dashboarder.backends.grafana - INFO - New team quotas_01_proj_1 created in Grafana 2024-04-14T03:01:48.445422043Z 04/14/2024 03:01:48 AM - osm_mon.dashboarder.backends.grafana - INFO - New team quotas_01_proj_2 created in Grafana 2024-04-14T03:01:54.893171785Z 04/14/2024 03:01:54 AM - osm_mon.dashboarder.backends.grafana - INFO - New user quotas_01_user created in Grafana 2024-04-14T03:01:55.105065171Z 04/14/2024 03:01:55 AM - osm_mon.dashboarder.backends.grafana - INFO - User quotas_01_user is assigned Editor permission 2024-04-14T03:01:55.236719377Z 04/14/2024 03:01:55 AM - osm_mon.dashboarder.backends.grafana - INFO - User quotas_01_user is assigned Editor permission 2024-04-14T03:01:58.238677196Z 04/14/2024 03:01:58 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard folder quotas_01_proj_1 created 2024-04-14T03:01:58.429337275Z 04/14/2024 03:01:58 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard quotas_01_proj_1 is created in Grafana 2024-04-14T03:01:58.503358902Z 04/14/2024 03:01:58 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard folder quotas_01_proj_2 created 2024-04-14T03:01:58.713117053Z 04/14/2024 03:01:58 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard quotas_01_proj_2 is created in Grafana 2024-04-14T03:01:59.624612454Z 04/14/2024 03:01:59 AM - osm_mon.dashboarder.dashboarder - ERROR - Exception processing message: 2024-04-14T03:01:59.624645004Z Traceback (most recent call last): 2024-04-14T03:01:59.624652040Z File "/usr/lib/python3/dist-packages/osm_mon/dashboarder/dashboarder.py", line 116, in _user_msg 2024-04-14T03:01:59.624658532Z project_data = values["changes"]["add_project_role_mappings"] 2024-04-14T03:01:59.624663984Z KeyError: 'add_project_role_mappings' 2024-04-14T03:02:01.331216065Z 04/14/2024 03:02:01 AM - osm_mon.dashboarder.backends.grafana - INFO - Grafana team updated {"message":"Team updated"} 2024-04-14T03:02:11.513306518Z 04/14/2024 03:02:11 AM - osm_mon.dashboarder.dashboarder - ERROR - Exception processing message: 2024-04-14T03:02:11.513348601Z Traceback (most recent call last): 2024-04-14T03:02:11.513358942Z File "/usr/lib/python3/dist-packages/osm_mon/dashboarder/dashboarder.py", line 129, in _user_msg 2024-04-14T03:02:11.513367577Z project_new_name = values["changes"]["name"] 2024-04-14T03:02:11.513375992Z KeyError: 'name' 2024-04-14T03:02:41.144700216Z 04/14/2024 03:02:41 AM - osm_mon.dashboarder.backends.grafana - INFO - Team quotas_01_proj_2 deleted in Grafana 2024-04-14T03:02:42.844297646Z 04/14/2024 03:02:42 AM - osm_mon.dashboarder.backends.grafana - INFO - User quotas_01_user deleted in Grafana 2024-04-14T03:02:44.396589684Z 04/14/2024 03:02:44 AM - osm_mon.dashboarder.backends.grafana - INFO - Team quotas_01_proj_3 deleted in Grafana 2024-04-14T03:02:59.074720226Z 04/14/2024 03:02:59 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-14T03:11:31.107387420Z 04/14/2024 03:11:31 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-14T03:27:04.631089045Z 04/14/2024 03:27:04 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana 2024-04-14T03:52:40.330328604Z 04/14/2024 03:52:40 AM - osm_mon.dashboarder.backends.grafana - INFO - Dashboard admin is created in Grafana