2024-04-23T17:52:30.528193325Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:52:34.539705878Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:52:38.540165527Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:52:42.569172940Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:52:46.592166384Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:52:50.596212339Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:52:54.606011162Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:52:58.608803997Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:02.612135877Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:06.616097643Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:10.621372000Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:14.625939525Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:18.629934108Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:22.642930491Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:26.657944239Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:30.660495171Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:34.668668544Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:38.675436070Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:42.681548073Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:46.697933661Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:50.716287072Z nc: kafka (10.105.57.182:9092): Operation timed out 2024-04-23T17:53:56.732736139Z kafka (10.105.57.182:9092) open 2024-04-23T17:53:56.854465031Z nc: bad address 'ro' 2024-04-23T17:53:59.879265285Z kafka (10.105.57.182:9092) open 2024-04-23T17:53:59.960329685Z nc: bad address 'ro' 2024-04-23T17:54:02.955270952Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:02.966936299Z nc: bad address 'ro' 2024-04-23T17:54:05.972730892Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:06.025778992Z nc: bad address 'ro' 2024-04-23T17:54:09.045773796Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:09.053194321Z nc: bad address 'ro' 2024-04-23T17:54:12.068943384Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:12.090584553Z nc: bad address 'ro' 2024-04-23T17:54:15.109400128Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:15.123418098Z nc: bad address 'ro' 2024-04-23T17:54:18.174530471Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:18.182235195Z nc: bad address 'ro' 2024-04-23T17:54:21.190051642Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:21.195032142Z nc: bad address 'ro' 2024-04-23T17:54:24.221042062Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:24.225506719Z nc: bad address 'ro' 2024-04-23T17:54:27.229177470Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:27.232956311Z nc: bad address 'ro' 2024-04-23T17:54:30.240382382Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:30.351384165Z nc: bad address 'ro' 2024-04-23T17:54:33.365305476Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:33.367890666Z nc: bad address 'ro' 2024-04-23T17:54:36.381494543Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:36.497990537Z nc: bad address 'ro' 2024-04-23T17:54:39.503130704Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:39.514165071Z nc: bad address 'ro' 2024-04-23T17:54:42.525144976Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:42.541168596Z nc: bad address 'ro' 2024-04-23T17:54:45.550006173Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:45.555119214Z nc: bad address 'ro' 2024-04-23T17:54:48.567981562Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:48.573675218Z nc: bad address 'ro' 2024-04-23T17:54:51.585170852Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:51.604458271Z nc: bad address 'ro' 2024-04-23T17:54:54.606915755Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:54.616735831Z nc: bad address 'ro' 2024-04-23T17:54:57.623984295Z kafka (10.105.57.182:9092) open 2024-04-23T17:54:57.638077561Z nc: bad address 'ro' 2024-04-23T17:55:00.702350230Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:00.734883323Z nc: bad address 'ro' 2024-04-23T17:55:03.763436549Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:03.776626070Z nc: bad address 'ro' 2024-04-23T17:55:06.781079454Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:06.844540821Z nc: bad address 'ro' 2024-04-23T17:55:09.854448565Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:09.930739944Z nc: bad address 'ro' 2024-04-23T17:55:12.934435087Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:12.938159244Z nc: bad address 'ro' 2024-04-23T17:55:15.946506389Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:15.955789151Z nc: bad address 'ro' 2024-04-23T17:55:18.961397064Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:18.968255084Z nc: bad address 'ro' 2024-04-23T17:55:21.978605151Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:21.987484693Z nc: bad address 'ro' 2024-04-23T17:55:25.005820472Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:25.018843713Z nc: bad address 'ro' 2024-04-23T17:55:28.071587958Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:28.097470808Z nc: bad address 'ro' 2024-04-23T17:55:31.105993166Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:31.177954559Z nc: bad address 'ro' 2024-04-23T17:55:34.194874923Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:34.200920512Z nc: bad address 'ro' 2024-04-23T17:55:37.214588069Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:37.292820464Z nc: bad address 'ro' 2024-04-23T17:55:40.304935966Z kafka (10.105.57.182:9092) open 2024-04-23T17:55:40.310540614Z ro (10.244.0.17:9090) open 2024-04-23T17:55:40.318385858Z mongodb-k8s (10.244.0.46:27017) open 2024-04-23T17:56:02.867881877Z Loaded configuration:{'globalConfig': {'loglevel': 'DEBUG', 'logfile': None, 'nologging': False}, 'timeout': {'nsi_deploy': 7200, 'vca_on_error': 300, 'ns_deploy': 7200, 'ns_terminate': 1800, 'ns_heal': 1800, 'charm_delete': 600, 'primitive': 1800, 'ns_update': 1800, 'progress_primitive': 600, 'migrate': 1800, 'operate': 1800, 'verticalscale': 1800}, 'RO': {'host': 'ro', 'ng': False, 'port': 9090, 'uri': 'http://ro:9090/', 'tenant': 'osm', 'loglevel': 'ERROR', 'logfile': None, 'logger_name': 'lcm.roclient'}, 'VCA': {'host': 'vca', 'port': 17070, 'user': 'admin', 'secret': 'secret', 'cloud': 'lxd-cloud', 'k8s_cloud': 'k8scloud', 'helmpath': '/usr/local/bin/helm', 'helm3path': '/usr/local/bin/helm3', 'kubectlpath': '/usr/bin/kubectl', 'jujupath': '/usr/local/bin/juju', 'public_key': None, 'ca_cert': None, 'api_proxy': None, 'apt_mirror': None, 'eegrpcinittimeout': None, 'eegrpctimeout': None, 'eegrpc_tls_enforce': False, 'eegrpc_pod_admission_policy': 'baseline', 'loglevel': 'DEBUG', 'logfile': None, 'ca_store': '/etc/ssl/certs/osm-ca.crt', 'client_cert_path': '/etc/ssl/lcm-client/tls.crt', 'client_key_path': '/etc/ssl/lcm-client/tls.key', 'kubectl_osm_namespace': 'osm', 'kubectl_osm_cluster_name': '_system-osm-k8s', 'helm_ee_service_port': 50050, 'helm_max_initial_retry_time': 600, 'helm_max_retry_time': 30, 'helm_ee_retry_delay': 10}, 'database': {'driver': 'mongo', 'host': 'mongo', 'port': 27017, 'uri': 'mongodb://mongodb-k8s:27017/?replicaSet=rs0', 'name': 'osm', 'replicaset': None, 'user': None, 'password': None, 'commonkey': 'ruXUur2IjBhuhD89CmEyqIsqVLDqnxzJ', 'loglevel': 'DEBUG', 'logfile': None, 'logger_name': 'lcm.db'}, 'storage': {'driver': 'mongo', 'path': '/app/storage', 'loglevel': 'DEBUG', 'logfile': None, 'logger_name': 'lcm.fs', 'collection': 'files', 'uri': 'mongodb://mongodb-k8s:27017/?replicaSet=rs0'}, 'message': {'driver': 'kafka', 'path': '/app/storage/kafka', 'host': 'kafka', 'port': 9092, 'loglevel': 'DEBUG', 'logfile': None, 'group_id': 'lcm-server', 'logger_name': 'lcm.msg'}, 'tsdb': {'driver': 'prometheus', 'path': '/etc/prometheus', 'uri': 'http://prometheus:9090/', 'loglevel': 'DEBUG', 'logfile': None, 'logger_name': 'lcm.prometheus'}} 2024-04-23T17:56:02.910812460Z 2024-04-23T17:56:02 CRITICAL lcm lcm.py:140 starting osm/lcm version 9.0.0.post162+g9cd5345 2020-04-08 2024-04-23T17:56:03.087142495Z 2024-04-23T17:56:03 INFO lcm.db dbmongo.py:144 Connected to database osm version 1.2 2024-04-23T17:56:03.101066479Z /usr/local/lib/python3.10/dist-packages/gridfs/grid_file.py:881: UserWarning: use an explicit session with no_cursor_timeout=True otherwise the cursor may still timeout after 30 minutes, for more info see https://mongodb.com/docs/v4.4/reference/method/cursor.noCursorTimeout/#session-idle-timeout-overrides-nocursortimeout 2024-04-23T17:56:03.101082498Z super().__init__( 2024-04-23T17:56:03.361682647Z 2024-04-23T17:56:03 INFO lcm lcm.py:220 Connected to RO version 9.0.0.post172+ge31d74b\n" new-generation version True 2024-04-23T17:56:03.361957841Z 2024-04-23T17:56:03 INFO lcm.ns n2vc_juju_conn.py:83 Initializing N2VC juju connector... 2024-04-23T17:56:03.378737486Z 2024-04-23T17:56:03 INFO lcm.ns n2vc_juju_conn.py:89 N2VC juju connector initialized 2024-04-23T17:56:03.378804790Z 2024-04-23T17:56:03 DEBUG lcm.ns lcm_helm_conn.py:114 Initialize helm N2VC connector 2024-04-23T17:56:03.381021952Z 2024-04-23T17:56:03 DEBUG lcm.ns lcm_helm_conn.py:115 initial vca_config: {'host': 'vca', 'port': 17070, 'user': 'admin', 'secret': 'secret', 'cloud': 'lxd-cloud', 'k8s_cloud': 'k8scloud', 'helmpath': '/usr/local/bin/helm', 'helm3path': '/usr/local/bin/helm3', 'kubectlpath': '/usr/bin/kubectl', 'jujupath': '/usr/local/bin/juju', 'public_key': None, 'ca_cert': None, 'api_proxy': None, 'apt_mirror': None, 'eegrpcinittimeout': None, 'eegrpctimeout': None, 'eegrpc_tls_enforce': False, 'eegrpc_pod_admission_policy': 'baseline', 'loglevel': 'DEBUG', 'logfile': None, 'ca_store': '/etc/ssl/certs/osm-ca.crt', 'client_cert_path': '/etc/ssl/lcm-client/tls.crt', 'client_key_path': '/etc/ssl/lcm-client/tls.key', 'kubectl_osm_namespace': 'osm', 'kubectl_osm_cluster_name': '_system-osm-k8s', 'helm_ee_service_port': 50050, 'helm_max_initial_retry_time': 600, 'helm_max_retry_time': 30, 'helm_ee_retry_delay': 10} 2024-04-23T17:56:03.381054155Z 2024-04-23T17:56:03 DEBUG lcm.ns lcm_helm_conn.py:120 Initial retry time: 600 2024-04-23T17:56:03.381057682Z 2024-04-23T17:56:03 DEBUG lcm.ns lcm_helm_conn.py:123 Retry time: 30 2024-04-23T17:56:03.381082089Z 2024-04-23T17:56:03 INFO lcm.ns k8s_helm_base_conn.py:74 Initializing K8S Helm connector 2024-04-23T17:56:03.381530151Z 2024-04-23T17:56:03 INFO lcm.ns k8s_helm3_conn.py:70 K8S Helm3 connector initialized 2024-04-23T17:56:03.381568524Z 2024-04-23T17:56:03 INFO lcm.ns lcm_helm_conn.py:136 Helm N2VC connector initialized 2024-04-23T17:56:03.381616021Z 2024-04-23T17:56:03 INFO lcm.ns k8s_helm_base_conn.py:74 Initializing K8S Helm connector 2024-04-23T17:56:03.382110876Z 2024-04-23T17:56:03 INFO lcm.ns k8s_helm3_conn.py:70 K8S Helm3 connector initialized 2024-04-23T17:56:03.382144310Z 2024-04-23T17:56:03 DEBUG lcm.ns k8s_juju_conn.py:68 Initializing K8S Juju connector 2024-04-23T17:56:03.384871423Z 2024-04-23T17:56:03 DEBUG lcm.ns k8s_juju_conn.py:75 K8S Juju connector initialized 2024-04-23T17:56:03.422703754Z 2024-04-23T17:56:03 INFO lcm.k8scluster k8s_helm_base_conn.py:74 Initializing K8S Helm connector 2024-04-23T17:56:03.423090809Z 2024-04-23T17:56:03 INFO lcm.k8scluster k8s_helm3_conn.py:70 K8S Helm3 connector initialized 2024-04-23T17:56:03.423103293Z 2024-04-23T17:56:03 DEBUG lcm.k8scluster k8s_juju_conn.py:68 Initializing K8S Juju connector 2024-04-23T17:56:03.426294523Z 2024-04-23T17:56:03 DEBUG lcm.k8scluster k8s_juju_conn.py:75 K8S Juju connector initialized 2024-04-23T17:56:03.433073132Z 2024-04-23T17:56:03 INFO lcm.vca n2vc_juju_conn.py:83 Initializing N2VC juju connector... 2024-04-23T17:56:03.435422424Z 2024-04-23T17:56:03 INFO lcm.vca n2vc_juju_conn.py:89 N2VC juju connector initialized 2024-04-23T17:56:03.442354939Z 2024-04-23T17:56:03 INFO lcm.k8srepo k8s_helm_base_conn.py:74 Initializing K8S Helm connector 2024-04-23T17:56:03.442734395Z 2024-04-23T17:56:03 INFO lcm.k8srepo k8s_helm3_conn.py:70 K8S Helm3 connector initialized 2024-04-23T17:56:03.442950941Z 2024-04-23T17:56:03 DEBUG lcm lcm.py:688 Task kafka_read Enter with worker_id=680e6a0af92e 2024-04-23T17:56:03.443045171Z 2024-04-23T17:56:03 DEBUG lcm lcm.py:247 Task kafka_ping Enter 2024-04-23T17:56:03.982324318Z Topic nslcmops not found in cluster metadata 2024-04-23T17:56:03.982396564Z Topic vca not found in cluster metadata 2024-04-23T17:56:03.982400891Z Topic k8scluster not found in cluster metadata 2024-04-23T17:56:03.982403976Z Topic k8srepo not found in cluster metadata 2024-04-23T17:56:03.982407023Z Topic pla not found in cluster metadata 2024-04-23T17:56:07.649186376Z 2024-04-23T17:56:07 DEBUG lcm lcm.py:301 Task kafka_read receives sdn echo: dummy message 2024-04-23T17:56:07.649177049Z dummy message 2024-04-23T17:56:07.737609299Z 2024-04-23T17:56:07 DEBUG lcm lcm.py:301 Task kafka_read receives nsi echo: dummy message 2024-04-23T17:56:07.737769639Z dummy message 2024-04-23T17:56:07.824748061Z dummy message 2024-04-23T17:56:07.824774810Z 2024-04-23T17:56:07 DEBUG lcm lcm.py:301 Task kafka_read receives ns echo: dummy message 2024-04-23T17:56:07.824923735Z 2024-04-23T17:56:07 DEBUG lcm lcm.py:301 Task kafka_read receives wim_account echo: dummy message 2024-04-23T17:56:07.824937803Z dummy message 2024-04-23T17:56:07.826077373Z 2024-04-23T17:56:07 DEBUG lcm lcm.py:301 Task kafka_read receives vim_account echo: dummy message 2024-04-23T17:56:07.826099952Z dummy message