Update from master
Merged the following from master into paas branch:
Add OSM-POL integration tests
Change-Id: I140b9eb271c0f03520660b676e075b3f0d62a128
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Add OSM-MON integration tests
Change-Id: I3199869880d0c9ce0784dcc623c844dd39f1180a
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Bug 2218: Fix command for `juju run-action`
Change-Id: Ife2e8e9f532f3c67c7e2f71d3f77d3e4e7dc5257
Signed-off-by: Daniel Arndt <daniel.arndt@canonical.com>
Update the artifacts stored in stage2
This change updates the patterns of the artifacts to be stored by the
method `archive` in `ci_helper.groovy`.
The pattern "dists/*.gz" and "dists/*Packages" corresponding to index
files for debian repos are no longer required.
The pattern "dist/*.whl" corresponding to Python wheel files is now
required, since it is an additional artifact generated in stage2.
Change-Id: Id87fcb98b2d79a9bd0b64fdaca44da8acd9e1cb1
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
ntegration of OSM Charms with new MongoDB
Change-Id: I9e723dc94ff4c5b7e691179be4e9e3c7b43b6ab0
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Charm cleanup
Removal of obsolete charm code
Change-Id: Ifc5e83457cf580d8b236a636328470c527c5c3a9
Signed-off-by: Mark Beierl <mark.beierl@canonical.com>
Integration tests for NG UI charm
Change-Id: I3c8958d54aeed84faf1ed2194bc818c1691cf755
Signed-off-by: Daniel Arndt <daniel.arndt@canonical.com>
Fix unit tests for NG-UI charm
Change-Id: If5b98446bb589a3346bcaf1d260a3ad2c5affd3b
Signed-off-by: Daniel Arndt <daniel.arndt@canonical.com>
Set K8s 1.26 in charmed OSM installation
storage is deprecated: replaced by hostpath-storage
Change-Id: I11dd6fc2c18f89c289ad80da696929a7c0236d63
Signed-off-by: Patricia Reinoso <patricia.reinoso@canonical.com>
Remove duplicated lines in Airflow Dockerfile
Change-Id: Iaeb200d498c01e53a7748293d39b6d9a0ba3cfa9
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Fix docker tag in stage3 to coexist with periodic clean-up
Change-Id: I1ce9a5de84e0bcedd7abaecfa0fb6d753b853cb7
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Pin Charmed Operator Framework version for charms
Change-Id: Iff5659151e5678298b72e54b7b22a375bc7b7ebf
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Update base image for Airflow to 2.5.2
Change-Id: Id73a0de10b80a4154e1816c5695d3c96de1b03fe
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Update base image for Airflow to support Python 3.10
Change-Id: I4d0bd5be38faff10de4bd2dbaaa9a6010ab12732
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Remove checks for copyright in charms
This patch removes the flake8 copyright plugin and configuration.
Change-Id: I65e362748e16efbc48055370f8f1590d4910c000
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Update bundle (standalone and HA) to use MongoDB charm from edge channel
Change-Id: Ie60a105a58c5838db90129f1d6d896907675a405
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Update Dockerfile and stage-test script to run tests for charms
This patch updates Dockerfile to use Ubuntu 20.04 as base for building
and testing charms.
This patch updates stage-test script to execute testing for charms.
Tests will be executed only for charms modified by the review.
This patch updates tox configuration for charms setting the python
interpreter to python3.8.
Change-Id: Ib9046b78d6520188cc51ac776fe60ea16479f11c
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Adding documentation to OSM bundles
Change-Id: I94b2d7467f4fba40b625acaf545dc20fc6079f8c
Signed-off-by: Guillermo Calvino <guillermo.calvino@canonical.com>
Partial revert of 13026
The *.gz and *Packages are actually used in the creation of
the debian repository for the installers.
Change-Id: I56ba0ce478fba9bcaeb58d6f2abaf235a4eab78a
Signed-off-by: Mark Beierl <mark.beierl@canonical.com>
Minor indentation fixes in MON and POL K8s manifests
Change-Id: Ib96f1655df650587fc6255d5f98986e1332bbb2f
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Integration tests for VCA Integrator Operator
Change-Id: I2bc362961edb19f3a0696c779aa9eeaacc361572
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Signed-off-by: Mark Beierl <mark.beierl@canonical.com>
LCM integration tests: use RO charm from charmhub instead of building it
locally
Change-Id: I3c1aba9227d9ef5c28f559447da63035214c8ea1
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Feature 10981: installation of AlertManager as part of NG-SA
Change-Id: I99bb5785081df4395be336f323d5d4ac3dfd68b6
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Feature 10981: installation of webhook translator as part of NG-SA
Change-Id: I5318460103a6b89b37931bf661618251a3837d04
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Remove unnecessary Makefile related to old docker image build process
Change-Id: Icc304cfe7124979584405ec6635ce2c7a9861eac
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Update tools/local-build.sh to run python http server instead of qhttp
Change-Id: Id9857656e18e1487da7123e076bf00c0b9869d25
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Add Dockerfile for Webhook translator
Change-Id: Id9a787e0fd3fd953b1b2ace190cdca6a77199f27
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Replace OSM_STACK_NAME by OSM_NAMESPACE in installers scripts
Change-Id: I5ce4bdc392fd64b4bed7479768b91adba53c67e4
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Update helm version to 3.11.3
Change-Id: Ic95f32cd1fc311bf93a817da90f48a17d7c2bd13
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Add nohup to http.server in tools/local-build.sh
Change-Id: Ic21b33c22c069d6145ba9d60c7e3cebb75f99664
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Feature 10981: auto-scaling alerts rules for AlertManager
Change-Id: I7e8c3f7b1dd3201b75848ae6264eaa2375a5b06b
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
Feature 10981: fix CMD in webhook Dockerfile
Change-Id: If8332c12c2f065c0a4d195873e24a98aa34b0ed4
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
Feature 10981: remove mon and pol for ng-sa installation
This change removes the deployment of POL for NG-SA installation.
In addition, it deploys a reduced MON, which will only run
mon-dashboarder. A new K8s manifest (ng-mon.yaml )file has been created
for the purpose.
Change-Id: I243a2710d7b883d505ff4b4d012f7d67920f0e73
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Feature 10981: extended Prometheus sidecar to dump alerts rules in config files
Change-Id: Ic454c894b60d0b2b88b6ea81ca35a0bf4d54ebac
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
OSM DB Update Charm
Initial load of code for the osm-update-db-operator charm
Change-Id: I2884249efaaa86f614df6c286a69f3546489b523
Signed-off-by: Mark Beierl <mark.beierl@canonical.com>
Improve stage-test script: Split charms list according to tox envlist.
For newer charms the tox envlist includes lint, unit and integration: for these charms execute only lint and unit tests.
For older charms the tox envlist includes black, cover, flake8, pylint, yamllint, safety: for these charms execute all tests.
Change-Id: I6cfbe129440be1665f63572a1879060eccd822fd
Signed-off-by: Dario Faccin <dario.faccin@canonical.com>
Signed-off-by: Mark Beierl <mark.beierl@canonical.com>
diff --git a/installers/charm/osm-mon/config.yaml b/installers/charm/osm-mon/config.yaml
index 0163151..cb2eb99 100644
--- a/installers/charm/osm-mon/config.yaml
+++ b/installers/charm/osm-mon/config.yaml
@@ -96,7 +96,7 @@
After enabling the debug-mode, execute the following command to get the information you need
to start debugging:
- `juju run-action get-debug-mode-information <unit name> --wait`
+ `juju run-action <unit name> get-debug-mode-information --wait`
The previous command returns the command you need to execute, and the SSH password that was set.
diff --git a/installers/charm/osm-mon/lib/charms/data_platform_libs/v0/data_interfaces.py b/installers/charm/osm-mon/lib/charms/data_platform_libs/v0/data_interfaces.py
new file mode 100644
index 0000000..b3da5aa
--- /dev/null
+++ b/installers/charm/osm-mon/lib/charms/data_platform_libs/v0/data_interfaces.py
@@ -0,0 +1,1130 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Library to manage the relation for the data-platform products.
+
+This library contains the Requires and Provides classes for handling the relation
+between an application and multiple managed application supported by the data-team:
+MySQL, Postgresql, MongoDB, Redis, and Kakfa.
+
+### Database (MySQL, Postgresql, MongoDB, and Redis)
+
+#### Requires Charm
+This library is a uniform interface to a selection of common database
+metadata, with added custom events that add convenience to database management,
+and methods to consume the application related data.
+
+
+Following an example of using the DatabaseCreatedEvent, in the context of the
+application charm code:
+
+```python
+
+from charms.data_platform_libs.v0.data_interfaces import (
+ DatabaseCreatedEvent,
+ DatabaseRequires,
+)
+
+class ApplicationCharm(CharmBase):
+ # Application charm that connects to database charms.
+
+ def __init__(self, *args):
+ super().__init__(*args)
+
+ # Charm events defined in the database requires charm library.
+ self.database = DatabaseRequires(self, relation_name="database", database_name="database")
+ self.framework.observe(self.database.on.database_created, self._on_database_created)
+
+ def _on_database_created(self, event: DatabaseCreatedEvent) -> None:
+ # Handle the created database
+
+ # Create configuration file for app
+ config_file = self._render_app_config_file(
+ event.username,
+ event.password,
+ event.endpoints,
+ )
+
+ # Start application with rendered configuration
+ self._start_application(config_file)
+
+ # Set active status
+ self.unit.status = ActiveStatus("received database credentials")
+```
+
+As shown above, the library provides some custom events to handle specific situations,
+which are listed below:
+
+- database_created: event emitted when the requested database is created.
+- endpoints_changed: event emitted when the read/write endpoints of the database have changed.
+- read_only_endpoints_changed: event emitted when the read-only endpoints of the database
+ have changed. Event is not triggered if read/write endpoints changed too.
+
+If it is needed to connect multiple database clusters to the same relation endpoint
+the application charm can implement the same code as if it would connect to only
+one database cluster (like the above code example).
+
+To differentiate multiple clusters connected to the same relation endpoint
+the application charm can use the name of the remote application:
+
+```python
+
+def _on_database_created(self, event: DatabaseCreatedEvent) -> None:
+ # Get the remote app name of the cluster that triggered this event
+ cluster = event.relation.app.name
+```
+
+It is also possible to provide an alias for each different database cluster/relation.
+
+So, it is possible to differentiate the clusters in two ways.
+The first is to use the remote application name, i.e., `event.relation.app.name`, as above.
+
+The second way is to use different event handlers to handle each cluster events.
+The implementation would be something like the following code:
+
+```python
+
+from charms.data_platform_libs.v0.data_interfaces import (
+ DatabaseCreatedEvent,
+ DatabaseRequires,
+)
+
+class ApplicationCharm(CharmBase):
+ # Application charm that connects to database charms.
+
+ def __init__(self, *args):
+ super().__init__(*args)
+
+ # Define the cluster aliases and one handler for each cluster database created event.
+ self.database = DatabaseRequires(
+ self,
+ relation_name="database",
+ database_name="database",
+ relations_aliases = ["cluster1", "cluster2"],
+ )
+ self.framework.observe(
+ self.database.on.cluster1_database_created, self._on_cluster1_database_created
+ )
+ self.framework.observe(
+ self.database.on.cluster2_database_created, self._on_cluster2_database_created
+ )
+
+ def _on_cluster1_database_created(self, event: DatabaseCreatedEvent) -> None:
+ # Handle the created database on the cluster named cluster1
+
+ # Create configuration file for app
+ config_file = self._render_app_config_file(
+ event.username,
+ event.password,
+ event.endpoints,
+ )
+ ...
+
+ def _on_cluster2_database_created(self, event: DatabaseCreatedEvent) -> None:
+ # Handle the created database on the cluster named cluster2
+
+ # Create configuration file for app
+ config_file = self._render_app_config_file(
+ event.username,
+ event.password,
+ event.endpoints,
+ )
+ ...
+
+```
+
+### Provider Charm
+
+Following an example of using the DatabaseRequestedEvent, in the context of the
+database charm code:
+
+```python
+from charms.data_platform_libs.v0.data_interfaces import DatabaseProvides
+
+class SampleCharm(CharmBase):
+
+ def __init__(self, *args):
+ super().__init__(*args)
+ # Charm events defined in the database provides charm library.
+ self.provided_database = DatabaseProvides(self, relation_name="database")
+ self.framework.observe(self.provided_database.on.database_requested,
+ self._on_database_requested)
+ # Database generic helper
+ self.database = DatabaseHelper()
+
+ def _on_database_requested(self, event: DatabaseRequestedEvent) -> None:
+ # Handle the event triggered by a new database requested in the relation
+ # Retrieve the database name using the charm library.
+ db_name = event.database
+ # generate a new user credential
+ username = self.database.generate_user()
+ password = self.database.generate_password()
+ # set the credentials for the relation
+ self.provided_database.set_credentials(event.relation.id, username, password)
+ # set other variables for the relation event.set_tls("False")
+```
+As shown above, the library provides a custom event (database_requested) to handle
+the situation when an application charm requests a new database to be created.
+It's preferred to subscribe to this event instead of relation changed event to avoid
+creating a new database when other information other than a database name is
+exchanged in the relation databag.
+
+### Kafka
+
+This library is the interface to use and interact with the Kafka charm. This library contains
+custom events that add convenience to manage Kafka, and provides methods to consume the
+application related data.
+
+#### Requirer Charm
+
+```python
+
+from charms.data_platform_libs.v0.data_interfaces import (
+ BootstrapServerChangedEvent,
+ KafkaRequires,
+ TopicCreatedEvent,
+)
+
+class ApplicationCharm(CharmBase):
+
+ def __init__(self, *args):
+ super().__init__(*args)
+ self.kafka = KafkaRequires(self, "kafka_client", "test-topic")
+ self.framework.observe(
+ self.kafka.on.bootstrap_server_changed, self._on_kafka_bootstrap_server_changed
+ )
+ self.framework.observe(
+ self.kafka.on.topic_created, self._on_kafka_topic_created
+ )
+
+ def _on_kafka_bootstrap_server_changed(self, event: BootstrapServerChangedEvent):
+ # Event triggered when a bootstrap server was changed for this application
+
+ new_bootstrap_server = event.bootstrap_server
+ ...
+
+ def _on_kafka_topic_created(self, event: TopicCreatedEvent):
+ # Event triggered when a topic was created for this application
+ username = event.username
+ password = event.password
+ tls = event.tls
+ tls_ca= event.tls_ca
+ bootstrap_server event.bootstrap_server
+ consumer_group_prefic = event.consumer_group_prefix
+ zookeeper_uris = event.zookeeper_uris
+ ...
+
+```
+
+As shown above, the library provides some custom events to handle specific situations,
+which are listed below:
+
+- topic_created: event emitted when the requested topic is created.
+- bootstrap_server_changed: event emitted when the bootstrap server have changed.
+- credential_changed: event emitted when the credentials of Kafka changed.
+
+### Provider Charm
+
+Following the previous example, this is an example of the provider charm.
+
+```python
+class SampleCharm(CharmBase):
+
+from charms.data_platform_libs.v0.data_interfaces import (
+ KafkaProvides,
+ TopicRequestedEvent,
+)
+
+ def __init__(self, *args):
+ super().__init__(*args)
+
+ # Default charm events.
+ self.framework.observe(self.on.start, self._on_start)
+
+ # Charm events defined in the Kafka Provides charm library.
+ self.kafka_provider = KafkaProvides(self, relation_name="kafka_client")
+ self.framework.observe(self.kafka_provider.on.topic_requested, self._on_topic_requested)
+ # Kafka generic helper
+ self.kafka = KafkaHelper()
+
+ def _on_topic_requested(self, event: TopicRequestedEvent):
+ # Handle the on_topic_requested event.
+
+ topic = event.topic
+ relation_id = event.relation.id
+ # set connection info in the databag relation
+ self.kafka_provider.set_bootstrap_server(relation_id, self.kafka.get_bootstrap_server())
+ self.kafka_provider.set_credentials(relation_id, username=username, password=password)
+ self.kafka_provider.set_consumer_group_prefix(relation_id, ...)
+ self.kafka_provider.set_tls(relation_id, "False")
+ self.kafka_provider.set_zookeeper_uris(relation_id, ...)
+
+```
+As shown above, the library provides a custom event (topic_requested) to handle
+the situation when an application charm requests a new topic to be created.
+It is preferred to subscribe to this event instead of relation changed event to avoid
+creating a new topic when other information other than a topic name is
+exchanged in the relation databag.
+"""
+
+import json
+import logging
+from abc import ABC, abstractmethod
+from collections import namedtuple
+from datetime import datetime
+from typing import List, Optional
+
+from ops.charm import (
+ CharmBase,
+ CharmEvents,
+ RelationChangedEvent,
+ RelationEvent,
+ RelationJoinedEvent,
+)
+from ops.framework import EventSource, Object
+from ops.model import Relation
+
+# The unique Charmhub library identifier, never change it
+LIBID = "6c3e6b6680d64e9c89e611d1a15f65be"
+
+# Increment this major API version when introducing breaking changes
+LIBAPI = 0
+
+# Increment this PATCH version before using `charmcraft publish-lib` or reset
+# to 0 if you are raising the major API version
+LIBPATCH = 7
+
+PYDEPS = ["ops>=2.0.0"]
+
+logger = logging.getLogger(__name__)
+
+Diff = namedtuple("Diff", "added changed deleted")
+Diff.__doc__ = """
+A tuple for storing the diff between two data mappings.
+
+added - keys that were added
+changed - keys that still exist but have new values
+deleted - key that were deleted"""
+
+
+def diff(event: RelationChangedEvent, bucket: str) -> Diff:
+ """Retrieves the diff of the data in the relation changed databag.
+
+ Args:
+ event: relation changed event.
+ bucket: bucket of the databag (app or unit)
+
+ Returns:
+ a Diff instance containing the added, deleted and changed
+ keys from the event relation databag.
+ """
+ # Retrieve the old data from the data key in the application relation databag.
+ old_data = json.loads(event.relation.data[bucket].get("data", "{}"))
+ # Retrieve the new data from the event relation databag.
+ new_data = {
+ key: value for key, value in event.relation.data[event.app].items() if key != "data"
+ }
+
+ # These are the keys that were added to the databag and triggered this event.
+ added = new_data.keys() - old_data.keys()
+ # These are the keys that were removed from the databag and triggered this event.
+ deleted = old_data.keys() - new_data.keys()
+ # These are the keys that already existed in the databag,
+ # but had their values changed.
+ changed = {key for key in old_data.keys() & new_data.keys() if old_data[key] != new_data[key]}
+ # Convert the new_data to a serializable format and save it for a next diff check.
+ event.relation.data[bucket].update({"data": json.dumps(new_data)})
+
+ # Return the diff with all possible changes.
+ return Diff(added, changed, deleted)
+
+
+# Base DataProvides and DataRequires
+
+
+class DataProvides(Object, ABC):
+ """Base provides-side of the data products relation."""
+
+ def __init__(self, charm: CharmBase, relation_name: str) -> None:
+ super().__init__(charm, relation_name)
+ self.charm = charm
+ self.local_app = self.charm.model.app
+ self.local_unit = self.charm.unit
+ self.relation_name = relation_name
+ self.framework.observe(
+ charm.on[relation_name].relation_changed,
+ self._on_relation_changed,
+ )
+
+ def _diff(self, event: RelationChangedEvent) -> Diff:
+ """Retrieves the diff of the data in the relation changed databag.
+
+ Args:
+ event: relation changed event.
+
+ Returns:
+ a Diff instance containing the added, deleted and changed
+ keys from the event relation databag.
+ """
+ return diff(event, self.local_app)
+
+ @abstractmethod
+ def _on_relation_changed(self, event: RelationChangedEvent) -> None:
+ """Event emitted when the relation data has changed."""
+ raise NotImplementedError
+
+ def fetch_relation_data(self) -> dict:
+ """Retrieves data from relation.
+
+ This function can be used to retrieve data from a relation
+ in the charm code when outside an event callback.
+
+ Returns:
+ a dict of the values stored in the relation data bag
+ for all relation instances (indexed by the relation id).
+ """
+ data = {}
+ for relation in self.relations:
+ data[relation.id] = {
+ key: value for key, value in relation.data[relation.app].items() if key != "data"
+ }
+ return data
+
+ def _update_relation_data(self, relation_id: int, data: dict) -> None:
+ """Updates a set of key-value pairs in the relation.
+
+ This function writes in the application data bag, therefore,
+ only the leader unit can call it.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ data: dict containing the key-value pairs
+ that should be updated in the relation.
+ """
+ if self.local_unit.is_leader():
+ relation = self.charm.model.get_relation(self.relation_name, relation_id)
+ relation.data[self.local_app].update(data)
+
+ @property
+ def relations(self) -> List[Relation]:
+ """The list of Relation instances associated with this relation_name."""
+ return list(self.charm.model.relations[self.relation_name])
+
+ def set_credentials(self, relation_id: int, username: str, password: str) -> None:
+ """Set credentials.
+
+ This function writes in the application data bag, therefore,
+ only the leader unit can call it.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ username: user that was created.
+ password: password of the created user.
+ """
+ self._update_relation_data(
+ relation_id,
+ {
+ "username": username,
+ "password": password,
+ },
+ )
+
+ def set_tls(self, relation_id: int, tls: str) -> None:
+ """Set whether TLS is enabled.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ tls: whether tls is enabled (True or False).
+ """
+ self._update_relation_data(relation_id, {"tls": tls})
+
+ def set_tls_ca(self, relation_id: int, tls_ca: str) -> None:
+ """Set the TLS CA in the application relation databag.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ tls_ca: TLS certification authority.
+ """
+ self._update_relation_data(relation_id, {"tls_ca": tls_ca})
+
+
+class DataRequires(Object, ABC):
+ """Requires-side of the relation."""
+
+ def __init__(
+ self,
+ charm,
+ relation_name: str,
+ extra_user_roles: str = None,
+ ):
+ """Manager of base client relations."""
+ super().__init__(charm, relation_name)
+ self.charm = charm
+ self.extra_user_roles = extra_user_roles
+ self.local_app = self.charm.model.app
+ self.local_unit = self.charm.unit
+ self.relation_name = relation_name
+ self.framework.observe(
+ self.charm.on[relation_name].relation_joined, self._on_relation_joined_event
+ )
+ self.framework.observe(
+ self.charm.on[relation_name].relation_changed, self._on_relation_changed_event
+ )
+
+ @abstractmethod
+ def _on_relation_joined_event(self, event: RelationJoinedEvent) -> None:
+ """Event emitted when the application joins the relation."""
+ raise NotImplementedError
+
+ @abstractmethod
+ def _on_relation_changed_event(self, event: RelationChangedEvent) -> None:
+ raise NotImplementedError
+
+ def fetch_relation_data(self) -> dict:
+ """Retrieves data from relation.
+
+ This function can be used to retrieve data from a relation
+ in the charm code when outside an event callback.
+ Function cannot be used in `*-relation-broken` events and will raise an exception.
+
+ Returns:
+ a dict of the values stored in the relation data bag
+ for all relation instances (indexed by the relation ID).
+ """
+ data = {}
+ for relation in self.relations:
+ data[relation.id] = {
+ key: value for key, value in relation.data[relation.app].items() if key != "data"
+ }
+ return data
+
+ def _update_relation_data(self, relation_id: int, data: dict) -> None:
+ """Updates a set of key-value pairs in the relation.
+
+ This function writes in the application data bag, therefore,
+ only the leader unit can call it.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ data: dict containing the key-value pairs
+ that should be updated in the relation.
+ """
+ if self.local_unit.is_leader():
+ relation = self.charm.model.get_relation(self.relation_name, relation_id)
+ relation.data[self.local_app].update(data)
+
+ def _diff(self, event: RelationChangedEvent) -> Diff:
+ """Retrieves the diff of the data in the relation changed databag.
+
+ Args:
+ event: relation changed event.
+
+ Returns:
+ a Diff instance containing the added, deleted and changed
+ keys from the event relation databag.
+ """
+ return diff(event, self.local_unit)
+
+ @property
+ def relations(self) -> List[Relation]:
+ """The list of Relation instances associated with this relation_name."""
+ return [
+ relation
+ for relation in self.charm.model.relations[self.relation_name]
+ if self._is_relation_active(relation)
+ ]
+
+ @staticmethod
+ def _is_relation_active(relation: Relation):
+ try:
+ _ = repr(relation.data)
+ return True
+ except RuntimeError:
+ return False
+
+ @staticmethod
+ def _is_resource_created_for_relation(relation: Relation):
+ return (
+ "username" in relation.data[relation.app] and "password" in relation.data[relation.app]
+ )
+
+ def is_resource_created(self, relation_id: Optional[int] = None) -> bool:
+ """Check if the resource has been created.
+
+ This function can be used to check if the Provider answered with data in the charm code
+ when outside an event callback.
+
+ Args:
+ relation_id (int, optional): When provided the check is done only for the relation id
+ provided, otherwise the check is done for all relations
+
+ Returns:
+ True or False
+
+ Raises:
+ IndexError: If relation_id is provided but that relation does not exist
+ """
+ if relation_id is not None:
+ try:
+ relation = [relation for relation in self.relations if relation.id == relation_id][
+ 0
+ ]
+ return self._is_resource_created_for_relation(relation)
+ except IndexError:
+ raise IndexError(f"relation id {relation_id} cannot be accessed")
+ else:
+ return (
+ all(
+ [
+ self._is_resource_created_for_relation(relation)
+ for relation in self.relations
+ ]
+ )
+ if self.relations
+ else False
+ )
+
+
+# General events
+
+
+class ExtraRoleEvent(RelationEvent):
+ """Base class for data events."""
+
+ @property
+ def extra_user_roles(self) -> Optional[str]:
+ """Returns the extra user roles that were requested."""
+ return self.relation.data[self.relation.app].get("extra-user-roles")
+
+
+class AuthenticationEvent(RelationEvent):
+ """Base class for authentication fields for events."""
+
+ @property
+ def username(self) -> Optional[str]:
+ """Returns the created username."""
+ return self.relation.data[self.relation.app].get("username")
+
+ @property
+ def password(self) -> Optional[str]:
+ """Returns the password for the created user."""
+ return self.relation.data[self.relation.app].get("password")
+
+ @property
+ def tls(self) -> Optional[str]:
+ """Returns whether TLS is configured."""
+ return self.relation.data[self.relation.app].get("tls")
+
+ @property
+ def tls_ca(self) -> Optional[str]:
+ """Returns TLS CA."""
+ return self.relation.data[self.relation.app].get("tls-ca")
+
+
+# Database related events and fields
+
+
+class DatabaseProvidesEvent(RelationEvent):
+ """Base class for database events."""
+
+ @property
+ def database(self) -> Optional[str]:
+ """Returns the database that was requested."""
+ return self.relation.data[self.relation.app].get("database")
+
+
+class DatabaseRequestedEvent(DatabaseProvidesEvent, ExtraRoleEvent):
+ """Event emitted when a new database is requested for use on this relation."""
+
+
+class DatabaseProvidesEvents(CharmEvents):
+ """Database events.
+
+ This class defines the events that the database can emit.
+ """
+
+ database_requested = EventSource(DatabaseRequestedEvent)
+
+
+class DatabaseRequiresEvent(RelationEvent):
+ """Base class for database events."""
+
+ @property
+ def endpoints(self) -> Optional[str]:
+ """Returns a comma separated list of read/write endpoints."""
+ return self.relation.data[self.relation.app].get("endpoints")
+
+ @property
+ def read_only_endpoints(self) -> Optional[str]:
+ """Returns a comma separated list of read only endpoints."""
+ return self.relation.data[self.relation.app].get("read-only-endpoints")
+
+ @property
+ def replset(self) -> Optional[str]:
+ """Returns the replicaset name.
+
+ MongoDB only.
+ """
+ return self.relation.data[self.relation.app].get("replset")
+
+ @property
+ def uris(self) -> Optional[str]:
+ """Returns the connection URIs.
+
+ MongoDB, Redis, OpenSearch.
+ """
+ return self.relation.data[self.relation.app].get("uris")
+
+ @property
+ def version(self) -> Optional[str]:
+ """Returns the version of the database.
+
+ Version as informed by the database daemon.
+ """
+ return self.relation.data[self.relation.app].get("version")
+
+
+class DatabaseCreatedEvent(AuthenticationEvent, DatabaseRequiresEvent):
+ """Event emitted when a new database is created for use on this relation."""
+
+
+class DatabaseEndpointsChangedEvent(AuthenticationEvent, DatabaseRequiresEvent):
+ """Event emitted when the read/write endpoints are changed."""
+
+
+class DatabaseReadOnlyEndpointsChangedEvent(AuthenticationEvent, DatabaseRequiresEvent):
+ """Event emitted when the read only endpoints are changed."""
+
+
+class DatabaseRequiresEvents(CharmEvents):
+ """Database events.
+
+ This class defines the events that the database can emit.
+ """
+
+ database_created = EventSource(DatabaseCreatedEvent)
+ endpoints_changed = EventSource(DatabaseEndpointsChangedEvent)
+ read_only_endpoints_changed = EventSource(DatabaseReadOnlyEndpointsChangedEvent)
+
+
+# Database Provider and Requires
+
+
+class DatabaseProvides(DataProvides):
+ """Provider-side of the database relations."""
+
+ on = DatabaseProvidesEvents()
+
+ def __init__(self, charm: CharmBase, relation_name: str) -> None:
+ super().__init__(charm, relation_name)
+
+ def _on_relation_changed(self, event: RelationChangedEvent) -> None:
+ """Event emitted when the relation has changed."""
+ # Only the leader should handle this event.
+ if not self.local_unit.is_leader():
+ return
+
+ # Check which data has changed to emit customs events.
+ diff = self._diff(event)
+
+ # Emit a database requested event if the setup key (database name and optional
+ # extra user roles) was added to the relation databag by the application.
+ if "database" in diff.added:
+ self.on.database_requested.emit(event.relation, app=event.app, unit=event.unit)
+
+ def set_endpoints(self, relation_id: int, connection_strings: str) -> None:
+ """Set database primary connections.
+
+ This function writes in the application data bag, therefore,
+ only the leader unit can call it.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ connection_strings: database hosts and ports comma separated list.
+ """
+ self._update_relation_data(relation_id, {"endpoints": connection_strings})
+
+ def set_read_only_endpoints(self, relation_id: int, connection_strings: str) -> None:
+ """Set database replicas connection strings.
+
+ This function writes in the application data bag, therefore,
+ only the leader unit can call it.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ connection_strings: database hosts and ports comma separated list.
+ """
+ self._update_relation_data(relation_id, {"read-only-endpoints": connection_strings})
+
+ def set_replset(self, relation_id: int, replset: str) -> None:
+ """Set replica set name in the application relation databag.
+
+ MongoDB only.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ replset: replica set name.
+ """
+ self._update_relation_data(relation_id, {"replset": replset})
+
+ def set_uris(self, relation_id: int, uris: str) -> None:
+ """Set the database connection URIs in the application relation databag.
+
+ MongoDB, Redis, and OpenSearch only.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ uris: connection URIs.
+ """
+ self._update_relation_data(relation_id, {"uris": uris})
+
+ def set_version(self, relation_id: int, version: str) -> None:
+ """Set the database version in the application relation databag.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ version: database version.
+ """
+ self._update_relation_data(relation_id, {"version": version})
+
+
+class DatabaseRequires(DataRequires):
+ """Requires-side of the database relation."""
+
+ on = DatabaseRequiresEvents()
+
+ def __init__(
+ self,
+ charm,
+ relation_name: str,
+ database_name: str,
+ extra_user_roles: str = None,
+ relations_aliases: List[str] = None,
+ ):
+ """Manager of database client relations."""
+ super().__init__(charm, relation_name, extra_user_roles)
+ self.database = database_name
+ self.relations_aliases = relations_aliases
+
+ # Define custom event names for each alias.
+ if relations_aliases:
+ # Ensure the number of aliases does not exceed the maximum
+ # of connections allowed in the specific relation.
+ relation_connection_limit = self.charm.meta.requires[relation_name].limit
+ if len(relations_aliases) != relation_connection_limit:
+ raise ValueError(
+ f"The number of aliases must match the maximum number of connections allowed in the relation. "
+ f"Expected {relation_connection_limit}, got {len(relations_aliases)}"
+ )
+
+ for relation_alias in relations_aliases:
+ self.on.define_event(f"{relation_alias}_database_created", DatabaseCreatedEvent)
+ self.on.define_event(
+ f"{relation_alias}_endpoints_changed", DatabaseEndpointsChangedEvent
+ )
+ self.on.define_event(
+ f"{relation_alias}_read_only_endpoints_changed",
+ DatabaseReadOnlyEndpointsChangedEvent,
+ )
+
+ def _assign_relation_alias(self, relation_id: int) -> None:
+ """Assigns an alias to a relation.
+
+ This function writes in the unit data bag.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ """
+ # If no aliases were provided, return immediately.
+ if not self.relations_aliases:
+ return
+
+ # Return if an alias was already assigned to this relation
+ # (like when there are more than one unit joining the relation).
+ if (
+ self.charm.model.get_relation(self.relation_name, relation_id)
+ .data[self.local_unit]
+ .get("alias")
+ ):
+ return
+
+ # Retrieve the available aliases (the ones that weren't assigned to any relation).
+ available_aliases = self.relations_aliases[:]
+ for relation in self.charm.model.relations[self.relation_name]:
+ alias = relation.data[self.local_unit].get("alias")
+ if alias:
+ logger.debug("Alias %s was already assigned to relation %d", alias, relation.id)
+ available_aliases.remove(alias)
+
+ # Set the alias in the unit relation databag of the specific relation.
+ relation = self.charm.model.get_relation(self.relation_name, relation_id)
+ relation.data[self.local_unit].update({"alias": available_aliases[0]})
+
+ def _emit_aliased_event(self, event: RelationChangedEvent, event_name: str) -> None:
+ """Emit an aliased event to a particular relation if it has an alias.
+
+ Args:
+ event: the relation changed event that was received.
+ event_name: the name of the event to emit.
+ """
+ alias = self._get_relation_alias(event.relation.id)
+ if alias:
+ getattr(self.on, f"{alias}_{event_name}").emit(
+ event.relation, app=event.app, unit=event.unit
+ )
+
+ def _get_relation_alias(self, relation_id: int) -> Optional[str]:
+ """Returns the relation alias.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+
+ Returns:
+ the relation alias or None if the relation was not found.
+ """
+ for relation in self.charm.model.relations[self.relation_name]:
+ if relation.id == relation_id:
+ return relation.data[self.local_unit].get("alias")
+ return None
+
+ def _on_relation_joined_event(self, event: RelationJoinedEvent) -> None:
+ """Event emitted when the application joins the database relation."""
+ # If relations aliases were provided, assign one to the relation.
+ self._assign_relation_alias(event.relation.id)
+
+ # Sets both database and extra user roles in the relation
+ # if the roles are provided. Otherwise, sets only the database.
+ if self.extra_user_roles:
+ self._update_relation_data(
+ event.relation.id,
+ {
+ "database": self.database,
+ "extra-user-roles": self.extra_user_roles,
+ },
+ )
+ else:
+ self._update_relation_data(event.relation.id, {"database": self.database})
+
+ def _on_relation_changed_event(self, event: RelationChangedEvent) -> None:
+ """Event emitted when the database relation has changed."""
+ # Check which data has changed to emit customs events.
+ diff = self._diff(event)
+
+ # Check if the database is created
+ # (the database charm shared the credentials).
+ if "username" in diff.added and "password" in diff.added:
+ # Emit the default event (the one without an alias).
+ logger.info("database created at %s", datetime.now())
+ self.on.database_created.emit(event.relation, app=event.app, unit=event.unit)
+
+ # Emit the aliased event (if any).
+ self._emit_aliased_event(event, "database_created")
+
+ # To avoid unnecessary application restarts do not trigger
+ # “endpoints_changed“ event if “database_created“ is triggered.
+ return
+
+ # Emit an endpoints changed event if the database
+ # added or changed this info in the relation databag.
+ if "endpoints" in diff.added or "endpoints" in diff.changed:
+ # Emit the default event (the one without an alias).
+ logger.info("endpoints changed on %s", datetime.now())
+ self.on.endpoints_changed.emit(event.relation, app=event.app, unit=event.unit)
+
+ # Emit the aliased event (if any).
+ self._emit_aliased_event(event, "endpoints_changed")
+
+ # To avoid unnecessary application restarts do not trigger
+ # “read_only_endpoints_changed“ event if “endpoints_changed“ is triggered.
+ return
+
+ # Emit a read only endpoints changed event if the database
+ # added or changed this info in the relation databag.
+ if "read-only-endpoints" in diff.added or "read-only-endpoints" in diff.changed:
+ # Emit the default event (the one without an alias).
+ logger.info("read-only-endpoints changed on %s", datetime.now())
+ self.on.read_only_endpoints_changed.emit(
+ event.relation, app=event.app, unit=event.unit
+ )
+
+ # Emit the aliased event (if any).
+ self._emit_aliased_event(event, "read_only_endpoints_changed")
+
+
+# Kafka related events
+
+
+class KafkaProvidesEvent(RelationEvent):
+ """Base class for Kafka events."""
+
+ @property
+ def topic(self) -> Optional[str]:
+ """Returns the topic that was requested."""
+ return self.relation.data[self.relation.app].get("topic")
+
+
+class TopicRequestedEvent(KafkaProvidesEvent, ExtraRoleEvent):
+ """Event emitted when a new topic is requested for use on this relation."""
+
+
+class KafkaProvidesEvents(CharmEvents):
+ """Kafka events.
+
+ This class defines the events that the Kafka can emit.
+ """
+
+ topic_requested = EventSource(TopicRequestedEvent)
+
+
+class KafkaRequiresEvent(RelationEvent):
+ """Base class for Kafka events."""
+
+ @property
+ def bootstrap_server(self) -> Optional[str]:
+ """Returns a a comma-seperated list of broker uris."""
+ return self.relation.data[self.relation.app].get("endpoints")
+
+ @property
+ def consumer_group_prefix(self) -> Optional[str]:
+ """Returns the consumer-group-prefix."""
+ return self.relation.data[self.relation.app].get("consumer-group-prefix")
+
+ @property
+ def zookeeper_uris(self) -> Optional[str]:
+ """Returns a comma separated list of Zookeeper uris."""
+ return self.relation.data[self.relation.app].get("zookeeper-uris")
+
+
+class TopicCreatedEvent(AuthenticationEvent, KafkaRequiresEvent):
+ """Event emitted when a new topic is created for use on this relation."""
+
+
+class BootstrapServerChangedEvent(AuthenticationEvent, KafkaRequiresEvent):
+ """Event emitted when the bootstrap server is changed."""
+
+
+class KafkaRequiresEvents(CharmEvents):
+ """Kafka events.
+
+ This class defines the events that the Kafka can emit.
+ """
+
+ topic_created = EventSource(TopicCreatedEvent)
+ bootstrap_server_changed = EventSource(BootstrapServerChangedEvent)
+
+
+# Kafka Provides and Requires
+
+
+class KafkaProvides(DataProvides):
+ """Provider-side of the Kafka relation."""
+
+ on = KafkaProvidesEvents()
+
+ def __init__(self, charm: CharmBase, relation_name: str) -> None:
+ super().__init__(charm, relation_name)
+
+ def _on_relation_changed(self, event: RelationChangedEvent) -> None:
+ """Event emitted when the relation has changed."""
+ # Only the leader should handle this event.
+ if not self.local_unit.is_leader():
+ return
+
+ # Check which data has changed to emit customs events.
+ diff = self._diff(event)
+
+ # Emit a topic requested event if the setup key (topic name and optional
+ # extra user roles) was added to the relation databag by the application.
+ if "topic" in diff.added:
+ self.on.topic_requested.emit(event.relation, app=event.app, unit=event.unit)
+
+ def set_bootstrap_server(self, relation_id: int, bootstrap_server: str) -> None:
+ """Set the bootstrap server in the application relation databag.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ bootstrap_server: the bootstrap server address.
+ """
+ self._update_relation_data(relation_id, {"endpoints": bootstrap_server})
+
+ def set_consumer_group_prefix(self, relation_id: int, consumer_group_prefix: str) -> None:
+ """Set the consumer group prefix in the application relation databag.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ consumer_group_prefix: the consumer group prefix string.
+ """
+ self._update_relation_data(relation_id, {"consumer-group-prefix": consumer_group_prefix})
+
+ def set_zookeeper_uris(self, relation_id: int, zookeeper_uris: str) -> None:
+ """Set the zookeeper uris in the application relation databag.
+
+ Args:
+ relation_id: the identifier for a particular relation.
+ zookeeper_uris: comma-seperated list of ZooKeeper server uris.
+ """
+ self._update_relation_data(relation_id, {"zookeeper-uris": zookeeper_uris})
+
+
+class KafkaRequires(DataRequires):
+ """Requires-side of the Kafka relation."""
+
+ on = KafkaRequiresEvents()
+
+ def __init__(self, charm, relation_name: str, topic: str, extra_user_roles: str = None):
+ """Manager of Kafka client relations."""
+ # super().__init__(charm, relation_name)
+ super().__init__(charm, relation_name, extra_user_roles)
+ self.charm = charm
+ self.topic = topic
+
+ def _on_relation_joined_event(self, event: RelationJoinedEvent) -> None:
+ """Event emitted when the application joins the Kafka relation."""
+ # Sets both topic and extra user roles in the relation
+ # if the roles are provided. Otherwise, sets only the topic.
+ self._update_relation_data(
+ event.relation.id,
+ {
+ "topic": self.topic,
+ "extra-user-roles": self.extra_user_roles,
+ }
+ if self.extra_user_roles is not None
+ else {"topic": self.topic},
+ )
+
+ def _on_relation_changed_event(self, event: RelationChangedEvent) -> None:
+ """Event emitted when the Kafka relation has changed."""
+ # Check which data has changed to emit customs events.
+ diff = self._diff(event)
+
+ # Check if the topic is created
+ # (the Kafka charm shared the credentials).
+ if "username" in diff.added and "password" in diff.added:
+ # Emit the default event (the one without an alias).
+ logger.info("topic created at %s", datetime.now())
+ self.on.topic_created.emit(event.relation, app=event.app, unit=event.unit)
+
+ # To avoid unnecessary application restarts do not trigger
+ # “endpoints_changed“ event if “topic_created“ is triggered.
+ return
+
+ # Emit an endpoints (bootstap-server) changed event if the Kakfa endpoints
+ # added or changed this info in the relation databag.
+ if "endpoints" in diff.added or "endpoints" in diff.changed:
+ # Emit the default event (the one without an alias).
+ logger.info("endpoints changed on %s", datetime.now())
+ self.on.bootstrap_server_changed.emit(
+ event.relation, app=event.app, unit=event.unit
+ ) # here check if this is the right design
+ return
diff --git a/installers/charm/osm-mon/metadata.yaml b/installers/charm/osm-mon/metadata.yaml
index ee2f2f9..5bd1236 100644
--- a/installers/charm/osm-mon/metadata.yaml
+++ b/installers/charm/osm-mon/metadata.yaml
@@ -58,7 +58,7 @@
interface: kafka
limit: 1
mongodb:
- interface: mongodb
+ interface: mongodb_client
limit: 1
keystone:
interface: keystone
diff --git a/installers/charm/osm-mon/pyproject.toml b/installers/charm/osm-mon/pyproject.toml
index d0d4a5b..16cf0f4 100644
--- a/installers/charm/osm-mon/pyproject.toml
+++ b/installers/charm/osm-mon/pyproject.toml
@@ -50,7 +50,3 @@
# D100, D101, D102, D103: Ignore missing docstrings in tests
per-file-ignores = ["tests/*:D100,D101,D102,D103,D104"]
docstring-convention = "google"
-# Check for properly formatted copyright header in each file
-copyright-check = "True"
-copyright-author = "Canonical Ltd."
-copyright-regexp = "Copyright\\s\\d{4}([-,]\\d{4})*\\s+%(author)s"
diff --git a/installers/charm/osm-mon/requirements.txt b/installers/charm/osm-mon/requirements.txt
index cb303a3..398d4ad 100644
--- a/installers/charm/osm-mon/requirements.txt
+++ b/installers/charm/osm-mon/requirements.txt
@@ -17,7 +17,7 @@
#
# To get in touch with the maintainers, please contact:
# osm-charmers@lists.launchpad.net
-ops >= 1.2.0
+ops < 2.2
lightkube
lightkube-models
# git+https://github.com/charmed-osm/config-validator/
diff --git a/installers/charm/osm-mon/src/charm.py b/installers/charm/osm-mon/src/charm.py
index 176f896..db72dfe 100755
--- a/installers/charm/osm-mon/src/charm.py
+++ b/installers/charm/osm-mon/src/charm.py
@@ -22,7 +22,7 @@
#
# Learn more at: https://juju.is/docs/sdk
-"""OSM NBI charm.
+"""OSM MON charm.
See more: https://charmhub.io/osm
"""
@@ -30,6 +30,7 @@
import logging
from typing import Any, Dict
+from charms.data_platform_libs.v0.data_interfaces import DatabaseRequires
from charms.kafka_k8s.v0.kafka import KafkaRequires, _KafkaAvailableEvent
from charms.observability_libs.v1.kubernetes_service_patch import KubernetesServicePatch
from charms.osm_libs.v0.utils import (
@@ -46,7 +47,7 @@
from ops.main import main
from ops.model import ActiveStatus, Container
-from legacy_interfaces import KeystoneClient, MongoClient, PrometheusClient
+from legacy_interfaces import KeystoneClient, PrometheusClient
HOSTPATHS = [
HostPath(
@@ -85,7 +86,7 @@
def __init__(self, *args):
super().__init__(*args)
self.kafka = KafkaRequires(self)
- self.mongodb_client = MongoClient(self, "mongodb")
+ self.mongodb_client = DatabaseRequires(self, "mongodb", database_name="osm")
self.prometheus_client = PrometheusClient(self, "prometheus")
self.keystone_client = KeystoneClient(self, "keystone")
self.vca = VcaRequires(self)
@@ -151,9 +152,7 @@
def _on_get_debug_mode_information_action(self, event: ActionEvent) -> None:
"""Handler for the get-debug-mode-information action event."""
if not self.debug_mode.started:
- event.fail(
- "debug-mode has not started. Hint: juju config mon debug-mode=true"
- )
+ event.fail("debug-mode has not started. Hint: juju config mon debug-mode=true")
return
debug_info = {
@@ -176,20 +175,24 @@
self.on.vca_data_changed: self._on_config_changed,
self.on.kafka_available: self._on_config_changed,
self.on["kafka"].relation_broken: self._on_required_relation_broken,
+ self.mongodb_client.on.database_created: self._on_config_changed,
+ self.on["mongodb"].relation_broken: self._on_required_relation_broken,
# Action events
self.on.get_debug_mode_information_action: self._on_get_debug_mode_information_action,
}
- for relation in [
- self.on[rel_name] for rel_name in ["mongodb", "prometheus", "keystone"]
- ]:
+ for relation in [self.on[rel_name] for rel_name in ["prometheus", "keystone"]]:
event_handler_mapping[relation.relation_changed] = self._on_config_changed
- event_handler_mapping[
- relation.relation_broken
- ] = self._on_required_relation_broken
+ event_handler_mapping[relation.relation_broken] = self._on_required_relation_broken
for event, handler in event_handler_mapping.items():
self.framework.observe(event, handler)
+ def _is_database_available(self) -> bool:
+ try:
+ return self.mongodb_client.is_resource_created()
+ except KeyError:
+ return False
+
def _validate_config(self) -> None:
"""Validate charm configuration.
@@ -209,7 +212,7 @@
if not self.kafka.host or not self.kafka.port:
missing_relations.append("kafka")
- if self.mongodb_client.is_missing_data_in_unit():
+ if not self._is_database_available():
missing_relations.append("mongodb")
if self.prometheus_client.is_missing_data_in_app():
missing_relations.append("prometheus")
@@ -219,9 +222,7 @@
if missing_relations:
relations_str = ", ".join(missing_relations)
one_relation_missing = len(missing_relations) == 1
- error_msg = (
- f'need {relations_str} relation{"" if one_relation_missing else "s"}'
- )
+ error_msg = f'need {relations_str} relation{"" if one_relation_missing else "s"}'
logger.warning(error_msg)
raise CharmError(error_msg)
@@ -236,9 +237,7 @@
environment = {
# General configuration
"OSMMON_GLOBAL_LOGLEVEL": self.config["log-level"],
- "OSMMON_OPENSTACK_DEFAULT_GRANULARITY": self.config[
- "openstack-default-granularity"
- ],
+ "OSMMON_OPENSTACK_DEFAULT_GRANULARITY": self.config["openstack-default-granularity"],
"OSMMON_GLOBAL_REQUEST_TIMEOUT": self.config["global-request-timeout"],
"OSMMON_COLLECTOR_INTERVAL": self.config["collector-interval"],
"OSMMON_EVALUATOR_INTERVAL": self.config["evaluator-interval"],
@@ -249,7 +248,7 @@
"OSMMON_MESSAGE_PORT": self.kafka.port,
# Database configuration
"OSMMON_DATABASE_DRIVER": "mongo",
- "OSMMON_DATABASE_URI": self.mongodb_client.connection_string,
+ "OSMMON_DATABASE_URI": self._get_mongodb_uri(),
"OSMMON_DATABASE_COMMONKEY": self.config["database-commonkey"],
# Prometheus/grafana configuration
"OSMMON_PROMETHEUS_URL": f"http://{self.prometheus_client.hostname}:{self.prometheus_client.port}",
@@ -288,6 +287,9 @@
},
}
+ def _get_mongodb_uri(self):
+ return list(self.mongodb_client.fetch_relation_data().values())[0]["uris"]
+
def _patch_k8s_service(self) -> None:
port = ServicePort(SERVICE_PORT, name=f"{self.app.name}")
self.service_patcher = KubernetesServicePatch(self, [port])
diff --git a/installers/charm/osm-mon/tests/integration/test_charm.py b/installers/charm/osm-mon/tests/integration/test_charm.py
new file mode 100644
index 0000000..c5807e9
--- /dev/null
+++ b/installers/charm/osm-mon/tests/integration/test_charm.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python3
+# Copyright 2022 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# For those usages not covered by the Apache License, Version 2.0 please
+# contact: legal@canonical.com
+#
+# To get in touch with the maintainers, please contact:
+# osm-charmers@lists.launchpad.net
+#
+# Learn more about testing at: https://juju.is/docs/sdk/testing
+
+import asyncio
+import logging
+import shlex
+from pathlib import Path
+
+import pytest
+import yaml
+from pytest_operator.plugin import OpsTest
+
+logger = logging.getLogger(__name__)
+
+METADATA = yaml.safe_load(Path("./metadata.yaml").read_text())
+MON_APP = METADATA["name"]
+KAFKA_CHARM = "kafka-k8s"
+KAFKA_APP = "kafka"
+KEYSTONE_CHARM = "osm-keystone"
+KEYSTONE_APP = "keystone"
+MARIADB_CHARM = "charmed-osm-mariadb-k8s"
+MARIADB_APP = "mariadb"
+MONGO_DB_CHARM = "mongodb-k8s"
+MONGO_DB_APP = "mongodb"
+PROMETHEUS_CHARM = "osm-prometheus"
+PROMETHEUS_APP = "prometheus"
+ZOOKEEPER_CHARM = "zookeeper-k8s"
+ZOOKEEPER_APP = "zookeeper"
+VCA_CHARM = "osm-vca-integrator"
+VCA_APP = "vca"
+APPS = [KAFKA_APP, ZOOKEEPER_APP, KEYSTONE_APP, MONGO_DB_APP, MARIADB_APP, PROMETHEUS_APP, MON_APP]
+
+
+@pytest.mark.abort_on_fail
+async def test_mon_is_deployed(ops_test: OpsTest):
+ charm = await ops_test.build_charm(".")
+ resources = {"mon-image": METADATA["resources"]["mon-image"]["upstream-source"]}
+
+ await asyncio.gather(
+ ops_test.model.deploy(
+ charm, resources=resources, application_name=MON_APP, series="focal"
+ ),
+ ops_test.model.deploy(KAFKA_CHARM, application_name=KAFKA_APP, channel="stable"),
+ ops_test.model.deploy(MONGO_DB_CHARM, application_name=MONGO_DB_APP, channel="edge"),
+ ops_test.model.deploy(MARIADB_CHARM, application_name=MARIADB_APP, channel="stable"),
+ ops_test.model.deploy(PROMETHEUS_CHARM, application_name=PROMETHEUS_APP, channel="stable"),
+ ops_test.model.deploy(ZOOKEEPER_CHARM, application_name=ZOOKEEPER_APP, channel="stable"),
+ )
+ cmd = f"juju deploy {KEYSTONE_CHARM} {KEYSTONE_APP} --resource keystone-image=opensourcemano/keystone:12"
+ await ops_test.run(*shlex.split(cmd), check=True)
+
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=APPS,
+ )
+ assert ops_test.model.applications[MON_APP].status == "blocked"
+ unit = ops_test.model.applications[MON_APP].units[0]
+ assert unit.workload_status_message == "need kafka, mongodb, prometheus, keystone relations"
+
+ logger.info("Adding relations for other components")
+ await ops_test.model.add_relation(KAFKA_APP, ZOOKEEPER_APP)
+ await ops_test.model.add_relation(MARIADB_APP, KEYSTONE_APP)
+
+ logger.info("Adding relations")
+ await ops_test.model.add_relation(MON_APP, MONGO_DB_APP)
+ await ops_test.model.add_relation(MON_APP, KAFKA_APP)
+ await ops_test.model.add_relation(MON_APP, KEYSTONE_APP)
+ await ops_test.model.add_relation(MON_APP, PROMETHEUS_APP)
+
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=APPS,
+ status="active",
+ )
+
+
+@pytest.mark.abort_on_fail
+async def test_mon_scales_up(ops_test: OpsTest):
+ logger.info("Scaling up osm-mon")
+ expected_units = 3
+ assert len(ops_test.model.applications[MON_APP].units) == 1
+ await ops_test.model.applications[MON_APP].scale(expected_units)
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=[MON_APP], status="active", wait_for_exact_units=expected_units
+ )
+
+
+@pytest.mark.abort_on_fail
+@pytest.mark.parametrize(
+ "relation_to_remove", [KAFKA_APP, MONGO_DB_APP, PROMETHEUS_APP, KEYSTONE_APP]
+)
+async def test_mon_blocks_without_relation(ops_test: OpsTest, relation_to_remove):
+ logger.info("Removing relation: %s", relation_to_remove)
+ # mongoDB relation is named "database"
+ local_relation = relation_to_remove
+ if relation_to_remove == MONGO_DB_APP:
+ local_relation = "database"
+ await asyncio.gather(
+ ops_test.model.applications[relation_to_remove].remove_relation(local_relation, MON_APP)
+ )
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(apps=[MON_APP])
+ assert ops_test.model.applications[MON_APP].status == "blocked"
+ for unit in ops_test.model.applications[MON_APP].units:
+ assert unit.workload_status_message == f"need {relation_to_remove} relation"
+ await ops_test.model.add_relation(MON_APP, relation_to_remove)
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=APPS,
+ status="active",
+ )
+
+
+@pytest.mark.abort_on_fail
+async def test_mon_action_debug_mode_disabled(ops_test: OpsTest):
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=APPS,
+ status="active",
+ )
+ logger.info("Running action 'get-debug-mode-information'")
+ action = (
+ await ops_test.model.applications[MON_APP]
+ .units[0]
+ .run_action("get-debug-mode-information")
+ )
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(apps=[MON_APP])
+ status = await ops_test.model.get_action_status(uuid_or_prefix=action.entity_id)
+ assert status[action.entity_id] == "failed"
+
+
+@pytest.mark.abort_on_fail
+async def test_mon_action_debug_mode_enabled(ops_test: OpsTest):
+ await ops_test.model.applications[MON_APP].set_config({"debug-mode": "true"})
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=APPS,
+ status="active",
+ )
+ logger.info("Running action 'get-debug-mode-information'")
+ # list of units is not ordered
+ unit_id = list(
+ filter(
+ lambda x: (x.entity_id == f"{MON_APP}/0"), ops_test.model.applications[MON_APP].units
+ )
+ )[0]
+ action = await unit_id.run_action("get-debug-mode-information")
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(apps=[MON_APP])
+ status = await ops_test.model.get_action_status(uuid_or_prefix=action.entity_id)
+ message = await ops_test.model.get_action_output(action_uuid=action.entity_id)
+ assert status[action.entity_id] == "completed"
+ assert "command" in message
+ assert "password" in message
+
+
+@pytest.mark.abort_on_fail
+async def test_mon_integration_vca(ops_test: OpsTest):
+ await asyncio.gather(
+ ops_test.model.deploy(VCA_CHARM, application_name=VCA_APP, channel="beta"),
+ )
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=[VCA_APP],
+ )
+ controllers = (Path.home() / ".local/share/juju/controllers.yaml").read_text()
+ accounts = (Path.home() / ".local/share/juju/accounts.yaml").read_text()
+ public_key = (Path.home() / ".local/share/juju/ssh/juju_id_rsa.pub").read_text()
+ await ops_test.model.applications[VCA_APP].set_config(
+ {
+ "controllers": controllers,
+ "accounts": accounts,
+ "public-key": public_key,
+ "k8s-cloud": "microk8s",
+ }
+ )
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=APPS + [VCA_APP],
+ status="active",
+ )
+ await ops_test.model.add_relation(MON_APP, VCA_APP)
+ async with ops_test.fast_forward():
+ await ops_test.model.wait_for_idle(
+ apps=APPS + [VCA_APP],
+ status="active",
+ )
diff --git a/installers/charm/osm-mon/tests/unit/test_charm.py b/installers/charm/osm-mon/tests/unit/test_charm.py
index 3ea173a..33598fe 100644
--- a/installers/charm/osm-mon/tests/unit/test_charm.py
+++ b/installers/charm/osm-mon/tests/unit/test_charm.py
@@ -37,6 +37,7 @@
mocker.patch("charm.KubernetesServicePatch", lambda x, y: None)
harness = Harness(OsmMonCharm)
harness.begin()
+ harness.container_pebble_ready(container_name)
yield harness
harness.cleanup()
@@ -71,19 +72,21 @@
relation_id = harness.add_relation("mongodb", "mongodb")
harness.add_relation_unit(relation_id, "mongodb/0")
harness.update_relation_data(
- relation_id, "mongodb/0", {"connection_string": "mongodb://:1234"}
+ relation_id,
+ "mongodb",
+ {"uris": "mongodb://:1234", "username": "user", "password": "password"},
)
relation_ids.append(relation_id)
# Add kafka relation
relation_id = harness.add_relation("kafka", "kafka")
harness.add_relation_unit(relation_id, "kafka/0")
- harness.update_relation_data(relation_id, "kafka", {"host": "kafka", "port": 9092})
+ harness.update_relation_data(relation_id, "kafka", {"host": "kafka", "port": "9092"})
relation_ids.append(relation_id)
# Add prometheus relation
relation_id = harness.add_relation("prometheus", "prometheus")
harness.add_relation_unit(relation_id, "prometheus/0")
harness.update_relation_data(
- relation_id, "prometheus", {"hostname": "prometheus", "port": 9090}
+ relation_id, "prometheus", {"hostname": "prometheus", "port": "9090"}
)
relation_ids.append(relation_id)
# Add keystone relation
diff --git a/installers/charm/osm-mon/tox.ini b/installers/charm/osm-mon/tox.ini
index 56c095b..64bab10 100644
--- a/installers/charm/osm-mon/tox.ini
+++ b/installers/charm/osm-mon/tox.ini
@@ -21,7 +21,7 @@
[tox]
skipsdist=True
skip_missing_interpreters = True
-envlist = lint, unit
+envlist = lint, unit, integration
[vars]
src_path = {toxinidir}/src/
@@ -29,6 +29,7 @@
all_path = {[vars]src_path} {[vars]tst_path}
[testenv]
+basepython = python3.8
setenv =
PYTHONPATH = {toxinidir}:{toxinidir}/lib:{[vars]src_path}
PYTHONBREAKPOINT=ipdb.set_trace
@@ -53,14 +54,13 @@
black
flake8
flake8-docstrings
- flake8-copyright
flake8-builtins
pyproject-flake8
pep8-naming
isort
codespell
commands =
- codespell {toxinidir}/. --skip {toxinidir}/.git --skip {toxinidir}/.tox \
+ codespell {toxinidir} --skip {toxinidir}/.git --skip {toxinidir}/.tox \
--skip {toxinidir}/build --skip {toxinidir}/lib --skip {toxinidir}/venv \
--skip {toxinidir}/.mypy_cache --skip {toxinidir}/icon.svg
# pflake8 wrapper supports config from pyproject.toml
@@ -85,8 +85,8 @@
description = Run integration tests
deps =
pytest
- juju
+ juju<3
pytest-operator
-r{toxinidir}/requirements.txt
commands =
- pytest -v --tb native --ignore={[vars]tst_path}unit --log-cli-level=INFO -s {posargs}
+ pytest -v --tb native --ignore={[vars]tst_path}unit --log-cli-level=INFO -s {posargs} --cloud microk8s