From: David Garcia Date: Mon, 9 Mar 2020 07:38:17 +0000 (+0100) Subject: Add OSM charms X-Git-Tag: release-v8.0-start~48 X-Git-Url: https://osm.etsi.org/gitweb/?a=commitdiff_plain;h=82c5ffa15e1abb4c7ff58dc489f22cbe276c78c3;p=osm%2Fdevops.git Add OSM charms Change-Id: Iec1c915c77ac24080bfc8813eecc1970ee8c0213 Signed-off-by: David Garcia --- diff --git a/.gitignore b/.gitignore index 0c9f6308..e3ed0f85 100644 --- a/.gitignore +++ b/.gitignore @@ -35,3 +35,6 @@ osm-devops_*.post* *.local local +installers/charm/**/release/ +__pycache__ +.tox diff --git a/installers/charm/README.md b/installers/charm/README.md new file mode 100644 index 00000000..0fb31dbf --- /dev/null +++ b/installers/charm/README.md @@ -0,0 +1,169 @@ + + +# OSM Charms and interfaces + +**Description**: This document describes the high-level view of the OSM Charms and interfaces. An important note is that these charms Kubernetes Charms, so they must be deployed on top of a Kubernetes Cloud using Juju. + +## Folder tree + +In the current directory, there is one folder "interfaces" that has all the interfaces of the OSM components, which are basically two: osm-nbi, and osm-ro. + +Additionally, we can see six folders that contain each OSM core components: lcm-k8s, mon-k8s, nbi-k8s, pol-k8s, ro-k8s, and ui-k8s. + +Then, we can see a folder "bundle" which has the templates for the OSM bundles in single instance and HA. + +The "layers" folder include one common layer for all the osm charms (osm-common) + +```txt + +├── bundles +│ ├── osm +│ └── osm-ha +├── interfaces +│ ├── osm-nbi +│ └── osm-ro +├── layers +│ └── osm-common +├── lcm-k8s +├── mon-k8s +├── nbi-k8s +├── pol-k8s +├── ro-k8s +└── ui-k8s + +``` + +## Charms + +All the charms have a very similar structure. This subsection explains the purpose of each file inside the charms, as well as basic steps to get started. + +The folder structure for each charm looks like this: + +```txt +-k8s/ +├── config.yaml +├── icon.svg +├── layer.yaml +├── metadata.yaml +├── reactive +│ ├── .py +│ └── spec_template.yaml +├── README.md +├── .gitignore +├── .yamllint.yaml +└── tox.ini +``` + +Purpose of each file: + +- **config.yaml**: YAML file that include all the configurable options for the charm. +- **icon.svg**: SVG icon. This is the icon that will appear in the Charm Store. +- **layer.yaml**: YAML file with the layers that the charm needs. All the OSM Charms need at least the following layers: caas-base, status, leadership, and osm-common. If charms provide or require interfaces, which all of them do, those interfaces should be specified in this file too. +- **metadata.yaml**: YAML file that describe the top level information of the charm: name, description, series, interfaces that provide/require, needed storage, and deployment type. +- **reactive/\.py**: Python file that implements the actual logic to the charm. +- **reactive/spec_template.yaml**: Pod spec template to be used by the pods. +- **README.md**: This file describes how to build the charm, how to prepare the environment to test it with Microk8s. +- **.gitignore**: Typical Git Ignore file, to avoid pushing unwanted files to upstream. +- **.yamllint.yaml**: YAML file to specify the files to exclude from the yamllint test that tox.ini does. +- **tox.ini**: Includes basic functions to build the charms, and check the linting. + +## Interfaces + +Each interface needs at least three files: + +- **interface.yaml:** Metadata of the interface: name, maintainer, and summary. +- **provides.py:** Code for the charm that provides the interface. +- **requires.py:** Code for the charm that requires the interface. + +Additionally, there are also files for copyright and a README that explains how to use the interface. + +# Steps for testing + +## Dependencies + +```bash +sudo apt install tox -y +``` + +## Check the syntax of the charms + +```bash +./lint.sh +``` + +## Build all the charms + +```bash +./build.sh +``` + +## Generate bundle + +```bash +# Generate bundle from built charms +python3 generate_bundle.py --local --destination osm.yaml +# Help +python3 generate_bundle.py --help +``` + +## Install VCA + +```bash +sudo snap install juju --classic +juju bootstrap localhost osm-lxd +``` + +## Generate overlay + +> NOTE: This will be removed once the installer is merged. + +```bash +sudo snap install osmclient +sudo snap alias osmclient.osm osm +sudo snap connect osmclient:juju-client-observe +sudo snap connect osmclient:ssh-public-keys +sudo snap connect osmclient:network-control +osmclient.overlay # Execute the commands printed by this command to enable native charms +``` + +## Bootstrap Juju controller in Microk8s + +```bash +sudo snap install microk8s --classic +sudo usermod -a -G microk8s ubuntu +sudo chown -f -R ubuntu ~/.kube +newgrp microk8s +microk8s.status --wait-ready +microk8s.enable storage dns # (metallb) is optional +juju bootstrap microk8s osm-k8s +``` + +## Deploy OSM with charms + +```bash +juju add-model osm +juju deploy ./osm.yaml --overlay vca-overlay.yaml +``` + +## Wait until Charms are deployed + +```bash +watch -c juju status --color # Wait until every application is in active state +export OSM_HOSTNAME= +osm ns-list +# ... +``` diff --git a/installers/charm/build.sh b/installers/charm/build.sh new file mode 100755 index 00000000..17eea948 --- /dev/null +++ b/installers/charm/build.sh @@ -0,0 +1,25 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#!/bin/bash +set -eux +function build() { + cd $1 && tox -e build && cd .. +} + +build 'lcm-k8s' +build 'mon-k8s' +build 'nbi-k8s' +build 'pol-k8s' +build 'ro-k8s' +build 'ui-k8s' diff --git a/installers/charm/bundles/osm-ha/bundle.yaml b/installers/charm/bundles/osm-ha/bundle.yaml new file mode 100644 index 00000000..212a5208 --- /dev/null +++ b/installers/charm/bundles/osm-ha/bundle.yaml @@ -0,0 +1,230 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +description: A high-available OSM cluster. +bundle: kubernetes +applications: + zookeeper-k8s: + charm: "cs:~charmed-osm/zookeeper-k8s" + channel: "stable" + scale: 3 + series: kubernetes + storage: + database: 100M + options: + # client-port: 2181 + server-port: 2888 + leader-election-port: 3888 + zookeeper-units: 3 + annotations: + gui-x: 0 + gui-y: 1100 + mariadb-k8s: + charm: "cs:~charmed-osm/mariadb-k8s" + channel: "stable" + scale: 3 + series: kubernetes + storage: + database: 300M + options: + password: manopw + root_password: osm4u + user: mano + database: database + mysql_port: "3306" + query-cache-type: "OFF" + query-cache-size: 0 + ha-mode: true + annotations: + gui-x: -500 + gui-y: -400 + kafka-k8s: + charm: "cs:~charmed-osm/kafka-k8s" + channel: "stable" + scale: 3 + series: kubernetes + storage: + database: 200M + options: + advertised-hostname: "kafka-k8s" + advertised-port: 9092 + kafka-units: 3 + zookeeper-units: 3 + zookeeper-service-name: zookeeper-k8s-endpoints + annotations: + gui-x: 0 + gui-y: 600 + mongodb-k8s: + charm: "cs:~charmed-osm/mongodb-k8s" + channel: "stable" + scale: 3 + series: kubernetes + storage: + database: 300M + options: + advertised-port: 27017 + replica-set: rs0 + namespace: osm + service-name: mongodb-k8s-endpoints + cluster-domain: cluster.local + enable-sidecar: true + annotations: + gui-x: 0 + gui-y: 100 + nbi-k8s: + charm: "cs:~charmed-osm/nbi-k8s" + channel: "%(channel)s" + scale: 3 + series: kubernetes + storage: + packages: 100M + options: + log_level: "INFO" + DATABASE_COMMONKEY: osm + annotations: + gui-x: 0 + gui-y: -400 + ro-k8s: + charm: "cs:~charmed-osm/ro-k8s" + channel: "%(channel)s" + scale: 3 + series: kubernetes + storage: + log: 50M + options: + vim_database: "mano_vim_db" + ro_database: "mano_db" + OPENMANO_TENANT: "osm" + annotations: + gui-x: -500 + gui-y: 600 + ui-k8s: + charm: "cs:~charmed-osm/ui-k8s" + channel: "%(channel)s" + scale: 3 + series: kubernetes + options: + mysql_database: lwui + annotations: + gui-x: 500 + gui-y: -400 + lcm-k8s: + charm: "cs:~charmed-osm/lcm-k8s" + channel: "%(channel)s" + scale: 3 + series: kubernetes + storage: + packages: 100M + options: + vca_host: vca + vca_port: 17070 + vca_user: admin + vca_password: secret + vca_pubkey: pubkey + vca_cacert: cacert + vca_apiproxy: apiproxy + use_external_vca: true + DATABASE_COMMONKEY: osm + annotations: + gui-x: -500 + gui-y: 100 + mon-k8s: + charm: "cs:~charmed-osm/mon-k8s" + channel: "%(channel)s" + scale: 3 + series: kubernetes + storage: + database: 100M + options: + OSMMON_OPENSTACK_DEFAULT_GRANULARITY: 300 + OSMMON_GLOBAL_REQUEST_TIMEOUT: 10 + OSMMON_GLOBAL_LOGLEVEL: INFO + OSMMON_DATABASE_COMMONKEY: osm + OSMMON_COLLECTOR_INTERVAL: 30 + OSMMON_EVALUATOR_INTERVAL: 30 + vca_host: vca + vca_user: admin + vca_password: secret + vca_cacert: cacert + use_external_vca: true + annotations: + gui-x: 500 + gui-y: 100 + pol-k8s: + charm: "cs:~charmed-osm/pol-k8s" + channel: "%(channel)s" + scale: 3 + series: kubernetes + storage: + database: 100M + options: + log_level: INFO + annotations: + gui-x: -500 + gui-y: 1100 + prometheus-k8s: + charm: "cs:~charmed-osm/prometheus-k8s" + channel: "stable" + scale: 3 + series: kubernetes + storage: + database: 100M + options: + advertised-port: 9090 + web-subpath: / + default-target: "mon-k8s:8000" + annotations: + gui-x: 500 + gui-y: 600 + grafana-k8s: + charm: "cs:~charmed-osm/grafana-k8s" + channel: "stable" + scale: 3 + series: kubernetes + annotations: + gui-x: 500 + gui-y: 1100 + +relations: + - - "kafka-k8s:zookeeper" + - "zookeeper-k8s:zookeeper" + - - "ro-k8s:mysql" + - "mariadb-k8s:mysql" + - - "nbi-k8s:mongo" + - "mongodb-k8s:mongo" + - - "nbi-k8s:kafka" + - "kafka-k8s:kafka" + - - "lcm-k8s:kafka" + - "kafka-k8s:kafka" + - - "lcm-k8s:mongo" + - "mongodb-k8s:mongo" + - - "mon-k8s:kafka" + - "kafka-k8s:kafka" + - - "mon-k8s:mongo" + - "mongodb-k8s:mongo" + - - "pol-k8s:kafka" + - "kafka-k8s:kafka" + - - "pol-k8s:mongo" + - "mongodb-k8s:mongo" + - - "lcm-k8s:ro" + - "ro-k8s:ro" + - - "prometheus-k8s:prometheus" + - "mon-k8s:prometheus" + - - "grafana-k8s:prometheus" + - "prometheus-k8s:prometheus" + - - "ui-k8s:mysql" + - "mariadb-k8s:mysql" + - - "ui-k8s:nbi" + - "nbi-k8s:nbi" + - - "prometheus-k8s:prometheus" + - "nbi-k8s:prometheus" diff --git a/installers/charm/bundles/osm/bundle.yaml b/installers/charm/bundles/osm/bundle.yaml new file mode 100644 index 00000000..fc3dbcb4 --- /dev/null +++ b/installers/charm/bundles/osm/bundle.yaml @@ -0,0 +1,230 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +description: Single instance OSM bundle +bundle: kubernetes +applications: + zookeeper-k8s: + charm: 'cs:~charmed-osm/zookeeper-k8s' + channel: 'stable' + scale: 1 + series: kubernetes + storage: + database: 100M + options: + # client-port: 2181 + server-port: 2888 + leader-election-port: 3888 + zookeeper-units: 1 + annotations: + gui-x: 0 + gui-y: 1100 + mariadb-k8s: + charm: 'cs:~charmed-osm/mariadb-k8s' + channel: 'stable' + scale: 1 + series: kubernetes + storage: + database: 50M + options: + password: manopw + root_password: osm4u + user: mano + database: database + mysql_port: "3306" + query-cache-type: "OFF" + query-cache-size: 0 + ha-mode: false + annotations: + gui-x: -500 + gui-y: -400 + kafka-k8s: + charm: 'cs:~charmed-osm/kafka-k8s' + channel: 'stable' + scale: 1 + series: kubernetes + storage: + database: 100M + options: + advertised-hostname: "kafka-k8s" + advertised-port: 9092 + kafka-units: 1 + zookeeper-units: 1 + zookeeper-service-name: zookeeper-k8s-endpoints + annotations: + gui-x: 0 + gui-y: 600 + mongodb-k8s: + charm: 'cs:~charmed-osm/mongodb-k8s' + channel: 'stable' + scale: 1 + series: kubernetes + storage: + database: 50M + options: + advertised-port: 27017 + replica-set: rs0 + namespace: osm + service-name: mongodb-k8s-endpoints + cluster-domain: cluster.local + enable-sidecar: false + annotations: + gui-x: 0 + gui-y: 100 + nbi-k8s: + charm: '%(prefix)s/nbi-k8s%(suffix)s' + channel: '%(channel)s' + scale: 1 + series: kubernetes + storage: + packages: 50M + options: + log_level: "INFO" + DATABASE_COMMONKEY: osm + annotations: + gui-x: 0 + gui-y: -400 + ro-k8s: + charm: '%(prefix)s/ro-k8s%(suffix)s' + channel: '%(channel)s' + scale: 1 + series: kubernetes + storage: + log: 50M + options: + vim_database: 'mano_vim_db' + ro_database: 'mano_db' + OPENMANO_TENANT: 'osm' + annotations: + gui-x: -500 + gui-y: 600 + ui-k8s: + charm: '%(prefix)s/ui-k8s%(suffix)s' + channel: '%(channel)s' + scale: 1 + series: kubernetes + options: + mysql_database: lwui + annotations: + gui-x: 500 + gui-y: -400 + lcm-k8s: + charm: '%(prefix)s/lcm-k8s%(suffix)s' + channel: '%(channel)s' + scale: 1 + series: kubernetes + storage: + packages: 50M + options: + vca_host: vca + vca_port: 17070 + vca_user: admin + vca_password: secret + vca_pubkey: pubkey + vca_cacert: cacert + vca_apiproxy: apiproxy + use_external_vca: false + DATABASE_COMMONKEY: osm + annotations: + gui-x: -500 + gui-y: 100 + mon-k8s: + charm: '%(prefix)s/mon-k8s%(suffix)s' + channel: '%(channel)s' + scale: 1 + series: kubernetes + storage: + database: 100M + options: + OSMMON_OPENSTACK_DEFAULT_GRANULARITY: 300 + OSMMON_GLOBAL_REQUEST_TIMEOUT: 10 + OSMMON_GLOBAL_LOGLEVEL: INFO + OSMMON_DATABASE_COMMONKEY: osm + OSMMON_COLLECTOR_INTERVAL: 30 + OSMMON_EVALUATOR_INTERVAL: 30 + vca_host: vca + vca_user: admin + vca_password: secret + vca_cacert: cacert + use_external_vca: false + annotations: + gui-x: 500 + gui-y: 100 + pol-k8s: + charm: '%(prefix)s/pol-k8s%(suffix)s' + channel: '%(channel)s' + scale: 1 + series: kubernetes + storage: + database: 100M + options: + log_level: INFO + annotations: + gui-x: -500 + gui-y: 1100 + prometheus-k8s: + charm: 'cs:~charmed-osm/prometheus-k8s' + channel: 'stable' + scale: 1 + series: kubernetes + storage: + database: 50M + options: + advertised-port: 9090 + web-subpath: / + default-target: "mon-k8s:8000" + annotations: + gui-x: 500 + gui-y: 600 + grafana-k8s: + charm: 'cs:~charmed-osm/grafana-k8s' + channel: 'stable' + scale: 1 + series: kubernetes + annotations: + gui-x: 500 + gui-y: 1100 + +relations: + - - 'kafka-k8s:zookeeper' + - 'zookeeper-k8s:zookeeper' + - - 'ro-k8s:mysql' + - 'mariadb-k8s:mysql' + - - 'nbi-k8s:mongo' + - 'mongodb-k8s:mongo' + - - 'nbi-k8s:kafka' + - 'kafka-k8s:kafka' + - - 'lcm-k8s:kafka' + - 'kafka-k8s:kafka' + - - 'lcm-k8s:mongo' + - 'mongodb-k8s:mongo' + - - 'mon-k8s:kafka' + - 'kafka-k8s:kafka' + - - 'mon-k8s:mongo' + - 'mongodb-k8s:mongo' + - - 'pol-k8s:kafka' + - 'kafka-k8s:kafka' + - - 'pol-k8s:mongo' + - 'mongodb-k8s:mongo' + - - 'lcm-k8s:ro' + - 'ro-k8s:ro' + - - 'prometheus-k8s:prometheus' + - 'mon-k8s:prometheus' + - - 'grafana-k8s:prometheus' + - 'prometheus-k8s:prometheus' + - - 'ui-k8s:mysql' + - 'mariadb-k8s:mysql' + - - 'ui-k8s:nbi' + - 'nbi-k8s:nbi' + - - 'prometheus-k8s:prometheus' + - 'nbi-k8s:prometheus' diff --git a/installers/charm/generate_bundle.py b/installers/charm/generate_bundle.py new file mode 100644 index 00000000..95ffeaa7 --- /dev/null +++ b/installers/charm/generate_bundle.py @@ -0,0 +1,65 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import json +import argparse + +CHANNEL_LIST = [ + "stable", + "candidate", + "edge", +] +BUNDLE_PREFIX = "cs:~charmed-osm" +DEFAULT_BUNDLE = "bundles/osm/bundle.yaml" +HA_BUNDLE = "bundles/osm-ha/bundle.yaml" + +parser = argparse.ArgumentParser(description="Process some arguments.") + +parser.add_argument("--channel", help="Channel from the Charm Store") +parser.add_argument("--destination", help="Destination for the generated bundle") +parser.add_argument("--ha", help="Select HA bundle", action="store_true") +parser.add_argument("--local", help="Path to the bundle directory", action="store_true") +parser.add_argument("--store", help="Path to the bundle directory", action="store_true") + +args = parser.parse_args() +print(args) +if not args.local and not args.store: + raise Exception("--local or --store must be specified") +if args.local and args.store: + raise Exception("Both --local and --store cannot be specified. Please choose one.") +if not args.destination: + raise Exception("--destination must be specified") +if args.channel and not args.channel in CHANNEL_LIST: + raise Exception( + "Channel {} does not exist. Please choose one of these: {}".format( + args.channel, CHANNEL_LIST + ) + ) +channel = args.channel if args.channel else "stable" +path = HA_BUNDLE if args.ha else DEFAULT_BUNDLE +destination = args.destination +prefix = "." if args.local else BUNDLE_PREFIX +suffix = "/release" if args.local else "" + +data = { + "channel": channel, + "prefix": prefix, + "suffix": suffix, +} + +with open(path) as template: + bundle_template = template.read() + template.close() +with open("{}".format(destination), "w") as text_file: + text_file.write(bundle_template % data) + text_file.close() diff --git a/installers/charm/interfaces/osm-nbi/README.md b/installers/charm/interfaces/osm-nbi/README.md new file mode 100644 index 00000000..8fb9523f --- /dev/null +++ b/installers/charm/interfaces/osm-nbi/README.md @@ -0,0 +1,63 @@ + + +# Overview + +This interface layer handles communication between Mongodb and its clients. + +## Usage + +### Provides + +To implement this relation to offer an nbi: + +In your charm's metadata.yaml: + +```yaml +provides: + nbi: + interface: osm-nbi +``` + +reactive/mynbi.py: + +```python +@when('nbi.joined') +def send_config(nbi): + nbi.send_connection( + unit_get('private-address'), + get_nbi_port() + ) +``` + +### Requires + +If you would like to use an nbi from your charm: + +metadata.yaml: + +```yaml +requires: + nbi: + interface: osm-nbi +``` + +reactive/mycharm.py: + +```python +@when('nbi.ready') +def nbi_ready(): + nbi = endpoint_from_flag('nbi.ready') + if nbi: + for unit in nbi.nbis(): + add_nbi(unit['host'], unit['port']) +``` diff --git a/installers/charm/interfaces/osm-nbi/copyright b/installers/charm/interfaces/osm-nbi/copyright new file mode 100644 index 00000000..dd9405e3 --- /dev/null +++ b/installers/charm/interfaces/osm-nbi/copyright @@ -0,0 +1,16 @@ +Format: http://dep.debian.net/deps/dep5/ + +Files: * +Copyright: Copyright 2020, Canonical Ltd., All Rights Reserved. +License: Apache License 2.0 + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + . + http://www.apache.org/licenses/LICENSE-2.0 + . + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/installers/charm/interfaces/osm-nbi/interface.yaml b/installers/charm/interfaces/osm-nbi/interface.yaml new file mode 100644 index 00000000..ec8ee862 --- /dev/null +++ b/installers/charm/interfaces/osm-nbi/interface.yaml @@ -0,0 +1,16 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: osm-nbi +summary: Interface for relating to a OSM Northbound Interface +maintainer: '"Adam Israel" ' diff --git a/installers/charm/interfaces/osm-nbi/provides.py b/installers/charm/interfaces/osm-nbi/provides.py new file mode 100644 index 00000000..7ff31994 --- /dev/null +++ b/installers/charm/interfaces/osm-nbi/provides.py @@ -0,0 +1,44 @@ +#!/usr/bin/python +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from charms.reactive import RelationBase +from charms.reactive import hook +from charms.reactive import scopes + + +class OsmNBIProvides(RelationBase): + scope = scopes.GLOBAL + + @hook("{provides:osm-nbi}-relation-joined") + def joined(self): + self.set_state("{relation_name}.joined") + + @hook("{provides:osm-nbi}-relation-changed") + def changed(self): + self.set_state("{relation_name}.ready") + + @hook("{provides:osm-nbi}-relation-{broken,departed}") + def broken_departed(self): + self.remove_state("{relation_name}.ready") + self.remove_state("{relation_name}.joined") + + @hook("{provides:osm-nbi}-relation-broken") + def broken(self): + self.set_state("{relation_name}.removed") + + def send_connection(self, host, port=9999): + conv = self.conversation() + conv.set_remote("host", host) + conv.set_remote("port", port) diff --git a/installers/charm/interfaces/osm-nbi/requires.py b/installers/charm/interfaces/osm-nbi/requires.py new file mode 100644 index 00000000..a5e8e29d --- /dev/null +++ b/installers/charm/interfaces/osm-nbi/requires.py @@ -0,0 +1,56 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from charms.reactive import RelationBase +from charms.reactive import hook +from charms.reactive import scopes + + +class OsmNBIRequires(RelationBase): + scope = scopes.GLOBAL + + @hook("{requires:osm-nbi}-relation-joined") + def joined(self): + conv = self.conversation() + conv.set_state("{relation_name}.joined") + + @hook("{requires:osm-nbi}-relation-changed") + def changed(self): + conv = self.conversation() + if self.nbis(): + conv.set_state("{relation_name}.ready") + else: + conv.remove_state("{relation_name}.ready") + + @hook("{requires:osm-nbi}-relation-departed") + def departed(self): + conv = self.conversation() + conv.remove_state("{relation_name}.ready") + conv.remove_state("{relation_name}.joined") + + def nbis(self): + """Return the NBI's host and port. + + [{ + 'host': , + 'port': , + }] + """ + nbis = [] + for conv in self.conversations(): + port = conv.get_remote("port") + host = conv.get_remote("host") or conv.get_remote("private-address") + if host and port: + nbis.append({"host": host, "port": port}) + return nbis diff --git a/installers/charm/interfaces/osm-ro/README.md b/installers/charm/interfaces/osm-ro/README.md new file mode 100644 index 00000000..eb6413a3 --- /dev/null +++ b/installers/charm/interfaces/osm-ro/README.md @@ -0,0 +1,63 @@ + + +# Overview + +This interface layer handles communication between OSM's RO and its clients. + +## Usage + +### Provides + +To implement this relation to offer an ro: + +In your charm's metadata.yaml: + +```yaml +provides: + ro: + interface: osm-ro +``` + +reactive/myro.py: + +```python +@when('ro.joined') +def send_config(ro): + ro.send_connection( + unit_get('private-address'), + get_ro_port() + ) +``` + +### Requires + +If you would like to use a rodb from your charm: + +metadata.yaml: + +```yaml +requires: + ro: + interface: osm-ro +``` + +reactive/mycharm.py: + +```python +@when('ro.ready') +def ro_ready(): + ro = endpoint_from_flag('ro.ready') + if ro: + for unit in ro.ros(): + add_ro(unit['host'], unit['port']) +``` diff --git a/installers/charm/interfaces/osm-ro/copyright b/installers/charm/interfaces/osm-ro/copyright new file mode 100644 index 00000000..9270d6ca --- /dev/null +++ b/installers/charm/interfaces/osm-ro/copyright @@ -0,0 +1,16 @@ +Format: http://dep.debian.net/deps/dep5/ + +Files: * +Copyright: Copyright 2020, Canonical Ltd., All Rights Reserved. +License: Apache License 2.0 + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + . + http://www.apache.org/licenses/LICENSE-2.0 + . + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/installers/charm/interfaces/osm-ro/interface.yaml b/installers/charm/interfaces/osm-ro/interface.yaml new file mode 100644 index 00000000..9a12872b --- /dev/null +++ b/installers/charm/interfaces/osm-ro/interface.yaml @@ -0,0 +1,16 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: osm-ro +summary: Interface for relating to a OSM Resource Orchestrator +maintainer: '"Adam Israel" ' diff --git a/installers/charm/interfaces/osm-ro/provides.py b/installers/charm/interfaces/osm-ro/provides.py new file mode 100644 index 00000000..f5773194 --- /dev/null +++ b/installers/charm/interfaces/osm-ro/provides.py @@ -0,0 +1,44 @@ +#!/usr/bin/python +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from charms.reactive import RelationBase +from charms.reactive import hook +from charms.reactive import scopes + + +class OsmROProvides(RelationBase): + scope = scopes.GLOBAL + + @hook("{provides:osm-ro}-relation-joined") + def joined(self): + self.set_state("{relation_name}.joined") + + @hook("{provides:osm-ro}-relation-changed") + def changed(self): + self.set_state("{relation_name}.ready") + + @hook("{provides:osm-ro}-relation-{broken,departed}") + def broken_departed(self): + self.remove_state("{relation_name}.ready") + self.remove_state("{relation_name}.joined") + + @hook("{provides:osm-ro}-relation-broken") + def broken(self): + self.set_state("{relation_name}.removed") + + def send_connection(self, host, port=9090): + conv = self.conversation() + conv.set_remote("host", host) + conv.set_remote("port", port) diff --git a/installers/charm/interfaces/osm-ro/requires.py b/installers/charm/interfaces/osm-ro/requires.py new file mode 100644 index 00000000..fc8f0f4a --- /dev/null +++ b/installers/charm/interfaces/osm-ro/requires.py @@ -0,0 +1,56 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from charms.reactive import RelationBase +from charms.reactive import hook +from charms.reactive import scopes + + +class OsmRORequires(RelationBase): + scope = scopes.GLOBAL + + @hook("{requires:osm-ro}-relation-joined") + def joined(self): + conv = self.conversation() + conv.set_state("{relation_name}.joined") + + @hook("{requires:osm-ro}-relation-changed") + def changed(self): + conv = self.conversation() + if self.ros(): + conv.set_state("{relation_name}.ready") + else: + conv.remove_state("{relation_name}.ready") + + @hook("{requires:osm-ro}-relation-departed") + def departed(self): + conv = self.conversation() + conv.remove_state("{relation_name}.ready") + conv.remove_state("{relation_name}.joined") + + def ros(self): + """Return the NBI's host and port. + + [{ + 'host': , + 'port': , + }] + """ + ros = [] + for conv in self.conversations(): + port = conv.get_remote("port") + host = conv.get_remote("host") or conv.get_remote("private-address") + if host and port: + ros.append({"host": host, "port": port}) + return ros diff --git a/installers/charm/layers/osm-common/README.md b/installers/charm/layers/osm-common/README.md new file mode 100644 index 00000000..c55b97b0 --- /dev/null +++ b/installers/charm/layers/osm-common/README.md @@ -0,0 +1,17 @@ + + +# README + +WIP. Layer to share common functionality to write/deploy k8s charms for OSM demo diff --git a/installers/charm/layers/osm-common/layer.yaml b/installers/charm/layers/osm-common/layer.yaml new file mode 100644 index 00000000..6e8379ab --- /dev/null +++ b/installers/charm/layers/osm-common/layer.yaml @@ -0,0 +1,13 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. \ No newline at end of file diff --git a/installers/charm/layers/osm-common/lib/charms/osm/k8s.py b/installers/charm/layers/osm-common/lib/charms/osm/k8s.py new file mode 100644 index 00000000..9735517f --- /dev/null +++ b/installers/charm/layers/osm-common/lib/charms/osm/k8s.py @@ -0,0 +1,76 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from charmhelpers.core.hookenv import ( + network_get, + relation_id, + log, +) + + +def get_service_ip(endpoint): + try: + info = network_get(endpoint, relation_id()) + if 'ingress-addresses' in info: + addr = info['ingress-addresses'][0] + if len(addr): + return addr + else: + log("No ingress-addresses: {}".format(info)) + except Exception as e: + log("Caught exception checking for service IP: {}".format(e)) + + return None + + +def is_pod_up(endpoint): + """Check to see if the pod of a relation is up. + + application-vimdb: 19:29:10 INFO unit.vimdb/0.juju-log network info + + In the example below: + - 10.1.1.105 is the address of the application pod. + - 10.152.183.199 is the service cluster ip + + { + 'bind-addresses': [{ + 'macaddress': '', + 'interfacename': '', + 'addresses': [{ + 'hostname': '', + 'address': '10.1.1.105', + 'cidr': '' + }] + }], + 'egress-subnets': [ + '10.152.183.199/32' + ], + 'ingress-addresses': [ + '10.152.183.199', + '10.1.1.105' + ] + } + """ + try: + info = network_get(endpoint, relation_id()) + + # Check to see if the pod has been assigned it's internal and + # external ips + for ingress in info['ingress-addresses']: + if len(ingress) == 0: + return False + except: + return False + + return True diff --git a/installers/charm/layers/osm-common/metadata.yaml b/installers/charm/layers/osm-common/metadata.yaml new file mode 100644 index 00000000..6e8379ab --- /dev/null +++ b/installers/charm/layers/osm-common/metadata.yaml @@ -0,0 +1,13 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. \ No newline at end of file diff --git a/installers/charm/layers/osm-common/reactive/osm_common.py b/installers/charm/layers/osm-common/reactive/osm_common.py new file mode 100644 index 00000000..6e8379ab --- /dev/null +++ b/installers/charm/layers/osm-common/reactive/osm_common.py @@ -0,0 +1,13 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. \ No newline at end of file diff --git a/installers/charm/lcm-k8s/.yamllint.yaml b/installers/charm/lcm-k8s/.yamllint.yaml new file mode 100644 index 00000000..86049d25 --- /dev/null +++ b/installers/charm/lcm-k8s/.yamllint.yaml @@ -0,0 +1,25 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- + +extends: default + +yaml-files: + - '*.yaml' + - '*.yml' + - '.yamllint' +ignore: | + reactive/ + .tox + release/ diff --git a/installers/charm/lcm-k8s/README.md b/installers/charm/lcm-k8s/README.md new file mode 100644 index 00000000..7b8d3fd6 --- /dev/null +++ b/installers/charm/lcm-k8s/README.md @@ -0,0 +1,100 @@ + + +# Overview + +LCM for Juju CAAS + +## Testing + +The tests of this charm are done using tox and Zaza. + + + +### Prepare environment + +The machine in which the tests are run needs access to a juju k8s controller. The easiest way to approach this is by executing the following commands: + +``` +sudo apt install tox -y +sudo snap install microk8s --classic +sudo snap install juju + +microk8s.status --wait-ready +microk8s.enable storage dashboard dns + +juju bootstrap microk8s k8s-cloud +``` + +If /usr/bin/python does not exist, you should probably need to do this: + +``` +sudo ln -s /usr/bin/python3 /usr/bin/python +``` +### Build Charm + +**Download dependencies:** + +``` +mkdir -p ~/charm/layers ~/charm/build ~/charm/interfaces +git clone https://git.launchpad.net/canonical-osm ~/canonical-osm + +cd ~/charm/layers +git clone https://git.launchpad.net/charm-osm-lcm lcm-k8s +git clone https://git.launchpad.net/charm-osm-ro ro-k8s +git clone https://git.launchpad.net/charm-k8s-kafka kafka-k8s ~/charm/interfaces +git clone https://git.launchpad.net/charm-k8s-zookeeper zookeeper-k8s +git clone https://git.launchpad.net/charm-k8s-mongodb mongodb-k8s +git clone https://git.launchpad.net/charm-k8s-mariadb mariadb-k8s +git clone https://git.launchpad.net/charm-osm-ro ro-k8s + +cd ~/charm/interfaces +mv ~/canonical-osm/charms/interfaces/* . +git clone https://git.launchpad.net/interface-mongodb mongodb +git clone https://git.launchpad.net/interface-osm-ro osm-ro + +``` + +**Charm structure:** + +``` +├── config.yaml +├── icon.svg +├── layer.yaml +├── metadata.yaml +├── reactive +│ ├── spec_template.yaml +│ └── lcm.py +├── README.md +└── tox.ini +``` + +**Setup environment variables:** + +``` +export CHARM_LAYERS_DIR=~/charm/layers +export CHARM_BUILD_DIR=~/charm/build +export CHARM_INTERFACES_DIR=~/charm/interfaces +``` + +**Build:** + +``` +mkdir ~/charm/layers/lcm-k8s/tests/build +charm build ~/charm/layers/lcm-k8s +charm build ~/charm/layers/kafka-k8s +charm build ~/charm/layers/zookeeper-k8s +charm build ~/charm/layers/mongodb-k8s +charm build ~/charm/layers/mariadb-k8s +charm build ~/charm/layers/ro-k8s +mv ~/charm/build/* ~/charm/layers/lcm-k8s/tests/build/ +``` diff --git a/installers/charm/lcm-k8s/config.yaml b/installers/charm/lcm-k8s/config.yaml new file mode 100644 index 00000000..c27377fc --- /dev/null +++ b/installers/charm/lcm-k8s/config.yaml @@ -0,0 +1,64 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +options: + vca_host: + type: string + description: "The VCA host." + default: "admin" + vca_port: + type: int + description: "The VCA port." + default: 17070 + vca_user: + type: string + description: "The VCA user name." + default: "admin" + vca_password: + type: string + description: "The VCA user password." + default: "secret" + vca_pubkey: + type: string + description: "The VCA public key." + default: "secret" + vca_cacert: + type: string + description: "The VCA cacert." + default: "" + vca_apiproxy: + type: string + description: "The VCA api proxy (native charms)" + default: "" + vca_cloud: + type: string + description: "The VCA lxd cloud name" + default: "localhost" + advertised-hostname: + description: LCM Hostname + type: string + default: "lcm-k8s" + use_external_vca: + description: | + Boolean to deside whether LCM should wait + for an external VCA or not (using osm-vca interface) + type: boolean + default: false + DATABASE_COMMONKEY: + description: Database COMMONKEY + type: string + default: osm + image: + type: string + description: OCI image + default: opensourcemano/lcm:7 diff --git a/installers/charm/lcm-k8s/icon.svg b/installers/charm/lcm-k8s/icon.svg new file mode 100644 index 00000000..da31b4ab --- /dev/null +++ b/installers/charm/lcm-k8s/icon.svg @@ -0,0 +1,118 @@ + + + + + + + + image/svg+xml + + eclispe-che + + + + + + eclispe-che + Created with Sketch. + + + + + + + + + + + + + + + + + diff --git a/installers/charm/lcm-k8s/layer.yaml b/installers/charm/lcm-k8s/layer.yaml new file mode 100644 index 00000000..56b1af9a --- /dev/null +++ b/installers/charm/lcm-k8s/layer.yaml @@ -0,0 +1,23 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +includes: + - "layer:caas-base" + - 'layer:status' + - 'layer:leadership' + - "layer:osm-common" + - 'interface:kafka' + - 'interface:mongodb' + - 'interface:osm-ro' + +repo: https://code.launchpad.net/osm-k8s-bundle diff --git a/installers/charm/lcm-k8s/metadata.yaml b/installers/charm/lcm-k8s/metadata.yaml new file mode 100644 index 00000000..b44074c7 --- /dev/null +++ b/installers/charm/lcm-k8s/metadata.yaml @@ -0,0 +1,37 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: "lcm-k8s" +summary: "OSM LCM" +maintainers: + - "SolutionsQA " +description: | + A CAAS charm to deploy OSM's LCM. +tags: + - "application" +series: + - "kubernetes" +requires: + kafka: + interface: kafka + mongo: + interface: mongodb + ro: + interface: osm-ro +storage: + packages: + type: filesystem + location: /app/storage +deployment: + type: stateful + service: cluster diff --git a/installers/charm/lcm-k8s/reactive/lcm.py b/installers/charm/lcm-k8s/reactive/lcm.py new file mode 100644 index 00000000..85ec9c5b --- /dev/null +++ b/installers/charm/lcm-k8s/reactive/lcm.py @@ -0,0 +1,134 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +from charms.layer.caas_base import pod_spec_set +from charms.reactive import endpoint_from_flag +from charms.reactive import when, when_not, hook +from charms.reactive.flags import set_flag, clear_flag +from charmhelpers.core.hookenv import ( + log, + metadata, + config, +) +from charms import layer + + +@hook("upgrade-charm") +@when("leadership.is_leader") +def upgrade(): + clear_flag("lcm-k8s.configured") + + +@when("config.changed") +@when("leadership.is_leader") +def restart(): + clear_flag("lcm-k8s.configured") + + +@when_not("kafka.ready") +@when_not("lcm-k8s.configured") +def waiting_for_kafka(): + layer.status.waiting("Waiting for kafka to be ready") + + +@when_not("mongo.ready") +@when_not("lcm-k8s.configured") +def waiting_for_mongo(): + layer.status.waiting("Waiting for mongo to be ready") + + +@when_not("ro.ready") +@when_not("lcm-k8s.configured") +def waiting_for_ro(): + layer.status.waiting("Waiting for ro to be ready") + + +@when("kafka.ready", "mongo.ready", "ro.ready") +@when_not("lcm-k8s.configured") +@when("leadership.is_leader") +def configure(): + layer.status.maintenance("Configuring lcm container") + try: + kafka = endpoint_from_flag("kafka.ready") + mongo = endpoint_from_flag("mongo.ready") + osm_ro = endpoint_from_flag("ro.ready") + + if kafka and mongo and osm_ro: + kafka_units = kafka.kafkas() + kafka_unit = kafka_units[0] + + mongo_uri = mongo.connection_string() + log("Mongo URI: {}".format(mongo_uri)) + + ros = osm_ro.ros() + ro_unit = ros[0] + + if ( + mongo_uri + and kafka_unit["host"] + and kafka_unit["port"] + and ro_unit["host"] + and ro_unit["port"] + ): + spec = make_pod_spec( + ro_unit["host"], + ro_unit["port"], + kafka_unit["host"], + kafka_unit["port"], + mongo_uri, + ) + + log("set pod spec:\n{}".format(spec)) + pod_spec_set(spec) + layer.status.active("creating container") + set_flag("lcm-k8s.configured") + except Exception as e: + layer.status.blocked("k8s spec failed to deploy: {}".format(e)) + + +@when("lcm-k8s.configured") +def set_lcm_active(): + layer.status.active("ready") + + +def make_pod_spec(ro_host, ro_port, kafka_host, kafka_port, mongo_uri): + """Make pod specification for Kubernetes + + Args: + ro_host (str): RO hostname or IP + ro_port (str): RO Port + kafka_host (str): Kafka hostname or IP + kafka_port (int): Kafka port + mongo_uri (str): Mongo URI + Returns: + pod_spec: Pod specification for Kubernetes + """ + + with open("reactive/spec_template.yaml") as spec_file: + pod_spec_template = spec_file.read() + + cfg = config() + md = metadata() + + data = { + "name": md.get("name"), + "docker_image": cfg.get("image"), + "ro_host": ro_host, + "ro_port": ro_port, + "kafka_host": kafka_host, + "kafka_port": kafka_port, + "mongo_uri": mongo_uri, + } + data.update(cfg) + + return pod_spec_template % data diff --git a/installers/charm/lcm-k8s/reactive/spec_template.yaml b/installers/charm/lcm-k8s/reactive/spec_template.yaml new file mode 100644 index 00000000..1a270965 --- /dev/null +++ b/installers/charm/lcm-k8s/reactive/spec_template.yaml @@ -0,0 +1,49 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +version: 2 +containers: + - name: %(name)s + image: %(docker_image)s + ports: + # This is a fake port; lcm doesn't listen, since it's just reading + # and responding to the kafka bus. Fix this in post. + - containerPort: 9999 + protocol: TCP + config: + ALLOW_ANONYMOUS_LOGIN: 'yes' + OSMLCM_RO_HOST: %(ro_host)s + OSMLCM_RO_PORT: %(ro_port)s + OSMLCM_RO_TENANT: osm + + OSMLCM_MESSAGE_DRIVER: kafka + OSMLCM_MESSAGE_HOST: %(kafka_host)s + OSMLCM_MESSAGE_PORT: %(kafka_port)s + + OSMLCM_DATABASE_DRIVER: mongo + OSMLCM_DATABASE_URI: %(mongo_uri)s + OSMLCM_DATABASE_COMMONKEY: %(DATABASE_COMMONKEY)s + + OSMLCM_STORAGE_DRIVER: mongo + OSMLCM_STORAGE_PATH: /app/storage + OSMLCM_STORAGE_COLLECTION: files + OSMLCM_STORAGE_URI: %(mongo_uri)s + + OSMLCM_VCA_HOST: %(vca_host)s + OSMLCM_VCA_PORT: %(vca_port)s + OSMLCM_VCA_USER: %(vca_user)s + OSMLCM_VCA_PUBKEY: %(vca_pubkey)s + OSMLCM_VCA_SECRET: %(vca_password)s + OSMLCM_VCA_CACERT: %(vca_cacert)s + OSMLCM_VCA_APIPROXY: %(vca_apiproxy)s + OSMLCM_VCA_CLOUD: %(vca_cloud)s diff --git a/installers/charm/lcm-k8s/tox.ini b/installers/charm/lcm-k8s/tox.ini new file mode 100644 index 00000000..43934c26 --- /dev/null +++ b/installers/charm/lcm-k8s/tox.ini @@ -0,0 +1,64 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +[tox] +envlist = pep8 +skipsdist = True + +[testenv] +setenv = VIRTUAL_ENV={envdir} + PYTHONHASHSEED=0 +whitelist_externals = juju + sleep +passenv = HOME TERM CS_API_* OS_* AMULET_* +install_command = + pip install {opts} {packages} + +[testenv:build] +basepython = python3 +passenv=HTTP_PROXY HTTPS_PROXY NO_PROXY +setenv = CHARM_LAYERS_DIR = ../layers + CHARM_INTERFACES_DIR = ../interfaces/ +whitelist_externals = git + charm + rm + mv +commands = + rm -rf release + rm -rf ../interfaces/mongodb + git clone https://git.launchpad.net/interface-mongodb ../interfaces/mongodb + charm build . --build-dir /tmp + mv /tmp/lcm-k8s/ release/ + +[testenv:lint] +basepython = python3 +deps = + black + yamllint + flake8 +commands = + black --check --diff . --exclude "release/|.tox/" + yamllint . + flake8 reactive/ --max-line-length=100 + +[testenv:pep8] +basepython = python3 +deps=charm-tools +commands = charm-proof + +[testenv:func] +basepython = python3 +commands = python -c "print('Done')" + +[testenv:venv] +commands = {posargs} diff --git a/installers/charm/lint.sh b/installers/charm/lint.sh new file mode 100755 index 00000000..1a4fa803 --- /dev/null +++ b/installers/charm/lint.sh @@ -0,0 +1,27 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#!/bin/bash +set -eux +function lint() { + cd $1 + tox -e lint + cd .. +} + +lint 'lcm-k8s' +lint 'mon-k8s' +lint 'nbi-k8s' +lint 'pol-k8s' +lint 'ro-k8s' +lint 'ui-k8s' diff --git a/installers/charm/mon-k8s/.yamllint.yaml b/installers/charm/mon-k8s/.yamllint.yaml new file mode 100644 index 00000000..d24a69d9 --- /dev/null +++ b/installers/charm/mon-k8s/.yamllint.yaml @@ -0,0 +1,24 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +extends: default + +yaml-files: + - "*.yaml" + - "*.yml" + - ".yamllint" +ignore: | + reactive/ + .tox + release/ diff --git a/installers/charm/mon-k8s/README.md b/installers/charm/mon-k8s/README.md new file mode 100644 index 00000000..9b09230e --- /dev/null +++ b/installers/charm/mon-k8s/README.md @@ -0,0 +1,95 @@ + + +# Overview + +MON for Juju CAAS + +## Testing + +The tests of this charm are done using tox and Zaza. + + + +### Prepare environment + +The machine in which the tests are run needs access to a juju k8s controller. The easiest way to approach this is by executing the following commands: + +``` +sudo apt install tox -y +sudo snap install microk8s --classic +sudo snap install juju + +microk8s.status --wait-ready +microk8s.enable storage dashboard dns + +juju bootstrap microk8s k8s-cloud +``` + +If /usr/bin/python does not exist, you should probably need to do this: + +``` +sudo ln -s /usr/bin/python3 /usr/bin/python +``` +### Build Charm + +**Download dependencies:** + +``` +mkdir -p ~/charm/layers ~/charm/build ~/charm/interfaces +git clone https://git.launchpad.net/canonical-osm ~/canonical-osm + +cd ~/charm/layers +git clone https://git.launchpad.net/charm-osm-mon mon-k8s +git clone https://git.launchpad.net/charm-osm-ro ro-k8s +git clone https://git.launchpad.net/charm-k8s-kafka kafka-k8s ~/charm/interfaces +git clone https://git.launchpad.net/charm-k8s-zookeeper zookeeper-k8s +git clone https://git.launchpad.net/charm-k8s-mongodb mongodb-k8s + +cd ~/charm/interfaces +mv ~/canonical-osm/charms/interfaces/* . +git clone https://git.launchpad.net/interface-mongodb mongodb + +``` + +**Charm structure:** + +``` +├── config.yaml +├── icon.svg +├── layer.yaml +├── metadata.yaml +├── reactive +│ ├── spec_template.yaml +│ └── mon.py +├── README.md +└── tox.ini +``` + +**Setup environment variables:** + +``` +export CHARM_LAYERS_DIR=~/charm/layers +export CHARM_BUILD_DIR=~/charm/build +export CHARM_INTERFACES_DIR=~/charm/interfaces +``` + +**Build:** + +``` +mkdir ~/charm/layers/mon-k8s/tests/build +charm build ~/charm/layers/mon-k8s +charm build ~/charm/layers/kafka-k8s +charm build ~/charm/layers/zookeeper-k8s +charm build ~/charm/layers/mongodb-k8s +mv ~/charm/build/* ~/charm/layers/mon-k8s/tests/build/ +``` diff --git a/installers/charm/mon-k8s/config.yaml b/installers/charm/mon-k8s/config.yaml new file mode 100644 index 00000000..f8ea2358 --- /dev/null +++ b/installers/charm/mon-k8s/config.yaml @@ -0,0 +1,70 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +options: + advertised-hostname: + description: MON Hostname + type: string + default: "mon-k8s" + advertised-port: + description: MON Port + type: int + default: 8000 + OSMMON_OPENSTACK_DEFAULT_GRANULARITY: + description: Openstack default granularity + type: int + default: 300 + OSMMON_GLOBAL_REQUEST_TIMEOUT: + description: Global request timeout + type: int + default: 10 + OSMMON_GLOBAL_LOGLEVEL: + description: Global LOGLEVEL + type: string + default: INFO + OSMMON_DATABASE_COMMONKEY: + description: Database COMMONKEY + type: string + default: osm + OSMMON_COLLECTOR_INTERVAL: + description: Collector interval + type: int + default: 30 + OSMMON_EVALUATOR_INTERVAL: + description: Evaluator interval + type: int + default: 30 + vca_host: + type: string + description: "The VCA host." + default: "admin" + vca_user: + type: string + description: "The VCA user name." + default: "admin" + vca_password: + type: string + description: "The VCA user password." + default: "secret" + vca_cacert: + type: string + description: "The VCA cacert." + default: "" + use_external_vca: + description: Use an external VCA (deprecated) + type: boolean + default: false + image: + type: string + description: OCI image + default: opensourcemano/mon:7 diff --git a/installers/charm/mon-k8s/icon.svg b/installers/charm/mon-k8s/icon.svg new file mode 100644 index 00000000..da31b4ab --- /dev/null +++ b/installers/charm/mon-k8s/icon.svg @@ -0,0 +1,118 @@ + + + + + + + + image/svg+xml + + eclispe-che + + + + + + eclispe-che + Created with Sketch. + + + + + + + + + + + + + + + + + diff --git a/installers/charm/mon-k8s/layer.yaml b/installers/charm/mon-k8s/layer.yaml new file mode 100644 index 00000000..060f454d --- /dev/null +++ b/installers/charm/mon-k8s/layer.yaml @@ -0,0 +1,22 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +includes: + - "layer:caas-base" + - "layer:status" + - "layer:leadership" + - "layer:osm-common" + - "interface:kafka" + - "interface:mongodb" + - "interface:prometheus" +repo: https://code.launchpad.net/osm-k8s-bundle diff --git a/installers/charm/mon-k8s/metadata.yaml b/installers/charm/mon-k8s/metadata.yaml new file mode 100644 index 00000000..41c33c5d --- /dev/null +++ b/installers/charm/mon-k8s/metadata.yaml @@ -0,0 +1,37 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: mon-k8s +summary: "OSM Monitoring Interface (MON)" +maintainers: + - "SolutionsQA " +description: | + A CAAS charm to deploy OSM's Monitoring Interface (MON) +tags: + - "application" +series: + - "kubernetes" +requires: + kafka: + interface: kafka + mongo: + interface: mongodb + prometheus: + interface: prometheus +storage: + database: + type: filesystem + location: /app/database +deployment: + type: stateful + service: cluster diff --git a/installers/charm/mon-k8s/reactive/mon_k8s.py b/installers/charm/mon-k8s/reactive/mon_k8s.py new file mode 100644 index 00000000..9eae0b19 --- /dev/null +++ b/installers/charm/mon-k8s/reactive/mon_k8s.py @@ -0,0 +1,115 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +from charms.reactive import endpoint_from_flag +from charms.layer.caas_base import pod_spec_set +from charms.reactive import when, when_not, hook +from charms.reactive.flags import set_flag, clear_flag +from charmhelpers.core.hookenv import log, metadata, config +from charms import layer + + +@hook("upgrade-charm") +@when("leadership.is_leader") +def upgrade(): + clear_flag("mon-k8s.configured") + + +@when("config.changed") +@when("leadership.is_leader") +def restart(): + clear_flag("mon-k8s.configured") + + +@when_not("kafka.ready") +@when_not("mon-k8s.configured") +def waiting_for_kafka(): + layer.status.waiting("Waiting for kafka to be ready") + + +@when_not("mongo.ready") +@when_not("mon-k8s.configured") +def waiting_for_mongo(): + layer.status.waiting("Waiting for mongo to be ready") + + +@when_not("endpoint.prometheus.available") +@when_not("mon-k8s.configured") +def waiting_for_prometheus(): + layer.status.waiting("Waiting for prometheus to be ready") + + +@when("kafka.ready", "mongo.ready", "endpoint.prometheus.available") +@when_not("mon-k8s.configured") +@when("leadership.is_leader") +def configure(): + layer.status.maintenance("Configuring mon container") + try: + kafka = endpoint_from_flag("kafka.ready") + mongo = endpoint_from_flag("mongo.ready") + prometheus = endpoint_from_flag("endpoint.prometheus.available") + + if kafka and mongo and prometheus: + kafka_units = kafka.kafkas() + kafka_unit = kafka_units[0] + + mongo_uri = mongo.connection_string() + log("Mongo URI: {}".format(mongo_uri)) + + prometheus_url = prometheus.targets()[0]["targets"][0] + + if mongo_uri and kafka_unit["host"]: + + spec = make_pod_spec( + kafka_unit["host"], kafka_unit["port"], mongo_uri, prometheus_url + ) + + log("set pod spec:\n{}".format(spec)) + pod_spec_set(spec) + set_flag("mon-k8s.configured") + except Exception as e: + layer.status.blocked("k8s spec failed to deploy: {}".format(e)) + + +@when("mon-k8s.configured") +def set_mon_active(): + layer.status.active("ready") + + +def make_pod_spec(kafka_host, kafka_port, mongo_uri, prometheus_url): + """Make pod specification for Kubernetes + + Args: + kafka_host (str): Kafka hostname or IP + kafka_port (int): Kafka port + mongo_uri (str): Mongo URI + Returns: + pod_spec: Pod specification for Kubernetes + """ + + with open("reactive/spec_template.yaml") as spec_file: + pod_spec_template = spec_file.read() + + md = metadata() + cfg = config() + + data = { + "name": md.get("name"), + "docker_image": cfg.get("image"), + "kafka_host": kafka_host, + "kafka_port": kafka_port, + "mongo_uri": mongo_uri, + "prometheus_url": prometheus_url, + } + data.update(cfg) + return pod_spec_template % data diff --git a/installers/charm/mon-k8s/reactive/spec_template.yaml b/installers/charm/mon-k8s/reactive/spec_template.yaml new file mode 100644 index 00000000..c81e5a16 --- /dev/null +++ b/installers/charm/mon-k8s/reactive/spec_template.yaml @@ -0,0 +1,56 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +version: 2 +containers: + - name: %(name)s + image: %(docker_image)s + ports: + - containerPort: %(advertised-port)s + protocol: TCP + config: + ALLOW_ANONYMOUS_LOGIN: 'yes' + OSMMON_MESSAGE_DRIVER: kafka + OSMMON_MESSAGE_HOST: %(kafka_host)s + OSMMON_MESSAGE_PORT: %(kafka_port)s + + OSMMON_DATABASE_DRIVER: mongo + OSMMON_DATABASE_URI: %(mongo_uri)s + + OSMMON_OPENSTACK_DEFAULT_GRANULARITY: %(OSMMON_OPENSTACK_DEFAULT_GRANULARITY)s + OSMMON_GLOBAL_REQUEST_TIMEOUT: %(OSMMON_GLOBAL_REQUEST_TIMEOUT)s + OSMMON_GLOBAL_LOGLEVEL: %(OSMMON_GLOBAL_LOGLEVEL)s + OSMMON_VCA_HOST: %(vca_host)s + OSMMON_VCA_USER: %(vca_user)s + OSMMON_VCA_SECRET: %(vca_password)s + OSMMON_VCA_CACERT: %(vca_cacert)s + OSMMON_DATABASE_COMMONKEY: %(OSMMON_DATABASE_COMMONKEY)s + OSMMON_COLLECTOR_INTERVAL: %(OSMMON_COLLECTOR_INTERVAL)s + OSMMON_EVALUATOR_INTERVAL: %(OSMMON_EVALUATOR_INTERVAL)s + OSMMON_PROMETHEUS_URL: http://%(prometheus_url)s + kubernetes: + readinessProbe: + tcpSocket: + port: %(advertised-port)s + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 + livenessProbe: + tcpSocket: + port: %(advertised-port)s + initialDelaySeconds: 45 + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 \ No newline at end of file diff --git a/installers/charm/mon-k8s/tox.ini b/installers/charm/mon-k8s/tox.ini new file mode 100644 index 00000000..1954b2f9 --- /dev/null +++ b/installers/charm/mon-k8s/tox.ini @@ -0,0 +1,60 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +[tox] +envlist = pep8 +skipsdist = True + +[testenv] +setenv = VIRTUAL_ENV={envdir} + PYTHONHASHSEED=0 +whitelist_externals = juju + sleep +passenv = HOME TERM CS_API_* OS_* AMULET_* +install_command = + pip install {opts} {packages} + +[testenv:build] +basepython = python3 +passenv=HTTP_PROXY HTTPS_PROXY NO_PROXY +setenv = CHARM_LAYERS_DIR = ../layers + CHARM_INTERFACES_DIR = ../interfaces/ +whitelist_externals = git + charm + rm + mv +commands = + rm -rf release + rm -rf ../interfaces/mongodb + git clone https://git.launchpad.net/interface-mongodb ../interfaces/mongodb + charm build . --build-dir /tmp + mv /tmp/mon-k8s/ release/ + +[testenv:lint] +basepython = python3 +deps = + black + yamllint + flake8 +commands = + black --check --diff . --exclude "release/|.tox/" + yamllint . + flake8 reactive/ --max-line-length=100 + +[testenv:pep8] +basepython = python3 +deps=charm-tools +commands = charm-proof + +[testenv:venv] +commands = {posargs} diff --git a/installers/charm/nbi-k8s/.yamllint.yaml b/installers/charm/nbi-k8s/.yamllint.yaml new file mode 100644 index 00000000..d24a69d9 --- /dev/null +++ b/installers/charm/nbi-k8s/.yamllint.yaml @@ -0,0 +1,24 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +extends: default + +yaml-files: + - "*.yaml" + - "*.yml" + - ".yamllint" +ignore: | + reactive/ + .tox + release/ diff --git a/installers/charm/nbi-k8s/README.md b/installers/charm/nbi-k8s/README.md new file mode 100755 index 00000000..b4dc3d26 --- /dev/null +++ b/installers/charm/nbi-k8s/README.md @@ -0,0 +1,95 @@ + + +# Overview + +NBI for Juju CAAS + +## Testing + +The tests of this charm are done using tox and Zaza. + + + +### Prepare environment + +The machine in which the tests are run needs access to a juju k8s controller. The easiest way to approach this is by executing the following commands: + +``` +sudo apt install tox -y +sudo snap install microk8s --classic +sudo snap install juju + +microk8s.status --wait-ready +microk8s.enable storage dashboard dns + +juju bootstrap microk8s k8s-cloud +``` + +If /usr/bin/python does not exist, you should probably need to do this: + +``` +sudo ln -s /usr/bin/python3 /usr/bin/python +``` +### Build Charm + +**Download dependencies:** + +``` +mkdir -p ~/charm/layers ~/charm/build ~/charm/interfaces +git clone https://git.launchpad.net/canonical-osm ~/canonical-osm + +cd ~/charm/layers +git clone https://git.launchpad.net/charm-osm-nbi nbi-k8s +git clone https://git.launchpad.net/charm-osm-ro ro-k8s +git clone https://git.launchpad.net/charm-k8s-kafka kafka-k8s ~/charm/interfaces +git clone https://git.launchpad.net/charm-k8s-zookeeper zookeeper-k8s +git clone https://git.launchpad.net/charm-k8s-mongodb mongodb-k8s + +cd ~/charm/interfaces +mv ~/canonical-osm/charms/interfaces/* . +git clone https://git.launchpad.net/interface-mongodb mongodb + +``` + +**Charm structure:** + +``` +├── config.yaml +├── icon.svg +├── layer.yaml +├── metadata.yaml +├── reactive +│ ├── spec_template.yaml +│ └── nbi.py +├── README.md +└── tox.ini +``` + +**Setup environment variables:** + +``` +export CHARM_LAYERS_DIR=~/charm/layers +export CHARM_BUILD_DIR=~/charm/build +export CHARM_INTERFACES_DIR=~/charm/interfaces +``` + +**Build:** + +``` +mkdir ~/charm/layers/nbi-k8s/tests/build +charm build ~/charm/layers/nbi-k8s +charm build ~/charm/layers/kafka-k8s +charm build ~/charm/layers/zookeeper-k8s +charm build ~/charm/layers/mongodb-k8s +mv ~/charm/build/* ~/charm/layers/nbi-k8s/tests/build/ +``` diff --git a/installers/charm/nbi-k8s/config.yaml b/installers/charm/nbi-k8s/config.yaml new file mode 100755 index 00000000..129ff30b --- /dev/null +++ b/installers/charm/nbi-k8s/config.yaml @@ -0,0 +1,38 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +options: + advertised-hostname: + description: NBI Hostname + type: string + default: "nbi-k8s" + advertised-port: + description: NBI Port + type: int + default: 9999 + log_level: + description: "Log Level" + type: string + default: "DEBUG" + DATABASE_COMMONKEY: + description: Database COMMON KEY + type: string + default: osm + image: + type: string + description: OCI image + default: opensourcemano/nbi:7 + auth-backend: + type: string + description: Authentication backend ('internal' or 'keystone') + default: internal diff --git a/installers/charm/nbi-k8s/icon.svg b/installers/charm/nbi-k8s/icon.svg new file mode 100644 index 00000000..da31b4ab --- /dev/null +++ b/installers/charm/nbi-k8s/icon.svg @@ -0,0 +1,118 @@ + + + + + + + + image/svg+xml + + eclispe-che + + + + + + eclispe-che + Created with Sketch. + + + + + + + + + + + + + + + + + diff --git a/installers/charm/nbi-k8s/layer.yaml b/installers/charm/nbi-k8s/layer.yaml new file mode 100644 index 00000000..8bc365f9 --- /dev/null +++ b/installers/charm/nbi-k8s/layer.yaml @@ -0,0 +1,25 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +includes: + - "layer:caas-base" + - "layer:status" + - "layer:leadership" + - "layer:osm-common" + - "interface:kafka" + - "interface:mongodb" + - "interface:osm-nbi" + - "interface:prometheus" + - "interface:keystone" + +repo: https://code.launchpad.net/osm-k8s-bundle diff --git a/installers/charm/nbi-k8s/metadata.yaml b/installers/charm/nbi-k8s/metadata.yaml new file mode 100755 index 00000000..404cd2f0 --- /dev/null +++ b/installers/charm/nbi-k8s/metadata.yaml @@ -0,0 +1,42 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: "nbi-k8s" +summary: "OSM Northbound Interface (NBI)" +maintainers: + - "SolutionsQA " +description: | + A CAAS charm to deploy OSM's Northbound Interface (NBI). +tags: + - "application" +series: + - "kubernetes" +requires: + kafka: + interface: kafka + mongo: + interface: mongodb + keystone: + interface: keystone + prometheus: + interface: prometheus +provides: + nbi: + interface: osm-nbi +storage: + packages: + type: filesystem + location: /app/storage +deployment: + type: stateful + service: cluster diff --git a/installers/charm/nbi-k8s/reactive/nbi.py b/installers/charm/nbi-k8s/reactive/nbi.py new file mode 100644 index 00000000..edcdd4cc --- /dev/null +++ b/installers/charm/nbi-k8s/reactive/nbi.py @@ -0,0 +1,226 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +from charms.layer.caas_base import pod_spec_set +from charms.reactive import endpoint_from_flag +from charms.reactive import when, when_not, hook +from charms.reactive.flags import set_flag, clear_flag +from charmhelpers.core.hookenv import ( + log, + metadata, + config, +) +from charms import layer +from charms.osm.k8s import get_service_ip +import urllib.parse +import yaml +import traceback + + +@hook("upgrade-charm") +@when("leadership.is_leader") +def upgrade(): + clear_flag("nbi-k8s.configured") + + +@when("config.changed") +@when("leadership.is_leader") +def restart(): + clear_flag("nbi-k8s.configured") + + +@when("config.changed.auth-backend") +def clear_keystone_ready(): + clear_flag("keystone.ready") + + +@when_not("kafka.ready") +@when_not("nbi-k8s.configured") +def waiting_for_kafka(): + layer.status.waiting("Waiting for kafka to be ready") + + +@when_not("mongo.ready") +@when_not("nbi-k8s.configured") +def waiting_for_mongo(): + layer.status.waiting("Waiting for mongo to be ready") + + +@when_not("endpoint.prometheus.available") +@when_not("nbi-k8s.configured") +def waiting_for_prometheus(): + layer.status.waiting("Waiting for prometheus to be ready") + + +@when_not("keystone.ready") +@when_not("nbi-k8s.configured") +def waiting_for_keystone(): + auth_backend = config().get("auth-backend") + if auth_backend == "keystone": + layer.status.waiting("Waiting for Keystone to be ready") + else: + set_flag("keystone.ready") + + +@when("kafka.ready", "mongo.ready", "endpoint.prometheus.available", "keystone.ready") +@when_not("nbi-k8s.configured") +@when("leadership.is_leader") +def configure(): + layer.status.maintenance("Configuring nbi container") + try: + kafka = endpoint_from_flag("kafka.ready") + mongo = endpoint_from_flag("mongo.ready") + prometheus = endpoint_from_flag("endpoint.prometheus.available") + + if kafka and mongo and prometheus: + kafka_units = kafka.kafkas() + kafka_unit = kafka_units[0] + + mongo_uri = mongo.connection_string() + log("Mongo URI: {}".format(mongo_uri)) + + prometheus_uri = prometheus.targets()[0]["targets"][0] + + if ( + mongo_uri + and kafka_unit["host"] + and kafka_unit["port"] + and prometheus_uri + ): + spec = yaml.load( + make_pod_spec( + kafka_unit["host"], + kafka_unit["port"], + mongo_uri, + prometheus_uri, + ) + ) + + auth_backend = config().get("auth-backend") + + if auth_backend == "keystone": + keystone = endpoint_from_flag("keystone.ready") + if keystone: + keystone_units = keystone.keystones() + keystone_unit = keystone_units[0] + if ( + keystone_unit["host"] + and keystone_unit["port"] + and keystone_unit["user_domain_name"] + and keystone_unit["project_domain_name"] + and keystone_unit["username"] + and keystone_unit["password"] + and keystone_unit["service"] + ): + auth_keystone = { + "OSMNBI_AUTHENTICATION_BACKEND": "keystone", + "OSMNBI_AUTHENTICATION_AUTH_URL": keystone_unit["host"], + "OSMNBI_AUTHENTICATION_AUTH_PORT": keystone_unit[ + "port" + ], + "OSMNBI_AUTHENTICATION_USER_DOMAIN_NAME": keystone_unit[ + "user_domain_name" + ], + "OSMNBI_AUTHENTICATION_PROJECT_DOMAIN_NAME": keystone_unit[ + "project_domain_name" + ], + "OSMNBI_AUTHENTICATION_SERVICE_USERNAME": keystone_unit[ + "username" + ], + "OSMNBI_AUTHENTICATION_SERVICE_PASSWORD": keystone_unit[ + "password" + ], + "OSMNBI_AUTHENTICATION_SERVICE_PROJECT": keystone_unit[ + "service" + ], + } + spec["containers"][0]["config"].update(auth_keystone) + elif auth_backend == "internal": + spec["containers"][0]["config"][ + "OSMNBI_AUTHENTICATION_BACKEND" + ] = auth_backend + else: + layer.status.blocked( + "Unknown authentication method: {}".format(auth_backend) + ) + raise + log("set pod spec:\n{}".format(spec)) + pod_spec_set(spec) + set_flag("nbi-k8s.configured") + except Exception as e: + layer.status.blocked("k8s spec failed to deploy: {}".format(e)) + log("Error: {}".format(traceback.format_exc())) + + +@when("nbi-k8s.configured") +def set_nbi_active(): + layer.status.active("ready") + + +@when("nbi-k8s.configured", "nbi.joined") +def send_config(): + layer.status.maintenance("Sending NBI configuration") + try: + nbi = endpoint_from_flag("nbi.joined") + if nbi: + service_ip = get_service_ip("nbi") + if service_ip: + nbi.send_connection(service_ip, get_nbi_port()) + clear_flag("nbi.joined") + except Exception as e: + log("Fail sending NBI configuration: {}".format(e)) + + +def make_pod_spec(kafka_host, kafka_port, mongo_uri, prometheus_uri): + """Make pod specification for Kubernetes + + Args: + kafka_host (str): Kafka hostname or IP + kafka_port (int): Kafka port + mongo_uri (str): Mongo URI + prometheus_uri (str): Prometheus URI + Returns: + pod_spec: Pod specification for Kubernetes + """ + + with open("reactive/spec_template.yaml") as spec_file: + pod_spec_template = spec_file.read() + + md = metadata() + cfg = config() + prometheus_host, prometheus_port = parse_hostport(prometheus_uri) + data = { + "name": md.get("name"), + "docker_image": cfg.get("image"), + "mongo_uri": mongo_uri, + "kafka_host": "{}".format(kafka_host), + "kafka_port": "{}".format(kafka_port), + "prometheus_host": "{}".format(prometheus_host), + "prometheus_port": "{}".format(prometheus_port), + } + data.update(cfg) + + return pod_spec_template % data + + +def parse_hostport(uri): + if "//" in uri: + uri = uri.split("//")[1] + result = urllib.parse.urlsplit("//" + uri) + return result.hostname, result.port + + +def get_nbi_port(): + """Returns NBI port""" + cfg = config() + return cfg.get("advertised-port") diff --git a/installers/charm/nbi-k8s/reactive/spec_template.yaml b/installers/charm/nbi-k8s/reactive/spec_template.yaml new file mode 100644 index 00000000..18adec48 --- /dev/null +++ b/installers/charm/nbi-k8s/reactive/spec_template.yaml @@ -0,0 +1,41 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +version: 2 +containers: + - name: %(name)s + image: %(docker_image)s + ports: + - containerPort: %(advertised-port)s + protocol: TCP + config: + ALLOW_ANONYMOUS_LOGIN: 'yes' + + OSMNBI_MESSAGE_HOST: %(kafka_host)s + OSMNBI_MESSAGE_DRIVER: kafka + OSMNBI_MESSAGE_PORT: %(kafka_port)s + + OSMNBI_DATABASE_DRIVER: mongo + OSMNBI_DATABASE_URI: %(mongo_uri)s + OSMNBI_DATABASE_COMMONKEY: %(DATABASE_COMMONKEY)s + + OSMNBI_STORAGE_DRIVER: mongo + OSMNBI_STORAGE_PATH: /app/storage + OSMNBI_STORAGE_COLLECTION: files + OSMNBI_STORAGE_URI: %(mongo_uri)s + + OSMNBI_STATIC_DIR: /app/osm_nbi/html_public + + OSMNBI_PROMETHEUS_HOST: %(prometheus_host)s + OSMNBI_PROMETHEUS_PORT: %(prometheus_port)s + OSMNBI_LOG_LEVEL: %(log_level)s \ No newline at end of file diff --git a/installers/charm/nbi-k8s/tox.ini b/installers/charm/nbi-k8s/tox.ini new file mode 100644 index 00000000..537014a5 --- /dev/null +++ b/installers/charm/nbi-k8s/tox.ini @@ -0,0 +1,60 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +[tox] +envlist = pep8 +skipsdist = True + +[testenv] +setenv = VIRTUAL_ENV={envdir} + PYTHONHASHSEED=0 +whitelist_externals = juju + sleep +passenv = HOME TERM CS_API_* OS_* AMULET_* +install_command = + pip install {opts} {packages} + +[testenv:build] +basepython = python3 +passenv=HTTP_PROXY HTTPS_PROXY NO_PROXY +setenv = CHARM_LAYERS_DIR = ../layers + CHARM_INTERFACES_DIR = ../interfaces/ +whitelist_externals = git + charm + rm + mv +commands = + rm -rf release + rm -rf ../interfaces/mongodb + git clone https://git.launchpad.net/interface-mongodb ../interfaces/mongodb + charm build . --build-dir /tmp + mv /tmp/nbi-k8s/ release/ + +[testenv:lint] +basepython = python3 +deps = + black + yamllint + flake8 +commands = + black --check --diff . --exclude "release/|.tox/" + yamllint . + flake8 reactive/ --max-line-length=100 + +[testenv:pep8] +basepython = python3 +deps=charm-tools +commands = charm-proof + +[testenv:venv] +commands = {posargs} diff --git a/installers/charm/nbi-k8s/wheelhouse.txt b/installers/charm/nbi-k8s/wheelhouse.txt new file mode 100644 index 00000000..7c2badcd --- /dev/null +++ b/installers/charm/nbi-k8s/wheelhouse.txt @@ -0,0 +1,14 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +PyYAML \ No newline at end of file diff --git a/installers/charm/pol-k8s/.yamllint.yaml b/installers/charm/pol-k8s/.yamllint.yaml new file mode 100644 index 00000000..d24a69d9 --- /dev/null +++ b/installers/charm/pol-k8s/.yamllint.yaml @@ -0,0 +1,24 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +extends: default + +yaml-files: + - "*.yaml" + - "*.yml" + - ".yamllint" +ignore: | + reactive/ + .tox + release/ diff --git a/installers/charm/pol-k8s/README.md b/installers/charm/pol-k8s/README.md new file mode 100644 index 00000000..a0227ac7 --- /dev/null +++ b/installers/charm/pol-k8s/README.md @@ -0,0 +1,95 @@ + + +# Overview + +POL for Juju CAAS + +## Testing + +The tests of this charm are done using tox and Zaza. + + + +### Prepare environment + +The machine in which the tests are run needs access to a juju k8s controller. The easiest way to approach this is by executing the following commands: + +``` +sudo apt install tox -y +sudo snap install microk8s --classic +sudo snap install juju + +microk8s.status --wait-ready +microk8s.enable storage dashboard dns + +juju bootstrap microk8s k8s-cloud +``` + +If /usr/bin/python does not exist, you should probably need to do this: + +``` +sudo ln -s /usr/bin/python3 /usr/bin/python +``` +### Build Charm + +**Download dependencies:** + +``` +mkdir -p ~/charm/layers ~/charm/build ~/charm/interfaces +git clone https://git.launchpad.net/canonical-osm ~/canonical-osm + +cd ~/charm/layers +git clone https://git.launchpad.net/charm-osm-pol pol-k8s +git clone https://git.launchpad.net/charm-osm-ro ro-k8s +git clone https://git.launchpad.net/charm-k8s-kafka kafka-k8s ~/charm/interfaces +git clone https://git.launchpad.net/charm-k8s-zookeeper zookeeper-k8s +git clone https://git.launchpad.net/charm-k8s-mongodb mongodb-k8s + +cd ~/charm/interfaces +mv ~/canonical-osm/charms/interfaces/* . +git clone https://git.launchpad.net/interface-mongodb mongodb + +``` + +**Charm structure:** + +``` +├── config.yaml +├── icon.svg +├── layer.yaml +├── metadata.yaml +├── reactive +│ ├── spec_template.yaml +│ └── pol.py +├── README.md +└── tox.ini +``` + +**Setup environment variables:** + +``` +export CHARM_LAYERS_DIR=~/charm/layers +export CHARM_BUILD_DIR=~/charm/build +export CHARM_INTERFACES_DIR=~/charm/interfaces +``` + +**Build:** + +``` +mkdir ~/charm/layers/pol-k8s/tests/build +charm build ~/charm/layers/pol-k8s +charm build ~/charm/layers/kafka-k8s +charm build ~/charm/layers/zookeeper-k8s +charm build ~/charm/layers/mongodb-k8s +mv ~/charm/build/* ~/charm/layers/pol-k8s/tests/build/ +``` diff --git a/installers/charm/pol-k8s/config.yaml b/installers/charm/pol-k8s/config.yaml new file mode 100644 index 00000000..59f9d689 --- /dev/null +++ b/installers/charm/pol-k8s/config.yaml @@ -0,0 +1,26 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +options: + advertised-hostname: + description: POL Hostname + type: string + default: "pol-k8s" + log_level: + description: "Log Level" + type: string + default: "INFO" + image: + type: string + description: OCI image + default: opensourcemano/pol:7 diff --git a/installers/charm/pol-k8s/icon.svg b/installers/charm/pol-k8s/icon.svg new file mode 100644 index 00000000..da31b4ab --- /dev/null +++ b/installers/charm/pol-k8s/icon.svg @@ -0,0 +1,118 @@ + + + + + + + + image/svg+xml + + eclispe-che + + + + + + eclispe-che + Created with Sketch. + + + + + + + + + + + + + + + + + diff --git a/installers/charm/pol-k8s/layer.yaml b/installers/charm/pol-k8s/layer.yaml new file mode 100644 index 00000000..4ed526bf --- /dev/null +++ b/installers/charm/pol-k8s/layer.yaml @@ -0,0 +1,21 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +includes: + - "layer:caas-base" + - "layer:status" + - "layer:leadership" + - "layer:osm-common" + - "interface:kafka" + - "interface:mongodb" +repo: https://code.launchpad.net/osm-k8s-bundle diff --git a/installers/charm/pol-k8s/metadata.yaml b/installers/charm/pol-k8s/metadata.yaml new file mode 100644 index 00000000..9e8d646d --- /dev/null +++ b/installers/charm/pol-k8s/metadata.yaml @@ -0,0 +1,35 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: pol-k8s +summary: "OSM Policy Module (POL)" +maintainers: + - "SolutionsQA " +description: | + A CAAS charm to deploy OSM's Policy Module (POL) +tags: + - "application" +series: + - "kubernetes" +requires: + kafka: + interface: kafka + mongo: + interface: mongodb +storage: + database: + type: filesystem + location: /app/database +deployment: + type: stateful + service: cluster diff --git a/installers/charm/pol-k8s/reactive/pol_k8s.py b/installers/charm/pol-k8s/reactive/pol_k8s.py new file mode 100644 index 00000000..c31a3231 --- /dev/null +++ b/installers/charm/pol-k8s/reactive/pol_k8s.py @@ -0,0 +1,101 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +from charms.reactive import endpoint_from_flag +from charms.layer.caas_base import pod_spec_set +from charms.reactive import when, when_not, hook +from charms.reactive.flags import set_flag, clear_flag +from charmhelpers.core.hookenv import log, metadata, config +from charms import layer + + +@hook("upgrade-charm") +@when("leadership.is_leader") +def upgrade(): + clear_flag("pol-k8s.configured") + + +@when("config.changed") +@when("leadership.is_leader") +def restart(): + clear_flag("pol-k8s.configured") + + +@when_not("kafka.ready") +@when_not("pol-k8s.configured") +def waiting_for_kafka(): + layer.status.waiting("Waiting for kafka to be ready") + + +@when_not("mongo.ready") +@when_not("pol-k8s.configured") +def waiting_for_mongo(): + layer.status.waiting("Waiting for mongo to be ready") + + +@when("kafka.ready", "mongo.ready") +@when_not("pol-k8s.configured") +@when("leadership.is_leader") +def configure(): + layer.status.maintenance("Configuring pol container") + try: + kafka = endpoint_from_flag("kafka.ready") + mongo = endpoint_from_flag("mongo.ready") + + if kafka and mongo: + kafka_units = kafka.kafkas() + kafka_unit = kafka_units[0] + + mongo_uri = mongo.connection_string() + log("Mongo URI: {}".format(mongo_uri)) + + if mongo_uri and kafka_unit["host"]: + spec = make_pod_spec(kafka_unit["host"], kafka_unit["port"], mongo_uri) + + log("set pod spec:\n{}".format(spec)) + pod_spec_set(spec) + set_flag("pol-k8s.configured") + except Exception as e: + layer.status.blocked("k8s spec failed to deploy: {}".format(e)) + + +@when("pol-k8s.configured") +def set_pol_active(): + layer.status.active("ready") + + +def make_pod_spec(kafka_host, kafka_port, mongo_uri): + """Make pod specification for Kubernetes + + Args: + kafka_host (str): Kafka hostname or IP + kafka_port (int): Kafka port + mongo_host (str): Mongo URI + Returns: + pod_spec: Pod specification for Kubernetes + """ + + with open("reactive/spec_template.yaml") as spec_file: + pod_spec_template = spec_file.read() + + md = metadata() + cfg = config() + data = { + "name": md.get("name"), + "docker_image": cfg.get("image"), + "kafka_host": kafka_host, + "kafka_port": kafka_port, + "mongo_uri": mongo_uri, + } + data.update(cfg) + return pod_spec_template % data diff --git a/installers/charm/pol-k8s/reactive/spec_template.yaml b/installers/charm/pol-k8s/reactive/spec_template.yaml new file mode 100644 index 00000000..de7d4a21 --- /dev/null +++ b/installers/charm/pol-k8s/reactive/spec_template.yaml @@ -0,0 +1,48 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +version: 2 +containers: + - name: %(name)s + image: %(docker_image)s + ports: + # This is a fake port; lcm doesn't listen, since it's just reading + # and responding to the kafka bus. Fix this in post. + - containerPort: 80 + protocol: TCP + config: + ALLOW_ANONYMOUS_LOGIN: 'yes' + OSMPOL_MESSAGE_DRIVER: kafka + OSMPOL_MESSAGE_HOST: %(kafka_host)s + OSMPOL_MESSAGE_PORT: %(kafka_port)s + + OSMPOL_DATABASE_DRIVER: mongo + OSMPOL_DATABASE_URI: %(mongo_uri)s + + OSMPOL_GLOBAL_LOGLEVEL: %(log_level)s + # kubernetes: + # readinessProbe: + # exec: + # command: ["sh", "-c", "osm-pol-healthcheck || exit 1"] + # periodSeconds: 10 + # timeoutSeconds: 5 + # successThreshold: 1 + # failureThreshold: 3 + # livenessProbe: + # exec: + # command: ["sh", "-c", "osm-pol-healthcheck || exit 1"] + # initialDelaySeconds: 45 + # periodSeconds: 10 + # timeoutSeconds: 5 + # successThreshold: 1 + # failureThreshold: 3 \ No newline at end of file diff --git a/installers/charm/pol-k8s/tox.ini b/installers/charm/pol-k8s/tox.ini new file mode 100644 index 00000000..f8f69908 --- /dev/null +++ b/installers/charm/pol-k8s/tox.ini @@ -0,0 +1,60 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +[tox] +envlist = pep8 +skipsdist = True + +[testenv] +setenv = VIRTUAL_ENV={envdir} + PYTHONHASHSEED=0 +whitelist_externals = juju + sleep +passenv = HOME TERM CS_API_* OS_* AMULET_* +install_command = + pip install {opts} {packages} + +[testenv:build] +basepython = python3 +passenv=HTTP_PROXY HTTPS_PROXY NO_PROXY +setenv = CHARM_LAYERS_DIR = ../layers + CHARM_INTERFACES_DIR = ../interfaces/ +whitelist_externals = git + charm + rm + mv +commands = + rm -rf release + rm -rf ../interfaces/mongodb + git clone https://git.launchpad.net/interface-mongodb ../interfaces/mongodb + charm build . --build-dir /tmp + mv /tmp/pol-k8s/ release/ + +[testenv:lint] +basepython = python3 +deps = + black + yamllint + flake8 +commands = + black --check --diff . --exclude "release/|.tox/" + yamllint . + flake8 reactive/ --max-line-length=100 + +[testenv:pep8] +basepython = python3 +deps=charm-tools +commands = charm-proof + +[testenv:venv] +commands = {posargs} diff --git a/installers/charm/ro-k8s/.yamllint.yaml b/installers/charm/ro-k8s/.yamllint.yaml new file mode 100644 index 00000000..d24a69d9 --- /dev/null +++ b/installers/charm/ro-k8s/.yamllint.yaml @@ -0,0 +1,24 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +extends: default + +yaml-files: + - "*.yaml" + - "*.yml" + - ".yamllint" +ignore: | + reactive/ + .tox + release/ diff --git a/installers/charm/ro-k8s/README.md b/installers/charm/ro-k8s/README.md new file mode 100755 index 00000000..5571622e --- /dev/null +++ b/installers/charm/ro-k8s/README.md @@ -0,0 +1,91 @@ + + +# Overview + +OSM RO charm for Kubernetes + +## Testing + +The tests of this charm are done using tox and Zaza. + + + +### Prepare environment + +The machine in which the tests are run needs access to a juju k8s controller. The easiest way to approach this is by executing the following commands: + +``` +sudo apt install tox -y +sudo snap install microk8s --classic +sudo snap install juju + +microk8s.status --wait-ready +microk8s.enable storage dashboard dns + +juju bootstrap microk8s k8s-cloud +``` + +If /usr/bin/python does not exist, you should probably need to do this: + +``` +sudo ln -s /usr/bin/python3 /usr/bin/python +``` + +### Build Charm + +**Download dependencies:** + +``` +mkdir -p ~/charm/layers ~/charm/build ~/charm/interfaces +git clone https://git.launchpad.net/canonical-osm ~/canonical-osm + +cd ~/charm/layers +git clone https://git.launchpad.net/charm-k8s-mariadb mariadb-k8s +git clone https://git.launchpad.net/charm-osm-ro ro-k8s + +cd ~/charm/interfaces +mv ~/canonical-osm/charms/interfaces/* . +git clone https://git.launchpad.net/interface-osm-ro osm-ro + +``` + +**Charm structure:** + +``` +├── config.yaml +├── icon.svg +├── layer.yaml +├── metadata.yaml +├── reactive +│ ├── spec_template.yaml +│ └── ui.py +├── README.md +└── tox.ini +``` + +**Setup environment variables:** + +``` +export CHARM_LAYERS_DIR=~/charm/layers +export CHARM_BUILD_DIR=~/charm/build +export CHARM_INTERFACES_DIR=~/charm/interfaces +``` + +**Build:** + +``` +mkdir ~/charm/layers/ro-k8s/tests/build +charm build ~/charm/layers/mariadb-k8s +charm build ~/charm/layers/ro-k8s +mv ~/charm/build/* ~/charm/layers/ro-k8s/tests/build/ +``` diff --git a/installers/charm/ro-k8s/config.yaml b/installers/charm/ro-k8s/config.yaml new file mode 100755 index 00000000..c35d114d --- /dev/null +++ b/installers/charm/ro-k8s/config.yaml @@ -0,0 +1,34 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +options: + advertised-port: + description: RO port + type: int + default: 9090 + vim_database: + type: string + description: "The database name." + default: "mano_vim_db" + ro_database: + type: string + description: "The database name." + default: "mano_db" + OPENMANO_TENANT: + type: string + description: "Openmano Tenant" + default: "osm" + image: + type: string + description: OCI image + default: opensourcemano/ro:7 diff --git a/installers/charm/ro-k8s/icon.svg b/installers/charm/ro-k8s/icon.svg new file mode 100644 index 00000000..da31b4ab --- /dev/null +++ b/installers/charm/ro-k8s/icon.svg @@ -0,0 +1,118 @@ + + + + + + + + image/svg+xml + + eclispe-che + + + + + + eclispe-che + Created with Sketch. + + + + + + + + + + + + + + + + + diff --git a/installers/charm/ro-k8s/layer.yaml b/installers/charm/ro-k8s/layer.yaml new file mode 100644 index 00000000..73a7090b --- /dev/null +++ b/installers/charm/ro-k8s/layer.yaml @@ -0,0 +1,22 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +includes: + - "layer:caas-base" + - "layer:status" + - "layer:leadership" + - "layer:osm-common" + - "interface:juju-relation-mysql" + - "interface:osm-ro" + +repo: https://code.launchpad.net/osm-k8s-bundle diff --git a/installers/charm/ro-k8s/metadata.yaml b/installers/charm/ro-k8s/metadata.yaml new file mode 100755 index 00000000..3531e4b2 --- /dev/null +++ b/installers/charm/ro-k8s/metadata.yaml @@ -0,0 +1,36 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: "ro-k8s" +summary: "OSM Resource Orchestrator (RO)" +maintainers: + - "SolutionsQA " +description: | + A CAAS charm to deploy OSM's Resource Orchestrator (RO). +tags: + - "application" +series: + - "kubernetes" +provides: + ro: + interface: osm-ro +requires: + mysql: + interface: mysql +storage: + log: + type: filesystem + location: /var/log/osm +deployment: + type: stateful + service: cluster diff --git a/installers/charm/ro-k8s/reactive/ro.py b/installers/charm/ro-k8s/reactive/ro.py new file mode 100644 index 00000000..afd7f459 --- /dev/null +++ b/installers/charm/ro-k8s/reactive/ro.py @@ -0,0 +1,124 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +from charms.layer.caas_base import pod_spec_set +from charms.reactive import endpoint_from_flag +from charms.reactive import when, when_not, hook +from charms.reactive.flags import set_flag, clear_flag +from charmhelpers.core.hookenv import log, metadata, config +from charms import layer +from charms.osm.k8s import get_service_ip + + +@hook("upgrade-charm") +@when("leadership.is_leader") +def upgrade(): + clear_flag("ro-k8s.configured") + + +@when("config.changed") +@when("leadership.is_leader") +def restart(): + clear_flag("ro-k8s.configured") + + +@when_not("mysql.available") +@when_not("ro-k8s.configured") +def waiting_for_mysql(): + layer.status.waiting("Waiting for mysql to be ready") + + +@when("mysql.available") +@when_not("ro-k8s.configured") +@when("leadership.is_leader") +def configure(): + layer.status.maintenance("Configuring ro container") + try: + mysql = endpoint_from_flag("mysql.available") + + spec = make_pod_spec( + mysql.host(), + mysql.port(), + mysql.user(), + mysql.password(), + mysql.root_password(), + ) + + log("set pod spec:\n{}".format(spec)) + pod_spec_set(spec) + layer.status.active("creating container") + set_flag("ro-k8s.configured") + except Exception as e: + layer.status.blocked("k8s spec failed to deploy: {}".format(e)) + + +@when("ro-k8s.configured") +def set_ro_active(): + layer.status.active("ready") + + +@when("ro-k8s.configured", "ro.joined") +def send_config(): + layer.status.maintenance("Sending RO configuration") + try: + ro = endpoint_from_flag("ro.joined") + if ro: + service_ip = get_service_ip("ro") + if service_ip: + ro.send_connection( + service_ip, get_ro_port(), + ) + clear_flag("ro.joined") + except Exception as e: + log("Fail sending RO configuration: {}".format(e)) + + +def make_pod_spec( + mysql_host, mysql_port, mysql_user, mysql_password, mysql_root_password +): + """Make pod specification for Kubernetes + + Args: + mysql_name (str): RO DB name + mysql_host (str): RO DB host + mysql_port (int): RO DB port + mysql_user (str): RO DB user + mysql_password (str): RO DB password + Returns: + pod_spec: Pod specification for Kubernetes + """ + + with open("reactive/spec_template.yaml") as spec_file: + pod_spec_template = spec_file.read() + + md = metadata() + cfg = config() + + data = { + "name": md.get("name"), + "docker_image": cfg.get("image"), + "mysql_host": mysql_host, + "mysql_port": mysql_port, + "mysql_user": mysql_user, + "mysql_password": mysql_password, + "mysql_root_password": mysql_root_password, + } + data.update(cfg) + + return pod_spec_template % data + + +def get_ro_port(): + """Returns RO port""" + cfg = config() + return cfg.get("advertised-port") diff --git a/installers/charm/ro-k8s/reactive/spec_template.yaml b/installers/charm/ro-k8s/reactive/spec_template.yaml new file mode 100644 index 00000000..f4f7c9c3 --- /dev/null +++ b/installers/charm/ro-k8s/reactive/spec_template.yaml @@ -0,0 +1,55 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +version: 2 +service: + scalePolicy: serial +containers: + - name: %(name)s + image: %(docker_image)s + kubernetes: + readinessProbe: + httpGet: + path: /openmano/tenants + port: %(advertised-port)s + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 + livenessProbe: + tcpSocket: + port: %(advertised-port)s + initialDelaySeconds: 600 + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 + ports: + - containerPort: %(advertised-port)s + protocol: TCP + config: + ALLOW_ANONYMOUS_LOGIN: 'yes' + RO_DB_HOST: %(mysql_host)s + RO_DB_PORT: %(mysql_port)s + RO_DB_NAME: %(ro_database)s + RO_DB_USER: %(mysql_user)s + RO_DB_ROOT_PASSWORD: %(mysql_root_password)s + RO_DB_PASSWORD: %(mysql_password)s + + RO_DB_OVIM_PASSWORD: %(mysql_password)s + RO_DB_OVIM_HOST: %(mysql_host)s + RO_DB_OVIM_PORT: %(mysql_port)s + RO_DB_OVIM_USER: %(mysql_user)s + RO_DB_OVIM_ROOT_PASSWORD: %(mysql_root_password)s + RO_DB_OVIM_NAME: %(vim_database)s + OPENMANO_TENANT: %(OPENMANO_TENANT)s diff --git a/installers/charm/ro-k8s/tox.ini b/installers/charm/ro-k8s/tox.ini new file mode 100644 index 00000000..fef77c01 --- /dev/null +++ b/installers/charm/ro-k8s/tox.ini @@ -0,0 +1,63 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +[tox] +envlist = pep8 +skipsdist = True + +[testenv] +setenv = VIRTUAL_ENV={envdir} + PYTHONHASHSEED=0 +whitelist_externals = juju + sleep +passenv = HOME TERM CS_API_* OS_* AMULET_* +install_command = + pip install {opts} {packages} + + +[testenv:build] +basepython = python3 +passenv=HTTP_PROXY HTTPS_PROXY NO_PROXY +setenv = CHARM_LAYERS_DIR = ../layers + CHARM_INTERFACES_DIR = ../interfaces/ +whitelist_externals = git + charm + rm + mv +commands = + rm -rf release + rm -rf ../interfaces/juju-relation-mysql + rm -rf /tmp/canonical-osm + git clone https://git.launchpad.net/canonical-osm/ /tmp/canonical-osm + mv /tmp/canonical-osm/charms/interfaces/juju-relation-mysql ../interfaces/juju-relation-mysql + charm build . --build-dir /tmp + mv /tmp/ro-k8s/ release/ + +[testenv:lint] +basepython = python3 +deps = + black + yamllint + flake8 +commands = + black --check --diff . --exclude "release/|.tox/" + yamllint . + flake8 reactive/ --max-line-length=100 + +[testenv:pep8] +basepython = python3 +deps=charm-tools +commands = charm-proof + +[testenv:venv] +commands = {posargs} diff --git a/installers/charm/ui-k8s/.yamllint.yaml b/installers/charm/ui-k8s/.yamllint.yaml new file mode 100644 index 00000000..d24a69d9 --- /dev/null +++ b/installers/charm/ui-k8s/.yamllint.yaml @@ -0,0 +1,24 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +extends: default + +yaml-files: + - "*.yaml" + - "*.yml" + - ".yamllint" +ignore: | + reactive/ + .tox + release/ diff --git a/installers/charm/ui-k8s/README.md b/installers/charm/ui-k8s/README.md new file mode 100755 index 00000000..8cfa784a --- /dev/null +++ b/installers/charm/ui-k8s/README.md @@ -0,0 +1,117 @@ + + +# Overview + +UI for Juju CAAS + +## Testing + +The tests of this charm are done using tox and Zaza. + + + +### Prepare environment + +The machine in which the tests are run needs access to a juju k8s controller. The easiest way to approach this is by executing the following commands: + +``` +sudo apt install tox -y +sudo snap install microk8s --classic +sudo snap install juju + +microk8s.status --wait-ready +microk8s.enable storage dashboard dns + +juju bootstrap microk8s k8s-cloud +``` + +If /usr/bin/python does not exist, you should probably need to do this: + +``` +sudo ln -s /usr/bin/python3 /usr/bin/python +``` + +### Build Charm + +**Download dependencies:** + +``` +mkdir -p ~/charm/layers ~/charm/build ~/charm/interfaces +git clone --single-branch --branch master https://git.launchpad.net/canonical-osm ~/canonical-osm + +cd ~/charm/layers +git clone --single-branch --branch=master https://git.launchpad.net/charm-osm-ui ui-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-k8s-grafana grafana-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-k8s-prometheus prometheus-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-k8s-mariadb mariadb-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-osm-lcm lcm-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-osm-mon mon-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-osm-nbi nbi-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-osm-pol pol-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-osm-ro ro-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-k8s-keystone keystone-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-k8s-kafka kafka-k8s ~/charm/interfaces +git clone --single-branch --branch=master https://git.launchpad.net/charm-k8s-zookeeper zookeeper-k8s +git clone --single-branch --branch=master https://git.launchpad.net/charm-k8s-mongodb mongodb-k8s + +cd ~/charm/interfaces +mv ~/canonical-osm/charms/interfaces/* . +git clone https://git.launchpad.net/interface-mongodb mongodb +git clone https://git.launchpad.net/interface-osm-mon mon +git clone https://git.launchpad.net/interface-osm-nbi osm-nbi +git clone https://git.launchpad.net/interface-osm-ro osm-ro + +``` + +**Charm structure:** + +``` +├── config.yaml +├── icon.svg +├── layer.yaml +├── metadata.yaml +├── reactive +│ ├── spec_template.yaml +│ └── ui.py +├── README.md +└── tox.ini +``` + +**Setup environment variables:** + +``` +export CHARM_LAYERS_DIR=~/charm/layers +export CHARM_BUILD_DIR=~/charm/build +export CHARM_INTERFACES_DIR=~/charm/interfaces +``` + +**Build:** + +``` +mkdir ~/charm/layers/ui-k8s/tests/build +export https_proxy=\$PROXY +charm build ~/charm/layers/ui-k8s +charm build ~/charm/layers/grafana-k8s +charm build ~/charm/layers/prometheus-k8s +charm build ~/charm/layers/mariadb-k8s +charm build ~/charm/layers/lcm-k8s +charm build ~/charm/layers/mon-k8s +charm build ~/charm/layers/nbi-k8s +charm build ~/charm/layers/pol-k8s +charm build ~/charm/layers/ro-k8s +charm build ~/charm/layers/keystone-k8s +charm build ~/charm/layers/kafka-k8s +charm build ~/charm/layers/zookeeper-k8s +charm build ~/charm/layers/mongodb-k8s +mv ~/charm/build/* ~/charm/layers/ui-k8s/tests/build/ +``` diff --git a/installers/charm/ui-k8s/config.yaml b/installers/charm/ui-k8s/config.yaml new file mode 100755 index 00000000..1f6338e0 --- /dev/null +++ b/installers/charm/ui-k8s/config.yaml @@ -0,0 +1,30 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +options: + advertised-hostname: + description: Advertised hostname + type: string + default: "ui" + advertised-port: + description: UI Port + type: int + default: 80 + mysql_database: + description: Mysql Database for UI + type: string + default: lwui + image: + type: string + description: OCI image + default: opensourcemano/light-ui:7 diff --git a/installers/charm/ui-k8s/icon.svg b/installers/charm/ui-k8s/icon.svg new file mode 100644 index 00000000..0177a695 --- /dev/null +++ b/installers/charm/ui-k8s/icon.svg @@ -0,0 +1,119 @@ + + + + + + + image/svg+xml + + eclispe-che + + + + + + eclispe-che + Created with Sketch. + + + + + + + + + + + + + + + + + diff --git a/installers/charm/ui-k8s/layer.yaml b/installers/charm/ui-k8s/layer.yaml new file mode 100644 index 00000000..40a4a70a --- /dev/null +++ b/installers/charm/ui-k8s/layer.yaml @@ -0,0 +1,22 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +includes: + - "layer:caas-base" + - "layer:status" + - "layer:leadership" + - "layer:osm-common" + - "interface:juju-relation-mysql" + - "interface:osm-nbi" + +repo: https://code.launchpad.net/osm-k8s-bundle diff --git a/installers/charm/ui-k8s/metadata.yaml b/installers/charm/ui-k8s/metadata.yaml new file mode 100755 index 00000000..5c3fe9ea --- /dev/null +++ b/installers/charm/ui-k8s/metadata.yaml @@ -0,0 +1,31 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +name: "ui-k8s" +summary: "OSM User Interface (UI)" +maintainers: + - "SolutionsQA " +description: | + A CAAS charm to deploy OSM's Northbound Interface (NBI). +tags: + - "application" +series: + - "kubernetes" +requires: + mysql: + interface: mysql + nbi: + interface: osm-nbi +deployment: + type: stateless + service: cluster diff --git a/installers/charm/ui-k8s/reactive/spec_template.yaml b/installers/charm/ui-k8s/reactive/spec_template.yaml new file mode 100644 index 00000000..2bc6f230 --- /dev/null +++ b/installers/charm/ui-k8s/reactive/spec_template.yaml @@ -0,0 +1,42 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +version: 2 +service: + scalePolicy: serial +containers: + - name: %(name)s + image: %(docker_image)s + ports: + - containerPort: %(advertised-port)s + protocol: TCP + config: + ALLOW_ANONYMOUS_LOGIN: 'yes' + OSM_SERVER: %(nbi_host)s + OSMUI_SQL_DATABASE_URI: mysql://root:%(mysql_root_password)s@%(mysql_host)s:%(mysql_port)s/%(mysql_database)s + kubernetes: + readinessProbe: + tcpSocket: + port: %(advertised-port)s + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 + livenessProbe: + tcpSocket: + port: %(advertised-port)s + initialDelaySeconds: 30 + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 \ No newline at end of file diff --git a/installers/charm/ui-k8s/reactive/ui.py b/installers/charm/ui-k8s/reactive/ui.py new file mode 100644 index 00000000..46e57094 --- /dev/null +++ b/installers/charm/ui-k8s/reactive/ui.py @@ -0,0 +1,122 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +from charms.layer.caas_base import pod_spec_set +from charms.reactive import endpoint_from_flag +from charms.reactive import when, when_not, hook +from charms.reactive.flags import set_flag, clear_flag +from charmhelpers.core.hookenv import ( + log, + metadata, + config, +) +from charms import layer + + +@hook("upgrade-charm") +@when("leadership.is_leader") +def upgrade(): + clear_flag("ui-k8s.configured") + + +@when("config.changed") +@when("leadership.is_leader") +def restart(): + clear_flag("ui-k8s.configured") + + +@when_not("mysql.available") +@when_not("ui-k8s.configured") +def waiting_for_mysql(): + layer.status.waiting("Waiting for mysql to be available") + + +@when_not("nbi.ready") +@when_not("ui-k8s.configured") +def waiting_for_nbi(): + layer.status.waiting("Waiting for nbi to be available") + + +@when("mysql.available", "nbi.ready") +@when_not("ui-k8s.configured") +@when("leadership.is_leader") +def configure(): + + layer.status.maintenance("Configuring ui container") + try: + mysql = endpoint_from_flag("mysql.available") + nbi = endpoint_from_flag("nbi.ready") + nbi_unit = nbi.nbis()[0] + nbi_host = "{}".format(nbi_unit["host"]) + spec = make_pod_spec( + mysql.host(), + mysql.port(), + mysql.user(), + mysql.password(), + mysql.root_password(), + nbi_host, + ) + log("set pod spec:\n{}".format(spec)) + pod_spec_set(spec) + set_flag("ui-k8s.configured") + except Exception as e: + layer.status.blocked("k8s spec failed to deploy: {}".format(e)) + + +@when("ui-k8s.configured") +def set_ui_active(): + layer.status.active("ready") + + +def make_pod_spec( + mysql_host, mysql_port, mysql_user, mysql_password, mysql_root_password, nbi_host +): + """Make pod specification for Kubernetes + + Args: + mysql_name (str): UI DB name + mysql_host (str): UI DB host + mysql_port (int): UI DB port + mysql_user (str): UI DB user + mysql_password (str): UI DB password + nbi_uri (str): NBI URI + Returns: + pod_spec: Pod specification for Kubernetes + """ + + with open("reactive/spec_template.yaml") as spec_file: + pod_spec_template = spec_file.read() + + md = metadata() + cfg = config() + + data = { + "name": md.get("name"), + "docker_image": cfg.get("image"), + "mysql_host": mysql_host, + "mysql_port": mysql_port, + "mysql_user": mysql_user, + "mysql_password": mysql_password, + "mysql_root_password": mysql_root_password, + "nbi_host": nbi_host, + } + data.update(cfg) + + return pod_spec_template % data + + +def get_ui_port(): + """Returns UI port""" + cfg = config() + return cfg.get("advertised-port") diff --git a/installers/charm/ui-k8s/tox.ini b/installers/charm/ui-k8s/tox.ini new file mode 100644 index 00000000..50375f39 --- /dev/null +++ b/installers/charm/ui-k8s/tox.ini @@ -0,0 +1,62 @@ +# Copyright 2020 Canonical Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +[tox] +envlist = pep8 +skipsdist = True + +[testenv] +setenv = VIRTUAL_ENV={envdir} + PYTHONHASHSEED=0 +whitelist_externals = juju + sleep +passenv = HOME TERM CS_API_* OS_* AMULET_* +install_command = + pip install {opts} {packages} + +[testenv:build] +basepython = python3 +passenv=HTTP_PROXY HTTPS_PROXY NO_PROXY +setenv = CHARM_LAYERS_DIR = ../layers + CHARM_INTERFACES_DIR = ../interfaces/ +whitelist_externals = git + charm + rm + mv +commands = + rm -rf release + rm -rf ../interfaces/juju-relation-mysql + rm -rf /tmp/canonical-osm + git clone https://git.launchpad.net/canonical-osm/ /tmp/canonical-osm + mv /tmp/canonical-osm/charms/interfaces/juju-relation-mysql ../interfaces/juju-relation-mysql + charm build . --build-dir /tmp + mv /tmp/ui-k8s/ release/ + +[testenv:lint] +basepython = python3 +deps = + black + yamllint + flake8 +commands = + black --check --diff . --exclude "release/|.tox/" + yamllint . + flake8 reactive/ --max-line-length=100 + +[testenv:pep8] +basepython = python3 +deps=charm-tools +commands = charm-proof + +[testenv:venv] +commands = {posargs}