# Helm-chart based Execution Environments

## Introduction to Execution Environments in OSM

OSM's Execution Environments (EE) provide a runtime framework to run day-1 and day-2 primitives, as well as metrics collection for NFs. These EE provide the means for NF-specific management code to run it into a dedicated helm chart, which is deployed into OSM's system cluster. From there, the EE interacts with the managed NF (e.g. via SSH), providing a NF-agnostic mean to manage NFs by OSM.

OSM communicates with its EE to trigger actions via gRPC calls, which are handled by a fronted component (running in a pod) in its constituent helm chart. In order to ease the NF onboarding tasks, there is already a helm chart template available including a fronted element which implements the gRPC interface required by OSM.

This chart template (called `eechart`) is oriented to create a chart with two sub-components: the gRCP front-end (exposed via a Kubernetes service), and an optional back-end, in charge of the interaction with the NF, where onboarder's code required for NF operations is included. This template is oriented to make unnecessary knowing low-level implementation details of the chart structure or gRPC implementation, although all the details are also available for advanced users in the corresponding [repo of the GRPC pod image](https://osm.etsi.org/gitlab/vnf-onboarding/docker-api-fe/).

The purpose of this document is to provide the guidelines for adding primitives based on EE so that it can be operated at runtime by the end-user. The following sections will explain how to create easily EE primitives for your NF using the available template.

## Overview of the EE template

To understand how an EE works, we will retrieve the packages of the template and we will analyse them in detail, describing the structure and how they can be adapted to the needs of specific NFs.

First, we will clone the [OSM packages repository](https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages), which contains all OSM's sample packages. 

```bash
export PACKAGES_FOLDER=$HOME/packages
git clone --recursive https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git ${PACKAGES_FOLDER}
```

Then, we will use the packages `sample_ee_vnf` and `sample_ee_ns` as template. These sample packages define a simple network service consisting of a VNF that has a single Ubuntu-based VDU with two network interfaces, one of them connected to the management network. The VNF also includes various sample day-1 and day-2 primitives based on EE that will be discussed below.

### VNF descriptor template

When defining a NF package, the central file is the NF descriptor. The descriptor models the internal structure of the NF as well as the available day-1 and day-2 primitives, following the [OSM's IM](http://osm-download.etsi.org/repository/osm/debian/ReleaseELEVEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html). In the case of the EE template, this file corresponds to `sample_ee_vnf/sample_ee_vnfd.yaml`.

A pre-requirement to define EE-based primitives is including the appropriate reference in the descriptor to the own EE, and then a reference to the corresponding day-1 and day-2 primitives that are hosted in the EE.

The first is achieved by adding an `execution-environment-list` block, where one or more EEs (there might be more than one) are indicated. This can be seen in the following excerpt of the descriptor template:

```yaml
vnfd:
  description: Basic execution environment example
  df:
  ...
    lcm-operations-configuration:
      operate-vnf-op-config:
        day1-2:
        ...
          execution-environment-list:
          - external-connection-point-ref: vnf-mgmt-ext
            helm-chart: eechart
            id: sample_ee
```

In the example, just before that part, the day-2 primitives and their parameters are defined under `config-primitive` block. Every primitive must refer to an EE (`execution-environment-ref`), has a `name` and optionally a `parameter` list. A parameter is formed by a pair of tag (`name`) and its type (`data-type`).

```yaml
- config-primitive:
  - execution-environment-primitive: run_script
    execution-environment-ref: sample_ee
    name: run_script
    parameter:
    - data-type: STRING
      name: file
    - data-type: STRING
      name: parameters
  - execution-environment-primitive: ansible_playbook
    execution-environment-ref: sample_ee
    name: ansible_playbook
    parameter:
    - data-type: STRING
      name: playbook-name
    - data-type: STRING
      name: app
  - execution-environment-primitive: ping
    execution-environment-ref: sample_ee
    name: ping
    ...
```

If these day-2 primitives are also expected to be executed as day-1 operations, they must be referenced in the block `initial-config-primitive`, where their values would be also set. The `seq` attribute specifies the order in which they will be executed sequentially when the VNF is instantiated (the instantiation will not be successful if any of these primitives fails). The section where day-1 primitives are defined is shown in the following excerpt of the NF template descriptor:

```yaml
initial-config-primitive:
- execution-environment-ref: sample_ee
  name: config
  parameter:
  - name: ssh-hostname
    value: <rw_mgmt_ip>
  - name: ssh-username
    value: ubuntu
  seq: 1
- execution-environment-ref: sample_ee
  name: run_script
  parameter:
  - name: file
    value: install_nginx.sh
  seq: 2
- execution-environment-ref: sample_ee
  name: ansible_playbook
  parameter:
    - name: playbook-name
      value: playbook.yaml
    - name: app
      value: ntp
  seq: 3
  ...
```

In this template, the `config` day-1 primitive (see above) is used to initialise a set of configuration variables with their values, so it must be the first in sequence. In this descriptor, the SSH parameters to connect to the VDU from EE are also set. The `<rw_mgmt_ip>` value in `ssh-hostname` is replaced by the management IP of the NF when the NF is instantiated by OSM. SSH authentication is performed with public/private key pairs, which are transparently managed by OSM if `ssh-access` is configured in the descriptor (the EE generates its own keys, which are injected by OSM in the corresponding VDUs):

```yaml
lcm-operations-configuration:
  operate-vnf-op-config:
    day1-2:
    ...
      config-access:
        ssh-access:
          default-user: ubuntu
          required: true
```

### Helm chart template

In addition to the descriptor, a NF that uses helm chart EE must include the chart definition under the `helm-charts` subdirectory of the package. The NF package template that we are using in this guide already includes the pre-created `eechart` chart, which can be found under the `helm-charts` folder:

```text
helm-charts
└── eechart
    ├── Chart.yaml
    ├── charts
    ├── source
    │   ├── install.sh
    │   ├── install_nginx.sh
    │   ├── mylib.py
    │   ├── playbook.yaml
    │   ├── run_ssh.sh
    │   └── vnf_ee.py
    ├── templates
    └── values.yaml
```

`Chart.yaml`, `values.yaml` and the files in the `templates` and `charts` folders are related to the chart. In this template, they are designed to contain all the necessary configuration data to deploy the gRPC server in the OSM k8s cluster (the name and repository of the chart, type of service, resources, etc.) so that they do not require any modification to define custom primitives, as we will see shortly. Advanced users, however, might what to evolve then to e.g. customize the list of components included in the helm chart, for reusing pre-existing vendor containers, referencing other charts, etc.
As previously discussed, the chart included in the template is designed to be used as-is for the commonest cases of primitives, with minor customizations in the own package. Thus, the chart will use some parts of the own NF package to customize the EE for almost any potential use:

- Any files in the `source` folder of the package will be injected into the fronted pod of the EE when the NF is instantiated. This mechanism is quite useful to include any files required to support the mechanisms of the primitive (e.g. a playbook for an Ansible-based primitive).
- `install.sh`: Bash script to be executed into the EE when the container is created (only once). This script is useful to install additional software (e.g. Python libraries or apt packages) or make some initial configuration on the fronted container, without the need of re-creating the frontend container image from scratch (although it is always possible). In this sample NF package template, the script updates the OS, installs Ansible, the ping binary, and some Python libraries needed by the primitives of the example:

  ```bash
  #!/bin/bash
  ##
  # Licensed under the Apache License, Version 2.0 (the "License"); you may
  # not use this file except in compliance with the License. You may obtain
  # a copy of the License at
  #
  #         http://www.apache.org/licenses/LICENSE-2.0
  #
  # Unless required by applicable law or agreed to in writing, software
  # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
  # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
  # License for the specific language governing permissions and limitations
  # under the License.
  ##

  echo "Updating operating system"
  apt-get update

  # Install ansible libraries
  echo "Installing ansible"
  apt-get install -y software-properties-common
  apt-add-repository --yes --update ppa:ansible/ansible
  apt install -y ansible

  # Install library to execute command remotely by ssh
  echo "Installing asynssh"
  python3 -m pip install asyncssh

  # Install ping system command
  apt install -y iputils-ping

  # Install HTTP python library
  python3 -m pip install requests
  ```

- `vnf_ee.py`: Python file that includes the code to be called from when a primitive is called. `vnf_ee.py` defines a `VnfEE` class that includes one method for each primitive to be executed on the EE. You can modify, delete or add new methods here, but keep in mind that method names must match the name of the primitive in descriptor (the - character should be avoided). User code to interact with the VNF might be inserted here, although, for the sake of readability and maintainability, it is advised to include just invocations to code residing in other files of the source folder. In all cases, the methods must return a status code to back-end reporting the state of success (`OK`) or failure (`ERROR`) of the operation, and a text description in a `yield` call. The structure of VnfEE class is the following:

  ```python
  class VnfEE:

    def __init__(self, config_params):
        self.logger = logging.getLogger('osm_ee.vnf')
        self.config_params = config_params

    # config method saves SSH access parameters (host and username) for future use by other methods.
    # It is mandatory in any case.
    async def config(self, id, params):
        self.logger.debug("Execute action config, params: {}".format(params))
        # Config action is special, params are merged with previous config calls
        self.config_params.update(params)
        required_params = ["ssh-hostname"]
        self._check_required_params(self.config_params, required_params)
        yield "OK", "Configured"

    # This method implements the "run_script" primitive. Uncomment and modify it if a primitive requires executing a user
    # script in the VDU. It needs "file" parameter (file name to run) and optionally "parameters" (command-line arguments).
    async def run_script(self, id, params):
        self.logger.debug("Execute action run_script, params: '{}'".format(params))
        ...
        if return_code != 0:
            yield "ERROR", "return code {}: {}".format(return_code, stderr.decode())
        else:
            yield "OK", stdout.decode()
      ...
  ```

- The other files in the source directory are specific to each primitive and will be invoked from the `vnf_ee.py` file as we will see shortly.

## Update of the NF template package

After reviewing the package structure of the sample NF package, the next step would be the adaptation of the `vnf_ee.py` (and other auxiliary files invoked from it) to add those primitives. The `sample_ee_vnf` template already includes some examples of different types of day-1/day-2 primitives, which just need to be uncommented uncomment and adapted to implement them adapted to your needs. Some common types of primitives can be easily implemented from this template, as will be described in the following sections.

### EXAMPLE 1: Invoking a pre-existing script to execute remote commands against the NF

Some primitives can be implemented simply by running a script on a VDU of the NF via SSH. That script may exist in the VM or could be copied using SFTP/SCP from the EE. In this sample package, this a sample primitive of this kind called `run_script` is provided. This primitive runs a helper bash script (`run_ssh.sh`) on the own EE pod, which, in turn, transfers and executes a user script to the VDU via SSH, which will be run in the end. Both scripts are part of the package and are located in the `source` folder, as described before.

If you want to try a sample primitive of this type (or create your own primitive that runs a custom script), you just need to uncomment the lines of the `run_script` method in `VnfEE` class of the `vnf_ee.py` file:

```python
class VnfEE:
    SSH_SCRIPT = "/app/EE/osm_ee/vnf/run_ssh.sh"
    ...
    async def run_script(self, id, params):
        self.logger.debug("Execute action run_script, params: '{}'".format(params))
        self._check_required_params(params, ["file"])

        command = "bash " + self.SSH_SCRIPT + " " + self.config_params["ssh-hostname"] + " " + self.config_params["ssh-username"] + " " + params["file"]
        if "parameters" in params:
            command += " \"" + params.get("parameters", "") + "\""
        self.logger.debug("Command: '{}'".format(command))
        return_code, stdout, stderr = await util_ee.local_async_exec(command)
        if return_code != 0:
            yield "ERROR", "return code {}: {}".format(return_code, stderr.decode())
        else:
            yield "OK", stdout.decode()
```

The primitive needs two parameters: the name of the script to be executed on the VDU (`file`) and its command-line arguments (`parameters`), if any.

In case you wanted your primitive to be available also as day-2 primitive, you should also uncomment the lines related to the `run_script` primitive in the `config-primitive` block of the descriptor.

In case you planed to create your own custom primitive, please remind that the method must return `OK` or `ERROR` codes, and a description in a `yield` call.

```yaml
- execution-environment-primitive: run_script
  execution-environment-ref: sample_ee
  name: run_script
  parameter:
  - data-type: STRING
    name: file
  - data-type: STRING
    name: parameters
```

For example, you could include the following day-1 operation to install a NGINX server in a NF's VDU (by calling the `install_nginx.sh` script), just by uncommenting the following lines under `initial-config-primitive`:

```yaml
- execution-environment-ref: sample_ee
  name: run_script
  parameter:
  - name: file
    value: install_nginx.sh
  seq: 2
```

As in the general case, all files in `helm-charts/eechart/source` will be copied into the EE container, including the `run_ssh.sh` and `install_nginx.sh` files. When OSM requests to execute the `run_script` primitive, the back-end will call the `run_script` method, which will invoke the `run_ssh.sh` script in the EE. This script reads the SSH configuration parameters from the shell, the name of the second script (`install_nginx.sh`) and its runtime parameters (if any), and connects to the VDU via SSH for copying and running the script into it. The `run_ssh.sh` exit status is also relevant: a non-zero value indicates an error, so the Python method must return `ERROR` instead of `OK` in `yield`.

The template is designed so that you do not need to modify the `run_ssh.sh` script, just only create more scripts in `source` directory like `install_nginx.sh`. The final script to install the NGINX server in the NF may be as simple as this one:

```bash
#!/usr/bin/env bash

set -eux
sudo -s <<EOF
apt update
apt install -y nginx
systemctl status nginx
EOF
```

Regarding the `install.sh` file, in this case it is not required for the EE to install additional software (the `ssh` command is included in original Ubuntu 18.04 image), so it would remain as it is (note that the sample `install.sh` file includes some installations to support some of the examples that might be removed if not needed for your own NF).

### EXAMPLE 2: Invoking a local command in the EE

If you want to implement a NF primitive that requires running a command locally in the own EE, you can take the `ping` primitive of the template as starting point, which you can uncomment in the sample descriptor (and modify and adapt to your needs, if required). In the provided example, the primitive pings the NF's management IP address from the EE:

```yaml
- execution-environment-primitive: ping
  execution-environment-ref: sample_ee
  name: ping
```

The method in `vnf_ee.py` simply calls the `ping` command and returns `OK` or `ERROR` code and description in a `yield` call, as required:

```python
async def ping(self, id, params):
    self.logger.debug("Execute action ping, params: '{}'".format(params))

    command = "ping -c 3 " + self.config_params["ssh-hostname"]
    return_code, stdout, stderr = await util_ee.local_async_exec(command)
    if return_code != 0:
        yield "ERROR", "return code {}: {}".format(return_code, stderr.decode())
    else:
        yield "OK", stdout.decode()
```

This primitive does not need additional files from the `source` directory. However, since the `ping` binary is not included by default in the docker image of Ubuntu 18.04, its installation is specified in the `helm-charts/eechart/source/install.sh` script, as provided in the template package (note the `-y` flag in the `apt` command):

```bash
# Install ping system command
apt install -y iputils-ping
```

### EXAMPLE 3: Invoking a pre-existing Ansible playbook

Another easy way to implement primitives is by invoking Ansible playbooks. Thus, you can add a primitive of that type simply by uncommenting the `ansible_playbook` block in the NF descriptor of the template package. This primitive requires two parameters: the playbook file name (in the provided example, `playbook-name`) and a variable that is required for the sample playbook (`app` - this is only required in this specific playbook):

```yaml
- execution-environment-primitive: ansible_playbook
  execution-environment-ref: sample_ee
  name: ansible_playbook
  parameter:
  - data-type: STRING
    name: playbook-name
  - data-type: STRING
    name: app
```
The `ansible_playbook` method in `vnf_ee.py` file simply reads the playbook file and app parameters and calls a function to execute it. This function is imported from `osm_ee.util.util_ansible` and just needs the SSH configuration data stored by the `config` primitive for accessing VDU. As always, in case of error, the value returned by `yield` will be `ERROR` along with a description of the failure.

```python
import osm_ee.util.util_ansible as util_ansible

class VnfEE:
    PLAYBOOK_PATH = "/app/EE/osm_ee/vnf"
    ...
    async def ansible_playbook(self, id, params):
      self.logger.debug("Execute action ansible_playbook, params: '{}'".format(params))

      try:
          self._check_required_params(params, ["playbook-name"])
          params["ansible_user"] = self.config_params["ssh-username"]
          inventory = self.config_params["ssh-hostname"] + ","
          playbook = self.PLAYBOOK_PATH + "/" + params["playbook-name"]
          os.environ["ANSIBLE_HOST_KEY_CHECKING"] = "False"
          return_code, stdout, stderr = await util_ansible.execute_playbook(playbook, inventory, params)
          status = "OK" if return_code == 0 else "ERROR"
          yield status, stdout + stderr
      except Exception as e:
          self.logger.debug("Error executing ansible playbook: {}".format(repr(e)))
          yield "ERROR", str(e)
```

The playbook file included in the sample package is called `playbook.yaml` and, unsurprisingly, it is located in `source` directory of the chart. In this example, the playbook tries to install the package specified in the `app` parameter using the `apt` command:

```yaml
---
- hosts: all
  become: yes
  tasks:
  - name: Wait 120 seconds, but only start checking after 10 seconds
    wait_for_connection:
      delay: 20
      timeout: 120
  - name: Install packages
    apt:
      name:
      - "{{ app }}"
      state: latest
      cache_valid_time: 3600
```

As this primitive requires the Ansible software to be available in the EE and the provided Docker Ubuntu does not include it by default, the `install.sh` script takes care of its installation:

```bash
# Install ansible libraries
echo "Installing ansible"
apt-get install -y software-properties-common
apt-add-repository --yes --update ppa:ansible/ansible
apt install -y ansible
```

### EXAMPLE 4: Invoking custom Python code

It is also possible to implement a day-1/day-2 primitive by coding it in Python as a method of the `VnfEE` class. In the simplest case, it might be possible to embed a small python code inside the method (although, for more advanced cases, for the sake of readability, it is highly recommended just including there the invocation of methods imported from an external Python file also injected from the `source` folder).

In the provided template, a simple implementation of this case is provided by the `http_check` primitive, which performs an HTTP GET request from the EE to the port 80 of a NF's VDU's. The primitive, which requires no parameters, looks like this after uncommenting it in descriptor:

```yaml
- execution-environment-primitive: http_check
  execution-environment-ref: sample_ee
  name: http_check
```

The `http_check` method in the `vnf_ee.py` file takes the `ssh-hostname` variable (previously stored by the `config` primitive) to compose the URL, and uses the well-known `requests` library to create the call. As usual, the method returns an error code (`OK` or `ERROR`) and a text description in a `yield` call:

```python
import requests
...
async def http_check(self, id, params):
    self.logger.debug("Execute action http_check, params: '{}'".format(params))

    try:
        session = requests.Session()
        url = 'http://' + self.config_params["ssh-hostname"]
        self.logger.debug("HTTP GET {}...".format(url))
        req = session.get(url)
        self.logger.debug("{}".format(req.text))
        if req.status_code == 200:
            yield "OK", req.text
        else:
            yield "ERROR", req.text
    except Exception as e:
        self.logger.error("HTTP error: {}".format(repr(e)))
        yield "ERROR", str(e)
```

As expected, the `requests` library is not pre-installed and has to be installed in EE's container. The `install.sh` script takes care of the installation following the same procedure described in the previous examples:

```bash
# Install HTTP python library
python3 -m pip install requests
```

The case of a complex primitive that requires invoking a user's Python library may be required relatively often. In this case, the sample `touch` primitive just does that. It creates an empty file in the VDU with the path passed as a parameter, relying on a user-created library that must be in the `source` directory. The fragment in the descriptor looks like this:

```yaml
- execution-environment-primitive: touch
  execution-environment-ref: sample_ee
  name: touch
  parameter:
  - data-type: STRING
    name: file
```

In the example, the `VnfEE::touch` method calls the `ssh_exec` function from the imported library `mylib` for running a remote `touch` command via SSH.

```python
import osm_ee.vnf.mylib as mylib
...
async def touch(self, id, params):
    self.logger.debug("Execute action touch, params: '{}'".format(params))

    self._check_required_params(params, ["file"])
    command = "touch" + " " + params["file"]
    return_code, description = await mylib.ssh_exec(self.config_params["ssh-hostname"], self.config_params["ssh-username"], command)
    if return_code != 0:
        yield "ERROR", description
    else:
        yield "OK", description
```

`mylib.py` is a user library and needs to be located in the `source` folder (like all the others files). Its content look like this:

```python
import logging
import asyncio
import asyncssh

logger = logging.getLogger("osm_ee.vnf")

async def ssh_exec(host: str, user: str, command: str
                           ) -> (int, str):
    """
        Execute a remote command via SSH.
    """

    try:
        async with asyncssh.connect(host,
                                    username=user,
                                    known_hosts=None) as conn:
            logger.debug("Executing command '{}'".format(command))
            result = await conn.run(command)
            logger.debug("Result: {}".format(result))
            return result.exit_status, result.stderr
    except Exception as e:
        logger.error("Error: {}".format(repr(e)))
        return -1, str(e)
```

Since the library imports `asyncssh`, it must be installed by the `install.sh` script:

```bash
# Install library to execute command remotely by ssh
echo "Installing asynssh"
python3 -m pip install asyncssh
```

### EXAMPLE 5: Including a dependent sub-chart

Sometimes the NF is provided by a vendor that also supplies its own helm-chart to manage it. This chart can be included in the EE to be deployed as a subchart dependent on `eechart`. In this case, the main chart `eechart` can talk to a service exposed by the subchart, which will also be in the same namespace of the OSM cluster, to run some actions, being the subchart responsible of operating the NF. In other cases, the inclusion of a subchart could be useful to rely on some functionality provided by that subchart. For instance, a MySQL DB subchart could be useful to save the state of the EE. The main chart `eechart` can talk to a service exposed by the subchart to do some tasks, e.g. reading or writing to/from the DB.

In the sample package there is a primitive that operates on a service exposed by a subchart included in `eechart`. The primitive is `check_database` and the subchart is `mysql`. The subchart deploys a MySQL database and the primitive accesses the database to perform a simple query. The first step will be to include the subchart under `charts` folder in `helm-charts/eechart` directory. In the provided template package MySQL chart is in the `charts.sample` folder. Just move the `mysql-8.8.26.tgz` file to the `charts` directory to deploy it as a subchart beside `eechart`. This zipped file contains the MySQL chart files and was downloaded from the [bitnami repository](https://charts.bitnami.com/bitnami), so you could use another chart by downloading it from a repository with the `helm pull` command.

```bash
cd sample_ee_vnf/helm-charts/eechart
mv charts.sample/mysql-8.8.26.tgz charts/
```

In addition to moving the subchart file to the directory, some changes need to be made to the `eechart` chart for its integration. For example, it is needed to specify the MySQL's user and password to be able to access MySQL from the primitive's code executed in the back-end. There are variables in the MySQL chart that allow you to set, among other things, the access parameters, and they can be set from `eechart` values file. That is why the following lines have been added at the end of the `eechart/values.yaml` file:

```yaml
mysql:
  auth:
    rootPassword: "123456"
  fullnameOverride: "eechart-mysql"
```

These values are grouped into variables of a secret in `eechart/templates/secret.yaml` file:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: {{ include "eechart.fullname" . }}
type: Opaque
data:
  mysql_host: {{ .Values.mysql.fullnameOverride | b64enc | quote }}
  mysql_user: {{ "root" | b64enc | quote }}
  mysql_password: {{ .Values.mysql.auth.rootPassword | b64enc | quote }}
```

And finally, the secret is shared with the container as environment variables in `eechart/templates/statefulset.yaml` file. Therefore, in the EE container, `mysql_host`, `mysql_user` and `mysql_password` environment variables will be available and indicate the host, user and password to access MySQL.

```yaml
containers:
  - name: {{ .Chart.Name }}
    ...
    envFrom:
    - secretRef:
        name: {{ include "eechart.fullname" . }}
```

Once the changes have been made in `eechart`, the primitive must be added in the descriptor. In this case the `check_database` operation will be a day-1 primitive without parameters, so this fragment will have to be uncommented in the descriptor under the `initial-config-primitive` block:

```yaml
- execution-environment-ref: sample_ee
  name: check_database
  seq: 7
```

The method in `vnf_ee.py` calls the `mysql_query` function from `mylib`, an imported user library located in `source` directory. It returns `OK` or `ERROR` code and description in a `yield` call, as required. This Python function needs the host name, user and password for connecting to MySQL, which are taken from environment variables, as commented before, and executes the query `SHOW DATABASES`.

```python
class VnfEE:
    ...
    async def check_database(self, id, params):
        self.logger.debug("Execute action check_database, params: '{}'".format(params))

        host = os.getenv('mysql_host')
        user = os.getenv('mysql_user')
        password = os.getenv('mysql_password')
        retries = 3
        query = "SHOW DATABASES"
        return_code, description = mylib.mysql_query(host, user, password, retries, query)
        if return_code != 0:
            yield "ERROR", description
        else:
            yield "OK", description
```

Finally, as `mylib` imported library requires `mysql-connector-python` (a MySQL client Python package), it must be installed from the `install.sh` script:

```bash
# Install MySQL library
python3 -m pip install mysql-connector-python
```

## Updating NF and NS template packages to adapt them to your needs

In addition to the cases covered by examples above, you may want to edit the NF and NS descriptors and update the name of the NF and NS, change the images, network interfaces, memory sizes, etc. in any of the VDUs.

For final versions, it is also advised to remove or comment any primitives in the NF descriptor template and the methods from `vnf_ee.py` file that will not be used, as well as any related files under the `helm-charts/eechart/source` folder, and removing any unused software from the `install.sh` script. Also note that for final versions, it is highly advisable using a specific container image for the frontend with all required software preinstalled, to avoid hot installations upon instantiation (those are only convenient during development).

## Testing the NF and NS packages

As usual in OSM, the first step to use NF and NS package is onboarding them into the OSM system:

```bash
osm nfpkg-create {PACKAGES_FOLDER}/sample_ee_vnf
osm nspkg-create {PACKAGES_FOLDER}/sample_ee_ns
```

Then, you can instantiate the NS following the usual commands (`$VIM_TARGET` is the VIM's name in OSM and `$VIM_EXT_NET` is management network's name in that VIM):

```bash
osm ns-create --ns_name sample_ee --nsd_name sample_ee-ns --vim_account $VIM_TARGET --config "{vld: [ {name: mgmtnet, vim-network-name: $VIM_EXT_NET} ] }"
```

### Executing your day-2 primitives

For instance, for executing the `run_script` primitive as day-2 operation, you can run the `osm ns-action` command when NS instance is ready. As always, the `action_name` parameter contains the primitive's name, and `params` value is a YAML/JSON inline string with the parameters required by the primitive (if applicable):

```bash
$ osm ns-list
+------------------+--------------------------------------+---------------------+----------+-------------------+---------------+
| ns instance name | id                                   | date                | ns state | current operation | error details |
+------------------+--------------------------------------+---------------------+----------+-------------------+---------------+
| sample_ee        | 354b0009-05ca-4565-850f-03d029601753 | 2022-02-22T16:32:11 | READY    | IDLE (None)       | N/A           |
+------------------+--------------------------------------+---------------------+----------+-------------------+---------------+

$ osm ns-action --action_name run_script --vnf_name sample_ee --params '{file: install_nginx.sh, parameters: ""}' sample_ee
926b9375-732c-464a-98d1-59678b4de655
```

You can also see the history of operations over the NS instance:

```bash
$ osm ns-op-list sample_ee
+--------------------------------------+-------------+-------------+-----------+---------------------+--------+
| id                                   | operation   | action_name | status    | date                | detail |
+--------------------------------------+-------------+-------------+-----------+---------------------+--------+
| 27bc5366-06d6-4562-b6e2-ba39a01d6dde | instantiate | N/A         | COMPLETED | 2022-02-22T16:32:11 | -      |
| 926b9375-732c-464a-98d1-59678b4de655 | action      | run_script  | COMPLETED | 2022-02-22T16:34:41 | -      |
+--------------------------------------+-------------+-------------+-----------+---------------------+--------+
```

Details of the day-2 operation are also available:

```bash
$ osm ns-op-show 926b9375-732c-464a-98d1-59678b4de655
+-----------------------+------------------------------------------------------------------------------------------------------+
| field                 | value                                                                                                |
+-----------------------+------------------------------------------------------------------------------------------------------+
| _id                   | "926b9375-732c-464a-98d1-59678b4de655"                                                               |
| id                    | "926b9375-732c-464a-98d1-59678b4de655"                                                               |
| operationState        | "COMPLETED"                                                                                          |
| queuePosition         | 0                                                                                                    |
| stage                 | ""                                                                                                   |
| errorMessage          | ""                                                                                                   |
| detailedStatus        | null                                                                                                 |
| statusEnteredTime     | 1645547699.5537114                                                                                   |
| nsInstanceId          | "354b0009-05ca-4565-850f-03d029601753"                                                               |
| lcmOperationType      | "action"                                                                                             |
| startTime             | 1645547681.2144706                                                                                   |
| isAutomaticInvocation | false                                                                                                |
...
+-----------------------+------------------------------------------------------------------------------------------------------+
```

### Debugging

The logs generated by the day-1 or day-2 operations can be reviewed by accessing the EE in OSM's system cluster. Thus, a pod named `eechart-<id>` (the same prefix that chart name in VNFD) in the `osm` namespace would contain the runtime of the sample EE:

```bash
$ kubectl -n osm get pods
NAME                            READY   STATUS    RESTARTS   AGE
eechart-0031195008-0            1/1     Running   0          89m
...

$ kubectl -n osm logs pod/eechart-0031195008-0
Install additional libraries
Updating libraries
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
...
Starting frontend server
DEBUG:osm_ee.util:Execute local command: ssh-keygen -q -t rsa -N '' -f /root/.ssh/id_rsa
DEBUG:osm_ee.util:Return code: 0
DEBUG:osm_ee:Generated ssh_key, return_code: 0
...
```

It is also possible to access the container and check the EE directory structure and its contents. The files included in the `helm-charts/eechart/source` folder in the NF package should now exist in the `/app/EE/osm_ee/vnf` folder in the container filesystem:

```bash
$ kubectl -n osm exec -ti pod/eechart-0031195008-0 -- bash
root@eechart-0031195008-0:/app/EE# ls -l osm_ee/vnf
total 0
lrwxrwxrwx 1 root root 17 Feb 21 15:53 install.sh -> ..data/install.sh
lrwxrwxrwx 1 root root 23 Feb 21 15:53 install_nginx.sh -> ..data/install_nginx.sh
lrwxrwxrwx 1 root root 15 Feb 21 15:53 mylib.py -> ..data/mylib.py
lrwxrwxrwx 1 root root 20 Feb 21 15:53 playbook.yaml -> ..data/playbook.yaml
lrwxrwxrwx 1 root root 17 Feb 21 15:53 run_ssh.sh -> ..data/run_ssh.sh
lrwxrwxrwx 1 root root 16 Feb 21 15:53 vnf_ee.py -> ..data/vnf_ee.py