Open Source MANO (OSM) is an ETSI-hosted open source community delivering a production-quality MANO stack for NFV, capable of consuming openly published information models, available to everyone, suitable for all VNFs, operationally significant and VIM-independent. OSM is aligned to NFV ISG information models while providing first-hand feedback based on its implementation experience.
OSM follows a regular cadence of two releases per year, alternating between Long Term Support (LTS) releases such as Release TWELVE (2 years support) and Standard releases (6 months support) such as this release, Release THIRTEEN. **Release THIRTEEN** introduces a new scalable architecture for service assurance and closed-loop operations leveraging on cloud-native version of Apache Airflow and Prometheus. This architecture is prepared to cover the most demanding service assurance scenarios such as auto-healing and auto-scaling in large clouds and multiple edge sites. Release THIRTEEN incorporates new workflows for getting the state of Network Functions (NF), Network Services (NS) and VIM, and will gradually incorporate new capabilities in the next releases. In addition, Release THIRTEEN includes significant improvements in other key areas:
OSM follows a regular cadence of two releases per year, alternating between Long Term Support (LTS) releases such as Release FOURTEEN or Release TWELVE (2 years support) and Standard releases (6 months support). This release, **Release FOURTEEN**, offers a new scalable architecture for service assurance based on Apache Airflow and Prometheus and includes closed loops for auto-scaling and the handling of alerts coming from the Network Functions. In addition, the new architecture enables scheduled monitoring and service-assurance workflows, as well as alarm-based on-demand workflows. This capability will be key to supporting new telco cloud use cases in future Releases.
-**Execution environments.** Day-2 operations are a key capability in OSM, which provides added value to the mere deployment of NS and NF through the use of Execution Environments (EE), isolated environments that allow the execution of code to perform operations on the NF and NS. Release THIRTEEN provides an improved secured communication channel between helm-based EE and NF, the capability to upgrade helm-based EE and a new naming convention for Juju application in Juju-based EE.
-**NS deployment.** OSM has proved to be successful in production environments in the deployment of NS. This release incorporates new capabilities for NS deployment such as the capability to make persistent volumes of NF permanent in the VIM, the ability to store CA certificates as part of VIM registration and update, or the automatic WIM selection for inter-DC networks.
-**Internal LCM evolution.** LCM module in OSM is responsible of managing the workflows associated to life cycle events of VNF and NS such as instantiation, termination, scaling, healing and upgrading. In this Release, we have initiated the adoption of the Saga-based pattern for workflow management in selected operations. In addition, this release incorporates the communication channel between LCM and RO via Kafka.
-**OSM installation experience.** The community installer will now be able to auto-detect installations of OSM behind a web proxy and perform the appropriate configurations, easing the global installation experience. In addition, the internal process in OSM to generate the installation SW has incorporated the automatic publication of charms for charm-based installation.
-**OSM client.** Finally, this release includes some improvements in the OSM client, such as a more convenient registration of Prometheus-based telemetry systems as part of VIM registration and update, the refactoring of OSM client commands, and an installation procedure for OSM client in Windows and Red Hat Linux distros in addition to Ubuntu Linux distros.
Release FOURTEEN brings significant improvements in security, usability, platform management, infrastructure modelling and lifecycle management of network functions:
For the full list of new features, please refer to the [Release Notes](https://osm-download.etsi.org/ftp/osm-13.0-thirteen/OSM_Release_THIRTEEN_Release_Notes.pdf). For a comprehensive overview of OSM functionalities, you can also refer to the [OSM White Papers and Release Notes of previous releases](https://osm.etsi.org/wikipub/index.php/Release_notes_and_whitepapers).
-**Security enhancements**. Release FOURTEEN introduced secured helm-based Execution Environments (EE) based on authenticated gRPC communication channel between EE and NF, and the use of pod admission policies to restrict EE permissions running in OSM cluster. In addition, Pycrypto library has been PycryptoDome, a more modern and secure cryptography library.
-**Usability and platform management**. This release includes relevant enhancements in user management such as the generation of standard audit logs for every user operation from any entity interacting with OSM North Bound Interface. In addition, retry and password expiry policies have been added to user management.
-**Infra modelling and NF lifecycle**. OSM has proved to be successful in production environments in the deployment of NS. This release goes even further and incorporates new capabilities for NS deployment such as the capability to attach multiple volumes to different VM, the capability to re-use existing flavors, the simultaneous support of IPv4 and IPv6 in VM network interfaces, or the ability to provide instantiation parameter to Juju-based Kubernetes Deployment Units. In addition, the release has optimized the interactions with VIMs to make deployment faster and more reliable. Finally, the release includes a new connector for Transport-API WIMs, which will enable the use of TeraFlow SDN or any other WIM in the future to build optical transport connectivity between datacenters.
-**OSM installation experience**. This release has modified the community installer to install all OSM components with a Helm chart rather than plain Kubernetes manifests, which will allow much simpler upgrades of OSM services from now on.
-**OSM client**. Finally, this release includes some improvements in the OSM client, such as the support of different output formats for some commands and the replacement of Pycurl library by the well-known Requests library for HTTP interaction. Both features were developed by participants of the recent [OSM Hackfest for developers held in Castelldefels in June 2023](https://osm.etsi.org/wikipub/index.php/OSM15_Hackfest).
For the full list of new features, please refer to the [Release Notes](https://osm-download.etsi.org/ftp/osm-14.0-fourteen/OSM_Release_FOURTEEN_Release_Notes.pdf). For a comprehensive overview of OSM functionalities, you can also refer to the [OSM White Papers and Release Notes of previous releases](https://osm.etsi.org/wikipub/index.php/Release_notes_and_whitepapers).
**OSM in Practice**:
...
...
@@ -37,14 +39,16 @@ In order for OSM to work, it is assumed that:
All you need to run OSM is a single server or VM with the following requirements:
- MINIMUM: 2 CPUs, 6 GB RAM, 40GB disk and a single interface with Internet access
- RECOMMENDED: 2 CPUs, 8 GB RAM, 40GB disk and a single interface with Internet access
- Base image: [Ubuntu20.04 (64-bit variant required)](http://releases.ubuntu.com/20.04/)
- RECOMMENDED: 4 CPUs, 16 GB RAM, 80GB disk and a single interface with Internet access
- MINIMUM: 2 CPUs, 8 GB RAM, 50GB disk and a single interface with Internet access
- An additional installation option is the [Charmed Installation](03-installing-osm.md#charmed-installation) which will install OSM on Kubernetes with charms.
- You can also run OSM using a pre-built [vagrant](https://app.vagrantup.com/osm/boxes/releaseeight) image. You can find here detailed instruction on [how to install OSM in Vagrant](03-installing-osm.md#vagrant-installation)
- For other special installation options, please refer to the [specific chapter on installation options](03-installing-osm.md).
-[Ubuntu22.04 server image (64-bit variant required)](http://releases.ubuntu.com/22.04/)
**Reminder**: Although OSM could work with other base images, the only recommended are the ones above, since these are the images used in our daily tests.
...
...
@@ -30,7 +30,7 @@ Hence, it is assumed that:
Once you have one host available with the characteristics above, you just need to trigger the OSM installation by:
@@ -359,7 +359,7 @@ osm ns-create --ns_name hf-basic --nsd_name hackfest_basic-ns --vim_account open
### Specify IP profile information and IP for a NS VLD <a name="specify-ip-profile-information-and-ip-for-a-ns-vld">
In a generic way, the mapping can be specified in the following way, where `datanet` is the name of the network in the NS descriptor, ip-profile is where you have to fill the associated parameters from the data model ( [NS data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.html) ), and vnfd-connection-point-ref is the reference to the connection point:
In a generic way, the mapping can be specified in the following way, where `datanet` is the name of the network in the NS descriptor, ip-profile is where you have to fill the associated parameters from the data model ( [NS data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.html) ), and vnfd-connection-point-ref is the reference to the connection point:
### Specify IP profile information for an internal VLD of a VNF
In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, `internal` is the name of internal-vld in the VNF descriptor and ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)):
In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, `internal` is the name of internal-vld in the VNF descriptor and ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)):
In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor, ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)), `id1` is the internal-connection-point id and `a.b.c.d` is the IP that you have to specify for this scenario:
In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor, ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)), `id1` is the internal-connection-point id and `a.b.c.d` is the IP that you have to specify for this scenario:
@@ -471,7 +471,7 @@ You can try it using one of the examples of the hackfest (**packages: [hackfest_
```bash
osm ns-create --ns_name hf-basic --nsd_name hackfest_basic-ns
With the previous hackfest example, according [VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html) you will add in VNF Descriptor:
With the previous hackfest example, according [VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html) you will add in VNF Descriptor:
```yaml
volumes:
...
...
@@ -1373,7 +1373,7 @@ The diagram below shows the `slice_basic_ns` and `slice_basic_middle_ns`, its co
### Creating a Network Slice Template (NST)
Based on the OSM information model for Network slice templates [here](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/nst.html) it is possible to start writing the YAML descriptor for the NST.
Based on the OSM information model for Network slice templates [here](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/nst.html) it is possible to start writing the YAML descriptor for the NST.
@@ -7,10 +7,7 @@ If you want to learn more, these additional contents are highly recommended:
-**[VNF Onboarding Guide](https://osm.etsi.org/docs/vnf-onboarding-guidelines).** This guide includes all that you might need to onboard your VNF or applications in OSM.
-**[OSM Developer Guide][developer-guide].** If you want to contribute to OSM development, it is highly recommended reading this guide first.
### How to install OSM Client in Ubuntu 20.04 (RECOMMENDED)
### How to install OSM Client in Ubuntu 22.04 (RECOMMENDED)
OSM client is installed by default in the host where OSM is installed, but it can be also installed as a standalone client in an Ubuntu 20.04 system, following the procedure below:
OSM client is installed by default in the host where OSM is installed, but it can be also installed as a standalone client in an Ubuntu 22.04 system, following the procedure below:
```bash
# Clean the previous repos that might exist
...
...
@@ -154,13 +154,12 @@ sudo sed -i "/osm-download.etsi.org/d" /etc/apt/sources.list
OSM client can be easily installed in Windows by installing an Ubuntu distro on Linux with [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/install).
Once WSL is installed with an Ubuntu 20.04 distro, you can install OSM following the instruccions in (#how-to-install-osm-client-in-ubuntu-2004)
Once WSL is installed with an Ubuntu 22.04 distro, you can install OSM following the instruccions in (#how-to-install-osm-client-in-ubuntu-2204)
### How to install OSM Client directly in Windows with Conda and Git
...
...
@@ -300,7 +297,7 @@ Make sure that aliases for Python are disabled in Windows Configuration. Go to S
Open Git Bash and run the following commands to create a Conda environment with Python 3.8 and initialize all shells to work with Conda:
```bash
conda create -n osm-env python=3.8
conda create -n osm-env python=3.10
conda init --all
# Logout
```
...
...
@@ -310,13 +307,12 @@ Then install OSM client as follows:
```bash
# Install conda and install some packages via conda (which will install dependent libraries)
conda activate osm-env
conda install pycurl==7.45.1
# Upgrade pip to the latest version (with sudo, to install it globally for all users)
python -m pip install-U pip
# Decide which version to use (e.g., v13.0)
export OSM_CLIENT_VERSION=v13.0
# Decide which version to use (e.g., v14.0)
export OSM_CLIENT_VERSION=v14.0
# Clone IM repo and checkut the desired version
git clone https://osm.etsi.org/gerrit/osm/IM
...
...
@@ -651,7 +647,7 @@ Assuming that you have installed python-osmclient package, it's pretty simple to
# Annex 10: Tutorial - Charmed OSM installation and deployment of a CNF
## Introduction
In this tutorial you will install OSM that will be used to deploy and configure a CNF in a K8s cluster. Along the way you will attach a VIM and a K8s cluster to OSM in order to be able to deploy workloads, and also upload VNF packages, so they can be used later on.
Since OSM Release THIRTEEN, OSM offers a scalable architecture for Service Assurance (SA) based on Apache Airflow and Prometheus, which includes closed loops for auto-scaling and the handling of alerts coming from the Network Functions.
### Goal and principles of the Service Assurance architecture
The main goal of the Service Architecture is to sense the state of the objects deployed by OSM and build telco cloud use cases with that information:
- Offering state indicators of the NS and their components, as well as the NFV infrastructure used by OSM.
- Providing capabilities to build closed-loops for scaling and healing
Metric visualization is not a main objective for the E2E Service Assurance architecture. While OSM provides visualization of the collected and derived metrics, it will always serve for the purpose of building advanced use cases.
The principles of the Service Assurance archicture are:
- Do not reinvent the wheel. Instead of building a new framework from scratch, OSM relies on well-known open source projects such as [Prometheus](https://prometheus.io/)[Apache Airflow](https://airflow.apache.org/), respectively for the storage and handling of metrics and alerts, and the execution of workflows.
- Modularity of tasks. Workflows must be split into smaller stages that should be run by the most appropriate component. Similary to the ETL paradigm in data analytics pipelines, inputs could come from anywhere, being processed by any component and outputs could go everywhere.
- Scalability. Task execution must be prepared to run in parallel, so that a collection of workers can run potentially any task, maximizing workload sharing between workers. This also avoids dedicated components for specific tasks. Bottlenecks can be globally managed by scaling the number of workers.
- Configurable schedule of workflows. This is specially important for metric collection, which could be very intensive in resource consumption.
- Easy to add new capabilities. The selected components provide a toolkit to easily incorporate new functionality, lowering the learning curve.
### Building blocks
Service Assurance architecture in OSM is characterized by the following:
- It relies on Apache Airflow and Prometheus stack as main building blocks. Prometheus TSDB and Grafana were already deployed in previous releases of OSM. Since Release THIRTEEN and FOURTEEN, Airflow, Prometheus AlertManager and PushGateway are also deployed.
- It deprecates the use of the old components MON and POL. With the new SA architecture, POL is removed, while the role of MON is restricted to a process to create Grafana dashboards per project and NS.
- Metrics are gathered by dedicated and scheduled workflows running in Airflow
- Enhanced metrics per NS or VNF can be derived in Prometheus from the collected metrics
- Closed-loops for scaling and healing are triggered by alerts generated by Prometheus Alert Manager, and conveniently addressed to Airflow endpoints to run the associated workflows.
- Webhook Translator is a new component that has been added to translate webhooks, allowing the translation of an alert generated by Alert Manager to the appropriate Airflow DAG endpoint, and enabling the capability to translate in the future external alerts from VNF or other systems to dedicated Airflow workflows.
#### Apache Airflow
[Apache Airflow](https://airflow.apache.org/) is an open-source workflow management platform for data engineering pipelines. It started at Airbnb in October 2014 as a solution to manage the company's increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface. From the beginning, the project was made open source, becoming an Apache Incubator project in March 2016 and a top-level Apache Software Foundation project in January 2019.
Airflow is written in Python, and workflows are created via Python scripts. Airflow is designed under the principle of "configuration as code". While other "configuration as code" workflow platforms exist using markup languages like XML, using Python allows developers to import libraries and classes to help them create their workflows.
Airflow provides a mechanism to run scheduled workflows as well as workflows on demand from an alert. Workflows, called DAGs, are devided in tasks which are run by workers, convenient triggered by the scheduler. In addition, Airflow provides a suitable web-based UI which allows to enable/disable, monitor and log the execution of DAGs.
The way to model workflows in Airflow is via Directed Acyclic Graphs (DAG). A DAG is a collection of all the tasks you want to run, organized in a way that reflects their relationships and dependencies. A DAG is defined in a Python script, which represents the DAG structure (tasks and their dependencies) as code. For example, a simple DAG could consist of three tasks: A, B, and C. It could say that A has to run successfully before B can run, but C can run anytime. It could say that task A times out after 5 minutes, and B can be restarted up to 5 times in case it fails. It might also say that the workflow will run every night at 10pm but shouldn’t start until a certain date.
In summary a DAG is:
- A collection of tasks, with flexibility to create dependencies between tasks
- Defined in Python
- DAGs can be dynamically created, for instance one per VIM or one per Network Service
- Tasks can be dynamically created inside a DAG, for instance
- Airflow DAGs are designed to scale since a collection of Airflow workers can run tasks in parallel
- Airflow DAGs can be scheduled independently
#### Prometheus Stack
[Prometheus](https://prometheus.io/) is a well-known complete open-source framework for monitoring, including metric collection and storage in a time-series database, following an HTTP pull model, metric query via HTTP with a dedicated language PromQL, metric derivation, alert generation and management, and metric visualization.
The Webhook Translator a new component that has been added to translate webhooks, allowing the translation of an alert generated by Alert Manager to the appropriate Airflow DAG endpoint, and enabling the capability to translate in the future external alerts from VNF or other systems to dedicated Airflow workflows.
- It is lightweight. A very small number of lines of code does the work.
- It is stateless. It only translates HTTP requests. No state for those translations. When running as a deployment, native scaling is achieved by means of Kubernetes services.
- It is simple. It is based on [FastAPI](https://fastapi.tiangolo.com/), a simple and fast framework for developing an HTTP REST API in Python.
The code of the webhook can be found in [Gerrit](https://osm.etsi.org/gerrit/osm/NG-SA) and in [Gitlab](https://osm.etsi.org/gitlab/osm/ng-sa/-/tree/master/osm_webhook_translator).
### Metric acquisition and derivation
The following figure illustrates the general workflow for metric acquisition and derivation.
Metrics are acquired in OSM with dedicated Airflow DAGs. In some cases, such as for the obtention of the NS topologies, a single DAG is used. For other cases such as the gathering of the VIM status, the VM status and the VM resource consumption metrics, a DAG per VIM is used. The following figure shows a screenshot of the DAGs as shown by Airflow UI.
Below a summary of the metrics acquired by OSM, summarizing its input, output and the software used for the process.
- NS topology:
- From Mongo DB to Prometheus
- Software used to acquire the metric: Airflow DAG + Prometheus PushGateway
- VM status:
- From MongoDB and VIM to Prometheus
- Software used to acquire the metric: Airflow DAG per VIM + Prometheus PushGateway
- VIM status
- From MongoDB and VIM to Prometheus
- Software used to acquire the metric: Airflow DAG per VIM + Prometheus PushGateway
- VM metrics (resource consumption)
- From MongoDB and VIM to Prometheus
- Software used to acquire the metric: Airflow DAG per VIM + Prometheus PushGateway
The following figure illustrates graphically the specific workflows for metric acquisition.
The metrics, as acquired by OSM, might not be useful per-se, since they are not related to OSM objects. For instance, the VM status or the VM resource consumption metrics are only useful when they relate to a VNF instance or a NS instance. [Prometheus recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) allow to derive metrics from existing metrics, computing them and saving the result as a new set of time series. The following figure shows a screenshot of the metrics derived in Prometheus.
Below a summary of the metrics derived by OSM, summarizing its input, output and the software used for the process.
- Extended VM status:
- From Prometheus (NS topology, VM status) to Prometheus
- Software used to derive the metric: Prometheus Recording Rules
- VNF status:
- From Prometheus (Extended VM status) to Prometheus
- Software used to derive the metric: Prometheus Recording Rules
- NS status:
- From Prometheus (Extended VM status) to Prometheus
- Software used to derive the metric: Prometheus Recording Rules
Finally, the following figure summarizes graphically the specific workflows for metric derivation.
### Closed loops in OSM
The following figure illustrates the general workflow for closed-loops in OSM that is used both for auto-healing and auto-scaling.
-**Prometheus alerts**. All monitoring information (status of VIMs and VNFs managed by OSM) is stored in Prometheus. Prometheus has a simple alert generation mechanism based on [alerting rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) that are configured in a similar way as the recording rules. The alerts are automatically triggered and stopped depending on their defining rule.
- For the case of auto-healing operation, a static rule is pre-defined, which generates an alert when the status of a VM is down for more than a given period.
- For the case of auto-scaling operations, a sidecar container deployed with Prometheus dynamicaly updates the Prometheus alert rules as they are created/deleted by OSM in MongoDB at instantiation/termination time.
-**AlertManager**. [AlertManager](https://prometheus.io/docs/alerting/latest/alertmanager/) allows managing alerts, including silencing, inhibition, aggregation and sending out notifications. The alerts generated by Prometheus are sent to AlertManager, which forwards them to a webhook. The use of AlertManager is necessary as an intermediate step because Prometheus cannot send alerts directly to a webhook. In addition, AlertManager includes mechanisms for silencing, inhibition, aggregation, etc., which could be leveraged by OSM in the future.
-**Webhook Translator**. AlertManager can send alerts to multiple external systems via, for example, HTTP, but does not integrate directly with Airflow, which is the system that will run the worflows or DAGs for healing and scaling. Airflow has its own format for webhooks. The Webhook Translator is added to translate HTTP requests: it receives an HTTP request from AlertManager and forwards it to a supported Airflow webhook in the format expected by Airflow.
-**Airflow DAG driven by webhook**. The last step in the pipeline is running the DAG which will process the alert and execute the necessary actions to carry out the closed-loop operation. In the case of auto-healing, when the firing alert about a VM arrives, the DAG will check if there is an auto-healing rule stored in MongoDB regarding to this VM/VDU. In that case, the DAG updates alert status in MongoDB and sends to Kafka a heal message that will be consumed by LCM, which finally performs the healing operation.
-**MongoDB**. The information (auto-healing and auto-scaling rules) used for closed-loop operations is stored in MongoDB in the collection `alerts`. LCM is responsible of inserting and deleting the alarm-related objects when an NS is instantiated or terminated.
