From 6899cf6a2ef3de99217331cbf3fe51c87fef65e2 Mon Sep 17 00:00:00 2001 From: garciadeblas Date: Tue, 25 Jul 2023 09:26:58 +0200 Subject: [PATCH 1/2] Update content and references for Release FOURTEEN Signed-off-by: garciadeblas --- 01-quickstart.md | 29 +++++++++-------- 03-installing-osm.md | 32 ++++++++++--------- 05-osm-usage.md | 10 +++--- 07-what-to-read-next.md | 5 +-- 10-osm-client-commands-reference.md | 48 +++++++++++++---------------- 11-osm-im.md | 12 ++++---- 20-tutorial.md | 3 +- index.md | 2 +- index.rst | 1 + navigation.md | 6 ++++ 10 files changed, 77 insertions(+), 71 deletions(-) diff --git a/01-quickstart.md b/01-quickstart.md index e63abef..99cfa67 100644 --- a/01-quickstart.md +++ b/01-quickstart.md @@ -2,15 +2,17 @@ Open Source MANO (OSM) is an ETSI-hosted open source community delivering a production-quality MANO stack for NFV, capable of consuming openly published information models, available to everyone, suitable for all VNFs, operationally significant and VIM-independent. OSM is aligned to NFV ISG information models while providing first-hand feedback based on its implementation experience. -OSM follows a regular cadence of two releases per year, alternating between Long Term Support (LTS) releases such as Release TWELVE (2 years support) and Standard releases (6 months support) such as this release, Release THIRTEEN. **Release THIRTEEN** introduces a new scalable architecture for service assurance and closed-loop operations leveraging on cloud-native version of Apache Airflow and Prometheus. This architecture is prepared to cover the most demanding service assurance scenarios such as auto-healing and auto-scaling in large clouds and multiple edge sites. Release THIRTEEN incorporates new workflows for getting the state of Network Functions (NF), Network Services (NS) and VIM, and will gradually incorporate new capabilities in the next releases. In addition, Release THIRTEEN includes significant improvements in other key areas: +OSM follows a regular cadence of two releases per year, alternating between Long Term Support (LTS) releases such as Release FOURTEEN or Release TWELVE (2 years support) and Standard releases (6 months support). This release, **Release FOURTEEN**, offers a new scalable architecture for service assurance based on Apache Airflow and Prometheus and includes closed loops for auto-scaling and the handling of alerts coming from the Network Functions. In addition, the new architecture enables scheduled monitoring and service-assurance workflows, as well as alarm-based on-demand workflows. This capability will be key to supporting new telco cloud use cases in future Releases. -- **Execution environments.** Day-2 operations are a key capability in OSM, which provides added value to the mere deployment of NS and NF through the use of Execution Environments (EE), isolated environments that allow the execution of code to perform operations on the NF and NS. Release THIRTEEN provides an improved secured communication channel between helm-based EE and NF, the capability to upgrade helm-based EE and a new naming convention for Juju application in Juju-based EE. -- **NS deployment.** OSM has proved to be successful in production environments in the deployment of NS. This release incorporates new capabilities for NS deployment such as the capability to make persistent volumes of NF permanent in the VIM, the ability to store CA certificates as part of VIM registration and update, or the automatic WIM selection for inter-DC networks. -- **Internal LCM evolution.** LCM module in OSM is responsible of managing the workflows associated to life cycle events of VNF and NS such as instantiation, termination, scaling, healing and upgrading. In this Release, we have initiated the adoption of the Saga-based pattern for workflow management in selected operations. In addition, this release incorporates the communication channel between LCM and RO via Kafka. -- **OSM installation experience.** The community installer will now be able to auto-detect installations of OSM behind a web proxy and perform the appropriate configurations, easing the global installation experience. In addition, the internal process in OSM to generate the installation SW has incorporated the automatic publication of charms for charm-based installation. -- **OSM client.** Finally, this release includes some improvements in the OSM client, such as a more convenient registration of Prometheus-based telemetry systems as part of VIM registration and update, the refactoring of OSM client commands, and an installation procedure for OSM client in Windows and Red Hat Linux distros in addition to Ubuntu Linux distros. +Release FOURTEEN brings significant improvements in security, usability, platform management, infrastructure modelling and lifecycle management of network functions: -For the full list of new features, please refer to the [Release Notes](https://osm-download.etsi.org/ftp/osm-13.0-thirteen/OSM_Release_THIRTEEN_Release_Notes.pdf). For a comprehensive overview of OSM functionalities, you can also refer to the [OSM White Papers and Release Notes of previous releases](https://osm.etsi.org/wikipub/index.php/Release_notes_and_whitepapers). +- **Security enhancements**. Release FOURTEEN introduced secured helm-based Execution Environments (EE) based on authenticated gRPC communication channel between EE and NF, and the use of pod admission policies to restrict EE permissions running in OSM cluster. In addition, Pycrypto library has been PycryptoDome, a more modern and secure cryptography library. +- **Usability and platform management**. This release includes relevant enhancements in user management such as the generation of standard audit logs for every user operation from any entity interacting with OSM North Bound Interface. In addition, retry and password expiry policies have been added to user management. +- **Infra modelling and NF lifecycle**. OSM has proved to be successful in production environments in the deployment of NS. This release goes even further and incorporates new capabilities for NS deployment such as the capability to attach multiple volumes to different VM, the capability to re-use existing flavors, the simultaneous support of IPv4 and IPv6 in VM network interfaces, or the ability to provide instantiation parameter to Juju-based Kubernetes Deployment Units. In addition, the release has optimized the interactions with VIMs to make deployment faster and more reliable. Finally, the release includes a new connector for Transport-API WIMs, which will enable the use of TeraFlow SDN or any other WIM in the future to build optical transport connectivity between datacenters. +- **OSM installation experience**. This release has modified the community installer to install all OSM components with a Helm chart rather than plain Kubernetes manifests, which will allow much simpler upgrades of OSM services from now on. +- **OSM client**. Finally, this release includes some improvements in the OSM client, such as the support of different output formats for some commands and the replacement of Pycurl library by the well-known Requests library for HTTP interaction. Both features were developed by participants of the recent [OSM Hackfest for developers held in Castelldefels in June 2023](https://osm.etsi.org/wikipub/index.php/OSM15_Hackfest). + +For the full list of new features, please refer to the [Release Notes](https://osm-download.etsi.org/ftp/osm-14.0-fourteen/OSM_Release_FOURTEEN_Release_Notes.pdf). For a comprehensive overview of OSM functionalities, you can also refer to the [OSM White Papers and Release Notes of previous releases](https://osm.etsi.org/wikipub/index.php/Release_notes_and_whitepapers). **OSM in Practice**: @@ -37,14 +39,16 @@ In order for OSM to work, it is assumed that: All you need to run OSM is a single server or VM with the following requirements: -- MINIMUM: 2 CPUs, 6 GB RAM, 40GB disk and a single interface with Internet access -- RECOMMENDED: 2 CPUs, 8 GB RAM, 40GB disk and a single interface with Internet access -- Base image: [Ubuntu20.04 (64-bit variant required)](http://releases.ubuntu.com/20.04/) +- RECOMMENDED: 4 CPUs, 16 GB RAM, 80GB disk and a single interface with Internet access +- MINIMUM: 2 CPUs, 8 GB RAM, 50GB disk and a single interface with Internet access +- Base image: Ubuntu22.04 + - [Ubuntu22.04 cloud image (64-bit variant required)](https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64-disk-kvm.img) + - [Ubuntu22.04 server image (64-bit variant required)](http://releases.ubuntu.com/22.04/) Once you have prepared the host with the previous requirements, all you need to do is: ```bash -wget https://osm-download.etsi.org/ftp/osm-13.0-thirteen/install_osm.sh +wget https://osm-download.etsi.org/ftp/osm-14.0-fourteen/install_osm.sh chmod +x install_osm.sh ./install_osm.sh ``` @@ -54,7 +58,7 @@ This will install a standalone Kubernetes on a single host, and OSM on top of it **TIP:** In order to facilitate potential trobleshooting later, it is recommended to save the full log of your installation process: ```bash -wget https://osm-download.etsi.org/ftp/osm-13.0-thirteen/install_osm.sh +wget https://osm-download.etsi.org/ftp/osm-14.0-fourteen/install_osm.sh chmod +x install_osm.sh ./install_osm.sh 2>&1 | tee osm_install_log.txt ``` @@ -77,7 +81,6 @@ Example: #### Other installation options - An additional installation option is the [Charmed Installation](03-installing-osm.md#charmed-installation) which will install OSM on Kubernetes with charms. -- You can also run OSM using a pre-built [vagrant](https://app.vagrantup.com/osm/boxes/releaseeight) image. You can find here detailed instruction on [how to install OSM in Vagrant](03-installing-osm.md#vagrant-installation) - For other special installation options, please refer to the [specific chapter on installation options](03-installing-osm.md). ### Checking your installation diff --git a/03-installing-osm.md b/03-installing-osm.md index e712f35..dfa95d1 100644 --- a/03-installing-osm.md +++ b/03-installing-osm.md @@ -4,11 +4,11 @@ In order to install OSM, you will need, at least, a single server or VM with the following requirements: -- MINIMUM: 2 CPUs, 6 GB RAM, 40GB disk and a single interface with Internet access -- RECOMMENDED: 2 CPUs, 8 GB RAM, 40GB disk and a single interface with Internet access -- Base image: - - Ubuntu20.04 cloud image (64-bit variant required) () - - Ubuntu20.04 server image (64-bit variant required) () +- RECOMMENDED: 4 CPUs, 16 GB RAM, 80GB disk and a single interface with Internet access +- MINIMUM: 2 CPUs, 8 GB RAM, 50GB disk and a single interface with Internet access +- Base image: Ubuntu22.04 + - [Ubuntu22.04 cloud image (64-bit variant required)](https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64-disk-kvm.img) + - [Ubuntu22.04 server image (64-bit variant required)](http://releases.ubuntu.com/22.04/) **Reminder**: Although OSM could work with other base images, the only recommended are the ones above, since these are the images used in our daily tests. @@ -30,7 +30,7 @@ Hence, it is assumed that: Once you have one host available with the characteristics above, you just need to trigger the OSM installation by: ```bash -wget https://osm-download.etsi.org/ftp/osm-13.0-thirteen/install_osm.sh +wget https://osm-download.etsi.org/ftp/osm-14.0-fourteen/install_osm.sh chmod +x install_osm.sh ./install_osm.sh ``` @@ -40,7 +40,7 @@ This will install a standalone Kubernetes on a single host, and OSM on top of it **TIP:** In order to facilitate potential troubleshooting later, it is recommended to save the full log of your installation process: ```bash -wget https://osm-download.etsi.org/ftp/osm-13.0-thirteen/install_osm.sh +wget https://osm-download.etsi.org/ftp/osm-14.0-fourteen/install_osm.sh chmod +x install_osm.sh ./install_osm.sh 2>&1 | tee osm_install_log.txt ``` @@ -104,7 +104,7 @@ Some cases where the Charmed installer might be more suitable: ![OSM Docker containers](assets/800px-OSM_charmed_standalone.png) ```bash -wget https://osm-download.etsi.org/ftp/osm-13.0-thirteen/install_osm.sh +wget https://osm-download.etsi.org/ftp/osm-14.0-fourteen/install_osm.sh chmod +x install_osm.sh ./install_osm.sh --charmed ``` @@ -116,7 +116,7 @@ This will install OSM on [microk8s](https://microk8s.io/) using Charms. For the installation using external components the following parameters can be added: ```bash -wget https://osm-download.etsi.org/ftp/osm-13.0-thirteen/install_osm.sh +wget https://osm-download.etsi.org/ftp/osm-14.0-fourteen/install_osm.sh chmod +x install_osm.sh ./install_osm.sh --charmed --k8s ~/.kube/config --vca --lxd --lxd-cred ``` @@ -295,7 +295,7 @@ OSM could be installed to a remote OpenStack infrastructure from the OSM standar The installation can be performed with the following command: ```bash -wget https://osm-download.etsi.org/ftp/osm-13.0-thirteen/install_osm.sh +wget https://osm-download.etsi.org/ftp/osm-14.0-fourteen/install_osm.sh chmod +x install_osm.sh ./install_osm.sh -O -N [--volume] [OSM installer options] ``` @@ -469,7 +469,7 @@ There are two methods of installing the OSM client: via a Snap, or a Debian Pack On systems that support snaps, you can install the OSM client with the following command: ```bash -sudo snap install osmclient --channel 11.0/stable +sudo snap install osmclient --channel 14.0/stable ``` There are tracks available for all releases. Omitting the channel will use the latest stable release version. @@ -481,13 +481,15 @@ In order to install the OSM Client in your local Linux machine, you should follo ```bash # Clean the previous repos that might exist sudo sed -i "/osm-download.etsi.org/d" /etc/apt/sources.list -wget -qO - https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/OSM%20ETSI%20Release%20Key.gpg | sudo apt-key add - -sudo add-apt-repository -y "deb [arch=amd64] https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN stable devops IM osmclient" +wget -qO - https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/OSM%20ETSI%20Release%20Key.gpg | sudo apt-key add - +sudo add-apt-repository -y "deb [arch=amd64] https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN stable devops IM osmclient" sudo apt-get update sudo apt-get install -y python3-pip sudo -H python3 -m pip install -U pip -sudo -H python3 -m pip install python-magic pyangbind verboselogs -sudo apt-get install python3-osmclient +sudo apt-get install -y python3-osm-im python3-osmclient +sudo apt-get install -y libmagic1 +python3 -m pip install -r /usr/lib/python3/dist-packages/osm_im/requirements.txt +python3 -m pip install -r /usr/lib/python3/dist-packages/osmclient/requirements.txt ``` ### Usage diff --git a/05-osm-usage.md b/05-osm-usage.md index 0bc0a95..c2f48cb 100644 --- a/05-osm-usage.md +++ b/05-osm-usage.md @@ -359,7 +359,7 @@ osm ns-create --ns_name hf-basic --nsd_name hackfest_basic-ns --vim_account open ### Specify IP profile information and IP for a NS VLD -In a generic way, the mapping can be specified in the following way, where `datanet` is the name of the network in the NS descriptor, ip-profile is where you have to fill the associated parameters from the data model ( [NS data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.html) ), and vnfd-connection-point-ref is the reference to the connection point: +In a generic way, the mapping can be specified in the following way, where `datanet` is the name of the network in the NS descriptor, ip-profile is where you have to fill the associated parameters from the data model ( [NS data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.html) ), and vnfd-connection-point-ref is the reference to the connection point: ```yaml --config '{vld: [ {name: datanet, ip-profile: {...}, vnfd-connection-point-ref: {...} } ] }' @@ -373,7 +373,7 @@ osm ns-create --ns_name hf-multivdu --nsd_name hackfest_multivdu-ns --vim_accoun ### Specify IP profile information for an internal VLD of a VNF -In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, `internal` is the name of internal-vld in the VNF descriptor and ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)): +In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, `internal` is the name of internal-vld in the VNF descriptor and ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)): ```yaml --config '{vnf: [ {member-vnf-index: vnf1, internal-vld: [ {name: internal, ip-profile: {...} ] } ] }' @@ -390,7 +390,7 @@ osm ns-create --ns_name hf-multivdu --nsd_name hackfest_multivdu-ns --vim_accoun #### Specify IP address for an interface -In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor, ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)), `id1` is the internal-connection-point id and `a.b.c.d` is the IP that you have to specify for this scenario: +In this scenario, the mapping can be specified in the following way, where `vnf1` is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor, ip-profile is where you have to fill the associated parameters from the data model ([VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html)), `id1` is the internal-connection-point id and `a.b.c.d` is the IP that you have to specify for this scenario: ```yaml --config '{vnf: [ {member-vnf-index: vnf1, internal-vld: [ {name: internal, ip-profile: {...}, internal-connection-point: [{id-ref: id1, ip-address: "a.b.c.d"}] ] } ] }' @@ -471,7 +471,7 @@ You can try it using one of the examples of the hackfest (**packages: [hackfest_ ```bash osm ns-create --ns_name hf-basic --nsd_name hackfest_basic-ns -With the previous hackfest example, according [VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html) you will add in VNF Descriptor: +With the previous hackfest example, according [VNF data model](https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html) you will add in VNF Descriptor: ```yaml volumes: @@ -1373,7 +1373,7 @@ The diagram below shows the `slice_basic_ns` and `slice_basic_middle_ns`, its co ### Creating a Network Slice Template (NST) -Based on the OSM information model for Network slice templates [here](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/nst.html) it is possible to start writing the YAML descriptor for the NST. +Based on the OSM information model for Network slice templates [here](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/nst.html) it is possible to start writing the YAML descriptor for the NST. ```yaml nst: diff --git a/07-what-to-read-next.md b/07-what-to-read-next.md index 3c36a6a..0fd22a1 100644 --- a/07-what-to-read-next.md +++ b/07-what-to-read-next.md @@ -7,10 +7,7 @@ If you want to learn more, these additional contents are highly recommended: - **[VNF Onboarding Guide](https://osm.etsi.org/docs/vnf-onboarding-guidelines).** This guide includes all that you might need to onboard your VNF or applications in OSM. - **[OSM Developer Guide][developer-guide].** If you want to contribute to OSM development, it is highly recommended reading this guide first. +- **[Sample VNF Packages](https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git).** - **[Contents from latest edition of OSM Hackfest][latest-hackfest].** -- **Sample VNF Packages.** This page is under elaboration, but you can find relevant contents in these locations: - - **[Examples from OSM Hackfests][latest-hackfest]** - - **[Analysis of some Hackfest VNFs](https://osm.etsi.org/wikipub/index.php/Examples_from_OSM_Hackfests)** - - **[ANNEX 6: Tests to validate VIM capabilities from OSM](14-tests-for-vim-validation.md)** - [OSM Workshops and Webinars](https://osm.etsi.org/wikipub/index.php/OSM_workshops_and_events) - [Release notes and white papers](https://osm.etsi.org/wikipub/index.php/Release_notes_and_whitepapers) diff --git a/10-osm-client-commands-reference.md b/10-osm-client-commands-reference.md index edff5a6..b2f13ef 100644 --- a/10-osm-client-commands-reference.md +++ b/10-osm-client-commands-reference.md @@ -144,9 +144,9 @@ Commands: ## How to install standalone OSM client -### How to install OSM Client in Ubuntu 20.04 (RECOMMENDED) +### How to install OSM Client in Ubuntu 22.04 (RECOMMENDED) -OSM client is installed by default in the host where OSM is installed, but it can be also installed as a standalone client in an Ubuntu 20.04 system, following the procedure below: +OSM client is installed by default in the host where OSM is installed, but it can be also installed as a standalone client in an Ubuntu 22.04 system, following the procedure below: ```bash # Clean the previous repos that might exist @@ -154,13 +154,12 @@ sudo sed -i "/osm-download.etsi.org/d" /etc/apt/sources.list # Install dependencies sudo apt-get update -sudo apt-get install -y libcurl4-openssl-dev libssl-dev software-properties-common apt-transport-https -sudo apt-get install -y python3.8 python3-setuptools python3-dev python3-pip +sudo apt-get install -y python3 python3-dev python3-pip # Add OSM debian repo -curl -q -o OSM-ETSI-Release-key.gpg https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/OSM%20ETSI%20Release%20Key.gpg +curl -q -o OSM-ETSI-Release-key.gpg https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/OSM%20ETSI%20Release%20Key.gpg sudo apt-key add OSM-ETSI-Release-key.gpg -sudo add-apt-repository -y "deb [arch=amd64] https://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN stable devops IM osmclient" +sudo add-apt-repository -y "deb [arch=amd64] https://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN stable devops IM osmclient" sudo apt-get update # Install OSM IM and osmclient packages from deb repo @@ -173,20 +172,19 @@ sudo -H python3 -m pip install -r /usr/lib/python3/dist-packages/osm_im/requirem sudo snap install charm --classic ``` -### How to install OSM Client in Ubuntu 20.04 via pip (ALTERNATIVE) +### How to install OSM Client in Ubuntu 22.04 via pip (ALTERNATIVE) ```bash # Install dependencies sudo apt-get update -sudo apt-get install -y libcurl4-openssl-dev libssl-dev software-properties-common apt-transport-https sudo apt-get install -y git wget make -sudo apt-get install -y python3.8 python3-setuptools python3-dev python3-pip +sudo apt-get install -y python3 python3-dev python3-pip # Upgrade pip to the latest version (with sudo, to install it globally for all users) sudo -H python3 -m pip install -U pip -# Decide which version to use (e.g., v13.0) -export OSM_CLIENT_VERSION=v13.0 +# Decide which version to use (e.g., v14.0) +export OSM_CLIENT_VERSION=v14.0 # Install OSM IM and its dependencies via pip (installed with sudo, to install it globally for all users) sudo -H python3 -m pip install -r "https://osm.etsi.org/gitweb/?p=osm/IM.git;a=blob_plain;f=requirements.txt;hb=${OSM_CLIENT_VERSION}" @@ -204,20 +202,20 @@ sudo -H python3 -m pip install ./osmclient sudo snap install charm --classic ``` -### How to install OSM Client in RHEL 8.4 +### How to install OSM Client in RHEL ```bash # Install dependencies sudo dnf upgrade -y sudo dnf install -y libcurl-devel openssl-devel sudo dnf install -y git wget make patch gcc -sudo dnf install -y python38 python38-devel +sudo dnf install -y python310 python310-devel # Upgrade pip to the latest version (with sudo, to install it globally for all users) sudo -H python3 -m pip install -U pip -# Decide which version to use (e.g., v13.0) -export OSM_CLIENT_VERSION=v13.0 +# Decide which version to use (e.g., v43.0) +export OSM_CLIENT_VERSION=v14.0 # Install OSM IM and its dependencies via pip (installed with sudo, to install it globally for all users) sudo -H python3 -m pip install -r "https://osm.etsi.org/gitweb/?p=osm/IM.git;a=blob_plain;f=requirements.txt;hb=${OSM_CLIENT_VERSION}" @@ -238,20 +236,19 @@ sudo chmod 755 /usr/sbin/charm ``` -### How to install OSM Client for developers in Ubuntu 20.04 +### How to install OSM Client for developers in Ubuntu 22.04 ```bash # Install dependencies sudo apt-get update -sudo apt-get install -y libcurl4-openssl-dev libssl-dev software-properties-common apt-transport-https sudo apt-get install -y git wget make -sudo apt-get install -y python3.8 python3-setuptools python3-dev python3-pip +sudo apt-get install -y python3 python3-setuptools python3-dev python3-pip # Upgrade pip to the latest version (with sudo, to install it globally for all users) sudo -H python3 -m pip install -U pip -# Decide which version to use (e.g., v13.0) -export OSM_CLIENT_VERSION=v13.0 +# Decide which version to use (e.g., v14.0) +export OSM_CLIENT_VERSION=v14.0 # Install OSM IM and its dependencies via pip (installed with sudo, to install it globally for all users) sudo -H python3 -m pip install -r "https://osm.etsi.org/gitweb/?p=osm/IM.git;a=blob_plain;f=requirements.txt;hb=${OSM_CLIENT_VERSION}" @@ -276,7 +273,7 @@ which osm OSM client can be easily installed in Windows by installing an Ubuntu distro on Linux with [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/install). -Once WSL is installed with an Ubuntu 20.04 distro, you can install OSM following the instruccions in (#how-to-install-osm-client-in-ubuntu-2004) +Once WSL is installed with an Ubuntu 22.04 distro, you can install OSM following the instruccions in (#how-to-install-osm-client-in-ubuntu-2204) ### How to install OSM Client directly in Windows with Conda and Git @@ -300,7 +297,7 @@ Make sure that aliases for Python are disabled in Windows Configuration. Go to S Open Git Bash and run the following commands to create a Conda environment with Python 3.8 and initialize all shells to work with Conda: ```bash -conda create -n osm-env python=3.8 +conda create -n osm-env python=3.10 conda init --all # Logout ``` @@ -310,13 +307,12 @@ Then install OSM client as follows: ```bash # Install conda and install some packages via conda (which will install dependent libraries) conda activate osm-env -conda install pycurl==7.45.1 # Upgrade pip to the latest version (with sudo, to install it globally for all users) python -m pip install -U pip -# Decide which version to use (e.g., v13.0) -export OSM_CLIENT_VERSION=v13.0 +# Decide which version to use (e.g., v14.0) +export OSM_CLIENT_VERSION=v14.0 # Clone IM repo and checkut the desired version git clone https://osm.etsi.org/gerrit/osm/IM @@ -651,7 +647,7 @@ Assuming that you have installed python-osmclient package, it's pretty simple to from osmclient import client from osmclient.common.exceptions import ClientException hostname = "127.0.0.1" -myclient = client.Client(host=hostname, sol005=True) +myclient = client.Client(host=hostname) resp = myclient.nsd.list() print yaml.safe_dump(resp) ``` diff --git a/11-osm-im.md b/11-osm-im.md index 06665b6..b037b97 100644 --- a/11-osm-im.md +++ b/11-osm-im.md @@ -14,21 +14,21 @@ Below you can find tree representations of the VNFD (VNF Descriptor), NSD (Netwo ### VNFD tree -* [Navigable Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html) +* [Navigable Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.html) -* [Plain Text Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.tree.txt) +* [Plain Text Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-vnfd.tree.txt) ### NSD tree -* [Navigable Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.html) +* [Navigable Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.html) -* [Plain Text Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.tree.txt) +* [Plain Text Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/etsi-nfv-nsd.tree.txt) ### NST tree -* [Navigable Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/nst.html) +* [Navigable Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/nst.html) -* [Plain Text Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseTHIRTEEN/docs/osm-im/osm_im_trees/nst.tree.txt) +* [Plain Text Version](http://osm-download.etsi.org/repository/osm/debian/ReleaseFOURTEEN/docs/osm-im/osm_im_trees/nst.tree.txt) ## OSM URN Namespace diff --git a/20-tutorial.md b/20-tutorial.md index 66c890f..2f67715 100644 --- a/20-tutorial.md +++ b/20-tutorial.md @@ -1,4 +1,5 @@ -# OSM installation tutorial +# Annex 10: Tutorial - Charmed OSM installation and deployment of a CNF + ## Introduction In this tutorial you will install OSM that will be used to deploy and configure a CNF in a K8s cluster. Along the way you will attach a VIM and a K8s cluster to OSM in order to be able to deploy workloads, and also upload VNF packages, so they can be used later on. diff --git a/index.md b/index.md index 229f6e2..74087da 100644 --- a/index.md +++ b/index.md @@ -25,6 +25,6 @@ 16. [ANNEX 7: Setting up an LXD Cluster](16-lxd-cluster.md) 17. [ANNEX 8: TACACS Based Authentication Support In OSM](18-tacacs-based-authentication.md) 18. [ANNEX 9: LTS Upgrade](19-lts-upgrade.md) -19. [OSM tutorial](20-tutorial.md) +19. [ANNEX 10: Tutorial - Charmed OSM installation and deployment of a CNF](20-tutorial.md) 20. [OSM reference](21-reference.md) diff --git a/index.rst b/index.rst index 05afe95..23f256a 100644 --- a/index.rst +++ b/index.rst @@ -30,4 +30,5 @@ Welcome to Open Source MANO's documentation! 18-tacacs-based-authentication.md 19-lts-upgrade.md 20-tutorial.md + 21-reference.md diff --git a/navigation.md b/navigation.md index 09a1b25..a9678f4 100644 --- a/navigation.md +++ b/navigation.md @@ -16,3 +16,9 @@ [ANNEX 4: Reference of OSM's Northbound Interface](12-osm-nbi.md) [ANNEX 5: OpenVIM installation](13-openvim-installation.md) [ANNEX 6: Tests to validate VIM capabilities from OSM](14-tests-for-vim-validation.md) +[ANNEX 6: Kubernetes installation and requirements](15-k8s-installation.md) +[ANNEX 7: Setting up an LXD Cluster](16-lxd-cluster.md) +[ANNEX 8: TACACS Based Authentication Support In OSM](18-tacacs-based-authentication.md) +[ANNEX 9: LTS Upgrade](19-lts-upgrade.md) +[ANNEX 10: Tutorial - Charmed OSM installation and deployment of a CNF](20-tutorial.md) +[OSM reference](21-reference.md) -- GitLab From 0277e93ecbd46203436a8ac0251496e29023863e Mon Sep 17 00:00:00 2001 From: garciadeblas Date: Tue, 25 Jul 2023 13:15:10 +0200 Subject: [PATCH 2/2] Features 10981: Adoption of new monitoring architecture for closed-loops Signed-off-by: garciadeblas --- 21-reference.md | 142 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) diff --git a/21-reference.md b/21-reference.md index 4498a99..c88b676 100644 --- a/21-reference.md +++ b/21-reference.md @@ -1,5 +1,147 @@ # OSM Reference +## Service Assurance Architecture + +Since OSM Release THIRTEEN, OSM offers a scalable architecture for Service Assurance (SA) based on Apache Airflow and Prometheus, which includes closed loops for auto-scaling and the handling of alerts coming from the Network Functions. + +### Goal and principles of the Service Assurance architecture + +The main goal of the Service Architecture is to sense the state of the objects deployed by OSM and build telco cloud use cases with that information: + +- Offering state indicators of the NS and their components, as well as the NFV infrastructure used by OSM. +- Providing capabilities to build closed-loops for scaling and healing + +Metric visualization is not a main objective for the E2E Service Assurance architecture. While OSM provides visualization of the collected and derived metrics, it will always serve for the purpose of building advanced use cases. + +The principles of the Service Assurance archicture are: + +- Do not reinvent the wheel. Instead of building a new framework from scratch, OSM relies on well-known open source projects such as [Prometheus](https://prometheus.io/) [Apache Airflow](https://airflow.apache.org/), respectively for the storage and handling of metrics and alerts, and the execution of workflows. +- Modularity of tasks. Workflows must be split into smaller stages that should be run by the most appropriate component. Similary to the ETL paradigm in data analytics pipelines, inputs could come from anywhere, being processed by any component and outputs could go everywhere. +- Scalability. Task execution must be prepared to run in parallel, so that a collection of workers can run potentially any task, maximizing workload sharing between workers. This also avoids dedicated components for specific tasks. Bottlenecks can be globally managed by scaling the number of workers. +- Configurable schedule of workflows. This is specially important for metric collection, which could be very intensive in resource consumption. +- Easy to add new capabilities. The selected components provide a toolkit to easily incorporate new functionality, lowering the learning curve. + +![Modular architecture for Service Assurance](assets/sa-arch-etl-paradigm.png) + +### Building blocks + +Service Assurance architecture in OSM is characterized by the following: + +- It relies on Apache Airflow and Prometheus stack as main building blocks. Prometheus TSDB and Grafana were already deployed in previous releases of OSM. Since Release THIRTEEN and FOURTEEN, Airflow, Prometheus AlertManager and PushGateway are also deployed. +- It deprecates the use of the old components MON and POL. With the new SA architecture, POL is removed, while the role of MON is restricted to a process to create Grafana dashboards per project and NS. +- Metrics are gathered by dedicated and scheduled workflows running in Airflow +- Enhanced metrics per NS or VNF can be derived in Prometheus from the collected metrics +- Closed-loops for scaling and healing are triggered by alerts generated by Prometheus Alert Manager, and conveniently addressed to Airflow endpoints to run the associated workflows. +- Webhook Translator is a new component that has been added to translate webhooks, allowing the translation of an alert generated by Alert Manager to the appropriate Airflow DAG endpoint, and enabling the capability to translate in the future external alerts from VNF or other systems to dedicated Airflow workflows. + +#### Apache Airflow + +[Apache Airflow](https://airflow.apache.org/) is an open-source workflow management platform for data engineering pipelines. It started at Airbnb in October 2014 as a solution to manage the company's increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface. From the beginning, the project was made open source, becoming an Apache Incubator project in March 2016 and a top-level Apache Software Foundation project in January 2019. + +Airflow is written in Python, and workflows are created via Python scripts. Airflow is designed under the principle of "configuration as code". While other "configuration as code" workflow platforms exist using markup languages like XML, using Python allows developers to import libraries and classes to help them create their workflows. + +Airflow provides a mechanism to run scheduled workflows as well as workflows on demand from an alert. Workflows, called DAGs, are devided in tasks which are run by workers, convenient triggered by the scheduler. In addition, Airflow provides a suitable web-based UI which allows to enable/disable, monitor and log the execution of DAGs. + +![Airflow architecture](assets/sa-arch-airflow-schema.png) + +The way to model workflows in Airflow is via Directed Acyclic Graphs (DAG). A DAG is a collection of all the tasks you want to run, organized in a way that reflects their relationships and dependencies. A DAG is defined in a Python script, which represents the DAG structure (tasks and their dependencies) as code. For example, a simple DAG could consist of three tasks: A, B, and C. It could say that A has to run successfully before B can run, but C can run anytime. It could say that task A times out after 5 minutes, and B can be restarted up to 5 times in case it fails. It might also say that the workflow will run every night at 10pm but shouldn’t start until a certain date. + +![Airflow DAGs and tasks](assets/sa-arch-airflow-relation-dags-tasks.png) + +In summary a DAG is: + +- A collection of tasks, with flexibility to create dependencies between tasks +- Defined in Python +- DAGs can be dynamically created, for instance one per VIM or one per Network Service +- Tasks can be dynamically created inside a DAG, for instance +- Airflow DAGs are designed to scale since a collection of Airflow workers can run tasks in parallel +- Airflow DAGs can be scheduled independently + +![Examples of Airflow DAGs](assets/sa-arch-example-dags-25.png) + +#### Prometheus Stack + +[Prometheus](https://prometheus.io/) is a well-known complete open-source framework for monitoring, including metric collection and storage in a time-series database, following an HTTP pull model, metric query via HTTP with a dedicated language PromQL, metric derivation, alert generation and management, and metric visualization. + +![Prometheus architecture](assets/sa-arch-prometheus-stack.png) + +#### Webhook Translator + +The Webhook Translator a new component that has been added to translate webhooks, allowing the translation of an alert generated by Alert Manager to the appropriate Airflow DAG endpoint, and enabling the capability to translate in the future external alerts from VNF or other systems to dedicated Airflow workflows. + +- It is lightweight. A very small number of lines of code does the work. +- It is stateless. It only translates HTTP requests. No state for those translations. When running as a deployment, native scaling is achieved by means of Kubernetes services. +- It is simple. It is based on [FastAPI](https://fastapi.tiangolo.com/), a simple and fast framework for developing an HTTP REST API in Python. +- It is independent from the source of the alert. +- No maintenance is expected. + +![Webhook Translator](assets/sa-arch-webhook-translator.png) + +The code of the webhook can be found in [Gerrit](https://osm.etsi.org/gerrit/osm/NG-SA) and in [Gitlab](https://osm.etsi.org/gitlab/osm/ng-sa/-/tree/master/osm_webhook_translator). + +### Metric acquisition and derivation + +The following figure illustrates the general workflow for metric acquisition and derivation. + +![Generic workflow for metric acquisition and derivation](assets/sa-arch-osm-metric-acquisition-and-derivation.png) + +Metrics are acquired in OSM with dedicated Airflow DAGs. In some cases, such as for the obtention of the NS topologies, a single DAG is used. For other cases such as the gathering of the VIM status, the VM status and the VM resource consumption metrics, a DAG per VIM is used. The following figure shows a screenshot of the DAGs as shown by Airflow UI. + +![Screenshot of Airflow DAGs](assets/sa-arch-airflow-dags-screenshot.png) + +Below a summary of the metrics acquired by OSM, summarizing its input, output and the software used for the process. + +- NS topology: + - From Mongo DB to Prometheus + - Software used to acquire the metric: Airflow DAG + Prometheus PushGateway +- VM status: + - From MongoDB and VIM to Prometheus + - Software used to acquire the metric: Airflow DAG per VIM + Prometheus PushGateway +- VIM status + - From MongoDB and VIM to Prometheus + - Software used to acquire the metric: Airflow DAG per VIM + Prometheus PushGateway +- VM metrics (resource consumption) + - From MongoDB and VIM to Prometheus + - Software used to acquire the metric: Airflow DAG per VIM + Prometheus PushGateway + +The following figure illustrates graphically the specific workflows for metric acquisition. + +![Specific workflows for metric acquisition](assets/sa-arch-osm-metric-acquisition-workflows.png) + +The metrics, as acquired by OSM, might not be useful per-se, since they are not related to OSM objects. For instance, the VM status or the VM resource consumption metrics are only useful when they relate to a VNF instance or a NS instance. [Prometheus recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) allow to derive metrics from existing metrics, computing them and saving the result as a new set of time series. The following figure shows a screenshot of the metrics derived in Prometheus. + +![Screenshot of Prometheus derived metrics](assets/sa-arch-prometheus-derived-metrics-screenshot.png) + +Below a summary of the metrics derived by OSM, summarizing its input, output and the software used for the process. + +- Extended VM status: + - From Prometheus (NS topology, VM status) to Prometheus + - Software used to derive the metric: Prometheus Recording Rules +- VNF status: + - From Prometheus (Extended VM status) to Prometheus + - Software used to derive the metric: Prometheus Recording Rules +- NS status: + - From Prometheus (Extended VM status) to Prometheus + - Software used to derive the metric: Prometheus Recording Rules + +Finally, the following figure summarizes graphically the specific workflows for metric derivation. + +![Specific workflows for metric derivation](assets/sa-arch-osm-metric-derivation-workflows.png) + +### Closed loops in OSM + +The following figure illustrates the general workflow for closed-loops in OSM that is used both for auto-healing and auto-scaling. + +![Generic workflow for closed loops in OSM](assets/sa-arch-osm-closed-loops.png) + +- **Prometheus alerts**. All monitoring information (status of VIMs and VNFs managed by OSM) is stored in Prometheus. Prometheus has a simple alert generation mechanism based on [alerting rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) that are configured in a similar way as the recording rules. The alerts are automatically triggered and stopped depending on their defining rule. + - For the case of auto-healing operation, a static rule is pre-defined, which generates an alert when the status of a VM is down for more than a given period. + - For the case of auto-scaling operations, a sidecar container deployed with Prometheus dynamicaly updates the Prometheus alert rules as they are created/deleted by OSM in MongoDB at instantiation/termination time. +- **AlertManager**. [AlertManager](https://prometheus.io/docs/alerting/latest/alertmanager/) allows managing alerts, including silencing, inhibition, aggregation and sending out notifications. The alerts generated by Prometheus are sent to AlertManager, which forwards them to a webhook. The use of AlertManager is necessary as an intermediate step because Prometheus cannot send alerts directly to a webhook. In addition, AlertManager includes mechanisms for silencing, inhibition, aggregation, etc., which could be leveraged by OSM in the future. +- **Webhook Translator**. AlertManager can send alerts to multiple external systems via, for example, HTTP, but does not integrate directly with Airflow, which is the system that will run the worflows or DAGs for healing and scaling. Airflow has its own format for webhooks. The Webhook Translator is added to translate HTTP requests: it receives an HTTP request from AlertManager and forwards it to a supported Airflow webhook in the format expected by Airflow. +- **Airflow DAG driven by webhook**. The last step in the pipeline is running the DAG which will process the alert and execute the necessary actions to carry out the closed-loop operation. In the case of auto-healing, when the firing alert about a VM arrives, the DAG will check if there is an auto-healing rule stored in MongoDB regarding to this VM/VDU. In that case, the DAG updates alert status in MongoDB and sends to Kafka a heal message that will be consumed by LCM, which finally performs the healing operation. +- **MongoDB**. The information (auto-healing and auto-scaling rules) used for closed-loop operations is stored in MongoDB in the collection `alerts`. LCM is responsible of inserting and deleting the alarm-related objects when an NS is instantiated or terminated. + ## EPA Parameters ### Virtual CPU -- GitLab