Merge remote-tracking branch 'origin/master' into rel11

_build

_build*/

local/

```bash

kubectl logs -n osm deployments/lcm           # for LCM

kubectl logs -n osm deployments/ng-ui         # for LW-UI

kubectl logs -n osm deployments/ng-ui         # for NG-UI

kubectl logs -n osm deployments/mon           # for MON

kubectl logs -n osm deployments/nbi           # for NBI

kubectl logs -n osm deployments/pol           # for POL

    -u <repo base>: use specified repository url for osm packages

    -k <repo key>:  use specified repository public key url

    --k8s_monitor:  install the OSM kubernetes monitoring with prometheus and grafana

    -m <MODULE>:    install OSM but only rebuild the specified docker images (LW-UI, NBI, LCM, RO, MON, POL, KAFKA, MONGO, PROMETHEUS, PROMETHEUS-CADVISOR, KEYSTONE-DB, NONE)

    -m <MODULE>:    install OSM but only rebuild the specified docker images (NG-UI, NBI, LCM, RO, MON, POL, KAFKA, MONGO, PROMETHEUS, PROMETHEUS-CADVISOR, KEYSTONE-DB, NONE)

    -o <ADDON>:     do not install OSM, but ONLY one of the addons (vimemu, elk_stack) (assumes OSM is already installed)

    --showopts:     print chosen options and exit (only for debugging)

    --uninstall:    uninstall OSM: remove the containers and delete NAT rules

# kubectl -n osm apply -f /etc/osm/docker/osm_pods/nbi.yaml

```

#### Upgrading NG UI in K8s

#### Upgrading Next Generation UI in K8s

```bash

git clone https://osm.etsi.org/gerrit/osm/NG-UI

#you can then work in the cloned repo, apply patches with git pull, etc.

docker build NG-UI -f NG-UI/docker/Dockerfile -t opensourcemano/light-ui:develop --no-cache

docker build NG-UI -f NG-UI/docker/Dockerfile -t opensourcemano/ng-ui:develop --no-cache

kubectl -n osm patch deployment ng-ui --patch '{"spec": {"template": {"spec": {"containers": [{"name": "ng-ui", "image": "opensourcemano/ng-ui:develop"}]}}}}'

kubectl -n osm scale deployment ng-ui --replicas=0

kubectl -n osm scale deployment ng-ui --replicas=1

```bash

kubectl logs -n osm deployments/lcm           # for LCM

kubectl logs -n osm deployments/ng-ui         # for LW-UI

kubectl logs -n osm deployments/ng-ui         # for NG-UI

kubectl logs -n osm deployments/mon           # for MON

kubectl logs -n osm deployments/nbi           # for NBI

kubectl logs -n osm deployments/pol           # for POL

More details about this option are in ellaboration and will be shared soon.

### VIM Emulator

#### Vim-emu: A NFV multi-PoP emulation platform

This emulation platform was created to support network service developers to locally prototype and test their network services in realistic end-to-end multi-PoP scenarios. It allows the execution of real network functions, packaged as Docker containers, in emulated network topologies running locally on the developer's machine. The emulation platform also offers OpenStack-like APIs for each emulated PoP so that it can integrate with MANO solutions, like OSM. The core of the emulation platform is based on [Containernet](https://containernet.github.io/).

The emulation platform `vim-emu` was previously developed as part of the EU H2020 project [SONATA](http://sonata-nfv.eu/) and is now developed as part of OSM's DevOps MDG.

##### Cite this work

If you plan to use this emulation platform for academic publications, please cite the following paper:

- M. Peuster, H. Karl, and S. v. Rossem: [**MeDICINE: Rapid Prototyping of Production-Ready Network Services in Multi-PoP Environments**](http://ieeexplore.ieee.org/document/7919490/). IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Palo Alto, CA, USA, pp. 148-153. doi: 10.1109/NFV-SDN.2016.7919490. (2016)

##### Scope

The following figure shows the scope of the emulator solution and its mapping to a simplified ETSI NFV reference architecture in which it replaces the network function virtualisation infrastructure (NFVI) and the virtualised infrastructure manager (VIM). The design of vim-emu is based on a tool called Containernet which extends the well-known Mininet emulation framework and allows us to use standard Docker containers as VNFs within the emulated network. It also allows adding and removing containers from the emulated network at runtime which is not possible in Mininet. This concept allows us to use the emulator like a cloud infrastructure in which we can start and stop compute resources (in the form of Docker containers) at any point in time.

![Vim-emu-etsi-mapping.png](assets/300px-Vim-emu-etsi-mapping.png)]

##### Architecture

The vim-emu system design follows a highly customizable approach that offers plugin interfaces for most of its components, like cloud API endpoints, container resource limitation models, or topology generators.

In contrast to classical Mininet topologies, vim-emu topologies do not describe single network hosts connected to the emulated network. Instead, they define available PoPs which are logical cloud data centers in which compute resources can be started at emulation time. In the most simplified version, the internal network of each PoP is represented by a single SDN switch to which compute resources can be connected. This can be done as the focus is on emulating multi-PoP environments in which a MANO system has full control over the placement of VNFs on different PoPs but limited insights about PoP internals. We extended Mininet's Python-based topology API with methods to describe and add PoPs. The use of a Python-based API has the benefit that developers can use scripts to define or algorithmically generate topologies.

Besides an API to define emulation topologies, an API to start and stop compute resources within the emulated PoPs is available. Von-emu uses the concept of flexible cloud API endpoints. A cloud API endpoint is an interface to one or multiple PoPs that provides typical infrastructure-as-a-service (IaaS) semantics to manage compute resources. Such an endpoint can be an OpenStack Nova or HEAT like interface, or a simplified REST interface for the emulator CLI. These endpoints can be easily implemented by writing small, Python-based modules that translate incoming requests (e.g., an OpenStack Nova start compute) to emulator specific requests (e.g., start Docker container in PoP1).

As illustrated in the following figure, our platform automatically starts OpenStack-like control interfaces for each of the emulated PoPs which allow MANO systems to start, stop and manage VNFs. Specifically, our system provides the core functionalities of OpenStack's Nova, Heat, Keystone, Glance, and Neutron APIs. Even though not all of these APIs are directly required to manage VNFs, all of them are needed to let the MANO systems believe that each emulated PoP in our platform is a real OpenStack deployment. From the perspective of the MANO systems, this setup looks like a real-world multi-VIM deployment, i.e., the MANO system's southbound interfaces can connect to the OpenStack-like VIM interfaces of each emulated PoP. A demonstration of this setup was presented at [IEEE NetSoft 2017](http://ieeexplore.ieee.org/abstract/document/8004250/).

![Vim-emu-setup.png](assets/400px-Vim-emu-setup.png)

##### Example: OSM using vim-emu

This section gives an end-to-end usage example that shows how to connect OSM to a vim-emu instance and how to on-board and instantiate an example network service with two VNFs on the emulated infrastructure. All given paths are relative to the vim-emu repository root. The same example is also available for the classic build of OSM: [vim-emu classic build walkthrough](https://osm.etsi.org/wikipub/index.php/VIM_emulator_classic_build_walkthrough).

<iframe src="https://www.youtube.com/embed/Iji6FFIKL0w?" width="640" height="360" frameborder="0" allowfullscreen="true" style="box-sizing: border-box;"></iframe>

###### Example service: `pingpong`

####### Source descriptors

- Ping VNF (default ubuntu:trusty Docker container): `vim-emu/examples/vnfs/ping_vnf/`

- Pong VNF (default ubuntu:trusty Docker container): `vim-emu/examples/vnfs/pong_vnf/`

- Network service descriptor (NSD): `vim-emu/examples/services/pingpong_ns/`

####### Pre-packed VNF and NS packages

- Ping VNF: `vim-emu/examples/vnfs/ping.tar.gz`

- Pong VNF: `vim-emu/examples/vnfs/pong.tar.gz`

- NSD: `vim-emu/examples/services/pingpong_nsd.tar.gz`

###### Walkthrough

####### Step 1: Install OSM and vim-emu

Install OSM together with the emulator.

```bash

$ ./install_osm.sh --vimemu

```

######## Step 1.1: Start the emulator

Check if the emulator is running:

```bash

$ docker ps | grep vim-emu

```

If not, start it with the following command:

```bash

$ docker run --name vim-emu -t -d --rm --privileged --pid='host' --network=netosm -v /var/run/docker.sock:/var/run/docker.sock vim-emu-img python examples/osm_default_daemon_topology_2_pop.py

```

######## Step 1.2: Configure environment

You need to set the correct environment variables, i.e., you need to get the IP address of the vim-emu container to be able to add it as a VIM to your OSM installation:

```bash

$ export VIMEMU_HOSTNAME=$(sudo docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' vim-emu)

```

####### Step 2: Attach OSM to vim-emu

```bash

# connect OSM to emulated VIM

$ osm vim-create --name emu-vim1 --user username --password password --auth_url http://$VIMEMU_HOSTNAME:6001/v2.0 --tenant tenantName --account_type openstack

# list vims

$ osm vim-list

+----------+--------------------------------------+

| vim name | uuid                                 |

+----------+--------------------------------------+

| emu-vim1 | a8175948-efcf-11e7-94ad-00163eba993f |

+----------+--------------------------------------+

```

####### Step 3: On-board example *pingpong* service

The example can be found in the vim-emu git repository: <https://osm.etsi.org/gitweb/?p=osm/vim-emu.git;a=summary>.

```bash

# Clone the vim-emu repository containing the pingpong example

$ git clone https://osm.etsi.org/gerrit/osm/vim-emu.git

# VNFs

$ osm vnfd-create vim-emu/examples/vnfs/ping.tar.gz

$ osm vnfd-create vim-emu/examples/vnfs/pong.tar.gz

# NS

$ osm nsd-create vim-emu/examples/services/pingpong_nsd.tar.gz

# You can now check OSM's GUI to see the VNFs and NS in the catalog. Or:

$ osm vnfd-list

+-----------+--------------------------------------+

| vnfd name | id                                   |

+-----------+--------------------------------------+

| ping      | 2c632bc7-15f6-4997-a581-b9032ea4672c |

| pong      | e6fe076d-9d1f-4f05-a641-44b3e09df961 |

+-----------+--------------------------------------+

$ osm nsd-list

+----------+--------------------------------------+

| nsd name | id                                   |

+----------+--------------------------------------+

| pingpong | 776746fe-7c48-4f0c-8509-67da1f8c0678 |

+----------+--------------------------------------+

```

####### Step 4: Instantiate example *pingpong* service

```bash

$ osm ns-create --nsd_name pingpong --ns_name test --vim_account emu-vim1

```

####### Step 5: Check service instance

```bash

# using OSM client

$ osm ns-list

+------------------+--------------------------------------+--------------------+---------------+-----------------+

| ns instance name | id                                   | operational status | config status | detailed status |

+------------------+--------------------------------------+--------------------+---------------+-----------------+

| test             | 566e6c36-5f42-4f3d-89c7-dadcca01ae0d | running            | configured    | done            |

+------------------+--------------------------------------+--------------------+---------------+-----------------+

```

####### Step 6: Interact with deployed VNFs

```bash

# connect to ping VNF container (in another terminal window):

$ sudo docker exec -it mn.dc1_test-1-ubuntu-1 /bin/bash

# show network config

#root@dc1_test-nsi:/# ifconfig

eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:03

          inet addr:172.17.0.3  Bcast:0.0.0.0  Mask:255.255.0.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:8 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:648 (648.0 B)  TX bytes:0 (0.0 B)

ping0-0   Link encap:Ethernet  HWaddr 4a:57:93:a0:d4:9d

          inet addr:192.168.100.3  Bcast:192.168.100.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

# ping the pong VNF over the attached management network

#root@dc1_test-1-ubuntu-1:/# ping 192.168.100.4

PING 192.168.100.4 (192.168.100.4) 56(84) bytes of data.

64 bytes from 192.168.100.4: icmp_seq=1 ttl=64 time=0.596 ms

64 bytes from 192.168.100.4: icmp_seq=2 ttl=64 time=0.070 ms

--- 192.168.100.4 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 999ms

rtt min/avg/max/mdev = 0.048/0.059/0.070/0.011 ms

```

####### Step 7: Shut down

```bash

# delete service instance

$ osm ns-delete test

```

####### (optional) Step 8: Check vim-emu and its status

```bash

# connect to vim-emu Docker container to see its logs ( do in another terminal window)

$ sudo docker logs -f vim-emu

# check if the emulator is running in the container

$ sudo docker exec vim-emu vim-emu datacenter list

+---------+-----------------+----------+----------------+--------------------+

#| Label   | Internal Name   | Switch   |   # Containers |   # Metadata Items |

+=========+=================+==========+================+====================+

| dc2     | dc2             | dc2.s1   |              0 |                  0 |

+---------+-----------------+----------+----------------+--------------------+

| dc1     | dc1             | dc1.s1   |              0 |                  0 |

+---------+-----------------+----------+----------------+--------------------+

# check running service

$ sudo docker exec vim-emu vim-emu compute list

+--------------+----------------------------+---------------+------------------+-------------------------+

| Datacenter   | Container                  | Image         | Interface list   | Datacenter interfaces   |

+==============+============================+===============+==================+=========================+

| dc1          | dc1_test.ping.1.ubuntu     | ubuntu:trusty | ping0-0          | dc1.s1-eth2             |

+--------------+----------------------------+---------------+------------------+-------------------------+

| dc1          | dc1_test.pong.2.ubuntu     | ubuntu:trusty | pong0-0          | dc1.s1-eth3             |

+--------------+----------------------------+---------------+------------------+-------------------------+

```

##### Build & Installation

There are multiple ways to install and use the emulation platform. The easiest way is the automated installation using the OSM installer. The bare-metal installation requires a freshly installed Ubuntu 16.04 LTS and is done by an ansible playbook. Another option is to use a nested Docker environment to run the emulator inside a Docker container.

###### Automated installation (with OSM)

The following command will install OSM as well as the emulator (as a Docker container) on a local machine. It is recommended to use a machine with Ubuntu 16.04.

```bash

$ ./install_osm.sh --vimemu

```

###### Manual installation (vim-emu only)

####### Option 1: Bare-metal installation

- Requires: Ubuntu 16.04 LTS

```bash

$ sudo apt-get install ansible git aptitude

```

######## Step 1: Containernet installation

```bash

$ cd

$ git clone https://github.com/containernet/containernet.git

$ cd ~/containernet/ansible

$ sudo ansible-playbook -i "localhost," -c local install.yml

```

######## Step 2: vim-emu installation

```bash

$ cd

$ git clone https://osm.etsi.org/gerrit/osm/vim-emu.git

$ cd ~/vim-emu/ansible

$ sudo ansible-playbook -i "localhost," -c local install.yml

```

####### Option 2: Nested Docker Deployment

This option requires a Docker installation on the host machine on which the emulator should be deployed.

```bash

$ git clone https://osm.etsi.org/gerrit/osm/vim-emu.git

$ cd ~/vim-emu

# build the container:

$ docker build -t vim-emu-img .

# run the (interactive) container:

$ docker run --name vim-emu -it --rm --privileged --pid='host' --network=netosm -v /var/run/docker.sock:/var/run/docker.sock vim-emu-img /bin/bash

# alternative: run container with emulator in service mode

$ docker run --name vim-emu -t -d --rm --privileged --pid='host' --network=netosm -v /var/run/docker.sock:/var/run/docker.sock vim-emu-img python examples/osm_default_daemon_topology_2_pop.py

```

##### Additional information and links

- Main publication: M. Peuster, H. Karl, and S. v. Rossem: [**MeDICINE: Rapid Prototyping of Production-Ready Network Services in Multi-PoP Environments**](http://ieeexplore.ieee.org/document/7919490/). IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Palo Alto, CA, USA, pp. 148-153. doi: 10.1109/NFV-SDN.2016.7919490. (2016)

- [Containernet](https://containernet.github.io/)

- [SONATA NFV](http://sonata-nfv.eu/)

- [SONATA wiki (more documentation)](https://github.com/sonata-nfv/son-emu/wiki)

- [Demo Video on YouTube](https://youtu.be/pFL9wDNOBho)

##### Contact

If you have questions, please use the OSM TECH mailing list: [OSM_TECH@LIST.ETSI.ORG](https://mail.google.com/mail/?view=cm&fs=1&tf=1&to=OSM_TECH@LIST.ETSI.ORG)

Direct contact: Manuel Peuster (Paderborn University) <manuel@peuster.de>

#### Known limitations of VIM Emulator

- VIM Emulator requires special VM images, suitable for running in a VIM Emulator environment.

- Day-1 and Day-2 procedures of OSM are a work in progress in VIM Emulator, and hence are not available as of the date of this publication.

### DevStack

DevStack is a series of extensible scripts used to quickly bring up a complete OpenStack environment based on the latest versions of everything from git master. It is used interactively as a development environment and as the basis for much of the OpenStack project’s functional testing.

The scaling descriptor is part of a VNFD. Like the example below shows, it mainly specifies:

- An existing metric to be monitored, which should be pre-defined in the monitoring-param list (`vnf-monitoring-param-ref`).

- The VDU to be scaled (vdu-id-ref) and the amount of instances to scale per event (`count`)

- The VDU to be scaled (`aspect-delta-details:deltas:vdu-delta:id`) and the amount of instances to scale per event (`number-of-instances`)

- The thresholds to monitor (`scale-in/out-threshold`)

- The minimum and maximum amount of **scaled instances** to produce.

- The VDU's (`vdu-profile:id`) minimum and maximum amount of **scaled instances** to produce

- The minimum time it should pass between scaling operations (`cooldown-time`)

- The minimum amount of scaled instances to produce (`max-scale-level`)

```yaml

scaling-group-descriptor:

-   name: "cpu_autoscaling_descriptor"

    min-instance-count: 0

    max-instance-count: 10

scaling-aspect:

-   aspect-delta-details:

        deltas:

        -   id: vdu01_autoscale-delta

            vdu-delta:

            -   id: vdu01

                number-of-instances: 1

    id: vdu01_autoscale

    max-scale-level: 1

    name: vdu01_autoscale

    scaling-policy:

    -   name: "cpu_scaling_policy"

        scaling-type: "automatic"

        cooldown-time: 120

    -   cooldown-time: 120

        name: cpu_scaling_policy

        scaling-criteria:

        -   name: "cpu_autoscaling_criteria"

        -   name: cpu_scaling_policy

            scale-in-relational-operation: LT

            scale-in-threshold: 20

            scale-in-relational-operation: "LT"

            scale-out-threshold: 80

            scale-out-relational-operation: "GT"

            vnf-monitoring-param-ref: "vnf01_cpu_util"

    vdu:

    -   count: 1

        vdu-id-ref: vdu01

            scale-out-relational-operation: GT

            scale-out-threshold: 60

            vnf-monitoring-param-ref: vnf01_cpu_util

        scaling-type: automatic

        threshold-time: 10

vdu-profile:

-   id: vdu01

    min-number-of-instances: 1

    max-number-of-intannces: 11

```

#### Example

1. Your VIM has an accesible 'public' network and a management network (in this case called "PUBLIC" and "vnf-mgmt")

2. Your VIM has the 'haproxy_ubuntu' and 'apache_ubuntu' images, which can be found [here](https://osm-download.etsi.org/ftp/osm-4.0-four/4th-hackfest/images/)

3. You run the following command to match your VIM metrics telemetry system's granularity, if different than 300s (recommended for this example is 60s or Gnocchi's `medium archive-policy`):

```bash

docker service update --env-add OS_DEFAULT_GRANULARITY=60 osm_mon

```

Get the descriptors:

```bash

wget https://osm-download.etsi.org/ftp/osm-4.0-four/4th-hackfest/packages/webserver_vimmetric_autoscale_nsd.tar.gz

wget https://osm-download.etsi.org/ftp/osm-4.0-four/4th-hackfest/packages/webserver_vimmetric_autoscale_vnfd.tar.gz

git clone https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git

```

Onboard them:

```bash

osm vnfd-create webserver_vimmetric_autoscale_vnfd.tar.gz

osm nsd-create webserver_vimmetric_autoscale_nsd.tar.gz

cd osm-packages

osm vnfd-create wiki_webserver_autoscale_vnfd

osm nsd-create wiki_webserver_autoscale_nsd

```

Launch the NS:

```bash

osm ns-create --ns_name web01 --nsd_name webserver_vimmetric_autoscale_ns --vim_account <VIM_ACCOUNT_NAME>|<VIM_ACCOUNT_ID>

osm ns-create --ns_name web01 --nsd_name wiki_webserver_autoscale_ns --vim_account <VIM_ACCOUNT_NAME>|<VIM_ACCOUNT_ID>

osm ns-list

osm ns-show web01

```

1. To ensure the NS is working, visit the Load balancer's IP at the public network using a browser, the page should show an OSM logo and active VDUs.

2. To check metrics at Prometheus, visit `http://[OSM_IP]:9091` and look for `osm_cpu_utilization` and `osm_average_memory_utilization` (initial values could take some some minutes depending on your telemetry system's granularity).

3. To check metrics at Grafana, just install the OSM preconfigured version (`./install_osm.sh -o pm_stack`) and visit `http://[OSM_IP]:3000` (`admin`/`admin`), you will find a sample dashboard (the two top charts correspond to this example).

3. To check metrics at Grafana, just visit `http://[OSM_IP]:3000` (`admin`/`admin`), you will find a sample dashboard (the two top charts correspond to this example).

4. To increase CPU in this example to auto-scale the web server, install Apache Bench in a client within reach (could be the OSM host) and run it towards `test.php`.

```bash

sudo apt install apache2-utils

ab -n 5000000 -c 2 http://[load-balancer-ip]/test.php

ab -n 5000000 -c 2 http://<load-balancer-ip>/test.php

# Can also be run in the HAproxy machine.

ab -n 10000000 -c 1000 http://<Private IP of Apache webserver>:8080/

# This will stress CPU to 100% and trigger a scale-out operation in POL.

# In this test, scaling will usually go up to 3 web servers before HAProxy spreads to load to reach a normal CPU level (w/ 60s granularity, 180s cooldown)

```

Any of the VMs can be accessed through SSH to further monitor (with `htop`, for example), and there is an HAProxy UI at port `http://[HAProxy_IP]:32700` (all credentials are `osm`/`osm2018`)

If HA proxy is not started

```bash

service haproxy status

sudo service haproxy restart

``` 

Any of the VMs can be accessed through SSH (credential: `ubuntu`/`osm2021`) to further monitor (with `htop`, for example), and there is an HAProxy UI at port `http://[HAProxy_IP]:32700` (credential: `osm`/`osm2018`)

## Using Network Slices