OSM Public Wiki - User contributions [en]

Common issues and troubleshooting

2019-10-03T14:16:13Z

Tierno: /* NBI Errors */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

juju_pubkey=$(cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub)
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with ''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" ''

===NBI Errors===
====Cannot login after migration to 6.0.2====
* '''ERROR''': NBI always return "UNAUTHORIZED" Cannot login neither with UI nor with CLI. CLI shows error "can't find a default project for this user" or "project admin not allowed for this user".
* '''CAUSE''': Normally after a migration to release 6.0.2 There is a slight incompatibility with users creared from older versions
* '''SOLUTION''': Delete user admin and reboot NBI so that a new compatible user is created by running these commands:
curl --insecure https://localhost:9999/osm/test/db-clear/users
docker service update osm_nbi --force

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

Common issues and troubleshooting

2019-10-03T14:14:55Z

Tierno: /* NBI Errors */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

juju_pubkey=$(cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub)
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with ''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" ''

===NBI Errors===
====Cannot login after migration to 6.0.2====
* '''ERROR''': NBI always return "UNAUTHORIZED" Cannot login neither with UI nor with CLI. CLI shows error "can't find a default project for this user" or "project admin not allowed for this user".
* '''CAUSE''': Normally after a migration to release 6.0.2 There is a slight incompatibility with users creared from older versions
* '''SOLUTION''': Delete user admin and reboot NBI so that a new compatible user is created. Run these commands: "curl --insecure https://localhost:9999/osm/test/db-clear/users"; "docker service update osm_nbi --force";

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

Common issues and troubleshooting

2019-10-03T14:13:55Z

Tierno: /* NBI Errors */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

juju_pubkey=$(cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub)
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with ''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" ''

===NBI Errors===
====can't find a default project for this user====
* '''ERROR''': NBI always return "UNAUTHORIZED" Cannot login neither with UI nor with CLI. CLI shows error "can't find a default project for this user" or "project admin not allowed for this user".
* '''CAUSE''': Normally after a migration to release 6.0.2 There is a slight incompatibility with users creared from older versions
* '''SOLUTION''': Delete user admin and reboot NBI so that a new compatible user is created. Run these commands: "curl --insecure https://localhost:9999/osm/test/db-clear/users"; "docker service update osm_nbi --force";

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

Common issues and troubleshooting

2019-10-02T08:43:13Z

Tierno: /* can't find a default project for this user */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

juju_pubkey=$(cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub)
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with ''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" ''

===NBI Errors===
====can't find a default project for this user====
* '''ERROR''': NBI always return "UNAUTHORIZED" "can't find a default project for this user" or "project admin not allowed for this user"
* '''CAUSE''': Normally after a migration to release 6.0.2 There is a slight incompatibility with users creared from older versions
* '''SOLUTION''': Delete user admin and reboot NBI so that a new compatible user is created. Run these commands: "curl --insecure https://localhost:9999/osm/test/db-clear/users"; "docker service update osm_nbi --force";

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

Developer HowTo

2019-10-01T09:51:52Z

Tierno:

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). Current installation uses Docker Swarm by default (there are Kubernetes distributions and an incoming option for installing on Kubernetes). The modules are:

* '''kafka''': Provides a Kafka bus used for OSM communication. This module relies on ''zookeeper''.
* '''zookeeper''': Used by ''kafka''.
* '''nbi''': North Bound Interface of OSM. Restful server that follows ETSI SOL005 interface. Relies on ''mongo'' database and ''kafka'' bus. For authentication it can optionally uses ''keystone''.
* '''keystone''': Used optionally for NBI authentication and RBAC. It stores the users, projects and role permissions. It relies on ''mysql''.
* '''lcm''': Provides the Live Cycle Management. It uses ''ro'' for resource orchestration and juju for configuration. It relies also on ''mongo''.
* '''ro''': Makes the Resource Orchestration, or VIM deployment. Relies on ''mysql''.
* '''light-ui''': Web user interface. It communicates with ''nbi''.
* '''mon''': Performs OSM monitoring. Relies on ''mongo'' and ''mysql''.
* '''mongo''': Common non relational database for OSM modules.
* '''mysql''': Relational database server used for ''ro'', ''keystone'', ''mon'' and ''pol''.
* '''pol''': Policy Manager for OSM.
* '''prometheus''': for monitoring .

In addition LCM and NBI, shares a common file system where packages are stored. In current installation it uses a shared docker volume called ''osm_package''.

This picture shows the modules with the port that they export, their relationships with the name of env variables that control them. For clarity not all the dependencies are depicted, as kafka, mysql, ...

<pre>
__________ ________
| | | |
| light-ui |OSM_SERVER _______ |keystone|
| :80 |----------------------> | |-------------------------> |:5000 |
|__________| | nbi | |________|
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | mysql |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

For debugging it is convenient running the target module directly at host maintaining the rest of dependent modules in containers. In the following sections it is described how to achieve that.

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

==NBI==

Install needed OSM packages IM, common:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
# pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Additionally you may want that docker 'light-ui' uses your local copy of NBI by one of:
* Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:
OSM_SERVER: <172.17.0.1> # nbi
# Get the needed address at your setup by 'ip a | grep docker0'
* ... Or update light-ui docker service (not persistent on reboot):
docker service update osm_light-ui --force --env-add "OSM_SERVER=172.17.0.1"
# Get the needed address at your setup by 'ip a | grep docker0'

Clone and install needed NBI packages
docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/nbi.env'
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:

Values are stored at '/etc/osm/docker/lcm.env' but here it is explained how to obtain these values.
* OSMLCM_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/lcm.env'
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_CACERT: To get this value run the following command in the OSM host:
** juju controllers --format json | jq -r '.controllers["osm"]["ca-cert"]'
* OSMLCM_VCA_PUBKEY: To get this value run the following command in the OSM host:
** cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

It is needed to edit the docker service in order to expose the database port for the 'mysql' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'mysql' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: localhost or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

NBI API Description

2019-09-30T14:23:28Z

Tierno:

__TOC__

== Introduction ==
OSM North Bound Interface is a REST-full following ETSI SOL005 standard. It admits both YAML/JSON formats.

By default it serves https (auto-signed certificate) on port 9999. Bearer authentication (with token) is used. Basic authentication or no authentication is also possible changing 'nbi.cfg' file.

For developping porpouses it admits web browser navigation using a basic http format.

== Request headers ==
'''Accept''': 'application/json','application/yaml' in most of the cases. Use 'text/html' for web navigation. Use 'text/plain','application/zip' for descriptor artifacts showing. This header field shall be present if the response is expected to have a non-empty message body.

'''Content-Type''': 'application/json','application/yaml' for most of the cases. For packages uploading use 'application/gzip','application/zip','application/x-gzip','application/binary'. For descriptor artifacts it can be used 'text/plain'. This header field shall be present if the request has a non-empty message body.

'''Authorization''': 'Bearer <token_id>'. The authorization token for the request. It is mandatory except when a token is requested.

'''Content-Range''': '<first byte>-<last byte>0/<total>'. It allows uploading a descriptor by chunks.

'''Transaction-Id''': '<id>'. When uploaded by chunk, the id returned by first chunk

'''Content-File-MD5''': '<md5>'. MD5 of uploaded package.

'''Content-Filename''': '<filename>'. Filename of the uploaded package.

'''Query-String-Format''': '<yaml>'. Parses query string values using yaml format.

== Response headers ==
'''Content-Type''': 'application/json','application/yaml' is the more normal. 'text/html' when web navigation. 'application/gzip','text/plain' for descriptor artifacts

'''Location''': '/<main_topic>/v1/<topic>/<id>'. Always that something is created, this header is present containing URL to this element.

'''WWW-Authenticate''': 'Bearer realm="{}"'. When Authentication error is obtained.

'''Retry-After''': '<time>'. (Future releases)

==RBAC==
From Release SIX, operations are authorized via RBAC (Role Base Access Control). Each user and project is assigned with a set of roles, and some of them must grant the intended operation. If administrative query string is used (see next section), role must grant the use of it in addition to the operation.

Role permissions are organized in a hierarchical tree, separated by colon, optionally ending with the http operation. The value of False means not granted, meanwhile with True it is granted. If this permission is missing, the immediate hierarchical permission applies. When there is not hierarchical at all, the defined "default" permission applies, and if "default" is missing, not granted is assumed. See these examples [https://osm.etsi.org/gitweb/?p=osm/NBI.git;a=blob;f=osm_nbi/roles_to_operations.yml initial roles ]; and [https://osm.etsi.org/gitweb/?p=osm/NBI.git;a=blob;f=osm_nbi/resources_to_operations.yml role permissions] for the list of rbac permission with their operation. In following sections the rbac permission granting this operation is indicated in bracket.

== Query String ==

Query string has the format: '?key=value&key2=value2&...'.

For GET method it allows filtering with this format, follows SOL005 section 4.3.2:
query string := ?<simpleFilterExpr>["&"<simpleFilterExpr>]*
simpleFilterExpr := <attrName>["."<attrName>]*["."<op>]"="<value>[","<value>]*
op := "eq" | "neq" (or "ne") | "gt" | "lt" | "gte" | "lte" | "cont" | "ncont"
equal (by default), not equal, greater than, less than, greater than or equal, less than or equal, contains, not contains
attrName := string. For filtering inside array, use the index element of the array, or add ANYINDEX to apply the filtering over the same array element

For PUT/POST/PATCH It allows override e.g. package content, with the values of this query string

=== Administrative Query String===

Some special keys used for administration are:
* '''FORCE'''(=True) (rbac: force). Forces and operation. It disables descriptor checking, dependency cheking. For delete, it allows deleting things that cannot done in normal operation because dependency. But beware that forcing deletion will not deletes created things at VIM level.

* '''PUBLIC'''(=True) (rbac: public). Allows getting public descriptor or instances (apart from those belonging to the project used login). For write/update operations, it used for set or unset a descriptor as public

* '''ADMIN'''' (rbac: admin). Allows making operations on behalves on a different project/s than the one used for login (=projectA,projectB,...). Also to show all descriptor or instances (just ADMIN). For update operations..

* '''SET_PROJECT'''=ProjectA,ProjectB,... (rbac: set_project). Allows adding an existing descriptor or instance to a project portfolio with an update operation.

== URL and Methods summary ==

North Bound Interface (O: OSM specific; X,5: SOL005 not OSM implemented yet; O5: OSM and SOL005)

<pre>
URL: /osm GET POST PUT DELETE PATCH
/nsd/v1
/ns_descriptors_content O O
/<nsdInfoId> O O O O
/ns_descriptors O5 O5
/<nsdInfoId> O5 O5 5
/nsd_content O5 O5
/nsd O
/artifacts[/<artifactPath>] O
/pnf_descriptors 5 5
/<pnfdInfoId> 5 5 5
/pnfd_content 5 5

/vnfpkgm/v1
/vnf_packages_content O O
/<vnfPkgId> O O
/vnf_packages O5 O5
/<vnfPkgId> O5 O5 5
/package_content O5 O5
/upload_from_uri X
/vnfd O5
/artifacts[/<artifactPath>] O5

/nslcm/v1
/ns_instances_content O O
/<nsInstanceId> O O
/ns_instances 5 5
/<nsInstanceId> O5 O5
instantiate O5
terminate O5
action O
scale O5
heal 5
/ns_lcm_op_occs 5 5
/<nsLcmOpOccId> 5 5 5
TO BE COMPLETED 5 5
/vnf_instances (also vnfrs for compatibility) O
/<vnfInstanceId> O

/pdu/v1
/pdu_descriptor O O
/<id> O O O O

/admin/v1
/tokens O O
/<id> O O
/users O O
/<id> O O O O
/projects O O
/<id> O O
/vim_accounts (also vims for compatibility) O O
/<id> O O O
/wim_accounts O O
/<id> O O O
/sdns O O
/<id> O O O

/nst/v1 O O
/netslice_templates_content O O
/<nstInfoId> O O O O
/netslice_templates O O
/<nstInfoId> O O O
/nst_content O O
/nst O
/artifacts[/<artifactPath>] O

/nsilcm/v1
/netslice_instances_content O O
/<SliceInstanceId> O O
/netslice_instances O O
/<SliceInstanceId> O O
instantiate O
terminate O
action O
/nsi_lcm_op_occs O O
/<nsiLcmOpOccId> O O O
</pre>

== Administrative Content Details ==

'''/admin/v1/tokens'''. Token is a temporal authorization for a user in a concrete project. Each request must contain it in the http header Authorization
* GET: List the valid granted tokens obtained by this user
* POST: Obtain a token. The 'username' and 'password' is mandatory only if http header does not contain Authorization. 'project' can be provided, if not, first project of available for this user is assigned. For changing project, a new token must be obtained.
** Example of request content: '{username: <user>, password: <pass>, project_id: <project>}'

'''/admin/v1/tokens/<id>'''
* GET: show token details
* DELETE: revoque token. Logout

'''/admin/v1/users'''. (rbac: users). Users may operate in several projects
* GET: Get users
* POST: Add a new user.
** Example of request content: '{username: <user>, password: <pass>, projects: [P1, P2, ...]}'

'''/admin/v1/users/<id>'''. (rbac: users:id).
* GET: Get user details
* PUT/PATCH: Edit user.
** Example of user edition:
*** Change password: '{password: <new_pass>}'
*** Change the list of available projects to a new one: '{projects: [P1, P2, ...]}'
*** Remove a project 'P' from user if exist: {projects: {"$'P'": null}}'
*** Add a project 'P' to user if not already exist: '{projects: {"$+'P'": "P"}}'
*** Set 'P' to be project by default, (adding it if not exist): '{projects: {"$'P'": null, "$+[0]": "P"}}'
* DELETE: Removes user.

'''/admin/v1/projects'''. (rbac: projects). A project contains its own separate space
* GET: Get projects
* POST: Add a new project.
** Example of request content: '{name: <P-name>, admin: <True/False>}'

'''/admin/v1/projects/<id>'''. (rbac: projects:id)
* GET: Get project details
* PUT/PATCH: Edit project.
** Example of project edition to change admin status: '{admin: <True/False>}'
* DELETE: Removes project.

'''/admin/v1/vim_accounts'''. (rbac: vims). Vim account is the deployment target
* GET: Get vim_accounts
* POST: Add a new vim_accounts.
** Example of request content: TODO

'''/admin/v1/vim_accounts/<id>'''. (rbac: vims:id)
* GET: Get vim_account details
* PUT/PATCH: Edit vim_account.
** Example of vim_account edition: TODO
* DELETE: Removes vim_account.

'''/admin/v1/wim_accounts'''. (rbac: wims). WIM are WAN Infrastructure Managers, that is in charge of inter VIM connectivity managemnt
* GET: Get wim_accounts
* POST: Add a new wim_account.
** Example of request content: TODO

'''/admin/v1/wim_accounts/<id>'''. (rbac: wims:id)
* GET: Get wim_account details
* PUT/PATCH: Edit wim_account.
** Example of wim_account edition: TODO
* DELETE: Removes wim_account.

'''/admin/v1/sdns'''. (rbac: sdn_controllers). SDN controllers in charge of VIM underlayed connectivity
* GET: Get sdns
* POST: Add a new sdn.
** Example of request content: TODO

'''/admin/v1/sdns/<id>'''. (rbac: sdn_controllers:id)
* GET: Get sdn details
* PUT/PATCH: Edit sdn.
** Example of sdn edition: TODO
* DELETE: Removes sdn.

== PDU Details ==
Physical Data Unit. This is catalogue of physical elements present at some vim_account/s. Deployment of a PNFD/HNFD implies using one of this pdu from the catalogue. Matching from PNFD/HNFD to PDU is done by 'pdu.type'

'''/pdu/v1/pdu_descriptor'''. (rbac: pduds)
* GET: List PDUs. Use query string for filtering.
* POST: Create a new PDU.
** Example of request content:
<pre>
name: descriptive name # mandatory
type: str # mandatory. maps VNFD.vdu.pdu-type
description: str
vim_accounts: [vim-account-id, ...] # mandatory
interfaces: # mandatory
- name: str # mandatory. Maps VNFD.vdu.interfaces.name
mgmt: boolean # mandatory. Indicates if is the management one
ip-address: ip # mandatory
vim-network-id: xxx
vim-network-name: xxx
</pre>

'''/pdu/v1/pdu_descriptor/<id>'''. (rbac: pduds:id)
* GET: show PDU details
* PUT: overwrite this PDU
* PATCH: edit this PDU
* DELETE: remove the PDU

== VNFD Details ==
Virtual Network Function Descriptor. This includes also Physical and Hybrid Network Funcitons Descriptors (PNFD/HNFD). (At VDU level, it is indicated if it is physical with the 'pdu-type' information element).

TBC

== NSD Details ==
Network Services Descriptor

TBC

== NSLCM Details ==
Network Services Life Ciclo Management

'''/nslcm/v1/ns_instances_content'''.
* GET: (rbac: ns_instances:get) Obtains a list of NS
* POST: (rbac: ns_instances:content:post) (Asynchronous). Creates and Instantiate a NS. It returns the nsInstanceId in the response header 'Location'. Example of request content:
<pre>
nsName: name # mandatory
nsdId: nsd-internal-id # mandatory
vimAccountId: internal-id # mandatory
wimAccountId: False/True/internal-id
additionalParamsForNs: {param1: value1, param2: value2}
additionalParamsForVnf: [member-vnf-index: "1", additionalParams: {param1: value1, param2: value2}]
ssh_keys: [ssh-key-pub1, ssh-key-pub2, ]
vnf: [ {member-vnf-index: "1", vimAccountId: vim-uuid, internal-vld: [], vdu: [] } ]
vld: [ {name: vld-name, ip-profile: {}, vnfd-connection-point-ref: [{}] }]
</pre>

'''/nslcm/v1/ns_instances_content/<nsInstanceId>'''. (rbac: ns_instances:id)
* GET: Obtains details of a NS
* DELETE: (Asynchronous) Terminates (or undeploy) and deletes (if termination is done successfully) a NS

'''/nslcm/v1/ns_instances'''. (rbac: ns_instances)
* GET: Obtains a list of NS
* POST: Creates a NS, but do not instantitate it. It returns the nsInstanceId in the response header 'Location'. Example of request content:
<pre>
nsName: name # mandatory
nsdId: nsd-internal-id # mandatory
vimAccountId: internal-id # mandatory
wimAccountId: False/True/internal-id
additionalParamsForNs: {param1: value1, param2: value2}
additionalParamsForVnf: [member-vnf-index: "1", additionalParams: {param1: value1, param2: value2}]
</pre>

'''/nslcm/v1/ns_instances/<nsInstanceId>'''. (rbac: ns_instances:id)
* GET: Obtains details of a NS
* DELETE: Deletes a NS. It fails if NS is in 'INSTANTIATED' state

'''/nslcm/v1/ns_instances/<nsInstanceId>/instantiate'''. (rbac: ns_instances:id:instantiate:post)
* POST: (Asynchronous). Deploys a NS that it is in NOT_INSTANTIATED state. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'. Example of request content:
<pre>
vimAccountId: internal-id # mandatory
wimAccountId: False/True/internal-id
ssh_keys: [ssh-key-pub1, ssh-key-pub2, ]
vnf: [ {member-vnf-index: "1", vimAccountId: vim-uuid, internal-vld: [], vdu: [] } ]
vld: [ {name: vld-name, ip-profile: {}, vnfd-connection-point-ref: [{}] }]
</pre>

'''/nslcm/v1/ns_instances/<nsInstanceId>/terminate'''. (rbac: ns_instances:id:terminate:post)
* POST: (Asynchronous). Undeploys a NS that it is in INSTANTIATED state. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'

'''/nslcm/v1/ns_instances/<nsInstanceId>/action'''. (rbac: ns_instances:id:action:post)
* POST: (Asynchronous). Performs a Day-2 operation over the NS. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'
** Example of content to executing a primitive:
<pre>
member_vnf_index: str # Needed for VNFD mapping NSD.constituent-vnfd.member-vnf-index. If missing a NSD is targeting
vdu_id: str # Needed if target is a VDU. Maps VNFD.vdu-id
vdu_count_index: int # Needed if the VDU has several replicas (count>1)
primitive: str # Mandatory. Maps [NSD.ns-configuration or VNFD.vnf-configuration or VNFD.vdu.vdu-configuration].config-primitive.name
primitive_params: dict # Optional. Maps [NSD.ns-configuration or VNFD.vnf-configuration or VNFD.vdu.vdu-configuration].config-primitive.parameter
param: value
...
</pre>
'''/nslcm/v1/ns_instances/<nsInstanceId>/scale'''. (rbac: ns_instances:id:scale:post)
* POST: (Asynchronous). Performs a manual scale OUT/IN over one of the VNFs of the NS. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'
** Example of content: '{scaleType: SCALE_VNF, scaleVnfData: {scaleVnfType: SCALE_OUT, scaleByStepData: {member-vnf-index: <nsd:constituent-vnfd:member-vnf-index>, scaling-group-descriptor: <vnfd:scaling-group-descriptor:name>} } }' # Use SCALE_IN instead of SCALE OUT depending of desired type.

'''/nslcm/v1/ns_instances/nsi_lcm_op_occs'''. (rbac: ns_instances:opps)
* GET: Obtains the list of Operations occurrences. It may be useful filtering by a concrete NS with '?nsInstanceId=<nsInstaceId>'

'''/nslcm/v1/ns_instances/nsi_lcm_op_occs/<nsiLcmOpOccId>'''. (rbac: ns_instances:opps:id)
* GET: Show details of an Operations occurrences.

'''/nslcm/v1/vnf_instances'''. (rbac: vnf_instances). (also vnfrs for compatibility):
* GET: Obtains the list of VNF. It may be useful filtering by a concrete NS with '?nsr-id-ref=<nsInstaceId>'

'''/nslcm/v1//vnf_instances/<vnfInstanceId>'''. (rbac: vnf_instances:id)
* GET: Show details of a concrete VNF

== NST Details ==
NetSlice Template descriptor

'''/nst/v1/netslice_templates_content'''
* GET: (rbac: slice_templates:get). Obtain list of Network Slice Templates
* POST (rbac: slice_templates:content:post). : Create a Network Slice Template and upload the content

'''/nst/v1/netslice_templates_content/<nstInfoId>''''. (rbac: slice_templates:id)
* GET: Show the detail of Network Slice Template
* PUT: Update the whole Network Slice Template
* DELETE: Remove a Network Slice template. The Network Slice template is removed only if state is NOT_IN_USE.

'''/nst/v1/netslice_templates'''. (rbac: slice_templates)
* GET: Obtain list of Network Slice Templates
* POST: Create an empty Network Slice Template. A PUT should be done afterwards to upload the package

'''/nst/v1/netslice_templates/<nstInfoId>'''. (rbac: slice_templates:id)
* GET: Show the detail of Network Slice Template descriptor used by OSM
* DELETE: Remove a Network Slice Template. The Network Slice template is removed only if state is NOT_IN_USE.
* PATCH: Update a Network Slice Template field

'''/nst/v1/netslice_templates/<nstInfoId>/nst_content'''. (rbac: slice_templates:content)
* GET: Show the detail of the source Network Slice Template package
* PUT: Update a Network Slice Template

'''/nst/v1/netslice_templates/<nstInfoId>/nst'''. (rbac: slice_templates:id:nst)
* GET: Show the detail of the source Network Slice Template descriptor

'''/nst/v1/netslice_templates/<nstInfoId>/artifacts[/<artifactPath>]'''. (rbac: slice_templates:id:nst_artifact)
* GET: Obtain Network Slice Templates Artifacts. This is useful for download the Network Slice Templates

== NSILCM Details ==
NetSlice Instance Life Cicle Management

'''/nsilcm/v1/netslice_instances_content'''
* GET: (rbac: slice_instances:get). Obtain the list of Network Slice Instances
* POST: (rbac: slice_instances:content:post) (Asynchronous). Creates and Instantiate a Network Slice Instance. It returns the netsliceInstanceId in the response header 'Location'. Example of request content:

<pre>
nstId: name of the Network Slice Template #mandatory
nsiName: name of the Network Slice Instance # mandatory
vimAccountId: internal-id # mandatory
ssh_keys: comma separated list of keys to inject to vnfs
nsiDescription: description of the Network Slice Instance
additionalParamsForNsi: {param: value, ...}
netslice-subnet: [ Same content as section #NSLCM_Details /nslcm/v1/ns_instances_content
],
netslice-vld: [
name: TEXT,
vim-network-name: TEXT or DICT with the name for each vim account: {vimAccountId: network-name, ...},
vim-network-id: TEXT or DICT with the id for each vim account {vimAccountId: network-id},
ip-profile: Profile of the vld
]
</pre>

'''/nsilcm/v1/netslice_instances_content/<SliceInstanceId>'''. (rbac: slice_instances:id)
* GET: Show details of a concrete Network Slice Instance
* DELETE: (Asynchronous) Terminates (or undeploy) and deletes (if termination is done successfully) a Network Slice Instance Record

'''/nsilcm/v1/netslice_instances'''. (rbac: slice_instances)
* GET: Obtain the list of Network Slice Instances
* POST: Create the Network Slice Instance but do not instantitate it. It returns the netsliceInstanceId in the response header 'Location'. Example of request content: Same content as section [[#NSILCM_Details|NSILCM Details /nsilcm/v1/netslice_instances_content]]

'''/nsilcm/v1/netslice_instances/<SliceInstanceId>'''. (rbac: slice_instances:id)
* GET: Show details of a concrete Network Slice Instance
* DELETE: Delete the Network Slice Instance Record, if it is not instantiated

'''/nsilcm/v1/netslice_instances/<SliceInstanceId>/instantiate'''. (rbac: slice_instances:id:instantiate:post)
* POST: (Asynchronous). Deploys a Network Slice Instance that it is in NOT_INSTANTIATED state. It returns the nsiLcmOpOccId (Operations Occurence Id) in the response header 'Location'. Example of request content: Same content as section [[#NSILCM_Details|NSILCM Details /nsilcm/v1/netslice_instances_content]]

'''/nsilcm/v1/netslice_instances/<SliceInstanceId>/terminate'''. (rbac: slice_instances:id:terminate:post)
* POST: (Asynchronous). Undeploys a Network Slice Instance that it is in INSTANTIATED state. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'

'''/nsilcm/v1/netslice_instances/nsi_lcm_op_occ'''. (rbac: slice_instances:opps)
* GET: Obtain the list of Network Slice Instances operations. It may be useful filtering by a concrete NSI with '?netsliceInstanceId=<netsliceInstanceId>'

'''/nsilcm/v1/netslice_instances/nsi_lcm_op_occ/<nsiLcmOpOccId>'''. (rbac: slice_instances:opps:id)
* GET: Show details of a concrete Network Slice Instance operation

Common issues and troubleshooting

2019-09-18T14:08:11Z

Tierno: /* Instantiation Errors */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

juju_pubkey=$(cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub)
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with ''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" ''

===NBI Errors===
====can't find a default project for this user====
* '''ERROR''': NBI always return "UNAUTHORIZED" "can't find a default project for this user"
* '''CAUSE''': Normally after a migration to release 6.0.2 There is a slight incompatibility with users creared from older versions
* '''SOLUTION''': Delete user admin and reboot NBI so that a new compatible user is created. Run these commands: "curl --insecure https://localhost:9999/osm/test/db-clear/users"; "docker service update osm_nbi --force";

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

Common issues and troubleshooting

2019-09-10T14:12:43Z

Tierno: /* Manual recovery of Juju */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

juju_pubkey=$(cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub)
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_PUBKEY.*$/OSMLCM_VCA_PUBKEY='$juju_pubkey'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with ''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" ''

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

NBI API Description

2019-08-29T13:47:29Z

Tierno:

__TOC__

== Introduction ==
OSM North Bound Interface is a REST-full following ETSI SOL005 standard. It admits both YAML/JSON formats.

By default it serves https (auto-signed certificate) on port 9999. Bearer authentication (with token) is used. Basic authentication or no authentication is also possible changing 'nbi.cfg' file.

For developping porpouses it admits web browser navigation using a basic http format.

== Request headers ==
'''Accept''': 'application/json','application/yaml' in most of the cases. Use 'text/html' for web navigation. Use 'text/plain','application/zip' for descriptor artifacts showing. This header field shall be present if the response is expected to have a non-empty message body.

'''Content-Type''': 'application/json','application/yaml' for most of the cases. For packages uploading use 'application/gzip','application/zip','application/x-gzip','application/binary'. For descriptor artifacts it can be used 'text/plain'. This header field shall be present if the request has a non-empty message body.

'''Authorization''': 'Bearer <token_id>'. The authorization token for the request. It is mandatory except when a token is requested.

'''Content-Range''': '<first byte>-<last byte>0/<total>'. It allows uploading a descriptor by chunks.

'''Transaction-Id''': '<id>'. When uploaded by chunk, the id returned by first chunk

'''Content-File-MD5''': '<md5>'. MD5 of uploaded package.

'''Content-Filename''': '<filename>'. Filename of the uploaded package.

'''Query-String-Format''': '<yaml>'. Parses query string values using yaml format.

== Response headers ==
'''Content-Type''': 'application/json','application/yaml' is the more normal. 'text/html' when web navigation. 'application/gzip','text/plain' for descriptor artifacts

'''Location''': '/<main_topic>/v1/<topic>/<id>'. Always that something is created, this header is present containing URL to this element.

'''WWW-Authenticate''': 'Bearer realm="{}"'. When Authentication error is obtained.

'''Retry-After''': '<time>'. (Future releases)

== Query string ==

Query string has the format: '?key=value&key2=value2&...'.

Some special keys are:
* '''FORMAT=True''' Forces and operation. It disables descriptor checkings, dependency cheking. For delete, it allows deleting things that cannot done in normal operation because dependency. But beware that forcing deletion will not deletes created things at VIM level.

For GET method it allows filtering with this format, follows SOL005 section 4.3.2:
query string := ?<simpleFilterExpr>["&"<simpleFilterExpr>]*
simpleFilterExpr := <attrName>["."<attrName>]*["."<op>]"="<value>[","<value>]*
op := "eq" | "neq" (or "ne") | "gt" | "lt" | "gte" | "lte" | "cont" | "ncont"
equal (by default), not equal, greater than, less than, greater than or equal, less than or equal, contains, not contains
attrName := string. For filtering inside array, use the index element of the array, or add ANYINDEX to apply the filtering over the same array element

For PUT/POST/PATCH It allows override e.g. package content, with the values of this query string

== URL and Methods summary ==

North Bound Interface (O: OSM specific; X,5: SOL005 not OSM implemented yet; O5: OSM and SOL005)

<pre>
URL: /osm GET POST PUT DELETE PATCH
/nsd/v1
/ns_descriptors_content O O
/<nsdInfoId> O O O O
/ns_descriptors O5 O5
/<nsdInfoId> O5 O5 5
/nsd_content O5 O5
/nsd O
/artifacts[/<artifactPath>] O
/pnf_descriptors 5 5
/<pnfdInfoId> 5 5 5
/pnfd_content 5 5

/vnfpkgm/v1
/vnf_packages_content O O
/<vnfPkgId> O O
/vnf_packages O5 O5
/<vnfPkgId> O5 O5 5
/package_content O5 O5
/upload_from_uri X
/vnfd O5
/artifacts[/<artifactPath>] O5

/nslcm/v1
/ns_instances_content O O
/<nsInstanceId> O O
/ns_instances 5 5
/<nsInstanceId> O5 O5
instantiate O5
terminate O5
action O
scale O5
heal 5
/ns_lcm_op_occs 5 5
/<nsLcmOpOccId> 5 5 5
TO BE COMPLETED 5 5
/vnf_instances (also vnfrs for compatibility) O
/<vnfInstanceId> O

/pdu/v1
/pdu_descriptor O O
/<id> O O O O

/admin/v1
/tokens O O
/<id> O O
/users O O
/<id> O O O O
/projects O O
/<id> O O
/vim_accounts (also vims for compatibility) O O
/<id> O O O
/wim_accounts O O
/<id> O O O
/sdns O O
/<id> O O O

/nst/v1 O O
/netslice_templates_content O O
/<nstInfoId> O O O O
/netslice_templates O O
/<nstInfoId> O O O
/nst_content O O
/nst O
/artifacts[/<artifactPath>] O

/nsilcm/v1
/netslice_instances_content O O
/<SliceInstanceId> O O
/netslice_instances O O
/<SliceInstanceId> O O
instantiate O
terminate O
action O
/nsi_lcm_op_occs O O
/<nsiLcmOpOccId> O O O
</pre>

== Administrative Content Details ==

'''/admin/v1/tokens'''. Token is a temporal authorization for a user in a concrete project. Each request must contain it in the http header Authorization
* GET: List the valid granted tokens obtained by this user
* POST: Obtain a token. The 'username' and 'password' is mandatory only if http header does not contain Authorization. 'project' can be provided, if not, first project of available for this user is assigned. For changing project, a new token must be obtained.
** Example of request content: '{username: <user>, password: <pass>, project_id: <project>}'

'''/admin/v1/tokens/<id>'''
* GET: show token details
* DELETE: revoque token. Logout

'''/admin/v1/users'''. Users may operate in several projects
* GET: Get users
* POST: Add a new user.
** Example of request content: '{username: <user>, password: <pass>, projects: [P1, P2, ...]}'

'''/admin/v1/users/<id>'''
* GET: Get user details
* PUT/PATCH: Edit user.
** Example of user edition:
*** Change password: '{password: <new_pass>}'
*** Change the list of available projects to a new one: '{projects: [P1, P2, ...]}'
*** Remove a project 'P' from user if exist: {projects: {"$'P'": null}}'
*** Add a project 'P' to user if not already exist: '{projects: {"$+'P'": "P"}}'
*** Set 'P' to be project by default, (adding it if not exist): '{projects: {"$'P'": null, "$+[0]": "P"}}'
* DELETE: Removes user.

'''/admin/v1/projects'''. A project contains its own separate space
* GET: Get projects
* POST: Add a new project.
** Example of request content: '{name: <P-name>, admin: <True/False>}'

'''/admin/v1/projects/<id>'''
* GET: Get project details
* PUT/PATCH: Edit project.
** Example of project edition to change admin status: '{admin: <True/False>}'
* DELETE: Removes project.

'''/admin/v1/vim_accounts'''. Vim account is the deployment target
* GET: Get vim_accounts
* POST: Add a new vim_accounts.
** Example of request content: TODO

'''/admin/v1/vim_accounts/<id>'''
* GET: Get vim_account details
* PUT/PATCH: Edit vim_account.
** Example of vim_account edition: TODO
* DELETE: Removes vim_account.

'''/admin/v1/wim_accounts'''. WIM are WAN Infrastructure Managers, that is in charge of inter VIM connectivity managemnt
* GET: Get wim_accounts
* POST: Add a new wim_account.
** Example of request content: TODO

'''/admin/v1/wim_accounts/<id>'''
* GET: Get wim_account details
* PUT/PATCH: Edit wim_account.
** Example of wim_account edition: TODO
* DELETE: Removes wim_account.

'''/admin/v1/sdns'''. SDN controllers in charge of VIM underlayed connectivity
* GET: Get sdns
* POST: Add a new sdn.
** Example of request content: TODO

'''/admin/v1/sdns/<id>'''
* GET: Get sdn details
* PUT/PATCH: Edit sdn.
** Example of sdn edition: TODO
* DELETE: Removes sdn.

== PDU Details ==
Physical Data Unit. This is catalogue of physical elements present at some vim_account/s. Deployment of a PNFD/HNFD implies using one of this pdu from the catalogue. Matching from PNFD/HNFD to PDU is done by 'pdu.type'

'''/pdu/v1/pdu_descriptor'''
* GET: List PDUs. Use query string for filtering.
* POST: Create a new PDU.
** Example of request content:
<pre>
name: descriptive name # mandatory
type: str # mandatory. maps VNFD.vdu.pdu-type
description: str
vim_accounts: [vim-account-id, ...] # mandatory
interfaces: # mandatory
- name: str # mandatory. Maps VNFD.vdu.interfaces.name
mgmt: boolean # mandatory. Indicates if is the management one
ip-address: ip # mandatory
vim-network-id: xxx
vim-network-name: xxx
</pre>

'''/pdu/v1/pdu_descriptor/<id>'''
* GET: show PDU details
* PUT: overwrite this PDU
* PATCH: edit this PDU
* DELETE: remove the PDU

== VNFD Details ==
Virtual Network Function Descriptor. This includes also Physical and Hybrid Network Funcitons Descriptors (PNFD/HNFD). (At VDU level, it is indicated if it is physical with the 'pdu-type' information element).

TBC

== NSD Details ==
Network Services Descriptor

TBC

== NSLCM Details ==
Network Services Life Ciclo Management

'''/nslcm/v1/ns_instances_content'''
* GET: Obtains a list of NS
* POST: (Asynchronous). Creates and Instantiate a NS. It returns the nsInstanceId in the response header 'Location'. Example of request content:
<pre>
nsName: name # mandatory
nsdId: nsd-internal-id # mandatory
vimAccountId: internal-id # mandatory
wimAccountId: False/True/internal-id
additionalParamsForNs: {param1: value1, param2: value2}
additionalParamsForVnf: [member-vnf-index: "1", additionalParams: {param1: value1, param2: value2}]
ssh_keys: [ssh-key-pub1, ssh-key-pub2, ]
vnf: [ {member-vnf-index: "1", vimAccountId: vim-uuid, internal-vld: [], vdu: [] } ]
vld: [ {name: vld-name, ip-profile: {}, vnfd-connection-point-ref: [{}] }]
</pre>

'''/nslcm/v1/ns_instances_content/<nsInstanceId>'''
* GET: Obtains details of a NS
* DELETE: (Asynchronous) Terminates (or undeploy) and deletes (if termination is done successfully) a NS

'''/nslcm/v1/ns_instances'''
* GET: Obtains a list of NS
* POST: Creates a NS, but do not instantitate it. It returns the nsInstanceId in the response header 'Location'. Example of request content:
<pre>
nsName: name # mandatory
nsdId: nsd-internal-id # mandatory
vimAccountId: internal-id # mandatory
wimAccountId: False/True/internal-id
additionalParamsForNs: {param1: value1, param2: value2}
additionalParamsForVnf: [member-vnf-index: "1", additionalParams: {param1: value1, param2: value2}]
</pre>

'''/nslcm/v1/ns_instances/<nsInstanceId>'''
* GET: Obtains details of a NS
* DELETE: Deletes a NS. It fails if NS is in 'INSTANTIATED' state

'''/nslcm/v1/ns_instances/<nsInstanceId>/instantiate'''
* POST: (Asynchronous). Deploys a NS that it is in NOT_INSTANTIATED state. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'. Example of request content:
<pre>
vimAccountId: internal-id # mandatory
wimAccountId: False/True/internal-id
ssh_keys: [ssh-key-pub1, ssh-key-pub2, ]
vnf: [ {member-vnf-index: "1", vimAccountId: vim-uuid, internal-vld: [], vdu: [] } ]
vld: [ {name: vld-name, ip-profile: {}, vnfd-connection-point-ref: [{}] }]
</pre>

'''/nslcm/v1/ns_instances/<nsInstanceId>/terminate'''
* POST: (Asynchronous). Undeploys a NS that it is in INSTANTIATED state. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'

'''/nslcm/v1/ns_instances/<nsInstanceId>/action'''
* POST: (Asynchronous). Performs a Day-2 operation over the NS. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'
** Example of content to executing a primitive:
<pre>
member_vnf_index: str # Needed for VNFD mapping NSD.constituent-vnfd.member-vnf-index. If missing a NSD is targeting
vdu_id: str # Needed if target is a VDU. Maps VNFD.vdu-id
vdu_count_index: int # Needed if the VDU has several replicas (count>1)
primitive: str # Mandatory. Maps [NSD.ns-configuration or VNFD.vnf-configuration or VNFD.vdu.vdu-configuration].config-primitive.name
primitive_params: dict # Optional. Maps [NSD.ns-configuration or VNFD.vnf-configuration or VNFD.vdu.vdu-configuration].config-primitive.parameter
param: value
...
</pre>
'''/nslcm/v1/ns_instances/<nsInstanceId>/scale'''
* POST: (Asynchronous). Performs a manual scale OUT/IN over one of the VNFs of the NS. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'
** Example of content: '{scaleType: SCALE_VNF, scaleVnfData: {scaleVnfType: SCALE_OUT, scaleByStepData: {member-vnf-index: <nsd:constituent-vnfd:member-vnf-index>, scaling-group-descriptor: <vnfd:scaling-group-descriptor:name>} } }' # Use SCALE_IN instead of SCALE OUT depending of desired type.

'''/nslcm/v1/ns_instances/nsi_lcm_op_occs'''
* GET: Obtains the list of Operations occurrences. It may be useful filtering by a concrete NS with '?nsInstanceId=<nsInstaceId>'

'''/nslcm/v1/ns_instances/nsi_lcm_op_occs/<nsiLcmOpOccId>'''
* GET: Show details of an Operations occurrences.

'''/nslcm/v1/vnf_instances''' (also vnfrs for compatibility):
* GET: Obtains the list of VNF. It may be useful filtering by a concrete NS with '?nsr-id-ref=<nsInstaceId>'

'''/nslcm/v1//vnf_instances/<vnfInstanceId>'''
* GET: Show details of a concrete VNF

== NST Details ==
NetSlice Template descriptor

'''/nst/v1/netslice_templates_content'''
* GET: Obtain list of Network Slice Templates
* POST: Create a Network Slice Template and upload the content

'''/nst/v1/netslice_templates_content/<nstInfoId>'''
* GET: Show the detail of Network Slice Template
* PUT: Update the whole Network Slice Template
* DELETE: Remove a Network Slice template. The Network Slice template is removed only if state is NOT_IN_USE.

'''/nst/v1/netslice_templates'''
* GET: Obtain list of Network Slice Templates
* POST: Create an empty Network Slice Template. A PUT should be done afterwards to upload the package

'''/nst/v1/netslice_templates/<nstInfoId>'''
* GET: Show the detail of Network Slice Template descriptor used by OSM
* DELETE: Remove a Network Slice Template. The Network Slice template is removed only if state is NOT_IN_USE.
* PATCH: Update a Network Slice Template field

'''/nst/v1/netslice_templates/<nstInfoId>/nst_content'''
* GET: Show the detail of the source Network Slice Template package
* PUT: Update a Network Slice Template

'''/nst/v1/netslice_templates/<nstInfoId>/nst'''
* GET: Show the detail of the source Network Slice Template descriptor

'''/nst/v1/netslice_templates/<nstInfoId>/artifacts[/<artifactPath>]'''
* GET: Obtain Network Slice Templates Artifacts. This is useful for download the Network Slice Templates

== NSILCM Details ==
NetSlice Instance Life Cicle Management

'''/nsilcm/v1/netslice_instances_content'''
* GET: Obtain the list of Network Slice Instances
* POST: (Asynchronous). Creates and Instantiate a Network Slice Instance. It returns the netsliceInstanceId in the response header 'Location'. Example of request content:

<pre>
nstId: name of the Network Slice Template #mandatory
nsiName: name of the Network Slice Instance # mandatory
vimAccountId: internal-id # mandatory
ssh_keys: comma separated list of keys to inject to vnfs
nsiDescription: description of the Network Slice Instance
additionalParamsForNsi: {param: value, ...}
netslice-subnet: [ Same content as section #NSLCM_Details /nslcm/v1/ns_instances_content
],
netslice-vld: [
name: TEXT,
vim-network-name: TEXT or DICT with the name for each vim account: {vimAccountId: network-name, ...},
vim-network-id: TEXT or DICT with the id for each vim account {vimAccountId: network-id},
ip-profile: Profile of the vld
]
</pre>

'''/nsilcm/v1/netslice_instances_content/<SliceInstanceId>'''
* GET: Show details of a concrete Network Slice Instance
* DELETE: (Asynchronous) Terminates (or undeploy) and deletes (if termination is done successfully) a Network Slice Instance Record

'''/nsilcm/v1/netslice_instances'''
* GET: Obtain the list of Network Slice Instances
* POST: Create the Network Slice Instance but do not instantitate it. It returns the netsliceInstanceId in the response header 'Location'. Example of request content: Same content as section [[#NSILCM_Details|NSILCM Details /nsilcm/v1/netslice_instances_content]]

'''/nsilcm/v1/netslice_instances/<SliceInstanceId>'''
* GET: Show details of a concrete Network Slice Instance
* DELETE: Delete the Network Slice Instance Record, if it is not instantiated

'''/nsilcm/v1/netslice_instances/<SliceInstanceId>/instantiate'''
* POST: (Asynchronous). Deploys a Network Slice Instance that it is in NOT_INSTANTIATED state. It returns the nsiLcmOpOccId (Operations Occurence Id) in the response header 'Location'. Example of request content: Same content as section [[#NSILCM_Details|NSILCM Details /nsilcm/v1/netslice_instances_content]]

'''/nsilcm/v1/netslice_instances/<SliceInstanceId>/terminate'''
* POST: (Asynchronous). Undeploys a Network Slice Instance that it is in INSTANTIATED state. It returns the nsLcmOpOccId (Operationts Occurence Id) in the response header 'Location'

'''/nsilcm/v1/netslice_instances/nsi_lcm_op_occ'''
* GET: Obtain the list of Network Slice Instances operations. It may be useful filtering by a concrete NSI with '?netsliceInstanceId=<netsliceInstanceId>'

'''/nsilcm/v1/netslice_instances/nsi_lcm_op_occ/<nsiLcmOpOccId>'''
* GET: Show details of a concrete Network Slice Instance operation

Common issues and troubleshooting

2019-08-22T09:35:15Z

Tierno: /* File juju_id_rsa.pub not found */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with ''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" ''

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

Common issues and troubleshooting

2019-08-22T09:34:18Z

Tierno: /* Slow deployment of charms */

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

=== Add User in Group ===
Add the non-root user used for installation in ''sudo , lxd, docker'' groups

This will skip below error :-

''Finished installation of juju
Password:
'''sg: failed to crypt password with previous salt: Invalid argument
ERROR No controllers registered.'''''

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

=====TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'=====

In some cases, a MTU mismatch between the host and docker interfaces will cause this error while running pip. You can check this by running `ifconfig` and comparing the MTU of your host interface and the docker_gwbridge interface.

# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the MTU value with that of your host interface from the previous step:
{
"mtu": 1458
}

# Restart docker
sudo service docker restart

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

===Instantiation Errors===
====File juju_id_rsa.pub not found====
* '''ERROR''': ERROR creating VCA model name 'xxxx': Traceback (most recent call last): File "/usr/lib/python3/dist-packages/osm_lcm/ns.py", line 822, in instantiate await ... [Errno 2] No such file or directory: '/root/.local/share/juju/ssh/juju_id_rsa.pub'
* '''CAUSE''': Normally a migration from release FIVE do not set properly the env for LCM
* '''SOLUTION''': Ensure variable '''OSMLCM_VCA_PUBKEY''' is properly set at file ''/etc/osm/docker/lcm.env''. The value must mutch with the output of this command ''cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub''. If not, add or change it. Restart OSM, or just LCM service with '''docker service update osm_lcm --force --env-add OSMLCM_VCA_PUBKEY="<value>" '''

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

For each container, logs can be found under:
/var/lib/docker/containers/DOCKER_ID/DOCKER_ID-json.log

And the DOCKER_ID can be obtained this way, e.g. for MON
docker ps -aqf "name=osm_mon" -n 1 --no-trunc

Developer HowTo

2019-08-20T15:25:32Z

Tierno: /* NBI */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). Current installation uses Docker Swarm by default (there are Kubernetes distributions and an incoming option for installing on Kubernetes). The modules are:

* '''kafka''': Provides a Kafka bus used for OSM communication. This module relies on ''zookeeper''.
* '''zookeeper''': Used by ''kafka''.
* '''nbi''': North Bound Interface of OSM. Restful server that follows ETSI SOL005 interface. Relies on ''mongo'' database and ''kafka'' bus. For authentication it can optionally uses ''keystone''.
* '''keystone''': Used optionally for NBI authentication and RBAC. It stores the users, projects and role permissions. It relies on ''mysql''.
* '''lcm''': Provides the Live Cycle Management. It uses ''ro'' for resource orchestration and juju for configuration. It relies also on ''mongo''.
* '''ro''': Makes the Resource Orchestration, or VIM deployment. Relies on ''mysql''.
* '''light-ui''': Web user interface. It communicates with ''nbi''.
* '''mon''': Performs OSM monitoring. Relies on ''mongo'' and ''mysql''.
* '''mongo''': Common non relational database for OSM modules.
* '''mysql''': Relational database server used for ''ro'', ''keystone'', ''mon'' and ''pol''.
* '''pol''': Policy Manager for OSM.
* '''prometheus''': for monitoring .

In addition LCM and NBI, shares a common file system where packages are stored. In current installation it uses a shared docker volume called ''osm_package''.

This picture shows the modules with the port that they export, their relationships with the name of env variables that control them. For clarity not all the dependencies are depicted, as kafka, mysql, ...

<pre>
__________ ________
| | | |
| light-ui |OSM_SERVER _______ |keystone|
| :80 |----------------------> | |-------------------------> |:5000 |
|__________| | nbi | |________|
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | mysql |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

For debugging it is convenient running the target module directly at host maintaining the rest of dependent modules in containers. In the following sections it is described how to achieve that.

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

==NBI==

Install needed OSM packages IM, common:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
# pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Additionally you may want that docker 'light-ui' uses your local copy of NBI by one of:
* Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:
OSM_SERVER: <172.17.0.1> # nbi
# Get the needed address at your setup by 'ip a | grep docker0'
* ... Or update light-ui docker service (not persistent on reboot):
docker service update osm_light-ui --force --env-add "OSM_SERVER=172.17.0.1"
# Get the needed address at your setup by 'ip a | grep docker0'

Clone and install needed NBI packages
docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/nbi.env'
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:

Values are stored at '/etc/osm/docker/lcm.env' but here it is explained how to obtain these values.
* OSMLCM_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/lcm.env'
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_CACERT: To get this value run the following command in the OSM host:
** juju controllers --format json | jq -r '.controllers["osm"]["ca-cert"]'
* OSMLCM_VCA_PUBKEY: To get this value run the following command in the OSM host:
** cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-08-20T10:34:40Z

Tierno: /* Introduction */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). Current installation uses Docker Swarm by default (there are Kubernetes distributions and an incoming option for installing on Kubernetes). The modules are:

* '''kafka''': Provides a Kafka bus used for OSM communication. This module relies on ''zookeeper''.
* '''zookeeper''': Used by ''kafka''.
* '''nbi''': North Bound Interface of OSM. Restful server that follows ETSI SOL005 interface. Relies on ''mongo'' database and ''kafka'' bus. For authentication it can optionally uses ''keystone''.
* '''keystone''': Used optionally for NBI authentication and RBAC. It stores the users, projects and role permissions. It relies on ''mysql''.
* '''lcm''': Provides the Live Cycle Management. It uses ''ro'' for resource orchestration and juju for configuration. It relies also on ''mongo''.
* '''ro''': Makes the Resource Orchestration, or VIM deployment. Relies on ''mysql''.
* '''light-ui''': Web user interface. It communicates with ''nbi''.
* '''mon''': Performs OSM monitoring. Relies on ''mongo'' and ''mysql''.
* '''mongo''': Common non relational database for OSM modules.
* '''mysql''': Relational database server used for ''ro'', ''keystone'', ''mon'' and ''pol''.
* '''pol''': Policy Manager for OSM.
* '''prometheus''': for monitoring .

In addition LCM and NBI, shares a common file system where packages are stored. In current installation it uses a shared docker volume called ''osm_package''.

This picture shows the modules with the port that they export, their relationships with the name of env variables that control them. For clarity not all the dependencies are depicted, as kafka, mysql, ...

<pre>
__________ ________
| | | |
| light-ui |OSM_SERVER _______ |keystone|
| :80 |----------------------> | |-------------------------> |:5000 |
|__________| | nbi | |________|
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | mysql |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

For debugging it is convenient running the target module directly at host maintaining the rest of dependent modules in containers. In the following sections it is described how to achieve that.

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

==NBI==

Install needed OSM packages IM, common:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
# pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Additionally you may want that docker 'light-ui' uses your local copy of NBI. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:

OSM_SERVER: <172.17.0.1> # nbi
# Get the needed address at your setup by 'ip a | grep docker0'

Clone and install needed NBI packages
docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/nbi.env'
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:

Values are stored at '/etc/osm/docker/lcm.env' but here it is explained how to obtain these values.
* OSMLCM_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/lcm.env'
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_CACERT: To get this value run the following command in the OSM host:
** juju controllers --format json | jq -r '.controllers["osm"]["ca-cert"]'
* OSMLCM_VCA_PUBKEY: To get this value run the following command in the OSM host:
** cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Common issues and troubleshooting

2019-06-18T09:06:33Z

Tierno:

__TOC__

== Installation==

'''RECOMMENDATION:''' save a log of your installation:
$ ./install_osm.sh 2>&1 | tee osm_install_log.txt

===Docker===
====Were all docker images successfully built?====

Although controlled by the installer, you can check that the following images exist:

$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
osm/light-ui latest 1988aa262a97 18 hours ago 710MB
osm/lcm latest c9ad59bf96aa 46 hours ago 667MB
osm/ro latest 812c987fcb16 46 hours ago 791MB
osm/nbi latest 584b4e0084a7 46 hours ago 497MB
osm/pm latest 1ad1e4099f52 46 hours ago 462MB
osm/mon latest b17efa3412e3 46 hours ago 725MB
wurstmeister/kafka latest 7cfc4e57966c 10 days ago 293MB
mysql 5 0d16d0a97dd1 2 weeks ago 372MB
mongo latest 14c497d5c758 3 weeks ago 366MB
wurstmeister/zookeeper latest 351aa00d2fe9 18 months ago 478MB

====Are all processes/services running?====
$ docker stack ps osm |grep -i running

10 docker containers should be running.

All the 10 services should have at least 1 replica: 1/1

$ docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
yuyiqh8ty8pv osm_kafka replicated 1/1 wurstmeister/kafka:latest *:9092->9092/tcp
y585906h5vy5 osm_lcm replicated 1/1 osm/lcm:latest
pcdi5vb86nt9 osm_light-ui replicated 1/1 osm/light-ui:latest *:80->80/tcp
i56jhl5k6re4 osm_mon replicated 1/1 osm/mon:latest *:8662->8662/tcp
p5wyjtne93hp osm_mongo replicated 1/1 mongo:latest
iz5uncfdzu23 osm_nbi replicated 1/1 osm/nbi:latest *:9999->9999/tcp
4ttw2v4z2g57 osm_pm replicated 1/1 osm/pm:latest
xbg6bclp2anw osm_ro replicated 1/1 osm/ro:latest *:9090->9090/tcp
sf7rayfolncu osm_ro-db replicated 1/1 mysql:5
5bl73dhj1xl0 osm_zookeeper replicated 1/1 wurstmeister/zookeeper:latest

====Docker image failed to build====
=====Err:1 http://archive.ubuntu.com/ubuntu xenial InRelease=====

In some cases, DNS resolution works on the host but fails when building the Docker container. This is caused when Docker doesn't automatically determine the DNS server to use.

Check if the following works:
docker run busybox nslookup archive.ubuntu.com

If it does not work, you have to configure Docker to use the available DNS.
# Get the IP address you’re using for DNS:
nmcli dev show | grep 'IP4.DNS'
# Create a new file, /etc/docker/daemon.json, that contains the following (but replace the DNS IP address with the output from the previous step:
{
"dns": ["192.168.24.10"]
}
# Restart docker
sudo service docker restart
# Re-run
docker run busybox nslookup archive.ubuntu.com
# Now you should be able to re-run the installer and move past the DNS issue.

====Problem deploying stack osm====
=====network netosm could not be found=====

The error is ''network "netosm" is declared as external, but could not be found. You need to create a swarm-scoped network before the stack is deployed''

It usually happens because a "docker system prune" is done with the stack stopped. The following script will create it:

#!/bin/bash
# Create OSM Docker Network ...
[ -z "$OSM_STACK_NAME" ] && OSM_STACK_NAME=osm
OSM_NETWORK_NAME=net${OSM_STACK_NAME}
echo Creating OSM Docker Network
DEFAULT_INTERFACE=$(route -n | awk '$1~/^0.0.0.0/ {print $8}')
DEFAULT_MTU=$(ip addr show $DEFAULT_INTERFACE | perl -ne 'if (/mtu\s(\d+)/) {print $1;}')
echo \# OSM_STACK_NAME = $OSM_STACK_NAME
echo \# OSM_NETWORK_NAME = $OSM_NETWORK_NAME
echo \# DEFAULT_INTERFACE = $DEFAULT_INTERFACE
echo \# DEFAULT_MTU = $DEFAULT_MTU
sg docker -c "docker network create --driver=overlay --attachable \
--opt com.docker.network.driver.mtu=${DEFAULT_MTU} \
${OSM_NETWORK_NAME}"

===Juju===
====Bootstrap hangs====

If the Juju bootstrap takes a long time, stuck at this status...

Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.14.0
Waiting for address
Attempting to connect to 10.71.22.78:22
Connected to 10.71.22.78
Running machine configuration script...

...it usually indicates that the LXD container with the Juju controller is having trouble connecting to the internet.

Get the name of the LXD container. It will begin with 'juju-' and end with '-0'.

lxc list
+-----------------+---------+---------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| juju-0383f2-0 | RUNNING | 10.195.8.57 (eth0) | | PERSISTENT | |
+-----------------+---------+---------------------+------+------------+-----------+

Next, tail the output of cloud-init to see where the bootstrap is stuck.

lxc exec juju-0383f2-0 -- tail -f /var/log/cloud-init-output.log

====Is Juju running?====

If running, you should see something like this:

$ juju status

Model Controller Cloud/Region Version SLA
default osm localhost/localhost 2.3.7 unsupported

====ERROR controller osm already exists====

Did OSM installation fail during juju installation with an error like "ERROR controller osm already exists" ?

$ ./install_osm.sh
...
ERROR controller "osm" already exists
ERROR try was stopped

### Jum Agu 24 15:19:33 WIB 2018 install_juju: FATAL error: Juju installation failed
BACKTRACE:
### FATAL /usr/share/osm-devops/jenkins/common/logging 39
### install_juju /usr/share/osm-devops/installers/full_install_osm.sh 564
### install_lightweight /usr/share/osm-devops/installers/full_install_osm.sh 741
### main /usr/share/osm-devops/installers/full_install_osm.sh 1033

Try to destroy the Juju controller and run the installation again:
$ juju destroy-controller osm --destroy-all-models -y
$ ./install_osm.sh

If it does not work, you can destroy Juju container and run the installation again
#Destroy the Juju container
lxc stop juju-*
lxc delete juju-*
#Unregister the controller since we’ve manually freed the resources associated with it
juju unregister -y osm
#Verify that there are no controllers
juju list-controllers
#Run the installation again
./install_osm.sh

===LXD===
====ERROR profile default: /etc/default/lxd-bridge has IPv6 enabled====

Make sure that you follow the instructions in the Quickstart.

When asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements, Please answer "y".

When dialog messages related to LXD configuration are shown, please answer in the following way:

* Do you want to configure the LXD bridge? Yes
* Do you want to setup an IPv4 subnet? Yes
* << Default values apply for next questions >>
* '''Do you want to setup an IPv6 subnet? No'''

==Configuration==

===VIMs===

====Is the VIM URL reachable and operational?====

When there are problems to access the VIM URL, an error message similar to the following is shown after attempts to instantiate network services:

Error: "VIM Exception vimmconnConnectionException ConnectFailure: Unable to establish connection to <URL>"

* In order to debug potential issues with the connection, in the case of an OpenStack VIM, you can install the OpenStack client in the OSM VM and run some basic tests. I.e.:
$ # Install the OpenStack client
$ sudo apt-get install python-openstackclient
$ # Load your OpenStack credentials. For instance, if your credentials are saved in a file named 'myVIM-openrc.sh', you can load them with:
$ source myVIM-openrc.sh
$ # Test if the VIM API is operational with a simple command. For instance:
$ openstack image list

If the openstack client works, then make sure that you can reach the VIM from the RO docker:
$ docker exec -it osm_ro.1.xxxxx bash
$ curl <URL_CONTROLLER>

''In some cases, the errors come from the fact that the VIM was added to OSM using names in the URL that are not Fully Qualified Domain Names (FQDN).''

When adding a VIM to OSM, you must use always FQDN or the IP addresses. It must be noted that “controller” or similar names are not proper FQDN (the suffix should be added). Non-FQDN names might be understood by docker’s dnsmasq as a docker container name to be resolved, which is not the case. In addition, all the VIM endpoints should also be FQDN or IP addresses, thus guaranteeing that all subsequent API calls can reach the appropriate endpoint.

Think of an NFV infrastructure with tens of VIMs, first you will have to use different names for each controller (controller1, controller2, etc.), then you will have to add to every machine trying to interact with the different VIMs, not only OSM, all those entries in the /etc/hosts file. This is bad practice.

However, it is useful to have a mean to work with lab environments using non-FQDN names. Three options here. Probably you are looking for the third one, but we recommend the first one:
* Option 1. Change the admin URL and/or public URL of the endpoints to use an IP address or an FQDN. You might find this interesting if you want to bring your Openstack setup to production.
* Option 2. Modify /etc/hosts in the docker RO container. This is not persistent after reboots or restarts of the osm docker stack.
* Option 3. Modify /etc/osm/docker/docker-compose.yaml in the host, adding extra_hosts in the ro section with the entries that you want to add to /etc/hosts in the RO docker:
ro:
extra_hosts:
controller: 1.2.3.4
Then restart the stack:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
This is persistent after reboots and restarts of the osm docker stack.

====Authentication====
'''What should I check if the VIM authentication is failing?'''

Typically, you will get the following error messsage:

Error: "VIM Exception vimconnUnexpectedResponse Unauthorized: The request you have made requieres authentication. (HTTP 401)"

If your OpenStack URL is based on HTTPS, OSM will check by default the authenticity of your VIM using the appropriate public certificate. The recommended way to solve this is by modifying /etc/osm/docker/docker-compose.yaml in the host, sharing the host file (e.g. /home/ubuntu/cafile.crt) by adding a volume to the ro section as follows:
ro:
...
volumes:
- /home/ubuntu/cafile.crt:/etc/osm/cafile.crt
Then, when creating the VIM, you should use the config option "ca_cert" as follows:
$ # Create the VIM with all the usual options, and add the config option to specify the certificate
$ osm vim-create VIM-NAME ... --config '{ca_cert: /etc/osm/cafile.crt}'

For casual testing, when adding the VIM account to OSM, you can use 'insecure: True' (without quotes) as part of the VIM config parameters:
$ osm vim-create VIM-NAME ... --config '{insecure: True}'

'''Is the VIM management network reachable from OSM (e.g. via ssh, port 22)?'''

The simplest check would consist on deploying a VM attached to the management network and trying to access it via e.g. ssh from the OSM host.

For instance, in the case of an OpenStack VIM you could try something like this:
$ openstack server create --image ubuntu --flavor m1.small --nic mgmtnet test

If this does not work, typically it is due to one of these issues:
* Security group policy in your VIM is blocking your traffic (contact your admin to fix it)
* IP address space in the management network is not routable from outside (or in the reverse direction, for the ACKs).

==Operational issues==

===Running out of disk space===
If you are upgrading frequently your OSM installation, you might face that your disk is running out of space. The reason is that the previous dockers and docker images might be consuming some disk space. Running the following two commands should be enough to clear your docker setup:
docker system prune
docker image prune

If you are still experiencing issues with disk space, logs in one of the dockers could be the cause of your issue. Check the containers that are consuming more space (typically kafka-exporter)
du -sk /var/lib/docker/containers/* |sort -n
docker ps |grep <CONTAINER_ID>

Then, remove the stack and redeploy it again after doing a prune:
docker stack rm osm_metrics
docker system prune
docker image prune
docker stack deploy -c /etc/osm/docker/osm_metrics/docker-compose.yml osm_metrics

===VCA (juju)===
====Status is not coherent with running NS====

In extraordinary situations, the output of "juju status" could show pending units that should have been removed when deleting a NS. In those situations, you can clean up VCA by following the procedure below:
juju status
juju remove-application <application>
juju resolved <unit> --no-retry # You'll likely have to run it several times, as it will probably have an error in the next queued hook.Once the last hook is marked resolved, the charm will continue its removal

The following page also shows [https://docs.jujucharms.com/2.1/en/charms-destroy how to remove different Juju objects]

====Dump Juju Logs ====

To dump the Juju debug-logs, run this command:
juju debug-log --replay --no-tail > juju-debug.log

====Manual recovery of Juju====

If juju gets in a corrupt state and you cannot run `juju status` or contact the juju controller, you might need to remove manually the controller and register again, making OSM aware of the new controller.

<nowiki>
# Stop and delete all juju containers, then unregister the controller
lxc list
lxc stop juju-* #replace "*" by the right values
lxc delete juju-* #replace "*" by the right values
juju unregister -y osm

# Create the controller again
sg lxd -c "juju bootstrap --bootstrap-series=xenial localhost osm"

# Get controller IP and update it in relevant OSM env files
controller_ip=$(juju show-controller osm|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}')
sudo sed -i 's/^OSMMON_VCA_HOST.*$/OSMMON_VCA_HOST='$controller_ip'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_HOST.*$/OSMLCM_VCA_HOST='$controller_ip'/' /etc/osm/docker/lcm.env

#Get juju password and feed it to OSM env files
function parse_juju_password {
password_file="${HOME}/.local/share/juju/accounts.yaml"
local controller_name=$1
local s='[[:space:]]*' w='[a-zA-Z0-9_-]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^$$s$:|\1|" \
-e "s|^$$s$$$w$$s:$s[\"']$.*$[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^$$s$$$w$$s:$s$.*$$s\$|\1$fs\2$fs\3|p" $password_file |
awk -F$fs -v controller=$controller_name '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
if (match(vn,controller) && match($2,"password")) {
printf("%s",$3);
}
}
}'
}
juju_password=$(parse_juju_password osm)
sudo sed -i 's/^OSMMON_VCA_SECRET.*$/OSMMON_VCA_SECRET='$juju_password'/' /etc/osm/docker/mon.env
sudo sed -i 's/^OSMLCM_VCA_SECRET.*$/OSMLCM_VCA_SECRET='$juju_password'/' /etc/osm/docker/lcm.env

#Restart OSM stack
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
</nowiki>

====Slow deployment of charms====

You can make deplyment of charms quicker by:
* Upgrading your LXD installation to use ZFS: [[LXD configuration for OSM Release FIVE]]
** After LXD re-installation, you might need to reinstall the juju controller: [[Common_issues_and_troubleshooting#Manual_recovery_of_Juju|Reinstall Juju controller]]
* Preventing Juju from running apt-get update && apt-get upgrade when starting a machine: [[Advanced_Charm_Development#Disable_OS_upgrades|Disable OS upgrades in charms]]
* Building periodically a custom image that will be used as base image for all the charms: [[Advanced_Charm_Development#Build_a_custom_cloud_image|Custom base image for charms]]

==Checking the logs==

You can check the logs of any container with the following commands:

docker logs $(docker ps -aqf "name=osm_mon" -n 1)
docker logs $(docker ps -aqf "name=osm_pol" -n 1)
docker logs $(docker ps -aqf "name=osm_lcm" -n 1)
docker logs $(docker ps -aqf "name=osm_nbi" -n 1)
docker logs $(docker ps -aqf "name=osm_light-ui" -n 1)
docker logs $(docker ps -aqf "name=osm_ro.1" -n 1)
docker logs $(docker ps -aqf "name=osm_ro-db" -n 1)
docker logs $(docker ps -aqf "name=osm_mongo" -n 1)
docker logs $(docker ps -aqf "name=osm_kafka" -n 1)
docker logs $(docker ps -aqf "name=osm_zookeeper" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone.1" -n 1)
docker logs $(docker ps -aqf "name=osm_keystone-db" -n 1)
docker logs $(docker ps -aqf "name=osm_prometheus" -n 1)

NBI API Description

2019-06-03T17:31:00Z

Tierno:

Developer HowTo

2019-05-28T08:26:13Z

Tierno:

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

==NBI==

Install needed OSM packages IM, common:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
# pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Additionally you may want that docker 'light-ui' uses your local copy of NBI. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:

OSM_SERVER: <172.17.0.1> # nbi
# Get the needed address at your setup by 'ip a | grep docker0'

Clone and install needed NBI packages
docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/nbi.env'
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:

Values are stored at '/etc/osm/docker/lcm.env' but here it is explained how to obtain these values.
* OSMLCM_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/lcm.env'
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_CACERT: To get this value run the following command in the OSM host:
** juju controllers --format json | jq -r '.controllers["osm"]["ca-cert"]'
* OSMLCM_VCA_PUBKEY: To get this value run the following command in the OSM host:
** cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-05-22T13:25:45Z

Tierno:

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

==NBI==

Install needed OSM packages IM, common:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
# pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Additionally you may want that docker 'light-ui' uses your local copy of NBI. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:

OSM_SERVER: <172.17.0.1> # nbi
# Get the needed address at your setup by 'ip a | grep docker0'

Clone and install needed NBI packages
docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/nbi.env'
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/lcm.env'
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

NBI API Description

2019-04-16T11:31:23Z

Tierno:

OSM instantiation parameters

2019-04-11T10:18:45Z

Tierno:

When creating a NS instance, it is possible to pass instantiation parameters to OSM using the "--config" option of the client or the "config" parameter of the UI. In this section we will illustrate through some of the existing examples how to specify those parameters using OSM client:

===Specify a VIM network name for a NS VLD===
In a generic way, the mapping can be specified in the following way, where 'vldnet' is the name of the network in the NS descriptor and 'netVIM1' is the VIM network that you want to use:
--config '{vld: [ {name: vldnet, vim-network-name: netVIM1} ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_vnfd.tar.gz hackfest1-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_nsd.tar.gz hackfest1-ns]; images: [https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%202%20-%20Creating%20a%20basic%20VNF%20and%20NS.pdf creating a basic VNF and NS]''') in the following way:

osm ns-create --ns_name hf1 --nsd_name hackfest1-ns --vim_account openstack1 --config '{vld: [ {name: mgmtnet, vim-network-name: mgmt} ] }'

===Specify a VIM network name for an internal VLD of a VNF===
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor and 'netVIM1' is the VIM network that you want to use:
--config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, vim-network-name: netVIM1} ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, vim-network-name: mgmt} ] } ] }'

===Specify IP profile information and IP for a NS VLD===
In a generic way, the mapping can be specified in the following way, where 'datanet' is the name of the network in the NS descriptor, ip-profile you have to fill the associated parameters with the data model ( [http://osm-download.etsi.org/ftp/osm-doc/nsd.html# NS data model] ), and vnfd-connection-point-ref is the reference to the connection point:
--config '{vld: [ {name: datanet, ip-profile: {...}, vnfd-connection-point-ref: {...} } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vld: [ {name: datanet, ip-profile: {ip-version: ipv4 ,subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}],dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true}}, vnfd-connection-point-ref: [ {member-vnf-index-ref: "1", vnfd-connection-point-ref: vnf-data, ip-address: "192.168.100.17"}]}]}'

===Specify IP profile information for an internal VLD of a VNF===
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor and ip-profile you have to fill the associated parameters with the data model ( [http://osm-download.etsi.org/ftp/osm-doc/vnfd.html VNF data model] ):

--config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {...} ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:

osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {ip-version: ipv4 ,subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}] ,dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true}}}]}]} '

===Specify IP address and/or MAC address for an interface===

====Specify IP address for an interface====
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor ,ip-profile you have to fill the associated parameters with the data model ( [http://osm-download.etsi.org/ftp/osm-doc/vnfd.html VNF data model] ), 'id1' is the internal-connection-point id and 'a.b.c.d' is the IP that you have to specify for this scenario:

--config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {...}, internal-connection-point: [{id-ref: id1, ip-address: "a.b.c.d"}] ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account ost4 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {ip-version: ipv4 ,subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}] ,dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true}}, internal-connection-point: [{id-ref: mgmtVM-internal, ip-address: "192.168.100.3"}]}]}]}'

====Specify MAC address for an interface====
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'id1' is the id of VDU in the VNF descriptor and 'interf1' is the name of the interface to which you want to add the MAC address:
--config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: id1, interface: [{name: interf1, mac-address: "aa:bb:cc:dd:ee:ff" }]} ] } ] } '
You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_vnfd.tar.gz hackfest1-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_nsd.tar.gz hackfest1-ns]; images: [https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%202%20-%20Creating%20a%20basic%20VNF%20and%20NS.pdf creating a basic VNF and NS]''') in the following way:
osm ns-create --ns_name hf12 --nsd_name hackfest1-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: hackfest1VM, interface: [{name: vdu-eth0, mac-address: "52:33:44:55:66:21"}]} ] } ] } '

====Specify IP address and MAC address for an interface====
In the following scenario, we will bring together the two previous cases

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:

osm ns-create --ns_name hf12 --nsd_name hackfest2-ns --vim_account ost4 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal , ip-profile: {ip-version: ipv4, subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}] , dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true} }, internal-connection-point: [ {id-ref: mgmtVM-internal, ip-address: "192.168.100.3"} ] }, ], vdu: [ {id: mgmtVM, interface: [{name: mgmtVM-eth0, mac-address: "52:33:44:55:66:21"}]} ] } ] } '

===Force floating IP address for an interface===
In a generic way, the mapping can be specified in the following way, where 'id1' is the name of the VDU in the VNF descriptor and 'interf1' is the name of the interface:
--config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: id1, interface: [{name: interf1, floating-ip-required: True }]} ] } ] } '

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vdu:[ {id: mgmtVM, interface: [{name: mgmtVM-eth0, floating-ip-required: True }]} ] } ] } '

Make sure 'vim-network-name' in NS Descriptor has access to the outside to be able to use the parameter 'floating-ip-required'.

===Multi-site deployments (specifying different VIM accounts for different VNFs)===
In this scenario, the mapping can be specified in the following way, where '1 and 2' are the member vnf index of the constituent vnfs in the NS descriptor, 'vim1 and vim2' are the names of vim accounts and 'netVIM1 and netVIM2' are the VIM networks that you want to use:

--config '{vnf: [ {member-vnf-index: "1", vim_account: vim1}, {member-vnf-index: "2", vim_account: vim2} ], vld: [ {name: datanet, vim-network-name: {vim1: netVIM1, vim2: netVIM2} } ] }'
# NOTE: From release SIX (current muster) add 'wim_account: False' (inside --config) to avoid wim network connectivity if you have not a WIM in your system

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf12 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vim_account: openstack1}, {member-vnf-index: "2", vim_account: openstack3} ], vld: [ {name: mgmtnet, vim-network-name: {openstack1: mgmt, openstack3: mgmt} } ] }'

===Specifying a volume ID for a VNF volume===

In a generic way, the mapping can be specified in the following way, where 'VM1' is the name of the VDU, 'Storage1' is the volume name in VNF descriptor and '05301095-d7ee-41dd-b520-e8ca08d18a55' is the volume id:

--config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: VM1, volume: [ {name: Storage1, vim-volume-id: 05301095-d7ee-41dd-b520-e8ca08d18a55} ] } ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_vnfd.tar.gz hackfest1-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_nsd.tar.gz hackfest1-ns]; images: [https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%202%20-%20Creating%20a%20basic%20VNF%20and%20NS.pdf creating a basic VNF and NS]''') in the following way:

With the previous hackfest example, according [http://osm-download.etsi.org/ftp/osm-doc/vnfd.html VNF data model] you will add in VNF Descriptor:
<nowiki>
volumes:
- name: Storage1
size: 'Size of the volume'
</nowiki>

osm ns-create --ns_name h1 --nsd_name hackfest1-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: hackfest1VM, volume: [ {name: Storage1, vim-volume-id: 8ab156fd-0f8e-4e01-b434-a0fce63ce1cf} ] } ] } ] }'

===Adding additonal parameters===

From release SIX (current master) additional user parameters can be added, and they land at vdu:cloud-init (Jinja2 format) and/or vnf-configuration primitives (enclosed by <>). Here is an example of VNF descriptor that uses two parameters called "touch_filename" and "touch_filename2"
<pre>
vnfd:
...
vnf-configuration:
config-primitive:
- name: touch
parameter:
- data-type: STRING
default-value: <touch_filename2>
name: filename
initial-config-primitive:
- name: config
parameter:
- name: ssh-hostname
value: <rw_mgmt_ip> # this paramater is internal
- name: ssh-username
value: ubuntu
- name: ssh-password
value: osm4u
seq: '1'
- name: touch
parameter:
- name: filename
value: <touch_filename>
seq: '2'
</pre>

And they can be provided with:
--config '{additionalParamsForVnf: [{member-vnf-index: "1", additionalParams: {touch_filename: your-value, touch_filename2: your-value2}}]}'

OSM instantiation parameters

2019-04-10T17:09:01Z

Tierno:

When creating a NS instance, it is possible to pass instantiation parameters to OSM using the "--config" option of the client or the "config" parameter of the UI. In this section we will illustrate through some of the existing examples how to specify those parameters using OSM client:

===Specify a VIM network name for a NS VLD===
In a generic way, the mapping can be specified in the following way, where 'vldnet' is the name of the network in the NS descriptor and 'netVIM1' is the VIM network that you want to use:
--config '{vld: [ {name: vldnet, vim-network-name: netVIM1} ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_vnfd.tar.gz hackfest1-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_nsd.tar.gz hackfest1-ns]; images: [https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%202%20-%20Creating%20a%20basic%20VNF%20and%20NS.pdf creating a basic VNF and NS]''') in the following way:

osm ns-create --ns_name hf1 --nsd_name hackfest1-ns --vim_account openstack1 --config '{vld: [ {name: mgmtnet, vim-network-name: mgmt} ] }'

===Specify a VIM network name for an internal VLD of a VNF===
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor and 'netVIM1' is the VIM network that you want to use:
--config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, vim-network-name: netVIM1} ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, vim-network-name: mgmt} ] } ] }'

===Specify IP profile information and IP for a NS VLD===
In a generic way, the mapping can be specified in the following way, where 'datanet' is the name of the network in the NS descriptor, ip-profile you have to fill the associated parameters with the data model ( [http://osm-download.etsi.org/ftp/osm-doc/nsd.html# NS data model] ), and vnfd-connection-point-ref is the reference to the connection point:
--config '{vld: [ {name: datanet, ip-profile: {...}, vnfd-connection-point-ref: {...} } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vld: [ {name: datanet, ip-profile: {ip-version: ipv4 ,subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}],dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true}}, vnfd-connection-point-ref: [ {member-vnf-index-ref: "1", vnfd-connection-point-ref: vnf-data, ip-address: "192.168.100.17"}]}]}'

===Specify IP profile information for an internal VLD of a VNF===
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor and ip-profile you have to fill the associated parameters with the data model ( [http://osm-download.etsi.org/ftp/osm-doc/vnfd.html VNF data model] ):

--config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {...} ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:

osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {ip-version: ipv4 ,subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}] ,dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true}}}]}]} '

===Specify IP address and/or MAC address for an interface===

====Specify IP address for an interface====
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'internal' is the name of internal-vld in the VNF descriptor ,ip-profile you have to fill the associated parameters with the data model ( [http://osm-download.etsi.org/ftp/osm-doc/vnfd.html VNF data model] ), 'id1' is the internal-connection-point id and 'a.b.c.d' is the IP that you have to specify for this scenario:

--config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {...}, internal-connection-point: [{id-ref: id1, ip-address: "a.b.c.d"}] ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account ost4 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal, ip-profile: {ip-version: ipv4 ,subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}] ,dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true}}, internal-connection-point: [{id-ref: mgmtVM-internal, ip-address: "192.168.100.3"}]}]}]}'

====Specify MAC address for an interface====
In this scenario, the mapping can be specified in the following way, where '1' is the member vnf index of the constituent vnf in the NS descriptor, 'id1' is the id of VDU in the VNF descriptor and 'interf1' is the name of the interface to which you want to add the MAC address:
--config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: id1, interface: [{name: interf1, mac-address: "aa:bb:cc:dd:ee:ff" }]} ] } ] } '
You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_vnfd.tar.gz hackfest1-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_nsd.tar.gz hackfest1-ns]; images: [https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%202%20-%20Creating%20a%20basic%20VNF%20and%20NS.pdf creating a basic VNF and NS]''') in the following way:
osm ns-create --ns_name hf12 --nsd_name hackfest1-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: hackfest1VM, interface: [{name: vdu-eth0, mac-address: "52:33:44:55:66:21"}]} ] } ] } '

====Specify IP address and MAC address for an interface====
In the following scenario, we will bring together the two previous cases

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:

osm ns-create --ns_name hf12 --nsd_name hackfest2-ns --vim_account ost4 --config '{vnf: [ {member-vnf-index: "1", internal-vld: [ {name: internal , ip-profile: {ip-version: ipv4, subnet-address: "192.168.100.0/24", gateway-address: "0.0.0.0", dns-server: [{address: "8.8.8.8"}] , dhcp-params: {count: 100, start-address: "192.168.100.20", enabled: true} }, internal-connection-point: [ {id-ref: mgmtVM-internal, ip-address: "192.168.100.3"} ] }, ], vdu: [ {id: mgmtVM, interface: [{name: mgmtVM-eth0, mac-address: "52:33:44:55:66:21"}]} ] } ] } '

===Force floating IP address for an interface===
In a generic way, the mapping can be specified in the following way, where 'id1' is the name of the VDU in the VNF descriptor and 'interf1' is the name of the interface:
--config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: id1, interface: [{name: interf1, floating-ip-required: True }]} ] } ] } '

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf2 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vdu:[ {id: mgmtVM, interface: [{name: mgmtVM-eth0, floating-ip-required: True }]} ] } ] } '

Make sure 'vim-network-name' in NS Descriptor has access to the outside to be able to use the parameter 'floating-ip-required'.

===Multi-site deployments (specifying different VIM accounts for different VNFs)===
In this scenario, the mapping can be specified in the following way, where '1 and 2' are the member vnf index of the constituent vnfs in the NS descriptor, 'vim1 and vim2' are the names of vim accounts and 'netVIM1 and netVIM2' are the VIM networks that you want to use:
--config '{vnf: [ {member-vnf-index: "1", vim_account: vim1}, {member-vnf-index: "2", vim_account: vim2} ], vld: [ {name: datanet, vim-network-name: {vim1: netVIM1, vim2: netVIM2} } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_vnfd.tar.gz hackfest2-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_2_nsd.tar.gz hackfest2-ns]; images:[https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%203%20-%20Modeling%20multi-VDU%20VNF%20v2.pdf modeling multi-VDU VNF]''') in the following way:
osm ns-create --ns_name hf12 --nsd_name hackfest2-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vim_account: openstack1}, {member-vnf-index: "2", vim_account: openstack3} ], vld: [ {name: mgmtnet, vim-network-name: {openstack1: mgmt, openstack3: mgmt} } ] }'

===Specifying a volume ID for a VNF volume===

In a generic way, the mapping can be specified in the following way, where 'VM1' is the name of the VDU, 'Storage1' is the volume name in VNF descriptor and '05301095-d7ee-41dd-b520-e8ca08d18a55' is the volume id:

--config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: VM1, volume: [ {name: Storage1, vim-volume-id: 05301095-d7ee-41dd-b520-e8ca08d18a55} ] } ] } ] }'

You can try it using one of the examples of the hackfest ('''descriptors: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_vnfd.tar.gz hackfest1-vnf], [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/packages/hackfest_1_nsd.tar.gz hackfest1-ns]; images: [https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/US1604.qcow2 ubuntu1604], presentation: [https://osm-download.etsi.org/ftp/osm-4.0-four/3rd-hackfest/presentations/20180626%20OSM%20Hackfest%20-%20Session%202%20-%20Creating%20a%20basic%20VNF%20and%20NS.pdf creating a basic VNF and NS]''') in the following way:

With the previous hackfest example, according [http://osm-download.etsi.org/ftp/osm-doc/vnfd.html VNF data model] you will add in VNF Descriptor:
<nowiki>
volumes:
- name: Storage1
size: 'Size of the volume'
</nowiki>

osm ns-create --ns_name h1 --nsd_name hackfest1-ns --vim_account openstack1 --config '{vnf: [ {member-vnf-index: "1", vdu: [ {id: hackfest1VM, volume: [ {name: Storage1, vim-volume-id: 8ab156fd-0f8e-4e01-b434-a0fce63ce1cf} ] } ] } ] }'

===Adding additonal parameters===

From release SIX (current master) additional user parameters can be added, and they land at vdu:cloud-init (Jinja2 format) and/or vnf-configuration primitives (enclosed by <>). Here is an example of VNF descriptor that uses two parameters called "touch_filename" and "touch_filename2"
<pre>
vnfd:
...
vnf-configuration:
config-primitive:
- name: touch
parameter:
- data-type: STRING
default-value: <touch_filename2>
name: filename
initial-config-primitive:
- name: config
parameter:
- name: ssh-hostname
value: <rw_mgmt_ip> # this paramater is internal
- name: ssh-username
value: ubuntu
- name: ssh-password
value: osm4u
seq: '1'
- name: touch
parameter:
- name: filename
value: <touch_filename>
seq: '2'
</pre>

And they can be provided with:
--config '{additionalParamsForVnf: [{member-vnf-index: "1", additionalParams: {touch_filename: your-value, touch_filename2: your-value2}}]}'

Workflow with OSM tools

2019-04-10T11:26:42Z

Tierno:

__TOC__

= Login to OSM portal =
# Go to OSM portal: https://osm.etsi.org/
# Login using your username and password

'''Note:'''
* If you are '''[[Participate|contributing on behalf of your company]]''', you should login with your '''ETSI Online Account (EOL)'''.
** If your company is not yet an '''[https://portal.etsi.org/TBSiteMap/OSM/ListofOSMMembers.aspx OSM Member or Participant]''', you can check here '''[https://osm.etsi.org/welcome/#join how to join OSM as an organization]'''
** If your company has '''already joined OSM but you do not have an EOL account''' yet, you can '''[http://webapp.etsi.org/createaccount/ request an EOL account]'''

* If you are an '''individual contributor''', you can '''[https://osm.etsi.org/index2.php?page=page_register.php create your OSM account online]'''.

If you need any help, contact us at [mailto:OSMsupport@etsi.org OSMsupport@etsi.org]

= Report a bug on Bugzilla =
# Go to the OSM Portal and click on '''Bugzilla''' menu on the portal menu bar. Or simply go to [https://osm.etsi.org/bugzilla OSM Bugzilla]
# Click on "new" on Bugzilla menu bar.
# Choose the product, e.g. "OSM".
# Complete the bug form. If you know, choose the component, e.g. UI, SO, RO, Documentation/Wiki, etc.
# Enter the bug summary, description, attachement, etc.
# Click on the "Submit Bug" button to confirm.
# A bug id is generated (e.g. Bug 6 -Small Error)

= Clone your project =
# Clone a '''git''' repository, for example: <pre> git clone https://osm.etsi.org/gerrit/osm/RO.git </pre>
# Enter your username
# Enter you password
# It is mandatory to install the Gerrit commit-msg hook for the cloned repository in order to automatically insert a Change-Id tag in commit messages. You can install it by entering the following command for RO repository <pre> curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg </pre> You then need to make the hook executable <pre> chmod u+x RO/.git/hooks/commit-msg </pre> More information about the Gerrit commit-msg hook can be found here https://git.eclipse.org/r/Documentation/cmd-hook-commit-msg.html

'''Note''':
* You can find all the project repository URLs on '''[https://osm.etsi.org/gitweb/ Gitweb]''' or on '''[https://osm.etsi.org/gerrit/ Gerrit]'''
** On Gitweb
**# Click on '''Gitweb''' menu on the portal menu bar.
**# Select a product to see the available cloning URLs
** On Gerrit
**# Click on '''Gerrit''' menu on the portal menu bar.
**# Click on "Projects" menu on the Gerrit menu bar.
**# Click on "List" menu.
**# Select a product to see the available cloning URLs.

= Configure your Git environment =
# Configure your git username globally: <pre> git config --global user.name <username> </pre>
# Configure your git email address: <pre> git config --global user.email <email> </pre>
# Check your git configuration: <pre> git config --list </pre>
'''Note: Your email address will be visible on commits to Git.
If you'd like to keep your email address private, you can mask your email as follows:
if your email is <name@company.com> you can set user.email to <hidden@company.com>
This allows other users to identify you as a contributor (with your user name) and
Git Stats to keep track of your company's contributions (with the email domain)

In case you are using git in the same computer for other open source projects, you can restrict your git variables to the local folder: <pre>git config --local user.name <username>
git config --local user.email <email> </pre>

= License Headers =
Newly contributed Source Code should be licensed under the [https://www.apache.org/licenses/LICENSE-2.0 Apache 2.0 license]. A [https://osm.etsi.org/gitweb/?p=osm/LCM.git;a=blob_plain;f=LICENSE;hb=HEAD LICENSE] file shall exist at he root of the repository and all source files should have the following LICENSE header:

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.

= Commit changes to your local project =
# Go to the "RO" folder <pre>cd osm/RO</pre>
# Make some changes on the source (e.g. add a line to the "example-file")
# Stage the change. <pre> git add example-file </pre>
# Commit the change to your local repository. You can add a bug id in your commit message to link the bug to your contribution <pre> git commit -s -m "Bug 2 fixed!!" </pre>
# You can see your commit message and update it, if needed, using the following command: <pre>git commit --amend </pre>

==Developer's Certificate of Origin==
* The "-s" parameter enables to Sign the contribution. It adds the following line at the end of your commit message:
<pre> Signed-off-by: Random J Developer <random@developer.example.org></pre>

* The sign-off is a simple line at the end of the explanation for the patch, which certifies that you wrote it or otherwise have the right to pass it on as a open-source patch. The rules are pretty simple: signing a contribution means that you can certify the below:

<pre>
Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
</pre>

= Push your contribution to Gerrit =
== Push your contribution to Gerrit using git push ==
1. To avoid merge conflicts, it is recommended to do a pull with rebase before pushing your contribution in order to merge latest changes made by other users.
git pull --rebase
'''Note''': you can force git to use always the --rebase option with:
git config --local pull.rebase true

2. Push your commits to '''Gerrit''' for Code Review using the following command:
git push origin HEAD:refs/for/master
#use 'v1.0' instead of 'master' for contributing to v1.0 branch
'''Note''': you can also configure git to push always to the desired branch.
git config --local remote.origin.push HEAD:refs/for/master
git config --local remote.origin.push HEAD:refs/for/v1.0

3. Enter your username.

4. Enter your password.

5. You can review your contribution on '''Gerrit''' web interface

'''Notes:'''
* The following actions are execution automatically after pushing any contribution to Gerrit:
** '''Gerrit''' reports the new contribution state to the corresponding bug in '''Bugzilla'''
** '''Gerrit''' notifies '''Jenkins''' to build the new contribution.

== Push your contribution to Gerrit using git review ==
Instead of using "git push", you could use "git review". Follow this guide ([[Using git-review to push and review changes]]) to install and configure git-review. You will also have to set the SSH URL for your remote and copy the commit-msg hook.
* To avoid merge conflicts, it is recommended to do a pull with rebase before pushing your contribution.
git pull --rebase
* Then, send your contribution to gerrit:
git review -c

= Gerrit Notifications =
Gerrit sends email notifications to:
*Owner of the change
*Reviewers of the change
*Project watchers

Each user can configure his own watched projects and branches in the "settings/watched project" menu. Email notifications can be sent for New Changes, New Patch Sets, All Comments, Submitted Changes and Abandoned Changes.

Link: [https://osm.etsi.org/gerrit/#/settings/projects https://osm.etsi.org/gerrit/#/settings/projects] (you can navigate there from the Gerrit console by clicking on the arrow by your login on the top right corner)

= Continuous Integration =
# '''Jenkins''' builds, tests the new contribution, and reports the result on its web interface.
# '''Jenkins''' votes on the contribution using '''“Code-Verif”''' (-1, +1) on '''Gerrit'''.

= Code Review =
# Other '''contributors''' and '''MDG Committers''' can comment the new contribution and vote using '''“Code-Review”''' (-1, 0, +1) on '''Gerrit'''.
# The '''MDG Leader''' can comment the new contribution and votes using '''“Code-Review”''' (-2, -1, 0, +1, +2) on '''Gerrit'''.

= Amending a contribution in Gerrit =
Before amending, make sure that you have the commit-msg hook installed in your local repo. For instance, if your user is "user" and the repo is the "devops" repo, the command to install the commit-msg hook would be:
$ scp -p -P 29418 user@osm.etsi.org:hooks/commit-msg devops/.git/hooks/

In that way, all patches will use the same Change Id, thus guaranteeing that they will be associated to the same change.

NOTE: If you don't setup your environment as above, and would like to update a gerrit review with your new/updated code, you MUST have the Change-Id (of the existing gerrit review) inside the commit message. If the commit update does not contain the Change-Id, the commit will will end up creating a new gerrit review, and your subsequent gerrit review will need to be abandoned (not ideal).

== Amending your last change/commit pushed to Gerrit ==
1. Get the latest changes from the repo and rebase your changes on top
git pull --rebase

2. Fix the code and any conflicts that might have appeared after pulling the code.

3. Add all updated files to the index.
git add <file>

4. Amend your commit. '''NOTE''': Don't use the -m flag to specify a commit message, since it will override the previous message and regenerate the Change-Id. Instead, use the text editor to change the commit message if needed, and keep the Change-Id line intact.
git commit --amend

5. Submit your change back to the repository. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git push origin HEAD:refs/for/master

== Amending one of your changes which has dependencies on subsequent changes==
This procedure will only work if you are the only person working on a set of commits and you don't expect others to push commits and rebase them to be dependent on yours.

1. Run 'git status' on the branch to know how many commits you are ahead of origin. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git status
On branch master
Your branch is ahead of 'origin/master' by 3 commits.

2. Do an interactive rebase over the last X commits and specify which commit you want to amend by changing the command of the commit from "pick" to "edit". Use "drop" to abandon. Re-order the lines to put commits on a different order.
git rebase -i HEAD~3
edit 0ab17ad Change in file1
pick 4995f46 Second change
pick baa85bf Third change

3. Fix the code in the commit, add the files, amend the commit, and continue rebasing:
vi file1.txt #Fix the code
git add file1.txt
git commit --amend
git rebase --continue

4. In case of conflicts with subsequent commits, you will have to solve them, and continue rebasing
vi file1.txt #Fix the conflict
git add file1.txt #Add the files to the staging area; no commit is required.
git rebase --continue

5. Repeat step 4 as many times as required, until rebase finishes. You can cancel the whole operation at any moment with "git rebase --abort"

6. Rebase your contribution before pushing
git pull --rebase

7. Push your changes to gerrit. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git push origin HEAD:refs/for/master

== Amending a change of another author using git ==
When amending code from a different author, reviewer and original author should be coordinated in order to make sure that changes made by the reviewer are not reverted by the original author later. For that reason, it is recommended to avoid using the local copy of the branch. Instead, it is recommended to create a new local branch to work on every specific change, following the procedure below:

1. Get the last code from the repository.
git fetch origin

2. Create a new branch to work on the code from the remote branch. You can use the review number and patchset as name. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git checkout -b <review-number>-<patchset> origin/master

3. Pull the patch on the created branch. (To find the command to execute you can open the corresponding change page on Gerrit UI, click on download menu, then copy the "pull" command.)
git pull <url> <ref>

4. Fix the code.

5. Add all updated files to the index.
git add <file>

6. Amend the commit. '''NOTE''': Don't use the -m flag to specify a commit message, since it will override the previous message and regenerate the Change-Id. Instead, use the text editor to change the commit message if needed, and keep the Change-Id line intact.
git commit --amend

7. Rebase your contribution before pushing
git pull --rebase

8. Submit your change back to the repository. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git push origin HEAD:refs/for/master

== Amending a change of another author using git-review ==
1. Follow the instructions in '''[[Using git-review to push and review changes|this link]]''' to install and configure git-review.

2. Download the change with git-review
$ git review -d 1280
Downloading refs/changes/80/1280/1 from gerrit
Switched to branch "review/garciadeblas/1280"

This will download the change, put it in a branch called review/AUTHOR/change (if the change has no tag, the sequence number will be used instead), and switch to that branch.

3. After that, you can amend the downloaded change to improve it. Finally, push it again:
$ git add # add the changes
$ git commit --amend # do not touch the Change-Id. The Change-Id is the way for gerrit to keep track what belongs to what development stream as a new patch set.
$ git commit --amend --author # if you want to mark the changes as yours.
$ git review -c -R # The -R is important, since it tells git-review to not rebase your change against master.

'''NOTE''': Don't use the -m flag to specify a commit message, since it will override the previous message and regenerate the Change-Id. Instead, use the text editor to change the commit message if needed, and keep the Change-Id line intact.

4. Delete the branch once you have finished:
$ git branch -D review/garciadeblas/1280

= Merging the contribution =
# The '''MDG Leader''' can "Submit" the change only if +1 '''"Code-Verif"''' (from '''Jenkins''') and at least one +2 '''"Code-Review"'''.
# '''Gerrit''' reports the result of the code review to the corresponding bug in '''Bugzilla'''

= Fixing the author name / email =
* Did you commit your changes with a wrong user name? Don't panic, see how [https://www.everythingcli.org/git-like-a-pro-rewrite-author-history/ this can be fixed]

= OSM CI/CD =
[[File:OSM_CI-CD_workflow.png|700px|OSM CI/CD Workflow]]

[https://osm.etsi.org/wikipub/images/1/16/OSM_CI-CD_pipeline.pdf Learn more]

{{Feedback}}

NBI API Description

2019-04-09T11:15:57Z

Tierno:

NBI API Description

2019-03-28T13:28:16Z

Tierno:

NBI API Description

2019-03-27T18:41:06Z

Tierno:

Developer HowTo for RO Module

2019-03-22T10:29:13Z

Tierno:

==Getting Started==
The RO Module of OSM is implemented by openmano. The recommended Linux distribution for development is Ubuntu 16.04 LTS Server. PyCharm is a nice and easy to use tool for development and debugging.

The step [[Release_1_Installation]] installs all OSM modules in containers. However, for development, a virtual machine may be more suitable. To install the openmano module you can run a script that installs the required packages, clone the project from ''git clones https://osm.etsi.org/gerrit/osm/RO'' and configures openmano client and other utility scripts at path
wget -O install-openmano.sh "https://osm.etsi.org/gitweb/?p=osm/RO.git;a=blob_plain;f=scripts/install-openmano.sh"
chmod +x install-openmano.sh
sudo ./install-openmano.sh -q --develop #-h for help

See also and follow [[Workflow_with_OSM_tools#Clone_your_project]]

New code features must be incorporated to master:
git checkout master
Generate a ".gitignore", you can use the ''.gitignore-common'' example that skips PyCharm and Eclipse files
cp RO/.gitignore-common RO/.gitignore
#edit to include your local files to ignore
Prepare your git environment to push with a proper user/email, push to gerrit. See and configure:
[[Workflow_with_OSM_tools#Configure_your_Git_environment]]
[[Workflow_with_OSM_tools#Commit_changes_to_your_local_project]]
[[ Workflow_with_OSM_tools#Push_your_contribution_to_Gerrit]]

==Programming Language==
The RO module uses Python2. However Python3 conventions for a future migration must be used as far as possible. For example:

{| class="wikitable"
|-
! BAD Python2
! OK Python2 compatible with python3
|-
| "format test string %s number %d" % (st, num)
| "format test string {} number {}".format(st, num)
|-
| print a, b, c
| print(a,b,c)
|-
| except Exception, e
| except Exception as e
|-
| if type(x) == X:
| if isinstance(x,X):
|}

In addition, '''DO NOT USE libraries not compatible with python3'''

Descriptors can be YAML (prefered because more readable and allow comments) or JSON

==Code Style==
Please follow PEP8 style guide for all the Python code. Lines up to 120 length

===Logging===
Use the appropriate logging levels when logging the messages. An example is shown below:
self.logger.debug("Changing state to %s", next_state)

Logging levels (general and per module) are specified at '''openmanod.cfg'''

Try to use few useful logs, not verbose, that brings useful information. For example, in case of fail getting a server, the complete URL should be provided.

Avoid several logs together

WRONG:
self.looger.debug("Entering in method A")
self.logger.debug("Contacting server"

RIGHT:
self.logger.debug("method A, contacting server %s", url)

When the traceback is needed(call stack that generate the exception) , use the "exc_info=True" parameter
self.logger.error("Exception %s when ...", exception, exc_info=True)

===Exceptions===
Code must be wrote in a way that functions and methods raise an exception when something goes wrong, instead of returning a negative or false value.

Example
WRONG
def get_ip_address():
...
if fail:
return False, "Fail because xxx"
return True, ip

...
result, ip = get_ip_address()
if not result:
return False, "Cannot get ip address..."

RIGHT
def get_ip_address():
...
if fail:
raise customException("Fail because ...")
return ip

...
try:
ip = get_ip_address()
...
except customException as e:
raise customException2(str(e))

==Directory Organization==
The code organized into the following high level directories: 
* '''/''' contains the entry server file ''openmanod'' and client ''openmano'' code 
* '''/osm_ro''' contains the RO server code files 
* '''/test''' contains scripts and code for testing 
* '''/database_utils''' contains scripts for database creation, dumping and migration 
* '''/scripts''' general scripts, as installation, execution, reporting 
* '''/scenarios''' examples and templates of network scnario descriptors 
* '''/vnfs''' examples and templates of VNF descriptors 

==RO Architecture==
[[File:OpenmanoArchitecture.png|400px]]

===RO Server modules===
The RO module contains the following modules
* '''openmanod''' is the main program. It reads the configuration file (openmanod.cfg) and execute the httpserver and wait for the end
* '''httpserver.py''' is a thread that implements the northbound API interface, uses python bottle module. Calls main engine methods to perform the tasks
* '''nfvo.py''' is the main engine, implementing all the methods for the creation, deletion and management of vnfs, scenarios and instances. Operations against a VIM are asynchornous. ACTIONs to be done are stored at database before returning ok to the client
* '''nfvo_db.py''' is the module in charge of database operations. Uses base '''db_base.py'''. Database is managed with MySQLdb python library
* '''openmano_schemas.py''' is a dictionary schemas used to validate API request and response content using jsonschema library
* '''vim_thread.py''' There is a thread per VIM and credentials. It performs basic tasks of creating/deleting VM, networks, flavors, etc. In addition it refreshes the VM and network status. It calls vimconn.py methods
* '''vimconn.py''' is the base class for the VIM plugin. It contains the definition of the methods to be implemented. The inherited '''vimconn_openstack.py''', '''vimconn_openvim.py''', '''vimconn_vmware''' and '''vimconn_aws''' implements the operations against the concrete VIM type. OpenStack plugin uses the python-*client libraries, meanwhile Openvim plugin uses direct http requests.
* '''console_proxy_thread.py''' is a thread that implements a TCP/IP proxy for the console access to a VIM

===RO Client modules===
Other modules not part of the server are:
* '''openmano''' is a CLI client
* '''openmanoclient.py''' is a client python library for managing openmano server

===ACTIONS and TASKS===
ACTIONS are a a group of tasks performed against a a concrete instance-scenarios (NS record). The creation and deletion of the instance-scenario itself is an action. As it is asynchonous, NBI returns an "Action_id" that can be used to check the status. For each action, nfvo.py generates individual tasks for the related VIMs. Tasks are both stored at database and sent to the related vim_thread.py. A task has a concrete relation with a VIM, e.g. create/delete a VM, a network, ... look for a flavor, network, ...

===Database content===
TODO

==Database changes==
Database schema can be changed if needed. It is recomended to use a graphical tool (e.g. Heidy) to change database and change it back and copy the SQL commands. Make these steps:

1. openmanod: increment __version__, version_date and database_version

2. database_utils/migrate_mano_db.sh. See the three "TODO"

2a. Increment LAST_DB_VERSION

2b. Annotate a comment to have a track of versions: [ $OPENMANO_VER_NUM -ge 50XX ] && DB_VERSION=XX #0.5.XX => XX

2c. Generate new methods; function upgrade_to_XX() and function downgrade_from_XX. Insert here the sql commands. Last sql command over schema_version is quite important to detect the database version.

Test several upgrades/downgrades to version 20 with "migrate_mano_db.sh" and "migrate_mano_db.sh 20"

==Package dependency osm-im==
It is under IM repository. It contains the OSM models. Used models at RO are under <IM>/models/yang/vnfd.yang and <IM>/models/yang/nsd.yang. More modules will be incorporated in the future.

When IM is maked, pyangbind generates python source files imported at RO. They are used to validate and load a yaml/json VNFD and NSD catalog descriptor.

Method pybindJSONDecoder.load_ietf_json from pyangbind is used to parse and load the descriptor. A structure of nested YANG classes are generated and used at <RO>/osm_ro/nfvo.py methods new_vnfd_v3 and new_nsd_v3. RO supports backward compatibility with old format descriptors. NBI (osm_ro/httpserver.py) uses .../v3/... at URL to diferenciate between old and new descriptors. CLI (openmano) detects automatically if the descriptor is in old format or in OSM format to send the request to the server using the old URL or the new "v3" URL

===Installation===
The OSM binary installation install this package. In case of needed you can install/update it manually by:

apt-get update
apt-get install make git python python-pip tox debhelper python-bitarray python-lxml
pip install --upgrade pip
# If not inside a container. Use "sudo -HE" to get root privileges for pip installation
pip install pyangbind stdeb
git clone https://osm.etsi.org/gerrit/osm/IM
make -C IM clean all
dpkg -i IM/deb_dist/python-pyang_*.deb
dpkg -i IM/deb_dist/python-pyangbind_*.deb
dpkg -i IM/deb_dist/python-osm-im_*.deb
# Use this line to check if it is installed and where
python -c 'import osm_im; print osm_im.__path__[0]'

==Package dependency lib-osm-openvim==

It is under openvim repository. RO uses a library for the SDN assist that is in charge of performing the underlay dataplane connectivity using an openflow controller. osm-openvim and lib-osm-openvim are diffent, though they share same pieces of code. lib-osm-openvim uses a different database name (mano_vim_db) so that openvim can be installed in the same virtual machine or container (openvim uses vim_db database name)

TODO complete where RO uses it

===Installation===
The OSM binary installation install this package. In case of needed you can install/update it manually by:

apt-get update
apt-get install make git python tox libmysqlclient-dev
git clone https://osm.etsi.org/gerrit/osm/openvim
git -C openvim checkout 005a9dc # this is temporal, because last version contains error at Makefile
make -C openvim lite
# Use this line to check if it is installed and where
python -c 'import lib_osm_openvim; print lib_osm_openvim.__path__[0]'
# Install database of ovim library
OSMLIBOVIM_PATH=`python -c 'import lib_osm_openvim; print lib_osm_openvim.__path__[0]'`
# -U and -P are the admin database user/password. Normally "-U root" without password "executed as root"
${OSMLIBOVIM_PATH}/database_utils/install-db-server.sh -U root [-P passwd] -u mano -p manopw -d mano_vim_db --updatedb

==CLI client==

The RO code contains a python CLI (openmano) that allows friently command execution. This CLI client can run on a separate machine where openmano server is running. "openmano config" indicates where the server is (by default localhost)

==Northbound Interface==

The RO uses a REST API with YAML/JSON content. The primitives are explained [[RO Northbound Interface|here]]

==Running Unit Tests==

===Launching RO===
Openmano can run as systemd service. (Can be installed with '''./scripts/install-openmano-service.sh -f RO''')

Or can be run inside a screen (the prefered method for developpers). The script to easily launch/remove is '''./scripts/service-openmano.sh'''. Execute with -h to see options

Note that the general OSM install script, installs openmano inside a container and code run at folder /opt/openmano (instead of $HOME/openmano)

===Tests===
Many of openmano operations rely on an external VIM. Without external infraestructure it is recomended to use openvim in "test" (fake) mode. Install it in the same machine where openmano is located with [[OpenVIM_installation_(Release_One)#Installation]]

Run ''./test/basictest'' for a initial test. Type -h to see options. For testing using openvim just run <pre> . ./test/basictest.sh --force --screen --init-openvim </pre>

==Creating a new VIM plugin==

Choose a name, eg. XXX

Create a new python module vimconn_XXX.py derived from vimconn.py class. Implement the relevant functions. You have several connectors already created to be used as example as openstack and openvim. Openstack connector uses the python openstack client libraries, meanwhile openvim connector uses direct http requests.

DO NOT change the method names, parameters or parameter content. ROuse the same methods for all the VIMs and they cannot be changed to accomodate VIM specifics. VIM specifics must be solved inside the connector.

The new module can need specific configuration for the VIM that it is passed as a dictionary in the ''config'' variable at constructor. For example, in the case of openstack, ''config'' variable is used for: enabling/disabling port_security_enable, specifying the name of the physical network used for dataplane, regions, etc. The ''config'' variable is the right place to specify those needed parameters not provided by ROat the methods that form part of the VIM configuration. See [[Openstack configuration (Release FIVE)#Add openstack toOSM]] and [[Configuring AWS for OSM Release TWO#Add AWS to OSM]] for examples

For integration with the main project, it is needed to add the new vitype to dictionary "vim_module" at file "osm_ro/vim_thread.py".

openmano tenant-create osm # if not already done
export OPENMANO_TENANT=osm
openmano datacenter-create <vim-name> <access-URL> --type=XXX --config <specific config>
# if fails the cause can be a library you need not installed or an error upon importing. Try tho import it in a python client
# example of <specific config> text: "{dataplane_physical_net: physnet_sriov, port_security_enabled: False}"
openmano datacenter-attach <vim-name> --vim-tenant-name=<tenant-to-use-by-openmano> --user=<tenant-user> --password=<tenant-password>

You can test it using the '''test/test_RO.py''' file:
# Create manually an image <image> and a management network <net-mgmt> at your VIM
./test/test_RO.py deploy --help
# e.g. ./test/test_RO.py deploy -n <net-mgmt> -t osm -i <image> -d <vim-name> --timeout=30 --test simple_linux,simple_multi_vnfc,simple_2_vnf
./test/test_RO.py vimconn --help

Developer HowTo

2019-03-05T11:02:50Z

Tierno:

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

==NBI==

Install needed OSM packages IM, common:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
# pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Additionally you may want that docker 'light-ui' uses your local copy of NBI. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:

OSM_SERVER: <172.17.0.1> # nbi
# Get the needed address at your setup by 'ip a | grep docker0'

Clone and install needed NBI packages
docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-02-28T12:48:24Z

Tierno: /* 7 Expose needed ports of docker services */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

==NBI==

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-02-28T12:46:20Z

Tierno: /* NBI */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , ''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092"

==NBI==

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-02-28T12:38:03Z

Tierno: /* LCM */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , ''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092"

==NBI==

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
sudo apt-get install python3-keystoneclient
pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-02-28T12:31:58Z

Tierno: /* 7 Expose needed ports of docker services */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , ''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092"

==NBI==

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
sudo apt-get install python3-keystoneclient
pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-02-28T12:30:41Z

Tierno: /* LCM */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

==NBI==

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
sudo apt-get install python3-keystoneclient
pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==LCM==

Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

Cone and install LCM depencencies
docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-02-28T12:14:12Z

Tierno: /* LCM */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

==NBI==

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
sudo apt-get install python3-keystoneclient
pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==LCM==

Install needed OSM modules if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors

docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2019-02-28T12:07:29Z

Tierno: /* 6 Install needed packages */

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with
pip3 install -e common --user # needed for LCM, NBI
pip3 install -e N2VC --user # needed for LCM
pip3 install -e IM --user # needed for NBI

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

==NBI==

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
sudo apt-get install python3-keystoneclient
pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==LCM==

docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

Install needed OSM modules:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

NBI API Description

2019-02-27T14:55:23Z

Tierno:

OSM Release FIVE

2019-02-27T13:24:32Z

Tierno:

[[File:6hackdate.png|350px|right|top|link=https://osm.etsi.org/wikipub/index.php/6th_OSM_Hackfest]]

Open Source MANO (OSM) is an ETSI-hosted open source community delivering a production-quality MANO stack for NFV, capable of consuming openly published information models, available to everyone, suitable for all VNFs, operationally significant and VIM-independent. OSM is aligned to NFV ISG information models while providing first-hand feedback based on its implementation experience.

Release FIVE brings a number of improvements over previous releases. For the full list of new features, please refer to the Rel FIVE White Paper or the [[OSM Release FIVE Documentation]]. For a comprehensive overview of OSM functionalities, you can also refer to the release notes of former releases ([https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseONE-FINAL.pdf ONE], [https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseTWO-FINAL.pdf TWO], [https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseTHREE-FINAL.pdf THREE],
[https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseFOUR-FINAL.pdf FOUR]).

'''OSM in Practice'''
{{#evu:https://www.youtube.com/watch?v=kCFxPV67Adw
|alignment=left
}}

__TOC__

=Assumptions about interaction with VIMs and VNFs=
The following figure illustrates OSM interaction with VIMs and VNFs:
* OSM talks to the VIM for the deployment of VNFs and VLs connecting them
* OSM talks to the VNFs deployed in a VIM to run day-0, day-1 and day-2 configuration primitives.

[[File:Osmtopology.png|600px|OSM Release 4]]

In order for OSM to work, it is assumed that:
* Each VIM has an API endpoint reachable from OSM
* Each VIM has a so called management network which provides IP address to VNFs
* That management network is reachable from OSM

=Install OSM Release FIVE=

This release expands the cloud-native, ''dockerized'' OSM installation with the kafka bus for asynchronous communications, a lighter orchestrator with Network Services and Slicing capabilities, performance/fault/policy management features, a SOL005-aligned NBI, an enhanced GUI with a service composer, and OSM client leveraging the unified and improved REST API that the NBI exposes.

== Default installation procedure ==

All you need to run OSM Release FIVE is a single server or VM with the following requirements:
* MINIMUM: 2 CPUs, 4 GB RAM, 20GB disk and a single interface with Internet access
* RECOMMENDED: 2 CPUs, 8 GB RAM, 40GB disk and a single interface with Internet access
* Ubuntu16.04 (64-bit variant required) as base image (http://releases.ubuntu.com/16.04/)

Once you have prepared the host with the previous requirements, all you need to do is:
wget https://osm-download.etsi.org/ftp/osm-5.0-five/install_osm.sh
chmod +x install_osm.sh
./install_osm.sh

'''TIP:''' In order to facilitate potential trobleshooting later, it is recommended to save the full log of your installation process:
wget https://osm-download.etsi.org/ftp/osm-5.0-five/install_osm.sh
chmod +x install_osm.sh
./install_osm.sh 2>&1 | tee osm_install_log.txt

You will be asked if you want to proceed with the installation and configuration of LXD, juju, docker CE and the initialization of a local docker swarm, as pre-requirements. Please answer "y".

Then, some dialog messages related to LXD configuration will be shown. This is what you have to answer:
* Do you want to configure the LXD bridge? '''Yes'''
* Do you want to setup an IPv4 subnet? '''Yes'''
* ''<< '''Default values''' apply for next questions >>''
* Do you want to setup an IPv6 subnet? '''No'''

If you want to run docker commands, you will also need to either reload the shell or run the following command to add your user to the 'docker' group:
newgrp docker

===Installation including optional components===

You can include optional components in your installation by adding the following flags:
* '''VIM Emulator:''' --vimemu (more information [[VIM emulator|here]])
* '''Performance Management additional stack (Grafana only):''' --pm_stack (more information [[OSM Performance Management|here]])
* '''Fault Management features with ELK:''' --elk_stack (more information [[OSM Fault Management|here]])

Example:
./install_osm.sh --elk_stack --pm_stack --vimemu

===Other installation options===
* You can also run OSM using a pre-built [https://app.vagrantup.com/osm/boxes/releasefive vagrant] image: [[How_to_run_OSM_on_Vagrant]]
* For other special installation options, please refer to the [[OSM_Release_FIVE_Documentation#Alternative_installation_methods | release documentation]].

==Checking your installation==

After some time, you will get a fresh OSM Release FIVE installation with its latest, pre-built docker images which are built daily. You can access to the UI in the following URL (user:admin, password: admin): http://1.2.3.4, replacing 1.2.3.4 with the IP address of your host.

[[File:Osm_lwb_ui_login.png|600px|OSM home]]

[[File:Osm_lwb_ui.png|600px|OSM Release FIVE installation result]]

As a result of the installation, thirteen docker containers are created in the host (without considering optional stacks). You can check they are running by issuing the following commands:
docker stack ps osm |grep -i running
docker service ls

[[File:Osm_containers_rel5.png|600px|OSM Docker containers]]

At any time, you can quickly relaunch your deployment by using the pre-built docker images, like this:
docker stack rm osm && sleep 60 # The sleep is for making sure the stack removal finishes before redeploying
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

To check the logs of any container:
docker service logs osm_lcm # for LCM

OSM client, a python-based CLI for OSM, will be available as well in the host machine.
Via the OSM client, you can manage descriptors, NS and VIM complete lifecycle.

=Adding VIM accounts=
Before proceeding, make sure that you have a site with a VIM configured to run with OSM. Different kinds of VIMs are currently supported by OSM:
*'''OpenVIM.''' Check the following link to know how to install and use openvim for OSM: [[OpenVIM installation | OpenVIM installation]]. Openvim must run in 'normal' mode (not test or fake) to have real virtual machines reachable from OSM.
*'''OpenStack.''' Check the following link to learn how to configure OpenStack to be used by OSM: [[Openstack_configuration | Openstack configuration]]
*'''VMware vCloud Director.''' Check the following link to learn how to configure VMware VCD to be used by OSM: [[Configuring_VMware_vCloud_Director | Configuring VMware vCloud Director]]
*'''Amazon Web Services (AWS).''' Check the following link to learn how to configure AWS (EC2 and Virtual Private Cloud) to be used by OSM: [[Configuring_AWS | Configuring AWS for OSM]]
OSM can manage external SDN controllers to perform the dataplane underlay network connectivity on behalf of the VIM. See [[EPA and SDN assist]]

==Adding VIMs through OSM client==
===OpenVIM site===
Execute the following command, using the appropriate parameters (e.g. site name: "openvim-site", IP address: 10.10.10.10, VIM tenant: "osm")
osm vim-create --name openvim-site --auth_url http://10.10.10.10:9080/openvim --account_type openvim \
--description "Openvim site" --tenant osm --user dummy --password dummy

===Openstack site===
Execute the following command, using the appropriate parameters (e.g. site name: "openstack-site", IP address: 10.10.10.11, VIM tenant: "admin", user: "admin", password: "userpwd")
osm vim-create --name openstack-site --user admin --password userpwd \
--auth_url http://10.10.10.11:5000/v2.0 --tenant admin --account_type openstack

For advanced options, please refer to the [[Openstack_configuration#Add_openstack_to_OSM | OpenStack Setup Guide]].

===VMware vCloud Director site===
*Execute the following command, using the appropriate parameters (e.g. site name: "vmware-site", IP address: 10.10.10.12, VIM tenant: "vmware-tenant", user: "osm", password: "osm4u", admin user: "admin", admin password: "adminpwd", organization: "orgVDC")
osm vim-create --name vmware-site --user osm --password osm4u --auth_url https://10.10.10.12 \
--tenant vmware-tenant --account_type vmware \
--config '{admin_username: user, admin_password: passwd, orgname: organization, nsx_manager: "http://10.10.10.12",
nsx_user: user, nsx_password: userpwd,"vcenter_port": port, "vcenter_user":user, "vcenter_password":password,
"vcenter_ip": 10.10.10.14}'

For advanced options, please refer to the [[Configuring_VMware_vCloud_Director#Add_vCloud_using_OSM_Client | vCloud Director Setup Guide]].

===VMware Integrated Openstack (VIO) site===
Execute the following command, using the appropriate parameters (e.g. site name: "openstack-site-vio4", IP address: 10.10.10.12, VIM tenant: "admin", user: "admin", password: "passwd")
osm vim-create --name VIO --user admin --password passwd --auth_url https://10.10.10.12:5000/v3 --tenant admin \
--account_type openstack --config '{use_floating_ip: True, insecure: true, vim_type: VIO, APIversion: v3.3,
dataplane_physical_net: dvs-46, "use_internal_endpoint":true,"dataplane_net_vlan_range":["31-35","37-39"]}'

With respect to Openstack, the additional configuration for VIO is the following:
* '''vim_type''': Set to "VIO" to use VMware Integrated openstack as VIM
* '''use_internal_endpoint''': When true it allows use of private API endpoints
* '''dataplane_physical_net''': The configured network_vlan_ranges at neutron for the SRIOV (binding direct) and passthrough (binding direct-physical) networks, e.g. 'physnet_sriov' in the above configuration. In case of VMware Integrated Openstack (VIO) provide moref ID of distributed virtual switch, e.g 'dvs-46' in above configuration.
* '''dataplane_net_vlan_range''': In case of VMware Integrated Openstack (VIO) provide vlan ranges for the SRIOV (binding direct) networks in format ['start_ID - end_ID']

For common options, you may refer to the general [[Openstack_configuration#Add_openstack_to_OSM | OpenStack Setup Guide]].

===Amazon Web Services (AWS) site===
*Execute the following command, using the appropriate parameters (e.g. site name: "aws-site", VIM tenant: "admin", user: "XXX", password: "YYY")
osm vim-create --name aws-site --user XXX --password YYY --auth_url https://aws.amazon.com --tenant admin \
--account_type aws --config '{region_name: eu-central-1, flavor_info: {t2.nano: {cpus: 1, disk: 100, ram: 512},
t2.micro: {cpus: 1, disk: 100, ram: 1024}, t2.small: {cpus: 1, disk: 100, ram: 2048},
m1.small: {cpus: 1, disk: 160, ram: 1741}}}'

For advanced options, please refer to the [[Configuring_AWS#Add_AWS_to_OSM | AWS Setup Guide]].

==Adding VIMs through GUI==
Just access the ''VIM Accounts'' tab, click the ''New VIM'' button and fill the parameters accordingly.

[[File:Osmvim.png|600px|AddingVIMUI]]

=Deploying your first Network Service=
In this example we will deploy the following Network Service, consisting of two simple VNFs based on CirrOS connected by a simple VLD.

[[File:cirros_2vnf_ns.png|500px|NS with 2 CirrOS VNF]]

Before going on, download the required VNF and NS packages from this URL: https://osm-download.etsi.org/ftp/osm-3.0-three/examples/cirros_2vnf_ns/

==Onboarding a VNF==

The onboarding of a VNF in OSM involves adding the corresponding VNF package to the system. This process also assumes, as a pre-condition, that the corresponding VM images are available in the VIM(s) where it will be instantiated.

===Uploading VM image(s) to the VIM(s)===
In this example, only a vanilla CirrOS 0.3.4 image is need. It can be obtained from the following link: http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img

If not available, it would be required to upload the image into the VIM. Instructions differ from one VIM to another:
*In Openstack:
openstack image create --file="./cirros-0.3.4-x86_64-disk.img" --container-format=bare --disk-format=qcow2 cirros034
*In OpenVIM:
#copy your image to the NFS shared folder (e.g. /mnt/openvim-nfs)
cp ./cirros-0.3.4-x86_64-disk.img /mnt/openvim-nfs/
openvim image-create --name cirros034 --path /mnt/openvim-nfs/cirros-0.3.4-x86_64-disk.img

===VNF package onboarding===

*From the UI:
**Go to Projects --> Admin --> VNF Packages (''Open List'')
**Click on the Onboard VNFD button
**Drag and drop the VNF package file cirros_vnf.tar.gz in the importing area.
[[File:vnfd_onboard_r4.png|600px|Onboarding a VNF]]
*From OSM client:
osm vnfd-create cirros_vnf.tar.gz
osm vnfd-list

==Onboarding a NS==
*From the UI:
**Go to Projects --> Admin --> NS Packages (''Open List'')
**Click on the Onboard NSD button
**Drag and drop the NS package file cirros_2vnf_ns.tar.gz in the importing area.
[[File:nsd_onboard_r4.png|600px|Onboarding a NS]]
*From OSM client:
osm nsd-create cirros_2vnf_ns.tar.gz
osm nsd-list

==Instantiating the NS==
*From the UI:
**Go to Projects --> Admin --> NS Packages (''Open List'')
**Next the NS descriptor to be instantiated, click on Launch
[[File:nsd_list.png|600px|Instantiating a NS (step 1)]]
**Fill the form, adding at least a name and selecting the VIM
[[File:new_ns.png|600px|Instantiating a NS (step 2)]]
* From OSM client:
osm ns-create --nsd_name cirros_2vnf_ns --ns_name <ns-instance-name> --vim_account <data-center-name>
osm ns-list

'''Instantiation parameters can be specified using both CLI and UI. There are some examples collected in this wiki page: [[OSM instantiation parameters]].'''

==Using old descriptors from Release TWO==

Beware that old Release TWO descriptors cannot be directly used by OSM Release FIVE.

You should use this '''[[Creating your own VNF package#Migrating old descriptors to release FIVE|conversion tool]]''' to convert them to Release FIVE format and '''[[Creating your own VNF package|create the corresponding package]]'''.

=What's next?=

If you want to learn more, you can refer to the rest of '''[[OSM_Release_FIVE_Documentation | OSM documentation]]'''.

{{Feedback}}

NBI API Description

2019-02-27T07:53:46Z

Tierno:

NBI API Description

2019-02-27T07:28:24Z

Tierno:

OSM Release FIVE Documentation

2019-02-27T07:05:33Z

Tierno:

[[File:6hackdate.png|320px|right|top|link=https://osm.etsi.org/wikipub/index.php/6th_OSM_Hackfest]]

== Release FIVE Quickstart ==
* In order to get started quickly and get familiar with the most common operations, you can refer to the [[OSM_Release_FIVE | Release FIVE Quickstart page]].

== VNF and NS examples ==
* [[Examples from OSM Hackfests]]

== Onboarding your own VNF and NS ==
* [[Creating_your_own_VNF_package | Create your own VNF package]]
* [[Creating_your_own_VNF_charm_(Release_THREE) | Creating your own VNF charm]]
** Some [[Example_VNF_Charms | examples of VNF Charms]]
** [[VNF_Primitives | Default VNF Primitives]]
* [[OSM_Information_Model | OSM Information Model]]
* [[Reference_VNF_and_NS_Descriptors | Information to create VNF and NS Descriptors]]
* [[OSM_Release_FOUR#Using_old_descriptors_from_Release_TWO | How to onboard old Release TWO descriptors]]

== Specific installation methods ==
* [[How_to_run_OSM_on_Vagrant | How to run OSM on Vagrant]]
* [[How to upgrade the OSM Platform]]
* [[How to install OSM client | How to install OSM client separately]]
* [[Advanced OSM installation procedures]]
* [[How to install OSM in Amazon EC2 | How to install OSM in Amazon EC2]]

== EPA and SDN Assist ==
* [[EPA_and_SDN_assist | How-to guide]]
* [[Openstack_configuration_(Release_FOUR)#Configure_Openstack_for_full_EPA_support_in_OSM | OpenStack configuration for EPA and SDN Assist]]

== Reference material and self training ==
* [[OSM client]]:
** [[OSM_client#Installation | Installation]]
** [[OSM Client commands]]
** [[Enable autocompletion in OSM Client]]
* [[OSM instantiation parameters]]
* Using Monitoring and Autoscaling capabilities:
** [[OSM_Performance_Management | OSM Performance Management]]
** [[OSM_Fault_Management | OSM Fault Management]]
** [[OSM_Autoscaling | OSM Autoscaling]]
* 4th OSM Hackfest - full training material:
** [[4th_OSM_Hackfest#Agenda | Table of contents]]
** [https://osm-download.etsi.org/ftp/osm-4.0-four/4th-hackfest/presentations/ Documentation]
** [https://osm-download.etsi.org/ftp/osm-4.0-four/4th-hackfest/packages/ NS and VNF Packages] and [https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/ VM images]
* [[NBI_API_Description| Descriptrion of the OSM North Bound Interface REST API]]
* [[VIM_emulator | How to use the VIM emulator]]
* [[OSM_E2E_tests | Tests to validate VIM capabilities]]
* [[OSM_workshops_and_events | OSM Workshops videos]]
* [[OSM_Technical_Videos | Other technical videos and demos]]
* [https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseFOUR-FINAL.pdf Release FOUR White Paper]
* [https://osm.etsi.org/images/OSM_White_Paper_Experience_implementing_NFV_specs_final.pdf OSM's End User Advisory Group White Paper]

== Troubleshooting ==
* [[How to know the version of your current OSM installation]]
* [[Common issues and troubleshooting]]
* Have you detected any bug? Check [[How_to_report_issues_(Release_THREE) | this guide]] to see how to report issues.

== OSM Scope, Features and Integration Guidelines ==

* [[OSM Scope, Functionality, Operation and Integration Guidelines]]
* [[Release notes and whitepapers]]

== How to contribute ==
* [https://osm.etsi.org/about/how-to-join Joining the OSM community]
* Developer guides:
** [[Workflow_with_OSM_tools | Workflow with OSM tools (git, gerrit, bugzilla, etc.) (MUST READ for developers)]]
** [[Developer HowTo | Developer how-to (MUST READ for developers)]]
** [[Developer_HowTo_for_RO_Module | Developer how-to for RO]]
**[[How_to_upgrade_the_OSM_Platform#Upgrading_a_specific_component_to_use_your_own_cloned_repo_.28e.g._for_developing_purposes.29 | How to work with your own cloned repo for a specific component]]
* [https://opensourcemano.slack.com/ OSM Slack Workplace]
* Upcoming Hackfests and events:
** [[5th_OSM_Hackfest|5th OSM Hackfest - Barcelona]] - 4-8 Feb 2019

{{Feedback}}

NBI API Description

2019-02-21T10:01:30Z

Tierno:

NBI API Description

2019-02-14T12:18:46Z

Tierno:

NBI API Description

2019-02-12T09:51:28Z

Tierno:

NBI API Description

2019-02-11T16:16:48Z

Tierno:

NBI API Description

2019-02-11T11:19:21Z

Tierno: Created page with "__TOC__ == Introduction == OSM North Bound Interface is a REST-full following ETSI SOL005 standard. It admits both YAML/JSON formats. By default it serves https (auto-signed..."

Developer HowTo

2018-12-18T11:40:11Z

Tierno:

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.

<pre>
__________
| |
| light-ui |OSM_SERVER _______
| :80 |----------------------> | |
|__________| | nbi |
OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST _______
.............. <--------------------|_______|-------------------------> | |
. volume: . | |
. osm_osm_ . | mongo |
. packages . OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST | :27017|
.............. <--------------------| |-------------------------> |_______|
| lcm |
************** OSMLCM_VCA_HOST| |OSMLCM_RO_HOST
* lxd: juju * <--------------------|_______|--------------|
* controller * |
************** _______ | _______
| | <------------- | |
| ro | | ro-db |
| :9090 |RO_DB_HOST | :3306 |
|_______|-------------------------> |_______|

_______ _______ _______ _________
| | | | | | | |
| mon | | pm | | kafka |KAFKA_ZOOKEEPER_CONNECT |zookeeper|
| :8662 | | | | :9092 |-------------------------> | :2181 |
|_______| |_______| |_______| |_________|

</pre>

==General steps==
=== 1 Shutdown the container you want to debug===

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.

docker service scale osm_lcm=0

=== 2 Clone the module===

git clone https://osm.etsi.org/gerrit/osm/LCM

=== 3 Install the module===

Inside the folder where the module is cloned type the following command:

pip3 install -e . # try with --user if get permission errors

'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'

'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

=== 4 Setup the IDE===

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

[[File:PythonInterpreter.jpg|500px|top]]

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

[[File:PyCharmConfiguration.jpg|600px]]

=== 5 Configure it to interact with other modules===

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":

a.b.c.d mongo ro kafka nbi ro-db

=== 6 Install needed packages===

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are:
n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

=== 7 Expose needed ports of docker services===

To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
# ports:
# - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"

==NBI==

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI # try with --user if get permission errors
sudo apt-get install python3-keystoneclient
pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

Install needed OSM modules:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==LCM==

docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM # try with --user if get permission errors

Python interpreter: Python3
Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG

Install needed OSM modules:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common # try with --user if get permission errors
pip3 install aiokafka==0.4.* pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service '''mongo''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]

==RO==

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines:
ports:
- "3306:3306"
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
OSMLCM_RO_HOST: <172.17.0.1> # ro
# Get the needed address at your setup by 'ip a | grep docker0'

Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code:

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: ro or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Developer HowTo

2018-12-14T14:25:53Z

Tierno:

Workflow with OSM tools

2018-11-23T11:16:10Z

Tierno:

__TOC__

= Login to OSM portal =
# Go to OSM portal: https://osm.etsi.org/
# Login using your username and password

'''Note:'''
* If you are '''[[Participate|contributing on behalf of your company]]''', you should login with your '''ETSI Online Account (EOL)'''.
** If your company is not yet an '''[https://portal.etsi.org/TBSiteMap/OSM/ListofOSMMembers.aspx OSM Member or Participant]''', you can check here '''[https://osm.etsi.org/welcome/#join how to join OSM as an organization]'''
** If your company has '''already joined OSM but you do not have an EOL account''' yet, you can '''[http://webapp.etsi.org/createaccount/ request an EOL account]'''

* If you are an '''individual contributor''', you can '''[https://osm.etsi.org/index2.php?page=page_register.php create your OSM account online]'''.

If you need any help, contact us at [mailto:OSMsupport@etsi.org OSMsupport@etsi.org]

= Report a bug on Bugzilla =
# Go to the OSM Portal and click on '''Bugzilla''' menu on the portal menu bar. Or simply go to [https://osm.etsi.org/bugzilla OSM Bugzilla]
# Click on "new" on Bugzilla menu bar.
# Choose the product, e.g. "OSM".
# Complete the bug form. If you know, choose the component, e.g. UI, SO, RO, Documentation/Wiki, etc.
# Enter the bug summary, description, attachement, etc.
# Click on the "Submit Bug" button to confirm.
# A bug id is generated (e.g. Bug 6 -Small Error)

= Clone your project =
# Clone a '''git''' repository, for example: <pre> git clone https://osm.etsi.org/gerrit/osm/RO.git </pre>
# Enter your username
# Enter you password
# It is madatory to install the Gerrit commit-msg hook for the cloned repository in order to automatically insert a Change-Id tag in commit messages. You can install it by entering the following command for RO repository <pre> curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg </pre> You then need to make the hook executable <pre> chmod u+x RO/.git/hooks/commit-msg </pre> More information about the Gerrit commit-msg hook can be found here https://git.eclipse.org/r/Documentation/cmd-hook-commit-msg.html

'''Note''':
* You can find all the project repository URLs on '''[https://osm.etsi.org/gitweb/ Gitweb]''' or on '''[https://osm.etsi.org/gerrit/ Gerrit]'''
** On Gitweb
**# Click on '''Gitweb''' menu on the portal menu bar.
**# Select a product to see the available cloning URLs
** On Gerrit
**# Click on '''Gerrit''' menu on the portal menu bar.
**# Click on "Projects" menu on the Gerrit menu bar.
**# Click on "List" menu.
**# Select a product to see the available cloning URLs.

= Configure your Git environment =
# Configure your git username globally: <pre> git config --global user.name <username> </pre>
# Configure your git email address: <pre> git config --global user.email <email> </pre>
# Check your git configuration: <pre> git config --list </pre>
'''Note: Your email address will be visible on commits to Git.
If you'd like to keep your email address private, you can mask your email as follows:
if your email is <name@company.com> you can set user.email to <hidden@company.com>
This allows other users to identify you as a contributor (with your user name) and
Git Stats to keep track of your company's contributions (with the email domain)

In case you are using git in the same computer for other open source projects, you can restrict your git variables to the local folder: <pre>git config --local user.name <username>
git config --local user.email <email> </pre>

= License Headers =
Newly contributed Source Code should be licensed under the [https://www.apache.org/licenses/LICENSE-2.0 Apache 2.0 license]. A [https://osm.etsi.org/gitweb/?p=osm/LCM.git;a=blob_plain;f=LICENSE;hb=HEAD LICENSE] file shall exist at he root of the repository and all source files should have the following LICENSE header:

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.

= Commit changes to your local project =
# Go to the "RO" folder <pre>cd osm/RO</pre>
# Make some changes on the source (e.g. add a line to the "example-file")
# Stage the change. <pre> git add example-file </pre>
# Commit the change to your local repository. You can add a bug id in your commit message to link the bug to your contribution <pre> git commit -s -m "Bug 2 fixed!!" </pre>
# You can see your commit message and update it, if needed, using the following command: <pre>git commit --amend </pre>

==Developer's Certificate of Origin==
* The "-s" parameter enables to Sign the contribution. It adds the following line at the end of your commit message:
<pre> Signed-off-by: Random J Developer <random@developer.example.org></pre>

* The sign-off is a simple line at the end of the explanation for the patch, which certifies that you wrote it or otherwise have the right to pass it on as a open-source patch. The rules are pretty simple: signing a contribution means that you can certify the below:

<pre>
Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
</pre>

= Push your contribution to Gerrit =
== Push your contribution to Gerrit using git push ==
1. To avoid merge conflicts, it is recommended to do a pull with rebase before pushing your contribution in order to merge latest changes made by other users.
git pull --rebase
'''Note''': you can force git to use always the --rebase option with:
git config --local pull.rebase true

2. Push your commits to '''Gerrit''' for Code Review using the following command:
git push origin HEAD:refs/for/master
#use 'v1.0' instead of 'master' for contributing to v1.0 branch
'''Note''': you can also configure git to push always to the desired branch.
git config --local remote.origin.push HEAD:refs/for/master
git config --local remote.origin.push HEAD:refs/for/v1.0

3. Enter your username.

4. Enter your password.

5. You can review your contribution on '''Gerrit''' web interface

'''Notes:'''
* The following actions are execution automatically after pushing any contribution to Gerrit:
** '''Gerrit''' reports the new contribution state to the corresponding bug in '''Bugzilla'''
** '''Gerrit''' notifies '''Jenkins''' to build the new contribution.

== Push your contribution to Gerrit using git review ==
Instead of using "git push", you could use "git review". Follow this guide ([[Using git-review to push and review changes]]) to install and configure git-review. You will also have to set the SSH URL for your remote and copy the commit-msg hook.
* To avoid merge conflicts, it is recommended to do a pull with rebase before pushing your contribution.
git pull --rebase
* Then, send your contribution to gerrit:
git review -c

= Gerrit Notifications =
Gerrit sends email notifications to:
*Owner of the change
*Reviewers of the change
*Project watchers

Each user can configure his own watched projects and branches in the "settings/watched project" menu. Email notifications can be sent for New Changes, New Patch Sets, All Comments, Submitted Changes and Abandoned Changes.

Link: [https://osm.etsi.org/gerrit/#/settings/projects https://osm.etsi.org/gerrit/#/settings/projects] (you can navigate there from the Gerrit console by clicking on the arrow by your login on the top right corner)

= Continuous Integration =
# '''Jenkins''' builds, tests the new contribution, and reports the result on its web interface.
# '''Jenkins''' votes on the contribution using '''“Code-Verif”''' (-1, +1) on '''Gerrit'''.

= Code Review =
# Other '''contributors''' and '''MDG Committers''' can comment the new contribution and vote using '''“Code-Review”''' (-1, 0, +1) on '''Gerrit'''.
# The '''MDG Leader''' can comment the new contribution and votes using '''“Code-Review”''' (-2, -1, 0, +1, +2) on '''Gerrit'''.

= Amending a contribution in Gerrit =
Before amending, make sure that you have the commit-msg hook installed in your local repo. For instance, if your user is "user" and the repo is the "devops" repo, the command to install the commit-msg hook would be:
$ scp -p -P 29418 user@osm.etsi.org:hooks/commit-msg devops/.git/hooks/

In that way, all patches will use the same Change Id, thus guaranteeing that they will be associated to the same change.

NOTE: If you don't setup your environment as above, and would like to update a gerrit review with your new/updated code, you MUST have the Change-Id (of the existing gerrit review) inside the commit message. If the commit update does not contain the Change-Id, the commit will will end up creating a new gerrit review, and your subsequent gerrit review will need to be abandoned (not ideal).

== Amending your last change/commit pushed to Gerrit ==
1. Get the latest changes from the repo and rebase your changes on top
git pull --rebase

2. Fix the code and any conflicts that might have appeared after pulling the code.

3. Add all updated files to the index.
git add <file>

4. Amend your commit. '''NOTE''': Don't use the -m flag to specify a commit message, since it will override the previous message and regenerate the Change-Id. Instead, use the text editor to change the commit message if needed, and keep the Change-Id line intact.
git commit --amend

5. Submit your change back to the repository. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git push origin HEAD:refs/for/master

== Amending one of your changes which has dependencies on subsequent changes==
This procedure will only work if you are the only person working on a set of commits and you don't expect others to push commits and rebase them to be dependent on yours.

1. Run 'git status' on the branch to know how many commits you are ahead of origin. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git status
On branch master
Your branch is ahead of 'origin/master' by 3 commits.

2. Do an interactive rebase over the last X commits and specify which commit you want to amend by changing the command of the commit from "pick" to "edit".
git rebase -i HEAD~3
edit 0ab17ad Change in file1
pick 4995f46 Second change
pick baa85bf Third change

3. Fix the code in the commit, add the files, amend the commit, and continue rebasing:
vi file1.txt #Fix the code
git add file1.txt
git commit --amend
git rebase --continue

4. In case of conflicts with subsequent commits, you will have to solve them, and continue rebasing
vi file1.txt #Fix the conflict
git add file1.txt #Add the files to the staging area; no commit is required.
git rebase --continue

5. Repeat step 4 as many times as required, until rebase finishes.

6. Rebase your contribution before pushing
git pull --rebase

7. Push your changes to gerrit. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git push origin HEAD:refs/for/master

== Amending a change of another author using git ==
When amending code from a different author, reviewer and original author should be coordinated in order to make sure that changes made by the reviewer are not reverted by the original author later. For that reason, it is recommended to avoid using the local copy of the branch. Instead, it is recommended to create a new local branch to work on every specific change, following the procedure below:

1. Get the last code from the repository.
git fetch origin

2. Create a new branch to work on the code from the remote branch. You can use the review number and patchset as name. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git checkout -b <review-number>-<patchset> origin/master

3. Pull the patch on the created branch. (To find the command to execute you can open the corresponding change page on Gerrit UI, click on download menu, then copy the "pull" command.)
git pull <url> <ref>

4. Fix the code.

5. Add all updated files to the index.
git add <file>

6. Amend the commit. '''NOTE''': Don't use the -m flag to specify a commit message, since it will override the previous message and regenerate the Change-Id. Instead, use the text editor to change the commit message if needed, and keep the Change-Id line intact.
git commit --amend

7. Rebase your contribution before pushing
git pull --rebase

8. Submit your change back to the repository. '''NOTE''': use the appropriate branch instead of master, if you are working on a different branch.
git push origin HEAD:refs/for/master

== Amending a change of another author using git-review ==
1. Follow the instructions in '''[[Using git-review to push and review changes|this link]]''' to install and configure git-review.

2. Download the change with git-review
$ git review -d 1280
Downloading refs/changes/80/1280/1 from gerrit
Switched to branch "review/garciadeblas/1280"

This will download the change, put it in a branch called review/AUTHOR/change (if the change has no tag, the sequence number will be used instead), and switch to that branch.

3. After that, you can amend the downloaded change to improve it. Finally, push it again:
$ git add # add the changes
$ git commit --amend # do not touch the Change-Id. The Change-Id is the way for gerrit to keep track what belongs to what development stream as a new patch set.
$ git commit --amend --author # if you want to mark the changes as yours.
$ git review -c -R # The -R is important, since it tells git-review to not rebase your change against master.

'''NOTE''': Don't use the -m flag to specify a commit message, since it will override the previous message and regenerate the Change-Id. Instead, use the text editor to change the commit message if needed, and keep the Change-Id line intact.

4. Delete the branch once you have finished:
$ git branch -D review/garciadeblas/1280

= Merging the contribution =
# The '''MDG Leader''' can "Submit" the change only if +1 '''"Code-Verif"''' (from '''Jenkins''') and at least one +2 '''"Code-Review"'''.
# '''Gerrit''' reports the result of the code review to the corresponding bug in '''Bugzilla'''

= Fixing the author name / email =
* Did you commit your changes with a wrong user name? Don't panic, see how [https://www.everythingcli.org/git-like-a-pro-rewrite-author-history/ this can be fixed]

= OSM CI/CD =
[[File:OSM_CI-CD_workflow.png|700px|OSM CI/CD Workflow]]

[https://osm.etsi.org/wikipub/images/1/16/OSM_CI-CD_pipeline.pdf Learn more]

{{Feedback}}

Developer HowTo

2018-10-29T13:16:24Z

Tierno:

Openstack configuration (Release THREE)

2018-09-25T11:21:00Z

Tierno:

__TOC__

=Configure openstack for OSM (basic)=

'''1. Guarantee that Openstack API endpoints are reachable from OSM (particularly from RO container)'''

'''2. Create a management network, with DHCP enabled, reachable from OSM (particularly from VCA container)'''

You need to create a management network, with DHCP enabled, and guarantee that this management network is reachable from OSM. The network is used by the VCA (Juju) for configuring the VNFs once they are running. It is recommended to create a provider network, isolated from Openstack. For instance, in order to create a provider network using physical interface em1 and VLAN 500 and with CIDR 10.208.0.0/24, you should run the following commands:
neutron net-create mgmt --provider:network_type=vlan --provider:physical_network=physnet_em1 --provider:segmentation_id=500 --shared
neutron subnet-create --name subnet-mgmt mgmt 10.208.0.0/24 --allocation-pool start=10.208.0.2,end=10.208.0.254

'''3. Create a valid tenant/user'''

You need to create a tenant/user with rights to create/delete flavors. The easiest way is to create a user and assign it the role "admin". Another option is to change the general flavor management policies at file ''/etc/nova/policy.json'' to allow flavor creation per user.

'''4. Upload images'''

OSM will create the needed images into openstack at deployment time, but the process takes a long time and the provided path at VNFD must be accesible by OSM. For this reason, it is recommended to upload the images of the VNFs manually at openstack, using the following command:

openstack image create --file="./cirros-0.3.4-x86_64-disk.img" --container-format=bare --disk-format=qcow2 --public cirros034

'''5. Modify default security group or create a new one'''

By default openstack apply the "default" security group that blocks any incoming traffic to the VM. However ssh access might be needed by VCA. You must modify the default security group to allow TCP port 22, or create a new security group and configure RO to use this security group when datacenter is addeed (see [[Openstack_configuration_(Release_TWO)#Add_openstack_to_OSM|Add openstack to OSM]] )

=Configure Openstack for full EPA support in OSM =

Besides the instructions above for any Openstack, you should do extra configuration to configure Openstack for running VNFs which use SRIOV interfaces.

'''DO NOT consider this as the final configuration guide for Openstack with EPA. Please check Openstack docs and Openstack downstream distros' docs to get a further understanding and the up-to-date required configuration.'''

'''Note: The configuration shown below works with Openstack Newton, and it might not work with later versions.'''

* The compute nodes need to have a whitelist for the interfaces with SRIOV and passthrough enabled, and those interface need to be associated to a physical network label e.g. “physnet”. This can be done in the file /etc/nova/nova.conf :
pci_passthrough_whitelist=[{"devname": "p3p1", "physical_network": "physnet"}, {"devname": "p3p2", "physical_network": "physnet"}]

* The neutron controller needs to be updated to add “sriovnicswitch” to the mechanism_drivers. This can be done in the file “/etc/neutron/plugins/ml2/ml2_conf.ini”
mechanism_drivers =openvswitch,sriovnicswitch

* The neutron controller needs to be updated to set the vlans to be used for the defined physical network label. This can be done in the file “/etc/neutron/plugins/ml2/ml2_conf.ini”. For instance, to set the vlans from 2000 to 3000:
network_vlan_ranges =physnet:2000:3000

* The neutron controller needs to be updated to allow the supported NIC vendor’s product ID. This can be done in the file “/etc/neutron/plugins/ml2/ml2_conf.ini” to
[ml2_sriov]
supported_pci_vendor_devs = 8086:10ed

* The nova controller needs to be updated to allow proper scheduling of SR-IOV and Passthrough devices, by adding the PciPassthroughFilter filter to the list of filters. This can be done in the file “/etc/nova/nova.conf”:
scheduler_available_filters=nova.scheduler.filters.all_filters
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,CoreFilter, PciPassthroughFilter

The previous configuration has been collected from the links below. Please check them to get more details:
* https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/network_functions_virtualization_product_guide/
* https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/network_functions_virtualization_planning_guide/
* https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/network_functions_virtualization_configuration_guide/
* https://docs.openstack.org/newton/networking-guide/config-sriov.html

=Add openstack to OSM=

There is a parameter called '''--config''' used to supply general configuration options. This paremeter can be used when creating the datacenter using the OSM client or when adding the datacenter directly to the RO (both at datacenter creation and datacenter attachment to openmano tenant -the latter prevails-).

The following configuration can be added:

* '''security_groups''': To be used for the deployment
* '''availability_zone''': To be used for the deployment. It can be:
** a single availability zone (all deployments will land in that zone), e.g. 'availability_zone: controller'
** several availability zones, which enables affinity and anti-affinity deployments, e.g. 'availability_zone: [zone1, zone2]'
* '''region_name''': The region where the VM must be deployed.
* '''insecure''': (By default false). When true it allows authorization over a non trusted certificate over https
* '''use_existing_flavors''': (By default false). Set to "True" to use the closer flavor with enough resources instead of creating a new flavor with the exact requirements. This option does not work for EPA (cpu pinning, huge pages, ...) where openmano still tries to create a flavor with the needed extra expects. Use this options when you do not have admin credentials (Available from future v2.0.2 version)
* '''vim_type''': Set to "VIO" to use VMware Integrated openstack as VIM
* '''use_internal_endpoint''': Set to True to force using internal endpoints
For openstack API v3. (It is needed to provide domain variables)

* '''APIversion''': (By default it used v3 if the keystone auth-url ends with "v3"). Set to "v3.3" or "3" to use this openstack API version
* '''project_domain_id''', '''project_domain_name''': If non of them are provided "default" is used for project_domain_id
* '''user_domain_id''', '''user_domain_name''': If non of them are provided "default" is used for user_domain_id

ADVANCED configuration:

* '''keypair''': To be added in addition to the keypair allocated on the VNF descriptor. Provide the name of an openstack keypair
* '''dataplane_physical_net''': The physical network label used in Openstack both to identify SRIOV and passthrough interfaces (nova configuration) and also to specify the VLAN ranges used by SR-IOV interfaces (neutron configuration). In case of VMware Integrated Openstack (VIO) provide moref ID of distributed virtual switch.
* '''use_floating_ip''': (By default false). If boolean True (note: not string 'True') a management interface of a VNFD is automatically assigned a floating_ip -if possible-. If a string is suplied instead of a boolean, it indicates the network/floating ip pool to use. The prefered method is to use a provider network
* '''dataplane_net_vlan_range''': In case of VMware Integrated Openstack (VIO) provide vlan ranges for the SRIOV (binding direct) networks in format ['start_ID - end_ID']
* '''microversion''': This is an Openstack-only parameter that allows to specify a specific microversion to be used in nova. When using 'microversion: 2.32', it enables the use of Virtual Device Role Tagging, which allows to identify each VM interface with a tag (the tag will be the name of the interface in the VNFD) and convey that information to the VM as metadata. This implementation approach is due to the warning message in https://developer.openstack.org/api-guide/compute/microversions.html where it is stated that microversion backwards compatibility is not guaranteed and clients should always require a specific microversion. This functionality was introduced would not work with Openstack versions previous to Newton.

The content of config is a yaml format text. The recomendation is to use a comma separated list between curly brackets {} and quotes. For instance, the following config is used to add a VMware Integrated Openstack where floating IP addresses will be assigned to VNF mgmt interfaces, all deployments must land in the availability_zone "controller", and the vlan ranges for the SRIOV networks will be 1-5 and 7-10:
--config='{use_floating_ip: True, availability_zone: zone1, dataplane_net_vlan_range: ["1-5" , "7-10"]}}'

==Using the OSM client==
osm vim-create --name openstack-site --user admin --password userpwd --auth_url http://10.10.10.11:5000/v2.0 --tenant admin --account_type openstack --config='{security_groups: default, keypair: mykey}'

==Adding directly in the RO==
openmano datacenter-create openstack-site http://10.10.10.11:5000/v2.0 --type openstack --description "OpenStack site" --config='{security_groups: default, keypair: mykey}'
openmano datacenter-attach openstack-site --user=admin --password=userpwd --vim-tenant-name=admin --config='{availability_zone: one}'

NOTE: In case of adding the datacenter to the RO, after doing it, go to the GUI:
ACCOUNTS > OSMOPENMANO > Click 'REFRESH STATUS' button

Developer HowTo

2018-08-31T11:47:09Z

Tierno:

Developer HowTo

2018-08-31T11:40:28Z

Tierno: /* RO */