Developer HowTo: Difference between revisions

From OSM Public Wiki
Jump to: navigation, search
No edit summary
No edit summary
 
(38 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''THIS PAGE IS DEPRECATED'''. OSM Developer Guide has been moved to a new location: '''https://osm.etsi.org/docs/developer-guide/'''
---
The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.
==Introduction==
==Introduction==


Components run in a separate docker container (except juju controller that uses a lxd container). Normally you want to debug one component directly at host instead of on a container and use the rest of dependent containers.
Modules run in separate docker containers (except juju controller that uses a lxd container). Current installation uses Docker Swarm by default (there are Kubernetes distributions and an incoming option for installing on Kubernetes). The modules are:
 
* '''kafka''': Provides a Kafka bus used for OSM communication. This module relies on ''zookeeper''.
* '''zookeeper''': Used by ''kafka''.
* '''nbi''': North Bound Interface of OSM. Restful server that follows ETSI SOL005 interface. Relies on ''mongo'' database and ''kafka'' bus. For authentication it can optionally uses ''keystone''.
* '''keystone''': Used optionally for NBI authentication and RBAC. It stores the users, projects and role permissions. It relies on ''mysql''.
* '''lcm''': Provides the Live Cycle Management. It uses ''ro'' for resource orchestration and juju for configuration. It relies also on ''mongo''.
* '''ro''': Makes the Resource Orchestration, or VIM deployment. Relies on ''mysql''.
* '''light-ui''': Web user interface. It communicates with ''nbi''.
* '''mon''': Performs OSM monitoring. Relies on ''mongo'' and ''mysql''.
* '''mongo''': Common non relational database for OSM modules.
* '''mysql''': Relational database server used for ''ro'', ''keystone'', ''mon'' and ''pol''.
* '''pol''': Policy Manager for OSM.
* '''prometheus''': for monitoring .
 
 
In addition LCM and NBI, shares a common file system  where packages are stored. In current installation it uses a shared docker volume called ''osm_package''.


These are the components with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted.
This picture shows the modules with the port that they export, their relationships with the name of env variables that control them. For clarity not all the dependencies are depicted, as kafka, mysql, ...  


<pre>
<pre>
     __________
     __________                                                             ________
     |          |  
     |          |                                                            |        |
     | light-ui |OSM_SERVER              _______
     | light-ui |OSM_SERVER              _______                           |keystone|
     | :80      |----------------------> |      |
     | :80      |----------------------> |      |-------------------------> |:5000  |
     |__________|                        | nbi  |
     |__________|                        | nbi  |                          |________|
                     OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST        _______  
                     OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST        _______  
     .............. <--------------------|_______|-------------------------> |      |
     .............. <--------------------|_______|-------------------------> |      |
Line 24: Line 44:
     **************                      _______              |            _______
     **************                      _______              |            _______
                                         |      | <-------------            |      |
                                         |      | <-------------            |      |
                                         | ro    |                          | ro-db |   
                                         | ro    |                          | mysql |   
                                         | :9090 |RO_DB_HOST                | :3306 |
                                         | :9090 |RO_DB_HOST                | :3306 |
                                         |_______|-------------------------> |_______|
                                         |_______|-------------------------> |_______|
Line 35: Line 55:


</pre>
</pre>
 
For debugging it is convenient running the target module directly at host maintaining the rest of dependent modules in containers. In the following sections it is described how to achieve that.
 
==General steps==
==General steps==
===  1.- Shutdown the module you want to use===
===  1 Shutdown the container you want to debug===


First thing, you need to stop the module you want to develop. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.  
First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again.  


  docker service scale osm_lcm=0
  docker service scale osm_lcm=0


=== 2.- Clone the module===
=== 2 Clone the module===


  git clone https://osm.etsi.org/gerrit/osm/LCM
  git clone https://osm.etsi.org/gerrit/osm/LCM


=== 3.- Install the module===
=== 3 Install the module===


Inside the folder where the module is cloned type the following command:
Inside the folder where the module is cloned type the following command:


  pip3 install -e .
  pip3 install -e . # try with --user if get permission errors
 
 
'''Note''': pip3 can be installed with 'sudo apt-get install python3-pip'
 
'''Note''': it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'


=== 4.- Setup the IDE===
=== 4 Setup the IDE===


For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:
For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:
Line 63: Line 90:
[[File:PyCharmConfiguration.jpg|600px]]
[[File:PyCharmConfiguration.jpg|600px]]


=== 5.- Other modules addresses===
=== 5 Configure it to interact with other modules===


You need to provide your module the ip addresses of the modules it is going to communicate to. For that you can configure your "/etc/hosts" file.
You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.


In case you are using the same server where the rest of the modules are located, use "127.0.0.1" as the IP address of the modules:
In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1":


  127.0.0.1 localhost <vm name> mongo ro kafka
  127.0.0.1 localhost mongo ro kafka nbi ro-db


In case you use a different server, you need to provide the IP address of the host where OSM is running:
In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d":


  IP-OSM  mongo ro kafka
  a.b.c.d mongo ro kafka nbi ro-db


=== 6.- Install packages needed===
=== 6 Install needed packages===


Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.
Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.
Line 83: Line 110:
  common: git clone https://osm.etsi.org/gerrit/osm/common
  common: git clone https://osm.etsi.org/gerrit/osm/common
  IM: git clone https://osm.etsi.org/gerrit/osm/IM
  IM: git clone https://osm.etsi.org/gerrit/osm/IM
Install them with
pip3 install -e common --user  # needed for LCM, NBI
pip3 install -e N2VC --user  # needed for LCM
pip3 install -e IM --user  # needed for NBI
=== 7 Expose needed ports of docker services===
To expose '''mongo''' database (service osm_mongo), needed for NBI, LCM, MON:
docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list
Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack:
sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
#    ports:
#    - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section
Similarly, '''ro-db''' (service osm_ro-db) needed for RO, uses ports "3306:3306"
... And , '''kafka''' (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service


==NBI==
==NBI==


Install needed OSM packages IM, common:
# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common  # try with --user if get permission errors
# pip3 install aiokafka==0.4.*  pymongo==3.7.* pycrypto pyyaml==3.*
Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]
Additionally you may want that docker 'light-ui' uses your local copy of NBI by one of:
* Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:
OSM_SERVER: <172.17.0.1>    # nbi
# Get the needed address at your setup by 'ip a | grep docker0'
* ... Or update light-ui docker service (not persistent on reboot):
docker service update osm_light-ui --force --env-add "OSM_SERVER=172.17.0.1"
# Get the needed address at your setup by 'ip a | grep docker0'
Clone and install needed NBI packages
  docker service scale osm_nbi=0
  docker service scale osm_nbi=0
  git clone https://osm.etsi.org/gerrit/osm/NBI
  git clone https://osm.etsi.org/gerrit/osm/NBI
  pip3 install -e .
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
  pip3 install -e NBI  # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1


Python interpreter: Python3
Python interpreter: Python3
Line 94: Line 185:


Environment variables:
Environment variables:
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage
* OSMNBI_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/nbi.env'
** In case you develop in a different server than OSM, then it only works if LCM is also running in develop mode.
* OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
* OSMNBI_DATABASE_HOST: Mongo IP in case it is not configured in "/etc/hosts" file
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** In case you develop in the same server as OSM is installed, then it should be "127.0.0.1"
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
* OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
* OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public


OSM modules:
==LCM==
 
Install needed OSM packages if not done:
# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC  # try with --user if get permission errors
# osm_common:
  git clone https://osm.etsi.org/gerrit/osm/common
  git clone https://osm.etsi.org/gerrit/osm/common
  cd common
  pip3 install -e common  # try with --user if get permission errors
  pip3 install .


==LCM==
Change docker service '''mongo''' and '''kafka''' to expose ports. See [[Developer_HowTo#7 Expose needed ports of docker services]]


Cone and install LCM depencencies
  docker service scale osm_lcm=0
  docker service scale osm_lcm=0
  git clone https://osm.etsi.org/gerrit/osm/LCM
  git clone https://osm.etsi.org/gerrit/osm/LCM
  pip3 install -e .
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
  pip3 install -e LCM  # try with --user if get permission errors


Python interpreter: Python3
Python interpreter: Python3
Line 114: Line 218:


Environment variables:
Environment variables:
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage
 
** In case you develop in a different server than OSM, then it only works if NBI is also running in develop mode.
Values are stored at '/etc/osm/docker/lcm.env' but here it is explained how to obtain these values.
* OSMLCM_DATABASE_HOST: Mongo IP in case it is not configured in "/etc/hosts" file
* OSMLCM_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/lcm.env'
** In case you develop in the same server as OSM is installed, then it should be "127.0.0.1"
* OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
* OSMLCM_RO_HOST: RO IP in case it is not configured in "/etc/hosts" file
** Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
** In case you develop in the same server as OSM is installed, then it should be "127.0.0.1"
** Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
* OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See [[Developer_HowTo#5 Configure it to interact with other modules]]
* OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
* OSMLCM_VCA_CACERT: To get this value run the following command in the OSM host:
** juju controllers --format json | jq -r '.controllers["osm"]["ca-cert"]'
* OSMLCM_VCA_PUBKEY: To get this value run the following command in the OSM host:
** cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
* OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
** grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
* OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
** In case you run it in the same server as OSM use the following command:
** In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
*** juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
** In case you use a different server than OSM:
** In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
*** IP address from OSM
*** Configure the following ip-table rule in OSM host (not persistent on reboot):
*** Configure the following ip-table rule in OSM host:
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
**** sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
*** or creates a ssh tunnel inside OSM host (just temporal until session is closed):
**** ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
* OSMLCM_GLOBAL_LOGLEVEL: DEBUG


OSM modules:
==RO==
  git clone https://osm.etsi.org/gerrit/osm/N2VC
 
  cd N2VC
It is needed to edit the docker service in order to expose the database port for the 'mysql' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'mysql' section the following two lines:
pip3 install .
    ports:
      - "3306:3306"
git clone https://osm.etsi.org/gerrit/osm/common
Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file  '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM:
  cd common
      OSMLCM_RO_HOST: <172.17.0.1>    # ro
  pip3 install .
      # Get the needed address at your setup by 'ip a | grep docker0'
 
 
Then, relaunch service:
docker stack rm osm
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
 
Remove docker service, clone code:
 
docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
  git clone https://osm.etsi.org/gerrit/osm/RO
  curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
  cp RO/.gitignore-common RO/.gitignore
 
Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container:
  sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop
 
Python interpreter: Python2
Script: $INSTALLATION_FOLDER/RO/openmanod
 
Environment variables:
* RO_DB_HOST, RO_DB_OVIM_HOST: localhost or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
* RO_LOG_LEVEL: DEBUG

Latest revision as of 17:20, 17 February 2021

THIS PAGE IS DEPRECATED. OSM Developer Guide has been moved to a new location: https://osm.etsi.org/docs/developer-guide/

---

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.

Introduction

Modules run in separate docker containers (except juju controller that uses a lxd container). Current installation uses Docker Swarm by default (there are Kubernetes distributions and an incoming option for installing on Kubernetes). The modules are:

  • kafka: Provides a Kafka bus used for OSM communication. This module relies on zookeeper.
  • zookeeper: Used by kafka.
  • nbi: North Bound Interface of OSM. Restful server that follows ETSI SOL005 interface. Relies on mongo database and kafka bus. For authentication it can optionally uses keystone.
  • keystone: Used optionally for NBI authentication and RBAC. It stores the users, projects and role permissions. It relies on mysql.
  • lcm: Provides the Live Cycle Management. It uses ro for resource orchestration and juju for configuration. It relies also on mongo.
  • ro: Makes the Resource Orchestration, or VIM deployment. Relies on mysql.
  • light-ui: Web user interface. It communicates with nbi.
  • mon: Performs OSM monitoring. Relies on mongo and mysql.
  • mongo: Common non relational database for OSM modules.
  • mysql: Relational database server used for ro, keystone, mon and pol.
  • pol: Policy Manager for OSM.
  • prometheus: for monitoring .


In addition LCM and NBI, shares a common file system where packages are stored. In current installation it uses a shared docker volume called osm_package.

This picture shows the modules with the port that they export, their relationships with the name of env variables that control them. For clarity not all the dependencies are depicted, as kafka, mysql, ... 

     __________                                                              ________
    |          |                                                            |        |
    | light-ui |OSM_SERVER               _______                            |keystone|
    | :80      |----------------------> |       |-------------------------> |:5000   |
    |__________|                        | nbi   |                           |________|
                     OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST        _______ 
    .............. <--------------------|_______|-------------------------> |       |
    . volume:    .                                                          |       |
    . osm_osm_   .                                                          | mongo |
    . packages   .   OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST       | :27017|
    .............. <--------------------|       |-------------------------> |_______|
                                        | lcm   |
    **************       OSMLCM_VCA_HOST|       |OSMLCM_RO_HOST
    * lxd: juju  * <--------------------|_______|--------------|
    * controller *                                             |
    **************                       _______               |             _______
                                        |       | <-------------            |       |
                                        | ro    |                           | mysql |  
                                        | :9090 |RO_DB_HOST                 | :3306 |
                                        |_______|-------------------------> |_______|

     _______     _______                 _______                             _________
    |       |   |       |               |       |                           |         |
    | mon   |   | pm    |               | kafka |KAFKA_ZOOKEEPER_CONNECT    |zookeeper|  
    | :8662 |   |       |               | :9092 |-------------------------> | :2181   |       
    |_______|   |_______|               |_______|                           |_________|

For debugging it is convenient running the target module directly at host maintaining the rest of dependent modules in containers. In the following sections it is described how to achieve that.

General steps

1 Shutdown the container you want to debug

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again. 

docker service scale osm_lcm=0

2 Clone the module

git clone https://osm.etsi.org/gerrit/osm/LCM

3 Install the module

Inside the folder where the module is cloned type the following command: 

pip3 install -e .  # try with --user if get permission errors


Note: pip3 can be installed with 'sudo apt-get install python3-pip'

Note: it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

4 Setup the IDE

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

PythonInterpreter.jpg

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

PyCharmConfiguration.jpg

5 Configure it to interact with other modules

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1": 

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d": 

a.b.c.d mongo ro kafka nbi ro-db

6 Install needed packages

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are: 

n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

Install them with 

pip3 install -e common --user  # needed for LCM, NBI
pip3 install -e N2VC --user  # needed for LCM
pip3 install -e IM --user  # needed for NBI

7 Expose needed ports of docker services

To expose mongo database (service osm_mongo), needed for NBI, LCM, MON: 

docker service update osm_mongo --publish-add 27017:27017
#NOTE: Not sure if this is persistent upon reboot
# check exposed ports by:
docker service list

Alternativelly you can modify the docker osm stack by editing file '/etc/osm/docker/docker-compose.yaml', adding/uncommenting the exposed ports; and restart the stack: 

sudo vi /etc/osm/docker/docker-compose.yaml
# add/uncomment at section mongo:
#    ports:
#    - "27017:27017"
docker stack rm osm && sleep 60
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm
# make again the service scale to 0 of step 1 in this section

Similarly, ro-db (service osm_ro-db) needed for RO, uses ports "3306:3306"

... And , kafka (service osm_kafka) needed for LCM, NBI uses ports "9092:9092". Note: for kafka sometimes this is not enough and other modules has errrors connecting to kafka controller. If this is the case modify docker-compose adding kafka bus and relaunch service

NBI

Install needed OSM packages IM, common: 

# osm_im:
# Install Information Model vnfd.py and nsd.py files for pyangbind validation at e.g. parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone https://osm.etsi.org/gerrit/osm/IM
mkdir IM/osm_im
cd IM/models/yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-yang-types%402013-07-15.yang -O ./ietf-yang-types.yang
wget -q https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types%402013-07-15.yang -O ./ietf-inet-types.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/vnfd.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nsd.py nsd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../osm_im/nst.py nst.yang
cd ../../..
pip3 install -e IM
# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common  # try with --user if get permission errors
# pip3 install aiokafka==0.4.*  pymongo==3.7.* pycrypto pyyaml==3.*

Change docker service mongo and kafka to expose ports. See Developer_HowTo#7 Expose needed ports of docker services

Additionally you may want that docker 'light-ui' uses your local copy of NBI by one of:

  • Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'light-ui' section the required IP address from docker to the VM:
OSM_SERVER: <172.17.0.1>    # nbi
# Get the needed address at your setup by 'ip a | grep docker0'
  • ... Or update light-ui docker service (not persistent on reboot):
docker service update osm_light-ui --force --env-add "OSM_SERVER=172.17.0.1"
# Get the needed address at your setup by 'ip a | grep docker0'

Clone and install needed NBI packages 

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
# configure gerrit commit-msg hook
curl -Lo NBI/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x NBI/.git/hooks/commit-msg
cp NBI/.gitignore-common NBI/.gitignore
pip3 install -e NBI  # try with --user if get permission errors
# sudo apt-get install python3-keystoneclient
# pip3 install cherrypy==18.0.0 pyyaml==3.* keystoneauth1

Python interpreter: Python3 Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:

  • OSMNBI_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/nbi.env'
  • OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
    • Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
    • Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
  • OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file
  • OSMNBI_STATIC_DIR: <Absolute path of NBI>/osm_nbi/html_public

LCM

Install needed OSM packages if not done: 

# N2VC:
git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC  # try with --user if get permission errors

# osm_common:
git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common  # try with --user if get permission errors

Change docker service mongo and kafka to expose ports. See Developer_HowTo#7 Expose needed ports of docker services

Cone and install LCM depencencies 

docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
# configure gerrit commit-msg hook
curl -Lo LCM/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x LCM/.git/hooks/commit-msg
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM  # try with --user if get permission errors

Python interpreter: Python3 Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:

Values are stored at '/etc/osm/docker/lcm.env' but here it is explained how to obtain these values.

  • OSMLCM_DATABASE_COMMONKEY: must be the same value used by NBI. Get it with 'cat /etc/osm/docker/lcm.env'
  • OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
    • Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
    • Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
  • OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file. See Developer_HowTo#5 Configure it to interact with other modules
  • OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
  • OSMLCM_VCA_CACERT: To get this value run the following command in the OSM host:
    • juju controllers --format json | jq -r '.controllers["osm"]["ca-cert"]'
  • OSMLCM_VCA_PUBKEY: To get this value run the following command in the OSM host:
    • cat $HOME/.local/share/juju/ssh/juju_id_rsa.pub
  • OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
    • grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
  • OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
    • In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
      • juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
    • In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
      • Configure the following ip-table rule in OSM host (not persistent on reboot):
        • sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
      • or creates a ssh tunnel inside OSM host (just temporal until session is closed):
        • ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>
  • OSMLCM_GLOBAL_LOGLEVEL: DEBUG

RO

It is needed to edit the docker service in order to expose the database port for the 'mysql' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'mysql' section the following two lines: 

   ports:
     - "3306:3306"

Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' section the required IP address from docker to the VM: 

     OSMLCM_RO_HOST: <172.17.0.1>    # ro
     # Get the needed address at your setup by 'ip a | grep docker0'


Then, relaunch service: 

docker stack rm osm 
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code: 

docker service scale osm_ro=0
# clone and configure gerrit commit-msg hook
git clone https://osm.etsi.org/gerrit/osm/RO
curl -Lo RO/.git/hooks/commit-msg http://osm.etsi.org/gerrit/tools/hooks/commit-msg
chmod u+x RO/.git/hooks/commit-msg
cp RO/.gitignore-common RO/.gitignore

Install needed packages using its own installation script. Though it is not recomended, you can skip the '--no-db' option for installing a local database alternativelly to use the database at 'ro-db' container: 

sudo ./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2 Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:

  • RO_DB_HOST, RO_DB_OVIM_HOST: localhost or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
  • RO_LOG_LEVEL: DEBUG
Retrieved from "http:///wikipub/index.php?title=Developer_HowTo&oldid=6109"