Developer HowTo

From OSM Public Wiki
Revision as of 11:13, 28 August 2018 by Tierno (talk | contribs) (→‎NBI)
Jump to: navigation, search

The aim of this entry is to provide a guide for developers to set up their environment, in order to ease the development with the different OSM modules.

Introduction

Modules run in separate docker containers (except juju controller that uses a lxd container). For debugging it is convinient running the target module directly at host maintaining the rest of dependent modules in containers.

These are the modules with the port that they export, their relationships with the name of env variables that control them. For clarity the kafka dependency of lcm, nbi, mon and pm is not depicted. 

     __________
    |          | 
    | light-ui |OSM_SERVER               _______
    | :80      |----------------------> |       |
    |__________|                        | nbi   |
                     OSMNBI_STORAGE_PATH| :9999 |OSMNBI_DATABASE_HOST        _______ 
    .............. <--------------------|_______|-------------------------> |       |
    . volume:    .                                                          |       |
    . osm_osm_   .                                                          | mongo |
    . packages   .   OSMLCM_STORAGE_PATH _______ OSMLCM_DATABASE_HOST       | :27017|
    .............. <--------------------|       |-------------------------> |_______|
                                        | lcm   |
    **************       OSMLCM_VCA_HOST|       |OSMLCM_RO_HOST
    * lxd: juju  * <--------------------|_______|--------------|
    * controller *                                             |
    **************                       _______               |             _______
                                        |       | <-------------            |       |
                                        | ro    |                           | ro-db |  
                                        | :9090 |RO_DB_HOST                 | :3306 |
                                        |_______|-------------------------> |_______|

     _______     _______                 _______                             _________
    |       |   |       |               |       |                           |         |
    | mon   |   | pm    |               | kafka |KAFKA_ZOOKEEPER_CONNECT    |zookeeper|  
    | :8662 |   |       |               | :9092 |-------------------------> | :2181   |       
    |_______|   |_______|               |_______|                           |_________|

General steps

1 Shutdown the container you want to debug

First thing, you need to stop the module you want to debug. As OSM uses a docker service do not manually stop the container because it will be automatically relaunched again. Scale it to 0 for stopping and to 1 for running again. 

docker service scale osm_lcm=0

2 Clone the module

git clone https://osm.etsi.org/gerrit/osm/LCM

3 Install the module

Inside the folder where the module is cloned type the following command: 

pip3 install -e .  # try with --user if get permission errors


Note: pip3 can be installed with 'sudo apt-get install python3-pip'

Note: it is recomended pip3 version 9.0.3. Check version with 'pip3 --version' and fix it if needed with 'pip3 install pip==9.0.3'

4 Setup the IDE

For this tutorial we will use PyCharm as IDE. First thing, we will set "Python3" as default python interpreter:

PythonInterpreter.jpg

Next we will configure a new debug environment. For that we will go to the "Run" tab "Edit configurations". In the new window that appears we will need to configure the script and the environment parameters.

PyCharmConfiguration.jpg

5 Configure it to interact with other modules

You need to feed the IP addresses of the modules it is going to communicate to. For that, you can use your "/etc/hosts" file.

In case the module under development is running in the same server where the rest of the modules are located, use "127.0.0.1" (localhost) as the IP address of those modules. For instance, in the following example we have added mongo, ro and kafka to the line containing "127.0.0.1": 

127.0.0.1 localhost mongo ro kafka nbi ro-db

In case the module under development is running in a different server from the rest of modules, you will need to provide the IP address of that server. For instance, in the following example we have added a new line with the name resolution for mongo ro and kafka to IP address "a.b.c.d": 

a.b.c.d mongo ro kafka nbi ro-db

6 Install needed packages

Is it possible that you will need to install some additional packages in your server. If needed use the commands "pip3 install" or "apt-get install" for that.

Some modules imports another modules from OSM. The modules needed are: 

n2vc: git clone https://osm.etsi.org/gerrit/osm/N2VC
common: git clone https://osm.etsi.org/gerrit/osm/common
IM: git clone https://osm.etsi.org/gerrit/osm/IM

NBI

docker service scale osm_nbi=0
git clone https://osm.etsi.org/gerrit/osm/NBI
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e NBI  # try with --user if get permission errors

Python interpreter: Python3 Script: $INSTALLATION_FOLDER/NBI/osm_nbi/nbi.py

Environment variables:

  • OSMNBI_STORAGE_PATH: Path of the docker volume for filestorage. Both LCM and NBI must share the same path. You can either:
    • Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
    • Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant write permissions to pycharm on it with (path can be different at your envioron) 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes; sudo chmod -R o+w /var/lib/docker/volumes/osm_osm_packages/_data'
  • OSMNBI_DATABASE_HOST: Mongo IP in case host 'mongo' is not at "/etc/hosts" file

Create the vnfd_catalog.py and nsd_catalog.py files at NBI/osm_nbi folder from the yang OSM Information Model: 

# At parent folder where NBI is cloned
pip3 install pyang
git clone https://github.com/robshakir/pyangbind
pip3 install -e pyangbind
git clone  https://osm.etsi.org/gerrit/osm/IM
cd IM/models/yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../../NBI/osm_nbi/vnfd_catalog.py vnfd.yang
pyang --plugindir ../../../pyangbind/pyangbind/plugin -f pybind -o ../../../NBI/osm_nbi/nsd_catalog.py nsd.yang


OSM modules: 

git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common  # try with --user if get permission errors

LCM

docker service scale osm_lcm=0
git clone https://osm.etsi.org/gerrit/osm/LCM
cp LCM/.gitignore-common LCM/.gitignore
pip3 install -e LCM  # try with --user if get permission errors

Python interpreter: Python3 Script: $INSTALLATION_FOLDER/LCM/osm_lcm/lcm.py

Environment variables:

  • OSMLCM_STORAGE_PATH: Path of the docker volume for filestorage
    • Both LCM and NBI must share the same path. You can either:
      • Create a folder and debug both NBI and LCM at the same time (needed if you develop in a different server than OSM); or
      • Use the docker volume. Discover local path (Mountpoint) with 'docker volume inspect osm_osm_packages' and grant read permissions to pycharm on it with 'sudo chmod o+rx /var/lib/docker /var/lib/docker/volumes'
  • OSMLCM_DATABASE_HOST: Mongo IP in case 'mongo' host it is not at "/etc/hosts" file
  • OSMLCM_RO_HOST: RO IP in case 'ro' host it is not at "/etc/hosts" file
  • OSMLCM_VCA_SECRET: To get this value run the following command in the OSM host:
    • grep password /home/ubuntu/.local/share/juju/accounts.yaml |awk '{print $2}'
  • OSMLCM_VCA_HOST: Will be different depending on where your develop environment is running:
    • In case you run it in the same server as OSM use the following command to get the IP (<VCA_IP>):
      • juju show-controller|grep api-endpoints|awk -F\' '{print $2}'|awk -F\: '{print $1}'
    • In case you use a different server than OSM use the IP address of OSM host (<OSM_IP>). But in addition you need to redirect inside the OSM host, the port 17070 to the VCA container by one of:
      • Configure the following ip-table rule in OSM host (not persistent on reboot):
        • sudo iptables -t nat -A PREROUTING -p tcp -d <OSM_IP> --dport 17070 -j DNAT --to <VCA_IP>:17070
      • or creates a ssh tunnel inside OSM host (just temporal until session is closed):
        • ssh -L 0.0.0.0:17070:<VCA_IP>:17070 root@<VCA_IP>


OSM modules: 

git clone https://osm.etsi.org/gerrit/osm/N2VC
pip3 install -e N2VC  # try with --user if get permission errors

git clone https://osm.etsi.org/gerrit/osm/common
pip3 install -e common  # try with --user if get permission errors

RO

DRAFT: not tested

It is needed to edit the docker service in order to expose the database port for the 'ro-db' container. Edit file '/etc/osm/docker/docker-compose.yaml' (recomended to make a backup copy), and uncomment at 'ro-db' section the following two lines: 

   ports:
     - "3306:3306"

Additionally you may want that docker 'lcm' uses your local copy of RO. Edit again file '/etc/osm/docker/docker-compose.yaml', and set at 'lcm' the required IP address: 

     OSMLCM_RO_HOST: <127.17.0.1>    # ro
     # Check this address at your setup by 'ip a | grep docker0'


Then, relauch service: 

docker stack rm osm 
docker stack deploy -c /etc/osm/docker/docker-compose.yaml osm

Remove docker service, clone code and install needed packages: 

docker service scale osm_ro=0
git clone https://osm.etsi.org/gerrit/osm/RO
# This will install required packages. Do not use --no-db option for installing a local database instead of using 'ro-db' container
./RO/scripts/install-openmano.sh --no-db --noclone --develop

Python interpreter: Python2 Script: $INSTALLATION_FOLDER/RO/openmanod

Environment variables:

  • RO_DB_HOST, RO_DB_OVIM_HOST: 127.0.0.1 or <OSM_IP> depending if running on the same server or not. Not needed if alternativelly you use a local database copy.
Retrieved from "http:///wikipub/index.php?title=Developer_HowTo&oldid=3542"