From: ramonsalguer Date: Wed, 8 Apr 2020 17:44:07 +0000 (+0200) Subject: Initial contribution: Sandbox for Robot tests X-Git-Url: https://osm.etsi.org/gitweb/?a=commitdiff_plain;h=refs%2Fchanges%2F78%2F8778%2F1;p=osm%2Fsandboxes.git Initial contribution: Sandbox for Robot tests Change-Id: I8e49b402afcfc4fd92e94ac01ef5d99eb158f618 Signed-off-by: ramonsalguer --- diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ea9b9a1 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +*/.vagrant +*.log diff --git a/README.md b/README.md index 9f14b6f..6e4c37f 100644 --- a/README.md +++ b/README.md @@ -1,79 +1,62 @@ - -# Project Title - -One Paragraph of project description goes here +limitations under the License. -## Getting Started +## Goal -These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system. +The goal is adopting a methodology for local testing that is fully repeatable and well-known by the Depops MDL team in OSM. -### Prerequisites +This means that: -What things you need to install the software and how to install them +- Local testers should use (and rebuild regularly) the appropriate sandbox to perform tests and/or contributions to test plans, so that potential issues can be reliably diagnosed and solved. +- This means that all known changes in the installation procedures of any sandbox will be captured in the companion "provisioner" script (typically, `provisioner.sh`), so that the exact procedure can be well-known and testable by everyone. +- In case something that was known to work in a sandbox started to fail: + - It should be reported as a bug if there were no further contributions to the sandbox, since it is likely to be a gap in the knowledge/testing of the corresponding provisioning procedure, that might have changed over time (e.g new dependencies, change of repositories, hardcoded locations, etc.) + - In case new contributions (e.g. new tests, new patches, etc.) were not functional in the corresponding sandbox, the contributor should review them locally first, since they are likely to have issues. -``` -Give examples -``` +## Pre-requirements -### Installing +- VirtualBox and Vagrant locally installed. +- Host properly configured to use SSH and Git with OSM. + - NOTE: In the case of a Windows host, this means: + - Having your SSH keys (public and private in the `%userprofile%/.ssh` folder (the one equivalent to `~/.ssh` in Linux/Mac), + - Having Git installed and properly configured in Windows, i.e. check that `%userprofile%/.gitconfig` exists and has the proper configuration. + - In particular, it is required that the `user.name` there contains your EOL account name (i.e. avoid using a full name there). Beware that only the global profile is supported. -A step by step series of examples that tell you how to get a development env running +## Basic use of Vagrant -Say what the step will be +To build a sandbox from scratch: +```bash +cd folder_of_the_sandbox +vagrant up ``` -Give the example -``` - -And repeat - -``` -until finished -``` - -End with an example of getting some data out of the system or using it for a little demo -## Running the tests +NOTE: All the examples that follow are suppossed to be run in the same sandbox's folder. -Explain how to run the automated tests for this system +To access the VM via SSH: +```bash +vagrant ssh ``` -Give an example -``` - -## Deployment - -Add additional notes about how to deploy this on a live system - -## Built With - -* [Python](www.python.org/) - The language used -## Contributing +To destroy the VM: -Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us. - -## Versioning - -We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://osm.etsi.org/gitweb/?p=osm/sandboxes.git;a=tags). - -## License +```bash +vagrant destroy +``` -This project is licensed under the Apache2 License - see the [LICENSE.md](LICENSE) file for details +To rebuild and reprovision the VM: -## Acknowledgments +```bash +vagrant destroy && vagrant up +``` diff --git a/osm-tests/.gitignore b/osm-tests/.gitignore new file mode 100644 index 0000000..9fae18b --- /dev/null +++ b/osm-tests/.gitignore @@ -0,0 +1,13 @@ +packages/*.gz +packages/*.tgz +packages/*.yaml +packages/*.yml +images/*.qcow +images/*.qcow2 +images/*.img +images/*.raw +results/*.html +results/*.xml +localconfig/*.rc +localconfig/*.sh +!localconfig/TEMPLATE* diff --git a/osm-tests/README.md b/osm-tests/README.md new file mode 100644 index 0000000..39ed936 --- /dev/null +++ b/osm-tests/README.md @@ -0,0 +1,208 @@ +# README: Vagrant sandbox for Robot tests + +Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +## HOW TO USE IT (TL;DR) + +1. Prepare your local environment: + 1. Upload all required images to your VIM (e.g. `openstack image create ...`). + - Hint: you can download first all relevant testing images from OSM by: + ```bash + cd images + ./download_images.sh + ``` + 2. Add the corresponding VIM target to your OSM (`osm vim-create ...`). + 3. Copy all packages (NF, NS, NSI) required for the tests to the `packages/` folder. Hint: + ```bash + cd packages + ./download_packages.sh + ``` +2. Provide credentials for your VIM+OSM environment: + - Create/Edit `localconfig/envconfig.rc` (and `localconfig/envprovisioning.sh` if needed). + - HINT: First time, use `localconfig/TEMPLATE-envconfig.rc` and (if needed) `localconfig/TEMPLATE-envprovisioning.sh` as templates. +3. (OPTIONAL) In case you needed to apply a specific Devops patch-set, define it at `baseconfig/patchconfig.rc`. +4. Start the sandbox for its first time provisioning with `vagrant up`. +5. Do `vagrant ssh` to enter in the VM. +6. Run your first test: + ```bash + cd robot + robot -d /vagrant/results testsuite/hackfest_basic.robot + ``` + +## INTRODUCTION + +This Vagrant sandbox provides a simple self-contained environment suitable for running consistently E2E tests with OSM. + +Using this sandbox for local E2E testing has a number of advantages: + +- It enables the end user to run and validate easily in your local environment the set of Robot tests available in ETSI OSM and enables you to contribute back more easily. +- It provides a well-known software environment, with all the required dependencies properly pre-validated. +- It guarantees that there are no hardcoded dependencies with the local tester environment or with OSM's CI/CD in terms of versions, destinations, paths, credentials, etc. All the test conditions can be completely defined using environment variables if needed. +- Its provisioner script (`provisioner.sh`) can be used as-is for provisioning other types of environments, such as a Docker containers. +- Likewise, the provisioner script can serve as documentation/validation of different installation processes maintained by OSM (client, Robot, etc.). + +Here is the sandbox what Vagrant creates for you: + +![Diagram of the functionality of the Sandbox](assets/image-20200406234414818.png) + +As it can be seen, the provisioning of the sandbox installs several pieces of software: + +- OSM Client. +- Robot Framework and all library dependencies required to run the full test suite. +- Other complementary utilities to perform the tests or ease troubleshooting. + +In addition, the provision of the sandbox requires a number of inputs: + +- The collection of E2E Robot tests for OSM. +- Details of OSM instance(s) available for testing, part of the DUT. +- Details of VIM(s) available for testing, part of the DUT. +- Location of VNF and NS packages required to run the tests. +- Location of VM images required to run the tests (future use). + +## PRE-REQUIREMENTS + +### Available infrastructure (DUT) + +- There is at least one VIM available and reachable. +- There is at least one OSM instance available and reachable. +- The OSM instance(s) has (have) at least one target added per VIM. + +### Software pre-requirements already covered by Vagrant + +- `authorized_keys` at `~/.ssh` with proper permissions. +- `id_rsa`, `id_rsa.pub` at `~/.ssh` with proper permissions. +- A functional `.gitconfig` file at `~` with proper permissions. +- Base configuration files at `${BASE_CONFIG_FOLDER}` (for Vagrant, `~/baseconfig`): + - `seedconfig.rc`, where most basic configuration is made. + - `patchconfig.rc`, where a Gerrit patch-set for Devops is selected. This file is optional. +- Local environment configuration files at `${LOCAL_CONFIG_FOLDER}` (for Vagrant, `~/localconfig`) + - `envprovisioning.sh`, where local provisioning tasks are scripted. This file is optional. + - `envconfig.rc`, where the DUT (VIM and OSM) are defined with environment variables. +- All packages required for the tests are already placed at `${PACKAGES_FOLDER}`. + - In Vagrant, `/vagrant/packages`. +- All images required for the tests are already placed at `${IMAGES_FOLDER}`. + - In Vagrant, `/vagrant/images`. + - This folder is intended for future use, but not relevant for the current test suite. + +### Software pre-requirements to be fulfilled by the user + +Users **should edit** the information about the DUT (local VIM and available OSM) before provisioning the sandbox. + +In the case of Vagrant, the `localconfig` sub-folder of the Vagrant sandbox definition (i.e. at the host) contains the relevant files to be edited: + +1. `envprovisioning.sh` is used as "provisioner" to retrieve some the local environment data, i.e. it is a script intended to retrieve information from some well-known locations, such as local Git repos, that will be referred lated by `envconfig.rc`. +2. `envconfig.rc` contains the minimal set of environment variables required to run successfully the suite of Robot tests. + +Once the vagrant sandbox is created, these files will be automatically copied into the VM to `${LOCAL_CONFIG_FOLDER}`) so that they can complete the configuration of the test environment. + +## Configuration setup + +### Tester setup and test suite configuration + +Setup of testing conditions, independent from the local environment. This configuration is defined in two configuration files, stored at `${BASE_CONFIG_FOLDER}` (for Vagrant, `~/baseconfig`). + +#### Base configuration: `seedconfig.rc` + +This file contains common base configuration for the test suite, independent of the environment. + +This repo provides a configuration customized for Vagrant sandboxes. If that is your case, please do not edit. However, the existing file can be used as template for creating a `seedconfig.rc` for other types of environments (e.g. Docker). + +Hare is the sample of `seedconfig.rc` for Vagrant with the environment variables commented: + +```bash +# Base folder for all operations +export BASE_FOLDER=/home/vagrant +# EOL account for SSH operations. Required for password-less operations with Git. +# Comment to undefine and it will use HTTP instead +export ETSIUSERNAME=$(git config user.name) +# Folder where Robot tests are stored +export ROBOT_DEVOPS_FOLDER="${HOME}/devops/robot-systest" +# Folder to save alternative DUT environments (optional) +export ENVIRONMENTS_FOLDER=environments +# Folder where all required packages are stored +export PACKAGES_FOLDER="/vagrant/packages" +# Folder where all required images are stored (intended for future use) +#export IMAGES_FOLDER="/vagrant/images" +# Folder where test results should be exported +export ROBOT_REPORT_FOLDER="/vagrant/results" +``` + +#### Devops patch configuration (optional): `patchconfig.rc` + +This file, when exists, defines an optional Gerrit patch-set to be applied over the current Devops master. + +This configuration is optional and mostly intended for OSM developers. Do not use if you plan to run community-approved test suites. + +Here is the sample of `patchconfig.rc` with the environment variable commented: + +```bash +# URL to be used in a subsequent `pull` command to apply the patchset +export DEVOPS_PATCH=refs/changes/51/8751/1 +``` + +### DUT configuration (local environment) + +Setup of the local environment for the tests. This configuration is defined in configuration files stored at `${LOCAL_CONFIG_FOLDER}` (for Vagrant, `~/localconfig`). + +#### Local environment provisioning (optional): `envprovisioning.sh` + +This file, which is optional, is intended to be used as a script to retrieve information about the local environment, which might include cloning repositories, downloading credential files, etc. Typically, this is used to fetch sensitive information. There are two typical cases: + +- `openstack.rc` + - The provisioner, besides fetching it, would append the info to `${LOCAL_CONFIG_FOLDER}/envconfig.rc`. +- `clouds.yaml` + - The provisioner, besides fetching it, should copy the file to `${BASE_FOLDER}`. + +Here is an example of `envprovisioning.sh` with relevant operations commented: + +```bash +ssh-keyscan mygit.com >> ~/.ssh/known_hosts # Often needed for non public repos +git clone git@mygit.com:local-environment-data/vim-and-infra.git +cp vim-and-infra/openstack/clouds.yaml "${BASE_FOLDER}"/ # Copy of credentials to base folder +``` + +#### Local environment configuration: `envconfig.rc` + +This file, which is **mandatory**, is where the local DUT is completely defined with environment variables. + +```bash +# VIM setup +export OS_USERNAME=username +export OS_PASSWORD=password +export OS_TENANT_NAME=projectName +export OS_AUTH_URL=https://identityHost:portNumber/v2.0 +export OS_TENANT_ID=tenantIDString +export OS_REGION_NAME=regionName +export OS_CACERT=/path/to/cacertFile +# OSM location +export OSM_HOSTNAME="192.168.1.22" +# Details of the VIM target for OSM +export VIM_TARGET="vim002-fortville-physnet" +export VIM_MGMT_NET="management" +# Optional: location of alternative `envconfig.rc`-like files +export ENVIRONMENTS_FOLDER=environments +``` + +Alternatively, here is another example `envconfig.rc` where the VIM is defined with a credentials file, `clouds.yaml`: + +```bash +# VIM setup +export CLOUDS_PATH=${BASE_FOLDER} # Location of `clouds.yaml` +export OS_CLOUD=vim-002 +# OSM location +export OSM_HOSTNAME="192.168.1.22" +# Details of the VIM target for OSM +export VIM_TARGET="vim002-fortville-physnet" +export VIM_MGMT_NET="management" +# Optional: location of alternative `envconfig.rc`-like files +export ENVIRONMENTS_FOLDER=environments +``` diff --git a/osm-tests/Vagrantfile b/osm-tests/Vagrantfile new file mode 100644 index 0000000..ff4e717 --- /dev/null +++ b/osm-tests/Vagrantfile @@ -0,0 +1,112 @@ +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# -*- mode: ruby -*- +# vi: set ft=ruby : + +# All Vagrant configuration is done below. The "2" in Vagrant.configure +# configures the configuration version (we support older styles for +# backwards compatibility). Please don't change it unless you know what +# you're doing. +Vagrant.configure("2") do |config| + # The most common configuration options are documented and commented below. + # For a complete reference, please see the online documentation at + # https://docs.vagrantup.com. + + # Every Vagrant development environment requires a box. You can search for + # boxes at https://vagrantcloud.com/search. + config.vm.box = "ubuntu/bionic64" + + # Disable automatic box update checking. If you disable this, then + # boxes will only be checked for updates when the user runs + # `vagrant box outdated`. This is not recommended. + # config.vm.box_check_update = false + + # Create a forwarded port mapping which allows access to a specific port + # within the machine from a port on the host machine. In the example below, + # accessing "localhost:8080" will access port 80 on the guest machine. + # NOTE: This will enable public access to the opened port + # config.vm.network "forwarded_port", guest: 80, host: 8080 + + # Create a forwarded port mapping which allows access to a specific port + # within the machine from a port on the host machine and only allow access + # via 127.0.0.1 to disable public access + # config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1" + + # Create a private network, which allows host-only access to the machine + # using a specific IP. + # config.vm.network "private_network", ip: "192.168.33.10" + + # Create a public network, which generally matched to bridged network. + # Bridged networks make the machine appear as another physical device on + # your network. + # config.vm.network "public_network" + + # Share an additional folder to the guest VM. The first argument is + # the path on the host to the actual folder. The second argument is + # the path on the guest to mount the folder. And the optional third + # argument is a set of non-required options. + # config.vm.synced_folder "../data", "/vagrant_data" + + # Provider-specific configuration so you can fine-tune various + # backing providers for Vagrant. These expose provider-specific options. + # Example for VirtualBox: + # + # config.vm.provider "virtualbox" do |vb| + # # Display the VirtualBox GUI when booting the machine + # vb.gui = true + # + # # Customize the amount of memory on the VM: + # vb.memory = "1024" + # end + # + # View the documentation for the provider you are using for more + # information on available options. + + # Enable provisioning with a shell script. Additional provisioners such as + # Ansible, Chef, Docker, Puppet and Salt are also available. Please see the + # documentation for more information about their specific syntax and use. + # config.vm.provision "shell", inline: <<-SHELL + # apt-get update + # apt-get install -y apache2 + # SHELL + + # Feeds some configuration from the host to `vagrant` user in VM: + # - Private key + # - Public key + # - Public key in `authorized_keys` + # - Git configuration + #-------------------------------------------------------------- + config.vm.provision "shell", privileged: false, inline: <<-SHELL + echo -e '#{File.read("#{Dir.home}/.ssh/id_rsa")}' > '/home/vagrant/.ssh/id_rsa' + echo -e '#{File.read("#{Dir.home}/.ssh/id_rsa.pub")}' > '/home/vagrant/.ssh/id_rsa.pub' + echo -e '#{File.read("#{Dir.home}/.ssh/id_rsa.pub")}' >> '/home/vagrant/.ssh/authorized_keys' + echo -e '#{File.read("#{Dir.home}/.gitconfig")}' > '/home/vagrant/.gitconfig' + chmod 600 /home/vagrant/.ssh/id_rsa /home/vagrant/.ssh/id_rsa.pub /home/vagrant/.gitconfig + SHELL + + # Feeds config files for setting up Robot tests and local environment into the VM + config.vm.provision "shell", privileged: false, inline: <<-SHELL + sudo apt-get update -y + sudo apt-get install dos2unix + cp -r /vagrant/baseconfig /home/vagrant/ + cp -r /vagrant/localconfig /home/vagrant/ + dos2unix /home/vagrant/baseconfig/*.rc + dos2unix /home/vagrant/localconfig/*.rc + dos2unix /home/vagrant/localconfig/*.sh + SHELL + + # Performs the provision of the Vagrant Sandbox for Robot tests + config.vm.provision :shell, privileged: false, path: "provisioner.sh" +end diff --git a/osm-tests/assets/2020406 Vagrant sandbox for Robot.pptx b/osm-tests/assets/2020406 Vagrant sandbox for Robot.pptx new file mode 100644 index 0000000..298e3cd Binary files /dev/null and b/osm-tests/assets/2020406 Vagrant sandbox for Robot.pptx differ diff --git a/osm-tests/assets/2020406 Vagrant sandbox for Robot.svg b/osm-tests/assets/2020406 Vagrant sandbox for Robot.svg new file mode 100644 index 0000000..5a75e88 --- /dev/null +++ b/osm-tests/assets/2020406 Vagrant sandbox for Robot.svg @@ -0,0 +1 @@ +OSM ClientRobot FrameworkOther utilities (OpenStack client, yq, etc.)Robot tests for OSMVM / Docker / LXCOSM instance detailsVNF and NS Packages(location)VIM credentialsVM images(location)DUT \ No newline at end of file diff --git a/osm-tests/assets/image-20200406234414818.png b/osm-tests/assets/image-20200406234414818.png new file mode 100644 index 0000000..58811f1 Binary files /dev/null and b/osm-tests/assets/image-20200406234414818.png differ diff --git a/osm-tests/baseconfig/.void b/osm-tests/baseconfig/.void new file mode 100644 index 0000000..e69de29 diff --git a/osm-tests/baseconfig/patchconfig.rc b/osm-tests/baseconfig/patchconfig.rc new file mode 100644 index 0000000..46787cf --- /dev/null +++ b/osm-tests/baseconfig/patchconfig.rc @@ -0,0 +1,21 @@ +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +####################################################################################################### +# patchconfig.rc +####################################################################################################### +# (OPTIONAL) Additional configuration to add optional Devops patches (only if needed) + +# URL to be used in a subsequent `pull` command to apply the patchset +#export DEVOPS_PATCH=refs/changes/51/8751/7 diff --git a/osm-tests/baseconfig/seedconfig.rc b/osm-tests/baseconfig/seedconfig.rc new file mode 100644 index 0000000..7fcf884 --- /dev/null +++ b/osm-tests/baseconfig/seedconfig.rc @@ -0,0 +1,42 @@ +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +####################################################################################################### +# seedconfig.rc: COMMON BASE CONFIGURATION FOR THE ENVIRONMENT +####################################################################################################### +# This configuration is customized for Vagrant sandboxes. If that is your case, please do not edit. +# This file can be used as template for creating a `seedconfig.rc` +# for other types of environments (e.g. Docker) + +# Base folder for all operations +export BASE_FOLDER=/home/vagrant + +# EOL account for SSH operations. Required for password-less operations with Git. +# Comment to undefine and it will use HTTP instead +export ETSIUSERNAME=$(git config user.name) + +# Folder where Robot tests are stored +export ROBOT_DEVOPS_FOLDER="${HOME}/devops/robot-systest" + +# Folder to save alternative DUT environments (optional) +export ENVIRONMENTS_FOLDER=environments + +# Folder where all required packages are stored +export PACKAGES_FOLDER="/vagrant/packages" + +# Folder where all required images are stored (intended for future use) +export IMAGES_FOLDER="/vagrant/images" + +# Folder where test results should be exported +export ROBOT_REPORT_FOLDER="/vagrant/results" diff --git a/osm-tests/images/.void b/osm-tests/images/.void new file mode 100644 index 0000000..e69de29 diff --git a/osm-tests/images/download_images.sh b/osm-tests/images/download_images.sh new file mode 100644 index 0000000..a6dfc0d --- /dev/null +++ b/osm-tests/images/download_images.sh @@ -0,0 +1,16 @@ +#!/bin/bash +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +wget -nd -r -P . -A qcow,qcow2,img,raw https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/ diff --git a/osm-tests/localconfig/.void b/osm-tests/localconfig/.void new file mode 100644 index 0000000..e69de29 diff --git a/osm-tests/localconfig/TEMPLATE-envconfig.rc b/osm-tests/localconfig/TEMPLATE-envconfig.rc new file mode 100644 index 0000000..ae215ff --- /dev/null +++ b/osm-tests/localconfig/TEMPLATE-envconfig.rc @@ -0,0 +1,34 @@ +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +####################################################################################################### +# envconfig.rc +####################################################################################################### +# Local environment configuration: VIM(s), OSM(s), etc. + +# VIM setup +export OS_USERNAME=username +export OS_PASSWORD=password +export OS_TENANT_NAME=projectName +export OS_AUTH_URL=https://identityHost:portNumber/v2.0 +export OS_TENANT_ID=tenantIDString +export OS_REGION_NAME=regionName +export OS_CACERT=/path/to/cacertFile # Only if required for the VIM +# OSM location +export OSM_HOSTNAME="192.168.1.22" +# Details of the VIM target for OSM +export VIM_TARGET="vim002-physnet1" +export VIM_MGMT_NET="management" +# Optional: location of other alternative `envconfig.rc`-like files +export ENVIRONMENTS_FOLDER=environments diff --git a/osm-tests/localconfig/TEMPLATE-envprovisioning.sh b/osm-tests/localconfig/TEMPLATE-envprovisioning.sh new file mode 100644 index 0000000..c2d67d7 --- /dev/null +++ b/osm-tests/localconfig/TEMPLATE-envprovisioning.sh @@ -0,0 +1,22 @@ +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +####################################################################################################### +# envprovisioning.sh +####################################################################################################### +# (OPTIONAL) Local environment provisioning (e.g. cloning of local repos and credentials) + +ssh-keyscan mylocalgitserver.com >> ~/.ssh/known_hosts # Often needed for non public repos +git clone git@mylocalgitserver.com:local-environment-data/local-infra-info.git +cp local-infra-info/openstack/clouds.yaml "${BASE_FOLDER}"/ # Copy credentials to base folder diff --git a/osm-tests/packages/.void b/osm-tests/packages/.void new file mode 100644 index 0000000..e69de29 diff --git a/osm-tests/packages/download_packages.sh b/osm-tests/packages/download_packages.sh new file mode 100644 index 0000000..946ceb9 --- /dev/null +++ b/osm-tests/packages/download_packages.sh @@ -0,0 +1,16 @@ +#!/bin/bash +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +wget -nd -r -P . -A gz,tgz,yaml,yml -R *qcow*,*fedora20* https://osm-download.etsi.org/ftp/osm-5.0-five/6th-hackfest/packages/ diff --git a/osm-tests/provisioner.sh b/osm-tests/provisioner.sh new file mode 100644 index 0000000..650d366 --- /dev/null +++ b/osm-tests/provisioner.sh @@ -0,0 +1,135 @@ +#!/usr/bin/env bash + +# Copyright 2020 Telefónica Investigación y Desarrollo S.A.U. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +####################################################################################################### +# PRE-REQUIREMENTS FOR THE ENVIRONMENT: +####################################################################################################### +# - There is at least one VIM available and reachable. +# - There is at least one OSM instance available and reachable. +# - The OSM instance(s) has (have) already at least one target added per VIM. + +####################################################################################################### +# SOFTWARE PRE-REQUIREMENTS: (already covered for Vagrant) +####################################################################################################### +# - `authorized_keys` at `~/.ssh` with proper permissions +# - `id_rsa`, `id_rsa.pub` at `~/.ssh` with proper permissions +# - A functional `.gitconfig` file at `~` with proper permissions +# - `seedconfig.rc` and `patchconfig.rc` copied to `~/baseconfig` +# - `envprovisioning.sh` and `envconfig.rc` copied to `~/localconfig` + +####################################################################################################### +# CONFIGURATION SEEDING +####################################################################################################### + +# Folders where configuration is stored +BASE_CONFIG_FOLDER=baseconfig +LOCAL_CONFIG_FOLDER=localconfig # Default path. It can be reset dinamically by `seedconfig.rc` or `envprovisioning.sh` if needed + +# Base configuration +if [ -f ${BASE_CONFIG_FOLDER}/seedconfig.rc ] +then + cat ${BASE_CONFIG_FOLDER}/seedconfig.rc >> defaultenv.rc + source ${BASE_CONFIG_FOLDER}/seedconfig.rc +else + >&2 echo ################################################################################ + >&2 echo ERROR: Base configuration file ${BASE_CONFIG_FOLDER}/seedconfig.rc is missing. + >&2 echo Please check README.md for details. + >&2 echo Once fixed, rebuild your environment. E.g. for Vagrant: + >&2 echo vagrant destroy && vagrant up + >&2 echo ################################################################################ + exit 1 +fi + +# (OPTIONAL) Devops patch configuration +if [ -f ${BASE_CONFIG_FOLDER}/patchconfig.rc ] +then + cat ${BASE_CONFIG_FOLDER}/patchconfig.rc >> defaultenv.rc + source ${BASE_CONFIG_FOLDER}/patchconfig.rc +fi + +# (OPTIONAL) Local environment provisioning (e.g. cloning of local repos) +if [ -f ${LOCAL_CONFIG_FOLDER}/envprovisioning.sh ] +then + source ${LOCAL_CONFIG_FOLDER}/envprovisioning.sh +fi + +# Local environment configuration: VIM(s), OSM(s), credentials, etc. +if [ -f ${LOCAL_CONFIG_FOLDER}/envconfig.rc ] +then + cat ${LOCAL_CONFIG_FOLDER}/envconfig.rc >> defaultenv.rc + source ${LOCAL_CONFIG_FOLDER}/envconfig.rc +else + >&2 echo ################################################################################ + >&2 echo WARNING: Local configuration file ${BASE_CONFIG_FOLDER}/envconfig.rc is missing. + >&2 echo Please check README.md for details. + >&2 echo If it is an error, once fixed, rebuild your environment. E.g. for Vagrant: + >&2 echo vagrant destroy && vagrant up + >&2 echo Otherwise, you should add manually the appropriate environment variables later. + >&2 echo ################################################################################ +fi + +#------------------------------------------------------------------------------------------------------ + +# Installs OSM client +sudo sed -i "/osm-download.etsi.org/d" /etc/apt/sources.list +wget -qO - https://osm-download.etsi.org/repository/osm/debian/ReleaseSEVEN/OSM%20ETSI%20Release%20Key.gpg | sudo apt-key add - +sudo add-apt-repository -y "deb [arch=amd64] https://osm-download.etsi.org/repository/osm/debian/ReleaseSEVEN stable devops IM osmclient" +sudo apt-get update +sudo apt-get install -y python3-pip +sudo -H python3 -m pip install python-magic pyangbind verboselogs +sudo apt-get install -y python3-osmclient + +# Installs OpenStack client +##For Train version, uncomment the following two lines: +##sudo add-apt-repository -y cloud-archive:train +##sudo apt-get update +sudo apt-get install -y python3-openstackclient # Installs Queens by default + +# Installs Robot and all dependencies required for the tests + +sudo -H python3 -m pip install --ignore-installed haikunator requests pyvcloud progressbar pathlib robotframework robotframework-seleniumlibrary robotframework-requests robotframework-SSHLibrary +curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - +sudo add-apt-repository -y "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" +sudo apt-get install -y google-chrome-stable chromium-chromedriver +ln -s ${ROBOT_DEVOPS_FOLDER} robot + +# Clones Devops repo to retrieve all Robot tests from OSM community +ssh-keyscan -p 29418 osm.etsi.org >> ~/.ssh/known_hosts +if [ -n "${ETSIUSERNAME}" ] # If possible, uses ETSI's user name to make further contributions easier +then + git clone "ssh://${ETSIUSERNAME}@osm.etsi.org:29418/osm/devops" && \ + (cd "devops" && curl https://osm.etsi.org/gerrit/tools/hooks/commit-msg > .git/hooks/commit-msg ; chmod +x .git/hooks/commit-msg) +else + git clone "https://osm.etsi.org/gerrit/osm/devops" +fi + +# if applicable, adds additional patches to devops repo (refer to `patchconfig.rc`) +[ -n "${DEVOPS_PATCH}" ] && git -C devops pull https://osm.etsi.org/gerrit/osm/devops ${DEVOPS_PATCH} + +# Installs some additional packages to ease interactive troubleshooting +sudo apt-get install -y osm-devops +sudo snap install charm --classic +sudo snap install yq + +# Copies VIM credentials in `clouds.yaml` (if applicable) to a proper location +if [ -f ${CLOUDS_PATH}/clouds.yaml ]; then + sudo mkdir -p /etc/openstack + sudo cp ${CLOUDS_PATH}/clouds.yaml /etc/openstack/ + rm ${CLOUDS_PATH}/clouds.yaml +fi + +# Sets default environment to load automatically in `.bashrc` +cat defaultenv.rc >> ~/.bashrc diff --git a/osm-tests/results/.void b/osm-tests/results/.void new file mode 100644 index 0000000..e69de29