|
|
# Install OSM using GitLab CICD
|
|
|
|
|
|
Using the CICD project as a testbed (https://osm.etsi.org/gitlab/devops/cicd), create a gitlab-ci.yml file that will run on an OpenStack VM to:
|
|
|
1. Install OSM
|
|
|
1. Run TST010 Robot Suite
|
|
|
1. Upload report as GitLab artifact
|
|
|
1. Store all OSM container logs in files
|
|
|
1. Upload OSM container logs as artifacts
|
|
|
|
|
|
## GitLab CI Overview
|
|
|
|
|
|
Reference: https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html
|
|
|
|
|
|
GitLab creates new environments for every job, which means that in order to accomplish this task, we will need to create one single job that performs all the steps. If we create an install job, followed by a run robot test job, the first VM will be destroyed before the second job start. Fortunately, GitLab provides some steps within a job that we can use to help keep things clean:
|
|
|
|
|
|
- `before_script`: this is run first, and probably the best place to perform the installation.
|
|
|
- `script`: this is the main part of the job, and probably the best place to run Robot. This will not run if the before_script encounters an error.
|
|
|
- `after_script`: the last step, which is always run, even if the previous steps fail. This step is probably the best place for collecting logs from K8s.
|
|
|
|
|
|
There is also an `artifacts` declaration, where we can say what files should be stored once the job completes. It takes either a directory, or a list of files. Probably something like this is best:
|
|
|
|
|
|
```
|
|
|
after_script:
|
|
|
- mkdir artifacts
|
|
|
- cp -r /robot-systest/conformance-tests/reports artifacts/
|
|
|
- kubectl logs -n osm nbi --timestamps=true 2>&1 > artifacts/nbi.log
|
|
|
|
|
|
artifacts:
|
|
|
paths:
|
|
|
- artifacts
|
|
|
```
|
|
|
|
|
|
## GitLab Runner on OpenStack
|
|
|
|
|
|
A minor issue was noted when trying to upload artifacts: the VM complained about not having the gitlab runner software installed. Therefore as part of the initial steps in the job, we will need to do the following:
|
|
|
```
|
|
|
before_script:
|
|
|
- curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
|
|
|
- sudo apt-get install gitlab-runner
|
|
|
```
|
|
|
|
|
|
## Notes
|
|
|
|
|
|
- The deploy stage is what we will use for the job
|
|
|
- Artifacts examples:
|
|
|
- QA: https://osmqa.etsi.org/gitlab/osm/MON/-/jobs/4307/artifacts/browse
|
|
|
- Prod: https://osm.etsi.org/gitlab/osm/mon/-/jobs/17206/artifacts/browse
|
|
|
- gitlab-ci.yml: https://osm.etsi.org/gitlab/devops/cicd/-/blob/master/templates/python-coverage.yml
|
|
|
- Stage 3 job that does similar things to what we want to do
|
|
|
- Install OSM: https://osm.etsi.org/gitlab/osm/devops/-/blob/master/jenkins/ci-pipelines/ci_stage_3.groovy#L457
|
|
|
- `-p http://172.21.1.1:5000` this means to use the Docker Cache at 172.21.1.1
|
|
|
- T - `tee` command - splits input into printing on stdout and putting a copy into the named file
|
|
|
- `2>&1` - redirect STDERR to STDOUT: https://stackoverflow.com/questions/876239/how-to-redirect-and-append-both-standard-output-and-standard-error-to-a-file-wit
|
|
|
- Extracting k8s container logs
|
|
|
- https://osm.etsi.org/gitlab/osm/devops/-/blob/master/jenkins/ci-pipelines/ci_stage_3.groovy#L125 (for K8s installer)
|
|
|
- https://osm.etsi.org/gitlab/osm/devops/-/blob/master/jenkins/ci-pipelines/ci_stage_3.groovy#L118 (for Charmed installer)
|
|
|
- Mark's sample merge request: https://osm.etsi.org/gitlab/-/ide/project/devops/cicd/tree/beierlm-master-patch-34527/-/.gitlab-ci.yml/
|
|
|
- Things to do
|
|
|
- import ssh id so we can ssh in, look for `Instance gitlab-builder-6-project-53-concurrent-0-job-17290 is running on address 172.21.249.237` to get the IP address
|
|
|
- sleep to keep the vm alive
|
|
|
- Mark to look into creating a snapshot of a VM that already has OSM installed on it, and show how to update the gitlab-ci.yml to use that image
|
|
|
- run robot test using docker: https://osm.etsi.org/gitlab/osm/devops/-/blob/master/jenkins/ci-pipelines/ci_stage_3.groovy#L94
|
|
|
|
|
|
```
|
|
|
docker run --env OSM_HOSTNAME=${osmHostname} --env PROMETHEUS_HOSTNAME=${prometheusHostname} ${PROMETHEUS_PORT_VAR} ${JUJU_PASSWORD_VAR} --env-file ${envfile} -v ${clouds}:/etc/openstack/clouds.yaml -v ${osmRSAfile}:/root/osm_id_rsa -v ${kubeconfig}:/root/.kube/config -v ${tempdir}:/robot-systest/reports -v ${portmappingfile}:/root/port-mapping.yaml ${hostfilemount} opensourcemano/tests:testing-daily -c -t ${testName}
|
|
|
```
|
|
|
- do not use port-mapping.yaml as it is only for OSM robot tests. |
|
|
\ No newline at end of file |