Publication of Charms in CI/CD Pipeline
Proposers
- Mark Beierl (Canonical)
- Guillermo Calviño (Canonical)
Description
Design of a method to automatically detect when a git commit changes any of the OSM charms, followed by building and publishing the changes to the Charm Store.
At this time, the source code for all the OSM charms are kept in ETSI OSM’s Git instance, however, publication of the charms is still a manual process. This should be automated so that charms are part of the CI/CD pipeline like any other code in OSM.
When a new patch is uploaded to Gerrit, the charm needs to be built, but should not get published to a well known track. For snaps, we use latest/edge/GERRIT_REFSPEC as the track to receive unmerged code.
Publication of Charms for New Patches (pre-merge)
When a new patch is uploaded to Gerrit, the charm needs to be built, but should not get published to a well known track. For snaps, we use latest/edge/GERRIT_REFSPEC as the track to receive unmerged code.
Example
Building a new RO charm based off this change: https://osm.etsi.org/gerrit/c/osm/devops/+/12272
- Gerrit receives new patch and initiates a new Jenkins stage_1 job
- Stage_1 detects that this is a new change, not a merge, and will start the devops stage 2 job
- Stage 2 creates a docker container to run tests, create .deb package, etc
- Stage 2 checks if the change involves charm code (tbd)
- Stage 2 invokes a charmcraft pack in the osm-ro charm directory
- On success, stage 2 issues charmcraft upload of the new revision and notes the revision number
- Stage 2 issues a charmcraft release osm-ro --revision= --channel=latest/edge/refs-changes-72-12272-3
- Stage 2 starts the stage_3 job to complete the Robot test suite.
Publication of Charms for Merged Patches (post-merge/submit)
Once a patch in Gerrit is merged, a build job runs to publish the changed artifact to a well known location. For snaps, we use latest/edge (or v12.0/edge, etc, depending on the branch).
Example
Publication of the RO charm from the Patch example:
- Gerrit receives +2 and Submit code from the MDL, and initiates new Jenkins stage_1 job
- Stage 1 detects this is a merge and starts the devops stage 2 merge job
- Stage 2 merge creates a docker container to run tests, create .deb package, etc
- Stage 2 checks if the change involves charm code
- Stage 2 invokes a charmcraft pack in the osm-ro charm directory
- On success, stage 2 issues charmcraft upload of the new revision and notes the revision number
- Stage 2 issues a charmcraft release osm-ro --revision= --channel=latest/edge/merged
Promotion from Edge to Beta (daily)
Once the daily job has been completed successfully, stage 3 checks to see if the revision in Edge is newer than the revision in Beta. If it is, the latest Edge version gets promoted to Beta as a final step in the daily job.
Example
Promotion of osm-ro from edge to beta:
- Stage 3 job is started by timer to perform the daily robot test
- Stage 3 installs OSM, and runs the robot suite against it
- On successful completion of Robot, stage 3 lists the released revisions for osm-ro, architecture ubuntu 22.04 (amd64).
- Edge revision and Beta revisions are compared. If edge > beta, release the edge revision to beta.
Consideration for Edge Track
Currently the following releases deploy from Edge:
- 10.1.0 LTS
- 10.1.1 LTS
- 10.1.2 LTS
- 12.0.0 LTS
This presents a challenge to the use of Edge for holding merged changes until the daily job runs, therefore it makes more sense to use “latest/edge/merged” instead of the unadorned “latest/edge” for the staging of merged changes until the daily runs.
The downsides to using edge/merged:
- These tracks are typically not accessible unless you log in
- It is different than what we do for snaps
- Branch tracks like this are set to automatically expire in 30 days. This should not be an issue, unless the daily has not run for > 30 days.
Demo or definition of done
New changes to charms or bundles in devops are automatically published to Charmstore.