Gitlab CI

The goal of this chapter is to demonstrate the possibility for a Gitlab instance to interact with our redpesk factory.

In order to do that, we will take the following example: we want to trigger an automatic build and test of an application, on a merge request opening in the GIT repository of this application. If a problem appears in the build or in the test, the merge request is “blocked” and cannot be merged. This allows the maintainer of the application to have a first automatic check that the new code added by the merge request does not lead to regressions.

Gitlab environment

For this example we assume that our applications are hosted on gitlab.com. Each “application” corresponds to a Gitlab project and they are under the same group.

Gitlab applications overview

In this example, we can see that 3 Gitlab projects exist:

“Helloworld Binding”: the famous one, coming from redpesk OS (sources available here)
“Redpesk CI: this project will be used to store the common CI configurations, potentially used by all the projects of the group
“Redtest Helloworld Api”: another application that can be built and tested in the redpesk factory (sources available here)

redpesk factory set-up

On the redpesk factory side, the corresponding applications have been created. Each application URL points to the corresponding Gitlab project seen in the previous paragraph.

redpesk factory applications overview

Additional applications are present (starting with “python3”). They are, in fact, dependencies necessary to run “redtest-helloworld-api” tests.

Before going any further, make sure that the build and the test of your applications are successful on the branch you have chosen.

Gitlab group set-up

In this paragraph we can have two cases. If you are using the redpesk community factory (https://community-app.redpesk.bzh) and the public Gitlab instance you will have to use the gitlab.com CI runners so please refer to the first paragraph. If you have your On-Premise redpesk factory you can have a dedicated Gitlab CI runner in your factory, then, please refer to the second paragraph.

N.B.: keep in mind that gitlab.com limits the CI/CD time per month for free users (cf. https://about.gitlab.com/pricing/).

Having your own runner can be a way of having unlimited CI/CD time. If you want to have your own redpesk factory and/or your own CI runner(s), feel free to contact us.

Set-up with gitlab.com CI runner(s)

In the project Redpesk CI in your Gitlab group, add the .before_after_scripts_ci.yml file.

# IoT.bzh Company - redpesk
#
# This file is a generic file containing the script to be played before and after each job.
# This allows one to manage these scripts in a unique file
#
# In order to work correctly, this file needs to be included in the '.gitlab-ci.yml' file
# of the project.
#
# For more information please refer to:
#    - https://docs.gitlab.com/ee/ci/yaml/includes.html#use-default-configuration-from-an-included-configuration-file
#

# Set-up the different variables needed by the pipeline
variables:
    RP_INSTALL_RPCLI_URL: "https://raw.githubusercontent.com/redpesk-devtools/redpesk-sdk-tools/master/install-redpesk-sdk.sh"

default:
  # Use a Fedora 36 docker image (compatible to use rp-cli in it)
  image: fedora:36

  # Define the "before script" to use by default
  before_script:
    # Install `rp-cli` since we are in an empty docker
    - dnf install -y wget
    - wget -O - "$RP_INSTALL_RPCLI_URL" | bash -s -- --rp-cli --no-recommends
    - rp-cli connections add "$RP_REMOTE_FACTORY_NAME" --url "$RP_REMOTE_FACTORY_URL" --token "$RP_REMOTE_FACTORY_TOKEN"
    - rp-cli connections use "$RP_REMOTE_FACTORY_NAME"

  # Define the "after script" to use by default
  after_script:
    - >
        if ([ $CI_JOB_STATUS == 'success' ] && [ $CI_JOB_STAGE == 'test' ]) || [ $CI_JOB_STATUS == 'failed' ] || [ $CI_JOB_STATUS == 'canceled' ]; then
          rp-cli applications update "$RP_FACTORY_APP" -p "$RP_FACTORY_PROJECT" --source-rev "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
        fi

Firstly, this file allows one to select a Fedora 36 Docker’s image, to install rp-cli in it and to configure it to talk with the redpesk factory you are using (more information about the environment variables used in the next paragraphs).

After the jobs, under certain conditions, this configuration allows the resetting of the application source revision to its original value (in the redpesk factory).

Set-up with redpesk factory CI runner(s)

If you have your own factory, Iot.bzh can set-up for you (or can show you how) a Gitlab CI runner inside your redpesk factory. This allows you to have unlimited CI time.

Once the runner is set-up, with rp-cli installed, you will need to register this runner in your Gitlab group. In order to do that, go under CI/CD tab of the Gitlab group and click on “Runners”.

Then, click on “Register a group runner” on the top right. You will be able to get a token. This token needs to be entered inside the CI runner. Please send it to the Iot.bzh team, so we can associate the CI runner with your Gitlab group. Please also provide a list of tags to associate with this runner (useful for the CI configuration here below).

Get registration token

Once the CI runner is registered, you will be able to find it under the CI/CD section of the Gitlab group.

Finally, in the project Redpesk CI in your Gitlab group, add the .before_after_scripts_ci.yml file.

# IoT.bzh Company - redpesk
#
# This file is a generic file containing the script to be played before and after each job.
# This allows one to manage these scripts in a unique file
#
# In order to work correctly, this file needs to be included in the '.gitlab-ci.yml' file
# of the project.
#
# For more information please refer to:
#    - https://docs.gitlab.com/ee/ci/yaml/includes.html#use-default-configuration-from-an-included-configuration-file
#

default:
  # Specify the use of our own runner
  tags:
    - redpesk
    - my-company
    - iotbzh

  # Define the "before script" to use by default
  before_script:
    - rp-cli connections add "$RP_REMOTE_FACTORY_NAME" --url "$RP_REMOTE_FACTORY_URL" --token "$RP_REMOTE_FACTORY_TOKEN"
    - rp-cli connections use "$RP_REMOTE_FACTORY_NAME"

  # Define the "after script" to use by default
  after_script:
    - >
        if ([ $CI_JOB_STATUS == 'success' ] && [ $CI_JOB_STAGE == 'test' ]) || [ $CI_JOB_STATUS == 'failed' ] || [ $CI_JOB_STATUS == 'canceled' ]; then
          rp-cli applications update "$RP_FACTORY_APP" -p "$RP_FACTORY_PROJECT" --source-rev "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
        fi

The “tags” in the file allow the selection of the CI runner provided in the redpesk stack.

The before_script commands configure the rp-cli binary installed on the custom CI runner. The after_script commands make sure that the application setting (especially the source revision) are not kept after the pipeline run.

Group variables set-up

Wherever your CI runner is, you need to declare some environment variables that are used in the different jobs. On the group level the following common variables need to be declared:

GIT_STRATEGY
- Description: Gitlab global variable that defines how the sources are fetched at the beginning of each CI job. We set it to ‘none’ since we do not want to fetch sources in the runner
- Value: none
RP_REMOTE_FACTORY_NAME
- Description: Name of the factory you want to use, you can put whatever you want, it has no real impact other than display
- Value: my-great-factory
RP_REMOTE_FACTORY_URL
- Description: URL of the redpesk factory you are using (ex: https://community-app.redpesk.bzh)
- Value: https://community-app.redpesk.bzh
RP_REMOTE_FACTORY_TOKEN
- Description: Token generated from the webUI, allowing the runner to connect to the factory (cf. rp-cli configuration). The value here below is only an example and cannot be used.
- Value: 93e56690-62ac-4803-8528-ef1b443d5e24_1ba1007f-ddaa-44b8-846c-aac59d3c49fd

Gitlab project set-up

Gitlab CI configuration

In each project repository, add the .gitlab-ci.yml file at the repository root:

# IoT.bzh Company - redpesk
#
# This file allows, on a merge request, one to request a distant redpesk factory
# to build the source branch of the merge request and to run the corresponding tests.
# The results of these jobs will be available in the merge request UI
#

# Specify that this pipeline is on merge request only
workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

include:
  - project: 'test-ci-banano/redpesk-ci'
    ref: main
    file: '/.before_after_scripts_ci.yml'

build1:
  stage: build
  script:
    - rp-cli applications update "$RP_FACTORY_APP" -p "$RP_FACTORY_PROJECT" --source-rev "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
    - rp-cli applications build "$RP_FACTORY_APP" -p "$RP_FACTORY_PROJECT" --comment "[Gitlab CI] Started for MR '$CI_MERGE_REQUEST_IID', from author '$CI_COMMIT_AUTHOR'"

test1:
  stage: test
  script:
    - rp-cli applications test "$RP_FACTORY_APP" -p "$RP_FACTORY_PROJECT" --comment "[Gitlab CI] Started for MR '$CI_MERGE_REQUEST_IID', from author '$CI_COMMIT_AUTHOR'" -v

This file specifies that the workflow is played on merge request event only. It also includes the common configuration file located in the “Redpesk CI” project (that is in the same Gitlab group).

Finally, on the second half of the file the two jobs that will be run on each merge request opening: a build and a test in the redpesk factory, are described.

Project variables

In each project, additional Gitlab variables need to be declared in order for the CI jobs to work correctly. These variables can be declared under Settings/CI/CD/Variables.

Set project CI/CD variables

Two variables need to be defined per project:

RP_FACTORY_APP
- Description: Slug of the application in the redpesk factory
- Example value: helloworld-binding
RP_FACTORY_PROJECT
- Description: Slug of the project in the redpesk factory
- Example value: gitlab-ci-project

Merge request settings

If the success of the pipeline is mandatory to merge the merge request, you should check the case “Pipelines must succeed” in the “Merge requests” section of the project settings.

Merge request project settings

Enjoy your Gitlab CI set-up

You should now be able to enjoy the CI set-up! Open a merge request and check that the pipeline has started!

Merge request with pipeline

Pipeline details

redpesk factory releases

Continuous delivery

Projects management

Containers management

Images management

Applications management

Security audit tests

Monitoring

redpesk command line interface (rp-cli)

Local builder

Test Framework

SOTA

External Triggers