Application services

Introduction

The redpesk factory supports the concept of services. Services can be used to automate actions during specific application processes.
For example the download_sources service can be enabled to automatically download source files which are specified via a http, https or ftp url in SourceX statement in spec file.
This section provides information on how to configure your application services to ease application management through redpesk.

Each packaging branch has its own set of services which perform actions during a specific application process.

after the application’s service has been modified.
after the application’s specfile has been submitted.
after the applications’s archive file has been submitted.
after anything has been submitted within application git packaging files.

NOTE: application packaging files are regenerated (like tarball, specfile, …) before application build start.
Consequently any above listed actions may be triggered if a change is detected.

These services can be set either by using redpesk-cli or the web UI.
Each service has its own set of parameters which allows you to customize the action performed by the service.

The redpesk factory services are divided into two categories:

Common: always deployed in redpesk factory stacks.
Additional: deployed on-demand in redpesk factory stacks.
(please contact redpesk support if a service is not available on the factory stack you use)

The sub-chapters below describe actions performed by these redpesk factory services.

Common services

Exclude architecture(s)

Architecture(s) must be declared within redpesk project and are excluded from builds by using Excludearch macro.

Service name: excludearch
Parameters:
- arches:
  - Type: array
  - Description: list of architectures to exclude
Invoked: after the application’s specfile has been submitted

See a use case in chapter below.

Automatic version/release update of an application

Automatically increases the release each time a build succeeds. When automatic version is ticked, set the version according to the nearest Git source tag (X.X.X) found from the application revision in the application sources.

Service name: setverrel
Parameters:
- version:
  - Type: boolean
  - Description: automatically retrieve the version according to git tag, git sha and/or date
- release:
  - type: boolean
  - Description: automatically increment the release part (incremental +1 number for each build)
Invoked: after the application’s archive file has been submitted

See a use case in chapter below.

Specfile linter

Use a specfile linter to detect common specfile syntax errors.

Service name: specfile_linter
Parameters:
- enable:
  - Type: boolean
  - Description: enable linter execution while writing specfile
Invoked: after the application’s specfile file has been submitted

See a use case in chapter below.

Additional services

Propagate commits in packaging branches

Propagate each commit made in the current packaging branches in each target branch specified.

Service name: branch_alignment
Parameters:
- branches:
  - Type: array
  - Description: branches in which current packaging branch commits are propagated
Invoked: after anything has been submitted

See a use case in chapter below.

Download Specfile sources

Download Specfile sources when specified by a URL, and automatically add them in the git package.

Service name: download_sources
Parameters:
- force:
  - Type: boolean
  - Description: overwrite
Invoked: after the application’s specfile file has been submitted

See a use case in chapter below.

Fetch go dependencies

Fetch go dependencies according to go.mod and go.sum files and pack them into a vendor archive, allowing full offline builds of go applications.

Service name: go_dependency
Parameters:
- archive:
  - Type: string
  - Description: Specify the Go application source archive that contains go.mod and go.sum. Values: app-x.y.z.tar.gz. Default: None, will use autodetection
- compression:
  - Type: string
  - Description: Specify the compression method for the generated vendor tarball
Invoked: after the application’s archive file has been submitted

See a use case in chapter below.

Import an external git package

Import an external git package repository. Can be used only during application creation.

Service name: import_gitpkg
Parameters:
- url:
  - Type: string
  - Description: URL of the external git package repository to import
Invoked: after the application’s service has been modified

See a use case in chapter below.

Import an external git reference

Import an external git reference (branch/tag) as a redpesk application branch.

Service name: import_reference
Parameters:
- url:
  - Type: string
  - Description: URL
- ref:
  - Type: string
  - Description: Git reference
- branch:
  - Type: string
  - Description: Packaging branch
Invoked: after the application’s service has been modified

See a use case in chapter below.

Import an external SRPM

Import an external SRPM as a redpesk application branch.

Service name: import_srpm
Parameters:
- url:
  - Type: string
  - Description: URL
- branch:
  - Type: string
  - Description: Packaging branch
- tag:
  - Type: string
  - Description: Tag to set at the commit import
Invoked: after the application’s service has been modified

See a use case in chapter below.

Fetch nodejs dependencies

Fetch nodejs dependencies according to a package-lock.json file and pack them into an archive, allowing full offline builds of nodejs applications.

Service name: node_dependency
Parameters:
- archive:
  - Type: string
  - Description: Specify the Nodejs application source archive that contains package-lock.json. Values: app-x.y.z.tar.gz. Default: None, will use autodetection
- cpio:
  - Type: string
  - Description: Cpio file name to store all tarballs in
- alternative_dependencies_server
  - Type: string
  - Description: Alternative server URL to npm download dependencies
- ignore_lockfile_version
  - Type: boolean
  - Description: Ignore the lockfileVersion of input file
Invoked: after the application’s archive file has been submitted

See a use case in chapter below.

Fetch rust dependencies

Fetch rust dependencies according to a Cargo.toml file and pack them into a vendor archive, allowing full offline builds of rust applications.

Service name: rust_dependency
Parameters:
- archive:
  - Type: string
  - Description: Specify the Rust application source archive that contains Cargo.toml. Values: app-x.y.z.tar.gz. Default: None, will use autodetection
- compression:
  - Type: string
  - Description: Specify the compression method for the generated vendor tarball
Invoked: after the application’s archive file has been submitted

See a use case in chapter below.

Usage

WebUI

The services described above can be found for each application settings tab in a section named Application Services block.

services

redpesk-cli

Here is an example of retrieving and updating a service’s value with command line tool named redpesk-cli.

# List all available services (add --verbose option to get more details)
> rpcli applications services
Name                     Summary                         Version Parameters
download_sources         Download Specfile sources       2.0.1   force
excludearch              Exclude architecture(s)         2.0.0   arches
import_srpm              Import an external SRPM         2.1.0   branch, tag, url
rust_dependency          Fetch rust dependencies         1.0.1   archive, compression, config
setverrel                version-release update          2.0.0   release, version
specfile_linter          Specfile linter                 2.1.0   enable

# Retrieve application services value
> rpcli applications get --project test mustach -v
...
Services:

 - Name: specfile_linter
 - Parameters:
    enable:	 true

# Set automatic version in the setverrel service
> rpcli applications update --project test mustach --service "setverrel={'version':true}"
# Disable the setverrel service
> rpcli applications update --project test mustach --disable-services setverrel

NOTE: use Service name field mentioned in this documentation to select the right service to setup

Use cases

This section provide a usage examples for each service previously described by using redpesk-cli. The project with the following attribute will be used:

# Project information
> rpcli projects get test --verbose
...
Name:			 test
Slug:			 test
Type:			 standard-applications
Parent project:	 test
Description:     test project
Distributions:	 redpesk-lts-batz-2.0-update [Opt]
Architectures:	 x86_64 [Opt], aarch64 [Opt]
...

Exclude architecture(s)

Let’s consider the mustach application coming from the samples.

> rpcli applications add-from-sample --project test mustach

The test project has the architecture x86_64 as optional and so we can exclude it from all the next builds launched in the mustach application.

# Activate excludearch service for x86_64
> rpcli applications update --project test mustach --service "excludearch={'arches':['x86_64']}"

This service will add in the autogenerated specfile an ExcludeArch macro which prevents the factory from building the application for x86_64 architecture.

...
Name:           mustach
...
Version:        1.2.9
ExcludeArch:    x86_64
...

Ensure x86_64 architecture is ignored by launching a build within redpesk-lts-batz-2.0-update distribution.

# Launch a build
> rpcli applications build --project test --distribution redpesk-lts-batz-2.0-update mustach
# Get the build information
> rpcli applications builds get mustach
...
Status:		            done
Subtask aarch64 status:	closed
...

Automatic version/release update of an application

Let’s consider the mustach application coming from the samples.

> rpcli applications add-from-sample --project test mustach

During this use case, the mustach specfile will be modified so set it as internal in application settings.

> rpcli applications update mustach --project test --specfile-type internal

This service introduces two notions:

Manual/Automatic version
Manual/Automatic release

Manual Version

# Activate setverrel service for manual version
> rpcli applications update --project test mustach --service "setverrel={'version':false}"
# Download mustach specfile
> rpcli applications files download --project test mustach mustach.spec
# Change mustach specfile version to 0.0.0
> sed -i 's/Version:.*/Version:\t0.0.0/' mustach.spec
# Upload modified mustach specfile
> rpcli applications files upload --project test mustach mustach.spec
# Regenerate mustach archive
> rpcli applications refresh-files-pkg --project test mustach

To see what happened in the mustach application, take a look at the mustach archive version made by the factory.

> rpcli applications files list --project test mustach
...
mustach-0.0.0.tar.bz2
...

And also the version of the autogenerated specfile

Name:           mustach
...
Version:        0.0.0

Here the package version does not give information about what has been archived nor what is about to be built.

When manual version is enabled, the version set in the application specfile is set in the autogenerated specfile and also in the autogenerated application tarball.

Automatic version

# Activate setverrel service for manual version
> rpcli applications update --project test mustach --service "setverrel={'version':true}"
# Set application source revision to branch master
> rpcli applications update --project test mustach --source-rev master

To see what happened in the mustach application, take a look at the mustach archive version made by the factory.

> rpcli applications files list --project test mustach
...
mustach-1.2.9+20240607+2+gbfadeda.tar.bz2
...

And also the version of the autogenerated specfile

...
Name:           mustach
...
Version:        1.2.9+20240607+2+gbfadeda
...

Here the package version gives the following information about what has been archived and also what is about to be built:

The nearest git tag under the form X.X.X of branch master is the tag 1.2.9.
The latest commit of master branch that is archived if from the 7th June 2024.
The latest commit of master branch that is archived is 2 commits ahead of tag 1.2.9.
The latest commit of master branch that is archived has bfadeda as short SHA.

When automatic version is enabled, the source repository set in the application settings leads the version of your packaging. It gives information of what has been built or what is contained in your tarball without having to decompress it.

Manual release

# Activate setverrel service for manual release
> rpcli applications update --project test mustach --service "setverrel={'release':false}"
# Download mustach specfile
> rpcli applications files download --project test mustach mustach.spec
# Change mustach specfile release to 1
> sed -i 's/Release:.*/Release:\t1%{?dist}/' mustach.spec
# Upload modified mustach specfile
> rpcli applications files upload --project test mustach mustach.spec

To see what happened in the mustach application, take a look at the release of the autogenerated specfile.

...
Name:           mustach
...
Release:	    1%{?dist}
...

When manual release is enabled, the release set in the application specfile is set in the autogenerated specfile and so will be taken into account in the next application build.

Automatic release

# Activate setverrel service for automatic release
> rpcli applications update --project test mustach --service "setverrel={'release':true}"
# Build the application
> rpcli applications build --project test --distribution redpesk-lts-batz-2.0-update mustach
# Get the application build information
> rpcli applications builds get mustach --verbose
...
Subtask:			 x86_64
[RPM files]
  mustach-0.0.0-1.test.rpbatz.x86_64.rpm
  ...
Subtask:			 aarch64
[RPM files]
  mustach-0.0.0-1.test.rpbatz.aarch64.rpm

Here we can see the release of the package is set to 1 such as in the autogenerated specfile. Let’s see what happens when launching a build without changing anything in the mustach application.

# Build the application
> rpcli applications build --project test --distribution redpesk-lts-batz-2.0-update mustach
# Get the application build information
> rpcli applications builds get mustach --verbose
...
Subtask:			 x86_64
[RPM files]
  mustach-0.0.0-2.test.rpbatz.x86_64.rpm
  ...
Subtask:			 aarch64
[RPM files]
  mustach-0.0.0-2.test.rpbatz.aarch64.rpm

Here we can see that without changing the release in the autogenerated specfile, it has been automatically increased while building the application.

Specfile linter

Let’s consider the mustach application coming from the samples.

> rpcli applications add-from-sample --project test mustach

During this use case, the mustach specfile will be modified therefore set it as internal in application settings.

> rpcli applications update mustach --project test --specfile-type internal

Now activate the linter service and introduce an error in the specfile.

# Activate setverrel service for automatic release
> rpcli applications update --project test mustach --service "specfile_linter={'enable':true}"
# Download mustach specfile
> rpcli applications files download --project test mustach mustach.spec
# Remove Name definition from specfile
> sed -i '/Name:.*/d' mustach.spec
# Upload modified mustach specfile
> rpcli applications files upload --project test mustach mustach.spec
----- File(s) Upload requested by user ----
Uploading mustach.spec [KO]
Error: an error occurred while updating "files" - can't parse specfile error: Name field must be present in package: (main package)

Either way, the specfile is parsed before building the application. So if the specfile is written incorrectly, this service is here to save time by notifying you before launching the build.

Propagate commits in packaging branches

Let’s consider the mustach application coming from the samples.

> rpcli applications add-from-sample --project test mustach
# List mustach packaging branches
> rpcli applications branches list --project test mustach --verbose
* main	 [ACTIVE],[DEFAULT],[PROTECTED]	 (redpesk-lts-batz-2.0-update)

The mustach application has one branch named main. Let’s activate the branch_alignment service to create and link the branch next to main.

# Activate branch_alignment service
> rpcli applications update --project test mustach --service "branch_alignment={'branches':['main','next']}"
# List mustach packaging branches
> rpcli applications branches list --project test mustach --verbose
* main	 [ACTIVE],[DEFAULT],[PROTECTED]	 (redpesk-lts-batz-2.0-update)
  next	 				                 ()
# Checkout to branch next
> rpcli applications branch --project test mustach --destination next
# Create an empty file toto
> touch toto
# Upload the file toto
> rpcli applications files upload mustach toto
# Checkout to branch main
> rpcli applications branch --project test mustach --destination main
# List mustach files on branch main
> rpcli applications files list --project test mustach
...
toto

From now on, the packaging branch main and next are linked. It means that each modification made in one of them is propagated to the other.

Download Specfile sources

Let’s consider the mustach application coming from the samples.

> rpcli applications add-from-sample --project test mustach

During this use case, the mustach specfile will be modified therefore set it as internal in application settings.

> rpcli applications update mustach --project test --specfile-type internal

# Activate download_sources service
> rpcli applications update --project test mustach --service "download_sources={'force':false}"

Let’s imagine while building the mustach application some headers or libraries from the helloworld-binding version 1.1.1 are needed.

# Download mustach specfile
> rpcli applications files download --project test mustach mustach.spec
# Add helloworld-binding URL as a Sources
sed '/Source:.*/a Source1:\thttps://github.com/redpesk-samples/helloworld-binding/archive/refs/tags/1.1.1.tar.gz' mustach.spec
# Upload modified mustach specfile
> rpcli applications files upload --project test mustach mustach.spec
# List mustach files to see the helloworld-binding-1.1.1 archive
> rpcli applications files list --project test mustach
...
1.1.1.tar.gz
...

From now on, the helloworld-binding is set in the specfile and its content is available when building.

Fetch go dependencies

Let’s consider an application whose sources are written in golang named golang-app.

Golang satisfies dependencies by downloading modules from their sources into the module cache when building an application. However there is no network available while building an application in the redpesk factory. Fortunately golang offers a workaround with the vendoring build mode (documentation).

The first step to use such a build mode is to create this vendor archive.

# Activate go_dependency service
> rpcli applications update --project test golang-app --service "go_dependency={'archive':''}"

By leaving the archive parameter empty, the service will process the factory autogenerated archive created from the golang-app sources URL.

# List golang-app files
> rpcli applications files list --project test golang-app
...
golang-app-0.0.0.tar.gz
...
vendor.tar.bz2
...

Now that the vendor is created within the redpesk application, the second step is to use it while building.
So first declare it in the specfile :

Name: golang-app
Version: 0.0.0
Release: 1%{?dist}
...
Source0: %{name}-%{version}.tar.gz
Source1: vendor.tar.bz2
...

Then use the build mode vendor when building the golang-app.

Name: golang-app
Version: 0.0.0
Release: 1%{?dist}
...
%build
...
go build -mod=vendor
...

Import an external git package

Let’s imagine for whatever reason we need the package can-utils not yet available in redpesk-lts-batz-2.0-update distribution. You can either wait for it to be added or you can build your own it in your redpesk project. After that can-utils will be available either during an other project’s application build or an image build.

# Activate import_gitpkg service
> rpcli applications add --project test --name can-utils --pkg-name can-utils --service "import_gitpkg={'url':'https://src.fedoraproject.org/rpms/can-utils.git'}"
# List can-utils packaging branches
> rpcli applications branches list --project test can-utils --verbose
* main		 [ACTIVE],[DEFAULT],[PROTECTED]	 (redpesk-lts-batz-2.0-update)
  epel7		 				 ()
  epel9		 				 ()
  ...

By using this service, the whole can-utils branches have been imported. Because redpesk-lts-batz-2.0-update is aligned with RHEL9, the packaging branch epel9 has to be linked with the redpesk-lts-batz-2.0-update distribution.

> rpcli applications branch --project test can-utils --distributions redpesk-lts-batz-2.0-update --destination epel9

After that the application can-utils can be built within redpesk-lts-batz-2.0-update and once it is done, can-utils will be available in the test project either as BuildRequires or package in an image kickstart.

Import an external git reference

Let’s imagine for whatever reason we need the package can-utils not yet available in redpesk-lts-batz-2.0-update distribution. You can either wait for it to be added or you can build your own in your redpesk project. After that can-utils will be available either during an other project’s application build or an image build.

Because the test project is built within redpesk-lts-batz-2.0, aligned with RHEL9, and can-utils is coming from epel, we want to build, and so import the branch epel9.

# Activate import_reference service
> rpcli applications add --project test --name can-utils --pkg-name can-utils --service "import_reference={'url':'https://src.fedoraproject.org/rpms/can-utils.git','ref':'epel9'}"
# List can-utils packaging branches
> rpcli applications branches list --project test can-utils --verbose
* epel9	 [ACTIVE],[DEFAULT],[PROTECTED]	 (redpesk-lts-batz-2.0-update)

After that the application can-utils can be built against redpesk-lts-batz-2.0-update and once it is done, can-utils will be available in the test project either as BuildRequires or package in an image kickstart.

Import an external SRPM

Let’s imagine for whatever reason we need the package can-utils not yet available in redpesk-lts-batz-2.0-update distribution. You can either wait for it to be added or you can build your own in your redpesk project. After that can-utils will be available either during an other project application build or an image build.

Because the test project is built within redpesk-lts-batz-2.0, aligned with RHEL9, and can-utils is coming from epel, we want to build, and so import a source RPM from epel9.

# Activate import_srpm service
> rpcli applications add --project test --name can-utils --pkg-name can-utils --service "import_srpm={'url':'https://dl.fedoraproject.org/pub/epel/9/Everything/source/tree/Packages/c/can-utils-2023.03-1.el9.src.rpm','branch':'epel9'}"
# List can-utils packaging branches
> rpcli applications branches list --project test can-utils --verbose
* epel9	 [ACTIVE],[DEFAULT],[PROTECTED]	 (redpesk-lts-batz-2.0-update)

After that the application can-utils can be built against redpesk-lts-batz-2.0-update and once it is done, can-utils is available in the test project either as BuildRequires or package in an image kickstart.

Fetch nodejs dependencies

Let’s consider an application whose sources are written in nodejs named nodejs-app.

Nodejs satisfies dependencies by downloading modules from their sources into the module cache when building an application. However there is no network available while building an application in the redpesk factory. Fortunately nodejs modules can be packed in an archive and served locally (hence the usage of local-npm-registry) when building.

The first step to use such a build mode is to create this vendor archive.

# Activate go_dependency service
> rpcli applications update --project test nodejs-app --service "node_dependency={'archive':''}"

By leaving the archive parameter empty, the service will process the factory autogenerated archive created from the nodejs-app sources URL.

# List nodejs-app files
> rpcli applications files list --project test nodejs-app
...
nodejs-app-0.0.0.tar.gz
...
vendor_node.cpio
...

Now that the vendor is created within the redpesk application, the second step is to use it while building. So first declare it in the specfile.

Name: nodejs-app
Version: 0.0.0
Release: 1%{?dist}
...
Source0: %{name}-%{version}.tar.gz
Source1: node_vendor.cpio
...

Then add as BuildRequires local-npm-registry which chips the software which will serve locally what is inside the node vendor archive.

Name: nodejs-app
Version: 0.0.0
Release: 1%{?dist}
...
BuildRequires: local-npm-registry
...

Then unpack and serve what’s inside node vendor archive.

Name: nodejs-app
Version: 0.0.0
Release: 1%{?dist}
...
%prep
cpio -i < %{SOURCE1}
%setup
local-npm-registry %{_builddir} install --no-optional
...

After that npm can be used in the specfile build section to build nodejs-app

Fetch rust dependencies

Let’s consider an application whose sources are written in rust named rust-app.

Rust satisfies dependencies by downloading modules from their sources into the module cache when building an application. However there is no network available while building an application in the redpesk factory. Fortunately rust offers a workaround with an offline build mode (documentation).

The first step to use such a build mode is to create this vendor archive along with its cargo_config file associated.

# Activate rust_dependency service
> rpcli applications update --project test rust-app --service "rust_dependency={'archive':''}"

By leaving the archive parameter empty, the service will process the factory autogenerated archive created from the rust-app sources URL.

# List rust-app files
> rpcli applications files list --project test rust-app
...
rust-app-0.0.0.tar.gz
cargo_config
vendor.tar.bz2
...

Now that the vendor archive and its cargo_config have been created within the redpesk application, the second step is to use them while building. So first declare them in the specfile.

Name: rust-app
Version: 0.0.0
Release: 1%{?dist}
...
Source0: %{name}-%{version}.tar.gz
Source1: vendor.tar.bz2
Source2: cargo_config
...

Copy the cargo_config in rust sources root to notify where dependencies are stored locally while building.

Name: rust-app
Version: 0.0.0
Release: 1%{?dist}
...
%prep
%setup
mkdir .cargo
cp %{SOURCE2} .cargo/config
...

Then use the offline build mode when building the rust-app.

Name: rust-app
Version: 0.0.0
Release: 1%{?dist}
...
%build
cargo build --offline
...

redpesk factory releases

Continuous delivery

Projects management

Containers management

Images management

Applications management

Security audit tests

Monitoring

redpesk command line interface (rp-cli)

Local builder

Test Framework

SOTA

External Triggers

Application services

Introduction

Common services

Exclude architecture(s)

Automatic version/release update of an application

Specfile linter

Additional services

Propagate commits in packaging branches

Download Specfile sources

Fetch go dependencies

Import an external git package

Import an external git reference

Import an external SRPM

Fetch nodejs dependencies

Fetch rust dependencies

Usage

WebUI

redpesk-cli

Use cases

Exclude architecture(s)

Automatic version/release update of an application

Manual Version

Automatic version

Manual release

Automatic release

Specfile linter

Propagate commits in packaging branches

Download Specfile sources

Fetch go dependencies

Import an external git package

Import an external git reference

Import an external SRPM

Fetch nodejs dependencies

Fetch rust dependencies