redpesk integration tests: redtests

redtests are enhanced ptests. Ptests come from the Yocto environment, they are described in the Yocto wiki.

The main difference between redtests and ptests is the output format of the tests. While a ptest wants a “simple test” format (result: testname), a redtest requires the tests output to comply with the Test Anything Protocol. The main benefit is the ability to see test progress during the tests run.

Here below, an exhaustive list of what is needed by the redpesk® infrastructure to run redtests.

  • In addition to the application main package, the spec file needs to generate a second package, whose name ends with -redtest. For instance, in the case of a package named helloworld-service, the redtest package is named helloworld-service-redtest. This package contains the file structure necessary to run the tests.

  • In the installation section of the spec file, the content of the “-redtest” package needs to be installed under the /usr/libexec/redtest/<package> directory. In the case of the helloworld-service-redtest package, the directory path is /usr/libexec/redtest/helloworld-service.

  • Inside the test package (and therefore under /usr/libexec/redtest/<package>), a script or a binary named run-redtest must be present. This script is executed by the redpesk® infrastructure on the target to run the tests. In the helloworld-service-redtest package, the redpesk® infrastructure calls the script/binary file named /usr/libexec/redtest/helloworld-service/run-redtest. It is important to notice that this file needs to be executable!

  • The run-redtest file can test anything you like, there are only two restrictions: standard output (stdout) contents need to be in T.A.P. format and the output needs to be saved as a “.tap” file under /var/log/redtest/<package> directory. In the helloworld-service-redtest package, the log file is saved under /var/log/redtest/helloworld-service. The name of the “.tap” file does not need to abide by any specific format.

Once the two packages built, the integration tests can be triggered automatically or manually (depending on the setting). During this process, the redpesk® infrastructure installs the two packages on a target, and runs the tests. While they are running, a live console is available on the webUI: it contains the standard output (stdout) and the standard error (stderr). Moreover, at the end of the tests, the results are available along with a zip file containing the two outputs (stout and stderr) and the tests logs file.

A “Quickstart guide” is available in order to setup your application to be “redtests ready”.