ci-templates

Ready to include templates for common CI jobs.

Usage

Autopep8

By including this template in your project, its Python files will be linted with autopep8 for each Merge Request:

• Only the changed files will be checked with black.
• If they are not properly formatted according to black, the CI will fail.
• An assist-MR will be created to automatically fix the formatting.
• Upon merging this assist-MR, the changed files are checked again and the CI should pass.

To use the autopep8 CI template, follow these steps:

1. Add CI Bot as a member to your project in at least the Developer role.

2. Lastly, include the following snippet in your .gitlab-ci.yml file:

   include:
     - project: 'omegacen/ci-templates'
       ref: v5
       file: '/templates/autoformat/autopep8.yml'

The variables that can be changed are:

This CI template cannot be used together with the black CI template.

Black

By including this template in your project, its Python files will be linted with black 19.10b0 for each Merge Request:

• Only the changed files will be checked with black.
• If they are not properly formatted according to black, the CI will fail.
• An assist-MR will be created to automatically fix the formatting.
• Upon merging this assist-MR, the changed files are checked again and the CI should pass.

The default line length is 120, although this can be changed via the BLACK_LINE_LENGTH variable.

To use the black CI template, follow these steps:

1. Add CI Bot as a member to your project in at least the Developer role.

2. Lastly, include the following snippet in your .gitlab-ci.yml file:

   include:
     - project: 'omegacen/ci-templates'
       ref: v5
       file: '/templates/autoformat/black.yml'

The variables that can be changed are:

This CI template cannot be used together with the autopep8 CI template.

Conda

By including this template, the CI will automatically build one or more conda recipes in the repository and upload the built package our conda channel, https://conda.astro-wise.org.

The following jobs are added to the CI pipeline by including this template

• conda_build, which builds the package.
• conda_test, which tests the built package.
• conda_upload, which upload the package to the conda channel.

All jobs are run for MRs, tags, master, develop, and the default branch. The conda_upload job is only run for tags, and for the master, develop, release branches and the default branch. For conda_upload, the branch or job needs to be protected.

To use this template, do the following. First, include one or more conda recipes in your repository. For a repository that produces a single conda package, the file name of the recipe should be conda-recipes/meta.yaml. For more information on how to make a conda recipe, see the conda-wise documentation.

Next, include the following snippet in your .gitlab-ci.yml file:

include:
  - project: 'omegacen/ci-templates'
    ref: v5
    file: '/templates/conda/all.yml'

The complete list of variables is given below.

If set, $CONDA_PACKAGE_VERSION has to be used as the version of the conda package.

The conda_build job exposes the following environment variables to all subsequent CI jobs:

• $CONDA_PACKAGE_NAME
• $CONDA_PACKAGE_VERSION
• $CONDA_PACKAGE_BUILD_STRING

If the conda_build job builds multiple conda packages, the above variables refer to the first conda package built.

It can take long to resolve the dependencies when creating the environment for the conda_test job, especially for packages with many dependencies. The resolve can be sped up by using the solver from mamba, either by using mamba directly by setting CONDA_BUILD_COMMAND to mambabuild or by using the experimental conda feature by setting CONDA_BUILD_EXTRA_ARGS to --experimental-solver=libmamba.

Conda build

It is possible to only include the template for the build stage, in case you do not want to automatically upload your build recipe but only want to test it. In that case, you must include the conda-build.yml template instead of conda.yml:

include:
  - project: 'omegacen/ci-templates'
    ref: v5
    file: '/templates/conda/build.yml'

Monthly Merge

This template performs an automatic release merge and tag based on a schedule (usually monthly). It performs the following steps:

• Merge the develop branch into the master branch,
• Bump the version in specified files according to CalVer YYYY.MM.MICRO,
• Create a tag on the master branch.

To use it, follow these steps:

1. Include the following snippet in your .gitlab-ci.yml file:

   include:
     - project: 'omegacen/ci-templates'
       ref: v5
       file: '/templates/monthlymerge.yml'

2. Next, create a pipeline schedule in your project. Set it to the first of the month. This pipeline schedule should ideally be owned by CI Bot. This user should a maintainer in your repository (ask a GitLab admin to do this for you after you've created the pipeline schedule). Also add a variable called MM_RUN_JOB with a value of 1 to the pipeline schedule.

3. Next, protect the tags created by this template via a wildcard (20* will do).

4. Lastly, add the private key of CI Bot to the CI variables as a protected variable called MM_PRIVATE_KEY.

The complete list of variables is given below.

SonarQube

By including this template, SonarQube source code analysis will automatically be run on the default branch and on merge requests.

To use it, follow these steps:

1. Add CI Bot as a member to your project in at least the Developer role.

2. Lastly, include the following snippet in your .gitlab-ci.yml file:

   include:
     - project: 'omegacen/ci-templates'
       ref: v5
       file: '/templates/sonarqube.yml'

3. Optionally, you can add one or more Project badges. In your GitLab project, go to Settings -> General -> Badges. Then add the following:

   • Name: Quality Gate Status
   • Link: https://sonar.astro-wise.org/dashboard?id=%{project_path}
   • Badge image URL: https://sonar.astro-wise.org/api/project_badges/measure?project=%{project_path}&metric=alert_status

   This adds a Quality Gate badge. There are more metric available; go to the SonarQube project page and click "Get project badges" for a list.

The complete list of variables for this CI template is given below.

LaTeX

This template builds a pdf file from latex sources. For feature branches and merge requests it will also generate a pdf that highlights the differences with the default branch.

To use it, include the following snippet in your .gitlab-ci.yml file:

include:
  - project: 'omegacen/ci-templates'
    ref: v5
    file: '/templates/latex.yml'

The filename of the main tex file is determined automatically. This can be overruled by specifying the FILENAME_TEX variable.

The complete list of variables for this CI template is given below.

Controlling when jobs runs

By including a CI template, the jobs in it get automatically run under the right circumstances. E.g., by including the conda template, a conda package is build and tested for each MR to the main branches, and for each commit on the main branches a conda packages is build, tested, and released.

However, if you want more control over when a particular job runs, you can use the CI_AWE_RUN_<JOBNAME> and CI_AWE_SKIP_<JOBNAME> variables to run or to skip a job, respectively. The full list of variables is as follows:

The latest version of the CI templates is in the master branch. In addition there are release branches with naming convention v<X>. The master branch is always merged into the latest release branch.

Breaking changes increase the version number of the release branch. So backwards compatibility is guaranteed when you include a template from a release branch. However, including a template from the master branch might lead to breaking changes every now and then.