Rust Binary Release with Gitlab CI

Intro

For a recent project, I wrote a backend web service in 🦀 Rust 🦀 using the web framework Axum. Rust code compiles to a binary for a specific architecture. This simplifies deployment as you don't need to rely on a runtime environment.

I had not previously used Gitlab CI for hosting binary artifacts and it was a bit less obvious than I thought it would be. In this post, I will show you how to write a Gitlab CI pipeline to build and release a Rust binary. Hopefully this will help you save a few pain cycles.

Config File

To utilise Gitlab CI, you need to create a file named .gitlab-ci.yml in the root of your project. This file contains the configuration for your CI pipeline. This pipeline is configured to execute when a commit is pushed to the default branch.

Stages

The CI pipeline is split into stages, with each stage containing a series of tasks to complete. Stages can run in parallel or sequentially depending on the configuration.

There are 4 stages in this pipeline.

• prepare - Generate environment variables for future stages.
• test - Execute tests.
• build - Build the binary artifacts.
• release - Create a release on Gitlab.

The pipeline stages and their dependencies can be visualise as follows:

Stages are defined using the stages keyword.

stages:
  - prepare
  - test
  - build
  - release
  
prepare-job:
  stage: prepare
  ...
tests-job:
  stage: test
  ...
build-job:
  stage: build
  ...
release-job:
  stage: release
  ...
upload-job:
  stage: release
  ...

Each stage has one or more jobs. Jobs (named *-job here) are linked to the stage using the stage keyword.

Prepare

The prepare stage is used to generate environment variables which will be used in future stages.

prepare-job:
  stage: prepare
  image: rust
  rules:
    - if: $CI_COMMIT_TAG
      when: never # Do not run this job when a tag is created manually
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run this job when commits are pushed or merged to the default branch
  script:
    - echo "======== PREPARE JOB ========"
    - echo "PKG_VERSION=$(awk -F ' = ' '$1 ~ /version/ { gsub(/["]/, "", $2); printf("%s",$2) }' Cargo.toml)" >> variables.env
    - echo "PKG_REGISTRY_URL=${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic" >> variables.env
    - echo $PKG_VERSION
  artifacts:
    reports:
      dotenv: variables.env

Points worth noting:

• Uses the rust Docker image.
• Extracts the project version number from the Cargo.toml file.
• Generates the URL for the Gitlab package registry.
• Writes the environment variables to a file named variables.env.
• Exports the environment variables as an artifact that can be used in future stages.

Test

The test stage is used to execute the projects test cases.

tests-job:
  stage: test
  image: rust
  script:
    - echo "======== TESTS JOB ========"
    - apt update
    - apt install lld clang -y
    - cargo test

Build

The build stage, is where we compile the Rust code into a binary. The binary is then exported as an artifact for use in future stages.

build-job:
  stage: build
  image: ubuntu:20.04
  needs:
    - job: tests-job
    - job: prepare-job
      artifacts: true
  script:
    - echo "======== BUILD JOB ========"
    - apt update
    - apt install lld clang curl -y
    - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
    - source "$HOME/.cargo/env"
    - cargo build --release
    - mkdir bin
    - mv target/release/asm-web bin/asm-web-v${PKG_VERSION}
    - ls -la bin/
  artifacts:
    paths:
      - bin/

Points worth noting:

• Uses the ubuntu:20.04 Docker image as I need to compile to an older version of glibc and matches my target deployment OS version. Because of this, I also need to install lld and clang as well as rust.
• Requries the prepare and test stages to complete before executing.
• Uses the prepare stage artifact to extract the project version number.
• Compiles the Rust code in --release mode into a binary.
• Creates a directory named bin and moves the binary into it.
• Exports the /bin directory as an artifact that can be used in future stages.

Release

The release stage is how we publish our project and has 2 steps. The release-job and the upload-job.

The release-job creates a release on Gitlab.

release-job:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  needs:
    - job: prepare-job
      artifacts: true
    - job: build-job
  script:
    - echo "======== RELEASE JOB ========"
  rules:
    - if: $CI_COMMIT_TAG
      when: never # Do not run this job when a tag is created manually
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run this job when commits are pushed or merged to the default branch
  release: 
    tag_name: "v$PKG_VERSION" 
    description: "Awesome Web v$PKG_VERSION"
    ref: "$CI_COMMIT_SHA" # The tag is created from the pipeline SHA.
    assets:
      links:
        - name: "v$PKG_VERSION"
          url: "${PKG_REGISTRY_URL}/x86_64-unknown-linux-gnu/v${PKG_VERSION}/asm-web"

Points worth noting:

• Uses the release-cli Docker image.
• Depends on the prepare and build stages.
• Creates a tag based on the Cargo.toml project version.
• Creates a release named after the tag.
• Creates a link to the binary hosted on the Package Registry.

The upload-job uploads the binary to the Gitlab package registry.

upload-job:
  stage: release
  image: curlimages/curl:latest
  needs:
    - job: prepare-job
      artifacts: true
    - job: build-job
      artifacts: true
  rules:
    - if: $CI_COMMIT_TAG
      when: never # Do not run this job when a tag is created manually
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run this job when commits are pushed or merged to the default branch
  script:
    - echo "======== UPLOAD JOB ========"
    - ls -la bin/
    - 'curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file bin/asm-web-v${PKG_VERSION} ${PKG_REGISTRY_URL}/x86_64-unknown-linux-gnu/v${PKG_VERSION}/asm-web'

Points worth noting:

• Uses the curl Docker image.
• Depends on the prepare and build stages.
• Creates an asset in the Package Registry based on the Cargo.toml project version.

For this project, I am using the generic package registry. The URL requires the format: /projects/:id/packages/generic/:package_name/:package_version/:file_name which is explained below.

• id - The project ID. This can be found on the main project page.
• package_name - The name of the package. In this case I am using x86_64-unknown-linux-gnu which matches the rust target.
• package_version - The version of the package. This is extracted from the Cargo.toml file.
• file_name - The name of the binary. In this case I am using asm-web.

Bonus Round

To download a binary from the Gitlab project registry via the API, we need to use a token. For my purpose, I am using a Project Access Token. Project access tokens require the read_api scope and at least the Reporter role.

The docs show a curl example. I am deploying via Ansible. So an example Ansible task is shown below.

- name: "DOWNLOAD BINARY FROM GITLAB PACKAGE REGISTRY"
  ansible.builtin.get_url:
    url: "https://gitlab.com/api/v4/projects/{{ gitlab_project_id }}/packages/generic/{{ compilation_target }}/{{ binary_version }}/{{ binary_name }}"
    headers:
      "PRIVATE-TOKEN": "{{ project_access_token }}"
    dest: "{{ app_config_dir }}/{{ binary_name }}"
    owner: root
    group: root
    mode: u=rx,g=rx,o=rx

Improvements

As I get more familiar with Gitlab CI I will update this post with any improvements I make. Some things I would like to do are:

• Create a custom docker image with all the tools I need installed. This will speed up the build process as I won't have to install the tools in every stage and build.
• Rust can automatically build project documentation. I would like to add a stage to build the documentation and host it on Gitlab pages.

Outro

In this post I showed you how to write a Gitlab CI pipeline to build and release a Rust binary. This took me way longer than I thought it would, so if you are reading this, I hope you found it useful and helped you save some time. If you have suggestions and/or examples for improvements please hit me up via the links down there 👇🏾. Thanks for tuning in 💖