Rust - Crate - Grcov demo

When your projects grow complex, ensuring that every single line of code is covered by your tests becomes critical.

But how do you verify such a thing efficiently?

Fortunately, the Rust ecosystem offers powerful crates for this purpose.

This tutorial will guide you through setting up grcov for reliable code coverage.

First of all

The complete source code for this tutorial is available on GitHub:

• https://github.com/badprog/badprog-grcov-demo

All required software will be installed within our Docker environment (Dockerfile).

Versions used:

• rustc: 1.91.1 --> https://rust-lang.org/
• rustup: 1.28.2 (available with your Rust version)
• llvm: 21.2.2 (available with your Rust version)
• grcov: 0.10.5 --> https://github.com/mozilla/grcov
• python: 3.13.5 --> https://www.python.org/

We will divide our project structure into several key parts:

• Dockerfile
• LLVM
• Devcontainer
• Grcov
• Python Script
• Rust Code
• Pipeline

The project

The Dockerfile

Using a coverage tool like grcov requires specific compiler flags and the correct dependencies.

Our Dockerfile handles this setup:

We start from a base Rust image.

We install required components using rustup component add, such as:

• llvm-tools-preview
• clippy
• rustfmt

We install the grcov crate using cargo install with the --locked parameter.

We install Python.

Finally, we set up a dedicated non-root user for security within the container.

Understanding LLVM and instrumentation

To understand why our setup needs LLVM, let's quickly review its role in the compilation process.

The LLVM infrastructure is the backend used by the Rust compiler (rustc):

• Frontend step: rustc reads the .rs files and translates the code into LLVM Intermediate Representation (IR).
  This IR code is generic (CPU agnostic).

• Optimizing step: The LLVM optimizer takes the IR code and applies various passes to reduce size and increase execution speed.
  The IR remains CPU agnostic but is now optimal.

• Backend step: The LLVM backend translates the optimized IR code into the real machine code (assembly) specific to the target CPU.
  This final executable binary is CPU-specific and can only be executed on that particular architecture.

The devcontainer.json configuration

To generate coverage data, the Rust compiler must know that the code needs to be "instrumented".

Instrumentation means adding counters directly into the binary during compilation.

This configuration is managed via environment variables in your devcontainer.json file:

• RUSTFLAGS: -Cinstrument-coverage
  It tells rustc to insert LLVM counters into the compiled binary.

• LLVM_PROFILE_FILE: ${containerWorkspaceFolder}/coverage-files/cargo-test-%p-%m.profraw
  It tells the instrumented binary where to save the raw coverage data (.profraw files) upon exit.
  The %p (PID) and %m (thread count) format specifiers ensure unique file names.

The grcov Crate

Grcov is a crate originally created by Mozilla.

• Role: Its primary goal is to aggregate the raw coverage data generated by LLVM and present it in a human-readable report (e.g., HTML) showing coverage percentages.

• Mechanism: Grcov is not an instrument itself.
  It depends entirely on the LLVM code coverage toolchain for data.
  It reads the raw .profraw files, correlates the counter data with the source code using the binary's debug information, and creates the report.

How Instrumentation Works?

When the binary is instrumented and executed:

• LLVM inserts instructions at the beginning of each function, after each conditional branch (if, else, match), and at the beginning and end of each line of code.

• These instructions simply increment a counter in the binary's memory.

• Upon successful execution, the binary uses these counter values to write the raw data (.profraw) showing exactly how many times each part of the program was executed.

The Python Script

The Python script orchestrates the entire four-step pipeline:

1. cargo clean: Cleans up previous builds to ensure the binary is compiled and instrumented from scratch.

2. cargo test: Compiles and launches the instrumented test binary (due to RUSTFLAGS).
   This execution generates the raw .profraw files.

3. llvm-profdata merge: This LLVM tool is called to combine all the generated .profraw files (one per test/thread) into a single, usable .profdata file.

4. grcov: Reads the final .profdata file and generates the final report (in our case, in HTML format).

The script also uses the get_llvm_profdata_path() function to dynamically locate the necessary llvm-profdata binary within the Rust toolchain path.

It makes thus the script portable across different operating systems and architectures (e.g., finding the path using the rustc --print host-tuple command).

The Rust code structure

The project code is divided into three parts to demonstrate various testing types:

• Functions (src/): Simple functions to check if a parameter was provided to the executable.

• Unit tests (in start.rs): Check the isolated functions, verifying success and failure logic.

• Integration tests (tests/): Placed in a separate folder, these tests check the complete binary execution, verifying its behavior when launched with and without the required parameter.

Executing the pipeline

First, you have to build and reopen your project with devcontainer.

Then to run the entire coverage pipeline, simply execute the Python script:

python check_cover.py

Upon successful completion, you can verify your coverage report here:

With this tutorial, you should achieve 100% coverage for both lines and functions.

If the percentage is sometimes inaccurate due to caching issues, restart your IDE, perform a manual cleanup, and restart the process.

Good job, you did it!