[Code Coverage] Measuring Branch Coverage (@Source Code Level)

Contents

1. Introduction
2. Installing Dependencies
3. Approach to Solution
4. Understanding the Concept of Branch Coverage
5. Summary
6. References

---

1. Introduction

Measuring coverage of a software at source code level is to test how much of existing branches have been fallen through with given test inputs. Therefore, existing branches are first identified. Execution of software comes after for measuring the branch coverage.

---

2. Installing Dependencies

clang

clang, "provides a language front-end and tooling infrastructure for languages in the C language family for the LLVM project" [1]. We use clang to insert probes within a software built with C language in order to measure the coverage. It is best to search for documentations of each class when using clang api.

Installation Steps

• Automatic installation script [2]

$ bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"

• Installing all llvm packages

$ wget https://apt.llvm.org/llvm.sh
$ chmod +x llvm.sh
$ sudo ./llvm.sh <version number> all

• Path configuration (adding to .bashrc file)

export PATH="/usr/lib/llvm-<version>/bin:$PATH"

• Checking installation

$ clang --version

 

---

3. Approach to Solution

Most straight forward approach to measure branch coverage at source code level is to insert probes within the source code to invoke user defined functions to record whether the given branch has been taken (then branch) or not taken (else branch). Figure 1 shows an example of a probed inserted within an if condition (red colored texts). The concept of inserting such probes within the source code can be applied to other class of statements suches while, for, switch, and etc.

 

<Figure 1>

In more detail, the implementations of user defined functions should be inserted at the beginning of the target source code. This enables such insertion of probes within conditions to be executed for measuring coverage of each branch.

---

4. Understanding the Concept of Branch Coverage

Source Code Level vs. Binary Level

There are different type of coverage measurement based on the level of testing. Source Code Level and Binary Level. Here is an example of the confusion.

Given the following code: confusion.c

#include <stdio.h>

int main ( int argc, char *argv[] ) {
    int x = argc;

    if ( x >= 1 && x <= 5 ) {
        printf("x is between 1 and 5\n");
    } else {
        printf("x is not between 1 and 5\n");
    }

    return 0;
}

At source code level, two branches are identified. One when the if branch is taken and the other when the if branch is not taken.

 

However... at binary level (using gcov), it is shown that there are total of four branches. These four branches can be understood as when the first condition in the if branch is taken and not taken, and when the second condition of the if branch is taken and not taken.

 

<Figure 2>

Unconditional Jump (optimization)

In cases of uncondtional jump, corresponding branches can be ignore. This is due to the optimization of a compiler. During optimization such branches are taken as unconditional jump, not being considered as any sort of branch. Figure 3 shows some example code of an unconditional jump.

<Figure 3>

"Branches Executed" vs. "Taken At Least Once"

There is a big difference between branches executed and taken at least once. Looking back to the results of gcov in figure 2, branch executed has 100% coverage while taken at least once has 50%. Looking into the source code (figure 4), it is easily understood why such measurement is measured. As gcov measure branch coverage at binary level, both conditions results to true in which the branch of both condition not being taken is not covered.

 

<Figure 4>

It is quite important to understand this difference when measuring branch coverage. It is very tempting to measure coverage with branch execution because of its high percentage, however, it is not as precise as measuring branch coverage whether or not it was taken or not.

---

5. Summary

To measure the branch coverage of a target software at source code level, such probes to handle the measurement is inserted within the target source code. When this instrumented source code is executed with given test inputs, branch coverage is dynamically measured during the execution of the program.

 

Implementations can be found in my github repository ccov.

---

6. References

[1] https://clang.llvm.org/get_started.html

Clang - Getting Started

[2] https://apt.llvm.org/

LLVM Debian/Ubuntu packages

[3] PPT - Containing questions and misunderstandings dealt during study

kcov-v1.pptx

0.25MB

[4] PDF - Questions answerd after presentation

kcov-v1.pdf

3.69MB