gradle - SonarQube, JaCoCo: (11 of 22 conditions) when they are supposed to be 3 (or 4)

Question

I am lost on how SonarQube calculates conditions covered by tests.

Versions of tools used: * JaCoCo 0.8.1 * SonarQube 7.4

This is my groovy code

    boolean condition1(boolean b1, boolean b2) {
        !b1 || !b2
    }

    boolean condition2(boolean b1, boolean b2) {
        b1 || b2
    }

    boolean condition3(boolean b1, boolean b2) {
        !b1 && !b2
    }

    boolean condition4(boolean b1, boolean b2) {
        b1 && b2
    }

    boolean condition5(boolean b1, boolean b2) {
        b1 && !b2
    }

    boolean condition6(boolean b1, boolean b2, boolean b3) {
        b1 && b2 && b3
    }

Here are the tests

    void "test condition 1"() {
        expect:
        service.condition1(c1,c2)

        where:
        c1    | c2
        true  | true
        true  | false
        false | true
        false | false
    }

    void "test condition 2"() {
        expect:
        service.condition2(c1,c2)

        where:
        c1    | c2
        true  | true
        true  | false
        false | true
        false | false
    }

    void "test condition 3"() {
        expect:
        service.condition3(c1,c2)

        where:
        c1    | c2
        true  | true
        true  | false
        false | true
        false | false
    }

    void "test condition 4"() {
        expect:
        service.condition4(c1,c2)

        where:
        c1    | c2
        true  | true
        true  | false
        false | true
        false | false
    }

    void "test condition 5"() {
        expect:
        service.condition5(c1,c2)

        where:
        c1    | c2
        true  | true
        true  | false
        false | true
        false | false
    }

    void "test condition 6"() {
        expect:
        service.condition6(c1, c2, c3)

        where:
        c1    | c2    | c3
        true  | true  | true
        true  | true  | false
        true  | false | true
        true  | false | false
        false | true  | true
        false | true  | false
        false | true  | true
        false | true  | false
        false | false | false
    }

The code coverage report says those conditions are not satisfied and the followings are the only info I get

condition1. (11 of 22 conditions)
condition2. (7 of 14 conditions)
condition3. (11 of 22 conditions)
condition4. (7 of 14 conditions)
condition5. (9 of 18 conditions)
condition6. (11 of 22 conditions)

That means I am not able to reach 100% of covered tests although I believe logically did.

I am aware of SonarQube documentation https://docs.sonarqube.org/latest/user-guide/metric-definitions/ where it says

On each line of code containing some boolean expressions, the condition coverage simply answers the following question: 'Has each boolean expression been evaluated both to true and false?'. This is the density of possible conditions in flow control structures that have been followed during unit tests execution

Anyone has an idea on how this actually works and what I am doing wrong here?

Answer 1

From similar questions (such as "Why is JaCoCo not covering my String switch statements?" and "How does assert groupType != null contain 4 branches")

JaCoCo performs analysis of bytecode

thus similarly - take a look at bytecode.

For Example.groovy

class Example {
    boolean condition2(boolean b1, boolean b2) {
        b1 || b2
    }
}

Groovy compiler (groovyc --version)

Groovy compiler version 3.0.0-rc-1
Copyright 2003-2019 The Apache Software Foundation. http://groovy-lang.org/

generates (groovyc Example.groovy) following bytecode (javap -v -p Example.class)

  public boolean condition2(boolean, boolean);
    descriptor: (ZZ)Z
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=4, args_size=3
         0: invokestatic  #20                 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
         3: astore_3
         4: invokestatic  #38                 // Method org/codehaus/groovy/runtime/BytecodeInterface8.isOrigZ:()Z
         7: ifeq          25
        10: getstatic     #40                 // Field __$stMC:Z
        13: ifne          25
        16: invokestatic  #43                 // Method org/codehaus/groovy/runtime/BytecodeInterface8.disabledStandardMetaClass:()Z
        19: ifne          25
        22: goto          42
        25: iload_1
        26: ifne          33
        29: iload_2
        30: ifeq          37
        33: iconst_1
        34: goto          38
        37: iconst_0
        38: ireturn
        39: nop
        40: nop
        41: athrow
        42: iload_1
        43: ifne          50
        46: iload_2
        47: ifeq          54
        50: iconst_1
        51: goto          55
        54: iconst_0
        55: ireturn
        56: nop
        57: nop
        58: nop
        59: nop
        60: nop
        61: nop
        62: nop
        63: nop
        64: athrow
      LineNumberTable:
        line 2: 4
        line 3: 25
        line 4: 39
        line 3: 42
        line 4: 56
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0      56     0  this   LExample;
            0      56     1    b1   Z
            0      56     2    b2   Z

which contains 14 branches (2 branches per each conditional jump instruction ifeq and ifne), and your tests cover only half of them (execution path where first ifeq at offset 7 jumps to offset 25), which is absolutely consistent with what is reported by JaCoCo and hence shown in SonarQube.

And following discussion seems to be relevant here - http://groovy.329449.n5.nabble.com/Branch-coverage-issues-td5686725.html , because compilation of the same using --indy option (groovyc --indy Example.groovy) produces following bytecode

  public boolean condition2(boolean, boolean);
    descriptor: (ZZ)Z
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=3, args_size=3
         0: iload_1
         1: ifne          8
         4: iload_2
         5: ifeq          12
         8: iconst_1
         9: goto          13
        12: iconst_0
        13: ireturn
        14: nop
        15: nop
        16: nop
        17: nop
        18: nop
        19: nop
        20: nop
        21: nop
        22: athrow
      LineNumberTable:
        line 3: 0
        line 4: 14
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0      14     0  this   LExample;
            0      14     1    b1   Z
            0      14     2    b2   Z

which contains only 4 branches.