Testing os.Exit scenarios in Go with coverage information (coveralls.io/Goveralls)

Question

This question: How to test os.exit scenarios in Go (and the highest voted answer therein) sets out how to test os.Exit() scenarios within go. As os.Exit() cannot easily be intercepted, the method used is to reinvoke the binary and check the exit value. This method is described at slide 23 on this presentation by Andrew Gerrand (one of the core members of the Go team); the code is very simple and is reproduced in full below.

The relevant test and main files look like this (note that this pair of files alone is an MVCE):

package foo

import (
    "os"
    "os/exec"
    "testing"
)

func TestCrasher(t *testing.T) {
    if os.Getenv("BE_CRASHER") == "1" {
        Crasher() // This causes os.Exit(1) to be called
        return
    }
    cmd := exec.Command(os.Args[0], "-test.run=TestCrasher")
    cmd.Env = append(os.Environ(), "BE_CRASHER=1")
    err := cmd.Run()
    if e, ok := err.(*exec.ExitError); ok && !e.Success() {
        fmt.Printf("Error is %v
", e)
    return
    }
    t.Fatalf("process ran with err %v, want exit status 1", err)
}

and

package foo

import (
    "fmt"
    "os"
)

// Coverage testing thinks (incorrectly) that the func below is
// never being called
func Crasher() {
    fmt.Println("Going down in flames!")
    os.Exit(1)
}

However, this method appears to suffer certain limitations:

1. Coverage testing with goveralls / coveralls.io does not work - see for instance the example here (the same code as above but put into github for your convenience) which produces the coverage test here, i.e. it does not record the test functions being run. NOTE that you don't need to those links to answer the question - the above example will work fine - they are just there to show what happens if you put the above into github, and take it all the way through travis to coveralls.io

2. Rerunning the test binary appears fragile.

Specifically, as requested, here is a screenshot (rather than a link) for the coverage failure; the red shading indicates that as far as coveralls.io is concerned, Crasher() is not being called.

Is there a way around this? Particularly the first point.

At a golang level the problem is this:

• The Goveralls framework runs go test -cover ..., which invokes the test above.

• The test above calls exec.Command / .Run without -cover in the OS arguments

• Unconditionally putting -cover etc. in the argument list is unattractive as it would then run a coverage test (as the subprocess) within a non-coverage test, and parsing the argument list for the presence of -cover etc. seems a heavy duty solution.

• Even if I put -cover etc. in the argument list, my understanding is that I'd then have two coverage outputs written to the same file, which isn't going to work - these would need merging somehow. The closest I've got to that is this golang issue.

---

Summary

What I am after is a simple way to run go coverage testing (preferably via travis, goveralls, and coveralls.io), where it is possible to both test cases where the tested routine exits with OS.exit(), and where the coverage of that test is noted. I'd quite like it to use the re-exec method above (if that can be made to work) if that can be made to work.

The solution should show coverage testing of Crasher(). Excluding Crasher() from coverage testing is not an option, as in the real world what I am trying to do is test a more complex function, where somewhere deep within, under certain conditions, it calls e.g. log.Fatalf(); what I am coverage testing is that the tests for those conditions works properly.

Answer 1

Waitting for answers