c - Unit testing for failed malloc()

Question

What is the best way for unit testing code paths involving a failed malloc()? In most instances, it probably doesn't matter because you're doing something like

thingy *my_thingy = malloc(sizeof(thingy));
if (my_thingy == NULL) {
  fprintf(stderr, "We're so screwed!
");
  exit(EXIT_FAILURE);
} 

but in some instances you have choices other than dying, because you've allocated some extra stuff for caching or whatever, and you can reclaim that memory.

However, in those instances where you can try to recover from a failed malloc() that you're doing something tricky and error prone in a code path that's pretty unusual, making testing especially important. How do you actually go about doing this?

Answer 1

I saw a cool solution to this problem which was presented to me by S. Paavolainen. The idea is to override the standard malloc(), which you can do just in the linker, by a custom allocator which

1. reads the current execution stack of the thread calling malloc()
2. checks if the stack exists in a database that is stored on hard disk
   1. if the stack does not exist, adds the stack to the database and returns NULL
   2. if the stack did exist already, allocates memory normally and returns

Then you just run your unit test many times: this system automatically enumerates through different control paths to malloc() failure and is much more efficient and reliable than e.g. random testing.