logging - Bash: log stdout and stderr while preserving order and provenance

Question

It's fairly simple to log both the stdout and the stderr of a command to a log file:

./foo.sh &> log.txt

The problem is that when inspecting the log file, one doesn't know anymore which line was coming from which stream. This could be fixed by redirecting stdout and stderr to two separate files, but then the chronology and interleaving of the output is lost.

An other solution would be to redirect to three files. One with the stdout, one with the stderr, and one with both combined. Something like:

./foo.sh 2> >(tee stderr | tee -a combined) 1> >(tee stdout | tee -a combined)

But that is not be very elegant to have so many files (and this command still dumps a copy of the output on the shell).

I found an interesting bash function that would color only stderr messages in red:

color()(set -o pipefail;"$@" 2>&1>&3|sed $'s,.*,e[31m&e[m,'>&2)3>&1

but it doesn't preserve the order of the output and the result is unreadable in a text editor. Given the following program for foo.sh:

for i in 1 2; do
   for j in 1 2; do
     printf '%s
' "out $i"
   done
   for k in 1 2; do
     printf '%s
' "err $i" >&2
   done
done

Running color ./foo.sh produces:

out 1
out 1
out 2
out 2
[31merr 1[m
[31merr 1[m
[31merr 2[m
[31merr 2[m

How could one easily end up with something such as this in a single log file ?

@| out 1
@| out 1
$| err 1
$| err 1
@| out 2
@| out 2
$| err 2
$| err 2

Answer 1

maybe you are looking for script it records both stdout, stderr and the commands...it starts a new shell in which it records everything (or use -c _cmd_)

 $ script tx1

your color() function breaks order because sed is buffering...