linux - How to run a command in a bash script from a loop with different arguments simultaneously in the background, while formatting the output

Question

The following snippet will run within the script, but it takes a long time for each loop.

#!/bin/bash
….
some_command $A $B $C | awk ‘{print$1}’ | while read -r var1; do
    printf "
$var1 
"
    printf "
"
    other_command $var1
    printf "
"
done
….

I tried running this, but the printf statements will run before the other_command, which is used to make the output a little more readable.

#!/bin/bash
….
some_command $A $B $C | awk ‘{print$1}’ | while read -r var1; do
    printf "
$var1 
"
    printf "
"|
    other_command $var1 &
    printf "
"
done
wait 
….

If I run just other_command with the & in loop I get the desired result but it it not very readable.

Answer 1

The idea is to move the long running command into a function, and call this function multiple times as child processes. This way they can run in parallel. Inside the function, the output from the command is first written into a local variable, and only printed in one go after the command is done.

#!/bin/bash

function do_something () {
  local OUTPUT="$(other_command "$var1")"
  printf "
%s

%s
" "$1" "$OUTPUT";
}

some_command "$A" "$B" "$C" | while read -r "var1" "_";
do
  do_something "$var1" &
done

wait

Please note that the order of the output will (probably) be different on every call, which is inherent to parallel execution.