quoting - Double quotes inside quotes in bash

Question

I need pass $var_path to bash script inside single quotes and then get commnd executed on remote host. I know that single quotes do not allow passing variables, so tried double quoting, but getting error in sed. Assume this happens because its template uses " as well.

var="Test text"
var_path="/etc/file.txt"
echo "$var"|ssh root@$host 'cat - > /tmp/test.tmp && sed -n "/]/{:a;n;/}/b;p;ba}" $var_path > /tmp/new.conf.tmp'

so with ""

var="Test text"
    var_path="/etc/file.txt"
    echo "$var"|ssh root@$host "cat - > /tmp/test.tmp && sed -n "/]/{:a;n;/}/b;p;ba}" $var_path > /tmp/new.conf.tmp"

Errors from sed

sed: -e expression #1, char 0: unmatched `{'
./script.sh: line 4: n: command not found
./script.sh: line 4: /}/b: No such file or directory
./script.sh: line 4: p: command not found

If $var_path used directly without substitution script works as expected.

Answer 1

Arguments parsed as part of the command to send to the remote system in SSH in the are concatenated with spaces and then passed to the remote shell (in a manner similar to "${SHELL:-sh}" -c "$*"). Fortunately, bash has the built-in printf %q operation (an extension, so not available in all other POSIX shells) to make strings eval-safe, and thus ssh-safe, if your remote SHELL is bash; see the end of this answer for a workaround using Python's shlex module to generate a command safe in non-bash shells.

So, if you have a command that works locally, the first step is to put it into an array (notice also the quotes around the expansion of "$var_path" -- these are necessary to have an unambiguous grouping):

cmd=( sed -n '/]/{:a;n;/}/b;p;ba}' "$var_path" )

...which you can run locally to test:

"${cmd[@]}"

...or transform into an eval-safe string:

printf -v cmd_str '%q ' "${cmd[@]}"

...and then run it locally with ssh...

ssh remote_host "$cmd_str"

...or test it locally with eval:

eval "$cmd_str"

Now, your specific use case has some exceptions -- things that would need to be quoted or escaped to use them as literal commands, but which can't be quoted if you want them to retain their special meaning. &&, | and > are examples. Fortunately, you can work around this by building those parts of the string yourself:

ssh remote_host "cat - >/tmp/test.tmp && $cmd_str >/tmp/new.conf.tmp"

...which is equivalent to the local array expansion...

cat - >/tmp/test.tmp && "${cmd[@]}" >/tmp/new.conf.tmp

...or the local eval...

eval "cat - >/tmp/test.tmp && $cmd_str >/tmp/new.conf.tmp"

---

Addendum: Supporting Non-Bash Remote Shells

One caveat: printf %q is guaranteed to quote things in such a way that bash (or ksh, if you're using printf %q in ksh locally) will evaluate them to exactly match the input. If you had a target system with a shell which didn't support extensions to POSIX such as $'', this guarantee would no longer be available, and more interesting techniques become necessary to guarantee robustness in the face of the full range of possible inputs.

Fortunately, the Python standard library includes a function to escape arbitrary strings for any POSIX-compliant shell:

quote_string() {
  python -c '
import sys
try:
  from pipes import quote  # Python 2
except ImportError:
  from shlex import quote  # Python 3

print(" ".join([quote(x) for x in sys.argv[1:]]))
' "$@"
}

Thereafter, when you need an equivalent to printf '%q ' "$@" that works even when the remote shell is not bash, you can run:

cmd_str=$(quote_string "${cmd[@]}")