Working with local and remote variables in ssh session

Question

I'm writing a script that should scp me some files after they have been processed on a remote server. To do that I need variables from the local server that executes the script and later some variables of the remote server as well.

I tried several ways I found on Stack Exchange, none worked for me so far.

This list the initial script:

#!/bin/bash

set -x

export someID=$1

LOG=/folder/subfolder1/$someID/tmp/getResult.log
export DIR=/folder/subfolder1/$someID
export rc=0
export lServ=`hostname`

case $lServ in
        test.server*)
            remoteServer=remote.test.server
        ;;
        prod.server*|backup.server*)
            remoteServer=remote.prod.server
        ;;
esac

cd $DIR

ssh -v $remoteServer << 'EOF' >> $LOG 2>&1

    cd $someID

    folders=`ls`

    for file in $folders; do
        case $file in
        *.1)
            scp -v $file/outputFolder/$file.someExt $lServ:$DIR/p.someExt
            rc=`expr $rc + $?`
        ;;
        *)
            export subID=`echo $file | awk -F"." '{print $2}'
            scp -v $file/outputFolder/$file.ext $lServ:$DIR/k$subID.someExt 
            rc=`expr $rc + $?`
        ;;
        esac
    done

    case $rc in
    0)
        cd ..
        rm -rfv $someID
        export rc=$?
    ;;
    *)
        echo "some Errormessage"
        rc=`expr $rc + 1`
    ;;
    esac
EOF

rc=`expr $rc + $?`

exit $rc

Due to company restrictions I can't post the real names or folder structure.

Solution 1

The missing piece in your script appears to be the variable rc. Your script (mostly) is running on your local machine, so you could

• make a temporary filename outside the redirected part,
• use that name inside the redirected part, and
• echo $rc into the temporary file, and
• read the variable outside the directed part.

For a temporary filename, that's often done with mktemp, so the pieces might look like

• outside

    mytemp=$(mktemp)
    trap "rm -f $mytemp" EXIT HUP INT QUIT

• inside

    # get my IP-address from `SSH_CLIENT`
    my_addr=\$(echo "\$SSH_CLIENT" |awk '{printf \$1;}')
    result=\$(mktemp)
    echo "\$rc" >\$result
    scp \$result \$my_addr:$mytemp
    rm -f \$result

• outside

    rc=$(cat $mytemp)

The reason why $someID is not passed from the local server to the remote is that your here-document beginning with

ssh -v $remoteServer << 'EOF' >> $LOG 2>&1

has quoted the 'EOF' marker, telling the shell to not expand the string before passing it to the ssh command. If you want the expansion, take away the quotes. However, when you do that, you will have to put a backslash before each $ character that you want the remote shell to expand.

It might be simpler to use the -q (quiet) option for ssh/scp, and reduce the output of your script to something that could easily be redirected into a variable.

Solution 2

You don't appear to need to run commands on the remote system except to examine files. So instead of running remote commands to examine files, mount the remote filesystem on the local machine, and use local commands to examine both remote and local files. To mount a remote filesystem that you access over SSH, use SSHFS.

mkdir remote
sshfs "$remoteServer": remote
cd "$remoteServer/$someID"
for file in */*; do …
fusermount -u remote

Comments

[Alessio]

you have to think about your variables (and command substitutions) in terms of local context or remote context. If you need to use a remote var/subst then backslash-escape it if it's in double-quotes, or use single-quotes if you don't need to insert local variables into the remote command. e.g. you're using a heredoc <<EOF above, which works like double-quotes, so if you want to use a remote variable in the heredoc script, you need to backslash-escape it. the same goes for command subst (i.e. backticks or $(...)). If not escaped, they will be local.

[Alessio]

BTW, these quoting rules aren't specific to ssh. They're the same if you, e.g., use shell vars directly in an awk script - you need to use double-quotes AND escape the awk $1, $2, etc field variables (it's usually easier to use single-quotes for the awk script AND then use -v awkvar="$shellvar" outside the awk script, to set an awk var for use inside the awk script).

[Alessio]

actually, i was wrong about the heredoc. you're using <<'EOF' - the single-quotes prevent interpolation of local variables. so to interpolate locals, you'll need to get rid of the single-quotes around EOF and then use backslashes to escape all remote variables and command substitutions.

[Galagenberg]

Hi, my first problem is that the variables set on the local server are not passed through to the remote server so it stops working properly at: cd $someID

[Galagenberg]

Thank you for your solution. I tried it but the systems I have to use are quite old and the requriements for sshfs are not met.