about how to run impala-shell within a shell script

Question

i have a problem when trying to execute this bash code:

function createImpalaPartition() {

period_id=$1;
database=$2
node=$3

actual_full=$(date -d@"$period_id" +%Y/%m/%d/%H/%M/)
template="use c2d;create EXTERNAL TABLE exptopology_$period_id (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/$actual_full'"

echo "template is $template";
#impala-shell -V -i $node -d $database -q $template
impala-shell -V -i $node -q $template
}

This is how i invoke it:

createImpalaPartition $actual $impalaDatabase $impalaNode

where

actual=$(date +'%s')
impalaDatabase="dbName"
impalaNode="name_of_the_node"

the execution of the script returns:

[core@dub-vcd-vms170 ~]$ createImpalaPartition $actual $impalaDatabase $impalaNode
template is use c2d;create EXTERNAL TABLE exptopology_1428326587 (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/2015/04/06/14/23/'
Error, could not parse arguments "c2d;create EXTERNAL TABLE exptopology_1428326587 (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/2015/04/06/14/23/'"
 Usage: impala_shell.py [options]

As you can see, i have to create tables using a shell script.

UPDATE:

following this link, i can see that impala-shell can be used that way, but i am not using the correct params.

I have used -f instead of -q, the same error remains. Can anybody help me, please?

Solution

You need to quote the expansion of $template for the entire string to be treated as a single argument to impala-shell:

impala-shell -V -i $node "$template"

where -V enables verbose output. For non-verbose (quiet) output replace -V with -q.

Comments

[Ani Menon]

With -q you cannot pass a USE statement and another query, fully qualify the names for any tables outside the default database. (Or use the -f option to pass a file with a USE statement followed by other queries.) Refer