JQ error: is not defined at <top-level> when trying to add values to jq template

Question

I have a .jq template that I want to update values under the samples list, formatted as:

{
  "test": "abc",
  "p": "1",
  "v": "1.0.0",
  "samples": [
    {
      "uptime": $uptime,
      "curr_connections": $curr_connections,
      "listen_disabled_num": $listen_disabled_num,
      "conn_yields": $conn_yields,
      "cmd_get": $cmd_get,
      "cmd_set": $cmd_set,
      "bytes_read": $bytes_read,
      "bytes_written": $bytes_writtem,
      "get_hits": $get_hits,
      "rejected_connections": $rejected_connections,
      "limit_maxbytes": $limit_maxbytes,
      "cmd_flush": $cmd_flush
    }
  ]
}

My shell script to do this is below, I am basically running a command to pull some memcached output stats and want to insert some of the results into the jq template as key/values.

JQ=`cat template.jq`

SAMPLES=(uptime curr_connections listen_disabled_num conn_yields cmd_get cmd_set cmd_flush bytes_read bytes_written get_hits rejected_connections limit_maxbytes)

for metric in ${SAMPLES[*]}
do
  KEY=$(echo stats | nc $HOST $PORT | grep $metric | awk '{print $2}')
  VALUE=$(echo stats | nc $HOST $PORT | grep $metric | awk '{print $3}')

  echo "Using KEY: $KEY with value: $VALUE"

  jq -n --argjson $KEY $VALUE -f template.jq
done

Not sure if this is the best way to handle this scenario, but I am getting a ton of errors such as:

jq: error: conn_yields/0 is not defined at <top-level>, line 12:
      "conn_yields": $conn_yields,
jq: error: cmd_get/0 is not defined at <top-level>, line 13:
      "cmd_get": $cmd_get,
jq: error: cmd_set/0 is not defined at <top-level>, line 14:
      "cmd_set": $cmd_set,

Solution

If you are going to invoke jq using -f template.jq, then each of the $-variables in template.jq will have to be set separately on the command-line, one by one. In your case, this does not look like a very happy option.

If you are stuck with template.jq as it is, then it will be hard slogging, though there are alternatives besides setting the $-variables on the command line.

Please see https://github.com/stedolan/jq/wiki/Cookbook#using-jq-as-a-template-engine in the jq Cookbook for an alternative to using $-variables. Consider for example the implications of this illustration of "destructuring":

jq -nc '{a:1,b:2} as {a: $a, b:$b} | [$a,$b]'
[1,2]  

Another alternative

In your particular case, you could replace all the "$" characters in template.jq with ".", and then pass in a JSON object with the appropriate keys; e.g. change $uptime to .uptime, and then include a key/value pair for uptime.

Comments

[nobrac]

Thanks for the feedback! When you say "set at the command line", do you mean to set them as Environment variables? @peak

[peak]

No. There is no relationship between jq $-variables and shell or environment variables. Please see the Cookbook and/or the jq manual for details.