Convert JSON Array to Bash

Question

I'm using JQ to fetch some JSON from a quiz database and I want to parse the results. I'm trying to save a resulting array in Bash as shown below but the format is that used in JavaScript/Python with square brackets rather than Bash style.

quiz=$(curl -s https://opentdb.com/api.php?amount=3)
answers=$(echo $quiz | jq '[ .results][0][0].incorrect_answers')
correct=$(echo $quiz | jq '[ .results][0][0].correct_answer')
answers+=( $correct )

An example of what answers looks like is below:

[ "Excitement", "Aggression", "Exhaustion" ]

The correct answer is never pushed to the array due to the wrong format.

How can I convert an array of the format shown above so that it is interpreted in my script as an array.

Example of output from curl (this is not hard-coded, question and answer is different every-time):

{
  "response_code": 0,
  "results": [
    {
      "category": "Entertainment: Television",
      "type": "multiple",
      "difficulty": "easy",
      "question": "Which company has exclusive rights to air episodes of the "The Grand Tour"?",
      "correct_answer": "Amazon",
      "incorrect_answers": [
        "Netflix",
        "BBC",
        "CCTV"
      ]
    },
    {
      "category": "Celebrities",
      "type": "multiple",
      "difficulty": "medium",
      "question": "How much weight did Chris Pratt lose for his role as Star-Lord in "Guardians of the Galaxy"?",
      "correct_answer": "60 lbs",
      "incorrect_answers": [
        "30 lbs",
        "50 lbs",
        "70 lbs"
      ]
    },
    {
      "category": "Animals",
      "type": "boolean",
      "difficulty": "easy",
      "question": "The Killer Whale is considered a type of dolphin.",
      "correct_answer": "True",
      "incorrect_answers": [
        "False"
      ]
    }
  ]
}

Solution 1

According to the stated goal "I want the bash array to include correct_answer and incorrect_answers" you expect to get an array of combined values.
No need to run 2 different jq commands, the whole combined sequence can be composed with a single jq expression:

For single-word array items:

$ declare -a answers=($(jq -r '[.results[0].correct_answer] + .results[0].incorrect_answers | @sh' <<<"$quiz"))

@sh:

The input is escaped suitable for use in a command-line for a POSIX shell. If the input is an array, the output will be a series of space-separated strings.

Results:

$ echo "${answers[2]}"
'BBC'
$ echo "${answers[0]}"
'Amazon'
$ echo "${answers[1]}"
'Netflix'
$ echo "${answers[@]}"
'Amazon' 'Netflix' 'BBC' 'CCTV'

---

To handle items with spaces (like those in .results[1]) use the following approach with readarray and @json option:

$ readarray -t answers < <(jq -r '[.results[1].correct_answer] + .results[1].incorrect_answers | .[] |@json' <<<"$quiz")

Results:

$ echo "${answers[1]}"
"30 lbs"
$ echo "${answers[0]}"
"60 lbs"
$ echo "${answers[@]}"
"60 lbs" "30 lbs" "50 lbs" "70 lbs"
$ echo "${answers[2]}"
"50 lbs"

Solution 2

I'd get jq to output the results line-wise. Then use the bash mapfile command to read the lines into an array

mapfile -t correct < <(jq -r '.results[] | .correct_answer' <<<"$quiz")
declare -p correct

declare -a correct=([0]="Amazon" [1]="60 lbs" [2]="True")

For the incorrect answers, since bash does not have multi-dimensional arrays, I'd get jq to output the array of incorrect answers as CSV:

mapfile -t incorrect < <(jq -r '.results[] | .incorrect_answers | @csv' <<<"$quiz")
declare -p incorrect

declare -a incorrect=([0]="\"Netflix\",\"BBC\",\"CCTV\"" [1]="\"30 lbs\",\"50 lbs\",\"70 lbs\"" [2]="\"False\"")

Then:

$ echo "q $i: ans=${correct[i]}; wrong=${incorrect[i]}"
q 1: ans=60 lbs; wrong="30 lbs","50 lbs","70 lbs"

Documentation:

• Process Substitution
• Here Strings

---

Assuming you're interacting with the user:

i=0
read -p "Answer to question $i: " answer

if [[ $answer == "${correct[i]}" ]]; then
    echo Correct
elif [[ ${incorrect[i]} == *"\"$answer\""* ]]; then
    echo Incorrect
else
    echo Invalid answer
fi

Keep in mind that within [[...]] the == operator is not a string equality operator, it is a pattern matching operator.

• the first test is a straightforward string comparison
• the second test sees if the CVS string holding the incorrect answers contains the answer in double quotes

Solution 3

Extending @RomanPerekhrest's answer (go upvote it now):

mapfile -t answers < <(jq -r '.results[] | [.correct_answer] + .incorrect_answers | @sh' <<<"$quiz")
declare -p answers

declare -a answers=([0]="'Amazon' 'Netflix' 'BBC' 'CCTV'" [1]="'60 lbs' '30 lbs' '50 lbs' '70 lbs'" [2]="'True' 'False'")

Then, you can use something like this

for i in "${!answers[@]}"; do
    declare -a "this_answers=( ${answers[i]} )"
    echo $i
    printf " > %s\n" "${this_answers[@]}"
done

0
 > Amazon
 > Netflix
 > BBC
 > CCTV
1
 > 60 lbs
 > 30 lbs
 > 50 lbs
 > 70 lbs
2
 > True
 > False

Solution 4

Because this question is lodged in my head, a little script to run the game:

#!/usr/bin/env bash

main() {
    local -a questions
    mapfile -t questions < <(
        get_quiz |
        jq -r '.results[] | [.question] + [.correct_answer] + .incorrect_answers | @sh'
    )

    for i in "${!questions[@]}"; do
        local -a q="( ${questions[i]} )"
        question $((i+1)) "${q[@]}"
    done
}

question() {
    local num=$1 question=$2 correct=$3
    local -a answers=("${@:3}")

    # shuffle the answers
    mapfile -t answers < <(printf "%s\n" "${answers[@]}" | sort -R)

    echo
    echo "Question #$num"
    PS3="${question//&quot;/\'} "

    select answer in "${answers[@]}"; do
        if [[ $answer == "$correct" ]]; then
            echo "Correct"
            break
        fi
        echo "Incorrect"
    done
}

get_quiz() {
    # this is where you'd do your curl call, but 
    # for now, the sample data is printed.
    cat << 'JSON'
    {
      "response_code": 0,
      "results": [
        {
          "category": "Entertainment: Television",
          "type": "multiple",
          "difficulty": "easy",
          "question": "Which company has exclusive rights to air episodes of the &quot;The Grand Tour&quot;?",
          "correct_answer": "Amazon",
          "incorrect_answers": [
            "Netflix",
            "BBC",
            "CCTV"
          ]
        },
        {
          "category": "Celebrities",
          "type": "multiple",
          "difficulty": "medium",
          "question": "How much weight did Chris Pratt lose for his role as Star-Lord in &quot;Guardians of the Galaxy&quot;?",
          "correct_answer": "60 lbs",
          "incorrect_answers": [
            "30 lbs",
            "50 lbs",
            "70 lbs"
          ]
        },
        {
          "category": "Animals",
          "type": "boolean",
          "difficulty": "easy",
          "question": "The Killer Whale is considered a type of dolphin.",
          "correct_answer": "True",
          "incorrect_answers": [
            "False"
          ]
        }
      ]
    }
JSON
}

main

View more solutions

Comments

[jesse_b]

So what do you want your bash array to contain?

[Philip Kirkbride]

@Jesse_b I want the bash array to include correct_answer and incorrect_answers.

[Philip Kirkbride]

In that case I apologize, I should have mentioned that the JSON I posted was an example of the format but I have no knowledge of the specific text.

[Angel Todorov]

Which you store in the quiz variable, which I use in my answer.

[Michael Homer]

Try this with .results[1] - the quotes are literal.

[Angel Todorov]

Oh yeah, that's much better than mine

[Philip Kirkbride]

This would be perfect, I'm getting black when I echo though.

[Angel Todorov]

Are you sure you have valid contents in $quiz?

[Philip Kirkbride]

Sorry I had deleted my quiz line. This works great thanks!

[RomanPerekhrest]

@MichaelHomer, ok, fixed

[RomanPerekhrest]

@PhilipKirkbride, please recheck the 2nd approach to cover items with spaces

[Michael Homer]

Is there something about the question that makes including the quotes desirable? I don't see it, but since this was accepted while they were also there...

[Philip Kirkbride]

This is awesome, beyond what I planned to do but I'll use this now!

[Angel Todorov]

Don't just use it. Understand it. Ask questions about bits you don't get.

[Philip Kirkbride]

Something I realized after accepting is that when the answers are more than one word each word seems to be a separate item in the array.

[RomanPerekhrest]

@PhilipKirkbride, that's why I wrote my previous comment to you. Not sure if that comment was even read

[Philip Kirkbride]

@RomanPerekhrest sorry missed it

[vijay]

Any way to handle values with newlines in them? I'm still looking. Looks like I'll have to write it myself.

[vijay]

See unix.stackexchange.com/a/630274/42620 for a way to handle newlines and other arbitrary values.