Downloading json file from json file curl

Question

I have a json file with the structure seen below:

{
    url: "https://mysite.com/myjsonfile",
    version_number: 69,
}

This json file is accessed from mysite.com/myrootjsonfile

I want to run a load data script to access mysite.com/myrootjsonfile and load the json content from the url field using curl and save the resulting content to local storage.

This is my attempt so far.

curl -o assets/content.json 'https://mysite.com/myrootjsonfile' | grep -Po '(?<="url": ")[^"]*' 

unfortunately, instead of saving the content from mysite.com/myjsonfile its saving the content from above: mysite.com/myrootjsonfile. Can anyone point out what i might be doing wrong? Bear in mind in a completely new to curl. Thanks!

Solution 1

It is saving the content from myrootjsonfile because that is what you are telling curl to do - to save that file to the location assets/content.json, and then greping stdin, which is empty. You need to use two curl commands, one to download the root file (and process it to find the URL of the second), and the second to download the actual content you want. You can use command substitution for this:

my_url=$(curl https://mysite.com/myrootjsonfile | grep -Po '(?<=url: )[^,]*')
curl -o assets/content.json "$my_url"

I also changed the grep regex - this one matches a string of non-comma characters which follow after "url: ".

Solution 2

Assuming you wished to save the file to assets/content.json, note that flags are case sensitive.

Use -o instead of -O to redirect the output to assets/content.json.