how to re-run the "curl" command automatically when the error occurs

linux bash curl

11,973

Solution 1

Here's a bash snippet I use to perform exponential back-off:

# Retries a command a configurable number of times with backoff.
#
# The retry count is given by ATTEMPTS (default 5), the initial backoff
# timeout is given by TIMEOUT in seconds (default 1.)
#
# Successive backoffs double the timeout.
function with_backoff {
  local max_attempts=${ATTEMPTS-5}
  local timeout=${TIMEOUT-1}
  local attempt=1
  local exitCode=0
  while (( $attempt < $max_attempts ))
  do
    if "[email protected]"
    then
      return 0
    else
      exitCode=$?
    fi
    echo "Failure! Retrying in $timeout.." 1>&2
    sleep $timeout
    attempt=$(( attempt + 1 ))
    timeout=$(( timeout * 2 ))
  done
  if [[ $exitCode != 0 ]]
  then
    echo "You've failed me for the last time! ([email protected])" 1>&2
  fi
  return $exitCode
}

Then use it in conjunction with any command that properly sets a failing exit code:

with_backoff curl 'http://monkeyfeathers.example.com/'

Solution 2

Perhaps this will help. It will try the command, and if it fails, it will tell you and pause, giving you a chance to fix run-my-script.

COMMAND=./run-my-script.sh 
until $COMMAND; do 
    read -p "command failed, fix and hit enter to try again."
done

11,973

erical

Updated on June 04, 2022

Comments

erical 6 months
Sometimes when I execute a bash script with the curl command to upload some files to my ftp server, it will return some error like:
```
56 response reading failed
```
and I have to find the wrong line and re-run them manually and it will be OK.

I'm wondering if that could be re-run automatically when the error occurs.

My scripts is like this:
```
#there are some files(A,B,C,D,E) in my to_upload directory,
# which I'm trying to upload to my ftp server with curl command
for files in `ls` ;
    do curl -T $files ftp.myserver.com --user ID:pw ;
done
```
But sometimes A,B,C, would be uploaded successfully, only D were left with an "error 56", so I have to rerun curl command manually. Besides, as Will Bickford said, I prefer that no confirmation will be required, because I'm always asleep at the time the script is running. :)
- Kevin almost 11 years
  
  You can set it up to run until it succeeds, but that won't help if you don't fix the problem.
- daxelrod almost 11 years
  
  I'm confused, are you just looking to rerun the failed curl commands until they succeed, or the whole script?
Will Bickford almost 11 years

You could also use sleep instead of read -p so you wouldn't have to confirm each retry.
Kevin almost 11 years

The point of reading is to give him time to fix whatever's wrong. That'll require an indeterminate time and interaction, so I don't think there will be a problem with requiring confirmation.
erical almost 11 years

Thanks for your help,phs:) I can feel this is a good script,but I'm a newbie in linux,so it will take me a little time to understand it.
erical almost 11 years

Hi,kevin.I think your solution can really help me.
Kevin almost 11 years

Glad to help. You could also combine it with phs's function above to run it a few times before asking you to fix it.
Kevin almost 11 years

Just put everything you want to rerun into a separate script, including phs's backoff if you want it, and run that script like my example.
erical almost 11 years

sorry，I made a mistake,I mean I am not try to rerun the whole script
Kevin almost 11 years

Well, either way, just put whatever part you want to rerun (phs's function, a simple curl call, or an external script) into $COMMAND.
erical almost 11 years

but I'm not sure which part should be rerun,please take a look at the new answer by me
drewish almost 11 years

I wish you could favorite answers because I'm sure I'll come back looking for this later.
phs almost 11 years

@user1076599 Just to get started, you could paste that into your script somewhere near the top, and remove the set +e and set -e lines. Then you'll be able to use it within your script. To get fancier, you might put it in your shell's login script (~/.profile, ~/.bash_profile or ~/.bashrc) instead.
erical almost 11 years

oh,I can't wait to try this,this time I hope that error 56 would happen again,the more the better
Meng over 7 years

The "while [[ $attempt < $max_attempts ]]" will cause the program to exit prematurely, because this is actually doing the string comparison rather than comparing integers. And it will always fail if the max_attempts is set to greater than 9. The solution is to use "while (( $attempt < $max_attempts ))" instead.
phs over 7 years

@MLin: Whoops! Thanks for the catch, incorporated.
Fernando Correia almost 5 years

The message ends with 2 dots. This fixes that and adds the unit: echo "Failure! Retrying in ${timeout}s..." 1>&2
Fernando Correia over 4 years

Use attempt=1 to perform exactly the number of attempts defined by $ATTEMPTS. With attempts=0 it does one extra. See gist.github.com/fernandoacorreia/…