What is the purpose of using shift in shell scripts?

shell-script arguments

253,326

Solution 1

shift is a bash built-in which kind of removes arguments from the beginning of the argument list. Given that the 3 arguments provided to the script are available in $1, $2, $3, then a call to shift will make $2 the new $1. A shift 2 will shift by two making new $1 the old $3. For more information, see here:

Solution 2

As goldilocks’ comment and humanity’s references describe, shift reassigns the positional parameters ($1, $2, etc.) so that $1 takes on the old value of $2, $2 takes on the value of $3, etc.^* The old value of $1 is discarded. ($0 is not changed.) Some reasons for doing this include:

It lets you access the tenth argument (if there is one) more easily. $10 doesn’t work – it’s interpreted as $1 concatenated with a 0 (and so might produce something like Hello0). After a shift, the tenth argument becomes $9. (However, in most modern shells, you can use ${10}.)
As the Bash Guide for Beginners demonstrates, it can be used to loop through the arguments. IMNSHO, this is clumsy; for is much better for that.
As in your example script, it makes it easy to process all of the arguments the same way except for a few. For example, in your script, $1 and $2 are text strings, while $3 and all other parameters are file names.

So here’s how it plays out. Suppose your script is called Patryk_script and it is called as

Patryk_script USSR Russia Treaty1 Atlas2 Pravda3

The script sees

$1 = USSR
$2 = Russia
$3 = Treaty1
$4 = Atlas2
$5 = Pravda3

The statement ostr="$1" sets variable ostr to USSR. The first shift statement changes the positional parameters as follows:

$1 = Russia
$2 = Treaty1
$3 = Atlas2
$4 = Pravda3

The statement nstr="$1" sets variable nstr to Russia. The second shift statement changes the positional parameters as follows:

$1 = Treaty1
$2 = Atlas2
$3 = Pravda3

And then the for loop changes USSR ($ostr) to Russia ($nstr) in the files Treaty1, Atlas2, and Pravda3.

There are a few problems with the script.

```
for file in $@; do
```
If the script is invoked as
```
Patryk_script USSR Russia Treaty1 "World Atlas2" Pravda3
```
it sees
```
$1 = USSR
$2 = Russia
$3 = Treaty1
$4 = World Atlas2
$5 = Pravda3
```
but, because $@ isn’t quoted, the space in World Atlas2 isn’t quoted, and the for loop thinks it has four files: Treaty1, World, Atlas2, and Pravda3. This should be either
```
for file in "$@"; do
```
(to quote any special characters in the arguments) or simply
```
for file do
```
(which is equivalent to the longer version).
```
eval "sed 's/"$ostr"/"$nstr"/g' $file"
```
There’s no need for this to be an eval, and passing unchecked user input to an eval can be dangerous. For example, if the script is invoked as
```
Patryk_script "'; rm *;'" Russia Treaty1 Atlas2 Pravda3
```
it will execute rm *! This is a big concern if the script can be run with privileges higher than those of the user who invokes it; e.g., if it can be run via sudo or invoked from a web interface. It’s probably not so important if you just use it as yourself, in your directory. But it can be changed to
```
sed "s/$ostr/$nstr/g" "$file"
```
This still has some risks, but they are much less severe.
if [ -f $file ], > $file.tmp and mv $file.tmp $file should be if [ -f "$file" ], > "$file.tmp" and mv "$file.tmp" "$file", respectively, to handle file names that might have spaces (or other funny characters) in them. (The eval "sed … command also mangles file names that have spaces in them.)

^* shift takes an optional argument: a positive integer that specifies how many parameters to shift. The default is one (1). For example, shift 4 causes $5 to become $1, $6 to become $2, and so on. (Note that the example in the Bash Guide for Beginners is wrong.) And so your script could be modified to say

ostr="$1"
nstr="$2"
shift 2

which might be considered to be more clear.

End Note / Warning:

The Windows Command Prompt (batch file) language also supports a SHIFT command, which does basically the same thing as the shift command in Unix shells, with one striking difference, which I’ll hide to try to prevent people from being confused by it:

A command like SHIFT 4 is an error, yielding an “Invalid parameter to SHIFT command” error message.

SHIFT /n, where n is an integer between 0 and 8, is valid — but it doesn’t shift n times. It shifts once, starting with the n th argument. So SHIFT /4 causes %5 (the fifth argument) to become %4, %6 to become %5, and so on, leaving arguments 0 through 3 alone.

Solution 3

The simplest explanation is this. Consider the command:

/bin/command.sh SET location Cebu
/bin/command.sh SET location Cebu, Philippines 6014

Without the help of shift you cannot extract the complete value of the location because this value could get arbitrarily long. When you shift two times, the args SET and location are removed such that:

x="$@"
echo "location = $x"

Take a very long stare at the $@ thing. That means, it can save the complete location into variable x despite the spaces and comma that it has. So in summary, we call shift and then later we retrieve the value of what is left from the variable $@.

UPDATE I am adding below a very short snippet showing the concept and usefulness of shift without which, it would be very,very difficult to extract the fields correctly.

#!/bin/sh
#

[ $# -eq 0 ] && return 0

a=$1
shift
b=$@
echo $a
[ -n "$b" ] && echo $b

Explanation: After the shift, the variable b shall contain the rest of the stuff being passed in, no matter of spaces or etc. The [ -n "$b" ] && echo $b is a protection such that we only print b if it has a content.

Solution 4

shift treat command line arguments as a FIFO queue, it popleft element every time it's invoked.

array = [a, b, c]
shift equivalent to
array.popleft
[b, c]
$1, $2,$3 can be interpreted as index of the array.
$# is the length of array

View more solutions

253,326

Patryk

Software Engineer C++/Go/shell/python coder Linux enthusiast Github profiles: https://github.com/pmalek https://github.com/pmalekn

Updated on September 18, 2022

Comments

Patryk almost 2 years

I have came across this script:

#! /bin/bash                                                                                                                                                                                           

if (( $# < 3 )); then
  echo "$0 old_string new_string file [file...]"
  exit 0
else
  ostr="$1"; shift
  nstr="$1"; shift  
fi

echo "Replacing \"$ostr\" with \"$nstr\""
for file in $@; do
  if [ -f $file ]; then
    echo "Working with: $file"
    eval "sed 's/"$ostr"/"$nstr"/g' $file" > $file.tmp 
    mv $file.tmp $file
  fi  
done

What is the meaning of the lines where they use shift? I presume the script should be used with at least arguments so...?

goldilocks over 9 years

+1 Note it's not rotating them, it's shifting them off the array (of arguments). Shift/unshift and pop/push are common names for this general kind of manipulation (see, e.g., bash's pushd and popd).
Angel Todorov over 9 years

The definitive reference: gnu.org/software/bash/manual/bashref.html#index-shift
mikeserv over 9 years

shift can be applied to while, until, and even for loops for handling arrays in much more subtle ways than can a simple for loop. I often find it useful in for loops like... set -f -- $args; IFS=$split; for arg do set -- $arg; shift "${number_of_fields_to_discard}" && fn_that_wants_split_arg "$arg"; done
humanityANDpeace over 9 years

@goldilocks very true. Instead of "rotate" I should have better found a way to use another word, something like "move the arguments forward in the argument variables". Anyway I hope the examples in the text and the link to the reference will have made it clear nonetheless.
Scott - Слава Україні over 6 years

(1) This is not the simplest explanation. It’s an interesting angle. (2) But as long as you want to concatenate a bunch of arguments (or all of them), you might want to get into the habit of using "$*" instead of "$@". (3) I was going to upvote your answer, but I didn’t, because you say “shift two times” but you don’t actually show it in code. (4) If you want a script to be able to take a multi-word value from the command line, it’s probably better to require the user to put the entire value into quotes. … (Cont’d)
Scott - Слава Україні over 6 years

(Cont’d) … You might not care today, but your approach doesn’t allow you to have multiple spaces (Philippines   6014), leading spaces, trailing spaces, or tabs. (5) BTW, you may also be able to do what you’re doing here in bash with x="${*:3}".
eigenfield over 6 years

I tried to decipher where your '*** doesn't allow multiple spaces ***' comment is coming from because this is not related to the shift command. You maybe referring to $@ versus $* subtle differences. But the fact still remains, that my snippet above can accurately extract the location no matter how many spaces it has either trailing or in the front it doesn't matter. After the shift, location will contain the correct location.
calandoa almost 5 years

Though the script mentions bash, the question is general to all shells, therefore the Posix standard prevails: pubs.opengroup.org/onlinepubs/9699919799/utilities/…