Create new file but add number if filename already exists in bash

Question

I found similar questions but not in Linux/Bash

I want my script to create a file with a given name (via user input) but add number at the end if filename already exists.

Example:

$ create somefile
Created "somefile.ext"
$ create somefile
Created "somefile-2.ext"

Solution 1

The following script can help you. You should not be running several copies of the script at the same time to avoid race condition.

name=somefile
if [[ -e $name.ext || -L $name.ext ]] ; then
    i=0
    while [[ -e $name-$i.ext || -L $name-$i.ext ]] ; do
        let i++
    done
    name=$name-$i
fi
touch -- "$name".ext

Solution 2

Easier:

touch file`ls file* | wc -l`.ext

You'll get:

$ ls file*
file0.ext  file1.ext  file2.ext  file3.ext  file4.ext  file5.ext  file6.ext

Solution 3

To avoid the race conditions:

name=some-file

n=
set -o noclobber
until
  file=$name${n:+-$n}.ext
  { command exec 3> "$file"; } 2> /dev/null
do
  ((n++))
done
printf 'File is "%s"\n' "$file"
echo some text in it >&3

And in addition, you have the file open for writing on fd 3.

With bash-4.4+, you can make it a function like:

create() { # fd base [suffix [max]]]
  local fd="$1" base="$2" suffix="${3-}" max="${4-}"
  local n= file
  local - # ash-style local scoping of options in 4.4+
  set -o noclobber
  REPLY=
  until
    file=$base${n:+-$n}$suffix
    eval 'command exec '"$fd"'> "$file"' 2> /dev/null
  do
    ((n++))
    ((max > 0 && n > max)) && return 1
  done
  REPLY=$file
}

To be used for instance as:

create 3 somefile .ext || exit
printf 'File: "%s"\n' "$REPLY"
echo something >&3
exec 3>&- # close the file

The max value can be used to guard against infinite loops when the files can't be created for other reason than noclobber.

Note that noclobber only applies to the > operator, not >> nor <>.

Remaining race condition

Actually, noclobber does not remove the race condition in all cases. It only prevents clobbering regular files (not other types of files, so that cmd > /dev/null for instance doesn't fail) and has a race condition itself in most shells.

The shell first does a stat(2) on the file to check if it's a regular file or not (fifo, directory, device...). Only if the file doesn't exist (yet) or is a regular file does 3> "$file" use the O_EXCL flag to guarantee not clobbering the file.

So if there's a fifo or device file by that name, it will be used (provided it can be open in write-only), and a regular file may be clobbered if it gets created as a replacement for a fifo/device/directory... in between that stat(2) and open(2) without O_EXCL!

Changing the

  { command exec 3> "$file"; } 2> /dev/null

to

  [ ! -e "$file" ] && { command exec 3> "$file"; } 2> /dev/null

Would avoid using an already existing non-regular file, but not address the race condition.

Now, that's only really a concern in the face of a malicious adversary that would want to make you overwrite an arbitrary file on the file system. It does remove the race condition in the normal case of two instances of the same script running at the same time. So, in that, it's better than approaches that only check for file existence beforehand with [ -e "$file" ].

For a working version without race condition at all, you could use the zsh shell instead of bash which has a raw interface to open() as the sysopen builtin in the zsh/system module:

zmodload zsh/system

name=some-file

n=
until
  file=$name${n:+-$n}.ext
  sysopen -w -o excl -u 3 -- "$file" 2> /dev/null
do
  ((n++))
done
printf 'File is "%s"\n' "$file"
echo some text in it >&3

Solution 4

Try something like this

name=somefile
path=$(dirname "$name")
filename=$(basename "$name")
extension="${filename##*.}"
filename="${filename%.*}"
if [[ -e $path/$filename.$extension ]] ; then
    i=2
    while [[ -e $path/$filename-$i.$extension ]] ; do
        let i++
    done
    filename=$filename-$i
fi
target=$path/$filename.$extension

Solution 5

Use touch or whatever you want instead of echo:

echo file$((`ls file* | sed -n 's/file\([0-9]*\)/\1/p' | sort -rh | head -n 1`+1))

Parts of expression explained:

• list files by pattern: ls file*
• take only number part in each line: sed -n 's/file\([0-9]*\)/\1/p'
• apply reverse human sort: sort -rh
• take only first line (i.e. max value): head -n 1
• combine all in pipe and increment (full expression above)

View more solutions

Comments

[heltonbiker]

It worked, but I changed initial i to 2, so as to be the "second" copy of the file. Thanks!

[PatriceG]

I used it with a else condition for the first passage in the loop

[Stephane Chazelas]

As commented already on @Choroba's answer [[ -e .... ]] does a stat(2), you need [[ -e ... || -L ... ]] to search for file existence, but even then there's a race condition, better is to open the file with O_EXCL for the system to guarantee you're not using an already existing file (or worse a symlink to a sensitive file).

[sorin]

This should be the accepted answer because it does take care of everything, including adding the suffix before the extension!

[Stephane Chazelas]

@sorin, it still fails for values of $name that start with -. It could fail for values of name like /file on systems where //file is special. It doesn't guard against $names that are broken symlinks. And has a TOCTOU race condition (note that it breaks in zsh where $path is the special array tied to $PATH)

[Tom Russell]

The following command does nothing when the contents above are copied into backup.sh: $ ./backup.sh blah.txt. I would expect a new file having a name of "blah2.txt" or some such.

[clearlight]

Wouldn't use this if the list could grow to a lot of files, as it will keep getting slower

[Uzi]

How will it work if I want the first output to be file1.ext?

[Ulrich Schwarz]

Nice, if (big if) you are sure you will never have non-consecutive numbers in there, i.e. if I ever delete file2.txt in your example, I'll now clobber file6.txt.

[tripleee]

Generally you should avoid using ls in scripts but this seems pretty harmless. For stylistic points, replace ls with printf '%s\n' (or even printf '.\n' which is robust against file names with newlines in them).

[tripleee]

This has a number of unfortunate antipatterns, including useless use of echo and the use of uppercase for private variables, which are reserved for system use.

[Maëlan]

This code has a race condition: the file name you decided upon might have been created and used by another program after [[ -e … ]] concludes that it does not exist, but before touch actually creates it. That’s a big pitfall, as it defeats the very purpose, and the issue may commonly arise from the moment you start using your script in parallel. See @StephaneChazelas’ answer for a solution that avoids (most) race conditions.

[Maëlan]

This is the best and only acceptable answer, as it is the only one that takes care of race conditions.

[Maëlan]

If I understand you correctly, there are two issues. Apart from the race condition in-between stat(2) and open(2) which is unlikely as you said, there is also the issue of non-regular files which happen to be writable (fifo, device… or symbolic link to one of these kinds). If "$file" already exists and is of this kind, then the loop will(?) stop instead of trying another filename. Shouldn’t that case be handled by combining the tentative redirection with a test && [ -f "$file" ] (coming after the redirection succeeded)? Or will the call to open(2) fail anyway?

[Stephane Chazelas]

@Maëlan, note that the race condition was already noted in the answer.

[Keithel]

In the touch command - why is $name in quotes, while its extension is not?: touch -- "$name".ext ? Does this serve any useful purpose?

[choroba]

@Keithel: $name should be double quoted if it should work for names containing spaces etc. Quoting .ext is possible, but doesn't change anything.

[Keithel]

Ok - so there is no specific need to leave the .ext out of the quotes. That's what I was wondering. Just seemed an odd way to write it.