KSH/BASH Maximum size of an Array

Question

What is the maximum size of array for ksh and bash scripting?

Example: Suppose I have an array of 10 elements. What will be the maximum number of string count that a particular index of an array can hold? What will be the maximum size of an array for the same?

I am new to Unix. I imagine this is a common question but I didn't manage to find an answer so I decided to ask here.

Solution 1

i=0
while true; do
    a[$i]=foo
    i=$((i+1))
    printf "\r%d " $i
done

This simple script shows on my systems (Gnu/Linux and Solaris):

• ksh88 limits the size to 2^12-1 (4095). (subscript out of range ). Some older releases like the one on HP-UX limit the size to 1023.

• ksh93 limits the size of a array to 2^22-1 (4194303), your mileage may vary.

• bash doesn't look to impose any hard-coded limit outside the one dictated by the underlying memory resources available. For example bash uses 1.3 GB of virtual memory for an array size of 18074340.

Note: I gave up with mksh which was too slow executing the loop (more than one hundred times slower than zsh, ksh93 and bash.)

Solution 2

There is no maximum limit on the size of an array, nor any requirement that members be indexed or assigned contiguously. Indexed arrays are referenced using integers (including arithmetic expressions (see Shell Arithmetic)) and are zero-based; associative arrays use arbitrary strings. Unless otherwise noted, indexed array indices must be non-negative integers.

http://www.gnu.org/software/bash/manual/html_node/Arrays.html

Solution 3

It depends on the implementation. In ksh there are documented "implementation defined limits" for indexed arrays. For ksh88 there are systems existing with a limit of 1023, for ksh93 the minimum limit required by an implementation is 4095. So you cannot count on having more available than that! (If you are only implementing for a specific system you can test your system limits as proposed in another answer here.)

Comments

[l0b0]

It is scary that you have to ask this question. Are you absolutely sure you can't handle the values in a while loop?

[Aman]

@l0b0 hmmm but I just ask it for knowledge purpose because I have a firm believe in the answers that I will get from here apart from that I have did prior lot of google but didn't get any satisfactory answer

[text]

Well actually the maximum size is the total amount of memory PLUS the total amount of swap space which will vary between system configurations.

[Jeight]

@mdpc That would be a hardware limitation and not a limitation of Bash/KSH. :)

[text]

Hmmm....I believe the OP asked for the MAXIMUM size of an array. :)

[Janis]

According to the ksh documentation you cannot count on the values examined here; it depends on the actual implementation on the system. (See my answer.)

[Janis]

@Jeight; This may be true for bash but not for ksh.

[Stéphane Chazelas]

what about pdksh or mksh or oksh?

[jlliagre]

@Janis Indeed, thanks. Answer updated to state this is implementation/configuration dependent.

[Aman]

@jlliagre suppose a string "ABCDEFGH" where each character is of 1 bytes then can i assume for ksh 93 and bash 4.2 array maximum capacity is 4194303 characters.

[jlliagre]

As I wrote, there is no hard-coded maximum capacity for bash. In any case, do you really need more than four millions element in a shell script array? Looks like you might be picking the wrong tool for the job.

[Janis]

@Stephane; I have no documentation inspected for those two variants of the original ksh. If it's not documented with those tools, I think that you should only (if at all!) rely on the original limits, or in case of single system development (as proposed) to check the supported range.

[Aman]

@jlliagre I got the concept :)

[Stéphane Chazelas]

Note that with ksh, the limit is on the subscript. You can't do a[4096]=x regardless of whether the other 4096 elements are set or not (in ksh, contrary to zsh, arrays are sparse, they're more like associative arrays with keys limited to positive integers than arrays). In mksh, there's no limit except that numbers wrap at 2^32 so your test is not valid there.

[jlliagre]

@StéphaneChazelas Assuming unlimited memory resources and a linear behavior, mksh would take more than 100 days only to initialize an 2^32 elements array on my core i5 CPU so the user's patience might be the the main limiting factor. bash, which is slower than ksh93 and zsh to run that loop will take 27 hours for the same 2^32 iterations.