How to determine the buffer size for BufferedOutputStream's write method

Question

I am trying to copy a file using the following code:

1:

        int data=0;
        byte[] buffer = new byte[4096];
        while((data = bufferedInputStream.read())!=-1){
            bufferedOutputStream.write(data);
        }

2:

        byte[] buffer = new byte[4096];
        while(bufferedInputStream.read(buffer)!=-1){
            bufferedOutputStream.write(buffer);
        }

Actual size of file is 3892028 bytes(on windows). The file will be uploaded by the user thro struts2 fileupload. Uploaded file size is exactly same as that of windows. When I try to copy the uploaded file from temporary folder, the copied file varies in size and the time taken also varies(it is negligible). Please find the below readings.

Without using buffer(Code 1)
Time taken 77
3892028
3891200

Buffer size 1024(Code 2)
Time taken 17
3892028
3891200

Buffer size 4096(Code 2)
Time taken 18
3892028
3891200

Buffer size 10240(Code 2)
Time taken 14
3892028
3901440

Buffer size 102400(Code 2)
Time taken 9
3892028
3993600

If I increase the buffer size further, time taken increases, again it is negligible. So my questions are,

1. Why the file size changes?
2. Is there any subtle consequences due to this size variation?
3. What is the best way to accomplish this functionality(copying a file)?

I don't know what is going beneath? Thanks for any suggestion. Edit: I have flush() and close() method calls. Note: I have trimmed my code to make it simpler.

Solution 1

1. Why the file size changes?

You forgot to `flush()` (and `close()`): bufferedOutputStream.flush()

Also you should pass the number of bytes read to write method:

bufferedOutputStream.write(data, 0, bytesRead);

1. What is the best way to accomplish this functionality(copying a file)?

• FileUtils.copyFile()
• IOUtils.copy()

Both from apache-commons IO.

Solution 2

The problem is, BufferedInputStream.read(byte[]) reads as much as it can into the buffer. So if the stream contains only 1 byte, only the first byte of byte array will be filled. However, BufferedInputStream.write(byte[]) writes all the given bytes into the stream, meaning it will still write full 4096 bytes, containing 1 byte from current iteration and 4095 remaining bytes from previous iteration.

What you need to do, is save the amount of bytes that were read, and then write the same amount.

Example:

int lastReadCnt = 0;
byte[] buffer = new byte[4096];
while((lastReadCnt = bufferedInputStream.read(buffer))!=-1){
    bufferedOutputStream.write(buffer, 0, lastReadCnt);
}

References:

• Java 6: InputStream: read(byte[],int,int)
• Java 6: OutputStream: write(byte[],int,int)

Comments

[boumbh]

I don't understand why the generated files are smaller than the original ones in the first three examples. It is the same result as if the code was while(bufferedInputStream.read(buffer)==buffer.length){ .

[bezmax]

@boumbh And that phenomenon had happened because he hadn't flushed his BufferedOutputStream. Default size of the buffer is 8192 bytes, and it is flushed automatically whenever filled. And the last pack of data written didn't fill the buffer of the stream, therefore it was never flushed to the underlying stream. Also, in comment to the other answer, Ahamed mentioned that he does have flush and close. However, I guess that he never flushed/closed the BufferedOutputStream, but directly the underlying one. Therefore the buffered one was never notified and never flushed it's unfilled buffer.