Easy way to write contents of a Java InputStream to an OutputStream

Question

I was surprised to find today that I couldn't track down any simple way to write the contents of an InputStream to an OutputStream in Java. Obviously, the byte buffer code isn't difficult to write, but I suspect I'm just missing something which would make my life easier (and the code clearer).

So, given an InputStream in and an OutputStream out, is there a simpler way to write the following?

byte[] buffer = new byte[1024];
int len = in.read(buffer);
while (len != -1) {
    out.write(buffer, 0, len);
    len = in.read(buffer);
}

Solution 1

Java 9

Since Java 9, InputStream provides a method called transferTo with the following signature:

public long transferTo(OutputStream out) throws IOException

As the documentation states, transferTo will:

Reads all bytes from this input stream and writes the bytes to the given output stream in the order that they are read. On return, this input stream will be at end of stream. This method does not close either stream.

This method may block indefinitely reading from the input stream, or writing to the output stream. The behavior for the case where the input and/or output stream is asynchronously closed, or the thread interrupted during the transfer, is highly input and output stream specific, and therefore not specified

So in order to write contents of a Java InputStream to an OutputStream, you can write:

input.transferTo(output);

Solution 2

As WMR mentioned, org.apache.commons.io.IOUtils from Apache has a method called copy(InputStream,OutputStream) which does exactly what you're looking for.

So, you have:

InputStream in;
OutputStream out;
IOUtils.copy(in,out);
in.close();
out.close();

...in your code.

Is there a reason you're avoiding IOUtils?

Solution 3

If you are using Java 7, Files (in the standard library) is the best approach:

/* You can get Path from file also: file.toPath() */
Files.copy(InputStream in, Path target)
Files.copy(Path source, OutputStream out)

Edit: Of course it's just useful when you create one of InputStream or OutputStream from file. Use file.toPath() to get path from file.

To write into an existing file (e.g. one created with File.createTempFile()), you'll need to pass the REPLACE_EXISTING copy option (otherwise FileAlreadyExistsException is thrown):

Files.copy(in, target, StandardCopyOption.REPLACE_EXISTING)

Solution 4

I think this will work, but make sure to test it... minor "improvement", but it might be a bit of a cost at readability.

byte[] buffer = new byte[1024];
int len;
while ((len = in.read(buffer)) != -1) {
    out.write(buffer, 0, len);
}

Solution 5

Using Guava's ByteStreams.copy():

ByteStreams.copy(inputStream, outputStream);

View more solutions

Comments

[Aaron Digulla]

I suggest a buffer of at least 10KB to 100KB. That's not much and can speed up copying large amounts of data tremendously.

[Kevin Parker]

This did the trick for me when writing a file. I did not realize I needed to get the actual length readd from the input stream and use that to output to a file.

[Raekye]

This is not good for single-threaded code as it could deadlock; see this question stackoverflow.com/questions/484119/…

[Raekye]

There are copy and toByteArray methods in docs.guava-libraries.googlecode.com/git-history/release/java‌​doc/… (guava calls input/output streams as "byte streams" and readers/writers as "char streams")

[Jeremy Logan]

I'm avoiding it for this mobile app I'm building cause it'd quintuple the size of the app to save a measly 5 lines of code.

[Matt Sheppard]

I don't think this actually solves the problem since one end is a path. While you can get a path for a file, as far as I am aware you can't get one for any generic stream (e.g. one over the network).

[user207421]

May be of some use how? He already has an input stream and an output stream. How will adding another one of each help exactly?

[user207421]

Using a large buffer is indeed a good idea but not because files are mostly > 1k, it is to amortize the cost of system calls.

[Erik Kaplun]

+1 still turned out to be useful in my case though, as I have a real file on the output side!

[Warren Dew]

@basZero Or using a try with resources block.

[Aritz]

As far as I'm concerned, te first method you say is not available with that signature. You have to add the CopyOptions parameter.

[user1079877]

CopyOptions is arbitrary! You can put it here if you want it.

[Jim Tough]

If you're already using the Guava library, Andrejs has recommended the ByteStreams class below. Similar to what IOUtils does, but avoids adding Commons IO to your project.

[WonderCsabo]

Do not forget to close the streams after that!

[rupps]

if you already use guava libraries it's a good idea, but if not, they are a mammoth library with thousands of methods 'google-way-of-doing-everything-different-to-the-standard'. I'd keep away from them

[Don Cheadle]

now this is what I was looking for! JDK to the rescue, no need for another library

[phil294]

you might want to say while(len > 0) instead of != -1, because the latter could also return 0 when using the read(byte b[], int off, int len)-method, which throws an exception @ out.write

[Joshua Pinter]

FYI, Files is NOT available in Android's Java 1.7. I got stung by this: stackoverflow.com/questions/24869323/…

[Joshua Pinter]

Great function, thanks. Would you need to put the close() calls in finally blocks, though?

[Jordan LaPrise]

@JoshPinter It wouldn't hurt.

[rfornal]

Can you please explain why this is the right answer?

[Christoffer Hammarström]

@Blauhirn: That would be incorrect, as it is entirely legal according to the InputStream contract for read to return 0 any number of times. And according to the OutputStream contract, the write method must accept a length of 0, and should only throw an exception when len is negative.

[Sled]

@fiXedd You can use Maven Shade to strip unneeded classes from the final .jar, thus incurring only a modest increase in file jar size

[ɲeuroburɳ]

You can save a line by changing the while to a for and putting one of the variables in for's init section: e.g., for (int n ; (n = in.read(buf)) != -1 ;) out.write(buf, 0, n);. =)

[Ti Strga]

Amusingly, the JDK also has a Files.copy() which takes two streams, and is what all the other Files.copy() functions forward to in order to do the actual work of copying. However, it's private (since it doesn't actually involve Paths or Files at that stage), and looks exactly like the code in the OP's own question (plus a return statement). No opening, no closing, just a copy loop.

[Cel Skeggs]

You probably should both include a finally block and not swallow exceptions in an actual implementation. Also, closing an InputStream passed to a method is sometimes unexpected by the calling method, so one should consider if it's the behavior they want.

[Brian de Alwis]

Your suggestion causes a 0-byte write on the first iteration. Perhaps least do: for(int n = 0; (n = in.read(buffer)) > 0;) { out.write(buffer, 0, n); }

[Bohemian]

@BriandeAlwis You are right about the first iteration being incorrect. The code has been fixed (IMHO in a cleaner way than your suggestion) - see edited code. Thx for caring.

[user207421]

@Blauhim read() can only return zero if you supplied a length of zero, which would be a programming error, and a stupid condition to loop forever on. And write() does not throw an exception if you provide a zero length.

[user207421]

Flushing inside the loop is highly counter-productive.

[Maarten Bodewes]

Kind of obvious, but if the class doesn't have too many dependencies, you can also simply copy the source code for liberal licenses (like those used for both Guava and Apache). First read the license though (disclaimer, IANAL etc.).

[user3792852]

If you are using a dependency manager, like Maven, and you're avoiding IOUtils and Guava for size reasons, you might want to have a look at your dependency hierarchy. Maybe one of your dependencies transitively includes one of these without notice, they are very famous after all. If that's the case, you might as well use them yourself , you're including them anyway.

[Andreas]

@ChristofferHammarström It is not legal for InputStream.read(byte[] b) to return 0 (unless you're stupid like EJP said): If the length of b is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at the end of the file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

[Christoffer Hammarström]

@Andreas: Thanks! I never realized. I could imagine use cases though where you'd want to pass a zero length buffer, or when it happens for other reasons. So just like i always wear my seat belt, i will still always check for -1. Stopping reading and discarding remaining data is at least as much an error as passing an empty buffer.

[Prabhakar]

Why catch the Exception when IOException suffices?

[user207421]

There is no other reason:only a zero-length buffer or a zero length parameter. The meaning of your final sentence escapes me.

[ﾤ ﾡ]

catch(Exception ex){} — this is top-notch

[Hong]

This is the best answer if you are already using Guava which has become indispensable for me.

[ZhekaKozlov]

@Hong You should use Files.copy as much as possible. Use ByteStreams.copy only if both of streams are not FileInputStream/FileOutputStream.

[ZhekaKozlov]

You should preferFiles.copy as much as possible. It is implemented in native code and therefore can be faster. transferTo should be used only if both streams are not FileInputStream/FileOutputStream.

[Hong]

@ZhekaKozlov Thank you for the tip. In my case, the input stream is from an Android app's resource (drawable).

[Dragas]

Aye. Shame this particular call is private and there's no other option but to copy it out into your own utilities class, since it's possible that you're not dealing with files, but rather 2 sockets at once.

[Stuart Marks]

For Java 9 and later, use InputStream.transferTo(OutputStream). stackoverflow.com/a/39440936/1441122

[mochadwi]

Files.copy doesn't have backward support for API below 26 @user1079877

[user207421]

Why is 'removing the buffering semantics from the code' a good idea?

[Archimedes Trajano]

It means I don't write the buffering logic myself, I use the one built into the JDK which is usually good enough.

[Adrian Baker]

"mammoth"? 2.7MB with a very small set of dependencies, and an API that carefully avoids duplicating the core JDK.

[The Impaler]

@ZhekaKozlov Unfortunately Files.copy does not handle any input/output streams but it's specifically designed for file streams.

[The Impaler]

!= -1 or > 0? Those predicates are not quite the same.

[IPP Nerd]

!= -1 means not-end-of-file. This is not an iteration but a while-do-loop in disguise: while((n = inputStream.read(buffer)) != -1) do { outputStream.write(buffer, 0,n) }

[09Q71AO534]

Might cause Vulnerability issues if we use this code. One of the best practice is found here.Please modify accordingly.

[09Q71AO534]

Might cause Vulnerability issues if we use this code. One of the best practice is found here.Please modify accordingly.

[Stijn Haezebrouck]

If the length of buffer is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at the end of the file, the value -1 is returned; otherwise, at least one byte is read and stored into buffer. - so indeed, in this code, a return of 0 cannot occur as the size of the buffer is larger.

[Garret Wilson]

"And do not use inputStream.transferTo(...) because is too generic. Your code performance will be better if you control your buffer memory." That sounds plausible, and indeed my own code originally tried to pick buffer sizes based upon known transfer size. But I'm reading the answer may be more complicated based in part upon drive block sizes and CPU cache. Have you done any real-world tests to back up your claim that custom buffer sizes perform better than InputStream.transferTo(OutputStream)? If so I'd be interested to see them. Performance is tricky.

[Unmitigated]

Have you actually seen how transferTo is implemented?

[Prof]

Also only available on >API 26

[Ali Dehghani]

@ZhekaKozlov It seems that Files.copy(in, out) is also using the transferTo method under the hood. So it seems there is no native code unless JVM provides an instrinsic for Files.copy(in, out)

[Osagui Aghedo]

You have the answer in your own question. Just create your own Utility class and have a static method with the example code you have in your question. Any library that does what you want is gonna do that behind the scenes anyways. Why get messy with libraries?

[Ermiya Eskandary]

This is the right answer, thank you