How do I use GZipStream with System.IO.MemoryStream?

Question

I am having an issue with this test function where I take an in memory string, compress it, and decompress it. The compression works great, but I can't seem to get the decompression to work.

//Compress
System.IO.MemoryStream outStream = new System.IO.MemoryStream();                
GZipStream tinyStream = new GZipStream(outStream, CompressionMode.Compress);
mStream.Position = 0;
mStream.CopyTo(tinyStream);

//Decompress    
outStream.Position = 0;
GZipStream bigStream = new GZipStream(outStream, CompressionMode.Decompress);
System.IO.MemoryStream bigStreamOut = new System.IO.MemoryStream();
bigStream.CopyTo(bigStreamOut);

//Results:
//bigStreamOut.Length == 0
//outStream.Position == the end of the stream.

I believe that bigStream out should at least have data in it, especially if my source stream (outStream) is being read. is this a MSFT bug or mine?

Solution 1

What happens in your code is that you keep opening streams, but you never close them.

• In line 2, you create a GZipStream. This stream will not write anything to the underlying stream until it feels it’s the right time. You can tell it to by closing it.

• However, if you close it, it will close the underlying stream (outStream) too. Therefore you can’t use mStream.Position = 0 on it.

You should always use using to ensure that all your streams get closed. Here is a variation on your code that works.

var inputString = "“ ... ”";
byte[] compressed;
string output;

using (var outStream = new MemoryStream())
{
    using (var tinyStream = new GZipStream(outStream, CompressionMode.Compress))
    using (var mStream = new MemoryStream(Encoding.UTF8.GetBytes(inputString)))
        mStream.CopyTo(tinyStream);

    compressed = outStream.ToArray();
}

// “compressed” now contains the compressed string.
// Also, all the streams are closed and the above is a self-contained operation.

using (var inStream = new MemoryStream(compressed))
using (var bigStream = new GZipStream(inStream, CompressionMode.Decompress))
using (var bigStreamOut = new MemoryStream())
{
    bigStream.CopyTo(bigStreamOut);
    output = Encoding.UTF8.GetString(bigStreamOut.ToArray());
}

// “output” now contains the uncompressed string.
Console.WriteLine(output);

Solution 2

This is a known issue: http://blogs.msdn.com/b/bclteam/archive/2006/05/10/592551.aspx

I have changed your code a bit so this one works:

var mStream = new MemoryStream(new byte[100]);
var outStream = new System.IO.MemoryStream();

using (var tinyStream = new GZipStream(outStream, CompressionMode.Compress))
{
    mStream.CopyTo(tinyStream);           
}

byte[] bb = outStream.ToArray();

//Decompress                
var bigStream = new GZipStream(new MemoryStream(bb), CompressionMode.Decompress);
var bigStreamOut = new System.IO.MemoryStream();
bigStream.CopyTo(bigStreamOut);

Solution 3

The way to compress and decompress to and from a MemoryStream is:

public static Stream Compress(
    Stream decompressed, 
    CompressionLevel compressionLevel = CompressionLevel.Fastest)
{
    var compressed = new MemoryStream();
    using (var zip = new GZipStream(compressed, compressionLevel, true))
    {
        decompressed.CopyTo(zip);
    }

    compressed.Seek(0, SeekOrigin.Begin);
    return compressed;
}

public static Stream Decompress(Stream compressed)
{
    var decompressed = new MemoryStream();
    using (var zip = new GZipStream(compressed, CompressionMode.Decompress, true))
    {
        zip.CopyTo(decompressed);
    }

    decompressed.Seek(0, SeekOrigin.Begin);
    return decompressed;
}

This leaves the compressed / decompressed stream open and as such usable after creating it.

Solution 4

Another implementation, in VB.NET:

Imports System.Runtime.CompilerServices
Imports System.IO
Imports System.IO.Compression

Public Module Compressor

    <Extension()> _
    Function CompressASCII(str As String) As Byte()

        Dim bytes As Byte() = Encoding.ASCII.GetBytes(str)

        Using ms As New MemoryStream

            Using gzStream As New GZipStream(ms, CompressionMode.Compress)

                gzStream.Write(bytes, 0, bytes.Length)

            End Using

            Return ms.ToArray

        End Using

    End Function

    <Extension()> _
    Function DecompressASCII(compressedString As Byte()) As String

        Using ms As New MemoryStream(compressedString)

            Using gzStream As New GZipStream(ms, CompressionMode.Decompress)

                Using sr As New StreamReader(gzStream, Encoding.ASCII)

                    Return sr.ReadToEnd

                End Using

            End Using

        End Using

    End Function

    Sub TestCompression()

        Dim input As String = "fh3o047gh"

        Dim compressed As Byte() = input.CompressASCII()

        Dim decompressed As String = compressed.DecompressASCII()

        If input <> decompressed Then
            Throw New ApplicationException("failure!")
        End If

    End Sub

End Module

Solution 5

If you are attempting to use the MemoryStream (e.g. passing it into another function) but receiving the Exception "Cannot access a closed Stream." then there is another GZipStream constructor you can use that will help you.

By passing in a true to the leaveOpen parameter, you can instruct GZipStream to leave the stream open after disposing of itself, by default it closes the target stream (which I didn't expect). https://msdn.microsoft.com/en-us/library/27ck2z1y(v=vs.110).aspx

using (FileStream fs = File.OpenRead(f))
using (var compressed = new MemoryStream())
{
    //Instruct GZipStream to leave the stream open after performing the compression.
    using (var gzipstream = new GZipStream(compressed, CompressionLevel.Optimal, true))
        fs.CopyTo(gzipstream);

    //Do something with the memorystream
    compressed.Seek(0, SeekOrigin.Begin);
    MyFunction(compressed);
}

View more solutions

Comments

[MerickOWA]

+1 good answer Timwi. Just to add to this, GZip has some internal buffering of data it needs to do in order to compress. It can't know that its done receiving data until you close it and therefore it doesn't spit out the last few bytes and decompression of the partial stream fails.

[Almo]

I think we're on .NET 3.5 (working with Unity), so .CopyTo doesn't exist yet. Looking elsewhere on SO for how to copy from one stream to another: stackoverflow.com/questions/230128/…

[MikeT]

Thanks for this, i've been having trouble figuring out exactly how to arrange the streams to get the correct output in both direction

[muffi]

Even if your post is 6 years old and I am not the questioneer - it helped me today. Thank you!

[bounav]

Shouldn't new GZipStream(outStream, CompressionMode.Compress, true) be used to leave the stream opened so that the using statement can close it as suggested in @briantyler's answer?