Reusing a filestream

Question

In the past I've always used a FileStream object to write or rewrite an entire file after which I would immediately close the stream. However, now I'm working on a program in which I want to keep a FileStream open in order to allow the user to retain access to the file while they are working in between saves. ( See my previous question).

I'm using XmlSerializer to serialize my classes to a from and XML file. But now I'm keeping the FileStream open to be used to save (reserialized) my class instance later. Are there any special considerations I need to make if I'm reusing the same File Stream over and over again, versus using a new file stream? Do I need to reset the stream to the beginning between saves? If a later save is smaller in size than the previous save will the FileStream leave the remainder bytes from the old file, and thus create a corrupted file? Do I need to do something to clear the file so it will behave as if I'm writing an entirely new file each time?

Solution 1

Your suspicion is correct - if you reset the position of an open file stream and write content that's smaller than what's already in the file, it will leave trailing data and result in a corrupt file (depending on your definition of "corrupt", of course).

If you want to overwrite the file, you really should close the stream when you're finished with it and create a new stream when you're ready to re-save.

I notice from your linked question that you are holding the file open in order to prevent other users from writing to it at the same time. This probably wouldn't be my choice, but if you are going to do that, then I think you can "clear" the file by invoking stream.SetLength(0) between successive saves.

Solution 2

There are various ways to do this; if you are re-opening the file, perhaps set it to truncate:

using(var file = new FileStream(path, FileMode.Truncate)) {
    // write
}

If you are overwriting the file while already open, then just trim it after writing:

file.SetLength(file.Position); // assumes we're at the new end

I would try to avoid delete/recreate, since this loses any ACLs etc.

Solution 3

Another option might be to use SetLength(0) to truncate the file before you start rewriting it.

Solution 4

Recently ran into the same requirement. In fact, previously, I used to create a new FileStream within a using statement and overwrite the previous file. Seems like the simple and effective thing to do.

using (var stream = new FileStream(path, FileMode.Create, FileAccess.Write)
{
   ProtoBuf.Serializer.Serialize(stream , value);
}

However, I ran into locking issues where some other process is locking the target file. In my attempt to thwart this I retried the write several times before pushing the error up the stack.

int attempt = 0;
while (true)
{
   try
   {
      using (var stream = new FileStream(path, FileMode.Create, FileAccess.Write)
      {
         ProtoBuf.Serializer.Serialize(stream , value);
      }
      break;
   }
   catch (IOException)
   {
      // could be locked by another process
      // make up to X attempts to write the file
      attempt++;
      if (attempt >= X)
      {
         throw;
      }
      Thread.Sleep(100);
   }
}

That seemed to work for almost everyone. Then that problem machine came along and forced me down the path of maintaining a lock on the file the entire time. So in lieu of retrying to write the file in the case it's already locked, I'm now making sure I get and hold the stream open so there are no locking issues with later writes.

int attempt = 0;
while (true)
{
   try
   {
      _stream = new FileStream(path, FileMode.Open, FileAccess.ReadWrite, FileShare.Read);
      break;
   }
   catch (IOException)
   {
      // could be locked by another process
      // make up to X attempts to open the file
      attempt++;
      if (attempt >= X)
      {
         throw;
      }
      Thread.Sleep(100);
   }
}

Now when I write the file the FileStream position must be reset to zero, as Aaronaught said. I opted to "clear" the file by calling _stream.SetLength(0). Seemed like the simplest choice. Then using our serializer of choice, Marc Gravell's protobuf-net, serialize the value to the stream.

_stream.SetLength(0);
ProtoBuf.Serializer.Serialize(_stream, value);

This works just fine most of the time and the file is completely written to the disk. However, on a few occasions I've observed the file not being immediately written to the disk. To ensure the stream is flushed and the file is completely written to disk I also needed to call _stream.Flush(true).

_stream.SetLength(0);
ProtoBuf.Serializer.Serialize(_stream, value);
_stream.Flush(true);

View more solutions

Comments

[Eric Anastas]

What would your choice be then? I don't want allow two users to open the file at the same time thinking they both have write access and each overwriting the other's saves?

[Eric Anastas]

Yes I'm just rewriting the entire file. So would you just suggest keeping the stream open to lock out access to the file, and then quickly closing and reopening the stream when I need to write the file again?

[Aaronaught]

One of the more common answers is to create a zero-length "lock file". This can be combined with holding a FileStream open indefinitely, of course, but a lot of apps opt for only the lock file because it can be overridden if necessary. I guess it depends on the nature of your app and sharing environment - maybe it really is the best option.

[Eric Anastas]

Set-length(0) seems to do what I'm trying to do.

[No Refunds No Returns]

Why do you want to keep the file open? What business purpose does it serve?

[mpen]

The problem with lock files is that if the application crashes and doesn't delete the lock file, then you can't reopen the file later unless you manually delete the lock file -- which isn't obvious to end users. I had this problem with firefox on linux awhile back.

[Despertar]

He does not want another process to grab the file after it is closed and to to write to it overwriting changes that another process is currently making.

[bsobaid]

ok, so using _fileStream.SetLength(0) overwrites the file but its extremely slow. I ran it 500,000 times and it took 35 seconds. I used streamwrite as shown in my original post, it was less than a second but it appended, which is not what I want

[Aaronaught]

@bsobaid: File I/O is slow. Deleting an existing file and creating a new one may be faster, but is also more risky and requires more defensive programming. Why do you need to write and empty the same file 500,000 times, anyway? Something is definitely wrong with that design.