How to read and write data into same file simultaneously

Question

I have read many a posts where-in they speak about reading and writing into the file NOT simultaneously using JavaME. I have a special use case scenarios where-in my log file (maybe full file or just portion of the file) is uploaded to the server on regular basis. This must continue without hampering the current logging of the application in this same file.

The code sample is a under:

boolean writing = true;
boolean reading = true;
void main() {
    new Thread("THREAD-FILE-READ") {
        public void run() {
            InputStream instream = getFileInStream();
            if (null != instream) {
                while (reading) {
                    try {
                        try {
                            synchronized(READ_LOCK) {
                                READ_LOCK.wait();
                            }
                        } catch (InterruptedException ex) {
                            ex.printStackTrace();
                        }

                        if (writtenCharsLen > 0) {
                            byte[] bytes = new byte[writtenCharsLen];
                            instream.read(bytes, 0, writtenCharsLen);
                            System.out.println("Read="+new String(bytes));
                            bytes = null;
                            writtenCharsLen = 0;
                        }
                    } catch (IOException ioe) {
                        ioe.printStackTrace();
                    }
                }
            }
            closeStream(instream);
        }
    }.start();

    new Thread("THREAD-FILE-WRITE") {
        public void run() {
            OutputStream outstream = getFileOutStream();
            if (null != outstream) {
                while (writing) {
                    try {
                        byte[] str = randomString();
                        if (null != str) {
                            writtenCharsLen = str.length;
                            System.out.println("Write=" + new String(str));
                            outstream.write(str);
                            str = null;
                        }
                    } catch (IOException ex) {
                        ex.printStackTrace();

                    } finally {
                        notifyReadStream();
                    }

                    try {
                        synchronized(WRITE_LOCK) {
                            WRITE_LOCK.wait();
                        }
                    } catch (InterruptedException ex) {
                        ex.printStackTrace();
                    }
                }
            }
            closeStream(outstream );
        }
    }.start();

}

void notifyReadStream() {
    try {
        synchronized (READ_LOCK) {
            READ_LOCK.notify();
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
}

void notifyWriteStream() {
    try {
        synchronized (WRITE_LOCK) {
            WRITE_LOCK.notify();
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
}

In the above code I will replace sop-read and sop-write with proper calls to network IO methods.

PS: Since this piece of code will run of multiple files and multitude of devices i need the modification as compressed as possible to keep my runtime heap as low as possible. Also this piece of code will run till the application life cycle hence closing and opening the file in middle is out of consideration.

Present Undesired Result: The read and write threads are showing running sop's for read and write. The read thread is reading from the position the writing thread has written. I am not facing any exception in this code but the result is undesired. I have also tried synchronizing read and write streams but that is throwing IllegalMonitorStateException

Expected Result: Reading of the stream must be triggered after writing into the stream is completed, also the read thread must be able to read from any position in the file.

Any help / pointers is useful?

EDIT: I was able to synchronize the read and the write streams using different monitors but i still feel, i could have done better using single monitor. Will try it sometime later.

Solution

I will attack this problem:

Present Undesired Result: The read and write threads are showing running sop's for read and write. The read thread is reading from the position the writing thread has written. I am not facing any exception in this code but the result is undesired. I have also tried synchronizing read and write streams but that is throwing IllegalMonitorStateException.

If you have synchronized the access using monitors i.e. the reader calls someObject.wait() and the writer calls someObject.notify(), remember that you have to wrap these calls in a synchronized block on someObject:

synchronized(someObject) {
    someObject.wait();
}


synchronized(someObject) {
    someObject.notify();
}

This is the cause for IllegalMonitorStateException.

Comments

[gnat]

how comfortable are you generally with multithreaded code? Eg do topics in Jenkov's tutorial look familiar? given J2ME limitations I'd say it's the best relevant intro I've seen

[Vimal]

@gnat, Thanks for your link it was really insightful to read it. And yes I am pretty comfortable with multi-threaded mobile environments and development. Please check my question description, I have edited the synchronous implementation. The next problem that I am working on now is how to smartly, and the conventional way where-in multiple variables are used to keep a log of number of bytes added/removed, read enough bytes from the file from the last position where I have already uploaded to server. I am digging at an optimized approach for this implementation.

[gnat]

for that next problem, did you consider BlockingQueue? there's a chapter in Jenkov's with explanation and code. If I understand correctly, "THREAD-FILE-WRITE" would enqueue and "THREAD-FILE-READ" would dequeue

[Vimal]

@gnat, yes I've. As you said it will enqueue and dequeue, which I'm able to achieve by sequentializing my monitors. My immediate issue now is how to efficiently read enough bytes from the file which are written by the writer to be sent to the server. I know I can do it by using variables that store the current and new position of the bytes in the file but I'm aiming at an optimized approach bcoz (a) its IO operation which are resource intensive & (b) this will occur as many time the read queue is notified. I'm thinking of some simpler method to resolve aforesaid problem.

[gnat]

I see. Looks like you're heading for quite some fun. :) Speaking in terms of queue, you have to design what, when and how to en/dequeue right?

[Vimal]

Thanks for your inputs! But moving writeCharsLen... after the write to stream didn't solve the OP, as it is still an asynchronous read from stream. Next, the threads can not yield as it is a continue process of reading/writing from/into a file which will eventually interface with the server to upload data.

[Vimal]

Thanks for your response! With this approach i can not wait on same monitor in these threads. Any yes, i have synchronized my monitor which is why it was throwing IllegalMonitorStateException, as i need to wait till the next write on the write thread and wait till the next read in the read thread. Now I am working on having individual monitors for each thread, will keep you all updated.

[Tudor]

@Vimal: You have to synchronize on exactly the same object on which you call wait/notify. But why can't you wait/notify on the same monitor in both threads?

[Vimal]

If i use the same monitor in these threads than on monitor.notify() it notifies both the threads and again reading and writing is asynchronous.

[Tudor]

@Vimal but I thought you wanted to make only the reader to wait until the writing is complete?

[Vimal]

if the writer doesn't wait then it will endless keep writing the data in to the output file. So the need is it should write to the file only when new log is available. Once the writer writes the data into the file the reader must immediately read it and send it to the server. Hence the OP.