final transient fields and serialization

Question

Is it possible to have final transient fields that are set to any non-default value after serialization in Java? My usecase is a cache variable — that's why it is transient. I also have a habit of making Map fields that won't be changed (i.e. contents of the map is changed, but object itself remains the same) final. However, these attributes seem to be contradictory — while compiler allows such a combination, I cannot have the field set to anything but null after unserialization.

I tried the following, without success:

• simple field initialization (shown in the example): this is what I normally do, but the initialization doesn't seem to happen after unserialization;
• initialization in constructor (I believe this is semantically the same as above though);
• assigning the field in readObject() — cannot be done since the field is final.

In the example cache is public only for testing.

import java.io.*;
import java.util.*;

public class test
{
    public static void main (String[] args) throws Exception
    {
        X  x = new X ();
        System.out.println (x + " " + x.cache);

        ByteArrayOutputStream  buffer = new ByteArrayOutputStream ();
        new ObjectOutputStream (buffer).writeObject (x);
        x = (X) new ObjectInputStream (new ByteArrayInputStream (buffer.toByteArray ())).readObject ();
        System.out.println (x + " " + x.cache);
    }

    public static class X implements Serializable
    {
        public final transient Map <Object, Object>  cache = new HashMap <Object, Object> ();
    }
}

Output:

test$X@1a46e30 {}
test$X@190d11 null

Solution 1

The short answer is "no" unfortunately - I've often wanted this. but transients cannot be final.

A final field must be initialized either by direct assignment of an initial value or in the constructor. During deserialization, neither of these are invoked, so initial values for transients must be set in the 'readObject()' private method that's invoked during deserialization. And for that to work, the transients must be non-final.

(Strictly speaking, finals are only final the first time they are read, so there are hacks that are possible that assign a value before it is read, but for me this is going one step too far.)

Solution 2

You can change the contents of a field using Reflection. Works on Java 1.5+. It will work, because serialization is performed in a single thread. After another thread access the same object, it shouldn't change the final field (because of weirdness in the memory model & reflaction).

So, in readObject(), you can do something similar to this example:

import java.lang.reflect.Field;

public class FinalTransient {

    private final transient Object a = null;

    public static void main(String... args) throws Exception {
        FinalTransient b = new FinalTransient();

        System.out.println("First: " + b.a); // e.g. after serialization

        Field f = b.getClass().getDeclaredField("a");
        f.setAccessible(true);
        f.set(b, 6); // e.g. putting back your cache

        System.out.println("Second: " + b.a); // wow: it has a value!
    }

}

Remember: Final is not final anymore!

Solution 3

Yes, this is easily possible by implementing the (apparently little known!) readResolve() method. It lets you replace the object after it is deserialized. You can use that to invoke a constructor that will initialize a replacement object however you want. An example:

import java.io.*;
import java.util.*;

public class test {
    public static void main(String[] args) throws Exception {
        X x = new X();
        x.name = "This data will be serialized";
        x.cache.put("This data", "is transient");
        System.out.println("Before: " + x + " '" + x.name + "' " + x.cache);

        ByteArrayOutputStream buffer = new ByteArrayOutputStream();
        new ObjectOutputStream(buffer).writeObject(x);
        x = (X)new ObjectInputStream(new ByteArrayInputStream(buffer.toByteArray())).readObject();
        System.out.println("After: " + x + " '" + x.name + "' " + x.cache);
    }

    public static class X implements Serializable {
        public final transient Map<Object,Object> cache = new HashMap<>();
        public String name;

        public X() {} // normal constructor

        private X(X x) { // constructor for deserialization
            // copy the non-transient fields
            this.name = x.name;
        }

        private Object readResolve() {
            // create a new object from the deserialized one
            return new X(this);
        }
    }
}

Output -- the string is preserved but the transient map is reset to an empty (but non-null!) map:

Before: test$X@172e0cc 'This data will be serialized' {This data=is transient}
After: test$X@490662 'This data will be serialized' {}

Solution 4

The general solution to problems like this is to use a "serial proxy" (see Effective Java 2nd Ed). If you need to retrofit this to an existing serialisable class without breaking serial compatibility, then you will need to do some hacking.

Solution 5

Five years later, I find my original answer unsatisfactory after I stumbled across this post via Google. Another solution would be using no reflection at all, and use the technique suggested by Boann.

It also makes use of the GetField class returned by ObjectInputStream#readFields() method, which according to the Serialization specification must be called in the private readObject(...) method.

The solution makes field deserialization explicit by storing the retrieved fields in a temporary transient field (called FinalExample#fields) of a temporary "instance" created by the deserialization process. All object fields are then deserialized and readResolve(...) is called: a new instance is created but this time using a constructor, discarding the temporary instance with the temporary field. The instance explicitly restores each field using the GetField instance; this is the place to check any parameters as would any other constructor. If an exception is thrown by the constructor it is translated to an InvalidObjectException and deserialization of this object fails.

The micro-benchmark included ensures that this solution is not slower than default serialization/deserialization. Indeed, it is on my PC:

Problem: 8.598s Solution: 7.818s

Then here is the code:

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InvalidObjectException;
import java.io.ObjectInputStream;
import java.io.ObjectInputStream.GetField;
import java.io.ObjectOutputStream;
import java.io.ObjectStreamException;
import java.io.Serializable;

import org.junit.Test;

import static org.junit.Assert.*;

public class FinalSerialization {

    /**
     * Using default serialization, there are problems with transient final
     * fields. This is because internally, ObjectInputStream uses the Unsafe
     * class to create an "instance", without calling a constructor.
     */
    @Test
    public void problem() throws Exception {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(baos);
        WrongExample x = new WrongExample(1234);
        oos.writeObject(x);
        oos.close();
        ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
        ObjectInputStream ois = new ObjectInputStream(bais);
        WrongExample y = (WrongExample) ois.readObject();
        assertTrue(y.value == 1234);
        // Problem:
        assertFalse(y.ref != null);
        ois.close();
        baos.close();
        bais.close();
    }

    /**
     * Use the readResolve method to construct a new object with the correct
     * finals initialized. Because we now call the constructor explicitly, all
     * finals are properly set up.
     */
    @Test
    public void solution() throws Exception {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(baos);
        FinalExample x = new FinalExample(1234);
        oos.writeObject(x);
        oos.close();
        ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
        ObjectInputStream ois = new ObjectInputStream(bais);
        FinalExample y = (FinalExample) ois.readObject();
        assertTrue(y.ref != null);
        assertTrue(y.value == 1234);
        ois.close();
        baos.close();
        bais.close();
    }

    /**
     * The solution <em>should not</em> have worse execution time than built-in
     * deserialization.
     */
    @Test
    public void benchmark() throws Exception {
        int TRIALS = 500_000;

        long a = System.currentTimeMillis();
        for (int i = 0; i < TRIALS; i++) {
            problem();
        }
        a = System.currentTimeMillis() - a;

        long b = System.currentTimeMillis();
        for (int i = 0; i < TRIALS; i++) {
            solution();
        }
        b = System.currentTimeMillis() - b;

        System.out.println("Problem: " + a / 1000f + "s Solution: " + b / 1000f + "s");
        assertTrue(b <= a);
    }

    public static class FinalExample implements Serializable {

        private static final long serialVersionUID = 4772085863429354018L;

        public final transient Object ref = new Object();

        public final int value;

        private transient GetField fields;

        public FinalExample(int value) {
            this.value = value;
        }

        private FinalExample(GetField fields) throws IOException {
            // assign fields
            value = fields.get("value", 0);
        }

        private void readObject(ObjectInputStream stream) throws IOException,
                ClassNotFoundException {
            fields = stream.readFields();
        }

        private Object readResolve() throws ObjectStreamException {
            try {
                return new FinalExample(fields);
            } catch (IOException ex) {
                throw new InvalidObjectException(ex.getMessage());
            }
        }

    }

    public static class WrongExample implements Serializable {

        private static final long serialVersionUID = 4772085863429354018L;

        public final transient Object ref = new Object();

        public final int value;

        public WrongExample(int value) {
            this.value = value;
        }

    }

}

A note of caution: whenever the class refers to another object instance, it might be possible to leak the temporary "instance" created by the serialization process: the object resolution occurs only after all sub-objects are read, hence it is possible for subobjects to keep a reference to the temporary object. Classes can check for use of such illegally constructed instances by checking that the GetField temporary field is null. Only when it is null, it was created using a regular constructor and not through the deserialization process.

Note to self: Perhaps a better solution exists in five years. See you then!

View more solutions

Comments

[Admin]

Thanks. I suspected it was that way too, but wasn't sure I didn't miss something.

[Admin]

Well, it looks too messy, I guess it's easier to give up on final here ;)

[Pindatjuh]

You can also implement a TransientMap, which you mark final but not transient. Every property, however, in the map must be transient, and hence the map is not serialized, but still existing on unserialization (and empty).

[Admin]

Wouldn't call this easy. The copy constructor is not automatic, so if I have 20 fields, 2 of them transient, I need to selectively copy 18 fields in the copy constructor. However, this does indeed achieve what I wanted.

[Rudi Wijaya]

Your answer "transients cannot be final" is incorrect: please explain Hibernate source code with final transient all over it: github.com/hibernate/hibernate-orm/blob/4.3.7.Final/…

[Christian Hujer]

Actually the answer is wrong. transient fields can be final. But in order to get that working for something else than default values (false / 0 / 0.0 / null), you want to implement not only readObject() but also readResolve(), or use Reflection.

[Pindatjuh]

Note that this only seems to work for primitive values. After testing with Object values, an InternalError is thrown as the GetField object is not expected to escape the readObject method. Therefore this answer reduces to the answer of Boann and adds nothing new.

[Admin]

@mdma Here I posted new question that about transient final working fine as regular. stackoverflow.com/questions/37614088/…

[Holger]

@doublep: actually, deserialization is the reason why this possibility exists. That’s also the reason why it doesn’t work for static final fields, static fields are never (de-)serialized, hence, there is no need for such a feature.

[Jules]

Don't suppose you could expand on this answer, could you? I'm afraid I don't have the book in question...

[Jules]

@user1803551 That's not exactly helpful. Answers here are supposed to provide an actual description of how to solve the problem, not just a pointer to a google search.

[dmatej]

This works but it is also somehow insecure. I don't like it. All instances of the class will have the field accessible, but you need to work with only one concrete instance and only for one single operation.

[Lenny Primak]

It's easy w/Lombok and @Builder(toBuilder = true)

[mjs]

Helped a bit to understand.