Java: object to byte[] and byte[] to object converter (for Tokyo Cabinet)

Question

I need to convert objects to a byte[] to be stored in the Tokyo Cabinet key-value store. I also need to unbyte the byte[] to an Object when reading from the key-value store.

Are there any packages out there that will help me with this task? Or would the best solution to implement it myself?

Solution 1

public static byte[] serialize(Object obj) throws IOException {
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    ObjectOutputStream os = new ObjectOutputStream(out);
    os.writeObject(obj);
    return out.toByteArray();
}
public static Object deserialize(byte[] data) throws IOException, ClassNotFoundException {
    ByteArrayInputStream in = new ByteArrayInputStream(data);
    ObjectInputStream is = new ObjectInputStream(in);
    return is.readObject();
}

Solution 2

If your class extends Serializable, you can write and read objects through a ByteArrayOutputStream, that's what I usually do.

Solution 3

Use serialize and deserialize methods in SerializationUtils from commons-lang.

Solution 4

You can look at how Hector does this for Cassandra, where the goal is the same - convert everything to and from byte[] in order to store/retrieve from a NoSQL database - see here. For the primitive types (+String), there are special Serializers, otherwise there is the generic ObjectSerializer (expecting Serializable, and using ObjectOutputStream). You can, of course, use only it for everything, but there might be redundant meta-data in the serialized form.

I guess you can copy the entire package and make use of it.

Solution 5

You can use ObjectMapper

        ObjectMapper objectMapper = new ObjectMapper();
        ObjectClass object = objectMapper.readValue(data, ObjectClass.class);

View more solutions

Comments

[Dean J]

All types included as variables in your class (and all types in those types, and so on) also need to be Serializable.

[Aviad Ben Dov]

Actually he'll need an ObjectOutputStream to wrap the BAOS too... But yeah, that's the easiest way to go.

[Tom Anderson]

I'd consider using something other than Java's built-in serialization, too - JBoss serialization, JSerial, Avro, etc, or an XML format like XStream or Javolution marshalling plus gzip. Standard serialization is not particularly fast, and although its marginal space efficiency is good, there's quite a bit of per-stream overhead.

[Cacho Santa]

don't forget to call is.close and in.close.

[Thomas Mueller]

is.close and in.close are not needed in this case because it's an in-memory stream. The only thing it would do here is make it slower and more verbose. But I understand the IDE might flag the code above because the stream is not closed.

[Cacho Santa]

I don't think it will make slower, it is only an empty method so the time difference must be very low. But you are right: Closing a ByteArrayOutputStream has no effect.Java Doc Awesome :D

[Arthur Eirich]

What if the object to be serialized is of type InputStream?

[Thomas Mueller]

@ArthurEirich Generally it would still work, but it might not do what you expect.

[Arthur Eirich]

@ThomasMueller Unfortnately, "os.writeObject(obj);" would cause a NotSerializableException as InputStream does not implement the Serializable interface =(

[Thomas Mueller]

@ArthurEirich OK, it would only work if the InputStream implementation implements Serializable.

[Zala Janaksinh]

@all if the object is Parcelable then what is solution ??

[Thomas Mueller]

@ZalaJanaksinh that's a different question. This question is about Java, not Android.

[Jenya Kirmiza]

are there any memory limitations using this method?

[Thomas Mueller]

Yes, those operations can run out of memory, specially deserialize: it's relatively easy to construct a (small) byte array that will be deserialized into a extremely large object, which will cause out of memory. There is also a security risk to deserialize any byte array, but this answer is not about security aspects.

[Safeer]

this method is causing "IOException: java.io.StreamCorruptedException: invalid stream header: 7B227374", any idea why its doing so?

[Thomas Mueller]

@Safeer probably because you tried to deserialize, but the input (the byte array) contains a JSON text file starting with {"st.... That won't work. You can only deserialize things that were serialized with the above method. To read a JSON file, try searching for "convert JSON to Java object" or similar.

[CharlieShi]

code not work for offheap cache like ohc. when performing benchmark, the code will throw nullpointer exception.

[Thomas Mueller]

@CharlieShi I think you are not using it correctly. Please ask a new question.

[tgabb]

link is broken!