Jackson De/Serializing Date-to-String-to-Date in generic Maps

Question

There are many examples of Jackson to/from java.util.Date code but they all seem to leverage POJO annotation. I have generic Maps of scalars that I wish to de/serialize to JSON. This is the current deserializer setup; very simple:

public class JSONUtils {
    static {
        DateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");

        mapper = new ObjectMapper();

        mapper.configure(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS, true);
        mapper.setDateFormat(df);  // this works for outbounds but has no effect on inbounds

        mapper.getDeserializationConfig().with(df); // Gave this a shot but still does not sniff strings for a format that we declare should be treated as java.util.Date                           
  }
  public static Map<String,Object> parseJSON(InputStream is) {
    Map<String,Object> data = null;

    try {
        data = mapper.readValue(is, Map.class);
    } catch(Exception e) {
      // ...
    }

    return data;
}

I grok that a dateserializer can turn java.util.Date into a ISO 8601-ish string. It's going the other way that puzzles me. Clearly, in a JSON doc with no context, a string is a string so I cannot know if it was once a date. So I am prepared to duck type this and examine all strings being deserialized and if they smell like YYYY-MM-DDTHH:MM:SS.sss datetimes, then I will make a java.util.Date instead of just passing back a String. So given:

{ "name": "buzz",
  "theDate": "2013-09-10T12:00:00.000"
}

will yield

Map<String,Object> m = mapper.readValue(is, Map.class);
Object o1 = m.get("name");   // o1 is instanceof String
Object o2 = m.get("theDate");  // o2 is instanceof Date

But this means that the deserializer has to return two different types and I have not been able to figure out how to do this in Jackson. Does anyone know of a good, compact example that will sniff for date-like strings and turn them into Dates, leaving others as Strings?

Solution 1

I've been looking for the answer on a related subject recently and come up with the following solution, thanks to Justin Musgrove and his article Custom jackson date deserializer. Basically, the idea is to replace standard deserializer for Object.class that will convert any string in the specified format to the Date object or fallback to the standard behaviour otherwise. Obviously, this operation comes at cost of extra processing, so you'd want to keep a dedicated instance of ObjectMapper configured for this and only use it when absolutely necessary or if prepared doing second pass anyway.

Note that the Date string format in your example has no timezone component, which may cause some issues, but I leave the format as requested. You can use a parser of your choice in place of the FastDateFormat from Apache Commons Lang. I actually use Instant in my case.

CustomObjectDeserializer.java

import java.io.IOException;

import org.apache.commons.lang3.time.FastDateFormat;

import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.JsonTokenId;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer;

public class CustomObjectDeserializer extends UntypedObjectDeserializer {
    private static final long serialVersionUID = 1L;

    private static final FastDateFormat format = FastDateFormat.getInstance("yyyy-MM-dd'T'HH:mm:ss.SSS");

    public CustomObjectDeserializer() {
        super(null, null);
    }

    @Override
    public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
        if (p.getCurrentTokenId() == JsonTokenId.ID_STRING) {
            try {
                String value = p.getText();
                // put your own parser here
                return format.parse(value);
            } catch (Exception e) {
                return super.deserialize(p, ctxt);
            }
        } else {
            return super.deserialize(p, ctxt);
        }
    }

}

JSONUtils.java

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Map;

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;

public class JSONUtils {
    private static final ObjectMapper mapper = new ObjectMapper();

    static {
        mapper.configure(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS, true);

        SimpleModule module = new SimpleModule("DateConverter");
        // register a new deserializer extending and replacing UntypedObjectDeserializer 
        module.addDeserializer(Object.class, new CustomObjectDeserializer());
        mapper.registerModule(module);
    }

    public static Map<String, Object> parseJSON(InputStream is) {
        Map<String, Object> data = null;

        try {
            data = mapper.readValue(is, Map.class);
        } catch (Exception e) {
            // ...
            e.printStackTrace();
        }

        return data;
    }

    public static void main(String[] args) throws Exception {
        String input = "{\"name\": \"buzz\", \"theDate\": \"2013-09-10T12:00:00.000\"}";
        InputStream is = new ByteArrayInputStream(input.getBytes(StandardCharsets.UTF_8));

        Map<String, Object> m = mapper.readValue(is, Map.class);
        Object o1 = m.get("name"); // o1 is instanceof String
        Object o2 = m.get("theDate"); // o2 is instanceof Date
        System.out.println(o1.getClass().getName() + " : " + o1);
        System.out.println(o2.getClass().getName() + " : " + o2);
    }
}

Solution 2

If you have a POJO, you can easy use annotation on get and set method with serializer and deserializer.

following an example that serialize and deserialize objects in different ways: List<POJO> to String, String to Map and Map to List<POJO> again. Obviously, in the map the Date values are as String.

This solution is thread safe because uses org.joda.time.format.DateTimeFormat and org.joda.time.format.DateTimeFormatter, you can find more info herein this post How to deserialize JS date using Jackson? and this link http://fahdshariff.blogspot.co.uk/2010/08/dateformat-with-multiple-threads.html

My POJO:

@JsonAutoDetect
public class QueueTask implements Serializable {

    private static final long serialVersionUID = -4411796657106403937L;

    public enum ActivitiQueueStatus {

        IN_PROGRESS(AsyncProcessingWorkflowContentModel.InProgressTask.TYPE.getLocalName()), //
        IN_QUEUE(AsyncProcessingWorkflowContentModel.InQueueTask.TYPE.getLocalName());

        private String value;

        private ActivitiQueueStatus(final String value) {
            this.value = value;
        }

        public static ActivitiQueueStatus enumOf(final String value) {
            for (ActivitiQueueStatus enum_i : values()) {
                if (enum_i.value.equals(value))
                    return enum_i;
            }
            throw new IllegalArgumentException("value '" + value + "' is not a valid enum");
        }
    }

    private String user;

    private Date creationDate;

    private int noRowsSelected;

    private ActivitiQueueStatus status;


    public String getUser() {
        return user;
    }

    public void setUser(String user) {
        this.user = user;
    }

    @JsonSerialize(using = JsonDateSerializer.class)
    public Date getCreationDate() {
        return creationDate;
    }

    @JsonDeserialize(using = JsonDateDeSerializer.class)
    public void setCreationDate(Date creationDate) {
        this.creationDate = creationDate;
    }

    public int getNoRowsSelected() {
        return noRowsSelected;
    }

    public void setNoRowsSelected(int noRowsSelected) {
        this.noRowsSelected = noRowsSelected;
    }

    public ActivitiQueueStatus getStatus() {
        return status;
    }

    public void setStatus(ActivitiQueueStatus status) {
        this.status = status;
    }

}

My Serializer:

@Component
public class JsonDateDeSerializer extends JsonDeserializer<Date> {

    // use joda library for thread safe issue
    private static final DateTimeFormatter dateFormat = DateTimeFormat.forPattern("dd/MM/yyyy hh:mm:ss");

    @Override
    public Date deserialize(final JsonParser jp, final DeserializationContext ctxt) throws IOException, JsonProcessingException {
        if (jp.getCurrentToken().equals(JsonToken.VALUE_STRING))
            return dateFormat.parseDateTime(jp.getText().toString()).toDate();
        return null;
    }

}

and Deserializer:

@Component
public class JsonDateSerializer extends JsonSerializer<Date> {

    // use joda library for thread safe issue
    private static final DateTimeFormatter dateFormat = DateTimeFormat.forPattern("dd/MM/yyyy hh:mm:ss");

    @Override
    public void serialize(final Date date, final JsonGenerator gen, final SerializerProvider provider) throws IOException, JsonProcessingException {

        final String formattedDate = dateFormat.print(date.getTime());

        gen.writeString(formattedDate);
    }

}

My Service:

public class ServiceMock {

    // mock this parameter for usage.
    public List<QueueTask> getActiveActivities(QName taskStatus) {
        final List<QueueTask> listToReturn = new LinkedList<QueueTask>();

        final SimpleDateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss");
        Date d1 = null, d2 = null, d3 = null, d4 = null, d5 = null;

        try {
            d1 = dateFormat.parse("01/02/2013 12:44:44");
            d2 = dateFormat.parse("21/12/2013 16:44:44");
            d3 = dateFormat.parse("21/12/2013 16:45:44");
            d4 = dateFormat.parse("21/12/2013 16:44:46");
            d5 = dateFormat.parse("11/09/2013 16:44:44");
        } catch (ParseException e) {
        }

        QueueTask dataSet = new QueueTask();
        dataSet = new QueueTask();
        dataSet.setUser("user_b");
        dataSet.setStatus(ActivitiQueueStatus.enumOf("placeInQueue"));
        dataSet.setNoRowsSelected(500);
        dataSet.setCreationDate(d1);
        listToReturn.add(dataSet);

        dataSet = new QueueTask();
        dataSet.setUser("user_d");
        dataSet.setStatus(ActivitiQueueStatus.enumOf("placeInQueue"));
        dataSet.setNoRowsSelected(300);
        dataSet.setCreationDate(d2);
        listToReturn.add(dataSet);

        dataSet = new QueueTask();
        dataSet.setUser("user_a");
        dataSet.setStatus(ActivitiQueueStatus.enumOf("inProgress"));
        dataSet.setNoRowsSelected(700);
        dataSet.setCreationDate(d3);
        listToReturn.add(dataSet);

        dataSet = new QueueTask();
        dataSet.setUser("user_k");
        dataSet.setStatus(ActivitiQueueStatus.enumOf("inProgress"));
        dataSet.setNoRowsSelected(700);
        dataSet.setCreationDate(d4);
        listToReturn.add(dataSet);

        dataSet = new QueueTask();
        dataSet.setUser("user_l");
        dataSet.setStatus(ActivitiQueueStatus.enumOf("inProgress"));
        dataSet.setNoRowsSelected(700);
        dataSet.setCreationDate(d5);
        listToReturn.add(dataSet);

        return listToReturn;
    }

}

MAIN usage:

public class SerializationServiceTest {

    private static final Logger LOGGER = LoggerFactory.getLogger(OUPQueueStatusServiceIT.class);

    public void testGetActiveActivitiesSerialization() throws Exception {
        LOGGER.info("testGetActiveActivitiesSerialization - start");

        ServiceMock mockedService = new ServiceMock();

        // AsyncProcessingWorkflowContentModel.InProgressTask.TYPE is an QName, mock this calling
        List<QueueTask> tasks = mockedService.getActiveActivities(AsyncProcessingWorkflowContentModel.InProgressTask.TYPE);
        assertNotNull(tasks);
        assertTrue(tasks.size() == 5);
        assertNotNull(tasks.get(0).getUser());
        assertNotNull(tasks.get(0).getCreationDate());
        assertNotNull(tasks.get(0).getStatus());
        assertNotNull(tasks.get(0).getNoRowsSelected());

        final ObjectMapper mapper = new ObjectMapper();
        final String jsonString = mapper.writeValueAsString(tasks);

        assertNotNull(jsonString);
        assertTrue(jsonString.contains("creationDate"));

        // test serialization from string to Map
        final List<Map<String, Object>> listOfMap = mapper.readValue(jsonString, new TypeReference<List<Map<String, Object>>>() {
        });
        assertNotNull(listOfMap);

        final DateFormat formatter = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss");
        for (Map<String, Object> map_i : listOfMap) {
            // check date value
            assertTrue(map_i.containsKey("creationDate"));
            final Date date = formatter.parse("" + map_i.get("creationDate"));
            assertNotNull(date);

            assertNotNull(map_i.get("user"));
            assertNotNull(map_i.get("status"));
            assertNotNull(ActivitiQueueStatus.valueOf("" + map_i.get("status")));
            assertNotNull(map_i.get("noRowsSelected"));
        }

        // test de-serialization
        List<QueueTask> deserializedTaskList = mapper.convertValue(listOfMap, new TypeReference<List<QueueTask>>() {
        });

        assertNotNull(deserializedTaskList);
        assertTrue(deserializedTaskList.size() == 5);
        for (QueueTask t : deserializedTaskList) {
            assertNotNull(t.getUser());
            assertNotNull(t.getCreationDate());
            assertNotNull(t.getDownloadType());
            assertNotNull(t.getStatus());
        }
        LOGGER.info("testGetActiveActivitiesSerialization - end");
    }

    public static void main(String[] args) throws Exception {
        new SerializationServiceTest().SerializationServiceTest();
    }

}

Solution 3

After some weeks poking around on this (and no other comments or answers), I now believe what I seek is NOT possible in Jackson. Deserialization of JSON into a Map with ducktyping for dates must occur after-the-fact. There is no way to interpose the parse stream, sniff the string for YYYY-MM-DDTHH:MM:SS.SSS and upon match substitute a Date object instead of String. You must let Jackson build the Map, then outside of Jackson go back to the top and walk the Map, sniffing for dates.

I will add that since I have a very specific duck I am looking for, the fastest implementation to turn the String into a Date is a hand-rolled thing about 120 lines long that validates and sets up the proper integer m-d-y-h-m-s-ms for Calendar then calls getTime(). 10,000,000 conversions takes 4240 millis, or about 2.3m/sec.

Before the joda-time lobby pipes up, yes, I tried that first:

// This is set up ONCE, outside the timing loop:
DateTimeFormatter format = ISODateTimeFormat.dateHourMinuteSecondMillis();

// These are in the timing loop:
while(loop) {
    DateTime time = format.parseDateTime("2013-09-09T14:45:00.123");
    Date d = time.toDate();
}

takes about 9630 millis to run, about 1.04m/sec; half the speed. But that's still WAY faster than the "out of the box use javax" option:

java.util.Calendar c2 = javax.xml.bind.DatatypeConverter.parseDateTime(s);
Date d = c2.getTime();

This takes 30428 mills to run, about .33m/sec -- almost 7x slower than the handroll.

SimpleDateFormat is not thread-safe so therefore was not considered in for use in converter utility where I cannot make any assumptions about the callers.

Comments

[nmorenor]

Thought not sure why the editor is missing some of the syntax here, I hope it help.

[Buzz Moschetti]

This is a good example of how to use the annotations features. These features allow the developer to specifically assign a de/serializer scheme to a particular field. But my problem is different. The Maps are generic. There are no annotations, only a Map carrying key-value pairs, where the value can be String, Int, Long, BigDecimal, Date, Map, and List. Jackson out of the box supports nice de/serialization of map-of-map constructs and mapper.configure(DeserializationFeature.USE_BIG_DECIMAL_FOR_‌​FLOATS, true) licks the float problem. This leaves Dates.

[Buzz Moschetti]

Oh -- I'd like to add that post-processing a map-of-maps by walking the objects with instanceof is wicked expensive. I'd much rather inject the logic into Jackson at the point where it is deserializing a set of chars.

[nmorenor]

According with Jackson documentation there is a SerializationFeature WRITE_DATE_KEYS_AS_TIMESTAMPS and a setDateFormat method on the ObjectMapper. Not sure if it is exactly what you are looking for, but it sounds like a good start fasterxml.github.io/jackson-databind/javadoc/2.2.0/com/… fasterxml.github.io/jackson-databind/javadoc/2.2.0/com/…

[Buzz Moschetti]

This was the first thing I tried because it seemed so... easy and appropriate (sidestepping the DateFormat sync issue for a moment). So setDateFormat() works great on the way out - Dates are rendered as I wish. But although the docs say "void setDateFormat(DateFormat dateFormat) Method for configuring DateFormat to use when serializing time values as Strings, and deserializing from JSON Strings." it does not seem to sniff JSON Strings for the supplied DateFormat and convert them back to java.util.Date.

[Buzz Moschetti]

Did a -1 because the answer is for a different question.