How do I serialize & deserialize CSV properly?

Question

I've been trying to serialize an object to a CSV String but the object contains a List and @JsonUnwrapped doesn't work on List objects.

Expected sample output:

color,part.name\n
red,gearbox\n
red,door\n
red,bumper

Actual output:

com.fasterxml.jackson.core.JsonGenerationException: Unrecognized column 'name':

Here is my code: (Most of it is the 2 POJO's)

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import com.fasterxml.jackson.annotation.JsonRootName;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlElementWrapper;
import java.io.IOException;
import static java.util.Arrays.asList;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;

public class NestedWrapping {

@JsonRootName("Car")
@JsonInclude(JsonInclude.Include.NON_DEFAULT)
@JsonAutoDetect(fieldVisibility = JsonAutoDetect.Visibility.ANY, getterVisibility = JsonAutoDetect.Visibility.NONE, setterVisibility = JsonAutoDetect.Visibility.NONE)
@JsonPropertyOrder({"color"})
public static class Car {

    @JsonProperty("color")
    private String color;

    @JsonFormat(shape = JsonFormat.Shape.STRING)
    @JacksonXmlElementWrapper(useWrapping = false)
    private List<Part> parts;

    public String getColor() {
        return color;
    }

    public void setColor(String color) {
        this.color = color;
    }

    public List<Part> getParts() {
        return parts;
    }

    public void setParts(List<Part> parts) {
        this.parts = parts;
    }

}

@JsonInclude(JsonInclude.Include.NON_DEFAULT)
@JsonAutoDetect(fieldVisibility = JsonAutoDetect.Visibility.ANY, getterVisibility = JsonAutoDetect.Visibility.NONE, setterVisibility = JsonAutoDetect.Visibility.NONE)
@JsonPropertyOrder({
    "name"
})
public static class Part {

    @JsonProperty("name")
    private String name;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

}

public static void main(String args[]) {
    try {
        Car car = new Car();
        car.setColor("red");
        Part part1 = new Part();
        part1.setName("geabox");
        Part part2 = new Part();
        part2.setName("door");
        Part part3 = new Part();
        part3.setName("bumper");
        car.setParts(asList(part1, part2, part3));
        System.out.println("serialized: " + serialize(car, Car.class, true));
    } catch (IOException ex) {
        Logger.getLogger(NestedWrapping.class.getName()).log(Level.SEVERE, null, ex);
    }
}

public static final synchronized String serialize(final Object object, final Class type, final Boolean withHeaders) throws IOException {
    CsvMapper csvMapper = new CsvMapper();
    CsvSchema csvSchema;
    if (withHeaders) {
        csvSchema = csvMapper.schemaFor(type).withHeader();
    } else {
        csvSchema = csvMapper.schemaFor(type).withoutHeader();
    }
    return csvMapper.writer(csvSchema).writeValueAsString(object);
}

}

Nothing I try seems to work, I've read every post on stackoverflow and github about the topic but I can't find a working solution.

Sorry about any pointless annotations that I've left behind for no reason and if you answer with code, please feel free to remove them.

Solution 1

From the error, I would like to believe that it has something to do with your schema for a Car, which has the columns of {"color"} taken from @JsonPropertyOrder on Car and not a "name" value.

You probably want to add "parts" in there, but you would get the same error that "name" is not part of that schema.

After a few changes to your code, I was able to serialize and deserialize a Car object.

Part

Here, after some other changes it needed a constructor with a single String value, so add that

@JsonPropertyOrder({"name"})
public static class Part {
    @JsonProperty("name")
    private String name;

    public Part() {
        this("");
    }

    public Part(String partJSON) {
        // TODO: Unserialize the parameter... it is a serialized Part string... 
        this.name = partJSON;
    }

Car

Here, you will need to implement a method that will convert the List<Part> into a CSV-readable format manually.

Such a method would look like this

@JsonGetter("parts")
public String getPartString() {
    String separator = ";";
    StringBuilder sb = new StringBuilder();

    Iterator<Part> iter = this.parts.iterator();
    while (iter.hasNext()) {
        Part p = iter.next();
        sb.append(p.getName());

        if (iter.hasNext())
            sb.append(separator);
    }

    return sb.toString();
} 

Also, don't forget to fix the schema at the top of the class

@JsonPropertyOrder({"color", "parts"})
public static class Car {

    @JsonProperty("color")
    private String color;
    @JsonProperty("parts")
    private List<Part> parts;

    public Car() {
        this.parts = new ArrayList<>();
    }

serialize

You can change your serialize method to take the type of the class as a generic type parameter instead of an explicit Class like so.

public static final synchronized <T> String serialize(final T object, final Boolean withHeaders) throws IOException {
    CsvMapper csvMapper = new CsvMapper();
    CsvSchema csvSchema = csvMapper.schemaFor(object.getClass());

    if (withHeaders) {
        csvSchema = csvSchema.withHeader();
    } else {
        csvSchema = csvSchema.withoutHeader();
    }

    return csvMapper.writer(csvSchema).writeValueAsString(object);
}

main - writer

Now, if you serialize a Car, you should see

color,parts
red,gearbox;door;bumper

main - reader

And reading that CSV string and looping over the Car.getParts()

Car car = mapper.readerFor(Car.class).with(csvSchema).readValue(csv);

for (Part p : car.getParts()) {
    System.out.println(p.getName());
}

gearbox
door
bumper

Solution 2

Full working CSV Serialize & Deserialize solution:

import com.fasterxml.jackson.annotation.JsonGetter;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;
import java.io.IOException;
import java.util.ArrayList;
import static java.util.Arrays.asList;
import java.util.Iterator;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;

public class NestedWrapping {

    @JsonPropertyOrder({"color", "parts"})
    public static class Car {

        @JsonProperty("color")
        private String color;

        @JsonProperty("parts")
        private List<Part> parts;

        public String getColor() {
            return color;
        }

        public void setColor(String color) {
            this.color = color;
        }

        public List<Part> getParts() {
            return parts;
        }

        public void setParts(List<Part> parts) {
            this.parts = parts;
        }

        public Car() {
            this.parts = new ArrayList<>();
        }

        @JsonGetter("parts")
        public String getPartString() {
            String separator = ";";
            StringBuilder sb = new StringBuilder();

            Iterator<Part> iter = this.parts.iterator();
            while (iter.hasNext()) {
                Part p = iter.next();
                sb.append(p.getName());

                if (iter.hasNext()) {
                    sb.append(separator);
                }
            }

            return sb.toString();
        }

        @Override
        public String toString() {
            return "Car{" + "color=" + color + ", parts=" + parts + '}';
        }

    }

    @JsonPropertyOrder({
        "name"
    })
    public static class Part {

        @JsonProperty("name")
        private String name;

        public Part() {
        }

        public Part(String name) {
            this.name = name;
        }

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        @Override
        public String toString() {
            return "Part{" + "name=" + name + '}';
        }

    }

    public static void main(String args[]) {
        try {
            Car car = new Car();
            car.setColor("red");
            Part part1 = new Part();
            part1.setName("geabox");
            Part part2 = new Part();
            part2.setName("door");
            Part part3 = new Part();
            part3.setName("bumper");
            car.setParts(asList(part1, part2, part3));
            String serialized = serialize(car, Car.class, true);
            System.out.println("serialized: " + serialized);
            List<Car> deserializedCars = (List) deserialize(serialized, Car.class, true);
            for (Car deserializedCar : deserializedCars) {
                System.out.println("deserialized: " + deserializedCar.toString());
            }
        } catch (IOException ex) {
            Logger.getLogger(NestedWrapping.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    public static final synchronized String serialize(final Object object, final Class type, final Boolean withHeaders) throws IOException {
        CsvMapper csvMapper = new CsvMapper();
        CsvSchema csvSchema;
        if (withHeaders) {
            csvSchema = csvMapper.schemaFor(type).withHeader();
        } else {
            csvSchema = csvMapper.schemaFor(type).withoutHeader();
        }
        return csvMapper.writer(csvSchema).writeValueAsString(object);
    }

    public static final synchronized List<Object> deserialize(final String csv, final Class type, final Boolean hasHeaders) throws IOException {
        CsvMapper csvMapper = new CsvMapper();
        CsvSchema csvSchema;
        if (hasHeaders) {
            csvSchema = csvMapper.schemaFor(type).withHeader();
        } else {
            csvSchema = csvMapper.schemaFor(type).withoutHeader();
        }
        MappingIterator<Object> mappingIterator = csvMapper.readerFor(type).with(csvSchema).readValues(csv);
        List<Object> objects = new ArrayList<>();
        while (mappingIterator.hasNext()) {
            objects.add(mappingIterator.next());
        }
        return objects;
    }

}

Comments

[OneCricketeer]

Any reason you are using CSV instead of JSON? As far as I can tell, you have many Parts in one Car, and I don't think CSV is the best format to represent that

[Hooli]

I'm using XML and JSON as well but I need to be able to represent it in all 3 formats. The other two work fine already.

[Hooli]

@SotiriosDelimanolis: Done. Same concept as a database JOIN basically.

[Hooli]

@StaxMan: Any ideas?

[OneCricketeer]

I don't see how you could get 3 rows from what you provided since you only serialize one Car object. Each row is one object. Therefore, your data would look could look like red,"gearbox,door,bumper", but I am not sure how Jackson CSV represents lists.

[Hooli]

Apologies for the delayed response. Thank you for clarifying everything. I have gone ahead and posted the full solution below for anyone that's looking for a working CSV serialize/deserialize solution.

[OneCricketeer]

Welcome. Doesn't look like you took my comment on the serialize method