convert CSV lines into Javascript objects

Question

I have a simple csv file

people.csv:

fname, lname, uid, phone, address
John, Doe, 1, 444-555-6666, 34 dead rd
Jane, Doe, 2, 555-444-7777, 24 dead rd
Jimmy, James, 3, 111-222-3333, 60 alive way

What I want to do it get each line of the CSV, convert it to a JavaScript object, store them into an array, and then convert the array into a JSON object.

server.js:

var http = require('http');
var url  = require('url');
var fs = require('fs');

var args = process.argv;
var type = args[2] || 'text';
var arr = []; 
var bufferString; 

function csvHandler(req, res){
  fs.readFile('people.csv',function (err,data) {

  if (err) {
    return console.log(err);
  }

  //Convert and store csv information into a buffer. 
  bufferString = data.toString(); 

  //Store information for each individual person in an array index. Split it by every newline in the csv file. 
  arr = bufferString.split('\n'); 
  console.log(arr); 

  for (i = 0; i < arr.length; i++) { 
    JSON.stringify(arr[i]); 
  }

  JSON.parse(arr); 
  res.send(arr);  
});
}

//More code ommitted

My question is if I am actually converting that CSV lines into Javascript objects when I call the .split('\n') method on bufferString or is there another way of doing so?

Solution 1

By doing this:

arr = bufferString.split('\n'); 

you will have an array containing all rows as string

["fname, lname, uid, phone, address","John, Doe, 1, 444-555-6666, 34 dead rd",...]

You have to break it again by comma using .split(','), then separate the headers and push it into an Javascript Object:

var jsonObj = [];
var headers = arr[0].split(',');
for(var i = 1; i < arr.length; i++) {
  var data = arr[i].split(',');
  var obj = {};
  for(var j = 0; j < data.length; j++) {
     obj[headers[j].trim()] = data[j].trim();
  }
  jsonObj.push(obj);
}
JSON.stringify(jsonObj);

Then you will have an object like this:

[{"fname":"John",
  "lname":"Doe",
  "uid":"1",
  "phone":"444-555-6666",
  "address":"34 dead rd"
 }, ... }]

See this FIDDLE

Solution 2

Using ES6/ES7 and some functional programming guidelines:

• All variables are const (immutability)
• Use map/reduce instead of while/for
• All functions are Arrow
• No dependencies

// Split data into lines and separate headers from actual data
// using Array spread operator
const [headerLine, ...lines] = data.split('\n');

// Split headers line into an array
// `valueSeparator` may come from some kind of argument
// You may want to transform header strings into something more
// usable, like `camelCase` or `lowercase-space-to-dash`
const valueSeparator = '\t';
const headers = headerLine.split(valueSeparator);

// Create objects from parsing lines
// There will be as much objects as lines
const objects = lines
  .map( (line, index) =>
    line
      // Split line with value separators
      .split(valueSeparator)

      // Reduce values array into an object like: { [header]: value }
      // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/reduce
      .reduce(

        // Reducer callback 
        (object, value, index) => ({
          ...object,
          [ headers[index] ]: value,
        }),

        // Initial value (empty JS object)
        {}
      )
  );

console.log("Objects:", objects);

For CSV files using , as separator and quotes string values, you can use this version:

// Split data into lines and separate headers from actual data
// using Array spread operator
const [headerLine, ...lines] = data.split('\n');

// Use common line separator, which parses each line as the contents of a JSON array
const parseLine = (line) => JSON.parse(`[${line}]`);

// Split headers line into an array
const headers = parseLine(headerLine);

// Create objects from parsing lines
// There will be as much objects as lines
const objects = lines
  .map( (line, index) =>

    // Split line with JSON
    parseLine(line)

      // Reduce values array into an object like: { [header]: value } 
      .reduce( 
        (object, value, index) => ({
          ...object,
          [ headers[index] ]: value,
        }),
        {}
      ) 
  );

return objects;

Note: For big files, it would be better to work with streams, generators, iterators, etc.

Comments

[GGMU]

Thank you for this. Could you just explain what the second line does exactly? Wha do you mean by headers?

[alandarev]

@GGMU arr[0] is the first line, that is where the headers are: fname, lname, ... and split(','); explodes the string into array using symbol , as deliminator.

[alandarev]

Judge wise before adding a framework to the depedency list. Especially if all you need is CSV reader.

[GGMU]

Ah got it. Thank you. Exactly what I was looking for.

[Ali Salti]

it's good for the example only. if you have a different csv with line breaks in the fields (2 lines address or something it) the code will mess up badly

[Corné]

That link is now dead.