underscore/lodash unique by multiple properties

Question

I have an array of objects with duplicates and I'm trying to get a unique listing, where uniqueness is defined by a subset of the properties of the object. For example,

{a:"1",b:"1",c:"2"}

And I want to ignore c in the uniqueness comparison.

I can do something like

_.uniq(myArray,function(element) { return element.a + "_" + element+b});

I was hoping I could do

_.uniq(myArray,function(element) { return {a:element.a, b:element.b} });

But that doesn't work. Is there something like that I can do, or do I need to create a comparable representation of the object if I'm comparing multiple properties?

Solution 1

There doesn't seem to be a straightforward way to do this, unfortunately. Short of writing your own function for this, you'll need to return something that can be directly compared for equality (as in your first example).

One method would be to just .join() the properties you need:

_.uniqBy(myArray, function(elem) { return [elem.a, elem.b].join(); });

Alternatively, you can use _.pick or _.omit to remove whatever you don't need. From there, you could use _.values with a .join(), or even just JSON.stringify:

_.uniqBy(myArray, function(elem) {
    return JSON.stringify(_.pick(elem, ['a', 'b']));
});

Keep in mind that objects are not deterministic as far as property order goes, so you may want to just stick to the explicit array approach.

P.S. Replace uniqBy with uniq for Lodash < 4

Solution 2

Use Lodash's uniqWith method:

_.uniqWith(array, [comparator])

This method is like _.uniq except that it accepts comparator which is invoked to compare elements of array. The order of result values is determined by the order they occur in the array. The comparator is invoked with two arguments: (arrVal, othVal).

When the comparator returns true, the items are considered duplicates and only the first occurrence will be included in the new array.

---

Example:
I have a list of locations with latitude and longitude coordinates -- some of which are identical -- and I want to see the list of locations with unique coordinates:

const locations = [
  {
    name: "Office 1",
    latitude: -30,
    longitude: -30
  },
  {
    name: "Office 2",
    latitude: -30,
    longitude: 10
  },
  {
    name: "Office 3",
    latitude: -30,
    longitude: 10
  }
];

const uniqueLocations = _.uniqWith(
  locations,
  (locationA, locationB) =>
    locationA.latitude === locationB.latitude &&
    locationA.longitude === locationB.longitude
);

// Result has Office 1 and Office 2

Solution 3

Here there's the correct answer

javascript - lodash - create a unique list based on multiple attributes.

FYI var result = _.uniqBy(list, v => [v.id, v.sequence].join());

Solution 4

late to the party but I found this in lodash docs.

var objects = [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }, { 'x': 1, 'y': 2 }];
 
_.uniqWith(objects, _.isEqual);
// => [{ 'x': 1, 'y': 2 }, { 'x': 2, 'y': 1 }]

Solution 5

I do think that the join() approach is still the simplest. Despite concerns raised in the previous solution, I think choosing the right separator is the key to avoiding the identified pitfalls (with different value sets returning the same joined value). Keep in mind, the separator need not be a single character, it can be any string that you are confident will not occur naturally in the data itself. I do this all the time and am fond of using '~!$~' as my separator. It can also include special characters like \t\r\n etc.

If the data contained is truly that unpredictable, perhaps the max length is known and you could simply pad each element to its max length before joining.

View more solutions

Comments

[friedi]

And why are you trying to do the second attempt? The first one is working, right?

[Jeff Storey]

Yes the first is working but it feels a bit hacky to have to do the string concatenation. Trying to understand if there's a more natural way to do this.

[dandavis]

objects are always unique, so you need to compare by individual property values, not by whole objects. using a string compare can work with certain data but not others, for example: with numerical strings like shown, you risk colliding {a:"1"} with {a:1}.s

[dandavis]

_.uniq([{a:"1",b:"1",c:"2"},{a:"1",b:"2",c:"2"},{a:"1",b:"1"‌​,c:"2"}], JSON.stringify); JSON order is not guaranteed, but i can't see why this wouldn't work within a single browser.

[Jeff Storey]

In my particular case, I'm only comparing strings. @dandavis I don't want to compare all of the attributes, only a subset of them

[mu is too short]

@dandavis I think it would be better to pack the properties into arrays before JSONifying them. Or write your own version of _.uniq that uses _.isEqual instead of ===.

[Koushik Chatterjee]

@JeffStorey you want to use _unique strictly, or you want to do a more functional solution? like creating a comparator function and combine reduce/find or combile filter/find?

[mu is too short]

Using join('') is full of holes (such as [1,23] and [12,3]).

[voithos]

@muistooshort: Good point - I suppose the standard comma-delimited join would be better.

[dandavis]

@voithos: but what if the data contains a comma? or if a number is/isnt quoted?

[voithos]

@dandavis: I suppose you could do JSON.stringify on the array, instead of using join. Really, though, it would be easier to just rewrite uniq to take into consideration multiple properties.

[nils petersohn]

lodash 4 comes with _.uniqWith(myArray, _.isEqual)

[vinhboy]

@nilspetersohn Thanks, works for me. But out of curiosity, what does the regular .uniq use, if not _.isEqual?

[user1477388]

@vinhboy it uses SameValueZero. Ref. lodash.com/docs/4.17.15#uniq Ref. lodash.com/docs/3.10.1#uniq

[eddy]

This should be the answer

[Franco]

This works but needs a Return inside the function

[Reed Dunkle]

@Franco the arrow function is using an implicit return. Read more here

[Stanislau Listratsenka]

What's about complexity? i'm not sure but presumably uniqWIth has O(n^2) but uniqBy has O(n)

[Reed Dunkle]

@StanislauListratsenka I think they're two different tools for two different purposes. If all you need is uniqBy, I'd use that. In my opinion, this use case prefers uniqWith. I'm not sure about the complexity. At first glance, it seems like it would be 2N, not n^2, which is the same as the solutions that use uniqBy. But I'm not 100% sure. Let me know if you discover anything in the source code.