Python: Combining two lists and removing duplicates in a functional programming way

Question

I'm trying to write a function that would combine two lists while removing duplicate items, but in a pure functional way. For example:

a = [1,2,2]
b = [1,3,3,4,5,0]
union(a,b) --> [1,2,3,4,5,0]

The imperative form of the code would be:

def union(a,b):
    c = []
    for i in a + b:
        if i not in c:
            c.append(i)
    return c

I've tried several approaches, but couldn't find a way to do that without using a loop to go over the items - what am I missing?

Solution 1

list(set(a + b))

This combines two lists a and b and using set takes only unique vales and then we can make it back to list.

Solution 2

If you want to keep the order you can use collections.OrderedDict, otherwise just use set. These data structures use hash values of their items for preserving them, thus they don't keep the duplicates.

In [11]: from collections import OrderedDict

In [12]: list(OrderedDict.fromkeys(a+b))
Out[12]: [1, 2, 3, 4, 5, 0]

Solution 3

Have you tried using sets?

>>> a = [1,2,2]
>>> b = [1,3,3,4,5,0]
>>> list(set(a).union(set(b)))
[0, 1, 2, 3, 4, 5]

Solution 4

To combine the two lists:

a = [1,2,2]
b = [1,3,3,4,5,0]

Using sets:

union = set(a) | set(b)
# -> set([0, 1, 2, 3, 4, 5])

Using comprehension list:

union = a + [x for x in b if x not in a]
# -> [1, 2, 2, 3, 3, 4, 5, 0]

Using loop, without duplicates, preserving order:

union = []
for x in a + b:
    if x not in union:
        union.append(x)
# -> [1, 2, 3, 4, 5, 0]

View more solutions

Comments

[Mazdak]

The set doesn't preserve the order.

[Moinuddin Quadri]

OP never asked that he want to preserve the order?

[DanielR]

Unfortunately I am looking to preserve the order, so this doesn't solve my problem, although I do appreciate the simplicity of the solution

[Moinuddin Quadri]

@DanielR: Then You must mention that in your question. We can't guess what you desire.