Python function to merge unique values form multiple lists to one list

Question

I am pretty new to Python. I am trying to write a function that will merge unique values in separate lists into one list. I keep getting a result of a tuple of lists. Ultimately I would like to have one list of unique values from my three lists -a,b,c. Can anyone give me a hand with this?

def merge(*lists):
    newlist = lists[:]
    for x in lists:
        if x not in newlist:
            newlist.extend(x)
    return newlist

a = [1,2,3,4]
b = [3,4,5,6]
c = [5,6,7,8]

print(merge(a,b,c))

I am getting a Tuple of Lists

([1, 2, 3, 4], [3, 4, 5, 6], [5, 6, 7, 8])

Solution 1

You may just need sets:

>>> a = [1,2,3,4]
>>> b = [3,4,5,6]
>>> c = [5,6,7,8]
>>>
>>> uniques = set( a + b + c )
>>> uniques
set([1, 2, 3, 4, 5, 6, 7, 8])
>>>

Solution 2

If you don't care about them being in the original order, the easiest and likely fasted way is to use set functions:

>>> set().union(a, b, c)
{1, 2, 3, 4, 5, 6, 7, 8}

If you do care about the original order (sets happen to preserve it in this case, but aren't guaranteed to), then you can fix your original attempt by realising that the argument lists contains a tuple of all of the original lists you passed in. This means that iterating over it gets you each of those lists one at a time, rather than the elements in them - you can fix this by using the itertools module:

for x in itertools.chain.from_iterable(lists):
   if x not in newlist:
      newlist.append(x)

Also, you would want newlist to start out as an empty list rather than a copy of the input lists.

Comments

[daboross]

+1 for using union instead of adding the lists together beforehand. This means that you can use other iterables instead of just lists.

[g.d.d.c]

Note that depending on the number of unique values the time to check x not in newlist will grow for each iteration.