Python unique list using set

24,529
>>> from collections import OrderedDict
>>> items = [1, 2, 3, 'a', 2, 4, 'a']
>>> OrderedDict.fromkeys(items).keys()
[1, 2, 3, 'a', 4]
Share:
24,529
user1744238
Author by

user1744238

Updated on August 23, 2022

Comments

  • user1744238
    user1744238 over 1 year

    Possible Duplicate:
    How do you remove duplicates from a list in Python whilst preserving order?

    What I am trying to do is write a method that takes a list as an argument and uses a set to return a copy of the list where each element only occurs once, as well as having the elements in the new list occur in order of their first occurrence in the original list. I HAVE to use a set for this, however, I can't make it so that the output is in the right order while having a quick result. If I put something like this:

    def unique(a):
    
    return list(set(a))
    

    and passed a list with millions of elements, it would give me a result quickly, but it wouldn't be ordered. So what I have right now is this:

    def unique(a):
    b = set(a)
    c = {}
    d = []
    for i in b:
        c[a.index(i)] = i
    for i in c:
        d.append(c[i])
    return d
    

    This gives me the result I want, but not fast enough. If I pass a list with a million elements, I could be waiting for half an hour, whereas the one liner up there takes less than a second. How could I solve this problem?

  • user1744238
    user1744238 over 11 years
    Thanks, but I NEED to incorporate a set somehow. Is there any way to do that?
  • jamylak
    jamylak over 11 years
    @user1744238 My solution is the most efficient you can get but if for some reason you really need to use a set I'm guessing for an assignment then see this: stackoverflow.com/questions/480214/…
  • thierrybm
    thierrybm over 10 years
    A similar way using a set (but once again the answer above is better, without using a set): items = [1, 2, 3, 'a', 2, 4, 'a']; s = set(); filter(lambda i: not i in s and not s.add(i), items)