How to Remove Duplicates Items from a Python List

In this tutorial, we are going to learn to remove duplicate elements from a Python list. We will do this with the following methods.

First Method 

Remove duplicates items from the List using Set

First of all, we convert the list into sets in Python. We know that a Set is not arranged in order collection of data items that are unique and mutable. So if we transfer the list to Set of python then it automatically removes the duplicate items.

First let us create a list and print the list, in the following list 2 appears 4 times, 5 appears 3 times20 appears 2 times.

li=[2, 3, 5, 2, 6, 7, 8, 6, 2, 5, 10, 20, 30, 10, 20, 5, 2]
print(li)

Output

[2, 3, 5, 2, 6, 7, 8, 6, 2, 5, 10, 20, 30, 10, 20, 5, 2]

Now we convert the list Set and then to list again as follows

set1= set(li)
list1 = list(set1)
print(list1)

Output

[2, 3, 5, 6, 7, 8, 10, 20, 30]

So in this way, we will get our desire result. See all the duplicate elements remove.

Second Method to remove duplicates items from List

In the following method, we first convert dictionary keys with the help of dict.fromKeys() method/ function which will remove the duplicate items and the this dictionary keys converted back to list with the help of list function

li=[2, 3, 5, 2, 6, 7, 8, 6, 2, 5, 10, 20, 30, 10, 20, 5, 2]
# print the original list

print(li)
dict1 = dict.fromkeys(li)
newList = list(dict1)
print(newList)

Output

[2, 3, 5, 6, 7, 8, 10, 20, 30]

Third Method

In the third method, we remove the duplicates elements from the list by using for loop with if condition using not in keys.

li=[2, 3, 5, 2, 6, 7, 8, 6, 2, 5, 10, 20, 30, 10, 20, 5, 2]
print("The original list is: " , li)
list1 = []
for i in li:
    if i not in list1:
        list1.append(i)
# print the unique list after removal
print(" list : " , list1)

The original list is :  [2, 3, 5, 2, 6, 7, 8, 6, 2, 5, 10, 20, 30, 10, 20, 5, 2]

Unique list :  [2, 3, 5, 6, 7, 8, 10, 20, 30] 

Using List Comprehension

a = [1, 2, 3, 3, 5, 9, 6, 2, 8, 5, 2, 3, 5, 7, 3, 5, 8, 8, 3]

b = []

[b.append(item) for item in a if item not in b]

print(b)

Output

[1, 2, 3, 5, 9, 6, 8, 7]

SHARE THIS