Merge two arrays and remove duplicates in Java

Question

I am having trouble removing the duplicates from two arrays that have been merged into one. I have written the following code that merges the arrays, yet I'm not sure how to remove the duplicates from the final array. Assume the arrays are already sorted.

public static int[] merge(int[] list1, int[] list2) {
    int[] result = new int[list1.length + list2.length];

    int i = 0;
    int j = 0;

    for (int k = 0; k < (list1.length + list2.length); k++) {
        if (i >= list1.length) {
            result[k] = list2[j];
            j++;
        } 
        else if (j >= list2.length) {
            result[k] = list1[i];
            i++;
        } 
        else {
            if (list1[i] < list2[j]) {
                result[k] = list1[i];
                i++;
            } else {
                result[k] = list2[j];
                j++;
            }
        }
    }
    return result;
}

Solution 1

Ok, someone hated all the answers. Here's another attempt that combines two stackoverflow q's, combining arrays and removing dupes.

This one runs a good deal faster than my earlier attempt on two lists of a million ints.

public int[] mergeArrays2(int[] arr1, int[] arr2){
    int[] merged = new int[arr1.length + arr2.length];
    System.arraycopy(arr1, 0, merged, 0, arr1.length);
    System.arraycopy(arr2, 0, merged, arr1.length, arr2.length);

    Set<Integer> nodupes = new HashSet<Integer>();

    for(int i=0;i<merged.length;i++){
        nodupes.add(merged[i]);
    }

    int[] nodupesarray = new int[nodupes.size()];
    int i = 0;
    Iterator<Integer> it = nodupes.iterator();
    while(it.hasNext()){
        nodupesarray[i] = it.next();
        i++;
    }



    return nodupesarray;
}

console output:

INFO [main] (TestMergeArray.java:40) - creating two lists of a million ints
DEBUG [main] (TestMergeArray.java:41) - list 1 size : 1000000
DEBUG [main] (TestMergeArray.java:42) - list 2 size : 1000000
INFO [main] (TestMergeArray.java:56) - now merging
INFO [main] (TestMergeArray.java:59) - done, final list size is 864975

Solution 2

this clearer lambda solution is slightly slower because of the (un)boxing
requires Java 8 or above

public static int[] mergedistinct( int[] array1, int[] array2 ) {
  Stream<Integer> s1 = IntStream.of( array1 ).boxed();
  Stream<Integer> s2 = IntStream.of( array2 ).boxed();
  return( Stream.concat( s1, s2 ).distinct().mapToInt( i -> i ).toArray() );
}

[1, 2, 3, 5, 4, 7, 8]

if you need the array sorted:

…
return( Stream.concat( s1, s2 ).distinct().sorted().mapToInt( i -> i ).toArray() );

Solution 3

Here's a technique that iterates the arrays just once and does not use a hash to detect duplicates.

import java.util.List;
import java.util.ArrayList;
import java.util.Arrays;

public class SortedMerge {
    public static int[] merge(int[] array1, int[] array2) {
        int[] a;
        int[] b;
        List<Integer> c = new ArrayList<Integer>();
        int i = 0;
        int j = 0;

        // b is longer than a
        if (array1.length > array2.length) {
            a = array2;
            b = array1;
        } else {
            a = array1;
            b = array2;
        }

        while (j < b.length) {
            int bb = b[j];

            if (i < a.length) {
                int aa = a[i];

                if (aa > bb) {
                    c.add(bb);
                    j++;
                } else {
                    c.add(aa);
                    i++;
                    if (aa == bb) {
                        j++;
                    }
                }
            } else {
                c.add(bb);
                j++;
            }
        }
        // java 8 List<Integer> to int[]
        return c.stream().mapToInt(Integer::intValue).toArray();
    }

    public static void main(String[] args) throws Exception {
        int[] array1 = new int[]{3, 5, 8, 11, 14};
        int[] array2 = new int[]{1, 2, 3, 4, 6, 8, 14, 15, 17};
        int[] c = merge(array1, array2);

        for (int i = 0; i < c.length; i++) {
            System.out.format("%d,", c[i]);
        }
        System.out.println();
        // output> 1,2,3,4,5,6,8,11,14,15,17,
    }
}

Comments

[Patrick Parker]

this answer does not take into account the fact that user is merging two sorted arrays and wants to maintain the sort.