Apr 3, 2014

Remove Duplicates In A List Without A Loop

I had worked on a past project ColdFusion project that required me to process large amounts of list data from a ColdFusion web service. Part of the process involved removing duplicates, which was using a cfloop to loop through the list. They had a limit to the amount they could process since this was a web service and taking too long to respond back to the calling server can cause some timeout issues.

I was able to increase the amount that could be processed and increase performance by not not using a loop at all.

        lstNumbers = "1513,5845,9548,6955,5214,7845,3256,4588,1259,3578,9589,1513,3256,8599,5411,5698,8526,4785";
        writeOutput("List Length: " & #listLen(lstNumbers)# & "<br />");
        arrNumbers = listToArray(lstNumbers);
        writeOutput("Array Length: " & arrayLen(arrNumbers) & "<br />");
        objHashSet = createObject("java", "java.util.HashSet");
        arrCleanNumbers = objHashSet.toArray();
        writeOutput("Array Length No Duplicates: " & arrayLen(arrCleanNumbers) & "<br />");
        lstCleanNumbers = arrayToList(arrCleanNumbers);
        writeOutput("List Length No Duplicates: " & listLen(lstCleanNumbers));


I first created an array and then initiated the java HashSet object. I then converted my array to a java array, doing this will automatically remove the duplicates in the array. You can then create a list from this array. Simple, fast and no looping required.