google apps script - GAS - Issue with Removing Duplicates from Array (Single or MultiColumn)

Question

This is an extension of this question... The difference is that question is looking at 1-column ranges and this one is asking for 1 or 2-column ranges.

I have a couple of columns with a set of numbers I am prepping for a query. Some of these numbers have leading zeros and I will need to keep it on the list with leading zeros and without leading zeros.

What I have done so far is to create a column of values with leading and without leading zeros. Here is an example of the array when I getValues on the column(s).

[[1],[2],[001],[002],[1],[2]] for a single column or [[1,1],[2,1],[001,1],[002,1],[1,1],[2,1]] for multiple columns

The end result should be... [[1],[2],[001],[002]] or [[1,1],[2,1],[001,1],[002,1]], respectively.

The last two were dropped because they were duplicates and I only need it in the array once.

Here is what I am trying but I am having issues:

var array = sh.getRange(1,sh.getLastColumn(),sh.getLastRow(),1).getValues();
var uniqueArray = removeDuplicates(array)

function removeDuplicates(myArray){
  var newArray = [];
  myArray.forEach(function(x){
    if(newArray.indexOf(x[0]) === -1){
      newArray.push(x[0]);
    }                   
  });
}

Error: The array comes back as null and then when I try to get uniqueArray.length it will give me TypeError: Cannot read property 'length' of undefined

I've also tried:

var uniqueArray = Array.from(new Set(array));

This seems like it would be less taxing and I like it but it returns all values. It doesn't drop the duplicates.

What am I doing wrong and what is the best approach? How can I fix this?

Answer 1

Use Map to create a hash map with unique keys:

const remDups = (arr2d, colToCompare = 0) => [
  ...arr2d
    .reduce(
      (map, row) =>
        map.has(row[colToCompare]) ? map : map.set(row[colToCompare], row),
      new Map()
    )
    .values(),
];
console.log(
  remDups([
    [1, 1],
    [2, 1],
    ['001', 1],
    ['002', 1],
    [1, 2],
    [2, 3],
  ])
);