Using map() on an iterator

Question

Say we have a Map: let m = new Map();, using m.values() returns a map iterator.

But I can't use forEach() or map() on that iterator and implementing a while loop on that iterator seems like an anti-pattern since ES6 offer functions like map().

So is there a way to use map() on an iterator?

Solution 1

The simplest and least performant way to do this is:

Array.from(m).map(([key,value]) => /* whatever */)

Better yet

Array.from(m, ([key, value]) => /* whatever */))

Array.from takes any iterable or array-like thing and converts it into an array! As Daniel points out in the comments, we can add a mapping function to the conversion to remove an iteration and subsequently an intermediate array.

Using Array.from will move your performance from O(1) to O(n) as @hraban points out in the comments. Since m is a Map, and they can't be infinite, we don't have to worry about an infinite sequence. For most instances, this will suffice.

There are a couple of other ways to loop through a map.

Using forEach

m.forEach((value,key) => /* stuff */ )

Using for..of

var myMap = new Map();
myMap.set(0, 'zero');
myMap.set(1, 'one');
for (var [key, value] of myMap) {
  console.log(key + ' = ' + value);
}
// 0 = zero
// 1 = one

Solution 2

You could define another iterator function to loop over this:

function* generator() {
    for (let i = 0; i < 10; i++) {
        console.log(i);
        yield i;
    }
}

function* mapIterator(iterator, mapping) {
    for (let i of iterator) {
        yield mapping(i);
    }
}

let values = generator();
let mapped = mapIterator(values, (i) => {
    let result = i*2;
    console.log(`x2 = ${result}`);
    return result;
});

console.log('The values will be generated right now.');
console.log(Array.from(mapped).join(','));

Now you might ask: why not just use Array.from instead? Because this will run through the entire iterator, save it to a (temporary) array, iterate it again and then do the mapping. If the list is huge (or even potentially infinite) this will lead to unnecessary memory usage.

Of course, if the list of items is fairly small, using Array.from should be more than sufficient.

Solution 3

Other answers here are... Weird. They seem to be re-implementing parts of the iteration protocol. You can just do this:

function* mapIter(iterable, callback) {
  for (let x of iterable) {
    yield callback(x);
  }
}

and if you want a concrete result just use the spread operator ....

[...mapIter([1, 2, 3], x => x**2)]

Solution 4

This simplest and most performant way is to use the second argument to Array.from to achieve this:

const map = new Map()
map.set('a', 1)
map.set('b', 2)

Array.from(map, ([key, value]) => `${key}:${value}`)
// ['a:1', 'b:2']

This approach works for any non-infinite iterable. And it avoids having to use a separate call to Array.from(map).map(...) which would iterate through the iterable twice and be worse for performance.

Solution 5

There is a proposal, that brings multiple helper functions to Iterator: https://github.com/tc39/proposal-iterator-helpers (rendered)

You can use it today by utilizing core-js-pure:

import { from as iterFrom } from "core-js-pure/features/iterator";

// or if it's working for you (it should work according to the docs,
// but hasn't for me for some reason):
// import iterFrom from "core-js-pure/features/iterator/from";

let m = new Map();

m.set("13", 37);
m.set("42", 42);

const arr = iterFrom(m.values())
  .map((val) => val * 2)
  .toArray();

// prints "[74, 84]"
console.log(arr);

View more solutions

Comments

[ktilcu]

Can maps have an infinite length?

[shinzou]

How can a finite amount of memory hold an infinite data structure?

[hraban]

it doesn't, that's the point. Using this you can create "data streams" by chaining an iterator source to a bunch of iterator transforms and finally a consumer sink. E.g. for streaming audio processing, working with huge files, aggregators on databases, etc.

[hraban]

@ktilcu for an iterator: yes. a .map on an iterator can be thought of as a transform on the generator, which returns an iterator itself. popping one element calls the underlying iterator, transforms the element, and returns that.

[ktilcu]

I think the Array.from portion guarantees the finite set. I do understand how a generator could provide a lazy and infinite sequence but thats not part of this question.

[hraban]

The problem with this answer is it turns what could be an O(1) memory algorithm into an O(n) one, which is quite serious for larger datasets. Aside from, of course, requiring finite, non-streamable iterators. The title of the question is "Using map() on an iterator", I disagree that lazy and infinite sequences are not part of the question. That's precisely how people use iterators. The "map" was only an example ("Say.."). The good thing about this answer is its simplicity, which is very important.

[ktilcu]

@hraban Thanks for adding to this discussion. I can update the answer to include a few caveats just so future travelers have the info front and center. When it comes down to it we will often have to make the decision between simple and optimal performance. I will usually side with simpler (to debug, maintain, explain) over performance.

[ruX]

This is so typical for js, a lot of hand waiving to archive simple result

[Joel Malone]

I like this answer. Can anyone recommend a library that offers Array-like methods on iterables?

[sudo]

Generator/iterator support in JS is very lacking. In Python it's easy to deal with these things. I use Py to process massive data sets streamed to/from Postgres databases, and it's just as easy as using simple arrays. Different use cases, though. Usually in JS you aren't doing any long-running synchronous tasks like that. But it would be nice to be able to use those annoying iterator interfaces without converting to arrays.

[Daniel]

@ktilcu You can instead call Array.from(m, ([key,value]) => /* whatever */) (notice the mapping function is inside the from) and then no intermediate array is created (source). It still moves from O(1) to O(n), but at least iteration and mapping happen in just one full iteration.

[Jaka Jančar]

mapIterator() does not guarantee that underlying iterator will be properly closed (iterator.return() called) unless the return value's next was called at least once. See: repeater.js.org/docs/safety

[flying sheep]

Nice! This is how JS’s APIs should have been done. As always, Rust gets it right: doc.rust-lang.org/std/iter/trait.Iterator.html

[cowlicks]

Why are you manually using the iterator protocol instead of just a for .. of .. loop?

[sheean]

@cowlicks I have no idea! Maybe there were compatibility issues or I was confused believing it was a TypeScript-only construct. I've updated it now to use a for .. of .. loop instead as this is much clearer.

[chpio]

"As always, Rust gets it right" sure... There's a standardization proposal for all sort of helper functions for the iterator interface github.com/tc39/proposal-iterator-helpers You can use it today with corejs by importing the from fn from "core-js-pure/features/iterator" which returns the "new" iterator.

[bodo]

Is there some way to write this with fp-ts? Haven’t used it myself, but this seems like the thing it is meant for!?!

[Brian]

How would this look in typescript?