map/grep is not a real iterator?

I always thought map/grep(and many functions in List::MoreUtils) is the way to functional program in perl5. but a small test tell me it's wrong:

test1: perl -e" do{print if $_ %1000 == 0} for(1..100_000_000);" #run successfully

test2: perl -e" map{print if $_ %1000 ==0} 1..100_000_000" #out of memory

seems map function create a array to accommodate all data from list. As I know, an iterator is just an cursor to traverse a lists which can be infinite or finite. but the present implementation of map sets feed list a limit to a maximum array, and I think it's wrong.

Please point out if any mistakes.


Good question!

You might not want to use a map when you have a void
context for reasons of clarity. See this SO question.

Also, Perl 5 doesn't have lazy lists unless you use an iterator and there are
several options to achieve that listed at this SO question.

I'm sure others that know more about the Perl internals can add to this.

> test2: perl -e" map{print if $_ %1000 ==0} 1..100_000_000"

that example is not relevant. map has to _return_ 100M elements so it need memory.

that does not mean memory taken for 1..100_000_000 range

If I may self-promote, I wrote a python-esque generator module called Generator::Object, based on Coro, which may serve you if you really need lazy evaluation.

The input list is as much the killer as the output list is. You can test that using:

perl -e'map{print if $_ %1000==0; ()} 1..100_000_000'

which of course will return the empty list. The problem is that for map Perl will first build a list of the numbers 1 to 100 million as SVs, but for ($x..$y) though has been optimized in the Perl compiler.

map/grep are no iterators at all. They just read the whole list, apply the function to every element and return a new list.

Perl itself doesn't have special iterator support. But you still can write something as an iterator, but you have to do it on your own.

For example you create a function that returns a function that uses an closure. Or you just use OOP for keep tracking of the current state and so on.

Leave a comment

About xiaoyafeng

user-pic I blog about Perl.