Introduction
Stream is the lazy counterpart to Enum. The functions look almost
the same (map, filter, take, …) but they do not do the work
right away. Instead, they build up a recipe that runs only when you
finally ask for a concrete result, usually by piping into Enum.to_list/1
or any other Enum function.
When does that matter? Two common cases:
-
Huge collections where computing every intermediate step would eat too much memory.
-
Infinite sequences (file lines, clock ticks, generated IDs), where you only want the first few items and stopping early is essential.
Consider this example of using Stream to multiply each element in a
list by 2:
list = [1, 2, 3, 4]
Stream.map(list, fn x -> x * 2 end)
# => #Stream<[enum: [1, 2, 3, 4], funs: [#Function<49.101170868/1 in Stream.map/2>]]>You might notice that the output is not a list but a Stream. To retrieve the final result, you will need to convert the stream back into a list:
list = [1, 2, 3, 4]
list
|> Stream.map(fn x -> x * 2 end)
|> Enum.to_list()
# => [2, 4, 6, 8]The Stream module can be used with Elixir’s pipe operator (|>) to create a sequence of transformations. The transformations only get executed when the stream is converted into a list or another enumerable.
This is how you would use Stream to multiply each element by 3 and then filter out odd numbers:
list = [1, 2, 3, 4]
list
|> Stream.map(fn x -> x * 3 end)
|> Stream.filter(fn x -> rem(x, 2) == 0 end)
|> Enum.to_list()
# => [6, 12]Commonly Used Stream Functions
Stream offers a lot of useful functions, similar to Enum. All are listed at the official Stream documentation. Here are some of the most commonly used functions to give you an idea of what’s available.
Stream.map/2
The Stream.map/2 function is used to transform each element in an enumerable using a provided function. The result is a new Stream that can be evaluated later.
list = [1, 2, 3, 4]
list
|> Stream.map(fn x -> x * 2 end)
|> Enum.to_list()
# => [2, 4, 6, 8]The &1 shorthand can be used as follows:
list = [1, 2, 3, 4]
list
|> Stream.map(&(&1 * 2))
|> Enum.to_list()
# => [2, 4, 6, 8]More details can be found at the official Elixir Stream.map/2 documentation.
Stream.filter/2
The Stream.filter/2 function filters out elements based on a provided function, resulting in a new Stream.
list = [1, nil, 2, nil, 3]
list
|> Stream.filter(fn x -> x != nil end)
|> Enum.to_list()
# => [1, 2, 3]Using the &1 shorthand:
list = [1, nil, 2, nil, 3]
list
|> Stream.filter(&(&1 != nil))
|> Enum.to_list()
# => [1, 2, 3]More details can be found at the official Elixir Stream.filter/2 documentation.
Reducing a stream with Enum.reduce/3
Stream itself does not have a reduce function, because reducing
always walks the entire collection and therefore cannot stay lazy.
Instead, you build up a lazy pipeline with Stream and finish it off
with Enum.reduce/3 (or Enum.to_list/1, Enum.sum/1, etc.):
list = [1, 2, 3, 4]
list
|> Stream.map(&(&1 * 2))
|> Enum.reduce(0, fn x, acc -> x + acc end)
# => 20
Enum.reduce/3 is Elixir’s version of what other languages call
fold. The accumulator can be a number, a list, a map, anything you
like. It is the most general Enum function, every other Enum function
could be written in terms of it.
|
Stream.take/2
The Stream.take/2 function generates a new stream that takes the first n items from the original stream.
list = [1, 2, 3, 4]
list
|> Stream.take(2)
|> Enum.to_list()
# => [1, 2]More details can be found at the official Elixir Stream.take/2 documentation.
Stream.drop/2
The Stream.drop/2 function generates a new stream that drops the first n items from the original stream.
list = [1, 2, 3, 4]
list
|> Stream.drop(2)
|> Enum.to_list()
# => [3, 4]More details can be found at the official Elixir Stream.drop/2 documentation.
Stream.concat/1,2
The Stream.concat/1,2 function generates a new stream that concatenates two streams or a stream of streams.
Stream.concat([1, 2], [3, 4])
|> Enum.to_list()
# => [1, 2, 3, 4]More details can be found at the official Elixir Stream.concat/1,2 documentation.
Stream.cycle/1
The Stream.cycle/1 function generates an infinite stream repeating the given enumerable.
Stream.cycle([1, 2])
|> Stream.take(5)
|> Enum.to_list()
# => [1, 2, 1, 2, 1]More details can be found at the official Elixir Stream.cycle/1 documentation.