Enumerables
An enumerable is any data structure that can be traversed or iterated over. It
can be lists like [1, 2, 3], maps like %{a: 1, b: 2} and ranges like 1..3. All
these structures can be processed using the functions provided by the Enum
module.
Introduction
The Enum module is your Swiss-army knife for working with collections.
Anything a for loop would do in another language (transform, filter,
group, sort, add up, pick an element) you do in Elixir with a function
from Enum.
The big idea is always the same: Enum takes a collection and a small
function that describes what to do with each element, and gives you back
a new collection or a single value.
Consider this example of using Enum to multiply each element in a list
by 2:
list = [1, 2, 3, 4]
Enum.map(list, fn x -> x * 2 end)
# => [2, 4, 6, 8]The Enum.map function takes two arguments: the enumerable (list in this case) and a transformation function for each element.
For further enhancement, Elixir’s pipe operator (|>) can be used with Enum functions for cleaner and more readable code. Here’s an example:
list = [1, 2, 3, 4]
list
|> Enum.map(fn x -> x * 3 end)
|> Enum.filter(fn x -> rem(x, 2) == 0 end)
# => [6, 12]This statement takes a list, multiplies each element by 3 using Enum.map, and then filters out the odd numbers using Enum.filter. The use of the pipe operator makes the code flow naturally and easier to read.
|
You can also use the |
Enum functions are eager; they execute immediately and return a result. If memory usage is a concern with very large collections, consider using the Stream module for lazy computation.
|
Commonly Used Enum Functions
Enum offers a ton of useful functions. All are listed at the official Enum documentation. Here are some of the most commonly used functions to give you an idea of what’s available.
Enum.map/2
The Enum.map/2 function is used to transform each element in an enumerable using a provided function.
list = [1, 2, 3, 4]
Enum.map(list, fn x -> x * 2 end)
# => [2, 4, 6, 8]The &1 shorthand can be used as follows:
list = [1, 2, 3, 4]
list |> Enum.map(&(&1 * 2))
# => [2, 4, 6, 8]More details can be found at the official Elixir Enum.map/2 documentation.
Enum.filter/2
The Enum.filter/2 function filters out elements based on a provided function.
list = [1, nil, 2, nil, 3]
Enum.filter(list, fn x -> x != nil end)
# => [1, 2, 3]Using the &1 shorthand:
list = [1, nil, 2, nil, 3]
list |> Enum.filter(&(&1 != nil))
# => [1, 2, 3]More details can be found at the official Elixir Enum.filter/2 documentation.
Enum.reduce/2,3
The Enum.reduce/2,3 function reduces an enumerable to a single value.
list = [1, 2, 3, 4]
Enum.reduce(list, 0, fn x, acc -> x + acc end)
# => 10
The use of reduce/3 and it’s accumulator is similar to the fold function in other languages. It can be tricky to use.
|
More details can be found at the official Elixir Enum.reduce/2 documentation.
Enum.sort/1,2
The Enum.sort/1,2 function sorts the elements in an enumerable.
list = [4, 2, 3, 1]
Enum.sort(list)
# => [1, 2, 3, 4]You can provide a comparator function:
list = [4, 2, 3, 1]
Enum.sort(list, fn a, b -> a > b end)
# => [4, 3, 2, 1]More details can be found at the official Elixir Enum.sort/2 documentation.
Enum.at/2,3
Returns the element at the given index (zero based) or a default value.
list = [1, 2, 3, 4]
Enum.at(list, 2)
# Output: 3More details can be found at the official Elixir Enum.at/2,3 documentation.
Enum.concat/1,2
Concatenates the collection of enumerable(s) given.
Enum.concat([[1, 2], [3, 4]])
# Output: [1, 2, 3, 4]More details can be found at the official Elixir Enum.concat/1,2 documentation.
Enum.count/1,2
Counts the enumerable items, optionally, using the provided function.
list = [1, 2, 3, 4]
Enum.count(list)
# Output: 4More details can be found at the official Elixir Enum.count/1,2 documentation.
Enum.find/2,3
Finds the first element for which the provided function returns a truthy value.
list = [1, 2, 3, 4]
Enum.find(list, fn x -> x > 2 end)
# Output: 3More details can be found at the official Elixir Enum.find/2,3 documentation.
Enum.group_by/2,3
Groups all items in the enumerable by the given function.
list = [{:apple, "fruit"}, {:carrot, "vegetable"}, {:banana, "fruit"}]
Enum.group_by(list, fn {_name, type} -> type end)
# %{
# "fruit" => [apple: "fruit", banana: "fruit"],
# "vegetable" => [carrot: "vegetable"]
# }
IEx prints {:apple, "fruit"} as apple: "fruit" when every
element in the list is a two-tuple starting with an atom, that is just
the keyword list syntax. The data is still a list of tuples, the
display is nicer to read.
|
More details can be found at the official Elixir Enum.group_by/2,3 documentation.
Enum.join/1,2
Joins all the items in the enumerable into a single string.
list = ["Hello", "World"]
Enum.join(list, " ")
# Output: "Hello World"More details can be found at the official Elixir Enum.join/1,2 documentation.
Enum.max/1
Returns the maximum value in the enumerable.
list = [1, 2, 3, 4]
Enum.max(list)
# Output: 4More details can be found at the official Elixir Enum.max/1 documentation.
Enum.min/1
Returns the minimum value in the enumerable.
list = [1, 2, 3, 4]
Enum.min(list)
# Output: 1More details can be found at the official Elixir Enum.min/1 documentation.
Enum.random/1
Selects a random element from the enumerable.
list = [1, 2, 3, 4]
Enum.random(list)
# Output: Random value from the listMore details can be found at the official Elixir Enum.random/1 documentation.
Enum.reject/2
Filters out the items in the enumerable for which the provided function returns a truthy value.
list = [1, 2, 3, 4]
Enum.reject(list, fn x -> x < 3 end)
# Output: [3, 4]More details can be found at the official Elixir Enum.reject/2 documentation.
Enum.sum/1
Returns the sum of all items in the enumerable.
list = [1, 2, 3, 4]
Enum.sum(list)
# Output: 10More details can be found at the official Elixir Enum.sum/1 documentation.
Introduction
Stream is the lazy counterpart to Enum. The functions look almost
the same (map, filter, take, …) but they do not do the work
right away. Instead, they build up a recipe that runs only when you
finally ask for a concrete result, usually by piping into Enum.to_list/1
or any other Enum function.
When does that matter? Two common cases:
-
Huge collections where computing every intermediate step would eat too much memory.
-
Infinite sequences (file lines, clock ticks, generated IDs), where you only want the first few items and stopping early is essential.
Consider this example of using Stream to multiply each element in a
list by 2:
list = [1, 2, 3, 4]
Stream.map(list, fn x -> x * 2 end)
# => #Stream<[enum: [1, 2, 3, 4], funs: [#Function<49.101170868/1 in Stream.map/2>]]>You might notice that the output is not a list but a Stream. To retrieve the final result, you will need to convert the stream back into a list:
list = [1, 2, 3, 4]
list
|> Stream.map(fn x -> x * 2 end)
|> Enum.to_list()
# => [2, 4, 6, 8]The Stream module can be used with Elixir’s pipe operator (|>) to create a sequence of transformations. The transformations only get executed when the stream is converted into a list or another enumerable.
This is how you would use Stream to multiply each element by 3 and then filter out odd numbers:
list = [1, 2, 3, 4]
list
|> Stream.map(fn x -> x * 3 end)
|> Stream.filter(fn x -> rem(x, 2) == 0 end)
|> Enum.to_list()
# => [6, 12]Commonly Used Stream Functions
Stream offers a lot of useful functions, similar to Enum. All are listed at the official Stream documentation. Here are some of the most commonly used functions to give you an idea of what’s available.
Stream.map/2
The Stream.map/2 function is used to transform each element in an enumerable using a provided function. The result is a new Stream that can be evaluated later.
list = [1, 2, 3, 4]
list
|> Stream.map(fn x -> x * 2 end)
|> Enum.to_list()
# => [2, 4, 6, 8]The &1 shorthand can be used as follows:
list = [1, 2, 3, 4]
list
|> Stream.map(&(&1 * 2))
|> Enum.to_list()
# => [2, 4, 6, 8]More details can be found at the official Elixir Stream.map/2 documentation.
Stream.filter/2
The Stream.filter/2 function filters out elements based on a provided function, resulting in a new Stream.
list = [1, nil, 2, nil, 3]
list
|> Stream.filter(fn x -> x != nil end)
|> Enum.to_list()
# => [1, 2, 3]Using the &1 shorthand:
list = [1, nil, 2, nil, 3]
list
|> Stream.filter(&(&1 != nil))
|> Enum.to_list()
# => [1, 2, 3]More details can be found at the official Elixir Stream.filter/2 documentation.
Reducing a stream with Enum.reduce/3
Stream itself does not have a reduce function, because reducing
always walks the entire collection and therefore cannot stay lazy.
Instead, you build up a lazy pipeline with Stream and finish it off
with Enum.reduce/3 (or Enum.to_list/1, Enum.sum/1, etc.):
list = [1, 2, 3, 4]
list
|> Stream.map(&(&1 * 2))
|> Enum.reduce(0, fn x, acc -> x + acc end)
# => 20
Enum.reduce/3 is Elixir’s version of what other languages call
fold. The accumulator can be a number, a list, a map, anything you
like. It is the most general Enum function, every other Enum function
could be written in terms of it.
|
Stream.take/2
The Stream.take/2 function generates a new stream that takes the first n items from the original stream.
list = [1, 2, 3, 4]
list
|> Stream.take(2)
|> Enum.to_list()
# => [1, 2]More details can be found at the official Elixir Stream.take/2 documentation.
Stream.drop/2
The Stream.drop/2 function generates a new stream that drops the first n items from the original stream.
list = [1, 2, 3, 4]
list
|> Stream.drop(2)
|> Enum.to_list()
# => [3, 4]More details can be found at the official Elixir Stream.drop/2 documentation.
Stream.concat/1,2
The Stream.concat/1,2 function generates a new stream that concatenates two streams or a stream of streams.
Stream.concat([1, 2], [3, 4])
|> Enum.to_list()
# => [1, 2, 3, 4]More details can be found at the official Elixir Stream.concat/1,2 documentation.
Stream.cycle/1
The Stream.cycle/1 function generates an infinite stream repeating the given enumerable.
Stream.cycle([1, 2])
|> Stream.take(5)
|> Enum.to_list()
# => [1, 2, 1, 2, 1]More details can be found at the official Elixir Stream.cycle/1 documentation.
Enum vs Stream
Think about Enum and Stream as two different chefs in a kitchen who are asked to prepare a large meal.
The Enum Chef (Eager Chef):
The Enum chef is eager to get the job done. He tries to cook everything at once. He gets every ingredient, every pot and pan, and starts cooking immediately. This is great if you’re not cooking a lot of food, because everything gets done fast.
But what if the meal or the number of meals is huge? Well, then the Enum chef might run into trouble. His kitchen (or our computer’s memory) might not have enough room for all the food he’s trying to cook at once. He might get overwhelmed because he’s trying to do too much at once.
The Stream Chef (Lazy Chef):
The Stream chef, on the other hand, is more laid-back. He doesn’t start cooking until it’s absolutely necessary. He prepares each dish one at a time, using only the ingredients and cookware needed for each dish. Once a dish is ready, he moves on to the next one.
If the meal is huge, the Stream chef handles it better because he only works on one dish at a time, which means he doesn’t need a lot of room in his kitchen. He’s more efficient with large meals because he can handle them piece by piece.
Comparing the Chefs:
-
Speed: The Enum chef (Eager chef) works faster when the meal is small because he cooks everything at once. But the Stream chef (Lazy chef) could be faster for large meals because he efficiently handles them one dish at a time. You could use a stopwatch to see who finishes cooking first.
-
Kitchen Space (Memory): The Stream chef (Lazy chef) uses his kitchen space more efficiently because he only prepares one dish at a time. This difference becomes obvious when they’re asked to prepare a large meal. You could look at how messy their kitchens are to see who uses space better.
So, when you’re choosing between Enum and Stream in your Elixir code, think about the size of your "meal" (your data), and pick the chef that suits your needs best.