Elixir - Looping through and adding to map

25,129

This is a nice challenge to have and solving it will definitely give you some insight into functional programming.

The solution for such problems in functional languages is usually reduce (often called fold). I will start with a short answer (and not a direct translation) but feel free to ask for a follow up.

The approach below will typically not work in functional programming languages:

map = %{}
Enum.each [1, 2, 3], fn x ->
  Map.put(map, x, x)
end
map

The map at the end will still be empty because we can't mutate data structures. Every time you call Map.put(map, x, x), it will return a new map. So we need to explicitly retrieve the new map after each enumeration.

We can achieve this in Elixir using reduce:

map = Enum.reduce [1, 2, 3], %{}, fn x, acc ->
  Map.put(acc, x, x)
end

Reduce will emit the result of the previous function as accumulator for the next item. After running the code above, the variable map will be %{1 => 1, 2 => 2, 3 => 3}.

For those reasons, we rarely use each on enumeration. Instead, we use the functions in the Enum module, that support a wide range of operations, eventually falling back to reduce when there is no other option.

EDIT: to answer the questions and go through a more direct translation of the code, this what you can do to check and update the map as you go:

Enum.reduce blogs, %{}, fn blog, history ->
  posts = get_posts(blog)
  Enum.reduce posts, history, fn post, history ->
    if Map.has_key?(history, post.url) do
      # Return the history unchanged
      history
    else
      do_thing(post)
      Map.put(history, post.url, true)
    end
  end
end

In fact, a set would be better here, so let's refactor this and use a set in the process:

def traverse_blogs(blogs) do
  Enum.reduce blogs, HashSet.new, &traverse_blog/2
end

def traverse_blog(blog, history) do
  Enum.reduce get_posts(blog), history, &traverse_post/2
end

def traverse_post(post, history) do
  if post.url in history do
    # Return the history unchanged
    history
  else
    do_thing(post)
    HashSet.put(history, post.url)
  end
end
Share:
25,129
William Dunne
Author by

William Dunne

Updated on July 09, 2022

Comments

  • William Dunne
    William Dunne over 1 year

    I'm rebuilding something in Elixir from some code I built in C#.

    It was pretty hacked together, but works perfectly (although not on Linux, hence rebuild).

    Essentially what it did was check some RSS feeds and see if there was any new content. This is the code:

    Map historic (URL as key, post title as value).
    List<string> blogfeeds
    while true
    for each blog in blogfeeds
       List<RssPost> posts = getposts(blog)
       for each post in posts
            if post.url is not in historic
               dothing(post)
               historic.add(post)
    

    I am wondering how I can do Enumeration effectively in Elixir. Also, it seems that my very process of adding things to "historic" is anti-functional programming.

    Obviously the first step was declaring my list of URLs, but beyond that the enumeration idea is messing with my head. Could someone help me out? Thanks.

  • William Dunne
    William Dunne over 8 years
    Okay awesome, thanks that helps. However, I still don't know how I would handle the checking if an item is new (not in the map) with this approach? Its something happening fairly often and I would prefer to not use something like a DB, although I would if I had to. It makes a lot of sense though, thank you.
  • GavinBrelstaff
    GavinBrelstaff over 8 years
    How about combining two functions from the Enum module: member? and filter. That way you could compile the list of all items not a member of the existing list then do something.
  • William Dunne
    William Dunne over 8 years
    How would you add to the filter? Whenever it finds a unique post it adds it to the list (no longer unique) and does a function with it. Is it possible for a function to call itself?
  • José Valim
    José Valim over 8 years
    I have edited the question for a more complete answer!
  • W.M.
    W.M. about 7 years
    Thank you very much @José Valim