Why are there two kinds of functions in Elixir?

Question

I'm learning Elixir and wonder why it has two types of function definitions:

• functions defined in a module with def, called using myfunction(param1, param2)
• anonymous functions defined with fn, called using myfn.(param1, param2)

Only the second kind of function seems to be a first-class object and can be passed as a parameter to other functions. A function defined in a module needs to be wrapped in a fn. There's some syntactic sugar which looks like otherfunction(&myfunction(&1, &2)) in order to make that easy, but why is it necessary in the first place? Why can't we just do otherfunction(myfunction))? Is it only to allow calling module functions without parenthesis like in Ruby? It seems to have inherited this characteristic from Erlang which also has module functions and funs, so does it actually comes from how the Erlang VM works internally?

It there any benefit having two types of functions and converting from one type to another in order to pass them to other functions? Is there a benefit having two different notations to call functions?

Solution 1

Just to clarify the naming, they are both functions. One is a named function and the other is an anonymous one. But you are right, they work somewhat differently and I am going to illustrate why they work like that.

Let's start with the second, fn. fn is a closure, similar to a lambda in Ruby. We can create it as follows:

x = 1
fun = fn y -> x + y end
fun.(2) #=> 3

A function can have multiple clauses too:

x = 1
fun = fn
  y when y < 0 -> x - y
  y -> x + y
end
fun.(2) #=> 3
fun.(-2) #=> 3

Now, let's try something different. Let's try to define different clauses expecting a different number of arguments:

fn
  x, y -> x + y
  x -> x
end
** (SyntaxError) cannot mix clauses with different arities in function definition

Oh no! We get an error! We cannot mix clauses that expect a different number of arguments. A function always has a fixed arity.

Now, let's talk about the named functions:

def hello(x, y) do
  x + y
end

As expected, they have a name and they can also receive some arguments. However, they are not closures:

x = 1
def hello(y) do
  x + y
end

This code will fail to compile because every time you see a def, you get an empty variable scope. That is an important difference between them. I particularly like the fact that each named function starts with a clean slate and you don't get the variables of different scopes all mixed up together. You have a clear boundary.

We could retrieve the named hello function above as an anonymous function. You mentioned it yourself:

other_function(&hello(&1))

And then you asked, why I cannot simply pass it as hello as in other languages? That's because functions in Elixir are identified by name and arity. So a function that expects two arguments is a different function than one that expects three, even if they had the same name. So if we simply passed hello, we would have no idea which hello you actually meant. The one with two, three or four arguments? This is exactly the same reason why we can't create an anonymous function with clauses with different arities.

Since Elixir v0.10.1, we have a syntax to capture named functions:

&hello/1

That will capture the local named function hello with arity 1. Throughout the language and its documentation, it is very common to identify functions in this hello/1 syntax.

This is also why Elixir uses a dot for calling anonymous functions. Since you can't simply pass hello around as a function, instead you need to explicitly capture it, there is a natural distinction between named and anonymous functions and a distinct syntax for calling each makes everything a bit more explicit (Lispers would be familiar with this due to the Lisp 1 vs. Lisp 2 discussion).

Overall, those are the reasons why we have two functions and why they behave differently.

Solution 2

I don't know how useful this will be to anyone else, but the way I finally wrapped my head around the concept was to realize that elixir functions aren't Functions.

Everything in elixir is an expression. So

MyModule.my_function(foo) 

is not a function but the expression returned by executing the code in my_function. There is actually only one way to get a "Function" that you can pass around as an argument and that is to use the anonymous function notation.

It is tempting to refer to the fn or & notation as a function pointer, but it is actually much more. It's a closure of the surrounding environment.

If you ask yourself:

Do I need an execution environment or a data value in this spot?

And if you need execution use fn, then most of the difficulties become much clearer.

Solution 3

I may be wrong since nobody mentioned it, but I was also under the impression that the reason for this is also the ruby heritage of being able to call functions without brackets.

Arity is obviously involved but lets put it aside for a while and use functions without arguments. In a language like javascript where brackets are mandatory, it is easy to make the difference between passing a function as an argument and calling the function. You call it only when you use the brackets.

my_function // argument
(function() {}) // argument

my_function() // function is called
(function() {})() // function is called

As you can see, naming it or not does not make a big difference. But elixir and ruby allow you to call functions without the brackets. This is a design choice which I personally like but it has this side effect you cannot use just the name without the brackets because it could mean you want to call the function. This is what the & is for. If you leave arity appart for a second, prepending your function name with & means that you explicitly want to use this function as an argument, not what this function returns.

Now the anonymous function is bit different in that it is mainly used as an argument. Again this is a design choice but the rational behind it is that it is mainly used by iterators kind of functions which take functions as arguments. So obviously you don't need to use & because they are already considered arguments by default. It is their purpose.

Now the last problem is that sometimes you have to call them in your code, because they are not always used with an iterator kind of function, or you might be coding an iterator yourself. For the little story, since ruby is object oriented, the main way to do it was to use the call method on the object. That way, you could keep the non-mandatory brackets behaviour consistent.

my_lambda.call
my_lambda.call()
my_lambda_with_arguments.call :h2g2, 42
my_lambda_with_arguments.call(:h2g2, 42)

Now somebody came up with a shortcut which basically looks like a method with no name.

my_lambda.()
my_lambda_with_arguments.(:h2g2, 42)

Again, this is a design choice. Now elixir is not object oriented and therefore call not use the first form for sure. I can't speak for José but it looks like the second form was used in elixir because it still looks like a function call with an extra character. It's close enough to a function call.

I did not think about all the pros and cons, but it looks like in both languages you could get away with just the brackets as long as you make brackets mandatory for anonymous functions. It seems like it is:

Mandatory brackets VS Slightly different notation

In both cases you make an exception because you make both behave differently. Since there is a difference, you might as well make it obvious and go for the different notation. The mandatory brackets would look natural in most cases but very confusing when things don't go as planned.

Here you go. Now this might not be the best explanation in the world because I simplified most of the details. Also most of it are design choices and I tried to give a reason for them without judging them. I love elixir, I love ruby, I like the function calls without brackets, but like you, I find the consequences quite misguiding once in a while.

And in elixir, it is just this extra dot, whereas in ruby you have blocks on top of this. Blocks are amazing and I am surprised how much you can do with just blocks, but they only work when you need just one anonymous function which is the last argument. Then since you should be able to deal with other scenarios, here comes the whole method/lambda/proc/block confusion.

Anyway... this is out of scope.

Solution 4

I've never understood why explanations of this are so complicated.

It's really just an exceptionally small distinction combined with the realities of Ruby-style "function execution without parens".

Compare:

def fun1(x, y) do
  x + y
end

To:

fun2 = fn
  x, y -> x + y
end

While both of these are just identifiers...

• fun1 is an identifier that describes a named function defined with def.
• fun2 is an identifier that describes a variable (that happens to contain a reference to function).

Consider what that means when you see fun1 or fun2 in some other expression? When evaluating that expression, do you call the referenced function or do you just reference a value out of memory?

There's no good way to know at compile time. Ruby has the luxury of introspecting the variable namespace to find out if a variable binding has shadowed a function at some point in time. Elixir, being compiled, can't really do this. That's what the dot-notation does, it tells Elixir that it should contain a function reference and that it should be called.

And this is really hard. Imagine that there wasn't a dot notation. Consider this code:

val = 5

if :rand.uniform < 0.5 do
  val = fn -> 5 end
end

IO.puts val     # Does this work?
IO.puts val.()  # Or maybe this?

Given the above code, I think it's pretty clear why you have to give Elixir the hint. Imagine if every variable de-reference had to check for a function? Alternatively, imagine what heroics would be necessary to always infer that variable dereference was using a function?

Solution 5

There's an excellent blog post about this behavior: link

Two types of functions

If a module contains this:

fac(0) when N > 0 -> 1;
fac(N)            -> N* fac(N-1).

You can’t just cut and paste this into the shell and get the same result.

It’s because there is a bug in Erlang. Modules in Erlang are sequences of FORMS. The Erlang shell evaluates a sequence of EXPRESSIONS. In Erlang FORMS are not EXPRESSIONS.

double(X) -> 2*X.            in an Erlang module is a FORM

Double = fun(X) -> 2*X end.  in the shell is an EXPRESSION

The two are not the same. This bit of silliness has been Erlang forever but we didn’t notice it and we learned to live with it.

Dot in calling fn

iex> f = fn(x) -> 2 * x end
#Function<erl_eval.6.17052888>
iex> f.(10)
20

In school I learned to call functions by writing f(10) not f.(10) - this is “really” a function with a name like Shell.f(10) (it’s a function defined in the shell) The shell part is implicit so it should just be called f(10).

If you leave it like this expect to spend the next twenty years of your life explaining why.

View more solutions

Comments

[Cyclone]

I am also learning Elixir, and this is the first issue I encountered that gave me pause, something about it just seemed inconsistent. Great explanation, but to be clear… is this the result of an implementation issue, or does it reflect deeper wisdom regarding the use and passing of functions? Since anonymous functions can match based on argument values, it seems like it would be useful to be able to also match on the number of arguments (and consistent with function pattern matching elsewhere).

[José Valim]

It is not an implementation constraint in the sense that it could also work as f() (without the dot).

[José Valim]

You can match on the number of arguments by using is_function/2 in a guard. is_function(f, 2) checks it has arity of 2. :)

[Cyclone]

@JoséValim I found the related discussion on the mailing list, have you come to any conclusion? Personally I like the idea of just forcing () on 0-arity calls and remove the .. If named functions were forced to do the same too make things more consistent I'd be OK with that too. Or how about alternate syntax for parameterless invocation? LiveScript uses a bang: function!

[jsvisa]

@JoséValim You said def get a new variable scope, actually fn also get a new scope. x = 2 fun = fn y when y < 0 -> x = 4 end fun.(-2) IO.puts x #=> 2 it doesn't change the x value. In my opinion, in fn we defined a new varible x, so the outer x is isolated.

[José Valim]

Right. def gets a new empty scope. I will amend that. Thanks!

[CMCDragonkai]

I would have preferred function invocations without dots for both named and anonymous function. It makes it confusing sometimes and you forget whether a particular function was anonymous or named. There's also more noise when it comes to currying.

[lud]

I do not understand because Erlang allows to capture a function/arity in a variable. But it still doesn't need a dot

[José Valim]

In Erlang, anonymous function calls and regular functions are already syntactically different: SomeFun() and some_fun(). In Elixir, if we removed the dot, they would be the same some_fun() and some_fun() because variables use the same identifier as function names. Hence the dot.

[lud]

@JoséValim Right, i'm so blind. Thanks for your anwser !

[RubenLaguna]

It's clear that MyModule.my_function(foo) is an expression but MyModule.my_function "could" have been an expression that returns a function "object". But since you need to tell the arity, you would need something like MyModule.my_function/1 instead. And I guess that they decided it was better to use a &MyModule.my_function(&1) syntax instead that allows to express the arity (and serves other purposes too). Still unclear why there is a () operator for named functions and a .() operator for unnamed functions

[Admin]

I'm not sure of the usefulness in directly answering the OP question, but the link provided (including the comments section) is actually a great read IMO for the target audience of the question, i.e those of us new to & learning Elixir+Erlang.

[mljrg]

@JoséValim My interpretation of this "dot issue" is because in Elixir named functions and anonymous functions both use lowercase names, whereas in Erlang the first are lowercase, and the second begin with an uppercase letter (i.e., they are stored in a variable). So in Elixir something is needed to distinguish them, and for that, the dot was chosen. I believe this was a syntatic hazard in Elixir.

[jnmandal]

I think you want: &MyModule.my_function/1 thats how you can pass it around as a function.

[Nawaz]

"This is also why Elixir uses a dot for calling anonymous functions." The answer, in the present form, does not really explain why Elixir needs a dot when calling a function. I understand the limitation of anonymous functions in comparison of named functions, from arity point of view. I also understand why Elixir have the syntax hello/1 when passing a function. But I don't see how these concepts explain the dot when calling a function. Also, if a function f takes a function g (anonymous or otherwise) as argument, how would f invoke g? Would it use dot or not?

[Nawaz]

@mljrg: "So in Elixir something is needed to distinguish them, and for that, the dot was chosen"... Why does Elixir need to distinguish them (from invocation point of view)? Why does it even matter whether it was created with an anonymous function or named function?

[mljrg]

@Nawaz I believe it is related with parsing. The dot helps the compiler distinguish between a function and a variable holding a reference to a function. In Erlang you do not have this need because the letter case permits to distinguish between lowercase names related to declared functions, and capitalized variables holding references to functions. This need to distinguish both situations is not unique to Elixir and Erlang; for example, Common Lisp requires to use FUNCALL to call anonymous functions. Other languages like Scheme and Haskell do not make this distinction.

[Nawaz]

@mljrg: That is not a genuine or inherent problem. A variable could be made to behave like a function if the designers of languages want to. Also, as I asked before, if a function f takes a function g (anonymous or otherwise) as argument, how would f invoke g? Would it use dot or not?

[mljrg]

@Nawaz. There must exist some difficulty or firm design decision for doing so in Elixir. Anyway, I have come to accept syntax "as is" in most languages instead of complaining too much; in fact, I came to appreciate Elixir syntax "as is". You really have to start using a language for a while to forget about its "syntax nuances". In case of function f, if it gets a function in an argument g, you must call it as g.(...).

[AnilRedshift]

The link is dead now :(