Lambda expression and method overloading doubts

Question

OK, so method overloading is-a-bad-thing™. Now that this has been settled, let's assume I actually want to overload a method like this:

static void run(Consumer<Integer> consumer) {
    System.out.println("consumer");
}

static void run(Function<Integer, Integer> function) {
    System.out.println("function");
}

In Java 7, I could call them easily with non-ambiguous anonymous classes as arguments:

run(new Consumer<Integer>() {
    public void accept(Integer integer) {}
});

run(new Function<Integer, Integer>() {
    public Integer apply(Integer o) { return 1; }
});

Now in Java 8, I'd like to call those methods with lambda expressions of course, and I can!

// Consumer
run((Integer i) -> {});

// Function
run((Integer i) -> 1);

Since the compiler should be able to infer Integer, why don't I leave Integer away, then?

// Consumer
run(i -> {});

// Function
run(i -> 1);

But this doesn't compile. The compiler (javac, jdk1.8.0_05) doesn't like that:

Test.java:63: error: reference to run is ambiguous
        run(i -> {});
        ^
  both method run(Consumer<Integer>) in Test and 
       method run(Function<Integer,Integer>) in Test match

To me, intuitively, this doesn't make sense. There is absolutely no ambiguity between a lambda expression that yields a return value ("value-compatible") and a lambda expression that yields void ("void-compatible"), as set out in the JLS §15.27.

But of course, the JLS is deep and complex and we inherit 20 years of backwards compatibility history, and there are new things like:

Certain argument expressions that contain implicitly typed lambda expressions (§15.27.1) or inexact method references (§15.13.1) are ignored by the applicability tests, because their meaning cannot be determined until a target type is selected.

from JLS §15.12.2

The above limitation is probably related to the fact that JEP 101 wasn't implemented all the way, as can be seen here and here.

Question:

Who can tell me exactly what parts of the JLS specifies this compile-time ambiguity (or is it a compiler bug)?

Bonus: Why were things decided this way?

Update:

With jdk1.8.0_40, the above compiles and works fine

Solution 1

I think you found this bug in the compiler: JDK-8029718 (or this similar one in Eclipse: 434642).

Compare to JLS §15.12.2.1. Identify Potentially Applicable Methods:

…

• A lambda expression (§15.27) is potentially compatible with a functional interface type (§9.8) if all of the following are true:

  • The arity of the target type's function type is the same as the arity of the lambda expression.

  • If the target type's function type has a void return, then the lambda body is either a statement expression (§14.8) or a void-compatible block (§15.27.2).

  • If the target type's function type has a (non-void) return type, then the lambda body is either an expression or a value-compatible block (§15.27.2).

Note the clear distinction between “void compatible blocks” and “value-compatible blocks”. While a block might be both in certain cases, the section §15.27.2. Lambda Body clearly states that an expression like () -> {} is a “void compatible block”, as it completes normally without returning a value. And it should be obvious that i -> {} is a “void compatible block” too.

And according to the section cited above, the combination of a lambda with a block that is not value-compatible and target type with a (non-void) return type is not a potential candidate for the method overload resolution. So your intuition is right, there should be no ambiguity here.

Examples for ambiguous blocks are

() -> { throw new RuntimeException(); }
() -> { while (true); }

as they don’t complete normally, but this is not the case in your question.

Solution 2

This bug has already been reported in the JDK Bug System: https://bugs.openjdk.java.net/browse/JDK-8029718. As you can check the bug has been fixed. This fix syncs javac with the spec in this aspect. Right now javac is correctly accepting the version with implicit lambdas. To get this update, you need to clone javac 8 repo.

What the fix does is to analyze the lambda body and determine if it's void or value compatible. To determine this you need to analyze all return statements. Let's remember that from the spec (15.27.2), already referenced above:

• A block lambda body is void-compatible if every return statement in the block has the form return.
• A block lambda body is value-compatible if it cannot complete normally (14.21) and every return statement in the block has the form return Expression.

This means that by analyzing the returns in the lambda body you can know if the lambda body is void compatible but to determine if it's value compatible you also need to do a flow analysis on it to determine that it can complete normally (14.21).

This fix also introduces a new compiler error for cases when the body is neither void nor value compatible, for example if we compile this code:

class Test {
    interface I {
        String f(String x);
    }

    static void foo(I i) {}

    void m() {
        foo((x) -> {
            if (x == null) {
                return;
            } else {
                return x;
            }
        });
    }
}

the compiler will give this output:

Test.java:9: error: lambda body is neither value nor void compatible
    foo((x) -> {
        ^
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
1 error

I hope this helps.

Comments

[Syam S]

Just to clarify. When you say run(i -> {}); that 'i' could be Consumer<Integer> or Function<Integer, Integer>. That ambiguity is obvious right?

[Lukas Eder]

@SyamS: i is the first (and only) argument to either Consumer.accept() or Function.apply(). This, per se, might be ambiguous. But given that one lambda evaluates to a "value-compatible" type (Function) and the other evaluates to a "void-compatible" type (Consumer), I'd intuitively think that there is no ambiguity

[Syam S]

Sorry if this sounds silly, but function overloading generally depends only on the input type. Its doesn't check for return types. So in this case accept and apply both take one argument of type Integer. So it looks ambiguous to me. :) Does lambda look for return type for inference?

[Holger]

It did work with earlier versions (e.g. beta 102 and earlier).

[jacobhyphenated]

@SyamS That does seem like an ambiguity, but the compiler can figure out run((Integer i) -> {}) is a Consumer. So although it could be both Function<Integer,Integer> or Consumer<Integer>, Consumer<Integer> is the best match and the compiler uses that. The question is, why does the compiler only do this when you specify (Integer i) and not just i.

[Lukas Eder]

@SyamS: i -> {} can never evaluate to Function, because it is "void-compatible". i -> 1 can never evaluate to Consumer, because it is "value-compatible". For each call, only one of the overloaded methods is even applicable in my opinion. As @jacobhyphenated also pointed out, the ambiguity can be resolved by explicitly specifying identical function argument types (Integer i).

[jacobhyphenated]

@LukasEder You are correct. If I try to specify run((Consumer<Integer>) (Integer i) -> {1}); it will not compile. That leads me to believe that this must be a compiler bug since there really is no ambiguity between the two lambdas.

[Lukas Eder]

@jacobhyphenated: I would wish so. I dearly want to overload two such methods in the jOOQ API, without creating a hassle for Java 8 users :-) But I think that your line of thought is not yet a formal proof for this being a bug. I suspect that this is really a JLS limitation.

[Syam S]

Thanks both of you. I've just started learning Java 8. Sometime back I saw a video by Brian Goetz, Lambda : A peek under the hood. He was talking about all these design consideration. I didn't watch it fully since I couldn't understand most of it. May be that will help you. You can find the video in youtube.com/watch?v=9JRDbjQRhRw

[Lukas Eder]

@SyamS: I somewhat doubt that Brian Goetz would've bothered boring the JAX's general audience with such language details, that might have been discussed somewhere in the depths of the lambda-dev mailing list :-)

[ggovan]

The right hand side of the lambda expression cannot be reasoned about with out type information. i->i.thing() could be void or a value, we don't know unless we know what i is. It seems like the compiler is unwilling to reason about this, despite both lambdas having Integer parameters.

[Lukas Eder]

@ggovan: I think that might be the right clue here. I might probably be confused by the fact that my concrete lambdas are non-ambiguous (i -> 1 and i -> {}), whereas lambdas in general could be ambiguous (i -> intFunction() and i -> voidFunction()). Now proove it, and I'll accept your answer :-)

[ggovan]

For a lambda i->i.thing() you cannot tell whether this is void or value without knowing the type of i. Could that be the problem rather than a compiler bug.

[Lukas Eder]

Thanks. I was there, too, but then I saw JLS §15.12.2.5, which deals only with "explicitly typed lambda expressions", not with "implicitly typed lambda expressions". It looks as though my code falls through all of §15.12.2.5 to reach " Otherwise, the method invocation is ambiguous, and a compile-time error occurs."

[Holger]

@ggovan: we are talking about i->{} which is clearly void type and i->1 which is clearly value type. Just as given in the question…

[Holger]

@Lukas Eder: §15.12.2.5 is about “Choosing the Most Specific Method” which applies when more than one method is applicable but due to the “void vs value type” there is only one method applicable thus that section does not apply to our situation (well, it should be irrelevant here…).

[Lukas Eder]

@Holger: I know, it shouldn't be applicable. But check again my blockquoted section: "Certain argument expressions that contain implicitly typed lambda expressions (§15.27.1) or inexact method references (§15.13.1) are ignored by the applicability tests, because their meaning cannot be determined until a target type is selected." Maybe, the applicability section is simply skipped for implicitly typed lambda expressions

[ZhongYu]

This is the correct answer. The bug has been fixed - bugs.openjdk.java.net/browse/JDK-8029718

[ZhongYu]

@LukasEder the lambda expression is used in 15.12.2.1, so that we got only one potentially applicable method (); it's ignored in (15.12.2.2), and the method is found to be applicable. since this is the only applicable method, (15.12.2.5) does not apply.

[Lukas Eder]

@zhong.j.yu: That sounds very reasonable, thanks. I will double-check this with an early access build of 8u20. Why did you delete your answer?

[ZhongYu]

@LukasEder I think Holger's answer is perfect.

[ZhongYu]

there's been hot debates on whether run(i->1) should compile. There's no ambiguity or difficulty here, since i is obviously Integer. unfortunately they decided not to support it at this time, but leaving the door open for future consideration (if enough people need this feature)

[Lukas Eder]

@zhong.j.yu: ... although, Dan Smith seemed to have disagreed that this particular case was a bug

[ZhongYu]

@LukasEder no I think his point(1) applies here, in which he agrees that it was a bug

[Lukas Eder]

@zhong.j.yu: But he clearly refers to 1) "The example in the description" and 2) "The Function vs. Consumer example is not a bug". The latter pretty much corresponds to my own example, I think?

[ZhongYu]

@LukasEder in the (2) example the lambda body is a statement expression, but in your question the lambda body is a block. the bug is concerned with a block body; see the original report that triggered the bug - mail.openjdk.java.net/pipermail/lambda-dev/2013-November/…

[Lukas Eder]

But both my example calls don't compile, regardless if we have a void-compatible or expression-compatible lambda... And I would say that the (2) example is also a void-compatible lambda, no? System.gc() returns void

[Lukas Eder]

@zhong.j.yu: I suspect that this debate is also the one you've cited here?

[ZhongYu]

@LukasEder it was a long and confusing discussion starting from mail.openjdk.java.net/pipermail/lambda-spec-observers/… - I don't recommend you read it:) nobody knew what each other were talking about.

[ZhongYu]

@LukasEder in example (2), without knowing the type of lambda parameter s, javac will not compile the lambda body, so the return type is unknown at that point. If the lambda is explicit - foo( (String s)->System.gc() ), it'll be fine. but, one may argue that s is obviously String anyway... hence the debate I mentioned in another comment.

[ZhongYu]

@LukasEder in your example, the lambda body is a block, javac at least knows whether it returns void, just by the structure of the code.

[Lukas Eder]

@Holger: I have added a link to the bug to your answer, as zhong's hypothesis has been confirmed on lambda-dev

[Stephan Herrmann]

Just to close a loop here: in Eclipse this bug was deliberately inserted just to mimic the behavior of javac. This has been reverted soon after the javac bug was fixed (see the references in the answer).