When a Method Reference Reuses a Stream

Java method references are often used as a more concise alternative to lambda expressions. In many cases, the two forms are interchangeable and choosing between them is simply a matter of readability.

However, there are situations where replacing a lambda expression with a method reference changes the behaviour of the program. One such case involves Java Streams and the Iterable interface.

Making a Stream Iterable

Java Streams provide an iterator() method but do not implement the Iterable interface. This means they cannot be used directly in an enhanced for loop.

Since Iterable is a functional interface, we can adapt a Stream using a lambda expression:

Iterable<String> values = () -> createStream().iterator();

Alternatively, we can use a method reference:

Iterable<String> values = createStream()::iterator;

Both expressions compile and produce an Iterable<String> that can be used in an enhanced for loop:

for (String value : values) {
System.out.println(value);
}

Despite their similarity, the two implementations behave differently when iterated more than once.

Comparing the Two Approaches

To demonstrate the difference, we use a method that creates a Stream and prints a message whenever it is called:

static Stream<String> createStream() {
System.out.println(" Creating a new stream.");
return Stream.of("Example value");
}

We also need a method that iterates over an Iterable twice:

static void iterateTwice(final Iterable<String> values) {
System.out.println(" First iteration:");
for (final var value : values) {
System.out.println(" " + value);
}
System.out.println(" Second iteration:");
for (final var value : values) {
System.out.println(" " + value);
}
}

Now we can compare the lambda expression and the method reference:

public static void main() {
System.out.println("Lambda:");
final Iterable<String> withLambda =
() -> createStream().iterator();
iterateTwice(withLambda);
System.out.println();
System.out.println("Method reference:");
final Iterable<String> withMethodReference =
createStream()::iterator;
try {
iterateTwice(withMethodReference);
} catch (final IllegalStateException e) {
System.out.println(" Expected exception: " + e);
}
}

Running this code produces the following output:

Lambda:
First iteration:
Creating a new stream.
Example value
Second iteration:
Creating a new stream.
Example value
Method reference:
Creating a new stream.
First iteration:
Example value
Second iteration:
Expected exception: java.lang.IllegalStateException: stream has already been operated upon or closed

The lambda expression allows both iterations to complete successfully. The method reference throws an IllegalStateException during the second iteration.

There is also a difference in when the Stream is created. With the lambda expression, a new Stream is created at the beginning of each iteration. With the method reference, the Stream is created immediately when the Iterable is assigned.

Understanding the Difference

The difference comes from when Java evaluates the expressions involved.

Consider the lambda expression:

() -> createStream().iterator()

The body of the lambda is executed whenever the iterator() method of the Iterable is called. Consequently, every call creates a new Stream and returns its iterator. Its behaviour is equivalent to the following anonymous class:

Iterable<String> values = new Iterable<>() {
@Override
public Iterator<String> iterator() {
return createStream().iterator();
}
};

Each iteration uses an independent Stream instance.

The method reference behaves differently:

createStream()::iterator

This is a method reference to an instance method of a particular object. Java evaluates the expression to the left of :: when the method reference is created. In this case, createStream() is called immediately, and the resulting Stream instance is retained as the receiver of the method reference. The behaviour is therefore equivalent to:

final Stream<String> stream = createStream();
Iterable<String> values = new Iterable<>() {
@Override
public Iterator<String> iterator() {
return stream.iterator();
}
};

Every call to iterator() operates on the same Stream instance.

Streams Can Only Be Consumed Once

Java Streams are designed for a single traversal. Once a Stream has been consumed, it cannot be reused for another traversal. This restriction also applies when consuming a Stream through its iterator.

In the method-reference example, the first iteration consumes the Stream. The second iteration attempts to obtain another iterator from the same Stream instance, which results in an IllegalStateException.

The lambda expression avoids this problem because it creates a new Stream for each call to iterator().

Using a Method Reference Without Reusing the Stream

It is possible to preserve the behaviour of the lambda expression while still using a method reference. Instead of referencing the iterator() method of an existing Stream, we can introduce a static method that creates a new iterator:

private static Iterator<String> createIterator() {
return createStream().iterator();
}

The Iterable can then be defined as:

Iterable<String> values = Main::createIterator;

In this case, the method reference points to a static method. Each invocation of iterator() calls createIterator(), which creates a new Stream. Both iterations now work as expected.

For this particular example, the original lambda expression is arguably simpler:

Iterable<String> values = () -> createStream().iterator();

It also makes the creation of a new Stream for each iteration more explicit.

A General Difference Between Lambdas and Method References

The behaviour demonstrated here is not specific to Streams. An expression of the form

someExpression()::someMethod

evaluates someExpression() immediately when the method reference is created. The resulting object is then used as the receiver whenever the referenced method is invoked.

In contrast, a lambda expression such as

() -> someExpression().someMethod()

evaluates someExpression() each time the lambda is invoked.

This distinction matters when the expression creates a new object, produces side effects, accesses mutable state, or returns a resource that cannot be reused.

It does not apply in the same way to every kind of method reference. For example, a reference to a static method such as SomeClass::someStaticMethod does not capture an instance produced by a receiver expression.

Conclusion

Lambda expressions and method references are not always semantically equivalent, even when they implement the same functional interface.

In this example, replacing

() -> createStream().iterator()

with

createStream()::iterator

changes when the Stream is created and whether a new Stream is available for each iteration.

Both expressions compile successfully, but only the lambda expression produces an Iterable that can be traversed repeatedly.

When refactoring lambda expressions into method references, it is therefore important to consider when the receiver expression is evaluated and whether the resulting object can safely be reused.

Help me with the Spiderman Operator

From time to time, I encounter a silly syntax in Java that I silently dubbed the “spiderman operator” because of all the syntactically pointing that’s going on. My problem is that it’s not very readable, I don’t know an alternative syntax for it and my programming style leads me more often to it than I am willing to ignore.

The spiderman operator looks like this:

x -> () -> x

In its raw form, it means that you have a function that takes x and returns a Supplier of x:

Function<X, Supplier<X>> rawForm = x -> () -> x;

That in itself is not very useful or mysterious, but if you take into account that the Supplier<X> is just one possible type you can return, because in Java, as long as the signature fits, the thing sits, it gets funnier.

A possible use case

Let’s define a type that is an interface with just one method:

public interface DomainValue {
    BigDecimal value();
}

In Java, the @FunctionalInterface annotation is not required to let the interface be, in fact, a functional interface. It only needs to have one method without implementation. How can we provide methods with implementation in Java interfaces. Default methods are the way:

@FunctionalInterface
public interface DomainValue {
    BigDecimal value();

    default String denotation() {
        return getClass().getSimpleName();
    }
}

Let’s say that we want to load domain values from a key-value-store with the following access method:

Optional<Double> loadEntry(String key)

If there is no entry with the given key or the syntax is not suitable to be interpreted as a double, the method returns Optional.emtpy(). Else it returns the double value wrapped in an Optional shell. We can convert it to our domain value like this:

Optional<DomainValue> myValue = 
    loadEntry("current")
        .map(BigDecimal::new)
        .map(x -> () -> x);

And there it is, the spiderman operator. We convert from Double to BigDecimal and then to DomainValue by saying that we want to convert our BigDecimal to “something that can supply a BigDecimal”, which is exactly what our DomainValue can do.

A bigger use case

Right now, the DomainValue type is nothing more than a mantle around a numerical value. But we can expand our domain to have more specific types:

public interface Voltage extends DomainValue {
}
public interface Power extends DomainValue {
    @Override
    default String denotation() {
        return "Electric power";
    }
}

Boring!

public interface Current extends DomainValue {
    default Power with(Voltage voltage) {
	return () -> value().multiply(voltage.value());
    }
}

Ok, this is maybe no longer boring. We can implement a lot of domain functionality just in interfaces and then instantiate ad-hoc types:

Voltage europeanVoltage = () -> BigDecimal.valueOf(220);
Current powerSupply = () -> BigDecimal.valueOf(2);
Power usage = powerSupply.with(europeanVoltage);

Or we load the values from our key-value-store:

Optional<Voltage> maybeVoltage = 
    loadEntry("voltage")
        .map(BigDecimal::new)
        .map(x -> () -> x);

Optional<Current> maybeCurrent = 
    loadEntry("current")
        .map(BigDecimal::new)
        .map(x -> () -> x);

You probably see it already: We have some duplicated code! The strange thing is, it won’t go away so easily.

The first call for help

But first I want to sanitize the code syntactically. The duplication is bad, but the spiderman operator is just unreadable.

If you have an idea how the syntax of the second map() call can be improved, please comment below! Just one request: Make sure your idea compiles beforehands.

Failing to eliminate the duplication

There is nothing easier than eliminating the duplication above: The code is syntactically identical and only the string parameter is different – well, and the return type. We will see how this affects us.

What we cannot do:

<DV extends DomainValue> Optional<DV> loadFor(String entry) {
    Optional<BigDecimal> maybeValue = load(entry);
    return maybeValue.map(x -> () -> x);
}

Suddenly, the spiderman operator does not compile with the error message:

The target type of this expression must be a functional interface

I can see the problem: Subtypes of DomainValue are not required to stay compatible to the functional interface requirement (just one method without implementation).

Interestingly, if we work with a wildcard for the generic, it compiles:

Optional<? extends DomainValue> loadFor(String entry) {
    Optional<BigDecimal> maybeValue = load(entry);
    return maybeValue.map(x -> () -> x);
}

The problem is that we still need to downcast to our specific subtype afterwards. But we can use this insight and move the downcast into the method:

<DV extends DomainValue> Optional<DV> loadFor(
	String entry,
	Class<DV> type
) {
	Optional<BigDecimal> maybeValue = load(entry);
	return maybeValue.map(x -> type.cast(x));
}

Which makes our code readable enough, but at the price of using reflection:

Optional<Voltage> european = loadFor("voltage", Voltage.class);
Optional<Current> powerSupply = loadFor("current", Current.class);

I’m not a fan of this solution, because downcasts are dangerous and reflection is dangerous, too. Mixing two dangerous things doesn’t neutralize the danger most of the time. This code will fail during runtime sooner or later, without any compiler warning us about it. If you don’t believe me, add a second method without implementation to the Current interface and see if the compiler warns you. Hint: This is what you will see at runtime:

java.lang.ClassCastException: Cannot cast java.math.BigDecimal to Current

But, to our surprise, it doesn’t even need a second method. The code above doesn’t work. Even if we reintroduce our spiderman operator (with an additional assignment to help the type inference), the cast won’t work:

<DV extends DomainValue> Optional<DV> loadFor(
    String entry,
    Class<DV> type
) {
    Optional<BigDecimal> maybeValue = load(entry);
    Optional<DomainValue> maybeDomainValue = maybeValue.map(x -> () -> x);
    return maybeDomainValue.map(x -> type.cast(x));
}

The ClassCastException just got a lot more mysterious:

java.lang.ClassCastException: Cannot cast Loader$$Lambda$8/0x00000008000028c0 to Current

My problem is that I am stuck. There is working code that uses the spiderman operator and produces code duplication, but there is no way around the duplication that I can think of. I can get objects for the supertype (DomainValue), but not for a specific subtype of it. If I want that, I have to accept duplication. Or am I missing something?

The second call for help

If you can think about a way to eliminate the duplication, please tell me (or us) in the comments. This problem doesn’t need to be solved for my peace of mind or the sanity of my code – the duplication is confined to a particular place.

Being used to roam nearly without boundaries in the Java syntax (25 years of thinking in Java will do that to you), this particular limitation hit hard. If you can give me some ideas, I would be grateful.