What could be kind of surprise even for some experienced Java programmers, the java.util.Stream interface extends java.lang.AutoCloseable. This (in theory) puts it on par with files, DB connections, and other resources requiring manual closing. Should we close all our Streams in the same way as any other resources?

In this post, I’d like to explain which Streams have to be closed and how the closing actually works. We will also go through some of the interesting under-documented properties of this feature.

How Java 8 API met AutoCloseable

AutoCloseable interface has been introduced in Java 7 as a part of try-with-resources construct, designed for easy resource cleanup right after they are not used anymore:

import java.sql.Connection;

// Java < 7
Connection connection = ...;
try {
    // do something with the connection
} finally {
    connection.close(); // explicitely close the connection!
}

// Java 7+
try (Connection connection = ...) {
    // do something with the connection
}
// no explicit close needed!

The goal was to simplify the cleanup process to avoid stuff like DB/network connections or files being left opened unnecessarily. Unfortunately, this change (probably due to backward compatibility reasons) did not introduce any compile-time checks, so such resource leaks are still possible. That’s why the docs for AutoCloseable state clearly:

For code that must operate in complete generality, or when it is known that the AutoCloseable instance requires resource release, it is recommended to use try-with-resources constructions.

Streams (introduced with Java 8) seem to be a great example of that “freedom of choice”, as the docs are mentioning them directly later:

However, when using facilities such as java.util.stream.Stream that support both I/O-based and non-I/O-based forms, try-with-resources blocks are in general unnecessary when using non-I/O-based forms.

The moral is straightforward - when a Stream is dealing with I/O operations (files, connections, etc.) it should be closed - like this:

try (Stream<String> linesStream = Files.lines(Paths.get("file.txt"))) {
    linesStream.forEach(System.out::println);            
} catch (IOException e) {
    // TODO: handle IOException...
}

It’s quite disappointing, that Streams API is not smart enough to perform the cleanup (basically by calling AutoCloseable::close() method) on its own, as a part of the terminal operation (like .collect() or sum()). This is rather counterintuitive especially because an attempt of reusing already “consumed” Stream ends up with the IllegalStateException:

Exception in thread "main" java.lang.IllegalStateException: 
    stream has already been operated upon or closed

As long as this is how it works, using try-with-resources for closing Streams is probably the best solution we have. Unfortunately, it’s also an additional thing we have to remember about.

JDK Streams requiring closing

As far as I know, there is no official list of JDK API methods requiring to close the Streams they produce. However, a simple search for the Stream’s onClose(Runnable closeHandler) method usages (a way of defining close handler, described in the next section) revealed the members of only two classes: java.nio.file.Files and java.util.Scanner. The search has been performed against Java 15.

Methods of the java.nio.file.Files that rely on manual Stream closing:

• Files.list(Path dir) [docs]
• Files.walk(Path start, int maxDepth, FileVisitOption... options) [docs]
• Files.find(Path start, int maxDepth, BiPredicate<Path, BasicFileAttributes> matcher, FileVisitOption... options) [docs]
• Files.lines(Path path) [docs]
• Files.lines(Path path, Charset cs) [docs]

Methods of the java.util.Scanner that rely on manual Stream closing:

• Scanner.tokens() [docs]
• Scanner.findAll(Pattern pattern) [docs]

At the time of writing, IntelliJ Idea does not provide any out-of-the-box inspection that could show the Streams, that we forgot to close. Luckily, Sonarlint plugin (and the SonarQube itself) both report such cases thanks to the S2095 rule.

Update 2021-06-23: As has been pointed out in a comment, IntelliJ IDEA (at least version 2021.1) does provide a dedicated inspection. However, it’s disabled by default and seems to work only in some IO-related cases (not in case of onClose presence in general). To enable the inspection, navigate to Settings > Editor > Inspections and type AutoCloseable used without 'try'-with-resources into the search box.

Stream closing 101

Enabling controlled closing of a Stream is as simple as defining onClose handler by passing appropriate java.lang.Runnable instance. However, please keep in mind that without the usage of try-with-resources (or equivalent) our close handler would not be executed.

Stream<String> closeable = List.of("a", "b", "c", "d", "e").stream()
    .onClose(() -> System.out.println("closed!"));

try (closeable) {
    String formatted = closeable.collect(Collectors.joining());
    System.out.println(formatted);
}

A result is quite predictable:

abcde
closed!

In fact, we can easily define multiple close handlers:

var multiCloseable = IntStream.of(1, 2, 3)
    .onClose(() -> System.out.println("first!"))
    .onClose(() -> System.out.println("second!"))
    .onClose(() -> System.out.println("third!"));
try (multiCloseable) {
    var sum = multiCloseable.sum();
    System.out.println(sum);
}

In such a case, the result is a bit more gripping:

6
first!
second!
third!

It looks like when more than one onClose handlers are defined, they are being executed in order of their appearance (FIFO).

Crash testing close handlers

Allowing multiple close handlers to be defined leads to the interesting question: what happens when one of the closing procedures fails? Let’s take our previous example into another ride, and modify the second close handler in a way, that will throw an exception.

var multiCloseable = IntStream.of(1, 2, 3)
    .onClose(() -> System.out.println("first!"))
    .onClose(() -> {
        throw new IllegalStateException("second has failed");
    })
    .onClose(() -> System.out.println("third!"));
try (multiCloseable) {
    System.out.println(multiCloseable.sum());
}

When one of the Stream’s close handlers throws an exception, the rest of them is still being executed in a standard order - exactly as we can see in the output:

6
first!
third!
Exception in thread "main" java.lang.IllegalStateException: second has failed

We can go even one step further, and let all 3 close handlers fail in the same way. We already know, that all defined handlers will be executed, but what (which?) exception will be reported?

var multiCloseable = IntStream.of(1, 2, 3)
    .onClose(() -> {
        throw new IllegalStateException("first has failed");
    })
    .onClose(() -> {
        throw new IllegalStateException("second has failed");
    })
    .onClose(() -> {
        throw new IllegalStateException("third has failed");
    });
try (multiCloseable) {
    System.out.println(multiCloseable.sum());
}

In case when more than one close handler fails, the returned exception is the first one being thrown. However, all other (next) exceptions are not lost, but reported as suppressed ones:

6
Exception in thread "main" java.lang.IllegalStateException: first has failed
	at com.mikemybytes.Playground.lambda$main$0(Playground.java:48)
	at java.base/java.util.stream.Streams$1.run(Streams.java:842)
	at java.base/java.util.stream.Streams$1.run(Streams.java:842)
	at java.base/java.util.stream.AbstractPipeline.close(AbstractPipeline.java:323)
	at com.mikemybytes.Playground.main(Playground.java:58)
	Suppressed: java.lang.IllegalStateException: second has failed
		at com.mikemybytes.Playground.lambda$main$1(Playground.java:51)
		at java.base/java.util.stream.Streams$1.run(Streams.java:846)
		... 3 more
	Suppressed: java.lang.IllegalStateException: third has failed
		at com.mikemybytes.Playground.lambda$main$2(Playground.java:54)
		at java.base/java.util.stream.Streams$1.run(Streams.java:846)
		... 2 more

That means, that with a bit of stack trace traversing we can obtain a complete set of all such exceptions being thrown in the meantime.

Streams vs flatMap

Update 2021-05-03: As František Řezáč and Manos Tzagkarakis have pointed me out on Twitter, flatMap behaves a bit differently in terms of stream closing. This paragraph has been added thanks to their suggestion.

Surprisingly, closeable Streams are handled differently when being passed to flatMap. As we can find in the docs, the flatMap method:

Returns a stream consisting of the results of replacing each element of this stream with the contents of a mapped stream produced by applying the provided mapping function to each element. Each mapped stream is closed after its contents have been placed into this stream. (If a mapped stream is null an empty stream is used, instead.)

Let’s illustrate that with an example:

var input1 = Stream.of("a", "b").onClose(() -> {
    System.out.println("closed 1");
});
var input2 = Stream.of("c", "d").onClose(() -> {
    System.out.println("closed 2");
});
var combined = Stream.of(input1, input2).onClose(() -> {
    System.out.println("closed combined");
});
var elements = combined.flatMap(s -> s).count(); // no closing here!
System.out.println("elements: " + elements);

Every Stream passed to the flatMap method will be closed automatically without a need of using try-with-resources (or calling close() directly) on the Stream that contains them - that’s why the output contains traces of closing both the input1 and input2 Streams:

closed 1
closed 2
elements: 4

Summary

Finally, let’s recap all of the discussed Stream closing properties:

• In general, all I/O-related Streams should be put inside try-with-resources construct to ensure automated resource cleanup. Pay close attention to the methods of java.nio.file.Files and java.util.Scanner.
• When unsure about if the particular Stream should be closed or not, look for onClose(Runnable closeHandler) calls within the Stream returning method.
• Single Stream could have multiple close handlers defined - they will be executed in the FIFO order.
• All close handlers defined for a Stream will be executed even in case of throwing an exception in any of them.
• In case when multiple exceptions have been thrown from close handlers, the returned stack trace will contain all of them. The first exception thrown will become the main one.
• Streams passed to flatMap will be closed regardless of closing the Stream that contains them.