The previous post was mostly about programming "in the small" where the primary concern is making sure the body of code in the method does what it's supposed to and doesn't do anything else. This blog post is about what to do when code doesn't work – how Scala signals failure and how to recover from it, based on some insightful discussions.
First, let's define what we mean by failure.
- Unexpected internal failure: the operation fails as the result of an unfulfilled expectation, such as a null pointer reference, violated assertions, or simply bad state.
- Expected internal failure: the operation fails deliberately as a result of internal state, i.e. a blacklist or circuit breaker.
- Expected external failure: the operation fails because it is told to process some raw input, and will fail if the raw input cannot be processed.
- Unexpected external failure: the operation fails because a resource that the system depends on is not there: there's a loose file handle, the database connection fails, or the network is down.
Java has one explicit construct for handling failure:
Exception. There's some difference of usage in Java throughout the years – IO and JDBC use checked exceptions throughout, while other API like
org.w3c.dom rely on unchecked exceptions. According to Clean Code, the best practice is to use unchecked exceptions in preference to checked exceptions, but there's still debate over whether unchecked exceptions are always appropriate.
Scala makes "checked vs unchecked" very simple: it doesn't have checked exceptions. All exceptions are unchecked in Scala, even
The way you catch an exception in Scala is by defining a
PartialFunction on it:
Or you can use control.Exception, which provides some interesting building blocks. The docs say "focuses on composing exception handlers", which means that this set of classes supplies most of the logic you would put into a catch or finally block.
control.Exception methods is fun and you can string together exception handling logic to create automatic resource management, or an automated exception logger. On the other hand, it's full of sharp things like
allCatch. Leave it alone unless you really need it.
Another important caveat is to make sure that you are catching the exceptions that you think you're catching. A common mistake (mentioned in Effective Scala) is to use a default case in the partial function:
This will catch absolutely everything, including OutOfMemoryError and other errors that would normally terminate the JVM.
If you want to catch "everything" that would normally happen, then use
Exceptions don't get mentioned very much in Scala, but they're still the bedrock for dealing with unexpected failure. For unexpected internal failure, there's a set of assertion methods called
assume, which all use throwables under the hood.
Option represents optional values, returning an instance of
Some(A) if A exists, or
None if it does not. It's ubiquitous in Scala code, to the point where it fades into invisibility. The cheat sheet is the best way to get a handle on it.
It's almost impossible to use
Option incorrectly, but there is one caveat:
Some(null) is valid. If you have code that returns null, wrap it in
Option() to convert it:
Either is a disjoint union construct. It returns either an instance of
Left[L] or an instance of
Right[R]. It's commonly used for error handling, where by convention
Left is used to represent failure and
Right is used to represent success. It's perfect for dealing with expected external failures such as parsing or validation.
Either is like
Option in that it makes an abstract idea explicit by introducing an intermediate object. Unlike
Option, it does not have a
flatMap method, so you can't use it in for comprehensions – not safely at any rate. You can use a left or right projection if you're not interested in handling failure:
More typically, you'll use
You're not limited to using
Either for parsing or validation, of course. You can use it for CQRS.
or arbitary binary choices:
Either is powerful, but it's trickier than
Option. In particular, it can lead to deeply nested code. It can also be misunderstood. Take the following Java lookup method:
Option, so we can use that. But what if the database goes down? Using the error reporting convention of
Either might suggest the following:
But this is awkward. If you return
Either because something might fail unexpectedly, then immediately half your API becomes littered with
Ah, but what if you're modifying a new object?
If you're dealing with expected failure and there's good odds that the operation will fail, then returning
Either is fine: create a case class representing failure
FailResult and use
Don't return exceptions through Either. If you want a construct to return exceptions, use Try.
Try is similar to
Either, but instead of returning any class in a
Right wrapper, it returns
Success[T]. It's an analogue for the try-catch block: it replaces try-catch's stack based error handling with heap based error handling. Instead of having an exception thrown and having to deal with it immediately in the same thread, it disconnects the error handling and recovery.
Try can be used in for comprehensions: unlike
Either, it implements
flatMap. This means you can do the following:
and if there's an exception returned from the first
Try, then the for comprehension will terminate early and return the
You can get access to the exception through pattern matching:
Try will let you recover from exceptions at any point in the chain, so you can defer recovery to the end:
recover in the middle:
There's also a
recoverWith method that will let you swap out a
You can mix
Try together to coerce methods that throw exceptions internally:
Try isn't always appropriate. If we go back to the first exception example, this is the
Note the kludge to get around the lack of a
finally block to close the stream. Victor Klang and Som Snytt suggested using a value class and
transform to enhance
Which is cleaner, at the cost of some magic.
Try was originally invented at Twitter to solve a specific problem: when using Future, the exception may be thrown on a different thread than the caller, and so can't be returned through the stack. By returning an exception instead of throwing it, the system is able to reify the bottom type and let it cross thread boundaries to the calling context.
Try is new enough that people are still getting comfortable with it. I think that it's a useful addition when try-catch blocks aren't flexible enough, but it does have a snag: returning
Try in a public API means exceptions must be dealt with by the caller. Using
Try also implies to the caller that the method has captured all non fatal exceptions itself. If you're doing this in your trait:
Try should be at the top to ensure exception capture:
Because exceptions must be dealt with the caller, you are placing more trust in the caller to handle or delegate a failure appropriately. With try-catch blocks, doing nothing means that the exception can pass up the stack to a top level exception handler. With
Try, exceptions must be either returned or handled by each method in the chain, just like checked exceptions.
To pass the exception along, use
Or to rethrow the exception up the stack if the return type is Unit:
And you want to avoid this:
If you have a system that needs specific error logging or error recovery, it's probably safer to stick to unchecked exceptions.
Exceptionto signal unexpected failure in purely functional code.
Optionto return optional values.
Option(possiblyNull)to avoid instances of
Eitherto report expected failure.
Eitherto return exceptions.
Tryrather than a catch block for handling unexpected failure.
Trywhen working with
Tryin a public API has a similiar effect as a checked exception. Consider using exceptions instead.