Error Handling in Scala

The previous post was mostly about programming "in the small" where the primary concern is making sure the body of code in the method does what it's supposed to and doesn't do anything else. This blog post is about what to do when code doesn't work – how Scala signals failure and how to recover from it, based on some insightful discussions.

First, let's define what we mean by failure.

• Unexpected internal failure: the operation fails as the result of an unfulfilled expectation, such as a null pointer reference, violated assertions, or simply bad state.
• Expected internal failure: the operation fails deliberately as a result of internal state, i.e. a blacklist or circuit breaker.
• Expected external failure: the operation fails because it is told to process some raw input, and will fail if the raw input cannot be processed.
• Unexpected external failure: the operation fails because a resource that the system depends on is not there: there's a loose file handle, the database connection fails, or the network is down.

Java has one explicit construct for handling failure: Exception. There's some difference of usage in Java throughout the years – IO and JDBC use checked exceptions throughout, while other API like org.w3c.dom rely on unchecked exceptions. According to Clean Code, the best practice is to use unchecked exceptions in preference to checked exceptions, but there's still debate over whether unchecked exceptions are always appropriate.

Exceptions

Scala makes "checked vs unchecked" very simple: it doesn't have checked exceptions. All exceptions are unchecked in Scala, even SQLException and IOException.

The way you catch an exception in Scala is by defining a PartialFunction on it:

val input = new BufferedReader(new FileReader(file))
try {
  try {
    for (line <- Iterator.continually(input.readLine()).takeWhile(_ != null)) {
      Console.println(line)
    }
  } finally {
    input.close()
  }
} catch {
  case e:IOException => errorHandler(e)
}

Or you can use control.Exception, which provides some interesting building blocks. The docs say "focuses on composing exception handlers", which means that this set of classes supplies most of the logic you would put into a catch or finally block.

Exception.handling(classOf[RuntimeException], classOf[IOException]) by println apply { 
  throw new IOException("foo") 
}

Using the control.Exception methods is fun and you can string together exception handling logic to create automatic resource management, or an automated exception logger. On the other hand, it's full of sharp things like allCatch. Leave it alone unless you really need it.

Another important caveat is to make sure that you are catching the exceptions that you think you're catching. A common mistake (mentioned in Effective Scala) is to use a default case in the partial function:

try {
  operation()
} catch {
  case _ => errorHandler(e)
}

This will catch absolutely everything, including OutOfMemoryError and other errors that would normally terminate the JVM.

If you want to catch "everything" that would normally happen, then use NonFatal:

import scala.util.control.NonFatal

try {
  operation()
} catch {
  case NonFatal(exc) => errorHandler(e)
}

Exceptions don't get mentioned very much in Scala, but they're still the bedrock for dealing with unexpected failure. For unexpected internal failure, there's a set of assertion methods called require, assert, and assume, which all use throwables under the hood.

Option

Option represents optional values, returning an instance of Some(A) if A exists, or None if it does not. It's ubiquitous in Scala code, to the point where it fades into invisibility. The cheat sheet is the best way to get a handle on it.

It's almost impossible to use Option incorrectly, but there is one caveat: Some(null) is valid. If you have code that returns null, wrap it in Option() to convert it:

val optionResult = Option(null) // optionResult is None.

Either

Either is a disjoint union construct. It returns either an instance of Left[L] or an instance of Right[R]. It's commonly used for error handling, where by convention Left is used to represent failure and Right is used to represent success. It's perfect for dealing with expected external failures such as parsing or validation.

case class FailResult(reason:String)

def parse(input:String) : Either[FailResult, String] = {
  val r = new StringTokenizer(input)
  if (r.countTokens() == 1) {
    Right(r.nextToken())
  } else {
    Left(FailResult("Could not parse string: " + input))
  }
}

Either is like Option in that it makes an abstract idea explicit by introducing an intermediate object. Unlike Option, it does not have a flatMap method, so you can't use it in for comprehensions – not safely at any rate. You can use a left or right projection if you're not interested in handling failure:

val rightFoo = for (outputFoo <- parse(input).right) yield outputFoo 

More typically, you'll use fold:

parse(input).fold(
  error => errorHandler(error),
  success => { ... }
)

You're not limited to using Either for parsing or validation, of course. You can use it for CQRS.

case class UserFault
case class UserCreatedEvent

def createUser(user:User) : Either[UserFault, UserCreatedEvent]

or arbitary binary choices:

def whatShape(shape:Shape) : Either[Square, Circle]

Either is powerful, but it's trickier than Option. In particular, it can lead to deeply nested code. It can also be misunderstood. Take the following Java lookup method:

public Foo lookup(String id) throws FooException // throw if not found or db exception

Scala has Option, so we can use that. But what if the database goes down? Using the error reporting convention of Either might suggest the following:

def lookup() : Either[FooException,Option[Foo]]

But this is awkward. If you return Either because something might fail unexpectedly, then immediately half your API becomes littered with Either[Throwable, T].

Ah, but what if you're modifying a new object?

def modify(inputFoo:Foo) : Either[FooException,Foo]

If you're dealing with expected failure and there's good odds that the operation will fail, then returning Either is fine: create a case class representing failure FailResult and use Either[FailResult,Foo].

Don't return exceptions through Either. If you want a construct to return exceptions, use Try.

Try

Try is similar to Either, but instead of returning any class in a Left or Right wrapper, it returns Failure[Throwable] or Success[T]. It's an analogue for the try-catch block: it replaces try-catch's stack based error handling with heap based error handling. Instead of having an exception thrown and having to deal with it immediately in the same thread, it disconnects the error handling and recovery.

Try can be used in for comprehensions: unlike Either, it implements flatMap. This means you can do the following:

val sumTry = for {
  int1 <- Try(Integer.parseInt("1"))
  int2 <- Try(Integer.parseInt("2"))
} yield {
  int1 + int2
}

and if there's an exception returned from the first Try, then the for comprehension will terminate early and return the Failure.

You can get access to the exception through pattern matching:

sumTry match {
  case Failure(thrown) => {
    Console.println("Failure: " + thrown)
  }
  case Success(s) => {
    Console.println(s)
  }
}

Or through failed:

if (sumTry.isFailure) {
  val thrown = sumTry.failed.get  
}

Try will let you recover from exceptions at any point in the chain, so you can defer recovery to the end:

val sum = for {
  int1 <- Try(Integer.parseInt("one"))
  int2 <- Try(Integer.parseInt("two"))
} yield {
  int1 + int2
} recover {
  case e => 0
}

Or recover in the middle:

val sum = for {
  int1 <- Try(Integer.parseInt("one")).recover { case e => 0 }
  int2 <- Try(Integer.parseInt("two"))
} yield {
  int1 + int2
} 

There's also a recoverWith method that will let you swap out a Failure:

val sum = for {
  int1 <- Try(Integer.parseInt("one")).recoverWith {
    case e: NumberFormatException => Failure(new IllegalArgumentException("Try 1 next time"))
  }
  int2 <- Try(Integer.parseInt("2"))
} yield {
  int1 + int2
}

You can mix Either and Try together to coerce methods that throw exceptions internally:

val either : Either[String, Int] = Try(Integer.parseInt("1")).transform({ i => Success(Right(i)) }, { e => Success(Left("FAIL")) }).get
Console.println("either is " + either.fold(l => l, r => r))    

Try isn't always appropriate. If we go back to the first exception example, this is the Try analogue:

val input = new BufferedReader(new FileReader(file))
val results = Seq(
  Try {
    for (line <- Iterator.continually(input.readLine()).takeWhile(_ != null)) {
      Console.println(line)
    }
  },
  Try(input.close())
)

results.foreach { result =>
  result.recover {
    case e:IOException => errorHandler(e)
  }
}

Note the kludge to get around the lack of a finally block to close the stream. Victor Klang and Som Snytt suggested using a value class and transform to enhance Try:

implicit class TryOps[T](val t: Try[T]) extends AnyVal {
  def eventually[Ignore](effect: => Ignore): Try[T] = {
    val ignoring = (_: Any) => { effect; t }
    t transform (ignoring, ignoring)
  }
}

Try(1 / 0).map(_ + 1) eventually { println("Oppa Gangnam Style") }

Which is cleaner, at the cost of some magic.

Try was originally invented at Twitter to solve a specific problem: when using Future, the exception may be thrown on a different thread than the caller, and so can't be returned through the stack. By returning an exception instead of throwing it, the system is able to reify the bottom type and let it cross thread boundaries to the calling context.

Try is new enough that people are still getting comfortable with it. I think that it's a useful addition when try-catch blocks aren't flexible enough, but it does have a snag: returning Try in a public API means exceptions must be dealt with by the caller. Using Try also implies to the caller that the method has captured all non fatal exceptions itself. If you're doing this in your trait:

def modify(foo:Foo) : Try[Foo]

Then Try should be at the top to ensure exception capture:

def modify(foo:Foo) : Try[Foo] = Try {
  Foo()
}

Because exceptions must be dealt with the caller, you are placing more trust in the caller to handle or delegate a failure appropriately. With try-catch blocks, doing nothing means that the exception can pass up the stack to a top level exception handler. With Try, exceptions must be either returned or handled by each method in the chain, just like checked exceptions.

To pass the exception along, use map:

def fooToString(foo:Foo) : Try[String] = {
  modify(foo).map { outFoo =>
   outFoo.toString()
  }
}

Or to rethrow the exception up the stack if the return type is Unit:

def doStuff : Unit = {
  val modifiedFoo = modify(foo).get // throws the exception if failure
}

And you want to avoid this:

modify(foo) match {
  case Failure(f) => { 
    // database failure?  don't care, swallow exception.
  } 
  case Success(s) => {
    ...
  }
}

If you have a system that needs specific error logging or error recovery, it's probably safer to stick to unchecked exceptions.

TL;DR

• Throw Exception to signal unexpected failure in purely functional code.
• Use Option to return optional values.
• Use Option(possiblyNull) to avoid instances of Some(null).
• Use Either to report expected failure.
• Use Try rather than Either to return exceptions.
• Use Try rather than a catch block for handling unexpected failure.
• Use Try when working with Future.
• Exposing Try in a public API has a similiar effect as a checked exception. Consider using exceptions instead.