Terse Systems

Redefining java.lang.System With Byte Buddy

| Comments

The previous post talked about using Java’s SecurityManager to prevent attackers from gaining access to sensitive resources. This is complicated by the fact that if the wrong permissions are granted, or a type confusion attack in the JVM is used, it’s possible to turn off SecurityManager by calling System.setSecurityManager(null).

There should be a way to tell the JVM that once a SecurityManager is set, it should never be unset. But this isn’t in the JVM itself right now, and adding it would mean redefining java.lang.System itself. So let’s go do that.

The example project is at https://github.com/wsargent/securityfixer.

The first step is to use the Java Instrumentation API. This will allow us to install a Java agent before the main program starts. In the Java agent, we’ll intercept the setSecurityManager method, and throw an exception if the security manager is already set.

The second step is Byte Buddy, a code generation tool that will create new bytecode representing the System class. Byte Buddy comes with an AgentBuilder that can be attached to the instrumentation instance. Byte Buddy uses ASM under the hood, but doesn’t require raw manipulation of byte code and class files the way that ASM does — instead, you write interceptors and Byte Buddy will generate the corresponding byte code. From there, an interceptor appended to the bootstrap class path will be loaded before the actual JVM System class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
public class SecurityFixerAgent {

    public static void premain(String arg, Instrumentation inst) {
        install(arg, inst);
    }

    public static void agentmain(String arg, Instrumentation inst) {
        install(arg, inst);
    }

    /**
     * Installs the agent builder to the instrumentation API.
     *
     * @param arg the path to the interceptor JAR file.
     * @param inst instrumentation instance.
     */
    static void install(String arg, Instrumentation inst) {
        appendInterceptorToBootstrap(arg, inst);
        AgentBuilder agentBuilder = createAgentBuilder(inst);
        agentBuilder.installOn(inst);
    }
}

The interceptor class lives in its own package and is relatively simple:

1
2
3
4
5
6
7
8
9
10
11
public class MySystemInterceptor {

    private static SecurityManager securityManager;

    public static void setSecurityManager(SecurityManager s) {
        if (securityManager != null) {
            throw new IllegalStateException("SecurityManager cannot be reset!");
        }
        securityManager = s;
    }
}

The interesting bit is the configuration of the AgentBuilder. Byte Buddy is set up out of the box for class transformation and adding new methods and dynamic classes, not redefinition of static methods, so we have to flip a bunch of switches to get the behavior we want:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
private static AgentBuilder createAgentBuilder(Instrumentation inst) {
     // Find me a class called "java.lang.System"
     final ElementMatcher.Junction<NamedElement> systemType = ElementMatchers.named("java.lang.System");

     // And then find a method called setSecurityManager and tell MySystemInterceptor to
     // intercept it (the method binding is smart enough to take it from there)
     final AgentBuilder.Transformer transformer =
             (b, typeDescription) -> b.method(ElementMatchers.named("setSecurityManager"))
                     .intercept(MethodDelegation.to(MySystemInterceptor.class));

     // Disable a bunch of stuff and turn on redefine as the only option
     final ByteBuddy byteBuddy = new ByteBuddy().with(Implementation.Context.Disabled.Factory.INSTANCE);
     final AgentBuilder agentBuilder = new AgentBuilder.Default()
             .withByteBuddy(byteBuddy)
             .withInitializationStrategy(AgentBuilder.InitializationStrategy.NoOp.INSTANCE)
             .withRedefinitionStrategy(AgentBuilder.RedefinitionStrategy.REDEFINITION)
             .withTypeStrategy(AgentBuilder.TypeStrategy.Default.REDEFINE)
             .type(systemType)
             .transform(transformer);

     return agentBuilder;
}

Finally, once that’s done, we can write a simple test class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
public class Main {

    // We don't want any ACTUAL security here when we turn on the security manager...
    static class SillyPolicy extends Policy {
        @Override
        public boolean implies(ProtectionDomain domain, Permission permission) {
            return true;
        }
    }

    public static void main(String[] args) throws Exception {
        // Programmer turns on security manager...
        Policy.setPolicy(new SillyPolicy());
        System.setSecurityManager(new SecurityManager());

        System.out.println("Security manager is set!");
        try {
            // Attacker tries to turn off security manager...
            System.setSecurityManager(null);

            // Happens on normal circumstances...
            System.err.println("ATTACK SUCCEEDED: Security manager was reset!");
        } catch (IllegalStateException e) {
            // Happens on agent redefinition of java.lang.System
            System.out.println("ATTACK FAILED: " + e.getMessage());
        }
    }

}

And then running it with the agent enabled and pointing to the interceptor will give us the behavior we want:

1
java -javaagent:agent/target/securityfixer-agent-1.0-SNAPSHOT.jar=bootstrap/target/securityfixer-bootstrap-1.0-SNAPSHOT.jar securityfixer.Main

Gives us the output:

1
2
Security manager is set!
ATTACK FAILED: SecurityManager cannot be reset!

There’s a lot more you can do with this, but it’s a good start.

Self-Protecting Sandbox Using SecurityManager

| Comments

TL:DR; Most sandbox mechanisms involving Java’s SecurityManager do not contain mechanisms to prevent the SecurityManager itself from being disabled, and are therefore “defenseless” against malicious code. Use a SecurityManager and a security policy as a system property on startup to cover the entire JVM, or use an “orthodox” sandbox as described below.

Background

Since looking at the Java Serialization vulnerability, I’ve been thinking about mitigations and solutions in the JVM. I started with a Java agent, notsoserial, as a way to disable Java serialization itself. But that just opened up a larger question — why should the JVM let you call runtime.exec to begin with? Poking into that question led me to Pro-Grade and looking at blacklists of security manager permissions as a solution.

The problem with blacklists is that there’s always a way to work around them. Simply disabling the “execute” file permission in the JVM didn’t mean anything by itself — what’s to say that it couldn’t simply be re-enabled? What prevents malicious code from turning off the SecurityManager?

I thought it was possible, but I didn’t know exactly how it could happen. I’ve never worked on a project that worked with the SecurityManager, or added custom security manager checks. Most of the time, the Java security manager is completely disabled on server side applications, or the application is given AllPermission, essentially giving it root access to the JVM.

Broken Sandboxes

All my suspicions were confirmed in a recent paper from a team at CMU, Evaluating the Flexibility of the Java Sandbox:

[D]evelopers regularly misunderstand or misuse Java security mechanisms, that benign programs do not use all of the vast flexibility afforded by the Java security model, and that there are clear differences between the ways benign and exploit programs interact with the security manager.

The team found that most of the policies that programmers put in place to “sandbox” code so that it did not have permissions to do things like execute files were not effective. In particular, they call out some security manager idioms as “defenseless” — they cannot prevent themselves from being sidestepped. In a nutshell, while sandboxed applications may not execute a script on the filesystem, they can still modify or disable the security manager.

The team looked through 36 applications that used the SecurityManager, to see how application programmers work with the security architecture.

Every single one of them failed.

All of these applications ran afoul of the Java sandbox’s flexibility even though they attempted to use it for its intended purpose. […] While Java does provide the building blocks for constraining a subset of an application with a policy that is stricter than what is imposed on the rest of the application, it is clear that it is too easy to get this wrong: we’ve seen no case where this goal was achieved in a way that is known to be free of vulnerabilities.

I think there are times when the disconnect between security professionals and application developers turns into a gulf. At the same time that the Oracle Secure Coding Guide has a section on access control, and the CERT guide talks about protecting sensitive operations with security manager checks, there’s very little about how to set up a secure environment that can use SecurityManager effectively at all.

Sandbox Defeating Permissions

Here are the permissions that make up a defenseless sandbox:

  • RuntimePermission("createClassLoader")
  • RuntimePermission("accessClassInPackage.sun")
  • RuntimePermission("setSecurityManager")
  • ReflectPermission("suppressAccessChecks")
  • FilePermission("<<ALL FILES>>", "write, execute")
  • SecurityPermission("setPolicy")
  • SecurityPermission("setProperty.package.access")

Given any one of these, sandboxed code can break out of the sandbox and work its way down the stack. The AllPermission in particular allows all of the above, and the team specifically calls out blacklists as ineffective:

[R]estricting just one permission but allowing all others results in a defenseless sandbox.

The prograde-example I showed earlier only restricts one permission, and allows all others. It’s absolutely better than nothing, but it can be worked around. To ensure that your security manager can defend itself, you should use pro-grade and add to add the following to the security policy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
priority "deny";

deny {
  permission java.io.FilePermission "<<ALL FILES>>", "write, execute";
  permission java.lang.RuntimePermission "createClassLoader";
  permission java.lang.RuntimePermission "accessClassInPackage.sun";
  permission java.lang.RuntimePermission "setSecurityManager";
  permission java.lang.reflect.ReflectPermission  "suppressAccessChecks";
  permission java.security.SecurityPermission "setPolicy";
  permission java.security.SecurityPermission "setProperty.package.access";

  // prevent mutable strings
  // http://slightlyrandombrokenthoughts.blogspot.com/2009/07/java-se-security-part-ii-immutability.html
  permission java.lang.RuntimePermission "charsetProvider";
};

Some of these permissions — such as the access to the entire ‘sun’ package — are ubiquitous in packages, so you may not be able to do this. Still, if you have an application server, you should look at the policy file and see if any of the above permissions are explicitly granted.

The Orthodox Sandbox

The team discusses an effective sandbox techniques in passing:

[…] Java provides an “orthodox” mechanism to achieve this goal while aligning with intended sandbox usage: a custom class loader that loads untrusted classes into a constrained protection domain. This approach is more clearly correct and enables a self-protecting sandbox.

Note: this is true, but also a bit misleading. The reference to a “constrained protection domain” is to ClassLoader.defineClass, which can take a ProtectionDomain. The SecureClassLoader maps a ProtectionDomain to a CodeSource, and the URLClassLoader maps an array of URL to the individual CodeSource — the end result is that SecurityPolicy can see that a protection domain has a custom class loader, and can return different permissions from there.

Rather than attack the problem from inside the application, the team implemented a JVMTI agent which is even more secure, able to stop all the exploits they tried. It does have some drawbacks though:

Unfortunately, to use the JVMTI events that allow us to enforce rule 2, the JIT must be off, which drastically slows down the execution of applets in the presence of our agent. Rule 1 adds about 1-2% overhead.

I decided I would write my own sandbox. Jens Nordahl wrote a blog post called Sandboxing plugins in Java that was extraordinarily helpful in laying out the foundations.

Implementation

So, here’s my example project. It starts up a sandbox, then tries to execute a script from within the sandbox. Add the execute permissions, and it works. Take out the execute permission, and it doesn’t.

The main class is small:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
object Main {

  private val logger = LoggerFactory.getLogger(Main.getClass)

  private val className = "com.tersesystems.sandboxexperiment.sandbox.ScriptRunner"

  def main(args: Array[String]) {
    val sm = System.getSecurityManager
    if (sm == null) {
      val homeDir = System.getProperty("user.dir")
      val scriptName = s"${homeDir}/../testscript.sh"
      logger.info("Starting security manager in the code")
      System.setSecurityManager(new SandboxSecurityManager(scriptName))
    } else {
      logger.error(s"Predefined security manager ${sm}")
      System.exit(-1)
    }

    try {
      val result = runSandboxCode[String]
      logger.info(s"result = $result")
    } catch {
      case e: AccessControlException =>
        logger.error("Cannot run untrusted code", e)
      case NonFatal(e) =>
        logger.error("Unexpected error", e)
      case other: Throwable =>
        logger.error("Don't know what happened", other)
    }
  }

  private def createSandboxClassLoader: SandboxClassLoader = {
    DoPrivilegedAction(new RuntimePermission("createClassLoader")) {
      new SandboxClassLoader(this.getClass.getClassLoader)
    }(AccessController.getContext)
  }

  /**
   * Uses reflection to instantiate the class which will try to execute shell code.
   */
  private def runSandboxCode[T]: T = {
    // Use a custom class loader to isolate the code...
    val sandboxClassLoader = createSandboxClassLoader
    val scriptRunnerClass = sandboxClassLoader.loadClass(className)
    val method = scriptRunnerClass.getMethod("run")
    val scriptRunnerInstance = scriptRunnerClass.newInstance()
    try {
      method.invoke(scriptRunnerInstance).asInstanceOf[T]
    } catch {
      case e: java.lang.reflect.InvocationTargetException =>
        throw e.getCause
    }
  }
}

and the SandboxPolicy uses the classloader’s type to determine the sandbox permissions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
class SandboxPolicy(scriptName: String) extends Policy {

  override def getPermissions(domain: ProtectionDomain): PermissionCollection = {
    val result: PermissionCollection = if (providerDomain == domain) {
      new AllPermission().newPermissionCollection()
    } else if (isSandbox(domain)) {
      sandboxPermissions
    } else {
      appPermissions
    }
    result
  }

  private def sandboxPermissions: Permissions = {
    val permissions = new Permissions()
    permissions.add(new PropertyPermission("*", "read"))

    // THIS IS THE LINE WHERE EVERYTHING HAPPENS!
    // DON'T COMMENT OUT PLZ KTHXBYE
    permissions.add(new java.io.FilePermission(scriptName, "execute"))
    permissions
  }

  private def isSandbox(domain: ProtectionDomain): Boolean = {
    domain.getClassLoader match {
      case cl: SandboxClassLoader =>
        true
      case other =>
        false
    }
  }
}

After that, the actual script runner is straightforward:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class ScriptRunner {
  def run() = executeScript()

  private def executeScript(): String = {
    // Slightly hacked up because we don't have access to Main here...
    val cwd = System.getProperty("user.dir")
    val script = s"${cwd}/../testscript.sh"

    // Okay, here we go!
    val runtime = Runtime.getRuntime
    val process = runtime.exec(script)
    val input = new BufferedReader(new InputStreamReader(process.getInputStream))
    val b = new StringBuffer()
    for (line <- Iterator.continually(input.readLine()).takeWhile(_ != null)) {
      b.append(line)
    }
    b.toString
  }
}

I also added a bunch of logging, probably more than necessary. Any privilege escalation has to go through the SecurityManager, and so an exploit may stand a good chance of being detected even if it succeeds, especially once you add Logstash and an intrusion detection tool.

Experience and Limitations

The big limitation of the sandbox is that it is incredibly limiting. Java’s sandbox is designed for completely untrusted code that can be downloaded from the Internet. It was not designed for protecting server-side code against privilege escalation exploits.

I found myself bumping into issues, mostly around what a security policy is capable of, and how plugin code must be packaged distinctly. There is no way to sandbox a particular class, for example — instead, you must identify a JAR or class location by URL, point a class loader at it, and then use reflection to instantiate the sandbox.

This means that to use a sandbox, there needs to be almost nothing on the classpath. From a comment on the article itself:

If a jar file is already on the classpath of the application i would not consider it a plugin. The premise here is that plugin code is less trustworthy than application code – so a plugin jar file should not be allowed on the classpath.

Best Practices

For new applications, you can sandbox code pretty easily: you can write a small launcher that sets up the SecurityManager and a list of plugins, and sets up sandboxes for each of them.

Because you have a custom class loader, you do have some options. You can forbid loading of sensitive classes (such as serialization) by overriding loadClass:

1
2
3
4
5
6
override def loadClass(name: String, resolve: Boolean): Class[_] = {
  if (forbiddenClasses.contains(name)) {
    throw new IllegalArgumentException("This functionality is disabled")
  }
  super.loadClass(name, resolve)
}

Or you can assign individual classes to different CodeSource instances, and give them different ProtectionDomain areas inside the classloader itself, although that would require some tweaking to ensure that the class could only be found in the given CodeSource.

For refactoring an existing application, it is probably easier to generate a policy with pro-grade’s policy generator: all the individual JAR files are known, so it’s a matter of assigning them them the minimal set of permissions. It is incredibly easy to create a policy that allows the SecurityManager to be disabled, but the out of the box policy for Tomcat seems to avoid the dangerous permissions and has generally good guidelines.

If using a policy file is not practical, the next best option is to override the class loader to forbid or allow certain packages. This is what SecurityManager itself does internally — when you add permissions to access a particular package using the “accessClassInPackage” permission, it’s actually feeding back into the class loader to do the check.

Further Reading

I have not covered a couple of areas. Notably, I haven’t discussed what happens when the sandbox wants to create its own class loader, and how you would manage that securely. I imagine you’d look through the classloader’s parents and if you see SandboxClassLoader in that list, you know it’s sandboxed… but I don’t know that there’s not an exploit in that.

Likewise, I haven’t gone into thread management very much. The AccessControlContext can be passed between different threads, but I don’t have the mechanics down. There is an article about secure thread collaboration across protection domains. However, I have figured out how to prevent sandboxed code from spawning threads — something the default SecurityManager does not prevent by default.

There are still performance concerns about the SecurityManager, which was not designed for a heavily multi-threaded environment. However, there is progress in this area: Peter Firmstone and Alexey V. Varlamov wrote a drop in replacement for the security manager designed for concurrency as part of Apache River, and there’s a completed project JEP-232 to improve secure application performance.

In addition to sandboxing, the SecurityManager can also be used to track JVM activity. Wealthfront added a modification to inspect IO in the JVM — the class itself is available at LessIOSecurityManager.java that can be set as a system property.

The ten year retrospective on Java Security by Li Gong makes for some interesting reading.

Other “orthodox” sandbox examples are thin on the ground. One example on Github is csm-sandbox, but I haven’t checked it myself. You can also see a sandbox example in Java Security, 2nd Edition, in Appendix D.

It would be nice if the specification allowed a policy file to be packaged with a signed JAR file, so everything didn’t have to be done in a single java.policy file that has to assign every single code source. For example, logback needs IO / network permissions, but does not need to execute scripts, so if it just came with a policy file that said so, it would be easy enough to incorporate that. Anything that minimizes the hassle of putting a policy file together would be a good thing.

Finally, JDK 1.9 is coming out soon. Modularization features should make isolation and boundaries easier to implement, and there’s a reference to Module System Security Features in the slides.

An Easy Way to Secure Java Applications

| Comments

One of the things that stands out in the Java Serialization exploit is that once a server side Java application is compromised, the next step is to gain shell access on the host machine. This is known as a Remote Code Execution, or RCE for short.

The interesting thing is that Java has had a way to restrict execution and prevent RCE almost since Java 1.1: the SecurityManager. With the SecurityManager enabled, Java code operates inside a far more secure sandbox that prevents RCE.

1
java -Djava.security.manager com.example.Hello

This runs with the default security policy in $JAVA_HOME/jre/lib/security/java.policy, which in JDK 1.8 is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
// Standard extensions get all permissions by default

grant codeBase "file:$/*" {
        permission java.security.AllPermission;
};

// default permissions granted to all domains

grant {
        // Allows any thread to stop itself using the java.lang.Thread.stop()
        // method that takes no argument.
        // Note that this permission is granted by default only to remain
        // backwards compatible.
        // It is strongly recommended that you either remove this permission
        // from this policy file or further restrict it to code sources
        // that you specify, because Thread.stop() is potentially unsafe.
        // See the API specification of java.lang.Thread.stop() for more
        // information.
        permission java.lang.RuntimePermission "stopThread";

        // allows anyone to listen on dynamic ports
        permission java.net.SocketPermission "localhost:0", "listen";

        // "standard" properies that can be read by anyone

        permission java.util.PropertyPermission "java.version", "read";
        permission java.util.PropertyPermission "java.vendor", "read";
        permission java.util.PropertyPermission "java.vendor.url", "read";
        permission java.util.PropertyPermission "java.class.version", "read";
        permission java.util.PropertyPermission "os.name", "read";
        permission java.util.PropertyPermission "os.version", "read";
        permission java.util.PropertyPermission "os.arch", "read";
        permission java.util.PropertyPermission "file.separator", "read";
        permission java.util.PropertyPermission "path.separator", "read";
        permission java.util.PropertyPermission "line.separator", "read";

        permission java.util.PropertyPermission "java.specification.version", "read";
        permission java.util.PropertyPermission "java.specification.vendor", "read";
        permission java.util.PropertyPermission "java.specification.name", "read";

        permission java.util.PropertyPermission "java.vm.specification.version", "read";
        permission java.util.PropertyPermission "java.vm.specification.vendor", "read";
        permission java.util.PropertyPermission "java.vm.specification.name", "read";
        permission java.util.PropertyPermission "java.vm.version", "read";
        permission java.util.PropertyPermission "java.vm.vendor", "read";
        permission java.util.PropertyPermission "java.vm.name", "read";
};

Take code like this, for example:

1
2
3
4
5
6
7
8
9
10
package com.example

object Hello {
  def main(args: Array[String]): Unit = {
    val runtime = Runtime.getRuntime
    val cwd = System.getProperty("user.dir")
    val process = runtime.exec(s"$cwd/testscript.sh")
    println("Process executed without security manager interference!")
  }
}

With the security manager enabled and using an additional policy file, it’s possible to enable or disable execute privileges cleanly:

1
2
3
4
5
6
7
8
grant {
  // You can read user.dir
  permission java.util.PropertyPermission "user.dir", "read";

  // Gets access to the current user directory script
  permission java.io.FilePermission "${user.dir}/testscript.sh", "execute";
  permission java.util.PropertyPermission "scala.control.noTraceSuppression", "read";
};

You can run this with:

1
java -Djava.security.manager -Djava.security.policy=security.policy com.example.Hello

Just comment out the FilePermission line, and you get an exception.

So far, so great. But it’s only enabled for applets, on the client side. It’s disabled on the server-side.

Why? Well, because (as you can see above) the default SecurityManager locks down the system to the point of uselessness. In order to make the system useful, it must have a custom java.security.policy file defined.

This policy implementation has several problems. The policy file itself is archaic. The security permissions are not laid out in any kind of logical order, and some permissions have options for wildcards while others do not. You can only “allow” behavior with whitelists, not deny it. And worst of all, the longer the list, the slower the application will run. There is a tutorial and list of permissions, but it’s not terribly helpful in practice. And the documentation guide was last updated in 2002.

It is possible to write a custom SecurityManager when you have untrusted code: this is what Scalatron does, for example. NOTE: This is NOT a secure implementation of a sandbox according to Ben Murphy.

However, if we want to prevent RCE, then we want a general purpose SecurityManager that allows almost everything, but can prevent scripts being run on the host. It wouldn’t be a perfect defense, but it would be a decent part of a defense in depth strategy.

It turns out that someone already did this!

Ondrej Lukas put together pro-grade, which adds a “deny” option as well as an “allow” option to Java policy files. There’s a presentation at Devoxx going over pro-grade, with slides and video.

Now, using pro-grade with the previous example, the following policy would lock down all execution access:

1
2
3
4
5
6
priority "grant";

deny {
  // https://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html#FilePermission
  permission java.io.FilePermission "<<ALL FILES>>", "execute";
};

Note that this is not a complete solution. I suspect you would need to deny several other permissions in addition to this to prevent code from working around this, and I don’t know which ones are relevant for a blacklist. But it’s a start, and it goes a long way towards hardening Java server-side applications very cheaply.

EDIT: Have done research, and you do need to set additional permissions to prevent the SecurityManager itself from being circumvented. Please see this post for the relevant permissions.

Pro-grade is very simple to set up with an appropriate policy. There’s a policy generator that can show all the needed permissions for an application, and a tutorial showing all the steps needed to set it up, and a permissions debugger to catch stray permissions after that.

The example project is available at prograde-example, and pro-grade can be integrated into your project from http://mvnrepository.com/artifact/net.sourceforge.pro-grade/pro-grade/1.1.1. Most people will want Maven:

1
2
3
4
5
<dependency>
  <groupId>net.sourceforge.pro-grade</groupId>
  <artifactId>pro-grade</artifactId>
  <version>1.1.1</version>
</dependency>

The really interesting thing about pro-grade is that it’s a transparent solution. While it’s nice to have a whitelist policy, using this technique you can add pro-grade onto an existing, already compiled project, and deny script execution.

With only minor modifications, pro-grade could be made to notify of violations (especially in setSecurityManager and other seldom touched areas), and appear to work while also silently mangling the operation elsewhere. All you need to do is implement PermissionDeniedListener with an SLF4J implementation.

There’s enough research in attacks, it’s nice to see some progress being made in defense as well.

The Right Way to Use SecureRandom

| Comments

How do you generate a secure random number in JDK 1.8? It depends.

The default:

1
2
3
SecureRandom random = new SecureRandom();
byte[] values = new byte[20];
random.nextBytes(values);

If you’re okay with blocking the thread:

1
2
3
SecureRandom random = SecureRandom.getInstanceStrong();
byte[] values = new byte[20];
random.nextBytes(values);

That’s really it.

Details

The difference between the first use case and the second: the first instance uses /dev/urandom. The second instance uses /dev/random. /dev/random blocks the thread if there isn’t enough randomness available, but /dev/urandom will never block.

Believe it or not, there is no advantage in using /dev/random over /dev/urandom. They use the same pool of randomness under the hood. They are equally secure. If you want to safely generate random numbers, you should use /dev/urandom.

The only time you would want to call /dev/random is when the machine is first booting, and entropy has not yet accumulated. Most systems will save off entropy before shutting down so that some is available when booting, so this is not an issue if you run directly on hardware.

However, it might be an issue if you don’t run directly on hardware. If you are using a container based solution like Docker or CoreOS, you may start off from an initial image, and so may not be able to save state between reboots — additionally, in a multi-tenant container solution, there is only one shared /dev/random which may block horribly. However, the work around in these cases is to seed /dev/random with a userspace solution, either using an entropy server for pollinate, or a CPU time stamp counter for haveged. Either way, by the time the JVM starts, the system’s entropy pool should already be up to the job.

Some people have a cryptographic habit of using /dev/random for seed generation, so there are some cases where it’s easier to use getInstanceStrong just to avoid argument or the hassle of a code review. However, that’s a workaround for a personnel issue, not a cryptographic argument.

How the default works

There is a full list of SecureRandom implementation available, which lists the preferences available for the “default” SecureRandom. For Linux and MacOS, the list is:

  1. NativePRNG** Sun
  2. SHA1PRNG** Sun
  3. NativePRNGBlocking Sun
  4. NativePRNGNonBlocking

There is an asterisk saying “On Solaris, Linux, and OS X, if the entropy gathering device in java.security is set to file:/dev/urandom or file:/dev/random, then NativePRNG is preferred to SHA1PRNG. Otherwise, SHA1PRNG is preferred.”

However, this doesn’t affect the list. When they say “entropy gathering device”, they mean “securerandom.source”, and grepping through java.security shows:

1
2
3
4
$ grep securerandom.source $JAVA_HOME/jre/lib/security/java.security
# specified by the "securerandom.source" Security property.  If an
# "securerandom.source" Security property.
securerandom.source=file:/dev/random

Yep, the line exists, so “NativePRNG” is preferred to “SHA1PRNG”. So, what does that mean? There’s an entry in Standard Names, but there’s also is a more specific note of what each algorithm does in the Sun Providers section:

  • SHA1PRNG (Initial seeding is currently done via a combination of system attributes and the java.security entropy gathering device)
  • NativePRNG (nextBytes() uses /dev/urandom, generateSeed() uses /dev/random)
  • NativePRNGBlocking (nextBytes() and generateSeed() use /dev/random)
  • NativePRNGNonBlocking (nextBytes() and generateSeed() use /dev/urandom)

The nextBytes method is the base method: when you call nextInt or nextLong, etc, it will call down to nextBytes under the hood. The generateSeed method is not needed for a Native PRNG of any type, but it IS useful to seed a user space PRNG such as SHA1PRNG. You can call setSeed on a NativePRNG, and it will use an internal SHA1PRNG that can be “mixed in” with /dev/urandom, but it’s not necessary.

The default SecureRandom, which is the no-args constructor version, will use “NativePRNG” under the hood:

1
2
3
final SecureRandom defaultRandom = new SecureRandom();
System.out.println("defaultRandom algorithm = " + defaultRandom.getAlgorithm());
// prints "defaultRandom algorithm = NativePRNG"

And then you’re golden: you have all the randomness you need, and you will never block.

How getInstanceStrong works

The getInstanceStrong method will call the strongest crypto source available on the system. It does this by looking for the $JAVA_HOME/jre/lib/security/java.security file, and looking up the value in securerandom.strongAlgorithms:

1
securerandom.strongAlgorithms=NativePRNGBlocking:SUN

This means that SecureRandom.getInstanceStrong() is equivalent to SecureRandom.getInstance("NativePRNGBlocking").

Again, /dev/random is no more secure than /dev/urandom.

Debugging

If you want to see what your SecureRandom implementation is doing, then you can turn on debugging:

1
-Djava.security.debug="provider,engine=SecureRandom"

There is a debugging page but this is still undocumented in 1.8. It’s fixed in JDK 1.9.

Using SHA1PRNG

If you’re on Windows, then SHA1PRNG is the preferred algorithm — however, the underlying NativeSeedGenerator called by generateSeed() calls out to the Windows Crypto API. Alternately, you may have to work with a library that explicitly checks for the SecureRandom algorithm of SHA1PRNG, as is mentioned in the code sample for PRNGFixes.

SHA1PRNG is a pure Java implementation which is not as strong as the algorithms used by approved DRBG mechanisms in NIST SP800-90.

If you have a SHA1PRNG, then you can seed it directly:

1
2
3
4
5
6
SecureRandom nativeRandom = SecureRandom.getInstance("NativePRNGNonBlocking"); // assuming Unix
byte[] seed = nativeRandom.generateSeed(55); // NIST SP800-90A suggests 440 bits for SHA1 seed
SecureRandom sha1Random = SecureRandom.getInstance("SHA1PRNG");
sha1Random.setSeed(seed);
byte[] values = new byte[20];
sha1Random.nextBytes(values); // SHA1PRNG, seeded properly

Or, if you just call nextBytes immediately, the very first call will self-seed from /dev/random with 20 bytes… which will block, but what the heck.

1
2
SecureRandom sha1Random = SecureRandom.getInstance("SHA1PRNG");
sha1Random.nextBytes(values); // SHA1PRNG, self seeded with 20 bytes from blocking OS

You can futz with the self-seeding by changing the entropy generator around, but it’s not really worth it unless you have code you can’t change. My recommendation is to use “NativePRNGNonBlocking” and have done with it.

Randomness problems with SHA1PRNG

SHA1PRNG is pseudo random. You can actually measure the amount of randomness in the system, and it turns out that if you run SHA1PRNG with a weak seed, the amount of randomness is traceable.

Yongge Wang and Tony Nicol wrote a fascinating paper On statistical distance based testing of pseudo random sequences and experiments with PHP and Debian OpenSSL — the Java bit is at “8.1. Java SHA1PRNG API based sequences”. They also have a statistical testing package and software at LILTest.

In practice, if you have any doubts about SHA1PRNG, you should use /dev/urandom.

Further Reading

For more of an overview of how SecureRandom works under the hood, please see:

JDK 1.8 made a number of security improvements. SecureRandom was JEP-123, and the developer logs go into sordid history:

Also, the story of Android’s OpenSSL PRNG compromise for bitcoin theft is fascinating:

There is a new version of SecureRandom coming in JDK 1.9, JEP-273, which adds SHA-512 and AES-256 based off NIST SP800-90.

Closing the Open Door of Java Object Serialization

| Comments

TL;DR

This is a long blog post, so please read carefully and all the way through before you come up with objections as to why it’s not so serious. Here’s the short version.

Java Serialization is insecure, and is deeply intertwingled into Java monitoring (JMX) and remoting (RMI). The assumption was that placing JMX/RMI servers behind a firewall was sufficient protection, but attackers use a technique known as pivoting or island hopping to compromise a host and send attacks through an established and trusted channel. SSL/TLS is not a protection against pivoting.

This means that if a compromised host can send a serialized object to your JVM, your JVM could also be compromised, or at least suffer a denial of service attack. And because serialization is so intertwingled with Java, you may be using serialization without realizing it, in an underlying library that you cannot modify.

To combat an attacker who has penetrated or bypassed initial layers of security, you need a technique called defense in depth.

Ideally, you should disable serialization completely using a JVM agent called notsoserial. This will give you a security bulkhead and you can add network monitoring to see if an attacker starts testing ports with serialized objects.

If you can’t disable serialization, then there are options for limiting your exposure until you can remove those dependencies. Please talk to your developers and vendors about using a different serialization format.

The Exploit

If you can communicate with a JVM using Java object serialization using java.io.ObjectInputStream, then you can send a class (technically bytes that cause instantiation of a class already on the classpath) that can execute commands against the OS from inside of the readObject method, and thereby get shell access. Once you have shell access, you can modify the Java server however you feel like.

This is a class of exploit called “deserialization of untrusted data”, aka CWE-502. It’s a class of bug that has been encountered from Python, PHP, and from Rails.

Chris Frohoff and Gabriel Lawrence presented a talk called Marshalling Pickles that talked about some exploits that are possible once you have access to Java object serialization.

Practical Attacks

A blog post by FoxGlove Security took the Marshalling Pickles talk and pointed out that it’s common for application servers to run ports with either RMI, or JMX, a management protocol that runs on top of RMI. An attacker with access to those ports could compromise the JVM.

The proposed fix was to identify all app servers containing commons-collections JAR and remove them.

The problem is, you don’t need to have commons-collection running — there’s a number of different pathways in. The ysoserial tool shows four different ways into the JVM using object serialization, and that’s only with the known libraries.

There are any number of Java libraries which could have viable exploits. It isn’t over.

Matthias Kaiser of Code White is doing further research in Exploiting Java Serialization, and says that more exploits are coming.

So, fixing this particular exploit doesn’t fix the real problem, nor does it explain why it exists.

The Real Problem

The real problem is that “deserialization of untrusted input” happens automatically, at the ObjectInputStream level, when readObject is called.

You need to check untrusted input first before deserializing it, a process called “validation” or “recognition” if you’re up on language security. But the specification is so powerful and complex that there isn’t a good way to securely validate Java serialized objects.

Isolation — only having a management port open inside your “secure” data center — isn’t enough.

Cryptography — using message authentication or encryption — isn’t enough.

Obscurity — hoping that this bug is too obscure to be used by attackers or that your system is beneath their notice — isn’t enough.

User level fixes — subclassing ObjectInputStream with a whitelist or wrapping your code in doPrivileged blocks — aren’t enough.

I’ll break all of this down in detail in the following sections.

EDIT: Charles Miller provides more context.

Why Isolation And Cryptography Aren’t Enough

It’s not a secret in the Java developer community that object serialization can load arbitrary classes. In fact, object serialization is the part of the reason that Java Applets are commonly disabled on browsers.

So, my first response to this exploit was “yes, this is why you don’t publicly expose your management ports to attackers on the Internet.” JMX and RMI are designed to work internally to the data center, and the assumption is that these ports are not publicly exposed. This is a commonly held tenet of server side applications — for example, Redis’s security policy is explicitly “it’s totally insecure to let untrusted clients access the system, please protect it from the outside world yourself.

To this end, developers typically assume that some combination of network isolation, sandboxing, containers and hypervisors will create a secure data center that will prevent attackers from gaining direct access to a port. In the event that ports need to be exposed outside the data center, there are options for RMI over TLS and JMX over TLS.

Rob Rodgers graciously and kindly corrected me on the problem inherent in this approach: it assumes that the attacker only ever attacks from the outside.

Firewalls are typically effective at preventing direct attacks from the outside, but they only cover one avenue of entrance. Rather than penetrate firewalls directly, attackers typically circumvent them by coming from another direction. They’ll use the backdoor — connecting through unsecured laptops that manage payroll, desktop machines, smartphones, etc. If they’re remote, they’ll set up targeted phishing emails that have links that download malware. Or, if they’re local, they’ll try physical solutions — using wifi pineapple to get into the wireless network like TJ Maxx, dropping USB keys in the parking lot and so forth.

This sounds complicated, but to an attacker, it’s about as simple as attaching a debugger to a running process and running through a breakpoint is to a developer. It’s just part of the job.

Most attacks in the data center are from a compromised host inside the firewall. This changes the characteristics of the attacker: rather than a man in the middle, the attacker is now a man on the edge. Once there, they can see all the traffic to and from that machine. They could possibly poke at the DHCP server or the DNS server and start impersonating other clients. They have all the access to credentials that they need.

This is a class of exploit that you don’t typically hear about in enterprise circles, because the whole idea is that there’s an “inside” and an “outside” and the “inside” is secure. It’s just not the case. The reality is that attackers typically manage to compromise one machine inside the corporate firewall, and then leverage that machine to gain access to others.

SSL/TLS doesn’t help here. If you are in a coffee shop and connecting to a data center, TLS will protect you against a “Man in the middle” attack, where the user does not have the private keys necessary to break into the TLS session. If you are inside the data center, then things are different.

TLS will not save you. The calls are coming from inside the house. No combination of MAC, encryption and digital signatures can prevent a compromised host from sending a serialized nastygram.

For the same reason, obfuscation (what is commonly meant by “security by obscurity”) is not a solution, because a compromised host will happy deobfuscate the data.

From the point of view of the attacker, the focus is on making the attack and attack vectors, which is why you don’t typically hear the phrase “compromised host” all that often. Brian Keefer suggest “lateral movement.” Blake Hyde suggests “beachhead.” Ben Tasker suggests “Pivot or Pivoting.”

RMI has long been known to be a juicy target for hackers — it does not work well with firewalls or NAT, which is why many companies will run HTTP proxies through firewalls directly to RMI. There’s a tutorial showing how to use Metasploit to gain access to RMI servers. And because JMX runs on RMI, there’s every reason for operations to enable it and access it on a remote port.

You must assume that your network is insecure and that you may be talking to compromised hosts. You must establish bulkheads, a technique known as defense in depth to prevent the infection from spreading.

Why Obscurity Isn’t Enough

You may be wondering why anyone would bother attacking you with this, especially if the attack is obscure and you don’t think you’re a big target. How real is this? Does anyone actually do this?

The problem is that while there aren’t that many attackers over all, they benefit hugely from automation. Your average attacker is far more likely to be a script kiddie than he is to be Mr. Robot. The problem is that the script kiddies have libraries of every possible exploit, and automation frameworks that will run through every exploit until something works. There’s even a Linux distribution known as Kali Linux, specifically written for systems penetration.

Security firms rely on “pen-testers” to audit companies for these vulnerabilities, and the industry has already added this one. Direct Defense put together a Burp Suite Extender called Super Serial with instructions on how to use it — this locates all serialized objects from the server in the Burp Suite network scanning tool.

The original exploit has already been refined: Trust Foundry has a blog post describing a “one click exploit” and has an executable jar file on Github.

EDIT: As of December 3rd, Impervia reports 645 attacks using this vulnerability from 503 different IPs.

So, relying on obscurity of the bug won’t help: it’s already been packaged and broadcast. And relying on your company’s obscurity… well, attacks are common, cheap, and mostly automated. Maybe you’ll be lucky for a while, but saying you won’t be attacked is like saying you won’t get spam in your inbox. It may be obscure to you, but it’s not obscure to them.

And the end result? Well, look at Sony. Look at Target. Look at TJ Maxx. Look at Fandango and Credit Karma. Security breaches are real, and they have consequences.

The Fallacy of Trusted Input

The larger issue here is the implicit assumption that “untrusted input” implies that there can be trusted input. Really, it should be “unvalidated input.” Trust — actual, no kidding, trust — is rare. You must trust the JVM class loader and your initial configuration files because you can’t start the application without them, but everything after that is runtime input.

Barring actual custom hardware such as a trusted platform module or hardware security module, all runtime input can be compromised. Files can be rewritten. Network input can be spoofed. Anything that read in from I/O is “untrusted” in the sense that it has not been validated.

But even a object that has gone through validation is still untrustworthy: it could ask for things it has no right to, have faked credentials, etc. Trust or “untrust” are besides the point — input should have to jump some hurdles before it may be accepted for processing.

The Ideal Validation Scenario

If you are working in a microservices / domain driven design context, you build validation in an anti corruption layer around your bounded context (essentially a fancy word for “your app up to the point it has to work with I/O”). Everything outside the anti corruption layer is unvalidated. Everything inside the boundary is validated. This matches up well with language security principles. Here, we want full recognition before processing.

To this end, you use strongly typed objects called Value Objects to represent your validated input. As an example, the input to an “address service” might be “1 Market St, San Francisco CA 94111”. This is a String — a raw, unvalidated, out of the box type. Raw types are broken. What you want is an Address — this means that you want the Address parser to validate and create a series of AddressLine, City and ZipCode objects. You never want to expose a raw type like String in your domain. You especially don’t want to take raw types in your public APIs. If all your methods only use types created from validation, then you can limit your exposure. (Java suffers from not having value types, but they’re showing up in JDK 1.9, finally.)

This is the ideal. In theory, you can take the raw bytes of every object that you received, parse and validate it, and only return an object after validation, instead of having ObjectInputStream create an object out of the blue.

Why Validation Is Hard

The reality is not so fun. ObjectInputStream will let you hook into it, and subclass it. The tough part is validating the input given that it’s happening from inside ObjectInputStream.

Full on validation… turns out to be tricky. Very tricky. Sami Koivu has a couple of great blog posts in 2010(!) on why complex+powerful is a bad combination for security and breaking defensive serialization — I won’t go into great detail, but the problem is that the serialization logic is so complex and powerful that “secure validation while deserializing is extremely difficult” and (according to Koivu) not only does the CERT guide not get it right, but even Josh Bloch doesn’t get it right.

The Look-ahead Java deserialization solution that has been frequently mentioned suffers from this — the validation it suggests is a whitelist inside of ObjectInputStream.resolveClass — at this point you’ve already taken a bite out of the apple. Unless you are extraordinarily careful, even seemingly harmless whitelisted classes that are used in serialized objects can be used in an attack.

Why Whitelisting Isn’t Enough

Whitelisting for allowed classes can prevent unknown classes from being called, but it can’t prevent pathological classes from being deserialized. Wouter Coekaerts shows a denial of service attack using nested HashSets that would not be caught by whitelisting. Likewise, if you whitelist java.net.URL, you can’t stop a specially constructed object doing a series of blocking network lookups based off the hashCode method.

Nor can whitelisting stop exploitation of underlying serialization bugs. It’s a big, hairy piece of code, and there’s no real way to slice off a piece of it.

You can, of course, implement your own custom parser and ignore ObjectInputStream completely. That will work for simple data objects, and you can throw away anything you don’t recognize. That’s a lot of work though, and of course the more complex the parser, the more likely it is to have bugs.

Why User Level Solutions Aren’t Enough

You’re probably thinking all is not lost.

You’ll disable RMI and JMX ports, and move to JVM agent based solutions like Tapiki or Hyperic Sigar. Failing that, you’ll install Jolokia or jmxtrans and try to limit your exposure.

You’ll move to JSON (the lowest common denominator) or a faster, smaller, language independent binary protocol that does validation and schema resolution: something like Protocol Buffers / Capn Proto / Thrift / Avro. Or, if you’re going to stick with Java, you’ll use Kryo, although in that case I suggest adding Chill and having setRegistrationRequired turned on.

Then, you would carefully read the secure coding guidelines and follow all of the recommendations in your code, and you’d be fine.

The problem is that no matter what you do in your own code, you can’t be sure that some library code, somewhere, isn’t ignoring that completely and using a raw ObjectInputStream anyway. Github shows 678,489 instances of ObjectInputStream in various projects. JMX and RMI are only one vector — unless you decompile all your libraries and check them, you don’t know that one of the libraries in your framework isn’t using deserialization for a disk storage format (as was common in 200x), or for session backup, or for a distributed caching scheme.

Even then — even if you scan all of your code and can verify that ObjectInputStream.readObject is never called anywhere within your codebase — you still could be vulnerable. If you have any code that leans heavily on reflection (which happens more often than not in Java frameworks), then you can still have ObjectInputStream instantiated, although it will take more work to feed it the right bytes and call readObject. And I don’t know what the impact is to sandboxed user supplied code that is supposed to run in a custom classloader, but I bet everyone will be reviewing code for a while.

EDIT: SRC:CLR has a list of 41 libraries that reference Apache Commons Collections and perform serialization, although they are careful to note they have not proved untrusted deserialization in all these libraries.

EDIT: Sijmen Ruwhof discusses scanning an enterprise organisation for the critical Java deserialization vulnerability. This is a useful way to find what systems have serialization exposed on their ports — either look for well known RMI / JMX ports and scan for the hexidecimal string “AC ED 00 05” or “rO0” — but it should be noted that this still only deals with known ports and vulnerable classes, and suggests depreciation of Java serializable objects long term.

Why SecurityManager Isn’t Enough

There is something that is supposed to manage the JVM at a system level — SecurityManager. There are two problems with this: SecurityManager is both too strict and too permissive.

The system security manager has to be specifically enabled with -Djava.security.manager, and is a blunt instrument that causes lots of things to stop working everywhere. But SecurityManager doesn’t limit ObjectInputStream in any significant way: you can limit implementations of subclasses of ObjectInputStream, but you can’t make it safe or turn it off.

You can create a custom SecurityManager with your own permissions and set it on a thread, but you still have the same problem: any custom checks will apply to your user level subclasses of ObjectInputStream and any libraries can use the base ObjectInputStream with no restrictions.

Hacking the JVM

The ideal solution is to override ObjectInputStream itself, which will fix object serialization everywhere in the JVM.

My previous solution was to override the bootclasspath with a custom implementation of ObjectInputStream, but this contravenes the Java binary license. There is a precedence for using this method for fixing Sun’s XML class bugs, but there is a better way.

The better way to do this, without violating the Java binary license, is to use Eirik Bjørsnøs’s notsoserial project, a Java agent that will hook into the JVM and prevent and/or control deserialization everywhere.

Your Best Option

Your best option is to turn off object serialization completely, everywhere in the JVM, for good. This means no RMI, and no JMX, but there are other options.

Use notsoserial with nothing in the whitelist.

Your Second Best Option

In the event that you can’t turn it off — find out what you can to squeeze the problem down until you can turn it off completely. If you do have reasons to have it on, then make sure that everything is logged, tracked and locked down as much as possible.

So, use notsoserial with whitelisting, by tracing your serialization. Keep network communication locked down with TLS client certificates, and investigate Jolokia or jmxtrans if you can.

Also, talk to your operations team about using Burp Suite or Haka to identify Java serialization across the network.

This might be a pain if you are using Java Mission Control / Java Flight Recorder in production.

Your Third Best Option

If you’re writing or working with a library that requires object serialization and you don’t have the option to control your clients or your endpoints, then you can work with a subclass of ObjectInputStream.

Take a look at ValidatingObjectInputStream or SerialKiller for an ObjectInputStream replacement.

Java’s best option

The best option is for Java itself to remove serialization. Unfortunately, the last attempt JEP 154 was an April Fool’s joke. There have been discussions to add an explicit serialization API, but I don’t know of any current movement.

Another route that Oracle could take is to enhance the SecurityManager to allow more fine-grained control of serialization. This would work when the SecurityManager was enabled, but not break any existing classes or code otherwise.

Videos!

Wait, there’s more!

Just because you’re done with Java object serialization using ObjectInputStream doesn’t mean that you’re done: you can deserialize directly to classes using XMLEncoder as well!

Fortunately, now you get why deserialization is bad, you can use the same techniques shown here to validate or disable XMLEncoder in the same way. The point here is that you need to think about all the inputs to your system, and ask at each point if there is a gatekeeper. If you don’t have validation built in, you’re allowing an open door into your system.