There's several different things going on in TLS, and the way that Java handles it with JSSE (Java Secure Socket Extension) is involved. The last post was all about working with
KeyStore. This post is all about the key manager.
A key manager is how TLS presents a certificate chain to a peer, and decrypts information using the private key associated with the certificate at the end of that chain. Think of it as a "private key manager" and the name makes more sense.
A key manager needs a source of private keys and certificate chains. This is provided by a
KeyManagerFactory, which provides a key manager with its source material. Once the key manager has been instantiated, it is effectively immutable: you can't add new private keys to it or alter its behavior. You also can't retrieve private key information from the key manager at all.
There are a very small number of private keys in the key manager – usually only one, which is the hostname. Private keys can have a password. JSSE assumes that a password is required for the private key, even if it is the empty string, so you can't pass
null in as a value, but oddly, there's no way to pass a password to a specific private key in a KeyManager. Instead, passwords are associated with key stores, and the password of a key is assumed to be the same as the password of the keystore. This has implications for the key manager API during initialization.
There are two implementations of
KeyManagerFactory, which go by "SunX509", which is the default value returned by
KeyManagerFactory.getDefaultAlgorithm(), and "NewSunX509", the new one. We'll show the
The SunX509 Key Manager
The "SunX509" key manager is the default. It is backed by the
sun.security.ssl.SunX509KeyManagerImpl class. The javadoc for the implementation spells out its behavior fairly clearly:
The backing KeyStore is inspected when this object is constructed. All key entries containing a PrivateKey and a non-empty chain of X509Certificate are then copied into an internal store. This means that subsequent modifications of the KeyStore have no effect on the X509KeyManagerImpl object. Note that this class assumes that all keys are protected by the same password.
The JSSE handshake code currently calls into this class via chooseClientAlias() and chooseServerAlias() to find the certificates to use. As implemented here, both always return the first alias returned by getClientAliases() and getServerAliases(). In turn, these methods are implemented by calling getAliases(), which performs the actual lookup.
Note that this class currently implements no checking of the local certificates. In particular, it is not guaranteed that:
- the certificates are within their validity period and not revoked
- the signatures verify
- they form a PKIX compliant chain.
- the certificate extensions allow the certificate to be used for the desired purpose.
Chains that fail any of these criteria will probably be rejected by the remote peer.
The default "SunX509" key manager takes one keystore, ideally consisting of a few
KeyStore.PrivateKeyEntry, and one password:
You then get the key manager, which is always the first element in the array. The public API is
The JSSE documentation says:
Note: A KeyManagerFactory implementation for the SunX509 algorithm is supplied by the SunJSSE provider. The KeyManager that it specifies is a javax.net.ssl.X509KeyManager implementation.
This is both true and useless – in Java 1.8, you need to use an instance of
X509ExtendedKeyManager to do anything useful. We'll get into this a bit more later.
The NewSunX509 Key Manager
The "NewSunX509" key manager is backed by X509KeyManagerImpl implementation. This is not the default key manager, but it is a bit smarter about how it selects out the keystore alias.
is central to the "NewSunX509" implementation, because it lets you use multiple private keys, and multiple keystores:
Why would a key manager have multiple private keys?
The answer is in Multiple and Dynamic Keystores:
If multiple certificates are available, it attempts to pick a certificate with the appropriate key usage and prefers valid to expired certificates.
An X.509 end entity certificate may have a key usage extension in the certificate. If "KeyUsage" is present, then it must equal "digitalSignature" for a client certificate, or "keyEncipherment" for the server certificate. Because a key manager is also used in the case where a server makes a Certificate Request to the client, there may be instances where the default SSLContext is being used in both a server and a client context and so must hand out different EE certificates in that case. The CheckResult code goes into detail on this if you're interested.
Another possible use case is that the key manager may be responsible for managing multiple hosts. This is more likely than it sounds, because TLS supports wildcard certificates, and so a key manager may implement Server Name Indication to return different certificates for different hostnames served from the same IP address. We'll describe this later in another section.
There are some things that the "NewSunX509" key manager promises that cannot be fulfilled without some significant customization and some spelunking of the source code.
From the Java documentation:
- it is based around the KeyStore.Builder API. This allows it to use other forms of KeyStore protection or password input (e.g. a CallbackHandler) or to have keys within one KeyStore protected by different keys.
- it can use multiple KeyStores at the same time.
- it is explicitly designed to accommodate KeyStores that change over the lifetime of the process.
- it makes an effort to choose the key that matches best, i.e. one that is not expired and has the appropriate certificate extensions.
What it means by "KeyStores that change over the lifetime of the process" is that it support PKCS11 devices as keystores. Specifically, you can have a smartcard / Yubikey, and swap out the smartcard while the keystore is running – the keystore is the same, but the private key entry points to a different device. You can't effectively swap out a PKCS12 or JKS keystore, even if you change the file that those keys were loaded from, as the implementation will cache entries internally. Likewise, the
KeyStore.Builder will always return the same memoized keystore instance once it has been built. If you want to swap out a private key or certificate chain, you're going to have to write a KeyStore provider like the PKCS11 provider.
What it means by "keys within one KeyStore protected by different keys"… it means "protected by different passwords", but that isn't the case: if you use the KeyStore.Builder API with a PKCS12 file, you're still matching a keystore with a
PasswordProtection, and there is no way to match a password to a specific alias. If the keystore and all private key entries in the keystore have the same password, then everything works fine, but you still can't differentiate inside of a keystore.
Technically, the key manager does use
getProtectionParameter(alias) correctly, but the KeyStore.Builder doesn't pass the alias parameter along in either FileBuilder or the anonymous inner class of newInstance:
The end result is that even though the "NewSunX509" key manager works fine, you still can't use different passwords out of the box with the keystore builder API.
Another really strange thing about the "NewSunX509" key manager is that it prefixes aliases with numbers, so it's no longer the same alias as used in the keystore. This isn't listed in the code or even as javadoc – instead, it's a line comment. Given that the original alias name is still included, it's not terrible, but it does make it annoying when trying to match aliases in a keystore against the alias chosen by the key manager.
Add these two problems together, and you have to write a custom keystore builder with alias parsing code to get alias specific passwords to work:
Which is… bizarre. There's no mention of this in the documentation, but you'd think if you made a point of touting alias-specific passwords that you'd have a
KeyStore.Builder.newInstance factory method that took a password map to make it possible.
Using Domain Keystore
You can also use a domain keystore if you're not into the KeyStore.Builder API. The Domain KeyStore lets you aggregate several keystores together, meaning that a single keystore can be used for everything. The
sun.security.provider.DomainKeyStore implementation has more details and associated tests.
With some setting up of keystores:
Again, note that passwords are matched specifically to keystores, and not to the aliases inside of the keystores. I've always found it easiest to use
"".toCharArray() as the password everywhere, but the incoherence of password management still nags.
I also don't get why the DomainKeyStore configuration is tied directly to a text format. Ideally, you want to do configuration through plain POJO configuration objects, and then the parser just does the work of going from text to config objects. This gives you the flexibility to parse out in different formats (XML, JSON, natch) while making it easier to do type-safe configuration and unit tests based on sticking config objects together.
Since everything is handled at the KeyStore level, you can initialize the key manager factory with just a plain store and a null password on both "SunX509" and "NewSunX509":
This takes a bit more work to set up, but lets you switch and hide the logic better. However, because you can't pass in a KeyStore.Builder directly to a domain keystore, you also don't have access to alias specific passwords here at all.
Using PKCS11 with a KeyManager
Support for PKCS11 devices in Java is a neat feature, because in theory it means that private key material is never exposed in memory on the server. It's not always the case in practice, but PKCS11 is more secure than keeping private keys on your filesystem.
I haven't seen anyone really go through this end to end, so here's a quick description.
First, install OpenSC:
Then set up the OpenSC file
Import the private key store into the Yubikey from PKCS12:
Check that you can see it through keytool:
Once you're able to see it from the command line, you register the provider:
with the following in the DKS file:
And then you can fake out the callback handler with a password from somewhere else.
If you run into problems, add
-Djava.security.debug=sunpkcs11 to see what's going on internally.
Creating your own Key Manager
From time to time, you'll need to create your own key manager. Maybe you'll want to implement SNI, in which case Graham Edgecombe has an SniKeyManager. Or you'll want to set up a debugging proxy, in which case I recommend DebugX509ExtendedKeyManager. Or you may want to set up a simple AliasedX509ExtendedKeyManager. Or you may want to watch the filesystem for changes in the keystore, and reload the private keys with FileWatchingX509ExtendedKeyManager, or choose a certificate based off the alias.
There is no practical way to do custom development in JSSE without access to the underlying source and a debugger.
Here's what you need to set up your environment.
- Download an OpenJDK release from https://adoptopenjdk.net/.
- Clone the source code from AdoptOpenJDK github repository.
- Download IntelliJ IDEA, add the OpenJDK release and set the sources to point to
/jdk/src/share/classesas this is where all the Sun-specific classes are.
- Create a project using the OpenJDK release.
- Create your class extending the
- Write tests against your class.
- Use Ctrl-N to open up Sun-Specific classes and add breakpoints.
- Debug the test, and see what's going on under the hood.
You must extend
X509ExtendedKeyManager, which means you're locked into an abstract base class from the get-go. There's nothing you can do about this.
The JSSE documentation does attempt to point in this direction, saying:
If a key manager is not an instance of the X509ExtendedKeyManager class, then it will not work with the SSLEngine class.
For JSSE providers and key manager implementations, the X509ExtendedKeyManager class is highly recommended over the legacy X509KeyManager interface.
The documentation does not mention that if you create your own
X509ExtendedKeyManager, you must override the following methods for your specific implementation:
This is because
X509ExtendedKeyManager, instead of leaving those methods abstract, returns null.
Both the SunX509 implementation and the NewSunX509 implementation override this with their own logic, and it gets called in the normal code flow, so returning null is a useless behavior here, one that trips up everyone.
The key manager is complicated, but it is not magic. Most of the handshake and private key work is done internally in the handshake, and most customization is in choosing what private key entry to pick from what source. When in doubt, look at the source code, and then run it through a debugger and it'll start making sense.