TL;DR: Filters are used to determine whether an appender should log an event or not. Many loggers will log at INFO level far too often, and cause excessive logging. An filter using a custom evaluator can be used to create a budget for logging at different levels, and throttle logs down to a reasonable level.
This is part of the series of blog posts:
- Application Logging in Java: Creating a Logging Framework
- Application Logging in Java: Adding Configuration
- Application Logging in Java: Converters
- Application Logging in Java: Markers
- Application Logging in Java: Appenders
- Application Logging in Java: Logging Costs
- Application Logging in Java: Encoders
- Application Logging in Java: Tracing 3rd Party Code
- Application Logging in Java: Filters
- Application Logging in Java: Putting it all together
The design of Logback starts with the idea of generating a logging event first, and then sending that event to multiple appenders. However, appenders do have a check on what they can log; only an event that makes its way through the appender's filters can be logged. You may have no filters at all writing to the filesystem, but if you're appending to a centralized logger, you may filter by threshold level, or check for specific markers (which requires some custom code).
The logic for filters is interesting, because it it's not binary. Instead, filters return ACCEPT, DENY, or NEUTRAL. Neutral means that no decision is made here, and so you can run several filters together. This is useful when several appenders share filter logic and build up lists of filters in the appender, but it also means that if you're indicating a binary state, you may not want it to go into a filter directly.
The Logging Costs blog post discussed the expense of storing logs and the costs of excessive logging, and vaguely mentioned budget aware appenders as a solution to deal with excessive logging. Let's dig into that.
The threshold and level filters don't prevent excessive logging, per se. They don't keep track of the number of logging events that pass through the appender, or the amount of time between resets. However, circuit breakers are well known as a solution to limit excessive number of calls, and can be configured on an event count over time. If the circuit breaker is closed, then return true. If the circuit breaker is open, then return false.
It's in terse-logback in the
The nice thing about using an evaluator together with a filter is that it gives more flexibility about what to do when the circuit breaker is open.
For, example, you can prevent loggers from spamming at INFO level, by setting
OnMismatch to DENY and
OnMatch to NEUTRAL. This is useful for slimming down the logs while still getting occasional heartbeat and "processing packet" messages from services.
Or you can say "I only want to hear something from this appender if there's LOTS of output". This makes sense for high volume services that may have brief network partitions that resolve themselves very quickly.
This works very well with audio appenders, which you don't want going off several thousand times a second.
And that's how you write a budget aware appender.