Trac

ing

use cases

Execution time

system statuscurrent environment

parameters

issue recognition

progress

stackholders

developers

production timesystem engineers

managers

users

testers

lawyers DevOps

requirements

easy to use APIstackholder specific logs

readable logs

completenessperformance

client specific logging (UI, console, build job)

reliablecorrectness

use case specific logging (production,testing, debugging)

historic data

relevant logs

What to log

• reporter (who discovered)

• affected element

• severity

• message

• issue code

• additional data (e.g. exception object)

Log levels

• built-in: OFF, TRACE, DEBUG, INFO, WARN, ERROR, FATAL• Custom log levels• no level inheritance (as in log4j 1.x and logback), because

separation of logger configuration and logger

Log levels (2)<Configuration … status="INFO">

<Loggers>

<Root level="INFO">

<AppenderRef ref="SYSERROUT" />

</Root>

<Logger name="org.apache.logging.log4j2" level="INFO" />

<Logger name="my.package.MyIgnoreClass" level="OFF" />

</Loggers>

</Configuration>

Appenders<?xml version="1.0" encoding="UTF-8"?> <Configuration …>

<Appenders> <Console name="STDOUT" target="SYSTEM_OUT"> …</Console>

…</Appenders><Loggers>

<Logger name="my.package"> <appender-ref ref="STDOUT" /> </Logger>

</Loggers></Configuration>

Appenders

• ConsoleAppender• FileAppender

– RollingFileAppender– RandomAccessFileAppender

• Messaging– JMSAppender– SMTPAppender

• DB– JDBCAppender– JPAAppender– NoSQLAppender

• Remote– SocketAppender– SyslogAppender

Appenders

• Wrapper– FailoverAppender– AsyncAppender

<Async name="Async">

<AppenderRef ref="MyFile"/>

</Async>– RoutingAppender .. explained later with MDC– RewriteAppender .. explained later with Message API

FailoverAppender<Failover name="Failover" primary="RollingFile">

<Failovers>

<AppenderRef ref="Console"/>

</Failovers>

</Failover>

Layout

• PatternLayout

• HTMLLayout

• XMLLayout

• JSONLayout

• SerializedLayout

Pattern Layout<PatternLayout

pattern= " %date{dd.MM.yyyy HH:mm:ss,SSS} %5p %logger %m%n" />

%5p (oder %level) .. log level, maximal 5 Buchstaben, rechtbündig%logger .. logger name%c .. class name

%m .. log message%n .. platform dependent line break%M .. class method

Pattern Layout (2)

• %t .. Name of the thread• xException["none"|"short"|"full"|depth],[filters(packages)}• xThrowable["none"|"short"|"full"|depth],[filters(packages)}• xEx{"none"|"short"|"full"|depth],[filters(packages)}

– %xEx{short} .. Nur erste Zeile des Throwable– %xEx{n} .. n Zeilen

– %xEx{none} oder %xEx{0} .. unterdrücken • MDC{key} später erklärt

Properties<?xml version="1.0" encoding="UTF-8"?><Configuration …>

<Properties><Property name="defaultDatePattern">

%date{dd.MM.yyyy HH:mm:ss,SSS} </Property>

<Property name="defaultLoggerPattern"> %5p %logger %marker %m%n </Property>

<Property name="defaultLogPattern"> ${defaultDatePattern} ${defaultLoggerPattern} </Property>

</Properties> … </Configuration>

Nested Diagnostic Context (NDC)

• stack• context is stored per thread• use push and pop to add and remove information

NDC.push("processingLevel2");

log.info("success");

log4j.appender.CA.layout.ConversionPattern

=%d{yyyy-MM-dd HH:mm:ss.SSSS} %p %C %x = %m%n

%x .. Accessing information

Mapped Diagnostic Context (MDC)

• context is stored per thread• MDC.put(), MDC.get(), and MDC.remove()

MDC.put("userIP", req.getRemoteAddr());

MDC.put("userName", foo.getName());

log.info("success");

log4j.appender.CA.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss.SSSS} %p %X{userIP} %C %X{userName} = %m%n

%X{userName} .. retrieves value for key "userName"

Routing Appender<Console name="SYSERR"> … </Console>

<Routing name="ConsoleAppenderRouteIssueCode">

<Routes pattern="${ctx:issueCode}">

<Route>

<Console name="ConsoleAppenderWithIssueCode">

<PatternLayout pattern="${ctx:issueCode} -

${defaultLoggerPattern}"/>

</Console>

</Route>

<Route ref="SYSERR" key="${ctx:issueCode}" />

</Routes>

</Routing>

…

<AppenderRef ref="ConsoleAppenderRoute">

Lookup● $${ctx:myThreadContextPropertyKey}● ${ctx:myThreadMapContextPropertyKey}● ${sys:mySystemPropertyKey}● ${map:myMapMessageKey}● $${date:MM-dd-yyyy}● $${env:USER}● ${java:runtime}● $${jndi:logging/context-name}● define custom lookup plugins

Markersprivate static final Marker SQL_MARKER =

MarkerManager.getMarker("SQL");

private static final Marker UPDATE_MARKER = MarkerManager.getMarker("SQL_UPDATE", SQL_MARKER);

private static final Marker QUERY_MARKER = MarkerManager.getMarker("SQL_QUERY", SQL_MARKER);

…

logger.debug(QUERY_MARKER, "SELECT * FROM {}", table);

Filters

BurstFilter .. throttle logging when too many logs produced

ThresholdFilter .. filters by log level ranges

RegexFilter .. messages have to match regex

MarkerFilter .. log event filter by marker resp. its sub type

TimeFilter .. log only at certain times of day

MapFilter .. filters by entries in MapMessage

StructuredDataFilter .. special MapFilter

DynamicThresholdFilter .. z.B. debug only for user id X

ThreadContextMapFilter .. ThreadContext Map entries

CompositeFilter .. apply multiple filters

Regex Filter<?xml version="1.0" encoding="ASCII"?>

<Configuration name=„MyLogger" …>

…

<Filters>

<RegexFilter regex=".*stringToMatch.*" onMatch="DENY"

onMismatch="NEUTRAL"/>

</Filters>

…

</Configuration>

Thread Context Map Filter<Loggers>

…

<Root level="error">

<AppenderRef ref="RollingFile"/>

<ThreadContextMapFilter onMatch="ACCEPT"

onMismatch="NEUTRAL" operator="or">

<KeyValuePair key="foo" value="bar"/>

<KeyValuePair key="User2" value="WARN"/> </ThreadContextMapFilter>

</Root>

</Loggers>

Message API

• pass built-in or custom message objects• org.apache.logging.log4j.message

– ParameterizedMessage– MapMessage– SimpleMessage– ObjectMessage– Message

logger.info(new LoggedInMessage(map, user));

RewriteLogger logger = LogManager.getLogger(MyClass.class);

Map<String, String> msg = new HashMap<String, String>();

msg.put("creditcard", "123456789");

logger.info(new MapMessage(msg));

Rewrite (2)<Rewrite name="rewrite">

<AppenderRef ref="myweb"/>

<MapRewritePolicy mode="Update">

<KeyValuePair key="creditcard" value="XXXXXXXXXX"/>

</MapRewritePolicy>

</Rewrite>

<Logger …>

<Appender-Ref ref="rewrite" />

</Logger>

source: https://gomtiprabha.wordpress.com/2014/03/11/rewrite-appender-in-log4j2/

Flow tracingpublic String method(String input) {

logger.entry(input);

…

return logger.exit();

}

package my.custom.log4j2.plugins;

@Plugin(name = "Sandbox", type = "Core", elementType = "appender")

public class SandboxAppender extends AppenderBase {

private SandboxAppender(String name, Filter filter) {

super(name, filter, null);

}

public void append(LogEvent event) {

System.out.println(event.getMessage().getFormattedMessage());

}

@PluginFactory

public static SandboxAppender createAppender(

@PluginAttr("name") String name,

@PluginElement("filters") Filter filter) {

return new SandboxAppender(name, filter);

}

}

Plug-in infrastructure

Plug-in infrastructure (2)<configuration … packages="my.custom.log4j2.plugins">

…

<appenders>

<Sandbox />

</appender>

…

</configuration>

Reconfigure <?xml version="1.0" encoding="UTF-8"?>

<configuration monitorInterval="30">

...

</configuration>

• monitoring interval is a value in seconds• log4j 2 would reconfigure logging in case something has

changed in your configuration• log4j 2.0 does not lose logging events at the time of

reconfiguration

XInclude<?xml version="1.0" encoding="UTF-8"?>

<configuration …>

<ThresholdFilter level="debug"/>

<xi:include href="log4j-xinclude-appenders.xml" />

<xi:include href="log4j-xinclude-loggers.xml" />

</configuration>

Performance

source: https://logging.apache.org/log4j/2.x/manual/async.html

Logger configuration

• Centralize

• different context selectors– ThreadLocal– Classloader– JNDI– AsyncLogger

• Make all loggers async– Bundle

OSGi

• Configure BundleContextSelector– associates LoggerContexts with the ClassLoader of the

bundle that created the caller of the getLogger call

• Own logger API and central logger config– Derive Eclipse plug-in project from log4j2 jars– Define fragment project to define your own logger API

• In MANIFEST set – Eclipse-ExtensibleAPI: true

Programmatic access-Dlog4j.configurationFile= ...

LoggerContext ctx = (LoggerContext) LogManager.getContext(false);

Configuration config = ctx.getConfiguration();

LoggerConfig loggerConfig =

config.getLoggerConfig(LogManager.ROOT_LOGGER_NAME);

loggerConfig.setLevel(level);

ctx.updateLoggers(); // loggers refetch information from their LoggerConfig

Bad practice

• logging errors that has been already handled by application code (e.g. failed login)

• log messages with line breaks

• non unified formatted log messages (no tabs)

• long log messages (can be avoid by shorten the package name)

• in tests use logging where assertions would have been a better choice (as they really enforce expected behavior where logs can be ignored)

• not logging exception objects

• using custom appenders holding state

System.out.println(...)

System.err.println(...)

exception.printStackTrace()

FileWriter writer = ...;writer.write(...);writer.close();

LOGGER.error(...)

Bad practice (2)

Bad practice (3)

• use timestamps for every event

• log thread name

• log source of event (e.g. class name and message)

• log ids from domain (e.g. transaction id)

• concise and descriptive messages

• use rolling file appender (to restrict single log file size)

• log method inputs and output at debug resp. trace level

• log full exception objects

• log no more than necessary in production

Good practice

Good practice (2)

Log4J2 + Xtend Active Annotations

@Delegate (built-in)to derive custom loggers with a few customized methods

@Logto initialize instance log field with correct class as parameter

@TraceableTo annotate methods, so that logger.entry and logger.exit with input and output parameters are generated

Links

• Log4j2

• Apache Log4j 2.0 - Worth the Upgrade?

• The new log4j 2.0

• The Logging Olympics – A Race Between Today’s Top 5 Java Logging Frameworks