Last Updated on February 15th, 2024

When it comes to implementing Logging feature in our application, especially in Spring Boot based applications, Logback comes into the picture. Typically, logback configuration comes in the form of XML file (logback.xml) for the developers to use. It provides a powerful, responsive, and reliable solution in configuring the application’s logging. It ensures consistent and customizable logging behavior, simplify maintenance, and offers dynamic adjustments. All of them are crucial for effective application monitoring, debugging, and troubleshooting.

Logback is an open-source project with a strong engagement to reliability and robustness. Logback offers a broad range of configuration options through logback.xml, allowing us to customize our logging setup to accurately match our application’s requirement. We can control log levels, formats, destinations, and filtering rules. It is widely used in production environments by organizations of all sizes, verifying to its stability and reliability. In this article, we will talk about logback xml configuration examples and its related fundamental concepts.

What is Logback?

Logback is an open-source Java-based logging framework that acts as a useful tool for dealing with application logging. It makes application logging efficient, expandable, and easy to configure. It becomes a popular choice for Java developers because of its flexibility and robust features for logging application events, errors, and information. In a nutshell, Logback offers Java developers to manage and customize application logging efficiently.

To utilize the full power of Logback, we need to configure it correctly through the logback configuration which is known as logback.xml file.

What are the key features and aspects of Logback?

Let’s understand some primary concepts in Logback configuration:

1) Logging Levels: Logback supports various logging levels, including TRACE, DEBUG, INFO, WARN, and ERROR. This allows developers to control the expansiveness of log output, making it easier to focus on specific issues during debugging or monitoring.

2) Appenders: Logback supports multiple types of appenders, which determine where log messages need to be written. Common appenders include console appenders (logging to the console), file appenders (logging to files), rolling file appenders (rotating log files based on size or time), SMTP appenders (sending log messages via email), and more.

3) Layouts: Logback provides customizable layouts that define how log messages are formatted before being sent to an appender. We can customize log message formats to fulfil our specific requirements.

4) Filtering: Logback offers powerful filtering capabilities, allowing developers to fine-tune which log messages get processed based on criteria like log level, logger name, or message content.

5) Asynchronous Logging: Logback supports asynchronous logging, which dumps the process of writing log messages to a separate thread. This can enhance application performance, particularly in scenarios where logging might be resource-intensive.

6) Contextual Logging: Logback supports MDC (Mapped Diagnostic Context) and NDC (Nested Diagnostic Context), enabling the inclusion of context-specific information, such as user IDs or request IDs, in log messages. This feature is valuable for tracking in multi-threaded or distributed applications.

7) Configuration: Logback can be configured through XML configuration files, providing developers with fine-grained control over logging behavior. This allows for dynamic configuration changes without the need for application restarts.

8) Integration: Logback integrates seamlessly with various Java-based frameworks and libraries. It’s often the logging framework of choice in the Spring based applications.

9) Performance: Logback is known for its performance optimizations, making it suitable for high-throughput applications where efficient logging is critical.

What is Logback Configuration?

Logback configuration represents the process of setting up and customizing the behavior of the logging framework in an application. It includes specifying various parameters, such as log levels, log message formats, output destinations, and filtering rules, in order to generate, process, display and store the log messages.

How to Configure logback.xml in a Java/Spring Boot Application?

The primary mechanism for configuring Logback is through the use of a configuration file, typically named ‘logback.xml’ or ‘logback-spring.xml’. This configuration file is placed on the application’s classpath or in a specific directory, allowing Logback to read and apply the settings when the application starts.

In a Spring Boot application, we can place it under ‘src/main/resources’. You may also check a separate article on ‘Logging in Spring Boot’ to get to know how we can implement logging in Spring Boot Application.

What are the different components of logback.xml?

The logback.xml is a configuration file in Logback that is used to specify and configure various components that control the behavior of the logging framework. These components work together to determine how log messages are processed and where they are sent. Below are the different commonly found components in a logback.xml configuration:

1) <configuration>: This is the root element of the logback.xml file and encloses all other Logback configuration elements. It can include global settings for the entire logging context.

2) <appender>: The <appender> element defines individual appenders, which specify where log messages should be sent. Logback supports various types of appenders, such as <consoleAppender>, <fileAppender>, <rollingFileAppender>, and custom appenders. Each appender can have its own configuration settings.

3) <logger>: The <logger> element is used to configure loggers, which are responsible for generating log messages. Loggers are typically associated with specific parts of our application’s codebase. We can configure the log level, appender references, and optional filters for each logger.

4) <root>: The <root> element is a special logger that serves as the root of the logger hierarchy. It captures log messages that do not match any other logger’s criteria. It sets the default log level and can reference one or more appenders.

5) <filter>: Filters are used within appenders and loggers to selectively process log messages. Filters can be based on criteria such as log level, logger name, or message content. They allow us to control which log events are passed to a specific appender or logger.

6) <layout>: The <layout> element determines the format of log messages before they are sent to an appender. It specifies how log entries are structured, including timestamps, log levels, and the actual log message. Common layout options include <patternLayout>, <jsonLayout>, and custom layouts.

7) <encoder>: An alternative to specifying layouts directly within appenders, the <encoder> element can be used to configure encoding of log messages. Encoders are often used with asynchronous appenders. They define how log data is serialized for output.

8) <property>: Properties are used to define reusable values or variables within the configuration. They can be referenced elsewhere in the configuration to provide flexibility and avoid redundancy. For example, we can define a log file path as a property and use it in multiple appenders.

9) <include>: The <include> element allows you to include external configuration files within the logback.xml. This is useful for breaking down complex configurations into manageable pieces or for sharing common configuration across multiple applications.

10) <jmxConfigurator>: Logback supports JMX (Java Management Extensions) for dynamic configuration changes at runtime. The <jmxConfigurator> element enables JMX-based configuration management.

11) Comments: Comments can be added to the configuration file using <!– –>. Comments help document and explain different parts of the configuration for clarity and maintenance.

These components work together to define how log messages are collected, processed, and output in a Logback-based logging system. A well-structured logback.xml configuration is essential for successful logging and monitoring of Java applications.

What are Appenders in Logback xml?

Appenders are fundamental components of the Logback logging framework. They specify how the log messages are processed and handed over. Below is a short note on appenders in Logback:

1) Purpose: Appenders in Logback are accountable for determining where log messages should be sent or “appended” to. They serve as output destinations for log events generated by an application.

2) Types: Logback provides a variety of appender types, each suited for different use cases. Common appender types include:

• Console Appender: Sends log messages to the console or standard output.
• File Appender: Writes log messages to one or more log files on the file system.
• Rolling File Appender: A specialized file appender that supports log file rotation based on size or time.
• SMTP Appender: Sends log messages as emails, often used for error alerts.
• Socket Appender: For sending log messages over network sockets to a remote server or monitoring system.
• Database Appender: Stores log messages in a database, enabling structured log data storage.
• Custom Appenders: Developers can create custom appenders to suit their unique requirements, such as integration with specific external systems.

3) Configuration: Appenders are configured in the Logback configuration file (generally logback.xml or logback-spring.xml). The configuration specifies the appender type, parameters, and the layout pattern for log messages.

4) Layouts: Appenders work in conjunction with layouts, which determine the format of log messages before they are passed to the appender. Layouts allow developers to customize the appearance of log entries, including timestamps, log levels, and message content.

5) Multiple Appenders: Logback supports multiple appenders, which means that log messages can be sent to multiple destinations simultaneously. For example, an application can log messages to both the console and a log file for different purposes.

6) Filtering: Appenders can be combined with filters to preferably control which log messages are processed by a specific appender. Filters can be based on criteria such as log level, logger name, or message content.

7) Asynchronous Logging: Logback provides the ability to perform asynchronous logging, where log messages are arrived to a separate thread for writing. This can improve application performance, particularly when writing to slow or remote destinations like files or databases.

Logback appenders can be smoothly integrated into various Java frameworks and libraries. This makes Logback a popular choice for logging within the Java ecosystem.

What is the Rolling Policy in File Appenders?

A rolling policy is a key feature of a file appender in a logging framework like Logback or Log4j. It defines how log files are managed and rotated to prevent them from becoming excessively large. Here’s a short note on the rolling policy of a file appender:

Purpose: The rolling policy is intended to manage log files efficiently, ensuring that they do not grow endlessly. Large log files can be challenging to navigate and may consume excessive disk space. The rolling policy helps in keeping log files manageable and well organized.

Log File Rotation: The central concept of a rolling policy is log file rotation. When a log file reaches a defined size or time limit, the rolling policy triggers the creation of a new log file. The old log file is archived or renamed to prevent overwriting of existing log data.

Types of Rotation:

• Size-Based: Log files are rotated when they reach a defined maximum size (e.g., 10MB). This ensures that each log file remains relatively small.
• Time-Based: Log files are rotated at specific intervals, such as daily or hourly. Additionally, time-based rotation is useful for organizing logs by date also.
• Combined: Some rolling policies use a combination of size and time thresholds, enabling log files to roll over when either condition is met.

Archiving: Rolled-over log files are typically archived to a separate location or renamed with a timestamp or sequence number. This preserves historical log data for future reference or analysis.

Retention Policies: Application administrators can configure retention policies to determine how many archived log files to keep and for how long. Older log files may be automatically deleted or moved to long-term storage.

Preventing Data Loss: Rolling policies are designed to minimize the risk of data loss. When log files are rotated, the application continues to log to a new file, ensuring that log messages are not lost during the rotation process.

Configuration: The specific configuration of a rolling policy can be customized to match the application’s requirements. Developers can define the size thresholds, time intervals, file naming conventions, and archive locations.

logback xml Configuration Examples

Now, let’s explore Logback configurations for different scenarios.

Example#1: Console Appender

A common scenario is to log messages to the console. Here’s a logback.xml configuration for a console appender:

<configuration>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
       <encoder>
         <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
       </encoder>
    </appender>

    <root level="INFO">
      <appender-ref ref="CONSOLE"/>
    </root>
</configuration>

In the above configuration, we have defined an appender named “CONSOLE” of type ConsoleAppender. The encoder specifies the log message’s format using a pattern. The root logger is set to log messages at the INFO level to the “CONSOLE” appender.

Example#2: File Appender

The primary purpose of a file appender is to persist log messages in a file rather than displaying them in the console or sending them to other destinations.

To log messages to a file, use a file appender configuration:

<configuration>
    <appender name="FILE" class="ch.qos.logback.core.FileAppender">
       <file>myapp.log</file>
       <encoder>
         <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
       </encoder>
    </appender>

    <root level="INFO">
       <appender-ref ref="FILE"/>
    </root>
</configuration>

Here, we have defined an appender named “FILE” with a specified file name. Log messages will be written to myapp.log file.

Example#3: Rolling File Appender

File appenders often support log rotation, where new log entries are written to a new file when particular conditions are met, such as reaching a maximum file size or after a certain time period. This helps prevent log files from becoming too large and difficult to handle.

A rolling file appender creates new log files when the current one reaches a certain size or age. This example uses the RollingFileAppender:

<configuration>
    <appender name="ROLLING_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
      <file>myapp.log</file>
      <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
         <fileNamePattern>myapp-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
         <maxFileSize>10MB</maxFileSize>
         <maxHistory>30</maxHistory>
      </rollingPolicy>
      <encoder>
         <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
      </encoder>
    </appender>

    <root level="INFO">
       <appender-ref ref="ROLLING_FILE"/>
    </root>
</configuration>

Here, we have defined a rolling file appender named “ROLLING_FILE”. The rollingPolicy specifies that log files should roll over daily (%d{yyyy-MM-dd}) and when they reach 10MB (<maxFileSize>). <maxHistory> defines how many archived log files to keep.

Example#4: Logging to Multiple Appenders

We can log messages to multiple destinations in parallel. Here’s an example that logs to both the console and a file:

<configuration>
     <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
       <encoder>
          <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
       </encoder>
     </appender>

     <appender name="FILE" class="ch.qos.logback.core.FileAppender">
       <file>myapp.log</file>
       <encoder>
          <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
       </encoder>
     </appender>

     <root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="FILE"/>
     </root>
</configuration>

In this configuration, log messages are sent to both the “CONSOLE” and “FILE” appenders.

Example#5: Async Appender

Logging can be an expensive operation, especially when writing to files or remote systems. To improve application performance, you can use the asynchronous appender, which offloads log processing to a separate thread:

<appender name="ASYNC_FILE" class="ch.qos.logback.classic.AsyncAppender">
    <appender-ref ref="FILE"/>
</appender>

<root level="INFO">
   <appender-ref ref="ASYNC_FILE"/>
</root>

In this example, log messages are asynchronously written to the “FILE” appender, improving application responsiveness.

Example#6: SMTP Appender (Email Alerts)

For critical errors or warnings, we may want to receive email alerts. We can configure the SMTP appender to send emails when specific log events occur:

<appender name="EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
    <!-- Configure SMTP settings -->
    <smtpHost>smtp.example.com</smtpHost>
    <smtpPort>587</smtpPort>
    <username>your_username</username>
    <password>your_password</password>
    <SSL>true</SSL>

    <encoder>
       <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>

    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
       <level>ERROR</level>
    </filter>
</appender>

<root level="ERROR">
   <appender-ref ref="EMAIL"/>
</root>

This configuration sends email alerts for log events at the ERROR level or higher.

Example#7: Custom Log Levels

Sometimes, we need custom log levels for specific scenarios. We can define custom log levels in Logback:

<configuration>
   <!-- ... other appenders and configurations ... -->

   <root level="INFO">
      <appender-ref ref="CONSOLE"/>
      <appender-ref ref="FILE"/>
   </root>

   <logger name="com.example.special" level="SPECIAL">
       <appender-ref ref="SPECIAL_APPENDER"/>
   </logger>
</configuration>

Here, we define a custom log level “SPECIAL” for the logger with the name “com.example.special.”

Example#8: Logback Extensions

Logback offers extensions and add-ons for various use cases. For example, the “logstash-logback-encoder” extension provides structured logging for integration with Logstash and Elasticsearch. We can include these extensions in our logback.xml and configure them based on our requirements.

<dependency>
   <groupId>net.logstash.logback</groupId>
   <artifactId>logstash-logback-encoder</artifactId>
   <version>6.6</version>
</dependency>

<appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <!-- Configure Logstash settings -->
    <destination>localhost:4560</destination>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>

<root level="INFO">
   <appender-ref ref="LOGSTASH"/>
</root>

Example#9: Conditional Logging

We can conditionally enable or disable appenders or log levels based on particular criteria using Logback’s <if> and <then> elements. For example, we might want to enable debug logging in development but disable it in production:

<configuration>
   <!-- ... other appenders and configurations ... -->

   <if condition='property("spring.profiles.active").contains("dev")'>
     <then>
        <root level="DEBUG">
           <appender-ref ref="CONSOLE"/>
        </root>
     </then>
   </if>
   <if condition='property("spring.profiles.active").contains("prod")'>
     <then>
        <root level="INFO">
           <appender-ref ref="FILE"/>
        </root>
     </then>
   </if>
</configuration>

Example#10: Logback and Spring Profiles

Spring applications often use profiles to manage configuration settings for different environments (e.g., development, production). We can integrate Logback with Spring profiles to customize logging based on the active profile:

<springProperty scope="context" name="spring.profiles.active" source="spring.profiles.active" defaultValue="default"/>

<if condition='property("spring.profiles.active").contains("dev")'>
  <then>
     <root level="DEBUG">
        <appender-ref ref="CONSOLE"/>
     </root>
  </then>
</if>
<if condition='property("spring.profiles.active").contains("prod")'>
  <then>
     <root level="INFO">
        <appender-ref ref="FILE"/>
     </root>
  </then>
</if>

In this configuration, we read the active Spring profile and adjust the log levels and appenders accordingly.

Example#11: MDC (Mapped Diagnostic Context)

Mapped Diagnostic Context (MDC) is a useful feature for adding contextual information to log messages. For example, we can include a unique request ID in all log entries related to a particular HTTP request:

<appender name="FILE" class="ch.qos.logback.core.FileAppender">
    <file>myapp.log</file>
    <encoder>
       <pattern>%X{requestId} %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
</appender>

<root level="INFO">
   <appender-ref ref="FILE"/>
</root>

For example, in our code we can set the MDC context as below:

MDC.put("requestId", "12345");
logger.info("Processing request...");

Example#12: Conditional Logging Based on Logger Name

We can conditionally configure log levels or appenders based on the logger’s name. For example, we might want to log database-related code at a different level or to a different appender:

<logger name="com.example.persistence" level="DEBUG">
    <appender-ref ref="DB_APPENDER"/>
</logger>

In this configuration, any logger with the name starting with “com.example.persistence” will log messages at the DEBUG level to the “DB_APPENDER.”

Example#13: Change Logback Configuration Dynamically

Logback supports changing the configuration at runtime without restarting your application. We can use the JMXConfigurator or JRebel to reload configuration changes, making it handy for making adjustments in production environments without downtime.

Below is an example of a logback.xml configuration that demonstrates how to enable dynamic configuration in Logback using the JMXConfigurator:

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="true">

     <!-- Define a property for log file name -->
     <property name="logFileName" value="myapp.log" />

     <!-- Define a root logger with a ConsoleAppender -->
     <root level="INFO">
        <appender-ref ref="CONSOLE" />
     </root>

     <!-- Define a ConsoleAppender -->
     <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
           <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
     </appender>

     <!-- Enable JMX configuration -->
     <jmxConfigurator />

</configuration>

<jmxConfigurator />: This element enables dynamic configuration through JMX. With this configuration, we can use a JMX console or a JMX client to make runtime changes to Logback’s configuration.

To change the Logback configuration dynamically using JMX, we would connect to our application’s JMX server and navigate to the Logback MBean (Managed Bean). From there, we can adjust log levels, add or remove appenders, or make other configuration changes without requiring a restart of our application. Dynamic configuration is a powerful feature for managing logging in production environments with minimal disruption.

♥ Remember to test your Logback configuration thoroughly to ensure that it behaves as expected in different environments and under various conditions.

Logback XML Configuration Examples Using Miscellaneous Components

Let’s create an extensive logback.xml configuration that includes various elements commonly used for customized logging behavior in a Spring Boot application, along with explanations for each one:

<configuration>

    <!-- Define properties -->
    <property name="logDirectory" value="/logs"/>

    <!-- Define the appenders -->
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
           <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
         <file>${logDirectory}/myapp.log</file> <!-- The name of the log file -->
         <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
             <fileNamePattern>${logDirectory}/myapp.%d{yyyy-MM-dd}.log</fileNamePattern> <!-- Pattern for archived log files -->
             <maxHistory>7</maxHistory> <!-- Maximum number of archived log files to keep -->
         </rollingPolicy>
         <encoder>
             <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
         </encoder>
     </appender>

     <!-- Define custom logger levels -->
     <conversionRule conversionWord="COLOR" converterClass="org.springframework.boot.logging.logback.ColorConverter"/>
     <root level="INFO">
         <appender-ref ref="CONSOLE"/> <!-- Connect the CONSOLE appender to the root logger -->
         <appender-ref ref="FILE"/> <!-- Connect the FILE appender to the root logger -->
     </root>

     <!-- Define a custom logger -->
     <logger name="com.example.custom" level="DEBUG" additivity="false">
         <appender-ref ref="FILE"/>
     </logger>

     <!-- Define a logger that inherits settings from root logger -->
     <logger name="org.springframework" level="ERROR" additivity="true">
         <appender-ref ref="CONSOLE"/>
     </logger>

     <!-- Define a logger for a specific class -->
     <logger name="com.example.special.SpecialClass" level="WARN"/>

     <!-- Define log filters -->
     <turboFilter class="ch.qos.logback.classic.turbo.ThresholdFilter">
          <level>WARN</level>
     </turboFilter>

     <!-- Define a marker filter -->
     <filter class="ch.qos.logback.core.filter.MarkerFilter">
         <marker>IMPORTANT</marker>
         <onMatch>DENY</onMatch>
         <onMismatch>NEUTRAL</onMismatch>
     </filter>

     <!-- Define a masking converter -->
     <conversionRule conversionWord="MASK" converterClass="com.example.MaskingConverter"/>

     <!-- Define a shutdown hook -->
     <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>

</configuration>

Below are the explanations of the new elements in the logback.xml configuration. Elements such as <property>, <appender>, <conversionRule>, <root>, <logger>, <filter> etc. are explained in the previous examples. The explanations remain the same in this example as well.

<conversionRule>:

The <conversionRule> element is used to define custom conversion words or patterns that can be used within layout patterns. It allows us to create custom placeholders for log message data.

Conversion rules enable us to format and include additional information in log messages, making them more informative. For example, we can create a custom conversion word to include a specific piece of context data in log messages.

<turboFilter>:

The <turboFilter> element defines a turbo filter, which is a specialized filter in Logback that operates on a lower level than regular filters. Turbo filters are applied before regular filters and can be highly efficient.

Turbo filters are typically used for filtering log events based on criteria like log level or logger name before they are processed by other filters or appenders. They can help optimize log processing by quickly discarding irrelevant log events.

<markerFilter>:

The <markerFilter> element is used to configure a filter that filters log events based on the presence or absence of a specified marker (a user-defined tag or label).

Marker filters allow us to selectively include or exclude log messages that have been marked with specific markers. This is useful for categorizing and processing log events based on their associated markers, adding an extra layer of control over log routing.

<shutdownHook>:

The <shutdownHook> element specifies whether Logback should install a JVM shutdown hook to automatically shut down the logging system when the application terminates.

The shutdown hook ensures that Logback gracefully shuts down and flushes any pending log data before the application exits. It helps prevent log messages from being lost or incomplete during shutdown. It’s generally recommended to enable this feature to ensure proper log handling.

This above logback.xml configuration demonstrates a wide range of elements that can be utilized to customize and control logging behavior in a Spring Boot application. While not all elements are required for every application, understanding and using these options can highly enhance our ability to manage and analyze log data successfully.

---

Source: https://logback.qos.ch/manual/index.html