Development of Java Logging Part 1: Past and Present

By Shangzuo

Preface

Initially, I wrote this article to share my knowledge of Java logging with my team, as I often encountered irrational logging configurations and diverse logging methods when working with others. However, as I prepared and added more details, I realized that most articles on logging only focus on one aspect, either discussing the history and principles or addressing package conflicts, without providing a comprehensive overview of Java logging. As a result, this article has become more detailed than initially intended. To make it easier to read, I will present it as a series.

1. Introduction

Today, logging has evolved into three abstract layers: the interface layer, implementation layer, and adaptation layer.

• Interface layer: Also known as the logging facade, it's simply an interface that defines the interface and relies on other layers for implementation.

• Implementation layer: This is a tool that actually works and can record log content. Note that it's not an implementation of the above interface, as it doesn't perceive or directly implement the interface; it's an independent implementation..

• Adaptation layer: Typically referred to as an Adapter, it implements the above interface. Since the interface layer and adaptation layer are created by different entities, they can't directly match each other. As David Wheeler once said, "All problems in computer science can be solved by another level of indirection," which is why we have the adaptation layer.

The adaptation layer has two capabilities: binding and bridging.

• Binding: Binds the interface layer to an implementation layer (implements an interface layer and calls the implementation layer).

• Bridging: Bridges the interface layer to another interface layer (implements an interface layer and calls the interface of another interface layer). This mainly facilitates low-cost migration between the interface layer and the adaptation layer for users.

If you find the above description too abstract and difficult to understand, feel free to skip it and come back to it after finishing the article.

Next, we'll review the history of Java logging in chronological order, which will help us understand the underlying principles and guide our future practices.

2. History

2.1 Standard Output (<1999)

Java didn't have a special logging tool at the beginning, and everyone used System.out and System.err to output logs. However, they are merely simple information outputs, lacking the ability to distinguish error levels, control output granularity, or provide any management and filtering capabilities. As Java engineering has advanced, their capabilities have become increasingly insufficient.

Although System.out and System.err have default output to the console, they can also save output to a file:

System.setOut(new PrintStream(new FileOutputStream("log.txt", true)));
System.out.println("This sentence will be output to the log.txt file");

System.setErr(new PrintStream(new FileOutputStream("error.txt", true)));
System.err.println("This sentence will be output to the error.txt file");

2.2 Log4j (1999)

In 1996, a European company called SEMPER decided to develop a tool for logging. After many times of upgrades, it eventually developed into Log4j. The primary developer of this tool is a Russian programmer named Ceki Gülcü [2]. Remember his name, Ceki, because he will be mentioned frequently later on.

By 1999, Log4j had been widely used, and as users increased, the demands of the users became diversified. In 2001, Ceki decided to make Log4j open-source, aiming to further develop and improve it with the community. Shortly thereafter, the Apache Software Foundation (ASF) sent an invitation to Log4j, and Ceki joined Apache to continue his work on Log4j. As a result, Log4j was renamed Apache Log4j [3] and began to develop rapidly.

Log4j provides more powerful capabilities than System.out, and even many of its ideas are still widely accepted today, such as:

• Logs can be output to consoles, files, databases, and even remote servers and emails (referred to as Appender).

• The log output format (referred to as Layout) allows customization. For instance, error logs and regular logs can be displayed in different formats.

• Logs are divided into five levels (referred to as Level) from lowest to highest: debug, info, warn, error, and fatal. Before output, the allowed level of configuration is checked, and logs below this level will be ignored. Additionally, there are two special levels: all and off. The all level allows all logs to be output, while the off level completely disables log output.

• Log4j allows you to specify different loggers (referred to as Logger) within a project at any time. Each logger can be configured with its own logging location and log level.

• Log4j also supports configuration through properties or XML files.

With the success of Log4j, Apache incubated Log4Net [4], Log4cxx [5], and Log4php [6]. The open source community followed this trend and launched many projects such as Log4c [7], Log4cpp [8], and Log4perl [9]. This proves the influence of Log4j in the field of log processing.

However, Log4j had significant performance limitations and gradually declined after the introduction of Logback and Log4j 2. Eventually, in 2015, Apache announced the termination of Log4j development and fully migrated it to Log4j 2 [10] see [2.7 Log4j 2 (2012)].

2.3 JUL (2002.2)

As Java engineering advanced, Sun recognized the critical importance of logging and believed that this capability should be supported by the JRE-native. Therefore, in 1999, Sun submitted a JSR 047 [11] proposal with the title "Logging API Specification". However, it wasn't until two years later, in 2002, that the official Java logging system was released with Java 1.4. This system is called Java Logging API, JUL for short, and the package path is java.util.logging.

We will not introduce JUL here for details.

2.4 JCL (2002.8)

In addition to Log4j and JUL, there were other logging tools such as Apache Avalon [13] (a server-side development framework) and Lumberjack [14] (an open source logging tool running on JDK 1.2/1.3).

For independent and lightweight projects, developers can use a logging scheme based on their preferences. However, in most cases, a business system depends on numerous third-party tools and each of them uses its own logging implementation. When these tools are integrated, it inevitably leads to chaotic log management.

To address this problem, Apache introduced an interface called Jakarta Commons Logging 15 in 2002. The primary developer is also Ceki. This interface supports numerous logging tools, including Log4j, JUL, Apache Avalon, and Lumberjack. If developers want to print logs, they only need to call the JCL interface. The final log implementation is determined by the top-level business system. As we can see, this is a typical interface-implementation separation design.

However, since the implementations (Log4j, JUL) existed before the interface (JCL), JCL provided an adaptation layer to bridge the interface and the implementations. (The reasons for not using the latest version will be discussed in section [1.2.7 Log4j2 (2012)].)

Here is a brief introduction to several adaptation layers/implementation layers provided by JCL:

• AvalonLogger/LogKitLogger: Adaptation layer for binding Apache Avalon. Since Avalon has different log package names in different periods, there are two corresponding adaptation layers.

• Jdk13LumberjackLogger: Adaptation layer for binding Lumberjack.

• Jdk14Logger: Adaptation layer for binding JUL (since JUL is provided from JDK 1.4)

• Log4JLogger: Adaptation layer for binding Log4j.

• NoOpLog: It is JCL's built-in logging implementation, but it is an empty implementation that does nothing.

• SimpleLog: It is JCL's built-in logging implementation. Although it only has basic functions, it allows users to print logs even without reliance on other tools.

At that time, the project prefix Jakarta was used because it was part of the Jakarta Project [16], a joint initiative by Apache and Sun. (email [17]). Now JCL, as a sub-project of Apache Commons [18], is renamed Apache Commons Logging. Like our commonly used Commons Lang [19] and Commons Collections [20], JCL is also one of the components in Apache Commons Proper. However, the abbreviated JCL is retained and not changed to ACL.

2.5 Slf4j (2005)

Ceki, the developer of Log4j, started his own business in 2005 (blog [21], email [22]) and soon launched Simple Logging Facade for Java [23], abbreviated as SLF4J.

SLF4J is also an interface layer, and the interface design is very close to JCL. A great difference compared to JCL is the method of binding the logging implementation. JCL uses dynamic binding, which determines the appropriate logging implementation at runtime when logging is executed. In contrast, SLF4J uses static binding, that is, the logging implementation is determined at compile-time so that it has a better performance. This is the classloader problem that is often mentioned. For a more detailed discussion, see What is the issue with the runtime discovery algorithm of Apache Commons Logging [24] and Ceki's article Taxonomy of class loader problems encountered when using Jakarta Commons Logging [25].

Before SLF4J was launched, there had already been another interface layer, JCL, on the market. To allow users to choose, SLF4J introduced two bridging layers:

• jcl-over-slf4j: allows users who are using JCL to easily migrate to SLF4J. It seems that you are calling the JCL interface, but it is migrated to the SLF4J interface.

• slf4j-jcl: allows users who are using SLF4J to easily migrate to JCL.

SLF4J meets nearly all user cases by offering various adaptation layers.

Let's take a look at its offerings:

Articles introducing SLF4J often quote two diagrams on its official website (You can also refer to them if you are interested.):

I'd like to explain the name slf4j-log4j12, which stands for the adaptation layer of SLF4J + Log4j 1.2 (the last version of Log4j). Similarly, slf4j-jdk14 represents the adaptation layer of SLF4J + JDK 1.4 (that is, JUL).

2.6 Logback (2006)

However, Ceki did not limit his goal to SLF4J. As a founder of Log4j, he did know its problem. So, in 2006, Ceki introduced another logging implementation scheme: Logback [26]. In terms of ease of use, function, and performance, Logback surpasses Log4j.

Additionally, its native support for SLF4J without requiring extra adaptation layers has attracted many supporters. At present, Logback has become the most widely accepted logging implementation layer in the Java community (according to Logback's own statistics in 2021, the market share was 48% [27]).

Compared to Log4j, Logback provides many new features that we now are very familiar with:

• It supports cutting and rolling records of log files and asynchronous writing.

• For historical logs, it supports automatic cleaning by time or hard disk occupancy, and automatic compression to save hard disk space.

• The branch syntax is also supported. You can use <if>, <then>, <else> to configure different log output logic based on conditions. For example, you can determine to output more detailed log information only in the development environment.

• It provides a large number of log filters and can even identify each user through the login user session and output a separate log file.

• The exception stack supports printing jar package information so that we can know the specific file and line the call comes from, as well as the specific jar package the file comes from.

Logback is mainly composed of three parts:

• logback-core: the core implementation for logging and output.

• logback-classic: the adaptation layer that completes the implementation of the SLF4J interface.

• logback-access [28]: integrates Logback with Servlet containers (Tomcat and Jetty) so that their HTTP-access logs can also be output through the powerful Logback.

2.7 Log4j 2 (2012)

Apache introduced a new powerful product in 2012: Apache Log4j 2 [29], which has many highlights:

• With the plug-in structure [30], users can develop their own plug-ins to achieve Appender, Logger, and Filter extensions.

• Based on the asynchronous output of LMAX Disruptor [31], its performance increases by about 10 times compared to Logback in multi-threaded scenarios. Apache also publicizes this part as a main selling point. For details, see Log4j 2 Performance [32].

Log4j 2 mainly consists of two parts:

• log4j-core: the core implementation similar in function to logback-core.

• log4j-api: the interface layer similar in function to SLF4J, which only contains the interface definition of Log4j 2.

You'll find that Log4j 2 has a unique design, offering a third interface layer (log4j-api, even though it is its own interface) in addition to JCL and SLF4J. According to a section of API Separation [33] on the official website, this design allows users to use different interface layers and implementation layers simultaneously within a single project.

However, at present, Log4j 2 is generally regarded as an implementation layer and JCL or Slf4j is introduced as an interface layer. JCL released version 1.3.0 [34] at the end of 2023, nearly a decade later, which includes an adaptation layer for Log4j 2. That's the reason why we did not introduce the latest version of JCL in [1.2.4 JCL (2002.8)]. The log adaptation layers that have been "obsolete" were deprecated in this version.

In fact, the relationship between Logback and SLF4J is the same as that of log4j-core and log4j-api. At present, you can use Logback only through SLF4J. However, everyone will discuss them as two products respectively.

Although Log4j 2 has been available for ten years (this article was written in 2024), it still hasn't managed to surpass Logback. In my opinion, there are two main reasons:

• Although Log4j 2 carries the name Log4j, it is a completely rewritten logging system. Users cannot upgrade simply by changing the Log4j version number, which leads to a low willingness among existing users to upgrade.

• Log4j 2 was released six years after Logback but failed to offer sufficiently impressive or differentiated features. The two highlights mentioned before do not provide enough attraction for regular users. Meanwhile, the combination of SLF4J and Logback has already been powerful and has a significant first-mover advantage.

For example, Spring Boot was suggested to switch the logging system from Logback to Log4j2 [35], but this suggestion was declined by Phil Webb [36], a core contributor to Spring Boot. In his response, he provided several reasons, including Spring Boot needs to ensure backward compatibility to facilitate user upgrades, and switching to Log4j 2 would break it; the vast majority of users are not facing a logging performance issue, and the performance advantage alone of Log4j 2 is not a core concern for users; and "using Log4J2 when possible is very easy with Spring Boot" (refer to the official documentation for guidance [37] if you need).

2.8 spring-jcl (2017)

Since most applications are currently built based on Spring/Spring Boot, I would like to introduce the package spring-jcl [38]. Currently, Spring Boot uses the spring-jcl + Logback scheme.

Spring mentioned in its official Blog Logging Dependencies in Spring [39] that if it could "turn back the clock and start Spring as a new project, it would use a different logging dependency. Probably the first choice would be the Simple Logging Facade for Java (SLF4J)" instead of JCL as the default logging interface.

Now Spring wants to support SLF4J and ensure backward compatibility to support JCL at the same time, so it provides spring-jcl package from 5.0 (Spring Boot 2.0). It carries the name Spring, but the package name in the code is consistent with JCL (org.apache.commons.logging), and the function is naturally consistent with JCL. It additionally adapts SLF4J and places SLF4J in the first sequence of search so that it achieves the support for both JCL and SLF4J (you can refer to the section [1.2.4 JCL (2002.8)] for comparison).

If you create an application based on Spring Initializr [40], you don't need to worry about this package as it has already worked. However, if you encounter package conflicts during your project development or need to choose your logging interface and implementation, you can treat spring-jcl as JCL and confidently exclude it as needed.

2.9 Others

In addition to the logging schemes we mentioned above, there are some less common ones, such as:

• Flogger [41]: a logging interface layer introduced by Google in 2018. The first letter F means Fluent, which is its biggest feature: chain call (or streaming API, SLF4J 2.0 also supports Fluent API, which we will introduce in a subsequent article of this series).

• JBoss Logging [42]: introduced by RedHat circa 2010, with a complete interface layer, implementation layer, and adaptation layer.

• slf4j-reload4j [43]: a version introduced by Ceki based on Log4j 1.2.7 fork. It is aimed at addressing the security issues of Log4j. If your project is still using Log4j and you do not want to migrate it, it is recommended to replace it with this version. (However, not all security issues can be resolved. For details, refer to the reference link.)

Since these logging frameworks are rarely used in practical development, this article will not go into further detail about them.

3. Summary

The history has been introduced, but it does not end here. Two interfaces (JCL and SLF4J), four implementations (Log4j, JUL, Logback, and Log4j2), and countless adaptation layers are connected in series to form a network. I specially draw a diagram for it:

Explanation of this figure:

Modules in the same color have the same groupId, and you can refer to the specific value in the legend.
The adaptation layer of JCL is provided directly in its package. We have already introduced the details before and you can refer to [1.2.4 JCL (2002.8)] for review.
If you want to use Logback, you must also use SLF4J (referring to its adaptation layer is also included). Similarly, if you want to use Log4j 2, its log4j-api cannot be bypassed.

If you found the [1.1 Introduction] too abstract before, I recommend you review it now, and you'll gain more insights.

From this history, I also find several interesting details:

• For a long time before and after Log4j 2 was launched, SLF4J and Logback updated slowly because there were no competitors.

• Later products often provide support for their predecessors, while earlier products tend to ignore their successors.

• All problems in computer science can be solved by another level of indirection, or two if not (what the bridge layer does).

• Ceki has been contributing to Java logging for 25 years. (Of course, there are many such experts in the code field. For example, Linus Torvalds [44] has been maintaining Linux for 33 years, although he mainly participated as a product manager in the later period, and Bram Moolenaar [45] had maintained Vim for 32 years).

Reference

Disclaimer: The views expressed herein are for reference only and don't necessarily represent the official views of Alibaba Cloud.

Community

Development of Java Logging Part 1: Past and Present

Preface

1. Introduction

2. History

2.1 Standard Output (<1999)

2.2 Log4j (1999)

2.3 JUL (2002.2)

2.4 JCL (2002.8)

2.5 Slf4j (2005)

2.6 Logback (2006)

2.7 Log4j 2 (2012)

2.8 spring-jcl (2017)

2.9 Others

3. Summary

Reference

Read previous post:

Read next post:

Alibaba Cloud Community

You may also like

Comments

Alibaba Cloud Community

Related Products

Web Hosting Solution

YiDA Low-code Development Platform

Web Hosting

mPaaS