Spring Boot Logging using Log4J2

What is Logging?

Logging is defined as storing some key information about a process or application in a file or database. Like date and time, filename, some messages etc.

Why do we need to log in to our application:

Logging is essential in every robust application. If there will be any issue or you want to extract some old information then logging will help you definitely. You can store some custom messages for particular actions, it will help you to identify bugs in your application.

There are many approaches to store your logs like log4j, SLF4J, Log4j2. But I will prefer you to Log4j2 and now in this article, we will learn how to apply log4j2 in our spring-boot application and what are the key points to be taken care of while logging. In this article, we will also learn how to save your logs in a NoSql database like MongoDB.

Why should we use Log4j2?

• Log4j2 is an advanced version of the very popular library log4j. 
• It contains all features of log4j and added some more features.
• It handles both synchronous and asynchronous logging efficiently.

Configuring Log4j2:

Till now we see that log4j2 is a good logging framework but the most crucial thing is how we will implement it in our application. Click here to go to Log4j2 official documentation.

What if we are already using log4j then how would we upgrade to log4j2?

We will cover every point in this article. So, let’s start the fun.

Basic Log4j2 Configuration:

We need to add the following dependencies to our pom.xml file to start with Log4j2:

Note: if we are already using log4j then we can replace 1st dependency with the following dependency so you will need not update your logger declaration everywhere in your application:

Out of the box, Log4j2 will provide a default simple configuration of logs unless you define a custom configuration.

Now to use logs in your file you have to define an object of Log4j2 in your file.

Create an instance of Logger using LogManager class. Import Logger from org.apache.logging.log4j package.

Now you are ready to use basic logs in your application:

Log4j2 custom configuration:

In this section, we will learn how to print logs to the console, save them in a file, and NoSQL database. 

First of all, we need to create a log4j2.xml file in our classpath. Our log4j2 is configured using this file only.

Now let’s understand the tags that we used in our configuration file:

1. Configuration:
   1. It is the root element of our configuration file. We write all configurations under this tag.
2. Appenders:
   1. This tag contains all the appenders like ConsoleAppender, FileAppender, JDBCAppender, NoSqlAppender etc.
3. Loggers:
   1. This tag consists of all logger instances and root is a standard logger which outputs all messages whether it is system_out or manually logged messages.

In the next section, we will learn how to configure appenders for effective logging.

Configure Appenders:

In log4j2, Appenders are those who deliver log events to their destination.

There are some predefined appenders:

• 1. ConsoleAppender: Write logs to the system console.
  2. RollingFileAppender: Write logs to a defined file and rolls data according to a defined policy.
  3. AsyncAppender: It encapsulates other appenders and writes data in a separate thread.
  4. JDBCAppender: It writes logs to a relational database.

Log4j2 Layouts:

Layouts are used to define a particular pattern to write logs on their destination. Or we can say layouts are used to define how logs will be formatted.

Mostly used log4j2 layouts are :

1. PatternLayout: It configures logs in a string pattern.
2. JsonLayout: It is defined for writing logs in JSON format.
3. CsvLayout: It is defined for writing logs in CSV format.
4. HTMLLayout: It is used to write data in HTML format.
5. XMLLayout: It is used to write data in XML format.

PatternLayout:

It is a flexible and simple way to define a layout for logs. We only have to write the following line in our appender.

Components of PatternLayouts are :

• %d{yyyy-MM-dd HH:mm:ss}: It will print the date and time of the log event.
• %5-p: It means the priority of the logging event should be left-justified to a width of five characters.
• %c: It defines the class name of the log event.
• %L: It defines line numbers.
• %m: It writes the log message.

JSONLayout:

Logging data in JSON format has many advantages like it is easier to analyze JSON data in logging tools.

To configure our log4j2 to output Logs in JSON format we will have to add the following dependency to our pom.xml format.

To configure the appender to JSON layout we can simply define the below tag:

It will produce a well-formed JSON format of log events.

Configure Loggers:

As we all know ROOT is a standard logger in log4j2. In log4j2 we can also define additional Logger tag with several log levels, appenders or filters. 

Syntax to define a Logger is as follows:

Here in the Loggers element, we can define our root logger and other loggers with level and appenders as well.

Configure Log4j2 for MongoDB:

Here we are on our most important topics. I was working on a project and in my project, I had to log all events in MongoDB, and then I made my mind to write a blog on it so others can get help.

To achieve logs in mongo first we have to add the following dependencies in our pom.xml file:

After we configure our pom.xml it’s time to configure our log4j2.xml file. In this, we have to define an appender for  NoSql where we will set up our MongoDb credentials.

You can check out the official documentation of Log4j2 NoSql here.

Note: If your mongodb password contains characters like !$@ etc. then you have to escape these characters by url encoding your password.

Now you can log your messages to MongoDB. It works but now the problem starts when you try to write your exception in MongoDB.

Whenever you try to log an exception you will get the following error.

Problem:

Generally, the MongoDB driver allows different types of data to be written to the database. For non-standard structures, however, we need to register a codec class that will allow conversion of the log objects to the document format, which is the basic structure of the DB. An example of such a codec is the org.apache.logging.log4j.mongodb4.MongoDb4LevelCodec provided by log4j.

The logger is based on the MongoDb4DocumentObject class that implements the org.apache.logging.log4j.core.appender.nosql.NoSqlObject.NoSqlObject <org.bson.Document> interface. This allows you to create a native document that will be saved in the database. In case of errors logging, MongoDb4DocumentObject objects are added as stacktrace, which are not converted into a document during the saving process.

Solution: 

To overcome this problem We have to register a custom codec class. Now we have to copy the implementation provided by log4j and replace it with our custom class.

We have to replace the codec list in org.apache.logging.log4j.mongodb4.MongoDb4Provider class:

We have to create a class in our project and paste the content of org.apache.logging.log4j.mongodb4.MongoDb4Provider class and replace the codec list with the above. And we have to configure the package of the custom class in the log4j configuration file.

Log4j MDC :

The Map Diagnostic Context (MDC) is a map that is used to store the context data of a particular thread. We can store any important information which could be used to identify a particular application. Like user data or steps of the algorithm etc.

We have to simply put the key-value pair in MDC.

Syntax:

To extract info from MDC we use %X in our log4j2.xml file.

Asynchronous Logging:

Now when we are all set to write logs in the console, file, and even database but if we have high logging overhead then all we need to do is save logs using a separate thread in asynchronous mode. If we have a high CPU count on our server, asynchronous logging is best to reduce logging overhead on our system.

Now to achieve asynchronous logging we need to add the following dependency in our pom.xml file:

We can achieve async logging in many ways like by making complete logging asynchronous or we can use hybrid logging(sync+async).

To achieve complete asynchronous logging we will have to set  log4j2.contextSelector system property to org.apache.logging.log4j.core.async.AsyncLoggerContextSelector.

We can do this by setting VM arguments in the runtime configuration of our application.

To achieve hybrid logging we can use asyncLogger in our log4j2.xml file. You can check the official document of Log4j AsyncLogger here.

Note: In asyncLogger, you should always set property includeLocation=”true”, else if you are logging into a file it won’t get line number.

That’s it, now you are all set to log your messages.