Distributed Logging - Part1
Logging:
It means recording important application events (like requests, errors, warnings, debug info) so that we developers
can monitor and debug what the application is doing at runtime.
Before we go with Distributed Logging, lets first see, how its done in Single application:
Lets see, what it takes to implement the Log functionality:
1 @RestController
2 public class PaymentController {
3
4 //Get Logger object (Logback or Log4j2)
5 Logger log = [Link]([Link]);
6
7 @GetMapping("/payments")
8 public String getPayments() {
9
10 // Start Logging
11 [Link]("fetch the payments successfully");
12
13 return "successfully fetched all paym
14
That’s it.
But how? What exactly happened?
1. Dependency
When we add below dependency
1 <dependency>
2 <groupId>[Link]</groupId>
3 <artifactId>spring-boot-starter</artifactId>
4 </dependency>
5
It automatically brings below Jars:
[Link]
[Link]
[Link]
One question might comes to your mind:
Hey Shrayansh, so by default Springboot brings the "Logback" library. But what if I want to use "Log4j2".
Then I have to add the dependency of "Log4j2", like below:
1 <dependency>
2 <groupId>[Link]</groupId>
3 <artifactId>spring-boot-starter-log4j2</artifactId>
4 </dependency>
5
Now, in class path we have 2 implementation library of Slf4j:
Logback (default) one
Log4j2, which we have manually added.
Which implementation library will be used now?
1 /************* LoggerFactory (Framework code) *******/
2
3 private final static void bind() {
4 try {
5 List<SLF4JServiceProvider> providersList =
findServiceProviders();
6 reportMultipleBindingAmbiguity(providersList);
7 if (providersList != null && ![Link]()) {
8
9 //From List of providers, it will do get(0)
10 PROVIDER = [Link](0);
11 .
12 .
13 .
14 }
15 }
16
Also while starting the application, it will warns us about it:
Note: its just a warning not an error msg, so build will get success.
But for production, get(0) is not reliable, instead we should specifically exclude the default library, if we are manually
adding the library.
1 /*Excluding the starter logging library, which internally brings:
2 - logback-classic
3 - logback-core
4 */
5
6 <dependency>
7 <groupId>[Link]</groupId>
8 <artifactId>spring-boot-starter</artifactId>
9 <exclusions>
10 <exclusion>
11 <groupId>[Link]</groupId>
12 <artifactId>spring-boot-starter-logging</artifactId>
13 </exclusion>
14 </exclusions>
15 </dependency>
16
17 //Manually add the implementation library which we want to use.
18 <dependency>
19 <groupId>[Link]</groupId>
20 <artifactId>spring-boot-starter-log4j2</artifactId>
21 </dependency>
22
23
24
2. Get Logger Object
Logger log = [Link]([Link]);
LoggerFactory does:
Finds which slf4j implementation to use i.e. Logback or Log4j2 etc.
and It choose the implementation by seeing which library is present in the class path.
Invokes LoggerContext of that specific implementation of slf4j.
LoggerContext does:
Does caching based on Logger Name. Logger name could be class name or package name or any String.
Map<String, Logger> loggerCache;
If for a particular name, Logger object is already present, it re-use it else creates new and put into the cache.
Logger Object:
Few important fields present within Logger class are:
String name
Level level
Logger Parent
List<Logger> child
name: This is the name of the Logger. It can be :
Package name
Class name
Or any String
For a given logger name, only 1 logger object is created and reused.
Note: In real world application, the common practice is to create 1 logger per class.
1 // logger name = [Link]
2 Logger log = [Link]([Link]);
3
level:
For each Logger object, we configure a log level.
There are multiple level:
And each Log statement implicitly has its own level based on the method used.
[Link]("...");
[Link]("...");
[Link]("...");
and Logger prints a logs statement only if log statement level is same or higher than the Logger configured level.
1 @RestController
2 public class PaymentController {
3
4 Logger log = [Link]([Link]);
5
6 @GetMapping("/payments")
7 public String getPayments() {
8
9 [Link]("error log");
10 [Link]("warning log");
11 [Link]("info log");
12 [Link]("debug log");
13 [Link]("trace log");
14
15 return "successfully fetched all payments";
16 }
17 }
18
Now, lets change the Logger level through configuration:
1 @RestController
2 public class PaymentController {
3
4 Logger log = [Link]([Link]);
5
6 @GetMapping("/payments")
7 public String getPayments() {
8
9 [Link]("error log");
10 [Link]("warning log");
11 [Link]("info log");
12 [Link]("debug log");
13 [Link]("trace log");
14
15 return "successfully fetched all payments";
16 }
17 }
18
[Link]
1
2 [Link]=DEBUG
3
Logger parent
List<Logger> child:
Logger framework internally maintains hierarchy of Logger objects
1 //Logger name = [Link]
2 Logger log = [Link]([Link]);
3
Advantage of maintaining the Parent, Child relationship are:
1. Log level inheritance:
We don't need to configure each and every class.
For ex: I can set level on a Package logger and all of its child automatically inherit it.
1 @RestController
2 public class PaymentController {
3
4 Logger log = [Link]([Link]);
5
6 @GetMapping("/payments")
7 public String getPayments() {
8
9 [Link]("error log");
10 [Link]("warning log");
11 [Link]("info log");
12 [Link]("debug log");
13 [Link]("trace log");
14
15 return "successfully fetched all payments";
16 }
17 }
18
[Link]
1
2 /*This logger level is inherited by all its child loggers
3 unless those child loggers explicitly override the level.
4 */
5
6 [Link]=DEBUG
7
2. Override specific child behavior
1 [Link]=DEBUG
2
3 //I have set a different log level for a specific child logger under
"com".
4 [Link]=WARN
5
3. Accepted logs are propagated upwards:
If log is rejected : STOP (do not propagate it)
1 @RestController
2 public class PaymentController {
3
4 Logger log = [Link]([Link]);
5
6 @GetMapping("/payments")
7 public String getPayments() {
8
9 [Link]("trace log");
10
11 return "successfully fetched all payments";
12 }
13 }
14
1
2 [Link]=DEBUG
3
for ex: [Link]("trace log statement")
This log is rejected, as Logger level is DEBUG. So Debug and Upper level is only entertained rested is skipped.
Only accepted log is propagated upwards (till ROOT) and all appenders are executed.
By default is TRUE, but we can turn it FALSE also.
What is appender:
Appender is the component that decides where the log will go:
console
file
DB
kafka
Etc.
1 @RestController
2 public class PaymentController {
3
4 Logger log = [Link]([Link]);
5
6 @GetMapping("/payments")
7 public String getPayments() {
8
9 //This log statement is [Link] PaymentController Logger
level is INFO
10
11 [Link]("info log");
12
13 return "successfully fetched all payments";
14 }
15 }
16
Now, Once a child logger accepts a log event, it ALWAYS propagates upward, and all parent appenders execute,
regardless of parent levels.
And if you notice once thing, in all our previous example, we have not written any appenders and still we able to see
the longs on our CONSOLE. How?
As we know, appender is required to determine where the logs will go (console, file, DB etc..)
So how come its goes to CONSOLE, who is deciding it?
1 @RestController
2 public class PaymentController {
3
4 //Get Logger object (Logback or Log4j2)
5 Logger log = [Link]([Link]);
6
7 @GetMapping("/payments")
8 public String getPayments() {
9
10 // Start Logging
11 [Link]("fetch the payments successfully");
12
13 return "successfully fetched all paym
14
So, when we don’t specify the appender, accepted Log propagate upwards and at ROOT logger (which framework
provides) it has CONSOLE appender present, which print the logs on CONSOLE.