In Microservices Architecture based Applications, there are multiple services which interact with each other. In order to serve a client request, one request may involve multiple microservices call to get the response. If any exception occurs or any latency issue appears during such calls, then how will we identify the root cause of the issue? Of course, we will make use of Spring Cloud Sleuth & Zipkin to support distributed logging & Tracing. Therefore, we are going to learn about โHow to implement Distributed Logging Tracing using Sleuth Zipkinโ and related concepts accordingly.
If we are working on a Standard application, we can implement logging by adding one log file to identify any exception or other issues like response timings or latency etc. However, in case of Microservices based applications, there are multiple small applications. Therefore, we will have to maintain multiple log files such as at least one log file per application. In this case, correlating the logs to a particular request chain becomes difficult. Hence, identifying the root cause of the issue also becomes complicated.
In order to overcome from such kind of issues, we make use of Spring Cloud Sleuth and Zipkin. Letโs discuss our topic โHow to Implement Distributed Logging Tracing using Sleuth Zipkin?โ And related concepts accordingly.
What is Logging & Tracing?
Tracing : Tracing is a process of finding an execution path or a flow of multiple microservices to serve a request.
Logging : Logging is a process of admitting all log messages for a trace. These messages can be of any type such as DEBUG, INFO, WARN, ERROR etc.
Here, we will be implementing Logging and Tracing for multiple Microservices involved in serving one request. We also call it โDistributed Tracing and Loggingโ. Once it is implemented one time, then developer doesnโt need to check the code again to find the flow either for testing purpose or for debug purpose.
What is Sleuth and Zipkin used for ?
Sleuthย
Sleuth provides unique ids for the request flow. Further, the developer uses these ids to find out the flow of execution with the help of tools like Zipkin, ELK etc. Generally it has two ids : Trace id and Span Id. Trace Id is applicable for a complete flow. If the developer gets this Id, he/she can find the flow of execution in all microservices involved. However, Span Id is applicable to the flow of one microservice. If the developer gets this Id, he/she can find the flow of execution in a particular microservice. Moreover, there is a concept of parent id which is applicable to a particular microservice like span id. During the flow of execution, span id of the previous microservice becomes the parent id of the next microservice as shown below.
Zipkinย
We use Zipkin in two parts : Zipkin Client and Zipkin Server. Zipkin Client contains Sampler which collects data from microservices with the help of Sleuth and provides it to the Zipkin Server. In order to utilize the benefits of both tools, we should always add Zipkin Clientโs dependency along with Sleuth in every microservice. However, there must be only one centralized Zipkin Server, which collects all data from Zipkin Client and display it as a UI. Hence, after making a request developer should look into Zipkin Server to find trace id, span id and even flow of execution. From here itself, Open Log files to check Log lines that are related to the current trace Id.
How to download & setup Zipkin Server?
In order to check the flow of execution in microservices chain, we need to download & setup Zipkin Server in our system. Please follow below steps to get it done.
1) Download ZIPKIN from the Link below:
Zipkin Download
2) Extract to a folder โZipkinExecutableโ. The downloaded file name should be like โZipkinExecutable.zipโ
3) Open command prompt at the above location where your extracted folder exists.
For example: Letโs assume that your location is C:/Downloads/ZipkinExecutable/>
4) Run Jar file by using below command:
> java -jar zipkin-server-2.12.9-exec.jar
If Zipkin Server started successfully, you will see message something like โStarted ZipkinServer in 14.094 seconds (JVM running for 15.452)โ
5) Open your Browser and Enter URL:
http://localhost:9411/zipkin/
6) Click on โFind Tracesโ button. It will display โShowing:ย 0ย ofย 0โ
Other sources to get Zipkin jar
1) Download from zipkin.io page
2) You can also add following dependency to add zipkin-2.5.2.jar to your project. You can select the version as per your requirement.
<dependency>
<groupId>io.zipkin.zipkin2</groupId>
<artifactId>zipkin</artifactId>
<version>2.5.2</version>
</dependency>
How to Implement Distributed Logging & Tracing using Sleuth & Zipkin?
In order to implement Distributed Logging & Tracing, we will do three major tasks as below:
1. Develop three Microservices applications using Spring Boot which will interact with each other.
2. Implement distributed logging & tracing using Spring Cloud Sleuth
3. Test distributed logging & tracing using Zipkin Server
Letโs start with developing application step by step.
Step#1 : Create a Spring Boot Project using STS(Spring Tool Suite)
Here, we will use STS(Spring Tool Suite) to create our Spring Boot Project. If you are new to Spring Boot, visitย Internal Link to create a sample project in spring boot using STS. While creating a project in STS add starters โSleuthโ, โZipkin Clientโ, โSpring Webโ in order to get features of them.
Step#2: Modify application.properties file
server.port=8081 spring.application.name=microservice#1 logging.file.name=D:/myLogs/serviceOne.log #can include eureka details alsoโ-
Step#3: Create AppConfig.java to Setup Configurations
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.web.client.RestTemplate; import brave.sampler.Sampler;
@Configuration public class AppConfig { //Collects data from Sleuth and provides it to Zipkin Client @Bean public Sampler samplerOb() { //return Sampler.NEVER_SAMPLE; return Sampler.ALWAYS_SAMPLE; } //Creates RestTemplate Object @Bean public RestTemplate rt() { return new RestTemplate(); } }
Step#4: Write a RestController to implement a Microservice
Write a RestController like below:
import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; import org.springframework.web.client.RestTemplate; @RestController public class AppOneRestController { private Logger log = LoggerFactory.getLogger(AppOneRestController.class); @Autowired private RestTemplate restTemplate; @GetMapping("/m1") public String methodOne() { log.info("Inside microservice#1"); String response = restTemplate.getForObject("http://localhost:8082/m2", String.class); log.info("response by microservice#1 " + response); return "returning from microservice#1"; } }
Step#5: Write two more applications just like above
As similar to above code also define two more applications. For example, letโs say MyServiceTwo and MyServiceThree. However, Config Files will be similar but properties may differ. Also change the code of RestController accordingly.
AppConfig.java for Service #2
Same as for Service #1
application.properties for Service #2
server.port=8082 spring.application.name=microservice#2 logging.file.name=D:/myLogs/serviceTwo.log
RestController for Service #2
import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; import org.springframework.web.client.RestTemplate; @RestController public class AppTwoRestController { ย ย ย ย ย ย private Logger log = LoggerFactory.getLogger(AppTwoRestController.class); ย ย ย ย ย ย @Autowired ย ย ย ย ย ย private RestTemplate restTemplate; ย ย ย ย ย ย @GetMapping("/m2") ย ย ย ย ย ย public String methodTwo() { ย ย ย ย ย ย ย ย ย ย ย log.info("Inside microservice#2"); ย ย ย ย ย ย ย ย ย ย ย String response = restTemplate.getForObject("http://localhost:8083/m3", String.class); ย ย ย ย ย ย ย ย ย ย ย log.info("response by microservice#2 " + response); ย ย ย ย ย ย ย ย ย ย ย return "returning from microservice#2"; ย ย ย ย ย ย } }
ย
AppConfig.java for Service #3
Same as for Service #1
application.properties for Service #3
server.port=8083 spring.application.name=microservice#3 logging.file.name=D:/myLogs/serviceThree.log
RestController for Service #3
import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; import org.springframework.web.client.RestTemplate; @RestController public class AppThreeRestController { ย ย ย ย ย ย private Logger log = LoggerFactory.getLogger(AppThreeRestController.class); ย ย ย ย ย ย @Autowired ย ย ย ย ย ย private RestTemplate restTemplate; ย ย ย ย ย ย @GetMapping("/m3") ย ย ย ย ย ย public String methodThree() { ย ย ย ย ย ย ย ย ย ย ย ย log.info("Inside microservice#3"); ย ย ย ย ย ย ย ย ย ย ย ย log.info("done with microservice#3 " ); ย ย ย ย ย ย ย ย ย ย ย ย return "returning from microservice#3"; ย ย ย ย ย ย } }
How to test the application as a whole?
Lets follow below steps to test the whole application. Screenshots are also provided to make testing easier.
1) Start Zipkin Server
2) Start All Microservices one by one
3) Enter URL: http://localhost:8081/m1
4) Open a new browser window and enter http://localhost:9411/zipkin
5) Click on Find Trace. You will see details of three running microservices highlighted in green color.
6) Click on Blue color bar. You will see trace record of each microservice.
7) Click on any one option. Letโs click on โmicroservice#2โ. A new popup window will appear with all the details.
8) Click on โShow IDsโ, you will find details of traceId, spanId and parentId. Now, copy traceId from here.
9) Open Log files and search(ctrl+F) with traceId to see all log lines related to current request.
ย
For example, once we enter URL โhttp://localhost:9411/zipkinโ and click on โFind tracesโ button, we will see below screen.
ย
Now, Letโs click on blue bar just above the microservices highlighted in green color. We will get a screen something like below.
ย
In order to find out the trace Ids for a particular microservice click on any microservice as highlighted above. For example, we click on microservice#2. Now, We will get a popup window something like below.
Finally, copy the trace Id from here and look into respective log files to see the log messages. Furthermore, we have provided log messages generated after running this example for all three microservices in the next section.
Messages from Log Files
Below are the messages generated from Log files for respective Microservice.
Microservice#1 : Log messagesย
2021-04-21 23:53:27.880 INFO [microservice#1,663c732d1728af87,663c732d1728af87] 56972 --- [http-nio-8081-exec-3] c.d.s.zipkin.rest.AppOneRestController : Inside microservice#1 2021-04-21 23:53:28.064 INFO [microservice#1,663c732d1728af87,663c732d1728af87] 56972 --- [http-nio-8081-exec-3] c.d.s.zipkin.rest.AppOneRestController : response by microservice#1 returning from microservice#2
Microservice#2 : Log messagesย
2021-04-21 23:53:27.939 INFO [microservice#2,663c732d1728af87,ac095c1c5bce450e] 21528 --- [http-nio-8082-exec-1] c.d.s.zipkin.rest.AppTwoRestController : Inside microservice#2 2021-04-21 23:53:28.045 INFO [microservice#2,663c732d1728af87,ac095c1c5bce450e] 21528 --- [http-nio-8082-exec-1] c.d.s.zipkin.rest.AppTwoRestController : response by microservice#2 returning from microservice#3
Microservice#3 : Log messagesย
2021-04-21 23:53:28.020 INFO [microservice#3,663c732d1728af87,79a65229aacee4d2] 43752 --- [http-nio-8083-exec-1] c.d.s.z.rest.AppThreeRestController : Inside microservice#3 2021-04-21 23:53:28.021 INFO [microservice#3,663c732d1728af87,79a65229aacee4d2] 43752 --- [http-nio-8083-exec-1] c.d.s.z.rest.AppThreeRestController : done with microservice#3
We can see occurrences of trace id : โ663c732d1728af87โ in all three microservices as it is common for all. However, span Id is specific for each microservice. Moreover, since microservice#1 is the parent microservice, hence both ids are same for it.
Conclusion
After going through all the theoretical & example part of โHow to Implement Distributed Logging Tracing using Sleuth Zipkin?โ, finally, we should be able to implement logging & tracing concept using Sleuth and Zipkin as a whole. Similarly, we expect from you to further apply this knowledge in your project correspondingly. Additionally, having familiarity of terminologies explained in this article is very crucial to implement microservices for a developer particularly in Java. Furthermore, we will also be updating the article time to time accordingly if any need arises. In addition, feel free to provide your comments in comments section below.
ย
ย
ย
ย
ย
ย
ย
ย
ย
ย
Nice article.
But, It would be better if You put your code in GitHub so that We can compare the code.
Thanks