Skip to content

Detailed Logging Guide

Let's delve deeper into our logging practices, using the context of the "Feedback Request" feature to make the learning process more practical and relevant. This guide aims to detail how to apply logging best practices at each step of the flow, from creating the request to its approval and sending it to the evaluators.

Why is Logging Crucial? 🤔

In a distributed system like ours, logging is essential for:

  • Debugging and Troubleshooting: Quickly identifying and fixing errors.
  • Application Monitoring: Tracking the system's performance and health.
  • Business Analysis: Extracting insights about user behavior and feature usage.
  • Auditing: Tracking important events for security and compliance purposes.

General Logging Strategies 📝

  1. Structured Logging: Use a consistent format (preferably JSON) to facilitate log analysis and filtering.
  2. Appropriate Log Levels: Use DEBUG, INFO, WARN, and ERROR levels according to the event's severity.
  3. Rich Context: Include relevant information in the log, such as:
    • transactionId: To track the feedback request throughout the entire flow.
    • userId: ID of the collaborator who created the request.
    • pdmId: ID of the PDM who approved/rejected the request.
    • feedbackRequestId: ID of the feedback request.
  4. Business Event Logs: Record important events from a business perspective to generate KPIs and metrics.
  5. Privacy: Avoid logging sensitive information (e.g., passwords, credit card data).
  6. Performance: Excessive logging can impact performance. Be strategic in choosing what to log and at what level.

Logging in Action: Feedback Request 🚀

Let's detail how to apply logging at each step of the "Feedback Request" flow.

1. Request Creation (Collaborator)

  • Action: Collaborator accesses the feedback request list screen.
    • INFO: "User accessed the feedback request list screen", userId, transactionId
  • Action: Collaborator clicks the "New Request" button.
    • INFO: "User started creating a new feedback request", userId, transactionId
  • Action: Collaborator fills out the form.
    • DEBUG: (If necessary) Log the custom questions entered by the collaborator (be careful not to log sensitive information).
  • Action: Collaborator adds evaluators (name and email).
    • DEBUG: (If necessary) Log the added evaluators (be careful not to log sensitive information).
    • WARN: If the evaluator's email is from CI&T, log: "Attempt to add evaluator with CI&T email", userId, transactionId, evaluatorEmail
  • Action: Collaborator clicks the "Save Request" button.
    • INFO: "User saved the feedback request", userId, feedbackRequestId, transactionId
    • ERROR: If an error occurs while saving the request (e.g., database failure), log: "Failed to save feedback request", userId, transactionId, error, stackTrace

2. PDM Notification

  • Action: System sends a notification email to the PDM.
    • INFO: "Notification email sent to PDM", pdmId, feedbackRequestId, transactionId
    • ERROR: If an error occurs while sending the email, log: "Failed to send notification email to PDM", pdmId, feedbackRequestId, transactionId, error, stackTrace

3. PDM Approval/Rejection

  • Action: PDM accesses the screen listing feedbacks to be approved.
    • INFO: "PDM accessed the screen for pending feedback request approvals", pdmId, transactionId
  • Action: PDM approves or rejects the request.
    • INFO: "PDM approved the feedback request", pdmId, feedbackRequestId, transactionId
    • INFO: "PDM rejected the feedback request", pdmId, feedbackRequestId, transactionId, comments (PDM's comments)
    • ERROR: If an error occurs while approving/rejecting the request, log: "Failed to approve/reject feedback request", pdmId, feedbackRequestId, transactionId, error, stackTrace

4. Sending Email to Evaluators (If Approved)

  • Action: System sends an email to the evaluators.
    • INFO: "Feedback request email sent to evaluators", feedbackRequestId, numberOfEvaluators, transactionId
    • ERROR: If an error occurs while sending the email, log: "Failed to send feedback request email to evaluators", feedbackRequestId, transactionId, error, stackTrace

5. Editing the Request (If Rejected)

  • Action: Collaborator edits the rejected request.
    • INFO: "Collaborator started editing the rejected feedback request", userId, feedbackRequestId, transactionId
  • Action: Collaborator saves the edited request.
    • INFO: "Collaborator saved the edited feedback request", userId, feedbackRequestId, transactionId

Example of Structured Logs (JSON) 📊

Here are some examples of what structured JSON logs might look like:

{
  "timestamp": "2024-10-27T10:00:00.000Z",
  "level": "INFO",
  "message": "User accessed the feedback request list screen",
  "userId": "user123",
  "transactionId": "txn456"
}

{
  "timestamp": "2024-10-27T10:05:00.000Z",
  "level": "WARN",
  "message": "Attempt to add evaluator with CI&T email",
  "userId": "user123",
  "transactionId": "txn456",
  "evaluatorEmail": "email@ciandt.com"
}

{
  "timestamp": "2024-10-27T10:10:00.000Z",
  "level": "ERROR",
  "message": "Failed to save feedback request",
  "userId": "user123",
  "feedbackRequestId": "fbq789",
  "transactionId": "txn456",
  "error": "Database connection error",
  "stackTrace": "..."
}

Example of Structured Logs with Lombok (Spring Boot) 🛠️

To simplify the creation of structured logs in Spring Boot, we can use Lombok to automatically generate the logger:

import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import java.util.UUID;
import java.util.Map;

@Service
@Slf4j // Automatically generates the logger
public class FeedbackService {

    public void createRequest(String userId) {
        UUID transactionId = UUID.randomUUID();
        log.info("User started creating a new feedback request", Map.of(
            "userId", userId,
            "transactionId", transactionId
        ));
        // ...
    }

    public void notifyPdm(String pdmId, String feedbackRequestId) {
        UUID transactionId = UUID.randomUUID();
        log.info("Notification email sent to PDM", Map.of(
            "pdmId", pdmId,
            "feedbackRequestId", feedbackRequestId,
            "transactionId", transactionId
        ));
        // ...
    }

    public void addEvaluator(String userId, String evaluatorEmail) {
        if (evaluatorEmail.contains("@ciandt.com")) {
            UUID transactionId = UUID.randomUUID();
            log.warn("Attempt to add evaluator with CI&T email", Map.of(
                "userId", userId,
                "transactionId", transactionId,
                "evaluatorEmail", evaluatorEmail
            ));
            throw new IllegalArgumentException("Cannot add evaluators with a CI&T email");
        }
    }
}

Monitoring and Metrics 📈

With structured logs, we can monitor the application and extract important metrics:

  • Number of feedback requests created per day/week/month.
  • Average time for a PDM to approve/reject a request.
  • Number of emails sent to evaluators.
  • Number of errors when saving requests.

Tools and Technologies 🛠️

  • Spring Boot: Use SLF4J and Logback for structured logging. Use Lombok to simplify logger creation.
  • FastAPI: Use Python's logging module and configure a handler to format logs in JSON.
  • ELK Stack (Elasticsearch, Logstash, Kibana) or Grafana Loki: For collecting, storing, and analyzing logs.

Sending Logs to Grafana Loki 📤

Grafana Loki is an excellent option for centralizing and analyzing your logs. To send your logs to Loki, you need to configure an "appender" (for Logback) or a "handler" (for Python) that formats the logs in the expected format and sends them to the Loki server.

  1. Formatting Logs: Loki expects logs in a specific format, usually with a timestamp and a label that identifies the log source.
  2. Configuring Appenders/Handlers:
    • Logback (Spring Boot): You can use an appender like Loki4jAppender to send logs directly to Loki.
    • Python: You can create a custom handler that formats the logs and sends them to the Loki API.

Example of Logback Configuration for Loki:

  1. Add the Loki4jAppender dependency to your project:

    <dependency>
    <groupId>com.github.loki4j</groupId>
    <artifactId>loki4j-logback-appender</artifactId>
    <version>1.5</version>
</dependency>

  2. Configure logback.xml:

    <configuration>
    <appender name="Loki" class="com.github.loki4j.logback.Loki4jAppender">
        <url>http://localhost:3100/loki/api/v1/push</url> <!-- Replace with your Loki URL -->
        <format class="com.github.loki4j.logback.JsonFormatter">
            <jsonLayout>
                <pattern>
                    {
                        "timestamp": "%date{yyyy-MM-dd'T'HH:mm:ss.SSSZ}",
                        "level": "%level",
                        "message": "%message",
                        "logger": "%logger",
                        "thread": "%thread",
                        "transactionId": "%mdc{transactionId}", <!-- If you use MDC -->
                        "userId": "%mdc{userId}" <!-- If you use MDC -->
                    }
                </pattern>
            </jsonLayout>
        </format>
        <label class="com.github.loki4j.logback.StaticLabelProvider">
            <label>
                <name>app</name>
                <value>feedback-service</value> <!-- Your application's name -->
            </label>
        </label>
    </appender>

    <root level="INFO">
        <appender-ref ref="Loki"/>
    </root>
</configuration>

Creating Custom Appenders/Handlers:

If the existing options don't meet your needs, you can create your own appenders/handlers to format and send logs to Loki or any other analysis tool.

Conclusion 🎉

I hope this detailed guide helps you implement a robust and efficient logging system for the "Feedback Request" feature. Remember that logging is an essential skill for any developer, and practice makes perfect.

If you have any questions or suggestions, don't hesitate to share! We are in this journey together. 💪