Logs too sparse to trace issues, too verbose to afford — what to do when production troubleshooting always misses that one key detail?

Based on Xiyue Company's project delivery practices, logs are not about "the more complete the better", but rather "enough to reconstruct the complete path of a request". We usually recommend dividing logs into three layers: trace layer, business layer, and error layer. The trace layer is responsible for connecting the full journey of a request, the business layer records key state changes, and the error layer records exception stack traces and input/output parameters. This avoids both a flood of trivial logs and missing critical information during troubleshooting.

Why is missing one log line more damaging than having ten extra lines?

Many teams treat logs as a backend aid, only to realize key information is missing when a problem occurs. Two common extremes: one is hundreds of lines of info logs without order IDs or user IDs; the other is printing SQL parameters and encrypted fields verbatim, causing leaks. Both stray from the purpose of logs — observability.

Logs are not free. Each log entry must be serialized, written to disk, transmitted, and stored; by 2026, common practice is to bill based on volume. Experience range: keep log volume per request between 20 and 50 lines. If it exceeds 50 lines, check for duplicate or loop-based printing.

We once lowered business success logs to debug to reduce storage costs, only to spend three hours troubleshooting a payment callback issue. Lesson: don't sacrifice business state recording for storage cost.

What exactly should be logged? The three-layer log model

We divide logs into three layers by purpose: trace layer, business layer, and error layer. The rationale: troubleshooting requires knowing "what happened", not "every detail". Don't mix the three layers together, otherwise logs become a messy stew.

1. Trace layer: record the request's unique identifier (traceId), user identifier, call source, entry time, and duration. Note: trace layer logs must propagate context from the entry point, otherwise the full trace cannot be obtained.
2. Business layer: record core business state changes, such as an order changing from pending payment to paid, or inventory deduction success. Write operations must be logged; read operations only need logs on errors. It's recommended to name entries by business action, avoiding vague descriptions like "operation successful".
3. Error layer: record exception stack traces, input/output parameters at the time of error, error codes, and before/after states. This layer is best suited for warn or error levels. Do not record sensitive fields such as passwords, tokens, or full ID numbers; use desensitized placeholders instead.

The key to the three-layer log model is the "layering standard": the trace layer cares about "who am I", the business layer cares about "what was done", and the error layer cares about "why it failed". If all logs use info level without distinguishing between trace and business, you'll still have to guess during troubleshooting. As common practice in 2026, MDC is typically used to automatically inject traceId, so business logs don't need to repeat it.

How to tell if current logs are sufficient? The three-step verification method

Use the "three-step verification method" to spend ten minutes before a release checking whether logs are sufficient. The core is whether you can answer three questions: Can the request be located? Can the business sequence be understood? Can the error cause be found? All three steps are indispensable.

1. Step 1: Randomly pick a complete request from a recent real user and see if you can reconstruct its call chain from the logs. If you can only see a few isolated log entries, the trace layer is missing.
2. Step 2: Simulate a business exception (such as insufficient inventory) and check whether there are corresponding business state records and error context in the logs. If not, the business or error layer is not covered.
3. Step 3: Check whether the request start and end times are recorded. If there is only an exception stack trace without input/output parameters, locating the problem often requires reproducing the scenario, which is inefficient.

When executing, don't just look at log statements in code; look at real output. It's best to run a round in a test environment. If any of the three steps fails, rework and add logs.

Common pitfalls in logging and criteria to judge them

Here are four common pitfalls, each increasing troubleshooting time.

• Pitfall 1: All logs use info level. It's recommended that info only records key steps, debug records details, and error only records exceptions. Criterion: whether the level of each log reflects its importance.
• Pitfall 2: Logging inside loops. If a loop may execute tens of thousands of times, printing logs inside it can blow up storage. Criterion: whether the log frequency within a single loop is constant and low.
• Pitfall 3: Logging sensitive data in plaintext. Such as login passwords or the signature in payment result notifications. Criterion: whether desensitization is done before printing, or the data is filtered out.
• Pitfall 4: No unified traceId. In multi-service calls, logs are scattered across systems and cannot be linked. Criterion: whether a request ID can retrieve the complete call from the logging system.

Common practice in 2026 is to introduce structured logs (JSON). More fields mean higher storage cost. It's recommended to keep fields within 10 to 15; put extra information into the message field.

Log solution comparison: plain text logs vs structured logs — which is more convenient?

We compared plain text and structured logs across four dimensions:

• Readability: Plain text is more readable directly; structured logs (JSON) have hierarchy and are slightly more verbose to read, but with a log platform the difference is minimal.
• Search capability: Plain text only supports full-text search; structured logs can be filtered precisely by traceId, userId, or level, making a noticeable difference in troubleshooting efficiency.
• Storage overhead: Structured logs add field names and brackets, increasing volume by roughly 30%–60% for the same information, but they offer better filtering.
• Integration cost: Plain text requires zero modification; structured logs require defining field standards and adjusting output format, with 1–2 days of adaptation for legacy systems.

If your team already has a mature log platform, structured logs are the default choice; for single-machine deployments, plain text may suffice. The key is to define standards first to avoid inconsistent practices across services.

New services generally use structured logs. Legacy systems can be migrated incrementally, prioritizing adding traceId to core links. Criterion: can you quickly find all logs for a specific request in one file? If not, it's time to upgrade. Structured logging is not a silver bullet.

Applicable scenarios and boundaries

The three-layer log model suits web backends, microservices, and scheduled tasks, but is not suitable for ultra-large-scale gateways (which only record key metrics) or pure frontend projects. If you haven't adopted a log platform yet, start with the trace layer.

Additionally, for systems requiring long-term audits (such as payment or finance), consider log retention periods and compliance requirements. It's recommended to set different retention times based on business needs: for example, access logs for 30 days, audit logs for 180 days. The specific periods can follow your company's policies.

No need to be anxious about "incomplete logs" to the point of printing every variable. The benefit of logs is reduced troubleshooting time; the cost is storage and performance. When troubleshooting time becomes acceptable, there is no need to add more logs.

FAQ

Should logs be in Chinese or English?

It's recommended to use Chinese for business actions and English keywords for search. Keep the output format fixed for log platform grouping, and prioritize content readability.

What if the log level on production was accidentally set to info and there is too much data?

Prioritize keeping error and warn levels; keep only key steps for info. If there's still too much, use the log platform to filter first, then adjust levels in batch.

Should user ID and order ID be included in logs?

Yes. They are high-efficiency indexes for problem location; without them, you'll have to rely on time-based fuzzy searching. It's recommended to put them in MDC or structured fields.

What if logging too much affects performance?

First check whether logs are inside loops. If so, move them out or downgrade to debug. Asynchronous logging and batch flushing can also help, but reducing useless logs takes priority.

Should request parameters be printed in interface logs?

It's recommended to print desensitized input parameters to aid reproduction. However, exclude sensitive fields such as passwords, verification codes, and encryption keys, replacing them with asterisks.

If you're developing a new system, it's recommended to follow the "three-layer log model" and include traceId and key business fields from the first version, rather than waiting for an outage to add them. For legacy systems, first use the "three-step verification method" to check existing logs, then fill in the gaps for critical links. Logs are not the more the better; sufficient, searchable, and leak-free is the correct standard. When troubleshooting no longer relies on guessing or repeatedly checking code, this approach passes.