On this page
Define the event start and continuation rule
Write a rule that says which line starts an event and which lines continue it. A timestamp at the beginning of a line may be a start marker, but a stack trace can contain timestamps or lines that resemble headers. Some exporters use indentation, a delimiter, a record length, or a structured envelope instead. Record the source format and the exact rule used for the bounded capture.
RFC 5424 defines a syslog message structure rather than a universal multiline convention RFC Editor, (opens in a new tab). Do not apply a newline split to every log family and call the result events. If a capture mixes formats, partition it first and keep the parser or owner rule attached to each partition.
| Boundary field | Record | Stop when |
|---|---|---|
| start marker | timestamp, length, delimiter, or unknown | Every line looks like a start |
| continuation | indent, prefix, envelope, or rule | Continuation evidence is absent |
| source stream | file, socket, batch, or shard | Streams were concatenated |
| offset | line or byte range for event | Offsets were discarded |
| termination | next start, length, end marker, or EOF | Final event may be incomplete |
| format version | protocol or exporter version | Format inferred from examples |
Keep timestamp roles separate
A log can carry a time when the event happened, a time when it was observed by a collector, and a time when it was exported. These are not interchangeable. OpenTelemetry names time and observed time separately and also models severity, body, attributes, and trace context OpenTelemetry, (opens in a new tab). Store each field with its source and precision, then say which one orders the capture.
A timestamp order can disagree with arrival order because of buffering, clock skew, retries, or multiple writers. Do not sort away the original sequence. Keep source offsets, sequence fields, and transport order available so a reviewer can distinguish business chronology from delivery chronology.
| Field | Question | Do not infer |
|---|---|---|
| event time | When does the source say it happened? | The collector saw it then |
| observed time | When did the observer receive it? | Events are business-ordered |
| export time | When was it written or shipped? | Export order is event order |
| sequence | Does the source provide a sequence? | Timestamp ties are duplicates |
| timezone | What zone or offset is stated? | Local wall time is UTC |
| precision | seconds, milliseconds, or unknown | Missing precision is zero |
Preserve structured fields beside rendered text
Keep the raw or redacted event boundary together with structured fields such as severity, body, attributes, trace id, span id, and resource identity. A pretty-printed message can contain a newline that was originally one body value. Conversely, a structured exporter can split one logical event into several records. Record whether the review target is the body, the full log record, or the displayed line.
Use the timestamp and timezone guide for offset and database evidence, but keep boundary logic here. A correctly converted timestamp does not repair a line that was attached to the wrong event. Keep data provenance and redaction decisions separate from event parsing.
Structured review questions
- Which field identifies the event start and which field identifies continuation?
- Are body newlines escaped, encoded, or literal?
- Which attributes belong to the event and which belong to the resource?
- Can a trace or span identifier repeat across separate log events?
- Which source offsets are retained after multiline reconstruction?
Use a synthetic multiline fixture
The fixture below tests a start marker, two continuation lines, a second event, and a final unterminated event. It is a text ledger, not a log parser. Count events from the declared rule and keep the line ranges. Add a line that resembles a start marker inside a stack trace to test whether the rule is too broad.
Keep synthetic text free of real IP addresses, account names, stack traces, or message contents. The point is to test the boundary contract, not to show an application’s private failure. If a production sample is needed, use an approved redacted artifact and record the source scope.
| Fixture finding | Disposition | Next evidence |
|---|---|---|
| Indented lines attach to prior event | continue | Check the source continuation rule |
| Stack line resembles a start marker | hold | Add a stronger boundary field |
| Final event has no terminator | incomplete | Obtain the next boundary or mark EOF |
| Two source formats are mixed | partition | Review each format separately |
| Line offsets retained | continue conditionally | Compare reconstruction with source count |
Synthetic event boundary ledger
line 1 | 10:00:01Z INFO E1 start
line 2 | detail: first continuation
line 3 | detail: second continuation
line 4 | 10:00:02Z WARN E2 start
line 5 | detail: continuation
line 6 | 10:00:03Z INFO E3 start (capture ends)
expected events: E1 lines 1-3; E2 lines 4-5; E3 incomplete lines 6-endSeparate delivery order from event order
When logs are retried or replayed, duplicate delivery can coexist with one event identity, while two distinct events can share a timestamp. Record any source event id, sequence, retry marker, and collector offset. Do not claim exactly-once delivery from a unique line number or deduplicate solely on rendered text. A replay review is a boundary and identity question, not a count of unique strings.
Report a narrow observation such as “three bounded event starts and one incomplete final event.” If sequence, clock, or identity fields are absent, mark them unavailable. Keep the unknown visible rather than inventing a session count or causal order.
| Finding | Disposition | Do not infer |
|---|---|---|
| Start rule and offsets agree in fixture | bounded pass | Every real line is an event |
| Continuation marker missing | hold | Newline means new event |
| Timestamp order differs from source order | retain both orders | Clock order is delivery order |
| No stable event id | deduplication unavailable | Equal text is the same event |
| Final boundary absent | mark incomplete | Capture is complete |
Close with a qualified preparation step
Once framing, timestamp roles, structured fields, source offsets, and incomplete states are documented, keep the packet at technical evidence level. Use the data provenance log for collection and transformation history. Do not upload raw logs merely to test a multiline rule.
For a supported non-health collection, with authority to describe the log source and a controlled handling boundary, the qualified appraisal path may be a next preparation step. It does not parse logs, reconstruct sessions, create a quote, match a buyer, issue a legal opinion, or grant a license. Event-boundary evidence also does not prove completeness or value.
Takeaways
- Define event starts and continuations for each source format.
- Keep event, observed, and export times separate from source order.
- Retain structured fields and offsets beside reconstructed text.
- Use synthetic multiline lines to test incomplete and ambiguous boundaries.
- Report bounded event evidence without claiming universal deduplication or completeness.