Parse AI Coding JSONL Safely Before You Infer Completion

JSONL looks simple because every newline appears to separate one object. A live coding-agent transcript adds harder constraints: the final record may be incomplete, payloads may be unusually large, events may be duplicated, and an old completion can be invalidated by later activity.

Bound the reader before parsing

Use a streaming reader and an explicit line policy. Do not load an unbounded transcript into one string. Agent Island v1.7.1 uses platform-specific safeguards: the macOS Claude reader has a 64 MiB backstop, while the Windows reader skips a line above 1,000,000 characters before JSON parsing.

These are not identical policies and should not be reported as one shared limit. They reflect different released implementations. Both policies make malformed or partial records non-fatal for the rest of the file.

Parse only fields that can change state

Broad deserialization creates accidental dependencies on unrelated payloads. A state monitor needs a smaller projection.

For Claude events, useful fields include type, uuid, timestamp, message.stop_reason, isSidechain, isApiErrorMessage, and toolEndsTurn. For Codex events, the released readers inspect fields such as type, payload.type, payload.turn_id, role, timestamp, completed_at, and started_at.

read bounded line
  -> parse JSON
  -> project provider-specific state fields
  -> attach event identity and semantic time
  -> reduce ordered events into session state

Completion is a candidate, not a permanent fact

A completion-shaped event belongs to one turn. A later user message or start event means the session moved on. The reducer must therefore compare semantic order and allow later activity to supersede earlier completion.

This rule prevents a stale completion from pinning the monitor after the next turn has started. It also explains why the newest file timestamp alone is not enough: multiple turns live inside the same transcript.

Reject false completion envelopes

Rate-limit and API-error assistant envelopes are not successful completion. Sidechain or subagent completion is not automatically a main-thread handoff. Tool activity can also continue after a model stop marker.

Provider-specific guards should run before the generic completion reducer. Otherwise a syntactically valid line can produce a semantically wrong alarm.

Let file changes trigger a fresh reduction

Agent Island watches .jsonl files for changes and retains polling as a fallback. Event-driven observation reduces delay, while polling protects against missed watcher notifications. A partial line skipped now can be reconsidered after the next change.

Cache unchanged files by stable fingerprints such as path, size, and modification time, but invalidate that cache when the source changes. Caching is an optimization; it must not freeze an earlier state.

Tests that catch real parser mistakes

1. A blank or malformed line does not abort later valid records.
2. A partial final line becomes readable after a later append.
3. An oversized line follows the documented platform policy.
4. A duplicated event does not create a second alarm.
5. A later user or start event supersedes an older completion.
6. An API-error envelope does not become completed.
7. A sidechain finish does not page the user for the parent session.
8. A missed watcher event is recovered by polling.

Safe JSONL parsing is not just valid JSON handling. It is bounded I/O plus provider-aware event semantics, ordering, deduplication, and refresh behavior.