Skip to content

[kernel-1116] browser logging: Event Schema & Pipeline #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
archandatta wants to merge 10 commits into
base: main
Choose a base branch
from
Open

[kernel-1116] browser logging: Event Schema & Pipeline #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
a03a5bd
feat: add failing test stubs for events package
archandatta Mar 20, 2026
42f415f
feat: add BrowserEvent struct, CategoryFor, and truncateIfNeeded
archandatta Mar 20, 2026
fa67dff
feat: add RingBuffer with closed-channel broadcast fan-out
archandatta Mar 20, 2026
115c720
feat: add FileWriter per-category JSONL appender
archandatta Mar 20, 2026
f07e40d
feat: add Pipeline glue type sequencing truncation, file write, and r…
archandatta Mar 20, 2026
18fdb6d
review: fix truncateIfNeeded branch split, atomic.Pointer[string], Re…
archandatta Mar 27, 2026
997edb4
review: remove dead RingBuffer count field, fix FileWriter mutex doc,…
archandatta Mar 27, 2026
e5153da
chore: clean up maxS2RecordBytes comment
archandatta Mar 27, 2026
1644fe7
fix: serialise Pipeline.Publish to guarantee monotonic seq delivery o…
archandatta Mar 27, 2026
36cff2d
review
archandatta Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions server/lib/events/event.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
package events

import (
"encoding/json"
)

// maxS2RecordBytes is the maximum record size for the S2 event pipeline (1 MB).
const maxS2RecordBytes = 1_000_000

// EventCategory determines type of logging
type EventCategory string

const (
CategoryConsole EventCategory = "console"
CategoryNetwork EventCategory = "network"
CategoryPage EventCategory = "page"
CategoryInteraction EventCategory = "interaction"
CategoryLiveview EventCategory = "liveview"
CategoryCaptcha EventCategory = "captcha"
CategorySystem EventCategory = "system"
)

type Source string

const (
SourceCDP Source = "cdp"
SourceKernelAPI Source = "kernel_api"
SourceExtension Source = "extension"
SourceLocalProcess Source = "local_process"
)

type DetailLevel string

const (
DetailMinimal DetailLevel = "minimal"
DetailDefault DetailLevel = "default"
Copy link
Copy Markdown
Contributor

@rgarcia rgarcia Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if this is meant to be a concrete level, i'd consider renaming default to standard. minimal/standard/verbose/raw reads like an actual ladder, whereas default feels more like a policy alias whose meaning could vary by producer.

DetailVerbose DetailLevel = "verbose"
DetailRaw DetailLevel = "raw"
)

// BrowserEvent is the canonical event structure for the browser capture pipeline.
type BrowserEvent struct {
Copy link
Copy Markdown
Contributor

@rgarcia rgarcia Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: i wonder if Event would be a cleaner name than BrowserEvent. it feels more future-proof if extension/local-process/server-native producers end up as first-class peers.

CaptureSessionID string `json:"capture_session_id"`
Copy link
Copy Markdown
Contributor

@rgarcia rgarcia Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is capture_session_id intended to identify? this reads more like pipeline/native-producer lifecycle metadata than part of the portable event schema. if resume/subscription are meant to work across heterogeneous producers, i'd be careful about baking a producer-owned session id into every event.

Seq uint64 `json:"seq"`
Ts int64 `json:"ts"`
Type string `json:"type"`
Category EventCategory `json:"category"`
Source Source `json:"source"`
Copy link
Copy Markdown
Contributor

@rgarcia rgarcia Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is starting to feel like provenance wants to be an object rather than multiple top-level fields. i'd consider making source structured and moving cdp-specific context under source.metadata, e.g.

{
  "type": "console.log",
  "category": "console",
  "detail_level": "standard",
  "source": {
    "kind": "cdp",
    "event": "Runtime.consoleAPICalled",
    "metadata": {
      "target_id": "...",
      "cdp_session_id": "...",
      "frame_id": "...",
      "parent_frame_id": "..."
    }
  },
  "data": { ... }
}

that keeps the top level focused on stable cross-producer fields and makes non-cdp producers feel first-class too.

SourceEvent string `json:"source_event,omitempty"`
DetailLevel DetailLevel `json:"detail_level"`
TargetID string `json:"target_id,omitempty"`
CDPSessionID string `json:"cdp_session_id,omitempty"`
FrameID string `json:"frame_id,omitempty"`
ParentFrameID string `json:"parent_frame_id,omitempty"`
URL string `json:"url,omitempty"`
Data json.RawMessage `json:"data,omitempty"`
Truncated bool `json:"truncated,omitempty"`
}

// truncateIfNeeded marshals ev and returns the (possibly truncated) event together
func truncateIfNeeded(ev BrowserEvent) (BrowserEvent, []byte) {
data, err := json.Marshal(ev)
if err != nil {
return ev, data
Copy link
Copy Markdown

@cursor cursor bot Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marshal error silently writes corrupt JSONL line

Low Severity

truncateIfNeeded silently swallows json.Marshal errors and returns nil data (since Marshal returns nil on error). The caller in Pipeline.Publish passes this nil data to FileWriter.Write, which executes append(nil, '\n') and writes a bare newline to the JSONL file — a corrupt, non-JSON line. The function returns no error, so the pipeline cannot detect the failure. The first error path (return ev, data) and second (return ev, nil) are also inconsistent, suggesting an oversight.

Additional Locations (1)
Fix in Cursor Fix in Web

Copy link
Copy Markdown
Contributor

@Sayan- Sayan- Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

==

}
if len(data) <= maxS2RecordBytes {
return ev, data
}
ev.Data = json.RawMessage("null")
ev.Truncated = true
data, err = json.Marshal(ev)
if err != nil {
return ev, nil
}
return ev, data
}
Loading
Loading