-
Notifications
You must be signed in to change notification settings - Fork 33
feat(mastra): support Mastra v1 observability + map spans to gen_ai conventions #182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| ## [0.2.0] - 2026-06-03 | ||
| ### Feature | ||
| - Support Mastra v1 observability: `createFIObservability` / `createFIMastraExporter` wire Future AGI into the new `@mastra/observability` pipeline (Mastra v1 removed the `telemetry:` config the previous exporter relied on). | ||
| - Map Mastra spans to Future AGI's `gen_ai.*` conventions — span kind (`FISpanKind`) and `input.value` / `output.value` enrichment — so traces render fully in the Future AGI UI. | ||
| - Move the legacy v0.x `FITraceExporter` integration to the `@traceai/mastra/legacy` subpath. | ||
|
|
||
| ## [0.1.0] | ||
| ### Feature | ||
| - Initial release: `FITraceExporter` for Mastra v0.x (`telemetry:` config). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
149 changes: 149 additions & 0 deletions
149
typescript/packages/traceai_mastra/src/FIMastraSpanExporter.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,149 @@ | ||
| import { BaseExporter } from "@mastra/observability"; | ||
| import { SpanConverter } from "@mastra/otel-exporter"; | ||
| import { SpanType } from "@mastra/core/observability"; | ||
| import type { AnyExportedSpan, TracingEvent } from "@mastra/core/observability"; | ||
| import { FISpanKind } from "@traceai/fi-semantic-conventions"; | ||
| import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto"; | ||
| import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base"; | ||
| import type { ReadableSpan } from "@opentelemetry/sdk-trace-base"; | ||
|
|
||
| export interface FISpanExporterConfig { | ||
| /** Full OTLP traces endpoint URL. */ | ||
| endpoint: string; | ||
| /** Auth + extra headers for the export request. */ | ||
| headers: Record<string, string>; | ||
| /** Service name (also the resource `service.name`). */ | ||
| serviceName: string; | ||
| /** OTel resource attributes (must include `project_name` / `project_type`). */ | ||
| resourceAttributes: Record<string, string>; | ||
| /** Export request timeout in ms. */ | ||
| timeout?: number; | ||
| /** Spans per export batch. */ | ||
| batchSize?: number; | ||
| } | ||
|
|
||
| /** | ||
| * Mastra span type → Future AGI span kind (`gen_ai.span.kind`). | ||
| * | ||
| * The FI backend maps these (case-insensitively) to its observation types | ||
| * (llm / agent / tool / chain / ...). Mastra emits `gen_ai.operation.name` but | ||
| * NOT a span kind, so without this mapping every span renders as "unknown". | ||
| */ | ||
| const SPAN_KIND_BY_TYPE: Partial<Record<SpanType, FISpanKind>> = { | ||
| [SpanType.AGENT_RUN]: FISpanKind.AGENT, | ||
| [SpanType.MODEL_GENERATION]: FISpanKind.LLM, | ||
| [SpanType.MODEL_INFERENCE]: FISpanKind.LLM, | ||
| [SpanType.MODEL_STEP]: FISpanKind.CHAIN, | ||
| [SpanType.TOOL_CALL]: FISpanKind.TOOL, | ||
| [SpanType.MCP_TOOL_CALL]: FISpanKind.TOOL, | ||
| [SpanType.CLIENT_TOOL_CALL]: FISpanKind.TOOL, | ||
| [SpanType.WORKFLOW_RUN]: FISpanKind.CHAIN, | ||
| [SpanType.WORKFLOW_STEP]: FISpanKind.CHAIN, | ||
| [SpanType.WORKFLOW_CONDITIONAL]: FISpanKind.CHAIN, | ||
| [SpanType.WORKFLOW_PARALLEL]: FISpanKind.CHAIN, | ||
| [SpanType.WORKFLOW_LOOP]: FISpanKind.CHAIN, | ||
| [SpanType.GENERIC]: FISpanKind.CHAIN, | ||
| [SpanType.RAG_EMBEDDING]: FISpanKind.EMBEDDING, | ||
| [SpanType.RAG_VECTOR_OPERATION]: FISpanKind.RETRIEVER, | ||
| }; | ||
|
|
||
| /** | ||
| * Write `<key>.value` + `<key>.mime_type` the way the FI backend's `set_io_value` | ||
| * expects, so the trace UI renders the input/output. | ||
| */ | ||
| function setIoValue( | ||
| attrs: Record<string, unknown>, | ||
| key: "input" | "output", | ||
| value: unknown, | ||
| ): void { | ||
| if (value === undefined || value === null) return; | ||
| if (typeof value === "object") { | ||
| attrs[`${key}.value`] = JSON.stringify(value); | ||
| attrs[`${key}.mime_type`] = "application/json"; | ||
| } else { | ||
| attrs[`${key}.value`] = String(value); | ||
| attrs[`${key}.mime_type`] = "text/plain"; | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Enrich a converted OTLP span in place with the attributes Future AGI keys on: | ||
| * `gen_ai.span.kind` (from the Mastra span type) and `input.value`/`output.value` | ||
| * (from the span's input/output). Existing values are not overwritten. Exported | ||
| * for unit testing. | ||
| */ | ||
| export function enrichSpan(otelSpan: ReadableSpan, span: AnyExportedSpan): void { | ||
| const attrs = otelSpan.attributes as Record<string, unknown>; | ||
|
|
||
| // Span kind so the span isn't typed "unknown" in Future AGI. | ||
| const kind = SPAN_KIND_BY_TYPE[span.type]; | ||
| if (kind && attrs["gen_ai.span.kind"] == null) { | ||
| attrs["gen_ai.span.kind"] = kind; | ||
| } | ||
|
|
||
| // input.value / output.value so prompt / response / tool I/O renders. | ||
| // (SpanConverter puts these under gen_ai.*/mastra.* keys the backend doesn't | ||
| // surface in the I/O preview.) | ||
| const s = span as { input?: unknown; output?: unknown }; | ||
| if (attrs["input.value"] == null) setIoValue(attrs, "input", s.input); | ||
| if (attrs["output.value"] == null) setIoValue(attrs, "output", s.output); | ||
| } | ||
|
|
||
| /** | ||
| * Mastra v1 observability exporter for Future AGI. | ||
| * | ||
| * Reuses `@mastra/otel-exporter`'s {@link SpanConverter} to turn Mastra spans | ||
| * into OTLP spans (standard `gen_ai.*` conventions), then enriches each span with | ||
| * the attributes Future AGI keys on for display (`gen_ai.span.kind`, | ||
| * `input.value` / `output.value`), and ships OTLP http/protobuf to the collector. | ||
| */ | ||
| export class FIMastraSpanExporter extends BaseExporter { | ||
| name = "future-agi"; | ||
| private readonly converter: SpanConverter; | ||
| private readonly processor: BatchSpanProcessor; | ||
|
|
||
| constructor(config: FISpanExporterConfig) { | ||
| super(); | ||
| this.converter = new SpanConverter({ | ||
| format: "GenAI_v1_38_0", | ||
| packageName: "@traceai/mastra", | ||
| serviceName: config.serviceName, | ||
| // Only `resourceAttributes` is read off this config by the converter. | ||
| config: { resourceAttributes: config.resourceAttributes } as any, | ||
| }); | ||
| const exporter = new OTLPTraceExporter({ | ||
| url: config.endpoint, | ||
| headers: config.headers, | ||
| ...(config.timeout !== undefined ? { timeoutMillis: config.timeout } : {}), | ||
| }); | ||
| this.processor = new BatchSpanProcessor( | ||
| exporter, | ||
| config.batchSize !== undefined | ||
| ? { maxExportBatchSize: config.batchSize } | ||
| : undefined, | ||
| ); | ||
| } | ||
|
|
||
| protected async _exportTracingEvent(event: TracingEvent): Promise<void> { | ||
| if (event.type !== "span_ended") return; | ||
| const span = event.exportedSpan; | ||
| try { | ||
| const otelSpan = await this.converter.convertSpan(span); | ||
| enrichSpan(otelSpan, span); | ||
| this.processor.onEnd(otelSpan); | ||
| } catch (error) { | ||
| this.logger.error( | ||
| `[@traceai/mastra] Failed to export span ${span.id}`, | ||
| error as Error, | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| async flush(): Promise<void> { | ||
| await this.processor.forceFlush(); | ||
| } | ||
|
|
||
| async shutdown(): Promise<void> { | ||
| await this.processor.shutdown(); | ||
| } | ||
| } |
158 changes: 158 additions & 0 deletions
158
typescript/packages/traceai_mastra/src/FIObservability.ts
|
NVJKKartik marked this conversation as resolved.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| import { Observability } from "@mastra/observability"; | ||
| import { SpanType } from "@mastra/core/observability"; | ||
| import { | ||
| FIMastraSpanExporter, | ||
| type FISpanExporterConfig, | ||
| } from "./FIMastraSpanExporter.js"; | ||
|
|
||
|
|
||
| const DEFAULT_FI_BASE_URL = "https://api.futureagi.com"; | ||
| const FI_TRACES_PATH = "/tracer/v1/traces"; | ||
|
|
||
| export interface FIMastraExporterOptions { | ||
| /** Service name (resource `service.name`). Defaults to `"mastra-app"`. */ | ||
| serviceName?: string; | ||
| /** Future AGI API key. Defaults to `process.env.FI_API_KEY`. */ | ||
| apiKey?: string; | ||
| /** Future AGI secret key. Defaults to `process.env.FI_SECRET_KEY`. */ | ||
| secretKey?: string; | ||
| /** | ||
| * Full traces endpoint URL. When set, overrides `baseUrl`. | ||
| * Defaults to `${baseUrl}/tracer/v1/traces`. | ||
| */ | ||
| endpoint?: string; | ||
| /** | ||
| * Base collector URL. The `/tracer/v1/traces` path is appended. | ||
| * Defaults to `process.env.FI_BASE_URL` or `https://api.futureagi.com`. | ||
| */ | ||
| baseUrl?: string; | ||
| /** Extra headers merged into the export request. */ | ||
| headers?: Record<string, string>; | ||
| /** | ||
| * Future AGI project name. This is the `project_name` resource attribute the | ||
| * Future AGI collector keys on — WITHOUT it, traces are ingested but no project | ||
| * is created and nothing shows in the dashboard. Defaults to `FI_PROJECT_NAME` | ||
| * env, then (via {@link createFIObservability}) the `serviceName`. | ||
| */ | ||
| projectName?: string; | ||
| /** | ||
| * Future AGI project type. `"observe"` for continuous tracing (default), | ||
| * `"experiment"` for evaluation runs with project versions. | ||
| */ | ||
| projectType?: "observe" | "experiment"; | ||
| /** Extra OTel resource attributes merged onto every span's resource. */ | ||
| resourceAttributes?: Record<string, string>; | ||
| /** Export request timeout in ms. */ | ||
| timeout?: number; | ||
| /** Spans per export batch. */ | ||
| batchSize?: number; | ||
| } | ||
|
|
||
| /** @internal Exported for unit testing; not part of the public package API. */ | ||
| export function resolveResourceAttributes( | ||
| options: FIMastraExporterOptions, | ||
| ): Record<string, string> { | ||
| const projectName = options.projectName ?? process.env.FI_PROJECT_NAME; | ||
| return { | ||
| // Future AGI keys the project on these resource attributes. | ||
| ...(projectName ? { project_name: projectName } : {}), | ||
| project_type: options.projectType ?? "observe", | ||
| ...(options.resourceAttributes ?? {}), | ||
| }; | ||
| } | ||
|
|
||
| /** @internal Exported for unit testing; not part of the public package API. */ | ||
| export function resolveEndpoint(options: FIMastraExporterOptions): string { | ||
| if (options.endpoint) return options.endpoint; | ||
| const base = options.baseUrl ?? process.env.FI_BASE_URL ?? DEFAULT_FI_BASE_URL; | ||
| return base.replace(/\/+$/, "") + FI_TRACES_PATH; | ||
| } | ||
|
|
||
| /** @internal Exported for unit testing; not part of the public package API. */ | ||
| export function resolveAuth(options: FIMastraExporterOptions): { | ||
| apiKey: string; | ||
| secretKey: string; | ||
| } { | ||
| const apiKey = options.apiKey ?? process.env.FI_API_KEY; | ||
| const secretKey = options.secretKey ?? process.env.FI_SECRET_KEY; | ||
| if (!apiKey || !secretKey) { | ||
| throw new Error( | ||
| "[@traceai/mastra] Missing Future AGI credentials. Set FI_API_KEY and " + | ||
| "FI_SECRET_KEY environment variables, or pass { apiKey, secretKey } to " + | ||
| "createFIMastraExporter()/createFIObservability().", | ||
| ); | ||
| } | ||
| return { apiKey, secretKey }; | ||
| } | ||
|
|
||
| /** | ||
| * Create a Mastra v1 observability exporter pre-configured for Future AGI. | ||
| * | ||
| */ | ||
| export function createFIMastraExporter( | ||
| options: FIMastraExporterOptions = {}, | ||
| ): FIMastraSpanExporter { | ||
| const { apiKey, secretKey } = resolveAuth(options); | ||
| const config: FISpanExporterConfig = { | ||
| endpoint: resolveEndpoint(options), | ||
| headers: { | ||
| "x-api-key": apiKey, | ||
| "x-secret-key": secretKey, | ||
| ...(options.headers ?? {}), | ||
| }, | ||
| serviceName: options.serviceName ?? "mastra-app", | ||
| resourceAttributes: resolveResourceAttributes(options), | ||
| ...(options.timeout !== undefined ? { timeout: options.timeout } : {}), | ||
| ...(options.batchSize !== undefined ? { batchSize: options.batchSize } : {}), | ||
| }; | ||
| return new FIMastraSpanExporter(config); | ||
| } | ||
|
|
||
| export interface FIObservabilityOptions extends FIMastraExporterOptions { | ||
| /** | ||
| * Mastra span types to drop before export. Defaults to `[SpanType.MODEL_CHUNK]` | ||
| * — per-chunk streaming spans that are pure noise in an observability backend. | ||
| * Pass `[]` to export everything. | ||
| */ | ||
| excludeSpanTypes?: SpanType[]; | ||
| } | ||
|
|
||
| /** | ||
| * Create a ready-to-use Mastra v1 {@link Observability} instance wired to Future AGI. | ||
| * | ||
| * @example | ||
| * ```ts | ||
| * import { Mastra } from "@mastra/core"; | ||
| * import { createFIObservability } from "@traceai/mastra"; | ||
| * | ||
| * export const mastra = new Mastra({ | ||
| * agents: { ... }, | ||
| * observability: createFIObservability({ serviceName: "my-app" }), | ||
| * }); | ||
| * ``` | ||
| */ | ||
| export function createFIObservability( | ||
| options: FIObservabilityOptions = {}, | ||
| ): Observability { | ||
| const { | ||
| serviceName = "mastra-app", | ||
| excludeSpanTypes = [SpanType.MODEL_CHUNK], | ||
| ...exporterOptions | ||
| } = options; | ||
| return new Observability({ | ||
| configs: { | ||
| otel: { | ||
| serviceName, | ||
| excludeSpanTypes, | ||
| exporters: [ | ||
| createFIMastraExporter({ | ||
| ...exporterOptions, | ||
| serviceName, | ||
| // Default the FI project to the service name so a project is created. | ||
| projectName: exporterOptions.projectName ?? serviceName, | ||
| }), | ||
| ], | ||
| }, | ||
| }, | ||
| }); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.