Co-VibeDocs

Engineering

Verified Agent Telemetry#

Co-Vibe follows Observal's methodology, not its AGPL implementation code: observe the harness/proxy/provider boundary and parse structured telemetry server-side instead of trusting an agent to summarize its own spend.

Trusted Sources#

  • Codex/OpenAI: Codex can emit JSON/OTel events whose usage fields include usage.input_tokens, cached-input details, output tokens, reasoning output, and usage.total_tokens. Co-Vibe accepts OpenAI response payloads and Codex OTel span attributes, and the local companion submits Codex Desktop response.completed split counters from numeric fields only, then persists the parsed counters exactly.
  • Claude: Claude Code/SDK exposes result usage and observability metrics. Co-Vibe accepts Claude SDK/API usage shapes, including modelUsage, input_tokens, output_tokens, cache_creation_input_tokens, and cache_read_input_tokens. The project Claude Code hook reads transcript JSONL usage records and submits only those counters through the verified telemetry tool.
  • Cursor: The Co-Vibe Cursor hook and covibe-local watch sweep counter-only usage from Cursor's agent transcripts and local state database (per-session token splits, bubble token counts, Cursor's own cost accounting) and submit it as direct_counts. See cursor-integration.md.
  • Proxy/harness: A Co-Vibe wrapper may submit direct_counts only when it captured counters from a structured harness/provider event.

Trust Rules#

  • covibe_ingest_agent_telemetry is the only usage-metric path. It parses model, token, cache, and total counters server-side, stores a SHA-256 hash of the raw payload, and marks the span verified = 1.
  • Claude Code hooks may start and end agent sessions, but token/cost metrics still enter only through covibe_ingest_agent_telemetry.
  • Co-Vibe never stores raw prompts, model responses, or complete telemetry payloads in agent_spans.
  • Submitted cost is never trusted blindly. Co-Vibe recomputes cost when a known model has a versioned server pricing entry, uses provider-estimated cost only when pricing is unknown, and flags cost_mismatch = 1 when submitted cost disagrees with parsed usage.
  • Token and model attribution are deterministic for supported payloads. Dollar cost is an estimate unless the source is a provider billing/export system.

Supported Payload Kinds#

  • openai_response
  • codex_otel_span
  • anthropic_message
  • claude_sdk_result
  • direct_counts

Use telemetry_source to describe where the structured event came from: harness_captured, proxy_captured, provider_reported, or otel_captured.

Codex export format#

Codex emits OTel-style spans. Co-Vibe accepts payload_kind codex_otel_span and reads counters from the span attributes object (or, if the file is flat, from the root object). Only these fields are read — derive nothing else:

  • model: gen_ai.response.model (falls back to gen_ai.request.model)
  • input tokens: gen_ai.usage.input_tokens (or codex.turn.token_usage.input_tokens)
  • output tokens: gen_ai.usage.output_tokens (or codex.turn.token_usage.output_tokens)
  • cache-read tokens: gen_ai.usage.cache_read.input_tokens (or codex.turn.token_usage.cached_input_tokens)
  • total tokens: codex.usage.total_tokens (or codex.turn.token_usage.total_tokens); omitted totals are summed from the counters above
  • source event id: codex.event.id, then gen_ai.response.id, then span_id/id

Provider is always normalized to openai. Export counters only — no prompts, messages, responses, or transcript text. Write one JSON object (or an array of them) per run to .covibe/telemetry/*.json:

json
{
  "span_id": "codex-span-1",
  "attributes": {
    "gen_ai.response.model": "gpt-5-codex",
    "gen_ai.usage.input_tokens": 1200,
    "gen_ai.usage.cache_read.input_tokens": 800,
    "gen_ai.usage.output_tokens": 350,
    "codex.usage.total_tokens": 2350
  }
}

Flush the inbox (the --base-url value is the Co-Vibe origin only):

bash
npm exec -- covibe-local telemetry \
  --inbox .covibe/telemetry \
  --base-url <origin> \
  --payload-kind codex_otel_span \
  --telemetry-source otel_captured \
  --agent-type codex \
  --agent-name "Codex"

For the example above (gpt-5-codex, 400 uncached input + 800 cache-read + 350 output) the server prices the span itself: cost 0.0041, cost_source = server_pricing, verified = 1. No submitted cost is trusted.

Codex local response sync#

covibe-local watch also checks Codex Desktop's local SQLite databases when present. This is an external telemetry source, not Co-Vibe app storage: the companion reads the local Codex thread database only to map the current git repo root to Codex thread ids, then extracts numeric response.completed counters from the local Codex logs database. It submits split direct_counts only: response.id, response.model, usage.input_tokens, usage.output_tokens, usage.input_tokens_details.cached_tokens, and usage.total_tokens.

The first run primes ignored .covibe/codex-usage-state.json with existing response ids and does not backfill historical responses. Later runs submit events like:

json
{
  "provider": "openai",
  "model": "gpt-5.5",
  "input_tokens": 194,
  "output_tokens": 6,
  "cache_read_tokens": 181,
  "total_tokens": 200,
  "source_event_id": "codex-response:resp-2"
}

Run it once manually with:

bash
npm exec -- covibe-local telemetry \
  --base-url <origin> \
  --codex \
  --agent-type codex \
  --agent-name "Codex Local"

Or run continuous sync:

bash
npm exec -- covibe-local watch --base-url <origin>

The companion does not submit or store raw Codex logs, prompts, responses, transcript text, or SSE bodies. If the split logs database or sqlite3 reader is unavailable, no Codex usage event is submitted; watch continues with snapshots, inbox flushes, and heartbeats.

Cursor export format#

Cursor installs wired by covibe-local setup capture usage automatically (see cursor-integration.md); the manual format below also serves custom wrappers. Cursor exports counter-only usage, so Co-Vibe accepts payload_kind direct_counts and reads flat fields straight from the JSON object:

  • provider: provider (or the --agent/tool provider field)
  • model: model (or the tool model field)
  • input tokens: input_tokens
  • output tokens: output_tokens
  • cache-read tokens: cache_read_tokens
  • cache-write tokens: cache_write_tokens
  • total tokens: total_tokens (omitted totals are summed from the counters above)
  • source event id: source_event_id (or id)

Provider/model must be in the payload (or passed as flags) because Co-Vibe needs them to price the span. Export counters only. Write one JSON object (or array) per run to .covibe/telemetry/*.json:

json
{
  "provider": "anthropic",
  "model": "claude-sonnet-4-6",
  "input_tokens": 900,
  "output_tokens": 300,
  "cache_read_tokens": 200,
  "cache_write_tokens": 150,
  "total_tokens": 1550,
  "source_event_id": "cursor-span-1"
}

Flush the inbox:

bash
npm exec -- covibe-local telemetry \
  --inbox .covibe/telemetry \
  --base-url <origin> \
  --payload-kind direct_counts \
  --telemetry-source harness_captured \
  --agent-type cursor \
  --agent-name "Cursor"

For the example above (claude-sonnet-4-6) the server prices it cache-aware: cost 0.007823, cost_source = server_pricing, verified = 1. When a model has no server pricing entry, cost_source falls back to provider_estimate (if the payload carried total_cost_usd/cost_usd) or unknown.

Local Import#

For Codex, OpenAI wrappers, Cursor wrappers, or CI harnesses that can export usage JSON, developers can submit one file or flush the local telemetry inbox. Use the install command shown in Agent Setup. Hosted builds serve the companion from /downloads/co-vibe.tgz, so the default command looks like:

bash
npm install --save-dev https://your-co-vibe-host/downloads/co-vibe.tgz \
  && printf "Co-Vibe MCP token: " \
  && read -rs COVIBE_MCP_TOKEN \
  && printf "\n" \
  && export COVIBE_MCP_TOKEN \
  && npm exec -- covibe-local setup --base-url https://your-co-vibe-host \
  && npm exec -- covibe-local doctor --base-url https://your-co-vibe-host

Then submit the usage file:

bash
npm exec -- covibe-local telemetry \
  --base-url https://your-co-vibe-host \
  --file usage.json \
  --payload-kind codex_otel_span \
  --telemetry-source otel_captured \
  --agent-type codex \
  --agent-name "Codex" \
  --latency-ms 450

The setup, doctor, and telemetry --base-url values must be the Co-Vibe origin only; credential-bearing URLs, callback URLs, query strings, hashes, and other paths are rejected before local config is written. The setup block is chained so telemetry import comes after a successful install, setup, and doctor check.

For recurring local capture, write one usage JSON file per run into .covibe/telemetry and flush the inbox:

bash
npm exec -- covibe-local telemetry \
  --base-url https://your-co-vibe-host \
  --inbox .covibe/telemetry \
  --payload-kind direct_counts \
  --telemetry-source harness_captured \
  --agent-type cursor \
  --agent-name "Cursor"

The command accepts one JSON object or an array of usage events. It rejects files that look like raw prompts, messages, transcripts, content, inputs, outputs, responses, queries, or completions; export usage counters only. Use --latency-ms when the harness has elapsed runtime metadata for the usage event, and --cost-usd only for provider-estimated cost. Numeric telemetry flags must be finite non-negative values and fail before submission when invalid. Successfully submitted inbox files move to .covibe/telemetry/sent so they are not double-counted; failed ingest leaves inbox files in place for retry.

View as .md