Customer Readiness Tests#

These focused checks back the hosted and local companion parts of build-loop-tests.md.

Local Companion Setup Test#

Prove hosted customers can prepare a repo without hand-editing config.

Verify Settings → Agent connection shows the hosted package command npm install --save-dev <host>/downloads/co-vibe.tgz, or an intentional COVIBE_COMPANION_INSTALL_COMMAND override. If using public npm, verify npm view co-vibe version.
Run npm exec -- covibe-local setup --base-url <host> and approve the browser device prompt; verify setup mints one token per detected agent into ~/.covibe/credentials.json (or pass --token / export COVIBE_MCP_TOKEN for automation).
Verify .mcp.json contains the Co-Vibe stdio bridge.
Verify .claude/settings.json contains the Claude Code session hooks.
Verify the raw MCP token is not written into either config file.
Verify existing non-Co-Vibe MCP servers and Claude hooks are preserved.
Verify .covibe/telemetry exists for Codex/Cursor/provider usage JSON.
Verify setup prints status checks and submits the first snapshot inside a git repo.
Run npm exec -- covibe-local doctor --base-url <host> and verify it rechecks Node, git, token, token-safe MCP config, config cwd, Claude hooks, and server reachability without rewriting config; stale base URLs, stale working directories, or raw token values must fail.
Verify missing token is a hard failure before local config is written and a missing or hung running server is a bounded warning.
Run npm exec -- covibe-local snapshot --base-url <host> inside a git repo for manual resyncs, and verify API error messages do not echo raw token-shaped values.
Verify Co-Vibe stores branch, dirty count, ahead/behind, changed file paths, unpushed file paths, and diff hashes without file contents.
Submit a second snapshot from another session with an overlapping dirty or unpushed file.
Verify the snapshot response returns a warning to the agent and stores repo.snapshot_conflict.
Write counter-only usage JSON into .covibe/telemetry, flush with npm exec -- covibe-local telemetry --base-url <host> --inbox .covibe/telemetry, verify the file archives under .covibe/telemetry/sent after success, verify failed ingest leaves the file in the inbox for retry, and verify raw input/output/response-style payload envelopes are rejected before ingest.
Run npm exec -- covibe-local telemetry --base-url <host> --codex with local Codex state and logs databases; verify the first run primes .covibe/codex-usage-state.json without backfilling old totals, then a later response.completed log submits input, output, cache-read, and total counters through covibe_ingest_agent_telemetry; if the logs database is absent, verify no Codex usage event is submitted.
Run npm exec -- covibe-local watch --base-url <host> --once --session-id <id> and verify it submits a snapshot, flushes telemetry inbox files, syncs Codex split usage counters when present, heartbeats the session, rejects invalid numeric flags, and does not submit file contents, transcripts, prompts, responses, raw SSE bodies, aggregate thread counters, or logs.
Verify the Settings → Agent connection command does not prompt for or put a raw token on the setup command line (setup uses the browser device approval instead), chains the copied commands so setup/doctor do not run after an install failure, does not start a foreground watch loop (setup installs the background machine service by default; --no-service skips it, and a temporary npx-cache install is refused with a hint), shows a separate copyable npm exec -- covibe-local watch --base-url <host> command, and verify setup still accepts --token without COVIBE_MCP_TOKEN in the environment for automation, uses it for readiness/snapshot calls, accepts --flag=value syntax, and does not print or store it.
Verify setup rejects credential-bearing, callback, path, query string, and hash --base-url values before writing .mcp.json or Claude settings.
Verify unknown setup flags and extra positional arguments fail before writing .mcp.json or Claude settings.
Run npm run smoke:package and verify a packed package installs into a clean external repo, preserves existing config, exposes covibe-local --help, covibe-local doctor, covibe-mcp --help, and covibe-claude-hook --help, writes package-backed MCP/Claude hooks, creates .covibe/telemetry, exposes npm exec -- covibe-mcp, runs npm exec -- covibe-local watch --base-url <host> --once, and does not store or print the raw token.

Hosted Deployment Canary Test#

Prove the hosted product is ready before customers connect local companions.

Run npm run check:deploy to verify the Docker and standalone output contract.
Run npm run readiness:customer with Docker, WorkOS env, and enough free disk; verify its preflight fails fast, reports every unavailable prerequisite together, and gives safe docker system df / docker builder prune cleanup guidance, including Docker reclaimable-space hints when Docker responds. Use npm run readiness:customer -- --preflight-only for quick prerequisite-only checks before launching the longer build/test/container sequence.
Run npm run smoke:container where Docker is available to prove the hosted image boots and fails closed without WorkOS env.
For our hosted Docker path, verify docker-compose.yml uses WorkOS mode, /app/data persistence, and either a configured external HTTPS load balancer or the built-in Caddy proxy profile: docker compose --profile proxy up -d --build.
Run npm run db:postgres:init, npm run smoke:postgres, npm run smoke:postgres:runtime, and npm run doctor:production with the deployed WorkOS environment. Real hosted origins and localhost rehearsals must use PostgreSQL; the local companion does not run a database.
Before a permanent domain exists, optionally run npm run readiness:hosted:https-rehearsal with authenticated ngrok to prove the public HTTPS contract against a temporary WorkOS-mode container.
Run npm run readiness:handoff -- https://... against the permanent customer origin. It must reject localhost, private-network, HTTP, callback, path, query, hash, credential-bearing URLs, and temporary tunnel domains. For https://co-vibe.dev, do not run this until security reviews are done and Cloud SQL/PostgreSQL env is configured. It must run a fast /api/health, /api/mcp, and /downloads/co-vibe.tgz preflight plus doctor:production before the long local gate, including exact compact MCP tools, sign-in reachability, protected API rejection, unauthenticated token creation rejection, hidden local-only routes, malformed MCP envelope rejection, and missing/invalid MCP-token rejection, plus token-shaped response leak checks, and companion package identity before customer readiness and permanent hosted readiness.
Verify /api/health returns ok, checks.auth = "workos", and at least 8 MCP tools.
Verify /api/mcp lists exactly the eight compact public tools and no legacy tools.
Verify /downloads/co-vibe.tgz is the minimal co-vibe companion package without unexpected dependencies or app files, installs in a throwaway repo, covibe-local --help, covibe-mcp --help, and covibe-claude-hook --help run, covibe-local setup writes token-safe config, covibe-local doctor passes token/config/server/hook checks, invalid-token manual covibe-local snapshot --base-url <host> fails without echoing the raw token, covibe-local watch --base-url <host> --once starts the separate sync loop, setup reports the first snapshot result without file contents, and covibe-mcp exposes the public tools.
Verify /auth/sign-in responds with a redirect to the hosted WorkOS AuthKit page (the canary accepts a 3xx with a Location header).
After a first sign-in with a fresh user, verify /onboarding creates the tenant and first workspace and lands on the dashboard; a user invited to a tenant that already has members sees the access-pending panel instead.
Verify /api/state, token list, and token creation reject unauthenticated users.
Verify local-dev-only routes return 404 in WorkOS mode.
Verify /api/mcp rejects a missing or invalid MCP token and does not echo the raw token.