ShipGauge

Introduction

Pre-ship gate for cost and quality — metadata only, never your prompts

ShipGauge is the control plane for shipping LLM features safely: track cost per feature, block regressions in CI, and enforce budgets in production.

Who it’s for

Indie builders and small teams shipping AI features who need:

  • Per-feature spend visibility (feature_id) without standing up a full observability stack
  • A CI gate that fails merges when quality or cost regresses
  • Soft alerts and hard budget blocks in production

60-second mental model

  1. Instrument your app with the Node or Python SDK (or the ingest API). Every call is tagged with a feature_id.
  2. Gate pull requests with eval suites + shipgauge gate check (optional required GitHub check).
  3. Control production spend with daily/monthly budgets and alerts.
SDK / API  →  ShipGauge project  →  Overview spend
                 ↓
          Eval suites + CI gate
                 ↓
          Budget rules + Slack/email

How ShipGauge differs

Observability tools help you see traces and prompts. Eval suites help you measure quality. ShipGauge is the decision layer: whether this change should ship, and whether this feature can keep spending.

Privacy promise

ShipGauge stores metadata only — model, tokens, cost, latency, feature and customer ids. Prompts and completions are never sent to ShipGauge.

Next