Project metrics

Project metrics are custom application and eval measurements that show up in the Veryfront Metrics panel. They are separate from runtime traces and logs: the runtime installs the OpenTelemetry meter provider, while project code emits named instruments through veryfront/metrics. For runtime observability APIs and OTLP setup, see veryfront/observability.

Emit metrics

Use the SDK hook from app, agent, tool, task, workflow, and eval code:

import { metrics } from "veryfront/metrics";

metrics.counter("signup_total", 1, {
  plan: "pro",
  source: "checkout",
});

metrics.histogram("checkout_duration_seconds", 1.24, {
  step: "payment",
});

metrics.gauge("queue_depth", 42, {
  queue: "email",
});

Use counters for totals, histograms for durations and sizes, and gauges for current values. Prefer stable names with a prefix of your own so they are easy to discover in Studio. Names that start with ai_gateway_ or vf_ are reserved for platform metrics, except the vf_eval_ names used for eval metrics. When code runs inside Veryfront, the SDK adds request-scoped labels for project_id, project_slug, environment, and branch for preview requests. Preview requests without an explicit branch use branch="main". User code should not provide or trust those labels for isolation; the platform-owned request context wins. In shared Veryfront runtimes, service_name, service_version and deployment_environment name the runtime service, its release and the platform deployment, such as staging, that exported the sample.

Emit eval metrics

Eval definitions still use veryfront/eval metrics for pass/fail, scores, and reports. Add project metrics when you also want aggregate dashboards:

import { datasets, evalAgent, metrics as evalMetrics } from "veryfront/eval";
import { metrics } from "veryfront/metrics";

export default evalAgent({
  target: "agent:support",
  dataset: datasets.inline([
    { id: "q1", input: "Capital of France?", reference: "Paris" },
  ]),
  metrics: [evalMetrics.answer.exactMatch().gate()],
  async check(ctx) {
    const passed = ctx.record.output.text?.includes("Paris") === true;

    metrics.counter("vf_eval_result_total", 1, {
      eval_id: "support",
      metric: "answer.exactMatch",
      outcome: passed ? "pass" : "fail",
    });

    metrics.histogram("vf_eval_duration_ms", ctx.record.durationMs, {
      eval_id: "support",
    });
  },
});

Label policy

Metric labels become query dimensions. Keep them low-cardinality and safe:

Good labels	Avoid
`environment`, `branch`, `service`, `route`, `status`, `outcome`, `model`, `provider`, `eval_id`	User IDs, email addresses, prompts, outputs, request IDs, session IDs, raw URLs, secrets

Use a small allowlist per metric. Do not put tenant identity, project identity, credentials, or personally identifiable data into user-supplied labels. Project, environment, and preview branch labels are injected by the platform; values that project code passes for them are ignored. In Veryfront runtimes, the SDK drops a sample instead of exporting it when:

the metric name does not match [A-Za-z_][A-Za-z0-9_.:]* or is longer than 128 characters
it carries more than 16 labels of its own, a label name that does not match [A-Za-z_][A-Za-z0-9_.]* or is longer than 128 characters, or a label value longer than 256 characters
it would start a new name and label combination after the project has created 500 in the current runtime process

Shared Veryfront runtimes send project metrics through the platform with the project’s credential, and the platform files them under that project. It drops samples whose name is reserved for platform metrics.

Query sparse counters

Each runtime process exports its own cumulative totals and is identified by the service.instance.id resource attribute, so sum across instances after taking per-series differences. When a counter series is new, or has been idle for more than five minutes, the SDK exports its previous total just before the new one. That keeps counters from scheduled jobs countable: for events emitted once a day, max_over_time(...) - min_over_time(...) over the window counts every increment, where increase() extrapolates from too few samples:

sum by (outcome) (
  max_over_time(job_items_total[1d]) - min_over_time(job_items_total[1d])
)

Relationship to OpenTelemetry

veryfront/metrics writes to the active OpenTelemetry metrics API. Export routing is owned by the runtime process:

Shared Veryfront runtimes use platform-owned OTel env vars and filter project-supplied telemetry routing keys.
Dedicated project runtimes may use deployment environment variables because they run in their own process boundary.
Local or customer-cloud deployments can use any Prometheus-compatible backend that the runtime config points at; Studio should treat the backend as an implementation detail.

Regular OpenTelemetry traces and metrics describe runtime behavior. Project metrics describe product, app, and eval behavior inside one project.

Relationship to eval exporters

Langfuse, LangSmith, Braintrust, and similar systems should use explicit eval report exporters from veryfront/extensions/eval. Those exporters receive the completed, redacted EvalReport only when an eval run selects them. Project metrics are aggregate signals for dashboards and alerts. They are not the report transport and should not include eval inputs, outputs, traces, or judge evidence.

MCP posture

Do not expose arbitrary raw metric writes over MCP. Agents that need to create metrics should call project code or a typed project tool that uses veryfront/metrics; that keeps project scoping, label policy, rate limits, and redaction in one framework path.

Verify it worked

Deploy or run the code path that emits the metric, then open the project Metrics panel in Studio and query the metric name, for example vf_eval_result_total. If no series appears, check that metrics export is enabled for the runtime environment and that the selected time range includes the emitted sample. In shared Veryfront runtimes, platform telemetry env vars control export. In dedicated runtimes, check the deployment environment variables for the project runtime process.

Getting Started

Concepts

Guides

API Reference

Emit metrics

Emit eval metrics

Label policy

Query sparse counters

Relationship to OpenTelemetry

Relationship to eval exporters

MCP posture

Verify it worked

​Emit metrics

​Emit eval metrics

​Label policy

​Query sparse counters

​Relationship to OpenTelemetry

​Relationship to eval exporters

​MCP posture

​Verify it worked

Emit metrics

Emit eval metrics

Label policy

Query sparse counters

Relationship to OpenTelemetry

Relationship to eval exporters

MCP posture

Verify it worked