veryfront/eval

Import

import {
  createEvalReport,
  createEvalSourceDocument,
  deriveEvalId,
  discoverEvals,
  evalAgent,
  findEvalById,
} from "veryfront/eval";

Examples

import { datasets, evalAgent, metrics } from "veryfront/eval";

export default evalAgent({
  target: "agent:researcher",
  dataset: datasets.inline([
    { id: "q1", input: "Capital of France?", reference: "Paris" },
  ]),
  metrics: [
    metrics.answer.contains({ text: "Paris" }).gate(),
    metrics.agent.noFailedTools().gate(),
  ],
});

Exports

Functions

Name	Description	Source
`createEvalReport`	Create a JSON-serializable eval report from executed records.	source
`createEvalSourceDocument`	Create the normalized Eval document Studio can list, inspect, and edit.	source
`deriveEvalId`	Derive the stable `eval:<path>` ID for an eval file.	source
`discoverEvals`	Discover eval definitions from a project eval directory.	source
`evalAgent`	Define a V1 eval that targets a Veryfront agent.	source
`findEvalById`	Discover and return one eval definition by ID.	source
`isEvalDefinition`	Check whether a value is a normalized eval definition.	source
`runEval`	Execute an eval locally with injected target adapters.	source
`summarizeEvalRecords`	Summarize eval records into pass/fail and metric aggregates.	source

Types

Name	Description	Source
`CreateEvalSourceDocumentOptions`	Options for creating a Studio source document from a discovered eval.	source
`DiscoveredEval`	Eval definition discovered from project source.	source
`EvalAgentAdapter`	Adapter used by `runEval` to execute V1 agent targets.	source
`EvalAgentAdapterContext`	Context passed to an agent adapter when `runEval` executes an example.	source
`EvalAgentAdapterResult`	Agent adapter result normalized into an eval record.	source
`EvalAgentInput`	Input accepted by `evalAgent`.	source
`EvalCheckContext`	Context passed to an eval definition’s `check` callback.	source
`EvalDataset`	Dataset loader used by an eval definition.	source
`EvalDatasetLoadContext`	Context passed to dataset loaders.	source
`EvalDefinition`	First-class eval definition discovered from project source.	source
`EvalDiscoveryOptions`	Options for project-local eval discovery.	source
`EvalDiscoveryResult`	Result returned by eval discovery.	source
`EvalEditableField`	Form-editable Eval source field name.	source
`EvalExample`	Normalized dataset example used by eval runners and reports.	source
`EvalExampleInput`	Dataset example shape accepted by eval definitions.	source
`EvalExpect`	Built-in expectation helpers available inside `check`.	source
`EvalExpectation`	Fluent severity helpers for `check` expectations.	source
`EvalMetric`	Metric contract used by eval definitions.	source
`EvalMetricContext`	Optional runtime context passed to metric evaluators.	source
`EvalMetricFamily`	Metric family used for grouping report summaries.	source
`EvalMetricResult`	Result emitted by a metric or check assertion.	source
`EvalMetricSummary`	Aggregate pass/fail summary for one metric.	source
`EvalMetricThreshold`	Numeric threshold attached to score-based metrics.	source
`EvalRecord`	One executed example and repetition inside an eval report.	source
`EvalReport`	JSON-serializable report produced by `runEval`.	source
`EvalReportSummary`	Aggregate pass/fail summary for one eval report.	source
`EvalRun`	V2-ready Eval run projection.	source
`EvalSeverity`	How a metric result affects the final eval result.	source
`EvalSource`	Source location for a discovered eval definition.	source
`EvalSourceDocument`	Studio-editable Eval source document.	source
`EvalSourcePatch`	Eval source patch submitted by Studio forms.	source
`EvalSourceReference`	Source location for an Eval definition.	source
`EvalStudioCapability`	Capability string Studio uses for Eval read and write access.	source
`EvalTargetKind`	Primitive kind an eval can execute. V1 supports agent targets.	source
`EvalToolCall`	Tool call metadata captured during one eval record.	source
`EvalTrace`	Trace metadata captured for one eval record.	source
`EvalUsage`	Token and cost usage captured for one eval record.	source
`RunEvalOptions`	Options for running an eval locally.	source

Constants

Name	Description	Source
`datasets`	Dataset factories for inline, JSON, and JSONL eval examples.	source
`getEvalEditableFieldSchema`	Schema for an editable Eval source field name.	source
`getEvalRunSchema`	Schema for V2-ready Eval run projections.	source
`getEvalSourceDocumentSchema`	Schema for a Studio-editable Eval source document.	source
`getEvalSourcePatchSchema`	Schema for a source patch submitted from an Eval editor.	source
`getEvalSourceReferenceSchema`	Schema for an Eval source reference.	source
`getEvalStudioCapabilitySchema`	Schema for Eval Studio capabilities.	source
`metrics`	Metric factories for deterministic answers, agent behavior, operations, and judges.	source

Getting Started

Concepts

Guides

API Reference

Import

Examples

Exports

Functions

Types

Constants

​Import

​Examples

​Exports

​Functions

​Types

​Constants

Import

Examples

Exports

Functions

Types

Constants