Run evaluation

veryfront__run_eval

Requires project.evals.run.

Access

No external-service connection is required. Use a Veryfront credential with access to the requested resources. Call type: Can change resources.

Arguments

Field	Type	Required	Description and constraints
`project_reference`	`string`	No	Project identifier or slug that scopes the tool call. Minimum length: `1`.
`eval_id`	`string`	Yes	Eval to run. Minimum length: `1`.
`source_target_kind`	`string`	No	Source the eval definition is read from: project or `preview_branch`. Defaults to project. Values: `"project"`, `"preview_branch"`.
`target_branch_id`	`string or null`	No	Preview branch the eval definition is read from; required when `source_target_kind` is `preview_branch`.
`runtime_target_kind`	`string`	No	Runtime the eval runs on: `main_branch`, environment or `preview_branch`. Defaults to `main_branch`. Values: `"main_branch"`, `"environment"`, `"preview_branch"`.
`target_environment_id`	`string or null`	No	Environment the eval runs on; required when `runtime_target_kind` is environment.
`runtime_target_branch_id`	`string or null`	No	Preview branch the eval runs on, independent of the source branch; for a `preview_branch` runtime.
`input`	`value`	No	Eval run input: any JSON value (object, array, string, number, boolean or null).
`config`	`object`	No	Run configuration object. Its `eval_id` is always set to the eval being run.
`idempotency_key`	`string`	No	Retry key for this eval run, 1-128 characters. Overrides the HTTP Idempotency-Key header. Unkeyed root calls remain supported; supply the same key to retry safely. Minimum length: `1`. Maximum length: `128`.

Example request

Send this JSON-RPC request through an initialized MCP client. Replace sample identifiers and values before calling the tool. See MCP client setup.

request.json

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "veryfront__run_eval",
    "arguments": {
      "eval_id": "example"
    }
  }
}

Result

The published output shape is summarized below. Inspect both the MCP response content and isError when handling a call.

Field	Type	Required	Description and constraints
`kind`	`string`	Yes	Always eval-run. Value: `"eval-run"`.
`run_id`	`string`	Yes	Run identifier.
`eval_id`	`string`	Yes	Eval the run executes.
`status`	`string`	Yes	Run status: pending, running, waiting, completed, failed or cancelled. Values: `"pending"`, `"running"`, `"waiting"`, `"completed"`, `"failed"`, `"cancelled"`.
`target_kind`	`string`	Yes	What the eval runs against: always agent. Value: `"agent"`.
`target`	`string`	Yes	Identifier of the agent the eval runs against.
`source`	`object`	No	Source of the eval definition the run executes, when known.
`summary`	`value or null`	No	Result summary of a finished run: records, pass rate and metric results; null until it finishes.
`report_path`	`string or null`	Yes	Path of the run report, or null until the report is written.
`error`	`value or null`	No	Error of a failed run, or null.
`metadata`	`object`	Yes	Free-form JSON object stored with the run.
`created_at`	`string`	Yes	When the run was created, in ISO 8601 format.
`started_at`	`string or null`	Yes	When the run started, in ISO 8601 format, or null.
`completed_at`	`string or null`	Yes	When the run finished, in ISO 8601 format, or null.

Full schemas

{
  "type": "object",
  "properties": {
    "project_reference": {
      "type": "string",
      "minLength": 1,
      "description": "Project identifier or slug that scopes the tool call."
    },
    "eval_id": {
      "type": "string",
      "minLength": 1,
      "description": "Eval to run."
    },
    "source_target_kind": {
      "type": "string",
      "enum": [
        "project",
        "preview_branch"
      ],
      "description": "Source the eval definition is read from: project or preview_branch. Defaults to project."
    },
    "target_branch_id": {
      "anyOf": [
        {
          "type": "string",
          "format": "uuid",
          "description": "Provide the target branch id."
        },
        {
          "type": "null",
          "description": "Provide the target branch id."
        }
      ],
      "description": "Preview branch the eval definition is read from; required when source_target_kind is preview_branch."
    },
    "runtime_target_kind": {
      "type": "string",
      "enum": [
        "main_branch",
        "environment",
        "preview_branch"
      ],
      "description": "Runtime the eval runs on: main_branch, environment or preview_branch. Defaults to main_branch."
    },
    "target_environment_id": {
      "anyOf": [
        {
          "type": "string",
          "format": "uuid",
          "description": "Provide the target environment id."
        },
        {
          "type": "null",
          "description": "Provide the target environment id."
        }
      ],
      "description": "Environment the eval runs on; required when runtime_target_kind is environment."
    },
    "runtime_target_branch_id": {
      "anyOf": [
        {
          "type": "string",
          "format": "uuid",
          "description": "Provide the runtime target branch id."
        },
        {
          "type": "null",
          "description": "Provide the runtime target branch id."
        }
      ],
      "description": "Preview branch the eval runs on, independent of the source branch; for a preview_branch runtime."
    },
    "input": {
      "description": "Eval run input: any JSON value (object, array, string, number, boolean or null)."
    },
    "config": {
      "type": "object",
      "additionalProperties": {
        "description": "Additional structured values supplied to the tool."
      },
      "description": "Run configuration object. Its eval_id is always set to the eval being run."
    },
    "idempotency_key": {
      "type": "string",
      "minLength": 1,
      "maxLength": 128,
      "description": "Retry key for this eval run, 1-128 characters. Overrides the HTTP Idempotency-Key header. Unkeyed root calls remain supported; supply the same key to retry safely."
    }
  },
  "required": [
    "eval_id"
  ],
  "additionalProperties": false,
  "$schema": "http://json-schema.org/draft-07/schema#",
  "description": "Arguments for the run_eval tool."
}

{
  "type": "object",
  "properties": {
    "kind": {
      "type": "string",
      "const": "eval-run",
      "description": "Always eval-run."
    },
    "run_id": {
      "type": "string",
      "description": "Run identifier."
    },
    "eval_id": {
      "type": "string",
      "description": "Eval the run executes."
    },
    "status": {
      "type": "string",
      "enum": [
        "pending",
        "running",
        "waiting",
        "completed",
        "failed",
        "cancelled"
      ],
      "description": "Run status: pending, running, waiting, completed, failed or cancelled."
    },
    "target_kind": {
      "type": "string",
      "const": "agent",
      "description": "What the eval runs against: always agent."
    },
    "target": {
      "type": "string",
      "description": "Identifier of the agent the eval runs against."
    },
    "source": {
      "type": "object",
      "properties": {
        "file_path": {
          "type": "string",
          "description": "Source file path of the eval definition."
        },
        "export_name": {
          "type": "string",
          "description": "Name of the export that declares the eval in its source file."
        }
      },
      "required": [
        "file_path",
        "export_name"
      ],
      "additionalProperties": false,
      "description": "Source of the eval definition the run executes, when known."
    },
    "summary": {
      "anyOf": [
        {
          "description": "The summary returned by the tool."
        },
        {
          "type": "null",
          "description": "The summary returned by the tool."
        }
      ],
      "description": "Result summary of a finished run: records, pass rate and metric results; null until it finishes."
    },
    "report_path": {
      "type": [
        "string",
        "null"
      ],
      "description": "Path of the run report, or null until the report is written."
    },
    "error": {
      "anyOf": [
        {
          "description": "Error details returned by the tool."
        },
        {
          "type": "null",
          "description": "Error details returned by the tool."
        }
      ],
      "description": "Error of a failed run, or null."
    },
    "metadata": {
      "type": "object",
      "additionalProperties": {
        "description": "Additional structured values returned by the tool."
      },
      "description": "Free-form JSON object stored with the run."
    },
    "created_at": {
      "type": "string",
      "description": "When the run was created, in ISO 8601 format."
    },
    "started_at": {
      "type": [
        "string",
        "null"
      ],
      "description": "When the run started, in ISO 8601 format, or null."
    },
    "completed_at": {
      "type": [
        "string",
        "null"
      ],
      "description": "When the run finished, in ISO 8601 format, or null."
    }
  },
  "required": [
    "kind",
    "run_id",
    "eval_id",
    "status",
    "target_kind",
    "target",
    "report_path",
    "metadata",
    "created_at",
    "started_at",
    "completed_at"
  ],
  "additionalProperties": false,
  "description": "Eval run.",
  "$schema": "http://json-schema.org/draft-07/schema#"
}

Get started

APIs

REST API

GraphQL API

MCP API

Access

Arguments

Example request

Result

Full schemas

​Access

​Arguments

​Example request

​Result

​Full schemas

Access

Arguments

Example request

Result

Full schemas