Create eval - Veryfront

veryfront__create_eval

Access

No external-service connection is required. Use a Veryfront credential with access to the requested resources. Call type: Can change resources.

Arguments

Field	Type	Required	Description and constraints
`project_reference`	`string`	No	Project identifier or slug that scopes the tool call. Minimum length: `1`.
`source_target_kind`	`string`	No	Provide the source target kind. Values: `"project"`, `"preview_branch"`.
`target_branch_id`	`string or null`	No	Provide the target branch id.
`name`	`string`	Yes	Human-readable name for the target record. Minimum length: `1`.
`description`	`string`	No	Human-readable description that clarifies the request.
`target`	`string`	Yes	Provide the target. Minimum length: `1`.
`dataset`	`object`	Yes	Structured dataset details used by the tool.
`repetitions`	`integer`	No	Provide the repetitions.
`tags`	`string[]`	No	Provide one or more tag values.
`metadata`	`object`	No	Additional structured metadata for the request.
`metrics`	`object[]`	No	Provide one or more metric values. Default: `[]`.

Example request

Send this JSON-RPC request through an initialized MCP client. Replace sample identifiers and values before calling the tool. See MCP client setup.

request.json

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "veryfront__create_eval",
    "arguments": {
      "name": "example",
      "target": "example",
      "dataset": {
        "kind": "inline",
        "editable": false,
        "dynamic": false
      }
    }
  }
}

Result

The published output shape is summarized below. Inspect both the MCP response content and isError when handling a call.

Field	Type	Required	Description and constraints
`id`	`string`	Yes	Unique identifier for the returned record.
`name`	`string`	Yes	Human-readable name for the returned record.
`description`	`string or null`	Yes	Human-readable description returned by the tool.
`target_kind`	`string`	Yes	The target kind returned by the tool. Value: `"agent"`.
`target`	`string`	Yes	The target returned by the tool.
`source_path`	`string`	Yes	The source path returned by the tool.
`export_name`	`string`	Yes	The export name returned by the tool.
`dataset`	`value`	No	The dataset returned by the tool.
`metrics`	`value[]`	Yes	List of metrics returned by the tool.
`repetitions`	`integer`	Yes	The repetitions returned by the tool.
`tags`	`string[]`	Yes	List of tags returned by the tool.
`metadata`	`object`	Yes	Additional structured metadata returned by the tool.
`editable_fields`	`string[]`	Yes	List of editable fields returned by the tool.
`dynamic_fields`	`string[]`	Yes	List of dynamic fields returned by the tool.
`capabilities`	`string[]`	Yes	List of capabilities returned by the tool.
`source`	`string`	Yes	The source returned by the tool.
`edit_mode`	`string`	Yes	The edit mode returned by the tool. Values: `"structured"`, `"source_only"`.
`source_only_reasons`	`string[]`	Yes	List of source only reasons returned by the tool.

Full schemas

{
  "type": "object",
  "properties": {
    "project_reference": {
      "type": "string",
      "minLength": 1,
      "description": "Project identifier or slug that scopes the tool call."
    },
    "source_target_kind": {
      "type": "string",
      "enum": [
        "project",
        "preview_branch"
      ],
      "description": "Provide the source target kind."
    },
    "target_branch_id": {
      "anyOf": [
        {
          "type": "string",
          "format": "uuid",
          "description": "Provide the target branch id."
        },
        {
          "type": "null",
          "description": "Provide the target branch id."
        }
      ],
      "description": "Provide the target branch id."
    },
    "name": {
      "type": "string",
      "minLength": 1,
      "description": "Human-readable name for the target record."
    },
    "description": {
      "type": "string",
      "description": "Human-readable description that clarifies the request."
    },
    "target": {
      "type": "string",
      "minLength": 1,
      "description": "Provide the target."
    },
    "dataset": {
      "type": "object",
      "properties": {
        "kind": {
          "type": "string",
          "enum": [
            "inline",
            "json",
            "jsonl",
            "dynamic"
          ],
          "description": "Provide the kind."
        },
        "path": {
          "type": "string",
          "description": "Provide the path."
        },
        "examples": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string",
                "minLength": 1,
                "description": "Unique identifier for the target record."
              },
              "input": {
                "description": "Provide the input."
              },
              "reference": {
                "description": "Provide the reference."
              },
              "metadata": {
                "type": "object",
                "additionalProperties": {
                  "description": "Additional structured values supplied to the tool."
                },
                "description": "Additional structured metadata for the request."
              }
            },
            "required": [
              "id"
            ],
            "additionalProperties": false,
            "description": "One example item supplied to the tool."
          },
          "description": "Provide one or more example values."
        },
        "editable": {
          "type": "boolean",
          "description": "Provide the editable."
        },
        "dynamic": {
          "type": "boolean",
          "description": "Provide the dynamic."
        }
      },
      "required": [
        "kind",
        "editable",
        "dynamic"
      ],
      "additionalProperties": false,
      "description": "Structured dataset details used by the tool."
    },
    "repetitions": {
      "type": "integer",
      "exclusiveMinimum": 0,
      "description": "Provide the repetitions."
    },
    "tags": {
      "type": "array",
      "items": {
        "type": "string",
        "description": "One tag item supplied to the tool."
      },
      "description": "Provide one or more tag values."
    },
    "metadata": {
      "type": "object",
      "additionalProperties": {
        "description": "Additional structured values supplied to the tool."
      },
      "description": "Additional structured metadata for the request."
    },
    "metrics": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string",
            "minLength": 1,
            "description": "Human-readable name for the target record."
          },
          "family": {
            "type": "string",
            "enum": [
              "answer",
              "agent",
              "ops",
              "judge",
              "knowledge",
              "check"
            ],
            "description": "Provide the family."
          },
          "severity": {
            "type": "string",
            "enum": [
              "gate",
              "soft",
              "budget"
            ],
            "description": "Provide the severity."
          },
          "threshold": {
            "type": "object",
            "properties": {
              "min": {
                "type": "number",
                "description": "Provide the min."
              },
              "max": {
                "type": "number",
                "description": "Provide the max."
              }
            },
            "additionalProperties": false,
            "description": "Structured threshold details used by the tool."
          },
          "config": {
            "type": "object",
            "additionalProperties": {
              "description": "Additional structured values supplied to the tool."
            },
            "description": "Structured config details used by the tool."
          },
          "editable": {
            "type": "boolean",
            "description": "Provide the editable."
          },
          "dynamic": {
            "type": "boolean",
            "description": "Provide the dynamic."
          }
        },
        "required": [
          "name",
          "family",
          "severity",
          "editable",
          "dynamic"
        ],
        "additionalProperties": false,
        "description": "One metric item supplied to the tool."
      },
      "default": [],
      "description": "Provide one or more metric values."
    }
  },
  "required": [
    "name",
    "target",
    "dataset"
  ],
  "additionalProperties": false,
  "$schema": "http://json-schema.org/draft-07/schema#",
  "description": "Arguments for the create_eval tool."
}

{
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "description": "Unique identifier for the returned record."
    },
    "name": {
      "type": "string",
      "description": "Human-readable name for the returned record."
    },
    "description": {
      "type": [
        "string",
        "null"
      ],
      "description": "Human-readable description returned by the tool."
    },
    "target_kind": {
      "type": "string",
      "const": "agent",
      "description": "The target kind returned by the tool."
    },
    "target": {
      "type": "string",
      "description": "The target returned by the tool."
    },
    "source_path": {
      "type": "string",
      "description": "The source path returned by the tool."
    },
    "export_name": {
      "type": "string",
      "description": "The export name returned by the tool."
    },
    "dataset": {
      "description": "The dataset returned by the tool."
    },
    "metrics": {
      "type": "array",
      "items": {
        "description": "One metric item returned by the tool."
      },
      "description": "List of metrics returned by the tool."
    },
    "repetitions": {
      "type": "integer",
      "exclusiveMinimum": 0,
      "description": "The repetitions returned by the tool."
    },
    "tags": {
      "type": "array",
      "items": {
        "type": "string",
        "description": "One tag item returned by the tool."
      },
      "description": "List of tags returned by the tool."
    },
    "metadata": {
      "type": "object",
      "additionalProperties": {
        "description": "Additional structured values returned by the tool."
      },
      "description": "Additional structured metadata returned by the tool."
    },
    "editable_fields": {
      "type": "array",
      "items": {
        "type": "string",
        "description": "One editable field item returned by the tool."
      },
      "description": "List of editable fields returned by the tool."
    },
    "dynamic_fields": {
      "type": "array",
      "items": {
        "type": "string",
        "description": "One dynamic field item returned by the tool."
      },
      "description": "List of dynamic fields returned by the tool."
    },
    "capabilities": {
      "type": "array",
      "items": {
        "type": "string",
        "description": "One capability item returned by the tool."
      },
      "description": "List of capabilities returned by the tool."
    },
    "source": {
      "type": "string",
      "description": "The source returned by the tool."
    },
    "edit_mode": {
      "type": "string",
      "enum": [
        "structured",
        "source_only"
      ],
      "description": "The edit mode returned by the tool."
    },
    "source_only_reasons": {
      "type": "array",
      "items": {
        "type": "string",
        "description": "One source only reason item returned by the tool."
      },
      "description": "List of source only reasons returned by the tool."
    }
  },
  "required": [
    "id",
    "name",
    "description",
    "target_kind",
    "target",
    "source_path",
    "export_name",
    "metrics",
    "repetitions",
    "tags",
    "metadata",
    "editable_fields",
    "dynamic_fields",
    "capabilities",
    "source",
    "edit_mode",
    "source_only_reasons"
  ],
  "additionalProperties": false,
  "$schema": "http://json-schema.org/draft-07/schema#",
  "description": "Structured result returned by the create_eval tool."
}