Create an Auto Monitor Setup

Trigger a process that attempts to create a Traceloop monitor using the provided selector criteria

POST /v2/auto-monitor-setups

Create an Auto Monitor Setup

cURL

curl --request POST \
  --url https://app.traceloop.com/api/v2/auto-monitor-setups \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "external_id": "<string>",
  "evaluators": [
    "<string>"
  ],
  "evaluator_configs": [
    {}
  ],
  "selector": [
    {}
  ]
}
'

An auto monitor setup is an asynchronous process that attempts to create a Traceloop monitor based on the selector you provide. The selector is matched against your incoming spans — when a match is found, the monitor is created and the specified evaluators begin running on those spans.

All API requests require authentication. Pass your API key as a Bearer token in the Authorization header. See Authentication for details.

Request Body

external_id · string · required

Unique identifier for the auto monitor setup. Used to reference it in future requests (get, update, delete).

Example: "my-agent-monitor-1"

evaluators · string[]

List of evaluator slugs to run on matched spans.

Example: ["answer-relevancy", "toxicity-detector"]

See the full list of available slugs in the Evaluator Slugs reference.

Either evaluators or evaluator_configs must be provided on create. If both are provided, evaluator_configs wins and this field is ignored. Use evaluator_configs when you need to pin a per-evaluator scope.

evaluator_configs · object[]

List of per-evaluator configurations. Use this instead of evaluators when you need to control the granularity at which an individual evaluator runs.

Each entry has the following fields:

Field	Type	Required	Description
slug	string	Yes	Evaluator slug — same set accepted by the evaluators array. See the Evaluator Slugs reference.
scope	string	No	Granularity at which the evaluator runs. One of session, trace, span. When omitted, no scope is persisted and the downstream monitor applies its own default.

Example:

[
  { "slug": "char-count", "scope": "session" },
  { "slug": "toxicity",   "scope": "trace"   },
  { "slug": "pii" }
]

selector · object[]

An array of filter rules used to match spans. Each rule specifies an attribute key, a value to match, and a source indicating where the attribute lives. Only spans matching all provided rules will be evaluated.

Each rule has the following fields:

Field	Type	Required	Description
key	string	Yes	The attribute key to filter on (e.g., gen_ai.system, service.name)
value	string	Conditional	The value to match against. Required for equals, not_equals, contains, not_contains operators
values	string[]	Conditional	List of values to match against. Required for in, not_in operators
source	string	Yes	Where the attribute lives: span_attributes or resource_attributes
operator	string	No	Comparison operator. Defaults to equals. One of: equals, not_equals, contains, not_contains, exists, not_exists, in, not_in

All available span and resource attributes can be found in your Traceloop traces page.

Example Request

Using the simple evaluators array (no per-evaluator scope):

curl -X POST https://api.traceloop.com/v2/auto-monitor-setups \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "external_id": "my-agent-monitor-1",
    "evaluators": ["answer-relevancy", "toxicity-detector"],
    "selector": [
      {"key": "gen_ai.system", "value": "openai", "source": "span_attributes"},
      {"key": "gen_ai.request.model", "value": "gpt-4o", "source": "span_attributes"}
    ]
  }'

Using evaluator_configs to pin a scope per evaluator:

curl -X POST https://api.traceloop.com/v2/auto-monitor-setups \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "external_id": "my-agent-monitor-1",
    "selector": [
      {"key": "gen_ai.system", "value": "openai", "source": "span_attributes"}
    ],
    "evaluator_configs": [
      {"slug": "char-count", "scope": "session"},
      {"slug": "toxicity",   "scope": "trace"},
      {"slug": "pii"}
    ]
  }'

Response

201 Created

Returns the created auto monitor setup object. The init_rules array reflects the stored selector — same shape as the input selector.

Each entry in evaluators optionally includes a scope field (session, trace, or span) when a scope was set for that evaluator via evaluator_configs. When no scope was provided at create time, scope is omitted from the evaluator object (it is never returned as null).

The status field indicates the current state of the setup process:

Status	Description
pending	The setup has been created and is queued for processing
in_progress	The setup is actively being processed
waiting_for_spans	The setup is ready and waiting for enough spans to be received before attempting to process
completed	All monitors have been successfully created
partial	Some monitors were created but others failed
failed	The setup process failed

{
  "id": "cmm...",
  "external_id": "my-agent-monitor-1",
  "org_id": "c108269c-...",
  "project_id": "cm9v2g95l...",
  "env_project_id": "cm9v2ga9i...",
  "init_rules": [
    {
      "key": "gen_ai.system",
      "value": "openai",
      "source": "span_attributes",
      "operator": "equals"
    },
    {
      "key": "gen_ai.request.model",
      "value": "gpt-4o",
      "source": "span_attributes",
      "operator": "equals"
    }
  ],
  "evaluators": [
    {
      "evaluator_type": "answer-relevancy",
      "scope": "trace",
      "input_schema": [
        { "type": "string", "name": "completion", "description": "The LLM response to evaluate" },
        { "type": "string", "name": "context", "description": "The context for the answer" },
        { "type": "string", "name": "question", "description": "The original question" }
      ],
      "output_schema": [
        { "type": "float", "name": "answer_relevancy_score", "description": "Relevancy score (0-1)" }
      ],
      "status": "pending"
    },
    {
      "evaluator_type": "toxicity-detector",
      "input_schema": [
        { "type": "string", "name": "text", "description": "The text to analyze for toxicity" }
      ],
      "output_schema": [
        { "type": "boolean", "name": "is_toxic", "description": "Whether the text is toxic" }
      ],
      "status": "pending"
    }
  ],
  "status": "pending",
  "created_at": "2026-01-15T10:30:00Z",
  "updated_at": "2026-01-15T10:30:00Z"
}

400 Bad Request

Returned when the request body is invalid. Common validation errors:

• Neither evaluators nor evaluator_configs is set on a Create request — one of evaluators or evaluator_configs is required.
• An entry in evaluator_configs is missing its slug.
• An entry in evaluator_configs has a scope that is not one of session, trace, or span.
• A provided evaluator slug is not recognized — unknown evaluator slug "<slug>".

{
  "error": "one of evaluators or evaluator_configs is required"
}

500 Internal Server Error

{
  "error": "internal server error"
}