---
title: "Alert rules and notification channels"
slug: "alert-rules-notifications"
source: "https://app.cloudpe.com/help/alert-rules-notifications"
updated: "2026-08-30T19:03:44.326Z"
---

# Alert rules and notification channels

## Overview

Monitoring turns the metrics your VM agents stream into notifications. You define **Alert Rules** (a metric, an operator, a threshold and a duration), attach one or more notification **Channels** (email, Slack or a generic webhook), and use **Silences** to suppress delivery during planned maintenance without losing alert state.

Alert rules are scoped to a single VM, a VM group, a project, or the whole organization. Each rule tracks state per VM through a small state machine — `ok` → `pending` (threshold breached, duration not yet met) → `alerting` (fire notification) → `resolved` (recovery notification) → `ok`. A condition that clears before the duration elapses never notifies.

The first time an agent registers for an organization, a set of default organization-level rules is seeded covering CPU, memory and disk at critical and high severities. Seeding happens once — if you delete the defaults, they are not re-created by the next agent.

Monitoring is in beta; behaviour and the console layout may change.

## Before you start

- Monitoring is a beta feature. The **Monitoring** entry sits under the PLATFORM group in the dashboard sidebar.
- At least one VM must have the monitoring agent installed and reporting. Rules evaluate against metrics in the regional time-series store, so a VM with no active agent produces no alerts. See [Install the VM monitoring agent](/help/vm-monitoring-agent-install).
- Monitoring APIs require the `vms:read` permission on your account.
- Decide where notifications should go before you create rules — a rule with no channel attached will change state but deliver nothing.
- Have the destination ready: recipient addresses for an email channel, an incoming webhook URL for Slack, or an HTTPS endpoint (and optional shared secret for HMAC signing) for a webhook channel.

## Steps

### 1. Create a notification channel

1. Go to **Monitoring** in the sidebar and open the **Channels** tab.
2. Click **Add Channel**.
3. Pick the channel type and fill in its configuration:
   - `email` — one or more recipient addresses.
   - `slack` — a Slack incoming webhook URL.
   - `webhook` — an HTTPS endpoint, plus an optional secret used to sign the payload with HMAC.
4. Save the channel, then send a test notification from the channel's row to confirm delivery before you rely on it.

![](/kb/monitoring/alert-rules-notifications-02-channels.png)

Channels are organization-scoped and shared across all rules in the organization. Deleting a channel is a soft delete.

### 2. Create an alert rule

1. Open the **Alert Rules** tab and click **Create Rule**.
2. Choose the scope: a single VM, a VM group, a project, or the whole organization. The rule is evaluated separately for every VM the scope resolves to.
3. Choose the metric. Alert rules support `cpu_usage_pct`, `cpu_iowait_pct`, `mem_usage_pct` and `disk_usage_pct`.
4. Choose an operator (`>`, `>=`, `<`, `<=`, `==`) and a threshold value.
5. Set the duration the condition must hold before the rule fires. Shorter durations react faster; longer durations filter out spikes.
6. Set the severity: `critical`, `warning` or `info`.
7. Attach the channels that should receive the fire and resolve notifications, then save.

![](/kb/monitoring/alert-rules-notifications-01-alert-rules.png)

Firing rules appear on the **Active Alerts** tab while they are in the `pending` or `alerting` state; the **Fleet Health** tab shows agent coverage across your VMs.

### 3. Silence alerts during maintenance

1. Open the **Silences** tab and click **Create Silence**.
2. Scope the silence to a VM, a project, the organization, or one specific rule.
3. Set the start and end of the window and save.

![](/kb/monitoring/alert-rules-notifications-03-silences.png)

While a silence covers a VM, its project, its organization or the rule itself, notifications are suppressed. Alert state is still evaluated and tracked underneath, so recovery is detected correctly and a still-breaching condition is visible on **Active Alerts**.

## API

Notification channels, alert rules, alerts and silences are available as REST endpoints. All paths are under `/api/v1/`.

| Method | Path |
|---|---|
| `GET` | `/notification-channels` |
| `POST` | `/notification-channels` |
| `PATCH` | `/notification-channels/{id}` |
| `DELETE` | `/notification-channels/{id}` |
| `POST` | `/notification-channels/{id}/test` |
| `GET` | `/alert-rules` |
| `POST` | `/alert-rules` |
| `GET` | `/alert-rules/{id}` |
| `PATCH` | `/alert-rules/{id}` |
| `DELETE` | `/alert-rules/{id}` |
| `GET` | `/alerts` |
| `GET` | `/alerts/history?period=24h` |
| `GET` | `/alert-silences` |
| `POST` | `/alert-silences` |
| `DELETE` | `/alert-silences/{id}` |

Create an email notification channel:

```bash
curl -X POST https://app.cloudpe.com/api/v1/notification-channels \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
        "name": "Ops on-call",
        "type": "email",
        "config": { "emails": ["ops@example.com"] }
      }'
```

Send a test notification to that channel:

```bash
curl -X POST https://app.cloudpe.com/api/v1/notification-channels/<id>/test \
  -H "Authorization: Bearer <API_KEY>"
```

Create an alert rule:

```bash
curl -X POST https://app.cloudpe.com/api/v1/alert-rules \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
        "metric": "cpu_usage_pct",
        "operator": ">",
        "threshold": 90,
        "duration_seconds": 300,
        "severity": "warning",
        "scope_type": "org"
      }'
```

List currently active alert instances:

```bash
curl https://app.cloudpe.com/api/v1/alerts \
  -H "Authorization: Bearer <API_KEY>"
```

Note the wire vocabulary if you are scripting against the API: operators are `>`, `>=`, `<`, `<=`, `==`; scopes are `vm`, `vm_group`, `project`, `org`; the active firing state is `alerting`; and the list filter for enabled rules is `enabled_only`.

## Limits & billing

- Metric allowlist: only `cpu_usage_pct`, `cpu_iowait_pct`, `mem_usage_pct` and `disk_usage_pct` can be used in a rule. Disk and network throughput/IOPS counters, load averages and swap usage are not usable as rule metrics — creating a rule with one of them is rejected with HTTP 422, and any pre-existing rules using them were disabled rather than deleted.
- Repeat suppression: while a rule instance stays in `alerting`, repeat notifications are only re-sent once the rule's repeat interval has elapsed since the last notification.
- Inhibition: if a VM's agent is `disconnected`, notifications for that VM's other rules are suppressed. State continues to be tracked so recovery still resolves correctly.
- Fail-closed evaluation: if the regional metrics store for a rule's scope cannot be queried, no state change is made for that rule in that cycle. An instance already in `alerting` is deliberately left alerting rather than cleared on an absence of data.
- Alerting, notification channels and silences are part of the monitoring feature and are not billed separately from the resources they watch.

## Troubleshooting

- A rule never fires: confirm the VM's agent is active on the **Fleet Health** tab. A `disconnected` agent both stops producing metrics and inhibits notifications for that VM.
- The rule shows as firing but nothing arrives: check that a channel is attached to the rule, and that no silence covers the VM, its project, the organization or that rule. Send a test notification from the **Channels** tab to isolate a delivery problem from an evaluation problem.
- A rule you created earlier is disabled and will not re-enable: it probably uses a metric outside the supported list. Recreate it with one of the supported metrics.
- Repeated notifications stopped for a still-firing alert: this is expected. Repeats resume only after the rule's repeat interval elapses; the alert remains visible on **Active Alerts**.

## FAQ

**Does a silence stop the alert from being evaluated?**
No. Evaluation and state tracking continue; only notification delivery is suppressed. Recovery is still detected while a silence is active.

**Can one rule notify several destinations?**
Yes. Attach as many channels as you need to a rule; channels are organization-scoped and can be reused across rules.

**Why did my alert clear itself without a resolve notification?**
A rule that breaches the threshold but recovers before the duration elapses moves from `pending` back to `ok`. That transition never notifies.

**Do I get the default rules automatically?**
Default organization-level CPU, memory and disk rules are seeded the first time an agent registers for your organization. If your organization already has any alert rules — including deleted ones — seeding is skipped.

**Can I alert on network or disk throughput?**
Not today. Those counters are not available as rule metrics.

## Related

- [Install the VM monitoring agent](/help/vm-monitoring-agent-install)