---
title: "Deploying AI inference endpoints"
slug: "ai-inference-endpoints"
source: "https://app.cloudpe.com/help/ai-inference-endpoints"
updated: "2026-08-30T19:03:44.101Z"
---

# Deploying AI inference endpoints

## Overview

AI Inference lets you deploy an open-source model from the HuggingFace Hub as a dedicated inference API endpoint running on a GPU VM. CloudPe provisions the GPU VM from a pre-baked vLLM image, starts the model server for you, and attaches a floating IP so the endpoint is reachable over HTTP. The served API follows the OpenAI API format, so most OpenAI-compatible SDKs work by pointing the client at your endpoint URL.

The feature is in beta and appears as **AI Inference** under the **PLATFORM** group in the dashboard navigation.

Each endpoint is a single GPU VM: there is no autoscaling and no load balancing between replicas. If you want to manage the GPU machine yourself instead, see [Deploying GPU virtual machines](/help/gpu-vm-deployment).

## Before you start

- Project — endpoints are scoped to a project, so pick or create the project you want the endpoint (and its GPU VM) to belong to.
- Permissions — creating an endpoint requires `vms:create`, listing endpoints requires `vms:read`, and deleting an endpoint requires `vms:delete`.
- GPU capacity — the endpoint runs on a GPU flavor, so the GPU type and flavor you choose must be available in a region where inference is configured.
- Wallet and billing — the underlying GPU VM is billed like any other GPU instance, so your account must be in good standing to provision one.
- HuggingFace token — required only if the model you want is gated on HuggingFace. The token is encrypted before storage and is used only to download the model onto the VM.
- Beta status — behaviour and available options may change while the feature is in beta.

## Steps

1. Open **AI Inference** from the navigation to reach the endpoint list at `/dashboard/inference`.

   ![](/kb/compute/ai-inference-endpoints-01-list.png)

2. Click **Deploy Model**. The **Deploy AI Model** wizard opens on the **Select Model** tab.

   ![](/kb/compute/ai-inference-endpoints-02-deploy.png)

3. Choose a model. Browse the **Popular Models** list, use **All** to clear the category filter, or search the HuggingFace Hub directly. Model cards show parameter count, download and like counts, and whether the model is gated.

4. If the model is gated, paste your HuggingFace access token in the token field on this step. Without it the download on the VM will fail.

5. Click **Next** to move to the **Configure** tab. Use **Cancel** at any point to abandon the wizard without creating anything.

6. On **Configure**, select the GPU type and then a GPU flavor. The region is derived from the flavor you pick. Select the project the endpoint belongs to and give the endpoint a name.

7. Submit the deployment. The endpoint is created immediately in a pending state and provisioning continues in the background: the GPU VM is created, a floating IP is allocated, then the model server starts.

8. Return to the endpoint list and watch the status badge. The lifecycle runs pending → provisioning → starting → running. Once it is running, copy the endpoint URL from the card and use it as the base URL in your client.

To tear an endpoint down, delete it from the list. Deletion destroys the associated GPU VM and releases its resources.

## API

All calls use bearer authentication with an API key.

List endpoints — `GET /api/v1/inference/endpoints`

```bash
curl https://app.cloudpe.com/api/v1/inference/endpoints \
  -H "Authorization: Bearer <API_KEY>"
```

Discover models — `GET /api/v1/inference/models/popular`, `GET /api/v1/inference/models/search`, `GET /api/v1/inference/models/{author}/{model_name}`

```bash
curl https://app.cloudpe.com/api/v1/inference/models/popular \
  -H "Authorization: Bearer <API_KEY>"
```

Create an endpoint — `POST /api/v1/inference/endpoints`

```bash
curl -X POST https://app.cloudpe.com/api/v1/inference/endpoints \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-inference-endpoint",
    "model_id": "<huggingface_model_id>",
    "project_id": "<project_id>",
    "region_id": "<region_id>",
    "flavor_id": "<flavor_id>",
    "gpu_id": "<gpu_id>",
    "hf_token": null,
    "vllm_config": null
  }'
```

`name`, `model_id`, `project_id`, `region_id`, `flavor_id` and `gpu_id` are required; the identifier fields must be UUIDs. `hf_token` and `vllm_config` are optional and may be null.

Get one endpoint — `GET /api/v1/inference/endpoints/{endpoint_id}`

```bash
curl https://app.cloudpe.com/api/v1/inference/endpoints/<endpoint_id> \
  -H "Authorization: Bearer <API_KEY>"
```

Check endpoint health — `GET /api/v1/inference/endpoints/{endpoint_id}/health`

```bash
curl https://app.cloudpe.com/api/v1/inference/endpoints/<endpoint_id>/health \
  -H "Authorization: Bearer <API_KEY>"
```

Delete an endpoint — `DELETE /api/v1/inference/endpoints/{endpoint_id}`

```bash
curl -X DELETE https://app.cloudpe.com/api/v1/inference/endpoints/<endpoint_id> \
  -H "Authorization: Bearer <API_KEY>"
```

## Limits & billing

- Billing is for the underlying GPU VM: an endpoint costs the same as the GPU flavor it runs on, for as long as it exists. There is no separate per-request inference metering yet.
- The GPU VM is billed from provisioning onward, including time spent downloading the model and starting the server — not only while the endpoint reports running.
- One endpoint equals one GPU VM. There is no autoscaling, no replica pool, and no endpoint versioning or rollback.
- The served endpoint URL is public and does not carry its own API-key authentication, so treat it as an internet-exposed service and restrict access at the firewall level if the model is sensitive.
- Text-generation models are supported; image and audio models are not.
- Deleting an endpoint is the only way to stop billing for it — there is no pause.

## Troubleshooting

| Error | Cause | Fix |
|---|---|---|
| `Model '{…}' not found` | The model ID does not exist on HuggingFace, or it is gated and no usable token was supplied. | Re-check the model ID from the search results and add a HuggingFace token with access to that model. |
| `Invalid project_id` | The project in the request does not exist or is not one you can deploy into. | Pick the project from the **Configure** step, or send a project ID your account has access to. |
| `Inference endpoint not found` | The endpoint ID is wrong, belongs to another user, or the endpoint was already deleted. | Re-list your endpoints and use the ID returned there. |
| `No inference config for this region` | Inference is not enabled in the region derived from the chosen flavor. | Choose a GPU flavor in a region where inference is available, or contact support. |
| `Region not found` | The region ID in the request does not match a known region. | Select the flavor in the wizard so the region is filled in automatically. |
| `Image not found` | The vLLM image is not registered for the selected flavor or region. | Choose a different GPU flavor, or contact support so the image can be mapped in that region. |

If an endpoint reaches an error state after provisioning, the most common causes are a failed model download (gated model or expired token) or a model too large for the GPU memory of the chosen flavor. Delete the endpoint and redeploy with a valid token or a larger flavor.

## FAQ

**How do I call the endpoint once it is running?**
Copy the endpoint URL from the endpoint card and use it as the base URL in an OpenAI-compatible client. The model server speaks the OpenAI API format, so existing chat and completion code usually works with just a base URL change.

**Why is my endpoint stuck in starting?**
The model weights are downloaded onto the VM the first time the server starts. Large models take longer. If the health probe never succeeds, the endpoint moves to an error state and you can inspect it with the endpoint health call.

**Can I tune the serving configuration?**
Yes — the create call accepts a `vllm_config` object. If you leave it empty, defaults are used.

**Can I use a private or gated model?**
Yes, supply a HuggingFace token that has access to it. The token is encrypted at rest and used only when the VM starts the model server.

**Does anyone else see my endpoint?**
Endpoints are scoped to the user and project that created them in the dashboard and API. The served URL itself, however, is publicly reachable.

**Can I run the model on a VM I control instead?**
Yes — provision a GPU instance and install your own stack. See [Deploying GPU virtual machines](/help/gpu-vm-deployment).

## Related

- [Deploying GPU virtual machines](/help/gpu-vm-deployment)
- [Creating and managing virtual machines](/help/create-manage-virtual-machines)