Deploying AI inference endpoints

Last updated 30 Aug 2026

Overview

AI Inference lets you deploy an open-source model from the HuggingFace Hub as a dedicated inference API endpoint running on a GPU VM. CloudPe provisions the GPU VM from a pre-baked vLLM image, starts the model server for you, and attaches a floating IP so the endpoint is reachable over HTTP. The served API follows the OpenAI API format, so most OpenAI-compatible SDKs work by pointing the client at your endpoint URL.

The feature is in beta and appears as AI Inference under the PLATFORM group in the dashboard navigation.

Each endpoint is a single GPU VM: there is no autoscaling and no load balancing between replicas. If you want to manage the GPU machine yourself instead, see Deploying GPU virtual machines.

Before you start

Project — endpoints are scoped to a project, so pick or create the project you want the endpoint (and its GPU VM) to belong to.
Permissions — creating an endpoint requires vms:create, listing endpoints requires vms:read, and deleting an endpoint requires vms:delete.
GPU capacity — the endpoint runs on a GPU flavor, so the GPU type and flavor you choose must be available in a region where inference is configured.
Wallet and billing — the underlying GPU VM is billed like any other GPU instance, so your account must be in good standing to provision one.
HuggingFace token — required only if the model you want is gated on HuggingFace. The token is encrypted before storage and is used only to download the model onto the VM.
Beta status — behaviour and available options may change while the feature is in beta.

Steps

Open AI Inference from the navigation to reach the endpoint list at /dashboard/inference.
Click Deploy Model. The Deploy AI Model wizard opens on the Select Model tab.
Choose a model. Browse the Popular Models list, use All to clear the category filter, or search the HuggingFace Hub directly. Model cards show parameter count, download and like counts, and whether the model is gated.
If the model is gated, paste your HuggingFace access token in the token field on this step. Without it the download on the VM will fail.
Click Next to move to the Configure tab. Use Cancel at any point to abandon the wizard without creating anything.
On Configure, select the GPU type and then a GPU flavor. The region is derived from the flavor you pick. Select the project the endpoint belongs to and give the endpoint a name.
Submit the deployment. The endpoint is created immediately in a pending state and provisioning continues in the background: the GPU VM is created, a floating IP is allocated, then the model server starts.
Return to the endpoint list and watch the status badge. The lifecycle runs pending → provisioning → starting → running. Once it is running, copy the endpoint URL from the card and use it as the base URL in your client.

To tear an endpoint down, delete it from the list. Deletion destroys the associated GPU VM and releases its resources.

API

All calls use bearer authentication with an API key.

List endpoints — GET /api/v1/inference/endpoints

curl https://app.cloudpe.com/api/v1/inference/endpoints \
  -H "Authorization: Bearer <API_KEY>"

Discover models — GET /api/v1/inference/models/popular, GET /api/v1/inference/models/search, GET /api/v1/inference/models/{author}/{model_name}

curl https://app.cloudpe.com/api/v1/inference/models/popular \
  -H "Authorization: Bearer <API_KEY>"

Create an endpoint — POST /api/v1/inference/endpoints

curl -X POST https://app.cloudpe.com/api/v1/inference/endpoints \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-inference-endpoint",
    "model_id": "<huggingface_model_id>",
    "project_id": "<project_id>",
    "region_id": "<region_id>",
    "flavor_id": "<flavor_id>",
    "gpu_id": "<gpu_id>",
    "hf_token": null,
    "vllm_config": null
  }'

name, model_id, project_id, region_id, flavor_id and gpu_id are required; the identifier fields must be UUIDs. hf_token and vllm_config are optional and may be null.

Get one endpoint — GET /api/v1/inference/endpoints/{endpoint_id}

curl https://app.cloudpe.com/api/v1/inference/endpoints/<endpoint_id> \
  -H "Authorization: Bearer <API_KEY>"

Check endpoint health — GET /api/v1/inference/endpoints/{endpoint_id}/health

curl https://app.cloudpe.com/api/v1/inference/endpoints/<endpoint_id>/health \
  -H "Authorization: Bearer <API_KEY>"

Delete an endpoint — DELETE /api/v1/inference/endpoints/{endpoint_id}

curl -X DELETE https://app.cloudpe.com/api/v1/inference/endpoints/<endpoint_id> \
  -H "Authorization: Bearer <API_KEY>"

Limits & billing

Billing is for the underlying GPU VM: an endpoint costs the same as the GPU flavor it runs on, for as long as it exists. There is no separate per-request inference metering yet.
The GPU VM is billed from provisioning onward, including time spent downloading the model and starting the server — not only while the endpoint reports running.
One endpoint equals one GPU VM. There is no autoscaling, no replica pool, and no endpoint versioning or rollback.
The served endpoint URL is public and does not carry its own API-key authentication, so treat it as an internet-exposed service and restrict access at the firewall level if the model is sensitive.
Text-generation models are supported; image and audio models are not.
Deleting an endpoint is the only way to stop billing for it — there is no pause.

Troubleshooting

Error	Cause	Fix
`Model '{…}' not found`	The model ID does not exist on HuggingFace, or it is gated and no usable token was supplied.	Re-check the model ID from the search results and add a HuggingFace token with access to that model.
`Invalid project_id`	The project in the request does not exist or is not one you can deploy into.	Pick the project from the Configure step, or send a project ID your account has access to.
`Inference endpoint not found`	The endpoint ID is wrong, belongs to another user, or the endpoint was already deleted.	Re-list your endpoints and use the ID returned there.
`No inference config for this region`	Inference is not enabled in the region derived from the chosen flavor.	Choose a GPU flavor in a region where inference is available, or contact support.
`Region not found`	The region ID in the request does not match a known region.	Select the flavor in the wizard so the region is filled in automatically.
`Image not found`	The vLLM image is not registered for the selected flavor or region.	Choose a different GPU flavor, or contact support so the image can be mapped in that region.

If an endpoint reaches an error state after provisioning, the most common causes are a failed model download (gated model or expired token) or a model too large for the GPU memory of the chosen flavor. Delete the endpoint and redeploy with a valid token or a larger flavor.

FAQ

How do I call the endpoint once it is running? Copy the endpoint URL from the endpoint card and use it as the base URL in an OpenAI-compatible client. The model server speaks the OpenAI API format, so existing chat and completion code usually works with just a base URL change.

Why is my endpoint stuck in starting? The model weights are downloaded onto the VM the first time the server starts. Large models take longer. If the health probe never succeeds, the endpoint moves to an error state and you can inspect it with the endpoint health call.

Can I tune the serving configuration? Yes — the create call accepts a vllm_config object. If you leave it empty, defaults are used.

Can I use a private or gated model? Yes, supply a HuggingFace token that has access to it. The token is encrypted at rest and used only when the VM starts the model server.

Does anyone else see my endpoint? Endpoints are scoped to the user and project that created them in the dashboard and API. The served URL itself, however, is publicly reachable.

Can I run the model on a VM I control instead? Yes — provision a GPU instance and install your own stack. See Deploying GPU virtual machines.

Did this guide answer your question?If you need customized assistance with your deployment, reach out to our team.

Contact Support