Cloud

Models without machines.

Podstack Inference is one OpenAI-compatible API for open and frontier models that scales to zero and bills per token.

Podstack Inference is one OpenAI-compatible API for open and frontier models. Endpoints scale to zero when idle and bill per token, so you pay for answers, not for GPUs waiting for questions.

Open Inference CloudRead the API docs
At a glance

Inference at a glance

API
OpenAI-compatible
Billing unit
per token
Idle cost
zero
Cold starts
warm pool
What it does

What Inference does

One API, many models

Open-weight and frontier models sit behind the same OpenAI-compatible endpoint, so switching a model is a string change in your client.

Scale to zero

An endpoint with no traffic costs nothing. Cached model weights and a small warm pool per active model keep the first request fast when traffic returns.

Per-token billing

Usage is metered in tokens against your Podstack wallet, with quotas you can set per key.

Keys, quotas and usage

Create API keys, cap spend per key and watch token usage in the console.

Runs on Token Factory

The same serving stack operators license as Token Factory powers Podstack Inference, so behaviour is identical on Podstack Cloud and in a partner data center.

Live on Podstack Cloud

Live on Podstack Cloud

5 GPU configurations ready for model serving · autoscaling endpoints

GPUVRAMAvailabilityAction
L424 GBLiveDeploy
RTX409024 GBLiveDeploy
L40S48 GBLive · 2 GPUsDeploy
A10080 GBLiveDeploy
H10080 GBLiveDeploy
From the terminal

From the terminal

One CLI, one SDK, one API

terminalsh
$ podstack gpu launch
? pick GPU › H100 80GB · 50% slice
✓ ssh ready · billed per GPU-minute
$ podstack train create --budget 25
→ LoRA job started · adapter saved when it finishes
$ podstack models list
llama-3-70b · mistral-7b · sdxl · +18
How it works

How Inference works

  1. 1

    Create a key

    Generate an API key in the Inference Cloud console and set a quota if you want one.

  2. 2

    Point your client at Podstack

    Change the base URL in any OpenAI-compatible SDK and choose a model.

  3. 3

    Send requests

    Chat, completion and embedding requests are billed per token as they are served.

How it bills

How Inference bills

per token

Inference

Input and output tokens are metered per model and charged to your Podstack wallet. Idle endpoints are free.

See pricing
Works with

Inference works with

FAQ

Questions about Inference

Is Podstack Inference compatible with the OpenAI SDK?

Yes, Podstack Inference exposes an OpenAI-compatible API, so existing clients work after changing the base URL and API key.

What does scale to zero mean here?

An endpoint with no traffic keeps no GPU allocated and costs nothing, and cached weights plus a warm pool bring it back quickly when requests arrive.

How is inference billed?

Inference is billed per token, split into input and output tokens, against your Podstack wallet.

Which models are available?

The catalogue includes open-weight models and frontier models behind the same API, and the current list is shown in the Inference Cloud console.

Can I limit spend on a key?

Yes, each API key can carry a quota, and usage per key is visible in the console.

Can I run the same serving stack in my own data center?

Yes, the serving stack behind Podstack Inference is licensed to operators as Token Factory, part of Podstack OS.

Start with Inference

Podstack Inference is one OpenAI-compatible API for open and frontier models that scales to zero and bills per token.

Open Inference CloudRead the API docs