Lutflow

Measure a running vLLM server's throughput and model its cost per processed token, including idle capacity. This local measurement path (Tier 0) needs no Lutflow account, hosted backend, Kubernetes agent or GPU sensor.

Choose your path

The rest of this page is the SDK path.

Before you install

You need Python 3.9+ for the CLI (the anthropic and all extras require Python 3.10+), an existing vLLM server whose /metrics endpoint is reachable and, for dollar figures, the hourly price you pay for its GPU capacity. Check the token counters first (change the URL if your server is elsewhere):

curl -fsS http://localhost:8000/metrics | grep -E 'vllm:(prompt|generation)_tokens_total'

If this command returns no counters, check the server URL and vLLM metrics configuration before continuing. No vLLM server yet? vLLM on GKE with one L4 sets one up by hand.

Install

Run lutflow with uv, or install it with uv or pipx. Each keeps it in its own isolated environment:

# uv: run it without installing anything permanently
uvx --from 'lutflow>=0.3.2,<0.4' lutflow --version

# uv: install the command for good
uv tool install 'lutflow>=0.3.2,<0.4'

# pipx: install the command for good
pipx install 'lutflow>=0.3.2,<0.4'

The version range makes the install fail instead of quietly picking an older release. With uvx, write every command as uvx --from 'lutflow>=0.3.2,<0.4' lutflow ...; after uv tool install or pipx install, the command is just lutflow. To use lutflow from your own Python code, install it with pip in a virtual environment, as shown in SDK: Install.

Check before measuring

From 0.3.1, lutflow check --endpoint http://localhost:8000 checks one exposition without generating load. A PASS is not a measurement or hardware certification; SDK: check explains the profile, exits and limits. Then measure over a window.

Measure

lutflow measure --endpoint http://localhost:8000 \
  --duration 300 --interval 10 \
  --gpu-hourly-rate 0.85 --rate-basis instance \
  --gpus-in-basis 1 --gpu-count 1 \
  --json measurement.json

Replace 0.85 with your real hourly price. instance means that price covers the entire machine; --gpus-in-basis states how many GPUs it covers and --gpu-count how many this deployment pays for. Use per_gpu when the price is for one GPU. Without a verified price, omit those four options and the dollar fields report unavailable. The command observes existing traffic and does not generate requests.

Read the result

Dollar figures combine measured counters with the hourly rate you provide. They are modeled estimates, not charges read from a billing account. Here, "useful tokens" means prompt plus generated tokens; it does not mean answers passed a quality check.

The report separates allocated economics for the whole window from active efficiency during productive intervals. If there is no useful work, cost per token is undefined. Power, energy, VRAM and GPU utilization are unavailable without a sensor; they are never shown as zero. If the engine restarts during the window, the command ends with exit code 1 and no report, provided the engine publishes its process identity. The JSON stays on your machine.

If your server has no traffic, see SDK: bounded probe for an explicit, confirmed synthetic-load measurement. Before relying on TPOT or p95, read the known limitations of the published release. The SDK also has separate in-process provider budget controls.

License

The Lutflow SDK is source-available under the Business Source License 1.1 (BSL 1.1), with an Additional Use Grant that excludes using it for a GPU Cost Management Service. It changes to the Apache License 2.0 on March 29, 2030. See Security & Trust for the terms and for data handling.