Lutflow

Lutflow provides AI Cost Intelligence for LLM inference. Track and enforce your LLM spend in-process, and compute cost and energy metrics from your telemetry export.

Choose Your Path

SDK (PyPI)

Offline cost measurement from GPU telemetry. Install via pip, measure energy per token.

Sentinel (Helm)

Kubernetes agent for live GPU telemetry. DaemonSet with Kafka export and budget enforcement.

Quick Install

SDK

pip install lutflow==0.2.0
lutflow measure --from-telemetry export.jsonl

Sentinel

helm install lutflow-sentinel oci://ghcr.io/lutflow/charts/lutflow-sentinel \
  --namespace lutflow --create-namespace \
  --set global.tenantId="my-tenant"

How It Works

Collect — SDK reads telemetry exports; Sentinel scrapes /metrics from vLLM/TGI/BentoML
Compute — Energy (joules) and cost ($) per 1000 tokens, segmented by concurrency
Export — Local measurement or Kafka streaming to your analytics pipeline
Enforce — Optional budget limits with configurable actions (warn, error, kill)

License

Lutflow is source-available under BSL 1.1. See Security & Trust for our guarantees.

Quickstart