All 42 regions operational · 99.983 % uptime last 30d
p95 global · 340 msTokens served last hr · 42.6 Mstatus.lumenproto.dev →
Open-source models · Lumen Protocol v2.4 is live

Low-latency inference, routed closer to your users.
// 42 edge regions · 340 ms p95

Lumen Protocol routes requests across 42 edge regions. We serve the open-source model family — Llama, Qwen, Mistral, DeepSeek, Nemotron — at latencies production AI apps can actually ship on. Pay per million tokens. No seats. No contracts under 10 billion.

Get an API key → Read the /v2 docs
Shipping in production at
lumenproto.dev ▸ playground ▸ live-stream
session · eu-west-1

POST /v2/chat/completions streaming

Live telemetry · last 60 s

TTFT · p50
142ms
TPS · stream
186tok/s
Success rate
99.98%
Regions in use
12/42
Models available under one API surface
// what you get

Six primitives, built around production serving.

Everything in the platform is designed around one promise: your p95 should not change when a region goes down. Everything else — streaming, tool use, structured output, moderation — is a composition of these six.

// 01⌁

Smart routing

Requests are routed by latency-weighted health, not geographic distance. If eu-west-1 is slow, we fail over to eu-central-2 before you notice.

// one endpoint, all regions
POST /v2/chat/completions
// 02⟐

Token streaming

Server-sent events, native. First token delivered in a median 142 ms on the US-West edge; last token flushed in 2.8 s on 1 024-token completions.

for await (const t of stream) …
// 03⌬

Structured output

Pass a JSON schema; we constrain decoding to match. 99.3 % schema-valid first-pass on the ArenaStruct benchmark.

response_format: {
  type: "json_schema", …
}
// 04⎔

Tool use

Native parallel function calling. Up to 16 tools per request. The model picks; you handle. We handle the retries.

tools: [{ name: "query_db", … }]
// 05∴

Caching, by request

Content-addressed prompt caching. 60 to 90 % discount on repeated prefix tokens. We cache you don't configure.

// automatic. nothing to toggle.
// 06◉

Private endpoints

Dedicated single-tenant deployments on request. Starts at 240 B tokens per month committed. Deployed in 48 hours.

GET /v2/tenants/me/endpoint
// benchmarks

Compared on p95 latency, head to head.

Public API endpoints, same prompt, same completion length, 500 requests per endpoint, measured from us-east-1. We publish our methodology and raw numbers; read the full report (PDF, 14 pages).

ProviderTTFT · p95TPS · streamCost · 1M in
Lumen Protocol340 ms186 tok/s$0.68
Provider A640 ms102 tok/s$0.90
Provider B480 ms142 tok/s$1.20
Provider C820 ms88 tok/s$0.72
Provider D1 180 ms74 tok/s$0.55

Methodology, in one glance.

  • 01

    Same model, same prompt

    Llama 3.1 70B · structured JSON prompt from the ArenaStruct public set.

  • 02

    Same geography

    All measurements from a single Fastly PoP in us-east-1. 500 requests each, warmed.

  • 03

    Same completion length

    Hard max 1 024 output tokens. Mean completion 912 tokens.

  • 04

    Public, reproducible

    The entire test harness is on GitHub. Clone it, re-run it, compare.

// pricing

Pay per million tokens. Nothing else.

No seats. No per-user fees. No contract minimum under 10 B tokens / month. Volume rolls over 90 days.

Developer

$0.68/ 1M in
$2.10 / 1M out · all shared models
  • All open-source models
  • Shared edge regions · 42 total
  • Pay as you go · no commit
  • Community support · Discord
  • Standard rate limits (300 rpm)
Start free · 100 K tokens

Scale

$0.48/ 1M in
$1.60 / 1M out · from 1 B tokens / mo
  • Everything in Developer
  • Priority routing · lower p95
  • Prompt caching · auto 60 – 90 % off
  • Dashboard · teams · audit log
  • Email support · 2 hr p95 response
Request a key →

Private

Talk// 10B+ tokens / mo
Dedicated single-tenant endpoint
  • Dedicated GPU allocation
  • Choice of 1 to 3 regions
  • Custom models · fine-tuned
  • 99.99 % SLA · signed
  • Slack Connect · dedicated engineer
Talk to our team →
// docs

From curl to production, in nine lines.

We maintain official SDKs for TypeScript, Python, Go, and Rust. The HTTP surface is OpenAI-compatible: pass Authorization: Bearer, change the base URL, you're done.

Read the full /v2 docs → Clone the quickstart repo → Model catalog →
// install the SDK
npm install @lumen/sdk

// one call, token streamed
import { Lumen } from "@lumen/sdk"
const lumen = new Lumen({ apiKey: process.env.LUMEN_KEY })

const stream = await lumen.chat.stream({
  model: "llama-3.1-70b",
  messages: [{ role: "user", content: "ping" }],
  temperature: 0.2,
})

for await (const token of stream) {
  process.stdout.write(token.text)
}