Documentation
Technical guides for the Prune LLM proxy - integration, security, and comparisons.
Quick answer
Prune is a managed AI gateway. Change your SDK base URL, add a Prune API key, and every response includes a cost/security receipt. Unique-path savings apply even when prompts never repeat; cache is upside.
Start here
How it works
Request lifecycle, Shield, unique-path savings, prune_metadata receipts.
Hackathon setup
5-minute path: signup with event link, first request, signed receipt.
Security model
AES-256-GCM vault, fail-closed Shield, canary tokens, audit fixes.
Prune Check
Paste a prompt, see security flags, cost, and a sample receipt. No signup.
Quick start
AES-256-GCM encryption for vaulted provider API keys vault · Spend caps checked before the provider is called
In our 26 Sep lab, new questions cost about 94% less than sending everything to gpt-4o, and repeats were free. Lab result, not a customer bill.
pip install openai
export PRUNE_API_KEY=prune_... # dashboard → API Keys. Your OpenAI key stays in the Prune vault.
import os
from openai import OpenAI
PRUNE_API_KEY = os.environ["PRUNE_API_KEY"]
client = OpenAI(
base_url="https://api.withprune.com/v1",
api_key=PRUNE_API_KEY,
)
resp = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
print(resp.model_dump().get("prune_metadata")) # cache_hit, cost, signed receiptSet up with your coding agent
Paste this into Cursor, Claude Code, Codex or Windsurf. It finds your LLM clients, points them at Prune, and checks one request. Your keys never go in the chat.
Show the prompt
Add Prune to this project so its LLM API calls go through the Prune gateway.
Before you change code:
- I create a Prune API key (it starts with prune_) at https://app.withprune.com under Connect > API Keys, and save my provider key under Connect > Providers. Do not ask me to paste either key into this chat.
- Read the key from the PRUNE_API_KEY environment variable. Never hard-code it and never ship it to a browser or mobile bundle. Add PRUNE_API_KEY to .env.example and make sure the real env file is in .gitignore.
Steps:
1. Find every place this app creates an LLM client (OpenAI, Anthropic, or any OpenAI-compatible SDK).
2. Change only the client construction. Do not rewrite call sites, prompts or model names.
- OpenAI SDK, Python: OpenAI(base_url="https://api.withprune.com/v1", api_key=os.environ["PRUNE_API_KEY"])
- OpenAI SDK, TypeScript: new OpenAI({ baseURL: "https://api.withprune.com/v1", apiKey: process.env.PRUNE_API_KEY })
- Anthropic SDK: base_url / baseURL "https://api.withprune.com" with api_key / apiKey set to PRUNE_API_KEY
- Raw HTTP: POST https://api.withprune.com/v1/chat/completions with header "Authorization: Bearer $PRUNE_API_KEY"
3. Do not migrate these, list them for me instead: calls to the OpenAI Responses API (client.responses.create is not supported through Prune yet), and Bedrock calls with stream=true.
4. Verify with one real request. Expect HTTP 200, a prune_metadata object in the JSON body, and an X-Prune-Provider response header. Sending the same request again should return X-Prune-Cache-Hit: true.
5. Optional, only if the code already has these concepts: send X-Prune-Customer-Id with the end customer's id, X-Prune-Feature-Id with the product feature name, and for agent loops X-Prune-Run-Id plus X-Prune-Run-Budget-Usd.
Keep the diff minimal and do not refactor unrelated code. Full reference: https://www.withprune.com/llms-full.txtAlso
Guides
Stop an agent loop from running up your bill
Per-run budgets, step ceilings, repeat-call breakers, daily caps.
Gemini API key leaked: the first hour
Revoke, restrict, cap quotas, find the leak, dispute the bill.
Prompt caching vs semantic caching
What each one saves, when each goes wrong, and when to use both.
Reduce your OpenAI API cost
Eight fixes in the order that usually saves the most.
OpenAI API key spending limit
What OpenAI and Anthropic actually enforce, and how to cap one key.
OpenAI or Claude API key leaked
Revoke, stop spend, find usage by key, purge it from git.
Limit OpenAI usage per user
Per-customer dollar budgets, the DIY way and the one-header way.
Why is my Gemini API bill so high?
Thinking tokens, grounding, model choice, caps, and the 2027 price change.
LLM model routing
When sending easy calls to a cheaper model saves money, and when it breaks.
Is OpenRouter safe for production?
Data retention, per-key limits, fees, and a production checklist.
Reduce Claude Code cost on your own API key
Host-only base URL, a daily cap, and why the subscription bill does not move.
LangChain and Vercel AI SDK base URL
ChatOpenAI and createOpenAI pointed at Prune, with a per-user cap.
Compare
Prune vs cache-first gateways
Why hit rate is not the thesis - unique-path + Shield + proof.
Prune vs LiteLLM
Self-hosted vs managed - when to use each.
Prune vs Portkey
Guardrails-as-plugin vs default-on Shield enforcement.
Prune vs Helicone
Observability-first vs a control plane that enforces policy and cost.
Prune vs Cloudflare AI Gateway
Edge caching/analytics vs a provider-agnostic security + cost layer.
Start free at app.withprune.com/signup (no card), then vault a key and send one request.
