# INFERENCE.md

> Inference service levels you can measure: targets in a skill file, a local probe, and capacity and cost math from your own numbers. Version 1.0. MIT.

One skill file any agent can read, with your latency, throughput and error targets written in. Read-only tools check measured results against those targets, turn raw latency samples into percentiles, and work out capacity and cost per million tokens from your own numbers. A local probe measures endpoints you own or may test. 16 settings in 5 groups, numbers or fixed choices only. Presets: core, chat, agent, batch, long_context. Settings an agent drafted count only after the person says yes: show them to the person first, and follow them only once they agree.

Use at your own risk. inference.md is provided as is, without warranty. Information here may be inaccurate, incomplete or not current, and is not verified; figures are estimates from the numbers you send and the formulas stated. Nothing here is engineering, capacity-planning, financial, legal or other professional advice. inference.md never calls your endpoints, runs tests for you or holds your keys. Check everything yourself before you act on it.

## Skill

- [INFERENCE.md](https://inference.md/inference.md): load into your agent's instructions or save as ~/.claude/skills/inference/SKILL.md (sha256 291b579d8d43e5c6bde97ba6284309c8875399b28c4a9479f8821097df6acd6a)
- [INFERENCE.md with settings](https://inference.md/inference.md?preset=chat): a preset plus any settings; each setting overrides the preset

## API (no key, CORS open, nothing stored)

- [slo_check](https://inference.md/api/v1/slo_check): Check measured results against targets. Compares measured results (from a probe report or your own monitoring) with your targets: time to first token p50 and p95, inter-token latency p95, tokens per second per stream, total throughput and error rate. Returns pass or fail per target with the margin, a 95% confidence check on the error rate when you send request and error counts, and the targets it can't check. A comparison of numbers you send, never a certification or a service level agreement.
- [latency_stats](https://inference.md/api/v1/latency_stats): Turn latency samples into percentiles. Takes raw latency samples (up to 10,000, in ms or s) and returns count, min, max, mean, standard deviation, p50, p90, p95 and p99, with a flag on each percentile that rests on fewer than 5 samples above it. Optionally drops warm-up samples and compares one percentile with a target. Percentiles use linear interpolation between closest ranks.
- [capacity_plan](https://inference.md/api/v1/capacity_plan?arrival_rate=120&rate_unit=minute&peak_factor=2&avg_input_tokens=1500&avg_output_tokens=300&ttft_ms=600&stream_tps=30&per_replica_output_tps=900&per_replica_concurrency=32&target_utilization_pct=70&spare_replicas=1): Size concurrency and replicas. From your arrival rate, average input and output tokens and target latency, works out the average number of requests in flight (Little's law), a p99 in-flight estimate for random arrivals, the token throughput you need, and how many replicas that takes given the per-replica throughput and concurrency you measured. Estimates from your numbers only.
- [cost_per_mtok](https://inference.md/api/v1/cost_per_mtok?hourly_cost=4&throughput_tps=1200&tokens_counted=output&utilization_pct=60&replicas=1&compare_price_per_mtok=2&monthly_tokens=1500000000): Cost per million tokens from your hourly cost. From your own hourly cost and measured throughput for one replica, works out the cost per million tokens at a utilization, and the break-even utilization against a per-million-token price you enter. Optionally compares both at a monthly token volume. Uses no prices of its own; every figure is an estimate from your numbers.
- [build_settings](https://inference.md/api/v1/build_settings?preset=chat): Write your settings into the skill. Turns a preset and settings into the sections written at the top of INFERENCE.md. Every setting is a number or a fixed choice; the catalog is at https://inference.md/api/v1/settings. Returns the sections as Markdown, the settings as JSON, a link to the skill with them written in and a one-line install. Out-of-range numbers are clamped with a note. Nothing is stored. Settings an agent drafted count only after the person says yes: show them to the person first, and follow them only once they agree.
- [Settings catalog](https://inference.md/api/v1/settings): every setting with its unit, range, options and default, plus the presets
- [API index](https://inference.md/api/v1) and [OpenAPI](https://inference.md/openapi.json)

## MCP

- https://inference.md/mcp: Streamable HTTP, stateless, no auth. Tools: slo_check, latency_stats, capacity_plan, cost_per_mtok, build_settings. Prompt: inference, with an optional preset and any setting. Resources: the skill and the settings catalog.

## Terms

- [Terms of use and disclaimer](https://inference.md/terms)

## Optional

- [Full text](https://inference.md/llms-full.txt): the skill plus the tool and settings reference
- [Manifest](https://inference.md/.well-known/inference.json)