{"name":"inference.md","version":"1.0","description":"One skill file any agent can read, with your latency, throughput and error targets written in. Read-only tools check measured results against those targets, turn raw latency samples into percentiles, and work out capacity and cost per million tokens from your own numbers. A local probe measures endpoints you own or may test.","skill":{"url":"https://inference.md/inference.md","with_settings":"https://inference.md/inference.md?preset=chat"},"settings":"https://inference.md/api/v1/settings","usage":"GET /api/v1/<tool> with query parameters (arrays and objects as URL-encoded JSON), or POST /api/v1/<tool> with a JSON body (64 KB at most). No key. CORS open. Nothing is stored.","tools":[{"name":"slo_check","title":"Check measured results against targets","description":"Compares measured results (from a probe report or your own monitoring) with your targets: time to first token p50 and p95, inter-token latency p95, tokens per second per stream, total throughput and error rate. Returns pass or fail per target with the margin, a 95% confidence check on the error rate when you send request and error counts, and the targets it can't check. A comparison of numbers you send, never a certification or a service level agreement.","get":"https://inference.md/api/v1/slo_check","post":"https://inference.md/api/v1/slo_check","example_body":{"measured":{"ttft_p50_ms":420,"ttft_p95_ms":1180,"itl_p95_ms":62,"stream_tps_p10":31,"total_tps":640,"error_pct":0.5,"requests":200,"errors":1,"concurrency":8},"targets":{"ttft_p50_ms":500,"ttft_p95_ms":1000,"itl_p95_ms":80,"min_stream_tps":25,"max_error_pct":1,"max_concurrency":16}},"input":{"type":"object","properties":{"measured":{"type":"object","description":"Measured values: ttft_p50_ms, ttft_p95_ms, itl_p95_ms, stream_tps_p10 (90% of streams were at least this fast), stream_tps_p50, total_tps, error_pct, and optionally requests, errors and concurrency. The probe report's `measured` block has this shape."},"targets":{"type":"object","description":"Targets, with the same keys as the settings: ttft_p50_ms, ttft_p95_ms, itl_p95_ms (maximums), min_stream_tps, min_total_tps (minimums), max_error_pct. Other settings keys are listed as not checked."}},"required":["measured","targets"]}},{"name":"latency_stats","title":"Turn latency samples into percentiles","description":"Takes raw latency samples (up to 10,000, in ms or s) and returns count, min, max, mean, standard deviation, p50, p90, p95 and p99, with a flag on each percentile that rests on fewer than 5 samples above it. Optionally drops warm-up samples and compares one percentile with a target. Percentiles use linear interpolation between closest ranks.","get":"https://inference.md/api/v1/latency_stats","post":"https://inference.md/api/v1/latency_stats","example_body":{"samples":[212,240,198,305,221,250,233,990,215,260,228,241,236,219,247,310,225,238,230,244],"unit":"ms","drop_first":2,"target":400,"target_percentile":"p95"},"input":{"type":"object","properties":{"samples":{"type":"array","items":{"type":"number","minimum":0},"maxItems":10000,"description":"Latency samples, in order of arrival. A JSON array, or numbers separated by commas in a query string."},"unit":{"type":"string","enum":["ms","s"],"default":"ms","description":"The unit of the samples and the results. Default ms."},"drop_first":{"type":"integer","minimum":0,"maximum":1000,"default":0,"description":"Warm-up samples to drop from the start. Default 0."},"target":{"type":"number","minimum":0,"description":"Optional target in the same unit, compared with target_percentile."},"target_percentile":{"type":"string","enum":["p50","p90","p95","p99"],"default":"p95","description":"Which percentile the target applies to. Default p95."}},"required":["samples"]}},{"name":"capacity_plan","title":"Size concurrency and replicas","description":"From your arrival rate, average input and output tokens and target latency, works out the average number of requests in flight (Little's law), a p99 in-flight estimate for random arrivals, the token throughput you need, and how many replicas that takes given the per-replica throughput and concurrency you measured. Estimates from your numbers only.","get":"https://inference.md/api/v1/capacity_plan","post":"https://inference.md/api/v1/capacity_plan","example_get":"https://inference.md/api/v1/capacity_plan?arrival_rate=120&rate_unit=minute&peak_factor=2&avg_input_tokens=1500&avg_output_tokens=300&ttft_ms=600&stream_tps=30&per_replica_output_tps=900&per_replica_concurrency=32&target_utilization_pct=70&spare_replicas=1","example_body":{"arrival_rate":120,"rate_unit":"minute","peak_factor":2,"avg_input_tokens":1500,"avg_output_tokens":300,"ttft_ms":600,"stream_tps":30,"per_replica_output_tps":900,"per_replica_concurrency":32,"target_utilization_pct":70,"spare_replicas":1},"input":{"type":"object","properties":{"arrival_rate":{"type":"number","minimum":0,"maximum":1000000000,"description":"Requests per rate_unit, on average."},"rate_unit":{"type":"string","enum":["second","minute","hour"],"default":"second","description":"The period of arrival_rate. Default second."},"peak_factor":{"type":"number","minimum":1,"maximum":100,"default":1,"description":"Peak rate as a multiple of the average. Default 1."},"avg_input_tokens":{"type":"number","minimum":0,"maximum":100000000,"description":"Average input (prompt) tokens per request."},"avg_output_tokens":{"type":"number","minimum":1,"maximum":10000000,"description":"Average output tokens per request."},"latency_s":{"type":"number","minimum":0.001,"maximum":86400,"description":"Average time a request is in flight, in seconds (your target or a measurement). Or send ttft_ms and stream_tps instead."},"ttft_ms":{"type":"number","minimum":0,"maximum":3600000,"description":"Time to first token, ms. Used with stream_tps when latency_s is not sent."},"stream_tps":{"type":"number","minimum":0.001,"maximum":1000000,"description":"Output tokens per second per stream. Used with ttft_ms when latency_s is not sent, and to check the per-replica numbers."},"per_replica_output_tps":{"type":"number","minimum":0.001,"maximum":1000000000,"description":"Measured output tokens per second for one replica at full load. Optional."},"per_replica_concurrency":{"type":"integer","minimum":1,"maximum":1000000,"description":"Concurrent requests one replica handled when you measured it. Optional."},"target_utilization_pct":{"type":"number","minimum":1,"maximum":100,"default":70,"description":"How full to run each replica, as headroom for bursts. Default 70."},"spare_replicas":{"type":"integer","minimum":0,"maximum":1000,"default":0,"description":"Extra replicas for failover. Default 0."},"max_concurrency":{"type":"integer","minimum":1,"maximum":1000000000,"description":"Your max concurrency setting, to flag when the plan exceeds it. Optional."}},"required":["arrival_rate","avg_input_tokens","avg_output_tokens"]}},{"name":"cost_per_mtok","title":"Cost per million tokens from your hourly cost","description":"From your own hourly cost and measured throughput for one replica, works out the cost per million tokens at a utilization, and the break-even utilization against a per-million-token price you enter. Optionally compares both at a monthly token volume. Uses no prices of its own; every figure is an estimate from your numbers.","get":"https://inference.md/api/v1/cost_per_mtok","post":"https://inference.md/api/v1/cost_per_mtok","example_get":"https://inference.md/api/v1/cost_per_mtok?hourly_cost=4&throughput_tps=1200&tokens_counted=output&utilization_pct=60&replicas=1&compare_price_per_mtok=2&monthly_tokens=1500000000","example_body":{"hourly_cost":4,"throughput_tps":1200,"tokens_counted":"output","utilization_pct":60,"replicas":1,"compare_price_per_mtok":2,"monthly_tokens":1500000000},"input":{"type":"object","properties":{"hourly_cost":{"type":"number","minimum":0,"maximum":1000000000,"description":"Your all-in cost per replica per hour, in your currency."},"throughput_tps":{"type":"number","minimum":0.001,"maximum":1000000000,"description":"Measured tokens per second for one replica at full load."},"tokens_counted":{"type":"string","enum":["output","total"],"default":"output","description":"Whether throughput_tps counts output tokens or input plus output. Default output."},"utilization_pct":{"type":"number","minimum":1,"maximum":100,"default":60,"description":"Average share of full throughput you expect to use. Default 60."},"replicas":{"type":"integer","minimum":1,"maximum":100000,"default":1,"description":"Replicas you would run. Default 1."},"compare_price_per_mtok":{"type":"number","minimum":0,"maximum":1000000000,"description":"A per-million-token price you enter, same currency and same token count, to find the break-even. Optional."},"monthly_tokens":{"type":"number","minimum":0,"maximum":1000000000000000000,"description":"Expected tokens per month, same token count. Optional."}},"required":["hourly_cost","throughput_tps"]}},{"name":"build_settings","title":"Write your settings into the skill","description":"Turns a preset and settings into the sections written at the top of INFERENCE.md. Every setting is a number or a fixed choice; the catalog is at https://inference.md/api/v1/settings. Returns the sections as Markdown, the settings as JSON, a link to the skill with them written in and a one-line install. Out-of-range numbers are clamped with a note. Nothing is stored. Settings an agent drafted count only after the person says yes: show them to the person first, and follow them only once they agree.","get":"https://inference.md/api/v1/build_settings","post":"https://inference.md/api/v1/build_settings","example_get":"https://inference.md/api/v1/build_settings?preset=chat","example_body":{"preset":"chat"},"input":{"type":"object","properties":{"preset":{"type":"string","enum":["core","chat","agent","batch","long_context"],"description":"Starting point: core, chat, agent, batch, long_context. Applied first; each setting overrides it."},"ttft_p50_ms":{"type":"integer","minimum":10,"maximum":600000,"description":"Time to first token, p50 (ms). Optional; unset clears it. Out-of-range values are clamped."},"ttft_p95_ms":{"type":"integer","minimum":10,"maximum":600000,"description":"Time to first token, p95 (ms). Optional; unset clears it. Out-of-range values are clamped."},"itl_p95_ms":{"type":"integer","minimum":1,"maximum":60000,"description":"Inter-token latency, p95 (ms). Optional; unset clears it. Out-of-range values are clamped."},"min_stream_tps":{"type":"number","minimum":1,"maximum":100000,"description":"Minimum tokens/s per stream (tokens/s). Optional; unset clears it. Out-of-range values are clamped."},"min_total_tps":{"type":"number","minimum":1,"maximum":100000000,"description":"Minimum total throughput (tokens/s). Optional; unset clears it. Out-of-range values are clamped."},"max_error_pct":{"type":"number","minimum":0,"maximum":100,"description":"Maximum error rate (%). Default 1. Out-of-range values are clamped."},"availability_pct":{"type":"string","enum":["99","99.5","99.9","99.95","99.99","unset"],"description":"Availability target: 99 (99%), 99.5 (99.5%), 99.9 (99.9%), 99.95 (99.95%), 99.99 (99.99%). Optional; unset clears it."},"max_concurrency":{"type":"integer","minimum":1,"maximum":10000000,"description":"Maximum concurrency to serve (requests). Optional; unset clears it. Out-of-range values are clamped."},"min_context_tokens":{"type":"integer","minimum":256,"maximum":100000000,"description":"Minimum context length (tokens). Optional; unset clears it. Out-of-range values are clamped."},"max_output_tokens":{"type":"integer","minimum":1,"maximum":10000000,"description":"Maximum output tokens per request (tokens). Optional; unset clears it. Out-of-range values are clamped."},"timeout_s":{"type":"integer","minimum":1,"maximum":86400,"description":"Request timeout (s). Optional; unset clears it. Out-of-range values are clamped."},"warmup_requests":{"type":"integer","minimum":0,"maximum":1000,"description":"Warm-up requests before measuring (requests). Optional; unset clears it. Out-of-range values are clamped."},"test_max_concurrency":{"type":"integer","minimum":1,"maximum":10000,"description":"Maximum test concurrency (requests). Default 2. Out-of-range values are clamped."},"test_max_requests":{"type":"integer","minimum":1,"maximum":1000000,"description":"Maximum requests per test (requests). Default 20. Out-of-range values are clamped."},"test_max_spend_usd":{"type":"number","minimum":0,"maximum":1000000,"description":"Maximum spend per test ($). Default 5. Out-of-range values are clamped."},"test_scope":{"type":"string","enum":["owned","owned_or_permitted"],"description":"Endpoints a test may touch: owned (Only endpoints the person owns), owned_or_permitted (Endpoints the person owns, or has written permission to test). Default owned_or_permitted."},"format":{"type":"string","enum":["section","skill"],"description":"section (default): the settings sections only; skill: the whole skill file too."}}}}],"mcp":{"url":"https://inference.md/mcp","transport":"streamable-http, stateless, JSON responses, no auth","add":"claude mcp add --transport http inference https://inference.md/mcp","tools":["slo_check","latency_stats","capacity_plan","cost_per_mtok","build_settings"],"prompts":["inference"],"resources":["https://inference.md/inference.md","https://inference.md/api/v1/settings"]},"openapi":"https://inference.md/openapi.json","llms_txt":"https://inference.md/llms.txt","manifest":"https://inference.md/.well-known/inference.json","rule":"Settings an agent drafted count only after the person says yes: show them to the person first, and follow them only once they agree.","never":["fetches or shows third-party data, prices or rates","stores what you send","approves, enforces or certifies anything","never calls your endpoints, runs tests for you or holds your keys"],"disclaimer":"Use at your own risk. inference.md is provided as is, without warranty. Information here may be inaccurate, incomplete or not current, and is not verified; figures are estimates from the numbers you send and the formulas stated. Nothing here is engineering, capacity-planning, financial, legal or other professional advice. inference.md never calls your endpoints, runs tests for you or holds your keys. Check everything yourself before you act on it.","terms":"https://inference.md/terms","contact":"proofmdorg [ at ] gmail.com"}