# Inference Costs

> The AI suite of product features is making SaaS Gross Margin look much closer to Service-based business models. Software used to benefit from a nearly negligible cost of fulfillment. Now it must account for multiple COGS levers that scale with usage. In order to keep your offering priced sustainably, inference costs need enough room to flex with the introduction of new models & use cases. To do this, calculate your expected customer volume first. Translate the volume to their token usage and relative cost of the model used. Finally, include any pricing for API calls or compute time. Stress test where your pricing breaks relative to user behavior or model consumption.

Published model on Paths: https://getpaths.io/p/inference-costs
Values below are the base case of the published snapshot.

## Drivers

- Customers: 250 - Customers sending model queries this month
- Requests per customer: 120 - Average model queries each customer sends this month
- Input tokens per request: 2,000 - Prompt tokens the provider bills on one request
- Output tokens per request: 500 - Completion tokens the provider bills on one request
- Input price per million: $2.50 - Provider price for one million input tokens
- Output price per million: $10.00 - Provider price for one million output tokens
- API calls per request: 1 - Billed API calls for one customer request, including retries
- Price per API call: $0.00 - Gateway or platform fee charged once per API call
- Compute seconds per request: 2.5 - GPU or CPU seconds used to serve one request
- Price per compute second: $0.00 - Price of one second of the compute that serves a request

## Formulas

- Customer requests = Customers * Requests per customer -> 30,000 - Model queries this month
- Output tokens = Customer requests * Output tokens per request -> 15,000,000 - Output tokens billed this month
- Input tokens = Customer requests * Input tokens per request -> 60,000,000 - Input tokens billed this month
- API calls = Customer requests * API calls per request -> 30,000 - API calls billed this month
- Compute seconds = Customer requests * Compute seconds per request -> 75,000 - Compute seconds billed this month
- Output token cost = Output tokens * Output price per million / 1000000 -> $150.00 - Spend on output tokens this month
- Input token cost = Input tokens * Input price per million / 1000000 -> $150.00 - Spend on input tokens this month
- API call cost = API calls * Price per API call -> $60.00 - Spend on per-call fees this month
- Compute cost = Compute seconds * Price per compute second -> $75.00 - Spend on compute time this month
- Token cost = Input token cost + Output token cost -> $300.00 - Combined spend on input and output tokens this month
- Inference cost = Token cost + API call cost + Compute cost -> $435.00 - Token, API call, and compute spend this month
- Cost per request = Inference cost / Customer requests -> $0.01 - Inference spend divided by model queries
- Cost per customer = Inference cost / Customers -> $1.74 - Inference spend divided by customers