改动 / #2970

#2970 serve: expose configured pricing in model listings

open · @lindestad · 0 评论 · 去 GitHub 看

Server & API

说明

Issue: #2969 

## Summary

Clients such as Hermes can discover token prices from an endpoint's model listing. Strata currently omits those rates, leaving Hermes without endpoint-provided pricing.

This adds an optional top-level `pricing` object to the run config and reports it in `/v1/models` and `/models`, including alias entries. Operators supply the rates; clients calculate usage estimates.

Existing configs need no changes. Without `pricing`, the model listing remains byte-identical to the pre-feature response. `null` and `{}` also omit pricing.

## What changed

- Validate pricing before loading the engine, then include it in model discovery.
- Accept finite, nonnegative numbers or decimal strings, including zero. Malformed blocks and invalid known rates produce a config error. Preserve unknown fields for client extensions.
- Document the configuration, units, and restart requirement.
- Add nine tests covering validation, startup, both endpoints, aliases, free rates, and unchanged listings without pricing.

Example top-level config entry:

```json
"pricing": {
  "prompt": "0.00000015",
  "completion": "0.00000047",
  "input_cache_read": "0.000000016",
  "input_cache_write": "0.0000002",
  "request": "0"
}
```

Token rates are USD per token; `request` is USD per API request. These names follow OpenRouter's pricing object. Hermes-style `cache_read` and `cache_write` names are also accepted. The example rates are operator-defined accounting values.

## Extra Notes

Validation: 25 tests passed with:

```sh
python -m unittest serve.test_model_pricing \
  serve.test_server.ModelAliases \
  serve.test_server.ConfigApiKey \
  serve.test_runconfig -q
```

Both endpoint response bodies were also compared directly against the pre-feature parent using the mock engine: no config file, an empty config, an existing model config, and aliases all remained byte-identical without pricing.

Related: #2204, #2370, and #2400 address usage tracking or dashboards. This change only exposes rates to external clients; it adds no usage storage, billing calculations, UI, engine changes, or dependencies.

本站相关内容

相关页面的快捷入口。