Polyweave
Repositories
1
 Projects Packages

@polyweave/costs (1.0.7)

Published 2026-07-17 19:53:05 +00:00 by Dvorak

Installation

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/
npm install @polyweave/costs@1.0.7
"@polyweave/costs": "1.0.7"

About this package

@polyweave/costs

Cost tracking, prediction, and budget enforcement for Polyweave adapters. Provides a frame-protocol cost tracker with anytime-valid statistical guarantees, a Gaussian conjugate ratio e-process for output/input token ratio estimation, a rolling-hash cache-hit estimator, per-provider pricing tables, and token estimation utilities.

Synopsis

import { createCostTracker, createRatioEProcess, createRollingHash, setCustomPricing, getModelCost } from '@polyweave/costs';
import { createOpenAIAdapter } from '@polyweave/openai';

const adapter = createOpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY });

const tracker = createCostTracker(adapter, {
  budget: 10.00,
  provider: 'openai',
  priorRatio: 1.5,
  alpha: 0.10,
  mode: 'conservative'
});

tracker.source.pipe(mySink);

// Query budget state
tracker.sink.write({ type: 'DESCRIBE' });
// → DESCRIPTION { budget: 10.00, totalCost: 0, ... }

// Predict cost before executing
tracker.sink.write({ type: 'COST_CHECK', content: { maxBudget: 2.00 } });
// → COST { estimatedCost, pExceeded, wouldBlock: false }

Install

npm install @polyweave/costs

Why

Cost enforcement in LLM pipelines needs more than a simple budget counter. The cost tracker uses:

  • Anytime-valid e-processes — the Gaussian conjugate e-process provides statistical guarantees on false-block probability. Unlike p-values, e-values remain valid under optional stopping and continuous monitoring.
  • Rolling-hash cache estimation — estimates KV-cache reuse across conversation turns by tracking which text windows have been seen before.
  • Provider pricing tables — built-in pricing for 50+ models across all major providers. Overridable via setCustomPricing.

These were extracted from @polyweave/core into a standalone package so consumers can use cost tracking without pulling in the full core dependency tree.

API

createCostTracker(innerDuplex, options)

Returns a wrapped duplex. Intercepts frames flowing through the adapter and tracks cumulative costs from RECEIPT frames. Blocks TEXT frames when budget is exceeded.

Parameter Type Default Description
innerDuplex IFrameDuplex Yes The adapter to wrap
budget number 0 Global spend limit in USD. 0 disables enforcement
provider string null Provider name for pricing lookups
priorRatio number 1.5 Prior mean for the output/input token ratio e-process
priorWeight number 4 Prior precision (pseudo-observations) for the e-process
alpha number 0.10 Confidence interval width for cost estimates
mode string 'conservative' Estimation mode: 'conservative' or 'mean'
cacheDiscount number 0.5 Cost reduction factor for cache-hit tokens (0–1)
sigma2 number 0.01 Observation noise variance for the e-process
onBudgetExceeded() function null Callback when budget is exceeded

Frame protocol:

Inbound (sink) Effect
TEXT / IMAGE / AUDIO Accumulates text for hashing; forwards to inner duplex
COST_CHECK { maxBudget } Returns COST frame with estimate without executing
DESCRIBE Returns DESCRIPTION with full budget state snapshot
PARAM { budget, prior_gpt-4o, ... } Runtime parameter overrides
RESET Resets all accumulators

Output frames added on source:

Frame Content
COST { estimatedCost, pExceeded, wouldBlock } Cost estimate based on e-process prediction
DESCRIPTION { budget, totalCost, inputTokens, outputTokens, providerBreakdown, eprocess, cache } Full budget snapshot
ERROR { code: 'BUDGET_EXCEEDED' } Budget has been exceeded and TEXT frames are blocked

Snapshot:

The DESCRIPTION response includes a detailed snapshot:

  • budget / budgetRemaining — USD limits
  • inputTokens / outputTokens — cumulative token counts
  • inputCost / outputCost / totalCost — cumulative USD costs
  • requestCount — number of requests processed
  • providerBreakdown — per-provider token and cost breakdown
  • eprocess — e-process posterior state (mean, precision, sd)
  • cache — rolling hash state for cache-hit estimation

createRatioEProcess(options)

Returns { update, estimate, tailProbability, testBudgetExceeded, getState }. A Gaussian conjugate e-process for tracking the output/input token ratio.

Parameter Type Default Description
priorRatio number 1.5 Prior mean (typical output tokens per input token)
priorWeight number 4 Prior precision in pseudo-observations
sigma2 number 0.01 Observation noise variance
alpha number 0.10 CI width for estimation and gating

Methods:

Method Description
update(inputTokens, outputTokens) Add an observation pair from a RECEIPT frame
estimate(inputTokens, mode?) Predict output tokens and cost for given input. mode: 'conservative' uses upper CI bound
tailProbability(theta) Posterior probability that ratio ≤ theta
testBudgetExceeded(thetaDanger) Returns true if e-value exceeds 1/alpha (budget likely exceeded)
getState() Returns { n, posteriorMean, posteriorSD, priorRatio, priorWeight }

Statistical properties:

  • The e-process is anytime-valid — you can peek at results and make decisions at any point without inflating false-positive rates
  • The posterior converges as RECEIPT frames arrive
  • testBudgetExceeded gates TEXT frame emission: if the e-process signals budget danger, the tracker blocks

createRollingHash(options)

Returns { match, commit, getState, reset }. Sliding window polynomial hash for cache-hit estimation.

Parameter Type Default Description
windowSize number 32 Sliding window size in characters
base number 31 Polynomial hash base

Methods:

Method Description
match(text) Returns { total, matched, ratio } — how many windows in text have been seen before
commit(text) Adds all windows from text to the history. Returns count of new hashes
getState() Returns { totalHashes, windowSize } — total unique hashes stored
reset() Clears all hash history

Use case: Before sending a request, match() the prompt text against the hash history. If the ratio is high, most of the prompt's KV-cache can reuse prior computation. The cacheDiscount in createCostTracker applies a cost reduction for cached tokens.

getModelCost(provider, model)

Returns the pricing entry for a provider-model pair from the built-in pricing tables.

const pricing = getModelCost('openai', 'gpt-4o');
// → { mtoksInput: 2.50, mtoksOutput: 10.00 }

Returns null if the provider or model is unknown.

calculateTokenCost(inputTokens, outputTokens, costConfig)

Calculates total cost in USD given token counts and a cost configuration.

const cost = calculateTokenCost(1000, 500, { mtoksInput: 2.50, mtoksOutput: 10.00 });
// → { inputCost: 0.0025, outputCost: 0.005, totalCost: 0.0075 }

Returns { inputCost: 0, outputCost: 0, totalCost: 0 } when costConfig is null/undefined.

setCustomPricing(overrides)

Overrides default pricing tables.

setCustomPricing({
  openai: {
    'gpt-5': { mtoksInput: 2.00, mtoksOutput: 15.00 }
  },
  anthropic: {
    'claude-4-5-sonnet': { mtoksInput: 3.50, mtoksOutput: 17.50 }
  }
});

Call before creating adapters. Pricing is global — all adapters share the same pricing tables.

createCostEstimator(provider)

Creates a provider-specific cost estimator. Used internally by adapters.

const estimator = createCostEstimator('openai');
const cost = estimator.estimate({ model: 'gpt-4o', inputTokens: 500 });
// → { inputCost, outputCost, totalCost, model }

createTokenEstimator(options)

Returns { update, estimate, getState, configure }. A Welford-style streaming estimator for the output/input token ratio with prior shrinkage — a lighter-weight alternative to createRatioEProcess.

Parameter Type Default Description
priorRatio number 1.5 Prior mean (typical output tokens per input token)
priorWeight number 4 Prior precision in pseudo-observations
alpha number 0.10 Confidence width (confidence level = 1/sqrt(alpha))
mode string 'conservative' 'conservative' (upper CI), 'aggressive' (lower CI, clamped ≥ 0.5), or 'mean'

Methods:

Method Description
update(inputTokens, outputTokens) Add an observation pair (ignored when inputTokens <= 0)
estimate(inputTokens) Predict output tokens for given input count
getState() Returns { n, rawN, mean, rawMean, variance, stdErr, confidenceLevel, alpha, priorRatio, priorWeight, mode, ci }
configure(opts) Update priorRatio, priorWeight, alpha, or mode at runtime

Distribution helpers

Pure statistical primitives for the inverse cost-modeling path (absorbed from @polyweave/optimization).

Export Signature Returns
normalPercentiles(mean, stddev) (number, number) Percentile map { p5, p10, p25, p50, p75, p90, p95, p99 } (min 1)
lognormalPercentiles(mean, stddev) (number, number) Same percentile map for a lognormal fit
inferDistributionFromPercentiles(percentiles) Object Fitted distribution parameters
createOnlineStats(options?) Object Stateful online mean/variance accumulator

Errors

  • getModelCost(provider, model) returns null for unknown providers or models — it does not throw.
  • calculateTokenCost(inputTokens, outputTokens, costConfig) returns a zero-cost breakdown when costConfig is missing — it does not throw.
  • The cost tracker emits ERROR { code: 'BUDGET_EXCEEDED' } frames when the budget is exceeded.

Tests

npm test

132 unit tests (npm test) plus 50 property-based tests (npm run test:property) covering cost tracker frame interception, budget enforcement, e-process update/estimate/validation, rolling hash match/commit, pricing table lookups, provider pricing entries, custom pricing overrides, distribution helpers, and token estimation.

Benchmarks

npm run bench

bench/cost-tracking.bench.js — cost tracker overhead, e-process update throughput, and rolling hash throughput.

Requirements

Node.js 22 or later. ESM only.

Peer dependency: @polyweave/core.

Dependencies

Development Dependencies

ID Version
@rigor/core *

Peer Dependencies

ID Version
@polyweave/core *

Keywords

polyweave cost tracking budget token-estimation
Details
npm
2026-07-17 19:53:05 +00:00
430
John Dvorak
SEE LICENSE IN LICENSE
latest
25 KiB
Assets (1)
costs-1.0.7.tgz 25 KiB
Versions (6) View all
1.0.7 2026-07-17
1.0.6 2026-07-11
1.0.3 2026-07-10
1.0.2 2026-07-10
1.0.1 2026-07-10