@polyweave/routing (1.0.8)

Published 2026-07-11 15:31:32 +00:00 by Dvorak

Set up this registry in your project .npmrc file:

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/

To install the package using npm, run the following command:

npm install @polyweave/routing@1.0.8

or add it to the package.json file:

"@polyweave/routing": "1.0.8"

For more information on the npm registry, see the documentation.

@polyweave/routing

Provider routing, service selection, failover, and cost-aware model selection for Polyweave.

Published as @polyweave/routing.

Synopsis

import { createRouter, createServiceSelector, createReliabilityRouter, createModelRouter } from '@polyweave/routing';
import { createCircuitBreakerWrapper, createRetryWrapper } from '@polyweave/core';
import { createOpenAIAdapter } from '@polyweave/openai';
import { createAnthropicAdapter } from '@polyweave/anthropic';

var gpt4o = createCircuitBreakerWrapper(
  createRetryWrapper(createOpenAIAdapter(), { maxAttempts: 3 }),
  { failureThreshold: 5 }
);

var claude = createCircuitBreakerWrapper(
  createRetryWrapper(createAnthropicAdapter(), { maxAttempts: 3 }),
  { failureThreshold: 5 }
);

var router = createModelRouter({
  'gpt-4o': gpt4o,
  'claude-sonnet': claude
}, {
  budget: 5.00,
  alphaCost: 0.10,
  alphaSuccess: 0.10,
  minSuccess: 0.5,
  banditWeight: 0.0,
  priors: { 'gpt-4o': 0.88, 'claude-sonnet': 0.85 }
});

router.source.pipe(mySink);
router.sink.write({ type: 'TEXT', content: { text: 'Hello' } });
router.sink.write({ type: 'REFUSE_DELIVERY', content: { model: 'gpt-4o', requestId: 'req1', code: 'REDFLAG_TRIGGERED' } });

Install

npm install @polyweave/routing

Why

Polyweave applications integrate with dozens of LLM providers — each with different cost structures, reliability profiles, and capabilities. Hard-coding which provider handles which request is brittle: providers fail, costs change, and cheaper models may produce worse outputs. The routing package provides four levels of adapter dispatch, from simple PARAM-based routing to cost-aware bandit model selection with anytime-valid e-processes. All routers follow the duplex pattern — they accept frames on the sink, route them to a selected adapter, and emit responses on the source — so they compose arbitrarily with every other Polyweave wrapper.

Progression of Sophistication

Router	Selection Criteria	Learning	Failure Handling
`createRouter`	PARAM frame `{ key: 'provider' }`	None	None
`createServiceSelector`	Static strategy (cost, round-robin, performance)	Simple error/latency tracking	None
`createReliabilityRouter`	Preference order	None	Circuit-breaker aware fallback
`createModelRouter`	Cost e-process + success bandit	Anytime-valid per-model e-processes	REFUSE_DELIVERY/FEEDBACK gating

API

`createModelRouter(models, options)`

Cost-aware model router. Maintains independent anytime-valid e-processes per model for both cost (output/input token ratio) and success rate. Routes to the cheapest model whose success posterior passes the MAKER viability gate.

var router = createModelRouter({
  'gpt-4o': createOpenAIAdapter(),
  'gpt-4o-mini': createOpenAIAdapter({ defaultModel: 'gpt-4o-mini' }),
  'claude-sonnet': createAnthropicAdapter(),
  'mistral-small': createMistralAdapter()
}, {
  budget: 10.00,
  alphaCost: 0.10,
  alphaSuccess: 0.10,
  minSuccess: 0.5,
  banditWeight: 0.2,
  quorumK: 3,
  priors: { 'gpt-4o': 0.90, 'claude-sonnet': 0.85 },
  priorWeights: { 'gpt-4o': 5, 'claude-sonnet': 3 }
});

Parameter	Type	Required	Default	Description
`models`	`Object`	Yes	—	Map of model name to IFrameDuplex adapter.
`budget`	`number`	No	`0`	Global spend limit in USD. `0` disables. When exceeded, emits `ERROR { code: 'BUDGET_EXCEEDED' }` + `END` on the next TEXT/IMAGE/AUDIO frame.
`alphaCost`	`number`	No	`0.10`	α for cost CI width (clamped to [0.001, 0.5] on PARAM update).
`alphaSuccess`	`number`	No	`0.10`	α for success-rate CI and viability gate (clamped to [0.001, 0.5] on PARAM update).
`minSuccess`	`number`	No	`0.5`	Minimum per-step success probability. Models where `tailProbability(minSuccess) > alphaSuccess` are excluded from viable candidates. Clamped to [0.01, 0.99] on PARAM update.
`banditWeight`	`number`	No	`0.0`	0–1 exploration weight. `0` = pure cost-first (cheapest viable model). `1` = pure UCB exploration (highest upper confidence bound). Blended score: `(1-w) * (1/cost) + w * UCB`.
`quorumK`	`number`	No	`0`	First-to-ahead-by-k quorum margin. `0` disables. Clamped to non-negative integer on PARAM update.
`defaultModel`	`string`	No	First key in `models`	Fallback model name used as initial `currentModel` for PARAM `model` targeting. Note: routing decisions use e-process selection, not this default — this only affects explicit PARAM `model` switching.
`priors`	`Object`	No	`{}`	Per-model prior success rates, e.g. `{ 'gpt-4o': 0.88 }`. Each used as `priorRatio` for the model's success e-process.
`priorWeights`	`Object`	No	`{}`	Per-model prior weights for success e-process (default 3 if not specified). Also supports `'<name>_ratio'` keys for cost e-process prior weights (default 4).
`name`	`string`	No	`'model-router'`	Duplex name for topology identification.

Additional properties on returned duplex:

Property	Type	Description
`getState()`	`function`	Returns `{ models: { <name>: { success: RatioEProcessState, cost: RatioEProcessState } }, budget, totalSpent, banditWeight, quorumK }`.
`reset()`	`function`	Clears request map, resets totalSpent to 0, and recreates all success and cost e-processes with their original priors and weights.

Frame protocol (inbound on sink):

Frame on sink	Effect
`TEXT` / `IMAGE` / `AUDIO`	Route to selected model. Emits `META { selectedModel, pSuccess, estimatedCost }` on source before relay.
`COST_CHECK`	Returns `COST` frame with per-model candidacy breakdown (cost estimate, pSuccess, UCB score, viability tail, viable flag).
`DESCRIBE`	Returns `DESCRIPTION` with per-model state (success mean/SD, cost mean/SD, viability), budget, totalSpent, budgetRemaining, banditWeight, quorumK.
`REFUSE_DELIVERY { model, requestId, code }`	Feed failure (1 trial, 0 successes) to that model's success e-process. Uses `requestMap` to resolve model if `model` is absent. Codes: `MODEL_FAILED`, `REDFLAG_TRIGGERED`, `QUORUM_LOST`, or any string.
`FEEDBACK { model, requestId, success }`	Feed explicit success/failure. `success: true` → 1 trial, 1 success. `success: false` → 1 trial, 0 successes.
`PARAM { key, value }`	Supported keys: `budget`, `model`, `alphaCost`, `alphaSuccess`, `banditWeight`, `quorumK`, `minSuccess`, `prior_<model>` (sets that model's success prior). With `target`, forwards PARAM to that model's cost tracker.
`RESET`	Clears request map, resets totalSpent to 0, recreates all e-processes with original priors.

Frames emitted on source:

Frame	When
`META { selectedModel, pSuccess, estimatedCost }`	On routing decision for each TEXT/IMAGE/AUDIO frame.
`COST { selectedModel, candidates[], noViableModel, banditWeight }`	Response to COST_CHECK.
`DESCRIPTION { resource, candidates[], budget, totalSpent, budgetRemaining, banditWeight, quorumK }`	Response to DESCRIBE.
`ERROR { code: 'NO_VIABLE_MODEL', message }`	No model passes viability gate (tail probability exceeds alpha).
`ERROR { code: 'BUDGET_EXCEEDED', budget, totalSpent }`	Router budget exhausted on a TEXT/IMAGE/AUDIO request. Followed by END.

`createRouter(adapters, options)`

PARAM-based static router. Switches adapters via PARAM { key: provider, value: 'openai' }.

Parameter	Type	Required	Default	Description
`adapters`	`Object`	Yes	—	Map of adapter name to IFrameDuplex.
`paramKey`	`string`	No	`'provider'`	PARAM key that triggers a route switch. When a PARAM frame arrives with this key, the value selects the new active adapter. Non-matching keys are ignored silently.
`defaultAdapter`	`string`	No	First key in `adapters`	Initial active adapter name.

Aliases: createRouterWrapper and createRouterTransform (exported from index.js) are both identical to createRouter.

If the PARAM value does not match any key in adapters, the switch is silently ignored (current adapter unchanged). If the current adapter does not exist in adapters when a TEXT/IMAGE/AUDIO frame arrives, the router emits ERROR { code: 'NO_ADAPTER' } followed by END.

`createServiceSelector(adapters, options)`

Static strategy-based selector. Routes by cost, performance, round-robin, or random. Throws at construction if adapters is falsy or empty: Error('Service selector requires at least one adapter').

Parameter	Type	Required	Default	Description
`adapters`	`Array`	Yes	—	Array of IFrameDuplex adapters. Must be non-empty.
`strategy`	`string`	No	`'round-robin'`	Selection strategy: `'cost'`, `'performance'`, `'round-robin'`, or `'random'`.
`selector`	`function`	No	—	Custom selector: `(frame, adapters, adapterStats)` → adapter. Overrides `strategy` when provided.
`costConfig`	`Map`	No	`new Map()`	Per-adapter cost configuration for `'cost'` strategy. Maps adapter instance to `{ mtoksInput, mtoksOutput }`.

The 'performance' strategy computes a score as errorRate * 1000 + avgLatency and selects the adapter with the lowest score. The 'cost' strategy estimates input cost as ceil(text.length / 4) / 1e6 * mtoksInput and selects the cheapest. All strategies track per-adapter stats: requestCount, errorCount, totalLatency, lastUsed, totalCost.

Returns: { source: IFrameTransform, sink: IFrameTransform, ended, adapters, stats: Map<adapter, AdapterStats> }.

`createReliabilityRouter(providers, options)`

Fallback router. Tries providers in preference order. Falls back on CIRCUIT_OPEN errors.

Parameter	Type	Required	Default	Description
`providers`	`Object`	Yes	—	Map of provider name to IFrameDuplex.
`fallbackOrder`	`Array`	No	`Object.keys(providers)`	Ordered array of provider names — tried in this sequence when the preferred provider fails.
`logPrefix`	`string`	No	`'[ReliabilityRouter]'`	Prefix for console.log messages about routing and fallback decisions.

Initial provider selection: Uses frame.metadata.provider if present, otherwise uses the first entry in fallbackOrder.

Fallback behavior: When a provider responds with ERROR { code: 'CIRCUIT_OPEN' }, the router rewrites the original frame and retries the next untried provider in fallbackOrder. Non-circuit ERROR frames (and TEXT/END frames) are forwarded to the source immediately and clean up the request tracker.

All-providers-exhausted: When all providers in fallbackOrder have been tried for a request, emits ERROR { code: 'ALL_PROVIDERS_FAILED', retryable: false, triedProviders: [...] }.

The reliability router implements a complete duplex interface (source + sink) with TOPOLOGY_SCAN, TOPOLOGY_REPORT, TEST_INJECT, and TEST_SNAPSHOT protocol support.

Error Conditions Summary

Router	Condition	Behavior
`createRouter`	Active adapter not found in `adapters` when TEXT/IMAGE/AUDIO arrives	Emits `ERROR { code: 'NO_ADAPTER' }` + `END`
`createServiceSelector`	`adapters` is falsy or empty at construction	Throws `Error('Service selector requires at least one adapter')`
`createServiceSelector`	Selected adapter has no `sink` at request time	Emits `ERROR { code: 'SERVICE_UNAVAILABLE' }`
`createReliabilityRouter`	All providers tried and all failed/returned CIRCUIT_OPEN	Emits `ERROR { code: 'ALL_PROVIDERS_FAILED', retryable: false, triedProviders }`
`createModelRouter`	Budget exhausted (`totalSpent >= budget > 0`) on TEXT/IMAGE/AUDIO	Emits `ERROR { code: 'BUDGET_EXCEEDED' }` + `END`
`createModelRouter`	No model passes viability gate (`tailProbability(minSuccess) <= alphaSuccess`)	Emits `ERROR { code: 'NO_VIABLE_MODEL' }` + `END`
`createModelRouter`	PARAM `model` target's cost tracker missing	Frame returned as-is (no-op)

Edge Cases

First-request behavior (createModelRouter): Before any REFUSE_DELIVERY/FEEDBACK frames have been received, all models have n = 0 and are considered viable (prior dominates). The first routing decision selects the cheapest model among all registered models.
Zero-viable-model state: If all models fail below the viability threshold, every subsequent TEXT/IMAGE/AUDIO frame emits NO_VIABLE_MODEL. Send RESET to restore priors or PARAM { key: 'prior_<model>', value: <higherRate> } to adjust individual models.
Bandit-weight extremes: banditWeight: 0 sorts viable models by cost only (greedy). banditWeight: 1 sorts by UCB score only (exploratory). Intermediate values blend both, weighted so cost dominates at low values and exploration dominates at high values.
PARAM routing with PARAM targeting: In createModelRouter, PARAM frames with a target field are forwarded to that model's cost tracker (not to the adapter). PARAM frames without target update router-level settings.
createRouter PARAM switching: If PARAM { key: paramKey } arrives with a value not in adapters, the current adapter is unchanged — no error is emitted.
createServiceSelector END frames: END frames go through selectAdapter() using the configured strategy (not a fixed pass-through), so they use the strategy-selected adapter.

Composition

All routers are duplexes — they compose with any wrapper. The recommended stack from outer to inner:

createModelRouter(           // selects cheapest viable model
  createServiceSelector([    // optional per-provider routing
    createCircuitBreakerWrapper(
      createRetryWrapper(
        createTimeoutWrapper(createOpenAIAdapter(), { timeout: 30000 }),
        { maxAttempts: 3 }
      ),
      { failureThreshold: 5 }
    ),
    // ... more providers
  ])
)

The model router wraps individual adapters. The reliability layers (circuit breaker, retry, timeout) wrap the adapter BEFORE the model router sees it. This means:

Circuit breaker opens → adapter returns ERROR → can still route to another model
Retry exhausts → adapter returns ERROR → can still route to another model
REFUSE_DELIVERY from downstream (redflag/quorum/judge) → feeds success e-process → adapts routing

Examples

Cost-aware routing with bandit exploration

import { createModelRouter } from '@polyweave/routing';
import { createCostTracker } from '@polyweave/core';
import { createFrameSink } from '@polyweave/core';

var router = createModelRouter({
  'gpt-4o': gpt4oAdapter,
  'gpt-4o-mini': miniAdapter
}, { budget: 2.00, banditWeight: 0.3 });

router.source.pipe(createFrameSink(function(frame) {
  if (frame.type === 'META') console.log('Routed to:', frame.content.selectedModel);
  if (frame.type === 'TEXT') console.log('Response:', frame.content.text);
}));

router.sink.write({ type: 'TEXT', content: { text: 'Summarize this document' } });

Quorum with redflag feedback

router.sink.write({ type: 'PARAM', content: { key: 'quorumK', value: '3' } });

// Quorum runs 3 calls to the selected model, votes on the response
// If redflag fires on the winner:
router.sink.write({ type: 'REFUSE_DELIVERY', content: {
  model: 'gpt-4o', requestId: 'req1', code: 'REDFLAG_TRIGGERED'
}});
// Router lowers that model's estimated success rate
// Next request may route to a different model

Prior injection from historical data

router.sink.write({ type: 'PARAM', content: { key: 'prior_gpt-4o', value: '0.92' } });
router.sink.write({ type: 'PARAM', content: { key: 'prior_mistral-small', value: '0.65' } });
// We have high confidence in gpt-4o from history but less in mistral
// Router will prefer gpt-4o until mistral proves itself via REFUSE_DELIVERY/FEEDBACK

Fallback routing with circuit breaker

import { createReliabilityRouter } from '@polyweave/routing';

var fallback = createReliabilityRouter({
  primary: primaryAdapter,
  backup: backupAdapter,
  emergency: emergencyAdapter
}, {
  fallbackOrder: ['primary', 'backup', 'emergency'],
  logPrefix: '[MyRouter]'
});

fallback.source.pipe(mySink);

// Primary gets the frames. If circuit opens → backup. If backup opens → emergency.
// If all fail → ALL_PROVIDERS_FAILED error.
fallback.sink.write({ type: 'TEXT', content: { text: 'Hello' } });

Tests

npm test

Runs node --test test/*.test.js. Covers: routing decisions by PARAM key, REFUSE_DELIVERY and FEEDBACK frame handling, MAKER viability gate (p < 0.5 exclusion), UCB bandit exploration, COST_CHECK per-model estimates, DESCRIBE state query, budget enforcement (BUDGET_EXCEEDED emission), prior injection via PARAM prior_<model>, RESET behavior, circuit-breaker fallback (CIRCUIT_OPEN → retry, ALL_PROVIDERS_FAILED), topology scan, foreign frame routing, nested router composition, property-based cost-aware router tests, and service selector statistics tracking. 9 test files.

Benchmarks

npm run bench

Provider selection throughput and circuit breaker performance benchmarks.

Requirements

Node.js 22 or later. ESM only.

Peer dependencies: @polyweave/core, @polyweave/costs.

Caveats

The model router's success e-process learns from REFUSE_DELIVERY and FEEDBACK frames only — if neither is sent, per-model success estimates remain at their priors indefinitely.
With banditWeight: 1.0 (pure UCB), the router may select untested models with wide confidence intervals. Use moderate values (0.1–0.3) for gentle exploration.
The MAKER viability gate at minSuccess: 0.5 is theoretically sound (Eq. 9 from Meyerson et al. 2025): below p=0.5, quorum actively converges on the wrong answer. Do not lower this below 0.5 when using quorum.
The reliability router's logPrefix default '[ReliabilityRouter]' emits console.log at each routing attempt. Set to empty string or null to suppress.
createServiceSelector with the 'cost' strategy estimates input cost from frame.content.text.length / 4 (rough token estimate) — this is an approximation, not actual tokenization.