Polyweave
Repositories
1
 Projects Packages

@polyweave/routing (1.0.3)

Published 2026-07-10 19:10:07 +00:00 by Dvorak

Installation

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/
npm install @polyweave/routing@1.0.3
"@polyweave/routing": "1.0.3"

About this package

routing

Provider routing, service selection, failover, and cost-aware model selection for Polyweave.

Published as @polyweave/routing.

Synopsis

import { createRouter, createServiceSelector, createReliabilityRouter, createModelRouter } from '@polyweave/routing';
import { createCircuitBreakerWrapper, createRetryWrapper } from '@polyweave/core';
import { createOpenAIAdapter } from '@polyweave/openai';
import { createAnthropicAdapter } from '@polyweave/anthropic';

var gpt4o = createCircuitBreakerWrapper(
  createRetryWrapper(createOpenAIAdapter(), { maxAttempts: 3 }),
  { failureThreshold: 5 }
);

var claude = createCircuitBreakerWrapper(
  createRetryWrapper(createAnthropicAdapter(), { maxAttempts: 3 }),
  { failureThreshold: 5 }
);

var router = createModelRouter({
  'gpt-4o': gpt4o,
  'claude-sonnet': claude
}, {
  budget: 5.00,
  alphaCost: 0.10,
  alphaSuccess: 0.10,
  minSuccess: 0.5,
  banditWeight: 0.0,
  priors: { 'gpt-4o': 0.88, 'claude-sonnet': 0.85 }
});

router.source.pipe(mySink);
router.sink.write({ type: 'TEXT', content: { text: 'Hello' } });
router.sink.write({ type: 'REFUSE_DELIVERY', content: { model: 'gpt-4o', requestId: 'req1', code: 'REDFLAG_TRIGGERED' } });

Install

npm install @polyweave/routing

Description

The routing package provides four levels of adapter dispatch, from simple PARAM-based routing to cost-aware bandit model selection. All routers follow the duplex pattern — they accept frames on the sink, route them to a selected adapter, and emit responses on the source.

Progression of sophistication:

Router Selection Criteria Learning Failure Handling
createRouter PARAM frame { key: 'provider' } None None
createServiceSelector Static strategy (cost, round-robin, performance) Simple error/latency tracking None
createReliabilityRouter Preference order None Circuit-breaker aware fallback
createModelRouter Cost e-process + success bandit Anytime-valid per-model e-processes REFUSE_DELIVERY/FEEDBACK gating

API

createModelRouter(models, options)

Cost-aware model router. Maintains independent anytime-valid e-processes per model for both cost (output/input token ratio) and success rate. Routes to the cheapest model whose success posterior passes the MAKER viability gate.

var router = createModelRouter({
  'gpt-4o': createOpenAIAdapter(),
  'gpt-4o-mini': createOpenAIAdapter({ defaultModel: 'gpt-4o-mini' }),
  'claude-sonnet': createAnthropicAdapter(),
  'mistral-small': createMistralAdapter()
}, {
  budget: 10.00,
  alphaCost: 0.10,
  alphaSuccess: 0.10,
  minSuccess: 0.5,
  banditWeight: 0.2,
  quorumK: 3,
  priors: { 'gpt-4o': 0.90, 'claude-sonnet': 0.85 },
  priorWeights: { 'gpt-4o': 5, 'claude-sonnet': 3 }
});
Parameter Type Default Description
models Object required Map of model name to duplex adapter
budget number 0 Global spend limit in USD. 0 disables.
alphaCost number 0.10 α for cost CI width
alphaSuccess number 0.10 α for success-rate CI and viability gate
minSuccess number 0.5 Minimum per-step success probability (MAKER gate — models below 0.5 are excluded since quorum converges on wrong answer)
banditWeight number 0.0 0–1. 0 = pure cost-first, 1 = pure UCB exploration. Values between blend both.
quorumK number 0 First-to-ahead-by-k quorum margin. 0 disables.
defaultModel string first key Fallback model name
priors Object {} Per-model prior success rates, e.g. { 'gpt-4o': 0.88 }
priorWeights Object {} Per-model prior weights for success e-process

Frame protocol:

Frame on sink Effect
TEXT / IMAGE / AUDIO Route to selected model, emit META { selectedModel, pSuccess, estimatedCost } on source
COST_CHECK Return COST frame with per-model candidacy breakdown
DESCRIBE Return DESCRIPTION with per-model state, budget, totals
REFUSE_DELIVERY { model, requestId, code } Feed failure to that model's success e-process. Codes: MODEL_FAILED, REDFLAG_TRIGGERED, QUORUM_LOST, or any string.
FEEDBACK { model, requestId, success } Feed explicit success/failure. success: true = successful step.
PARAM { budget } Set global budget. 0 disables.
PARAM { banditWeight } 0–1 UCB exploration weight
PARAM { quorumK } Quorum margin
PARAM { prior_<model>: value } Set per-model prior, e.g. prior_gpt-4o: 0.92
RESET Clear all per-model accumulators

Frames emitted on source:

Frame When
META { selectedModel, pSuccess, estimatedCost } On routing decision
COST { candidates, selectedModel } On COST_CHECK
DESCRIPTION { resource: 'model-router', candidates, budget } On DESCRIBE
ERROR { code: 'NO_VIABLE_MODEL' } No model passes viability gate
ERROR { code: 'BUDGET_EXCEEDED' } Router budget exhausted

createRouter(adapters, options)

PARAM-based static router. Switches adapters via PARAM { key: provider, value: 'openai' }.

Parameter Type Default Description
adapters Object required Map of adapter name to duplex
paramKey string 'provider' Which PARAM key triggers a route
defaultAdapter string first key Default adapter name

createServiceSelector(adapters, options)

Static strategy-based selector. Routes by cost, performance, round-robin, or random.

Parameter Type Default Description
adapters Array required Array of duplex adapters
strategy string 'round-robin' 'cost', 'performance', 'round-robin', 'random' or a custom selector(frame, adapters, stats) function
costConfig Map new Map() Per-adapter cost configuration for 'cost' strategy

createReliabilityRouter(providers, options)

Fallback router. Tries providers in preference order. Falls back on CIRCUIT_OPEN errors.

Parameter Type Default Description
providers Object required Map of provider name to duplex
fallbackOrder Array Object.keys(providers) Preference order for fallback

Composition

All routers are duplexes — they compose with any wrapper. The recommended stack from outer to inner:

createModelRouter(           // selects cheapest viable model
  createServiceSelector([    // optional per-provider routing
    createCircuitBreakerWrapper(
      createRetryWrapper(
        createTimeoutWrapper(createOpenAIAdapter(), { timeout: 30000 }),
        { maxAttempts: 3 }
      ),
      { failureThreshold: 5 }
    ),
    // ... more providers
  ])
)

The model router wraps individual adapters. The reliability layers (circuit breaker, retry, timeout) wrap the adapter BEFORE the model router sees it. This means:

  • Circuit breaker opens → adapter returns ERROR → can still route to another model
  • Retry exhausts → adapter returns ERROR → can still route to another model
  • REFUSE_DELIVERY from downstream (redflag/quorum/judge) → feeds success e-process → adapts routing

Examples

Cost-aware routing with bandit exploration

import { createModelRouter } from '@polyweave/routing';
import { createCostTracker } from '@polyweave/core';
import { createFrameSink } from '@polyweave/core';

var router = createModelRouter({
  'gpt-4o': gpt4oAdapter,
  'gpt-4o-mini': miniAdapter
}, { budget: 2.00, banditWeight: 0.3 });

router.source.pipe(createFrameSink(function(frame) {
  if (frame.type === 'META') console.log('Routed to:', frame.content.selectedModel);
  if (frame.type === 'TEXT') console.log('Response:', frame.content.text);
}));

router.sink.write({ type: 'TEXT', content: { text: 'Summarize this document' } });

Quorum with redflag feedback

router.sink.write({ type: 'PARAM', content: { key: 'quorumK', value: '3' } });

// Quorum runs 3 calls to the selected model, votes on the response
// If redflag fires on the winner:
router.sink.write({ type: 'REFUSE_DELIVERY', content: {
  model: 'gpt-4o', requestId: 'req1', code: 'REDFLAG_TRIGGERED'
}});
// Router lowers that model's estimated success rate
// Next request may route to a different model

Prior injection from historical data

router.sink.write({ type: 'PARAM', content: { key: 'prior_gpt-4o', value: '0.92' } });
router.sink.write({ type: 'PARAM', content: { key: 'prior_mistral-small', value: '0.65' } });
// We have high confidence in gpt-4o from history but less in mistral
// Router will prefer gpt-4o until mistral proves itself via REFUSE_DELIVERY/FEEDBACK

Tests

npm test

Tests cover: routing decisions, REFUSE_DELIVERY and FEEDBACK frame handling, MAKER viability gate (p < 0.5 exclusion), UCB bandit exploration, COST_CHECK per-model estimates, DESCRIBE state query, budget enforcement, prior injection via PARAM, and RESET. 25 tests across 3 test files.

Benchmarks

node benchmark/routing-bench.js

Provider selection throughput and circuit breaker performance benchmarks.

Requirements

Node.js 18 or later. ESM only.

Peer dependencies: @polyweave/core.

Caveats

  • The model router's success e-process learns from REFUSE_DELIVERY and FEEDBACK frames only — if neither is sent, the per-model success prior is never updated.
  • With banditWeight: 1.0 (pure UCB), the router may select untested models with wide confidence intervals. Use moderate values (0.1–0.3) for gentle exploration.
  • The MAKER viability gate at minSuccess: 0.5 is theoretically sound (Eq. 9 from Meyerson et al. 2025): below p=0.5, quorum actively converges on the wrong answer. Do not lower this below 0.5 when using quorum.

See Also

  • @polyweave/core — frame types, wrappers, cost tracker, e-processes
  • @polyweave/quorum — first-to-ahead-by-k voting
  • @polyweave/redflag — output quality gating
  • @polyweave/bt-harness — behavior tree execution with evaluator feedback

License

MIT

Dependencies

Development Dependencies

ID Version
@rigor/core *

Peer Dependencies

ID Version
@polyweave/core *
@polyweave/costs *

Keywords

polyweave routing failover selection
Details
npm
2026-07-10 19:10:07 +00:00
388
John Dvorak
SEE LICENSE IN LICENSE
12 KiB
Assets (1)
routing-1.0.3.tgz 12 KiB
Versions (6) View all
1.0.7 2026-07-20
1.0.8 2026-07-11
1.0.5 2026-07-11
1.0.4 2026-07-11
1.0.3 2026-07-10