Polyweave
Repositories
1
 Projects Packages

@polyweave/grpo (1.0.6)

Published 2026-07-11 15:26:20 +00:00 by Dvorak

Installation

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/
npm install @polyweave/grpo@1.0.6
"@polyweave/grpo": "1.0.6"

About this package

@polyweave/grpo

Training-Free GRPO (Group Relative Policy Optimization) loop as a bt-harness duplex tool. Group-based rollout optimization with experience library accumulation — no gradient training required.

Published as @polyweave/grpo.

Synopsis

import { createGrpoTool, GRPO_SPEC } from '@polyweave/grpo';

const tool = createGrpoTool({
  apiKey: process.env.API_KEY,
  model: 'deepseek-v4-flash',
  dataset: [
    { task: 'Sort array', expected: 'sorted array' },
    { task: 'Find max', expected: 'maximum value' }
  ],
  groupSize: 4,
  epochs: 3,
  initialExperiences: [{ item: 'example', output: 'result', score: 0.8 }]
});

tool.source.pipe({
  write(frame) {
    if (frame.type === 'TOOL_RESULT') {
      console.log('Experiences:', frame.content.experiences);
      console.log('Epochs completed:', frame.content.epochs);
    }
    if (frame.type === 'ERROR') console.error(frame.content.message);
  }
});

// Start training
tool.sink.write({ type: 'TOOL_CALL', content: {} });

// Override dataset/groupSize/epochs per run
tool.sink.write({ type: 'TOOL_CALL', content: { epochs: 10, groupSize: 8 } });

// Retrieve current experience library
console.log(tool.getExperiences());

// Close the underlying document store
tool.close();

Install

npm install @polyweave/grpo

Why

Training-Free GRPO implements group-based rollout optimization as a bt-harness duplex tool. It processes a dataset over multiple epochs, generating multiple candidate outputs per item (the "group"), scoring them, computing group advantage, and accumulating the best outputs into an experience library for future reference. The behavior tree spec orchestrates: generate candidates via grpo_rollout → score them via grpo_score → store results → summarize → compute advantage → update experience library. The experience library grows across epochs, enabling knowledge accumulation without gradient-based training. Results are always deterministic given the same LLM outputs — no non-determinism from the tool itself.

API

createGrpoTool(options)

Creates a Polyweave frame duplex tool for GRPO training.

Parameter Type Required Default Description
options.apiKey string No undefined API key for LLM calls (passed to bt-harness)
options.model string No 'deepseek-v4-flash' Model name for both agent and evaluator
options.dataset Array No [] Array of items to process (strings or objects)
options.groupSize number No 4 Number of candidate outputs generated per dataset item
options.epochs number No 1 Number of full passes over the dataset
options.initialExperiences Array No [] Initial experience entries [{ item, output, score }] — normalized via normalizeExperienceLibrary
options.createAdapter function No undefined Custom adapter factory (passed to bt-harness)
options.adapterConfig object No undefined Adapter factory configuration
options.debug — also options._debug boolean No false Enable debug output in bt-harness

TOOL_CALL args (override constructor options):

Arg Type Description
dataset Array Replace the dataset inline
groupSize number Override group size
epochs number Override epoch count

Return value: { source, sink, _toolName: 'grpo', getExperiences, close }

  • source — Pushable output stream
  • sink — Writable input accepting TICK, CANCEL, ABORT, DESCRIBE, TOOL_CALL
  • _toolName — 'grpo'
  • getExperiences() — Returns the current experience library as parsed JSON array
  • close() — Closes the underlying createDocumentStore()

Concurrency guard: If running is true, a second TOOL_CALL emits { type: 'ERROR', content: { message: 'Already running' } } and is rejected.

Internal state: Uses createDocumentStore() for document storage. Dataset items stored at grpo/item/{i}. Experience library at grpo/experiences. Rollout results at grpo/rollout/{i}. Scores at grpo/score/{i}. Summaries at grpo/summary/{i}. Advantage at grpo/advantage.

Frame Protocol

Sink (input frames):

Frame Type Behavior
TOOL_CALL Starts the GRPO training loop via handleToolCall. Overrides dataset/groupSize/epochs if provided in content. Creates a createBtHarness instance with the GRPO behavior tree spec and 7 tool definitions.
TICK Forwards TICK to the current bt-harness if running
CANCEL Calls currentHarness.cancel(), sets running = false, clears currentHarness
ABORT Calls currentHarness.abort(), sets running = false, clears currentHarness
DESCRIBE Emits DESCRIPTION with tool schema: { name: 'grpo', capabilities: ['write', 'execute'], schema: { properties: { dataset, groupSize, epochs } } }

Source (output frames):

Frame Type Emitted When Content
TOOL_RESULT bt-harness emits bt:tree_complete { experiences: Array, epochs: number (completed), itemsProcessed: number }
TOOL_RESULT bt-harness emits bt:tree_failed { experiences: Array, failed: true }
ERROR Already running { message: 'Already running' }
ERROR Invalid harness spec { message: harness.errors.join('; ') }
ERROR bt-harness emits bt:error { message: frame.content.message || 'grpo error' }
DESCRIPTION After DESCRIBE Tool metadata with schema

Behavior Tree Specification (GRPO_SPEC)

The GRPO_SPEC is a bt-harness spec string defining:

PARAMETERS: groupSize : number = 4
SYMBOLS: @epoch, @item_index, @total_items, @max_epochs
POSTCONDITION: when @epoch > @max_epochs

ACTIONS:
  rollout — Read item + experiences, generate groupSize outputs, score each, store
  summarize — Read rollouts, summarize each output
  advantage — Compute group advantage: critique + new experiences proposal
  update — Apply operations to experience library, increment @item_index
  advance_epoch — When @item_index >= @total_items, advance @epoch, reset @item_index

TREE: selector {
  sequence { condition more_items, action rollout, action summarize, action advantage, action update }
  sequence { action advance_epoch }
}

Internal tool calls (7 tools registered with bt-harness):

Tool Name Arguments Writes Responds
grpo_read { sectionId } — { sectionId, text }
grpo_rollout { index, output } grpo/rollout/{index} { index }
grpo_score { index, score, feedback } grpo/score/{index} as JSON { score, feedback } { index, score }
grpo_store_results {} — { accepted: true }
grpo_summarize { index, summary } grpo/summary/{index} { index }
grpo_advantage { critique, experiences } grpo/advantage as JSON { critique, experiences } { critique }
grpo_update { operations } Updates experience library via applyExperienceOperations { count }

Each internal tool ignores TICK, CANCEL, and ABORT frames. DESCRIBE returns a DESCRIPTION with tool metadata.

GRPO_SPEC

The behavior tree specification string (see above). Exported for use with custom bt-harness setups.

Tests

npm test

Runs node --test test/rigor.test.js test/grpo.test.js. Covers: tool creation with various option sets, group rollout with candidate scoring, experience library accumulation across epochs, initial experiences seeding and normalization, TOOL_CALL argument overrides, CANCEL/ABORT lifecycle handling, concurrency guard (double TOOL_CALL), and bt-harness spec parsing.

Requirements

Node.js 22 or later. ESM only.

Peer dependencies: @polyweave/core.

Dependencies: @polyweave/bt-harness, @polyweave/state, @polyweave/loop-utils.

Dependencies

Dependencies

ID Version
@polyweave/bt-harness *
@polyweave/loop-utils *
@polyweave/state *

Development Dependencies

ID Version
@rigor/core *

Peer Dependencies

ID Version
@polyweave/core *
@polyweave/errors *

Keywords

polyweave grpo training-free optimization experience behavior-tree
Details
npm
2026-07-11 15:26:20 +00:00
68
John Dvorak
SEE LICENSE IN LICENSE
latest
6.5 KiB
Assets (1)
grpo-1.0.6.tgz 6.5 KiB
Versions (3) View all
1.0.6 2026-07-11
1.0.3 2026-07-11
1.0.1 2026-07-10