@polyweave/grpo (1.0.3)

Published 2026-07-11 00:52:08 +00:00 by Dvorak

Set up this registry in your project .npmrc file:

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/

To install the package using npm, run the following command:

npm install @polyweave/grpo@1.0.3

or add it to the package.json file:

"@polyweave/grpo": "1.0.3"

For more information on the npm registry, see the documentation.

grpo

Training-Free GRPO loop as a bt-harness duplex tool. Group-based rollout optimization with experience library accumulation.

Published as @polyweave/grpo.

Synopsis

import { createGrpoTool } from '@polyweave/grpo';

const tool = createGrpoTool({
  apiKey: process.env.API_KEY,
  model: 'deepseek-v4-flash',
  dataset: trainingItems,
  groupSize: 4,
  epochs: 3,
  initialExperiences: [{ item: 'example', output: 'result', score: 0.8 }]
});

tool.sink.write({ type: 'TOOL_CALL', content: {} });
tool.source.pipe(mySink);

Install

npm install @polyweave/grpo

Description

Training-Free GRPO (Group Relative Policy Optimization) implements group-based rollout optimization as a bt-harness duplex tool. It processes a dataset over multiple epochs, generating multiple candidate outputs per item (the "group"), scoring them, and accumulating the best outputs into an experience library for future reference.

The behavior tree spec (GRPO_SPEC) orchestrates: generate candidates → score them via a scorer → promote the best candidate to the experience library. The experience library grows across epochs, enabling knowledge accumulation without gradient-based training. Parameters control group size (how many candidates per item) and number of epochs (full passes over the dataset).

API

`createGrpoTool(options)`

Creates a Polyweave frame duplex tool for GRPO training.

Parameters:

options.apiKey — API key for LLM calls.
options.model — Model name (default 'deepseek-v4-flash').
options.dataset — Array of items to process.
options.groupSize — Number of candidates generated per item (default 4).
options.epochs — Number of full passes over the dataset (default 1).
options.initialExperiences — Array of initial experience entries [{item, output, score}].
options.createAdapter — Custom adapter factory.
options.adapterConfig — Adapter factory configuration.
options.debug — Enable debug output.

TOOL_CALL args: { dataset, groupSize, epochs } — Can override constructor options.

Returns: { source, sink, _toolName: 'grpo', close }

Tool calls exposed to agents:

grpo_read(sectionId) — Read any document section.
grpo_generate(item, experiences) — Generate a candidate for an item using experience library.
grpo_score(item, candidate) — Score a generated candidate.
grpo_promote(candidate, score) — Promote a candidate to the experience library.

Events on source:

TOOL_RESULT — Final result with { experienceCount, itemsProcessed, epochs }.

`GRPO_SPEC`

The behavior tree specification string. Defines the generate → score → promote tree with group rollout across items and epochs.

Examples

// Multi-epoch training with custom group size
const tool = createGrpoTool({
  apiKey: process.env.API_KEY,
  dataset: [
    { task: 'Sort array', expected: 'sorted array' },
    { task: 'Find max', expected: 'maximum value' }
  ],
  groupSize: 8,
  epochs: 5
});

tool.sink.write({ type: 'TOOL_CALL', content: { epochs: 10, groupSize: 6 } });

// With initial experiences from a previous run
const tool = createGrpoTool({
  apiKey: process.env.API_KEY,
  dataset: newTasks,
  initialExperiences: previousRunExperiences,
  epochs: 2,
  groupSize: 4
});

tool.close();

Tests

npm test

Tests cover tool creation with various option sets, group rollout with candidate scoring, experience library accumulation across epochs, initial experiences seeding, TOOL_CALL argument overrides, and CANCEL/ABORT handling.

Requirements

Node.js 18 or later. ESM only.

Caveats

Requires a valid apiKey or createAdapter for LLM model access.
The experience library is stored in an in-memory document store and grows with each promoted candidate. Consider bounding or periodic cleanup for long-running training with large datasets.
GRPO is training-free in the sense that no gradient updates occur; improvement comes from accumulating and referencing past successful outputs.

License

MIT

Dependencies

ID	Version
@polyweave/bt-harness	*
@polyweave/loop-utils	*
@polyweave/state	*

Development Dependencies

ID	Version
@rigor/core	*

Peer Dependencies

ID	Version
@polyweave/core	*

polyweave grpo training-free optimization experience behavior-tree

Details

npm

2026-07-11 00:52:08 +00:00

John Dvorak

SEE LICENSE IN LICENSE

5.4 KiB

Assets (1)

grpo-1.0.3.tgz 5.4 KiB

Versions (3) View all

1.0.6

2026-07-11

1.0.3

2026-07-11

1.0.1

2026-07-10