Polyweave
Repositories
1
 Projects Packages

@polyweave/raptor (1.0.2)

Published 2026-07-10 23:34:24 +00:00 by Dvorak

Installation

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/
npm install @polyweave/raptor@1.0.2
"@polyweave/raptor": "1.0.2"

About this package

raptor

RAPTOR hierarchy indexing and retrieval on Polyweave state stores.

Published as @polyweave/raptor.

Synopsis

import {
  createRaptorIndex,
  createRaptorRetriever,
  buildRaptorHierarchy
} from '@polyweave/raptor';

const index = createRaptorIndex({
  summarizer: mySummarizerAdapter,
  textStore: myTextStore,
  treeStore: myTreeStore
});

index.indexDocuments(documents, (err, tree) => {
  const retriever = createRaptorRetriever({ index: tree });
  retriever.search('query text', 5, (err, results) => {
    console.log(results);
  });
});

Install

npm install @polyweave/raptor

Description

RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval) builds hierarchical document indexes on Polyweave state stores. It recursively clusters and summarizes document chunks into a tree, enabling both coarse-to-fine retrieval and abstractive traversal. Leaf nodes hold original document chunks; internal nodes hold LLM-generated summaries of their children.

The package provides three main components: the raptor indexer (createRaptorIndex) for building hierarchical trees from documents, the raptor retriever (createRaptorRetriever) for searching across tree levels, and the summary refiner BT (createSummaryRefinerBt) for iterative summary quality improvement using bt-harness feedback loops.

The retriever supports multi-level traversal strategies (top-down, bottom-up, breadth-first) and relevance scoring using configurable similarity functions (cosine, text overlap, HHEM-based decision). The indexer uses the @polyweave/uncertainty package for cosine similarity computation and HHEM (Hallucination Hallucination Evaluation Model) segmentation decisions.

API

createRaptorIndex(options)

Creates a RAPTOR hierarchical document indexer.

Parameters:

  • options.summarizer — LLM adapter (duplex or callback (group, context, callback)) for generating node summaries. Defaults to truncation-based summarizer.
  • options.textStore — Text store for indexing.
  • options.treeStore — Tree store for hierarchy structure.
  • options.chunkSize — Maximum number of documents per leaf cluster (default 4).
  • options.maxSummaryLength — Maximum summary character length (default 320).
  • options.similarityFn — Function (a, b) => number for clustering similarity.
  • options.embedder — Embedding adapter for cosine similarity computation.

Returns: { indexDocuments(docs, callback), getTree(), close }

createRaptorRetriever(options)

Creates a RAPTOR tree retriever.

Parameters:

  • options.index — RAPTOR tree from createRaptorIndex.
  • options.topK — Number of results to return (default 5).
  • options.strategy — 'top-down', 'bottom-up', or 'breadth' (default 'top-down').
  • options.similarityFn — Function for relevance scoring.
  • options.embedder — Embedding adapter.

Returns: { search(query, k, callback), searchSync(query, k) }

createSummaryRefinerBt(options)

Creates a behavior tree harness for iteratively refining summaries.

Parameters:

  • options.apiKey — API key for LLM calls.
  • options.model — Model name.
  • options.createAdapter — Custom adapter factory.

Returns: bt-harness-compatible tool for summary refinement.

buildRaptorHierarchy(documents, options)

Builds a complete RAPTOR tree hierarchy from documents. Convenience function that creates an index, indexes documents, and returns the tree structure.

Parameters:

  • documents — Array of { text, id?, metadata? } document objects.
  • options — Same options as createRaptorIndex.

Returns: Tree structure object.

Examples

// Build and search a RAPTOR index
const docs = [
  { id: '1', text: 'Machine learning is a subset of artificial intelligence...' },
  { id: '2', text: 'Deep learning uses neural networks with multiple layers...' }
];

const index = createRaptorIndex({ textStore, treeStore });
index.indexDocuments(docs, (err, tree) => {
  const retriever = createRaptorRetriever({
    index: tree,
    strategy: 'top-down',
    topK: 3
  });

  retriever.search('neural network architecture', 3, (err, results) => {
    results.forEach(r => console.log(r.text, r.score));
  });
});

// With custom summarizer and similarity
const index = createRaptorIndex({
  textStore,
  treeStore,
  summarizer: myLlmAdapter,
  similarityFn: (a, b) => customSimilarity(a, b),
  chunkSize: 6,
  maxSummaryLength: 500
});

Tests

npm test

Tests cover RAPTOR index creation, document indexing and tree building, multi-level retrieval with various strategies, summary generation and refinement, similarity scoring, chunk clustering, and behavioral tests for the summary refiner BT harness.

Requirements

Node.js 18 or later. ESM only.

Caveats

  • The default summarizer is a simple truncation; for meaningful hierarchical summaries, provide a real LLM summarizer adapter.
  • Tree depth grows logarithmically with document count. Very large document sets may produce deep trees with increased retrieval latency.
  • HHEM-based similarity requires the @polyweave/uncertainty package and an appropriate evaluation model.
  • Embedding-based clustering requires an embedding adapter; without one, text-overlap similarity is used.

License

MIT

Dependencies

Dependencies

ID Version
@polyweave/bt-harness *
@polyweave/core *
@polyweave/openai *
@polyweave/self-improving *
@polyweave/tools *
@polyweave/uncertainty *

Development Dependencies

ID Version
@rigor/core *

Peer Dependencies

ID Version
@polyweave/state *
@polyweave/state-search *

Keywords

polyweave raptor retrieval hierarchical memory
Details
npm
2026-07-10 23:34:24 +00:00
4
John Dvorak
SEE LICENSE IN LICENSE
13 KiB
Assets (1)
raptor-1.0.2.tgz 13 KiB
Versions (4) View all
1.0.6 2026-07-11
1.0.3 2026-07-11
1.0.2 2026-07-10
1.0.1 2026-07-10