Polyweave
Repositories
1
 Projects Packages

@polyweave/state (1.0.13)

Published 2026-07-11 15:33:35 +00:00 by Dvorak

Installation

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/
npm install @polyweave/state@1.0.13
"@polyweave/state": "1.0.13"

About this package

@polyweave/state

Transactionally versioned state stores with DVVSet conflict detection, lazy MVCC snapshot isolation, and kind-driven entity projections.

Synopsis

import {
  createFactGraph,
  createDocumentStore,
  createTreeStore,
  createTextStore,
  createClusterTreeStore,
  createTextEmbeddingVectorStore,
  createVectorIndex,
  createMultiStateStore,
  createMultiStateStoreFromSpec,
  dotsDominate,
  dotsConflicting,
  createFlatDotsRegistry,
  FrameTypes,
  CursorRegistry,
  createQueryFrame
} from '@polyweave/state';

const docs = createDocumentStore();
docs.sink.write({ type: 'DOC_UPDATE', content: { sectionId: 'a', text: 'Hello world' } });
const result = docs.getSection('a');

Install

npm install @polyweave/state

Why

Polyweave state stores provide transactionally versioned, conflict-aware storage with DVVSet-based version vectors, lazy MVCC snapshot isolation, and kind-driven entity projections. Eight specialized stores serve different data shapes — facts (triples), documents (sections), trees (wormhole-trie paths), full-text (BM25), cluster trees (QT clustering), embedding vectors, vector indexes (JAG/RNG/KDT/BKT), and multi-state federation — while sharing a unified push-stream protocol and transaction contract. Use them as the persistent, queryable backbone for agent blackboards, memory systems, and structured data pipelines.

Architecture

All stores share a push-stream protocol: write frames via sink.write(frame), subscribe to output via source.pipe(listener). Internal mutation is synchronous and single-threaded — there is no actual concurrency, only logical races detected through version vectors.

Version Vectors (DVVSet)

Every versioned unit carries a dots object { txnId: counter }. Dots are advanced on transactional writes only; non-txn writes do not increment them. A zero-allocation FlatDots variant (Uint32Array) is available for hot paths.

Conflict Modes

Mode Behavior Stores
Persistent Conflicting value stored alongside committed value. Multiple versions visible. Fact Graph, Document Store, Tree Store
Abortive Transaction aborts. Key appended to blocked set. Text Store, Cluster Tree Store, Vector Index
Delegated Conflict detection handled by individual stores wrapped by the federation layer. Multi-State Store

MVCC Snapshot Isolation

Fact Graph, Document Store, and Tree Store support lazy shadow chain MVCC. When any snapshot transaction is open, commits go to a shadow chain instead of mutating base state in-place. Snapshot transactions see state frozen at their start time; read-committed and non-txn queries see the full merged chain.

Stores (8)

1. Fact Graph

createFactGraph(options?: object) → { source, sink, getFact, getFactVersion, getMergedDots, compareAndSwap, checkConflicts, getConflicts, resolveConflict, searchStream, query, killSnapshot, getMvccState, describe, close }

A fact-based triple store (entity, relation, value) with temporal and spatial indexing, CocoEdge composite index sketch, and streaming lazy query.

Write frames: FACT_INSERT, FACT_UPDATE, FACT_DELETE Conflict mode: persistent | MVCC: snapshot isolation

Parameters

Name Type Required Default Description
options.now function No Date.now Timestamp provider
options.ttlSweepIntervalMs number No 0 TTL sweep interval (0 = disabled)
options.setTimeout function No setTimeout setTimeout provider
options.clearTimeout function No clearTimeout clearTimeout provider
options.defaultTimeout number No 120000 Default transaction timeout in ms
options.durability object No {} Durability config: { enabled, wrapperId, walStore }
options.indexHints object No {} Index hints: { sketch } for composite index
options.enableCompositeIndex boolean No true Enable composite index
options.enableQuerySketch boolean No per enableCompositeIndex Enable query sketch estimation
options.maxCursors number No 100 Max streaming cursors

Key Methods

Method Signature Description
getFact(factId) → fact | null Read a fact by ID
getFactVersion(factId, chainLimit?) → dots Current version dots for a fact
getMergedDots(factId, chainLimit?) → dots Merged dots (base + shadow chain)
compareAndSwap(factId, dots, value) → { accepted, dots } CAS write
checkConflicts(txnId) → { conflicts, mode } Pre-flight conflict check
getConflicts({ factId }) → conflicts Get conflicts for a fact
resolveConflict(factId, resolution) → void Resolve a conflict
searchStream(query, options) → void Streaming query
query(dsl) → results DSL query via query executor
killSnapshot(txnId) → boolean Kill a snapshot transaction
getMvccState() → mvcc state MVCC state snapshot
describe() → description Lightweight state summary
close() → void Close the store

2. Document Store

createDocumentStore(options?: object) → { source, sink, getSection, getSectionVersion, getMergedDots, compareAndSwap, checkConflicts, getConflicts, resolveConflict, querySectionsStream, getSnapshot, describe, close }

A section-based document store. Documents are composed of sections identified by sectionId.

Write frames: DOC_UPDATE Conflict mode: persistent | MVCC: snapshot isolation

Parameters

Name Type Required Default Description
options.now function No Date.now Timestamp provider
options.setTimeout function No setTimeout setTimeout provider
options.clearTimeout function No clearTimeout clearTimeout provider
options.defaultTimeout number No 120000 Default transaction timeout in ms
options.emitSnapshotOnCommit boolean No false Emit snapshot frames on commit
options.durability object No {} Durability config: { enabled, wrapperId, walStore }
options.lazyPointRead boolean No true Enable lazy point reads
options.maxCursors number No 100 Max streaming cursors

Key Methods

Method Signature Description
getSection(sectionId) → section | null Read a section by ID
getSectionVersion(sectionId, chainLimit?) → dots Current version dots
getMergedDots(sectionId, chainLimit?) → dots Merged dots (base + shadow chain)
compareAndSwap(sectionId, dots, value) → { accepted, dots } CAS write
checkConflicts(txnId) → { conflicts, mode } Pre-flight conflict check
getConflicts({ sectionId }) → conflicts Get conflicts for a section
resolveConflict(sectionId, resolution) → void Resolve a conflict
querySectionsStream(query, options) → void Streaming query
getSnapshot(options?) → snapshot Get snapshot
describe() → description Lightweight state summary
close() → void Close the store

3. Tree Store

createTreeStore(options?: object) → { source, sink, getCandidates, getPathVersion, getMergedDots, compareAndSwapPath, checkConflicts, getConflicts, resolveConflict, searchStream, describe, close }

A wormhole-trie-backed tree store. Paths are the versioned unit — rearranging the tree structure IS versioned data.

Write frames: TREE_UPSERT, TREE_LINK, TREE_UNLINK Conflict mode: persistent | MVCC: snapshot isolation

Parameters

Name Type Required Default Description
options.now function No Date.now Timestamp provider
options.setTimeout function No setTimeout setTimeout provider
options.clearTimeout function No clearTimeout clearTimeout provider
options.defaultTimeout number No 120000 Default transaction timeout in ms
options.durability object No {} Durability config: { enabled, wrapperId, walStore }
options.documentStore duplex No null Document store for persistence
options.persistToDocumentStore boolean No Boolean(documentStore) Auto-persist to doc store
options.snapshotMode 'copy' No 'copy' Snapshot mode
options.trackNodePaths boolean No true Track reverse node→path mapping
options.trackPathMap boolean No true Track path→nodeId mapping
options.compactNodes boolean No false Enable node compaction
options.maxPathKeyCache number No 2000 Path key cache size
options.maxCursors number No 100 Max streaming cursors

Key Methods

Method Signature Description
getCandidates(query, options) → candidates Query tree paths
getPathVersion(treeId, path, chainLimit?) → dots Current version dots for a path
getMergedDots(treeId, path, chainLimit?) → dots Merged dots (base + shadow chain)
compareAndSwapPath(treeId, path, dots, node) → { accepted, dots } CAS write to tree path
checkConflicts(txnId) → { conflicts, mode } Pre-flight conflict check
getConflicts({ treeId, path }) → conflicts Get path conflicts
resolveConflict(treeId, path, resolution) → void Resolve a conflict
searchStream(query, options) → void Streaming tree search
describe() → description Lightweight state summary
close() → void Close the store

4. Text Store

createTextStore(options?: object) → { source, sink, index, remove, search, searchStream, searchByEmbedding, get, getDocumentVersion, checkConflicts, isBlocked, resolveConflict, compareAndSwapDocument, describe, stats, close }

Full-text search with BM25 scoring, Porter stemming, stopword filtering, LSH-based fuzzy matching, and optional embedding support. Two-phase commit prevents partial inverted index corruption.

Write frames: TEXT_INDEX, TEXT_REMOVE Conflict mode: abortive | MVCC: no (two-phase commit, doc-level CAS)

Parameters

Name Type Required Default Description
options.now function No Date.now Timestamp provider
options.setTimeout function No setTimeout setTimeout provider
options.clearTimeout function No clearTimeout clearTimeout provider
options.defaultTimeout number No 120000 Default transaction timeout in ms
options.durability object No {} Durability config: { enabled, wrapperId, walStore }
options.analyzer object No DEFAULT_ANALYZER Tokenizer + normalizer
options.fuzzyThreshold number No 0.3 LSH fuzzy match similarity threshold
options.maxFuzzyExpansion number No 10 Max fuzzy-expanded terms per query
options.storeText boolean No true Store full text in memory
options.storeAttributesInMemory boolean No true Store attributes in memory
options.compactAfterIndex boolean No false Compact posting lists after index
options.enableFuzzy boolean No true Enable fuzzy matching
options.bm25 object No {} BM25 parameters: { k1?, b? }
options.bmw object No {} BMW parameters
options.documentStore duplex No null Document store for text offloading
options.persistToDocumentStore boolean No Boolean(documentStore) Auto-persist to doc store
options.lazyLoadDocumentText boolean No per storeText + documentStore Lazy load text from doc store

Key Methods

Method Signature Description
index(docId, text, options?) → void Index a document
remove(docId) → boolean Remove a document
search(query, options?) → results Full-text search
searchStream(query, options) → void Streaming search
searchByEmbedding(embedding, limit) → results Embedding-based search
get(docId) → doc | null Get document by ID
getDocumentVersion(docId) → dots Current version dots
checkConflicts(txnId) → { conflicts, mode } Conflict check
isBlocked(docId) → boolean Check if doc is blocked
resolveConflict(docId, resolution) → void Resolve a conflict
compareAndSwapDocument(docId, dots, value) → { accepted, dots } CAS write
describe() → description State summary
stats() → stats Store statistics
close() → void Close the store

Throws:

  • TXN_BEGIN with missing txnId: emits error on transaction timeout
  • Two-phase commit failure: TXN_CONFLICT frame, doc blocked

5. Cluster Tree Store

createClusterTreeStore(options?: object) → { source, sink, upsert, remove, buildTree, searchStream, isBlocked, resolveConflict, getEmbeddingVersion, describe, close }

Quality-threshold (QT) clustering over vector embeddings. Maintains a cohesive cluster tree with auto-rebuild on drift. Optional vector store for candidate retrieval; operates in tree-only mode without one.

Write frames: CLUSTER_UPSERT Conflict mode: abortive | MVCC: no (doc-level CAS)

Parameters

Name Type Required Default Description
options.vectorStore object No null Vector store with getCandidates() and getSignals()
options.treeStore duplex No null Tree store (auto-creates if null)
options.treeId string No 'cluster-tree:default' Tree identifier
options.maxLeafDocs number No 32 Max docs per leaf
options.targetLeafCount number No null Target leaf count
options.staleRatioThreshold number No 1.25 Staleness threshold for rebuild
options.staleLeafEventThreshold number No 0.2 Fraction of stale events
options.clusterSignalWeight number No 0.15 Signal weight in scoring
options.scoringMethod string No 'full-hybrid' Scoring method
options.nnCandidateLimit number No 12 NN candidate limit
options.nnMaxCheck number No 24 NN max check
options.buildNearestExactThreshold number No 1500 Threshold for exact nearest
options.durability object No {} Durability config
options.emitFrames boolean No true Emit frames on mutations
options.emitFrameTypes string[] No ['all'] Frame types to emit

Key Methods

Method Signature Description
upsert(docId, embedding, metadata?) → void Upsert a doc/embedding
remove(docId) → void Remove a doc
buildTree() → void Build/rebuild cluster tree
searchStream(query, options) → void Streaming search
isBlocked(docId) → boolean Check if doc is blocked
resolveConflict(docId, resolution) → void Resolve a conflict
getEmbeddingVersion(docId) → dots Get version dots for a doc
describe() → description State summary
close() → void Close the store

Frame Protocol

Input Frame Description
CLUSTER_UPSERT { docId, embedding, metadata?, attributes?, txnId? }
CLUSTER_REMOVE { docId, txnId? }
CLUSTER_BUILD_TREE { }
CLUSTER_SEARCH { query, limit?, docMask?, ... }
TXN_BEGIN / TXN_COMMIT / TXN_ABORT Standard transaction frames
Output Frame Description
CLUSTER_RESULT Search result
CLUSTER_UPSERT_DOTS Upsert response with dots
CLUSTER_BUILT Tree build complete
TXN_CONFLICT Abortive conflict
CLUSTER_CONFLICT Cluster-level conflict
CLUSTER_KEY_BLOCKED / CLUSTER_KEY_UNBLOCKED Block/unblock notification
DESCRIPTION Description response

6. Text Embedding Vector Store

createTextEmbeddingVectorStore(textStore: duplex, options?: object) → { getCandidates, getSignals, stats, supportsDocMask }

A thin read-only adapter wrapping a Text Store to expose a vector-index-like API surface. No state or conflict detection of its own. Uses textStore.searchByEmbedding() for candidate retrieval.

Parameter Type Required Default Description
textStore duplex Yes — Text store with searchByEmbedding() and get()
options.maskOverfetch number No 4 Overfetch multiplier when a docMask is active

Throws: Error('createTextEmbeddingVectorStore requires textStore.searchByEmbedding() and textStore.get()')

Returns:

  • getCandidates(query, searchOptions?) — Returns doc IDs based on embedding similarity
  • getSignals(query, docIds) — Returns Map<docId, cosineSimilarity> for given doc IDs
  • stats() — Delegates to textStore.stats()
  • supportsDocMask — Always true

7. Vector Index

createVectorIndex(options: object) → { source, sink, addBatch, deleteId, search, build, getRowVersion, isBlocked, resolveConflict, unblockAll, describe, close }

A low-level multi-algorithm vector index (JAG, RNG, KDT, BKT) with product quantization, paged sampling, and row-level CAS-based conflict detection.

Parameter Type Required Default Description
options.dataset DistanceMatrix Yes — Dataset backing the index
options.graph JAG/RNG graph Yes — Graph structure
options.treeKdt KDT tree No null KD-tree for seed candidates
options.treeBkt BKT tree No null BK-tree for seed candidates
options.distance function Yes — Distance function
options.quantizer PQ quantizer No null Product quantization
options.quantizedDataset Uint8Matrix No null Quantized dataset representation
options.useQuantized boolean No true Use quantized vectors for search
options.labelset Labelset No null Deleted row marker
options.workspacePool workspace pool No auto-created Search workspace pool
options.seedCandidates number No 0 Seed candidate count
options.seedMaxCheck number No 4096 Max seed checks
options.adaptiveSearch boolean No true Enable adaptive search
options.adaptiveMinConfidence number No 0.08 Min confidence threshold
options.adaptiveStabilityThreshold number No 0.9 Stability threshold
options.adaptiveMaxCheckMultiplier number No 8 Max check multiplier
options.adaptiveMaxStageCount number No 4 Max adaptive stages
options.adaptiveSeedGrowth number No 2 Seed growth factor
options.adaptiveMaxSeedCandidates number No 4096 Max seed candidates
options.maintenanceEnabled boolean No true Auto-maintenance
options.maintenanceDeleteRatio number No 0.35 Delete threshold for rebuild
options.maintenanceMinDeleted number No 4096 Min deletes for rebuild
options.maintenanceMinOps number No 2048 Min ops for maintenance
options.maintenanceMode 'rebuild' No 'rebuild' Maintenance mode

Throws:

  • Error('VectorIndex requires dataset') if dataset is missing
  • Error('VectorIndex requires graph') if graph is missing
  • Error('VectorIndex requires distance function') if distance is not a function

Key Methods

Method Signature Description
addBatch(rows) → void Add rows to the index
deleteId(id) → void Mark a row as deleted
search(query, k?) → results Search with optional docMask
build() → void Force rebuild
getRowVersion(id) → dots Get row version
isBlocked(id) → boolean Check if blocked
resolveConflict(id, resolution) → void Resolve conflict
unblockAll() → void Unblock all blocked rows
describe() → description State summary
close() → void Close the store

8. Multi-State Store

createMultiStateStore(options?: object) → { source, sink, search, query, queryStream, multihopQueryStream, getAttributes, getState, getCursorRegistry, getMvccState, getMultiTxnState, triggerRestore, insert, get, update, delete, defineProjection, searchEntities, describe, close }

A federation store wrapping text, tree, graph, and document stores for parallel retrieval. Provides unified lazy query with variable binding, multihop traversal, cross-store candidate aggregation, weighted scoring, filter resolution, and entity projections.

Write frames: All store frames + TXN_BEGIN, TXN_COMMIT, TXN_ABORT, KILL_SNAPSHOT, KILL_ALL_SNAPSHOTS, DESCRIBE Conflict mode: Delegated (abortive-dominant across stores) | MVCC: Snapshot forwarded

Parameters

Name Type Required Default Description
options.textStore duplex No auto-created Text store
options.treeStore duplex No auto-created Tree store
options.graphStore duplex No auto-created Fact graph
options.docStore duplex No auto-created Document store
options.stores object No {} Additional custom stores
options.nodeToDocIds function No defaultNodeToDocIds Node→doc mapping
options.predicates object No {} Predicate definitions
options.predicateStats object No {} Predicate statistics
options.candidateWeights object No {} Per-store candidate weights
options.signalWeights object No {} Signal scoring weights
options.defaultCandidateLimit number No 200 Default candidate fetch limit
options.defaultCandidateStores string[] No ['text'] Default stores to search
options.plannerEnabled boolean No false Enable query planner
options.ranker function No null Custom cross-store ranker
options.persistInsertedDocuments boolean No true Persist inserted docs
options.projections object No null Entity projection config
options.durability object No {} Durability config

Key Methods

Method Signature Description
search(query, options?) → results Cross-store search
query(query, options) → results Structured query
queryStream(query, options) → void Streaming structured query
multihopQueryStream(query, options) → void Multihop streaming query
getAttributes(docId) → attributes Get stored attributes for a doc
getState() → state Get internal state snapshot
getCursorRegistry() → CursorRegistry Cursor registry
getMvccState() → mvcc state Aggregated MVCC state
getMultiTxnState(txnId) → txn state Per-txn state
triggerRestore() → void Trigger durability restore
insert(kind, fields) → entity Insert entity (projection CRUD)
get(kind, docId, options?) → entity Get entity with optional relation expansion
update(kind, docId, fields) → void Update entity fields
delete(kind, docId) → void Delete entity
defineProjection(name, spec) → void Register a projection
searchEntities(kind, query, options?) → results Search entities with optional expansion
describe() → description State summary
close() → void Close all stores

Multi-Store Transactions

Atomic commit across all stores with a shared txnId. Abortive-dominant: a text store conflict aborts writes to all stores.

store.sink.write({ type: 'TXN_BEGIN', content: { txnId: 't1', isolation: 'snapshot' } });
store.sink.write({ type: 'FACT_INSERT', content: { ..., txnId: 't1', targetStore: 'graph' } });
store.sink.write({ type: 'TEXT_INDEX', content: { ..., txnId: 't1', targetStore: 'text' } });
store.sink.write({ type: 'TXN_COMMIT', content: { txnId: 't1' } });

Entity Projections

Entity projections let you define entity types as collections of fields and relations. The Multi-State Store derives which stores to create, which frames to emit, and how to read data back.

Field Kinds

Kind Stores Description
value graph String attribute
value:n graph Number attribute
value:b graph Boolean attribute
value:? graph Nullable string
content document+text Document section + text index
content:stored document Document section only
content:index text Text index only
location:/root tree Tree path, auto-set on insert
location:/root/link tree Directional TREE_LINK
embedding:768 vector+cluster Vector (768d) + clustered
embedding:768:tag vector+cluster Per-tag vector
embedding:768:tag:rng vector+cluster With RNG algorithm
ref:Entity graph Single entity reference
refs:Entity graph Multi-entity references
timestamp graph Auto-set ISO timestamp on insert
timestamp:update / ts:update graph Auto-set on insert + update
geometry:point graph GeoJSON Point coordinates
expiry:3600 graph Auto-set TTL (seconds)

CRUD Example

const ms = createMultiStateStore({ projections: {
  Article: { primaryKey: 'docId', fields: { title: 'value', body: 'content' } }
}});

ms.insert('Article', { docId: 'art:1', title: 'ML', body: 'Intro...' });
const raw = ms.get('Article', 'art:1');
const expanded = ms.get('Article', 'art:1', { expand: ['authors'] });
ms.update('Article', 'art:1', { title: 'Deep Learning' });
ms.delete('Article', 'art:1');

createMultiStateStoreFromSpec(spec, registry?, options?)

Creates a multi-state store from a declarative spec with store references, candidate sets, and search profiles.

createMultiStateStoreFromSpec(spec, registry?, options?) → multiStateStore

Adds searchProfile(profileName, query, options) and getSpecInfo() to the returned store.

compileMultiStateSpec(spec, registry?, options?)

Compiles a spec into the options object passed to createMultiStateStore.

Transaction Contract

Method Signature Purpose
checkConflicts(txnId) → { conflicts, mode } Non-destructive pre-flight conflict check
getXxxVersion(key, chainLimit?) → dots Current dots for a key
getMergedDots(key, chainLimit?) → dots Merged dots (base + shadow chain, MVCC stores)
isBlocked(key) → boolean Whether a key is locked (abortive stores)
killSnapshot(txnId) → boolean Kill a snapshot transaction (MVCC stores)
describe() → description Lightweight state summary

Frame Protocol (all stores except Vector Index)

store.sink.write({ type: 'TXN_BEGIN', content: { txnId } });
store.sink.write({ type: 'TXN_COMMIT', content: { txnId } });
store.sink.write({ type: 'TXN_ABORT', content: { txnId } });

Snapshot transactions on MVCC stores:

store.sink.write({ type: 'TXN_BEGIN', content: { txnId, isolation: 'snapshot' } });

DESCRIBE Protocol

All stores handle DESCRIBE frames, returning DESCRIPTION with a lightweight state summary.

store.sink.write({ type: 'DESCRIBE', content: { streamId: 'req1' } });
// → DESCRIPTION { content: { storeType, ...counts, mvcc, conflicts, blocked } }

DVVSet Version Vectors (lib/dvv-set.js)

Object-based (zero-allocation)

Export Signature Description
dotsDominate(a, b) (dot, dot) → boolean a strictly dominates b
dotsEqual(a, b) (dot, dot) → boolean a is equal to b
dotsConflicting(a, b) (dot, dot) → boolean Neither dominates the other
advanceDot(dots, id) (dot, string) → dot Increment counter for id
dotsDescribe(dots) (dot) → object Copy dots into plain object
mergeDots(a, b) (dot, dot) → dot Pointwise max merge
dotsCount(dots) (dot) → number Count active replicas
dotsIsEmpty(dots) (dot) → boolean True if no active replicas

FlatDots (Uint32Array-based, zero-GC)

createFlatDotsRegistry() → {
  register(replicaId), replicaCount(), create(),
  dominate(a, b), equal(a, b), advance(dots, idx),
  conflicting(a, b), merge(a, b), copy(dots),
  isEmpty(dots), toObject(dots)
}

Maps replica IDs to sequential integer indices. Dots are dense Uint32Array — no hash lookups, cache-friendly.

Remote Stores (remote/remote-stores.js)

Local-first duplex wrappers with remote fallback. All follow the same pattern: try local first, if empty policy says "empty", fall back to remote.

Export Required Options Throws if missing
createTreeRemoteStoreDuplex({ local, remote, cache?, propagateWrites?, emptyPolicy? }) local Error('Tree remote store requires local duplex.')
createDocumentRemoteStoreDuplex({ local, remote, cacheResponse?, propagateWrites?, emptyPolicy? }) local Error('Document remote store requires local duplex.')
createFactRemoteStoreDuplex({ local, remote, cacheFacts?, propagateWrites?, emptyPolicy? }) local Error('Fact remote store requires local duplex.')
createTextRemoteStoreDuplex({ local, remote, propagateWrites?, emptyPolicy? }) local Error('Text remote store requires local duplex.')

Error output: Emits STATE_NOT_FOUND error frames when remote returns empty.

File Stores (remote/)

Export Description
createDocumentFileStore(dirPath, options?) File-backed document store
createTreeFileStore(dirPath, options?) File-backed tree store
createTextFileStore(dirPath, options?) File-backed text store
createFactFileStore(dirPath, options?) File-backed fact graph
createFileLayout(dirs?, options?) File layout manager for multi-store

Streaming Infrastructure (streaming.js)

Frame-based streaming protocol for lazy query results. Pure push-stream, no async/await.

Export Description
FrameTypes Constants: { QUERY, RESPONSE, META, NEXT, CONTINUE, CLOSE, FINISH, ERROR }
CursorRegistry Manages streaming query cursors; constructor accepts { maxCursors }
createQueryFrame(query, options) Create a QUERY frame with { cursorId, limit, offset, filters }
createResponseFrame(cursorId, results, metadata) Create a RESPONSE frame
createNextFrame(cursorId) Create a NEXT frame to request more results
createCloseFrame(cursorId) Create a CLOSE frame
createMetaFrame(content) Create a META frame
createStreamingErrorFrame(cursorId, message) Create an ERROR frame
createStreamingSearch(store, options) High-level streaming search harness
createStreamingFrameHandler(handlers) Frame handler factory
collectStreamResults(source, timeoutMs) Collect streaming results synchronously
StreamingPropertyTests Property test helper
runStreamingPropertyTests() Run streaming property tests

Additional Exports

Export Source Description
createEmptyAttributes() attribute-schema.js Creates an empty attributes bucket
normalizeAttributes(raw, schema?) attribute-schema.js Normalizes raw attributes
mergeNormalizedAttributes(target, source) attribute-schema.js Merges normalized attributes
createMultihopQueryExecutor(options) multi-state/multihop-query-executor.js Multihop traversal executor
createJAGGraph(options) vector-store/graph-jag.js JAG graph construction
createCoordinatorReplayState(options) multi-state/durability-schema.js Coordinator replay state
applyCoordinatorDelta(state, delta) multi-state/durability-schema.js Apply coordinator delta
createDurableStoreOp(type, store, delta) multi-state/durability-schema.js Durable store operation
serializeCoordinatorReplayState(state) multi-state/durability-schema.js Serialize replay state
MultiStateDurabilityDeltaType multi-state/durability-schema.js Delta type enum
resolveEmptyPolicy(policy) remote/empty-policy.js Resolve empty-check policy

Design Decisions

  • In-place commit is safe: JavaScript is single-threaded and synchronous at sink.write().
  • Kind-driven store selection: The caller declares what a field means ('value', 'content', 'embedding:768'); the system chooses stores, frame types, and read strategies.
  • Relations are first-class: Named predicates with inverses and cardinality, stored as fact graph facts, expanded on read.
  • Tree-store path-level dots: The versioned unit is treeId||pathKey, because rearranging the tree IS versioned data.
  • Text-store two-phase commit: Phase 1 checks all ops against committed dots; Phase 2 applies atomically.
  • MVCC shadow chain over clone: Zero-allocation shadow chain. No clone cost for reads when the chain is empty.
  • Conflicts are per-key: Only keys that were read AND written trigger conflict checks.
  • Dual API: Every store exposes both a direct API and a frame protocol. Both are first-class.
  • Federated compatible: All stores are { source, sink } duplexes following the push-stream contract.

Tests

npm test

489 tests across 51 test files covering: DVVSet conflict detection (object and FlatDots), lazy MVCC snapshot isolation, transactional commit/rollback/abort, conflict resolution, kind-driven entity projections with CRUD + relation expansion, streaming cursors, store-level property tests (js-rigor), text store BM25 correctness (BEIR SciFact evaluation), cluster tree store, embedding vector store, vector index (JAG comprehensive, multi-attribute, NOT logic), multi-state attribute handling, multihop query execution, remote stores, WAL crash recovery, durability schemas, and performance analysis.

Requirements

  • Node.js 22+
  • ESM only

Dependencies: heapify, porter-stemmer Peer dependencies: @polyweave/core, @polyweave/durability

Dependencies

Dependencies

ID Version
heapify ^0.6.0
porter-stemmer ^0.9.1

Development Dependencies

ID Version
@rigor/core *
fast-check *
tape *

Peer Dependencies

ID Version
@polyweave/core *
@polyweave/durability *
@polyweave/errors *

Keywords

polyweave state memory store
Details
npm
2026-07-11 15:33:35 +00:00
445
John Dvorak
SEE LICENSE IN LICENSE
latest
238 KiB
Assets (1)
state-1.0.13.tgz 238 KiB
Versions (5) View all
1.0.13 2026-07-11
1.0.10 2026-07-11
1.0.9 2026-07-11
1.0.8 2026-07-10
1.0.3 2026-07-10