@polyweave/state (1.0.8)

Published 2026-07-10 20:51:52 +00:00 by Dvorak

Set up this registry in your project .npmrc file:

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/

To install the package using npm, run the following command:

npm install @polyweave/state@1.0.8

or add it to the package.json file:

"@polyweave/state": "1.0.8"

For more information on the npm registry, see the documentation.

@polyweave/state

Transactionally versioned state stores with DVVSet-based conflict detection, lazy MVCC snapshot isolation, and kind-driven entity projections with declared relations.

Synopsis

import { createMultiStateStore, createDocumentStore, createTextStore, createTreeStore, createFactGraph } from '@polyweave/state';

var store = createMultiStateStore({ name: 'main' });
var docs = store.createDocumentStore({ name: 'docs' });

docs.sink.write({ type: 'DOC_UPDATE', content: { id: 'a', text: 'Hello world' } });
var result = docs.get('a');

Install

npm install @polyweave/state

Architecture

All stores share a push-stream protocol: write frames via sink.write(frame), subscribe to output via source.pipe(listener). Internal mutation is synchronous and single-threaded — there is no actual concurrency, only logical races detected through version vectors.

Version Vectors (DVVSet)

Every versioned unit (fact, section, tree path, document) carries a dots object { txnId: counter }. Dots are advanced on transactional writes only; non-txn writes do not increment them.

Shared primitives in src/lib/dvv-set.js:

dotsDominate(a, b)   → a strictly dominates b: a includes b and has additional causal knowledge
dotsEqual(a, b)      → a == b
dotsConflicting(a, b) → neither a ≥ b nor b ≥ a (fork)
advanceDot(dots, id)  → dots[id] = (dots[id] || 0) + 1

A zero-allocation FlatDots variant (Uint32Array) is available for hot paths.

Conflict Modes

Mode	Behavior	Stores
Persistent	Conflicting value stored alongside committed value. Multiple versions visible. System continues. Resolution is higher-order.	Fact Graph, Document Store, Tree Store
Abortive	Transaction aborts. Key appended to blocked set. All writes (including non-txn) rejected until resolved.	Text Store, Cluster Tree Store, Vector Index
Delegated	Conflict detection handled by the individual stores wrapped by the federation layer.	Multi-State Store

Conflict frames standardized via src/lib/conflict-frames.js: emitTxnConflict, emitKeyBlocked, emitKeyUnblocked, resolveConflict.

MVCC Snapshot Isolation

Fact Graph, Document Store, and Tree Store support lazy shadow chain MVCC via src/lib/mvcc.js. When any snapshot transaction is open, all commits go to a shadow chain instead of mutating the base state in-place. When the last snapshot closes, the chain condenses into the base. Snapshot transactions see state frozen at their start time; read-committed and non-txn queries see the full merged chain. Shadow commits are WAL-logged for crash recovery.

Stores (8)

1. Fact Graph

createFactGraph(options?)

A fact-based triple store (entity, relation, value) with temporal and spatial indexing, composite index sketch, and streaming lazy query.

Write frames	`FACT_INSERT`, `FACT_UPDATE`, `FACT_DELETE`
Conflict mode	persistent
MVCC	snapshot isolation
Key API	`getFact(factId)`, `getFactVersion(factId, chainLimit?)`, `getMergedDots(factId, chainLimit?)`, `compareAndSwap(factId, dots, value)`, `checkConflicts(txnId)`, `getConflicts({ factId })`, `resolveConflict(factId, resolution)`, `searchStream(query, options)`, `query(dsl)`, `killSnapshot(txnId)`, `getMvccState()`

2. Document Store

createDocumentStore(options?)

A section-based document store. Documents are composed of sections identified by sectionId. Supports streaming query and persistence integration.

Write frames	`DOC_UPDATE`
Conflict mode	persistent
MVCC	snapshot isolation
Key API	`getSection(sectionId)`, `getSectionVersion(sectionId, chainLimit?)`, `getMergedDots(sectionId, chainLimit?)`, `compareAndSwap(sectionId, dots, value)`, `checkConflicts(txnId)`, `getConflicts({ sectionId })`, `resolveConflict(sectionId, resolution)`, `querySectionsStream(query, options)`

3. Tree Store

createTreeStore(options?)

A wormhole-trie-backed tree store. Paths are the versioned unit. Rearranging the tree structure IS versioned data.

Write frames	`TREE_UPSERT`, `TREE_LINK`, `TREE_UNLINK`
Conflict mode	persistent
MVCC	snapshot isolation
Key API	`getCandidates(query, options)`, `getPathVersion(treeId, path, chainLimit?)`, `getMergedDots(treeId, path, chainLimit?)`, `compareAndSwapPath(treeId, path, dots, node)`, `checkConflicts(txnId)`, `getConflicts({ treeId, path })`, `resolveConflict(treeId, path, resolution)`, `searchStream(query, options)`

4. Text Store

createTextStore(options?)

Full-text search with BM25 scoring, inverted index, and optional embedding support. Two-phase commit: Phase 1 dry-runs all operations against committed dots, Phase 2 applies. No partial inverted index corruption.

Write frames	`TEXT_INDEX`, `TEXT_REMOVE`
Conflict mode	abortive
MVCC	no (two-phase commit, doc-level CAS)
Key API	`index(docId, text, options?)`, `remove(docId)`, `search(query, options?)`, `searchStream(query, options)`, `getDocumentVersion(docId)`, `checkConflicts(txnId)`, `isBlocked(docId)`, `resolveConflict(docId, resolution)`, `compareAndSwapDocument(docId, dots, value)`

5. Cluster Tree Store

createClusterTreeStore(options?)

Quality-threshold (QT) clustering over vector embeddings. Maintains a cohesive cluster tree with auto-rebuild on drift.

Write frames	`CLUSTER_UPSERT`
Conflict mode	abortive
MVCC	no (doc-level CAS)
Key API	`upsert(docId, embedding, metadata?)`, `remove(docId)`, `buildTree()`, `searchStream(query, options)`, `isBlocked(docId)`, `resolveConflict(docId, resolution)`, `getEmbeddingVersion(docId)`

6. Text Embedding Vector Store

createTextEmbeddingVectorStore(textStore)

A thin read-only adapter that wraps a Text Store, exposing a vector-index-like API surface. No state or conflict detection of its own.

Write frames	N/A (passes through to Text Store)
Conflict mode	N/A
MVCC	N/A
Key API	`getCandidates(query)`, `getSignals()`, `stats()`, `supportsDocMask()`

7. Vector Index

createVectorIndex(options?)

A low-level multi-algorithm vector index (JAG, RNG, KDT, BKT) with product quantization, paged sampling, and row-level CAS-based conflict detection.

Write frames	N/A (direct method calls: `addBatch`, `deleteId`)
Conflict mode	abortive (return-value conflicts)
MVCC	no (row-level CAS)
Key API	`addBatch(rows)`, `deleteId(id)`, `search(query, k?)`, `build()`, `getRowVersion(id)`, `isBlocked(id)`, `resolveConflict(id, resolution)`, `unblockAll()`

8. Multi-State Store

createMultiStateStore({ textStore, treeStore, graphStore, docStore, projections?, ... })

A federation store that wraps text, tree, graph, and document stores for parallel retrieval. Delegates writes to the appropriate underlying store. Provides unified lazy query with variable binding, multihop traversal, cross-store candidate aggregation, weighted scoring, filter resolution, and entity projections with kind-driven fields and declared relations.

Supports multi-store transactions — atomic commit across all stores with a shared txnId. Snapshot isolation forwarded to MVCC-capable stores; abortive-dominant atomicity.

Hybrid Memory + Disk Offload (recommended)

For balanced memory usage, keep graph/tree indexes in memory and offload heavy text/document payloads via file-backed WAL durability.

import { FileWALStore } from '@polyweave/core';
import {
  createDocumentStore,
  createFactGraph,
  createMultiStateStore,
  createTextStore,
  createTreeStore
} from '@polyweave/state';

const walStore = new FileWALStore('./data/polyweave');
walStore.initialize();

const docStore = createDocumentStore({
  durability: { enabled: true, wrapperId: 'docs', walStore }
});

const textStore = createTextStore({
  // Keep BM25 index in memory, offload raw document text to document-store
  storeText: false,
  documentStore: docStore,
  persistToDocumentStore: true,
  lazyLoadDocumentText: true,
  durability: { enabled: true, wrapperId: 'text', walStore }
});

const ms = createMultiStateStore({
  textStore,
  docStore,
  graphStore: createFactGraph(),   // in-memory hot index
  treeStore: createTreeStore(),     // in-memory topology
  durability: { enabled: true, wrapperId: 'multi', walStore }
});

Write frames	All store frames + `TXN_BEGIN`, `TXN_COMMIT`, `TXN_ABORT`, `KILL_SNAPSHOT`, `KILL_ALL_SNAPSHOTS`, `DESCRIBE`
Conflict mode	Delegated (abortive-dominant across stores)
MVCC	Snapshot forwarded to MVCC stores; chainIndices captured per store
Key API	`search(query, options?)`, `query(query, options)`, `queryStream(query, options)`, `multihopQueryStream(query, options)`, `getAttributes(docId)`, `getState()`, `getCursorRegistry()`, `getMvccState()`, `getMultiTxnState(txnId)`, `triggerRestore()`

Entity Projections

Entity projections let you define entity types as collections of fields and relations. The Multi-State Store derives which stores to create, which frames to emit, and how to read data back. The caller never names a store — they name what the field means.

Defining a Projection

const ms = createMultiStateStore({
  // Pre-created stores (optional — auto-created from kinds if omitted)
  graphStore: createFactGraph(),
  textStore: createTextStore(),

  projections: {
    Author: {
      primaryKey: 'docId',
      fields: {
        name: 'value',
        bio:  'content',
      },
      relations: {
        authored: { to: 'Article', inverse: 'authors' }
      }
    },
    Article: {
      primaryKey: 'docId',
      fields: {
        title:   'value',
        body:    'content',
        created: 'timestamp',
        vec:     'embedding:768',
      },
      relations: {
        authors: { to: 'Author', inverse: 'authored', many: true }
      }
    }
  }
});

Field Kinds

Each field declares a kind — a shorthand string or { kind, ... } object. The system derives store routing and frame construction automatically.

Kind	Stores	Description
`value`	graph	String attribute (fact: `entity=$docId, relation=$field`)
`value:n`	graph	Number attribute
`value:b`	graph	Boolean attribute
`value:?`	graph	Nullable string
`content`	document+text	Document section + text index
`content:stored`	document	Document section only (no search)
`content:index`	text	Text index only (searchable, not in doc)
`location:/root`	tree	Tree path, auto-set on insert
`location:/root/link`	tree	Directional TREE_LINK
`embedding:768`	vector+cluster	Vector (768d) + auto-clustered
`embedding:768:tag`	vector+cluster	Per-tag vector + cluster
`embedding:768:tag:rng`	vector+cluster	With RNG algorithm
`ref:Entity`	graph	Single entity reference
`refs:Entity`	graph	Multi-entity references
`timestamp`	graph	Auto-set ISO timestamp on insert
`timestamp:update` / `ts:update`	graph	Auto-set on insert + update
`geometry:point`	graph	GeoJSON Point coordinates
`expiry:3600`	graph	Auto-set TTL (seconds)

Field overrides via object form:

fields: {
  author:  { kind: 'value', relation: 'creator' },
  body:    { kind: 'content', bm25: { k1: 0.8 } },
  path:    { kind: 'location', root: '/articles', template: '/articles/$docId' },
  emb:     { kind: 'embedding', dim: 768, tag: 'img', algorithm: 'rng' },
  created: { kind: 'timestamp', relation: 'published_at' },
}

Relations

Relations are named predicates between entity types, distinct from fields:

Property	Description
`to`	Target entity type
`inverse`	Inverse predicate name on the target
`many`	`true` (default) for array values, `false` for single value

relations: {
  // Shorthand — just the target type name
  authored: 'Article',

  // Full spec with inverse and cardinality
  authors: { to: 'Author', inverse: 'authored', many: true },
}

CRUD API

// Insert — atomic across all stores
const article = ms.insert('Article', {
  docId: 'art:1',
  title: 'Machine Learning',
  body: 'An introduction...',
  authors: ['auth:1', 'auth:2']
});

// Get — raw values
const raw = ms.get('Article', 'art:1');
// { docId: 'art:1', title: '...', body: '...', authors: ['auth:1', 'auth:2'], _dots: {...}, _sources: {...} }

// Get with relation expansion
const expanded = ms.get('Article', 'art:1', { expand: ['authors'] });
// { docId: 'art:1', title: '...', authors: [{ docId: 'auth:1', name: 'Alice' }, ...] }

// Expand all relations
const full = ms.get('Article', 'art:1', { expand: '*' });

// Update — replaces only specified fields
ms.update('Article', 'art:1', { title: 'Deep Learning' });

// Delete — removes from all stores + relation facts
ms.delete('Article', 'art:1');

// Search with expansion
const hits = ms.searchEntities('Article', { text: 'learning' }, { expand: ['authors'] });

Validation

At projection registration time, the system validates:

Field kinds — must be a recognized kind string or { kind, ... } object
Relation targets — must reference a registered projection
Inverse consistency — inverse predicate must exist on the target entity
Name collisions — field name must not duplicate a relation name on the same entity

Auto-Store Creation

When projections are provided without pre-created stores, the MS auto-creates:

graph (fact graph)
document (document store)
text (text store)
tree (tree store)
cluster:* (cluster tree per tag)

Vector stores require pre-built datasets and are NOT auto-created. Pre-created stores take precedence.

Store Contract

DESCRIBE Protocol

All stores handle the DESCRIBE frame, returning a DESCRIPTION frame with a lightweight state summary. The Multi-State Store aggregates across all registered stores.

// Frame protocol (works on any store or MS)
store.sink.write({ type: 'DESCRIBE', content: { streamId: 'req1' } });
// → DESCRIPTION { content: { storeType, ...counts, mvcc, conflicts, blocked } }

// Direct API
store.describe()  →  { storeType, factCount, transactionCount, mvcc, ... }
ms.describe()     →  { storeType: 'multiState', stores: { graph: {...}, text: {...} }, ... }

Transaction Contract

Method	Signature	Purpose
`checkConflicts(txnId)`	`→ { conflicts, mode }`	Non-destructive pre-flight conflict check
`getXxxVersion(key, chainLimit?)`	`→ dots`	Current dots for a key
`getMergedDots(key, chainLimit?)`	`→ dots`	Merged dots (base + shadow chain, MVCC stores)
`isBlocked(key)`	`→ boolean`	Whether a key is locked (abortive stores)
`killSnapshot(txnId)`	`→ boolean`	Kill a snapshot transaction (MVCC stores)
`describe()`	`→ description object`	Lightweight state summary

Transaction Protocol

All stores (except Vector Index) follow the standard transaction protocol:

store.sink.write({ type: 'TXN_BEGIN', content: { txnId } });
// read + write operations with txnId...
store.sink.write({ type: 'TXN_COMMIT', content: { txnId } });
// or
store.sink.write({ type: 'TXN_ABORT', content: { txnId } });

Snapshot transactions on MVCC-enabled stores:

store.sink.write({ type: 'TXN_BEGIN', content: { txnId, isolation: 'snapshot' } });

Conflict detection is per-key: only keys that were both read AND written by a transaction are checked. Writes to unread keys skip conflict detection (last-writer-wins).

Multi-Store Transactions

const ms = createMultiStateStore({ graphStore: fg, textStore: ts, docStore: ds });

// Atomic write across 3 stores
ms.sink.write({ type: 'TXN_BEGIN', content: { txnId: 't1', isolation: 'snapshot' } });
ms.sink.write({ type: 'FACT_INSERT', content: { factId: 'f1', ..., txnId: 't1', targetStore: 'graph' } });
ms.sink.write({ type: 'TEXT_INDEX', content: { docId: 'd1', text: '...', txnId: 't1', targetStore: 'text' } });
ms.sink.write({ type: 'DOC_UPDATE', content: { sectionId: 's1', text: '...', txnId: 't1', targetStore: 'document' } });
ms.sink.write({ type: 'TXN_COMMIT', content: { txnId: 't1' } });
// All stores committed with dots { t1: 1 } — or all aborted if any abortive conflict

// Abortive-dominant: a text store conflict aborts the fact graph write too
ms.sink.write({ type: 'TXN_BEGIN', content: { txnId: 't2' } });
ms.sink.write({ type: 'TEXT_GET', content: { docId: 'd1', txnId: 't2', targetStore: 'text' } });
// ... external write to d1 ...
ms.sink.write({ type: 'TEXT_INDEX', content: { docId: 'd1', text: '...', txnId: 't2', targetStore: 'text' } });
ms.sink.write({ type: 'FACT_INSERT', content: { factId: 'f2', ..., txnId: 't2', targetStore: 'graph' } });
ms.sink.write({ type: 'TXN_COMMIT', content: { txnId: 't2' } });
// Text store conflict → TXN_CONFLICT(abortive), d1 blocked, f2 never written

// Kill snapshot across all MVCC stores
ms.sink.write({ type: 'KILL_SNAPSHOT', content: { txnId: 't1' } });
ms.sink.write({ type: 'KILL_ALL_SNAPSHOTS', content: {} });

// Aggregate MVCC state
ms.getMvccState();     // { hasActiveShadows, openCount, chainSize, openTxnIds, byStore }
ms.getMultiTxnState('t1'); // { txnId, isolation, subTxns, chainIndices, startedAt }

Library Modules

Module	Description
`src/lib/dvv-set.js`	DVVSet version vector primitives (zero-allocation + FlatDots Uint32Array variant)
`src/lib/mvcc.js`	Lazy MVCC shadow chain (beginSnapshot, endSnapshot, commitToShadow, condense)
`src/lib/conflict-frames.js`	Standard conflict frame factories
`src/lib/kind-registry.js`	Kind normalization + store detection from field kind specs
`src/state/multi-state/entity-projection.js`	Entity projection subsystem (fields + relations, CRUD, expansion)

Testing

# All state tests
node --test test/*.test.js               # 553 tests

# Entity projection tests (65 tests)
node --test test/entity-projection.test.js
node --test test/entity-projection.property.test.js
node --test test/entity-relation.test.js

# Multi-state store tests (19 tests)
node --test test/multi-state-store.test.js

# Store-specific tests
node --test test/fact-graph-conflict.test.js
node --test test/mvcc.test.js
node --test test/mvcc.property.test.js

Design Decisions

In-place commit is safe: JavaScript is single-threaded and synchronous at sink.write(). No concurrent mutation exists.
Kind-driven store selection: The caller declares what a field means ('value', 'content', 'embedding:768'); the system chooses stores, frame types, and read strategies. Never name a store.
Relations are first-class: Relations between entities are declared separately from fields, with named predicates, inverses, and cardinality. They're stored as fact graph facts and expanded on read.
Tree-store path-level dots: The versioned unit is treeId||pathKey, not nodeId, because rearranging the tree is versioned data.
Text-store two-phase commit: Phase 1 checks all ops against committed dots; Phase 2 applies atomically.
MVCC shadow chain over clone: Zero-allocation shadow chain. No clone cost for reads when the chain is empty or the query path is unaffected.
Conflicts are per-key: Only keys that were read AND written trigger conflict checks.
Dual API: Every store exposes both a direct API (store.getFact(), store.describe()) and a frame protocol (sink.write, source.pipe). Both are first-class; the frame protocol enables push-stream composition.
Federated compatible: All stores are { source, sink } duplexes following the push-stream contract.

Tests

npm test

64 tests covering DVVSet conflict detection, lazy MVCC snapshot isolation, transactional commit/rollback/abort, conflict resolution, kind-driven entity projections, streaming cursors, and store-level property tests.

Requirements

Node.js 18 or later. ESM only.

License

MIT

Dependencies

ID	Version
heapify	^0.6.0
porter-stemmer	^0.9.1

Development Dependencies

ID	Version
@rigor/core	*
fast-check	*
tape	*

Peer Dependencies

ID	Version
@polyweave/core	*
@polyweave/durability	*

polyweave state memory store

Details

npm

2026-07-10 20:51:52 +00:00

362

John Dvorak

SEE LICENSE IN LICENSE

236 KiB

Assets (1)

state-1.0.8.tgz 236 KiB

Versions (5) View all

1.0.13

2026-07-11

1.0.10

2026-07-11

1.0.9

2026-07-11

1.0.8

2026-07-10

1.0.3

2026-07-10

@polyweave/state (1.0.8)

Installation

About this package

@polyweave/state

Synopsis

Install

Architecture

Version Vectors (DVVSet)

Conflict Modes

MVCC Snapshot Isolation

Stores (8)

1. Fact Graph

2. Document Store

3. Tree Store

4. Text Store

5. Cluster Tree Store

6. Text Embedding Vector Store

7. Vector Index

8. Multi-State Store

Hybrid Memory + Disk Offload (recommended)

Entity Projections

Defining a Projection

Field Kinds

Relations

CRUD API

Validation

Auto-Store Creation

Store Contract

DESCRIBE Protocol

Transaction Contract

Transaction Protocol

Multi-Store Transactions

Library Modules

Testing

Design Decisions

Tests

Requirements

See Also

License

Dependencies

Keywords