core/docs/internal/ast-README.md

AST Module - DSL Compiler for Zanzibar-Graph

This module provides a complete Abstract Syntax Tree (AST) system for parsing and compiling the Evidence DSL into rule configurations that interface with the zanzibar-graph authorization system.

Overview

The AST module consists of three main components:

1. Parser - Converts DSL text into AST nodes
2. Generator - Converts AST nodes into rule configurations
3. Compiler - Orchestrates the parsing and generation process

Architecture

DSL Text → Parser → AST Nodes → Generator → Rule Configs → setRelationConfig()

Features

• Complete DSL Support - Supports all DSL constructs from the specification
• Modular Design - Clean separation of concerns with pluggable components
• Error Handling - Comprehensive error reporting and validation
• Program Management - Support for multiple compiled programs
• Rule Generation - Automatic conversion to existing rule system
• Extensible - Easy to add new node types and generators

Quick Start

import { DSLCompiler } from './src/ast/index.js';

// Create compiler with arbiter instance
const compiler = new DSLCompiler(arbiter);

// Compile DSL text
const result = compiler.compile(dslText, 'my-program');

if (result.success) {
  console.log(`Generated ${result.generatedCount} rules`);
} else {
  console.error('Compilation errors:', result.errors);
}

Core Components

1. AST Nodes (/nodes/)

The AST nodes represent the parsed structure of the DSL:

• BaseNode - Base class for all AST nodes
• ProgramNode - Root node containing all definitions
• DefinitionNode - Type definitions with fields and behaviors
• FactNode - Fact definitions with parameters and properties
• EvidenceNode - Evidence definitions with bodies
• MeasureNode - Measure definitions for value collection
• ExpressionNode - Expressions, variables, and literals
• And many more...

2. Parser (/parser/)

The DSL parser converts text into AST nodes:

import { DSLParser } from './parser/DSLParser.js';

const parser = new DSLParser();
const program = parser.parse(dslText);

3. Generator (/generator/)

The rule generator converts AST nodes into rule configurations:

import { RuleGenerator } from './generator/RuleGenerator.js';

const generator = new RuleGenerator(arbiter);
const result = generator.generateRules(program);

4. Compiler (DSLCompiler.js)

The main compiler orchestrates the entire process:

import { DSLCompiler } from './DSLCompiler.js';

const compiler = new DSLCompiler(arbiter);
const result = compiler.compile(dslText, 'program-name');

Usage Examples

Basic Compilation

const dsl = `
  definition User {
    role: string
    isActive: boolean
  }
  
  fact hasRole(user: User, role: string) CACHE eager
  
  evidence canRead(user: User, doc: Document) {
    hasRole(user, 'admin')
  }
`;

const result = compiler.compile(dsl, 'auth');

Complex DSL with Defeasible Logic

const complexDSL = `
  evidence canAccessCritical(user: User, resource: Resource) {
    // Strict requirement
    ALWAYS user.isActive
    
    // Defeasible access
    WHEN hasRole(user, 'admin') UNLESS isSuspended(user)
    
    // Requirements
    REQUIRES hasClearance(user, resource.level)
    
    // Fusion evidence
    fusion majority {
      user.isTrusted
      user.hasRecentActivity
    }
  }
`;

const result = compiler.compile(complexDSL, 'critical-access');

Multiple Programs

const programs = {
  'auth': `
    fact hasRole(user: User, role: string)
    evidence canRead(user: User, doc: Document) {
      hasRole(user, 'admin')
    }
  `,
  'finance': `
    fact hasBalance(user: User, amount: number)
    evidence canWithdraw(user: User, amount: number) {
      hasBalance(user, amount)
    }
  `
};

const result = compiler.compileMultiple(programs);

API Reference

DSLCompiler

Methods

• compile(dslText, programName) - Compile DSL text into rules
• compileMultiple(programs) - Compile multiple programs
• validate(dslText) - Validate DSL without compilation
• getCompiledProgram(name) - Get compiled program by name
• getAllCompiledPrograms() - Get all compiled programs
• removeCompiledProgram(name) - Remove compiled program
• clearCompiledPrograms() - Clear all programs
• getCompilationStats() - Get compilation statistics

Properties

• arbiter - The arbiter instance
• parser - The DSL parser
• generator - The rule generator

DSLParser

Methods

• parse(dslText) - Parse DSL text into AST
• getErrors() - Get parser errors

RuleGenerator

Methods

• generateRules(program) - Generate rules from AST
• getErrors() - Get generator errors
• getGeneratedRules() - Get generated rules

Supported DSL Constructs

Type Definitions

definition User {
  role: string
  isActive: boolean
  lastActive: timestamp BEHAVES {
    decaying down hourly
  } CACHE lazy
}

Facts

fact hasRole(user: User, role: string) CACHE eager
fact isMember(user: User, group: Group) transitive CACHE lazy

Evidence

evidence canRead(user: User, doc: Document) {
  // Direct evidence
  owns(user, doc)
  
  // Pattern matching
  isMember(user, *group) {
    canRead(group, doc)
  } limit 5
  
  // Defeasible logic
  WHEN hasRole(user, 'admin') UNLESS isSuspended(user)
}

Measures

measure userBalance(user: User) {
  user.balance
} PROVIDES number

measure userPermissions(user: User) {
  fusion max {
    user.role.permissions
    user.group.permissions
  }
} PROVIDES Permission[]

Error Handling

The compiler provides comprehensive error handling:

const result = compiler.compile(dslText);

if (!result.success) {
  console.error('Compilation failed:');
  result.errors.forEach(error => console.error(`  - ${error}`));
}

if (result.warnings.length > 0) {
  console.warn('Warnings:');
  result.warnings.forEach(warning => console.warn(`  - ${warning}`));
}

Testing

Run the test suite:

import { runDSLCompilerTests } from './tests/DSLCompiler.test.js';

const testResults = runDSLCompilerTests(arbiter);
console.log('Test Results:', testResults);

Examples

See the examples directory for comprehensive usage examples:

• examples/DSLExample.js - Complete usage examples
• tests/DSLCompiler.test.js - Test suite

Integration with Existing System

The AST module integrates seamlessly with the existing zanzibar-graph system:

1. Parser converts DSL text to AST nodes
2. Generator converts AST nodes to rule configurations
3. Compiler applies rules via arbiter.setRelationConfig()

The generated rules are compatible with all existing rule types:

• Direct rules
• Computed rules
• Parent rules
• Tuple-to-userset rules
• Similarity rules
• Multi-hop rules
• Logical operators
• Defeasible logic

Extensibility

The AST system is designed to be extensible:

1. Add new node types by extending BaseNode
2. Add new parsers by extending DSLParser
3. Add new generators by extending RuleGenerator
4. Add new compilers by extending DSLCompiler

Performance Considerations

• Lazy evaluation - AST nodes are created on demand
• Efficient parsing - Token-based parsing with minimal memory usage
• Rule caching - Generated rules are cached for reuse
• Batch processing - Support for compiling multiple programs at once

Future Enhancements

• Incremental compilation - Only recompile changed parts
• Parallel processing - Compile multiple programs in parallel
• Advanced optimizations - Rule optimization and simplification
• IDE support - Language server protocol support
• Visualization - AST visualization tools

Contributing

When contributing to the AST module:

1. Follow the existing code structure
2. Add comprehensive tests for new features
3. Update documentation
4. Ensure backward compatibility
5. Follow the established patterns

License

This module is part of the zanzibar-graph project and follows the same license terms.