core/docs/internal/ast-README.md

# AST Module - DSL Compiler for Zanzibar-Graph

This module provides a complete Abstract Syntax Tree (AST) system for parsing and compiling the Evidence DSL into rule configurations that interface with the zanzibar-graph authorization system.

## Overview

The AST module consists of three main components:

1. **Parser** - Converts DSL text into AST nodes
2. **Generator** - Converts AST nodes into rule configurations
3. **Compiler** - Orchestrates the parsing and generation process

## Architecture

```
DSL Text → Parser → AST Nodes → Generator → Rule Configs → setRelationConfig()
```

## Features

- **Complete DSL Support** - Supports all DSL constructs from the specification
- **Modular Design** - Clean separation of concerns with pluggable components
- **Error Handling** - Comprehensive error reporting and validation
- **Program Management** - Support for multiple compiled programs
- **Rule Generation** - Automatic conversion to existing rule system
- **Extensible** - Easy to add new node types and generators

## Quick Start

```javascript
import { DSLCompiler } from './src/ast/index.js';

// Create compiler with arbiter instance
const compiler = new DSLCompiler(arbiter);

// Compile DSL text
const result = compiler.compile(dslText, 'my-program');

if (result.success) {
  console.log(`Generated ${result.generatedCount} rules`);
} else {
  console.error('Compilation errors:', result.errors);
}
```

## Core Components

### 1. AST Nodes (`/nodes/`)

The AST nodes represent the parsed structure of the DSL:

- **BaseNode** - Base class for all AST nodes
- **ProgramNode** - Root node containing all definitions
- **DefinitionNode** - Type definitions with fields and behaviors
- **FactNode** - Fact definitions with parameters and properties
- **EvidenceNode** - Evidence definitions with bodies
- **MeasureNode** - Measure definitions for value collection
- **ExpressionNode** - Expressions, variables, and literals
- **And many more...**

### 2. Parser (`/parser/`)

The DSL parser converts text into AST nodes:

```javascript
import { DSLParser } from './parser/DSLParser.js';

const parser = new DSLParser();
const program = parser.parse(dslText);
```

### 3. Generator (`/generator/`)

The rule generator converts AST nodes into rule configurations:

```javascript
import { RuleGenerator } from './generator/RuleGenerator.js';

const generator = new RuleGenerator(arbiter);
const result = generator.generateRules(program);
```

### 4. Compiler (`DSLCompiler.js`)

The main compiler orchestrates the entire process:

```javascript
import { DSLCompiler } from './DSLCompiler.js';

const compiler = new DSLCompiler(arbiter);
const result = compiler.compile(dslText, 'program-name');
```

## Usage Examples

### Basic Compilation

```javascript
const dsl = `
  definition User {
    role: string
    isActive: boolean
  }
  
  fact hasRole(user: User, role: string) CACHE eager
  
  evidence canRead(user: User, doc: Document) {
    hasRole(user, 'admin')
  }
`;

const result = compiler.compile(dsl, 'auth');
```

### Complex DSL with Defeasible Logic

```javascript
const complexDSL = `
  evidence canAccessCritical(user: User, resource: Resource) {
    // Strict requirement
    ALWAYS user.isActive
    
    // Defeasible access
    WHEN hasRole(user, 'admin') UNLESS isSuspended(user)
    
    // Requirements
    REQUIRES hasClearance(user, resource.level)
    
    // Fusion evidence
    fusion majority {
      user.isTrusted
      user.hasRecentActivity
    }
  }
`;

const result = compiler.compile(complexDSL, 'critical-access');
```

### Multiple Programs

```javascript
const programs = {
  'auth': `
    fact hasRole(user: User, role: string)
    evidence canRead(user: User, doc: Document) {
      hasRole(user, 'admin')
    }
  `,
  'finance': `
    fact hasBalance(user: User, amount: number)
    evidence canWithdraw(user: User, amount: number) {
      hasBalance(user, amount)
    }
  `
};

const result = compiler.compileMultiple(programs);
```

## API Reference

### DSLCompiler

#### Methods

- `compile(dslText, programName)` - Compile DSL text into rules
- `compileMultiple(programs)` - Compile multiple programs
- `validate(dslText)` - Validate DSL without compilation
- `getCompiledProgram(name)` - Get compiled program by name
- `getAllCompiledPrograms()` - Get all compiled programs
- `removeCompiledProgram(name)` - Remove compiled program
- `clearCompiledPrograms()` - Clear all programs
- `getCompilationStats()` - Get compilation statistics

#### Properties

- `arbiter` - The arbiter instance
- `parser` - The DSL parser
- `generator` - The rule generator

### DSLParser

#### Methods

- `parse(dslText)` - Parse DSL text into AST
- `getErrors()` - Get parser errors

### RuleGenerator

#### Methods

- `generateRules(program)` - Generate rules from AST
- `getErrors()` - Get generator errors
- `getGeneratedRules()` - Get generated rules

## Supported DSL Constructs

### Type Definitions

```typescript
definition User {
  role: string
  isActive: boolean
  lastActive: timestamp BEHAVES {
    decaying down hourly
  } CACHE lazy
}
```

### Facts

```typescript
fact hasRole(user: User, role: string) CACHE eager
fact isMember(user: User, group: Group) transitive CACHE lazy
```

### Evidence

```typescript
evidence canRead(user: User, doc: Document) {
  // Direct evidence
  owns(user, doc)
  
  // Pattern matching
  isMember(user, *group) {
    canRead(group, doc)
  } limit 5
  
  // Defeasible logic
  WHEN hasRole(user, 'admin') UNLESS isSuspended(user)
}
```

### Measures

```typescript
measure userBalance(user: User) {
  user.balance
} PROVIDES number

measure userPermissions(user: User) {
  fusion max {
    user.role.permissions
    user.group.permissions
  }
} PROVIDES Permission[]
```

## Error Handling

The compiler provides comprehensive error handling:

```javascript
const result = compiler.compile(dslText);

if (!result.success) {
  console.error('Compilation failed:');
  result.errors.forEach(error => console.error(`  - ${error}`));
}

if (result.warnings.length > 0) {
  console.warn('Warnings:');
  result.warnings.forEach(warning => console.warn(`  - ${warning}`));
}
```

## Testing

Run the test suite:

```javascript
import { runDSLCompilerTests } from './tests/DSLCompiler.test.js';

const testResults = runDSLCompilerTests(arbiter);
console.log('Test Results:', testResults);
```

## Examples

See the examples directory for comprehensive usage examples:

- `examples/DSLExample.js` - Complete usage examples
- `tests/DSLCompiler.test.js` - Test suite

## Integration with Existing System

The AST module integrates seamlessly with the existing zanzibar-graph system:

1. **Parser** converts DSL text to AST nodes
2. **Generator** converts AST nodes to rule configurations
3. **Compiler** applies rules via `arbiter.setRelationConfig()`

The generated rules are compatible with all existing rule types:
- Direct rules
- Computed rules
- Parent rules
- Tuple-to-userset rules
- Similarity rules
- Multi-hop rules
- Logical operators
- Defeasible logic

## Extensibility

The AST system is designed to be extensible:

1. **Add new node types** by extending `BaseNode`
2. **Add new parsers** by extending `DSLParser`
3. **Add new generators** by extending `RuleGenerator`
4. **Add new compilers** by extending `DSLCompiler`

## Performance Considerations

- **Lazy evaluation** - AST nodes are created on demand
- **Efficient parsing** - Token-based parsing with minimal memory usage
- **Rule caching** - Generated rules are cached for reuse
- **Batch processing** - Support for compiling multiple programs at once

## Future Enhancements

- **Incremental compilation** - Only recompile changed parts
- **Parallel processing** - Compile multiple programs in parallel
- **Advanced optimizations** - Rule optimization and simplification
- **IDE support** - Language server protocol support
- **Visualization** - AST visualization tools

## Contributing

When contributing to the AST module:

1. Follow the existing code structure
2. Add comprehensive tests for new features
3. Update documentation
4. Ensure backward compatibility
5. Follow the established patterns

## License

This module is part of the zanzibar-graph project and follows the same license terms.