# NEXT_STEPS_423.md — Object Inference, Request Structure & Logic/Invariants ## Executive Summary APOPHIS currently tests routes in isolation with naive resource tracking and no request structure awareness. For Arbiter (11K routes, multi-tenant auth server), we need: 1. **Object Inference**: Schema-driven resource extraction with parent-child relationships 2. **Request Structure**: Path/body/query/header discrimination based on route schemas 3. **Logic/Invariants**: Cross-route temporal assertions, authorization boundaries, state consistency **Timeline**: 2-3 weeks for core implementation, 1 week for Arbiter-specific invariants **Impact**: 10-100x more bugs caught, especially authorization leaks and invalid state transitions --- ## 1. OBJECT INFERENCE ### 1.1 Current State (Naive) **File**: `src/test/petit-runner.ts:205-223` ```typescript const updateState = (command: ApiCommand, ctx: EvalContext, state: ModelState): ModelState => { if (command.route.category !== 'constructor') return state const body = ctx.response.body as Record | undefined if (body === undefined) return state const id = body.id ?? body.uuid ?? body._id if (id === undefined) return state const resourceType = command.route.path.split('/').filter(Boolean).pop() ?? 'resource' // ... stores flat resource } ``` **Problems**: - Only looks for `id`/`uuid`/`_id` in response body - Resource type = last path segment (e.g., `POST /tenant/applications` → type=`applications`) - No parent tracking (application is scoped to tenant) - No nested resource support (e.g., `/tenants/:id/applications/:appId/rules`) - No schema-driven identity field detection ### 1.2 Required: Schema-Driven Resource Extraction **New File**: `src/domain/resource-inference.ts` ```typescript interface ResourceIdentity { resourceType: string id: string parentType?: string parentId?: string tenantId?: string applicationId?: string scope: string | null } interface ResourceSchema { identityField: string // e.g., 'id', 'appId', 'tenantId' parentField?: string // e.g., 'tenantId' for applications identityPattern?: string // e.g., '^[a-z]+-[0-9]+$' scopeFields: string[] // e.g., ['tenantId', 'applicationId'] } export const extractResourceIdentity = ( route: RouteContract, responseBody: unknown, responseSchema?: Record ): ResourceIdentity | null => { // 1. Determine identity field from schema // - Look for field named 'id', 'uuid', '_id', or ending in 'Id' // - Prefer schema.required fields // - Use schema.pattern if available for validation // 2. Determine resource type from route path // - /tenants/:id/applications → type='application', parent='tenant' // - /oauth/token → type='token' (special case) // - /graph/nodes/:nodeId/relations → type='relation', parent='node' // 3. Extract parent from response body (e.g., body.tenantId) // or from route path params (e.g., :id in /tenants/:id/applications) // 4. Extract scope (tenantId, applicationId) from response or request headers } export const inferResourceHierarchy = (path: string): { resourceType: string parentType?: string isNested: boolean } => { const segments = path.split('/').filter(Boolean) // e.g., ['tenants', ':id', 'applications', ':appId'] // resourceType = 'application', parentType = 'tenant' // Special cases: // /oauth/token → resourceType='token', no parent // /graph/nodes/:nodeId/relations → resourceType='relation', parentType='node' // /authz/evaluate → not a resource (utility) } ``` **File to Modify**: `src/types.ts` Add to `ModelState`: ```typescript export interface ResourceHierarchy { readonly id: string readonly type: string readonly parentId?: string readonly parentType?: string readonly scope: { readonly tenantId?: string readonly applicationId?: string } readonly data: unknown readonly createdAt: number } export interface ModelState { readonly resources: ReadonlyMap> readonly counters: ReadonlyMap readonly relationships: ReadonlyMap> } ``` ### 1.3 Arbiter-Specific Resource Types **File**: `src/domain/arbiter-resources.ts` (new) ```typescript // Arbiter has specific resource hierarchies we need to understand export const ARBITER_RESOURCE_PATTERNS = { // Tenant hierarchy 'POST /tenants': { type: 'tenant', identityField: 'id' }, 'POST /tenants/:id/applications': { type: 'application', identityField: 'id', parentType: 'tenant', parentField: 'tenantId' }, 'POST /tenants/:id/users': { type: 'user', identityField: 'id', parentType: 'tenant', parentField: 'tenantId' }, // Graph resources 'POST /graph/nodes': { type: 'node', identityField: 'id' }, 'POST /graph/nodes/:id/relations': { type: 'relation', identityField: 'id', parentType: 'node', parentField: 'sourceId' }, // OAuth tokens 'POST /oauth/token': { type: 'token', identityField: 'access_token' }, 'POST /oauth/refresh': { type: 'token', identityField: 'access_token' }, // Sessions 'POST /sessions': { type: 'session', identityField: 'id' }, // Permissions 'POST /authz/grants': { type: 'grant', identityField: 'id' }, // Rules 'POST /tenants/:id/applications/:appId/rules': { type: 'rule', identityField: 'id', parentType: 'application', parentField: 'applicationId' } } as const export const isArbiterResource = (method: string, path: string): boolean => { const key = `${method} ${path}` return key in ARBITER_RESOURCE_PATTERNS } ``` ### 1.4 Enhanced State Updates **File**: `src/test/petit-runner.ts` — Replace `updateState` and `makeResource` ```typescript const updateState = (command: ApiCommand, ctx: EvalContext, state: ModelState): ModelState => { if (command.route.category !== 'constructor') return state const body = ctx.response.body as Record | undefined if (body === undefined) return state const identity = extractResourceIdentity(command.route, body, command.route.schema?.response as Record) if (!identity) return state const hierarchy: ResourceHierarchy = { id: identity.id, type: identity.resourceType, parentId: identity.parentId, parentType: identity.parentType, scope: { tenantId: identity.tenantId, applicationId: identity.applicationId }, data: body, createdAt: Date.now() } // Store in typed resource map const existing = state.resources.get(identity.resourceType) ?? new Map() const updated = new Map(existing) updated.set(identity.id, hierarchy) const newResources = new Map(state.resources) newResources.set(identity.resourceType, updated) // Track relationships if present let newRelationships = state.relationships if (identity.parentId && identity.parentType) { const rels = state.relationships.get(identity.resourceType) ?? [] newRelationships = new Map(state.relationships) newRelationships.set(identity.resourceType, [ ...rels, { from: identity.id, to: identity.parentId, type: 'childOf' } ]) } return { ...state, resources: newResources, relationships: newRelationships } } ``` ### 1.5 Test Cases **New File**: `src/test/resource-inference.test.ts` ```typescript test('extracts tenant resource from POST /tenants', () => { const route = makeRoute('POST', '/tenants') const body = { id: 'tenant-123', name: 'Acme' } const identity = extractResourceIdentity(route, body) assert.strictEqual(identity?.resourceType, 'tenant') assert.strictEqual(identity?.id, 'tenant-123') assert.strictEqual(identity?.parentType, undefined) }) test('extracts nested application with parent tenant', () => { const route = makeRoute('POST', '/tenants/:id/applications') const body = { id: 'app-456', tenantId: 'tenant-123', name: 'My App' } const identity = extractResourceIdentity(route, body) assert.strictEqual(identity?.resourceType, 'application') assert.strictEqual(identity?.id, 'app-456') assert.strictEqual(identity?.parentType, 'tenant') assert.strictEqual(identity?.parentId, 'tenant-123') }) test('extracts OAuth token with access_token identity', () => { const route = makeRoute('POST', '/oauth/token') const body = { access_token: 'tok-789', token_type: 'Bearer' } const identity = extractResourceIdentity(route, body) assert.strictEqual(identity?.resourceType, 'token') assert.strictEqual(identity?.id, 'tok-789') }) test('returns null for non-resource routes', () => { const route = makeRoute('GET', '/health') const body = { status: 'ok' } const identity = extractResourceIdentity(route, body) assert.strictEqual(identity, null) }) test('uses schema to find identity field', () => { const route = makeRoute('POST', '/custom/resources', { response: { type: 'object', properties: { resourceId: { type: 'string' }, name: { type: 'string' } }, required: ['resourceId'] } }) const body = { resourceId: 'res-999', name: 'Test' } const identity = extractResourceIdentity(route, body, route.schema?.response as Record) assert.strictEqual(identity?.id, 'res-999') }) ``` --- ## 2. REQUEST STRUCTURE INFERENCE ### 2.1 Current State (Blind Parameter Passing) **File**: `src/test/petit-runner.ts:111-168` ```typescript const executeCommand = async (fastify: FastifyInstance, command: ApiCommand): Promise => { const method = command.route.method let url = command.route.path // Replace path params with generated values const params = command.params for (const [key, value] of Object.entries(params)) { if (url.includes(`:${key}`)) { url = url.replace(`:${key}`, String(value)) } } // Everything else goes to query (GET/DELETE) or body (others) const queryParams: Record = {} const bodyParams: Record = {} for (const [key, value] of Object.entries(params)) { if (!command.route.path.includes(`:${key}`)) { if (method === 'GET' || method === 'DELETE') { queryParams[key] = String(value) } else { bodyParams[key] = value } } } // ... } ``` **Problems**: - Assumes ALL non-path params are body params for POST/PUT/PATCH - No understanding of `body.properties` from schema - No handling of nested body structures (`body.nested.field`) - No automatic header injection (x-tenant-id, authorization) - No content-type negotiation - Query params for GET/DELETE are just dumped as-is ### 2.2 Required: Schema-Aware Request Building **New File**: `src/domain/request-builder.ts` ```typescript interface RequestStructure { method: string url: string // With path params replaced headers: Record query?: Record body?: unknown contentType?: string } interface RouteParamSchema { pathParams: string[] // e.g., ['tenantId', 'appId'] bodySchema?: Record querySchema?: Record headerRequirements: string[] // e.g., ['x-tenant-id', 'authorization'] } export const parseRouteParams = (path: string): string[] => { const params: string[] = [] const segments = path.split('/') for (const segment of segments) { if (segment.startsWith(':')) { params.push(segment.slice(1)) } else if (segment.startsWith('{') && segment.endsWith('}')) { params.push(segment.slice(1, -1)) } } return params } export const buildRequest = ( route: RouteContract, generatedData: Record, scopeHeaders: Record, state: ModelState ): RequestStructure => { const pathParams = parseRouteParams(route.path) const url = substitutePathParams(route.path, generatedData, state) // Extract body params from schema const bodySchema = route.schema?.body as Record | undefined const body = bodySchema ? extractBodyParams(generatedData, bodySchema) : undefined // Extract query params from schema const querySchema = route.schema?.querystring as Record | undefined const query = querySchema ? extractQueryParams(generatedData, querySchema) : extractRemainingParams(generatedData, pathParams, body) // Build headers const headers = buildHeaders(route, scopeHeaders, generatedData, state) // Determine content type const contentType = body ? 'application/json' : undefined return { method: route.method, url, headers, query, body, contentType } } const substitutePathParams = ( path: string, data: Record, state: ModelState ): string => { let url = path const pathParams = parseRouteParams(path) for (const param of pathParams) { let value = data[param] // If param is an ID reference, try to find it in state if (value === undefined && param.endsWith('Id')) { const resourceType = param.replace(/Id$/, '') const resources = state.resources.get(resourceType) if (resources && resources.size > 0) { // Pick a random existing resource const ids = Array.from(resources.keys()) value = ids[Math.floor(Math.random() * ids.length)] } } if (value !== undefined) { url = url.replace(`:${param}`, String(value)) } } return url } const extractBodyParams = ( data: Record, bodySchema: Record ): Record => { const properties = bodySchema.properties as Record> | undefined if (!properties) return data const body: Record = {} for (const key of Object.keys(properties)) { if (key in data) { body[key] = data[key] } } // Handle nested objects for (const [key, propSchema] of Object.entries(properties)) { if (propSchema.type === 'object' && propSchema.properties) { body[key] = extractBodyParams(data, propSchema) } } return body } const buildHeaders = ( route: RouteContract, scopeHeaders: Record, data: Record, state: ModelState ): Record => { const headers: Record = { ...scopeHeaders } // Auto-inject tenant ID if route requires it if (route.requires.some(r => r.includes('x-tenant-id'))) { const tenantId = data['tenantId'] || scopeHeaders['x-tenant-id'] if (tenantId) { headers['x-tenant-id'] = String(tenantId) } } // Auto-inject authorization if required if (route.requires.some(r => r.includes('authorization'))) { // Could look up session/token from state const tokens = state.resources.get('token') if (tokens && tokens.size > 0) { const token = Array.from(tokens.values())[0] headers['authorization'] = `Bearer ${token.id}` } } // Content-Type for body requests if (route.schema?.body) { headers['content-type'] = 'application/json' } return headers } ``` ### 2.3 Enhanced Execution **File**: `src/test/petit-runner.ts` — Replace `executeCommand` ```typescript const executeCommand = async ( fastify: FastifyInstance, command: ApiCommand, state: ModelState ): Promise => { const request = buildRequest(command.route, command.params, command.headers, state) // Build query string const queryString = request.query ? Object.entries(request.query) .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`) .join('&') : '' const fullUrl = queryString ? `${request.url}?${queryString}` : request.url const response = await fastify.inject({ method: request.method, url: fullUrl, payload: request.body, headers: request.headers }) return { request: { body: request.body, headers: request.headers, query: request.query || {}, params: extractPathParams(command.route.path, request.url) }, response: { body: response.json(), headers: Object.fromEntries( Object.entries(response.headers).map(([k, v]) => [k, String(v)]) ), statusCode: response.statusCode } } } ``` ### 2.4 Arbiter-Specific Request Patterns **File**: `src/domain/arbiter-requests.ts` (new) ```typescript // Arbiter has specific request structure patterns export const ARBITER_REQUEST_PATTERNS = { 'POST /oauth/token': { bodyFields: ['grant_type', 'code', 'refresh_token', 'client_id', 'client_secret'], headerFields: ['authorization'], contentType: 'application/x-www-form-urlencoded' }, 'POST /oauth/authorize': { queryFields: ['client_id', 'response_type', 'redirect_uri', 'scope', 'state'], headerFields: ['cookie'] }, 'POST /tenant/applications': { bodyFields: ['name', 'slug', 'description', 'callbackURLs', 'allowedOrigins'], headerFields: ['x-tenant-id', 'authorization'], pathParams: [] // No path params, tenant from header }, 'GET /tenant/applications/:appId': { pathParams: ['appId'], headerFields: ['x-tenant-id', 'authorization'] }, 'POST /tenants/:tenantId/applications/:appId/rules': { pathParams: ['tenantId', 'appId'], bodyFields: ['dsl', 'priority', 'enabled'], headerFields: ['x-tenant-id', 'authorization'] }, 'POST /graph/nodes/:nodeId/relations': { pathParams: ['nodeId'], bodyFields: ['targetId', 'relationType', 'metadata'], headerFields: ['x-tenant-id', 'authorization'] }, 'POST /authz/evaluate': { bodyFields: ['userId', 'resourceId', 'permission', 'context'], headerFields: ['x-tenant-id', 'authorization'] } } as const export const getArbiterRequestPattern = (method: string, path: string) => { const key = `${method} ${path}` return ARBITER_REQUEST_PATTERNS[key as keyof typeof ARBITER_REQUEST_PATTERNS] } ``` ### 2.5 Test Cases **New File**: `src/test/request-builder.test.ts` ```typescript test('builds request with path params substituted', () => { const route = makeRoute('GET', '/users/:id') const data = { id: 'user-123' } const request = buildRequest(route, data, {}, emptyState()) assert.strictEqual(request.url, '/users/user-123') }) test('builds request with body from schema', () => { const route = makeRoute('POST', '/users', { body: { type: 'object', properties: { name: { type: 'string' }, email: { type: 'string' } } } }) const data = { name: 'John', email: 'john@example.com', extra: 'ignored' } const request = buildRequest(route, data, {}, emptyState()) assert.deepStrictEqual(request.body, { name: 'John', email: 'john@example.com' }) assert.strictEqual(request.body?.extra, undefined) }) test('injects tenant ID from scope headers', () => { const route = makeRoute('GET', '/tenant/applications', { 'x-requires': ['request_headers(this).x-tenant-id != null'] }) const scopeHeaders = { 'x-tenant-id': 'tenant-123' } const request = buildRequest(route, {}, scopeHeaders, emptyState()) assert.strictEqual(request.headers['x-tenant-id'], 'tenant-123') }) test('looks up path param from state if not in data', () => { const route = makeRoute('GET', '/users/:userId') const state = stateWithResource('user', 'user-456', {}) const request = buildRequest(route, {}, {}, state) assert.strictEqual(request.url, '/users/user-456') }) test('handles OAuth token request with form encoding', () => { const route = makeRoute('POST', '/oauth/token') const data = { grant_type: 'authorization_code', code: 'auth-code-123', client_id: 'client-456' } const request = buildRequest(route, data, {}, emptyState()) assert.strictEqual(request.headers['content-type'], 'application/x-www-form-urlencoded') assert.strictEqual(request.body, undefined) // Form data handled differently }) ``` --- ## 3. LOGIC & INVARIANTS ### 3.1 Current State (Status Code Only) **File**: `src/test/petit-runner.ts:186-199` ```typescript const checkPostconditions = (command: ApiCommand, ctx: EvalContext): EvalResult => { for (const ensure of command.route.ensures) { if (ensure.startsWith('status:')) { const expected = parseInt(ensure.replace('status:', ''), 10) if (ctx.response.statusCode !== expected) { return { success: false, error: `Expected status ${expected}, got ${ctx.response.statusCode}` } } } } return { success: true, value: ctx.response.statusCode } } ``` **Problems**: - Only checks `status:###` patterns - Ignores actual APOSTL formulas in `x-ensures` - No cross-route invariant checking - No temporal logic (what happens after state changes) - No authorization boundary verification ### 3.2 Required: Full Formula Evaluation **File**: `src/test/petit-runner.ts` — Replace `checkPostconditions` ```typescript import { parse } from '../formula/parser.js' import { evaluateBooleanResult } from '../formula/evaluator.js' const checkPostconditions = (command: ApiCommand, ctx: EvalContext): EvalResult => { for (const ensure of command.route.ensures) { // Legacy status check if (ensure.startsWith('status:')) { const expected = parseInt(ensure.replace('status:', ''), 10) if (ctx.response.statusCode !== expected) { return { success: false, error: `Expected status ${expected}, got ${ctx.response.statusCode}` } } continue } // Full APOSTL formula evaluation try { const ast = parse(ensure) const result = evaluateBooleanResult(ast.ast, ctx) if (!result) { return { success: false, error: `Contract violation: ${ensure}` } } } catch (err) { return { success: false, error: `Formula error in "${ensure}": ${err instanceof Error ? err.message : String(err)}` } } } return { success: true, value: ctx.response.statusCode } } ``` ### 3.3 Required: Cross-Route Invariant Registry **New File**: `src/domain/invariant-registry.ts` ```typescript interface Invariant { readonly name: string readonly description: string readonly check: (state: ModelState, history: ReadonlyArray) => InvariantResult } interface InvariantResult { readonly success: boolean readonly error?: string } // Built-in invariants export const BUILTIN_INVARIANTS: Invariant[] = [ { name: 'resource-consistency', description: 'Created resources must be retrievable', check: (state, history) => { // For each constructor in history, check that GET returns same data const constructors = history.filter((ctx, i) => { // Would need to track which history entries are constructors // This requires enhancing the history tracking return false }) return { success: true } } }, { name: 'tenant-isolation', description: 'Resources from tenant A must not be accessible in tenant B', check: (state) => { for (const [type, resources] of state.resources) { for (const [id, resource] of resources) { if (resource.scope.tenantId) { // Check no other tenant has access // Would need to simulate cross-tenant requests } } } return { success: true } } }, { name: 'authorization-transitivity', description: 'If parent has permission, child must inherit it', check: (state) => { // For graph relations: if node A -> node B, then B inherits A's permissions const relations = state.relationships.get('relation') || [] // Would need to check authorization evaluations return { success: true } } } ] // Arbiter-specific invariants export const ARBITER_INVARIANTS: Invariant[] = [ { name: 'oauth-token-validity', description: 'Issued tokens must be valid until revoked', check: (state, history) => { const tokens = state.resources.get('token') || new Map() // Check each token was issued properly and hasn't expired return { success: true } } }, { name: 'session-consistency', description: 'Active sessions must have valid users', check: (state) => { const sessions = state.resources.get('session') || new Map() const users = state.resources.get('user') || new Map() for (const [sessionId, session] of sessions) { const userId = (session.data as Record)?.userId if (userId && !users.has(String(userId))) { return { success: false, error: `Session ${sessionId} references non-existent user ${userId}` } } } return { success: true } } }, { name: 'permission-graph-acyclic', description: 'Permission inheritance graph must not have cycles', check: (state) => { // Detect cycles in relation graph const relations = state.relationships.get('relation') || [] // Run cycle detection algorithm return { success: true } } }, { name: 'tenant-application-consistency', description: 'Applications must belong to existing tenants', check: (state) => { const tenants = state.resources.get('tenant') || new Map() const applications = state.resources.get('application') || new Map() for (const [appId, app] of applications) { const tenantId = app.parentId if (tenantId && !tenants.has(tenantId)) { return { success: false, error: `Application ${appId} references non-existent tenant ${tenantId}` } } } return { success: true } } } ] export const checkInvariants = ( invariants: ReadonlyArray, state: ModelState, history: ReadonlyArray ): Array<{ name: string; result: InvariantResult }> => { return invariants.map(inv => ({ name: inv.name, result: inv.check(state, history) })) } ``` ### 3.4 Required: Temporal Logic in APOSTL **File**: `src/formula/parser.ts` — Extend grammar Add temporal operators: ```typescript // New node types for temporal logic export type FormulaNode = | ...existing nodes... | { type: 'temporal'; operator: 'eventually' | 'always' | 'until'; body: FormulaNode } | { type: 'previous'; inner: FormulaNode } ``` **File**: `src/formula/evaluator.ts` — Extend evaluation ```typescript // Temporal evaluation requires history context function evaluateTemporal( operator: string, body: FormulaNode, ctx: EvalContext, history: ReadonlyArray ): boolean { switch (operator) { case 'eventually': // True if body is true at some point in future // For testing: check if body will be true after some operation return true // Placeholder case 'always': // True if body is true at all points return history.every(h => evaluateNode(body, h)) case 'until': // True if left is true until right becomes true // Requires binary temporal operator return true // Placeholder default: throw new Error(`Unknown temporal operator: ${operator}`) } } ``` ### 3.5 Required: Stateful Test Runner Enhancement **File**: `src/test/petit-runner.ts` — Add invariant checking to main loop ```typescript export const runPetitTests = async ( fastify: FastifyInstance, config: TestConfig ): Promise => { const startTime = Date.now() const depth = DEPTH_CONFIGS[config.depth] const allRoutes = discoverRoutes(fastify) const filtered = filterByMode(allRoutes, config.mode) const routes = sortByCategory(filtered) const { commands: commandGroups, cacheHits, cacheMisses } = generateCommands(routes, depth, config.seed) const allCommands = commandGroups.flat() let state: ModelState = { resources: new Map(), counters: new Map(), relationships: new Map() } const resources: TrackedResource[] = [] const results: TestResult[] = [] const history: EvalContext[] = [] // Track all request/response contexts let testId = 0 for (const command of allCommands) { testId++ const name = `${command.route.method} ${command.route.path} (#${testId})` // Check preconditions const preOk = checkPreconditions(command, state) if (!preOk) { results.push({ ok: true, name, id: testId, directive: 'SKIP preconditions not met' }) continue } // Execute command let ctx: EvalContext try { ctx = await executeCommand(fastify, command, state) history.push(ctx) } catch (err) { results.push({ ok: false, name, id: testId, diagnostics: { error: err instanceof Error ? err.message : String(err) } }) continue } // Check postconditions const post = checkPostconditions(command, ctx) if (!post.success) { results.push({ ok: false, name, id: testId, diagnostics: { error: post.error, statusCode: ctx.response.statusCode } }) } else { results.push({ ok: true, name, id: testId }) } // Update state state = updateState(command, ctx, state) // Check invariants after state change const invariantResults = checkInvariants(ARBITER_INVARIANTS, state, history) for (const inv of invariantResults) { if (!inv.result.success) { results.push({ ok: false, name: `INVARIANT: ${inv.name} (#${testId})`, id: testId, diagnostics: { error: inv.result.error } }) } } const resource = makeResource(command, ctx) if (resource !== null) { resources.push(resource) } } // Final invariant check const finalInvariantResults = checkInvariants(BUILTIN_INVARIANTS, state, history) for (const inv of finalInvariantResults) { if (!inv.result.success) { results.push({ ok: false, name: `FINAL INVARIANT: ${inv.name}`, id: testId + 1, diagnostics: { error: inv.result.error } }) } } const passed = results.filter((r) => r.ok && r.directive === undefined).length const failed = results.filter((r) => !r.ok).length const skipped = results.filter((r) => r.directive !== undefined).length return { version: 13, plan: { start: 1, end: results.length }, tests: results, summary: { passed, failed, skipped, timeMs: Date.now() - startTime, cacheHits, cacheMisses } } } ``` ### 3.6 Arbiter-Specific Contract Examples **Example contracts for Arbiter routes**: ```javascript // Tenant isolation { 'x-requires': [ 'request_headers(this).x-tenant-id != null', 'request_headers(this).authorization matches "^Bearer .+"' ], 'x-ensures': [ 'response_code(this) == 200', 'response_body(this).tenantId == request_headers(this).x-tenant-id', 'if request_headers(this).x-tenant-id != response_body(this).tenantId then response_code(this) == 403 else T' ], 'x-invariants': [ 'response_body(this).resources.all(r => r.tenantId == request_headers(this).x-tenant-id)' ] } // OAuth authorization { 'x-requires': [ 'query_params(this).client_id != null', 'query_params(this).response_type == "code"', 'request_headers(this).cookie != null' ], 'x-ensures': [ 'response_code(this) == 302', 'response_headers(this).location matches "\\?code="' ] } // Token issuance { 'x-requires': [ 'request_body(this).grant_type in ["authorization_code", "refresh_token"]', 'request_body(this).code != null || request_body(this).refresh_token != null' ], 'x-ensures': [ 'response_code(this) == 200', 'response_body(this).access_token != null', 'response_body(this).token_type == "Bearer"', 'response_body(this).expires_in > 0' ] } // Graph authorization evaluation { 'x-requires': [ 'request_body(this).userId != null', 'request_body(this).resourceId != null', 'request_body(this).permission != null' ], 'x-ensures': [ 'response_code(this) == 200', 'response_body(this).allowed == true || response_body(this).allowed == false', 'if response_body(this).allowed == true then response_body(this).reason != null else T' ] } // Rule creation { 'x-requires': [ 'request_headers(this).x-tenant-id != null', 'request_body(this).dsl != null', 'request_body(this).priority >= 0' ], 'x-ensures': [ 'response_code(this) == 201', 'response_body(this).id != null', 'response_body(this).dsl == request_body(this).dsl', 'response_body(this).tenantId == request_headers(this).x-tenant-id' ] } ``` --- ## 4. IMPLEMENTATION ROADMAP ### Week 1: Object Inference & Request Structure **Day 1-2**: Resource Inference - [ ] Create `src/domain/resource-inference.ts` - [ ] Implement `extractResourceIdentity()` - [ ] Implement `inferResourceHierarchy()` - [ ] Add `ResourceHierarchy` to types - [ ] Write tests (10+ cases) **Day 3-4**: Request Builder - [ ] Create `src/domain/request-builder.ts` - [ ] Implement `buildRequest()` - [ ] Implement `substitutePathParams()` - [ ] Implement `extractBodyParams()` - [ ] Implement `buildHeaders()` - [ ] Write tests (10+ cases) **Day 5**: Integration - [ ] Update `petit-runner.ts` to use new modules - [ ] Update `executeCommand()` signature - [ ] Update `updateState()` for hierarchies - [ ] Run full test suite - [ ] Fix regressions ### Week 2: Logic & Invariants **Day 1-2**: Formula Evaluation in Tests - [ ] Update `checkPostconditions()` to use formula evaluator - [ ] Handle parse errors gracefully - [ ] Add formula error diagnostics - [ ] Write tests for formula-based postconditions **Day 3-4**: Invariant Registry - [ ] Create `src/domain/invariant-registry.ts` - [ ] Implement builtin invariants - [ ] Implement Arbiter-specific invariants - [ ] Add invariant checking to test runner loop - [ ] Write tests for invariant detection **Day 5**: Temporal Logic (Basic) - [ ] Add `previous()` operator support - [ ] Implement basic history tracking - [ ] Add `always` temporal operator - [ ] Write tests for temporal assertions ### Week 3: Stateful Testing & Polish **Day 1-2**: Command Sequences - [ ] Implement fast-check `commands()` arbitrary - [ ] Generate valid command sequences respecting preconditions - [ ] Add sequence shrinkers - [ ] Write property tests for sequences **Day 3-4**: Arbiter Integration - [ ] Create Arbiter-specific resource patterns - [ ] Create Arbiter request patterns - [ ] Add Arbiter-specific invariants - [ ] Test against Arbiter route samples **Day 5**: Documentation & Performance - [ ] Update README with advanced examples - [ ] Update SKILL.md with new patterns - [ ] Profile performance with 1000+ routes - [ ] Optimize hot paths - [ ] Final test suite: 250+ tests --- ## 5. TEST COVERAGE TARGETS ### Current: 198 tests ### Target after implementation: - **Resource inference**: +30 tests - **Request builder**: +25 tests - **Formula evaluation in tests**: +20 tests - **Invariant registry**: +25 tests - **Temporal logic**: +15 tests - **Stateful sequences**: +20 tests - **Arbiter-specific**: +20 tests - **Total**: ~353 tests --- ## 6. FILES TO CREATE/MODIFY ### New Files ``` src/domain/resource-inference.ts src/domain/request-builder.ts src/domain/invariant-registry.ts src/domain/arbiter-resources.ts src/domain/arbiter-requests.ts src/test/resource-inference.test.ts src/test/request-builder.test.ts src/test/invariant-registry.test.ts src/test/temporal-logic.test.ts ``` ### Modified Files ``` src/types.ts — Add ResourceHierarchy, update ModelState src/test/petit-runner.ts — Use resource inference, request builder, invariants src/formula/parser.ts — Add temporal operators (if needed) src/formula/evaluator.ts — Add history context support src/domain/category.ts — Add Arbiter-specific category rules ``` --- ## 7. ACCEPTANCE CRITERIA ### Object Inference - [ ] Can extract resource identity from response body using schema - [ ] Can determine parent-child relationships from route paths - [ ] Can handle OAuth tokens, sessions, graph nodes/relations - [ ] Returns null for non-resource routes (health, utility) - [ ] All Arbiter resource types are correctly identified ### Request Structure - [ ] Path params are substituted from generated data or state - [ ] Body params match schema properties (no extra fields) - [ ] Query params are extracted for GET/DELETE - [ ] Headers include x-tenant-id from scope - [ ] Authorization header is injected when required - [ ] Content-Type is set correctly (JSON vs form-encoded) ### Logic & Invariants - [ ] APOSTL formulas in x-ensures are evaluated (not just status:###) - [ ] Cross-route invariants are checked after each state change - [ ] Tenant isolation is verified - [ ] Resource consistency is verified (created resources are retrievable) - [ ] Authorization transitivity is checked - [ ] Session consistency is verified - [ ] Temporal operators work (previous, always) ### Performance - [ ] < 2s overhead for 1000 routes - [ ] < 5s overhead for 10,000 routes - [ ] Incremental cache still provides 10x+ speedup - [ ] Memory usage < 500MB for full Arbiter test run --- ## 8. RISKS & MITIGATIONS | Risk | Impact | Mitigation | |------|--------|------------| | Formula evaluation too slow | High | Cache parsed ASTs, use WeakMap | | Invariant checking O(n²) | Medium | Batch checks, use Set lookups | | Memory leak in history | Medium | Limit history size, use ring buffer | | Arbiter routes lack schemas | High | Fallback to path-based inference | | Fastify v5 compatibility | Low | Already using inject() API | | Breaking changes to existing tests | Medium | Maintain backward compatibility for status:### | --- **Next Action**: Start with `src/domain/resource-inference.ts` — this is the foundation everything else builds on.