apophis-fastify/docs/extensions/EXTENSION-ARCHITECTURE.md

APOPHIS v1.1 Architecture — Hybrid Core + Extensions

Status: Architecture Specification

Date: 2026-04-24

Scope: v1.1 First-Class Features & Extension Ecosystem

---

1. Philosophy: Core HTTP vs Extensions

First-class: Standard HTTP features that require deep integration with APOPHIS core:

• Schema-to-arbitrary integration (teaching fast-check to generate custom data)
• Request builder integration (constructing specialized payloads)
• HTTP executor integration (handling specialized responses)
• APOSTL parser/evaluator integration (new operations)

Extensions: Specialized protocols or features with heavy dependencies that should be opt-in:

• Different protocols (WebSockets, not HTTP)
• Heavy dependencies (Protobuf, MessagePack)
• Protocol-specific features such as SSE

This split keeps common HTTP testing in core while moving specialized protocols out of the default path.

---

2. First-Class Features (v1.1 Core)

2.1 Multipart File Uploads

Module: Core — src/infrastructure/multipart.ts, src/domain/multipart-generator.ts

Schema Annotations:

schema: {
  body: {
    type: 'object',
    'x-content-type': 'multipart/form-data',
    'x-multipart-fields': {
      description: { type: 'string', maxLength: 500 }
    },
    'x-multipart-files': {
      avatar: {
        maxSize: 5 * 1024 * 1024,
        mimeTypes: ['image/jpeg', 'image/png'],
        maxCount: 1
      }
    }
  }
}

APOSTL Operations:

// request_files(this).avatar.count == 1
// request_files(this).avatar.size <= 5242880
// request_files(this).avatar.mimetype matches "image/(jpeg|png)"
// request_fields(this).description != null

Core Integration Points:

1. Schema-to-arbitrary: Detect x-content-type: multipart/form-data, generate { fields: {...}, files: [...] }
2. Request builder: Convert generated data to multipart payload on RequestStructure
3. HTTP executor: Build FormData from request.multipart, inject via Fastify
4. Parser: Add request_files, request_fields to VALID_HEADERS
5. Evaluator: Add multipart operations to resolveOperation

2.2 Streaming / NDJSON

Module: Core — src/infrastructure/stream-collector.ts

Schema Annotations:

schema: {
  response: {
    200: {
      type: 'object',
      'x-streaming': true,
      'x-stream-format': 'ndjson',
      'x-stream-max-chunks': 100,
      'x-stream-timeout': 5000
    }
  }
}

APOSTL Operations:

// response_body(this) — array of parsed chunks
// stream_chunks(this) — alias for response_body(this)
// stream_duration(this) — total stream time in ms

Core Integration Points:

1. Contract extraction: Extract x-streaming, x-stream-format, x-stream-max-chunks, x-stream-timeout
2. HTTP executor: After inject, check if route has streaming config. If so:
   • Read response payload as string
   • Split by \n
   • JSON.parse each line (for NDJSON)
   • Respect maxChunks and timeoutMs
   • Store result in EvalContext.response.body and EvalContext.response.chunks
3. Parser: Add stream_chunks, stream_duration to VALID_HEADERS
4. Evaluator: Add streaming operations to resolveOperation

---

3. Extension System (v1.1+ Ecosystem)

The extension system handles features that don't require core HTTP integration.

3.1 Extension Interface

export interface ApophisExtension {
  /** Unique name. Used for state isolation and error attribution. */
  name: string
  
  /** APOSTL headers this extension adds. Used for parser validation. */
  headers?: string[]
  
  /** APOSTL predicates exposed by this extension. */
  predicates?: Record<string, PredicateResolver>
  
  /** Lifecycle hooks. */
  hooks?: {
    onBuildRequest?: Hook<RequestBuildContext, void>
    onBeforeRequest?: Hook<ExecutionContext, void>
    onAfterRequest?: Hook<ExecutionContext, void>
    onSuiteStart?: Hook<{ routes: RouteContract[] }, void>
    onSuiteEnd?: Hook<{ summary: TestSummary }, void>
    onViolation?: Hook<{ violation: ContractViolation }, void>
  }
  
  /** Severity: 'fatal' (block test), 'warn' (log, don't block). Default: 'fatal'. */
  severity?: 'fatal' | 'warn'
  
  /** Redaction: fields to mask in violation output. */
  redactFields?: string[]
  
  /** Initial state for this extension. Passed to hooks/predicates. */
  state?: Record<string, unknown>
}

3.2 Extension Registration

await fastify.register(apophis, {
  extensions: [
    sseExtension,
    createSerializerExtension(mySerializerRegistry),
    websocketExtension,
  ]
})

3.3 Extensions Available

SSE Extension

Module: src/extensions/sse/

export const sseExtension: ApophisExtension = {
  name: 'sse',
  headers: ['sse_events'],
  predicates: {
    sse_events: (ctx) => {
      const events = ctx.evalContext.response.sseEvents ?? []
      if (ctx.accessor.length === 0) return { value: events, success: true }
      
      const idx = parseInt(ctx.accessor[0], 10)
      const event = events[idx]
      if (!event) return { value: null, success: true }
      
      if (ctx.accessor[1] === 'event') return { value: event.event, success: true }
      if (ctx.accessor[1] === 'data') return { value: event.data, success: true }
      if (ctx.accessor[1] === 'id') return { value: event.id, success: true }
      if (ctx.accessor[1] === 'retry') return { value: event.retry, success: true }
      
      return { value: event, success: true }
    }
  }
}

Serializers Extension

Module: src/extensions/serializers/

export interface Serializer {
  readonly name: string
  encode(data: unknown): Buffer
  decode(buffer: Buffer): unknown
}

export interface SerializerRegistry {
  get(name: string): Serializer | undefined
  register(name: string, serializer: Serializer): void
}

export const createSerializerExtension = (registry: SerializerRegistry): ApophisExtension => ({
  name: 'serializers',
  hooks: {
    onBuildRequest: async (ctx) => {
      const serializerName = ctx.route.serializer?.name
      if (!serializerName) return
      
      const serializer = registry.get(serializerName)
      if (!serializer) return
      
      // Modify request: encode body, set content-type
      ctx.request.body = serializer.encode(ctx.request.body)
      ctx.request.headers = {
        ...ctx.request.headers,
        'content-type': `application/x-${serializerName}`,
      }
    },
    onAfterRequest: async (ctx) => {
      const serializerName = ctx.route.serializer?.name
      if (!serializerName) return
      
      const serializer = registry.get(serializerName)
      if (!serializer) return
      
      // Modify response: decode body
      const rawBody = Buffer.from(JSON.stringify(ctx.evalContext.response.body))
      ctx.evalContext.response.body = serializer.decode(rawBody)
    }
  }
})

WebSockets Extension

Module: src/extensions/websocket/

Note: WebSockets are fundamentally different from HTTP. They require a dedicated runner, not just hooks.

export const websocketExtension: ApophisExtension = {
  name: 'websocket',
  headers: ['ws_message', 'ws_state'],
  predicates: {
    ws_message: (ctx) => {
      const msg = ctx.evalContext.ws?.message ?? null
      if (ctx.accessor.length === 0) return { value: msg, success: true }
      if (!msg) return { value: null, success: true }
      
      if (ctx.accessor[0] === 'type') return { value: msg.type, success: true }
      if (ctx.accessor[0] === 'payload') return { value: msg.payload, success: true }
      if (ctx.accessor[0] === 'direction') return { value: msg.direction, success: true }
      
      return { value: msg, success: true }
    },
    ws_state: (ctx) => {
      return { value: ctx.evalContext.ws?.state ?? null, success: true }
    }
  },
  hooks: {
    onSuiteStart: async ({ routes }) => {
      // Pre-validate all WS contracts
      const wsRoutes = routes.filter(r => r.ws !== undefined)
      for (const route of wsRoutes) {
        validateWebSocketContract(route.ws!)
      }
    }
  }
}

WebSocket runner: Invoked by plugin separately from HTTP runners:

// In plugin/index.ts
const buildContract = (fastify, scope) => async (opts) => {
  const httpSuite = await runPetitTests(fastify, opts, scope)
  const wsSuite = await runWebSocketTests(fastify, opts, scope) // From extension
  return mergeSuites(httpSuite, wsSuite)
}

---

4. Core Changes (Phase 1)

4.1 Parser Extensibility

Current: VALID_HEADERS is hardcoded. Extensions can't add headers.

Solution: Extensions register headers. Parser validates against registered + core headers.

// src/formula/parser.ts
const CORE_HEADERS: OperationHeader[] = [
  'request_body', 'response_body', 'response_code',
  'request_headers', 'response_headers', 'query_params', 'cookies', 'response_time',
  'redirect_count', 'redirect_url', 'redirect_status',
  'timeout_occurred', 'timeout_value',
  // v1.1 first-class
  'request_files', 'request_fields', 'stream_chunks', 'stream_duration',
]

// ExtensionRegistry provides additional headers
function getValidHeaders(registry?: ExtensionRegistry): string[] {
  const extensionHeaders = registry 
    ? registry.extensions.flatMap(e => e.headers ?? [])
    : []
  return [...CORE_HEADERS, ...extensionHeaders]
}

// In parseOperation, validate against getValidHeaders()

4.2 Evaluator Extensibility

Current: resolveOperation checks core operations only.

Solution: Check extension predicates BEFORE core operations.

function resolveOperation(node, ctx, extensionRegistry, route) {
  const { header, accessor } = node
  
  // 1. Check extension predicates FIRST
  if (extensionRegistry) {
    const resolver = extensionRegistry.resolvePredicate(header)
    if (resolver) {
      const ownerName = extensionRegistry.getPredicateOwner(header)
      const extState = ownerName ? (extensionRegistry.getState(ownerName) ?? {}) : {}
      const result = resolver({ route, evalContext: ctx, accessor: accessor ?? [], extensionState: extState })
      if (result && typeof result.then !== 'function') {
        return (result as PredicateResult).value
      }
    }
  }
  
  // 2. Fall back to core operations
  switch (header) {
    // ... core cases ...
  }
}

4.3 HTTP Executor Hooks

Current: executeHttp is a monolithic function.

Solution: Add onTransformResponse hook point for extensions that need to modify responses.

export interface ResponseTransformContext {
  responseBody: unknown
  evalContext: EvalContext
  route: RouteContract
}

export type ResponseTransformHook = (ctx: ResponseTransformContext) => EvalContext | Promise<EvalContext>

// In executeHttp:
let ctx = buildEvalContext(request, response, route)

// Apply extension response transforms
for (const ext of (extensionRegistry?.extensions ?? [])) {
  if (ext.hooks?.onAfterRequest) {
    await ext.hooks.onAfterRequest({
      route,
      request,
      evalContext: ctx,
      extensionState: extensionRegistry?.getState(ext.name) ?? {},
    })
  }
}

---

5. Implementation Order

Phase 1: Core Extension Points (1-2 days)

1. Make parser accept registered headers (CORE_HEADERS + extension headers)
2. Make evaluator check extension predicates before core operations
3. Add response transform hook point to HTTP executor
4. Test: Core operations still work; extension predicates resolve

Phase 2A: Multipart (First-Class, 2-3 days)

1. Add MultipartFile, MultipartPayload types
2. Add multipart schema-to-arbitrary handler
3. Add multipart request builder support
4. Add multipart HTTP executor support (FormData construction)
5. Add request_files, request_fields to parser/evaluator
6. Extract multipart config from schema in contract.ts
7. Test: src/test/multipart.test.ts (10+ tests)

Phase 2B: Streaming (First-Class, 2-3 days)

1. Add chunks, streamDurationMs to EvalContext.response
2. Add streaming config extraction from schema
3. Add stream collection to HTTP executor (NDJSON parsing)
4. Add stream_chunks, stream_duration to parser/evaluator
5. Test: src/test/streaming.test.ts (8+ tests)

Phase 2C: Extension System Polish (1 day)

1. Document extension registration API
2. Add extensions: ApophisExtension[] to ApophisOptions
3. Wire extension headers into parser
4. Wire extension predicates into evaluator

Phase 3: Extensions (Parallel, after Phase 2C)

• SSE Extension (2-3 days)
• Serializers Extension (2-3 days)
• WebSockets Extension (1-2 weeks)

Phase 4: Integration (2-3 days)

1. Run full test suite
2. Update README
3. Verify benchmarks

---

6. File Layout

src/
  # Core v1.1 First-Class Features
  infrastructure/
    http-executor.ts         # ADD: multipart FormData, stream collection
    multipart.ts            # NEW: FormData construction
    stream-collector.ts     # NEW: NDJSON chunk parsing
  domain/
    schema-to-arbitrary.ts   # ADD: multipart schema handler
    request-builder.ts       # ADD: multipart payload construction
    contract.ts             # ADD: multipart/streaming config extraction
  formula/
    parser.ts               # MODIFY: extensible VALID_HEADERS
    evaluator.ts            # MODIFY: extension predicate check
  types.ts                  # ADD: MultipartFile, MultipartPayload, stream fields
  
  # Extension System
  extension/
    types.ts                # ADD: headers, onTransformResponse to interface
    registry.ts             # ADD: collect extension headers
  
  # Extensions (opt-in)
  extensions/
    sse/                    # SSE extension module
    serializers/            # Serializer extension module
    websocket/              # WebSocket extension module

---

7. Test Strategy

First-Class Features: Red-Green-Refactor

// Example: Multipart
// 1. Test: Parser accepts request_files(this).avatar.size
// 2. Implement: Add request_files to VALID_HEADERS
// 3. Test: Evaluator resolves request_files
// 4. Implement: Add multipart operations to resolveOperation
// 5. Test: Schema-to-arbitrary generates fake files
// 6. Implement: Add multipart handler to convertSchemaInternal
// 7. Test: Request builder constructs multipart payload
// 8. Implement: Add multipart support to buildRequest
// 9. Test: HTTP executor sends multipart request
// 10. Implement: Build FormData in executeHttp
// 11. Test: Integration — upload route works end-to-end
// 12. Implement: Full flow

Extensions: Self-Contained Tests

Each extension module has its own test.ts:

// src/extensions/sse/test.ts
import { test } from 'node:test'
import assert from 'node:assert'
import { sseExtension } from './extension.js'

test('sse: predicate returns events', () => {
  const resolver = sseExtension.predicates!.sse_events
  const result = resolver({
    route: mockRoute,
    evalContext: { response: { sseEvents: [{ event: 'update', data: {} }] } },
    accessor: [],
    extensionState: {},
  })
  assert.strictEqual((result.value as any[]).length, 1)
})

---

8. Backward Compatibility

All v1.1 changes are additive:

• Routes without multipart/streaming annotations work unchanged
• Extensions are opt-in via extensions: [...] option
• Existing APOSTL formulas work unchanged
• No breaking changes to public API

Migration path:

// v1.0
await fastify.register(apophis)

// v1.1 (no changes required for existing code)
await fastify.register(apophis)

// v1.1 with extensions
await fastify.register(apophis, {
  extensions: [sseExtension, serializerExtension, websocketExtension]
})

---

9. Risk Assessment

Risk	Mitigation
Parser changes break existing formulas	Comprehensive regression tests before parser modification
Multipart adds heavy deps	Only use native FormData/Blob (no external deps)
Streaming tests are flaky	Mock streams for unit tests; integration tests with deterministic timeouts
Extension conflicts	Namespacing by extension name; ExtensionRegistry.getState(name) isolates state
WebSocket extension too large	Split into sub-workstreams: client, runner, stateful, validation

---

10. Success Criteria

Criterion	Verification
Multipart upload routes tested	multipart.test.ts passes
Streaming routes tested	streaming.test.ts passes
Extension predicates work	Extension test.ts files pass
No regression	Full source and CLI test suites pass
Benchmark targets met	benchmark.test.ts passes
Documentation updated	README covers multipart and streaming

---

11. Quick Reference: First-Class vs Extension

Feature	Type	Core Files	Tests	Effort
Multipart	First-class	multipart.ts, schema-to-arbitrary.ts, request-builder.ts, http-executor.ts, parser.ts, evaluator.ts	multipart.test.ts	2-3 days
Streaming	First-class	stream-collector.ts, http-executor.ts, parser.ts, evaluator.ts, contract.ts	streaming.test.ts	2-3 days
SSE	Extension	src/extensions/sse/*	src/extensions/sse/test.ts	2-3 days
Serializers	Extension	src/extensions/serializers/*	src/extensions/serializers/test.ts	2-3 days
WebSockets	Extension	src/extensions/websocket/*	src/extensions/websocket/test.ts	1-2 weeks