chore: crush git history - reborn from consolidation on 2026-03-10

2026-03-10 00:00:00 -07:00
commit d278c4b105
313 changed files with 87549 additions and 0 deletions
@@ -0,0 +1,609 @@
+# APOPHIS DX Improvement Plan
+## Getting Started, Error Context, Cache/CI Docs, and Human-Readable Output
+
+---
+
+## 1. GETTING STARTED GUIDE
+
+### Goal
+A complete "Hello World" to "Production Ready" guide that a developer can follow in 15 minutes.
+
+### Structure
+
+#### 1.1 Installation (30 seconds)
+```bash
+npm install apophis-fastify
+# peer deps: fastify, @fastify/swagger
+```
+
+#### 1.2 Minimal Setup (2 minutes)
+```typescript
+import Fastify from 'fastify'
+import apophisPlugin from 'apophis-fastify'
+
+const fastify = Fastify()
+
+// APOPHIS needs @fastify/swagger for spec generation
+await fastify.register(import('@fastify/swagger'), {})
+await fastify.register(apophisPlugin, {
+  validateRuntime: true, // optional: validates contracts on every request
+})
+
+fastify.get('/health', {
+  schema: {
+    response: {
+      200: {
+        type: 'object',
+        properties: { status: { type: 'string' } }
+      }
+    }
+  }
+}, async () => ({ status: 'ok' }))
+
+await fastify.ready()
+
+// Run contract tests
+const result = await fastify.apophis.test({ mode: 'all', depth: 'quick' })
+console.log(result.summary)
+```
+
+#### 1.3 Your First Contract (5 minutes)
+Explain the mental model:
+- **Requires** (preconditions): What must be true BEFORE the request
+- **Ensures** (postconditions): What must be true AFTER the response
+- **Invariants**: What must ALWAYS be true across requests
+
+```typescript
+fastify.post('/users', {
+  schema: {
+    'x-category': 'constructor',  // creates a resource
+    'x-requires': [],             // no preconditions
+    'x-ensures': [
+      'status:201',
+      'response_body(this).id != null',
+      'response_body(this).email == request_body(this).email',
+    ],
+    body: {
+      type: 'object',
+      properties: {
+        email: { type: 'string', format: 'email' },
+        name: { type: 'string', minLength: 1 }
+      },
+      required: ['email', 'name']
+    },
+    response: {
+      201: {
+        type: 'object',
+        properties: {
+          id: { type: 'string' },
+          email: { type: 'string' },
+          name: { type: 'string' }
+        }
+      }
+    }
+  }
+}, async (req, reply) => {
+  reply.status(201)
+  return { id: 'user-123', email: req.body.email, name: req.body.name }
+})
+```
+
+#### 1.4 Complete CRUD Example (7 minutes)
+Show a full resource lifecycle:
+- POST /users (constructor)
+- GET /users/:id (observer — reads the resource)
+- PUT /users/:id (mutator — updates the resource)
+- DELETE /users/:id (destructor — deletes the resource)
+
+Demonstrate:
+- How constructors populate the state
+- How observers verify state
+- How mutators maintain invariants
+- How cleanup works
+
+#### 1.5 Running in CI (1 minute)
+```yaml
+# .github/workflows/contracts.yml
+- run: npm test
+  env:
+    APOPHIS_CHANGED_ROUTES: "${{ steps.changes.outputs.routes }}"
+```
+
+### Files to Create
+- `docs/getting-started.md` — Full guide
+- `docs/examples/crud-api.ts` — Complete working example
+- `docs/examples/minimal.ts` — Single route example
+
+---
+
+## 2. RICH ERROR CONTEXT SYSTEM
+
+### Current State (Bad)
+```
+Contract violation: response_body(this).id != null
+```
+
+No context. No request body. No response body. No status code. No suggestion.
+
+### Target State (Good)
+```
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ CONTRACT VIOLATION: POST /users
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+Formula:
+  response_body(this).id != null
+
+Expected:
+  id to be non-null
+
+Actual:
+  id = undefined
+
+Request:
+  POST /users
+  Content-Type: application/json
+  
+  {
+    "email": "alice@example.com",
+    "name": "Alice"
+  }
+
+Response:
+  HTTP/1.1 201 Created
+  content-type: application/json
+  
+  {
+    "email": "alice@example.com",
+    "name": "Alice"
+    // id is MISSING
+  }
+
+Suggestion:
+  Your handler returned a 201 but forgot to include 'id' in the
+  response body. Ensure your constructor routes return the created
+  resource with its generated identifier.
+
+Stack:
+  at validatePostconditions (src/domain/contract-validation.ts:39)
+  at runSequence (src/test/stateful-runner.ts:167)
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+```
+
+### Implementation Plan
+
+#### Phase 1: Structured Error Objects
+Replace string errors with rich error types:
+
+```typescript
+// src/types.ts
+export interface ContractViolation {
+  readonly type: 'contract-violation'
+  readonly route: { method: string; path: string }
+  readonly formula: string
+  readonly formulaType: 'status' | 'apostl'
+  readonly request: {
+    body: unknown
+    headers: Record<string, string>
+    query: Record<string, unknown>
+    params: Record<string, unknown>
+  }
+  readonly response: {
+    statusCode: number
+    headers: Record<string, string>
+    body: unknown
+  }
+  readonly context: {
+    expected: string
+    actual: string
+    diff?: string
+  }
+  readonly suggestion?: string
+  readonly stack?: string
+}
+```
+
+#### Phase 2: Smart Suggestions Engine
+Add a suggestions module that maps common failures to actionable fixes:
+
+```typescript
+// src/domain/error-suggestions.ts
+export const getSuggestion = (violation: ContractViolation): string | undefined => {
+  // Status code mismatch
+  if (violation.formulaType === 'status') {
+    return `Expected status ${violation.context.expected}, got ${violation.context.actual}. Check your route handler's reply.status() call.`
+  }
+  
+  // Null field
+  if (violation.formula.includes('!= null') && violation.context.actual === 'undefined') {
+    const field = extractField(violation.formula)
+    return `Field '${field}' is missing from the response. Ensure your handler returns all required fields.`
+  }
+  
+  // Equality mismatch
+  if (violation.formula.includes('==')) {
+    return `Expected values to match. Check for typos, case sensitivity, or missing transformations.`
+  }
+  
+  // Authorization
+  if (violation.formula.includes('authorization') || violation.formula.includes('tenant')) {
+    return `This route may require authentication headers. Check your scope configuration.`
+  }
+  
+  return undefined
+}
+```
+
+#### Phase 3: Diff Generation
+For equality comparisons, show a visual diff:
+
+```typescript
+// src/domain/error-formatter.ts
+export const formatDiff = (expected: unknown, actual: unknown): string => {
+  if (typeof expected === 'string' && typeof actual === 'string') {
+    // String diff
+    return `Expected: "${expected}"\nActual:   "${actual}"\nDiff:     ${generateCharDiff(expected, actual)}`
+  }
+  
+  if (typeof expected === 'number' && typeof actual === 'number') {
+    return `Expected: ${expected}\nActual:   ${actual}\nDelta:    ${actual - expected}`
+  }
+  
+  // Object diff (shallow)
+  return `Expected: ${JSON.stringify(expected, null, 2)}\nActual:   ${JSON.stringify(actual, null, 2)}`
+}
+```
+
+#### Phase 4: Stack Traces
+Capture the call stack at the point of failure:
+
+```typescript
+// In validatePostconditions
+const stack = new Error().stack
+return {
+  success: false,
+  error: new ContractViolation({
+    // ... fields
+    stack: cleanStack(stack),
+  })
+}
+```
+
+### Files to Create/Modify
+- `src/types.ts` — Add `ContractViolation` interface
+- `src/domain/error-suggestions.ts` — Suggestion engine
+- `src/domain/error-formatter.ts` — Human-readable formatter
+- `src/domain/contract-validation.ts` — Return structured errors
+- `src/test/tap-formatter.ts` — Format violations in TAP output
+
+---
+
+## 3. CACHE/CI DOCUMENTATION
+
+### Goal
+Clear documentation for CI/CD integration with practical examples.
+
+### Content
+
+#### 3.1 Cache Overview
+Explain:
+- What gets cached (schema → arbitrary mappings, generated commands)
+- Where it lives (`.apophis-cache.json` in project root)
+- When it invalidates (schema hash mismatch, explicit hints)
+- Performance impact (12x speedup on warm cache)
+
+#### 3.2 CI/CD Integration Patterns
+
+**Pattern A: Git-based Route Detection**
+```bash
+# Detect changed routes from git diff
+CHANGED=$(git diff --name-only HEAD~1 | grep 'routes/' | sed 's|routes/||' | paste -sd ',' -)
+APOPHIS_CHANGED_ROUTES="$CHANGED" npm test
+```
+
+**Pattern B: Manual Hints File**
+```json
+// .apophis-hints.json
+{
+  "changed": ["/users", "POST /orders"],
+  "reason": "PR #123: Updated user and order endpoints"
+}
+```
+
+**Pattern C: Full Cache Reset**
+```bash
+# Nuclear option: rebuild everything
+rm .apophis-cache.json
+npm test
+```
+
+**Pattern D: Monorepo Support**
+```bash
+# Per-package cache
+APOPHIS_CACHE_FILE="./packages/api/.apophis-cache.json" npm test
+```
+
+#### 3.3 GitHub Actions Example
+```yaml
+name: Contract Tests
+
+on: [push, pull_request]
+
+jobs:
+  contracts:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      
+      - name: Detect changed routes
+        id: changes
+        run: |
+          if [ "${{ github.event_name }}" = "pull_request" ]; then
+            CHANGED=$(git diff --name-only ${{ github.base_ref }} | grep -E 'routes/|schema/' || true)
+            echo "routes=$CHANGED" >> $GITHUB_OUTPUT
+          fi
+      
+      - name: Run contract tests
+        run: npm test
+        env:
+          APOPHIS_CHANGED_ROUTES: ${{ steps.changes.outputs.routes }}
+      
+      - name: Upload cache artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: apophis-cache
+          path: .apophis-cache.json
+```
+
+#### 3.4 Cache Configuration API
+```typescript
+// Programmatic control
+import { invalidateRoutes, invalidateCache } from 'apophis-fastify/incremental/cache'
+
+// Before test run
+invalidateRoutes(['/users'])  // Invalidate specific routes
+invalidateCache()             // Clear everything
+```
+
+### Files to Create
+- `docs/cache-and-ci.md` — Complete guide
+- `docs/examples/github-actions.yml` — Working workflow
+- `docs/examples/gitlab-ci.yml` — GitLab example
+
+---
+
+## 4. HUMAN-READABLE FAST-CHECK OUTPUT
+
+### Current State (Bad)
+```
+Property failed after 42 tests
+Counterexample: [{"name":"","email":"a@b.c"}]
+```
+
+### Target State (Good)
+```
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ PROPERTY TEST FAILURE: POST /users
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+Fast-check found a counterexample after 42 generated test cases:
+
+Generated Input:
+  {
+    "name": "",           ← empty string (violates minLength: 1)
+    "email": "a@b.c"      ← valid email format
+  }
+
+Request:
+  POST /users
+  Content-Type: application/json
+  
+  { "name": "", "email": "a@b.c" }
+
+Response:
+  HTTP/1.1 400 Bad Request
+  
+  { "error": "Name is required" }
+
+Contract Violation:
+  Postcondition: status:201
+  Expected: 201 Created
+  Actual:   400 Bad Request
+
+Analysis:
+  Your schema requires name to have minLength: 1, but the
+  generated test case produced an empty string. Your handler
+  correctly rejected it with 400, but the contract expects 201.
+  
+  Fix: Either:
+    1. Remove minLength constraint from schema if empty names are valid
+    2. Update contract to expect 400 for invalid input
+    3. Add x-category: 'utility' if this is a validation endpoint
+
+Shrunk: 3 times (from 128-character string to empty string)
+Seed: 12345 (re-run with APOPHIS_SEED=12345 to reproduce)
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+```
+
+### Implementation Plan
+
+#### Phase 1: Counterexample Formatter
+```typescript
+// src/test/counterexample-formatter.ts
+export interface FormattedCounterexample {
+  readonly route: { method: string; path: string }
+  readonly generatedInput: Record<string, unknown>
+  readonly request: { body: unknown; headers: Record<string, string> }
+  readonly response: { statusCode: number; body: unknown }
+  readonly contractViolation: ContractViolation
+  readonly shrinkCount: number
+  readonly seed: number
+}
+
+export const formatCounterexample = (example: FormattedCounterexample): string => {
+  // Build human-readable output
+}
+```
+
+#### Phase 2: Route Context in Errors
+When fast-check finds a failure, include the route context:
+
+```typescript
+// In stateful-runner.ts, catch fast-check errors
+try {
+  await fc.assert(prop, { numRuns, seed })
+} catch (err) {
+  if (err instanceof fc.Error) {
+    const formatted = formatFastCheckError(err, results)
+    console.error(formatted)
+  }
+}
+```
+
+#### Phase 3: Analysis Engine
+Auto-analyze failures and suggest fixes:
+
+```typescript
+// src/test/failure-analyzer.ts
+export const analyzeFailure = (
+  cmd: ApiOperation,
+  ctx: EvalContext,
+  violation: ContractViolation
+): string => {
+  // 400 status with 201 expectation
+  if (ctx.response.statusCode === 400 && violation.formula === 'status:201') {
+    return `Your handler rejected valid input. Check schema constraints match contract expectations.`
+  }
+  
+  // Missing field
+  if (violation.formula.includes('!= null') && violation.context.actual === 'undefined') {
+    const field = extractField(violation.formula)
+    return `Response missing '${field}'. Check your handler returns all required fields.`
+  }
+  
+  // Schema mismatch
+  return `Schema and contract may be out of sync. Review both for consistency.`
+}
+```
+
+### Files to Create
+- `src/test/counterexample-formatter.ts` — Format fast-check failures
+- `src/test/failure-analyzer.ts` — Auto-analyze and suggest fixes
+- `src/test/error-renderer.ts` — Terminal-friendly rendering with box drawing
+
+---
+
+## 5. ERROR SYSTEM ARCHITECTURE
+
+### Design Principles
+1. **Structured over String**: All errors are objects, not strings
+2. **Context-Rich**: Every error includes request, response, and contract context
+3. **Actionable**: Every error includes a suggestion for how to fix it
+4. **Traceable**: Every error includes a stack trace and route identifier
+5. **Diff-Friendly**: Equality failures show visual diffs
+6. **Reproducible**: Every error includes the seed needed to reproduce
+
+### Error Flow
+```
+Test Execution
+    ↓
+Contract Validation (contract-validation.ts)
+    ↓
+Structured Error Object (ContractViolation)
+    ↓
+Suggestion Engine (error-suggestions.ts)
+    ↓
+Diff Generation (error-formatter.ts)
+    ↓
+TAP Output (tap-formatter.ts)
+    ↓
+Console/CI Reporter
+```
+
+### Error Types
+```typescript
+export type ApophisError =
+  | ContractViolation
+  | FormulaParseError
+  | FormulaEvalError
+  | PreconditionError
+  | InvariantError
+  | TestGenerationError
+```
+
+---
+
+## 6. IMPLEMENTATION ORDER
+
+### Week 1: Foundation ✅ COMPLETE
+- [x] Create `ContractViolation` type in `src/types.ts`
+- [x] Update `contract-validation.ts` to return structured errors
+- [x] Create `error-suggestions.ts` with basic suggestion engine
+- [x] Update `tap-formatter.ts` to render rich diagnostics
+- [x] Add tests for new error system
+- [x] Fix `extractContract` null schema crash (`contract.ts:21`)
+- [x] Fix `hashSchema` circular reference stack overflow (`hash.ts:24`)
+- [x] Fix cleanup manager signal listener leak (`cleanup-manager.ts:48`)
+- [x] Block dangerous accessors (`__proto__`, `constructor`, `prototype`) in formula evaluator
+- [x] Normalize empty arrays to singletons in `extractContract`
+- [x] Fix build output path (`tsconfig.json` rootDir)
+- [x] Document route registration order requirement in README
+- [x] Add violation deduplication in test output (PETIT + stateful runners)
+- [x] Fix HEAD route noise in test generation
+- [x] Add clean stack traces filtered to user code
+
+**Status**: Error type chain tightened. `EvalResult` uses `error: string` with optional `violation?: ContractViolation`. Runners check `post.violation`. All 246 tests passing. Hardened against null schemas, circular references, prototype pollution, signal leaks, and duplicate failures.
+
+### Week 2: Getting Started ✅ COMPLETE
+- [x] Write `docs/getting-started.md`
+- [x] Create `docs/examples/minimal.ts`
+- [x] Create `docs/examples/crud-api.ts`
+- [ ] Add screenshots/GIFs of test output
+- [x] Update README with quick-start section
+
+### Week 3: Cache/CI Docs
+- [ ] Write `docs/cache-and-ci.md`
+- [ ] Create GitHub Actions example
+- [ ] Create GitLab CI example
+- [ ] Document `APOPHIS_CHANGED_ROUTES`
+- [ ] Document `.apophis-hints.json`
+
+### Week 4: Fast-Check Formatter ✅ COMPLETE
+- [x] Create `counterexample-formatter.ts`
+- [x] Create `failure-analyzer.ts`
+- [x] Create `error-renderer.ts` with box drawing
+- [x] Integrate with stateful runner
+- [x] Add tests for formatting
+
+### Week 5: Production Hardening ✅ COMPLETE
+- [x] Regex DoS protection with `safe-regex`
+- [x] Standard logging with `pino` (APOPHIS_LOG_LEVEL)
+- [x] Environment-aware cache (disabled in production/test)
+- [x] Lazy cache loading (no sync file I/O at module load)
+- [x] Fastify prefix support in route discovery
+- [x] Signal handler deduplication (global Map)
+- [x] Add `dispose()` method to CleanupManager
+- [x] Remove all `console.log` from production code
+- [x] Stryker mutation testing (contract-validation: 70%, error-suggestions: 68.7%)
+- [x] Fix flaky property test (schema-to-arbitrary)
+- [x] 345 tests passing
+
+### Week 6: Scope Isolation ✅ COMPLETE
+- [x] Implement scope filtering in `petit-runner.ts`
+- [x] Implement scope filtering in `stateful-runner.ts`
+- [x] Add scope headers to test requests via `buildRequest`
+- [x] Tests for multi-scope scenarios
+
+**Status**: Scope isolation fully implemented. Routes with `x-scope` annotation are filtered by the `scope` test parameter. Scope headers from `ScopeRegistry` are passed to test requests. 249 tests passing.
+
+---
+
+## 7. SUCCESS METRICS
+
+- [ ] New user can go from `npm install` to passing contract tests in < 15 minutes
+- [ ] Error messages include request/response context 100% of the time
+- [ ] 80% of contract violations include an actionable suggestion
+- [ ] CI integration documented for GitHub Actions, GitLab CI, and CircleCI
+- [ ] Fast-check failures formatted with route context and analysis
+- [ ] All examples in documentation are tested and working
+- [ ] README has a "Getting Started" section above the fold