apophis-fastify/docs/attic/root-history/FEEDBACK-arbiter-integration.md

Feedback for Apophis Team: Real-World Integration Challenges

Context

We're integrating Apophis v1.1 into Arbiter, a multi-tenant identity platform with complex auth, graph-based permissions, and LinkedDataFragment responses. The goal is to use Apophis for contract testing of our Fastify routes.

Issues Encountered

1. APOSTL Syntax: Mandatory else Clause is Undocumented ✅ FIXED in v2.0

Status: Resolved. APOPHIS v2.0 replaced APOSTL with Justin (plain JavaScript expressions).

What we wrote (v1.x):

if response_code(this) == 201 then response_body(this).data.ok == true else T

What v2.0 uses:

statusCode == 201 ? response.body.data.ok == true : true
// or simply:
!(statusCode == 201) || response.body.data.ok == true

Resolution: Justin uses standard JS ternary operators (? :) and boolean logic. No mandatory else clause, no custom syntax to learn.

2. Unclear Value Proposition vs Fastify Schema Validation

It took us time to understand what Apophis adds on top of Fastify's built-in JSON Schema validation.

Fastify already provides:

• Request body/query/params validation (via Ajv)
• Response serialization (via fast-json-stringify)
• Error formatting

We initially thought Apophis would:

• Validate responses against schemas (it doesn't — Fastify only serializes, doesn't validate responses)
• Replace our need for separate test files (it partially does, but only for behavioral contracts)

What Apophis actually adds:

• Behavioral contracts (x-ensures) for side effects and state changes
• Property-based test generation from schemas
• Stateful testing (constructor → mutator → destructor sequences)

Suggestion: Clarify in the "Getting Started" docs that Apophis is for behavioral contracts, not structural validation. Show a clear comparison table.

3. Testing Authenticated Routes is Underspecified

Our routes require:

• JWT tokens in Authorization header
• Tenant context (x-tenant-id header)
• Permission checks via graph-based auth
• Session cookies

The problem: Apophis generates requests programmatically, but there's no clear pattern for:

• Injecting auth tokens into generated requests
• Setting up prerequisite state (create user → login → get token → test route)
• Handling token refresh or session management

We tried:

• Using scopes to inject headers, but this is static and can't handle dynamic tokens
• Using x-requires for preconditions, but it's unclear how to satisfy them

Suggestion: Document a pattern for authenticated routes. Examples:

// Option 1: Dynamic scope setup
await app.apophis.scope.register('authed', {
  headers: async () => ({ 
    'Authorization': `Bearer ${await getTestToken()}` 
  })
})

// Option 2: Test hooks
await app.apophis.contract({
  beforeEach: async (req) => {
    req.headers['Authorization'] = await getTestToken()
  }
})

4. Running Against Real Server is Difficult

The docs show examples with inline route definitions, but we want to test our actual production routes.

Challenges:

• Server bootstraps databases, WAL stores, ledger connections
• Routes have complex dependency injection
• We need to clean up between tests (file system conflicts, port binding)

Example error:

Error: EEXIST: file already exists, mkdir 'server-data/wal.log'

Suggestion: Provide a guide for "Testing Existing Fastify Apps" that covers:

• Bootstrapping the server in test mode
• Cleaning up resources between runs
• Configuring Apophis after server creation but before ready()

5. Contract Debugging is Hard

When contracts fail, the output is verbose but not actionable.

Example output:

{
  "formula": "statusCode == 400 ? response.body.error != null : true",
  "context": {
    "expected": "non-null value",
    "actual": "undefined (field missing)"
  }
}

Problems:

• We don't see the actual request that was generated
• We don't see the full response body
• No suggestion for how to fix the contract

Suggestion: Include in failure output:

• The generated request (method, path, body)
• The full response body
• A suggestion like "Field 'error' missing from response. Check your handler returns error details."

6. No Clear CI/CD Pattern

We want to run Apophis in CI, but:

• How do we handle database migrations/seeding?
• How do we ensure deterministic runs (seed management)?
• How do we fail the build on contract violations?

Suggestion: Add a CI/CD section to docs with GitHub Actions example that shows:

- name: Contract Tests
  run: |
    npm run db:migrate:test
    npm run test:contracts
    # Exit code should be non-zero if contracts fail

What Works Well

• Schema-driven test generation is powerful
• x-category auto-categorization reduces boilerplate
• check() for single-route validation is useful
• Integration with @fastify/swagger is seamless

Recommendations

1. Make else optional in APOSTL conditionals ✅ Fixed in v2.0 — Justin uses standard JS ternary operators
2. Add "Auth Patterns" guide with examples for JWT, sessions, API keys
3. Improve error messages with request/response context and fix suggestions
4. Document real-world integration (existing Fastify apps, not just toy examples)
5. Add CI/CD examples with database setup and deterministic testing

Our Current Workaround

We're using Apophis for:

• Schema discovery and validation
• Contract syntax checking
• Documentation generation

But for authenticated routes, we're writing traditional E2E tests with fastify.inject() because we can control auth headers and setup/teardown more easily.

Update: APOPHIS v2.0 Resolutions

APOPHIS v2.0 (released 2026-04-25) addresses all feedback items:

1. ✅ APOSTL else clause: Replaced with Justin (standard JS ternary ? :)
2. ✅ Value proposition: Documentation now clearly distinguishes structural vs behavioral validation
3. ✅ Auth patterns: Extension system allows dynamic header injection via onBuildRequest hook
4. ✅ Real-world integration: Guide added for testing existing Fastify apps with complex bootstrapping
5. ✅ Contract debugging: Failure output now includes generated request, full response, and fix suggestions
6. ✅ CI/CD patterns: GitHub Actions example with database migrations and deterministic seeds

Recommended next steps for Arbiter integration:

• Migrate contracts from APOSTL to Justin using the migration guide
• Use the Extension Plugin System for Arbiter-specific predicates (graph_check, partial_graph, budget_check)
• Register Arbiter extension to inject S2S headers and handle preflight/finalize lifecycle

See docs/extensions/EXTENSION-PLUGIN-SYSTEM.md for the Arbiter extension example.