apophis-fastify/docs/attic/root-history/FEEDBACK-arbiter-integration.md

# Feedback for Apophis Team: Real-World Integration Challenges

## Context

We're integrating Apophis v1.1 into Arbiter, a multi-tenant identity platform with complex auth, graph-based permissions, and LinkedDataFragment responses. The goal is to use Apophis for contract testing of our Fastify routes.

## Issues Encountered

### 1. ~~APOSTL Syntax: Mandatory `else` Clause is Undocumented~~ ✅ FIXED in v2.0

**Status**: Resolved. APOPHIS v2.0 replaced APOSTL with Justin (plain JavaScript expressions).

**What we wrote (v1.x):**
```apostl
if response_code(this) == 201 then response_body(this).data.ok == true else T
```

**What v2.0 uses:**
```javascript
statusCode == 201 ? response.body.data.ok == true : true
// or simply:
!(statusCode == 201) || response.body.data.ok == true
```

**Resolution**: Justin uses standard JS ternary operators (`? :`) and boolean logic. No mandatory `else` clause, no custom syntax to learn.

### 2. Unclear Value Proposition vs Fastify Schema Validation

It took us time to understand what Apophis adds on top of Fastify's built-in JSON Schema validation.

**Fastify already provides:**
- Request body/query/params validation (via Ajv)
- Response serialization (via fast-json-stringify)
- Error formatting

**We initially thought Apophis would:**
- Validate responses against schemas (it doesn't — Fastify only serializes, doesn't validate responses)
- Replace our need for separate test files (it partially does, but only for behavioral contracts)

**What Apophis actually adds:**
- Behavioral contracts (`x-ensures`) for side effects and state changes
- Property-based test generation from schemas
- Stateful testing (constructor → mutator → destructor sequences)

**Suggestion:** Clarify in the "Getting Started" docs that Apophis is for *behavioral* contracts, not structural validation. Show a clear comparison table.

### 3. Testing Authenticated Routes is Underspecified

Our routes require:
- JWT tokens in Authorization header
- Tenant context (x-tenant-id header)
- Permission checks via graph-based auth
- Session cookies

**The problem:** Apophis generates requests programmatically, but there's no clear pattern for:
- Injecting auth tokens into generated requests
- Setting up prerequisite state (create user → login → get token → test route)
- Handling token refresh or session management

**We tried:**
- Using `scopes` to inject headers, but this is static and can't handle dynamic tokens
- Using `x-requires` for preconditions, but it's unclear how to satisfy them

**Suggestion:** Document a pattern for authenticated routes. Examples:
```typescript
// Option 1: Dynamic scope setup
await app.apophis.scope.register('authed', {
  headers: async () => ({ 
    'Authorization': `Bearer ${await getTestToken()}` 
  })
})

// Option 2: Test hooks
await app.apophis.contract({
  beforeEach: async (req) => {
    req.headers['Authorization'] = await getTestToken()
  }
})
```

### 4. Running Against Real Server is Difficult

The docs show examples with inline route definitions, but we want to test our actual production routes.

**Challenges:**
- Server bootstraps databases, WAL stores, ledger connections
- Routes have complex dependency injection
- We need to clean up between tests (file system conflicts, port binding)

**Example error:**
```
Error: EEXIST: file already exists, mkdir 'server-data/wal.log'
```

**Suggestion:** Provide a guide for "Testing Existing Fastify Apps" that covers:
- Bootstrapping the server in test mode
- Cleaning up resources between runs
- Configuring Apophis after server creation but before `ready()`

### 5. Contract Debugging is Hard

When contracts fail, the output is verbose but not actionable.

**Example output:**
```json
{
  "formula": "statusCode == 400 ? response.body.error != null : true",
  "context": {
    "expected": "non-null value",
    "actual": "undefined (field missing)"
  }
}
```

**Problems:**
- We don't see the actual request that was generated
- We don't see the full response body
- No suggestion for how to fix the contract

**Suggestion:** Include in failure output:
- The generated request (method, path, body)
- The full response body
- A suggestion like "Field 'error' missing from response. Check your handler returns error details."

### 6. No Clear CI/CD Pattern

We want to run Apophis in CI, but:
- How do we handle database migrations/seeding?
- How do we ensure deterministic runs (seed management)?
- How do we fail the build on contract violations?

**Suggestion:** Add a CI/CD section to docs with GitHub Actions example that shows:
```yaml
- name: Contract Tests
  run: |
    npm run db:migrate:test
    npm run test:contracts
    # Exit code should be non-zero if contracts fail
```

## What Works Well

- Schema-driven test generation is powerful
- `x-category` auto-categorization reduces boilerplate
- `check()` for single-route validation is useful
- Integration with `@fastify/swagger` is seamless

## Recommendations

1. ~~**Make `else` optional** in APOSTL conditionals~~ ✅ Fixed in v2.0 — Justin uses standard JS ternary operators
2. **Add "Auth Patterns" guide** with examples for JWT, sessions, API keys
3. **Improve error messages** with request/response context and fix suggestions
4. **Document real-world integration** (existing Fastify apps, not just toy examples)
5. **Add CI/CD examples** with database setup and deterministic testing

## Our Current Workaround

We're using Apophis for:
- Schema discovery and validation
- Contract syntax checking
- Documentation generation

But for authenticated routes, we're writing traditional E2E tests with `fastify.inject()` because we can control auth headers and setup/teardown more easily.

## Update: APOPHIS v2.0 Resolutions

**APOPHIS v2.0 (released 2026-04-25) addresses all feedback items:**

1. ✅ **APOSTL `else` clause**: Replaced with Justin (standard JS ternary `? :`)
2. ✅ **Value proposition**: Documentation now clearly distinguishes structural vs behavioral validation
3. ✅ **Auth patterns**: Extension system allows dynamic header injection via `onBuildRequest` hook
4. ✅ **Real-world integration**: Guide added for testing existing Fastify apps with complex bootstrapping
5. ✅ **Contract debugging**: Failure output now includes generated request, full response, and fix suggestions
6. ✅ **CI/CD patterns**: GitHub Actions example with database migrations and deterministic seeds

**Recommended next steps for Arbiter integration:**
- Migrate contracts from APOSTL to Justin using the [migration guide](docs/getting-started.md#migration-from-v1x)
- Use the Extension Plugin System for Arbiter-specific predicates (`graph_check`, `partial_graph`, `budget_check`)
- Register Arbiter extension to inject S2S headers and handle preflight/finalize lifecycle

See `docs/extensions/EXTENSION-PLUGIN-SYSTEM.md` for the Arbiter extension example.