Files
apophis-fastify/docs/getting-started.md
T
John Dvorak 31530fe899
CI / test (20.x) (push) Failing after 1m55s
CI / test (22.x) (push) Failing after 35s
(mess) Stuffing commit.
2026-05-20 16:09:43 -07:00

284 lines
8.8 KiB
Markdown

# Getting Started with APOPHIS
Get from install to your first behavioral bug in 10 minutes.
APOPHIS is inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only validating request and response shape, encode intended behavior as executable contracts and let the tool find violations automatically.
## Prerequisites
- Node.js 20.x or 22.x
- A Fastify app with `@fastify/swagger` registered
## Step 1: Install
```bash
npm install apophis-fastify fastify @fastify/swagger
```
## Step 2: Scaffold
```bash
apophis init --preset safe-ci
```
This creates:
- `apophis.config.js` — config with a `quick` profile
- `APOPHIS.md` — preset-specific guidance
- Package script: `npm run apophis:verify`
## Step 3: Add One Behavioral Contract
Pick one important route. Add an `x-ensures` clause that checks behavior across operations:
```javascript
import crypto from 'crypto';
app.post('/users', {
schema: {
'x-category': 'constructor',
'x-ensures': [
// BEHAVIORAL: Creating a user must make it retrievable
'response_code(GET /users/{response_body(this).id}) == 200'
]
}
}, async (request, reply) => {
const { name } = request.body;
const id = `usr-${crypto.createHash('sha256').update(name).digest('hex').slice(0, 8)}`;
reply.status(201);
return { id, name };
});
```
> **Warning:** Using `Date.now()` or `Math.random()` in handlers breaks determinism and replay. Use a stable function of the input instead. APOPHIS does not proactively detect nondeterministic handlers; it warns only when a replay diverges from the original run.
## Step 4: Run Verify
```bash
apophis verify --profile quick --routes "POST /users"
```
## Example Failure
If your `GET /users/:id` handler has a bug (always returns 404), APOPHIS catches it:
```text
Contract violation
POST /users
Profile: quick
Seed: 42
Expected
response_code(GET /users/{response_body(this).id}) == 200
Observed
GET /users/usr-7d865e returned 404
Why this matters
The resource created by POST /users is not retrievable.
Replay
apophis replay --artifact reports/apophis/failure-2026-04-28T12-30-22Z.json
Next
Check the create/read consistency for POST /users and GET /users/{id}.
```
## Step 5: Replay and Fix
Copy the replay command and run it:
```bash
apophis replay --artifact reports/apophis/failure-2026-04-28T12-30-22Z.json
```
Fix the bug in your handler. Re-run verify. The failure should now pass.
## Behavioral vs Structural Contracts
APOPHIS contracts should verify **behavior**, not structure. Fastify and `@fastify/swagger` already enforce status codes, required fields, and types. Behavioral contracts catch what schemas cannot:
| Structural (avoid) | Behavioral (prefer) |
|---|---|
| `status:200` | `response_body(this) == request_body(this)` |
| `response_body(this).id != null` | `response_code(GET /users/{response_body(this).id}) == 200` |
| `response_body(this).name != null` | `response_body(GET /users/{id}).name == previous(response_body(this).name)` |
**Good behavioral patterns (from the paper):**
- **Constructor precondition**: Resource must not exist before creation
```apostl
response_code(GET /users/{request_body(this).email}) == 404
```
- **Round-trip equality**: POST response matches the request body
```apostl
response_body(this) == request_body(this)
```
- **Cross-route retrievability**: Creating a resource makes it readable via GET
```apostl
response_code(GET /users/{response_body(this).id}) == 200
```
- **State-change verification**: DELETE causes subsequent GET to return 404
```apostl
response_code(GET /users/{request_params(this).id}) == 404
```
- **Previous state preservation**: DELETE returns the last known state
```apostl
response_body(this) == previous(response_body(GET /users/{request_params(this).id}))
```
- **Invariant over collections**: All resources satisfy a cross-resource constraint
```apostl
for t in response_body(GET /tournaments) :-
response_body(GET /tournaments/{t.id}/players).length <= t.capacity
```
**Anti-patterns to avoid:**
- Checking status codes (handled by schema validation)
- Checking field existence (handled by schema validation)
- Checking field types (handled by schema validation)
## Next Steps
- Add more routes to your profile: `apophis verify --profile quick --routes "POST /users,PUT /users/:id"`
- Use wildcards to match route patterns: `apophis verify --routes 'POST /api/*'`
- Run all routes: `apophis verify --profile quick`
- Run only changed routes in CI: `apophis verify --profile ci --changed`
- Requires a git repository.
- Use machine-readable output in CI: `apophis verify --profile ci --format json-summary`
- Add observe mode for runtime drift detection: see [observe.md](observe.md)
- Add qualify mode for scenario, stateful, and chaos checks: see [qualify.md](qualify.md)
## Variants
Test the same route with different headers or content types:
```javascript
await fastify.apophis.contract({
variants: [
{ name: 'json', headers: { accept: 'application/json' } },
{ name: 'xml', headers: { accept: 'application/xml' } }
]
})
```
Or declare variants in the route schema:
```javascript
app.get('/users', {
schema: {
'x-variants': [
{ name: 'json', headers: { accept: 'application/json' } }
]
}
})
```
## Plugin Options
When registering the APOPHIS plugin, you can pass these options:
```javascript
await fastify.register(apophis, {
// Swagger config passthrough (if @fastify/swagger is not already registered)
swagger: { openapi: { info: { title: 'API', version: '1.0.0' } } },
// Runtime contract validation hooks: 'off', 'warn', or 'error'
// Only active in non-production environments
runtime: 'warn',
// Automatically clean up tracked resources after tests
cleanup: true,
// Global timeout in milliseconds for all requests
timeout: 5000,
// Tenant isolation scopes
scopes: {
tenant1: { headers: { 'x-tenant-id': '1' } },
tenant2: { headers: { 'x-tenant-id': '2' } },
},
// Auth and protocol extensions
extensions: [jwtAuth, apiKeyAuth],
// Plugin hook-phase contracts
pluginContracts: {
'rate-limit': { appliesTo: 'POST /users', ensures: ['status != 429'] },
},
// Outbound dependency contracts
outboundContracts: {
'payment-api': {
target: 'https://payments.example.com',
method: 'POST',
response: { 200: { type: 'object', properties: { id: { type: 'string' } } } }
}
}
})
```
## Schema Annotations
APOPHIS reads these OpenAPI schema extensions:
| Annotation | Location | Description |
|---|---|---|
| `x-category` | Top-level | Route classification: `constructor`, `mutator`, `observer`, `destructor`, `utility` |
| `x-ensures` | Top-level or `response[statusCode]` | Post-condition contracts (APOSTL formulas) |
| `x-requires` | Top-level or `response[statusCode]` | Pre-condition contracts (APOSTL formulas) |
| `x-variants` | Top-level | Request variants for content-type negotiation or feature flags |
| `x-timeout` | Top-level or `response[statusCode]` | Per-route timeout in milliseconds |
| `x-outbound` | Top-level | Outbound dependency contracts for this route |
| `x-streaming` | Top-level | Mark route as streaming (populates `chunks` and `streamDurationMs` in eval context) |
| `x-validate-runtime` | Top-level or `response[statusCode]` | Toggle runtime validation for this route (default: true) |
| `x-extension-config` | Top-level | Per-route config for extensions (e.g., `{ jwt: { verify: false } }`) |
Annotations can be placed on the top-level schema or nested inside `response[statusCode]`. Nested annotations take precedence for that status code.
## Programmatic API
After registration, `fastify.apophis` provides:
```javascript
// Run contract tests for all routes
const suite = await fastify.apophis.contract({ runs: 50, seed: 42 })
// Run stateful tests
const stateful = await fastify.apophis.stateful({ runs: 50, seed: 42 })
// Run a single scenario
const scenario = await fastify.apophis.scenario({
name: 'oauth-basic',
steps: [...]
})
// Check a single route
const result = await fastify.apophis.check('GET', '/users/:id')
// Get enriched OpenAPI spec with contract metadata
const spec = fastify.apophis.spec()
// Clean up tracked resources
await fastify.apophis.cleanup()
// Test-only utilities (NODE_ENV=test only)
fastify.apophis.test.registerPluginContracts('name', spec)
fastify.apophis.test.registerOutboundContracts({ ... })
fastify.apophis.test.enableOutboundMocks({ mode: 'example' })
fastify.apophis.test.disableOutboundMocks()
const calls = fastify.apophis.test.getOutboundCalls('payment-api')
```
## Config Reference
For the full configuration reference, see [CLI Reference](cli.md).
## Monorepo Workspaces
Use `--workspace` to run verify or doctor across all packages:
```bash
apophis verify --workspace --profile quick --format json
```
See [CLI Reference](cli.md) for workspace output format and exit codes.