Initial public release of Apophis — invariant-driven automated API testing

This commit is contained in:
John Dvorak
2026-03-10 00:00:00 -07:00
parent d278c4b105
commit 3ac1daf7e9
82 changed files with 3902 additions and 1098 deletions
+42 -36
View File
@@ -2,15 +2,7 @@
Deterministic contract verification for CI and local development.
## What Verify Does
`apophis verify` runs behavioral contracts against your Fastify routes:
1. Discovers routes from your Fastify app
2. Filters routes by profile config and CLI flags
3. Generates test data from JSON Schema
4. Executes routes and checks `x-ensures` contracts
5. Reports pass/fail with deterministic seed and replay command
APOPHIS implements the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable formulas, then verify them automatically with property-based generation and deterministic replay.
## When to Use It
@@ -79,6 +71,8 @@ apophis verify --routes "POST /users/*"
apophis verify --profile quick
```
`*` and `?` wildcards are supported in `--routes`.
## `--changed` Flag
Run only routes modified in the current git branch:
@@ -87,7 +81,7 @@ Run only routes modified in the current git branch:
apophis verify --profile ci --changed
```
If no routes changed, exits 0 with a message.
If no routes changed, exits 2 with a message.
## Failure Output Format
@@ -126,6 +120,8 @@ Next
apophis replay --artifact reports/apophis/failure-2026-04-28T12-30-22Z.json
```
Nondeterminism warnings appear in output when the same seed produces different results across runs. This indicates stateful behavior in your application that contracts cannot control.
## Machine Output for CI
Use concise formats to reduce log volume in large verify runs:
@@ -137,6 +133,7 @@ Use concise formats to reduce log volume in large verify runs:
```bash
# Extract only failed routes from full ndjson
# Note: route.failed events are only emitted for failures, not passed routes
apophis verify --profile quick --format ndjson | jq 'select(.type == "route.failed")'
# Write artifact to disk and parse the file instead of stdout
@@ -149,7 +146,7 @@ apophis verify --profile quick --format json --artifact-dir reports/apophis
|---|---|
| 0 | All contracts passed |
| 1 | One or more behavioral contracts failed |
| 2 | Config error or no routes matched |
| 2 | Config error, no routes matched, no contracts found, or not a git repo |
| 3 | Internal APOPHIS error |
| 130 | Interrupted (SIGINT) |
@@ -158,42 +155,51 @@ apophis verify --profile quick --format json --artifact-dir reports/apophis
```javascript
// apophis.config.js
export default {
mode: 'verify',
profile: 'quick',
profiles: {
quick: {
name: 'quick',
mode: 'verify',
preset: 'safe-ci',
routes: ['POST /users']
},
ci: {
name: 'ci',
mode: 'verify',
preset: 'safe-ci',
routes: []
}
},
presets: {
'safe-ci': {
name: 'safe-ci',
depth: 'quick',
timeout: 5000,
parallel: false,
chaos: false,
observe: false
}
},
environments: {
local: {
name: 'local',
allowVerify: true,
allowObserve: true,
allowQualify: false,
allowChaos: false,
allowBlocking: true,
requireSink: false
runs: 10,
timeout: 5000
}
}
};
```
For the full config schema, see [CLI Reference](cli.md).
## Workspace Support
Run verify across all packages in a monorepo workspace:
```bash
apophis verify --workspace --profile quick --format json
```
Output includes per-package pass/fail summaries. Fails if any package fails.
## Test Budget
The `runs` field in your preset controls how many property-based tests execute per route. Default is 50. Lower for faster CI feedback, higher for deeper exploration:
```javascript
profiles: {
quick: {
mode: 'verify',
preset: 'safe-ci',
routes: ['POST /users']
}
},
presets: {
'safe-ci': {
runs: 10,
timeout: 5000
}
}
```