Initial public release of Apophis — invariant-driven automated API testing
This commit is contained in:
+42
-36
@@ -2,15 +2,7 @@
|
||||
|
||||
Deterministic contract verification for CI and local development.
|
||||
|
||||
## What Verify Does
|
||||
|
||||
`apophis verify` runs behavioral contracts against your Fastify routes:
|
||||
|
||||
1. Discovers routes from your Fastify app
|
||||
2. Filters routes by profile config and CLI flags
|
||||
3. Generates test data from JSON Schema
|
||||
4. Executes routes and checks `x-ensures` contracts
|
||||
5. Reports pass/fail with deterministic seed and replay command
|
||||
APOPHIS implements the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable formulas, then verify them automatically with property-based generation and deterministic replay.
|
||||
|
||||
## When to Use It
|
||||
|
||||
@@ -79,6 +71,8 @@ apophis verify --routes "POST /users/*"
|
||||
apophis verify --profile quick
|
||||
```
|
||||
|
||||
`*` and `?` wildcards are supported in `--routes`.
|
||||
|
||||
## `--changed` Flag
|
||||
|
||||
Run only routes modified in the current git branch:
|
||||
@@ -87,7 +81,7 @@ Run only routes modified in the current git branch:
|
||||
apophis verify --profile ci --changed
|
||||
```
|
||||
|
||||
If no routes changed, exits 0 with a message.
|
||||
If no routes changed, exits 2 with a message.
|
||||
|
||||
## Failure Output Format
|
||||
|
||||
@@ -126,6 +120,8 @@ Next
|
||||
apophis replay --artifact reports/apophis/failure-2026-04-28T12-30-22Z.json
|
||||
```
|
||||
|
||||
Nondeterminism warnings appear in output when the same seed produces different results across runs. This indicates stateful behavior in your application that contracts cannot control.
|
||||
|
||||
## Machine Output for CI
|
||||
|
||||
Use concise formats to reduce log volume in large verify runs:
|
||||
@@ -137,6 +133,7 @@ Use concise formats to reduce log volume in large verify runs:
|
||||
|
||||
```bash
|
||||
# Extract only failed routes from full ndjson
|
||||
# Note: route.failed events are only emitted for failures, not passed routes
|
||||
apophis verify --profile quick --format ndjson | jq 'select(.type == "route.failed")'
|
||||
|
||||
# Write artifact to disk and parse the file instead of stdout
|
||||
@@ -149,7 +146,7 @@ apophis verify --profile quick --format json --artifact-dir reports/apophis
|
||||
|---|---|
|
||||
| 0 | All contracts passed |
|
||||
| 1 | One or more behavioral contracts failed |
|
||||
| 2 | Config error or no routes matched |
|
||||
| 2 | Config error, no routes matched, no contracts found, or not a git repo |
|
||||
| 3 | Internal APOPHIS error |
|
||||
| 130 | Interrupted (SIGINT) |
|
||||
|
||||
@@ -158,42 +155,51 @@ apophis verify --profile quick --format json --artifact-dir reports/apophis
|
||||
```javascript
|
||||
// apophis.config.js
|
||||
export default {
|
||||
mode: 'verify',
|
||||
profile: 'quick',
|
||||
profiles: {
|
||||
quick: {
|
||||
name: 'quick',
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: ['POST /users']
|
||||
},
|
||||
ci: {
|
||||
name: 'ci',
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: []
|
||||
}
|
||||
},
|
||||
presets: {
|
||||
'safe-ci': {
|
||||
name: 'safe-ci',
|
||||
depth: 'quick',
|
||||
timeout: 5000,
|
||||
parallel: false,
|
||||
chaos: false,
|
||||
observe: false
|
||||
}
|
||||
},
|
||||
environments: {
|
||||
local: {
|
||||
name: 'local',
|
||||
allowVerify: true,
|
||||
allowObserve: true,
|
||||
allowQualify: false,
|
||||
allowChaos: false,
|
||||
allowBlocking: true,
|
||||
requireSink: false
|
||||
runs: 10,
|
||||
timeout: 5000
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
For the full config schema, see [CLI Reference](cli.md).
|
||||
|
||||
## Workspace Support
|
||||
|
||||
Run verify across all packages in a monorepo workspace:
|
||||
|
||||
```bash
|
||||
apophis verify --workspace --profile quick --format json
|
||||
```
|
||||
|
||||
Output includes per-package pass/fail summaries. Fails if any package fails.
|
||||
|
||||
## Test Budget
|
||||
|
||||
The `runs` field in your preset controls how many property-based tests execute per route. Default is 50. Lower for faster CI feedback, higher for deeper exploration:
|
||||
|
||||
```javascript
|
||||
profiles: {
|
||||
quick: {
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: ['POST /users']
|
||||
}
|
||||
},
|
||||
presets: {
|
||||
'safe-ci': {
|
||||
runs: 10,
|
||||
timeout: 5000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user