docs: add paper citations, fix pedagogical issues, improve SKILL.md
- Cite arxiv 2602.23922 (Invariant-Driven Automated Testing) in all major docs - Add Progressive Complexity section to SKILL.md for LLM guidance - Fix SKILL.md Fast Start example to use deterministic ID generation - Fix getting-started.md failure output inconsistency - Fix auth-patterns.md TypeScript syntax in JS doc - Fix fastify-structure.md Date.now() in test helper - Fix observe.md misleading workspace heading - Build: clean | Tests: 849 pass, 0 fail
This commit is contained in:
@@ -4,6 +4,8 @@ Behavioral confidence for Fastify services.
|
||||
|
||||
APOPHIS checks whether route behavior holds across operations, states, and protocol flows.
|
||||
|
||||
Inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only checking payload shape, APOPHIS encodes intended behavior as executable contracts and verifies them with property-based and stateful testing.
|
||||
|
||||
Supported Node.js versions: 20.x and 22.x.
|
||||
|
||||
```bash
|
||||
|
||||
@@ -7,6 +7,8 @@ description: Use this skill when adding or improving APOPHIS contract-driven tes
|
||||
|
||||
APOPHIS verifies API behavior across operations, state changes, protocol flows, and dependencies. Use it when schema validation is not enough to answer whether an endpoint did the right thing.
|
||||
|
||||
Inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable contracts, then verify them with property-based and stateful testing.
|
||||
|
||||
## When To Use
|
||||
|
||||
Use this skill when the operator asks to:
|
||||
@@ -76,6 +78,7 @@ When entering a Fastify codebase:
|
||||
import Fastify from 'fastify'
|
||||
import swagger from '@fastify/swagger'
|
||||
import apophis from 'apophis-fastify'
|
||||
import crypto from 'crypto'
|
||||
|
||||
const app = Fastify()
|
||||
await app.register(swagger)
|
||||
@@ -114,8 +117,9 @@ app.post('/users', {
|
||||
}
|
||||
}
|
||||
}, async (req, reply) => {
|
||||
const id = `usr-${crypto.createHash('sha256').update(req.body.email).digest('hex').slice(0, 8)}`
|
||||
reply.status(201)
|
||||
return { id: 'usr-1', ...req.body }
|
||||
return { id, ...req.body }
|
||||
})
|
||||
|
||||
await app.ready()
|
||||
@@ -350,6 +354,31 @@ Operator framing:
|
||||
|
||||
> The failing seed gives us a reproducible behavioral example. I'll replay it first so we can distinguish a real regression from source drift or nondeterministic app state.
|
||||
|
||||
## Progressive Complexity
|
||||
|
||||
Start simple and add depth only where it pays off:
|
||||
|
||||
**Level 1 — Status and shape**: Every route gets an expected status code and key field existence.
|
||||
```apostl
|
||||
status:201
|
||||
response_body(this).id != null
|
||||
```
|
||||
|
||||
**Level 2 — Cross-route behavior**: Constructors check retrievability; mutators check persistence.
|
||||
```apostl
|
||||
response_code(GET /users/{response_body(this).id}) == 200
|
||||
response_body(GET /users/{response_body(this).id}).email == request_body(this).email
|
||||
```
|
||||
|
||||
**Level 3 — Isolation and boundaries**: Tenant, auth, and idempotency checks.
|
||||
```apostl
|
||||
if request_headers(this).x-tenant-id != null then response_headers(this).x-tenant-id == request_headers(this).x-tenant-id else true
|
||||
```
|
||||
|
||||
**Level 4 — Protocol and dependency flows**: Variants, scenarios, outbound contracts, and chaos.
|
||||
|
||||
Add level 2 before level 4. Do not skip level 2 for resource APIs.
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
Do not:
|
||||
|
||||
@@ -144,7 +144,7 @@ See `docs/protocol-extensions-spec.md` for full JWT extension configuration.
|
||||
`getToken` runs per request. Handle refresh inline:
|
||||
|
||||
```javascript
|
||||
let cachedToken: string | null = null
|
||||
let cachedToken = null
|
||||
|
||||
const auth = createAuthExtension({
|
||||
name: 'jwt-with-refresh',
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
Inject controlled failures into contract tests to validate resilience guarantees.
|
||||
|
||||
Chaos testing applies the invariant-driven verification approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) under adverse conditions: if a contract must hold, it should still hold when dependencies fail, responses are delayed, or payloads are corrupted.
|
||||
|
||||
## Usage
|
||||
|
||||
```javascript
|
||||
|
||||
@@ -374,8 +374,10 @@ import { mkdirSync, rmSync } from 'fs'
|
||||
import { tmpdir } from 'os'
|
||||
import { join } from 'path'
|
||||
|
||||
let testCounter = 0
|
||||
|
||||
export function createTestWorkspace() {
|
||||
const dir = join(tmpdir(), `apophis-test-${Date.now()}`)
|
||||
const dir = join(tmpdir(), `apophis-test-${++testCounter}`)
|
||||
mkdirSync(dir, { recursive: true })
|
||||
|
||||
return {
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
Get from install to your first behavioral bug in 10 minutes.
|
||||
|
||||
APOPHIS is inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only validating request and response shape, encode intended behavior as executable contracts and let the tool find violations automatically.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20.x or 22.x
|
||||
@@ -70,7 +72,7 @@ Expected
|
||||
response_code(GET /users/{response_body(this).id}) == 200
|
||||
|
||||
Observed
|
||||
GET /users/usr-123 returned 404
|
||||
GET /users/usr-7d865e returned 404
|
||||
|
||||
Why this matters
|
||||
The resource created by POST /users is not retrievable.
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
APOPHIS is designed to be safe and predictable for LLM-generated Fastify services.
|
||||
|
||||
It applies the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) to LLM-assisted development: constrained vocabulary, deterministic replay, and executable contracts give coding agents a verifiable loop between generated changes and behavioral correctness.
|
||||
|
||||
## Why APOPHIS Is Good for LLM-Generated Services
|
||||
|
||||
Coding agents benefit from:
|
||||
|
||||
+4
-2
@@ -2,6 +2,8 @@
|
||||
|
||||
Runtime visibility and drift detection without blocking by default.
|
||||
|
||||
Observe extends the invariant framework from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) to production environments: contracts run continuously against live traffic to detect behavioral drift without affecting requests.
|
||||
|
||||
## What Observe Does
|
||||
|
||||
`apophis observe` validates your runtime observe configuration:
|
||||
@@ -153,9 +155,9 @@ observe: {
|
||||
}
|
||||
```
|
||||
|
||||
## Workspace Support
|
||||
## Monorepo Validation
|
||||
|
||||
For monorepos, use `apophis doctor --workspace` to validate observe configuration across all workspace packages.
|
||||
For monorepos, use `apophis doctor --workspace` to validate observe configuration across all workspace packages. `observe` itself does not support `--workspace`; use `doctor` to check config in each package.
|
||||
|
||||
## Mode Mismatch
|
||||
|
||||
|
||||
@@ -6,6 +6,8 @@
|
||||
|
||||
This specification defines protocol-specific extensions for APOPHIS, driven by the Arbiter team's requirements for testing OAuth 2.1, WIMSE S2S, Transaction Tokens (RFC 8693), SPIFFE/SPIRE, and related security protocols.
|
||||
|
||||
APOPHIS is grounded in [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021). Protocol extensions add domain-specific predicates (JWT, X.509, SPIFFE) to the core invariant framework.
|
||||
|
||||
Arbiter maintains 58 protocol conformance test files covering 138 behaviors across 7 specifications. These extensions bridge the gap between declarative APOSTL contracts and the domain-specific predicates required for security protocol validation.
|
||||
|
||||
### 1.1 Current Shipped vs Not-Shipped Snapshot
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
Run scenario, stateful, and chaos checks against non-production Fastify services.
|
||||
|
||||
Qualify extends the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) with multi-step protocol flows, stateful sequences, and controlled fault injection.
|
||||
|
||||
## What Qualify Does
|
||||
|
||||
`apophis qualify` runs deeper testing than verify:
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
Deterministic contract verification for CI and local development.
|
||||
|
||||
APOPHIS implements the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable formulas, then verify them automatically with property-based generation and deterministic replay.
|
||||
|
||||
## When to Use It
|
||||
|
||||
- **Local development**: Quick feedback on behavioral changes
|
||||
|
||||
Reference in New Issue
Block a user