diff --git a/README.md b/README.md index 296f826..7ebf9de 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ Behavioral confidence for Fastify services. APOPHIS checks whether route behavior holds across operations, states, and protocol flows. +Inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only checking payload shape, APOPHIS encodes intended behavior as executable contracts and verifies them with property-based and stateful testing. + Supported Node.js versions: 20.x and 22.x. ```bash diff --git a/SKILL.md b/SKILL.md index a826d8d..f61af95 100644 --- a/SKILL.md +++ b/SKILL.md @@ -7,6 +7,8 @@ description: Use this skill when adding or improving APOPHIS contract-driven tes APOPHIS verifies API behavior across operations, state changes, protocol flows, and dependencies. Use it when schema validation is not enough to answer whether an endpoint did the right thing. +Inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable contracts, then verify them with property-based and stateful testing. + ## When To Use Use this skill when the operator asks to: @@ -76,6 +78,7 @@ When entering a Fastify codebase: import Fastify from 'fastify' import swagger from '@fastify/swagger' import apophis from 'apophis-fastify' +import crypto from 'crypto' const app = Fastify() await app.register(swagger) @@ -114,8 +117,9 @@ app.post('/users', { } } }, async (req, reply) => { + const id = `usr-${crypto.createHash('sha256').update(req.body.email).digest('hex').slice(0, 8)}` reply.status(201) - return { id: 'usr-1', ...req.body } + return { id, ...req.body } }) await app.ready() @@ -350,6 +354,31 @@ Operator framing: > The failing seed gives us a reproducible behavioral example. I'll replay it first so we can distinguish a real regression from source drift or nondeterministic app state. +## Progressive Complexity + +Start simple and add depth only where it pays off: + +**Level 1 — Status and shape**: Every route gets an expected status code and key field existence. +```apostl +status:201 +response_body(this).id != null +``` + +**Level 2 — Cross-route behavior**: Constructors check retrievability; mutators check persistence. +```apostl +response_code(GET /users/{response_body(this).id}) == 200 +response_body(GET /users/{response_body(this).id}).email == request_body(this).email +``` + +**Level 3 — Isolation and boundaries**: Tenant, auth, and idempotency checks. +```apostl +if request_headers(this).x-tenant-id != null then response_headers(this).x-tenant-id == request_headers(this).x-tenant-id else true +``` + +**Level 4 — Protocol and dependency flows**: Variants, scenarios, outbound contracts, and chaos. + +Add level 2 before level 4. Do not skip level 2 for resource APIs. + ## Anti-Patterns Do not: diff --git a/docs/auth-patterns.md b/docs/auth-patterns.md index 8667f1f..804b24a 100644 --- a/docs/auth-patterns.md +++ b/docs/auth-patterns.md @@ -144,7 +144,7 @@ See `docs/protocol-extensions-spec.md` for full JWT extension configuration. `getToken` runs per request. Handle refresh inline: ```javascript -let cachedToken: string | null = null +let cachedToken = null const auth = createAuthExtension({ name: 'jwt-with-refresh', diff --git a/docs/chaos.md b/docs/chaos.md index 774cad2..44df7ed 100644 --- a/docs/chaos.md +++ b/docs/chaos.md @@ -2,6 +2,8 @@ Inject controlled failures into contract tests to validate resilience guarantees. +Chaos testing applies the invariant-driven verification approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) under adverse conditions: if a contract must hold, it should still hold when dependencies fail, responses are delayed, or payloads are corrupted. + ## Usage ```javascript diff --git a/docs/fastify-structure.md b/docs/fastify-structure.md index 571f6f0..428f20a 100644 --- a/docs/fastify-structure.md +++ b/docs/fastify-structure.md @@ -374,8 +374,10 @@ import { mkdirSync, rmSync } from 'fs' import { tmpdir } from 'os' import { join } from 'path' +let testCounter = 0 + export function createTestWorkspace() { - const dir = join(tmpdir(), `apophis-test-${Date.now()}`) + const dir = join(tmpdir(), `apophis-test-${++testCounter}`) mkdirSync(dir, { recursive: true }) return { diff --git a/docs/getting-started.md b/docs/getting-started.md index 9f6f197..e001100 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -2,6 +2,8 @@ Get from install to your first behavioral bug in 10 minutes. +APOPHIS is inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only validating request and response shape, encode intended behavior as executable contracts and let the tool find violations automatically. + ## Prerequisites - Node.js 20.x or 22.x @@ -70,7 +72,7 @@ Expected response_code(GET /users/{response_body(this).id}) == 200 Observed - GET /users/usr-123 returned 404 + GET /users/usr-7d865e returned 404 Why this matters The resource created by POST /users is not retrievable. diff --git a/docs/llm-safe-adoption.md b/docs/llm-safe-adoption.md index d51af63..bcd7628 100644 --- a/docs/llm-safe-adoption.md +++ b/docs/llm-safe-adoption.md @@ -2,6 +2,8 @@ APOPHIS is designed to be safe and predictable for LLM-generated Fastify services. +It applies the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) to LLM-assisted development: constrained vocabulary, deterministic replay, and executable contracts give coding agents a verifiable loop between generated changes and behavioral correctness. + ## Why APOPHIS Is Good for LLM-Generated Services Coding agents benefit from: diff --git a/docs/observe.md b/docs/observe.md index fcac7d3..3e00311 100644 --- a/docs/observe.md +++ b/docs/observe.md @@ -2,6 +2,8 @@ Runtime visibility and drift detection without blocking by default. +Observe extends the invariant framework from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) to production environments: contracts run continuously against live traffic to detect behavioral drift without affecting requests. + ## What Observe Does `apophis observe` validates your runtime observe configuration: @@ -153,9 +155,9 @@ observe: { } ``` -## Workspace Support +## Monorepo Validation -For monorepos, use `apophis doctor --workspace` to validate observe configuration across all workspace packages. +For monorepos, use `apophis doctor --workspace` to validate observe configuration across all workspace packages. `observe` itself does not support `--workspace`; use `doctor` to check config in each package. ## Mode Mismatch diff --git a/docs/protocol-extensions-spec.md b/docs/protocol-extensions-spec.md index 71eef6c..7309abb 100644 --- a/docs/protocol-extensions-spec.md +++ b/docs/protocol-extensions-spec.md @@ -6,6 +6,8 @@ This specification defines protocol-specific extensions for APOPHIS, driven by the Arbiter team's requirements for testing OAuth 2.1, WIMSE S2S, Transaction Tokens (RFC 8693), SPIFFE/SPIRE, and related security protocols. +APOPHIS is grounded in [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021). Protocol extensions add domain-specific predicates (JWT, X.509, SPIFFE) to the core invariant framework. + Arbiter maintains 58 protocol conformance test files covering 138 behaviors across 7 specifications. These extensions bridge the gap between declarative APOSTL contracts and the domain-specific predicates required for security protocol validation. ### 1.1 Current Shipped vs Not-Shipped Snapshot diff --git a/docs/qualify.md b/docs/qualify.md index 8783eca..345060b 100644 --- a/docs/qualify.md +++ b/docs/qualify.md @@ -2,6 +2,8 @@ Run scenario, stateful, and chaos checks against non-production Fastify services. +Qualify extends the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) with multi-step protocol flows, stateful sequences, and controlled fault injection. + ## What Qualify Does `apophis qualify` runs deeper testing than verify: diff --git a/docs/verify.md b/docs/verify.md index 22a68b4..a8523a7 100644 --- a/docs/verify.md +++ b/docs/verify.md @@ -2,6 +2,8 @@ Deterministic contract verification for CI and local development. +APOPHIS implements the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable formulas, then verify them automatically with property-based generation and deterministic replay. + ## When to Use It - **Local development**: Quick feedback on behavioral changes