Initial public release of Apophis — invariant-driven automated API testing

2026-03-10 00:00:00 -07:00
parent d278c4b105
commit 3ac1daf7e9
82 changed files with 3902 additions and 1098 deletions
@@ -0,0 +1,599 @@
+# APOPHIS Protocol Extensions Specification
+
+## Status: Active design; shipped baseline: v2.0.0; remaining targets listed per feature
+
+## 1. Overview
+
+This specification defines protocol-specific extensions for APOPHIS, driven by the Arbiter team's requirements for testing OAuth 2.1, WIMSE S2S, Transaction Tokens (RFC 8693), SPIFFE/SPIRE, and related security protocols.
+
+APOPHIS is grounded in [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021). Protocol extensions add domain-specific predicates (JWT, X.509, SPIFFE) to the core invariant framework.
+
+Arbiter maintains 58 protocol conformance test files covering 138 behaviors across 7 specifications. These extensions bridge the gap between declarative APOSTL contracts and the domain-specific predicates required for security protocol validation.
+
+### 1.1 Current Shipped vs Not-Shipped Snapshot
+
+**Shipped in v2.0.0:**
+
+- `contract({ variants })` for multi-header/media negotiation execution.
+- `fastify.apophis.scenario(...)` for multi-step capture/rebind flows.
+- `response_payload(this)` for JSON/LDF semantic payload access.
+- Chaos testing (`chaos` config) for resilience/failure-path validation.
+- Extension registration API (`extensions` plugin option).
+
+**Not shipped yet:**
+
+- Route-level `x-variants` schema extraction.
+
+Use the shipped foundations today. Route-level `x-variants` is follow-up work.
+
+### 1.2 Extension Registration
+
+Register extensions via the plugin options:
+
+```javascript
+await fastify.register(apophis, {
+  extensions: [
+    jwtExtension({ jwks: 'https://auth.example.com/.well-known/jwks.json' }),
+    x509Extension(),
+    spiffeExtension(),
+    tokenHashExtension()
+  ]
+});
+```
+
+Extensions are loaded at plugin registration time and validated before routes are processed.
+
+### 1.3 x-variants Status
+
+Route-level `x-variants` schema extraction is **not shipped** yet. Use call-site `contract({ variants })` instead:
+
+```javascript
+const suite = await fastify.apophis.contract({
+  depth: 'quick',
+  variants: [
+    { name: 'json', headers: { accept: 'application/json' } },
+    { name: 'ldf', headers: { accept: 'application/ld+json' } },
+  ],
+});
+```
+
+### 1.4 Protocol Packs Status
+
+Built-in protocol pack presets are **shipped**. Reference them by name in `apophis.config.js`:
+
+```javascript
+export default {
+  packs: ['oauth21'],
+  // User profiles and presets override pack defaults
+};
+```
+
+Available packs:
+- `oauth21` — OAuth 2.1 authorization code flow with PKCE
+- `rfc8628-device-auth` — Device Authorization Grant
+- `rfc8693-token-exchange` — Token Exchange
+
+Packs resolve during config loading and merge profiles/presets into the config. User config always takes precedence.
+
+---
+
+## 2. Design Principles
+
+### 2.1 Extension Architecture
+All protocol extensions follow the v1.1 extension architecture:
+
+```javascript
+await fastify.register(apophis, {
+  extensions: [
+    jwtExtension({ jwks: 'https://auth.example.com/.well-known/jwks.json' }),
+    x509Extension(),
+    spiffeExtension(),
+    tokenHashExtension()
+  ]
+});
+```
+
+### 2.2 Configuration Per Route
+Routes may need different validation keys or extraction sources:
+
+```javascript
+fastify.get('/wimse/wit', {
+  schema: {
+    'x-category': 'observer',
+    'x-extension-config': {
+      jwt: { verify: false, extractFrom: 'body' }
+    },
+    'x-ensures': [
+      'jwt_claims(this).sub != null',
+      'jwt_claims(this).cnf.jwk != null'
+    ]
+  }
+});
+```
+
+### 2.3 Test Data Seeding
+Stateful tests may need pre-existing resources:
+
+```javascript
+await fastify.apophis.seed([
+  { method: 'POST', url: '/oauth/clients', body: { client_id: 'test-client' } },
+  { method: 'POST', url: '/wimse/wit', body: { workload: 'frontend' } }
+]);
+
+const results = await fastify.apophis.stateful({ depth: 'standard' });
+```
+
+---
+
+## 3. JWT Extension
+
+### 3.1 Use Cases
+OAuth 2.1, Transaction Tokens, WIMSE S2S, SPIFFE JWT-SVID
+
+### 3.2 Predicates
+
+```apostl
+# Access JWT claims
+jwt_claims(this).sub              # subject
+jwt_claims(this).aud              # audience
+jwt_claims(this).iss              # issuer
+jwt_claims(this).exp              # expiration (numeric timestamp)
+jwt_claims(this).iat              # issued at (numeric timestamp)
+jwt_claims(this).jti              # JWT ID (for replay detection)
+jwt_claims(this).scope            # scope
+jwt_claims(this).cnf.jwk          # confirmation key (WIMSE)
+jwt_claims(this).txn              # transaction token ID
+
+# Access JWT header
+jwt_header(this).alg              # algorithm
+jwt_header(this).kid              # key ID
+jwt_header(this).typ              # type
+
+# Validation
+jwt_valid(this)                   # signature verifies against known key
+jwt_format(this) == "compact"     # compact vs JSON serialization
+```
+
+### 3.3 Configuration
+
+```javascript
+jwtExtension({
+  jwks: 'https://auth.example.com/.well-known/jwks.json',
+  extractFrom: 'authorization',
+  verify: true,
+})
+```
+
+### 3.4 Extension State
+The JWT extension maintains state across a test run:
+
+```javascript
+/**
+ * JWT extension state across a test run.
+ * @property {Set<string>} seenJtis - Track seen JTIs for replay detection
+ * @property {Map<string, DecodedJwt>} decodedCache - Cached decoded JWTs
+ */
+const jwtExtensionState = {
+  seenJtis: new Set(),
+  decodedCache: new Map()
+};
+```
+
+### 3.5 Example Contracts
+
+```apostl
+# OAuth 2.1: Token response contains required claims
+if response_code(this) == 200 then jwt_claims(this).sub != null else T
+if response_code(this) == 200 then jwt_claims(this).exp > jwt_claims(this).iat else T
+
+# WIMSE: WPT expiration must be short-lived
+if response_code(this) == 200 then jwt_claims(this).exp <= jwt_claims(this).iat + 30 else T
+
+# Transaction Tokens: Token type must be transaction_token
+if response_code(this) == 200 then jwt_claims(this).txn != null else T
+```
+
+### 3.6 Implementation Notes
+- Decode Base64URL without verification for claim inspection
+- Verify signatures using configured JWKS or key material
+- Support extracting JWT from multiple sources
+- Track `seen_jtis` for replay detection within a test run
+
+---
+
+## 4. Time Control Extension
+
+### 4.1 Problem
+Many protocol behaviors depend on time:
+- Token expiration (JWT `exp` claim)
+- Refresh token rotation windows
+- WIMSE WPT short TTL (≤30 seconds)
+- Challenge TTLs
+
+Current limitation: APOSTL has `response_time(this)` (wall clock duration) but no way to compare JWT timestamps to "now" or fast-forward time.
+
+### 4.2 Predicates
+
+```apostl
+# Compare JWT exp to current time (server time)
+jwt_claims(this).exp > now()
+jwt_claims(this).exp <= now() + 30
+
+# Time since previous request
+response_time(this) <= 5000  # already exists
+elapsed_since_previous(this) <= 30  # new: seconds since last request in stateful test
+```
+
+### 4.3 Server-Level Time Mocking
+
+```javascript
+await fastify.register(apophis, {
+  timeMock: true  // enables apophis.time control
+});
+
+// In tests or stateful sequences:
+await fastify.apophis.time.advance(30000);  // +30 seconds
+await fastify.apophis.time.set('2026-04-25T12:00:00Z');
+```
+
+### 4.4 Implementation
+
+```javascript
+/**
+ * Time control for deterministic testing.
+ * @property {function(number): void} advance - Advance simulated time by milliseconds
+ * @property {function(string): void} set - Set simulated time to specific ISO timestamp
+ * @property {function(): number} now - Get current simulated time
+ * @property {function(): void} reset - Reset to real time
+ */
+const timeControl = {
+  advance(ms) { /* ... */ },
+  set(isoString) { /* ... */ },
+  now() { return Date.now(); },
+  reset() { /* ... */ }
+};
+```
+
+The `now()` predicate returns simulated time when time mocking is enabled, or the host wall clock outside deterministic test mode. Deterministic runs must inject or freeze time.
+
+### 4.5 DST Testing Example
+
+```apostl
+# Test that tokens issued before DST transition work after
+if previous(jwt_claims(this).iat).hour == 1 then jwt_valid(this) == true else T
+```
+
+---
+
+## 5. Stateful Cross-Request Predicates
+
+### 5.1 Problem
+Protocols have multi-step flows where step N depends on step N-1:
+
+1. **OAuth 2.1 refresh token rotation:** First refresh succeeds and returns NEW token. Second refresh with OLD token fails.
+2. **Transaction token single-use:** First consumption succeeds. Second consumption with same token fails.
+3. **WIMSE WPT replay:** First verification succeeds. Second verification with same jti fails.
+
+Current limitation: `previous()` only compares values, not state transitions.
+
+### 5.2 Predicates
+
+```apostl
+# Check if token was seen in previous requests
+already_seen(this, jwt_claims(this).jti) == false
+
+# Check if token was consumed
+is_consumed(this, jwt_claims(this).jti) == false
+
+# Reference specific previous request by category
+previous(constructor).jwt_claims(this).refresh_token  # last constructor's refresh token
+previous(mutator).jwt_claims(this).txn                # last mutator's transaction token
+previous(observer).jwt_claims(this).jti               # last observer's JWT ID
+```
+
+### 5.3 Implementation
+
+Extension state tracks tokens across requests:
+
+```javascript
+/**
+ * Stateful extension state tracking tokens across requests.
+ * @property {Set<string>} seenTokens - Tokens observed in previous requests
+ * @property {Set<string>} consumedTokens - Tokens that have been consumed
+ * @property {Map<string, EvalContext>} categoryHistory - category -> last context
+ */
+const statefulExtensionState = {
+  seenTokens: new Set(),
+  consumedTokens: new Set(),
+  categoryHistory: new Map()
+};
+```
+
+### 5.4 Example Contracts
+
+```apostl
+# OAuth 2.1 refresh: new token must differ from old
+if response_code(this) == 200 then 
+  response_body(this).refresh_token != previous(request_body(this)).refresh_token 
+else T
+
+# Transaction token: single use
+if response_code(this) == 409 then 
+  response_body(this).error == "transaction_token_replay_detected" && 
+  already_seen(this, jwt_claims(this).jti) == true
+else T
+```
+
+---
+
+## 6. X.509 Extension
+
+### 6.1 Use Cases
+SPIFFE X509-SVID, mTLS certificate validation
+
+### 6.2 Predicates
+
+```apostl
+# Certificate properties
+x509_uri_sans(this)               # array of URI subject alternative names
+x509_uri_sans(this).length        # count of URI SANs
+x509_ca(this)                     # is CA certificate? (boolean)
+x509_expired(this)                # is expired? (boolean)
+x509_not_before(this)             # notBefore timestamp
+x509_not_after(this)              # notAfter timestamp
+
+# Chain validation (lightweight)
+x509_self_signed(this)            # is self-signed?
+x509_issuer(this)                 # issuer DN
+x509_subject(this)                # subject DN
+```
+
+### 6.3 Explicitly Out of Scope
+- `x509_chain_valid(this)` — APOPHIS does not implement RFC 5280 path validation. Applications may expose chain-validation results and test them as ordinary response behavior.
+
+### 6.4 Example Contracts
+
+```apostl
+# SPIFFE: X509-SVID must have exactly 1 URI SAN
+if response_code(this) == 200 then x509_uri_sans(this).length == 1 else T
+
+# SPIFFE: X509-SVID leaf must not be CA
+if response_code(this) == 200 then x509_ca(this) == false else T
+
+# SPIFFE: Certificate must not be expired
+if response_code(this) == 200 then x509_expired(this) == false else T
+```
+
+---
+
+## 7. SPIFFE Extension
+
+### 7.1 Use Cases
+SPIFFE ID validation, trust domain checks
+
+### 7.2 Predicates
+
+```apostl
+# SPIFFE ID parsing
+spiffe_parse(this).trustDomain    # trust domain string
+spiffe_parse(this).path           # path segments (array)
+spiffe_parse(this).path.length    # path depth
+spiffe_validate(this)             # boolean: valid SPIFFE ID?
+
+# Properties
+spiffe_id(this)                   # full SPIFFE ID string
+spiffe_trust_domain(this)         # alias for spiffe_parse(this).trustDomain
+```
+
+### 7.3 Example Contracts
+
+```apostl
+# SPIFFE: Trust domain must be lowercase
+if response_code(this) == 200 then spiffe_parse(this).trustDomain matches "^[a-z0-9.-]+$" else T
+
+# SPIFFE: Path must not be empty
+if response_code(this) == 200 then spiffe_parse(this).path.length > 0 else T
+
+# SPIFFE: ID must be valid
+if response_code(this) == 200 then spiffe_validate(this) == true else T
+```
+
+---
+
+## 8. Token Hash Extension
+
+### 8.1 Use Cases
+WIMSE S2S `ath` (access token hash), `tth` (transaction token hash), `oth` (other token hash)
+
+### 8.2 Predicates
+
+```apostl
+# Token hash validation
+ath_valid(this)                   # access token hash matches Authorization header
+tth_valid(this)                   # transaction token hash matches Txn-Token header
+oth_valid(this, "header-name")    # custom token hash matches named header
+
+# Raw hash computation
+token_hash(this, "sha256")        # SHA-256 hash of token from context
+```
+
+### 8.3 Example Contracts
+
+```apostl
+# WIMSE: If ath claim present, must match access token
+if jwt_claims(this).ath != null then ath_valid(this) == true else T
+
+# WIMSE: If tth claim present, must match transaction token
+if jwt_claims(this).tth != null then tth_valid(this) == true else T
+```
+
+---
+
+## 9. HTTP Signature Extension
+
+### 9.1 Use Cases
+WIMSE S2S detached HTTP signatures
+
+### 9.2 Predicates
+
+```apostl
+# Signature components
+signature_input(this)             # Signature-Input header parsed
+signature(this)                   # Signature header value
+signature_valid(this)             # signature verifies against key
+
+# Coverage
+signature_covers(this, "@method")          # covers HTTP method
+signature_covers(this, "@request-target")  # covers request target
+signature_covers(this, "authorization")    # covers auth header
+signature_covers(this, "txn-token")        # covers txn-token header
+```
+
+### 9.3 Example Contracts
+
+```apostl
+# WIMSE: Signature must cover @method and @request-target
+if response_code(this) == 200 then signature_covers(this, "@method") == true else T
+if response_code(this) == 200 then signature_covers(this, "@request-target") == true else T
+```
+
+---
+
+## 10. Request Context Extension
+
+### 10.1 Predicates
+
+```apostl
+# URL components
+request_url(this)                 # full URL
+request_url(this).path            # path only
+request_url(this).host            # host header
+
+# TLS info (when available)
+request_tls(this).cipher          # TLS cipher suite
+request_tls(this).version         # TLS version
+request_tls(this).client_cert     # client certificate (if mTLS)
+
+# Body hash (for content integrity)
+request_body_hash(this, "sha256") # SHA-256 of raw request body
+```
+
+### 10.2 Example Contracts
+
+```apostl
+# WIMSE audience validation: WPT aud claim must match request URL
+if response_code(this) == 200 then jwt_claims(this).aud == request_url(this) else T
+```
+
+---
+
+## 11. Priority Matrix
+
+| Feature | Impact | Effort | Priority | Protocols Needing It |
+|---------|--------|--------|----------|---------------------|
+| JWT extension (claims + validation) | Very High | Medium | **P0** | OAuth 2.1, WIMSE, Txn Tokens, SPIFFE |
+| Time control (`now()`, `advance()`) | Very High | Medium | **P0** | OAuth 2.1, WIMSE, Txn Tokens, CIBA |
+| Stateful predicates (`previous()`, `already_seen()`) | High | Medium | **P1** | OAuth 2.1, Txn Tokens, WIMSE |
+| X.509 extension (basic properties) | High | Low | **P1** | SPIFFE, WIMSE |
+| SPIFFE extension | Medium | Low | **P2** | SPIFFE |
+| Token hash extension | Medium | Low | **P2** | WIMSE |
+| HTTP signature extension | Medium | Medium | **P2** | WIMSE |
+| Request context (`request_url()`) | Medium | Low | **P2** | WIMSE |
+| Parallel execution | Low | High | **P3** | — |
+
+---
+
+## 12. Protocol Test Inventory
+
+| Protocol | Test File | Behaviors | Needs Extensions |
+|----------|-----------|-----------|------------------|
+| OAuth 2.1 | `oauth21-profile-conformance.test.js` | 13 | JWT, time control |
+| WIMSE S2S | `draft-wimse-s2s-protocol-conformance.test.js` | 31 | JWT, token hash, HTTP sig, X.509 |
+| Transaction Tokens | `draft-oauth-transaction-tokens-conformance.test.js` | 25 | JWT, time control, stateful |
+| SPIFFE/SPIRE | `spiffe-spire-conformance.test.js` | 24 | SPIFFE, X.509, JWT |
+| Token Exchange | `rfc8693-token-exchange-conformance.test.js` | 15 | JWT |
+| Device Auth | `rfc8628-device-authorization-conformance.test.js` | 12 | JWT |
+| CIBA | `ciba-conformance.test.js` | 18 | JWT, time control |
+
+**Total: 138 protocol behaviors across 7 specifications.**
+
+---
+
+## 13. Out of Scope
+
+We acknowledge these are too complex or inappropriate for Apophis:
+
+| Feature | Why Out of Scope |
+|---------|-----------------|
+| Replay detection across restarts | Cross-run replay detection requires application-owned persistent state. |
+| Full X.509 chain validation | Requires trust store, CRL/OCSP, and policy validation. Applications may expose the result for APOPHIS to check. |
+| Cryptographic algorithm implementation | Apophis should not implement crypto. It should verify signatures using existing libraries. |
+| Protocol state machines | Full state-machine extraction is still out of scope at route-schema level, but protocol flows are supported through `fastify.apophis.scenario(...)` and can be combined with `contract({ variants })` and APOSTL formulas. |
+| Network-level testing | TCP behavior, packet inspection, MTU issues. Out of scope for HTTP contract testing. |
+| Parallel execution for race detection | Can be tested with separate load testing tools. Not essential for contract testing. |
+
+---
+
+## 14. Implementation Plan
+
+### Phase 1: JWT + Time Control (P0) — Shipped in v2.0.0
+**Status**: Complete
+**Files**:
+- `src/extensions/jwt.ts` — JWT extension implementation
+- `src/extensions/time.ts` — Time control extension
+- `src/extensions/stateful.ts` — Stateful predicates extension
+- `src/test/protocol-extensions.test.ts` — Protocol extension tests
+- `src/test/cli/protocol-conformance-p2.test.ts` — Protocol conformance tests
+
+**Tests**:
+- Decode Base64URL claims without verification
+- Verify signatures against JWKS
+- Extract from multiple sources (header, body, query)
+- `seen_jtis` replay detection
+- `now()` predicate with mocked time
+- `apophis.time.advance()` in stateful tests
+
+### Phase 2: X.509 + SPIFFE (P1) — Shipped in v2.0.0
+**Status**: Complete
+**Files**:
+- `src/extensions/x509.ts` — X.509 extension
+- `src/extensions/spiffe.ts` — SPIFFE extension
+- `src/test/protocol-extensions.test.ts` — Protocol extension tests
+
+### Phase 3: Token Hash + HTTP Signature (P2) — Shipped in v2.0.0
+**Status**: Complete
+**Files**:
+- `src/extensions/token-hash.ts` — Token hash extension
+- `src/extensions/http-signature.ts` — HTTP signature extension
+- `src/test/protocol-extensions.test.ts` — Protocol extension tests
+
+### Phase 4: Request Context (P2) — Shipped in v2.0.0
+**Status**: Complete
+**Files**:
+- `src/extensions/request-context.ts` — Request context predicates
+- `src/test/protocol-extensions.test.ts` — Protocol extension tests
+
+---
+
+## 15. References
+
+### Codebase Citations
+- **Extension architecture**: `docs/extensions/EXTENSION-ARCHITECTURE.md`
+- **Extension types**: `src/extension/types.ts`
+- **Extension registry**: `src/extension/registry.ts`
+- **Formula parser**: `src/formula/parser.ts`
+- **Test runner**: `src/test/petit-runner.ts`
+
+### External References
+- JWT RFC 7519: https://tools.ietf.org/html/rfc7519
+- WIMSE S2S: https://datatracker.ietf.org/doc/draft-ietf-wimse-s2s-protocol/
+- Transaction Tokens (RFC 8693): https://tools.ietf.org/html/rfc8693
+- SPIFFE/SPIRE: https://spiffe.io/
+- OAuth 2.1: https://datatracker.ietf.org/doc/draft-ietf-oauth-v2-1/
+
+---
+
+*Document Version: 1.0*
+*Author: APOPHIS Architecture Team*
+*Date: 2026-04-25*
+*Source Feedback: docs/attic/root-history/FEEDBACK-protocol-extensions-wishlist.md*