Files
apophis-fastify/docs/protocol-extensions-spec.md
T

600 lines
19 KiB
Markdown
Raw Normal View History

# APOPHIS Protocol Extensions Specification
## Status: Active design; shipped baseline: v2.0.0; remaining targets listed per feature
## 1. Overview
This specification defines protocol-specific extensions for APOPHIS, driven by the Arbiter team's requirements for testing OAuth 2.1, WIMSE S2S, Transaction Tokens (RFC 8693), SPIFFE/SPIRE, and related security protocols.
APOPHIS is grounded in [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021). Protocol extensions add domain-specific predicates (JWT, X.509, SPIFFE) to the core invariant framework.
Arbiter maintains 58 protocol conformance test files covering 138 behaviors across 7 specifications. These extensions bridge the gap between declarative APOSTL contracts and the domain-specific predicates required for security protocol validation.
### 1.1 Current Shipped vs Not-Shipped Snapshot
**Shipped in v2.0.0:**
- `contract({ variants })` for multi-header/media negotiation execution.
- `fastify.apophis.scenario(...)` for multi-step capture/rebind flows.
- `response_payload(this)` for JSON/LDF semantic payload access.
- Chaos testing (`chaos` config) for resilience/failure-path validation.
- Extension registration API (`extensions` plugin option).
**Not shipped yet:**
- Route-level `x-variants` schema extraction.
Use the shipped foundations today. Route-level `x-variants` is follow-up work.
### 1.2 Extension Registration
Register extensions via the plugin options:
```javascript
await fastify.register(apophis, {
extensions: [
jwtExtension({ jwks: 'https://auth.example.com/.well-known/jwks.json' }),
x509Extension(),
spiffeExtension(),
tokenHashExtension()
]
});
```
Extensions are loaded at plugin registration time and validated before routes are processed.
### 1.3 x-variants Status
Route-level `x-variants` schema extraction is **not shipped** yet. Use call-site `contract({ variants })` instead:
```javascript
const suite = await fastify.apophis.contract({
depth: 'quick',
variants: [
{ name: 'json', headers: { accept: 'application/json' } },
{ name: 'ldf', headers: { accept: 'application/ld+json' } },
],
});
```
### 1.4 Protocol Packs Status
Built-in protocol pack presets are **shipped**. Reference them by name in `apophis.config.js`:
```javascript
export default {
packs: ['oauth21'],
// User profiles and presets override pack defaults
};
```
Available packs:
- `oauth21` — OAuth 2.1 authorization code flow with PKCE
- `rfc8628-device-auth` — Device Authorization Grant
- `rfc8693-token-exchange` — Token Exchange
Packs resolve during config loading and merge profiles/presets into the config. User config always takes precedence.
---
## 2. Design Principles
### 2.1 Extension Architecture
All protocol extensions follow the v1.1 extension architecture:
```javascript
await fastify.register(apophis, {
extensions: [
jwtExtension({ jwks: 'https://auth.example.com/.well-known/jwks.json' }),
x509Extension(),
spiffeExtension(),
tokenHashExtension()
]
});
```
### 2.2 Configuration Per Route
Routes may need different validation keys or extraction sources:
```javascript
fastify.get('/wimse/wit', {
schema: {
'x-category': 'observer',
'x-extension-config': {
jwt: { verify: false, extractFrom: 'body' }
},
'x-ensures': [
'jwt_claims(this).sub != null',
'jwt_claims(this).cnf.jwk != null'
]
}
});
```
### 2.3 Test Data Seeding
Stateful tests may need pre-existing resources:
```javascript
await fastify.apophis.seed([
{ method: 'POST', url: '/oauth/clients', body: { client_id: 'test-client' } },
{ method: 'POST', url: '/wimse/wit', body: { workload: 'frontend' } }
]);
const results = await fastify.apophis.stateful({ depth: 'standard' });
```
---
## 3. JWT Extension
### 3.1 Use Cases
OAuth 2.1, Transaction Tokens, WIMSE S2S, SPIFFE JWT-SVID
### 3.2 Predicates
```apostl
# Access JWT claims
jwt_claims(this).sub # subject
jwt_claims(this).aud # audience
jwt_claims(this).iss # issuer
jwt_claims(this).exp # expiration (numeric timestamp)
jwt_claims(this).iat # issued at (numeric timestamp)
jwt_claims(this).jti # JWT ID (for replay detection)
jwt_claims(this).scope # scope
jwt_claims(this).cnf.jwk # confirmation key (WIMSE)
jwt_claims(this).txn # transaction token ID
# Access JWT header
jwt_header(this).alg # algorithm
jwt_header(this).kid # key ID
jwt_header(this).typ # type
# Validation
jwt_valid(this) # signature verifies against known key
jwt_format(this) == "compact" # compact vs JSON serialization
```
### 3.3 Configuration
```javascript
jwtExtension({
jwks: 'https://auth.example.com/.well-known/jwks.json',
extractFrom: 'authorization',
verify: true,
})
```
### 3.4 Extension State
The JWT extension maintains state across a test run:
```javascript
/**
* JWT extension state across a test run.
* @property {Set<string>} seenJtis - Track seen JTIs for replay detection
* @property {Map<string, DecodedJwt>} decodedCache - Cached decoded JWTs
*/
const jwtExtensionState = {
seenJtis: new Set(),
decodedCache: new Map()
};
```
### 3.5 Example Contracts
```apostl
# OAuth 2.1: Token response contains required claims
if response_code(this) == 200 then jwt_claims(this).sub != null else T
if response_code(this) == 200 then jwt_claims(this).exp > jwt_claims(this).iat else T
# WIMSE: WPT expiration must be short-lived
if response_code(this) == 200 then jwt_claims(this).exp <= jwt_claims(this).iat + 30 else T
# Transaction Tokens: Token type must be transaction_token
if response_code(this) == 200 then jwt_claims(this).txn != null else T
```
### 3.6 Implementation Notes
- Decode Base64URL without verification for claim inspection
- Verify signatures using configured JWKS or key material
- Support extracting JWT from multiple sources
- Track `seen_jtis` for replay detection within a test run
---
## 4. Time Control Extension
### 4.1 Problem
Many protocol behaviors depend on time:
- Token expiration (JWT `exp` claim)
- Refresh token rotation windows
- WIMSE WPT short TTL (≤30 seconds)
- Challenge TTLs
Current limitation: APOSTL has `response_time(this)` (wall clock duration) but no way to compare JWT timestamps to "now" or fast-forward time.
### 4.2 Predicates
```apostl
# Compare JWT exp to current time (server time)
jwt_claims(this).exp > now()
jwt_claims(this).exp <= now() + 30
# Time since previous request
response_time(this) <= 5000 # already exists
elapsed_since_previous(this) <= 30 # new: seconds since last request in stateful test
```
### 4.3 Server-Level Time Mocking
```javascript
await fastify.register(apophis, {
timeMock: true // enables apophis.time control
});
// In tests or stateful sequences:
await fastify.apophis.time.advance(30000); // +30 seconds
await fastify.apophis.time.set('2026-04-25T12:00:00Z');
```
### 4.4 Implementation
```javascript
/**
* Time control for deterministic testing.
* @property {function(number): void} advance - Advance simulated time by milliseconds
* @property {function(string): void} set - Set simulated time to specific ISO timestamp
* @property {function(): number} now - Get current simulated time
* @property {function(): void} reset - Reset to real time
*/
const timeControl = {
advance(ms) { /* ... */ },
set(isoString) { /* ... */ },
now() { return Date.now(); },
reset() { /* ... */ }
};
```
The `now()` predicate returns simulated time when time mocking is enabled, or the host wall clock outside deterministic test mode. Deterministic runs must inject or freeze time.
### 4.5 DST Testing Example
```apostl
# Test that tokens issued before DST transition work after
if previous(jwt_claims(this).iat).hour == 1 then jwt_valid(this) == true else T
```
---
## 5. Stateful Cross-Request Predicates
### 5.1 Problem
Protocols have multi-step flows where step N depends on step N-1:
1. **OAuth 2.1 refresh token rotation:** First refresh succeeds and returns NEW token. Second refresh with OLD token fails.
2. **Transaction token single-use:** First consumption succeeds. Second consumption with same token fails.
3. **WIMSE WPT replay:** First verification succeeds. Second verification with same jti fails.
Current limitation: `previous()` only compares values, not state transitions.
### 5.2 Predicates
```apostl
# Check if token was seen in previous requests
already_seen(this, jwt_claims(this).jti) == false
# Check if token was consumed
is_consumed(this, jwt_claims(this).jti) == false
# Reference specific previous request by category
previous(constructor).jwt_claims(this).refresh_token # last constructor's refresh token
previous(mutator).jwt_claims(this).txn # last mutator's transaction token
previous(observer).jwt_claims(this).jti # last observer's JWT ID
```
### 5.3 Implementation
Extension state tracks tokens across requests:
```javascript
/**
* Stateful extension state tracking tokens across requests.
* @property {Set<string>} seenTokens - Tokens observed in previous requests
* @property {Set<string>} consumedTokens - Tokens that have been consumed
* @property {Map<string, EvalContext>} categoryHistory - category -> last context
*/
const statefulExtensionState = {
seenTokens: new Set(),
consumedTokens: new Set(),
categoryHistory: new Map()
};
```
### 5.4 Example Contracts
```apostl
# OAuth 2.1 refresh: new token must differ from old
if response_code(this) == 200 then
response_body(this).refresh_token != previous(request_body(this)).refresh_token
else T
# Transaction token: single use
if response_code(this) == 409 then
response_body(this).error == "transaction_token_replay_detected" &&
already_seen(this, jwt_claims(this).jti) == true
else T
```
---
## 6. X.509 Extension
### 6.1 Use Cases
SPIFFE X509-SVID, mTLS certificate validation
### 6.2 Predicates
```apostl
# Certificate properties
x509_uri_sans(this) # array of URI subject alternative names
x509_uri_sans(this).length # count of URI SANs
x509_ca(this) # is CA certificate? (boolean)
x509_expired(this) # is expired? (boolean)
x509_not_before(this) # notBefore timestamp
x509_not_after(this) # notAfter timestamp
# Chain validation (lightweight)
x509_self_signed(this) # is self-signed?
x509_issuer(this) # issuer DN
x509_subject(this) # subject DN
```
### 6.3 Explicitly Out of Scope
- `x509_chain_valid(this)` — APOPHIS does not implement RFC 5280 path validation. Applications may expose chain-validation results and test them as ordinary response behavior.
### 6.4 Example Contracts
```apostl
# SPIFFE: X509-SVID must have exactly 1 URI SAN
if response_code(this) == 200 then x509_uri_sans(this).length == 1 else T
# SPIFFE: X509-SVID leaf must not be CA
if response_code(this) == 200 then x509_ca(this) == false else T
# SPIFFE: Certificate must not be expired
if response_code(this) == 200 then x509_expired(this) == false else T
```
---
## 7. SPIFFE Extension
### 7.1 Use Cases
SPIFFE ID validation, trust domain checks
### 7.2 Predicates
```apostl
# SPIFFE ID parsing
spiffe_parse(this).trustDomain # trust domain string
spiffe_parse(this).path # path segments (array)
spiffe_parse(this).path.length # path depth
spiffe_validate(this) # boolean: valid SPIFFE ID?
# Properties
spiffe_id(this) # full SPIFFE ID string
spiffe_trust_domain(this) # alias for spiffe_parse(this).trustDomain
```
### 7.3 Example Contracts
```apostl
# SPIFFE: Trust domain must be lowercase
if response_code(this) == 200 then spiffe_parse(this).trustDomain matches "^[a-z0-9.-]+$" else T
# SPIFFE: Path must not be empty
if response_code(this) == 200 then spiffe_parse(this).path.length > 0 else T
# SPIFFE: ID must be valid
if response_code(this) == 200 then spiffe_validate(this) == true else T
```
---
## 8. Token Hash Extension
### 8.1 Use Cases
WIMSE S2S `ath` (access token hash), `tth` (transaction token hash), `oth` (other token hash)
### 8.2 Predicates
```apostl
# Token hash validation
ath_valid(this) # access token hash matches Authorization header
tth_valid(this) # transaction token hash matches Txn-Token header
oth_valid(this, "header-name") # custom token hash matches named header
# Raw hash computation
token_hash(this, "sha256") # SHA-256 hash of token from context
```
### 8.3 Example Contracts
```apostl
# WIMSE: If ath claim present, must match access token
if jwt_claims(this).ath != null then ath_valid(this) == true else T
# WIMSE: If tth claim present, must match transaction token
if jwt_claims(this).tth != null then tth_valid(this) == true else T
```
---
## 9. HTTP Signature Extension
### 9.1 Use Cases
WIMSE S2S detached HTTP signatures
### 9.2 Predicates
```apostl
# Signature components
signature_input(this) # Signature-Input header parsed
signature(this) # Signature header value
signature_valid(this) # signature verifies against key
# Coverage
signature_covers(this, "@method") # covers HTTP method
signature_covers(this, "@request-target") # covers request target
signature_covers(this, "authorization") # covers auth header
signature_covers(this, "txn-token") # covers txn-token header
```
### 9.3 Example Contracts
```apostl
# WIMSE: Signature must cover @method and @request-target
if response_code(this) == 200 then signature_covers(this, "@method") == true else T
if response_code(this) == 200 then signature_covers(this, "@request-target") == true else T
```
---
## 10. Request Context Extension
### 10.1 Predicates
```apostl
# URL components
request_url(this) # full URL
request_url(this).path # path only
request_url(this).host # host header
# TLS info (when available)
request_tls(this).cipher # TLS cipher suite
request_tls(this).version # TLS version
request_tls(this).client_cert # client certificate (if mTLS)
# Body hash (for content integrity)
request_body_hash(this, "sha256") # SHA-256 of raw request body
```
### 10.2 Example Contracts
```apostl
# WIMSE audience validation: WPT aud claim must match request URL
if response_code(this) == 200 then jwt_claims(this).aud == request_url(this) else T
```
---
## 11. Priority Matrix
| Feature | Impact | Effort | Priority | Protocols Needing It |
|---------|--------|--------|----------|---------------------|
| JWT extension (claims + validation) | Very High | Medium | **P0** | OAuth 2.1, WIMSE, Txn Tokens, SPIFFE |
| Time control (`now()`, `advance()`) | Very High | Medium | **P0** | OAuth 2.1, WIMSE, Txn Tokens, CIBA |
| Stateful predicates (`previous()`, `already_seen()`) | High | Medium | **P1** | OAuth 2.1, Txn Tokens, WIMSE |
| X.509 extension (basic properties) | High | Low | **P1** | SPIFFE, WIMSE |
| SPIFFE extension | Medium | Low | **P2** | SPIFFE |
| Token hash extension | Medium | Low | **P2** | WIMSE |
| HTTP signature extension | Medium | Medium | **P2** | WIMSE |
| Request context (`request_url()`) | Medium | Low | **P2** | WIMSE |
| Parallel execution | Low | High | **P3** | — |
---
## 12. Protocol Test Inventory
| Protocol | Test File | Behaviors | Needs Extensions |
|----------|-----------|-----------|------------------|
| OAuth 2.1 | `oauth21-profile-conformance.test.js` | 13 | JWT, time control |
| WIMSE S2S | `draft-wimse-s2s-protocol-conformance.test.js` | 31 | JWT, token hash, HTTP sig, X.509 |
| Transaction Tokens | `draft-oauth-transaction-tokens-conformance.test.js` | 25 | JWT, time control, stateful |
| SPIFFE/SPIRE | `spiffe-spire-conformance.test.js` | 24 | SPIFFE, X.509, JWT |
| Token Exchange | `rfc8693-token-exchange-conformance.test.js` | 15 | JWT |
| Device Auth | `rfc8628-device-authorization-conformance.test.js` | 12 | JWT |
| CIBA | `ciba-conformance.test.js` | 18 | JWT, time control |
**Total: 138 protocol behaviors across 7 specifications.**
---
## 13. Out of Scope
We acknowledge these are too complex or inappropriate for Apophis:
| Feature | Why Out of Scope |
|---------|-----------------|
| Replay detection across restarts | Cross-run replay detection requires application-owned persistent state. |
| Full X.509 chain validation | Requires trust store, CRL/OCSP, and policy validation. Applications may expose the result for APOPHIS to check. |
| Cryptographic algorithm implementation | Apophis should not implement crypto. It should verify signatures using existing libraries. |
| Protocol state machines | Full state-machine extraction is still out of scope at route-schema level, but protocol flows are supported through `fastify.apophis.scenario(...)` and can be combined with `contract({ variants })` and APOSTL formulas. |
| Network-level testing | TCP behavior, packet inspection, MTU issues. Out of scope for HTTP contract testing. |
| Parallel execution for race detection | Can be tested with separate load testing tools. Not essential for contract testing. |
---
## 14. Implementation Plan
### Phase 1: JWT + Time Control (P0) — Shipped in v2.0.0
**Status**: Complete
**Files**:
- `src/extensions/jwt.ts` — JWT extension implementation
- `src/extensions/time.ts` — Time control extension
- `src/extensions/stateful.ts` — Stateful predicates extension
- `src/test/protocol-extensions.test.ts` — Protocol extension tests
- `src/test/cli/protocol-conformance-p2.test.ts` — Protocol conformance tests
**Tests**:
- Decode Base64URL claims without verification
- Verify signatures against JWKS
- Extract from multiple sources (header, body, query)
- `seen_jtis` replay detection
- `now()` predicate with mocked time
- `apophis.time.advance()` in stateful tests
### Phase 2: X.509 + SPIFFE (P1) — Shipped in v2.0.0
**Status**: Complete
**Files**:
- `src/extensions/x509.ts` — X.509 extension
- `src/extensions/spiffe.ts` — SPIFFE extension
- `src/test/protocol-extensions.test.ts` — Protocol extension tests
### Phase 3: Token Hash + HTTP Signature (P2) — Shipped in v2.0.0
**Status**: Complete
**Files**:
- `src/extensions/token-hash.ts` — Token hash extension
- `src/extensions/http-signature.ts` — HTTP signature extension
- `src/test/protocol-extensions.test.ts` — Protocol extension tests
### Phase 4: Request Context (P2) — Shipped in v2.0.0
**Status**: Complete
**Files**:
- `src/extensions/request-context.ts` — Request context predicates
- `src/test/protocol-extensions.test.ts` — Protocol extension tests
---
## 15. References
### Codebase Citations
- **Extension architecture**: `docs/extensions/EXTENSION-ARCHITECTURE.md`
- **Extension types**: `src/extension/types.ts`
- **Extension registry**: `src/extension/registry.ts`
- **Formula parser**: `src/formula/parser.ts`
- **Test runner**: `src/test/petit-runner.ts`
### External References
- JWT RFC 7519: https://tools.ietf.org/html/rfc7519
- WIMSE S2S: https://datatracker.ietf.org/doc/draft-ietf-wimse-s2s-protocol/
- Transaction Tokens (RFC 8693): https://tools.ietf.org/html/rfc8693
- SPIFFE/SPIRE: https://spiffe.io/
- OAuth 2.1: https://datatracker.ietf.org/doc/draft-ietf-oauth-v2-1/
---
*Document Version: 1.0*
*Author: APOPHIS Architecture Team*
*Date: 2026-04-25*
*Source Feedback: docs/attic/root-history/FEEDBACK-protocol-extensions-wishlist.md*