John Dvorak
6c39bd0a6c
- Fix const inference bug: wrap inferred contracts with status-code guards - Add integration test for status-guarded contract inference - Tighten and deduplicate docs across verify, qualify, getting-started, cli - Fix broken cross-references and TypeScript→JavaScript conversions - Fix factual errors: license, Date.now(), sampling defaults, cache env - Add missing features: --workspace, --generation-profile, json-summary formats - Move stale extension docs (AUTH-RATE-LIMIT-REVISED, HTTP-EXTENSIONS) to attic - Update PLUGIN_CONTRACTS_SPEC status to Implemented - Build: clean | Tests: 849 pass, 0 fail
144 lines
3.1 KiB
Markdown
144 lines
3.1 KiB
Markdown
# Chaos Mode
|
|
|
|
Inject controlled failures into contract tests to validate resilience guarantees.
|
|
|
|
## Usage
|
|
|
|
```javascript
|
|
const result = await fastify.apophis.contract({
|
|
depth: 'standard',
|
|
chaos: {
|
|
probability: 0.1, // 10% of requests get chaos
|
|
delay: { probability: 1, minMs: 100, maxMs: 500 },
|
|
error: { probability: 1, statusCode: 503 },
|
|
dropout: { probability: 1 },
|
|
corruption: { probability: 1 },
|
|
},
|
|
});
|
|
```
|
|
|
|
## Event Types
|
|
|
|
### Delay
|
|
|
|
Adds artificial latency. Tests timeout contracts:
|
|
|
|
```apostl
|
|
timeout_occurred(this) == false
|
|
response_time(this) < 1000
|
|
```
|
|
|
|
### Error
|
|
|
|
Forces HTTP status codes. Tests error-handling contracts:
|
|
|
|
```apostl
|
|
if status:503 then response_body(this).retry_after != null
|
|
```
|
|
|
|
### Dropout
|
|
|
|
Simulates network failure (status 0). Tests fallback contracts:
|
|
|
|
```apostl
|
|
status:200 || status:0
|
|
```
|
|
|
|
### Corruption
|
|
|
|
Mutates response bodies. Tests parsing robustness:
|
|
|
|
```apostl
|
|
response_body(this).id != null
|
|
```
|
|
|
|
## Corruption Strategies
|
|
|
|
Built-in strategies are content-type agnostic:
|
|
|
|
| Strategy | Effect |
|
|
|----------|--------|
|
|
| `truncate` | Cuts response body short |
|
|
| `malformed` | Invalidates structural boundaries (e.g., unclosed JSON, bad headers) |
|
|
| `field-corrupt` | Replaces a random field value with corrupted data |
|
|
|
|
Extension strategies can add content-type-specific behavior if needed.
|
|
|
|
## Custom Corruption via Extensions
|
|
|
|
```javascript
|
|
const myExtension = {
|
|
name: 'custom-corrupt',
|
|
corruptionStrategies: {
|
|
'application/vnd.api+json': (data) => ({
|
|
...data,
|
|
corrupted: true,
|
|
}),
|
|
'text/*': (data) => `CORRUPTED:${String(data)}`,
|
|
},
|
|
};
|
|
|
|
await fastify.register(apophis, {
|
|
extensions: [myExtension],
|
|
});
|
|
```
|
|
|
|
Extension strategies take precedence over built-ins. Wildcard patterns (`text/*`) match any subtype.
|
|
|
|
## Environment Guard
|
|
|
|
Low-level contract chaos APIs require `NODE_ENV=test`. For CLI qualification, environment policy controls whether chaos gates may run.
|
|
|
|
```
|
|
Error: chaos is only available in test environment. Set NODE_ENV=test to enable quality features.
|
|
```
|
|
|
|
## Interpreting Results
|
|
|
|
Failed tests include chaos events in diagnostics:
|
|
|
|
```json
|
|
{
|
|
"statusCode": 503,
|
|
"error": "Contract violation: status:200",
|
|
"chaosEvents": [
|
|
{
|
|
"type": "error",
|
|
"injected": true,
|
|
"details": {
|
|
"statusCode": 503,
|
|
"reason": "Chaos error: overridden 200 with 503"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
1. **Start small**: `probability: 0.05` (5% of requests)
|
|
2. **Test one failure mode at a time**: Comment out other chaos types
|
|
3. **Verify contracts handle chaos**: `if status:503 then response_body(this).error != null`
|
|
4. **Use seeds for reproducibility**: `seed: 42` makes chaos deterministic
|
|
|
|
## Example: Testing Retry Logic
|
|
|
|
```javascript
|
|
fastify.get('/data', {
|
|
schema: {
|
|
'x-ensures': [
|
|
'if status:503 then response_headers(this).retry-after != null',
|
|
'redirect_count(this) <= 3',
|
|
],
|
|
},
|
|
}, handler);
|
|
|
|
// Test
|
|
const result = await fastify.apophis.contract({
|
|
chaos: {
|
|
probability: 0.2,
|
|
error: { probability: 1, statusCode: 503 },
|
|
},
|
|
});
|
|
```
|