@push-stream-std/push-jsonl-parse (0.0.3)
Installation
@push-stream-std:registry=https://hub.kl1.tenere.ai/api/packages/push-stream-std/npm/npm install @push-stream-std/push-jsonl-parse@0.0.3"@push-stream-std/push-jsonl-parse": "0.0.3"About this package
push-jsonl-parse
Streaming JSONL parsing.
Published as @push-stream-std/push-jsonl-parse.
Synopsis
import { parseJsonl } from '@push-stream-std/push-jsonl-parse';
import { values } from '@push-stream-std/push-values';
import { collect } from '@push-stream-std/push-collect';
values(['{"id":1}\n{"id":2}\n', '{"id":3}\n'])
.pipe(parseJsonl())
.pipe(collect((err, results) => {
console.log(results); // [{id: 1}, {id: 2}, {id: 3}]
}));
Install
npm config set @push-stream-std:registry https://hub.kl1.tenere.ai/api/packages/push-stream-std/npm/
npm config set -- //hub.kl1.tenere.ai/api/packages/push-stream-std/npm/:_authToken "$PACKAGE_TOKEN"
npm install @push-stream-std/push-jsonl-parse
Description
A zero-dependency push-stream through operator that parses newline-delimited JSON (JSONL/NDJSON) with minimal overhead. Accumulates incoming string or Buffer chunks in an array (avoiding ConsString flatten/recreate cycles), only joining when a newline is detected. When the stream ends, any trailing partial line is parsed as a final JSON value if non-empty.
API
parseJsonl(options)
Creates a through stream that parses newline-delimited JSON from incoming data chunks.
Parameters:
options.maxBuffer(number, default0) — If greater than0, the stream will abort with an error when the accumulated buffer exceeds this byte count before a newline is found.
Returns: A push-stream through.
Tests
npm test
Tests cover single and multi-line JSONL parsing, chunked inputs across multiple writes, trailing partial lines at end, empty lines, Buffered inputs (toString('utf8')), malformed JSON error handling, and maxBuffer enforcement.
Requirements
Node.js 18 or later. ESM only.
License
MIT
Dependencies
Dependencies
| ID | Version |
|---|---|
| @streamparser/json | ^0.0.22 |
Development Dependencies
| ID | Version |
|---|---|
| fast-check | ^3.15.0 |
| ndjson | ^2.0.0 |
| stream-json | ^1.9.1 |