@polyweave/groq (1.0.7)

Published 2026-07-17 15:32:56 +00:00 by Dvorak

Set up this registry in your project .npmrc file:

@polyweave:registry=https://hub.kl1.tenere.ai/api/packages/Polyweave/npm/

To install the package using npm, run the following command:

npm install @polyweave/groq@1.0.7

or add it to the package.json file:

"@polyweave/groq": "1.0.7"

For more information on the npm registry, see the documentation.

@polyweave/groq

Groq LPU fast inference adapter for Polyweave. OpenAI-compatible chat completions and Whisper audio transcription. 500–1000 tokens/second.

Published as @polyweave/groq.

Synopsis

import { createGroqAdapter } from '@polyweave/groq';
import { createTextFrame, createFrameSink } from '@polyweave/core';

var adapter = createGroqAdapter({ apiKey: process.env.GROQ_API_KEY });

adapter.source.pipe(createFrameSink(function(frame) {
  if (frame.type === 'TEXT') console.log(frame.content.text);
  if (frame.type === 'RECEIPT') console.log('Cost:', frame.content.totalCost);
  if (frame.type === 'ERROR') console.error(frame.content.message);
}));

// Chat completion (streaming)
adapter.sink.write(createTextFrame('Explain quantum computing in one sentence'));

// Audio transcription
adapter.sink.write({ type: 'AUDIO', content: { data: audioBuffer } });

// Deferred API key
var adapter2 = createGroqAdapter({});
adapter2.source.pipe(createFrameSink(function(f) {
  if (f.type === 'TEXT') console.log(f.content.text);
}));
// Frames are buffered until key is set
adapter2.sink.write(createTextFrame('Hello'));
adapter2.sink.write({ type: 'PARAM', content: { key: 'apiKey', value: process.env.GROQ_API_KEY } });
// Buffered frames are now drained

Install

npm install @polyweave/groq

Why

Groq's LPU (Language Processing Unit) delivers the fastest LLM inference in the market — 500 to 1000 tokens per second. This adapter exposes Groq's OpenAI-compatible chat completions API and Whisper audio transcription as a Polyweave frame duplex. Supports streaming SSE, batch (non-streaming), tool/function calling, structured output (response_format), and progressive receipt emission during stream. No external HTTP library — uses Node.js built-in https/http. Audio transcription uses multipart form-data uploads.

Models: llama-3.3-70b-versatile, llama-3.1-8b-instant, openai/gpt-oss-120b, openai/gpt-oss-20b, qwen/qwen3-32b, meta-llama/llama-4-scout-17b-16e-instruct, whisper-large-v3, whisper-large-v3-turbo.

API

`createGroqAdapter(config)`

Creates a Groq frame duplex adapter.

Parameter	Type	Required	Default	Description
`config.apiKey`	`string`	No	`process.env.GROQ_API_KEY \|\| null`	Groq API key
`config.baseURL`	`string`	No	`'https://api.groq.com/openai/v1'`	Base URL override
`config.model`	`string`	No	`'llama-3.3-70b-versatile'`	Default model (settable via PARAM `model` key)
`config.budget`	`number`	No	`0`	Hard budget cap. TEXT/AUDIO frames rejected once `totalSpent >= budget`
`config.name`	`string`	No	`null`	Node name for PARAM targeting
`config.timeout`	`number`	No	`120000`	Request timeout in ms

Return value: IFrameDuplex with source and sink, plus setReady(key) and name.

Default params: { model: 'llama-3.3-70b-versatile', max_tokens: 1024, stream: true }

Buffering: If apiKey is not set at construction time, frames are buffered in frameBuffer until setReady(key) is called. Once ready, buffered frames are drained sequentially.

Budget enforcement: Before TEXT or AUDIO frames are processed, if budgetTotal > 0 and totalSpent >= budgetTotal, the adapter emits ERROR (code: 'BUDGET_EXCEEDED') + END and drops the frame.

Frame Protocol

Sink (input frames):

Frame Type	Content Properties	Behavior
`TEXT`	`content.text` (string)	Appends to `conversation` array as `{ role: 'user', content: text }`. Calls `sendChatRequest` with full conversation history.
`AUDIO`	`content.data` (buffer or base64 string)	Calls `sendAudioRequest` for Whisper transcription. Emits ERROR if `content.data` is missing.
`PARAM`	`content.key`, `content.value`	Sets params. Special keys: `apiKey` calls `setReady(v)`, `budget` sets `budgetTotal`. All other keys stored in `params`.
`COST_CHECK`	`content.maxBudget` (number, optional)	Enters cost check mode. Buffers frames until non-ESTIMATED_OUTPUT frame. Estimates cost, emits COST frame + END. If `maxBudget` exceeded, emits ERROR (`'Estimated cost $X exceeds budget $Y'`) + END.
`ESTIMATED_OUTPUT`	—	Silently consumed in cost check mode
`START`	—	Silently ignored
`END`	—	Silently ignored
`TOPOLOGY_SCAN`	`content.path` (array, optional)	Emits TOPOLOGY_REPORT with `{ kind: 'duplex', id: 'groq-adapter', receives: ['TEXT','AUDIO','PARAM','START','END'], generates: ['TEXT','AUDIO','TOOL_CALL','ERROR','END','RECEIPT'] }`, then passes through
`TEST_INJECT`	`content.frames` (array)	Recursively writes each frame
`TEST_SNAPSHOT`	—	Emits TEST_SNAPSHOT back

Source (output frames):

Frame Type	Emitted When	Content
`TEXT`	Stream delta or batch response `message.content`	`{ text }` with metadata `{ model, usage?, finishReason? }`
`TOOL_CALL`	Tool call in stream or batch response	`{ name, arguments }` with `id` = toolCallId
`AUDIO`	Audio transcription result	`{ data, mimeType: 'audio/mpeg', transcription, model }`
`RECEIPT`	Stream progress (every 50 tokens), stream end, batch end, audio transcription, error (partial)	`{ inputTokens, outputTokens, inputCost, outputCost, totalCost }` with `{ provider: 'groq', model, progressive, final, streamId }`
`COST`	After COST_CHECK completes	Extended cost frame with `bounds`, `accuracy`, `estimateSource`, `forwardEstimate`, `inputBounds`, `maxOutputTokens`
`ERROR`	Budget exceeded, HTTP errors, network errors, parse errors, stream errors, audio errors	`{ message, code }` (code: `'BUDGET_EXCEEDED'` or `'ADAPTER_ERROR'`)
`END`	After every completion or error	`{ id, streamId, metadata: { model, usage?, finishReason?, error?, transcription? } }`

PARAM keys:

model, max_tokens, temperature, top_p, stream, system, tools, tool_choice, response_format, audio_model, budget, apiKey

Stream tool call accumulation:

Tool calls in Groq streaming responses arrive across multiple deltas. The adapter accumulates partial tool call data in duplex._streamTC keyed by delta index. When finish_reason is present, accumulated tool calls are flushed as TOOL_CALL frames and duplex._streamTC is cleared.

Error conditions:

Condition	Error code	Frame type	Description
Budget exceeded (pre-check)	`'BUDGET_EXCEEDED'`	ERROR + END	TEXT/AUDIO dropped when `totalSpent >= budgetTotal`
Estimated cost exceeds budget	`'BUDGET_EXCEEDED'`	ERROR + END	COST_CHECK `maxBudget` exceeded, message includes actual vs budget
HTTP error from Groq API	`'ADAPTER_ERROR'`	ERROR + END	Status >= 400, message from `error.message` in response body
Request timeout	`'ADAPTER_ERROR'`	ERROR + END	`req.on('timeout')` → `req.destroy()` + error emit
Connection/network error	`'ADAPTER_ERROR'`	ERROR + END	`req.on('error')`, `err.message` propagated
Audio frame missing data	`'ADAPTER_ERROR'`	ERROR + END	`'Audio frame requires data'`
Audio API HTTP error	`'ADAPTER_ERROR'`	ERROR + END	Status >= 400 from transcription endpoint
JSON parse error	`'ADAPTER_ERROR'`	ERROR + END	Message: `'Parse: ...'`
Stream error event	`'ADAPTER_ERROR'`	ERROR + END	`e.error.message` from SSE stream
General catch in sink.write	`'ADAPTER_ERROR'`	ERROR + END	`err.message`

On any error, if duplex._outputTokenCount > 0, a final progressive RECEIPT is emitted for accumulated tokens, totalSpent is updated, and duplex._outputTokenCount is reset to 0 before ERROR + END.

`getModelCost(model)`

Returns the cost config { mtoksInput, mtoksOutput } or { perMinute } for a Groq model, or null if unknown. Does case-insensitive key match and bidirectional substring match.

`calculateTokenCost(model, inputTokens, outputTokens)`

Returns { inputCost, outputCost, totalCost } in dollars. Uses per-million-token rates from GROQ_COSTS. Returns zeros for unknown models.

`GROQ_COSTS`

Static cost table. Chat models: llama-3.1-8b-instant (0.05/0.08), llama-3.3-70b-versatile (0.59/0.79), openai/gpt-oss-120b (0.15/0.60), openai/gpt-oss-20b (0.075/0.30), qwen/qwen3-32b (0.29/0.59), meta-llama/llama-4-scout-17b-16e-instruct (0.10/0.40). Audio: whisper-large-v3 ($0.000111/min), whisper-large-v3-turbo ($0.00004/min).

Tests

npm test

Runs node --test test/rigor.test.js test/smoke.test.js test/streaming.test.js. Covers: adapter construction, PARAM accumulation, frame buffering before API key, budget enforcement, COST_CHECK flow, streaming SSE parsing, tool call accumulation across deltas, calculateTokenCost monotonicity, getModelCost lookup, and audio transcription multipart form-data construction.

Requirements

Node.js 22 or later. ESM only.

Peer dependencies: @polyweave/core, @polyweave/costs.

Development Dependencies

ID	Version
@rigor/core	*

Peer Dependencies

ID	Version
@polyweave/core	*
@polyweave/costs	*
@polyweave/errors	*

polyweave groq llm lpu fast-inference adapter

Details

npm

2026-07-17 15:32:56 +00:00

John Dvorak

SEE LICENSE IN LICENSE

latest

8.4 KiB

Assets (1)

groq-1.0.7.tgz 8.4 KiB

Versions (6) View all

1.0.7

2026-07-17

1.0.8

2026-07-11

1.0.5

2026-07-10

1.0.4

2026-07-10

1.0.3

2026-07-10

@polyweave/groq (1.0.7)

Installation

About this package

@polyweave/groq

Synopsis

Install

Why

API

`createGroqAdapter(config)`

Frame Protocol

Sink (input frames):

Source (output frames):

PARAM keys:

Stream tool call accumulation:

Error conditions:

`getModelCost(model)`

`calculateTokenCost(model, inputTokens, outputTokens)`

`GROQ_COSTS`

Tests

Requirements

Dependencies

Keywords