nonImageBinaryFateFor

nonImageBinaryFateFor(provider): NonImageBinaryFate

Defined in: _worktrees/docs-release/gaunt-sloth/packages/agent/src/middleware/binaryContentInjectionMiddleware.ts:122

The measured fate of a non-image binary block, per provider label.

Measured CFG-63, not inferred. Every arm below comes from handing the installed client the exact block createContentBlock emits, with globalThis.fetch replaced by a capture stub (no network, no vendor calls), and reading either the thrown error or the request body that was built. The value is the same for file, audio, video and binary, because what separates the arms is whether the converter has a branch for the block at all — not which format it carries.

xai-responses → silently-discarded. @langchain/xai@1.4.10 dist/converters/responses.js:30-33 — the human-message branch recognises text and image_url and returns { type:'input_text', text:'' } for everything else. The probe against https://api.x.ai/v1/responses built a body carrying no trace of the base64 payload for all four format types. There is no shape that would work instead: xAI’s Responses union offers only input_file carrying a Files-API file_id, and gth has no upload path to produce one.
anthropic → delivered-or-loud. @langchain/anthropic@1.5.8 dist/utils/content.js:78 (fromStandardFileBlock) converts a PDF to a document block; a non-PDF non-image mime throws Unsupported file mime type for file base64 source, audio throws Converter for anthropic does not implement fromStandardAudioBlock method, and video/binary throw from @langchain/core’s dispatcher.
openai, openrouter, deepseek, xai, huggingface → delivered-or-loud. All reach @langchain/openai@1.5.10 dist/converters/completions.js:72 (fromStandardFileBlock → a file part carrying file_data) and :34 (fromStandardAudioBlock → input_audio for wav/mp3, a throw otherwise). openai on the Responses path (GS2-74) converts a file at dist/converters/responses.js:1081-1097 into input_file instead; both wire paths deliver it.
google-genai, vertexai, google → delivered-or-loud. @langchain/google@0.2.3 dist/converters/messages.js:80 and :49 turn file and audio into inlineData; video and binary throw from the dispatcher.
ollama → delivered-or-loud. @langchain/ollama@1.3.0 dist/utils.js:92 throws Unsupported content type: <type> for every non-text/image_url part.
groq → delivered-or-loud, and this is the one arm measured on the SERVER rather than the client. @langchain/groq@1.3.1 dist/chat_models.js:82 assigns content: message.content verbatim, so the block reaches the wire unchanged and only Groq’s API can decide. Measured live 2026-09-08 against qwen/qwen3.8-27b with a PDF: the API rejects it — HTTP 400 invalid_request_error, naming the content-part types it accepts, text / image_url / document. The user reads that error, so nothing is discarded and nothing is refused here. Note what the rejection also reveals: Groq has a document part type, and gth emits the standard block’s file, which is not one of the three — so a shape Groq would accept plausibly exists and gth does not build it. Whether any Groq model then reads a document is a separate question and is NOT established; do not treat the accepted type as a working feature.

Everything else is unmeasured, and like imageBlockFor’s fallback arm it must stay permissive. Who lands there, precisely: any label this switch does not enumerate — a custom or fake provider, a future vendor package, a LangChain class whose _llmType() nobody has measured — plus the empty string, which resolveVisionProvider yields only when there is no llm or its _llmType() is missing or throws. (A module config supplying an already-built LLM does NOT generally give '': modelProviderType is unset there, so the _llmType() fallback runs and returns that class’s own label — which is exactly how xai-responses arrives above.) Refusing on a label nobody has measured would break configurations that work today, so this leaves a debugLog trace instead and an unenumerated label is discoverable in a /debug-dump.

Exported so each arm can be unit-tested directly.

Parameters

provider

string

Returns

NonImageBinaryFate