Embabel Agent Documentation/com.embabel.agent.spi.loop.streaming/LlmMessageStreamer

LlmMessageStreamer

fun interface LlmMessageStreamer

Framework-agnostic interface for streaming LLM inference.

Streaming counterpart of LlmMessageSender. Implementations handle the actual LLM communication (Spring AI, LangChain4j, etc.). The original stream contract remains the single abstract method for source and binary compatibility. The new streamInference method exposes the assembled terminal message required by an Embabel-owned streaming tool loop.

Key Differences from Non-Streaming:

  • Emits content incrementally, followed by the assembled assistant message

  • Does not execute tools; ToolLoop remains under Embabel's control

  • Provider adapters are responsible for assembling partial tool-call fragments

See also

for non-streaming equivalent

Functions

Link copied to clipboard
abstract fun stream(messages: List<Message>, tools: List<Tool>, toolCallInspectors: List<ToolCallInspector>): <Error class: unknown class><String>

Stream raw content chunks using the original provider-managed tool contract.

Link copied to clipboard
open fun streamInference(messages: List<Message>, tools: List<Tool>): <Error class: unknown class><LlmInferenceStreamEvent>

Stream one inference without delegating tool execution to the provider.