Embabel Agent Documentation/com.embabel.agent.rag.ingestion/ContentChunker

ContentChunker

class ContentChunker(maxChunkSize: Int = 1500, overlapSize: Int = 200, minChunkSize: Int = 2000)

Converts MaterializedContainerSection objects into Chunk objects with intelligent text splitting.

For container sections with small total content (aggregated from leaves), creates a single chunk containing all leaf content. For large leaf sections within containers, splits them individually into multiple chunks.

Parameters

maxChunkSize

Maximum characters per chunk (default: 1500)

overlapSize

Characters of overlap between chunks (default: 200)

minChunkSize

Minimum characters to warrant splitting (default: 2000)

Constructors

ContentChunker

constructor(maxChunkSize: Int = 1500, overlapSize: Int = 200, minChunkSize: Int = 2000)

Types

Companion

object Companion

SplitterConfig

data class SplitterConfig(val maxChunkSize: Int = 1500, val overlapSize: Int = 200, val minChunkSize: Int = 2000)

Configuration for the splitter

Functions

chunk

fun chunk(section: MaterializedContainerSection): List<Chunk>

Split a MaterializedContainerSection into one or more Chunks

splitSections

fun splitSections(sections: List<MaterializedContainerSection>): List<Chunk>

Split multiple MaterializedContainerSections into Chunks