DICE API Documentation/com.embabel.dice.pipeline/BatchedExtractionStrategy

BatchedExtractionStrategy

class BatchedExtractionStrategy : ExtractionExecutionStrategy, AutoCloseable

Processes chunks in fixed-size windows: each window fans out concurrently on the injected executor and is fully joined before the next window starts. This is the rate-limit lever — it caps in-flight extract() calls to batchSize at a time. Results are returned in input order.

BatchedExtractionStrategy(batchSize = 1) is the safe degrade-to-serial path until extractor thread-safety is verified (see ExtractionExecutionStrategy). A caller-supplied executor is never shut down here; a pool created by the no-executor constructor is shut down by close.

Parameters

batchSize

maximum concurrent extractions per window; must be positive

Constructors

BatchedExtractionStrategy

constructor(batchSize: Int = 5, executor: ExecutorService)

Use a caller-supplied executor; close will not shut it down.

constructor(batchSize: Int = 5)

Create an internal fixed pool sized to batchSize. The pool is owned by this strategy, so call close (or use it as a resource) when you are done with it.

Functions

open override fun close()

Shuts down an internally-created pool; a no-op when the executor was caller-supplied.

execute

open override fun <T> execute(chunks: List<<Error class: unknown class>>, extract: (<Error class: unknown class>) -> T): List<T?>

Apply extract to each chunk, returning results in INPUT ORDER. A null slot at index i means extract(chunks[i]) threw and was caught (see contract above).