ConfigurableModelProvider

class ConfigurableModelProvider constructor(llms: List<LlmService<*>>, embeddingServices: List<EmbeddingService>, properties: ConfigurableModelProviderProperties, roleResolvers: List<RoleResolver> = emptyList(), credentialLlmServiceFactories: List<CredentialLlmServiceFactory> = emptyList(), embeddingRoleResolvers: List<EmbeddingRoleResolver> = emptyList(), credentialEmbeddingServiceFactories: List<CredentialEmbeddingServiceFactory> = emptyList(), localModelCatalogs: List<LocalModelCatalog> = emptyList()) : ModelProvider

Take LLM definitions from configuration

Constructors

constructor(llms: List<LlmService<*>>, embeddingServices: List<EmbeddingService>, properties: ConfigurableModelProviderProperties, roleResolvers: List<RoleResolver> = emptyList(), credentialLlmServiceFactories: List<CredentialLlmServiceFactory> = emptyList(), embeddingRoleResolvers: List<EmbeddingRoleResolver> = emptyList(), credentialEmbeddingServiceFactories: List<CredentialEmbeddingServiceFactory> = emptyList(), localModelCatalogs: List<LocalModelCatalog> = emptyList())

What configuration says role means for this deployment, or null if it says nothing.

EXHAUSTIVE over ModelSelectionCriteria, deliberately and with no else, exactly as getLlm has always been.

open fun infoString(verbose: Boolean?, indent: Int): String

open override fun listModelNames(modelClass: Class<*>): List<String>

Every model of this class the deployment can serve - registered at startup, or being served by a local runner since.

open override fun listRoles(modelClass: Class<*>): List<String>

List the roles available for this class of model

Resolve any role in these options to a concrete model, applying whatever hyperparameters are configured against that role.