ConfigurableModelProvider

class ConfigurableModelProvider constructor(llms: List<LlmService<*>>, embeddingServices: List<<Error class: unknown class>>, properties: ConfigurableModelProviderProperties, roleResolvers: List<RoleResolver> = emptyList(), credentialLlmServiceFactories: List<CredentialLlmServiceFactory> = emptyList(), embeddingRoleResolvers: List<EmbeddingRoleResolver> = emptyList(), credentialEmbeddingServiceFactories: List<CredentialEmbeddingServiceFactory> = emptyList(), localModelCatalogs: List<LocalModelCatalog> = emptyList()) : ModelProvider

Take LLM definitions from configuration

Constructors

constructor(llms: List<LlmService<*>>, embeddingServices: List<<Error class: unknown class>>, properties: ConfigurableModelProviderProperties, roleResolvers: List<RoleResolver> = emptyList(), credentialLlmServiceFactories: List<CredentialLlmServiceFactory> = emptyList(), embeddingRoleResolvers: List<EmbeddingRoleResolver> = emptyList(), credentialEmbeddingServiceFactories: List<CredentialEmbeddingServiceFactory> = emptyList(), localModelCatalogs: List<LocalModelCatalog> = emptyList())

open override fun configuredOptionsForRole(role: String): <Error class: unknown class>?

What configuration says role means for this deployment, or null if it says nothing.

open override fun getEmbeddingService(criteria: <Error class: unknown class>): <Error class: unknown class>

EXHAUSTIVE over ModelSelectionCriteria, deliberately and with no else, exactly as getLlm has always been.

open override fun getLlm(criteria: <Error class: unknown class>): LlmService<*>

open fun infoString(verbose: Boolean?, indent: Int): String

open override fun listModelNames(modelClass: Class<*>): List<String>

Every model of this class the deployment can serve - registered at startup, or being served by a local runner since.

open override fun listModels(): List<<Error class: unknown class>>

open override fun listRoles(modelClass: Class<*>): List<String>

List the roles available for this class of model

open override fun resolveLlmOptions(llmOptions: <Error class: unknown class>): <Error class: unknown class>

Resolve any role in these options to a concrete model, applying whatever hyperparameters are configured against that role.