credential
Upper bound on LLM services built from user-supplied keys and held for reuse.
A cache bound is an operational concern: it trades memory against how often a deployment rebuilds a service for a key it has seen before, and the right number depends on how many distinct keys are concurrently active — which only the deployment knows. The default suits a deployment with tens to low hundreds of concurrent users; raise it if yours has more, and expect roughly one thin chat-client wrapper per entry.
Exceeding it is not an error. Least-recently-used entries are dropped and rebuilt on next use, so the only cost of setting it too low is repeated construction.
Must be at least 1, checked at startup. Zero or negative would evict every entry as soon as it was inserted, so the cache would silently never hold anything and every call would rebuild a service over the network - indistinguishable from a caching bug, and only visible as latency. Rejected rather than tolerated for that reason.