FeatureRecipe
semantic-caching-litellm
Realizes core/semantic-caching on the AI gateway core/gateway/litellm; a Release binds it in its Manifest. It writes the gateway's own cache setting into the configuration the gateway's Recipe renders, and emits no file. The gateway keeps its replies in the core/datastore/redis wired to its cache_store socket, reached over TLS while it publishes redis-tls, turns each prompt into a vector with embedding_model, an alias of its model map, and serves a stored reply whose prompt is at least similarity_threshold alike; the store removes a reply ttl_seconds after it was stored. Chat requests, streamed or not, and responses and messages requests are cached; completions, embeddings and every other kind of request are not. Replies are kept apart by the key each request presents, which the gateway verifies and a request cannot name for itself. Under scope end_user it also keeps apart the end users requests name, in an x-litellm-end-user-id or x-litellm-customer-id header or in the body's user, metadata.user_id, litellm_metadata.user or safety_identifier field. The gateway verifies none of them, so a caller naming another end user is served that end user's replies, and a request naming none shares one cache with every other request naming none under the same key. Under scope key the end user is left out, though the body's user field, a request parameter, still keeps apart requests naming different ones. The build halts unless the gateway's api_key slot is attached and its cache_store socket wired, on an embedding_model the map does not publish, on a second such attachment on one gateway, and when the attachment is carried by any component other than core/gateway/litellm.
출처
- 콘텐츠 다이제스트
- sha256:97de262b47f5ab2c8619e2f4a3ea9cd7fc0f6a63c34bc34ae90b82de0c7b2d85
- 게시일
- 2026-11-10T10:16:27Z
- 서명 ID
- https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
서명
- 공식application/vnd.dev.sigstore.bundle.v0.3+json
https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
sha256:415c43e0b733d9244922a09de2fc93c990f3fd833df0ce3bdf18c88be4f6b2fb
이 버전의 참조
oci://hub.aggrega.ai/featurerecipe/helm/semantic-caching-litellm:1.0.0- 대상
- —
- 구현 대상
- core/semantic-caching:1.0.0
제공
이 레시피가 소비자를 위해 배선하는 결정적 포인터 템플릿입니다.
선언된 provides 템플릿이 없습니다.
Workload
선언된 워크로드 템플릿이 없습니다.
로컬 변수
선언된 로컬 변수가 없습니다.
값 매핑
선언된 값 매핑이 없습니다.
관계
게시된 다른 엔터티와 주고받는 타입 지정 에지입니다. 게시된 대상이 없는 참조는 링크가 아닌 끊어진 에지로 표시됩니다.
사용함
- 구현구현
core/semantic-caching:1.0.0
사용됨
카탈로그에서 이 엔터티에 의존하는 것이 없습니다.