Feature
core/semantic-caching
Lets an AI gateway answer a prompt from the replies it stored earlier when the prompt is close enough in meaning to one it answered before, rather than calling the model again. The gateway turns each prompt into a vector with embedding_model, serves a stored reply whose prompt is at least similarity_threshold alike, and keeps a reply for ttl_seconds. scope decides who shares a cache. With key, every caller presenting the same key to the gateway is served from one cache, so an end user behind that key can be served a reply generated for another, tool calls included. The default, end_user, keeps each end user the gateway identifies apart. A match on meaning suits single prompts: on multi-turn or agentic traffic consecutive turns read alike, so a reply to an earlier turn can be served for a later one. It does not meet core/data-masking on one component. The developer attaches it in the Aggregate, with its inputs in values; it takes no infra. core/gateway/litellm, azure-api-management, apisix and traefik-hub realize it, the last three keeping scope key alone, apisix and traefik-hub for the callers each Release declares; litellm, apisix and traefik-hub keep the replies in a core/datastore/redis wired to the gateway, litellm requiring its administrative key. On core/gateway/kong-enterprise and higress every attachment halts, naming the caller separation each cache lacks, and on agentgateway, naming the reply cache it lacks; on any component but these seven an attachment halts naming the type, rendering nothing.
Köken
- İçerik özeti
- sha256:bac81b4175f269b56678e2188d363229d5d8028c18c7928702a321bc90c751c1
- Yayımlanma
- 2026-11-10T10:17:27Z
- İmzalama kimliği
- https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
İmza
- Resmiapplication/vnd.dev.sigstore.bundle.v0.3+json
https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
sha256:f7c90cc58a5dd4682daeea38a0b260125ad4758a8f36a5b1811b67ff7cacdfb6
Bu sürümdeki referans
oci://hub.aggrega.ai/feature/core/semantic-caching:1.0.0- Şunlara uygulanır
- gateway.aggrega.ai/intelligence=semantic
- Şunlarla çakışır
- —
Geliştirici parametreleri
Geliştiricinin bu özelliği eklerken ayarladığı değerler.
BaseComponents#/schemas/BaseFeatureParameter öğesini genişletir
| Ad | Tip | Varsayılan | Zorunlu | Açıklama |
|---|---|---|---|---|
| similarity_threshold | number (min 0, max 1) | — | evet | How alike a prompt must be to one answered before for the stored reply to be served: the cosine similarity of their two vectors, from 0 to 1, where 1 is identical. A higher value serves fewer and closer matches; which value suits depends on embedding_model. |
| embedding_model | string (min 1) | — | evet | The model that turns a prompt into the vector compared, named as the gateway knows it: an alias of the model map on an AI gateway, or on Azure API Management the id of a backend the instance holds for an embeddings model. Each realization refuses a name its gateway does not publish wherever the build can see what the gateway publishes; an instance's backends are not in the build, so the Azure API Management realization takes the id as written. |
| ttl_seconds | integer (min 1) | — | evet | How long a stored reply stays usable, in seconds. |
| scope | string (key | end_user) | end_user | hayır | Who shares one cache. end_user, the default: each end user the gateway identifies has a cache of its own, so none is served a reply generated for another. key: every caller presenting the same key to the gateway shares one cache, so a reply generated for one end user behind that key can be served to another sending a similar prompt. Each realization says how its gateway identifies an end user and what it does with a request naming none, and refuses end_user where its gateway cannot keep end users apart. |
Çıktı sözleşmesi
Bu özelliğin tüketicilere yayımladığı değerler.
Çıktı değeri bildirilmemiş.
Ek açıklamalar
Ek açıklama bildirilmemiş.
İlişkiler
Diğer yayımlanmış varlıklara giden ve onlardan gelen tipli kenarlar. Yayımlanmış bir hedefi olmayan bir başvuru, bağlantı yerine kopuk bir kenar olarak gösterilir.
Kullanır
- uygulanıruygulanır
gateway.aggrega.ai/intelligence=semantic
core/gateway/agentgateway v1.0.0 →core/gateway/apisix v1.0.0 →core/gateway/azure-api-management v1.0.0 →core/gateway/cloudflare-ai-gateway v1.0.0 →core/gateway/envoy-gateway v1.0.0 →core/gateway/higress v1.0.0 →core/gateway/kong-enterprise v1.0.0 →core/gateway/litellm v1.0.0 →core/gateway/traefik-hub v1.0.0 → - $refşemayı genişletir (values)
BaseComponents#/schemas/BaseFeatureParameter
Kullananlar
çakışır
uygular
uygular
uygular
uygular
uygular
uygular
uygular