FeatureRecipe
concurrency-limit-litellm
Realizes core/concurrency-limit on the AI gateway core/gateway/litellm; a Release binds it in its Manifest. Each Recipe of the gateway reads the attachment and writes max_in_flight as the max_parallel_requests of every entry of the model map, typed by hand and wired alike. The proxy keeps each entry's count in memory, where a record lapses ten minutes after it is made and a new count starts at zero while requests counted by the old one are in flight, so the Recipe also sets that lifetime to a century, keeping each count as long as the proxy runs. The proxy takes one of an entry's slots for each request it sends there, so at most max_in_flight requests are in flight to each entry at once. A request finding its entry full is answered 429: after the gateway's retries where its alias has one entry, and at once where it has several, even while another of them has room, since the proxy picks the entry before it counts; a fallback chain attached to the gateway sends it on to its alias's fallbacks instead. The count is each running proxy's own. The proxy keeps no total that holds — its one total is read only by a request limiter an environment variable switches on, which counts a request for its first second — so this recipe halts on scope gateway, the Feature's default. It halts on a model map of more than 100 entries, past which the store the counts are kept in, 200 records, can drop one, on a second such attachment, and on any component other than core/gateway/litellm; and the gateway refuses a request carrying priority, whose queue the century lifetime would keep.
출처
- 콘텐츠 다이제스트
- sha256:4bc0edd05182b96fdcbfdfec82f8337d4ba62bf3ded7553b7aef287c926b07d3
- 게시일
- 2026-11-10T10:15:18Z
- 서명 ID
- https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
서명
- 공식application/vnd.dev.sigstore.bundle.v0.3+json
https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
sha256:ff15716a5d82add0372a66d1e2cd887af5de7d3a9c6c19358c3a767e63e6fe9c
이 버전의 참조
oci://hub.aggrega.ai/featurerecipe/helm/concurrency-limit-litellm:1.0.0- 대상
- —
- 구현 대상
- core/concurrency-limit:1.0.0
제공
이 레시피가 소비자를 위해 배선하는 결정적 포인터 템플릿입니다.
선언된 provides 템플릿이 없습니다.
Workload
선언된 워크로드 템플릿이 없습니다.
로컬 변수
선언된 로컬 변수가 없습니다.
값 매핑
선언된 값 매핑이 없습니다.
관계
게시된 다른 엔터티와 주고받는 타입 지정 에지입니다. 게시된 대상이 없는 참조는 링크가 아닌 끊어진 에지로 표시됩니다.
사용함
- 구현구현
core/concurrency-limit:1.0.0
사용됨
카탈로그에서 이 엔터티에 의존하는 것이 없습니다.