Recipe
tgi
Text Generation Inference as a self-hosted inference server for local development via Docker Compose, synthesized from the Workload kind by the compose provider. A weights-pull step, running the runtime's own downloader from the serving image at the same tag, resolves the model pointer into a named model-cache volume before the server starts; the compiler never fetches weights. The server is started under the weights identifier, which is the model name published to consumers. An attached model_pull_token is rendered as the HF_TOKEN environment variable on the pull step, which is also the key the credential must publish under; unattached, weights are pulled anonymously. An attached api_key is rendered the same way as API_KEY on the serving service, and the server then requires it of every caller on its generation routes. A GPU count is honoured: the compose provider reserves that many NVIDIA GPUs for the serving service, which the Docker host must be able to hand out. Compose is single-host: the replica count and the GPU runtime class are authored all the same, and the build reports each as an unsupported-field warning naming the setting. provides.host is the compose service name, which a consumer on the same compose network resolves directly.
출처
- 콘텐츠 다이제스트
- sha256:957c5f867ba3afbaa0e06bd205048758f2a232c688cfeaf46213e2eba0daf764
- 게시일
- 2026-11-10T10:21:06Z
- 서명 ID
- https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
서명
- 공식application/vnd.dev.sigstore.bundle.v0.3+json
https://token.actions.githubusercontent.com/https://github.com/aggrega-ai/aggrega-hub/.github/workflows/catalog-publish-sign.yml@refs/heads/main
sha256:cc2dfa9a8128165b647879bb71193d014bd641b07e790e5766d18527afd8420d
이 버전의 참조
oci://hub.aggrega.ai/recipe/compose/tgi:1.0.0- 대상
- —
- 구현 대상
- core/ai-model/tgi:1.0.0
제공
이 레시피가 소비자를 위해 배선하는 결정적 포인터 템플릿입니다.
| 이름 | 템플릿 |
|---|---|
| host | {{ .context.instance_id }} |
| port | {{ .infra.port }} |
| endpoint | http://{{ .context.instance_id }}:{{ .infra.port }}/v1 |
| model | {{ .values.model }} |
| protocol | http |
| metrics_endpoint | http://{{ .context.instance_id }}:{{ .infra.port }}/metrics |
| metrics_service | |
| metrics_service_port | |
| metrics_path |
Workload
Workload · form: stateless · replicas: {{ · containers: main
로컬 변수
선언된 로컬 변수가 없습니다.
값 매핑
선언된 값 매핑이 없습니다.
관계
게시된 다른 엔터티와 주고받는 타입 지정 에지입니다. 게시된 대상이 없는 참조는 링크가 아닌 끊어진 에지로 표시됩니다.
사용함
- 구현구현계약 호환
core/ai-model/tgi:1.0.0
모든 필수 출력이 충족되며, 제공되는 모든 출력이 계약에 명시되어 있습니다.
사용됨
카탈로그에서 이 엔터티에 의존하는 것이 없습니다.