Prometheus — one server, two scrape targets
This example compiles a small, real stack — a Prometheus metrics server
scraping two Grafana dashboards servers — from one Developer-authored
Aggregate to two targets:
- Docker Compose for local development, and
- Kubernetes via Workload synthesis on the official
prom/prometheusimage.
The same aggregate.yaml drives both. Only the Recipe layer selected by the
Manifest changes per target.
Grafana serves its own metrics in the Prometheus text format at /metrics on
its web port, and it does so by default, so each Grafana is a real scrape
target. Both are the same Component type wired twice, so everything that
differs between the two scrape jobs comes from what is authored on each
connection and from where each Grafana says it serves its metrics.
Files
| File | Role |
|---|---|
aggregate.yaml | Developer intent: two Grafanas, team-dashboards and ops-dashboards (both core/observability/grafana), and a metrics server (core/observability/prometheus) that declares a scrape_target use of each. |
manifest-compose.yaml | Maps the Prometheus type to the Compose Recipe (compose/prometheus) and the Grafana type to the Grafana Compose Recipe (compose/grafana). |
manifest-helm-synth.yaml | Maps the Prometheus type to the synthesized helm Recipe (helm/prometheus) and the Grafana type to the synthesized Grafana helm Recipe (helm/grafana). Neither fetches a chart. |
release-compose.yaml / release-helm-synth.yaml | Operator records for the two targets. |
Wiring two targets on one socket
scrape_target is one socket carrying two connections, and each connection
becomes one scrape job:
metrics: type: core/observability/prometheus uses: scrape_target: team-dashboards: target: team-dashboards ops-dashboards: target: ops-dashboards values: job_name: ops-grafana scrape_interval: 30sThe team-dashboards connection authors no values, so its job takes what the
socket declares. The ops-dashboards connection sets two keys, and each key it
sets shapes that job alone — the team-dashboards job is untouched by them.
The target says where it serves metrics
A connection says how often Prometheus scrapes and under which job name; it
never says where. That is the target’s own fact, and the target publishes it:
each Grafana Recipe publishes metrics_endpoint, the full address of its
metrics — scheme, host, port and path, such as
http://team-dashboards:3000/metrics on Compose. Prometheus builds the scrape
job from that address: its scheme, its path, and its host and port as the
job’s target.
So a target that serves no metrics publishes no address, and wiring one to
scrape_target halts the build, naming it, rather than compiling a job that
would scrape nothing. A product whose metrics are off until a value turns them
on — many in the catalog take metrics_enabled — publishes its address while
they are on. On Kubernetes a target may instead publish what Prometheus
discovers its metrics by, a Service or a pod selector, when no single address
reaches all of them; the Compose realization has no cluster to discover from
and halts on such a target.
The socket admits servers, stateful servers and workers alike, so a database, a log store or an operator is wired the same way; whether it can be scraped is decided by what it publishes.
What it demonstrates
- One authoring surface, many targets. A single Aggregate is the source of truth for what the stack is. Each Manifest says how to realize it.
- One socket, many connections. Prometheus scrapes as many targets as the
Aggregate wires into
scrape_target. The Recipe emits one scrape job per connection, so the number of jobs follows the topology rather than the Recipe. - Per-connection configuration.
job_nameandscrape_intervalare declared on the socket’svalueSchema, which makes them properties of a connection rather than of the server. One Prometheus scraping several targets cannot express several cadences in a single Component-wide value. - Declared defaults fill the keys a connection omits.
scrape_intervalcarries a default on the socket’svalueSchema. A connection that omits it —team-dashboardshere — receives it, and a connection that sets it —ops-dashboards— overrides only what it set. - Scrape wiring through the String Pipe. Each Grafana Recipe’s
metrics_endpointis piped into Prometheus and rendered into that Grafana’s job: its scheme, its path and itstargetsentry. The compiler routes the string without interpreting it: each Recipe decides what address its own target publishes, and the compiler pipes it through unchanged. - Provider-idiomatic configuration. On both targets the server writes a
minimal
prometheus.ymlat start-up whose scrape jobs address the wired Grafanas, then execs Prometheus against it. On Compose this runs as a service; on Kubernetes the same container renders into a StatefulSet. - Persistent state. Prometheus stores its samples on a data volume so they
survive restarts — a named Compose volume locally, and a per-replica
PersistentVolumeClaim sized from
infra.storageon Kubernetes. Each Grafana keeps its users and dashboards on a volume of its own. - Reproducible images. Every Recipe here pins its image tag from
infra.engine_version, so the rendered workloads reference fixed images. - Lean single-purpose shape. On Kubernetes the Prometheus component renders as its StatefulSet, its ClusterIP and Headless Services, and its data volume claim — nothing else. Each Grafana renders as its own Deployment, Service and data volume claim.
Scaffold the example
You can build this example with the commands below, or have your AI assistant build it for you.
Write the example into a new folder named after it:
aggrega scaffold --example prometheus-scrapeIt then prints the commands to run next. The first changes into the new folder:
cd prometheus-scrapeBuild
This workspace carries no .aggrega-config.yaml, so the standard-library
entities resolve from the hub registry, hub.aggrega.ai, and no catalog flag is
needed:
# Local development (Docker Compose)aggrega build --release release-compose.yaml --auto-pull
# Kubernetes (Workload synthesis on prom/prometheus)aggrega build --release release-helm-synth.yaml --auto-pullRendered artifacts land under dist/. --auto-pull resolves each Release’s
dependencies into its lockfile during the build; aggrega pull --release release-compose.yaml once up front does the same and lets later builds run
without the flag. The synthesized target needs no external chart fetch — both
Recipes render the official images directly into Kubernetes resources.
With an AI assistant
An AI assistant set up with Aggrega’s tools can do the same: paste this prompt into it.
Use the Aggrega tools only, and create no folder yourself. If the Workspace folder holds no `prometheus-scrape` folder yet, call the `scaffold` tool once with `example` set to `prometheus-scrape` and no `dir`: it writes the example into a new `prometheus-scrape` folder. Then call the `build` tool once for each Release, in this order — `release-compose.yaml` and `release-helm-synth.yaml` — each time with `workspace_root` set to `prometheus-scrape`, `release` set to that file, `auto_pull` set to true and `write` set to true. Tell me which files each build wrote; they are under `prometheus-scrape/dist/`.What the compiled scrape configuration looks like
On Compose, the server writes a prometheus.yml holding one job per wired
Grafana, each addressing the metrics_endpoint that Grafana publishes — its
compose service name and port:
scrape_configs: - job_name: ops-grafana scrape_interval: 30s scheme: http metrics_path: "/metrics" static_configs: - targets: - "ops-dashboards:3001" - job_name: team-dashboards scrape_interval: 15s scheme: http metrics_path: "/metrics" static_configs: - targets: - "team-dashboards:3000"On Kubernetes, the same two jobs are written by the synthesized StatefulSet’s
start-up command, each addressing that Grafana’s Service, which the helm
provider names after the component and its default service key:
scrape_configs: - job_name: ops-grafana scrape_interval: 30s scheme: http metrics_path: "/metrics" static_configs: - targets: - "ops-dashboards-default:3001" - job_name: team-dashboards scrape_interval: 15s scheme: http metrics_path: "/metrics" static_configs: - targets: - "team-dashboards-default:3000"Both targets compile the same two jobs from the same two Aggregate edges: each
Grafana’s metrics_endpoint travels through its own scrape_target
connection, and the compiler pipes that string into the job without
interpreting it. What differs between the targets is the address each Grafana
publishes — a compose service name on one, a Service name on the other — and
the workload the configuration is written into — a Compose service on one, a
StatefulSet on the other. Here both Grafanas sit in the same scope as
Prometheus; on Kubernetes, a Grafana the Release places in another namespace
is addressed by its Service name qualified with that namespace.
Read the team-dashboards job against the ops-grafana job to see where every
value comes from. The ops-grafana job carries ops-grafana and 30s because
its connection authors both. The team-dashboards job carries
team-dashboards as its job_name because the Aggregate declares none and the
Recipe falls back to the connection alias — the name this Aggregate gave the
connection — and it carries 15s because that is the default the Component’s
scrape_target socket declares for the key that connection omitted. The
scheme, the metrics_path and the target of both jobs come from the address
each Grafana publishes, which is where Grafana serves its metrics, so neither
connection says anything about them. Setting a key on a connection overrides it
for that job and leaves the other job alone.
All of this happens on the consuming side. Per-connection configuration surfaces only in the render context of the node that authored the connection, never in the target’s, so wiring or tuning a connection into a Grafana never perturbs what that Grafana itself renders — its Deployment and its Service are the same bytes either way.
Grafana answers its metrics to any caller, so neither connection carries a
credential. For a target whose metrics take one, the connection carries it in
its credential slot, and Prometheus presents it as a bearer token read from a
file; a target that publishes metrics_credential_required true halts the
build while its connection carries none.
Run it (Compose)
aggrega build --release release-compose.yaml --auto-pulldocker compose -f dist/app/compose/docker-compose.yaml up -dThe Release gives each Grafana its own port — team-dashboards on 3000 and
ops-dashboards on 3001 — so both run side by side locally. Each Grafana
listens on the port its Release gives it, and each port travels in the address
that Grafana publishes into its own scrape job’s target.
Open http://127.0.0.1:9090 — the Prometheus expression browser, which the
Compose Recipe publishes on the host’s loopback address only. Under
Status → Targets the two jobs are listed under the names the compiled
configuration gives them, ops-grafana and team-dashboards, each addressing
its Grafana’s /metrics. A query for grafana_build_info, the gauge every
Grafana sets to 1 and labels with its version, returns one series per job. Tear
down with docker compose -f dist/app/compose/docker-compose.yaml down.