AI Features
The Aggrega AI Feature Set
To support the unique demands, volatility, and security threats of modern AI systems, Aggrega embeds specialized intelligence directly into its open-source AI Standard Library (The Catalog). It ensures that both generic cloud-native scaffolding and highly specialized AI-native reasoning engines coexist securely.
The core Go compiler is a content-neutral universal router; all AI-specific intelligence lives in pluggable WebAssembly (Wasm) policies and the JSON Schema 2020-12 declarations Platform Architects author on Catalog entities.
1. The Operational Cloud Ontology & AI Duck-Typing
To bridge the generic Open Application Model (OAM) with the strict realities of modern infrastructure, the component.aggrega.ai/type label maps exclusively to a closed set of Universal Cloud-Native Primitives. This semantic identity is what a socket selector, a Feature’s appliesTo and a governance policy read to know how a component treats state, network ingress, and lifecycles; the compiler matches it as an opaque string and does not branch on it, and how a component renders is decided by the Recipe the Manifest maps for it.
The Cloud-Native Types (The Infrastructure Base):
server: Long-running, stateless compute that expects inbound application traffic (e.g., an AI Gateway or a Chat UI).worker: Long-running, stateless compute that pulls from queues or runs background loops with zero inbound application ports (e.g., a LangGraph agent or Celery worker).task: Ephemeral compute that runs to completion and dies (e.g., DB migration scripts).function: Event-driven compute that scales to zero (e.g., AWS Lambda).stateful-server: In-cluster compute requiring stable network identities and permanent disks (e.g., a Qdrant vector database, a Kafka broker).resource: External, platform-managed infrastructure (e.g., AWS RDS PostgreSQL, S3 buckets, a hosted event bus).
The set is Aggrega’s own and it is closed. The published Component schema enumerates these values, and a document carrying anything else is refused when it loads — E0215 names the label and lists what it takes. That is what the axis buys: a Feature or a socket selector written against stateful-server means the same thing across every catalog it meets.
What a component is in your own vocabulary goes on your own key. A tier, a data zone, a service class, a finer split of the primitives — each is a label key you author beside the reserved one, since the Component schema accepts any further label key with a string value and every selector surface reads it identically. That is the same mechanism the AI duck-typing labels below already use; the reserved key is the shared floor, and everything above it is yours.
The operational type is not the browse category, and they are separate axes on purpose. A Kafka broker sits in the messaging category — where a user looks for it — and carries stateful-server as its type, because that is what infrastructure has to do for it. A selector written against one axis never matches on the other.
See entities/component for the Semantic Label Taxonomy tables — the Generic Cloud Layer, Specialized AI Layer, Cross-Cutters, and Dynamic Business Context labels Architects use to layer cognitive identity on top of the base types.
1.1 The Open-Source Catalog Standard: “Naked Base, Flavored Edges”
When applying these labels, there is a critical distinction between the Official Aggrega Open-Source Catalog (core/*) and a Private Enterprise Catalog (acme/*).
For the official open-source catalog to remain universally usable, it must strictly distinguish between Immutable Technical Physics and Subjective Cognitive Intent. If the general-purpose core/datastore/postgres is hardcoded with engine: relational, a governance verdict plugin configured to enforce datastore-type matching will halt the build when a developer enables the pgvector extension in their values and attempts to connect an AI Agent that requires a vector datastore.
To maintain an unopinionated core catalog, official components follow this standard: Keep the Base Naked, and Flavor the Edges.
- The Naked Base: The root-level open-source
Componenthardcodes only what is objectively true of the product and cannot be changed by anything an author configures — first and always the operational primitive (component.aggrega.ai/type). A Postgres database is objectively astateful-server; that fact cannot change. The same test admits a capability label where the product’s identity is that capability:core/datastore/qdrantandcore/datastore/weaviateare vector search engines and nothing an author writes makes them anything else, so each carriesdatastore.aggrega.ai/engine: vectoron the base, exactly ascore/datastore/rediscarrieskv. What the naked base refuses is the opposite case — a trait that depends on how a component is configured or used, which the third mechanism below exists for. - The Flavored Interfaces (Schema Inheritance): A Component may declare
extends, naming another Component as its base, and what that composes is structure: labels, annotations, attachment slots, the self-inject allowlist, topology sockets and namedassertions. The three data pillars do not compose — an extending Component writes the literal markerschema: inheritedand authors novalues,infraorprovidescontent of its own, so its contract is its base’s identically. That is why the mechanism narrows rather than varies: because the contract is unchanged, every Recipe written for the base already serves the specialization, and the specialization costs one catalog entity and one Manifest mapping line. What a specialization adds is therefore identity a selector or a policy reads — an audit tier, a data zone, a governance contact, or a socket whose selector admits less than the base’s. It cannot add a different contract, because it has no surface on which to author one. One consequence is easy to miss: the mapping line is still required. Recipe selection is by exact mapping key, so a specialization needs its ownmappings.components.typeentry even though it needs no Recipe of its own. See examples/extends-wrapper for a published wrapper that adds a reviewed-tier label and narrows a socket on it, together with the build that halts when an edge wired into that socket leaves the tier. A trait that depends on how a component is configured is not this mechanism’s subject: a specialization pins its base’s artifact and inherits its contract, so a trait true of the specialization is already true of the base, where the Naked Base rule puts it. Such a trait belongs to the Dynamic Mutator below. - The Dynamic Mutator (Wasm Promotion): If a trait is dictated by the developer’s payload (e.g.,
enable_vector: true), the enterprise uses the governance socket’slabel-transformrole at topology construction to dynamically read the opaquevaluesand automatically inject thedatastore.aggrega.ai/engine: vectorlabel into the IR Graph before policies run. This is where a configuration-dependent trait becomes available to strict policy evaluation — the developer’s declaredvaluesare what promote it, and the transform runs before any policy reads a label.
The Enterprise Exception: Because the compiler is unopinionated, enterprises authoring their own internal catalogs are free to break this rule. If a corporate security policy dictates that all internal databases are highly sensitive, the Platform Architect can and should hardcode compliance.aggrega.ai/data-class: restricted (or a company dictated similar label) directly onto their acme/postgres base component, so that every instance of it carries the class and a policy the enterprise attaches reads it on every one. The label itself enforces nothing: the compiler carries it as a string, and the attached policy is what halts a build. Where only some of an estate’s uses of a product hold that data, a specialization through extends carries the class instead — see examples/policy-data-class-route, whose claims assistant is the standard-library LangGraph agent plus one class label, beside the policy that reads it.
1.2 How Semantic Labels Power the Compiler Pipeline
To keep labels in the Platform Architect’s domain, Developers cannot write labels on their Component instances inside the Aggregate: the Aggregate schema gives a component instance no labels key. A node’s labels are its catalog entity’s, together with any that a label-transform governance attachment the operator configures adds during topology construction. Here is exactly how these labels drive the compilation pipeline without requiring custom Go code:
1. component.aggrega.ai/type (The Base Primitive)
- Who writes it? The Platform Architect (Domain 2). This is mandatory for every component in the catalog.
- Who consumes it? The Static Analyzer and Wasm Rendering Plugins.
- Compiler Use Case: This is the core operational cloud ontology. The Static Analyzer evaluates the
usesSchemaselector strings natively in Go to guarantee a Developer cannot attach an Agent to a standard Database if the socket requires a specific AI capability. Furthermore, DevOps uses this label to explicitly route stateful primitives (stateful-server,resource) into physical cluster boundaries that support persistent disks, while a governance verdict role can enforce that stateful components are never placed in ephemeral-only boundaries.
2. model.aggrega.ai/capability
- Who writes it? The Platform Architect.
- Who consumes it? The Static Analyzer (via the Kubernetes-style label selectors of
usesSchemasockets). - Compiler Use Case: When an Architect authors a
usesSchemaselector naming the required capability label, the compiler’s native label-matching engine rejects wiring that does not satisfy the selector before artifacts are generated. The compiler treats these labels as opaque strings; the semantic meaning of “text-generation” vs “embedding” lives in the Architect’s selector authoring.
3. component.aggrega.ai/hosting
- Who writes it? The Platform Architect.
- Who consumes it? The Policy Engine (a governance verdict role), Wasm Rendering Plugins, and the standard library’s own callers of a model or an agent: its agents, whose assertion reads it on each
modelconnection,core/ai-tool/agent-invocation, whose assertion reads it on itstarget_agentconnection, as does its Kubernetes realization, refusing a credential to anexternalagent reached overhttp, the AI gateway, whose assertion reads it on each connection to itsmodelsandcache_storesockets, andcore/gateway/kong,core/gateway/kong-enterprise,core/gateway/apisix,core/gateway/envoy-gateway,core/gateway/traefik-hub,core/gateway/agentgatewayandcore/gateway/higress, whose assertions read it on each connection to theirmodelssockets, and to the two Kongs’backend, thecache_storesockets of APISIX and Traefik Hub and thecachesockets of Envoy Gateway, Higress and Traefik Hub — see “The key a caller presents” below. - Compiler Use Case: It records who operates the component’s underlying resource, letting a policy an operator attaches judge Data Exfiltration risk; the compiler reads the value as a string and judges nothing. When an Agent whose class label says it holds
internaldata connects to a Model labeledmanaged— a service operated outside the Release’s own boundary — such a policy can read that the data leaves that boundary, and halt the build unless the connection is routed through an AI gateway carryingcore/data-masking, which masks the chat and completion text the gateway forwards. An agent’smodelsocket takes such a gateway in place of the model, so that route is an edge of the graph the policy reads, and a model the gateway serves from itsmodelssocket is an edge of its own, from the gateway to the model component, whose labels the policy reads — so the path from an agent through the gateway to amanagedmodel is in the graph end to end. An upstream typed by hand into the gateway’s model map is not: it is a string the compiler does not resolve, so a policy reading the graph sees the gateway and nothing behind it. examples/policy-data-class-route is such a policy: its settings class amanagedmodelpublic, and it halts a build where an assistant holdingrestricteddata reaches one with no masking gateway on the route. This is a cross-cutting trait: the sameself-hosted | managed | externalvalues describe a managed database or a self-run inference server just as they describe a model, so the label lives under the sharedcomponent.aggrega.ai/namespace rather than the model-specific one.
4. agent.aggrega.ai/autonomy
- Who writes it? The Platform Architect.
- Who consumes it? The Policy Engine (a governance verdict role).
- Compiler Use Case: It identifies the execution risk of the cognitive loop. If an agent is labeled
autonomous(meaning it runs background loops without user prompts) and connects to a highly sensitive database, the policy engine can halt the build unless an author attaches whatever approval or containment Feature the enterprise’s own rules require, so an unattended loop cannot reach that data unreviewed. - What an unlabeled component means on this axis, because the absence is deliberate. The label describes the execution risk of a cognitive loop, so a component that holds no loop of its own carries no value here rather than the lower of the two. A good many components in the agent category are unlabeled for exactly that reason: the supporting pieces an agent platform ships alongside its server — a browser console, a schema-migration run, a task worker, a code sandbox, an egress proxy — are in that category because it is where you look for them, while the loop they support lives in the platform’s server, which is where the label sits. The clearest case is a single product split across both treatments: a platform’s server carries
copilot, and the code sandbox it ships beside it carries nothing. The test is where the loop lives rather than what shape the piece takes, so a runtime that hosts agent code you supply is also unlabeled — the loop there is whatever you deploy into it, not something the product asserts. A selector naming either value therefore matches the components that hold a loop and no others, and one written to match any autonomy posture skips the supporting pieces rather than treating them as the least autonomous thing in the deployment.
5. gateway.aggrega.ai/intelligence
- Who writes it? The Platform Architect.
- Who consumes it? The Static Analyzer, and enterprise-authored linter Wasm plugins.
- Compiler Use Case: It dictates what features a router can physically support. The AI
fallback-chainFeature declaresappliesTo: gateway.aggrega.ai/intelligence=semantic, and the Static Analyzer evaluates that selector against the target component’s labels before any artifact is rendered. If a Developer attaches it to a standardl4-meshgateway the build halts with a selector violation naming the Feature and the selector it failed, because the Architect authored that selector knowing a standard TCP mesh cannot parse LLM JSON payloads to read the model a request names. The refusal reads off the absence ofsemanticrather than the presence ofl4-mesh, so it holds equally on a router that carries no intelligence label at all. The compiler treats the label as an opaque string, exactly as above: whatsemanticmeans lives in the Architect’s selector authoring. On a router that does carrysemanticthe attachment passes the selector, and what happens next is its realization’s: the standard library’s realization writes the chain into the AI gateway’s routing configuration. - What the standard library carries. These components carry
semantic:core/gateway/litellm, whose realization renders the proxy’s routing table: a model map over several upstream providers;core/gateway/cloudflare-ai-gateway, Cloudflare’s hosted gateway, which reads each model request it forwards — the model it names, the tokens it is billed for and, where the gateway’s own checks are on, its content;core/gateway/azure-api-management, the AI gateway of Azure API Management, a hosted service, which reads each model request on an API serving a model — the prompt its semantic cache compares, the tokens a reply reports and the content its safety check sends for inspection;core/gateway/apisix, whose AI proxy reads the body of each request on a route serving a model and the tokens the model’s reply reports;core/gateway/envoy-gateway, whose Envoy AI Gateway extension reads the body of each request on the route serving the models wired to it, to find the model it names, and the tokens each reply reports;core/gateway/kong-enterprise, Kong Gateway Enterprise, whose AI proxy reads the body of each request on a route serving a wired model — the model it names and the messages it carries — and the tokens each reply reports;core/gateway/traefik-hub, Traefik Hub, whose Chat Completion middleware reads the body of each request on the route serving a wired model, checks it against the chat completion schema, and reads the tokens each reply reports;core/gateway/agentgateway, agentgateway, which reads the body of each request on its model-serving paths to find the model it names, the way it chooses the wired model a request is sent to, and the tokens each reply reports; andcore/gateway/higress, Higress, whose AI proxy reads the body of each request on a route serving a wired model — the model it names and whether it streams — and whose plugins read the tokens each reply reports — so the AI Features that require this value resolve against any of them, and each realization says what its gateway honours, or halts the build where its gateway honours nothing of it; a Feature with no realization for a gateway halts the build there, naming its type. No component carriesl4-mesh, and the standard library’s other gateway components carry no intelligence label at all: the ones that route traffic terminate HTTP and route at layer 7, and one of them is a control plane that routes nothing at all, so neither value describes it. A layer-4 TCP mesh is a different product class again. That leaves the refusal above fully in force, because it reads off the absence ofsemantic: an attachment requiring that value halts on any of them. An Architect whose catalog does run a layer-4 mesh carriesl4-meshon that component, which makes the distinction selectable in both directions. - This axis names a product tier, which is a different question from what a component can do to traffic, and reading it as the latter is the mistake it invites.
semanticsays the router parses the request body, which is what a control acting on the model a request names needs — so it is the right axis for that. It says nothing about whether a router can copy traffic, carry a caller’s identity, check an upstream’s certificate, refuse a prompt that matches a list of patterns, or keep a budget over a period, and a Feature needing one of those has no business asking about the tier: most of the components that offer them are ordinary layer-7 proxies carrying no intelligence value at all, and where asemanticrouter does offer one, it offers it for reasons the tier does not record. So the tier reaches the wrong set in both directions — it excludes the plain proxies entirely, and it says nothing about which capabilities the AI gateway it does reach actually has. That question belongs to the data-path axis below, and the two compose — a Feature may require asemanticrouter that also mirrors, by naming both.
6. tool.aggrega.ai/impact
- Who writes it? The Platform Architect.
- Who consumes it? The Policy Engine and Wasm Rendering Plugins.
- Compiler Use Case: By moving this out of the developer’s mutable
valuesblock and into the immutable label identity, it allows the policy engine to evaluate strict, least-privilege configurations. A rendering plugin might generate standardinvokepermissions for aread-onlyweb-search tool, but demand strict cryptographic signature configurations to invoke amutatingSQL-drop tool.
7. compliance.aggrega.ai/data-class
- Who writes it? The Platform Architect of an estate’s own catalog (or dynamically injected via the governance
label-transformrole). - Who consumes it? The Policy Engine (a governance verdict role).
- Compiler Use Case: It gives the data a component holds a governance identity that a policy can follow along the graph. The compiler carries the label to the IR graph as a string and ranks nothing: it does not know that
restrictedis more sensitive thanpublic, or that either word means anything. A policy attached at their-verdictrole supplies that meaning in its own settings — which classes exist and how they rank — and reads the label on the components of the routes the compiler enumerates over theusesedges, so it can halt a build where a component holdingrestricteddata reaches a component of another class along a route with no Feature the policy counts as declassifying attached on it, such ascore/data-maskingon the AI gateway the route passes through. The enumeration is bounded per starting component and lists nothing past the bound, so a route the policy reads is real while the absence of one is not proof that none exists. examples/policy-data-class-route is that policy, written for the Rego and CEL engines compiled into the CLI and for OPA, with the halting builds and the completing one side by side. - What the standard library carries. No
core/Component carries this label. Which class a component’s data has is a statement about how an estate uses the component, not a property of the product true in every Release, so by the Naked Base rule it stays off the base: an estate carries it on its own Component — a specialization throughextends, as the worked example’s claims assistant is, or a base of its own under the Enterprise Exception above.
8. hardware.aggrega.ai/accelerator
- Who writes it? The Platform Architect. When packaging a local model component, they know it fundamentally requires a GPU to function.
- Who consumes it? DevOps (Domain 4).
- Compiler Use Case: DevOps explicitly routes GPU-labeled components into physical hardware pool boundaries in the
Releasevia thecomponents: {}map. The Developer simply asks for the cognitive model in theirAggregate, and DevOps places it into the expensive GPU cluster boundary, entirely preserving the Separation of Concerns. A governance verdict role can additionally enforce that any component carrying this label is never placed in a boundary lacking GPU infrastructure.
9. datastore.aggrega.ai/engine
- Who writes it? The Platform Architect, on datastore-category components (statically, on a flavored specialization, or injected by the
label-transformrole when developervaluespromote the role). The values in use across the catalog are:relational,document,kv,vector,graph,semantic-memory,search. Each value names a storage engine a product is —searchnames a search and analytics engine that also indexes vectors — so a trait that lives in an application’s own schema and access pattern rather than in any deployable store is not a value on this axis. - Who consumes it? The Static Analyzer (via selector strings) and the Policy Engine.
- Compiler Use Case: Selectors target storage semantics rather than products — an agent’s knowledge-base socket selects
datastore.aggrega.ai/engine in (vector, semantic-memory, search), and a policy engine can refuse SQL payload routing to a vector engine. AI-relevance across dual-use datastores is derived from this label, never stored as a separate boolean. - What the standard library carries.
core/datastore/qdrant,core/datastore/weaviate,core/datastore/chroma,core/datastore/pgvectorandcore/datastore/milvuscarryvector,core/datastore/cogneecarriessemantic-memory, andcore/datastore/opensearchandcore/datastore/elasticsearchcarrysearch, so the knowledge-base selector above resolves against each of them.core/datastore/redis,core/datastore/valkeyandcore/datastore/memcachedcarrykv. Every one of these labels sits on the component’s own base, because the product’s identity is that engine. Forcore/datastore/pgvectorthat identity is the artifact it pins: it runs the pgvector project’s own PostgreSQL image, which carries the extension, and every Recipe enables the extension in the database the component creates — so its vector trait does not depend on a setting, which is what separates it from the configured case under “Naked Base, Flavored Edges” above, where a general-purpose PostgreSQL gains the trait only when a developer turns an extension on.relational,documentandgraphhave no standard-library component, so a selector naming only values among them matches nothing here and is satisfied by a catalog that carries one.
10. pipeline.aggrega.ai/mode
- Who writes it? The Platform Architect, on pipeline-category components. Each value names how a product runs a multi-step job, and the values follow the components that carry them rather than a list set out in advance. The standard library carries two:
durableoncore/ai-pipeline/temporal, whose engine records every step’s outcome as an event history and replays it, so a job whose worker crashes resumes where it stopped; andreconciledoncore/ai-pipeline/argo-workflows, whose controller runs each step as a pod from a workflow resource held in the cluster’s own API. - Who consumes it? The Policy Engine and Wasm Rendering Plugins.
- Compiler Use Case: Lets policies and renderers tell pipeline engines apart by how a run survives a failure — a replayed event history or a reconciled cluster resource — rather than by product name.
11. dataplane.aggrega.ai/* (The Data-Path Capabilities)
- Who writes it? The Platform Architect, on any component that sits in the traffic path — a reverse proxy, an ingress gateway, a service mesh, an AI gateway.
- Who consumes it? The Static Analyzer, via a Feature’s
appliesToand a socket’susesSchemaselector. - Compiler Use Case: It answers what this component can do to traffic, which is what a runtime control actually needs to know and what a product-tier label cannot say.
core/traffic-shadowing, which copies live requests to a second service for evaluation, declaresappliesTo: dataplane.aggrega.ai/request-mirroring— the key alone — and the Static Analyzer evaluates that against the target’s labels before anything renders, so attaching it to a component that cannot mirror at all halts with a selector violation naming the Feature and the selector it failed. The compiler treats every one of these strings as opaque, exactly as it treats the rest. - Why the prefix names the traffic path rather than a kind of component. Mirroring, caller identity and upstream verification are not gateway properties — they are properties of anything standing in the request path, and a service mesh added to a catalog has all three. Naming the family after the component kind that happened to carry them first would oblige every later kind to restate the same vocabulary under its own prefix, leaving one capability with several names and no selector able to span them.
- Why the fidelity rides in the value. Two components can both mirror requests and still differ in the way a Feature cares about: one copies every matching request, the other copies a settable share of them. A key that only said whether a component mirrors would admit both and leave the difference to be discovered after rendering. Putting the fidelity in the value lets it be asked for in whichever of two ways fits the Feature. A Feature that cannot work at all without one fidelity names that value in its selector. A Feature that still works at the lower fidelity selects on the key alone, and its realization refuses the setting the component cannot honour: copying the whole stream works on every component that mirrors, so
core/traffic-shadowingattaches to all of them, and the realization for a component declaringfull-raterefuses a share below 100 in a message naming the share asked for and the fidelity the component declares — two things the selector violation does not say. The values are therefore derived from what shipping products implement rather than composed as a taxonomy:request-mirroringcarriesfull-rateandsampledbecause real proxies divide exactly that way, one offering a copy-rate setting and another offering none. - A value asserts a capability, never a configured state. The two read identically to a developer and only one of them is a property of the component. A proxy able to verify an upstream certificate carries
upstream-tls: trust-anchorwhether or not any deployment enables the check — the label is what a Feature selects on before anything is configured, so loading it with configuration would make it unreadable at the moment it is used. - What the standard library carries.
core/gateway/nginxcarriesrequest-mirroring: full-rate,upstream-tls: trust-anchorandconcurrency-cap: total, one cap on every request it forwards.core/gateway/contourcarriesrequest-mirroring: sampled,caller-identity: client-certificateandupstream-tls: trust-anchor.core/gateway/envoy-gatewaycarries the same three values ascore/gateway/contour,period-budget: tokens, counting the tokens each reply of a model it serves through Envoy AI Gateway reports, andconcurrency-cap: per-target, a cap on the requests in flight to each model it serves.core/gateway/kongcarriesupstream-tls: trust-anchorandprompt-guard: pattern-list, its AI Prompt Guard matching a model request’s messages against the lists its routes hold.core/gateway/tykcarriesupstream-pinning: public-key, whichcore/upstream-key-pinningselects.core/gateway/litellmcarriescaller-identity: injected-header,content-masking: entity-recognition,period-budget: cost,prompt-guard: pattern-list,upstream-tls: trust-anchorandconcurrency-cap: per-target, a cap on the requests in flight to each entry of its model map.core/gateway/cloudflare-ai-gatewaycarriesperiod-budget: cost.core/gateway/apisixcarriesperiod-budget: tokens, counting the tokens each model’s reply reports and pricing nothing, andprompt-guard: pattern-list.core/gateway/kong-enterprisecarriesperiod-budget: tokens, counting the tokens each model’s reply reports, andcontent-masking: entity-recognition.core/gateway/traefik-hubcarriescontent-masking: entity-recognitionandprompt-guard: pattern-list, its Content Guard middleware masking the matches of the patterns its configuration holds or the entity types a Presidio analyzer it calls detects, or refusing a request a pattern matches, andperiod-budget: cost, priced from the per-model price list its quota middleware ships with, which can count the tokens each reply reports instead.core/gateway/agentgatewaycarriescontent-masking: pattern-listandprompt-guard: pattern-list, its regular-expression guard masking or refusing the matches of the patterns its configuration holds and recognizing no entity types, andperiod-budget: cost, the budgets it keeps per API key in US dollars, priced from its model cost catalog, or in tokens.core/gateway/higresscarriescontent-masking: pattern-list, itsai-data-maskingplugin masking the matches of the patterns its configuration holds in the text of a chat request’s messages and recognizing no entity types, andperiod-budget: tokens, counting the prompt and completion tokens each reply reports together and pricing nothing.core/gateway/azure-api-managementcarriesperiod-budget: tokens, counting the prompt and completion tokens each reply reports together, with a separate count at each gateway of the instance, and pricing nothing.core/security/nemo-guardrailsandcore/security/trustyai-guardrails-gateway, two guards that stand in a model’s path and judge a request with a trained classifier, carryprompt-guard: classifier, which theguardsockets of the AI gateway and the NGINX gateway select. The AI gateways among them —core/gateway/litellm,core/gateway/cloudflare-ai-gateway,core/gateway/apisix,core/gateway/envoy-gateway,core/gateway/kong-enterprise,core/gateway/traefik-hub,core/gateway/agentgateway,core/gateway/higressandcore/gateway/azure-api-management— carrysemanticand the rest carry nogateway.aggrega.ai/intelligencevalue, which is the separation working rather than an exception to it: a Feature needing a mirror reaches the three components that have one, and a product-tier selector would reach only Envoy Gateway among them, for a reason that says nothing about mirroring, and a Feature needing a prompt guard, content masking or a period budget reaches an AI gateway on what it can do rather than on the tier it sits in, while one capping the requests in flight reaches the NGINX gateway, which carries no tier value, as it reaches Envoy Gateway and the AI gateway, and one refusing a request on a list of patterns reaches open-source Kong, which carries none either, as it reaches APISIX. A component with nothing to describe on one of these axes carries no value there rather than a weakest one, so a selector naming any value on that axis correctly does not attach to it.
12. security.aggrega.ai/text-anonymization
- Who writes it? The Platform Architect, on a service that takes part in masking sensitive text. Where that work is split across two services,
analyzergoes on the one that finds sensitive spans in a text and reports them, andanonymizeron the one that rewrites the spans it is handed;sanitizergoes on a service that does both in one call, finding the sensitive spans in the text it is sent and rewriting them itself, answering with the rewritten text. - Who consumes it? The Static Analyzer, via a socket’s
usesSchemaselector. - Compiler Use Case: It is how a consumer is handed the right address.
core/gateway/litellmdeclares apii_analyzersocket selectingsecurity.aggrega.ai/text-anonymization=analyzerand apii_anonymizersocket selecting theanonymizervalue, so wiring either service to the other’s socket halts with a selector violation naming the socket.core/gateway/kong-enterprisedeclares one socket,pii_sanitizer, selecting thesanitizervalue, andcore/gateway/traefik-hubone,pii_analyzer, selecting theanalyzervalue alone, because the Content Guard its chart Recipe writes forcore/data-maskingreplaces what the analyzer finds itself. The compiler routes the address the selected component publishes and checks nothing about what answers there. - What the standard library carries.
core/security/presidio-analyzercarriesanalyzerandcore/security/presidio-anonymizercarriesanonymizer. Both speak the Presidio HTTP API the AI gateway calls when acore/data-maskingattachment, or the gateway’s ownvalues.log_masking, lists entity types, and Traefik Hub’s Content Guard calls the analyzer alone when acore/data-maskingattachment on that gateway lists them.core/security/kong-ai-pii-servicecarriessanitizerand speaks Kong’s AI PII service API, which Kong Gateway Enterprise’s AI PII Sanitizer calls with every text it masks, its regular expressions included, while acore/data-maskingattachment is on that gateway. The sanitizer presents no credential to it and every text reaches it unmasked, so its address is an in-cluster one, not meant to be exposed beyond the cluster.
13. model.aggrega.ai/api
- Who writes it? The Platform Architect, on every component in the model category and on a gateway that serves a model API. The value names which model API answers at the address the component publishes:
openaiwhere it is the OpenAI API — the vendor’s own, or a server’s implementation of it —anthropicwhere it is Anthropic’s Messages API,geminiwhere it is Google’s Gemini API, andnativewhere it is the product’s own API and none of those, which a consumer reaches through that product’s client. Each value is read off the product’s own reference, and a component carries it whatever a given Release configures. - Who consumes it? The Static Analyzer, via a socket’s
usesSchemaselector, and the assertions of the AI gateway, ofcore/gateway/kong, ofcore/gateway/kong-enterprise, ofcore/gateway/apisix, ofcore/gateway/envoy-gateway, ofcore/gateway/traefik-hub, ofcore/gateway/agentgatewayand ofcore/gateway/higress, on each connection to theirmodelssockets. - Compiler Use Case: It is what lets one socket take either a model or a gateway serving models. The
modelsocket of every agent but the five runtimes named below, thetarget_modelsocket ofcore/ai-tool/model-invocationand the two model sockets ofcore/datastore/cogneeselectcomponent.aggrega.ai/category in (ai-model, gateway), model.aggrega.ai/api: the category requirement admits the two categories, and the key requirement admits only the components among them that carry the label, so wiring such a socket to a gateway that serves no model API halts with a selector violation naming the socket. A selector’s requirements all have to hold and its alternatives live inside one key’s set, so the one key both kinds of component answer to has to sit on every model component as well as on the gateway — which is why the label names an API rather than marking gateways alone. A gateway carrying it publishes theendpointandmodelkeys a model publishes, so a consumer’s Recipe reads either the same way. An agent whose Recipes deliver the variables of one model API names its value instead: themodelsockets ofcore/ai-agent/openai-agents-sdk,core/ai-agent/agent-frameworkandcore/ai-agent/mastraselectmodel.aggrega.ai/api=openai, that ofcore/ai-agent/claude-agent-sdkselectsmodel.aggrega.ai/api=anthropic, and that ofcore/ai-agent/google-adkselectsmodel.aggrega.ai/api=gemini, so wiring one of them to a model or a gateway serving another API halts with a selector violation naming the socket. The socket ofcore/ai-agent/google-adkalso narrows the category toai-model, refusing every gateway, because ADK’s client reads no address and would send a gateway’s credential to Google’s own address. The AI gateway reads the value rather than selecting on it: itsmodelssocket selectscomponent.aggrega.ai/category=ai-model, admitting every model, and because the gateway describes a model wired there to its proxy through the OpenAI API alone, its Component’s assertion halts the build on a connection whose target carries any value butopenai, naming the connection and saying to type that upstream into the gateway’s hand-typed model map instead.core/gateway/kong,core/gateway/kong-enterprise,core/gateway/apisix,core/gateway/envoy-gateway,core/gateway/traefik-hub,core/gateway/agentgatewayandcore/gateway/higressread it the same way on their ownmodelssockets, because each writes a wired model’s route, or on agentgateway its model entry, for the OpenAI API alone. The compiler treats the value as an opaque string and checks nothing about what answers at the address. - What the standard library carries.
core/gateway/litellm,core/gateway/kong,core/gateway/kong-enterprise,core/gateway/apisix,core/gateway/envoy-gateway,core/gateway/traefik-hub,core/gateway/agentgatewayandcore/gateway/higresscarryopenai, as do the self-hosted serverscore/ai-model/vllm,ollama,docker-model-runner,tgi,tei,nvidia-nimandlitellm, the managed APIscore/ai-model/openai,openai-embeddingandnvidia-api, and the brokercore/ai-model/openai-compatible, which is pointed at a service speaking that API.core/ai-model/anthropiccarriesanthropic, andcore/ai-model/geminicarriesgemini. Every other model component carriesnative.
14. tool.aggrega.ai/transport
- Who writes it? The Platform Architect, on a tool a consumer reaches over a transport it has to speak. The value names that transport.
- Who consumes it? The Static Analyzer, via a consumer’s tool socket.
- Compiler Use Case: It is how a consumer is handed only the tools it can talk to. A tool socket selects
component.aggrega.ai/category=ai-tool, tool.aggrega.ai/transport=http: the category requirement admits tools and nothing else, and the transport requirement admits only the tools reached over HTTP, so wiring such a socket to a tool that carries no such label halts with a selector violation naming the socket. How a tool is reached has to be a label rather than a published value, because a selector reads labels and never the content of a value. The compiler treats the value as an opaque string and checks nothing about what answers at the address. - What the standard library carries.
core/ai-tool/mcpcarrieshttp. It attaches a Model Context Protocol server to an agent, and the Manifest chooses how that server is realized:helm/mcpandcompose/mcprun an image serving the protocol over HTTP;helm/mcp-stdio-bridgeandcompose/mcp-stdio-bridgerun a package that speaks only stdio behind a pinned bridge serving it over HTTP;helm/mcp-remoteandcompose/mcp-remotereach a server someone else runs, and run nothing. Every realization publishes the samehost,port,endpoint,protocolandcredential_header, so a consumer’s wiring does not change with the realization. A server’s own credentials are environment variables, held in the tool’sserver_envslot: delivered to the bridge by reference on Kubernetes, and on Docker Compose written into the service’s environment as delivered, where a Compose variable reference keeps the credential out of the file; a remote server’s credential is held by the consumer on its own connection to the tool and presented under the headercredential_headernames,Authorizationunless the Release names another. Where a remote server reads its key from its own address, the consumer holds the whole address on that connection too, in anendpointslot filled on Kubernetes by a reference to a Secret and on Docker Compose by a Compose variable reference, so the key reaches no rendered file, while the tool’sinfra.endpointholds the address without the part that carries the key; examples/mcp-address-from-secret builds both.core/ai-agent/openai-agents-sdk,core/ai-agent/claude-agent-sdk,core/ai-agent/agent-framework,core/ai-agent/google-adkandcore/ai-agent/mastra, and the flow platformscore/ai-agent/flowiseandcore/ai-agent/n8n, declare a tool socket of this form,tools, and deliver each wired tool’s address, and the connection’s credential by reference, under variables named for the connection’s alias, while a connection writing aninject:block receives what that block dictates in place of the variables derived from what the tool publishes, what itscredentialandendpointslots hold staying under the alias’s name; a consumer of its own declares one the same way. Each of the three flow platforms carries a flow authored in the Aggregate into the platform at deploy, on both targets: Langflow loads the carried definition from its flows directory at each start, Flowise’s chatflow is seeded by a one-shot step signed in as the owner account the Release names, and n8n’s workflow is imported by an init step before the platform starts.core/ai-agent/langflowdeclares atoolssocket of the same form, which admits noinject:block, and delivers each connection through that flow instead, because Langflow’s MCP Tools component takes a server’s address only from Langflow’s own settings or the flow, never from a variable: its Recipes write the tool’s address into the MCP Tools node whose server is named for the connection’s alias, deliver the connection’s credential by reference as a variable that node’s header names — underAuthorization, a variable holdingBearerand the credential — and let the hosts of the wired tools and stores through Langflow’s guard against private addresses. All three platforms also declare avector_storesocket selectingdatastore.aggrega.ai/engine in (vector, semantic-memory, search), which delivers each wired store’s address — theendpointit publishes, forcore/datastore/pgvectora connection URI carrying no password, or one composed from its protocol, host and port — and the connection’s credential by reference, the same way; a store that also publishes its database and user, ascore/datastore/pgvectordoes, reaches Flowise and n8n in those parts as well, since neither platform’s PostgreSQL vector node takes a connection string; Langflow, whose PGVector component takes one string, receives such a store asAGGREGA_VECTOR_STORE_<ALIAS>_CONNECTION_STRING, the connection’s credential its password, by reference on Kubernetes and as delivered on Docker Compose. A Flowise chatflow reads a delivered variable through a Runtime variable of the same name created on Flowise’s Variables page, and an n8n workflow reads one through$envonly where the Aggregate sets the platform’sallow_env_access_in_nodesvalue, since by default its Recipes keep n8n’s block on environment access in nodes.
2. Port Abstraction (The Zero-Port Convention)
The open-source Aggrega standard-library catalog recommends omitting physical port declarations from Components’ cognitive layer, routing port selection through the Ops Recipe instead. Authors following this recommendation avoid port collisions across deployment targets, prevent backdoor APIs that bypass the Injection Contract, and keep network topography flexible across providers. The compiler itself is agnostic to ports — it pipes whatever values the resolved JSON Schema permits. The convention operates with an explicit escape hatch for legacy systems whose physical ports cannot move.
- The Cloud-Native Default (Dynamic Injection): For modern workloads,
Components omit physical ports entirely (no
portsorexposearrays) from theirschema:declaration. Platform Ops Recipes choose the internal TCP/UDP port and publish it as a namedprovideskey, which downstream consumers read through the.context.usesInjection Contract. - The Legacy Escape Hatch (Explicit Declaration): When a legacy component has an unchangeable, hardcoded port, the Platform Architect explicitly exposes
a
portfield within the Component’sschema:declaration. This gives the Developer a validated way to declare the physical constraint. The compiler carries the value through the pipeline unchanged; the Recipe consumes it via{{ .values.port }}. - Operational Ports (Out-of-Band): Ports used strictly by cluster infrastructure (e.g., Prometheus scraping, Envoy sidecar metrics) remain hidden from the Developer’s cognitive layer. They are injected out-of-band by Platform Ops via Features (e.g., attaching the
core/prometheus-exporterFeature, whose scrape port is operatorinfra).
3. Domain-Specific Schemas (Configuration Intelligence)
The developer’s values blocks (which are opaque to the Go compiler but strictly typed by the Component’s schema: declaration) are validated to support explicit AI workflows:
- Agent Links (Dependency Arrays): The standard library’s shared agent parameter schema carries
tools_allowedandagents_allowedasvaluesarrays naming the catalog Components an agent may invoke or delegate to — a documented convention the compiler does not evaluate. A link the compiler wires as a topological edge is ausessocket the agent’s Component declares in itsusesSchema, and that socket’s selector is what the build checks. - Deep Cognitive Memory Partitioning (CoALA): Rather than one flat memory bucket, an Architect can declare a
cognitive_memorysocket whose connections map Semantic, Episodic, and Procedural memory to distinct infrastructure targets as explicit topological edges, so policy engines can evaluate and mandate granular access guardrails (e.g., ensuring an agent can read Semantic memory, but cannot overwrite it if hijacked). The standard library declares no such socket. - Multimodal Pipelines: A Component’s
schema:can declare explicit execution stages such asdocument_chunking,embedding_generation, andstorage_routingas typedvalues, so a declarative data flow is validated before anything renders. The standard library declares no such stages.
4. Gateway Intelligence & Routing
The standard library’s AI gateways carry the controls for routing requests to LLM APIs: core/gateway/litellm, which runs in the cluster, and two hosted services — core/gateway/cloudflare-ai-gateway, Cloudflare’s hosted gateway, which its terraform Recipe creates in a Cloudflare account, and core/gateway/azure-api-management, the AI gateway of Azure API Management, whose terraform Recipe adds an API serving a model, with its backend and its policy, to an API Management instance that already exists. The helm and compose Recipes of each hosted gateway reference it where it is already configured. Some controls are values on a gateway itself and some are Features attached to it, and each Feature below says on which gateway the standard library realizes it. On a hosted gateway every control is written into the gateway’s own configuration — a setting of the Cloudflare gateway’s resource, an element of the API’s policy on Azure API Management — so only its terraform Recipe can carry one, and its helm and compose Recipes halt the build on any attachment. Below, “the AI gateway” is core/gateway/litellm, “the hosted gateway” is core/gateway/cloudflare-ai-gateway, and Azure API Management’s gateway is named in full.
Outside Aggrega, at run time
agent
its model socket wired to the gateway
AI gateway
with the controls attached to it
token budget
semantic cache
prompt-injection shield
model
a provider behind the gateway
An agent’s requests pass through the gateway its model socket is wired to, so every control attached to the gateway stands between the agent and the model. The three drawn here are the Features core/period-budget, core/semantic-caching and core/prompt-injection-shield, shown side by side for where they sit rather than as one gateway’s configuration: the entries below say which gateway realizes each control and what halts the build.
-
The models behind it: the AI gateway’s model map — the aliases it serves, each with the upstream a request for it is sent to — has two parts. A model component wired to the gateway’s
modelssocket, managed or self-hosted, is served under the connection’s alias, its entry written from theendpointandmodelthe model publishes, strings the compiler pipes into the gateway’s configuration and never resolves; the model is therefore a component of the graph, reached over an edge of its own. The gateway describes such a model to its proxy through the OpenAI API alone, so a connection reaching a model whosemodel.aggrega.ai/apiis notopenaihalts the build, naming the connection. The other part is the gateway’svalues.models, entries typed by hand in the proxy’s own provider/model notation: for an upstream the catalog does not model, for a provider whose own API is not the OpenAI API, and for several entries sharing one alias, which wired models cannot do, since each is served under its own connection’s alias. A wired entry presents the credential its own connection carries, as “The key a caller presents” below describes, and never the gateway’sprovider_key; every hand-typed entry presents the one credential in that slot. An alias both parts publish halts the build, naming it, and so does a gateway neither part gives a model. -
In an agent’s path: the gateway publishes an
endpointand amodel, the keys a model component publishes, and carriesmodel.aggrega.ai/api: openai, so an agent’smodelsocket takes the gateway directly and every control attached to it stands between that agent and the providers behind it. Themodelit publishes is the alias itsvalues.modelnames, which the build halts on when neither part of the map publishes it; it may be left out only where the gateway serves a single alias and an entry ofvalues.modelspublishes it, which is then the one published. A caller wanting another alias the map publishes asks the same endpoint for it by name.core/gateway/cloudflare-ai-gatewaypublishes the same two keys — the address of one of the gateway’s endpoints, composed from the Cloudflare account and the gateway’s id, and the model a caller asks for, which the gateway forwards unchecked — but carries nomodel.aggrega.ai/api, so an agent’smodelsocket does not take it.core/gateway/azure-api-managementpublishes them too — the address of its API, composed from the instance’s host and the API’s path, and the model a caller asks for, forwarded unchecked — and carries nomodel.aggrega.ai/apieither: the API answers the OpenAI chat completions request, but by default a caller presents an API Management subscription key in its own header rather than OpenAI’s bearer token, so a plain OpenAI client is not enough to call it. -
Kong in front of models:
core/gateway/kong, the open-source Kong gateway, serves the models wired to its ownmodelssocket through Kong’s AI proxy, and of the controls below it realizes prompt deny patterns alone, as Kong’s AI Prompt Guard on every route of each wired model. Each wired model gets a chat route and a completion route under/<alias>/v1, written from theendpointandmodelthe model publishes, and its Recipes import those routes into the gateway’s PostgreSQL backend before it starts. The route chooses the model, so a request is sent to the model its path names whatever model name it carries. The gateway presents each connection’s own credential as a bearer token its configuration reads from the gateway’s environment by name, and a connection reaching a model whosemodel.aggrega.ai/apiis notopenaihalts the build. It publishes the route of the alias itsvalues.modelnames as itsendpointand that alias as itsmodel, and carriesmodel.aggrega.ai/api: openai, so an agent’smodelsocket takes it as it takes the AI gateway. With nothing wired to that socket the gateway serves no model and publishes both keys empty, and an agent wired to it then reads no address and no model name. The import adds and updates routes and removes none, so a route whose connection is removed keeps answering, and the prompt guard acore/prompt-deny-patternsattachment wrote keeps refusing once the attachment is removed, until each is deleted through Kong’s Admin API. For the same reason the gateway declares no callers, and its model routes check no caller’s key: Kong tells callers apart only as the consumers an authentication plugin names, and a consumer and its key, once imported, would stay able to authenticate after the caller is removed from the Release, a rotated key leaving the old one working; nor can a key-auth key be held by reference, so a caller’s key could reach Kong only as a literal in a rendered file. -
Kong Gateway Enterprise in front of models:
core/gateway/kong-enterprise, Kong’s paid edition of the same database-backed gateway, serves the models wired to its ownmodelssocket on the routescore/gateway/kongserves, written from the same configuration and imported into its PostgreSQL backend by a step in the gateway’s own pod before the gateway starts, and publishes itsendpointandmodelas Kong does. Its one realization is a chart Recipe over Kong’s official Helm chart; there is no compose realization. It runs the image the Release names ininfra.image, version tag included, because Kong distributes the enterprise image under its own terms and the catalog therefore names none; the build halts while that key is unset or its tag carries no version. The licence reaches the gateway by reference to the Secret bound to its requiredlicenseslot, never as a value, and the build halts while nothing binds that slot. The compiler validates no licence and contacts no vendor, so a Release that builds cleanly says nothing about one: Kong reads the licence as a node starts, and a node with no valid licence behaves as with an expired one, its entity configuration read-only and its proxy traffic served with the configuration already in place. Kong sells its AI plugins as its AI Gateway Enterprise offering, an entitlement inside the licence that no build can see, and skips such a plugin rather than refusing it where the licence lacks that entitlement, so attachingcore/semantic-caching,core/period-budget,core/data-masking,core/fallback-chain,core/retry-policyorcore/target-selectionhalts the build until the Release setsinfra.ai_gateway_entitledtotrue, affirming that the licence carries it. Of the controls below it realizes the period budget and the fallback chain, each written into the routes the gateway imports, and it realizescore/data-maskingas “Data Exfiltration & Payload Interception” below describes; its realizations of the semantic cache, the retry policy and target selection halt every attachment, each naming why, and the concurrency limit, audit logging, prompt-injection screening, response validation and prompt deny patterns have no realization there, so an attachment of any of them halts the build. A changed route set rolls the gateway, and the import adds and updates routes and removes none, so a route whose connection is removed, and a plugin an AI Feature wrote once the Feature is removed, stay in the database until they are deleted through Kong’s Admin API; for the reasons the bullet above gives, it declares no callers either, and its model routes check no caller’s key. That a running gateway enforces any of these controls is not observed by any build. -
APISIX in front of models:
core/gateway/apisix, the open-source Apache APISIX gateway, serves the models wired to its ownmodelssocket through APISIX’s AI proxy where the Manifest binds its chart Recipe, which runs APISIX 3.18.0. Each wired model gets a chat route and a completion route under/<alias>/v1— the chat route alone while a period budget or prompt deny patterns are attached, since APISIX counts no tokens on a completion request and its prompt guard reads none — written from theendpointandmodelthe model publishes into a routes file rendered with the gateway, which it reads at start and again whenever the file changes. A gateway reading its routes from that file reads no etcd and runs no Admin API, so with a model wired anetcdconnection or anadmin_keyattachment halts the build, and with nothing wired both are required. The route chooses the model. The gateway presents each connection’s own credential as a bearer token its routes read from the gateway’s environment by name, and a connection reaching a model whosemodel.aggrega.ai/apiis notopenaihalts the build. It publishes the route of the alias itsvalues.modelnames as itsendpointand that alias as itsmodel, and carriesmodel.aggrega.ai/api: openai, so an agent’smodelsocket takes it as it takes the AI gateway; with nothing wired it publishes both keys empty. A route whose connection is removed is gone from the next render. A Release may name the gateway’s callers ininfra.callersand their keys through thecaller_keysslot, one Secret holding a data key per caller name: each caller is then a consumer in the routes file whose key is read from that Secret by name, and every route refuses with HTTP 401 a request presenting no declared key, asAuthorization: Bearer <key>— how an OpenAI client presents one — and removes the key from the request it forwards. With no callers the routes are open. Its workload Recipe, at APISIX 3.17.0, keeps its routes in etcd, renders none, and halts on a connection to that socket and oninfra.callers. Of the controls below it realizes the period budget, a semantic cache for the callers it declares, and prompt deny patterns. -
Envoy Gateway in front of models:
core/gateway/envoy-gateway, the open-source Envoy Gateway, serves the models wired to its ownmodelssocket through Envoy AI Gateway, whose controller,core/gateway/envoy-ai-gateway-operator, the gateway calls as its extension server through itsextensionsocket. A model wired with nothing on that socket halts the build, and with the controller wired the Release places the gateway inenvoy-gateway-systemor the build halts. Each wired model becomes a rule of one AI route attached to the gateway, chosen by the model name a request carries, with a backend written from theendpointandmodelthe model publishes, so every wired model is served under/v1at the gateway’s own address. The gateway presents each connection’s own credential as a bearer token, which Envoy AI Gateway’s controller reads from the Secret the credential names, and a connection reaching a model whosemodel.aggrega.ai/apiis notopenaihalts the build. While a model is wired, every route on the gateway takes the 50Mi per-connection request buffer Envoy AI Gateway configures. It publishes its address, qualified withenvoy-gateway-systemso it resolves from any namespace, as itsendpointand the alias itsvalues.modelnames as itsmodel, and carriesmodel.aggrega.ai/api: openai, so an agent’smodelsocket takes it as it takes the AI gateway; with nothing wired it publishes both keys empty. The controller’s webhook serves a certificate cert-manager issues, so cert-manager must already run in the cluster, unless the Release names, in the controller’sinfra.webhook_tls_secret, a Secret holding a certificate the operator supplies, which the webhook then serves. Of the controls below it realizes the period budget, the retry policy and the concurrency limit, each written onto the one traffic policy of the route serving its models. It does not realize the semantic cache or response validation, since Envoy AI Gateway keeps no reply to answer a later request with and checks no reply against a schema, nor the fallback chain: Envoy AI Gateway falls back by retrying onto further backends of a model’s rule, and the gateway writes its one traffic policy’s retry count onto every rule of the route serving its models, so a fallback list would get exactly its sends only where every wired model’s list has the same length. It realizescore/audit-loggingas its access log, as “Audit logging” below describes, and refuses by name what that log cannot keep: what a request cost, since the per-request cost Envoy AI Gateway can compute has no unit and the gateway keeps no prices; a caller the gateway verified, since nothing on that path verifies the caller; and a retention, since that log has no store and no retention setting. examples/envoy-ai-gateway-token-budget builds the whole path, with an agent reaching OpenAI’s managed model through the gateway. -
agentgateway in front of models:
core/gateway/agentgateway, the open-source agentgateway, serves the models wired to its ownmodelssocket from one configuration file its Recipes render. Each wired model is an entry of that file, named by the connection’s alias and written from theendpointandmodelthe model publishes as an OpenAI-compatible provider of chat completions, so every wired model is served under/v1at the gateway’s own address and the model name a request carries chooses it: the request reaches the model asking for the model name the model publishes, and a name no connection carries is answered with HTTP 404. The gateway presents each connection’s own credential as a bearer token, which the file names as a variable the gateway reads from its environment when it reads the file, a variable missing there stopping it from starting, and a connection reaching a model whosemodel.aggrega.ai/apiis notopenaihalts the build. It publishes that/v1base URL as itsendpointand the alias itsvalues.modelnames as itsmodel, and carriesmodel.aggrega.ai/api: openai, so an agent’smodelsocket takes it as it takes the AI gateway; with nothing wired it publishes both keys empty. Two Recipes run the same image, pinned by digest:agentgateway-chartrenders the project’s standalone Helm chart, fetched from the project’s registry, and realizes the controls below, andagentgatewayruns the image as one Docker Compose service and realizes none of them, nor callers, so an attachment there halts the build naming its type; reach for the chart on Kubernetes, where the build can reach that registry. On the chart a Release may name the gateway’s callers ininfra.callersand their keys through thecaller_keysslot, as on APISIX: each caller is an API key of the file, checked in strict mode and read from that Secret by name, so a model request presenting no declared key asAuthorization: Bearer <key>is refused with HTTP 401, and the key is removed from the request forwarded; with no callers the model routes are open. agentgateway puts each key the file names this way, a wired model’s on either Recipe and a caller’s on the chart, into the file’s text before it reads the file as YAML, so a key must be a plain token — letters, digits and-,.,_,~,+,/and=— and other characters can load as a different key or stop the gateway. The gateway reads its file again when it changes, so a caller removed from the Release is refused once the mounted file is updated. Whilecore/audit-logging, orcore/period-budgetwith scopecaller, is attached, the chart Recipe gives the gateway a store, a SQLite file on a volume of its own, and runs one replica. Of the controls below it realizes data masking, prompt deny patterns, the period budget, audit logging and the retry policy, each written into the file, and refuses by name each value it cannot keep. Data masking replaces each match of the patterns the attachment lists with<masked>, or underrefuseanswers the request or the reply with HTTP 403, in a request’s system and developer prompts, its other messages and the arguments and results of its tool calls, and in the message text of a reply that arrives whole; it recognizes no entity types and restores nothing. agentgateway does not mask a streamed reply, so while replies are masked a model request asking for a streamed reply is refused with HTTP 403 before it is forwarded, and so is any model request whose body is larger than the 2 MiB the gateway reads for that check, streamed or not. A refusal by a guard, a deny pattern’s included, is HTTP 403 with the body “The request was rejected due to inappropriate content”. The budget, the records and the retry are described under “Period budget”, “Audit logging” and “Resilience” below. Its realizations of the fallback chain, target selection, the semantic cache, response validation and prompt-injection screening halt every attachment, each naming why, and the concurrency limit does not attach to it, since it carries nodataplane.aggrega.ai/concurrency-capvalue. -
Traefik Hub in front of models:
core/gateway/traefik-hub, Traefik’s paid gateway, serves the models wired to its ownmodelssocket through Traefik Hub’s Chat Completion middleware. Its one realization is a chart Recipe over Traefik’s own Helm chart; there is no compose realization. It runs the image the Release names ininfra.image, tag included, because Traefik distributes Traefik Hub under its own terms and the catalog therefore names none; the build halts while that key is unset or names no tag. The licence token reaches the gateway by reference to the Secret bound to its requiredlicenseslot, under the data keytoken, never as a value, and the build halts while nothing binds that slot or it publishes another data key. Traefik requires outbound HTTPS toapi.traefik.io, where the gateway validates its token; from Traefik Hub v3.21 the gateway serves while that host cannot be reached, retrying in the background, and a token lasts a year by default, the gateway stopping once it expires. The chart Recipe turns off the chart’s export of the gateway’s error logs to Traefik’s platform and its check for a newer release. The compiler validates no token and contacts no vendor, so a Release that builds cleanly says nothing about whether the gateway starts. Each wired model gets a route at/<alias>/v1/chat/completions, sent to theendpointthe model publishes through a Service of type ExternalName and asking for themodelit publishes whatever model name a request carries, so the path chooses the model; it serves chat completions alone. The gateway presents each connection’s own credential as a bearer token, which Traefik Hub reads from the Secret the connection’s Chat Completion middleware names, and a connection reaching a model whosemodel.aggrega.ai/apiis notopenai, or whose endpoint names an address rather than a host, halts the build. It publishes the route of the alias itsvalues.modelnames as itsendpointand that alias as itsmodel, and carriesmodel.aggrega.ai/api: openai, so an agent’smodelsocket takes it as it takes the AI gateway; with nothing wired it publishes both keys empty. A Release may name the gateway’s callers ininfra.callersand the htpasswd hashes of their keys through thecaller_keysslot: Traefik Hub’s API Key middleware then heads every model route and refuses a request presenting no declared key asAuthorization: Bearer <key>, and the key is removed from the request forwarded; with no callers the routes are open. What Traefik Hub does with a caller’s entry naming a Secret or a data key that does not exist is not stated in its reference, so in the worst case the routes are as open as with no callers. Of the controls below it realizes the period budget, the semantic cache for the callers it declares, prompt deny patterns and data masking, through middlewares of Traefik Hub’s AI gateway, and the fallback chain and the retry policy, through Traefik’s own failover and retry, each written into the routes its chart Recipe renders. The AI gateway is an entitlement carried inside the licence, which no build can see, and what Traefik Hub does with a valid token whose licence lacks it is undocumented, so attaching any of the first four halts the build until the Release setsinfra.ai_gateway_entitledtotrue. Its realizations of prompt-injection screening, audit logging, target selection and response validation halt every attachment, each naming why, and the concurrency limit does not attach to it, since it carries nodataplane.aggrega.ai/concurrency-capvalue. That a running gateway enforces any of these controls is not observed by any build. -
Higress in front of models:
core/gateway/higress, the open-source Higress gateway, built on Envoy and configured through Kubernetes resources, serves the models wired to its ownmodelssocket through Higress’s AI proxy. Its one realization is a chart Recipe over the project’s ownhigress-coreHelm chart, which runs Higress’s controller, with its discovery service, and the gateway; there is no compose realization, since the project means its compose distribution mainly for local testing and that distribution runs several cooperating services configured through a store of their own. The Recipe pins the chart, the images and every plugin module it writes, and accepts ininfra.source_versiononly the chart versions its configuration is proven at; the gateway fetches those modules from the Higress plugin registry at run time, which the cluster must reach and the build cannot check. The controller watches only the Release’s namespace, under an IngressClass named for the gateway and that namespace, so two Higress gateways in one namespace collide. Each alias gets a route at/<alias>/v1, sent to theendpointthe alias’s model publishes through an entry of the gateway’s own registry and asking for themodelit publishes whatever model name a request carries, so the path chooses the model; connections naming one alias in theirservesvalue share its requests in equal shares. The gateway presents each connection’s own credential as a bearer token, which the Higress controller reads from the Secret the AI proxy’s configuration names, in the gateway’s namespace, and hands the gateway, so the key is in no rendered file; a Secret name holding a dot halts the build, since Higress reads the name up to its first dot, and a connection carrying no credential passes on the key the request presents. Higress’s AI proxy also copies theAuthorizationheader a client sends into anX-HI-ORIGINAL-AUTHheader, and every model route removes that copy before the request reaches the model, so a key a client presents never travels beside a connection’s own credential. Over https the gateway checks that a model’s certificate chains to the system’s trust anchors and names the model’s host, whatever host the client sent, and sends the model nothing otherwise: the Recipe marks the route of every alias whose connections are all https with Higress’shigress.io/backend-protocol: HTTPSannotation, so the gateway takes the server name from the host the AI proxy writes into each request — the model’s, qualified with the model’s namespace where it carries no dot — and checks the certificate against it. That annotation applies to every connection of an alias, so an alias mixing http and https connections halts the build, naming both; serving them under two aliases clears it. A connection reaching a model whosemodel.aggrega.ai/apiis notopenai, or whose endpoint names an address rather than a host, halts the build. It publishes the route of the alias itsvalues.modelnames as itsendpointand that alias as itsmodel, and carriesmodel.aggrega.ai/api: openai, so an agent’smodelsocket takes it as it takes the AI gateway; with nothing wired it publishes both keys empty. It declares no callers, so its model routes check no caller’s key, and a value needing a caller the gateway verified is refused by name. Of the controls below it realizes the period budget, audit logging and the retry policy, each written onto the model routes, and it realizescore/data-maskingas “Data Exfiltration & Payload Interception” below describes. Its realizations of response validation, the fallback chain, the semantic cache, prompt-injection screening and target selection halt every attachment, each naming why, and neither prompt deny patterns nor the concurrency limit attaches to it, since it carries nodataplane.aggrega.ai/prompt-guardordataplane.aggrega.ai/concurrency-capvalue. That a running gateway enforces any of these controls is not observed by any build. -
Azure API Management in front of a model:
core/gateway/azure-api-management, the AI gateway of Azure API Management, a hosted service, serves one model API through an API on an API Management instance that already exists. Its terraform Recipe reads the instance and adds the API, its chat completions operation, the backend it forwards to and the API’s policy, which removes the caller’s subscription key from what it forwards and authenticates to the model API asinfra.backend_authenticationsays: with the instance’s managed identity, with a provider key held in Key Vault, or with nothing. Its helm and compose Recipes reference an API configured outside the Release and halt the build on any attachment. Of the controls below its terraform Recipe realizes the semantic cache, the period budget, prompt-injection screening and the retry policy, each written into the API’s policy. It does not realize the concurrency limit, since Microsoft calls the cap its concurrency policy enforces never completely accurate, and lower when several capacity units run in a region, nor the fallback chain, since the API forwards to one backend and Azure’s own fallback, a backend pool’s priority groups, moves later requests off a backend whose circuit breaker has tripped rather than sending a failed request on to another model. Its realization ofcore/audit-logginghalts every attachment, naming what is out of its reach: the log itself, which a diagnostic setting on the instance turns on; what a request cost, whilerequire_costis on, since Azure’s model-request log records token counts and the model and no price; and a retention, whereretention_daysis given, since a row lasts as long as the Log Analytics workspace it lands in keeps it. That the policies apply and enforce as Microsoft documents is not observed by any build. -
The key a caller presents: A server that checks the key its callers present holds that key in a slot of its own, named
api_keyon each model-serving target that checks one —core/ai-model/vllm,teiandtgi, the LiteLLM model, and the AI gateway, where it is the key LiteLLM calls its master key — and each of its Recipes delivers the key as the variable the server reads.core/gateway/apisixandcore/gateway/agentgatewayeach hold one key for each caller its Release names ininfra.callers, in itscaller_keysslot: one Secret, holding each caller’s key under that caller’s name.core/gateway/traefik-hubholds its callers the same way, except that its Secret holds the htpasswd hash of each caller’s key, which its key check compares a presented key against, so it is not the Secret the caller presents its key from. A caller holds the key it presents on its own side, on the connection it presents it on, in a slot its socket declares: the standard-library agents declare one on theirmodelsocket,credential, the AI gateway on itsmodelssocket and on itscache_storesocket, where it is the password the gateway’s semantic cache presents to its store,core/gateway/kongandcore/gateway/kong-enterpriseon theirmodelssockets,core/gateway/apisixon itsmodelssocket and on itscache_storesocket, where it is likewise the password its semantic cache presents to its store,core/gateway/agentgatewayon itsmodelssocket,core/gateway/envoy-gatewayon itsmodelssocket and on itscachesocket, where it is the password the gateway’s rate-limit service presents to the store keeping a period budget’s count,core/gateway/higresson itsmodelssocket and on itscachesocket, where it is the password its token limit presents to the store keeping a period budget’s count, andcore/gateway/traefik-hubon itsmodelssocket, on itscachesocket, where it is the password its quota presents to the store keeping a period budget’s count, and on itscache_storesocket, where it is the password its semantic cache presents to its store — so an agent reaching a model, or a gateway reaching a model it serves, holds that connection’s key there. The AI gateway, the two Kongs, APISIX and agentgateway deliver each wired model’s key under a variable of its own, which that model’s entry or route in their configuration names; Envoy Gateway names the Secret in the security policy of the model’s backend, and Envoy AI Gateway’s controller reads the key from it; Traefik Hub names the Secret and the data key in the Chat Completion middleware of the model’s route, and reads the key from it. A connection reaching a model labelledcomponent.aggrega.ai/hosting: managedcarries one, or says in its ownvaluesthat the model takes none,unauthenticated: true; with neither, the caller’s assertion halts the build naming the connection and its target, and a model labelled otherwise draws no such halt. A key a caller’s Component declares in a slot of its own instead is one key for every connection it makes. Neither end sees the other’s attachments, so the server learns nothing of a caller’s key, and the two match because the Aggregate feeds both from one parameter: where both ends resolve secrets in one scope — one Kubernetes namespace, one Compose project — they reference one secret, and otherwise each holds its own, realized from the same store entry. vLLM, TEI and TGI each check a single key, so every caller presents the same key and the server cannot tell its callers apart, and TGI matches its key without regard to letter case; APISIX, agentgateway and Traefik Hub each check every declared caller’s key and tell their callers apart by it; the two Kongs declare no callers and check no caller’s key, for the reasons “Kong in front of models” above gives; the LiteLLM model and the AI gateway also accept the keys they issue, while their bookkeeping database is attached. Credential Model works the shape through in one Aggregate. -
Choosing between targets:
core/target-selectiondeclares how a gateway chooses among several targets that can serve the same request, with astrategyoflatency(the target observed to answer quickest) orcost(the target cheapest to call). The strategy is part of what the application asks of its gateway, so the developer attaches it in the Aggregate. On the AI gateway the targets are the entries of itsvalues.modelsthat publish the same alias — a wired model is served under its connection’s alias, which no other entry may publish — and the standard library realizeslatencyby writing the proxy’slatency-based-routinginto the routing configuration the gateway’s Recipe renders: the proxy sends each request to the entry whose last ten replies took the least time on average, per output token where a reply reports its tokens, or to its first token for a streamed request, and an entry not yet tried counts as quickest. Without the attachment the configuration names no strategy, and the proxy picks among those entries at random.costis refused by name there: the proxy prices an entry by looking its target up, exactly as written, in a price table that names OpenAI’s and Anthropic’s models without the provider prefix the gateway’s targets carry, and prices an entry it does not find at a fixed fallback, so the entries sharing an alias would be priced alike and chosen by their order. Oncore/gateway/kong-enterpriseevery attachment halts, because each model wired to it is served on routes of its own carrying that one model, which leaves nothing to choose between. Oncore/gateway/agentgatewayevery attachment halts too, naming why: agentgateway draws two of a model name’s targets at random and takes the one whose health, divided by the time it has taken to answer, scores higher, so the quickest target is taken only when it is drawn, and it chooses no target by cost. Oncore/gateway/higressevery attachment halts too, naming why: Higress can measure a service’s latency, but its load-balancer plugin runs after the AI proxy — which has already set the request’s model, address and credential for the candidate the route resolved — and writes its choice into a request header nothing routes by without an Envoy filter this catalog does not render, so the choice does not take effect, and it chooses no target by cost. Oncore/gateway/traefik-hubevery attachment halts too, naming why: Traefik picks by lowest response time only among the servers of one Service, which one model name and one credential cover, and between Services balances by weight or by a hash of the client’s address, never by latency, and none of its strategies chooses a target by cost; and each model wired to the gateway is served on a route of its own carrying that one model, which leaves nothing to choose between. On any other gateway an attachment halts the build. -
Resilience:
core/fallback-chainsends a request whose model fails on to other models:chainsnames, for each model alias that falls back, the aliases to try in order until one answers, a fallback that fails following its own entry in turn, andmax_hopsbounds how far along that chain a request is passed. The standard library realizes it on the AI gateway by writing both into the routing configuration the gateway’s Recipe renders, and halts the build on a name the gateway’s model map does not publish. A request falls back only once the gateway has given up on the alias it asked for: where one alias is published more than once, a failed call first moves to another of its entries as a retry, bounded by the gateway’s retry count.core/retry-policysets that count — how many times the gateway may send a request, the first send included (max_attempts), for a failure that succeeds on being sent again, such as a refused connection, a rate limit or a timeout — and the standard library realizes it on the AI gateway by writing it into the routing configuration the gateway’s Recipe renders. Oncore/gateway/envoy-gatewayit is realized as the retry of the traffic policy on the route serving the wired models, the same policy that carries a period budget’s rate limit, because Envoy Gateway attaches one traffic policy to a route: a request is sent again after a refused connection or stream, a reset or a reply of 429, 503 or 504, up tomax_attemptssends, and a model that answers nothing within the route’s request timeout is not sent again, because that timeout spans every send. Oncore/gateway/azure-api-managementit is realized as Azure’s retry around the forwarding, in the backend section of the API’s policy: a request answered with HTTP 429 or a 5xx status is sent again, its body held so it can be resent, one second after the previous send, up tomax_attemptssends. A refused or dropped connection, and a model that answers nothing within the forwarding’s timeout, are not sent again, because Azure ends the retry on such an error and hands it to the policy’s on-error section. Azure retries 1 to 50 times after the first send, so amax_attemptsof 1 writes no retry and one above 51 is refused by name. The retry takes the place of the forwarding the API would inherit, so while it is attached a forwarding setting made at a broader scope of the instance, such as a timeout, does not apply to the API, which forwards with Azure’s defaults. On the hosted gateway,core/retry-policyis realized by writing into the gateway’s resource its retry count, which names the sends after the first, from 1 to 5, somax_attemptsfrom 2 to 6 is honoured and any other value refused;core/fallback-chainis not realized there, because that gateway falls back between models only inside a route a request names in place of a model, so a request for a model never falls back. Oncore/gateway/kong-enterprise,core/fallback-chainis realized on its chart Recipe: the chat and completion routes of each aliaschainsgives an entry carry Kong’s AI Proxy Advanced in place of its AI proxy, with the alias’s model and the models its walk falls back to as targets in priority order — the alias, then its entry’s fallbacks in order, a fallback’s own entry followed in turn only while fewer thanmax_hopshave been counted. The balancer sends a request to each target of the walk once, in order, until one answers, moving on after a connection error or timeout, an invalid reply header or a reply of 429, 500, 502, 503 or 504, and returns every other reply as it came. The build halts on a name themodelssocket does not carry, on a walk reaching an alias twice, besidecore/retry-policy, since Kong keeps one retry count for the whole walk and cannot send to a model again before falling back, and on a gateway image older than 3.12.core/retry-policyis not realized there, and its realization halts every attachment naming why: each wired model is served on routes of its own carrying that one model, so a retry has to send the request to that same target again, and Kong’s reference reads both ways on whether its balancer does. Oncore/gateway/agentgateway,core/retry-policyis realized on the chart Recipe as a retry policy of the gateway’s own configuration, attached to the listener its model requests arrive on: a request is sent again, at once and to the same model, after a reply of 429, 500, 502, 503 or 504, a refused or reset connection, a connection taking longer than 11 seconds to open or a failed name lookup, up tomax_attemptssends. agentgateway sends a request once more than its policy’s attempts, so the policy is written asmax_attemptsless one,max_attempts1 writes none, and one above 256 is refused by name. A request whose body is larger than 64 KiB is sent once, and the configuration sets no timeout, so a model that accepts the connection and answers nothing is waited on rather than sent again.core/fallback-chainis not realized there, and its realization halts every attachment naming why: agentgateway fails over between models by evicting a failed one for the requests after it, while the request that failed is answered with its failure, so no request is walked along a chain. Oncore/gateway/higress,core/retry-policyis realized as the route’s own retry, written as Ingress annotations onto every model route: a request is sent again after a refused or reset connection, a 5xx reply or a 429, a POST included, up tomax_attemptssends;max_attempts1 writes the retry off, overriding the retry Higress’s control plane gives a route carrying no annotation, and one above 2147483648 is refused by name. A model that accepts the connection and then answers nothing or slowly is not sent again, since the route sets no per-try deadline, and a streamed request is sent again only before its reply has begun.core/fallback-chainis not realized there either, and its realization halts every attachment naming why: Higress sends a failed request on at most once, to one other destination — a route’s default backend, or the one fallback route its console writes for an AI route — so no request is walked along a chain. Oncore/gateway/traefik-hubboth are realized on the chart Recipe, and each halts the build on a Traefik Hub image older than v3.20, whose failover takes no middlewares on the services it tries and whose retry cannot send a POST again on a reply’s status — a retry policy of one send excepted.core/fallback-chainsends the route of each aliaschainsgives an entry through nested failovers of Traefik’s own, trying the models of the alias’s walk in order — the alias, then its entry’s fallbacks in order, a fallback’s own entry followed in turn only while fewer thanmax_hopshave been counted — each with its own model name, credential, quotas and retry. A model’s reply of 429 or of 500 to 504, after any retry of that model, moves the request on to the next; any other reply is returned as it came, the last model’s whatever it is, and a request body over 1 MiB is refused with HTTP 413 rather than sent. The build halts on a name themodelssocket does not carry and on a walk reaching an alias twice.core/retry-policyis Traefik’s retry middleware, last on each model route, or on each model a walk tries: a request is sent again at once after a connection that carried nothing to the model and after a reply of 429 or of 500 to 504, a POST included, up tomax_attemptssends, the first included, andmax_attempts1 writes nothing. A reply the gateway forms itself — a refusal of its key check, a guard, the masking or a quota — is never sent again, and neither is a streamed reply once it has begun. -
Period budget:
core/period-budgetbounds what the traffic through a gateway may consume over a period: alimit, theperiodit covers, ascope— everything the gateway serves, each caller or each model — and aunit— a cost in US dollars or tokens — and once what the gateway has counted over the period exceeds the limit, further model requests are refused. It attaches to any component declaringdataplane.aggrega.ai/period-budget. The standard library realizes it on the AI gateway, whose budget covers everything the gateway serves, is counted in US dollars, and starts again at the start of every period — a number of days from midnight UTC, or a calendar month from its first day: the realization writes the limit and the period into the configuration the gateway’s Recipe renders, refuses any other unit, scope or period by name —30damong them, which the gateway counts as a calendar month — and halts unless the gateway’sdatabase_urlandapi_keycredential slots are both attached, because without its bookkeeping database the gateway keeps no count and without an administrative key it checks no caller against the budget. Once the count exceeds the limit, the gateway refuses the administrative key’s model requests as well as every caller’s; its model listing and key management still answer that key. On the hosted gateway it is realized as a spend-limit rule on the gateway’s resource, counted in US dollars over a window that moves forward with time — the lastperiod, for models whose pricing the gateway knows — for everything the gateway serves: a budget per caller or per model, tokens and a calendar month are refused by name. Oncore/gateway/apisixit is realized on the chart Recipe, written onto the routes of the wired models as APISIX’s token rate limiting: counted in the tokens each model’s reply reports —total_tokens,prompt_tokensorcompletion_tokens— over a fixed window ofperiodthat opens with the first model request after the previous window closed rather than on the clock, for everything the gateway serves, for each wired model apart, or for each caller its Release declares ininfra.callersapart, the gateway telling them apart by the key each presents. Once the count reaches the limit the gateway answers model requests with HTTP 503 until the window closes; a request’s tokens are added once its reply completes, so the request that reaches the limit is served, and a reply its semantic cache serves counts nothing and is served past the limit. APISIX counts no tokens on a completion request, so while the budget is attached each wired model gets its chat route alone, and it keeps the count in the memory of the gateway process, so the chart runs one replica and a restart opens a new window. A budget in US dollars, one per caller whileinfra.callersdeclares none, a calendar month and a second budget on one gateway are refused by name, as is an attachment on its workload Recipe, which renders no route. Oncore/gateway/envoy-gatewayit is realized as Envoy Gateway’s usage-based rate limit, on the AI route serving the models wired to the gateway: the route declares the token countunitnames —total_tokens,prompt_tokensorcompletion_tokens— as a cost Envoy AI Gateway reads off each reply, and a traffic policy on that route charges it when the reply completes, againstlimittokens per one unit of Envoy Gateway’s rate limit, for everything the gateway serves or for each wired model apart. Each window is fixed on the UTC clock — a second, a minute, an hour, a day from midnight, a week from a Thursday, or 30 or 365 days counted from 1 January 1970 — and1mois a calendar month, under a setting of Envoy Gateway’s rate-limit service that then holds for every monthly rate limit that install keeps. The count is kept by the rate-limit service Envoy Gateway’s control plane creates and turns on for every gateway it serves, in the Redis-protocol store wired to the gateway’scachesocket. Once the count exceeds the limit the gateway answers model requests with HTTP 429 until the window ends; a request’s tokens are added once its reply completes, so the request that crosses the limit is served. While the store or the rate-limit service cannot be reached, or a check takes longer than Envoy’s rate-limit timeout, model requests are refused with HTTP 500, under a setting of Envoy Gateway’s control plane that then holds for every rate limit that install keeps, so a rate limit applied to it by hand fails closed too; a request admitted just before an outage is still served and not counted, because its tokens are charged after the reply, and a store keeping the count in memory starts every window again when it restarts. A budget in US dollars, one per caller — the gateway tells callers apart only by a header the caller writes itself, and nothing on that path verifies it — any other period and a second budget on one gateway are refused by name, as is an attachment with nothing wired to themodelsor thecachesocket. Oncore/gateway/azure-api-managementit is realized as Azure’s token limit, written into the API’s policy: counted in the prompt and completion tokens each reply reports, together, for everything the API serves or for each API Management subscription, whose key the gateway verifies — either a quota over a fixed window on the UTC clock, an hour, a day, a week, a calendar month or a calendar year (1h,1d,7d,1moor12mo; Microsoft does not say which day a week starts on), answered with HTTP 403 once exceeded, or a rate over one minute (1m), answered with HTTP 429 and counted over a sliding window on the classic tiers and as a token bucket on the v2 tiers. The count is the usage each reply reports, so concurrent requests can pass the limit, and each gateway of the instance keeps its own count. A budget in US dollars or in prompt or completion tokens alone, one per model — the model a request names is written by the caller, and several names can reach one model — one per subscription on an API that requires no subscription key, any other period and a limit that is not a whole number are refused by name. The Consumption tier runs no token limit. Oncore/gateway/kong-enterpriseit is realized on the chart Recipe, written into the routes the gateway imports as Kong’s AI Rate Limiting Advanced: counted in the tokens each model’s reply reports —total_tokens,prompt_tokensorcompletion_tokens— in fixed windows ofperiod, a number of seconds, minutes, hours or days, each request counted in the one window its time falls in, for everything the gateway serves or for each wired model apart, a request under scopemodelcounted against the alias it named whichever model answered it. Once the count reaches the limit the next model request is answered with HTTP 429 until the window ends, so the request that crosses the limit is served; Kong can also refuse a request early, on its own estimate of the prompt tokens it carries. The count is kept in the gateway’s PostgreSQL backend, shared by every replica and kept across a restart. A budget in US dollars — Kong counts cost only from a price its configuration gives each model, and the catalog supplies none — one per caller — Kong tells callers apart only as the consumers an authentication plugin names, and the gateway declares no callers, for the reasons “Kong in front of models” above gives — a calendar month, a second budget on one gateway and a gateway image older than 3.15 are refused by name, as is an attachment with nothing wired to themodelssocket. Oncore/gateway/agentgatewayit is realized on the chart Recipe, written into the gateway’s configuration and counted in the total tokens each reply reports: under scopecalleras a budget on the API key of each caller its Release declares ininfra.callers, over fixed windows ofperiodaligned to the Unix epoch, a day starting at midnight UTC, kept in a SQLite store on a volume the Recipe renders, with one replica; under scopegatewayormodelas a token bucket for everything the gateway serves or for each model name a request asks for, filled to the limit everyperiodfrom when the gateway read its configuration and kept in its memory, so a restart or a changed configuration fills it again. Once what was counted reaches the limit, model requests are answered with HTTP 429 until the window ends or the bucket is filled; the request that reaches it is served. A budget in US dollars — the gateway’s model cost catalog prices none of the models wired to it, so the budget would count nothing — in prompt or completion tokens alone, one per caller whileinfra.callersdeclares none, a calendar month, a limit that is not a whole number and a second budget on one gateway are refused by name, as is an attachment with nothing wired to themodelssocket; its compose Recipe has no realization of it. Oncore/gateway/traefik-hubit is realized on the chart Recipe as Traefik Hub’s quota middleware on the route of each wired model, keeping its count in the Redis-protocol store wired to the gateway’scachesocket: it counts the tokens each reply reports —total_tokens,prompt_tokensorcompletion_tokens— or, underusd, their cost in US dollars, priced from a list of models the middleware ships with, a model it has no price for being refused with HTTP 422, over a window that slides with time, what was counted at one moment freeing again one period later, for each wired model apart, or for everything the gateway serves while one model is wired. Once the count reaches the limit, model requests are refused with HTTP 429; a request’s count is added once its reply completes, so the request crossing the limit, and any served beside it, is served, and a reply the semantic cache serves counts nothing. Traefik’s reference says the quota serves requests uncounted while the store cannot be reached. A budget per caller — the quota tells requests apart by the client’s address, a request header or the request’s host, and none of them is the caller the gateway’s key check verified — one for the whole gateway while several models are wired, since Traefik’s reference does not say whether routes naming one quota share its count, a calendar month and any period but hours or days, or minutes or seconds making whole hours, are refused by name, as are a limit that is not a whole number of tokens and an attachment with nothing wired to themodelsor thecachesocket. Oncore/gateway/higressit is realized on the chart Recipe as Higress’s token limit plugin on the route of each wired model, keeping its count in the Redis-protocol store wired to the gateway’scachesocket: it adds the prompt and completion tokens each reply reports, together, once the reply completes, over a fixed window ofperiod— a second, a minute, an hour or a day — that opens when the first reply in it is counted, for everything the gateway serves or for each alias apart. While the count exceeds the limit, model requests are answered with HTTP 429 until the window ends, so the request crossing the limit is served. A reply reporting no usage counts nothing, so the Recipe asks a streamed request’s model to report its usage and refuses with HTTP 403 a streamed request turning that report off, a request whose content type neither containsapplication/jsonnor begins withmultipart/form-data, which the AI proxy forwards without asking for the report, and one whose body the gateway’s plugins do not read — a compressed body, a content type containingoctet-streamorgrpc, an upgrade; a client deliberately sending a body the gateway’s validation cannot decode gets past those refusals, and such a streamed request goes uncounted. With no masking attached, a request whose body exceeds the gateway’s 32 KiB buffer may be refused with HTTP 413, depending on when the store answers. While the store cannot be reached, model requests are served and not counted: the plugin has no setting that refuses them then. A budget in US dollars or in prompt or completion tokens alone, one per caller — the gateway declares no callers, so it verifies none — any other period, a limit that is not a whole number and a second budget on one gateway are refused by name, as is an attachment with nothing wired to themodelsor thecachesocket. -
Concurrency:
core/concurrency-limitcaps how many requests a gateway carries at once (max_in_flight), which keeps a burst of callers from becoming a queue at the model behind it.scopesays what one cap counts:gateway, the default, counts every request the gateway forwards together;modelgives each target the gateway sends requests to a count of its own, a target being each upstream entry the gateway routes to, so where one model name has several entries each is counted apart. The cap and its scope are capacity sizing the operator decides, set in the attachment’sinfra, so the operator attaches it in the Release. The cap is counted per running proxy, so where several replicas run each of them admits that many. It attaches to any component declaringdataplane.aggrega.ai/concurrency-cap, and a realization refuses by name the scope its gateway does not count. Oncore/gateway/nginxscopegatewayis realized as NGINX’slimit_conn, keyed on one word the same for every request, so NGINX counts every request it is processing as one total, kept in memory its worker processes share, and answers a request past the cap with HTTP 503; scopemodelis refused, since NGINX keeps no count for each upstream target, and so is amax_in_flightabove 65535, the mostlimit_conntakes. Oncore/gateway/envoy-gatewayscopemodelis realized as the circuit breaker of the traffic policy on the route serving the wired models, the policy that also carries a period budget’s rate limit and the retry policy’s retry: Envoy Gateway applies it to the backend of each rule of that route, and each wired model has a rule of its own, so at mostmax_in_flightrequests are sent to each wired model at once, and a request past the cap is answered with HTTP 503 and is not sent again; a send the retry policy repeats counts against the cap like the first. Envoy’s worker threads check the count and then add to it, so requests arriving together can take it briefly above the cap. Scopegatewayis refused, since Envoy Gateway keeps a cap for each backend and none on their total, and so is an attachment with nothing wired to themodelssocket. On the AI gateway scopemodelis realized as themax_parallel_requestsof every entry of its model map, typed by hand and wired alike: at mostmax_in_flightrequests are in flight to each entry at once, and a request finding its entry full is answered with HTTP 429 — after the gateway’s retries where its alias has one entry, and at once where it has several, even while another of them has room, since the proxy picks the entry before it counts — unlesscore/fallback-chainlists fallbacks for its alias, to which it is then sent on. The proxy keeps each count in memory, where by default a record lapses ten minutes after it is made and a new count then starts at zero, so the attachment also sets that lifetime to a century, keeping every count as long as the proxy runs. Because of that lifetime the gateway refuses a request carryingprioritywhile the cap is attached, even where the Release admits the field, since the proxy keeps a queue entry for every such request until its record lapses; and the build halts on a model map of more than 100 entries, past which the store the counts are kept in, which holds 200 records and drops one whenever it is full, could drop a count. Scopegatewayis refused there, since the proxy’s one total is read only by a request limiter an environment variable switches on, which counts a request for its first second. Nor does the hosted gateway realize it: the request limit its terraform Recipe writes into the gateway’s resource is a rate, a number of requests over an interval, rather than a cap on the requests in flight. -
Audit logging:
core/audit-logginghas a gateway keep a record of each model request it serves, naming the model that answered and the tokens the reply reports, and each realization says where the record lands. It attaches to any component carryinggateway.aggrega.ai/intelligence: semantic, and its three values are the operator’s, set in itsinfra, so the operator attaches it in the Release.require_cost, on by default, has the record also say what each request cost, in a currency the realization names.require_verified_caller, on by default, has it name the caller the gateway verified, a key the gateway issued or checked.retention_days, unset by default, has the gateway delete a record once it is that many days old; with none the gateway itself deletes no record, and a record lasts as long as the place it lands keeps it. A gateway that cannot keep a value refuses the attachment at build, naming the value, so an attachment leavingrequire_costandrequire_verified_callerat their defaults gets the cost and the verified caller, or a halt. Turningrequire_verified_calleroff accepts a record whose caller is whatever the request itself claims, or no caller at all: anyone sending a request can then have it recorded against another caller, so such a record does not show who sent a request. On the AI gateway the record is a row in the gateway’s bookkeeping database, and every value is honoured: the row says what the request cost in US dollars, priced from the table the gateway keeps per model — a model the table does not price is recorded as costing nothing — and names the key the gateway verified, with the user and team the key belongs to, while the end user a row names is whatever the request writes, unchecked. Without its bookkeeping database the gateway writes no row, and without an administrative key it checks no caller and writes no row for a request that presents no key and names no end user, so the build halts unless the gateway’sdatabase_urlandapi_keycredential slots are both attached; while the gateway’svalues.log_maskinglists what to mask, it records answered requests only. Oncore/gateway/envoy-gatewaythe record is a line of the gateway’s access log, which its Recipe writes while the attachment is present: one JSON line per request on the data plane’s standard output, carrying the fields Envoy Gateway writes by default, the model the request named, the model that answered and the prompt, completion and total tokens the reply reports, the route serving the models declaring those counts so that Envoy AI Gateway records them. A request the model did not answer carries no model that answered and no tokens, and the access log keeps nothing itself, so how long a line lasts is up to whatever collects that output. No field names a caller the gateway verified: the client’s address and the headers it sends are written as they arrive. The gateway keeps no prices, nothing on the route its Recipe renders verifies the caller, and the access log keeps no record it can delete, so its realization builds an attachment that turnsrequire_costandrequire_verified_calleroff and gives noretention_days, and otherwise halts naming each value it cannot keep, all of them in one failure; it halts too with nothing wired to themodelssocket. Oncore/gateway/agentgatewaythe record is a row the gateway writes into a SQLite store on a volume its chart Recipe renders while the attachment is present, with one replica, for each model request that passes its key, budget and authorization checks, a guard’s refusal included: the model the gateway asked the model server for — the name the wired model publishes, not the alias the caller sent — and the model that answered, the input, output and total tokens the reply reports, its status and timing, and, for each caller its Release declares ininfra.callers, that caller’s name, from the API key the gateway verified. A request refused for its key, by an authorization rule or by a caller’s budget leaves no row. No prompt or reply text is kept, and the rows are read through the gateway’s admin interface, which listens inside its pod.require_verified_calleris honoured while callers are declared;require_costis refused by name, since the gateway’s model cost catalog prices none of the models wired to it, and so arerequire_verified_callerwith no caller declared andretention_days, since the gateway deletes no record; its compose Recipe has no realization of it. Oncore/gateway/higressthe record is theai_logfield of the gateway’s access log line, which Higress’s statistics plugin fills while the attachment is present: the model that answered and the input, output and total tokens the reply reports, on the gateway’s standard output, with no prompt or reply text. The gateway declares no callers, prices nothing and keeps no record it could delete, so its realization builds an attachment that turnsrequire_costandrequire_verified_calleroff and gives noretention_days, and otherwise halts naming each value it cannot keep, all of them in one failure; it halts too with nothing wired to themodelssocket. Oncore/gateway/traefik-hubevery attachment halts, naming why: Traefik Hub keeps a record of a model request only as an OpenTelemetry span its AI middlewares export to a tracing backend — naming the model and, at its detailed level, the tokens and their cost in US dollars — and its realization wires no tracing backend, so the record would land nowhere; nor does the span name a caller, so it cannot name the one the gateway’s key check verified. Oncore/gateway/azure-api-managementevery attachment halts, as the gateway’s bullet above says, and no other gateway in the standard library realizes it, so an attachment on any other gateway halts naming the type. -
Semantic caching:
core/semantic-cachingdeclares a cache that answers a prompt with the reply stored for an earlier one close enough in meaning, rather than calling the model again.embedding_modelnames the model that turns each prompt into a vector, as the gateway publishes it, a stored reply is served when the two prompts are at leastsimilarity_thresholdalike, and it stays usable forttl_seconds.scopedecides who shares a cache:end_user, the default, keeps each end user the gateway identifies apart, whilekeyserves every caller presenting one key from one cache, so an end user behind that key can be served a reply generated for another.core/data-maskingdeclares a conflict with it, so the build halts on a component carrying both. The standard library realizes it on the AI gateway, which keeps the replies in acore/datastore/rediswired to itscache_storesocket, presenting to the store the password the credential on that connection holds where the store requires one, the store removing each reply oncettl_secondshave passed, and caches chat, responses and messages requests, not completions or embeddings. The realization halts the build unless the gateway’sapi_keyslot is attached, because without an administrative key the gateway checks no caller, on anembedding_modelthe gateway’s model map does not publish, and when nothing is wired tocache_store. A stored reply is served only to a request matching what it was stored under: the key the request presents, which the gateway verifies, with that key’s team and organization; underend_user, the end user the request names; and the model the request asks for with its other parameters. The gateway writes the key, team, organization and end user into the request itself, over any value a caller sent for them on a chat request; on a responses or messages request it removes a team, organization or end user field from the request’s own metadata and refuses one naming the key there, so no request can name its way into another key’s replies. A request’s own metadata can still move it among the replies of the key it presents: on a responses requestmodel_groupreplaces the model it asks for in what its reply is stored under; on a responses or messages requestcaching_groupslets requests naming the same group of model aliases share replies across them; and on a responses or chat requestredis_namespacekeeps requests naming a namespace apart from those naming none. The gateway reads an end user from a header or a body field the request itself names, and verifies none of them: underend_usera caller naming another end user is served that end user’s replies, and requests naming no end user share one cache with every other such request under the key they present. The hosted gateway does not realize it: its cache answers only a request identical to an earlier one. Oncore/gateway/azure-api-managementit is realized as Azure’s semantic cache lookup and store, written into the API’s policy:embedding_modelnames a backend the API Management instance already holds for an embeddings model, which the instance reaches with its managed identity, and the instance needs a Redis cache with RediSearch configured as its external cache, neither of which the build can see, so the build halts until the operator affirms that the cache serves RediSearch by setting the gateway’sinfra.cache_serves_vector_search; the store keeps a reply forttl_seconds; andsimilarity_thresholdis written as Azure’s score threshold, one minus the threshold, reading that score as a cosine distance, since Microsoft gives its range and its direction — lower requires closer prompts — but not its measure. A reply is stored and served apart per API and per API Management subscription, whose key the gateway verifies, so the realization keepskey, refuses it on an API that requires no subscription key, and refusesend_user: the API its Recipe renders validates no end user’s token, so the gateway has no verified end user to keep apart.core/gateway/contourandcore/gateway/nginxdo not realize it either, since they offer no semantic cache. Oncore/gateway/agentgatewayits realization halts every attachment, naming why: agentgateway keeps no cache of replies, and its prompt caching only marks where a provider that caches prompts itself may do so. Oncore/gateway/kong-enterpriseits realization halts every attachment, naming the scope asked for: Kong’s AI Semantic Cache has no setting that keeps one caller’s stored replies apart from another’s, and the gateway tells callers apart only as the consumers an authentication plugin names and declares no callers, for the reasons “Kong in front of models” above gives, so underend_userand underkeyalike every request would share one cache. Oncore/gateway/higressits realization halts every attachment, naming the scope asked for: Higress’s semantic cache stores each reply under one key prefix followed by the prompt text, and nothing in that key names a caller or an end user, while the gateway declares no callers, so every request would share one cache. Oncore/gateway/apisixit is realized on the chart Recipe, written onto each wired model’s chat route as APISIX 3.18.0’s semantic cache, for the callers its Release declares ininfra.callers: a reply is stored per route and per caller, under the key the caller presented, which the gateway verifies, so no caller is served a reply stored for another; the realization keepskeyand refusesend_userby name, because the gateway cannot tell apart the end users behind one key.embedding_modelnames a connection on the gateway’smodelssocket, whose endpoint, model and credential the cache embeds with. The cache embeds a request’s last user message and serves a stored reply whose message is at leastsimilarity_thresholdalike and whose earlier messages — a system prompt, earlier turns — are the same. The replies sit forttl_secondsin acore/datastore/rediswired to the gateway’scache_storesocket, to which the cache presents the password the credential on that connection holds, and a reply served by meaning is stored again under that request forttl_secondsmore. Completion requests are not cached, and a store or embedding failure is a miss: the request is still answered. The realization halts with no caller declared, with nothing wired tocache_storeor to themodelssocket, and on anembedding_modelno connection there carries. APISIX keys a stored reply by route and caller rather than by gateway, so two APISIX gateways wired to one store, or two Releases of one Aggregate pointing their gateway at one external store, can serve a cached reply to a caller of the same name on the same alias and model. Its workload Recipe, at APISIX 3.17.0, carries no semantic cache at all. Oncore/gateway/traefik-hubit is realized on the chart Recipe as Traefik Hub’s semantic cache on the route of each wired model, for the callers its Release declares ininfra.callers: a reply is stored per wired model and per caller, under the caller the gateway’s key check verified, each by its own key, so no caller is served a reply stored for another; the realization keepskeyand refusesend_userby name, because the gateway cannot tell apart the end users behind one key.embedding_modelnames a connection on the gateway’smodelssocket, whose endpoint, model and credential the cache embeds a request’s text with.similarity_thresholdis written as the cache’s distance bound, one minus the threshold, read as a cosine distance, since Traefik’s reference gives the bound’s direction but not its measure. A reply is kept forttl_secondsin acore/datastore/rediswired to the gateway’scache_storesocket, and a request too large to embed is answered uncached. The realization halts with no caller declared, with nothing wired tocache_storeor to themodelssocket, on anembedding_modelno connection there carries, and on a Traefik Hub image whose tag names no release from v3.19.11 within 3.19 or from v3.20.6, since an earlier release shares one cache among every caller of a route. -
Prompt-injection screening:
core/prompt-injection-shielddeclares that the model requests a gateway forwards are checked, before the model is called, for an attempt to override the instructions the model was given, at asensitivityofstandardorstrict— strict refusing more such attempts, and more legitimate requests with them. The check is a probabilistic classifier’s verdict: it misses some attacks — reworded, encoded or respelled ones among them — refuses some legitimate requests, and is known to be evadable, so it is one control beside others, and it does not judge the model’s reply, tool-call arguments or images. It attaches to any gateway, and a realization either sends the gateway’s requests through a guard wired to itsguardsocket, which needs nothing of the gateway but that it forwards to an address it is configured with, or turns on a check the gateway runs itself. Through a guard, the guard checks each request and forwards what it admits to the model, and attaching the Feature with nothing on that socket halts the build naming it. The standard library realizes it on the AI gateway and on the NGINX gateway, each withcore/security/nemo-guardrailsas the guard: a service speaking the OpenAI chat-completions API, whose rail sends the latest user message of a request to the classifier of thecore/security/trustyai-huggingface-detectorwired to it and refuses the request when that classifier’s score reaches the guard’s threshold. A refused request is answered by the guard itself, with an ordinary completion, and nothing of it reaches the model; when the guard or its detector cannot be reached the request is refused too. Each of the two halts the build unless the guard is that one, running the threshold the sensitivity names — 0.5 forstandardand 0.001 forstrict, the two values measured for the detector’s default classifier, which the gateway, seeing the guard and not the detector behind it, cannot check the model of. On the AI gateway every alias of the model map is sent to the guard, each target writtenopenai/<model>, so the map is the one typed by hand: a model wired to the gateway’smodelssocket would be sent to its own endpoint with its own connection’s credential, past the guard, and the realization halts the build while that socket holds a connection, naming it. The AI gateway refuses every request but a chat completion, the one kind the guard checks; on the NGINX gateway its one location forwards to the guard in place of the upstream, and the guard answers no other model route. Oncore/gateway/azure-api-managementit is realized as Azure’s own prompt shield, turned on in the API’s policy through the instance’s Azure AI Content Safety backend thatinfra.content_safety_backend_idnames, and the build halts while that key is unset: Azure AI Content Safety checks each request for a user attack before the model is called, ahead of a semantic cache’s lookup, and the gateway answers one it detects with HTTP 403, as it answers a request longer than the service accepts. The shield is on or off, sostandardis kept andstrictis refused by name, and it runs at Microsoft’s own sensitivity, not measured against the thresholdstandardnames on the guards. The instance reaches that backend with its managed identity, which needs the Cognitive Services User role on the service; the build sees neither. No gateway in the standard library but these three realizes it, and attaching it to another halts the build naming the type: most render no route a realization could send through a guard — they keep their routes in a store of their own, take them from the services behind them, or are a hosted service on which the standard library turns on no check of this kind — andcore/gateway/apisix, whose chart Recipe renders the routes of the models wired to it, andcore/gateway/envoy-gateway, whose Recipe renders the AI route of the models wired to it, have no realization of it. Oncore/gateway/agentgateway, whose Recipes render an entry for each model wired to it, the realization halts every attachment, naming why: agentgateway judges a request for such an attempt only through a service it calls — a webhook the operator runs, or Azure AI Content Safety, Google Model Armor or AWS Bedrock Guardrails — and the realization wires none of them. Oncore/gateway/higressthe realization halts every attachment too, naming why: Higress judges such an attempt only through a service it calls — Alibaba Cloud’s content-security service, or, through a plugin its source carries but its 2.2.5 plugin release does not publish, a Qwen3Guard model service the operator runs — each letting the request through when that call fails, and the realization wires neither. Oncore/gateway/traefik-hubthe realization halts every attachment as well, naming why: Traefik Hub judges such an attempt only through a service it calls — its LLM Guard and Parallel LLM Guard middlewares, which call a guard service or a model in a request and with conditions the realization would have to write, and whose work with a prompt-attack detector is not measured, so no sensitivity maps to a check there; or its Azure Prompt Shield, Amazon Bedrock Guardrails and GCP Model Armor middlewares, which call hosted services the realization does not wire. The guard reads only the latest user message, so a system message, an earlier turn or a tool result reaches the model unread, and only traffic crossing the gateway is screened: a workload calling the model’s address directly goes around it. -
Response Validation & Evals:
core/response-validationdeclares a check of a model’s reply against a JSON Schema before it reaches the caller:schemacarries the schema itself, and a reply that does not match it is refused rather than passed on — or, withinverton, a reply that does match. The schema is the attachment’s, the same for every request, not one the caller sends with its own request, so the developer attaches it in the Aggregate, writing the schema in itsvalues. No realization in the standard library keeps it: oncore/gateway/agentgatewayits realization halts every attachment, naming why — agentgateway’s response guards match patterns or call a content-safety or webhook service, and none of them takes a schema — and so does its realization oncore/gateway/higress, whose reply check answers a reply that passes with the bare JSON it finds in the reply’s message rather than the chat completion its OpenAI route returns, is skipped by any client sending the request headerX-HIGRESS-AI-JSON-RESP: true, and leaves a schema nested more than six levels deep unchecked; and so does its realization oncore/gateway/traefik-hub, since no middleware in Traefik Hub’s reference checks a reply against a schema — its Chat Completion middleware checks requests against the chat completion schema, its API management checks request bodies against an OpenAPI document, and its LLM Guard sends a reply to an outside service, which the realization does not wire; on any other gateway an attachment halts the build naming the type.core/traffic-shadowingcopies live requests to a second service and discards that service’s answers, so a candidate model or service can be evaluated under real traffic without any caller seeing its output. It attaches to any component declaringdataplane.aggrega.ai/request-mirroring, which the AI gateway does not; the standard library realizes it on the NGINX gateway, which copies the whole stream rather than a share of it, and halts on a smaller share by name.
5. Asynchronous Execution & HITL Infrastructure
Aggrega treats human-interactive frameworks (like AutoGen and LangGraph) as worker nodes. To support “sleeping” agents waiting for human input without losing state, Platform Architects can declare usesSchema sockets on these agents’ Components that strictly enforce the explicit binding of datastore-category components to them, selected by the datastore engine label — datastore.aggrega.ai/engine in (vector, semantic-memory, search) for a store the agent retrieves from by similarity, or datastore.aggrega.ai/engine=kv for one it checkpoints its run state into. The socket binds on the engine label rather than on a product name, so an author chooses which store arrives and the declaration still holds. Furthermore, for human-to-agent feedback loops (e.g., Slack integrations), a governance verdict role can be configured to reject the architecture unless an explicitly declared Gateway component is attached, so callback webhooks enter through a declared ingress boundary.
Agents built on a framework or SDK that an application is written around, rather than shipped as a product of its own, take one shape in the standard library. The Release names the application image, which the catalog does not pin. A model socket carries what the framework reads of the wired model — its address and its model name where the framework reads them, and the connection’s credential, by reference — to the variables that framework reads. Where the framework defines no server of its own, the component is a worker and opens no port: core/ai-agent/crewai and core/ai-agent/langgraph take this shape, and so do core/ai-agent/openai-agents-sdk, whose model socket takes only a model or gateway carrying model.aggrega.ai/api: openai, core/ai-agent/claude-agent-sdk, whose socket takes only one carrying model.aggrega.ai/api: anthropic, and core/ai-agent/agent-framework, for an application built on Microsoft Agent Framework, which serves nothing itself, whose socket takes only one carrying model.aggrega.ai/api: openai and delivers the variables Agent Framework’s Python OpenAI clients read; these three halt the build, naming infra.image, until the Release sets it. The OpenAI Agents SDK uploads a trace of each run to OpenAI by default, so core/ai-agent/openai-agents-sdk turns that upload off unless the Aggregate sets its upload_traces_to_openai value to true. Where the framework serves HTTP itself, the component is a server on the framework’s own port and publishes the address a caller reaches as host, port, endpoint and protocol, and it too halts the build, naming infra.image, until the Release sets it: core/ai-agent/google-adk serves ADK’s REST API on port 8000 and delivers the key ADK’s Gemini client sends to the Gemini API, its model socket taking only a model carrying model.aggrega.ai/api: gemini, such as core/ai-model/gemini, and never a gateway, whose address ADK’s client would not read; core/ai-agent/mastra serves the server Mastra builds on port 4111, ready on Kubernetes once its /health route answers, takes a model or gateway carrying model.aggrega.ai/api: openai, and, where the Aggregate names one of the application’s agents in its agent_id value, publishes the address of that agent’s Agent2Agent card, so core/ai-tool/agent-invocation can delegate to it. The routes of both servers answer any caller unless the application authenticates them, so a core/public-ingress or core/http-route attachment on either halts the build until the Release sets its infra.routes_authenticated to true, affirming that the application authenticates every request its routes take. The SDK and server runtimes and core/ai-agent/agent-framework also declare an optional tools socket, which delivers each wired tool’s address, and the connection’s credential by reference, under variables named for the connection’s alias. Because the application is the Release’s own code, the variables these agents deliver for that code to read are defaults: every socket of theirs admits an inject: block, through which the Aggregate names the variables, or the files, the application reads instead, while a variable the framework reads by itself and what an attachment on the connection delivers stay. The same holds for the agent’s own values: each component admits an inject: block of env and file entries on itself, and an entry whose template reads one of its own values and nothing else of the component — no other value, its infra, its metadata or its identity — replaces the default name a Recipe delivers that value under, such as CREWAI_ROLE for a CrewAI agent’s role. An import of a CrewAI project’s agent declarations, a LangGraph application’s langgraph.json, a Google ADK application’s agent configurations or an Agent Framework declarative agent drafts these agents — core/ai-agent/crewai, core/ai-agent/langgraph, core/ai-agent/google-adk and core/ai-agent/agent-framework — wired to the datastore the configuration names, and to the model it names where the agent’s model socket takes it, beside a draft Release that leaves the image the program runs in for the operator to name, and reads the configuration alone, never the program, as Importer Plugins describes.
6. Topological Security Validation & Runtime Enforcement
Because AI workloads introduce novel attack vectors (like prompt injection and autonomous data exfiltration), securing them requires bridging static architectural governance with active runtime enforcement. As an Ahead-of-Time (AOT) compiler, Aggrega does not intercept live network traffic or evaluate AI prompts. Instead, it solves complex security requirements through a two-part architectural handshake: Static Architecture Compilation (Compile-Time) and Execution Handoff (Deploy-Time).
A. Compile-Time: Policy Validation & Artifact Generation
The compiler constructs a flat Intermediate Representation (IR) graph containing all nodes, edges, and attached Features. Before any physical infrastructure is generated, the enterprise’s security posture is enforced by evaluating this static JSON graph against industry-standard policy engines at a governance verdict role.
Because the socket evaluates generic JSON data, it is not locked into a single policy language. Enterprises can compile and enforce their architectural rules using the standard that best fits their ecosystem:
- Open Policy Agent (OPA/Rego): A widely used language for evaluating complex, multi-hop graph relationships and calculating least-privilege access across distributed topologies. The Rego engine compiled into the CLI runs
.regofiles as they stand; a policy that needs OPA’s complete builtin surface is compiled with OPA’s own toolchain and runs behind an adapter guest. - Common Expression Language (CEL): The CEL engine compiled into the CLI runs rules written as single expressions over a document, the way a Kubernetes validating-admission policy is written.
- Kyverno: Evaluates static JSON/YAML manifests, allowing teams to write policies in native YAML rather than learning Rego. No Kyverno guest ships with the compiler, which runs policies only as WebAssembly guests and never invokes the Kyverno CLI, so Kyverno rules run behind a guest that wraps the engine.
- Custom Rust/TypeScript Validation: Security teams can write bespoke organizational policies in standard languages and compile them to Wasm to act as the validation gate.
Once the policy engine validates the graph, the compiler translates those abstract security requirements into physical configuration. When a developer attaches a security Feature — core/data-masking on the AI gateway, for instance — the compiler runs the pipeline step its realization names, an isolated WebAssembly (Wasm) plugin, and that step writes the setting the control needs into the configuration the gateway’s own Recipe already renders. A realization may instead leave the writing to the gateway’s own Recipe, which reads the attachment and writes the setting itself, as core/data-masking on core/gateway/kong-enterprise does, and one for another kind of control may emit a resource of its own; either way the output is a static artifact written from what the developer attached.
B. Deploy-Time & Runtime: Execution Handoff & Enforcement
Aggrega’s responsibility ends the exact moment it finishes processing the input and writes the configured static artifacts to the output directory. It is a pure compiler: it writes the configuration, and then control returns entirely to whatever system invoked it.
How those configurations are executed is highly flexible and completely up to the enterprise:
- The Artifact Handoff (Deploy-Time): External deployment controllers take over to push the artifacts into reality. This could be a strict GitOps controller (like ArgoCD or Flux) syncing files into a Kubernetes cluster, but it could just as easily be a standard CI/CD pipeline executing
terraform apply, or a developer running a local shell script against Docker Compose. - Runtime Enforcement: Once the configurations are applied to the target environment, the physical infrastructure takes over. The specialized tools configured by Aggrega (such as a guardrail sidecar, an active service mesh, or native cloud firewalls) boot up and begin their active work: physically intercepting payloads, verifying JWTs, or dropping malicious prompt injections.
7. How Aggrega Addresses AI Security Challenges
By decoupling architectural validation from runtime execution, Platform Teams can use Aggrega to explicitly model, validate, and generate configuration solutions to the AI vulnerabilities below.
A. Data Exfiltration & Payload Interception
- The Problem: An autonomous agent might attempt to leak restricted enterprise data to an unauthorized external API.
- The Aggrega Solution:
- Compile-Time (Validation & Generation): The Platform Team authors a policy (e.g., Rego) evaluated at a governance verdict role stating: “Every path that passes through an AI gateway must carry
core/data-masking.” If missing, the compiler halts. If present, the realization the Manifest binds writes the masking into the configuration the gateway already renders: oncore/gateway/litellm, guardrail entries that match the regular expressions the attachment lists, and that send the text to a Presidio analyzer and anonymizer wired to the gateway’spii_analyzerandpii_anonymizersockets for the entity types it lists; oncore/gateway/kong-enterprise, Kong’s AI PII Sanitizer on each wired model’s chat and completion routes, written by the gateway’s chart Recipe, which sends the text it masks, with the regular expressions and the entity types the attachment lists, to the Kong AI PII service wired to the gateway’spii_sanitizersocket; oncore/gateway/agentgateway, the gateway’s own regular-expression guards, written into its configuration by its chart Recipe, which replace each match of the patterns the attachment lists with<masked>, or underrefuseanswer the request or the reply with HTTP 403, and recognize no entity types; those guards do not mask a streamed reply, so while replies are masked the same Recipe writes a rule refusing with HTTP 403, before it is forwarded, a request asking for one, and any request whose body is larger than the 2 MiB the gateway reads for that rule, streamed or not; oncore/gateway/traefik-hub, a Content Guard of Traefik Hub’s AI gateway on each wired model’s route, written by its chart Recipe, which masks the matches of the patterns the attachment lists through its regular-expression engine, or of the entity types it lists through the Presidio analyzer wired to the gateway’spii_analyzersocket, one engine per attachment; oncore/gateway/higress, Higress’sai-data-maskingplugin on each wired model’s route, written by its chart Recipe, which replaces each match of the patterns the attachment lists in a chat request’s messages with the pattern’s name in brackets, and recognizes no entity types. A policy can also ask the narrower question — whether the masking sits between the restricted data and the provider — by reading each component’s attachments along the route rather than the route’s attachments as a whole, as examples/policy-data-class-route does. - Runtime (Physical Enforcement): Once the orchestrator deploys the generated artifacts, the gateway replaces each match in the request text it masks with a placeholder before forwarding it, so the model provider receives the placeholder; with
action: refusethe attachment forwards nothing of a request carrying a match and answers it with an error instead. A recognizer that cannot be reached or that errs fails the request, so its text is not passed on. What the masking does not reach is decided by two values whose defaults are the protection. On the AI gateway, while the attachment also masks replies, the gateway masks the reply’s text and the parts a model or its provider adds beside it — a refusal, citations and annotations, a web search’s query and results, fields under names of the provider’s own — or, underrefuse, sends none of a reply carrying a match; a streamed reply it masks by pattern reaches the caller whole, at the end. By default, what its masking does not reach in a request — a route such as image generation, a field such assafety_identifier, an image or a file in the conversation, a tool the provider runs itself — the AI gateway refuses with HTTP 400 before forwarding anything of it; an operator who accepts such text reaching the provider unmasked setsallow_unmasked_texton the attachment in the Release, the realization listing exactly what then passes, and content that is not text stays governed byallow_non_text_content. By default, what a masked reply carries that its masking cannot read — spoken audio and its transcript, an image, the probabilities of the tokens the model chose — is withheld, and a failed call reaches the caller with a fixed message in place of the one the provider returned;allow_unmasked_reply_partslets those reach the caller unmasked. A reply’s own text is never let through by it, but the parts it lets through can carry the reply’s words: a spoken reply’s transcript, the tokens behind the token probabilities, a provider’s error message quoting the reply. Oncore/gateway/kong-enterpriseeach match is replaced by a numbered placeholder, so an attachment listing patterns underredactsetsplaceholder_shows_name: false;refusereaches requests only, from Kong 3.12, which answers a request carrying a match with HTTP 400; and replies are masked from 3.12, a request asking for a streamed reply then being refused before it is forwarded, andallow_streamed_repliesrefused. Kong’s sanitizer reads the text of the user’s turns, a completion’s prompt and a reply’s message text, and Kong’s reference names no setting that refuses a request for what the sanitizer does not read or withholds the rest of a reply, so on that gateway each protection is refused by name at build time while it keeps its default, and an attachment builds once the Release relaxes them —allow_unmasked_textandallow_non_text_contentwhere requests are masked,allow_unmasked_reply_partsandmask_reasoning: falsewhere replies are — withdata-masking-kong-enterpriselisting what each then lets through. There a reply whose text does not arrive as the single string the OpenAI API returns passes unread, its own text reaching the caller unmasked; the guard is the build’s check on every model wired to that gateway, which halts on one whosemodel.aggrega.ai/apiis notopenai. Oncore/gateway/traefik-hubeach match is replaced by a run of*as long as the match, so an attachment listing patterns underredactsetsplaceholder_shows_name: false;refuseanswers HTTP 403 in place of the request or the reply; and a reply that is masked is held whole and sent at the end, a streamed one included, so an attachment masking entity types in replies setsallow_streamed_replies. Traefik’s reference does not say which fields of a chat request or reply Content Guard reads, and names no setting that refuses a request, or withholds part of a reply, for what it does not read, so on that gateway each protection is refused by name at build time while it keeps its default, and an attachment builds once the Release relaxes them —allow_unmasked_textandallow_non_text_contentwhere requests are masked,allow_unmasked_reply_partsandmask_reasoning: falsewhere replies are — withdata-masking-traefik-hublisting what each then lets through. An attachment listing both patterns and entity types, and a second masking attachment on one gateway, are refused, since Traefik’s reference does not say whether a second Content Guard on a route reads the text the first masked or the request as the caller sent it. Oncore/gateway/higressHigress’sai-data-maskingplugin replaces each match of the patterns the attachment lists, in the content and the reasoning of each message of a chat request, with the pattern’s name in brackets; it masks requests only, soscoperesponseandbothare refused, and it recognizes no entity types. The plugin reads nothing else of a request, so while the protections keep their defaults the gateway’s chart Recipe also writes Higress’s request-validation plugin to refuse with HTTP 403, before anything is forwarded, a request carrying what the masking does not read — a body not sent as JSON or not a chat completion, a message whose content is a list or null, a tool call or tool turn, a message’s name, any parameter but a chat completion’s non-text ones and the developer’s own tools, response format and stop sequences — and, whatever the attachment relaxes, a request whose body Higress’s plugins skip — a compressed body, a content type containingoctet-streamorgrpc, an upgrade — and a body holding the Unicode replacement character U+FFFD, which is how that validation reads a byte that is not valid UTF-8, since the masking plugin forwards a body that is not valid UTF-8 unmasked. A request whose body exceeds the gateway’s 32 KiB buffer is refused with HTTP 413. The validation reads only a body its JSON decoder accepts, so a client deliberately sending valid JSON that the decoder refuses — a number beyond what a double holds, for example — passes every refusal of its body: the masking plugin then masks only each message’s plain content and reasoning, so a listed value anywhere else in the body — a tool call’s arguments, a name, any other field — is forwarded unmasked, as is content written with JSON escape sequences, and if such a body is also not valid UTF-8, all of it is forwarded unmasked. A client deliberately nesting a body more than 127 levels deep, inside a field the validation admits such astools, sends valid JSON the validation decodes and admits, but the masking plugin’s JSON reader stops at that depth, so content written with JSON escape sequences in that body is forwarded unmasked as well. Masking is not sufficient on its own: text no pattern matches and no recognizer detects passes unchanged; on the AI gateway, tool-call arguments and tool results are masked value by value but the names of their keys are not; the developer’s own tool definitions, response formats and stop sequences pass as sent; the gateway and the recognizer services see the original text; what the gateway records about a call — in its own log, or in what it hands a logging integration — can still hold that text; and traffic that does not cross the gateway is not touched.
- Compile-Time (Validation & Generation): The Platform Team authors a policy (e.g., Rego) evaluated at a governance verdict role stating: “Every path that passes through an AI gateway must carry
B. Defending Against MCP Poisoning (Trust & Isolation)
- The Problem: If an agent is compromised via a poisoned Model Context Protocol (MCP) tool, it may autonomously execute destructive actions (like dropping a database or authorizing a transfer).
- The Aggrega Solution:
- Compile-Time (Validation & Generation): The developer attaches an approval-gate Feature to the sensitive tool in their
Aggregate.yaml. The standard library ships no such Feature, so this one is authored in the enterprise’s own catalog, together with the FeatureRecipe that renders it. The compiler validates the graph and translates the attachment into the configuration for an L7 proxy or webhook gateway. - Runtime (Physical Enforcement): When the hijacked agent attempts to invoke the tool, the deployed proxy pauses the request and physically blocks execution until a cryptographically signed approval is received from a human operator.
- Compile-Time (Validation & Generation): The developer attaches an approval-gate Feature to the sensitive tool in their
C. Solving the “Confused Deputy” (Context Propagation)
- The Problem: In multi-agent chains, a public-facing agent might be hijacked and command an internal, highly privileged agent to fetch sensitive data. Standard firewalls only see that “Agent A is talking to Agent B” and allow the traffic, completely losing the context of the original malicious user.
- The Aggrega Solution: Enterprises can layer two specific patterns:
- Caller-Identity Propagation, at two hops: The original caller’s identity has to survive two different hops, and each is protected at a different place. At the gateway hop, the developer attaches
core/context-propagationto the gateway a request enters through, which selects ondataplane.aggrega.ai/caller-identity. At compile-time, the attachment is realized in that gateway’s own configuration — on Envoy Gateway, a client traffic policy that verifies each client’s certificate against an authority the Release names and forwards the certificate’s details; on the LiteLLM AI gateway, a request hook that removes any identity header a caller supplied and adds the user and team of the virtual key the gateway verified, and its organization where the key names one, on requests to the model aliases the attachment lists indestinations. At runtime, the service behind the gateway receives who called rather than only that the gateway called. At the workload hop, a request one agent makes directly to another never crosses that gateway, so it is checked at the receiving agent’s own ingress, where a service mesh rejects any request lacking a signed token proving the original user’s intent (e.g., an Istio JWTAuthorizationPolicy). The standard library ships the gateway-hop Feature and no workload-hop Feature, so attachingcore/context-propagationprotects the traffic that passes through a gateway and leaves agent-to-agent calls to mesh configuration the enterprise supplies itself. An agent another party runs, whichcore/ai-tool/agent-invocationreaches over A2A ascore/ai-agent/a2a-remote, sits outside the Release altogether: that party checks each call against the security scheme the agent’s Agent Card declares, and the tool presents the credential held on its connection, read from a Secret by reference, under the request header the agent publishes. Where the card requires OAuth 2.0 client credentials or OpenID Connect, the connection holds a client secret instead, or as well, with the client id the operator writes on it, and the agent publishes the token or discovery address and the scopes its card declares; the tool’s own container obtains the token with them, refreshes it and presents it on each call; the catalog delivers only those inputs, and no build can check what the container does with them. Where the card requires mutual TLS, the connection holds a client certificate, and, where an authority the image does not already trust signs the agent’s server certificate, that authority too; each is mounted into the tool’s container as files from a Secret the connection names, and the container presents the certificate in the TLS handshake itself. So the tool reaches an agent whose card declares an API key in a header or a cookie, an HTTP authentication scheme whose value is fixed, such as Bearer or Basic, with the Secret holding the whole header value, OAuth 2.0 or OpenID Connect with the client-credentials grant, or mutual TLS. It does not reach one whose card requires a key in the query string, an HTTP scheme answering a challenge from the agent, such as Digest, or an OAuth 2.0 flow other than client credentials. Nor does it present a credential to an agent another party runs over plain HTTP: where that agent’s address ishttp, a credential or a client secret on the connection halts the build, naming the connection, because the credential, or the token obtained with the secret, would travel to it unencrypted. An agent running inside the Release stays reachable overhttpwith either, and the credential then crosses the cluster network unencrypted. - Payload Inspection Between Agents: A request one agent sends directly to another never crosses the AI gateway, so it is inspected at the receiving agent’s own ingress, by a guardrail engine running beside that agent (e.g., NeMo Guardrails). At compile-time, the FeatureRecipe realizing a Feature attached to the receiving agent renders the engine’s configuration. At runtime, the engine evaluates each request against rules the enterprise writes and refuses one those rules reject; how well it tells a hijacked request from a legitimate one belongs to the engine and the rules, because Aggrega compiles configuration and never evaluates a prompt. The standard library ships no Feature that places such an engine, so this pattern is authored in the enterprise’s own catalog. A gateway’s own pattern guard,
dataplane.aggrega.ai/prompt-guard: pattern-list, is narrower, andcore/prompt-deny-patternsturns it on — on the AI gateway, every message of a request read and letter case ignored, oncore/gateway/apisixandcore/gateway/kong, every message of every role read as one text, letter case counting and a list of patterns a request must match as well honoured, oncore/gateway/agentgateway, each message, tool-call argument and tool result read apart, letter case counting, and oncore/gateway/traefik-hub, the request read as a chat completion, of which Traefik’s reference does not list the fields read, letter case counting: it refuses a model request whose text literally matches one of the expressions the attachment lists — a paraphrase, another language or an encoding passes it — and only on traffic passing through the gateway, so a request sent directly between agents never meets it.core/prompt-injection-shieldis its classifier counterpart on the same hop, sending each model request a gateway forwards through a NeMo Guardrails guard wired to that gateway’sguardsocket, or through Azure AI Content Safety’s prompt shield on Azure API Management, and it too covers only traffic that crosses the gateway.
- Caller-Identity Propagation, at two hops: The original caller’s identity has to survive two different hops, and each is protected at a different place. At the gateway hop, the developer attaches
D. Topological Risk Analysis
- The Problem: Complex AI architectures with dozens of interconnected models, vectors, and tools are too massive for a human to manually audit for transitive vulnerabilities (e.g., “Can Agent A talk to Tool B, which talks to Agent C, which has access to Database D?”).
- The Aggrega Solution: This check runs ahead of time. Because Aggrega builds a complete IR graph of the intended topology in memory, a governance verdict role can run advanced pathfinding rules across the entire architecture. If the policy detects a continuous, transitive chain of edges originating from a public node and terminating at a highly sensitive database with none of the controls the enterprise requires attached anywhere along it, it immediately halts the build. The insecure architecture is blocked before a single line of infrastructure code is ever generated.
8. Observability & Compliance
To satisfy stringent regulatory requirements (e.g., the EU AI Act), Platform Architects can utilize Aggrega’s feature system to enforce AI observability as a structural infrastructure requirement rather than relying on application code:
- Audit Logging: Enterprise-authored Wasm plugins can configure gateway sidecars to asynchronously stream raw, unredacted prompts and responses to a secure WORM (Write Once, Read Many) storage bucket, bypassing standard application logs.
- LLM Tracing: Proxy features can configure the data-plane to extract and forward AI-specific OpenTelemetry span headers, ensuring complex agent reasoning loops can be visualized in external tools.
9. Feature Compatibility & Topological Safety
Because the core compiler is agnostic to physical infrastructure mechanics, it relies entirely on explicit declarations in the catalog to prevent the generation of incompatible configurations.
conflictsWith: A list, declared on a Feature’s catalog entity underinteroperability, of the Features it cannot be attached alongside. The compiler compares it with the Features each component ends up carrying once every scope has contributed — the component’s own attachments, those written on a subsystem or boundary that holds it, those on its Release component entry, and those annotation-driven expansion produced — and halts before rendering any physical artifacts when one component carries both (i.e., an author attaches, directly or through a scope, two Features that one Feature’s author explicitly declared incompatible). One declaration is enough, and Features are matched by name whatever their versions. The compiler reads nothing into the names: the incompatibility is the author’s declaration, and the check is a comparison of names. Feature atomicity and purity themselves remain a social contract — nothing checks that a Feature does one thing.
10. Dynamic Topology Mapping (Dependency Resolution)
To build the IR graph without hardcoded rules, the compiler reads it from three explicit declarations:
usesEdges: Developers link components in each instance’susesblock, naming the target component in a socket, and every edge of the graph comes from one. Avaluesentry that happens to name another component draws no edge: the compiler reads no meaning into a value.- Edge Targets: Every
usestarget must name a component the build holds; a target that does not exist halts the build, naming the socket it was wired into. - Cross-Node Type Safety: The
usesSchemaselector strings act as explicit out-of-band instructions, evaluated natively by the Go compiler’s label-matching engine during static analysis. They guarantee that developers cannot accidentally wire standard components into a declared socket that requires a specific AI-native capability or compliance label. The guarantee is scoped to declared sockets by construction: a selector lives in ausesSchemaentry, so an ad-hoc socket has none to evaluate. Each edge records whether its socket was catalog-declared, so an operator who needs the guarantee to be total can require it as a governance verdict.
11. Automated Design of Agentic Systems (ADAS)
Advanced AI systems feature Meta-Agents that write code to invent and deploy new agents on the fly. Because Aggrega requires an ahead-of-time (AOT) static architecture, it solves runtime infrastructure evolution through two explicit boundaries:
- The Pre-Provisioned Sandbox Boundary: For ephemeral tools invented at runtime, the Architect defines a feature (e.g.,
dynamic-execution-environment). When attached, the Wasm rendering plugin generates the configuration to provision a locked-down compute pool (like a WebAssembly runtime). The Meta-Agent injects new code into this sandbox, but its blast radius is bounded by the generated static network policies. - The Meta-Agent as the “Developer” Role (The GitOps Loop): If the Meta-Agent needs genuine new physical infrastructure, it acts as a Developer. It writes a new
Aggregate.yamland commits it to Git. This triggers the standard CI/CD pipeline. A governance verdict role statically analyzes the AI’s requested topology, safely halting the build if the AI hallucinated a vulnerability.
12. Model Serialization & Supply Chain Security
Deploying an open-source model often means downloading opaque .pkl or .gguf files that can contain malicious code payloads. Enterprises can treat model supply chain security as a configuration problem of the target platform. By defining a model_provenance feature in the Catalog, an enterprise’s Wasm rendering plugin can evaluate the configuration and translate it into runtime artifacts (like Kubernetes Kyverno policies), so the target cluster is configured to block the orchestrator from pulling unsigned or unverified model weights.