按需运行的任务:一个 Aggregate,四个目标
一个携带 core/manual-trigger 的任务在 Kubernetes、Docker Compose、Terraform 和 CloudFormation 上部署而不启动,旁边是一个不携带任何内容的任务,并附有在每个目标上启动一次运行的命令。
有些工作只有在有人要求时才会运行至完成:一次性的数据修复、按需的备份。core/manual-trigger 就是用于此的 Feature。挂接到一个没有调度的 core/compute/task 上时,它会部署该任务而不启动它,每次运行都由某人使用目标自己的命令手动启动。
此工作区为四个目标构建一个 Aggregate——通过 Helm 的 Kubernetes、Docker Compose、Terraform 和 CloudFormation——并在每个目标上展示该挂接在渲染输出中改变了什么,以及启动一次运行的命令。第二个任务不携带挂接,因此差异在同一次构建中即可看到。另外两个 Releases 预期会中止,本页引用了它们各自打印的内容。
它构建了什么
| 实例 | 目录中的 Component | 挂接 | 由什么启动运行 |
|---|---|---|---|
data-fix | core/compute/task | core/manual-trigger | 某人,手动,在每个目标上 |
report | core/compute/task | 无 | 它的调度,或者在 Docker Compose 上为栈的启动 |
挂接位于 Aggregate 中,紧挨着它所改变的任务:按需运行是 data-fix 本身的一部分,因此由开发者编写,并且该 Feature 不接受任何值,也不接受 infra。Release 不为它编写任何内容。
report 的调度是一个参数 report_schedule,Aggregate 以空默认值声明它,并将其读入任务的 schedule 值:
spec: parameters: report_schedule: type: string default: "" [… omitted: the parameter's description]
components: [… omitted: data-fix with its attachment, and the comment above each task] report: type: core/compute/task values: schedule: "{{ .parameters.report_schedule }}"它之所以是参数,是因为四个目标对没有挂接的任务处理方式不一致。Kubernetes 在有调度时将其作为 CronJob 按调度运行,没有调度时则在部署时运行一次。Docker Compose 没有调度器,因此每次栈启动时运行该任务一次,并拒绝调度。在 AWS 上,Terraform 或 CloudFormation 应用时没有任何东西会运行 Amazon ECS 任务,因此那里的任务需要调度或该挂接,两者都没有时会被拒绝。所以 Kubernetes、Terraform 和 CloudFormation 的 Releases 填写 report_schedule,而 Docker Compose 的 Release 将其留空。
data-fix 在全部四个目标上以相同方式构建,没有调度,也没有参数。
六个 Releases
| Release | Manifest | report_schedule | 构建结果 |
|---|---|---|---|
release-helm.yaml | manifest-helm.yaml | 30 2 * * 1-5 | 完成 |
release-compose.yaml | manifest-compose.yaml | 空 | 完成 |
release-terraform.yaml | manifest-terraform.yaml | 30 2 * * 1-5 | 完成 |
release-cloudformation.yaml | manifest-cloudformation.yaml | 30 2 * * 1-5 | 完成 |
release-compose-scheduled.yaml | manifest-compose.yaml | 30 2 * * 1-5 | 中止 |
release-terraform-unscheduled.yaml | manifest-terraform.yaml | 空 | 中止 |
每个 Manifest 将 core/compute/task 映射到其目标的 Recipe,并将 core/manual-trigger 绑定到其目标的 FeatureRecipe:helm/task 和 helm/manual-trigger、compose/task 和 compose/manual-trigger、terraform/task-ecs 和 terraform/manual-trigger,以及 cloudformation/task-ecs 和 cloudformation/manual-trigger。未为该挂接绑定 FeatureRecipe 的 Manifest 会中止构建,并指出该挂接。
每个 Release 在 infra.image 下指定每个任务运行的镜像,因为 core/compute/task 不打包任何软件。AWS 的 Releases 还在 infra.overrides 下指定每个任务运行所用的集群、子网、安全组和执行角色,并且在 report 有调度的情况下,指定调度启动每次运行时所用的角色。其中每个特定于账户的值都是占位符。
生成示例
你可以用下面的命令构建这个示例,也可以让你的 AI 助手替你构建。
将示例写入一个以其命名的新文件夹:
aggrega scaffold --example task-run-on-demand然后它会打印接下来要运行的命令。第一条命令进入新文件夹:
cd task-run-on-demand构建
工作区从 .aggrega-config.yaml 指定的 hub 注册表读取标准库目录。这里的每个 Recipe 都仅从目录渲染,因此一旦锁文件解析完毕,每次构建都可离线运行。
aggrega build --release release-helm.yaml --auto-pull # one Release在文档仓库的检出中(它包含由 aggrega scaffold --example 写出的副本所不具备的共享构建辅助脚本),./check.sh 会构建全部六个 Releases 并检查每个结果。它为每个 Release 打印一行,并且仅当每个都按本页所述结束时才以 0 退出:
RELEASE EXIT EXPECTED VERDICT----------------------------------------------------------------------------------------------------release-helm.yaml 0 complete OK: completed, the files the README showsrelease-compose.yaml 0 complete OK: completed, the files the README showsrelease-terraform.yaml 0 complete OK: completed, the files the README showsrelease-cloudformation.yaml 0 complete OK: completed, the files the README showsrelease-compose-scheduled.yaml 1 halt OK: halted with the expected refusalrelease-terraform-unscheduled.yaml 1 halt OK: halted with the expected refusal
check: every build ended as expected.借助 AI 助手
已配置 Aggrega 工具的 AI 助手也能完成同样的操作:把下面这条提示粘贴给它。
仅使用 Aggrega 工具,不要自行创建任何文件夹。如果工作区文件夹中还没有 `task-run-on-demand` 文件夹,请调用一次 `scaffold` 工具,将 `example` 设为 `task-run-on-demand`,且不设 `dir`:它会把示例写入新的 `task-run-on-demand` 文件夹。然后调用 `build` 工具,将 `workspace_root` 设为 `task-run-on-demand`,`release` 设为 `release-helm.yaml`,`auto_pull` 设为 true,`write` 设为 true。告诉我构建写入了哪些文件;它们位于 `task-run-on-demand/dist/` 下。Kubernetes
release-helm.yaml 写入两个文件,每个任务一个 CronJob,此外不写任何内容。data-fix 是一个保持挂起的 CronJob,其调度是开发者从未编写过的:
apiVersion: batch/v1kind: CronJobmetadata: name: data-fix [… omitted: the labels and annotations the build writes on every resource]spec: schedule: 0 0 30 2 * suspend: true jobTemplate: spec: template: [… omitted: the pod template's metadata, the same labels again] spec: containers: - name: main image: registry.example.com/orders-data-fix:2.0.1 [… omitted: the container's CPU and memory requests and limits] restartPolicy: NeverCronJob 需要调度,而 0 0 30 2 * 指的是 2 月 30 日,任何年份都没有这个日期:挂起时,该 CronJob 不启动任何运行;恢复后,它仍然没有可以启动运行的时间。如果没有该挂接,没有调度的任务会是一个 Job,它在部署后立即运行。
report 是一个按 Release 所填写的调度运行的 CronJob,且未被挂起:
[… omitted: apiVersion, kind and metadata, as in data-fix-cronjob.yaml under report's name]spec: schedule: 30 2 * * 1-5 jobTemplate: [… omitted: the job template, as in data-fix-cronjob.yaml with report's image]data-fix 的一次运行是从其 CronJob 创建的 Job,名称由你选择:
kubectl create job data-fix-run-1 --from=cronjob/data-fix同一命令在给定 --from=cronjob/report 时,会在 report 的计划运行之间启动一次额外运行。
Docker Compose
release-compose.yaml 写入一个 Compose 文件。每个任务都是一个服务,退出后 Compose 不会再次启动它;data-fix 还携带 profile manual,而 report 不携带任何 profile:
services: data-fix: image: registry.example.com/orders-data-fix:2.0.1 [… omitted: the labels the build writes on every service, and the service's network] restart: "no" [… omitted: the CPU and memory limits and reservations] profiles: - manual report: image: registry.example.com/orders-report:1.4.2 [… omitted: the labels and the network, as on data-fix] restart: "no" [… omitted: the CPU and memory limits and reservations, as on data-fix][… omitted: the network the services join]Compose 只有在某个 profile 被启用或该服务在命令行中被指定时,才会启动携带该 profile 的服务,因此 docker compose up 会运行 report 一次,并让 data-fix 保持停止。data-fix 的一次运行通过指定其服务来启动,--rm 会在运行结束后删除其容器:
docker compose -f dist/compose/docker-compose.yaml run --rm data-fixTerraform
release-terraform.yaml 写入一个根配置,每个任务一个 module 块,并在其旁边写入每个任务的模块。两个块的区别在于调度:
# Root configuration for this boundary. Each component rendered here is one# module block below, and the directory that block points at holds that# component's own files. One configuration means one dependency graph, which is# what lets a reference or a depends_on reach from one component to another.
module "data-fix" { source = "./modules/data-fix"
assign_public_ip = false cluster = "arn:aws:ecs:eu-west-1:123456789012:cluster/example-cluster" cpu = "256" environment = {} execution_role_arn = "arn:aws:iam::123456789012:role/example-ecs-task-execution" image = "registry.example.com/orders-data-fix:2.0.1" memory = "512" name = "data-fix" security_groups = ["sg-0fedcba9876543210"] subnet_ids = ["subnet-0123456789abcdef0"]}
module "report" { source = "./modules/report"
assign_public_ip = false cluster = "arn:aws:ecs:eu-west-1:123456789012:cluster/example-cluster" cpu = "256" environment = {} execution_role_arn = "arn:aws:iam::123456789012:role/example-ecs-task-execution" image = "registry.example.com/orders-report:1.4.2" memory = "512" name = "report" schedule_expression = "cron(30 2 ? * 2,3,4,5,6 *)" schedule_expression_timezone = "UTC" scheduler_role_arn = "arn:aws:iam::123456789012:role/example-scheduler" security_groups = ["sg-0fedcba9876543210"] subnet_ids = ["subnet-0123456789abcdef0"]}该模块只有在被给定表达式时才声明 EventBridge Scheduler 调度,因此 data-fix 的任务定义是独立存在的:
[… omitted: the provider requirement, the Region lookup, the task definition and the comments on each]resource "aws_scheduler_schedule" "this" { count = var.schedule_expression == null ? 0 : 1[… omitted: the rest of the schedule]Recipe 以 EventBridge Scheduler 自己的形式写入 report 的调度:30 2 * * 1-5 变为 cron(30 2 ? * 2,3,4,5,6 *),星期几从星期日为 1 开始编号,并使用 UTC,因为 Release 未指定时区。
CloudFormation
release-cloudformation.yaml 写入一个栈模板。它为每个任务包含一个任务定义,并仅为 report 包含一个调度:
{ "AWSTemplateFormatVersion": "2010-09-09", "Resources": { "DataFixTaskDefinition": { [… omitted: the provenance metadata the build writes on every resource] "Properties": { "ContainerDefinitions": [ { "Essential": true, "Image": "registry.example.com/orders-data-fix:2.0.1", "Name": "main" } ], "Cpu": "256", "ExecutionRoleArn": "arn:aws:iam::123456789012:role/example-ecs-task-execution", "Family": "data-fix", "Memory": "512", "NetworkMode": "awsvpc", "RequiresCompatibilities": [ "FARGATE" ] }, "Type": "AWS::ECS::TaskDefinition" }, "ReportSchedule": { [… omitted: the provenance metadata] "Properties": { "FlexibleTimeWindow": { "Mode": "OFF" }, "ScheduleExpression": "cron(30 2 ? * 2,3,4,5,6 *)", "ScheduleExpressionTimezone": "UTC", "Target": { "Arn": "arn:aws:ecs:eu-west-1:123456789012:cluster/example-cluster", "EcsParameters": { [… omitted: the launch type, the network configuration and the task count each run starts with] "TaskDefinitionArn": { "Ref": "ReportTaskDefinition" } }, "RoleArn": "arn:aws:iam::123456789012:role/example-scheduler" } }, "Type": "AWS::Scheduler::Schedule" }, "ReportTaskDefinition": { [… omitted: the same shape as DataFixTaskDefinition, with report's image and family] } }}栈中没有任何内容引用 DataFixTaskDefinition,因此没有任何东西会启动它。
在 Amazon ECS 上启动运行
在两个 AWS 目标上,data-fix 的一次运行都通过 Amazon ECS RunTask 操作启动,传入渲染出的任务定义族,以及 Release 在 infra.overrides 下指定的集群和网络配置:
aws ecs run-task \ --cluster arn:aws:ecs:eu-west-1:123456789012:cluster/example-cluster \ --task-definition data-fix \ --launch-type FARGATE \ --network-configuration 'awsvpcConfiguration={subnets=[subnet-0123456789abcdef0],securityGroups=[sg-0fedcba9876543210],assignPublicIp=DISABLED}'仅按族指定时,任务定义解析为其最新的活动修订版。
构建不做的事
构建写入文件后即停止。它不会在任何目标上启动任何运行:它不连接任何集群,不启动任何 Compose 栈,也不调用任何 AWS API,挂接只改变文件的内容。它也不写入任何脚本:本页上的每条命令都是目标自己的命令,由运维该目标的人运行,且没有一条位于 dist/ 中。挂接的各个实现在任何目标上都不会添加自己的文件;每个实现所做的改变都在任务自身的输出中,如上所示。
会中止的两个 Releases
release-compose-scheduled.yaml 在 Docker Compose 上填写 report_schedule,并且同时将 core/manual-trigger 挂接到 report。在 Kubernetes 和 AWS 上,有调度的任务无论是否有该挂接都会按调度运行;在没有调度器的 Docker Compose 上,有调度的任务无论是否挂接都没有实现。aggrega build --release release-compose-scheduled.yaml --auto-pull 会中止:
[… omitted: what resolving the lockfile prints, the same halt among it][CATALOG_ERROR E0033] Recipe/FeatureRecipe assertions caused by: Recipe "task" instance "report" assertion "schedule_refused": [… omitted: the template position and the assertion's own source]: error calling fail: assertion failed: the component "report" is scheduled "30 2 * * 1-5", and Docker Compose has no scheduler: nothing in a compose file starts a service at a time, so the run would happen at none of the times written. Map core/compute/task to helm/task in the Manifest, which runs it on its schedule as a Kubernetes CronJob; or remove the schedule from the component in the Aggregate to run it once at each docker compose up, or attach core/manual-trigger to it there to start each run with docker compose run --rmrelease-terraform-unscheduled.yaml 在 Terraform 上将 report_schedule 留空,因此 report 既没有调度也没有该挂接,AWS 上没有任何东西会启动它。这正是 core/manual-trigger 存在的用途,aggrega build --release release-terraform-unscheduled.yaml --auto-pull 会中止并指出它:
[… omitted: what resolving the lockfile prints, the same halt among it][CATALOG_ERROR E0033] Recipe/FeatureRecipe assertions caused by: Recipe "task-ecs" instance "report" assertion "schedule_translates": [… omitted: the template position and the assertion's own source]: error calling fail: assertion failed: the component "report" has no schedule and carries no core/manual-trigger attachment, so it runs once, and nothing on AWS starts a run that happens once: neither Terraform nor CloudFormation runs an Amazon ECS task when it applies, so the run would never happen. Give the component a schedule in the Aggregate; attach core/manual-trigger to it there to deploy its task definition alone, for the operator to start each run with aws ecs run-task; or map core/compute/task to helm/task in the Manifest, which runs it once on Kubernetes这里调度通过 Release 填写的参数到达任务,因此给该组件一个调度就意味着填写 report_schedule,正如 release-terraform.yaml 所做的那样。