I presented kro (Kube Resource Orchestrator, pronounced "crow") with a live EKS demo at Pod Weekly — RGDs, CEL-inferred dependency graphs, and cloud-plus-cluster orchestration in a single reconciliation loop. This is the write-up, including the parts a slide can't hold.
The Operator pattern is Kubernetes' superpower and its tax. Want to give your teams a clean WebApp API instead of a Deployment + Service + HPA + Ingress ritual? Today that means a CRD and a controller: Go expertise, a reconciliation loop, RBAC, leader election, upgrade strategy, monitoring — per abstraction. Large platform teams end up operating a fleet of bespoke controllers, each one infrastructure that exists to hide infrastructure. AWS's own customer research captured the mood precisely: "we often feel like we are building for Kubernetes rather than with Kubernetes."
Having built an internal developer platform for 40+ teams the hard way — Helm charts, Terraform modules, and a GitOps addon layer — I find this gap deeply familiar. The abstraction layer is the product; the controller plumbing is pure cost. kro's bet is that the plumbing can be generated.
kro (originally AWS Labs, Nov 2024; now donated to kubernetes-sigs under CNCF SIG Cloud Provider) installs one CRD: ResourceGraphDefinition. You write an RGD in YAML; at runtime kro validates it, generates a new CRD for your API, and spins up a dedicated micro-controller for it — reconciliation loop, status propagation, cascade delete included. No Go anywhere.
An RGD has two halves. The schema is your API surface, written in SimpleSchema — types, defaults, and validation in single readable lines instead of OpenAPI boilerplate. The resources are what gets created, as templates wired together with CEL expressions:
spec: schema: kind: WebApp # a brand-new Kubernetes API spec: name: string image: string | default="nginx:alpine" replicas: integer | default=2 hpa.enabled: boolean | default=false status: # bubble child status up via CEL availableReplicas: ${deployment.status.availableReplicas} resources: - id: deployment template: kind: Deployment spec.replicas: ${schema.spec.replicas} - id: service template: kind: Service # depends on deployment — inferred, not declared: spec.selector: ${deployment.spec.selector.matchLabels}
The developer-facing result from the live demo: a platform team defines that RGD once, and an application team deploys a production-shaped stack with eight lines of YAML. Deployment, Service, conditional HPA, status roll-up, and cascade delete — all conjured from kind: WebApp.
Most orchestration systems make you declare ordering (dependsOn, Helm hooks, sync waves). kro doesn't have an ordering field at all — it parses your CEL expressions and infers the dependency graph. If the Service references ${deployment.spec.selector.matchLabels}, the Service depends on the Deployment. That's the whole mechanism, and it produces real graph semantics:
Topological creation, reverse-order deletion — parallel branches (two Deployments referencing only a shared ConfigMap) are created simultaneously; diamond convergence (a Service referencing both) waits for both branches. Deletion unwinds the graph backwards, so no orphans.
Blocking on resolvability, not existence. A dependent resource waits until the referenced CEL field is resolvable — not merely until the parent object exists. That's a subtly stronger guarantee than most hand-written controllers bother with, and it's the same class of bug as the two-phase waiter problem in my fleet-upgrade doctrine: "exists" and "ready" are different facts.
Cycles fail at apply time with a clear error, and the computed order is inspectable: kubectl get rgd webapp-rgd -o jsonpath='{.status.topologicalOrder}'.
CEL is the right glue precisely because it's boring: no side effects, no loops, guaranteed termination, nanosecond evaluation, and type-checked at apply time against fetched OpenAPI schemas. Errors surface before any instance runs — not in a controller log at 2 a.m.
The demo's sharpest moment: AWS Controllers for Kubernetes (ACK) exposes AWS resources — S3, RDS, DynamoDB — as CRDs, and kro treats them like any other node in the graph. A CEL reference from a Deployment's env to a bucket's status field makes cloud provisioning and app deployment a single reconciliation loop:
resources: - id: bucket readyWhen: - ${bucket.status.ackResourceMetadata.arn != ""} # ready = ARN exists template: apiVersion: s3.services.k8s.aws/v1alpha1 kind: Bucket - id: deployment template: env: - name: S3_BUCKET_ARN value: "${bucket.status.ackResourceMetadata.arn}" # cloud status → app env
No glue code, no init-container hacks, no Terraform-then-Helm two-phase deploy. The CEL expression is the dependency edge. Live on the cluster: kubectl exec deploy/s3-demo-app -- env | grep S3_BUCKET_ARN returns a real ARN the pod learned from a resource AWS created seconds earlier.
The honest comparison, as I presented it: Helm packages and templates but is static — no runtime reconciliation, ordering via hook folklore. Crossplane reconciles and composes, with heavier machinery and readiness-check ordering. Custom operators do everything — at the price of owning Go controllers forever. kro fills the specific gap between "Helm for packaging" and "full operator": runtime reconciliation, auto-inferred ordering, CEL-defined status and conditionals (includeWhen), zero Go. It composes naturally with GitOps — RGDs are just resources, so ArgoCD applies platform definitions while app teams commit eight-line instances in their own repos. It would slot directly into the inventory-driven GitOps model as the abstraction layer above the addon fleet.
kro is experimental (v0.9.2 at the time of the talk), and I said so on stage: APIs aren't solidified, it's not an EKS managed addon, and Graph Revisions — the answer to "what happens to existing instances when an RGD schema changes?" — is still landing. Schema evolution is exactly where my inventory-schema-debt scars itch: once hundreds of instances depend on an API's shape, changing it is a migration. I wouldn't put kro under production workloads today.
What it's already excellent for: prototyping internal developer platform APIs. The expensive part of platform design isn't the controller — it's discovering the right abstraction. kro collapses that iteration loop from "write a controller, argue about it, rewrite it" to "edit YAML, re-apply, demo it after lunch." Design with kro now; decide later whether the graduated kro or a hardened operator carries it to production. The trajectory — AWS Labs → kubernetes-sigs → CNCF SIG Cloud Provider in under two years — suggests betting on the former.