~ / work / bulk-cicd-modernization

500+ pipelines upgraded, zero tickets_

When a shared CI/CD product needs an upgrade, every team that consumed it has a stale copy. This automation found every deployed pipeline stack across the fleet and brought them all to the latest version — 500+ pipelines patched in parallel — safely, idempotently, and with a paper trail.

role
designer & author
scale
500+ CI/CD pipelines · fleet-wide · multiple regions
stack
Python, boto3, AWS ServiceCatalog, CodePipeline, CloudFormation, STS

the problem

CI/CD infrastructure was distributed to teams as ServiceCatalog products. That's great for day one — and a versioning nightmare for day two. Each mandatory change (new security scanning stage, IMDSv2 enforcement, runtime upgrades) meant asking hundreds of teams to update their provisioned products. Most wouldn't. The fleet drifted.

the approach

 discover            plan                execute             verify
 ┌─────────────┐    ┌─────────────────┐  ┌───────────────┐  ┌──────────────┐
 │ enumerate   │───►│ diff provisioned│─►│ update product│─►│ pipeline runs│
 │ provisioned │    │ version vs      │  │ via Service-  │  │ green? stack │
 │ products in │    │ latest artifact │  │ Catalog API   │  │ healthy?     │
 │ every acct  │    │ skip if current │  │ per account   │  │ → CSV report │
 └─────────────┘    └─────────────────┘  └───────────────┘  └──────────────┘
        ▲
        STS assume-role per account · region-aware · resumable

The engine walks every account, finds provisioned CI/CD products, compares versions, and updates only what's stale. Every action lands in a per-account CSV — who was upgraded, who was skipped, who failed and why.

engineering details

Idempotent by default. Re-running the tool is always safe: current stacks are skipped, failed ones retried. Partial runs don't leave the fleet in an unknown state.

ServiceCatalog as the control plane. Upgrades flow through the same governed API teams used to provision — permissions, constraints, and history all stay intact.

Batteries for the long tail. Companion scripts handle the odd cases: security-group rule cleanup, IMDS enforcement on stray EC2, token generation for multi-account API access, cross-account principal generation.

Human-shaped rollout. Account cohorts (by CSV) let risky changes soak in a small ring before fleet-wide execution.

impact

A change that would have been hundreds of tickets and a quarter of chasing became a supervised run measured in hours. Compliance changes (scanning stages, IMDSv2) now reach 100% of the fleet, provably, with an audit CSV to show for it.

Pythonboto3ServiceCatalog CloudFormationIdempotencyAudit trails
← prev: fleet observability next: onboarding & multi-tenancy →