Note: This post is pure google bait designed to help others who run into the same problem. We hope it helps!
We had been advising clients to rebuild older ParallelCluster stacks — the ones whose cleanup Lambdas are templated on python3.9 — by mid-2026, ahead of the published Lambda block dates. Those dates have moved. AWS now blocks python3.9 function creates on February 1, 2027 and updates on March 3, 2027 so you have a lot more time to plan a refresh if you have not already done so.
TL;DR: Old Parallelcluster stacks will stop working
AWS deprecated the python3.9 Lambda runtime on December 15, 2025. ParallelCluster releases 3.3.0 through 3.13.x template their per-cluster CleanupResources and CleanupRoute53 Lambda functions on python3.9, and for much of the past year AWS’s published schedule said python3.9 function creates would be blocked on August 31, 2026 and updates on September 30, 2026. We had been advising clients with older clusters to rebuild by mid-2026 on that basis. AWS has since moved both dates: block-function-create is now February 1, 2027 and block-function-update is March 3, 2027. Nothing breaks in 2026.
Deprecated runtimes are never blocked from invoking, so an existing cluster keeps running, updating, and deleting normally until those dates. The operations that eventually fail are pcluster update-cluster against a python3.9-templated stack (after March 3, 2027) and pcluster create-cluster with a pre-3.14.0 release (after February 1, 2027). You can check a cluster’s exposure with two read-only CLI calls, and the fix is the rebuild you’d do for any pcluster version move: a new cluster on ≥ 3.14.0 (latest is 3.15.1), which templates python3.12.
What the python3.9 block does to ParallelCluster
ParallelCluster’s CloudFormation stack includes two small Lambda functions, CleanupResources (tears down resources CloudFormation can’t clean up natively on stack delete) and CleanupRoute53 (removes the cluster’s Route 53 hosted-zone records). Whether they’re templated on python3.9 depends on the ParallelCluster release that built the cluster:
- Affected: ParallelCluster ≥ 3.3.0 and < 3.14.0 (3.3.0 through 3.13.x). The CHANGELOG entry for 3.3.0 is “Upgrade Python to version 3.9 … and cluster Lambda resources”; every release up to 3.14.0 inherits it. A cluster created by any of these versions carries python3.9-templated cleanup Lambdas.
- Not affected: ParallelCluster ≥ 3.14.0 — the 3.14.0 CHANGELOG entry is “Upgrade Python runtime used by Lambda functions to Python 3.12 (from 3.9)”. python3.12’s deprecation isn’t until October 2028. The latest release, 3.15.1, is in this band.
- Not affected: ParallelCluster < 3.3.0 — predates the python3.9 cluster Lambdas (and has long been end-of-life for other reasons).
We confirmed both ends of the affected band against live clusters: a 3.12.0 cluster’s template pins Runtime: python3.9 for both functions, and a freshly built 3.15.0 cluster’s cleanup Lambdas are python3.12 out of the box.
The deprecation reaches those two functions through two specific gates — one on creating a python3.9 function, one on updating one. Each gates a different ParallelCluster operation:
| ParallelCluster operation | What it does to the cleanup Lambdas | Gated by | Behavior on/after that date |
|---|---|---|---|
pcluster create-cluster with a pre-3.14.0 release |
CloudFormation issues CreateFunction with Runtime: python3.9 |
block-create — Feb 1, 2027 | Lambda rejects the create; the stack fails and rolls back. New clusters can’t be built from pre-3.14.0 releases. |
pcluster update-cluster on an existing < 3.14.0 cluster, when the change touches those functions |
CloudFormation issues UpdateFunctionConfiguration / UpdateFunctionCode against a python3.9 function |
block-update — Mar 3, 2027 | Lambda rejects the update; the stack update fails and rolls back. |
pcluster delete-cluster |
Invokes the cleanup Lambdas | nothing — invoke is never blocked | Works indefinitely, on any version. |
| Running jobs, Slurm autoscaling, compute-fleet churn | nothing — that’s the Slurm scheduler plugin + EC2, not these Lambdas | n/a | Unaffected. |
The two operational consequences:
Existing clusters: update-cluster is the main operation affected. The cleanup Lambdas aren’t often called — jobs, queues, and autoscaling never call them. They’re only touched when CloudFormation reconciles the stack (update) or invoked (delete). Before March 3, 2027, an update-cluster that re-asserts Runtime: python3.9 succeeds. After that date, an update that causes CloudFormation to modify those functions fails and rolls back. Not every update touches the cleanup Lambdas, but you don’t control which ones do, and the failure would show up inside whatever change you were actually trying to make — a queue resize, an AMI swap, a capacity adjustment, etc.
New clusters: a rebuild on the same old version is blocked at create. After February 1, 2027, building with a pre-3.14.0 release fails at the CreateFunction step, so any replacement has to be ≥ 3.14.0. Recreating on the version you already run stops being an option.
One caveat on the failure modes above: we’re reasoning from AWS’s documented block semantics (create/update of a deprecated-runtime function is refused at the Lambda API, and CloudFormation surfaces that as a resource failure). We have not triggered a real block — today’s date is before both dates — so treat the exact rollback behavior as expected, not observed. The dates and the invoke-is-never-blocked behavior come directly from AWS’s runtime documentation.
The 3.9 runtime block dates have moved twice
The deprecation date and the enforcement dates are different things. Deprecation (December 15, 2025, for python3.9) means AWS stops patching the runtime; functions keep invoking indefinitely. What constrains a ParallelCluster stack are the two later milestones — block-function-create and block-function-update — and AWS has now published three different schedules for those:
| When AWS published it | Block function create | Block function update |
|---|---|---|
| Original schedule (announced ~mid-2025, the standard 30/60-day policy) | Jan 15, 2026 | Feb 15, 2026 |
| First extension (published through much of 2026) | Aug 31, 2026 | Sep 30, 2026 |
| Current (on AWS’s runtimes table today) | Feb 1, 2027 | Mar 3, 2027 |
The middle row is the schedule we had been working from: with creates blocked August 31, 2026, advising clients to rebuild by June or July was straightforward planning. The current schedule replaces it — roughly thirteen months more runway than the original dates. AWS explains the extensions on the runtime table itself:
For some runtimes, AWS is delaying the block-function-create and block-function-update dates beyond the usual 30 and 60 days after deprecation. AWS has made this change in response to customer feedback to give you more time to upgrade your functions.
Could the dates move again? Possibly — AWS labels them “subject to change,” and so far they’ve only moved later.
Unexpected discovery while working on v3.12.0
We were doing routine discovery on a client’s production ParallelCluster environment — an older 3.12.0 cluster running normally — as part of planning a refresh onto a current release.
When we tried to confirm that 3.12.0 was affected by the 3.9 block dates we got back results we did not expect …
- The enforcement dates had moved again. The August 31, 2026 block-create date we’d been planning around is now February 1, 2027, and block-update is March 3, 2027. That discovery is why this blog post was written.
- The live runtime and the IaC had diverged. The cluster’s live Lambda functions were running a current Python runtime, but the CloudFormation template that defines them still specified the deprecated one.
The first finding was simple/obvious.
The second finding was kind of interesting, the operators of this cluster had manually upgraded the runtimes of the lambda functions that parallelcluster-3.12.0 had created – this is also worth a bit of writing as there are some downstream consequences that come from changing the configuration of a cloudformation-created resource.
Live Runtime vs Templated Runtime (Drift caused by human operators)
Here we have what the CloudFormation template pins, then what the live function actually runs:
The template — generated by ParallelCluster 3.12.0 — pins Runtime: python3.9 for both cleanup Lambdas. But the live functions were running python3.14, with a LastModified timestamp roughly 13 months after the stack was last deployed. We didn’t make that change, so we pulled CloudTrail logs to see what did:
An admin from the cloud team had bumped the runtime in the AWS Console (UpdateFunctionConfiguration, runtime=python3.14, from a browser session), on both functions, thirty seconds apart — almost certainly in response to the python3.9 deprecation notice. For a Lambda that’s a managed output of someone else’s CloudFormation — which is what ParallelCluster’s cleanup functions are — it fixes the running function but not the template that defines it.
ParallelCluster re-renders the template from the pcluster release that owns the cluster, so the next pcluster update-cluster will set Runtime: python3.9 right back onto those functions — that’s what 3.12.0’s template says.
And another unexpected consequence, an accidental monitoring blindspot!
The console edit also has a side effect on visibility: Lambda’s deprecated-runtime notices (Trusted Advisor, Health Dashboard) key off the live runtime, so they go quiet once the live function moves to python3.14, while the template exposure remains. A cluster in this state passes every routine check — pcluster describe-cluster healthy, Slurm green, deletes work, no deprecation notices — and the python3.9 dependency only resurfaces on the next stack update.
Checking your cluster
Short answer: Any parallelcluster version earlier than 3.14.0 has the python3.9 dependency that will eventually stop working.
Here is how to verify via CLI commands:
Reading the result:
- Version tag
< 3.14.0→ the cluster’s cleanup Lambdas are templated on python3.9. You’re in the affected range. - Live runtime still
python3.9→ deprecated runtime, live, today. Updates after March 3, 2027 will fail, and the runtime is unpatched now. - Live runtime newer than
python3.9(like thepython3.14above) → someone bumped the live function out-of-band. The template still says python3.9, so the exposure returns on the nextupdate-cluster. The version tag, not the live runtime, is the authoritative signal.
You can also ask the template directly — aws cloudformation get-template --stack-name <cluster> --query TemplateBody --output text | grep -i runtime — and if it prints python3.9, the cluster is templated on a deprecated runtime regardless of what the live function reports.
Fixing the 3.9 runtime block issue
There is no in-place minor-version upgrade for ParallelCluster — the pcluster CLI version is bound to the cluster it manages, so a 3.12.0 cluster can’t be driven by a 3.15.1 CLI. Getting onto a non-deprecated Lambda runtime means the same thing as any pcluster version move: build a new cluster on a current release and cut over.
- Recommended: stand up a replacement cluster on ParallelCluster ≥ 3.14.0 (ideally 3.15.1, the latest). Its cleanup Lambdas are templated on python3.12, and you pick up everything else newer releases fixed. This is the blue/green path you’d take for any pcluster upgrade.
- If you’re staying on the old cluster for now: the hard cutoff is March 3, 2027, so there is still plenty of time to plan for a replacement or refresh effort.
Treat pcluster update-cluster on a python3.9-templated stack as a known constraint after that date, and schedule the rebuild before it — ideally before February 1, 2027, after which a same-version replacement cluster can no longer be created either.
Takeaways
A few things that apply beyond this specific deprecation:
1. “Running fine” and “still updatable” are separate questions
A deprecated-runtime Lambda invokes indefinitely, and a cluster on one runs indefinitely. Liveness tells you a thing currently works; it says nothing about whether the IaC that owns it can still be deployed.
2. Console edits to CloudFormation-managed resources create drift
The human admin runtime bump on that v3.12.0 cluster we worked on was a reasonable response to a deprecation notice — but the function is an output of ParallelCluster’s CloudFormation, so the console edit fixed the live resource while the template kept the old runtime, and the template wins on the next deploy. Before hot-fixing a live resource, check whether something owns it via IaC; if so, the fix belongs at the source (here, a newer ParallelCluster release). A template-vs-live runtime comparison is a cheap addition to a pre-upgrade checklist.
3. Put EOL dates on a calendar
We have an actual deadline — March 3, 2027 – but remember that AWS has moved this date twice already so it is worth confirming and re-confirming dates and assumptions when planning for a refresh or replacement.
Bottom Line
ParallelCluster releases 3.3.0 through 3.13.x template their CleanupResources and CleanupRoute53 Lambda functions on python3.9, which AWS deprecated on December 15, 2025. The enforcement dates that matter have moved twice and now sit at block-create February 1, 2027 and block-update March 3, 2027 — so nothing breaks in 2026, and clusters on those releases keep running, updating, and deleting normally until then. After those dates, pcluster create-cluster on a pre-3.14.0 release and pcluster update-cluster on a python3.9-templated stack will fail. Check your exposure with two CLI calls (the parallelcluster:version tag and the template’s Runtime: lines — the version tag is authoritative even if someone bumped the live function out-of-band), and plan the rebuild onto ≥ 3.14.0 (latest 3.15.1, templated on python3.12) before March 2027.
Sources: – AWS Lambda runtimes & deprecation schedule — python3.9: deprecated Dec 15 2025, block-create Feb 1 2027, block-update Mar 3 2027 (current values; verified against the live table) – Runtime use after deprecation — deprecated runtimes continue to invoke; create/update get blocked on schedule; AWS note on delaying block dates “in response to customer feedback” – Evidence the block dates moved: aws-samples issue #413 (Jul 2025) cites the original block-create Jan 15 2026 / block-update Feb 15 2026; community guides from early 2026 (CloudQuery, Joe Karlsson) cite the extended Aug 31 2026 / Sep 30 2026 dates — both since superseded by the current Feb 1 / Mar 3 2027 values – AWS ParallelCluster CHANGELOG — 3.3.0 (cluster Lambdas → python3.9), 3.14.0 (Lambda functions → python3.12) – Retrieve Lambda functions using a deprecated runtime
