Cloud Composer release notes

(Airflow 2.10.5) The apache-airflow-providers-google package was upgraded to version 17.2.0 in Cloud Composer 2 images and Cloud Composer 3 builds.

For more information about changes, see the apache-airflow-providers-google changelog from version 17.1.0 to version 17.2.0.

(Airflow 2.10.5) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 10.8.0 from version 10.7.0. For changes in other packages, see the preinstalled packages changelog.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.15 (default)
• composer-3-airflow-2.9.3-build.35

New images are available in Cloud Composer 2:

• composer-2.14.4-airflow-2.10.5 (default)
• composer-2.14.4-airflow-2.9.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.14 (default)
• composer-3-airflow-2.9.3-build.34

New images are available in Cloud Composer 2:

• composer-2.14.2-airflow-2.10.5 (default)
• composer-2.14.2-airflow-2.9.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.13 (default)
• composer-3-airflow-2.9.3-build.33

New images are available in Cloud Composer 2:

• composer-2.14.1-airflow-2.10.5 (default)
• composer-2.14.1-airflow-2.9.3

(Airflow 2.10.5) The apache-airflow-providers-google package was upgraded to version 17.1.0 in Cloud Composer 2 images and Cloud Composer 3 builds.

For more information about changes, see the apache-airflow-providers-google changelog from version 15.1.0 to version 17.1.0.

(Airflow 2.10.5) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 10.7.0 from version 10.6.1. For changes in other packages, see the preinstalled packages changelog.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.12 (default)
• composer-3-airflow-2.9.3-build.32

New images are available in Cloud Composer 2:

• composer-2.14.0-airflow-2.10.5 (default)
• composer-2.14.0-airflow-2.9.3

(Cloud Composer 2) Moved the update_fab_perms option from [webserver] to [fab] in the Airflow configuration. The corresponding deprecation warning is no longer generated in Airflow web server logs.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.11 (default)
• composer-3-airflow-2.9.3-build.31

New images are available in Cloud Composer 2:

• composer-2.13.9-airflow-2.10.5 (default)
• composer-2.13.9-airflow-2.9.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.10 (default)
• composer-3-airflow-2.9.3-build.30

New images are available in Cloud Composer 2:

• composer-2.13.8-airflow-2.10.5 (default)
• composer-2.13.8-airflow-2.9.3

We're changing the way we provide support dates for Airflow builds in Cloud Composer 3. Before this change, some Airflow builds had their end of support date listed as "To be announced" until a later Airflow version became available. We're deprecating this approach for all builds that are released after July 01, 2025.

We are now providing support dates that depend on a date when a particular Airflow build was released:

• We are introducing the standard support period of 12 months after the release date.
• All Airflow builds that didn't have an end of support date when this change was introduced are supported until July 1, 2026. Because of this change, some Airflow builds released before July 1, 2025 are supported for longer than the standard support period.
• All Airflow builds that had their support date shorter than 12 months are now supported for 12 months since the date of their initial release.
• All builds released after July 1, 2025 will use the standard support period.

We are gradually rolling out a change that switches the default version from Cloud Composer 2 to Cloud Composer 3 in the Cloud Composer API.

In regions where the change is rolled out, a Cloud Composer 3 environment is created by default when a version is not specified in Google Cloud CLI, Cloud Composer API, or Terraform. If you use automation scripts to provision Cloud Composer 2 environments, make sure that you explicitly specify a Cloud Composer 2 version.

In this release, the change is rolling out in the following regions: africa-south1, asia-northeast2, asia-south2, asia-southeast2, europe-southwest1, europe-west10, europe-west12, europe-west8, me-central1, me-central2, me-west1, southamerica-west1, and us-south1.

(Available without upgrading) During Cloud Composer 2 environment operations, a more informative error message is returned when an environment's web server has connectivity issues.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.9 (default)
• composer-3-airflow-2.9.3-build.29

New images are available in Cloud Composer 2:

• composer-2.13.7-airflow-2.10.5 (default)
• composer-2.13.7-airflow-2.9.3

Cloud Composer 1 and Cloud Composer 2 environments with version 2.0.x are approaching their end of life. We're planning to deprecate them in the following way:

• Starting September 15, 2025, you will no longer be able to create new Cloud Composer 1 environments.
• On September 15, 2026, all Cloud Composer 1 and Cloud Composer 2 version 2.0.x environments will reach their planned end of life, and you won't be able to use them.

Cloud Composer 2 environments with versions later than 2.1.0 and all Cloud Composer 3 environments are not affected by this deprecation.

We recommend planning migration to Cloud Composer 3 or upgrading your Cloud Composer 2 environments to a later version.

This release includes internal infrastructure improvements to Cloud Composer. There are no user-visible changes.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.8 (default)
• composer-3-airflow-2.9.3-build.28

New images are available in Cloud Composer 2:

• composer-2.13.6-airflow-2.10.5 (default)
• composer-2.13.6-airflow-2.9.3

(Airflow 2.10.5) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 10.5.0 from version 10.4.2. For changes in other packages, see the preinstalled packages changelog.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.7 (default)
• composer-3-airflow-2.9.3-build.27

New images are available in Cloud Composer 2:

• composer-2.13.5-airflow-2.10.5 (default)
• composer-2.13.5-airflow-2.9.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.6 (default)
• composer-3-airflow-2.9.3-build.26

New images are available in Cloud Composer 2:

• composer-2.13.4-airflow-2.10.5 (default)
• composer-2.13.4-airflow-2.9.3

(Cloud Composer 3) If a VPC network is attached to an environment, then all DNS addresses accessed by the Airflow components of the environment are resolved using the Cloud DNS configuration of the VPC network. In particular, Airflow workers that execute DAGs will resolve DNS addresses in this way.

(Cloud Composer 3) All newly created private DNS zones are immediately visible to a Cloud Composer environment. Previously, re-attaching a VPC network was required.

(Cloud Composer 2) In Cloud Composer versions 2.11.5 and later, log processing is switching to using OpenTelemetry instead of Fluentd.

This change was announced previously and is gradually rolling out over several releases. In this release, it's available in the following regions: asia-east1, asia-east2, asia-northeast2, asia-northeast3, asia-south2, asia-southeast2, australia-southeast1, australia-southeast2, europe-central2, europe-north1, europe-north2, europe-southwest1, europe-west, europe-west10, europe-west12, europe-west6, europe-west8, europe-west9, me-central1, me-central2, me-west1, northamerica-northeast2, northamerica-south1, southamerica-east1, southamerica-west1, us-east5, us-south1, us-west, and us-west3.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.5 (default)
• composer-3-airflow-2.9.3-build.25

New images are available in Cloud Composer 2:

• composer-2.13.3-airflow-2.10.5 (default)
• composer-2.13.3-airflow-2.9.3

(Airflow 2.10.5 and 2.9.3) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 10.4.2 from version 10.1.0.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.4 (default)
• composer-3-airflow-2.9.3-build.24

New images are available in Cloud Composer 2:

• composer-2.13.2-airflow-2.10.5 (default)
• composer-2.13.2-airflow-2.9.3

The default value of the [scheduler]max_tis_per_query Airflow configuration option is set to 128. This change fixes the issue with DAGs remaining in the scheduled state in some cases.

Cloud Composer 3 and Cloud Composer 2 no longer support files with gzip encoding (content-type=gzip).

Before this change, all files that had gzip encoding, including .py and .zip files, were processed inconsistently by the DAG processor. After this change, the DAG processor skips all files with gzip encoding. This change doesn't apply to regular .zip archives. Airflow still decompresses and processes DAG files contained in archives that have no specified encoding.

This change is gradually rolled out to all regions supported by Cloud Composer, except us-central1, us-east4 and europe-west1. It will be rolled out to the remaining regions in one of the future releases.

(Cloud Composer 3) Improved the computation of metrics related to snapshots. This change reduces the number of calls made by Cloud Composer to the Cloud Storage API.

(Cloud Composer 2) In Cloud Composer versions 2.11.5 and later, log processing is switching to using OpenTelemetry instead of Fluentd.

This change is gradually rolled out to the asia-south1 and africa-south1 regions. It will be rolled out to more regions in future releases. New and upgraded Cloud Composer 2 environments in the listed regions will get this change.

(Airflow 2.9.3) The apache-airflow-providers-google package was upgraded to version 15.1.0 in Cloud Composer 2 images and Cloud Composer 3 builds.

For more information about changes, see the apache-airflow-providers-google changelog from version 14.0.0 to version 15.1.0.

(Airflow 2.9.3) Changes in preinstalled packages:

• apache-airflow-providers-standard was upgraded to 1.0.0 from 0.4.0.
• aiosqlite was removed from preinstalled packages.
• json-merge-patch was removed from preinstalled packages.
• time-machine was removed from preinstalled packages.
• pyjwt was downgraded to 2.9.0 from 2.10.1.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.3 (default)
• composer-3-airflow-2.9.3-build.23

New images are available in Cloud Composer 2:

• composer-2.13.1-airflow-2.10.5 (default)
• composer-2.13.1-airflow-2.9.3

The following Cloud Composer versions have an extended upgrade timeline:

• composer-3-airflow-2.10.5-build.3
• composer-3-airflow-2.9.3-build.23
• composer-2.13.1-airflow-2.10.5
• composer-2.13.1-airflow-2.9.3

For newly created Cloud Composer 3 environments, the minimum amount of memory is changed to 2 GB.

For newly created environments, database retention policy is now enabled by default in Google Cloud CLI, API, and Terraform. Before this change, it was enabled by default only in Google Cloud Console.

Improved the environment liveness monitoring. This change addresses some cases of transient failures that caused "Liveness probe failed" warnings in the environment's logs.

(Airflow 2.10.5) The apache-airflow-providers-google package was upgraded to version 15.1.0 in Cloud Composer 2 images and Cloud Composer 3 builds.

For more information about changes, see the apache-airflow-providers-google changelog from version 14.0.0 to version 15.1.0.

(Airflow 2.10.5) Changes in preinstalled packages:

• apache-airflow-providers-standard was upgraded to 1.0.0 from 0.4.0.
• aiosqlite was removed from preinstalled packages.
• json-merge-patch was removed from preinstalled packages.
• time-machine was removed from preinstalled packages.

The default version of Airflow is changed to 2.10.5.

Airflow 2.10.2 is no longer included in Cloud Composer images and builds.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.2 (default)
• composer-3-airflow-2.9.3-build.22

New images are available in Cloud Composer 2:

• composer-2.13.0-airflow-2.10.5 (default)
• composer-2.13.0-airflow-2.9.3

The Deployment Manager API is no longer automatically enabled when you enable Cloud Composer API because this API isn't used by the Cloud Composer service.

Environments with Cloud Composer versions 2.0.* still rely on the Deployment Manager API for updates, upgrades, and environment deletion. It won't be possible to perform these operations if this API is disabled. We recommend to upgrade your 2.0.* environments to a later version to remove this dependency.

Starting from June 2025, the default version for new Cloud Composer environments changes from Cloud Composer 2 to Cloud Composer 3. New environments will use the latest default Airflow build (composer-3-airflow-2). Currently, the default version is composer-2-airflow-2.

Database retention policy is now enabled by default in Google Cloud console and remains disabled in Google Cloud CLI, API, and Terraform.

This feature helps to maintain the Airflow database size. You can enable or disable the database retention policy or adjust the retention period for new and existing environments.

The default environment's service account setting is gradually removed in Cloud Composer. After the change, you'll need to explicitly specify a service account when you create a new Cloud Composer environment. For more information about addressing the change, see the eariler announcement of this change.

In this release, the change is rolling out to the following regions: africa-south1, asia-northeast2, asia-south2, australia-southeast2, europe-north2, europe-southwest1, europe-west8, europe-west10, europe-west12, me-central1, me-central2, me-west1, northamerica-northeast2, northamerica-south1, southamerica-west1, us-east7, and us-south1. It will be rolled out to more regions in future releases.

Cloud Composer 2 environments now always use the environment's service account for performing PyPI packages installations:

• Existing Cloud Composer 2 environments that previously used the default Cloud Build service account now use the environment's service account instead.
• Cloud Composer 2 environments created in versions 2.10.2 and later already have this change.
• Cloud Composer 3 environments already use the environment's service account, and are not affected by this change.
• This change is gradually rolled out to all regions supported by Cloud Composer 2.

Cloud Composer now detects situations when asynchronous tasks are blocked in Airflow triggerers. If a trigger's execution is blocked for more than five minutes, Cloud Composer restarts the triggerer, which solves this transient issue.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.5-build.0
• composer-3-airflow-2.10.2-build.13 (default)
• composer-3-airflow-2.9.3-build.20

New images are available in Cloud Composer 2:

• composer-2.12.1-airflow-2.10.5
• composer-2.12.1-airflow-2.10.2 (default)
• composer-2.12.1-airflow-2.9.3

Support dates for previous Cloud Composer 3 builds are available. All Cloud Composer 3 builds with Airflow 2.10.2 are supported until April 17, 2026.

(Airflow 2.10.2 and 2.9.3) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 10.3.0 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-cncf-kubernetes changelog from version 10.1.0 to version 10.3.0.

(Airflow 2.10.2 and 2.9.3) Changes in preinstalled packages:

• apache-airflow-providers-postgres was upgraded to 6.1.0 from 5.14.0.
• apache-airflow-providers-smtp was upgraded to 2.0.0 from 1.9.0.
• types-requests was removed from preinstalled packages.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.12 (default)
• composer-3-airflow-2.9.3-build.19

New images are available in Cloud Composer 2:

• composer-2.12.0-airflow-2.10.2 (default)
• composer-2.12.0-airflow-2.9.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.11 (default)
• composer-3-airflow-2.9.3-build.18

New images are available in Cloud Composer 2:

• composer-2.11.5-airflow-2.10.2 (default)
• composer-2.11.5-airflow-2.9.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.10 (default)
• composer-3-airflow-2.9.3-build.17

New images are available in Cloud Composer 2:

• composer-2.11.4-airflow-2.10.2 (default)
• composer-2.11.4-airflow-2.9.3

Improved the forbidden APIs check. When a new environment is created, Cloud Composer now checks for forbidden APIs that are required by a specific major version of Cloud Composer.

(Airflow 2.10.2 and 2.9.3) Preinstalled packages were changed:

• virtualenv was downgraded from 20.29.1 to 20.28.1

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.9 (default)
• composer-3-airflow-2.9.3-build.16

New images are available in Cloud Composer 2:

• composer-2.11.3-airflow-2.10.2 (default)
• composer-2.11.3-airflow-2.9.3

(Cloud Composer 2) The list of permissions reported by dependency_permissions_check_count metric was updated. Now the metric reports more permissions that are required for a functioning environment. Permissions that weren't necessary were replaced or removed.

(New Cloud Composer environments only) Cloud Composer now enforces SSL connections to Cloud SQL instances.

This change implements a security compliance standard in Cloud Composer and enforces SSL usage. You can adopt other security standards by following recommendations listed in Vulnerability findings. Previously, it was possible to create a non-SSL connection to the Airflow database. After the change, the connection will be refused.

(Airflow 2.10.2 and 2.9.3) Preinstalled packages were changed:

• logbook was removed from preinstalled packages
• minimal-snowplow-tracker was removed from preinstalled packages
• mashumaro was downgraded from 3.15 to 3.14

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.8 (default)
• composer-3-airflow-2.9.3-build.15

New images are available in Cloud Composer 2:

• composer-2.11.2-airflow-2.10.2 (default)
• composer-2.11.2-airflow-2.9.3

Cloud Composer 2 is no longer available in Mexico (northamerica-south1). It is not possible to select this region when creating a new Cloud Composer 2 environment. Existing Cloud Composer 2 environments and both new and existing Cloud Composer 3 environments are not affected by this change.

(Available without upgrading) Starting from Cloud Composer version 2.11.0 with Airflow 2.10.2 and 2.9.3, the enable_task_context_logger Airflow configuration option is no longer blocked.

(Available without upgrading) In Cloud Composer 2 and Cloud Composer 1, a more meaningful error message is produced when the Composer Service Agent account doesn't have necessary permissions to access the network during the environment creation.

(Cloud Composer 3) The zone in the Environment.config.node_config.location field is no longer populated in the Cloud Composer API.

(Airflow 2.10.2 and 2.9.3) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 10.1.0 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-cncf-kubernetes changelog from version 10.0.1 to version 10.1.0.

(Airflow 2.10.2 and 2.9.3) The following preinstalled provider packages were upgraded to new major versions:

• apache-airflow-providers-apache-beam upgraded from 5.9.1 to 6.0.0
• apache-airflow-providers-dbt-cloud upgraded from 3.11.2 to 4.0.0
• apache-airflow-providers-hashicorp upgraded from 3.8.0 to 4.0.0
• apache-airflow-providers-http upgraded from 4.13.3 to 5.0.0
• apache-airflow-providers-mysql upgraded from 5.7.4 to 6.0.0
• apache-airflow-providers-sendgrid upgraded from 3.6.0 to 4.0.0
• apache-airflow-providers-sqlite upgraded from 3.9.1 to 4.0.0
• apache-airflow-providers-ssh upgraded from 3.14.0 to 4.0.0

(Airflow 2.10.2 and 2.9.3) The following preinstalled packages were downgraded:

• certifi downgraded from 2024.12.14 to 2024.8.30.
• charset-normalizer downgraded from 3.4.1 to 3.4.0.
• cryptography downgraded from 44.0.0 to 43.0.3.
• filelock downgraded from 3.16.1 to 3.15.4.
• google-auth downgraded from 2.37.0 to 2.35.0.
• keyring downgraded from 25.6.0 to 25.5.0.
• urllib3 downgraded from 2.3.0 to 2.2.3.
• zipp downgraded from 3.21.0 to 3.20.2.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.7 (default)
• composer-3-airflow-2.9.3-build.14

New images are available in Cloud Composer 2:

• composer-2.11.1-airflow-2.10.2 (default)
• composer-2.11.1-airflow-2.9.3

(New Cloud Composer 2 environments only) Cloud Composer 2 environments in versions 2.10.2 and later always use the environment's service account for performing PyPI packages installations. This change applies only to newly created environments, existing environments that are upgraded to 2.10.2 and later versions will not get this change.

(Cloud Composer 3 only) The /data folder is now synchronized with Airflow triggerers.

(Available without upgrading) Improved the error message generated when the Cloud Composer Service Agent service account is missing permissions on the project or on the environment's service account.

The Redis persistent disk is now automatically deleted together with the environment. This persistent disk is used by the Redis queue and stores only technical data.

The worker_autoscale Airflow configuration option is blocked in Cloud Composer 2. Previously, it was blocked only in Cloud Composer 3.

The importlib-resources package was removed from preinstalled packages.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.5 (default)
• composer-3-airflow-2.9.3-build.12

New images are available in Cloud Composer 2:

• composer-2.10.2-airflow-2.10.2 (default)
• composer-2.10.2-airflow-2.9.3

Cloud Composer 2.10.1 is a version with an extended upgrade timeline.

The COMPOSER_AGENT_BUILD_SERVICE_ACCOUNT environment variable is changed to reserved. This change improves the security of Cloud Composer environments.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.10.2-build.4 (default)
• composer-3-airflow-2.9.3-build.11

(Airflow 2.10.2 and 2.9.3) The apache-airflow-providers-google package was upgraded to version 10.26.0 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-google changelog from version 10.25.0 to version 10.26.0.

(Airflow 2.10.2 and 2.9.3) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 10.0.1 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-cncf-kubernetes changelog from version 9.0.1 to version 10.0.1.

(Available without upgrading) Fixed an issue where Airflow workers sometimes generated incomplete or unreadable output.

The default version of Airflow is changed to 2.10.2.

New images are available in Cloud Composer 2:

• composer-2.10.0-airflow-2.10.2 (default)
• composer-2.10.0-airflow-2.9.3

(Airflow 2.10.2 and 2.9.3) The apache-airflow-providers-google package was upgraded to version 10.25.0 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-google changelog from version 10.24.0 to version 10.25.0.

(Airflow 2.9.3 and 2.7.3) The docutils package was removed from preinstalled packages.

New images are available in Cloud Composer 2:

• composer-2.9.10-airflow-2.9.3 (default)
• composer-2.9.10-airflow-2.7.3

Cloud Composer 2.9.7 is a version with an extended upgrade timeline.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.3-build.5 (default)
• composer-3-airflow-2.7.3-build.21

(Airflow 2.9.3) The apache-airflow-providers-google package was upgraded to version 10.24.0 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-google changelog from version 10.23.0 to version 10.24.0.

(Airflow 2.9.3) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 9.0.0 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-cncf-kubernetes changelog from version 8.4.2 to version 9.0.0.

(Airflow 2.9.3 and 2.7.3) Changes in preinstalled packages:

• The grpcio package was downgraded from 1.66.2 to 1.65.5.
• The js2py package was removed from dependencies.
• The pyjsparser package was removed from dependencies.
• The tzlocal package was removed from dependencies.

Cloud Composer 2.9.8 images are available:

• composer-2.9.8-airflow-2.9.3 (default)
• composer-2.9.8-airflow-2.7.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.3-build.3 (default)
• composer-3-airflow-2.7.3-build.19

(Airflow 2.9.3 and 2.9.1) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 8.4.2 in Cloud Composer 2 images and Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-cncf-kubernetes changelog from version 8.4.1 to version 8.4.2.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.3-build.0
• composer-3-airflow-2.9.1-build.7 (default)
• composer-3-airflow-2.7.3-build.16

Cloud Composer 2.9.4 images are available:

• composer-2.9.4-airflow-2.9.3
• composer-2.9.4-airflow-2.9.1 (default)
• composer-2.9.4-airflow-2.7.3

Support dates for previous Cloud Composer 3 builds are available. All Cloud Composer 3 builds with Airflow 2.9.1 are supported until September 18, 2025.

The apache-airflow-providers-cncf-kubernetes package was upgraded to version 8.4.1 in Cloud Composer 2 images and Cloud Composer 3 builds with Airflow 2.9.1. For more information about changes, see the apache-airflow-providers-cncf-kubernetes changelog from version 8.3.4 to version 8.4.1.

The apache-airflow-providers-google package was upgraded to version 10.22.0 in Cloud Composer 2 images and Cloud Composer 3 builds with Airflow 2.9.1. For more information about changes, see the apache-airflow-providers-google changelog from version 10.21.0 to version 10.22.0.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.1-build.6 (default)
• composer-3-airflow-2.7.3-build.15

Cloud Composer 2.9.3 images are available:

• composer-2.9.3-airflow-2.9.1 (default)
• composer-2.9.3-airflow-2.7.3

The apache-airflow-providers-google package was upgraded to version 10.21.1 in Cloud Composer 2 images with Airflow 2.7.3 and 2.9.1, and in all latest Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-google changelog from version 10.21.0 to version 10.21.1.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.1-build.5
• composer-3-airflow-2.7.3-build.14

Cloud Composer 2.9.2 images are available:

• composer-2.9.2-airflow-2.9.1 (default)
• composer-2.9.2-airflow-2.7.3

(Airflow 2.7.3) Improved the reliability of connections between Airflow components and the Redis component.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.1-build.4
• composer-3-airflow-2.7.3-build.13

Cloud Composer 2.9.1 images are available:

• composer-2.9.1-airflow-2.9.1 (default)
• composer-2.9.1-airflow-2.7.3

(Airflow 2.9.1) Improved the reliability of connections between Airflow components and the Redis component.

The default version of Airflow is changed to 2.9.1.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.1-build.3
• composer-3-airflow-2.7.3-build.12

Cloud Composer 2.9.0 images are available:

• composer-2.9.0-airflow-2.9.1 (default)
• composer-2.9.0-airflow-2.7.3

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.1-build.2
• composer-3-airflow-2.7.3-build.11

Cloud Composer 2.8.8 images are available:

• composer-2.8.8-airflow-2.9.1
• composer-2.8.8-airflow-2.7.3 (default)

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.1-build.1
• composer-3-airflow-2.7.3-build.10

Cloud Composer 2.8.7 images are available:

• composer-2.8.7-airflow-2.9.1
• composer-2.8.7-airflow-2.7.3 (default)
• composer-2.8.7-airflow-2.6.3

Information about excluded Cloud Storage objects in the environment's bucket is no longer logged. This change reduces the usage of the Storage API during the synchronization of DAG files, which improves the performance of Airflow components and results in fewer Airflow component restarts. The change will gradually become available in all Cloud Composer environments.

(Airflow 2.9.1) Task context logging is disabled, and it is not possible to enable it.

(Airflow 2.9.1) Raw HTML code in DAG docs and DAG parameter descriptions is disabled by default.

(Airflow 2.9.1) Audit log permissions are revoked from all roles except Admin.

The apache-airflow-providers-google package was upgraded to version 10.21.0 in Cloud Composer 2 images with Airflow 2.7.3 and 2.9.1, and in all latest Cloud Composer 3 builds. For more information about changes, see the apache-airflow-providers-google changelog from version 10.18.0 to version 10.21.0.

The upgraded apache-airflow-providers-google package provides new operators and sensors for triggering DAGs across Cloud Composer environments:

• CloudComposerRunAirflowCLICommandOperator makes a call to the Airflow REST API of a Cloud Composer environment.
• CloudComposerDAGRunSensor checks if a DAG Run is completed.

When installing PyPI packages, if you want your builds to run with a custom service account, you can override the COMPOSER_AGENT_BUILD_SERVICE_ACCOUNT environment variable with the chosen service account. For more information, see Install Python dependencies.

New Airflow builds are available in Cloud Composer 3:

• composer-3-airflow-2.9.1-build.0
• composer-3-airflow-2.7.3-build.9

Cloud Composer 2.8.6 images are available:

• composer-2.8.6-airflow-2.9.1
• composer-2.8.6-airflow-2.7.3 (default)
• composer-2.8.6-airflow-2.6.3

Support dates for previous Cloud Composer 3 builds are available. All Cloud Composer 3 builds with Airflow 2.7.3 are supported until July 17, 2025.

A new Airflow build is available in Cloud Composer 3:

• composer-3-airflow-2.7.3-build.8

Cloud Composer 2.8.5 images are available:

• composer-2.8.5-airflow-2.7.3 (default)
• composer-2.8.5-airflow-2.6.3

New Cloud Composer 2 environments are gradually switched to using GKE 1.29 and PSC as a connectivity channel to the GKE control plane. The IP address for the PSC endpoint will be taken from the nodes IP range. This change might require using a larger IP range for the nodes when you create an environment.

A new Airflow build is available in Cloud Composer 3:

• composer-3-airflow-2.7.3-build.7

Cloud Composer 2.8.4 images are available:

• composer-2.8.4-airflow-2.7.3 (default)
• composer-2.8.4-airflow-2.6.3

(Airflow 2.7.3) The apache-airflow-providers-google package was upgraded to version 10.18.0. For more information about changes, see the apache-airflow-providers-google changelog from version 10.17.0 to version 10.18.0.

(Airflow 2.7.3) The apache-airflow-providers-cncf-kubernetes package was upgraded to version 8.3.1.

(Airflow 2.7.3) The apache-beam package was upgraded to version 2.56.0.

A new Airflow build is available in Cloud Composer 3:

• composer-3-airflow-2.7.3-build.6

Cloud Composer 2.8.3 images are available:

• composer-2.8.3-airflow-2.7.3 (default)
• composer-2.8.3-airflow-2.6.3

Cloud Composer 2.8.3 is a version with an extended upgrade timeline.

The google-cloud-bigquery package version was downgraded from 3.23.1 to 3.20.1 because of the #39541 issue in the public version of Airflow.

The dbt-core and dbt-bigquery packages were upgraded to version 1.8.1.

Cloud Composer 2.8.2 images are available:

• composer-2.8.2-airflow-2.7.3 (default)
• composer-2.8.2-airflow-2.6.3

Cloud Composer 2.8.1 images are available:

• composer-2.8.1-airflow-2.7.3 (default)
• composer-2.8.1-airflow-2.6.3

The Logs in Cloud Logging Only feature is enabled by default in new environments:

• New Cloud Composer environments now save Airflow task logs only in Cloud Logging by default.
• Existing environments are not changed. If you upgrade an existing environment to Cloud Composer 2.8.0, it keeps saving logs to the environment's bucket.
• You can enable and disable saving logs to the environment's bucket for an existing environment.

Cloud Composer 2.8.0 images are available:

• composer-2.8.0-airflow-2.7.3 (default)
• composer-2.8.0-airflow-2.6.3

The apache-airflow-providers-google package is upgraded to version 10.17.0. For more information about changes, see the apache-airflow-providers-google changelog from version 10.16.0 to version 10.17.0.

The apache-airflow-providers-cncf-kubernetes package was upgraded to version 8.1.0.

Cloud Composer 2.7.1 images are available:

• composer-2.7.1-airflow-2.7.3 (default)
• composer-2.7.1-airflow-2.6.3

(New environments only) Increased the default value of the [webserver]auto_refresh_interval Airflow configuration option to 15 seconds. Pages in the Airflow UI, such as the list of DAGs, now will update every 15 seconds.

The default version of Airflow is changed to 2.7.3.

Cloud Composer 2.7.0 images are available:

• composer-2.7.0-airflow-2.7.3 (default)
• composer-2.7.0-airflow-2.6.3

The default values of the following Airflow configuration options are changed to provide more optimized Cloud Composer environments. This change applies to new environments with Airflow 2.6.3 and 2.7.3 and to existing environments that do not use custom values for these parameters. For existing environments, the new default values will be used after an upgrade.

• [scheduler]job_heartbeat_sec to 30
• [scheduler]scheduler_health_check_threshold to 60
• [scheduler]scheduler_heartbeat_sec to 15

If you want to override the [scheduler]scheduler_heartbeat_sec option's value, then also adjust the [scheduler]scheduler_health_check_threshold option, as described in Cloud Composer documentation.

Cloud Composer 2.6.6 images are available:

• composer-2.6.6-airflow-2.7.3
• composer-2.6.6-airflow-2.6.3 (default)
• composer-2.6.6-airflow-2.5.3

The apache-airflow-providers-google package is upgraded to version 10.16.0 in images with Airflow 2.6.3, and images with Airflow 2.7.3 have this version. For more information about changes, see the apache-airflow-providers-google changelog from version 10.15.0 to version 10.16.0.

The apache-airflow-providers-cncf-kubernetes package was upgraded to version 8.0.1 in images with Airflow 2.6.3, and images with Airflow 2.7.3 have this version.

Cloud Composer 2.6.5 images are available:

• composer-2.6.5-airflow-2.7.3
• composer-2.6.5-airflow-2.6.3 (default)
• composer-2.6.5-airflow-2.5.3

Cloud Composer 2.6.5 is a version with an extended upgrade timeline.

Cloud Composer 2.6.4 images are available:

• composer-2.6.4-airflow-2.6.3 (default)
• composer-2.6.4-airflow-2.5.3

(Cloud Composer 2 in select regions) Cloud Composer shows the account selection and consent screens when opening Airflow UI for an environment.

If you have the "Don't allow users to access any third-party apps" setting enabled in Google Workspace, then configure access to the "Apache Airflow in Cloud Composer" app in Google Workspace, so that your Google Workspace users can still access Airflow UI in Cloud Composer environments. For more information, see Allow access to Airflow UI in Google Workspace.

(Cloud Composer 2 in select regions) Reduced the propagation time of the revoked Cloud IAM permission that blocks access to Airflow UI.

In new environments with Airflow 2.6.3, the default values of the following Airflow configuration options are changed to provide more optimized Cloud Composer environments:

• [scheduler]job_heartbeat_sec to 30
• [scheduler]scheduler_heartbeat_sec to 15

The apache-airflow-providers-google package is upgraded to version 10.15.0 in images with Airflow 2.6.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.14.0 to version 10.15.0.

Cloud Composer 2.6.3 images are available:

• composer-2.6.3-airflow-2.6.3 (default)
• composer-2.6.3-airflow-2.5.3

Starting February 27, 2024, in the us-central1, europe-west1, europe-west2, europe-west3, europe-west6, us-east1, and us-east4 regions it is possible to create new Cloud Composer 1 environments only in projects that already have Cloud Composer 1 environments.

In all other existing or newly created projects in these regions, it is possible to create only Cloud Composer 2 environments. This change is a part of the preparation for Cloud Composer 1 end of support, as communicated earlier and described in the Versioning overview.

Cloud Composer 2.6.2 images are available:

• composer-2.6.2-airflow-2.6.3 (default)
• composer-2.6.2-airflow-2.5.3

The apache-airflow-providers-google package is upgraded to version 10.14.0 in images with Airflow 2.6.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.13.1 to version 10.14.0.

Cloud Composer 2.6.1 images are available:

• composer-2.6.1-airflow-2.6.3 (default)
• composer-2.6.1-airflow-2.5.3

Data lineage integration is now enabled by default in newly created environments with Cloud Composer version 2.1.2 and later, if Data Lineage API is enabled in the environment's project. Existing and upgraded environments keep their current configuration.

(Available without upgrading) The default worker_concurrency in Airflow 2.6.3 and later versions is now calculated using a different formula. For more information, see Change worker concurrency.

Connections to the Redis environment component are now additionally secured with a password. Improved the reliability of the environment component responsible for metrics reporting (airflow-monitoring).

The apache-airflow-providers-google package is upgraded to version 10.13.1 in images with Airflow 2.6.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.12.0 to version 10.13.1.

Cloud Composer 2.6.0 images are available:

• composer-2.6.0-airflow-2.6.3 (default)
• composer-2.6.0-airflow-2.5.3

Cloud Composer 2.6.0 is a version with an extended upgrade timeline.

Sensitive Airflow configuration options are now hidden in Airflow UI by default. The default value of the [webserver]expose_config option is now set to non-sensitive-only.

Cloud Composer 2.5.4 images are available:

• composer-2.5.4-airflow-2.5.3
• composer-2.5.4-airflow-2.6.3 (default)

The apache-airflow-providers-google package is upgraded to version 10.12.0 in images with Airflow 2.6.3 and 2.5.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.11.1 to version 10.12.0.

Cloud Composer 2.5.3 images are available:

• composer-2.5.3-airflow-2.5.3
• composer-2.5.3-airflow-2.6.3 (default)

The apache-airflow-providers-google package is upgraded to version 10.11.1 in images with Airflow 2.6.3 and 2.5.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.10.1 to version 10.11.1.

The apache-airflow-providers-cncf-kubernetes package was upgraded to version 7.9.0.

Cloud Composer 2.5.2 images are available:

• composer-2.5.2-airflow-2.5.3
• composer-2.5.2-airflow-2.6.3 (default)

Quotas for snapshot operations are increased. Now it's possible to create up to 52 snapshots daily for a single environment. For scheduled snapshots taken at regular intervals, the minimum recommended interval is now 30 minutes (previously it was two hours).

The apache-airflow-providers-google package is upgraded to version 10.10.1 in images with Airflow 2.6.3 and 2.5.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.10.0 to version 10.10.1.

The apache-airflow-providers-cncf-kubernetes package was upgraded to version 7.6.0.

Cloud Composer 2.5.1 images are available:

• composer-2.5.1-airflow-2.5.3
• composer-2.5.1-airflow-2.6.3 (default)

Directories with names ending in .py are no longer synchronized. If your environment's bucket contains such directories, please rename them.

The apache-airflow-providers-google package is upgraded to version 10.10.0 in images with Airflow 2.6.3 and 2.5.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.9.0 to version 10.10.0.

The default version of Airflow is changed to 2.6.3.

Cloud Composer 2.5.0 images are available:

• composer-2.5.0-airflow-2.5.3
• composer-2.5.0-airflow-2.6.3 (default)

Airflow 2.6.3 consolidates the logic for handling tasks that are stuck in the queued state:

• The [kubernetes]worker_pods_pending_timeout, [celery]stalled_task_timeout, and [celery]task_adoption_timeout Airflow configuration options are deprecated and merged into the [scheduler]task_queued_timeout option.
• In Cloud Composer, the default value of the [scheduler]task_queued_timeout option is set to 40 minutes.
• If your environment uses a custom value for any of the deprecated Airflow configuration options, please clear the overrides before upgrading. If the values are not cleared, the longest timeout of all deprecated options is selected upon upgrading.
• If required, you can override the value of the [scheduler]task_queued_timeout option in your environment.
• For more information about other changes between Airflow versions 2.5.3 and 2.6.3, see Airflow release notes.

Cloud Composer 2.4.6 images are available:

• composer-2.4.6-airflow-2.6.3
• composer-2.4.6-airflow-2.5.3 (default)
• composer-2.4.6-airflow-2.4.3

Cloud Composer 2.4.5 images are available:

• composer-2.4.5-airflow-2.5.3 (default)
• composer-2.4.5-airflow-2.4.3

The cost of the environments.ExecuteAirflowCommand and environments.StopAirflowCommand operations is reduced from 100 to 25 quota units.

The apache-airflow-providers-google package is upgraded to version 10.9.0 in images with Airflow 2.5.3 and 2.4.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.7.0 to version 10.9.0.

Cloud Composer 2.4.4 images are available:

• composer-2.4.4-airflow-2.5.3 (default)
• composer-2.4.4-airflow-2.4.3

The apache-airflow-providers-google package is upgraded to version 10.7.0 in images with Airflow 2.5.3 and 2.4.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.5.0 to version 10.7.0.

Version 10.7.0 includes new Cloud Run, Cloud Batch and Dataplex Data Quality operators:

• Cloud Run is used to build and deploy scalable containerized apps written in any language (including Go, Python, Java, Node.js, .NET, and Ruby) on a fully managed platform. The newly provided Cloud Run Operators enable creating, executing, updating, deleting and listing Cloud Run Jobs.
• Cloud Batch is a fully managed batch service to schedule, queue, and execute batch jobs on Google's infrastructure. The provided Cloud Batch Operators enable submitting, listing and deleting batch jobs as well as listing a job's tasks.
• Dataplex is an intelligent data fabric that provides unified analytics and data management across your data lakes, data warehouses, and data marts. New Dataplex operators handle creating, updating, getting, deleting and running a Data Quality scan, getting a Data Quality Scan job, creating and deleting a zone, as well as creating and deleting an asset.

Cloud Composer 2.4.3 images are available:

• composer-2.4.3-airflow-2.5.3 (default)
• composer-2.4.3-airflow-2.4.3

Cloud Composer now generates a FAILED_PRECONDITION error message when an environment operation fails because the environment's cluster is missing.

Cloud Composer 2.4.2 images are available:

• composer-2.4.2-airflow-2.5.3 (default)
• composer-2.4.2-airflow-2.4.3

(Cloud Composer 2) An update that enforces VPC Service Controls perimeter settings for accessing Airflow UI and Airflow REST API is rolled out to all regions. As a result of this change, certain users that could previously access the Airflow UI or Airflow REST API might not be able to do it now.

If you experience problems with access to Airflow UI or Airflow REST API, update the perimeter or access level configuration for affected users. For more information on configuring VPC Service Controls see Configure VPC Service Controls.

The apache-airflow-providers-google package is upgraded to version 10.5.0 in images with Airflow 2.5.3 and 2.4.3. For more information about changes, see the apache-airflow-providers-google changelog from version 10.3.0 to version 10.5.0.

Cloud Composer 2.4.1 images are available:

• composer-2.4.1-airflow-2.5.3 (default)
• composer-2.4.1-airflow-2.4.3

The composer.environments.executeAirflowCommand permission is now required to run Airflow CLI commands through the gcloud environments run command:

• The composer.user and composer.environmentAndStorageObjectViewer roles do not have this permission and are not permitted to run Airflow CLI commands.

• This permission is added to the composer.admin and composer.environmentAndStorageObjectAdmin roles.

GCSFuse version updated to 1.01. Cloud Composer uses GCSFuse to sync files between the environment bucket and worker pods.

Cloud Composer 2.4.0 images are available:

• composer-2.4.0-airflow-2.5.3 (default)
• composer-2.4.0-airflow-2.4.3

The apache-airflow-providers-google package is upgraded to public version 10.3.0 in images with Airflow 2.5.3 and 2.4.3. Changes compared to version 2023.6.6+composer:

• Google Ads API default version updated to v14.
• scipy package is removed from dependencies.
• Several other small fixes and improvements.

For more information about changes, see the apache-airflow-providers-google changelog from version 10.1.1 to version 10.3.0.

(Cloud Composer 2) The default version of Airflow is changed to 2.5.3.

Cloud Composer 2.3.5 images are available:

• composer-2.3.5-airflow-2.5.3 (default)
• composer-2.3.5-airflow-2.4.3

This version is a version with an extended upgrade timeline.

(Available without upgrading) The default amount of memory available to the web server, schedulers, and workers is changed from 3.75 GB per CPU core to 4 GB per CPU core. This value is used only if you do not specify the amount of memory available to these environment components.

(Available without upgrading) Improved the performance of DAG UI in Private IP environments.

Cloud Composer 2.3.4 images are available:

• composer-2.3.4-airflow-2.5.3
• composer-2.3.4-airflow-2.5.1 (default)
• composer-2.3.4-airflow-2.4.3

Improved the error message when environment creation fails because of issues with GKE workloads.

Cloud Composer 2.3.3 images are available:

• composer-2.3.3-airflow-2.5.1 (default)
• composer-2.3.3-airflow-2.4.3

(Airflow 2.5.1) Changed the severity of triggerer watchdog messages from error to warning and updated the message's content to be more informative. This change improves debugging experience for triggers.

Cloud Composer 2.3.2 images are available:

• composer-2.3.2-airflow-2.5.1 (default)
• composer-2.3.2-airflow-2.4.3

Cloud Composer 2.3.1 images are available:

• composer-2.3.1-airflow-2.5.1 (default)
• composer-2.3.1-airflow-2.4.3

The apache-airflow-providers-google package in images with Airflow 2.5.1 and 2.4.3 was upgraded to version 2023.6.6+composer. This version is based on the public version 10.1.1, with additional fixes to some operators and upgrades to many SDK package dependencies (such as protobuf). Notable changes include:

• Google Ads default API changed from version 12 to 13.
• protobuf==4.22.5 is included, this is the first Cloud Composer version with protobuf version 4.x.
• In this version of the provider package, the deprecated delegate_to parameter is removed from all GCP operators, hooks, and triggers, as well as from Firestore and Gsuite transfer operators that interact with GCS. Impersonation can be achieved by utilizing the impersonation_chain parameter instead. The delegate_to param remains available only in Gsuite and marketing platform hooks and operators that don't interact with Google Cloud.

For a full list of changes in the apache-airflow-providers-google, see the changelog from version 8.9.0 to 10.1.1 on the apache-airflow-providers-google page.

The google-cloud-asset package is added to images with Airflow 2.5.1 and 2.4.3.

Cloud Composer 2.3.0 images are available:

• composer-2.3.0-airflow-2.5.1 (default)
• composer-2.3.0-airflow-2.4.3

(Cloud Composer 2) The deprecated [core]non_pooled_task_slot_count Airflow configuration option is replaced with the [core]default_pool_task_slot_count configuration option in the default Airflow configuration. Make sure to update your custom Airflow configuration overrides to use the new option instead of the deprecated one.

An improved error message is now displayed when a subnetwork with unsupported IPv4 ranges is used to create an environment in a shared VPC configuration.

Cloud Composer 2.2.1 images are available:

• composer-2.2.1-airflow-2.5.1 (default)
• composer-2.2.1-airflow-2.4.3

(Cloud Composer 2) Increased the default values of [core]dagbag_import_timeout to 120 seconds, and dag_file_processor_timeout to 300 seconds. The updated parameters allow the DAG Processor to parse more DAGs and provide more time to add parsed DAGs into the DAG bag.

(Cloud Composer 2) Increased the default value of [scheduler]zombie_detection_interval to 20 seconds. This change reduces the chance that Airflow skips retry attempts for a zombie task.

Cloud Composer 2.2.0 images are available:

• composer-2.2.0-airflow-2.5.1 (default)
• composer-2.2.0-airflow-2.4.3

(Cloud Composer 2) The default [celery]stalled_task_timeout value is set to 1200. Tasks that stay in the queued state for more than 20 minutes (1200 seconds) are now rescheduled.

(Airflow 2.4.3 and 2.5.1) Python packages upgraded:

• Added new packages: dbt-bigquery and google-cloud-documentai.
• The apache-airflow-providers-cncf-kubernetes package was upgraded to version 6.0.0.

(Cloud Composer 2) The default version of Airflow is changed to 2.5.1.

Cloud Composer 2.1.15 images are available:

• composer-2.1.15-airflow-2.5.1 (default)
• composer-2.1.15-airflow-2.4.3

Cloud Composer 2.1.14 and 1.20.12 images are available:

• composer-2.1.14-airflow-2.5.1
• composer-2.1.14-airflow-2.4.3 (default)
• composer-2.1.14-airflow-2.3.4
• composer-1.20.12-airflow-2.4.3
• composer-1.20.12-airflow-2.3.4
• composer-1.20.12-airflow-1.10.15

The apache-airflow-providers-google package in images with Airflow 2.3.4 and 2.4.3 was upgraded to 2023.4.13+composer. Changes compared to version 2023.3.14+composer:

• Update Google Display and Video 360 operators to use API v2.
• Update Google Campaign Manager operators to use API v4.
• Update google-cloud-dlp package to version 3.7.1 and adjust hooks and operators.

Cloud Composer 2.1.13 images are available:

• composer-2.1.13-airflow-2.4.3 (default)
• composer-2.1.13-airflow-2.3.4

GCSFuse version updated to 42.3. Cloud Composer uses GCSFuse to sync files between the environment bucket and worker pods.

Cloud Composer 2.1.12 images are available:

• composer-2.1.12-airflow-2.4.3 (default)
• composer-2.1.12-airflow-2.3.4

The default value for the dag_dir_list_interval Airflow configuration option is changed from 30 to 120 seconds.

Increased the timeout for environment operations performed by Cloud Build to 35 minutes.

Cloud Composer 2.1.11 and 1.20.11 images are available:

• composer-2.1.11-airflow-2.4.3 (default)
• composer-2.1.11-airflow-2.3.4
• composer-1.20.11-airflow-1.10.15
• composer-1.20.11-airflow-2.4.3
• composer-1.20.11-airflow-2.3.4

(Airflow 2.3.4 and 2.4.3 only) The Google Ads Python library (google-ads) package is updated to version 20.0.0.

(Airflow 2.3.4 and 2.4.3 only) The apache-airflow-providers-google package was upgraded to version 2023.3.14+composer. This version is based on the public version 8.9.0 with the following changes:

• Use google-ads==20.0.0

The source code for the apache-airflow-providers-google package version 2023.3.14+composer is available on GitHub.

(Composer 2 only) The instance_name field for Airflow Webservers in new environments is now by default set to the Composer Environment name.

Cloud Composer 1.20.1 and 2.1.11 are versions with an extended upgrade timeline.

Cloud Composer 2.1.10 and 1.20.10 images are available:

• composer-2.1.10-airflow-2.4.3 (default)
• composer-2.1.10-airflow-2.3.4
• composer-1.20.10-airflow-1.10.15
• composer-1.20.10-airflow-2.4.3
• composer-1.20.10-airflow-2.3.4

Cloud Composer 2.1.9 and 1.20.9 images are available:

• composer-2.1.9-airflow-2.4.3 (default)
• composer-2.1.9-airflow-2.3.4
• composer-2.1.9-airflow-2.2.5
• composer-1.20.9-airflow-1.10.15
• composer-1.20.9-airflow-2.4.3
• composer-1.20.9-airflow-2.3.4
• composer-1.20.9-airflow-2.2.5

Cloud Composer 2.1.8 and 1.20.8 images are available:

• composer-2.1.8-airflow-2.4.3 (default)
• composer-2.1.8-airflow-2.3.4
• composer-2.1.8-airflow-2.2.5
• composer-1.20.8-airflow-1.10.15
• composer-1.20.8-airflow-2.4.3
• composer-1.20.8-airflow-2.3.4
• composer-1.20.8-airflow-2.2.5

The default Composer version is changed to Cloud Composer 2.

The default Cloud Composer image used for operations is now the latest version of Cloud Composer 2, unless the image-version parameter is explicitly specified.

The composer-latest-airflow-* version aliases now point to Cloud Composer 2.

The apache-airflow-providers-google package is upgraded to the public version 8.9.0 in images with Airflow 2.4.3 and 2.3.4. For more information about changes, see the apache-airflow-providers-google page.

Cloud Composer 2.1.7 and 1.20.7 images are available:

• composer-2.1.7-airflow-2.4.3 (default)
• composer-2.1.7-airflow-2.3.4
• composer-2.1.7-airflow-2.2.5
• composer-1.20.7-airflow-1.10.15
• composer-1.20.7-airflow-2.4.3
• composer-1.20.7-airflow-2.3.4
• composer-1.20.7-airflow-2.2.5

(Cloud Composer 2) The default version of Airflow is changed to 2.4.3.

Cloud Composer 2.1.6 and 1.20.6 images are available:

• composer-2.1.6-airflow-2.4.3 (default)
• composer-2.1.6-airflow-2.3.4
• composer-2.1.6-airflow-2.2.5
• composer-1.20.6-airflow-1.10.15 (default)
• composer-1.20.6-airflow-2.4.3
• composer-1.20.6-airflow-2.3.4
• composer-1.20.6-airflow-2.2.5

Cloud Composer 1.20.5 and 2.1.5 images are available:

• composer-1.20.5-airflow-1.10.15 (default)
• composer-1.20.5-airflow-2.2.5
• composer-1.20.5-airflow-2.3.4
• composer-1.20.5-airflow-2.4.3
• composer-2.1.5-airflow-2.2.5
• composer-2.1.5-airflow-2.3.4 (default)
• composer-2.1.5-airflow-2.4.3

Images with Airflow 2.4.3 use the public version 8.6.0 of the apache-airflow-providers-google package. For more information about changes, see the package's page in the Airflow documentation.

(Airflow 2.2.5 only) The apache-airflow-providers-google package in images with Airflow 2.2.5 was upgraded to 2022.12.20+composer. Changes compared to version 2022.10.17+composer:

• Fix issue with the recursive error #25898 and #26175.
• Change log level to DEBUG for messages generated when a secret is not found in Secret Manager.
• Improve memory usage in Dataproc deferrable operators.

Cloud Composer 1.20.4 and 2.1.4 images are available:

• composer-1.20.4-airflow-1.10.15 (default)
• composer-1.20.4-airflow-2.2.5
• composer-1.20.4-airflow-2.3.4
• composer-1.20.4-airflow-2.4.3
• composer-2.1.4-airflow-2.2.5
• composer-2.1.4-airflow-2.3.4 (default)
• composer-2.1.4-airflow-2.4.3

Cloud Composer 1.20.3 and 2.1.3 images are available:

• composer-1.20.3-airflow-1.10.15 (default)
• composer-1.20.3-airflow-2.2.5
• composer-1.20.3-airflow-2.3.4
• composer-2.1.3-airflow-2.2.5
• composer-2.1.3-airflow-2.3.4 (default)

Cloud Composer 1.20.2 and 2.1.2 are versions with an extended upgrade timeline.

Cloud Composer 1.20.2 and 2.1.2 images are available:

• composer-1.20.2-airflow-1.10.15 (default)
• composer-1.20.2-airflow-2.2.5
• composer-1.20.2-airflow-2.3.4
• composer-2.1.2-airflow-2.2.5
• composer-2.1.2-airflow-2.3.4 (default)

(Available without upgrading) Allowed custom secondary IP range for pods is now narrower. You can now create Composer environments using IP ranges with /23 masks.

(New environments only) Airflow 2.3.4 is now the default Airflow version for Composer 2 environments.

(Cloud Composer 2) Starting with version 2.1.0, Composer environments no longer use Deployment Manager.

Images with Airflow 2.3.4 use the public version 8.4.0 of the apache-airflow-providers-google package. See the package page.

(Cloud Composer 2 with Private Service Connect) SQL proxy now gets restarted faster when connection issues are detected.

(Cloud Composer 2) You can now create Composer environments with labels that contain only digits.

Cloud Composer 1.19.15 and 2.0.32 images are available:

• composer-1.19.15-airflow-1.10.15 (default)
• composer-1.19.15-airflow-2.1.4
• composer-1.19.15-airflow-2.2.5
• composer-1.19.15-airflow-2.3.4
• composer-2.0.32-airflow-2.1.4
• composer-2.0.32-airflow-2.2.5
• composer-2.0.32-airflow-2.3.4

Cloud Composer 1.19.14 and 2.0.31 images are available:

• composer-1.19.14-airflow-1.10.15 (default)
• composer-1.19.14-airflow-2.1.4
• composer-1.19.14-airflow-2.2.5
• composer-1.19.14-airflow-2.3.3
• composer-2.0.31-airflow-2.1.4
• composer-2.0.31-airflow-2.2.5
• composer-2.0.31-airflow-2.3.3

The apache-airflow-providers-google package in images with Airflow 2.1.4 and 2.2.5 was upgraded to 2022.10.17+composer. Changes compared to version 2022.9.6+composer:

• Upgraded requirements for the google-cloud-container package from >=0.1.1,<2.0.0 to >=2.2.0,<3.0.0.
• Added changes for GKEHook and GKE unit tests from #22852, without pulling changes for DataprocCreateClusterOperator.

Cloud Composer 1.19.13 and 2.0.30 images are available:

• composer-1.19.13-airflow-1.10.15 (default)
• composer-1.19.13-airflow-2.1.4
• composer-1.19.13-airflow-2.2.5
• composer-1.19.13-airflow-2.3.3
• composer-2.0.30-airflow-2.1.4
• composer-2.0.30-airflow-2.2.5
• composer-2.0.30-airflow-2.3.3

Starting from Cloud Composer version 2.0.29, new private IP environments by default use Private Service Connect (instead of VPC peerings).

You can set HTTP_PROXY and HTTPS_PROXY environment variables in your environment. These standard Linux variables are used to route traffic through the specified proxies. The NO_PROXY variable by default is set to a list of Google domains so that they are excluded from proxying.

The result of the airflow_monitoring DAG is no longer automatically sent to XCom. This change improves monitoring performance.

Cloud Composer 1.19.12 and 2.0.29 images are available:

• composer-1.19.12-airflow-1.10.15 (default)
• composer-1.19.12-airflow-2.1.4
• composer-1.19.12-airflow-2.2.5
• composer-1.19.12-airflow-2.3.3
• composer-2.0.29-airflow-2.1.4
• composer-2.0.29-airflow-2.2.5
• composer-2.0.29-airflow-2.3.3

Images with Airflow 2.3.3 use the public version 8.1.0 of the apache-airflow-providers-google package. Other Cloud Composer images still use a custom version of this package, which is based on the public version 6.8.0.

When upgrading to Airflow 2.3, make sure that your environment is not affected by breaking changes introduced in the version 7.0.0 of this package.

Version 8.1.0 does not include some connectors, such as Dataform. Support for Dataform connectors will be available starting from version 8.3.0, which will be released together with Cloud Composer images with Airflow 2.3.4.

Cloud Composer 1.19.11 and 2.0.28 images are available:

• composer-1.19.11-airflow-1.10.15 (default)
• composer-1.19.11-airflow-2.1.4
• composer-1.19.11-airflow-2.2.5
• composer-1.19.11-airflow-2.3.3
• composer-2.0.28-airflow-2.1.4
• composer-2.0.28-airflow-2.2.5
• composer-2.0.28-airflow-2.3.3

The apache-airflow-providers-google package was upgraded to 2022.9.6+composer. Changes compared to version 2022.8.26+composer:

• Fix the Triggerer's async thread was blocked error in the deferrable mode for Composer Operators. (#25951)
• Fix a delay in Dataproc CreateBatch operator (#26126)
• Show better log for Datafusion operators when the response is empty (#26202)

Cloud Composer 1.19.10 and 2.0.27 images are available:

• composer-1.19.10-airflow-1.10.15 (default)
• composer-1.19.10-airflow-2.1.4
• composer-1.19.10-airflow-2.2.5
• composer-2.0.27-airflow-2.1.4
• composer-2.0.27-airflow-2.2.5

(Available without upgrading) When an environment is deleted, Cloud Composer automatically deletes the persistent disk of the environment's Redis queue.

(Available without upgrading) Cloud Composer now makes several attempts to create an environment when the Cloud Composer connection subnetwork is locked by another operation. This change improves the reliability when creating environments with Public Service Connect.

The apache-airflow-providers-google package was upgraded to 2022.8.26+composer. Changes compared to version 2022.8.23+composer:

• Added deferrable option to Dataproc operators to run the task asynchronously. (#25302)

  Cloud Composer team plans to add support for Deferrable operators in October, 2022.

Cloud Composer 1.19.9 and 2.0.26 images are available:

• composer-1.19.9-airflow-1.10.15 (default)
• composer-1.19.9-airflow-2.1.4
• composer-1.19.9-airflow-2.2.5
• composer-2.0.26-airflow-2.1.4
• composer-2.0.26-airflow-2.2.5

The apache-airflow-providers-google package was upgraded to 2022.8.23+composer. Changes compared to version 2022.8.16+composer:

• Added missing google-cloud-dataform package.

Cloud Composer uses a custom version of the apache-airflow-providers-google package. This custom version is based on the public version 6.8.0. For information about other changes compared to version 6.8.0, see release notes for the previous versions of this package.

(Available without upgrading) It is no longer possible to set environment variables with names http_proxy and https_proxy.

Cloud Composer 1.19.8 and 2.0.25 images are available:

• composer-1.19.8-airflow-1.10.15 (default)
• composer-1.19.8-airflow-2.1.4
• composer-1.19.8-airflow-2.2.5
• composer-2.0.25-airflow-2.1.4
• composer-2.0.25-airflow-2.2.5

The apache-airflow-providers-google package was upgraded to 2022.8.16+composer. Changes compared to version 2022.6.22+composer:

• Fix Vertex AI Custom Job training issue (#25367)
• New Operators for the Google Cloud Dataform service (#25587)

Cloud Composer uses a custom version of the apache-airflow-providers-google package. This custom version is based on the public version 6.8.0. For information about other changes compared to version 6.8.0, see release notes for the previous versions of this package.

The following packages are now preinstalled in Cloud Composer images with Airflow 2.2.5: google-cloud-firestore, firebase-admin, gcfs, dbt-core, apache-airflow-providers-dbt-cloud.

Cloud Composer 1.19.7 and 2.0.24 images are available:

• composer-1.19.7-airflow-1.10.15 (default)
• composer-1.19.7-airflow-2.1.4
• composer-1.19.7-airflow-2.2.5
• composer-2.0.24-airflow-2.1.4
• composer-2.0.24-airflow-2.2.5

Logs that are generated when processing individual DAG files are no longer written to the Airflow scheduler's disk. This change improves the scheduler's disk usage.

Cloud Composer 1.19.6 and 2.0.23 images are available:

• composer-1.19.6-airflow-1.10.15 (default)
• composer-1.19.6-airflow-2.1.4
• composer-1.19.6-airflow-2.2.5
• composer-2.0.23-airflow-2.1.4
• composer-2.0.23-airflow-2.2.5

Cloud Composer 1.19.5 and 2.0.22 images are available:

• composer-1.19.5-airflow-1.10.15 (default)
• composer-1.19.5-airflow-2.1.4
• composer-1.19.5-airflow-2.2.5
• composer-2.0.22-airflow-2.1.4
• composer-2.0.22-airflow-2.2.5

Cloud Composer 1.19.4 and 2.0.21 images are available:

• composer-1.19.4-airflow-1.10.15 (default)
• composer-1.19.4-airflow-2.1.4
• composer-1.19.4-airflow-2.2.5
• composer-2.0.21-airflow-2.1.4
• composer-2.0.21-airflow-2.2.5

Source code for the apache-airflow-providers-google package versions 2022.6.22+composer and 2022.5.18+composer is available on GitHub:

• 2022.6.22+composer
• 2022.5.18+composer

Cloud Composer 1.19.3 and 2.0.20 images are available:

• composer-1.19.3-airflow-1.10.15 (default)
• composer-1.19.3-airflow-2.1.4
• composer-1.19.3-airflow-2.2.5
• composer-2.0.20-airflow-2.1.4
• composer-2.0.20-airflow-2.2.5

Cloud Composer 1.19.2 and 2.0.19 are versions with an extended upgrade timeline.

Cloud Composer 1.19.2 and 2.0.19 images are available:

• composer-1.19.2-airflow-1.10.15 (default)
• composer-1.19.2-airflow-2.1.4
• composer-1.19.2-airflow-2.2.5
• composer-2.0.19-airflow-2.1.4
• composer-2.0.19-airflow-2.2.5

The apache-airflow-providers-google package upgraded to 2022.6.22+composer. Changes compared to version 2022.5.18+composer:

• Fixed a regression in BigQueryToGCPOpertor after changes to links were introduced in #24416.
• Fixed errors related to the usage of the service_account attribute by BeamRunJavaPipelineOperator.

Cloud Composer uses a custom version of the apache-airflow-providers-google package. This custom version is based on the public version 6.8.0. For information about other changes compared to version 6.8.0, see release notes for the previous versions of this package.

Cloud Composer 1.19.1 and 2.0.18 images are available:

• composer-1.19.1-airflow-1.10.15 (default)
• composer-1.19.1-airflow-2.1.4
• composer-1.19.1-airflow-2.2.5
• composer-2.0.18-airflow-2.1.4
• composer-2.0.18-airflow-2.2.5

Cloud Composer 1.18.12 is a version with an extended upgrade timeline.

Cloud Composer 1.19.0 and 2.0.17 images are available:

• composer-1.19.0-airflow-1.10.15 (default)
• composer-1.19.0-airflow-2.1.4
• composer-1.19.0-airflow-2.2.5
• composer-2.0.17-airflow-2.1.4
• composer-2.0.17-airflow-2.2.5

(Airflow 1.10.15) Upgraded apache-beam and google provider packages to version 2022.6.1:

• Support impersonation_chain parameter for Dataflow runner in Apache Beam operators
• Added missing project_id parameter for wait_for_job method in the Dataflow operators
• Added key_secret_project_id parameter which specifies a project with KeyFile

Cloud Composer 1.18.12 and 2.0.16 images are available:

• composer-1.18.12-airflow-1.10.15 (default)
• composer-1.18.12-airflow-2.1.4
• composer-1.18.12-airflow-2.2.5
• composer-2.0.16-airflow-2.1.4
• composer-2.0.16-airflow-2.2.5

Cloud Composer 1.18.11 and 2.0.15 images are available:

• composer-1.18.11-airflow-1.10.15 (default)
• composer-1.18.11-airflow-2.1.4
• composer-1.18.11-airflow-2.2.5
• composer-2.0.15-airflow-2.1.4
• composer-2.0.15-airflow-2.2.5

(Airflow 1) The google-cloud-bigquery package is upgraded from 1.28.0 to 2.13.0.

Cloud Composer uses a custom version of the apache-airflow-providers-google package, 2022.5.18+composer. This custom version is based on the public version 6.8.0.

Changes compared to version 6.8.0:

Breaking changes:

• Upgrade to support Google Ads v10 (#22965)

Features:

• [FEATURE] google provider - BigQueryInsertJobOperator log query (#23648)
• [FEATURE] google provider - split GkeStartPodOperator execute (#23518)
• Add exportContext.offload flag to CLOUD_SQL_EXPORT_VALIDATION. (#23614)
• Create links for BiqTable operators (#23164)
• implements #22859 - Add .sql as templatable extension (#22920)
• GCSFileTransformOperator: New templated fields 'source_object', 'destination_object' (#23328)

Bug Fixes

• Fix PostgresToGCSOperator does not allow nested JSON (#23063)
• Fix GCSToGCSOperator ignores replace parameter when there is no wildcard (#23340)
• update processor to fix broken download URLs (#23299)
• LookerStartPdtBuildOperator, LookerCheckPdtBuildSensor : fix empty materialization id handling (#23025)
• Change ComputeSSH to throw provider import error instead paramiko (#23035)
• Fix cancel_on_kill after execution timeout for DataprocSubmitJobOperator (#22955)
• Fix select * query xcom push for BigQueryGetDataOperator (#22936)
• MSSQLToGCSOperator fails: datetime is not JSON Serializable (#22882)
• Update credentials when using ADC in Compute Engine #23773

Misc changes

• Add Stackdriver assets and migrate system tests to AIP-47 (#23320)
• CloudTasks assets & system tests migration (AIP-47) (#23282)
• TextToSpeech assets & system tests migration (AIP-47) (#23247)
• Fix code-snippets in google provider (#23438)
• BigQuery assets (#23165)
• Remove redundant docstring in BigQueryUpdateTableSchemaOperator (#23349)
• Migrate gcs to new system tests design (#22778)
• add missing docstring in 'BigQueryHook.create_empty_table' (#23270)
• Cleanup Google provider CHANGELOG.rst (#23390)
• migrate system test gcs_to_bigquery into new design (#22753)
• Add example DAG for demonstrating usage of GCS sensors (#22808)

Cloud Composer 1.18.10 and 2.0.14 images are available:

• composer-1.18.10-airflow-1.10.15 (default)
• composer-1.18.10-airflow-2.1.4
• composer-1.18.10-airflow-2.2.3
• composer-1.18.10-airflow-2.2.5
• composer-2.0.14-airflow-2.1.4
• composer-2.0.14-airflow-2.2.3
• composer-2.0.14-airflow-2.2.5

If it is not possible to create an environment because of CMEK-related organization policies constraints/gcp.restrictCmekCryptoKeyProjects and constraints/gcp.restrictNonCmekServices, then such attempts fail with an error immediately.

It is now possible to use upper-case symbols in the versions of PyPI packages.

If it is not possible to create an environment because of constraints/compute.vmCanIpForward and compute.vmExternalIpAccessorganization policies, then such attempts fail with an error immediately.

Cloud Composer 1.18.9 and 2.0.13 images are available:

• composer-1.18.9-airflow-1.10.15 (default)
• composer-1.18.9-airflow-2.1.4
• composer-1.18.9-airflow-2.2.3
• composer-1.18.9-airflow-2.2.5
• composer-2.0.13-airflow-2.1.4
• composer-2.0.13-airflow-2.2.3
• composer-2.0.13-airflow-2.2.5

(Airflow 2) The default value for the [webserver]worker_refresh_interval Airflow configuration option is changed to 600 seconds.

(Cloud Composer 1) Increased the memory limit for GCSfuse on machine types that have more than 4 GB of memory. This change improves the stability of the syncing process between the environment's bucket and worker pods.

(Available without upgrading) The domain prefix for Private Service Connect subnetwork (connection_subnetwork) is now omitted in environment details.

Cloud Composer 1.18.8 and 2.0.12 images are available:

• composer-1.18.8-airflow-1.10.15 (default)
• composer-1.18.8-airflow-2.1.4
• composer-1.18.8-airflow-2.2.3
• composer-2.0.12-airflow-2.1.4
• composer-2.0.12-airflow-2.2.3

(Cloud Composer 2) When a new environment is created, Cloud Composer checks if the project's network setup allows connections to the Airflow web server domain, *.composer.cloud.google.com. An error is generated if it's not possible to connect to this domain.

Cloud Composer 1.18.7 and 2.0.11 images are available:

• composer-1.18.7-airflow-1.10.15 (default)
• composer-1.18.7-airflow-2.1.4
• composer-1.18.7-airflow-2.2.3
• composer-2.0.11-airflow-2.1.4
• composer-2.0.11-airflow-2.2.3

(Cloud Composer 2) Airflow webserver and worker-scheduler images in multiregional repositories are now tagged with their image version (for example, composer-2.0.10-airflow-2.1.4). This change is gradually backfilled to previous images as well.

(Airflow 1.10.15) Airflow Upgrade Checker updated to version 1.4.0.

Cloud Composer 1.18.6 and 2.0.10 images are available:

• composer-1.18.6-airflow-1.10.15 (default)
• composer-1.18.6-airflow-2.1.4
• composer-1.18.6-airflow-2.2.3
• composer-2.0.10-airflow-2.1.4
• composer-2.0.10-airflow-2.2.3

When syncing objects in an environment bucket to Airflow components, Cloud Composer now reports problems with file names. Such objects no longer cause the syncing process to fail. Objects in the environment's bucket named ".", having names ending in "/", and directories and objects having the same name are reported.

Airflow 2.2.3 packages are updated: added keyrings.google-artifactregistry-auth, updated apache-airflow-providers-google to version 6.6.0.

Cloud Composer 1.18.5 and 2.0.9 images are available:

• composer-2.0.9-airflow-2.2.3
• composer-2.0.9-airflow-2.1.4
• composer-1.18.5-airflow-2.2.3
• composer-1.18.5-airflow-2.1.4
• composer-1.18.5-airflow-1.10.15 (default)

The description of Composer Compute CPU SKUs was changed from "CPU" to "mCPU", to simplify the invoice interpretation. There are no changes in the actual Cloud Composer pricing model.

Improved the validation of custom IP ranges that are specified when an environment is created. The validation is more extensive and redundant error messages were removed.

(New Cloud Composer 1 environments) The minimum disk size for environment nodes is changed from 20 GB to 30 GB.

Cloud Composer 1.18.4 and 2.0.8 images are available:

• composer-2.0.8-airflow-2.2.3
• composer-2.0.8-airflow-2.1.4
• composer-1.18.4-airflow-2.2.3
• composer-1.18.4-airflow-2.1.4
• composer-1.18.4-airflow-1.10.15 (default)

Environment labels are now propagated to the environment's bucket.

Cloud Composer 1.18.3 and 2.0.7 images are available:

• composer-2.0.7-airflow-2.2.3
• composer-2.0.7-airflow-2.1.4
• composer-1.18.3-airflow-2.2.3
• composer-1.18.3-airflow-2.1.4
• composer-1.18.3-airflow-1.10.15 (default)

Improved the handling of errors generated during update and upgrade operations because of an invalid pip.conf configuration file.

(Airflow 1) The apache-airflow-backport-providers-google package is updated to version 2022.2.22.

Cloud Composer 1.18.2 and 2.0.6 images are available:

• composer-2.0.6-airflow-2.2.3
• composer-2.0.6-airflow-2.1.4
• composer-1.18.2-airflow-2.2.3
• composer-1.18.2-airflow-2.1.4
• composer-1.18.2-airflow-1.10.15 (default)

(New environments only) Temporary Pub/Sub topics created during environment operations are now labeled.

(Airflow 2) The google-cloud-datastore package was added to the list of preinstalled packages.

(Airflow 1) The apache-airflow-backport-providers-google package is updated to version 2022.2.11.

(Available without upgrading) Improved the handling of errors in DAG UI for tasks without a set operator.

Cloud Composer 1.18.1 and 2.0.5 images are available:

• composer-2.0.5-airflow-2.2.3
• composer-2.0.5-airflow-2.1.4
• composer-1.18.1-airflow-2.2.3
• composer-1.18.1-airflow-2.1.4
• composer-1.18.1-airflow-1.10.15 (default)

(Cloud Composer 1) Starting from this version, newly created environments use Python 3.8 as the default Python version. Python 3.6 is no longer available. If you upgrade an existing environment to 1.18.0 and later versions, the Python version changes from Python 3.6 to Python 3.8.

Cloud Composer 1.18.0 and 2.0.4 images are available:

• composer-2.0.4-airflow-2.2.3
• composer-2.0.4-airflow-2.1.4
• composer-1.18.0-airflow-2.2.3
• composer-1.18.0-airflow-2.1.4
• composer-1.18.0-airflow-1.10.15 (default)

(Airflow 2.2.3) Support for Deferrable Tasks is not available in Cloud Composer yet.

Cloud Composer 1.17.10 and 2.0.3 images are available:

• composer-2.0.3-airflow-2.2.3
• composer-2.0.3-airflow-2.1.4
• composer-2.0.3-airflow-2.0.2
• composer-1.17.10-airflow-2.2.3
• composer-1.17.10-airflow-2.1.4
• composer-1.17.10-airflow-2.0.2
• composer-1.17.10-airflow-1.10.15 (default)

Cloud Composer 1.17.9 and 2.0.2 images are available:

• composer-2.0.2-airflow-2.1.4
• composer-2.0.2-airflow-2.0.2
• composer-1.17.9-airflow-2.1.4
• composer-1.17.9-airflow-2.0.2
• composer-1.17.9-airflow-1.10.15 (default)

(Available without upgrading) Programmatic calls to Airflow API and Airflow UI in Cloud Composer 2 must now contain OAuth tokens with the https://www.googleapis.com/auth/cloud-platform scope. You can find an example of setting this scope in Make calls to Airflow REST API.

Cloud Composer 1.17.8 and 2.0.1 images are available:

• composer-2.0.1-airflow-2.1.4
• composer-2.0.1-airflow-2.0.2
• composer-1.17.8-airflow-2.1.4
• composer-1.17.8-airflow-2.0.2
• composer-1.17.8-airflow-1.10.15 (default)

Cloud Composer 2.0.0 images are available:

• composer-2.0.0-airflow-2.1.4
• composer-2.0.0-airflow-2.0.2

(Airflow 2.1.4) The apache-airflow-providers-hashicorp package is added to images with Airflow 2.1.4.

(Airflow 2.1.4) Backported the fix for the SerializedDagNotFound: DAG not found in serialized_dag table Airflow bug.

(Airflow 2.1.4) Users with the Admin role can now create users in the Airflow UI.

Cloud Composer 1.17.7 and 2.0.0-preview.7 images are available:

• composer-1.17.7-airflow-1.10.15 (default)
• composer-1.17.7-airflow-2.0.2
• composer-1.17.7-airflow-2.1.4
• composer-2.0.0-preview.7-airflow-2.0.2
• composer-2.0.0-preview.7-airflow-2.1.4

(Available without upgrading) Web server network access control settings can now be configured in Cloud Composer 2 environments.

(Cloud Composer 2) The /dags directory from the environment's bucket is no longer synced to the web server. With this change, the web server startup times are improved.

(Cloud Composer 2) The performance of syncing files to schedulers and workers was improved.

(Airflow 1.10.15) The apache-airflow-backport-providers-google package is updated:

• Wait mechanism for DataprocJobSensor was implemented to handle transient issues when a Job status is not available after creation.
• Support for impersonation_chain in the GKEStartPodOperator.
• New Operators for Dataproc Metastore and Dataproc Serverless.
• Fixed a bug in Dataflow hook when no jobs are returned.
• Google Secret Manager clients are no longer cached.
• Use correct project and location in the on_kill method for Dataflow operators.

New versions of Cloud Composer images:

• composer-1.17.6-airflow-1.10.15 (default)
• composer-1.17.6-airflow-2.0.2
• composer-1.17.6-airflow-2.1.4
• composer-2.0.0-preview.6-airflow-2.0.2
• composer-2.0.0-preview.6-airflow-2.1.4

(Cloud Composer 1) GCSfuse version updated to 37.0. Cloud Composer uses GCSfuse to sync files between the environment bucket and worker pods.

(New environments only) Default values for maintenance windows are from 00:00:00 to 04:00:00 (GMT) on Sunday, Friday, and Saturday every week.

New versions of Cloud Composer images:

composer-1.17.5-airflow-2.1.4 composer-1.17.5-airflow-2.1.2 composer-1.17.5-airflow-2.0.2 composer-1.17.5-airflow-1.10.15 (default) composer-1.17.5-airflow-1.10.12 composer-1.17.5-airflow-1.10.14

The scheduler heartbeat probe no longer checks for the availability of logs in Cloud Logging. This change reduces the chance of false positives.

(Cloud Composer 2 only) The termination grace period for workers is extended from 30 seconds to 30 minutes. This change does not apply to operations that re-create pods (for example, when changing environment variables), so that these operations do not require a long time to complete.

(New Cloud Composer 2 environments only) Access to the Airflow web server in Cloud Composer 2 is now additionally protected by Service Control API checks, for extended access control.

New versions of Cloud Composer images:

• composer-2.0.0-preview.5-airflow-2.1.4
• composer-2.0.0-preview.5-airflow-2.1.2
• composer-2.0.0-preview.5-airflow-2.0.2
• composer-1.17.4-airflow-2.1.4
• composer-1.17.4-airflow-2.1.2
• composer-1.17.4-airflow-2.0.2
• composer-1.17.4-airflow-1.10.15 (default)
• composer-1.17.4-airflow-1.10.14
• composer-1.17.4-airflow-1.10.12

Added the google-cloud-aiplatform package to Cloud Composer images with Airflow versions 2.1.2, 2.0.2, and 1.10.15.

(New environments only) Cloud Composer 2 environments create Autopilot clusters using the Regular release channel. Before this change, the Rapid channel was used.

(New environments only) Shielded Nodes and Secure Boot features are enabled for Cloud Composer 1 environment clusters.

(New environments only) Cloud Composer 1 environment creation no longer fails when the constraints/compute.requireShieldedVm policy is turned on.

New versions of Cloud Composer images:

• composer-1.17.3-airflow-2.1.2
• composer-1.17.3-airflow-2.0.2
• composer-1.17.3-airflow-1.10.15 (default)
• composer-1.17.3-airflow-1.10.14
• composer-1.17.3-airflow-1.10.12
• composer-2.0.0-preview.4-airflow-2.1.2
• composer-2.0.0-preview.4-airflow-2.0.2

Changes in the preinstalled apache-airflow-backport-providers-google package for Airflow 1.10.15:

• Dataflow job operators can be run in async mode.
• Dataflow Hook handles no Job Type.

New versions of Cloud Composer images:

• composer-1.17.2-airflow-2.1.2
• composer-1.17.2-airflow-2.0.2
• composer-1.17.2-airflow-1.10.15 (default)
• composer-1.17.2-airflow-1.10.14
• composer-1.17.2-airflow-1.10.12
• composer-2.0.0-preview.3-airflow-2.1.2 (default)
• composer-2.0.0-preview.3-airflow-2.0.2

Cloud Composer images with Airflow 2 now use Python 3.8.12.

Changes in the preinstalled apache-airflow-backport-providers-google package for Airflow 1.10.15:

• CloudDataFusionStartPipelineOperator can be run in async mode.
• Added a new sensor, DatafusionPipelineStateSensor.
• Fixes for the success_states and pipeline_timeout parameters in CloudDataFusionStartPipelineOperator.

New versions of Cloud Composer images:

• composer-1.17.1-airflow-2.1.2
• composer-1.17.1-airflow-2.0.2
• composer-1.17.1-airflow-1.10.15 (default)
• composer-1.17.1-airflow-1.10.14
• composer-1.17.1-airflow-1.10.12
• composer-2.0.0-preview.2-airflow-2.1.2
• composer-2.0.0-preview.2-airflow-2.0.2

The default value for the visibility-timeout Airflow configuration option, which determines the amount of time after which the task is re-picked by another worker, is increased from 6 hours to 7 days. This change fixes a problem with long-running tasks, when two Airflow workers attempt to write to one log file at the same time, causing tasks to fail with a logging exception.

Airflow has its own system for controlling task health, which is not dependent on visibility-timeout. If required, you can override the value of this configuration option for your environment.

If an Airflow configuration option is blocked in the image version specified for an upgrade operation, and this option has an override in your environment, the upgrade operation is rejected.

New versions of Cloud Composer images:

• composer-1.17.0-airflow-2.1.2
• composer-1.17.0-airflow-2.0.2
• composer-1.17.0-airflow-1.10.15 (default)
• composer-1.17.0-airflow-1.10.14
• composer-1.17.0-airflow-1.10.12
• composer-2.0.0-preview.1-airflow-2.1.2
• composer-2.0.0-preview.1-airflow-2.0.2

Cloud Composer 2 uses the following Cloud Composer images:

• composer-2.0.0-preview.0-airflow-2.1.2
• composer-2.0.0-preview.0-airflow-2.1.1
• composer-2.0.0-preview.0-airflow-2.0.2

(Airflow 2) Cloud Composer now uses a custom authentication backend for authentication in the stable Airflow REST API. The default value of the [api]auth_backend Airflow configuration option is changed to airflow.composer.api.backend.composer_auth. The default role for new users that authenticate through the stable Airflow REST API is defined by the [api]composer_auth_user_registration_role Airflow configuration option, which is set to Op by default.

New versions of Cloud Composer images:

• composer-1.16.16-airflow-1.10.12
• composer-1.16.16-airflow-1.10.14
• composer-1.16.16-airflow-1.10.15 (default)
• composer-1.17.0-preview.12-airflow-2.0.2
• composer-1.17.0-preview.12-airflow-2.1.1
• composer-1.17.0-preview.12-airflow-2.1.2

The default value for the [core]parallelism Airflow configuration option is set to match the maximum worker concurrency in Airflow. Before this change, some tasks could be present in the queue but not executed, and then fail after reaching the timeout.

(Airflow 2.1.1) Airflow configuration options that are deprecated in Airflow 2 are now blocked. In Airflow 2, these options are located in a different configuration section, but Airflow still uses the deprecated location as a fallback.

New versions of Cloud Composer images:

• composer-1.16.15-airflow-1.10.12
• composer-1.16.15-airflow-1.10.14
• composer-1.16.15-airflow-1.10.15 (default)
• composer-1.17.0-preview.11-airflow-2.0.2
• composer-1.17.0-preview.11-airflow-2.1.1

(Airflow 2.1.1) Downgraded colorlog dependency from 5.0.1 to 4.8.0

(Airflow 2.1.1) downgraded google-cloud-memcache dependency from 1.1.0 to 1.0.0

New versions of Cloud Composer images:

• composer-1.16.14-airflow-1.10.12
• composer-1.16.14-airflow-1.10.14
• composer-1.16.14-airflow-1.10.15 (default)
• composer-1.17.0-preview.10-airflow-2.0.2
• composer-1.17.0-preview.10-airflow-2.1.1

(Airflow 2) By default, Airflow performs two retries for a failed task. The default value for the core-default_task_retries Airflow configuration option is 2.

New versions of Cloud Composer images:

• composer-1.16.13-airflow-1.10.12
• composer-1.16.13-airflow-1.10.14
• composer-1.16.13-airflow-1.10.15 (default)
• composer-1.17.0-preview.9-airflow-2.0.2
• composer-1.17.0-preview.9-airflow-2.1.1

New versions of Cloud Composer images:

• composer-1.16.12-airflow-1.10.12
• composer-1.16.12-airflow-1.10.14
• composer-1.16.12-airflow-1.10.15 (default)
• composer-1.17.0-preview.8-airflow-2.0.2
• composer-1.17.0-preview.8-airflow-2.1.1

New versions of Cloud Composer images:

• composer-1.16.11-airflow-1.10.12
• composer-1.16.11-airflow-1.10.14
• composer-1.16.11-airflow-1.10.15 (default)
• composer-1.17.0-preview.7-airflow-2.0.2

New versions of Cloud Composer images:

• composer-1.16.10-airflow-1.10.15 (default)
• composer-1.16.10-airflow-1.10.14
• composer-1.16.10-airflow-1.10.12
• composer-1.17.0-preview.6-airflow-2.0.1
• composer-1.17.0-preview.6-airflow-2.0.2

New versions of Cloud Composer images:

• composer-1.17.0-preview.5-airflow-2.0.1
• composer-1.16.9-airflow-1.10.15 (default)
• composer-1.16.9-airflow-1.10.14
• composer-1.16.9-airflow-1.10.12

New versions of Cloud Composer images:

• composer-1.17.0-preview.4-airflow-2.0.1
• composer-1.16.8-airflow-1.10.15 (default)
• composer-1.16.8-airflow-1.10.14
• composer-1.16.8-airflow-1.10.12

New versions of Cloud Composer images:

• composer-1.17.0-preview.3-airflow-2.0.1
• composer-1.16.7-airflow-1.10.15
• composer-1.16.7-airflow-1.10.14 (default)
• composer-1.16.7-airflow-1.10.12

GCSfuse version was updated to 0.35.1 (latest release). Cloud Composer uses GCSfuse to sync files between the environment buckets and worker pods. The change improves the stability of the syncing process.

(Airflow 2) Preinstalled packages changed. Removed: google_cloud_build==2.0.0, mock==2.0.0, pbr==5.5.1. Downgraded overrides from 3.1.0 to 2.8.0.

For DAG runs that have long-running tasks with certain operators, task level logs are now periodically updated in the Airflow UI. Before this change, logs for such tasks were only available in Airflow UI after the task was completed.

It is now possible to create environments with CMEK encryption in projects with enabled domain restricted sharing. Before the fix, an error related to insufficient Cloud Pub/Sub permissions was generated.

New versions of Cloud Composer images:

• composer-1.17.0-preview.2-airflow-2.0.1
• composer-1.16.6-airflow-1.10.15
• composer-1.16.6-airflow-1.10.14 (default)
• composer-1.16.6-airflow-1.10.12

Increased the timeout for environment upgrade operations to support upgrades for databases up to 16 GB in size. If an upgrade operation times out and the Airflow database size is more than 10 GB, a warning message about the database size is generated.

New versions of Cloud Composer images:

• composer-1.17.0-preview.1-airflow-2.0.1
• composer-1.16.5-airflow-1.10.15
• composer-1.16.5-airflow-1.10.14 (default)
• composer-1.16.5-airflow-1.10.12

Error messages about PyPI package conflicts now contain links to corresponding cluster build logs.

New versions of Cloud Composer images:

• composer-1.17.0-preview.0-airflow-2.0.1
• composer-1.16.4-airflow-1.10.15
• composer-1.16.4-airflow-1.10.14 (default)
• composer-1.16.4-airflow-1.10.12

For new Cloud Composer environments with Airflow 2, SMTP configuration properties for Airflow have new default values:

• smtp_user is set to an empty value by default.
• smtp_password is set to an empty value by default.
• smtp_mail_from is set to a default value used by Airflow.

Improved the error message that is generated when the specified service account does not have enough permissions to run Airflow workloads.

Added troubleshooting information to error messages generated on Airflow web server deployment failures.

GKE clusters of new Cloud Composer environments use Container-Optimized OS with Containerd (cos_containerd) image type.

Kerberos client (krb5-user) package is pre-installed in Cloud Composer container images.

Some environment operations that failed because of networking problems are now retried instead of failing.

New versions of Cloud Composer images:

• composer-1.16.3-airflow-1.10.15
• composer-1.16.3-airflow-1.10.14 (default)
• composer-1.16.3-airflow-1.10.12