Skip to content

Add a GCS external payload storage driver - #3151

Open
diwu-sf wants to merge 14 commits into
temporalio:mainfrom
diwu-sf:codex/gcs-payload-storage
Open

diwu-sf wants to merge 14 commits into
temporalio:mainfrom
diwu-sf:codex/gcs-payload-storage

Conversation

@diwu-sf

@diwu-sf diwu-sf commented Oct 16, 2026 •

Copy link
Copy Markdown

What changed?

Added an experimental Google Cloud Storage driver for external payloads and an adapter for the official Google Cloud Storage Java client.

Object names follow the existing GCS drivers' encoding and namespace layout, with a namespace-preserving fallback for long names. Claims use object_name. Uploads leave existing objects unchanged, and downloads verify SHA-256 before decoding the complete payload, including metadata.

The application owns the Google client and executor. Includes setup documentation and automatic integration tests against fsouza/fake-gcs-server. Gradle downloads and caches a checksum-verified native binary; JUnit starts the server on a free local port and cleans up afterward.

Why?

Store large Temporal payloads in GCS without writing a custom storage driver.

Related to temporalio/features#924.

Breaking changes?

No existing SDK API changes. The optional Google client adapter brings newer protobuf and gRPC dependencies.

Server PR

Not required.

Testing

  • All 54 GCS tests passed without skips, including seven tests using the real Google client against fake GCS.
  • Integration coverage includes canonical claims, Unicode object names, a 6 MiB payload, missing objects and buckets, concurrent conditional uploads, integrity failures, empty namespaces, and long-name fallback.
  • ./gradlew --offline spotlessApply passed.
  • Both GCS modules passed test and assemble, including Javadocs, with Java 21 and Java 8 release compatibility checks.
  • Confirmed that the existing dev-server preparation task also prepares the cached GCS emulator for offline tests.
  • The full repository test suite was not run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants