Cloud Storage (GCS) - Package cloud.google.com/go/storage (v1.46.0)

Package storage provides an easy way to work with Google Cloud Storage. Google Cloud Storage stores data in named objects, which are grouped into buckets.

More information about Google Cloud Storage is available at https://cloud.google.com/storage/docs.

See https://pkg.go.dev/cloud.google.com/go for authentication, timeouts, connection pooling and similar aspects of this package.

Creating a Client

To start working with this package, create a Client:

ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
    // TODO: Handle error.
}

The client will use your default application credentials. Clients should be reused instead of created as needed. The methods of Client are safe for concurrent use by multiple goroutines.

You may configure the client by passing in options from the google.golang.org/api/option package. You may also use options defined in this package, such as WithJSONReads.

If you only wish to access public data, you can create an unauthenticated client with

client, err := storage.NewClient(ctx, option.WithoutAuthentication())

To use an emulator with this library, you can set the STORAGE_EMULATOR_HOST environment variable to the address at which your emulator is running. This will send requests to that address instead of to Cloud Storage. You can then create and use a client as usual:

// Set STORAGE_EMULATOR_HOST environment variable.
err := os.Setenv("STORAGE_EMULATOR_HOST", "localhost:9000")
if err != nil {
    // TODO: Handle error.
}

// Create client as usual.
client, err := storage.NewClient(ctx)
if err != nil {
    // TODO: Handle error.
}

// This request is now directed to http://localhost:9000/storage/v1/b
// instead of https://storage.googleapis.com/storage/v1/b
if err := client.Bucket("my-bucket").Create(ctx, projectID, nil); err != nil {
    // TODO: Handle error.
}

Please note that there is no official emulator for Cloud Storage.

Buckets

A Google Cloud Storage bucket is a collection of objects. To work with a bucket, make a bucket handle:

bkt := client.Bucket(bucketName)

A handle is a reference to a bucket. You can have a handle even if the bucket doesn't exist yet. To create a bucket in Google Cloud Storage, call BucketHandle.Create:

if err := bkt.Create(ctx, projectID, nil); err != nil {
    // TODO: Handle error.
}

Note that although buckets are associated with projects, bucket names are global across all projects.

Each bucket has associated metadata, represented in this package by BucketAttrs. The third argument to BucketHandle.Create allows you to set the initial BucketAttrs of a bucket. To retrieve a bucket's attributes, use BucketHandle.Attrs:

attrs, err := bkt.Attrs(ctx)
if err != nil {
    // TODO: Handle error.
}
fmt.Printf("bucket %s, created at %s, is located in %s with storage class %s\n",
    attrs.Name, attrs.Created, attrs.Location, attrs.StorageClass)

Objects

An object holds arbitrary data as a sequence of bytes, like a file. You refer to objects using a handle, just as with buckets, but unlike buckets you don't explicitly create an object. Instead, the first time you write to an object it will be created. You can use the standard Go io.Reader and io.Writer interfaces to read and write object data:

obj := bkt.Object("data")
// Write something to obj.
// w implements io.Writer.
w := obj.NewWriter(ctx)
// Write some text to obj. This will either create the object or overwrite whatever is there already.
if _, err := fmt.Fprintf(w, "This object contains text.\n"); err != nil {
    // TODO: Handle error.
}
// Close, just like writing a file.
if err := w.Close(); err != nil {
    // TODO: Handle error.
}

// Read it back.
r, err := obj.NewReader(ctx)
if err != nil {
    // TODO: Handle error.
}
defer r.Close()
if _, err := io.Copy(os.Stdout, r); err != nil {
    // TODO: Handle error.
}
// Prints "This object contains text."

Objects also have attributes, which you can fetch with ObjectHandle.Attrs:

objAttrs, err := obj.Attrs(ctx)
if err != nil {
    // TODO: Handle error.
}
fmt.Printf("object %s has size %d and can be read using %s\n",
    objAttrs.Name, objAttrs.Size, objAttrs.MediaLink)

Listing objects

Listing objects in a bucket is done with the BucketHandle.Objects method:

query := &storage.Query{Prefix: ""}

var names []string
it := bkt.Objects(ctx, query)
for {
    attrs, err := it.Next()
    if err == iterator.Done {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    names = append(names, attrs.Name)
}

Objects are listed lexicographically by name. To filter objects lexicographically, [Query.StartOffset] and/or [Query.EndOffset] can be used:

query := &storage.Query{
    Prefix: "",
    StartOffset: "bar/",  // Only list objects lexicographically >= "bar/"
    EndOffset: "foo/",    // Only list objects lexicographically < "foo/"
}

// ... as before

If only a subset of object attributes is needed when listing, specifying this subset using Query.SetAttrSelection may speed up the listing process:

query := &storage.Query{Prefix: ""}
query.SetAttrSelection([]string{"Name"})

// ... as before

ACLs

Both objects and buckets have ACLs (Access Control Lists). An ACL is a list of ACLRules, each of which specifies the role of a user, group or project. ACLs are suitable for fine-grained control, but you may prefer using IAM to control access at the project level (see Cloud Storage IAM docs.

To list the ACLs of a bucket or object, obtain an ACLHandle and call ACLHandle.List:

acls, err := obj.ACL().List(ctx)
if err != nil {
    // TODO: Handle error.
}
for _, rule := range acls {
    fmt.Printf("%s has role %s\n", rule.Entity, rule.Role)
}

You can also set and delete ACLs.

Conditions

Every object has a generation and a metageneration. The generation changes whenever the content changes, and the metageneration changes whenever the metadata changes. Conditions let you check these values before an operation; the operation only executes if the conditions match. You can use conditions to prevent race conditions in read-modify-write operations.

For example, say you've read an object's metadata into objAttrs. Now you want to write to that object, but only if its contents haven't changed since you read it. Here is how to express that:

w = obj.If(storage.Conditions{GenerationMatch: objAttrs.Generation}).NewWriter(ctx)
// Proceed with writing as above.

Signed URLs

You can obtain a URL that lets anyone read or write an object for a limited time. Signing a URL requires credentials authorized to sign a URL. To use the same authentication that was used when instantiating the Storage client, use BucketHandle.SignedURL.

url, err := client.Bucket(bucketName).SignedURL(objectName, opts)
if err != nil {
    // TODO: Handle error.
}
fmt.Println(url)

You can also sign a URL without creating a client. See the documentation of SignedURL for details.

url, err := storage.SignedURL(bucketName, "shared-object", opts)
if err != nil {
    // TODO: Handle error.
}
fmt.Println(url)

Post Policy V4 Signed Request

A type of signed request that allows uploads through HTML forms directly to Cloud Storage with temporary permission. Conditions can be applied to restrict how the HTML form is used and exercised by a user.

For more information, please see the XML POST Object docs as well as the documentation of BucketHandle.GenerateSignedPostPolicyV4.

pv4, err := client.Bucket(bucketName).GenerateSignedPostPolicyV4(objectName, opts)
if err != nil {
    // TODO: Handle error.
}
fmt.Printf("URL: %s\nFields; %v\n", pv4.URL, pv4.Fields)

Credential requirements for signing

If the GoogleAccessID and PrivateKey option fields are not provided, they will be automatically detected by BucketHandle.SignedURL and BucketHandle.GenerateSignedPostPolicyV4 if any of the following are true:

you are authenticated to the Storage Client with a service account's downloaded private key, either directly in code or by setting the GOOGLE_APPLICATION_CREDENTIALS environment variable (see Other Environments),
your application is running on Google Compute Engine (GCE), or
you are logged into gcloud using application default credentials with impersonation enabled.

Detecting GoogleAccessID may not be possible if you are authenticated using a token source or using option.WithHTTPClient. In this case, you can provide a service account email for GoogleAccessID and the client will attempt to sign the URL or Post Policy using that service account.

To generate the signature, you must have:

iam.serviceAccounts.signBlob permissions on the GoogleAccessID service account, and
the IAM Service Account Credentials API enabled (unless authenticating with a downloaded private key).

Errors

Errors returned by this client are often of the type googleapi.Error. These errors can be introspected for more information by using errors.As with the richer googleapi.Error type. For example:

var e *googleapi.Error
if ok := errors.As(err, &e); ok {
      if e.Code == 409 { ... }
}

Retrying failed requests

Methods in this package may retry calls that fail with transient errors. Retrying continues indefinitely unless the controlling context is canceled, the client is closed, or a non-transient error is received. To stop retries from continuing, use context timeouts or cancellation.

The retry strategy in this library follows best practices for Cloud Storage. By default, operations are retried only if they are idempotent, and exponential backoff with jitter is employed. In addition, errors are only retried if they are defined as transient by the service. See the Cloud Storage retry docs for more information.

Users can configure non-default retry behavior for a single library call (using BucketHandle.Retryer and ObjectHandle.Retryer) or for all calls made by a client (using Client.SetRetry). For example:

o := client.Bucket(bucket).Object(object).Retryer(
    // Use WithBackoff to change the timing of the exponential backoff.
    storage.WithBackoff(gax.Backoff{
        Initial:    2 * time.Second,
    }),
    // Use WithPolicy to configure the idempotency policy. RetryAlways will
    // retry the operation even if it is non-idempotent.
    storage.WithPolicy(storage.RetryAlways),
)

// Use a context timeout to set an overall deadline on the call, including all
// potential retries.
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()

// Delete an object using the specified strategy and timeout.
if err := o.Delete(ctx); err != nil {
    // Handle err.
}

Sending Custom Headers

You can add custom headers to any API call made by this package by using callctx.SetHeaders on the context which is passed to the method. For example, to add a custom audit logging header:

ctx := context.Background()
ctx = callctx.SetHeaders(ctx, "x-goog-custom-audit-<key>", "<value>")
// Use client as usual with the context and the additional headers will be sent.
client.Bucket("my-bucket").Attrs(ctx)

gRPC API

This package includes support for the Cloud Storage gRPC API. The implementation uses gRPC rather than the Default JSON & XML APIs to make requests to Cloud Storage. The Go Storage gRPC client is generally available. The Notifications, Serivce Account HMAC and GetServiceAccount RPCs are not supported through the gRPC client.

To create a client which will use gRPC, use the alternate constructor:

ctx := context.Background()
client, err := storage.NewGRPCClient(ctx)
if err != nil {
    // TODO: Handle error.
}
// Use client as usual.

Using the gRPC API inside GCP with a bucket in the same region can allow for Direct Connectivity (enabling requests to skip some proxy steps and reducing response latency). A warning is emmitted if gRPC is not used within GCP to warn that Direct Connectivity could not be initialized. Direct Connectivity is not required to access the gRPC API.

Dependencies for the gRPC API may slightly increase the size of binaries for applications depending on this package. If you are not using gRPC, you can use the build tag disable_grpc_modules to opt out of these dependencies and reduce the binary size.

The gRPC client emits metrics by default and will export the gRPC telemetry discussed in gRFC/66 and gRFC/78 to Google Cloud Monitoring. The metrics are accessible through Cloud Monitoring API and you incur no additional cost for publishing the metrics. Google Cloud Support can use this information to more quickly diagnose problems related to GCS and gRPC. Sending this data does not incur any billing charges, and requires minimal CPU (a single RPC every minute) or memory (a few KiB to batch the telemetry).

To access the metrics you can view them through Cloud Monitoring metric explorer with the prefix storage.googleapis.com/client. Metrics are emitted every minute.

You can disable metrics using the following example when creating a new gRPC client using WithDisabledClientMetrics.

The metrics exporter uses Cloud Monitoring API which determines project ID and credentials doing the following:

* Project ID is determined using OTel Resource Detector for the environment otherwise it falls back to the project provided by google.FindCredentials.

* Credentials are determined using Application Default Credentials. The principal must have roles/monitoring.metricWriter role granted. If not a logged warning will be emitted. Subsequent are silenced to prevent noisy logs.

Storage Control API

Certain control plane and long-running operations for Cloud Storage (including Folder and Managed Folder operations) are supported via the autogenerated Storage Control client, which is available as a subpackage in this module. See package docs at cloud.google.com/go/storage/control/apiv2 or reference the Storage Control API docs.

Constants

DeleteAction, SetStorageClassAction, AbortIncompleteMPUAction

const (

	// DeleteAction is a lifecycle action that deletes a live and/or archived
	// objects. Takes precedence over SetStorageClass actions.
	DeleteAction = "Delete"

	// SetStorageClassAction changes the storage class of live and/or archived
	// objects.
	SetStorageClassAction = "SetStorageClass"

	// AbortIncompleteMPUAction is a lifecycle action that aborts an incomplete
	// multipart upload when the multipart upload meets the conditions specified
	// in the lifecycle rule. The AgeInDays condition is the only allowed
	// condition for this action. AgeInDays is measured from the time the
	// multipart upload was created.
	AbortIncompleteMPUAction = "AbortIncompleteMultipartUpload"
)

NoPayload, JSONPayload

const (
	// Send no payload with notification messages.
	NoPayload = "NONE"

	// Send object metadata as JSON with notification messages.
	JSONPayload = "JSON_API_V1"
)

Values for Notification.PayloadFormat.

ObjectFinalizeEvent, ObjectMetadataUpdateEvent, ObjectDeleteEvent, ObjectArchiveEvent

const (
	// Event that occurs when an object is successfully created.
	ObjectFinalizeEvent = "OBJECT_FINALIZE"

	// Event that occurs when the metadata of an existing object changes.
	ObjectMetadataUpdateEvent = "OBJECT_METADATA_UPDATE"

	// Event that occurs when an object is permanently deleted.
	ObjectDeleteEvent = "OBJECT_DELETE"

	// Event that occurs when the live version of an object becomes an
	// archived version.
	ObjectArchiveEvent = "OBJECT_ARCHIVE"
)

Values for Notification.EventTypes.

ScopeFullControl, ScopeReadOnly, ScopeReadWrite

const (
	// ScopeFullControl grants permissions to manage your
	// data and permissions in Google Cloud Storage.
	ScopeFullControl = raw.DevstorageFullControlScope

	// ScopeReadOnly grants permissions to
	// view your data in Google Cloud Storage.
	ScopeReadOnly = raw.DevstorageReadOnlyScope

	// ScopeReadWrite grants permissions to manage your
	// data in Google Cloud Storage.
	ScopeReadWrite = raw.DevstorageReadWriteScope
)

Variables

ErrBucketNotExist, ErrObjectNotExist

var (
	// ErrBucketNotExist indicates that the bucket does not exist.
	ErrBucketNotExist = errors.New("storage: bucket doesn't exist")
	// ErrObjectNotExist indicates that the object does not exist.
	ErrObjectNotExist = errors.New("storage: object doesn't exist")
)

Functions

func ShouldRetry

func ShouldRetry(err error) bool

ShouldRetry returns true if an error is retryable, based on best practice guidance from GCS. See https://cloud.google.com/storage/docs/retry-strategy#go for more information on what errors are considered retryable.

If you would like to customize retryable errors, use the WithErrorFunc to supply a RetryOption to your library calls. For example, to retry additional errors, you can write a custom func that wraps ShouldRetry and also specifies additional errors that should return true.

func SignedURL

func SignedURL(bucket, object string, opts *SignedURLOptions) (string, error)

SignedURL returns a URL for the specified object. Signed URLs allow anyone access to a restricted resource for a limited time without needing a Google account or signing in. For more information about signed URLs, see https://cloud.google.com/storage/docs/accesscontrol#signed_urls_query_string_authentication If initializing a Storage Client, instead use the Bucket.SignedURL method which uses the Client's credentials to handle authentication.

Example

package main

import (
	"fmt"
	"io/ioutil"
	"time"

	"cloud.google.com/go/storage"
)

func main() {
	pkey, err := ioutil.ReadFile("my-private-key.pem")
	if err != nil {
		// TODO: handle error.
	}
	url, err := storage.SignedURL("my-bucket", "my-object", &storage.SignedURLOptions{
		GoogleAccessID: "[email protected]",
		PrivateKey:     pkey,
		Method:         "GET",
		Expires:        time.Now().Add(48 * time.Hour),
	})
	if err != nil {
		// TODO: handle error.
	}
	fmt.Println(url)
}

func WithDisabledClientMetrics

func WithDisabledClientMetrics() option.ClientOption

WithDisabledClientMetrics is an option that may be passed to [NewClient]. gRPC metrics are enabled by default in the GCS client and will export the gRPC telemetry discussed in gRFC/66 and gRFC/78 to Google Cloud Monitoring. The option is used to disable metrics. Google Cloud Support can use this information to more quickly diagnose problems related to GCS and gRPC. Sending this data does not incur any billing charges, and requires minimal CPU (a single RPC every few minutes) or memory (a few KiB to batch the telemetry).

The default is to enable client metrics. To opt-out of metrics collected use this option.

func WithJSONReads

func WithJSONReads() option.ClientOption

WithJSONReads is an option that may be passed to [NewClient]. It sets the client to use the Cloud Storage JSON API for object reads. Currently, the default API used for reads is XML, but JSON will become the default in a future release.

Setting this option is required to use the GenerationNotMatch condition. We also recommend using JSON reads to ensure consistency with other client operations (all of which use JSON by default).

Note that when this option is set, reads will return a zero date for [ReaderObjectAttrs].LastModified and may return a different value for [ReaderObjectAttrs].CacheControl.

func WithXMLReads

func WithXMLReads() option.ClientOption

WithXMLReads is an option that may be passed to [NewClient]. It sets the client to use the Cloud Storage XML API for object reads.

This is the current default, but the default will switch to JSON in a future release.

ACLEntity

type ACLEntity string

ACLEntity refers to a user or group. They are sometimes referred to as grantees.

It could be in the form of: "user-

Or one of the predefined constants: AllUsers, AllAuthenticatedUsers.

AllUsers, AllAuthenticatedUsers

const (
	AllUsers              ACLEntity = "allUsers"
	AllAuthenticatedUsers ACLEntity = "allAuthenticatedUsers"
)

ACLHandle

type ACLHandle struct {
	// contains filtered or unexported fields
}

ACLHandle provides operations on an access control list for a Google Cloud Storage bucket or object. ACLHandle on an object operates on the latest generation of that object by default. Selecting a specific generation of an object is not currently supported by the client.

func (*ACLHandle) Delete

func (a *ACLHandle) Delete(ctx context.Context, entity ACLEntity) (err error)

Delete permanently deletes the ACL entry for the given entity.

Example

package main

import (
	"context"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	// No longer grant access to the bucket to everyone on the Internet.
	if err := client.Bucket("my-bucket").ACL().Delete(ctx, storage.AllUsers); err != nil {
		// TODO: handle error.
	}
}

func (*ACLHandle) List

func (a *ACLHandle) List(ctx context.Context) (rules []ACLRule, err error)

List retrieves ACL entries.

Example

package main

import (
	"context"
	"fmt"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	// List the default object ACLs for my-bucket.
	aclRules, err := client.Bucket("my-bucket").DefaultObjectACL().List(ctx)
	if err != nil {
		// TODO: handle error.
	}
	fmt.Println(aclRules)
}

func (*ACLHandle) Set

func (a *ACLHandle) Set(ctx context.Context, entity ACLEntity, role ACLRole) (err error)

Set sets the role for the given entity.

Example

package main

import (
	"context"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	// Let any authenticated user read my-bucket/my-object.
	obj := client.Bucket("my-bucket").Object("my-object")
	if err := obj.ACL().Set(ctx, storage.AllAuthenticatedUsers, storage.RoleReader); err != nil {
		// TODO: handle error.
	}
}

ACLRole

type ACLRole string

ACLRole is the level of access to grant.

RoleOwner, RoleReader, RoleWriter

const (
	RoleOwner  ACLRole = "OWNER"
	RoleReader ACLRole = "READER"
	RoleWriter ACLRole = "WRITER"
)

ACLRule

type ACLRule struct {
	Entity      ACLEntity
	EntityID    string
	Role        ACLRole
	Domain      string
	Email       string
	ProjectTeam *ProjectTeam
}

ACLRule represents a grant for a role to an entity (user, group or team) for a Google Cloud Storage object or bucket.

Autoclass

type Autoclass struct {
	// Enabled specifies whether the autoclass feature is enabled
	// on the bucket.
	Enabled bool
	// ToggleTime is the time from which Autoclass was last toggled.
	// If Autoclass is enabled when the bucket is created, the ToggleTime
	// is set to the bucket creation time. This field is read-only.
	ToggleTime time.Time
	// TerminalStorageClass: The storage class that objects in the bucket
	// eventually transition to if they are not read for a certain length of
	// time. Valid values are NEARLINE and ARCHIVE.
	// To modify TerminalStorageClass, Enabled must be set to true.
	TerminalStorageClass string
	// TerminalStorageClassUpdateTime represents the time of the most recent
	// update to "TerminalStorageClass".
	TerminalStorageClassUpdateTime time.Time
}

Autoclass holds the bucket's autoclass configuration. If enabled, allows for the automatic selection of the best storage class based on object access patterns. See https://cloud.google.com/storage/docs/using-autoclass for more information.

BucketAttrs

type BucketAttrs struct {
	// Name is the name of the bucket.
	// This field is read-only.
	Name string

	// ACL is the list of access control rules on the bucket.
	ACL []ACLRule

	// BucketPolicyOnly is an alias for UniformBucketLevelAccess. Use of
	// UniformBucketLevelAccess is recommended above the use of this field.
	// Setting BucketPolicyOnly.Enabled OR UniformBucketLevelAccess.Enabled to
	// true, will enable UniformBucketLevelAccess.
	BucketPolicyOnly BucketPolicyOnly

	// UniformBucketLevelAccess configures access checks to use only bucket-level IAM
	// policies and ignore any ACL rules for the bucket.
	// See https://cloud.google.com/storage/docs/uniform-bucket-level-access
	// for more information.
	UniformBucketLevelAccess UniformBucketLevelAccess

	// PublicAccessPrevention is the setting for the bucket's
	// PublicAccessPrevention policy, which can be used to prevent public access
	// of data in the bucket. See
	// https://cloud.google.com/storage/docs/public-access-prevention for more
	// information.
	PublicAccessPrevention PublicAccessPrevention

	// DefaultObjectACL is the list of access controls to
	// apply to new objects when no object ACL is provided.
	DefaultObjectACL []ACLRule

	// DefaultEventBasedHold is the default value for event-based hold on
	// newly created objects in this bucket. It defaults to false.
	DefaultEventBasedHold bool

	// If not empty, applies a predefined set of access controls. It should be set
	// only when creating a bucket.
	// It is always empty for BucketAttrs returned from the service.
	// See https://cloud.google.com/storage/docs/json_api/v1/buckets/insert
	// for valid values.
	PredefinedACL string

	// If not empty, applies a predefined set of default object access controls.
	// It should be set only when creating a bucket.
	// It is always empty for BucketAttrs returned from the service.
	// See https://cloud.google.com/storage/docs/json_api/v1/buckets/insert
	// for valid values.
	PredefinedDefaultObjectACL string

	// Location is the location of the bucket. It defaults to "US".
	// If specifying a dual-region, CustomPlacementConfig should be set in conjunction.
	Location string

	// The bucket's custom placement configuration that holds a list of
	// regional locations for custom dual regions.
	CustomPlacementConfig *CustomPlacementConfig

	// MetaGeneration is the metadata generation of the bucket.
	// This field is read-only.
	MetaGeneration int64

	// StorageClass is the default storage class of the bucket. This defines
	// how objects in the bucket are stored and determines the SLA
	// and the cost of storage. Typical values are "STANDARD", "NEARLINE",
	// "COLDLINE" and "ARCHIVE". Defaults to "STANDARD".
	// See https://cloud.google.com/storage/docs/storage-classes for all
	// valid values.
	StorageClass string

	// Created is the creation time of the bucket.
	// This field is read-only.
	Created time.Time

	// Updated is the time at which the bucket was last modified.
	// This field is read-only.
	Updated time.Time

	// VersioningEnabled reports whether this bucket has versioning enabled.
	VersioningEnabled bool

	// Labels are the bucket's labels.
	Labels map[string]string

	// RequesterPays reports whether the bucket is a Requester Pays bucket.
	// Clients performing operations on Requester Pays buckets must provide
	// a user project (see BucketHandle.UserProject), which will be billed
	// for the operations.
	RequesterPays bool

	// Lifecycle is the lifecycle configuration for objects in the bucket.
	Lifecycle Lifecycle

	// Retention policy enforces a minimum retention time for all objects
	// contained in the bucket. A RetentionPolicy of nil implies the bucket
	// has no minimum data retention.
	//
	// This feature is in private alpha release. It is not currently available to
	// most customers. It might be changed in backwards-incompatible ways and is not
	// subject to any SLA or deprecation policy.
	RetentionPolicy *RetentionPolicy

	// The bucket's Cross-Origin Resource Sharing (CORS) configuration.
	CORS []CORS

	// The encryption configuration used by default for newly inserted objects.
	Encryption *BucketEncryption

	// The logging configuration.
	Logging *BucketLogging

	// The website configuration.
	Website *BucketWebsite

	// Etag is the HTTP/1.1 Entity tag for the bucket.
	// This field is read-only.
	Etag string

	// LocationType describes how data is stored and replicated.
	// Typical values are "multi-region", "region" and "dual-region".
	// This field is read-only.
	LocationType string

	// The project number of the project the bucket belongs to.
	// This field is read-only.
	ProjectNumber uint64

	// RPO configures the Recovery Point Objective (RPO) policy of the bucket.
	// Set to RPOAsyncTurbo to turn on Turbo Replication for a bucket.
	// See https://cloud.google.com/storage/docs/managing-turbo-replication for
	// more information.
	RPO RPO

	// Autoclass holds the bucket's autoclass configuration. If enabled,
	// allows for the automatic selection of the best storage class
	// based on object access patterns.
	Autoclass *Autoclass

	// ObjectRetentionMode reports whether individual objects in the bucket can
	// be configured with a retention policy. An empty value means that object
	// retention is disabled.
	// This field is read-only. Object retention can be enabled only by creating
	// a bucket with SetObjectRetention set to true on the BucketHandle. It
	// cannot be modified once the bucket is created.
	// ObjectRetention cannot be configured or reported through the gRPC API.
	ObjectRetentionMode string

	// SoftDeletePolicy contains the bucket's soft delete policy, which defines
	// the period of time that soft-deleted objects will be retained, and cannot
	// be permanently deleted. By default, new buckets will be created with a
	// 7 day retention duration. In order to fully disable soft delete, you need
	// to set a policy with a RetentionDuration of 0.
	SoftDeletePolicy *SoftDeletePolicy

	// HierarchicalNamespace contains the bucket's hierarchical namespace
	// configuration. Hierarchical namespace enabled buckets can contain
	// [cloud.google.com/go/storage/control/apiv2/controlpb.Folder] resources.
	// It cannot be modified after bucket creation time.
	// UniformBucketLevelAccess must also also be enabled on the bucket.
	HierarchicalNamespace *HierarchicalNamespace
}

BucketAttrs represents the metadata for a Google Cloud Storage bucket. Read-only fields are ignored by BucketHandle.Create.

BucketAttrsToUpdate

type BucketAttrsToUpdate struct {
	// If set, updates whether the bucket uses versioning.
	VersioningEnabled optional.Bool

	// If set, updates whether the bucket is a Requester Pays bucket.
	RequesterPays optional.Bool

	// DefaultEventBasedHold is the default value for event-based hold on
	// newly created objects in this bucket.
	DefaultEventBasedHold optional.Bool

	// BucketPolicyOnly is an alias for UniformBucketLevelAccess. Use of
	// UniformBucketLevelAccess is recommended above the use of this field.
	// Setting BucketPolicyOnly.Enabled OR UniformBucketLevelAccess.Enabled to
	// true, will enable UniformBucketLevelAccess. If both BucketPolicyOnly and
	// UniformBucketLevelAccess are set, the value of UniformBucketLevelAccess
	// will take precedence.
	BucketPolicyOnly *BucketPolicyOnly

	// UniformBucketLevelAccess configures access checks to use only bucket-level IAM
	// policies and ignore any ACL rules for the bucket.
	// See https://cloud.google.com/storage/docs/uniform-bucket-level-access
	// for more information.
	UniformBucketLevelAccess *UniformBucketLevelAccess

	// PublicAccessPrevention is the setting for the bucket's
	// PublicAccessPrevention policy, which can be used to prevent public access
	// of data in the bucket. See
	// https://cloud.google.com/storage/docs/public-access-prevention for more
	// information.
	PublicAccessPrevention PublicAccessPrevention

	// StorageClass is the default storage class of the bucket. This defines
	// how objects in the bucket are stored and determines the SLA
	// and the cost of storage. Typical values are "STANDARD", "NEARLINE",
	// "COLDLINE" and "ARCHIVE". Defaults to "STANDARD".
	// See https://cloud.google.com/storage/docs/storage-classes for all
	// valid values.
	StorageClass string

	// If set, updates the retention policy of the bucket. Using
	// RetentionPolicy.RetentionPeriod = 0 will delete the existing policy.
	//
	// This feature is in private alpha release. It is not currently available to
	// most customers. It might be changed in backwards-incompatible ways and is not
	// subject to any SLA or deprecation policy.
	RetentionPolicy *RetentionPolicy

	// If set, replaces the CORS configuration with a new configuration.
	// An empty (rather than nil) slice causes all CORS policies to be removed.
	CORS []CORS

	// If set, replaces the encryption configuration of the bucket. Using
	// BucketEncryption.DefaultKMSKeyName = "" will delete the existing
	// configuration.
	Encryption *BucketEncryption

	// If set, replaces the lifecycle configuration of the bucket.
	Lifecycle *Lifecycle

	// If set, replaces the logging configuration of the bucket.
	Logging *BucketLogging

	// If set, replaces the website configuration of the bucket.
	Website *BucketWebsite

	// If not empty, applies a predefined set of access controls.
	// See https://cloud.google.com/storage/docs/json_api/v1/buckets/patch.
	PredefinedACL string

	// If not empty, applies a predefined set of default object access controls.
	// See https://cloud.google.com/storage/docs/json_api/v1/buckets/patch.
	PredefinedDefaultObjectACL string

	// RPO configures the Recovery Point Objective (RPO) policy of the bucket.
	// Set to RPOAsyncTurbo to turn on Turbo Replication for a bucket.
	// See https://cloud.google.com/storage/docs/managing-turbo-replication for
	// more information.
	RPO RPO

	// If set, updates the autoclass configuration of the bucket.
	// To disable autoclass on the bucket, set to an empty &Autoclass{}.
	// To update the configuration for Autoclass.TerminalStorageClass,
	// Autoclass.Enabled must also be set to true.
	// See https://cloud.google.com/storage/docs/using-autoclass for more information.
	Autoclass *Autoclass

	// If set, updates the soft delete policy of the bucket.
	SoftDeletePolicy *SoftDeletePolicy
	// contains filtered or unexported fields
}

BucketAttrsToUpdate define the attributes to update during an Update call.

func (*BucketAttrsToUpdate) DeleteLabel

func (ua *BucketAttrsToUpdate) DeleteLabel(name string)

DeleteLabel causes a label to be deleted when ua is used in a call to Bucket.Update.

func (*BucketAttrsToUpdate) SetLabel

func (ua *BucketAttrsToUpdate) SetLabel(name, value string)

SetLabel causes a label to be added or modified when ua is used in a call to Bucket.Update.

BucketConditions

type BucketConditions struct {
	// MetagenerationMatch specifies that the bucket must have the given
	// metageneration for the operation to occur.
	// If MetagenerationMatch is zero, it has no effect.
	MetagenerationMatch int64

	// MetagenerationNotMatch specifies that the bucket must not have the given
	// metageneration for the operation to occur.
	// If MetagenerationNotMatch is zero, it has no effect.
	MetagenerationNotMatch int64
}

BucketConditions constrain bucket methods to act on specific metagenerations.

The zero value is an empty set of constraints.

BucketEncryption

type BucketEncryption struct {
	// A Cloud KMS key name, in the form
	// projects/P/locations/L/keyRings/R/cryptoKeys/K, that will be used to encrypt
	// objects inserted into this bucket, if no encryption method is specified.
	// The key's location must be the same as the bucket's.
	DefaultKMSKeyName string
}

BucketEncryption is a bucket's encryption configuration.

BucketHandle

type BucketHandle struct {
	// contains filtered or unexported fields
}

BucketHandle provides operations on a Google Cloud Storage bucket. Use Client.Bucket to get a handle.

Example

exists

package main

import (
	"context"
	"fmt"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}

	attrs, err := client.Bucket("my-bucket").Attrs(ctx)
	if err == storage.ErrBucketNotExist {
		fmt.Println("The bucket does not exist")
		return
	}
	if err != nil {
		// TODO: handle error.
	}
	fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}

func (*BucketHandle) ACL

func (b *BucketHandle) ACL() *ACLHandle

ACL returns an ACLHandle, which provides access to the bucket's access control list. This controls who can list, create or overwrite the objects in a bucket. This call does not perform any network operations.

func (*BucketHandle) AddNotification

func (b *BucketHandle) AddNotification(ctx context.Context, n *Notification) (ret *Notification, err error)

AddNotification adds a notification to b. You must set n's TopicProjectID, TopicID and PayloadFormat, and must not set its ID. The other fields are all optional. The returned Notification's ID can be used to refer to it. Note: gRPC is not supported.

Example

package main

import (
	"context"
	"fmt"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	b := client.Bucket("my-bucket")
	n, err := b.AddNotification(ctx, &storage.Notification{
		TopicProjectID: "my-project",
		TopicID:        "my-topic",
		PayloadFormat:  storage.JSONPayload,
	})
	if err != nil {
		// TODO: handle error.
	}
	fmt.Println(n.ID)
}

func (*BucketHandle) Attrs

func (b *BucketHandle) Attrs(ctx context.Context) (attrs *BucketAttrs, err error)

Attrs returns the metadata for the bucket.

Example

package main

import (
	"context"
	"fmt"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	attrs, err := client.Bucket("my-bucket").Attrs(ctx)
	if err != nil {
		// TODO: handle error.
	}
	fmt.Println(attrs)
}

func (*BucketHandle) BucketName

func (b *BucketHandle) BucketName() string

BucketName returns the name of the bucket.

func (*BucketHandle) Create

func (b *BucketHandle) Create(ctx context.Context, projectID string, attrs *BucketAttrs) (err error)

Create creates the Bucket in the project. If attrs is nil the API defaults will be used.

Example

package main

import (
	"context"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	if err := client.Bucket("my-bucket").Create(ctx, "my-project", nil); err != nil {
		// TODO: handle error.
	}
}

func (*BucketHandle) DefaultObjectACL

func (b *BucketHandle) DefaultObjectACL() *ACLHandle

DefaultObjectACL returns an ACLHandle, which provides access to the bucket's default object ACLs. These ACLs are applied to newly created objects in this bucket that do not have a defined ACL. This call does not perform any network operations.

func (*BucketHandle) Delete

func (b *BucketHandle) Delete(ctx context.Context) (err error)

Delete deletes the Bucket.

Example

package main

import (
	"context"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	if err := client.Bucket("my-bucket").Delete(ctx); err != nil {
		// TODO: handle error.
	}
}

func (*BucketHandle) DeleteNotification

func (b *BucketHandle) DeleteNotification(ctx context.Context, id string) (err error)

DeleteNotification deletes the notification with the given ID. Note: gRPC is not supported.

Example

package main

import (
	"context"

	"cloud.google.com/go/storage"
)

var notificationID string

func main() {
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		// TODO: handle error.
	}
	b := client.Bucket("my-bucket")
	// TODO: Obtain notificationID from BucketHandle.AddNotification
	// or BucketHandle.Notifications.
	err = b.DeleteNotification(ctx, notificationID)
	if err != nil {
		// TODO: handle error.
	}
}

func (*BucketHandle) GenerateSignedPostPolicyV4

func (b *BucketHandle) GenerateSignedPostPolicyV4(object string, opts *PostPolicyV4Options) (*PostPolicyV4, error)

GenerateSignedPostPolicyV4 generates a PostPolicyV4 value from bucket, object and opts. The generated URL and fields will then allow an unauthenticated client to perform multipart uploads.

This method requires the Expires field in the specified PostPolicyV4Options to be non-nil. You may need to set the GoogleAccessID and PrivateKey fields in some cases. Read more on the automatic detection of credentials for this method.

func (*BucketHandle) IAM

func (b *BucketHandle) IAM() *iam.

Cloud Storage (GCS) - Package cloud.google.com/go/storage (v1.46.0) Stay organized with collections Save and categorize content based on your preferences.

Creating a Client

Buckets

Objects

Listing objects

ACLs

Conditions

Signed URLs

Post Policy V4 Signed Request

Credential requirements for signing

Errors

Retrying failed requests

Sending Custom Headers

gRPC API

Storage Control API

Constants

DeleteAction, SetStorageClassAction, AbortIncompleteMPUAction

NoPayload, JSONPayload

ObjectFinalizeEvent, ObjectMetadataUpdateEvent, ObjectDeleteEvent, ObjectArchiveEvent

ScopeFullControl, ScopeReadOnly, ScopeReadWrite

Variables

ErrBucketNotExist, ErrObjectNotExist

Functions

func ShouldRetry

func SignedURL

Example

func WithDisabledClientMetrics

func WithJSONReads

func WithXMLReads

ACLEntity

AllUsers, AllAuthenticatedUsers

ACLHandle

func (*ACLHandle) Delete

Example

func (*ACLHandle) List

Example

func (*ACLHandle) Set

Example

ACLRole

RoleOwner, RoleReader, RoleWriter

ACLRule

Autoclass

BucketAttrs

BucketAttrsToUpdate

func (*BucketAttrsToUpdate) DeleteLabel

func (*BucketAttrsToUpdate) SetLabel

BucketConditions

BucketEncryption

BucketHandle

Example

exists

func (*BucketHandle) ACL

func (*BucketHandle) AddNotification

Example

func (*BucketHandle) Attrs

Example

func (*BucketHandle) BucketName

func (*BucketHandle) Create

Example

func (*BucketHandle) DefaultObjectACL

func (*BucketHandle) Delete

Example

func (*BucketHandle) DeleteNotification

Example

func (*BucketHandle) GenerateSignedPostPolicyV4

func (*BucketHandle) IAM

Cloud Storage (GCS) - Package cloud.google.com/go/storage (v1.46.0)