Cloud Spanner - Package cloud.google.com/go/spanner (v1.36.0)

Package spanner provides a client for reading and writing to Cloud Spanner databases. See the packages under admin for clients that operate on databases and instances.

See https://cloud.google.com/spanner/docs/getting-started/go/ for an introduction to Cloud Spanner and additional help on using this API.

See https://godoc.org/cloud.google.com/go for authentication, timeouts, connection pooling and similar aspects of this package.

Creating a Client

To start working with this package, create a client that refers to the database of interest:

ctx := context.Background()
client, err := spanner.NewClient(ctx, "projects/P/instances/I/databases/D")
if err != nil {
    // TODO: Handle error.
}
defer client.Close()

Remember to close the client after use to free up the sessions in the session pool.

To use an emulator with this library, you can set the SPANNER_EMULATOR_HOST environment variable to the address at which your emulator is running. This will send requests to that address instead of to Cloud Spanner. You can then create and use a client as usual:

// Set SPANNER_EMULATOR_HOST environment variable.
err := os.Setenv("SPANNER_EMULATOR_HOST", "localhost:9010")
if err != nil {
    // TODO: Handle error.
}
// Create client as usual.
client, err := spanner.NewClient(ctx, "projects/P/instances/I/databases/D")
if err != nil {
    // TODO: Handle error.
}

Simple Reads and Writes

Two Client methods, Apply and Single, work well for simple reads and writes. As a quick introduction, here we write a new row to the database and read it back:

_, err := client.Apply(ctx, []*spanner.Mutation{
    spanner.Insert("Users",
        []string{"name", "email"},
        []interface{}{"alice", "[email protected]"})})
if err != nil {
    // TODO: Handle error.
}
row, err := client.Single().ReadRow(ctx, "Users",
    spanner.Key{"alice"}, []string{"email"})
if err != nil {
    // TODO: Handle error.
}

All the methods used above are discussed in more detail below.

Keys

Every Cloud Spanner row has a unique key, composed of one or more columns. Construct keys with a literal of type Key:

key1 := spanner.Key{"alice"}

KeyRanges

The keys of a Cloud Spanner table are ordered. You can specify ranges of keys using the KeyRange type:

kr1 := spanner.KeyRange{Start: key1, End: key2}

By default, a KeyRange includes its start key but not its end key. Use the Kind field to specify other boundary conditions:

// include both keys
kr2 := spanner.KeyRange{Start: key1, End: key2, Kind: spanner.ClosedClosed}

KeySets

A KeySet represents a set of keys. A single Key or KeyRange can act as a KeySet. Use the KeySets function to build the union of several KeySets:

ks1 := spanner.KeySets(key1, key2, kr1, kr2)

AllKeys returns a KeySet that refers to all the keys in a table:

ks2 := spanner.AllKeys()

Transactions

All Cloud Spanner reads and writes occur inside transactions. There are two types of transactions, read-only and read-write. Read-only transactions cannot change the database, do not acquire locks, and may access either the current database state or states in the past. Read-write transactions can read the database before writing to it, and always apply to the most recent database state.

Single Reads

The simplest and fastest transaction is a ReadOnlyTransaction that supports a single read operation. Use Client.Single to create such a transaction. You can chain the call to Single with a call to a Read method.

When you only want one row whose key you know, use ReadRow. Provide the table name, key, and the columns you want to read:

row, err := client.Single().ReadRow(ctx, "Accounts", spanner.Key{"alice"}, []string{"balance"})

Read multiple rows with the Read method. It takes a table name, KeySet, and list of columns:

iter := client.Single().Read(ctx, "Accounts", keyset1, columns)

Read returns a RowIterator. You can call the Do method on the iterator and pass a callback:

err := iter.Do(func(row *Row) error {
   // TODO: use row
   return nil
})

RowIterator also follows the standard pattern for the Google Cloud Client Libraries:

defer iter.Stop()
for {
    row, err := iter.Next()
    if err == iterator.Done {
        break
    }
    if err != nil {
        // TODO: Handle error.
    }
    // TODO: use row
}

Always call Stop when you finish using an iterator this way, whether or not you iterate to the end. (Failing to call Stop could lead you to exhaust the database's session quota.)

To read rows with an index, use ReadUsingIndex.

Statements

The most general form of reading uses SQL statements. Construct a Statement with NewStatement, setting any parameters using the Statement's Params map:

stmt := spanner.NewStatement("SELECT First, Last FROM SINGERS WHERE Last >= @start")
stmt.Params["start"] = "Dylan"

You can also construct a Statement directly with a struct literal, providing your own map of parameters.

Use the Query method to run the statement and obtain an iterator:

iter := client.Single().Query(ctx, stmt)

Rows

Once you have a Row, via an iterator or a call to ReadRow, you can extract column values in several ways. Pass in a pointer to a Go variable of the appropriate type when you extract a value.

You can extract by column position or name:

err := row.Column(0, &name)
err = row.ColumnByName("balance", &balance)

You can extract all the columns at once:

err = row.Columns(&name, &balance)

Or you can define a Go struct that corresponds to your columns, and extract into that:

var s struct { Name string; Balance int64 }
err = row.ToStruct(&s)

For Cloud Spanner columns that may contain NULL, use one of the NullXXX types, like NullString:

var ns spanner.NullString
if err := row.Column(0, &ns); err != nil {
    // TODO: Handle error.
}
if ns.Valid {
    fmt.Println(ns.StringVal)
} else {
    fmt.Println("column is NULL")
}

Multiple Reads

To perform more than one read in a transaction, use ReadOnlyTransaction:

txn := client.ReadOnlyTransaction()
defer txn.Close()
iter := txn.Query(ctx, stmt1)
// ...
iter =  txn.Query(ctx, stmt2)
// ...

You must call Close when you are done with the transaction.

Timestamps and Timestamp Bounds

Cloud Spanner read-only transactions conceptually perform all their reads at a single moment in time, called the transaction's read timestamp. Once a read has started, you can call ReadOnlyTransaction's Timestamp method to obtain the read timestamp.

By default, a transaction will pick the most recent time (a time where all previously committed transactions are visible) for its reads. This provides the freshest data, but may involve some delay. You can often get a quicker response if you are willing to tolerate "stale" data. You can control the read timestamp selected by a transaction by calling the WithTimestampBound method on the transaction before using it. For example, to perform a query on data that is at most one minute stale, use

client.Single().
    WithTimestampBound(spanner.MaxStaleness(1*time.Minute)).
    Query(ctx, stmt)

See the documentation of TimestampBound for more details.

Mutations

To write values to a Cloud Spanner database, construct a Mutation. The spanner package has functions for inserting, updating and deleting rows. Except for the Delete methods, which take a Key or KeyRange, each mutation-building function comes in three varieties.

One takes lists of columns and values along with the table name:

m1 := spanner.Insert("Users",
    []string{"name", "email"},
    []interface{}{"alice", "[email protected]"})

One takes a map from column names to values:

m2 := spanner.InsertMap("Users", map[string]interface{}{
    "name":  "alice",
    "email": "[email protected]",
})

And the third accepts a struct value, and determines the columns from the struct field names:

type User struct { Name, Email string }
u := User{Name: "alice", Email: "[email protected]"}
m3, err := spanner.InsertStruct("Users", u)

Writes

To apply a list of mutations to the database, use Apply:

_, err := client.Apply(ctx, []*spanner.Mutation{m1, m2, m3})

If you need to read before writing in a single transaction, use a ReadWriteTransaction. ReadWriteTransactions may be aborted automatically by the backend and need to be retried. You pass in a function to ReadWriteTransaction, and the client will handle the retries automatically. Use the transaction's BufferWrite method to buffer mutations, which will all be executed at the end of the transaction:

_, err := client.ReadWriteTransaction(ctx, func(ctx context.Context, txn *spanner.ReadWriteTransaction) error {
    var balance int64
    row, err := txn.ReadRow(ctx, "Accounts", spanner.Key{"alice"}, []string{"balance"})
    if err != nil {
        // The transaction function will be called again if the error code
        // of this error is Aborted. The backend may automatically abort
        // any read/write transaction if it detects a deadlock or other
        // problems.
        return err
    }
    if err := row.Column(0, &balance); err != nil {
        return err
    }

    if balance <= 10 {
        return errors.New("insufficient funds in account")
    }
    balance -= 10
    m := spanner.Update("Accounts", []string{"user", "balance"}, []interface{}{"alice", balance})
    // The buffered mutation will be committed.  If the commit
    // fails with an Aborted error, this function will be called
    // again.
    return txn.BufferWrite([]*spanner.Mutation{m})
})

Structs

Cloud Spanner STRUCT (aka STRUCT) values (https://cloud.google.com/spanner/docs/data-types#struct-type) can be represented by a Go struct value.

A proto StructType is built from the field types and field tag information of the Go struct. If a field in the struct type definition has a "spanner:<field_name>" tag, then the value of the "spanner" key in the tag is used as the name for that field in the built StructType, otherwise the field name in the struct definition is used. To specify a field with an empty field name in a Cloud Spanner STRUCT type, use the spanner:"" tag annotation against the corresponding field in the Go struct's type definition.

A STRUCT value can contain STRUCT-typed and Array-of-STRUCT typed fields and these can be specified using named struct-typed and []struct-typed fields inside a Go struct. However, embedded struct fields are not allowed. Unexported struct fields are ignored.

NULL STRUCT values in Cloud Spanner are typed. A nil pointer to a Go struct value can be used to specify a NULL STRUCT value of the corresponding StructType. Nil and empty slices of a Go STRUCT type can be used to specify NULL and empty array values respectively of the corresponding StructType. A slice of pointers to a Go struct type can be used to specify an array of NULL-able STRUCT values.

DML and Partitioned DML

Spanner supports DML statements like INSERT, UPDATE and DELETE. Use ReadWriteTransaction.Update to run DML statements. It returns the number of rows affected. (You can call use ReadWriteTransaction.Query with a DML statement. The first call to Next on the resulting RowIterator will return iterator.Done, and the RowCount field of the iterator will hold the number of affected rows.)

For large databases, it may be more efficient to partition the DML statement. Use client.PartitionedUpdate to run a DML statement in this way. Not all DML statements can be partitioned.

Tracing

This client has been instrumented to use OpenCensus tracing (http://opencensus.io). To enable tracing, see "Enabling Tracing for a Program" at https://godoc.org/go.opencensus.io/trace. OpenCensus tracing requires Go 1.8 or higher.

DefaultRetryBackoff is used for retryers as a fallback value when the server did not return any retry information.

DefaultSessionPoolConfig is the default configuration for the session pool that will be used for a Spanner client, unless the user supplies a specific session pool config.

DisableGfeLatencyAndHeaderMissingCountViews disables GFEHeaderMissingCount and GFELatency metric

EnableGfeHeaderMissingCountView enables GFEHeaderMissingCount metric

EnableGfeLatencyAndHeaderMissingCountViews enables GFEHeaderMissingCount and GFELatency metric

EnableGfeLatencyView enables GFELatency metric

EnableStatViews enables all views of metrics relate to session management.

ErrCode extracts the canonical error code from a Go error.

ErrDesc extracts the Cloud Spanner error description from a Go error.

ExtractRetryDelay extracts retry backoff from a *spanner.Error if present.

NumericString returns a string representing a *big.Rat in a format compatible with Spanner SQL. It returns a floating-point literal with 9 digits after the decimal point.

ToSpannerError converts a general Go error to *spanner.Error. If the given error is already a *spanner.Error, the original error will be returned.

Spanner Errors are normally created by the Spanner client library from the returned APIError of a RPC. This method can also be used to create Spanner errors for use in tests. The recommended way to create test errors is calling this method with a status error, e.g. ToSpannerError(status.New(codes.NotFound, "Table not found").Err())

An ApplyOption is an optional argument to Apply.

ApplyAtLeastOnce returns an ApplyOption that removes replay protection.

With this option, Apply may attempt to apply mutations more than once; if the mutations are not idempotent, this may lead to a failure being reported when the mutation was applied more than once. For example, an insert may fail with ALREADY_EXISTS even though the row did not exist before Apply was called. For this reason, most users of the library will prefer not to use this option. However, ApplyAtLeastOnce requires only a single RPC, whereas Apply's default replay protection may require an additional RPC. So this option may be appropriate for latency sensitive and/or high throughput blind writing.

Priority returns an ApplyOptions that sets the RPC priority to use for the commit operation.

TransactionTag returns an ApplyOption that will include the given tag as a transaction tag for a write-only transaction.

BatchReadOnlyTransaction is a ReadOnlyTransaction that allows for exporting arbitrarily large amounts of data from Cloud Spanner databases. BatchReadOnlyTransaction partitions a read/query request. Read/query request can then be executed independently over each partition while observing the same snapshot of the database. BatchReadOnlyTransaction can also be shared across multiple clients by passing around the BatchReadOnlyTransactionID and then recreating the transaction using Client.BatchReadOnlyTransactionFromID.

Note: if a client is used only to run partitions, you can create it using a ClientConfig with both MinOpened and MaxIdle set to zero to avoid creating unnecessary sessions. You can also avoid excess gRPC channels by setting ClientConfig.NumChannels to the number of concurrently active BatchReadOnlyTransactions you expect to have.

AnalyzeQuery returns the query plan for statement.

Cleanup cleans up all the resources used by this transaction and makes it unusable. Once this method is invoked, the transaction is no longer usable anywhere, including other clients/processes with which this transaction was shared.

Calling Cleanup is optional, but recommended. If Cleanup is not called, the transaction's resources will be freed when the session expires on the backend and is deleted. For more information about recycled sessions, see https://cloud.google.com/spanner/docs/sessions.

Close marks the txn as closed.

Execute runs a single Partition obtained from PartitionRead or PartitionQuery.

PartitionQuery returns a list of Partitions that can be used to execute a query against the database.

PartitionQueryWithOptions returns a list of Partitions that can be used to execute a query against the database. The sql query execution will be optimized based on the given query options.

PartitionRead returns a list of Partitions that can be used to read rows from the database. These partitions can be executed across multiple processes, even across different machines. The partition size and count hints can be configured using PartitionOptions.

PartitionReadUsingIndex returns a list of Partitions that can be used to read rows from the database using an index.

PartitionReadUsingIndexWithOptions returns a list of Partitions that can be used to read rows from the database using an index. Pass a ReadOptions to modify the read operation.

PartitionReadWithOptions returns a list of Partitions that can be used to read rows from the database. These partitions can be executed across multiple processes, even across different machines. The partition size and count hints can be configured using PartitionOptions. Pass a ReadOptions to modify the read operation.

Query executes a query against the database. It returns a RowIterator for retrieving the resulting rows.

Query returns only row data, without a query plan or execution statistics. Use QueryWithStats to get rows along with the plan and statistics. Use AnalyzeQuery to get just the plan.

QueryWithOptions executes a SQL statment against the database. It returns a RowIterator for retrieving the resulting rows. The sql query execution will be optimized based on the given query options.

QueryWithStats executes a SQL statement against the database. It returns a RowIterator for retrieving the resulting rows. The RowIterator will also be populated with a query plan and execution statistics.

Read returns a RowIterator for reading multiple rows from the database.

ReadRow reads a single row from the database.

If no row is present with the given key, then ReadRow returns an error where spanner.ErrCode(err) is codes.NotFound.

ReadRowUsingIndex reads a single row from the database using an index.

If no row is present with the given index, then ReadRowUsingIndex returns an error where spanner.ErrCode(err) is codes.NotFound.

If more than one row received with the given index, then ReadRowUsingIndex returns an error where spanner.ErrCode(err) is codes.FailedPrecondition.

ReadRowWithOptions reads a single row from the database. Pass a ReadOptions to modify the read operation.

If no row is present with the given key, then ReadRowWithOptions returns an error where spanner.ErrCode(err) is codes.NotFound.

ReadUsingIndex calls ReadWithOptions with ReadOptions{Index: index}.

ReadWithOptions returns a RowIterator for reading multiple rows from the database. Pass a ReadOptions to modify the read operation.

BatchReadOnlyTransactionID is a unique identifier for a BatchReadOnlyTransaction. It can be used to re-create a BatchReadOnlyTransaction on a different machine or process by calling Client.BatchReadOnlyTransactionFromID.

MarshalBinary implements BinaryMarshaler.

UnmarshalBinary implements BinaryUnmarshaler.

Client is a client for reading and writing data to a Cloud Spanner database. A client is safe to use concurrently, except for its Close method.

NewClient creates a client to a database. A valid database name has the form projects/PROJECT_ID/instances/INSTANCE_ID/databases/DATABASE_ID. It uses a default configuration.

NewClientWithConfig creates a client to a database. A valid database name has the form projects/PROJECT_ID/instances/INSTANCE_ID/databases/DATABASE_ID.

Apply applies a list of mutations atomically to the database.

BatchReadOnlyTransaction returns a BatchReadOnlyTransaction that can be used for partitioned reads or queries from a snapshot of the database. This is useful in batch processing pipelines where one wants to divide the work of reading from the database across multiple machines.

Note: This transaction does not use the underlying session pool but creates a new session each time, and the session is reused across clients.

You should call Close() after the txn is no longer needed on local client, and call Cleanup() when the txn is finished for all clients, to free the session.

BatchReadOnlyTransactionFromID reconstruct a BatchReadOnlyTransaction from BatchReadOnlyTransactionID

Close closes the client.

DatabaseName returns the full name of a database, e.g., "projects/spanner-cloud-test/instances/foo/databases/foodb".

PartitionedUpdate executes a DML statement in parallel across the database, using separate, internal transactions that commit independently. The DML statement must be fully partitionable: it must be expressible as the union of many statements each of which accesses only a single row of the table. The statement should also be idempotent, because it may be applied more than once.

PartitionedUpdate returns an estimated count of the number of rows affected. The actual number of affected rows may be greater than the estimate.

PartitionedUpdateWithOptions executes a DML statement in parallel across the database, using separate, internal transactions that commit independently. The sql query execution will be optimized based on the given query options.

ReadOnlyTransaction returns a ReadOnlyTransaction that can be used for multiple reads from the database. You must call Close() when the ReadOnlyTransaction is no longer needed to release resources on the server.

ReadOnlyTransaction will use a strong TimestampBound by default. Use ReadOnlyTransaction.WithTimestampBound to specify a different TimestampBound. A non-strong bound can be used to reduce latency, or "time-travel" to prior versions of the database, see the documentation of TimestampBound for details.

ReadWriteTransaction executes a read-write transaction, with retries as necessary.

The function f will be called one or more times. It must not maintain any state between calls.

If the transaction cannot be committed or if f returns an ABORTED error, ReadWriteTransaction will call f again. It will continue to call f until the transaction can be committed or the Context times out or is cancelled. If f returns an error other than ABORTED, ReadWriteTransaction will abort the transaction and return the error.

To limit the number of retries, set a deadline on the Context rather than using a fixed limit on the number of attempts. ReadWriteTransaction will retry as needed until that deadline is met.

See https://godoc.org/cloud.google.com/go/spanner#ReadWriteTransaction for more details.

ReadWriteTransactionWithOptions executes a read-write transaction with configurable options, with retries as necessary.

ReadWriteTransactionWithOptions is a configurable ReadWriteTransaction.

See https://godoc.org/cloud.google.com/go/spanner#ReadWriteTransaction for more details.

Single provides a read-only snapshot transaction optimized for the case where only a single read or query is needed. This is more efficient than using ReadOnlyTransaction() for a single read or query.

Single will use a strong TimestampBound by default. Use ReadOnlyTransaction.WithTimestampBound to specify a different TimestampBound. A non-strong bound can be used to reduce latency, or "time-travel" to prior versions of the database, see the documentation of TimestampBound for details.

ClientConfig has configurations for the client.

CommitOptions provides options for commiting a transaction in a database.

CommitResponse provides a response of a transaction commit in a database.

Decoder is the interface implemented by a custom type that can be decoded from a supported type by Spanner. A code example:

type customField struct { Prefix string Suffix string }

// Convert a string to a customField value func (cf *customField) DecodeSpanner(val interface{}) (err error) { strVal, ok := val.(string) if !ok { return fmt.Errorf("failed to decode customField: %v", val) } s := strings.Split(strVal, "-") if len(s) > 1 { cf.Prefix = s[0] cf.Suffix = s[1] } return nil }

Encoder is the interface implemented by a custom type that can be encoded to a supported type by Spanner. A code example:

type customField struct { Prefix string Suffix string }

// Convert a customField value to a string func (cf customField) EncodeSpanner() (interface{}, error) { var b bytes.Buffer b.WriteString(cf.Prefix) b.WriteString("-") b.WriteString(cf.Suffix) return b.String(), nil }

Error is the structured error returned by Cloud Spanner client.

Deprecated: Unwrap any error that is returned by the Spanner client as an APIError to access the error details. Do not try to convert the error to the spanner.Error struct, as that struct may be removed in a future release.

Example: var apiErr *apierror.APIError _, err := spanner.NewClient(context.Background()) errors.As(err, &apiErr)

Error implements error.Error.

GRPCStatus returns the corresponding gRPC Status of this Spanner error. This allows the error to be converted to a gRPC status using status.Convert(error).

Unwrap returns the wrapped error (if any).

GenericColumnValue represents the generic encoded value and type of the column. See google.spanner.v1.ResultSet proto for details. This can be useful for proxying query results when the result types are not known in advance.

If you populate a GenericColumnValue from a row using Row.Column or related methods, do not modify the contents of Type and Value.

Decode decodes a GenericColumnValue. The ptr argument should be a pointer to a Go value that can accept v.