Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/python-bigquery-storage/issues
• Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python

If you are still having issues, please be sure to include as much information as possible:

Environment details

• OS type and version:
• Python version: python --version
• pip version: pip --version
• google-cloud-bigquery-storage version: pip show google-cloud-bigquery-storage

$ pip show google-cloud-bigquery-storage
Name: google-cloud-bigquery-storage
Version: 2.26.0
Summary: Google Cloud Bigquery Storage API client library
Home-page: https://github.com/googleapis/python-bigquery-storage
Author: Google LLC
Author-email: [email protected]
License: Apache 2.0
Location: /Users/film42/.pyenv/versions/3.10.9/envs/banzai-sept-2/lib/python3.10/site-packages
Requires: google-api-core, google-auth, proto-plus, protobuf
Required-by: 

Steps to reproduce

1. Use a STRING protobuf field to append a row with a DATETIME bigquery column type using the value '2024-10-11T00:17:35.479490+00:00'.

Please see previous issue #717 where @bhavitsharma ran into the same issue as I did.

Code example

datetime_string = dt.datetime.now(dt.timezone.utc).isoformat()

p = MyProto()
p.created_at = datetime_string

# perform a normal append ...

try:
  future = append_rows_stream.send(request)
  future.result()
except InvalidArgument as e:
  pass
  # How are you supposed to get the AppendRowsResponse object?

Docs here say that errors must be read from the response which includes an index, msg, and error type for each row. https://cloud.google.com/bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1#google.cloud.bigquery.storage.v1.AppendRowsResponse . But, there is no way to get access to the underlying response because on error this lib capture the code and message and throws away the rest:

If you crack open the source code of this lib and modify it to print the response.row_errors you get:

print(response.row_errors[499])

index: 499
code: FIELDS_ERROR
message: "Invalid date time value \'2024-10-11T00:17:35.479490+00:00\' on field \'MyProto.created_at\'"

Stack trace

Without access to the response to inspect each row, this is the only thing you see:

400 Errors found while processing rows. Please refer to the row_errors field for details. The list may not be complete because of the size limitations. Entity: projects/my_project/datasets/my_dataset/tables/my_table/streams/my_stream_id

In other words... nothing helpful.

I think it would be great if we could have something like:

try:
  append_rows_stream.send(request).result()
except InvalidArgument as e:
  # Provide access to the response on the InvalidArgument error...
  print(e.response.row_errors[499].message)

OR

try:
  future = append_rows_stream.send(request)
  future.result()
except InvalidArgument as e:
  # Allow this custom AppendRowsFuture object to retain access to the underlying response
  print(future.response().row_errors[499].message)