Skip to content

Conversation

@rcdexta
Copy link
Contributor

@rcdexta rcdexta commented Aug 15, 2025

The PR includes:

  • x-default-catalog header support in DuckDB server for setting default catalog on connection
  • GitHub Actions CI workflow with comprehensive tests
  • Organized test files in the tests directory
  • Tests for catalog headers, prepared statements, and basic functionality

Will work like:

try:
  conn = flightsql.connect(
      uri="grpc://localhost:31337",
      db_kwargs={
          "username": username,
          "password": password,
          DatabaseOptions.WITH_COOKIE_MIDDLEWARE.value: "true",
          # Set default catalog via custom header
          f"{DatabaseOptions.RPC_CALL_HEADER_PREFIX.value}x-default-catalog": "superduck",
      }
  )
  
  with conn.cursor() as cur:
      # Check current database
      cur.execute("SELECT current_database()")
      result = cur.fetchone()
      print(f"Current database: {result[0]}")

rcdexta and others added 27 commits July 25, 2025 23:35
Fix the multiple prepare statements error with fallback
This PR adds a `--log-format` parameter to gizmosql_server that allows switching between text (default) and JSON log formats. The JSON format is optimized for ingestion by log aggregation services like Datadog.

## Changes

- Added `--log-format` parameter to gizmosql_server CLI with options: text (default), json
- Updated all logging statements to output JSON when format is set to "json"
- Maintained backward compatibility with default text format
- Applied consistent logging across both DuckDB and SQLite backends

## JSON Log Format

When using `--log-format json`, all logs are output as single-line JSON objects with:
- `level`: Log level (INFO, WARNING, ERROR)
- `message`: Human-readable message
- Additional context fields like `query`, `component`, `version`, `engine`, etc.

## Example Usage

```bash
# Default text format (backward compatible)
gizmosql_server --database-filename data.db --print-queries

# JSON format for Datadog
gizmosql_server --database-filename data.db --print-queries --log-format json
```

## Testing

- Verified both text and JSON formats work correctly
- All SQL operations tested and working
- Query logging properly outputs in JSON format when enabled
- Backward compatibility maintained

Addresses issue gizmodata#25 for consistent Arrow logging and provides foundation for structured logging requested by users for Datadog integration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
feat: Add JSON log format support for Datadog integration
- Test workflow can run after CI build or independently
- Uses build artifacts when available, falls back to Docker
- No duplicate build logic
- Flexible setup for different environments
@prmoore77
Copy link
Member

hi @rcdexta - I believe this requirement is addressed by: #51 - where you can set/get the catalog and schema now...

@prmoore77
Copy link
Member

ah - I see the logging changes, those are good. Could you pull the latest main into your branch - and see if setting/getting the catalog/schema change is acceptable?

I don't think we should merge the ECR changes you have in the PR, though (those likely should stay in the fork)...

Thanks for the PR!

@rcdexta
Copy link
Contributor Author

rcdexta commented Aug 18, 2025

@prmoore77 #51 is definitely a cleaner approach, I am going to close this PR.

Thanks for the quick changes.

@rcdexta rcdexta closed this Aug 18, 2025
@tsathishkumar tsathishkumar deleted the feat/catalog-header-support branch November 3, 2025 05:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants