Skip to content

Conversation

justinianvoss22
Copy link
Contributor

@justinianvoss22 justinianvoss22 commented Apr 29, 2025

Description

  • Breaks up values used in datasource parameter. Sends values to receiver separately in order to process them separately.
  • Datasource is still required (as before). However, if datasource is not specified, the individual fields are used.
  • Allows for encryption, character escaping, and driver-specific processing of these fields.

Link to tracking issue

Fixes #39760

Testing

  • Unit tests:
    • scraper_test.go
    • config_test.go
    • receiver_test.go
  • Testing locally:
    • Set up agent with sqlreceiver running
    • Use this configuration:
            sqlquery:
              driver: postgres
              host: localhost
              port: 5432
              username: justinvoss
              database: bindplane18
              additional_params:
                sslmode: disable
              storage: file_storage
              queries:
                - sql: "select * from resources"
                  tracking_start_value: "10000"
                  logs:
                    - body_column: name
    
  • Using the old configuration should still work too:
           sqlquery:
               datasource: postgresql://justinvoss@localhost:5432/bindplane18?sslmode=disable
               driver: postgres
               queries:
                   - logs:
                       - attribute_columns:
                           - name
                           - kind
                       - body_column: data
                     sql: select * from resources```
                    
    
    

Documentation

  • Update README.md

@justinianvoss22 justinianvoss22 requested review from a team, crobert-1 and dmitryax as code owners April 29, 2025 20:05
Copy link

linux-foundation-easycla bot commented Apr 29, 2025

@justinianvoss22 justinianvoss22 marked this pull request as draft April 29, 2025 20:05
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 4 times, most recently from 451b0c0 to 67d4161 Compare April 30, 2025 17:45
@djaglowski djaglowski changed the title [receiver/sqlreceiver]: break up datasource parameter into struct [receiver/sqlqueryreceiver]: break up datasource parameter into struct Apr 30, 2025
@justinianvoss22
Copy link
Contributor Author

justinianvoss22 commented Apr 30, 2025

@dmitryax @crobert-1 Originally, I converted the datasource parameter from a string into a struct. I realized it might be better to have a separate parameter, datasource_config, in order to not break existing functionality. Let me know your thoughts on this.

@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch from 3cbfc3c to a9cad77 Compare April 30, 2025 18:17
@justinianvoss22 justinianvoss22 changed the title [receiver/sqlqueryreceiver]: break up datasource parameter into struct [receiver/sqlqueryreceiver]: adddatasource_config parameter Apr 30, 2025
@justinianvoss22 justinianvoss22 changed the title [receiver/sqlqueryreceiver]: adddatasource_config parameter [receiver/sqlqueryreceiver]: add datasource_config parameter Apr 30, 2025
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 2 times, most recently from 47b940e to fcf6f8f Compare May 1, 2025 15:07
@justinianvoss22
Copy link
Contributor Author

justinianvoss22 commented May 1, 2025

This URL: https://blogs.oracle.com/developers/post/connecting-a-go-application-to-oracle-database in receiver/sqlqueryreceiver/README.md is consistently failing the check-links PR check.

@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 5 times, most recently from 6d11cf9 to ba3feb5 Compare May 6, 2025 12:42
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch from ba3feb5 to 9ce48f0 Compare May 14, 2025 14:39
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch 2 times, most recently from fa6e884 to bb772e2 Compare May 22, 2025 15:26
@justinianvoss22 justinianvoss22 force-pushed the justin/bpop-2758-sql-query-receiver-breakup-data-source-parameter branch from 8e66f70 to 023c902 Compare June 27, 2025 15:00
@justinianvoss22
Copy link
Contributor Author

@crobert-1 Good to merge?

@crobert-1 crobert-1 added the ready to merge Code review completed; ready to merge by maintainers label Jun 27, 2025
@justinianvoss22
Copy link
Contributor Author

@crobert-1 I'm not authorized to push this branch, could I have you push this? Thanks!

@crobert-1
Copy link
Member

@crobert-1 I'm not authorized to push this branch, could I have you push this? Thanks!

Only contrib maintainers have permission to merge PRs, I don't have permission either. I've added the ready to merge label to signal to them that this is ready, it should be merged relatively soon. Feel free to post in #otel-collector-dev on CNCF Slack if it sits too long. 👍

@justinianvoss22
Copy link
Contributor Author

Ahh gotcha, thanks!

@mx-psi mx-psi merged commit 87aafa3 into open-telemetry:main Jun 27, 2025
187 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 27, 2025
Dylan-M pushed a commit to Dylan-M/opentelemetry-collector-contrib that referenced this pull request Aug 5, 2025
…arate fields (open-telemetry#39762)

#### Description
- Breaks up values used in `datasource` parameter. Sends values to
receiver separately in order to process them separately.
- Datasource is still required (as before). However, if `datasource` is
not specified, the individual fields are used.
- Allows for encryption, character escaping, and driver-specific
processing of these fields.

#### Link to tracking issue

Fixes
open-telemetry#39760

#### Testing
- Unit tests:
    - scraper_test.go
    - config_test.go
    - receiver_test.go
- Testing locally:
    - Set up agent with sqlreceiver running
    - Use this configuration:
    ```receivers:
            sqlquery:
              driver: postgres
              host: localhost
              port: 5432
              username: justinvoss
              database: bindplane18
              additional_params:
                sslmode: disable
              storage: file_storage
              queries:
                - sql: "select * from resources"
                  tracking_start_value: "10000"
                  logs:
                    - body_column: name
     ```
- Using the old configuration should still work too:
   ```receivers:
          sqlquery:
datasource:
postgresql://justinvoss@localhost:5432/bindplane18?sslmode=disable
              driver: postgres
              queries:
                  - logs:
                      - attribute_columns:
                          - name
                          - kind
                      - body_column: data
                    sql: select * from resources```
                   
   
#### Documentation
- Update README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal/sqlquery ready to merge Code review completed; ready to merge by maintainers receiver/sqlquery SQL query receiver waiting-for-code-owners

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[receiver/sqlqueryreceiver]: break up datasource parameter into separate fields

6 participants