Skip to content

Conversation

@dfellis
Copy link
Collaborator

@dfellis dfellis commented Jan 3, 2022

Not adding any extra functionality to the app until we have tests for it. Wanted to put this approach in front of you guys to see what you think.

I can't figure out how to do coverage tests on the CLI app, but I don't think that's too big of a problem?

@coveralls
Copy link

coveralls commented Jan 3, 2022

Coverage Status

Coverage decreased (-0.06%) to 98.141% when pulling 2f386fb on dfellis:master into cd47015 on uber:master.

CMakeLists.txt Outdated
message("${name}_test${test_number} - ${argfile}")
endif()

# No coverage possible for H3 CLI testing
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume coverage should be somehow possible (would need to configure the binaries to produce coverage, and include them in the considered source paths)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll study it, but okay if that's a separate PR?

@nrabinowitz
Copy link
Collaborator

Not a huge fan of this approach - seems like it will bloat the CMakeLists.txt file, and I don't like the idea that we're expanding the purpose of that file beyond build tasks.

Other options:

  • Put all tests in a shell script
  • Put all input/output in some text file, and make a test via CMake that reads the file, runs commands from input, and asserts output
  • Use a unit testing framework in other language, e.g. Node or Python

@isaacbrodsky
Copy link
Collaborator

@nrabinowitz Another option would be to define the tests in a separate CMake file but still in that framework.

@dfellis
Copy link
Collaborator Author

dfellis commented Jan 3, 2022

@nrabinowitz Another option would be to define the tests in a separate CMake file but still in that framework.

I personally like this compromise. The CMakeLists file does already handle a lot of stuff, and this would help us organize not just the test logic but other things like the fuzzers.

@nrabinowitz
Copy link
Collaborator

@nrabinowitz Another option would be to define the tests in a separate CMake file but still in that framework.

I personally like this compromise. The CMakeLists file does already handle a lot of stuff, and this would help us organize not just the test logic but other things like the fuzzers.

That seems okay by me. The main issue for me is just that the rest of the CMake file doesn't have actual data or need the kind of review that test cases do, whereas this does. Separating it, ideally with a header comment, should be fine (and I agree that other parts could be separated out as well).

add_h3_test(testH3CellAreaExhaustive src/apps/testapps/testH3CellAreaExhaustive.c)

add_h3_cli_test(testCliCellToLatLng "cellToLatLng -c 8928342e20fffff" "37.5012466151, -122.5003039349")
add_h3_cli_test(testCliLatLngToCell "latLngToCell --lat 20 --lng 123 -r 2" "824b9ffffffffff")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still feel like there's a substantive difference between tasks that run a test and tasks that are tests. This is fine for now, but I don't see how it scales up to a large number of tests - I'd much rather have inputs and outputs defined outside of CMake (in a text file, in yaml, or even as separate lines in a script file).

@dfellis dfellis merged commit b4a53e0 into uber:master Jan 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants