Fix GenAI image harm categories #1667
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Important
Looks good to me! 👍
Reviewed everything up to 5f0ee63 in 53 seconds. Click for details.
- Reviewed
78lines of code in2files - Skipped
0files when reviewing. - Skipped posting
2draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. instructor/utils.py:805
- Draft comment:
Good filtering of harm categories; consider renaming the loop variable (e.g. use 'category' instead of 'c') for clarity. - Reason this comment was not posted:
Confidence changes required:33%<= threshold85%None
2. tests/llm/test_genai/test_utils.py:37
- Draft comment:
The filtering logic for supported_categories is duplicated in multiple tests. Consider extracting it to a helper to adhere to DRY. - Reason this comment was not posted:
Confidence changes required:33%<= threshold85%None
Workflow ID: wflow_2Voj85ZfCGaDIgBB
You can customize 
|
I experience something related to this issue again on new version I think #1773. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Testing
pre-commit run --files instructor/utils.py tests/llm/test_genai/test_utils.pypytest tests/llm/test_genai/test_utils.py -qpytest tests/ -k "not docs" -q(fails: ModuleNotFoundError: No module named 'pytest_examples')https://chatgpt.com/codex/tasks/task_e_686c241a883c83268f3abe2ae0471f7e
Important
Filter unsupported image-related harm categories in
update_genai_kwargsfor Google GenAI and update tests accordingly.update_genai_kwargsininstructor/utils.pynow filters out unsupported image-related harm categories for Google GenAI.safety_settings.test_utils.pyto check for correct filtering of harm categories.safety_settingsonly includes supported categories.Nonevalues are not set in the result.pre-commitandpytestto ensure no errors ininstructor/utils.pyandtest_utils.py.This description was created by
for 5f0ee63. You can customize this summary. It will automatically update as commits are pushed.