Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: instructlab/eval
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.1.1
Choose a base ref
...
head repository: instructlab/eval
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.1.2
Choose a head ref
  • 3 commits
  • 3 files changed
  • 2 contributors

Commits on Aug 27, 2024

  1. Use single answer file and model list 

    Since phased training is using individual model names for judgment, it's hitting the case where multiple models are being loaded for evaluation.  We don't currently need that code path as it was inherited from FastChat so this change removes it and relies on the single answerfile and its model type.

Signed-off-by: Dan McPherson <[email protected]>
    @danmcp
    danmcp committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    deb8695 View commit details
    Browse the repository at this point in the history

  2. Fix e2e by removing old option 

    Signed-off-by: Dan McPherson <[email protected]>
    @danmcp
    danmcp committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    39b6960 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #110 from danmcp/singleanswerfile 

    Use single answer file and model list
    @danmcp
    danmcp authored Aug 27, 2024
    Configuration menu
    Copy the full SHA
    ff54038 View commit details
    Browse the repository at this point in the history
Loading