Breaking Changes

• LangCheck v0.3.0 supports and requires openai>=1
• The required function parameters for OpenAI-based metrics have changed. Please review the documentation to see the new API!

New Features

• Launched langcheck.augment with customizable English text augmentations to test robustness of LLMs! Check out the documentation to learn more.
  • langcheck.augment.rephrase()
  • langcheck.augment.synonym()
  • langcheck.augment.gender()
  • langcheck.augment.keyboard_typo()
  • langcheck.augment.ocr_typo()
  • langcheck.augment.remove_punctuation()
  • langcheck.augment.change_case()
• Improved the quality of explanations and speed of the OpenAI-based factual_consistency() metric
• More deterministic evaluations for OpenAI-based metrics
• Added progress bar when computing metrics – thanks @ThanhChinhBK!
• Added benchmarking for Japanese factual consistency metrics.

Bug Fixes

• Fixed "can't find Rust compiler" bug during installation for Python 3.10 by removing detoxify as a dependency and running their model directly (detoxify is unmaintained and pinned an old version of the tokenizers library)

Full Changelog: v0.2.0...v0.3.0