v1.0.0

Release Notes

• New metric structure is completed.
  • Custom metric support is improved and no longer required to extend datasets.Metric, rather uses jury.metrics.Metric.
  • Metric usage is unified with compute, preprocess and postprocess functions, which the only required implementation for custom metric is compute.
  • Both string and Metric objects can be passed to Jury(metrics=metrics) now in a mixed fashion.
  • load_metric function was rearranged to capture end score results and several metrics added accordingly (e.g. load_metric("squad_f1") will load squad metric which returns F1-score).
• Example notebook has added to example.
  • MT and QA tasks were illustrated.
  • Custom metric creation added as example.

Acknowledgments

@fcakyon @cemilcengiz @devrimcavusoglu