IdentityChain v0.1.0

What's in v0.1.0?

1. Support HumanEval-Mini-vHumanEvalPlus-Mini-v0.1.10
2. Support the DeepSeek Coder Model Family in examples/run_identity_chain_huggingface.py
3. Provide Alternative FIM Prompt

Full Changelog: https://github.com/marcusm117/IdentityChain/commits/v0.1.0

What's next?

We plan to support more evaluation benchmarks, provide examples for more models, and extend the framework to more programming languages

IdentityChain v0.0.1

What's in v0.0.1?

You can create an IdentityChain object to evaluate the Self-Consistency, NL-to-PL Accuracy, and PL-to-NL Accuracy of your own Code LLMs on two benchmarks: HumanEvalPlus and MBPP.

Full Changelog: https://github.com/marcusm117/IdentityChain/commits/v0.0.1

What's next?

We plan to support more evaluation benchmarks and more programming languages