Skip to content

Instantly share code, notes, and snippets.

@idoru

idoru/Mistral-7B-Instruct-v0.2-Lighteval.md

Created March 24, 2024 22:14
Show Gist options
  • Save idoru/fc9f8d5f53640b025c7cde4b449a57f3 to your computer and use it in GitHub Desktop.
Save idoru/fc9f8d5f53640b025c7cde4b449a57f3 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. idoru created this gist Mar 24, 2024.
    13 changes: 13 additions & 0 deletions Mistral-7B-Instruct-v0.2-Lighteval.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,13 @@
    ## Mistral-7B-Instruct-v0.2 - Lighteval

    | Task |Version| Metric |Value | |Stderr|
    |---------------------------|------:|--------------|-----:|---|-----:|
    |all | |qem |0.0000|± |0.0000|
    | | |truthfulqa_mc1|0.5288|± |0.0175|
    | | |truthfulqa_mc2|0.6827|± |0.0152|
    |leaderboard\|gsm8k\|0 | 0|qem |0.0000|± |0.0000|
    |leaderboard\|truthfulqa:mc\|0| 0|truthfulqa_mc1|0.5288|± |0.0175|
    | | |truthfulqa_mc2|0.6827|± |0.0152|


    Elapsed time: 00:29:48