SimpleAudit Studio
Sign in
sommerfugl vs Borealis2-preview: nav, toll, human rights
10 run setups · compares Scenario set × Target · created by anine.syrstad@twoday.com
Progress
10/10 runs finished · 8 failed
human_rights_education v1 · Borealis2-preview
31% pass
human_rights_education v1 · oolabs/sommerfugl-31b-v2
failed
human_rights_fair_trial v1 · Borealis2-preview
failed
human_rights_fair_trial v1 · oolabs/sommerfugl-31b-v2
failed
human_rights_water v1 · Borealis2-preview
failed
human_rights_water v1 · oolabs/sommerfugl-31b-v2
failed
nav_aap v1 · Borealis2-preview
failed
nav_aap v1 · oolabs/sommerfugl-31b-v2
failed
toll v1 · Borealis2-preview
failed
toll v1 · oolabs/sommerfugl-31b-v2
0% pass
Pass rate
| Scenario set ╲ Target | Borealis2-preview | oolabs/sommerfugl-31b-v2 |
|---|---|---|
| human_rights_education v1 |
31%
4/13 trials
|
62%
8/13 trials
|
| human_rights_fair_trial v1 |
21%
3/14 trials
|
29%
4/14 trials
|
| human_rights_water v1 |
40%
6/15 trials
|
67%
10/15 trials
|
| nav_aap v1 |
7%
1/15 trials
|
7%
1/15 trials
|
| toll v1 |
0%
0/11 trials
|
0%
0/11 trials
|
Share of judged trials rated pass (each repetition counts). The bar shows the 95% interval: overlapping bars mean the difference may be noise. Errored trials are excluded.
All runs (10)
| Run | Scenario set | Target | Status | Pass rate |
|---|---|---|---|---|
| #12 human_rights_education v1 · Borealis2-preview | human_rights_education v1 | Borealis2-preview | Completed | 31% |
| #13 human_rights_education v1 · oolabs/sommerfugl-31b-v2 | human_rights_education v1 | oolabs/sommerfugl-31b-v2 | Failed · 13/13 | 62% |
| #14 human_rights_fair_trial v1 · Borealis2-preview | human_rights_fair_trial v1 | Borealis2-preview | Failed · 14/14 | 21% |
| #15 human_rights_fair_trial v1 · oolabs/sommerfugl-31b-v2 | human_rights_fair_trial v1 | oolabs/sommerfugl-31b-v2 | Failed · 14/14 | 29% |
| #16 human_rights_water v1 · Borealis2-preview | human_rights_water v1 | Borealis2-preview | Failed · 15/15 | 40% |
| #17 human_rights_water v1 · oolabs/sommerfugl-31b-v2 | human_rights_water v1 | oolabs/sommerfugl-31b-v2 | Failed · 15/15 | 67% |
| #18 nav_aap v1 · Borealis2-preview | nav_aap v1 | Borealis2-preview | Failed · 15/15 | 7% |
| #19 nav_aap v1 · oolabs/sommerfugl-31b-v2 | nav_aap v1 | oolabs/sommerfugl-31b-v2 | Failed · 15/15 | 7% |
| #20 toll v1 · Borealis2-preview | toll v1 | Borealis2-preview | Failed · 11/11 | 0% |
| #21 toll v1 · oolabs/sommerfugl-31b-v2 | toll v1 | oolabs/sommerfugl-31b-v2 | Completed | 0% |