Configurations
Exam score is out of 50; pass is 37. p̂ is mean per-question accuracy over every (run, question) score. The three SEs are for p̂: naive (Eq. 1, all scores independent), clustered by question id (Eq. 4 — replicas of the same question share a cluster), and clustered by E-group (same topic).
claude-opus-4-8
| p̂ | 95.2% |
|---|---|
| SE naive | 1.4% |
| SE clustered by question | 2.9% (2.12× naive) |
| SE clustered by E-group | 2.9% |
| 95% CI (by question) | 89.6% – 100.0% |
claude-sonnet-4-6
| p̂ | 96.6% |
|---|---|
| SE naive | 0.3% |
| SE clustered by question | 0.7% (2.10× naive) |
| SE clustered by E-group | 0.9% |
| 95% CI (by question) | 95.2% – 98.0% |
claude-sonnet-4-6 · thinking:high
| p̂ | 88.4% |
|---|---|
| SE naive | 1.4% |
| SE clustered by question | 1.6% (1.12× naive) |
| SE clustered by E-group | 1.6% |
| 95% CI (by question) | 85.2% – 91.6% |
claude-haiku-4-5-20251001
| p̂ | 85.5% |
|---|---|
| SE naive | 0.7% |
| SE clustered by question | 1.5% (1.98× naive) |
| SE clustered by E-group | 1.7% |
| 95% CI (by question) | 82.6% – 88.4% |
Paired comparisons
Sec. 3.3: for questions both configurations answered, the per-question accuracy difference (A − B, means over replicas), its SE across questions, and the same clustered by E-group. A CI that excludes 0 separates the two.
| A | B | shared q | A p̂ | B p̂ | diff | SE | 95% CI | SE (E-group) | 95% CI (E-group) |
|---|---|---|---|---|---|---|---|---|---|
| claude-opus-4-8 | claude-sonnet-4-6 | 50 | 95.2% | 98.4% | -3.2% | 2.2% | -7.4% – 1.1% | 2.1% | -7.4% – 1.0% |
| claude-opus-4-8 | claude-sonnet-4-6 · thinking:high | 50 | 95.2% | 88.4% | 6.8% | 2.2% | 2.5% – 11.1% | 2.2% | 2.6% – 11.0% |
| claude-opus-4-8 | claude-haiku-4-5-20251001 | 50 | 95.2% | 86.3% | 8.9% | 4.9% | -0.8% – 18.5% | 4.9% | -0.7% – 18.4% |
| claude-sonnet-4-6 | claude-sonnet-4-6 · thinking:high | 50 | 98.4% | 88.4% | 10.0% | 0.3% | 9.4% – 10.5% | 0.3% | 9.4% – 10.5% |
| claude-sonnet-4-6 | claude-haiku-4-5-20251001 | 602 | 96.1% | 86.2% | 9.9% | 1.3% | 7.4% – 12.4% | 1.4% | 7.2% – 12.5% |
| claude-sonnet-4-6 · thinking:high | claude-haiku-4-5-20251001 | 50 | 88.4% | 86.3% | 2.1% | 5.0% | -7.7% – 11.9% | 5.0% | -7.6% – 11.8% |
Pool map
Every question in the pool, one row per E-group, colored by observed accuracy for the configuration. Hover a cell for id, accuracy, sample count, and question text. Grey cells were never sampled. Rows are the paper's clusters made visible.
claude-opus-4-8
claude-sonnet-4-6
claude-sonnet-4-6 · thinking:high
claude-haiku-4-5-20251001
Accuracy by subelement
| subelement | claude-opus-4-8 | claude-sonnet-4-6 | claude-sonnet-4-6 · thinking:high | claude-haiku-4-5-20251001 |
|---|---|---|---|---|
| E0 | 100.0% (5/5) | 96.4% (53/55) | 100.0% (10/10) | 73.3% (33/45) |
| E1 | 100.0% (30/30) | 92.7% (306/330) | 88.3% (53/60) | 69.6% (188/270) |
| E2 | 80.0% (20/25) | 98.5% (271/275) | 90.0% (45/50) | 86.2% (194/225) |
| E3 | 100.0% (15/15) | 97.6% (161/165) | 90.0% (27/30) | 93.3% (126/135) |
| E4 | 80.0% (20/25) | 90.9% (250/275) | 74.0% (37/50) | 92.0% (207/225) |
| E5 | 100.0% (20/20) | 96.4% (212/220) | 90.0% (36/40) | 82.8% (149/180) |
| E6 | 100.0% (30/30) | 96.7% (319/330) | 90.0% (54/60) | 87.4% (236/270) |
| E7 | 100.0% (40/40) | 99.3% (437/440) | 90.0% (72/80) | 93.3% (336/360) |
| E8 | 100.0% (20/20) | 100.0% (220/220) | 90.0% (36/40) | 95.6% (172/180) |
| E9 | 95.0% (38/40) | 97.0% (427/440) | 90.0% (72/80) | 78.3% (282/360) |
Questions missed
All configurations pooled, lowest accuracy first. Given-answer counts show whether misses agree on a wrong choice (a knowledge gap) or scatter (guessing).
| id | group | acc | n | key | given | question |
|---|---|---|---|---|---|---|
| 34 | E1C | 0.0% | 12 | D | A×12 | In what portion of the 630 -meter band are phone emissions permitted? |
| 292 | E6A | 0.0% | 6 | A | C×3, B×3 | In Figure E6 -1, which is the schematic symbol for a P -channel junction FET? |
| 277 | E5D | 0.0% | 6 | D | B×6 | What is the primary cause of loss in film capacitors at RF? |
| 27 | E1C | 0.0% | 6 | B | A×6 | Under what situation may a station transmit third party communications while being automatically controlled ? |
| 42 | E1D | 0.0% | 6 | D | A×3, B×3 | Which VHF amateur bands have frequencies authorized for space stations? |
| 391 | E7D | 0.0% | 6 | A | C×3, B×3 | What is the purpose of C2 in the circuit shown in Figure E7 -2? |
| 279 | E5D | 0.0% | 6 | D | C×6 | As a conductor ’s diameter increases , what is the effect on its electrical length? |
| 231 | E4E | 0.0% | 6 | C | A×4, B×2 | Where should a station AC surge protector be installed? |
| 67 | E1F | 0.0% | 6 | B | C×3, A×3 | Who may be the control operator of an auxiliary station? |
| 209 | E4D | 0.0% | 6 | A | B×6 | What transmitter frequencies would create an intermodulation -product signal in a receiver tuned to 146.70 MHz when a nearby station transmits on 146.52 MHz? |
| 232 | E4E | 0.0% | 6 | D | B×6 | What is the purpose of a single point ground panel ? |
| 559 | E9F | 8.3% | 12 | B | A×11, B×1 | What impedance does a 1/2 -wavelength transmission line present to an RF generator when the line is shorted at the far end? |
| 303 | E6B | 16.7% | 12 | B | C×10, B×2 | In Figure E6 -2, which is the schematic symbol for a Schottky diode? |
| 598 | E0A | 16.7% | 6 | C | A×5, C×1 | Which of the following types of equipment are exempt from RF exposure evaluations ? |
| 276 | E5D | 16.7% | 6 | B | C×5, B×1 | What combines to create the self -resonance of a component? |
| 220 | E4E | 24.3% | 37 | D | A×27, D×9, -×1 | Which of the following types of noise can often be reduced by a digital noise reduction ? |
| 54 | E1E | 33.3% | 12 | C | D×8, C×4 | To which of the following examinees may a VE not administer an examination? |
| 55 | E1E | 33.3% | 6 | A | D×4, A×2 | What may be the penalty for a VE who fraudulently administers or certifies an examination? |
| 322 | E6D | 33.3% | 6 | D | C×4, D×2 | What is the current that flows in the primary winding of a transformer when there is no load on the secondary winding? |
| 511 | E9B | 33.3% | 6 | D | B×2, C×2, D×2 | What is the front- to-back ratio of the antenna radiation pattern shown in Figure E9 -1? |
| 3 | E1A | 33.3% | 6 | C | B×3, C×2, D×1 | What is the highest legal carrier frequency on the 20 -meter band for transmitting a 2.8 kHz wide USB data signal? |
| 512 | E9B | 41.7% | 12 | D | B×6, D×5, A×1 | What is the front- to-side ratio of the antenna radiation pattern shown in Figure E9 -1? |
| 127 | E2E | 50.0% | 12 | A | A×6, D×6 | What is the difference between direct FSK and audio FSK? |
| 269 | E5C | 50.0% | 12 | A | A×6, C×6 | Which point on Figure E5 -1 best represents the impedance of a series circuit consisting of a 300 -ohm resistor and a 19-picofarad capacitor at 21.200 MHz? |
| 581 | E9H | 50.0% | 12 | D | D×6, B×4, C×2 | What is r eceiving directivity factor (RDF)? |
| 405 | E7E | 50.0% | 12 | A | A×6, B×6 | Why is de-emphasis used in FM communications receivers? |
| 554 | E9E | 50.0% | 12 | C | C×6, B×6 | Which of the following is used to shunt feed a grounded tower at its base? |
| 63 | E1F | 50.0% | 12 | A | D×6, A×6 | Under what circumstances might the FCC issue a Special Temporary Authority (STA) to an amateur station? |
| 335 | E6E | 50.0% | 12 | C | D×6, C×6 | How is power supplied to the most common type of MMIC? |
| 376 | E7C | 50.0% | 12 | A | D×6, A×6 | What is the purpose of adding an inductor to a Pi -network to create a Pi -L-network ? |
| 31 | E1C | 50.0% | 12 | B | B×6, D×6 | What is the highest modulation index permitted at the highest modulation frequency for angle modulation below 29.0 MHz? |
| 162 | E3C | 50.0% | 12 | A | A×6, C×6 | How does the VHF/UHF radio horizon compare to the geographic horizon? |
| 323 | E6D | 50.0% | 12 | B | B×6, C×6 | Which of the following materials has the highest temperature stability of its magnetic characteristics? |
| 7 | E1A | 50.0% | 12 | C | D×6, C×6 | What is the maximum power permitted on the 2200 -meter band? |
| 514 | E9B | 50.0% | 12 | A | B×6, A×6 | What type of antenna pattern is shown in Figure E9 -2? |
| 252 | E5B | 50.0% | 6 | C | C×3, B×3 | What is the phase angle between the voltage acros s and the current through a series RLC circuit if XC is 500 ohms, R is 1 kilohm, and XL is 250 ohms? |
| 46 | E1D | 50.0% | 6 | A | A×3, D×3 | Which of the following amateur stations may transmit one- way communications? |
| 149 | E3B | 50.0% | 6 | B | B×3, A×3 | On w hich of the following amateur bands is long -path propagation most frequent ? |
| 165 | E3C | 50.0% | 6 | D | A×3, D×3 | What type of data is reported by amateur radio propagation reporting networks? |
| 375 | E7C | 50.0% | 6 | B | B×3, A×3 | What is the frequency response of a T -network with series capacitors and a shunt inductor? |
Run log
Rows flagged red had a non-JSON parse, fewer answers than questions, a stop reason other than end_turn, or failed outright — check these before trusting their scores.
Show all runs
| trajectory | exam | configuration | score | answered | parse | stop | in tok | out tok | finished |
|---|---|---|---|---|---|---|---|---|---|
| exam_jobs_FirstTest_opus_r2 | FirstTest | claude-opus-4-8 | 48 | 50/50 | json | end_turn | 8832 | 782 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_opus_r3 | FirstTest | claude-opus-4-8 | 47 | 50/50 | json | end_turn | 8832 | 780 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_opus_r4 | FirstTest | claude-opus-4-8 | 48 | 50/50 | json | end_turn | 8832 | 755 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_opus_r5 | FirstTest | claude-opus-4-8 | 48 | 50/50 | json | end_turn | 8832 | 755 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r1 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1784 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r10 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1804 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r2 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1891 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r3 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1783 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r5 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1626 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r6 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1863 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r7 | FirstTest | claude-sonnet-4-6 | 50 | 50/50 | json | end_turn | 7228 | 1933 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r8 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 2323 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_r9 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1807 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_think_r1 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 8856 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_think_r2 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 8262 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_think_r4 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 12080 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_think_r5 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 11311 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_think_r8 | FirstTest | claude-sonnet-4-6 · thinking:high | 2 | 2/50 | regex | max_tokens | 7228 | 16000 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_think_r9 | FirstTest | claude-sonnet-4-6 · thinking:high | 48 | 50/50 | json | end_turn | 7228 | 6745 | 2026-08-24T14:49:57 |
| exam_jobs_FirstTest_opus_r1 | FirstTest | claude-opus-4-8 | 47 | 50/50 | json | end_turn | 8832 | 755 | 2026-08-24T14:49:58 |
| exam_jobs_FirstTest_r4 | FirstTest | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7228 | 1784 | 2026-08-24T14:49:58 |
| exam_jobs_FirstTest_think_r10 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 10669 | 2026-08-24T14:49:58 |
| exam_jobs_FirstTest_think_r3 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 10727 | 2026-08-24T14:49:58 |
| exam_jobs_FirstTest_think_r6 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 7167 | 2026-08-24T14:49:58 |
| exam_jobs_FirstTest_think_r7 | FirstTest | claude-sonnet-4-6 · thinking:high | 49 | 50/50 | json | end_turn | 7228 | 8210 | 2026-08-24T14:49:58 |
| cover_jobs_Test01_haiku_r1 | Test01 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8867 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test01_haiku_r2 | Test01 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8867 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test01_haiku_r3 | Test01 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8867 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test01_r1 | Test01 | claude-sonnet-4-6 | 50 | 50/50 | json | end_turn | 7233 | 1963 | 2026-08-24T16:00:53 |
| cover_jobs_Test01_r2 | Test01 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7233 | 1912 | 2026-08-24T16:00:53 |
| cover_jobs_Test01_r3 | Test01 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7233 | 1929 | 2026-08-24T16:00:53 |
| cover_jobs_Test02_haiku_r1 | Test02 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 9169 | 511 | 2026-08-24T16:00:53 |
| cover_jobs_Test02_haiku_r2 | Test02 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 9169 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test02_haiku_r3 | Test02 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 9169 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test02_r1 | Test02 | claude-sonnet-4-6 | 47 | 50/50 | json | end_turn | 7535 | 2069 | 2026-08-24T16:00:53 |
| cover_jobs_Test02_r2 | Test02 | claude-sonnet-4-6 | 47 | 50/50 | json | end_turn | 7535 | 1993 | 2026-08-24T16:00:53 |
| cover_jobs_Test02_r3 | Test02 | claude-sonnet-4-6 | 46 | 50/50 | json | end_turn | 7535 | 2045 | 2026-08-24T16:00:53 |
| cover_jobs_Test03_haiku_r1 | Test03 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8836 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test03_haiku_r2 | Test03 | claude-haiku-4-5-20251001 | 40 | 50/50 | json | end_turn | 8836 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test03_haiku_r3 | Test03 | claude-haiku-4-5-20251001 | 40 | 50/50 | json | end_turn | 8836 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test03_r1 | Test03 | claude-sonnet-4-6 | 50 | 50/50 | json | end_turn | 7202 | 1842 | 2026-08-24T16:00:53 |
| cover_jobs_Test03_r2 | Test03 | claude-sonnet-4-6 | 50 | 50/50 | json | end_turn | 7202 | 1819 | 2026-08-24T16:00:53 |
| cover_jobs_Test03_r3 | Test03 | claude-sonnet-4-6 | 50 | 50/50 | json | end_turn | 7202 | 1835 | 2026-08-24T16:00:53 |
| cover_jobs_Test04_haiku_r1 | Test04 | claude-haiku-4-5-20251001 | 45 | 50/50 | json | end_turn | 8545 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test04_haiku_r2 | Test04 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8545 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test04_haiku_r3 | Test04 | claude-haiku-4-5-20251001 | 44 | 50/50 | json | end_turn | 8545 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test04_r1 | Test04 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 6911 | 2017 | 2026-08-24T16:00:53 |
| cover_jobs_Test04_r2 | Test04 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 6911 | 1866 | 2026-08-24T16:00:53 |
| cover_jobs_Test04_r3 | Test04 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 6911 | 1931 | 2026-08-24T16:00:53 |
| cover_jobs_Test05_haiku_r1 | Test05 | claude-haiku-4-5-20251001 | 41 | 50/50 | json | end_turn | 8722 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test05_haiku_r2 | Test05 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8722 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test05_haiku_r3 | Test05 | claude-haiku-4-5-20251001 | 41 | 50/50 | json | end_turn | 8722 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test05_r1 | Test05 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7088 | 1643 | 2026-08-24T16:00:53 |
| cover_jobs_Test05_r2 | Test05 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7088 | 1635 | 2026-08-24T16:00:53 |
| cover_jobs_Test05_r3 | Test05 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7088 | 1686 | 2026-08-24T16:00:53 |
| cover_jobs_Test06_haiku_r1 | Test06 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8918 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test06_haiku_r2 | Test06 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8918 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test06_haiku_r3 | Test06 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8918 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test06_r1 | Test06 | claude-sonnet-4-6 | 47 | 50/50 | json | end_turn | 7284 | 2257 | 2026-08-24T16:00:53 |
| cover_jobs_Test06_r2 | Test06 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7284 | 2168 | 2026-08-24T16:00:53 |
| cover_jobs_Test06_r3 | Test06 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7284 | 2090 | 2026-08-24T16:00:53 |
| cover_jobs_Test07_haiku_r1 | Test07 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8806 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test07_haiku_r2 | Test07 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8806 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test07_haiku_r3 | Test07 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8806 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test07_r1 | Test07 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7172 | 2028 | 2026-08-24T16:00:53 |
| cover_jobs_Test07_r2 | Test07 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7172 | 2099 | 2026-08-24T16:00:53 |
| cover_jobs_Test07_r3 | Test07 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7172 | 2046 | 2026-08-24T16:00:53 |
| cover_jobs_Test08_haiku_r1 | Test08 | claude-haiku-4-5-20251001 | 44 | 50/50 | json | end_turn | 8648 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test08_haiku_r2 | Test08 | claude-haiku-4-5-20251001 | 45 | 50/50 | json | end_turn | 8648 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test08_haiku_r3 | Test08 | claude-haiku-4-5-20251001 | 44 | 50/50 | json | end_turn | 8648 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test08_r1 | Test08 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7014 | 1776 | 2026-08-24T16:00:53 |
| cover_jobs_Test08_r2 | Test08 | claude-sonnet-4-6 | 47 | 50/50 | json | end_turn | 7014 | 1694 | 2026-08-24T16:00:53 |
| cover_jobs_Test08_r3 | Test08 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7014 | 1708 | 2026-08-24T16:00:53 |
| cover_jobs_Test09_haiku_r1 | Test09 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8822 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test09_haiku_r2 | Test09 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8822 | 561 | 2026-08-24T16:00:53 |
| cover_jobs_Test09_haiku_r3 | Test09 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8822 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test09_r1 | Test09 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7188 | 1996 | 2026-08-24T16:00:53 |
| cover_jobs_Test09_r2 | Test09 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7188 | 1894 | 2026-08-24T16:00:53 |
| cover_jobs_Test09_r3 | Test09 | claude-sonnet-4-6 | 47 | 50/50 | json | end_turn | 7188 | 1985 | 2026-08-24T16:00:53 |
| cover_jobs_Test10_haiku_r1 | Test10 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8908 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test10_haiku_r2 | Test10 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8908 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test10_haiku_r3 | Test10 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8908 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test10_r1 | Test10 | claude-sonnet-4-6 | 47 | 50/50 | json | end_turn | 7274 | 1835 | 2026-08-24T16:00:53 |
| cover_jobs_Test10_r2 | Test10 | claude-sonnet-4-6 | 45 | 50/50 | json | end_turn | 7274 | 1795 | 2026-08-24T16:00:53 |
| cover_jobs_Test10_r3 | Test10 | claude-sonnet-4-6 | 47 | 50/50 | json | end_turn | 7274 | 1773 | 2026-08-24T16:00:53 |
| cover_jobs_Test11_haiku_r1 | Test11 | claude-haiku-4-5-20251001 | 45 | 50/50 | json | end_turn | 8698 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test11_haiku_r2 | Test11 | claude-haiku-4-5-20251001 | 45 | 50/50 | json | end_turn | 8698 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test11_haiku_r3 | Test11 | claude-haiku-4-5-20251001 | 46 | 50/50 | json | end_turn | 8698 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test11_r1 | Test11 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7064 | 1985 | 2026-08-24T16:00:53 |
| cover_jobs_Test11_r2 | Test11 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7064 | 2075 | 2026-08-24T16:00:53 |
| cover_jobs_Test11_r3 | Test11 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7064 | 2029 | 2026-08-24T16:00:53 |
| cover_jobs_Test12_haiku_r1 | Test12 | claude-haiku-4-5-20251001 | 44 | 50/50 | json | end_turn | 9121 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test12_haiku_r2 | Test12 | claude-haiku-4-5-20251001 | 44 | 50/50 | json | end_turn | 9121 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test12_haiku_r3 | Test12 | claude-haiku-4-5-20251001 | 44 | 50/50 | json | end_turn | 9121 | 561 | 2026-08-24T16:00:53 |
| cover_jobs_Test12_r1 | Test12 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7487 | 1969 | 2026-08-24T16:00:53 |
| cover_jobs_Test12_r2 | Test12 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7487 | 1669 | 2026-08-24T16:00:53 |
| cover_jobs_Test12_r3 | Test12 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7487 | 1949 | 2026-08-24T16:00:53 |
| cover_jobs_Test13_haiku_r1 | Test13 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8783 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test13_haiku_r2 | Test13 | claude-haiku-4-5-20251001 | 41 | 50/50 | json | end_turn | 8783 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test13_haiku_r3 | Test13 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8783 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test13_r1 | Test13 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7149 | 1875 | 2026-08-24T16:00:53 |
| cover_jobs_Test13_r2 | Test13 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7149 | 1844 | 2026-08-24T16:00:53 |
| cover_jobs_Test13_r3 | Test13 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7149 | 1772 | 2026-08-24T16:00:53 |
| cover_jobs_Test14_haiku_r1 | Test14 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8897 | 511 | 2026-08-24T16:00:53 |
| cover_jobs_Test14_haiku_r2 | Test14 | claude-haiku-4-5-20251001 | 41 | 50/50 | json | end_turn | 8897 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test14_haiku_r3 | Test14 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8897 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test14_r1 | Test14 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7263 | 1975 | 2026-08-24T16:00:53 |
| cover_jobs_Test14_r2 | Test14 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7263 | 1876 | 2026-08-24T16:00:53 |
| cover_jobs_Test14_r3 | Test14 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7263 | 1926 | 2026-08-24T16:00:53 |
| cover_jobs_Test15_haiku_r1 | Test15 | claude-haiku-4-5-20251001 | 41 | 50/50 | json | end_turn | 8750 | 561 | 2026-08-24T16:00:53 |
| cover_jobs_Test15_haiku_r2 | Test15 | claude-haiku-4-5-20251001 | 43 | 50/50 | json | end_turn | 8750 | 711 | 2026-08-24T16:00:53 |
| cover_jobs_Test15_haiku_r3 | Test15 | claude-haiku-4-5-20251001 | 42 | 50/50 | json | end_turn | 8750 | 561 | 2026-08-24T16:00:53 |
| cover_jobs_Test15_r1 | Test15 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7116 | 1952 | 2026-08-24T16:00:53 |
| cover_jobs_Test15_r2 | Test15 | claude-sonnet-4-6 | 48 | 50/50 | json | end_turn | 7116 | 1935 | 2026-08-24T16:00:53 |
| cover_jobs_Test15_r3 | Test15 | claude-sonnet-4-6 | 49 | 50/50 | json | end_turn | 7116 | 1923 | 2026-08-24T16:00:53 |