Darwin

LEXam-hard, all 518 open questions; single sample (temperature=1.0, top_p=0.95, top_k=20); thinking budget 32,768 tokens, responses truncated at 32K regenerated with a 120K budget (60 items); judged by DeepSeek-R1-0528 with the LEXam paper judge prompt (eval.yaml); score = mean of German and English mean grades x 100 (de 44.37, en 47.07); 1 item without a parsable grade counted as 0; bf16

Official MDPBench code and prompt, public set 2,720 pages (text edit distance, formula CDM, table TEDS). vLLM FP8, temperature 0, thinking on, 32,768 max tokens; pages with an empty thinking-mode answer re-read with thinking off (17 pages). Single run.

HMMT February 2026, all 33 problems; majority vote over 16 samples (maj@16) = 100.0; mean accuracy over 16 samples = 96.59; temperature=1.0, top_p=0.95, top_k=20; thinking budget 131,072 tokens; bf16

AIME 2026, all 30 problems; majority vote over 16 samples (maj@16) = 100.0; mean accuracy over 16 samples = 98.75; temperature=1.0, top_p=0.95, top_k=20; thinking budget 131,072 tokens; bf16

LEXam mcq_4_choices, all 1,655 items; majority vote over 4 samples (temperature=1.0, top_p=0.95, top_k=20); thinking budget 32,768 tokens; bf16. Single-sample 60.54, mean of 4 samples 61.42

Standard inference, Pass@1

4B class Gen-2 evolution, Pass@1

GPQA Diamond, all 198 items; greedy decoding (temperature=0), single sample (no voting / no test-time engine); weights released in FP8 (W8A8, compressed-tensors)

Standard inference, Pass@1

Standard inference, Pass@1

DELPHI cascade system, Pass@1

Darwin-DELPHI test-time engine, Pass@1

greedy decoding (temperature=0), single-sample (no voting / no test-time engine), max_tokens=16384, options shuffled seed=42; hardware: NVIDIA B200 x6 (TP2 x PP3), vLLM bfloat16

9B class, standard inference, Pass@1

General, zero-shot: never trained on the Typed Decisions train split, its workflows or its question schemas. Test split, 400 cases, 2,000 decisions, one request per case with the state and all five questions (README request shape), all answered, zero errors. Accuracy = agreement with the gold label; KL = KL(gold || prediction); Brier summed over options (scorer reproduces the README Uniform reference). One forward pass per question, no generated tokens.

Pipeline name: darwin_180b_rsi_r3_bf16_vllm_extract_oneshot_structured_output_file_nothink (vllm_extract provider, max_tokens 32768, temperature 0, json_object output, thinking disabled with chat_template_kwargs enable_thinking=false); served checkpoint: FINAL-Bench/Darwin-180B-RSI-R3 on vLLM 0.29.0 with online FP8 quantization, tensor parallel 2. Single run, 370 of 370 documents completed.