Dominika Długosz gives an oral presentation at EMNLP 2026

Dominika Długosz, PhD student at MLKD, presented the paper “The Importance of Being Statistically Earnest: A Critical Re-evaluation of GSM-Symbolic” as an oral presentation at EMNLP 2026 in Budapest on 26 October. The paper is joint work with Arlindo L. Oliveira and Natalia Díaz-Rodríguez.

Dominika Długosz at EMNLP 2026 in Budapest.

GSM-Symbolic is a benchmark whose authors reported that large language models score lower on reworded versions of school maths problems, and concluded that the models lack genuine reasoning ability. The paper argues that this conclusion rests on shaky statistical ground.

Re-evaluating 20 open-weight models with a more careful statistical analysis, the authors find that only 8 show a statistically significant change in performance. They also show that the benchmark's problems use systematically larger numbers than the original ones, which accounts for half of the remaining cases. The models that are affected fail in different, model-specific ways, so blanket claims about LLM reasoning risk being statistically premature.

Title slide of the talk.