What happens
On Intel CPUs with AMX-BF16, for example 4th Gen Xeon Scalable processors, the BF16 matrix multiplications that PyTorch sends to oneDNN run on AMX instructions. Those instructions treat denormal inputs as zero and flush denormal outputs to zero, whatever the flush-to-zero and denormals-are-zero settings say, so torch.set_flush_denormal(False) does not change it.
PyTorch only sends a BF16 matrix multiplication to oneDNN when m × k × n is larger than 16 × 16 × 16. Smaller products run in PyTorch's own CPU kernel, which keeps denormals by default. So on the same machine, the same tiny inputs can give a non-zero answer in a small matrix multiplication and zero in a larger one.
How the tester sees it
The data pattern probe feeds the matrix multiply subnormal inputs on purpose and records whether the device keeps or flushes them, so a flushing device is reported as exactly that, and never as a silent error.
Where it stands
It is reported to PyTorch as issue #200531, labelled triaged, with a documentation change ready that adds a section to the numerical accuracy notes and a note to the set_flush_denormal docstring.