On the Limitations of Fractal Dimension as a Measure of Generalization

The paper estimates the PH-dimension of the full optimization trajectory WS by regressing log E0^1(Wn) versus log n, using 5,000 post-convergence iterates as a finite sample to build multiple subsamples Wn; it does not set dimPH0(xS) for the finite sample equal to the run’s PH-dimension. This is explicit in their Experimental Design and the summary of Birdal et al.’s estimator linking the slope m of log E0^1(Wn) to dimPH0(WS) ≈ α/(1−m) (with α=1) . The model’s core claim—that because any finite set has dimPH0=0, the run-level PH-dimensions are identically zero—misapplies the definition by replacing WS with a single finite sample xS; the paper, in contrast, follows the theoretical setup where dimPH0 is a property of WS and is estimated via scaling of E0^α(Wn) over n (not taken as dimPH0(xS)) . Empirically, the paper finds: (i) PH dimensions correlate more weakly with generalization gap than simpler measures like the final parameter norm and the LR/BS ratio, per Table 1 and Fisher z-tests ; (ii) much apparent correlation disappears after conditioning on learning rate via partial correlations and conditional mutual information tests, depending on dataset and batch size ; and (iii) PH magnitudes exhibit a model-wise double descent pattern with width, aligned more with test accuracy than with the accuracy gap . These results contradict the model’s asserted “impossibility” of hyperparameter confounding and double descent under the stated measures. In short, the paper’s methodology and conclusions are consistent with the PH-dimension framework, whereas the model’s argument attacks a straw definition (dimPH0 of a finite sample) rather than the trajectory-level quantity actually used.