Attention Mechanisms in Dynamical Systems: A Case Study with Predator-Prey Models

The paper’s central claim is that, after training a linear softmax attention layer on noisy Lotka–Volterra (LV) trajectories via an MSE objective that multiplies each observation by its attention weight, the time index with highest attention coincides with the minimum of the normal derivative of a Lyapunov(-type) function V along the (so-called) limit cycle, and the lowest attention coincides with the maximum; this is stated as an “exact” correspondence and presented as robust (Result section and Fig. 5) , with code computing the “normal derivative” by dotting ∇V with the rotated tangent Jf (outward normal) and V, ∇V given explicitly (Eq. (1)–(2)) . However, for LV one has the exact identity Jf = xy ∇V, so the “normal derivative” ∂_n V equals ||∇V|| along any nontrivial closed orbit (hence strictly positive and varying); it is not learned by the training objective used. In fact, the paper’s loss encourages concentrating softmax weight on a single observation with large Euclidean norm of the state, not on a point minimizing ||∇V||. The candidate solution correctly identifies this mismatch, supplies the LV geometric identities, explains why the softmax-plus-MSE objective collapses to a support point of a linear functional on the sampled convex curve, and provides a concrete counterexample where the learned high-attention index is not a minimizer of ∂_n V. The paper also contains conceptual/confounding issues (e.g., repeatedly calling LV closed orbits a “limit cycle”) that undercut its claims . Therefore, the model’s critique is correct and the paper’s main claim is wrong as stated.