Speaker：Huiming Zhang (Beihang University)

Time：2026-1-11 16:30

Location：Conference Room S106 at Experiment Building at Haiyun Campus

Abstract:

Classical information-theoretic generalization bounds, which link generalization error to the mutual information between the algorithm's input and output, typically rely on sub-Gaussian assumptions or finite moment generating functions(MGFs). However, these assumptions are often violated in heavy-tailed scenarios, such as adversarial training, reinforcement learning with rare high-reward events, and financial modeling. In this work, we bridge this gap by establishing a comprehensive framework for generalization under heavy-tailed sub-Weibull regimes. We demonstrate that standard K-L divergence bounds are vacuous in these settings due to the unboundedness of extreme events. To overcome this, we introduce a novel decorrelation lemma based on Rényi divergence and a generalized Young-type inequality, which circumvents the need for MGFs. By combining these tools with a refined chaining technique on the space of measures, we derive Dudley-type generalization bounds that explicitly depend on the tail parameter and the Rényi information. Additionally, we establish new maximal inequalities and information-theoretic generalization bounds under sub-Weibullity of loss of data in machine learning. The work also explores the application of these results to large language models (LLM): providing tail-adaptive reward guarantees for Reinforcement Learning from Human Feedback in LLM alignment (mitigating catastrophic Goodhart effects where KL-regularization fails).