The target of this paper is to discuss the Kullback-Leibler divergence for the family of γ-order Generalized Normal distribution. It has emerged from the Euclidean Logarithmic Sobolev Inequality and due to an “extra” shape parameter is an extension for the multivariate Normal distribution. In particular, for different g-orders and variances, the corresponding divergences are evaluated. The evaluated results coincide with the existing ones, in the case of γ = 2, the classical Normal case.

γ-order Generalized Normal distribution, Kullback-Leibler divergence, entropy power.

Received: November 1, 2024; Accepted: December 10, 2024; Published: February 12, 2025

How to cite this article: Christos P. Kitsos and Ioannis S. Stamatiou, Kullback-Leibler divergence for the γ-order generalized normal  Far East Journal of Theoretical Statistics 69(1) (2025), 55‑79. https://doi.org/10.17654/0972086325003

This Open Access Article is Licensed under Creative Commons Attribution 4.0 International License

References:
[1] S. M. Ali and S. D. Silvey, A general class of coefficients of divergence of one distribution from another, Journal of the Royal Statistical Society: Series B 28(1) (1966), 131-142.
[2] T. W. Anderson, An Introduction to Multivariate Statistical Analysis, Wiley- Interscience, 3rd ed., 2003.
[3] T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley, New York, 2nd ed., 2006.
[4] A. Dytso, R. Bustin, H. V. Poor and S. Shamai, Analytical properties of generalized Gaussian distributions, J. Stat. Distrib. Appl. 5(1) (2018), 1-40.
[5] A. Etz, Technical notes on Kullback-Leibler divergence, PsyArXiv, 2019. https://doi.org/10.31234/osf.io/5vhzu.
[6] B. A. Frigyik, S. Srivastava and M. R. Gupta, Functional Bregman divergence and Bayesian estimation of distributions, IEEE Trans. Inform. Theory 54(11) (2008), 5130-5139.
[7] L. Gross, Logarithmic Sobolev inequalities, Amer. J. Math. 97(4) (1975), 1061-1083.
[8] G. E. Halkos and D. C. Kitsou, Weighted location differential tax in environmental problems, Environ. Econ. Policy Stud. 20 (2018), 1-15.
[9] H. Jeffreys, An invariant form for the prior probability in estimation problems, Proc. of the Royal Society of London, The Royal Society 186 (1946), 453-461.
[10] S. Karlin and H. M. Taylor, A First Course in Stochastic Processes, Academic Press, New York, 2nd ed., 1975.
[11] C. P. Kitsos, Generalizing the heat equation, REVSTAT, 2023. https://revstat.ine.pt/index.php/REVSTAT/article/view/517.
[12] C. P. Kitsos and I. S. Stamatiou, The -order generalized Chi-square distribution Research in Statistics 2(1) (2024), 1-9.
[13] C. P. Kitsos and I. S. Stamatiou, Laplace transformation for the -order generalized normal Far East Journal of Theoretical Statistics 68(1) (2024), 1-21.
[14] C. P. Kitsos and N. K. Tavoularis, Logarithmic Sobolev inequalities for information measures, IEEE Trans. Inform. Theory 55(6) (2009), 2554-2561.
[15] C. P. Kitsos and T. L. Toulias, New information measures for the generalized normal distribution, Information 1 (2010), 13-27.
[16] C. P. Kitsos and T. L. Toulias, On the family of the -ordered normal distributions, Far East Journal of Theoretical Statistics 35(2) (2011), 95-114.
[17] C. P. Kitsos and T. L. Toulias, Hellinger distance between generalized normal distributions, British Journal of Mathematics and Computer Science 21(2) (2017), 1-16.
[18] S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Statist. 22(1) (1951), 79-86.
[19] M. D. Pino, J. Dolbeault and I. Gentil, Nonlinear diffusions, hypercontractivity and the optimal -Euclidean logarithmic Sobolev inequality, J. Math. Anal. Appl. 293(2) (2004), 375-388.
[20] A. Pluchino, A. Rapisarda and C. Tsallis, Nonergodicity and central-limit time behavior for long-range Hamiltonians, EPL 80(26002) (2007), 1-6.
[21] A. Rényi, On measures of entropy and information, J. Neyman, ed., Berkeley Symp. on Math. Statist. and Prob., Vol. 4.1, 1961, pp. 547-561.
[22] T. Royen, Expansions of the multivariate chi-square distribution, J. Multivariate Anal. 38(2) (1991), 213-232.
[23] E. Schrödinger, Zur theorie der fall-und steigversuche an teilchen mit brownsher bewegung, Physikalische Zeitschrift 16 (1915), 289-295.
[24] S. Tiwary, Time evolution of entropy, in various scenarios, Eur. J. Phys. 41(2) (2020), 1-11.
[25] Y. L. Tong, The multivariate normal distribution, Springer Series in Statistics, Springer New York, 1st ed., 1990.
[26] T. L. Toulias and C. P. Kitsos, Generalizations of entropy and information measures, N. J. Daras and M. Th. Rassias, eds., Computation, Cryptography, and Network Security, Springer, 2015, pp. 493-524.
[27] F. B. Weissler, Logarithmic Sobolev inequalities for the heat-diffusion semigroup, Trans. Amer. Math. Soc. 237 (1978), 255-269.
[28] S. Yu, A. Zhang and H. Li, A review of estimating the shape parameter of generalized Gaussian distribution, Journal of Computational Information Systems 8(21) (2012), 9055-9064.