Optimum Probability Distribution for Minimum Redundancy of Source Coding

Abstract

In the present communication, we have obtained the optimum probability distribution with which the messages should be delivered so that the average redundancy of the source is minimized. Here, we have taken the case of various generalized mean codeword lengths. Moreover, the upper bound to these codeword lengths has been found for the case of Huffman encoding.

Cite this paper

O. Parkash and P. Kakkar, "Optimum Probability Distribution for Minimum Redundancy of Source Coding,"*Applied Mathematics*, Vol. 5 No. 1, 2014, pp. 96-105. doi: 10.4236/am.2014.51011.

O. Parkash and P. Kakkar, "Optimum Probability Distribution for Minimum Redundancy of Source Coding,"

References

[1] C. E. Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal, Vol. 27, 1948, pp. 379-423 (Part I) 623-656 (Part II).

[2] L. G. Kraft, “A Device for Quantizing Grouping and Coding Amplitude Modulated Pulses,” M.S. Thesis, MIT, Cambridge, 1949.

[3] L. L. Campbell, “A Coding Theorem and Renyi’s Entropy,” Information and Control, Vol. 8, No. 4, 1965, pp. 423-429.

[4] J. N. Kapur, “Entropy and Coding,” Mathematical Sciences Trust Society (MSTS), New Delhi, 1998.

[5] A. Renyi, “On Measures of Entropy and Information,” Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 1961, pp. 547-561.

[6] O. Parkash and P. Kakkar, “Development of Two New Mean Codeword Lengths,” Information Sciences, Vol. 207, 2012, pp. 90-97.

[7] D. Harte, “Multifractals: Theory and Applications,” Chapman and Hall, London, 2001.

[8] J. F. Bercher, “Source Coding with Escort Distributions and Renyi Entropy Bounds,” Physics Letters A, Vol. 373, No. 36, 2009, 3235-3238.

[9] J. F. Bercher, “Tsallis Distribution as a Standard Maximum Entropy Solution with ‘Tail’ Constraint,” Physics Letters A, Vol. 372, No. 35, 2008, pp. 5657-5659.

[10] C. Beck and F. Schloegl, “Thermodynamics of Chaotic Systems,” Cambridge University Press, Cambridge, 1993.

[11] D. A. Huffman, “A Method for the Construction of Minimum Redundancy Codes,” Proceedings of the Institute of Radio Engineers, Vol. 40, No. 10, 1952, pp. 1098-1101.