ICA  Vol.3 No.4 , November 2012
A Reward Functional to Solve the Replacement Problem
Abstract: The replacement problem can be modeled as a finite, irreducible, homogeneous Markov Chain. In our proposal the problem was modeled using a Markov decision process and then, the instance was optimized using dynamic programming. We proposed a new functional that includes a reward functional, that can be more helpful in processing industries because it considerate instances like incomes, maintenance costs, fixed costs to replace equipment, purchase price and salvage values; and this functional can be solved with dynamic programming and used to make effective decisions. Two theorems are proved related with this new functional. A numerical example is presented in order to demonstrate the utility of this proposal.
Cite this paper: E. Gress, O. Arango, J. Armenta and A. Reyes, "A Reward Functional to Solve the Replacement Problem," Intelligent Control and Automation, Vol. 3 No. 4, 2012, pp. 413-418. doi: 10.4236/ica.2012.34045.

[1]   A. Kristensen, “Textbook Notes of Herd Management: Dynamic Programming and Markov Process,” Dina Notat, 1996.

[2]   R. A. Howard, “Dynamic Programming and Markov Processes,” The MIT Press, Cambridge, 1960.

[3]   E. L. Sernik and S. Marcus, “Optimal Cost and Policy for Markovian Replacement Problem,” Journal of Optimization Theory and Applications, Vol. 71, No. 1, 1991, pp. 105-126. doi:10.1007/BF00940042

[4]   S. P. Sethi, G. Sorger and X. Zhou, “Stability of RealTime Lot Scheduling and Machine Replacement Policies with Quality Levels,” IEEE Transactions on Automatic Control, Vol. 45, No. 11, 2000, pp. 2193-2196. doi:10.1109/9.887687

[5]   J. D. Sherwin and B. Al-Najjar, “Practical Models for Conditions-Based Monitoring Inspection Intervals,” Journal of Quality in Maintenance Engineering, Vol. 5, No. 3, 1999, pp. 203-209. doi:10.1108/13552519910282665

[6]   E. Lewis, “Introduction to Reliability Theory,” Wiley, Singapore City, 1987.

[7]   S. Childress and P. Durango-Cohen, “On Parallel Machine Replacement Problems with General Replacement Cost Functions and Stochastic Deterioration,” Naval Research Logistics, Vol. 52, No. 5, 1999, pp. 402-419.

[8]   T. Cheng, “Optimal Replacement of Ageing Equipment Using Geometric Programming,” International Journal of Production Research, Vol. 30, No. 9, 1999, pp. 2151-2158. doi:10.1080/00207549208948142

[9]   N. Karabakal, C. Bean and J. Lohman, “Solving Large Replacement Problems with Budget Constraints,” The Engineering Economist, Vol. 4, No. 5, 2000, pp. 290-308. doi:10.1080/00137910008967554

[10]   T. Dohi, et al., “A Simulation Study on the Discounted Cost Distribution under Age Replacement Policy,” IEMS, Vol. 3, No. 2, 2004, pp. 134-139.

[11]   R. Bellman, “Equipment Replacement Policy,” Journal of Society for Industrial and Applied Mathematics, Vol. 3, No. 3, 1955, pp. 133-136.

[12]   D. White, “Dynamic Programming,” Holden Day, San Francisco, 1969.

[13]   D. Davidson, “An Overhaul Policy for Deteriorating Equipment,” In: A. Jardine, Ed., Operational Research in Maintenance, Manchester University Press, New York, 1970, pp. 1-24.

[14]   J. Walker, “Graphical Analysis for Machine Replacement,” International Journal of Operations and Production Management, Vol. 14, No. 10, 1992, pp. 54-63. doi:10.1108/01443579410067252

[15]   D. Bertsekas, “Dynamic Programming and Optimal Control,” Athena Scientific, Belmont, 2000.

[16]   L. Plá, C. Pomar and J. Pomar, “A Markow Sow Model Representing the Productive Lifespan of Herd Sows,” Agricultural Systems, Vol. 76, No. 1, 2004, pp. 253-272.

[17]   L. Nielsen and A. Kristensen, “Finding the K Best Policies in a Finite-Horizon Markov Decision Process,” European Journal of Operation Research, Vol. 175, No. 2, 2006 pp. 1164-1179. doi:10.1016/j.ejor.2005.06.011

[18]   L. Nielsen, et al., “Optimal Replacement Policies for Dairy Cows Based on Daily Yield Measurements,” Journal of Dairy Science, Vol. 93, No. 1, 2009, pp. 75-92. doi:10.3168/jds.2009-2209

[19]   F. Hillier and G. Lieberman, “Introduction to Operations Research,” Mc. Graw Hill Companies, Boston, 2002.

[20]   P. Schrijner and E. Doorn, “The Deviation Matrix of a Continuous-Time Markov Chain,” Probability in the Engineering and Informational Science, Vol. 16, No. 3, 2009, pp. 351-366.

[21]   C. Meyer, “Sensitivity of the Stationary Distribution of a Markov Chain,” SIAM Journal of Matrix Applications, Vol. 15, No. 3, 1994, pp. 715-728. doi:10.1137/S0895479892228900

[22]   M. Abbad, J. Filar and T. R. Bielecki, “Algorithms for Singularly Perturbed Markov Chain,” Proceedings of the 29th IEEE Conference, Honolulu, 5-7 December 1990, pp. 1402-1407.

[23]   E. Feinberg, “Constrained Discounted Decision Processes and Hamiltonian Cycles,” Mathematical Operational Research, Vol. 25, No. 1, 2000, pp. 130-140. doi:10.1287/moor.

[24]   S. Ross, “Applied Probability Models with Optimization Applications,” Holden Day, San Francisco, 1992.