In case of an epidemic arising from an infectious disease for which an effective vaccine is available, the desired policy is to increase the number of people who will choose this vaccine   . One of the common ways to analyze this issue is to study the potential motivation of individuals to choose the vaccination in a theoretic-game framework. In   , the authors presented a Nash vaccinating game played by cohorts of parents of babies who consider whether or not to vaccinate their offspring against pediatric diseases such as measles, mumps, rubella, polio, etc. The game proposed is played among a finite number of groups of parents, where parents in a group are considered to share common perceptions of the risk of vaccinating their offspring, as well as the risk of not treating. The game was further extended into a generalized Nash game in  and placed in a dynamic context in  . In  the authors studied a game with a mechanism reflecting the possibility of increasing the vaccine supplies for seasonal influenza in developing countries. The epidemiological diseases like influenza have extensively been studied by many researchers in the literature. In  a lab-computer simulation is used to examine the uptake of the influenza vaccine depending on age-dependent decision-making. Their study showed via a theoretical-game experiment that vaccinating young people is a more effective immunization policy against influenza than vaccinating the elderly. The authors in  have addressed a model that characterizes the feedback between the word-of-mouth and voluntary vaccination on the dynamics of an epidemic. The vaccination decision-making process has been presented in  with a new strategy to incentivize the vaccination and the treatment using the social network structure of the population. The suggested strategy is to subsidize both the vaccine and the medical treatment in order to motivate individuals to vaccinate. Relevant to the application of games in decision-making problems, a theoretical study of the operation of a supply chain with a single manufacturer and single retailer under uncertainty of information can be found in  . Along the same lines, the work in  pursues the notion of a supply chain under uncertain information in a game model of a competition between two heterogeneous suppliers working with the same retailer. In energy investment markets,  presents a model of a coalition game that characterizes a contract-based scheme for electricity trading between the aggregator who buys the electric power (in small-scale) from the main power source, and sales it to consumers.
The origin of game theory can be traced back to the works of Von Neumann   . The works of Nash  form the basis of classic 2-player noncooperative games. Since then, numerous works have been advanced in this area. Some of the most recent ones are for instance:  , studying 2-player zero-sum game with random payoff matrix to investigate the connections of the best payoff of 2 players under joint chance constraints; and  who have discussed a 2-person zero-sum game with infinite action space and bounded payoff functions. In symmetric games, the idea of finding a solution for a competitive contest between two players or more is strongly related to the concept of Nash equilibrium (NE). In nonzero asymmetric games, the Nash-Pareto equilibrium concept is required to improve the strategy of one of the players without making the opponent’s strategy worse  . The existence of Pareto equilibrium was discussed in  , in a game with multicriteria payoffs, neglecting the linear and convexity structure, in general, topological spaces. Pareto-efficient optimization is addressed in  to explore the significance of players’ cognition in the game. The authors introduced a new definition for Nash game with the observance of players’ beliefs. Consequently, optimizing the players’ belief can forthright the game to the Pareto-efficient equilibrium.
One way to incentivize players to act in a certain way is to consider the incentive as a parameter in the players’ payoffs. Exerting control on this parameter leads us to consider the game from an optimal control problem’s perspective. Starting in the 50’s, optimal control was the key to studying variational problems. The main contribution in this field was made by (Pontryagin, 1964) when he theorized the maximum principle theorem  . The optimal control problem has been extensively studied by many authors over the years. For instance,  and  discussed optimal control problems with necessary conditions. However,  and  looked at optimal control problems with sufficient conditions. The recent century has witnessed contributions by many scientists in enriching the concepts and literature of theoretic games and control problems. For example,  used a straightforward model where certain several suppositions will let integrate evolutionary games into the drug-dosage formulation as an optimal control problem. In  built a novel model to optimize cancer chemotherapy by studying it as an evolutionary game between cancer and the oncologist. In our most recent work, we applied optimal control to a replicator dynamics system of a norms game in a population, and we showed that the otherwise evolutionarily stable state of all defectors can be controlled towards a norm-dominated population  .
In this paper, we study and formulate an asymmetric game as in  , where the players have different levels of income, different probabilities of infection and differing costs of vaccinating. The suggested control is used as an incentive to encourage the players toward vaccination and away from a stable Nash-Pareto pair of strategies which has a cost depending probabilities of vaccinating for both player types. We find the optimal control values and the optimal strategies with the objective of maximizing the sum of the players who choose to vaccinate numerically.
The structure of this paper is presented as follows: In section 0 we present our formulation of the vaccination game inspired by  , where we implement a modification on its payoffs. Further, we find the game’s Nash equilibria and their stability characteristics after we formulate the game in an associated replicator dynamics form. We show that there exists on Nash-Pareto stable pair of mix strategies for the game. In Section 3 we incorporate the replicator dynamics equations into an optimal control problem where we seek to decrease the perceived costs of vaccinating in both groups. We show that this exogenous control results in gains in overall expected vaccine coverage in the population.
2. The Formulation of the Vaccination Game Model
We consider a game similar to the one in  , but where the pathogen is transmitted only via humans. The game considers two players in different positions where each player has two pure strategies: to vaccinate (V), or not vaccinate (NV). A player who decides to vaccinate must pay some “cost”, denoted by , which here reflects an amalgamation of psychological cost, time or monetary cost, and cost of side effects (if developed). The probability of getting infected, denoted by q, will cause a player to incur some “loss”, denoted by if the infection develops. Thus we consider the following asymmetric bimatrix game:
where is the income of Player i. This game has a bimatrix payoff, s.t. we can write the sub-matrices for each player in the game as follows:
If we consider this game in mixed strategies, denoted by and with being the probabilities of Player 1, respectively Player 2, to vaccinate, we write the expected payoff functions for each player as follows:
which imply further:
To solve the game in (1) we use the reaction curves method so we rewrite the expected payoffs in (3) as linear functions of their corresponding variables s.t. for we have:
To maximize and in (4) we need to find their stationary points:
The derivative of depend on the parameter y; therefore, the sign of depends on y to determine the growth in ,
Since is a linear function of y, then is achieved for
But implies that .
Same analysis can be made for and we get that is achieved for:
But then by follow the same analysis as above, we get: .
Thus, we have an equilibrium when & . The Nash equilibrium in mixed strategies can be now calculated from finding all points of x and y so that the equilibria are (Figure 1):
2.1. Replicator Dynamics and Nash-Pareto Pairs
Let us think now of our game as a game with two types of players, differentiated by income and vaccination costs, each with two pure strategies: . Then we think of the mixed strategy , with , as the fraction of population of type 1, who chooses pure strategy i. Similarly,
Figure 1. The blue curve shows the maximization of w.r.t the coordinates of . The orange curve shows the maximization of w.r.t the coordinates of . The point is the intersection of the two colored curves.
, with , represents the fraction of population of type 2 who chooses pure strategy .
It is known (see  ) that the replicator dynamics (RD) associated with our game is:
To simplify our study of the replicator dynamics associated with our game, we first agree to denote the pure strategies V, NV by indexes 1, respectively 2, for each player. Then we rescale (as in  , Chapter 11) our game (1) into a bimatrix game with the same Nash equilibria computed in the previous section, using the following definitions:
Definition 2.1 1) A game is a rescaling of a bimatrix game , denoted by , if there exist constants and so that
2) If , then the bimatrix game is called a g-zero-sum game (with ) if there exist suitable so that
It is known that Nash equilibrium points of a rescaled game are the same as those of the original bimatrix game. We show below that our bimatrix game can be rescaled to a g-zero-sum-game.
Proposition 1 The bimatrix game in (1) is a g-zero-sum-game for any .
Proof. We show first that there exists , a rescaling of as defined in (14), which simplifies the players’ matrices as an intermediary step. Then we show that is a g-zero-sum-game using (15). In (14) let us take . Then we take and ; respectively we take: and . So we have that
In this way the relations (14) can be written as:
We now show that the bimatrix game , i.e., that there exist so that
We solve this linear system and we find the following:
For we have:
Using these values in (15) of Definition 0.1, the bimatrix game is a (−1)-zero-sum-game:
Using now (16) and (14) we can conclude that is a g-zero-sum-game, since we found and so that
Hence the original bimatrix game (1) is a (−1)-zero-sum-game:
Nash-Pareto is a relaxation of the idea of evolutionarily stable strategy (ESS) for asymmetric games so that we can include mixed strategies. Suppose that our two subpopulations (of player of type 1, respectively of player of type 2) are in a state where . This state will not be stable in an evolutionary sense if there exists a neighboring state, say , such that both types can increase their mean payoff by deviating to . We then define the following:
Definition 2.2 Let be a state for our population in an asymmetric game with payoff matrices . Then is a Nash-Pareto pair for this game if the following two conditions hold:
2) For all states for which equality holds in condition a) above, we have,
if , and
The following result is known (see  ).
Theorem 2.1 If the pair is in the interior of the set , then is a Nash-Pareto pair of the bimatrix iff is a rescaled zero-sum-game. Moreover, the Nash-Pareto point is stable in the replicator dynamics (13).
Using Proposition 1 and Theorem 2.1 above we have that is indeed a Nash-Pareto point of our bimatrix game and is stable for the dynamics (13).
2.2. Nash-Pareto Equilibria and Evolutionary Stability
Let be a set of mixed vaccinating strategies for the players in our game so that and/or . Then, the reduced replicator dynamics associated with the dynamics (13) and to the asymmetric bimatrix game is given by (see  ):
Putting all computations together, we have the following explicit RD:
It is immediate to see that all three Nash equilibria found in (10) are critical points of the RD (18).
According to the analysis in  , we can show next that our Nash-Pareto pair in the reduced RD (18) dynamics is not asymptotically stable, as whenever and we automatically get , and not negative. This is good in the applied context of our game, as we started our analysis with the goal of increasing vaccine coverage in our population, which means we wish to increase the probabilities of vaccinating for both types of players. While the mixed strategy point is stable, we can next ask what can be done to nudge the population away from this state. In the next section, we use an optimal control problem associated with the reduced RD (18) where the control parameters are the vaccination costs .
3. Optimal Control of Vaccination Away from the Nash-Pareto State
We formulate the problem of controlling cost in the system (18) as an optimal control problem that is considered by Yosida in  . Moreover, we are going to follow the same formulation as in  to build up a classical optimal control problem where we want to minimize the following functional problem:
where is the Lagrangian function defined as , and ; the function is a lower-semi continuous function defined as:
And the set . Problem W is formulated subject to the following conditions:
where and . The control  .
In our case, we define the control such that . The control will reduce the cost of vaccination in our game (1) and in our RD system (18) as follows:
Thus, we study the following problem (for details see  ):
where , , , , and .
The proof of existence and uniqueness of the optimal solution for the problem can be found in  . We solve the problem numerically by using the steepest descent method to determine the first optimality conditions to (for more details see  and  ).
In all our simulations below, we consider so that . Also, we fix below the values of to be the same.
In Figure 2 (upper panel), we present the Nash-Pareto pairs as functions of the perceived costs , while in Figure 2 (lower panel) we present the optimally controlled strategies and for , as functions of the initial group costs and , which were evolved, correspondingly, from the initial values of . We allow both for all and we find that the variation in costs has a big impact onto the probability of getting vaccinated in each group, as well as on the expected coverage in the population, assuming both groups of players (of type 1 and of type 2) are equally sized.
We see that after time periods, we get that for initial values of costs: , the groups probabilities of vaccinating have changed from the starting Nash-Pareto pair of to the pair . Though group 2’s probability of vaccinating has slightly decreased, group 1’s has increased. This is an advantage when we look at these values from the point of view of the expected coverage in the population, which is roughly estimated as (without considering time lags between vaccination and vaccine uptake):
where is the fraction of population made up of players of type i for .
Figure 2. In Figure 2 (upper panel), we present the Nash-Pareto pairs as functions of the perceived costs , while in Figure 2 (lower panel) we present the optimally controlled strategies and for , as functions of the initial group costs and .
For an illustration, we look at the case where . In this case, we present in Figure 3 the expected overall coverages and . We can clearly see that for and , the value of , thus the optimal control applied in this case, with equal population fractions, results in higher expected coverage for the vaccine. In this case, the application of the control over a number of 6 time periods (years) resulted in a net growth of expected vaccine coverage of approximately 0.12% (see Figure 3).
Figure 3. Expected overall coverages and with equal size groups. The application of the control over a number of 6 time periods (years) resulted in a net growth of expected vaccine coverage of approximately 0.12%.
The comparison of coverage levels can, in fact, be conducted for any pair of in our analysis. The most desired optimally controlled states are those where the control leads to better coverage than the Nash-Pareto case. Moreover, our analysis can be easily expanded to two more cases: 1) players of type 1 are a majority (recall these are players with income level ; since , then we have a population where the lower income group is a majority); 2) players of type 1 are a minority.
We present the difference these proportions make on the discussion of our results from the perspective of the overall vaccine coverage. First, let us consider that and . In this case, we see (Figure 4) that the best coverage after applying the control results in a level of 83.47% for which is higher than the corresponding Nash-Pareto coverage level for which has a level of 75.38%. In this case, the application of the control over the same number of years resulted in a net growth of expected vaccine coverage of approximately 0.09%, and this is directly due to the population makeup and groups sizes (check Figure 4). Last but not least, we
Figure 4. Expected overall coverages and with and . The application of the control over a number of 6 time periods resulted in a net growth of expected vaccine coverage of approximately 0.09%.
present our case when and . We see from Figure 5 that the best coverage after applying the control results in a level of 78.32% for which is higher than the corresponding Nash-Pareto coverage level for which has a level of 75.38%. In this case, the application of the control over the same number of years resulted in a net growth of expected vaccine coverage of approximately 0.03% (see Figure 5).
In this paper we presented a 2-player asymmetric bimatrix game with two pure strategies, vaccinating or non-vaccinating, against a potentially infectious disease. We transformed and associated with our game a replicator dynamics system whose only mix strategy state is a Nash-Pareto stable pair. This means that over time, this mixed strategy of vaccination is likely to endure. From a public health perspective, the time-stability of the mixed Nash-Pareto pair is not desirable, as it is hoped that vaccine coverage can be increased in a population.
We showed that by introducing an exogenous control to decrease the perceived costs of vaccinating to all individuals in the population, an increase in the expected vaccine coverage can be achieved. We also showed that the net growth in the expected coverage depends on the population mix of players of type 1 and type 2, which is to be expected, as their vaccinating strategies are different.
Our game can be easily generalized in at least two directions: first, we can
Figure 5. Expected overall coverages and with and . The application of the control over a number of 6 time periods resulted in a net growth of expected vaccine coverage of approximately 0.03%.
consider more than 2 groups in our population makeup and we can consider these groups differentiated not by income, but by age, cost ( ) and loss ( ). Second, we can introduce differing mechanisms for cost control. Our work here presents the basics of how such a multiplayer vaccination game can be analyzed and controlled, with the scope of gaining net increases in overall expected vaccine coverage levels.
The first author acknowledges the support of the National Sciences and Engineering Research Council (NSERC) of Canada through the Discovery Grant #400684.
 Cojocaru, M.-G., Bauch, C.T. and Johnston, M.D. (2007) Dynamics of Vaccination Strategies via Projected Dynamical Systems. Bulletin of Mathematical Biology, 69, 1453-1476.
 Mamani, H., Chick, S.E. and Simchi-Levi, D. (2013) A Game-Theoretic Model of International Influenza Vaccination Coordination. Management Science, 59, 1650-1670.
 Chapman, G.B., Li, M., Vietri, J., Ibuka, Y., Thomas, D., Yoon, H. and Galvani, A.P. (2012) Using Game Theory to Examine Incentives in Influenza Vaccination Behavior. Psychological Science, 23, 1008-1015.
 Bhattacharyya, S., Bauch, C.T. and Breban, R. (2015) Role of Word-of-Mouth for Programs of Voluntary Vaccination: A Game-Theoretic Approach. Mathematical Biosciences, 269, 130-134.
 Li, Q., Li, M., Lv, L., Guo, C. and Lu, K. (2017) A New Prediction Model of Infectious Diseases with Vaccination Strategies Based on Evolutionary Game Theory, Chaos, Solitons & Fractals, 104, 51-60.
 Chen, L., Peng, J., Liu, Z.B. and Zhao, R.Q. (2017) Pricing and Effort Decisions for a Supply Chain with Uncertain Information. International Journal of Production Research, 55, 264-284.
 Liu, Z., Zhao, R., Liu, X. and Chen, L. (2017) Contract Designing for a Supply Chain with Uncertain Information Based on Confidence Level. Applied Soft Computing, 56, 617-631.
 Li, Z., Chen, L. and Nan, G. (2017) Small-Scale Renewable Energy Source Trading: A Contract Theory Approach. IEEE Transactions on Industrial Informatics, 14, 1491-1500.
 Cheng, J., Leung, J. and Lisser, A. (2016) Random-Payoff Two-Person Zero-Sum Game with Joint Chance Constraints. European Journal of Operational Research, 252, 213-219.
 Lu, H.S. (2008) The Existence of Pareto Equilibrium for Multiobjective Games in General Topological Spaces. International Conference on Intelligent Computation Technology and Automation, Hunan, 20-22 October 2008, Vol. 2, 776-780.
 Ren, J., Wong, K.K. and Hou, J. (2013) A Forward-Looking Nash Game and Its Application to Achieving Pareto-Efficient Optimization. Applied Mathematics, 4, 1609-1615.
 Jacobson, D.H., Lele, M.M. and Speyer, J.L. (1971) New Necessary Conditions of Optimality for Control Problems with State-Variable Inequality Constraints. Journal of Mathematical Analysis and Applications, 35, 255-284.
 de Pinho, M.D.R., Vinter, R.B. and Bertozzi, L. (1997) Necessary Conditions for Optimal Control Problems Involving Nonlinear Differential Algebraic Equations. Journal of Mathematical Analysis and Applications, 212, 493-516.
 Iwasaki, T. and Skelton, R.E. (1994) All Controllers for the General H Control Problem: LMI Existence Conditions and State Space Formulas. Automatica, 30, 1307-1317.
 Bewick, S., Yang, R. and Zhang, M. (2009) Embedding Evolutionary Game Theory into an Optimal Control Framework for Drug Dosage Design. Annual International Conference on Engineering in Medicine and Biology Society, Minneapolis, 3-6 September 2009, 6026-6029.
 Orlando, P.A., Gatenby, R.A. and Brown, J.S. (2012) Cancer Treatment as a Game: Integrating Evolutionary Game Theory into the Optimal Control of Chemotherapy. Physical Biology, 9, Article ID: 065007.
 Lasiecka, I. and Triggiani, R. (1983) Regularity of Hyperbolic Equations under L2 (0, T; L2 (T))-Dirichlet Boundary Terms. Applied Mathematics & Optimization, 10, 275-286.