In this paper, we study an asymmetric game that characterizes the intentions of players to adopt a vaccine. The game describes a decision-making process of two players differentiated by income level and perceived treatment cost, who consider a vaccination against an infectious disease. The process is a noncooperative game since their vaccination decision has a direct impact on vaccine coverage in the population. We introduce a replicator dynamics (RD) to investigate the players’ optimal strategy selections over time. The dynamics reveal the long-term stability of the unique Nash-Pareto equilibrium strategy of this game, which is an extension of the notion of an evolutionarily stable strategy pair for asymmetric games. This Nash-Pareto pair is dependent on perceived costs to each player type, on perceived loss upon getting infected, and on the probability of getting infected from an infected person. Last but not least, we introduce a payoff parameter that plays the role of cost-incentive towards vaccination. We use an optimal control problem associated with the RD system to show that the Nash-Pareto pair can be controlled to evolve towards vaccination strategies that lead to a higher overall expected vaccine coverage.
In case of an epidemic arising from an infectious disease for which an effective vaccine is available, the desired policy is to increase the number of people who will choose this vaccine [
The origin of game theory can be traced back to the works of Von Neumann [
One way to incentivize players to act in a certain way is to consider the incentive as a parameter in the players’ payoffs. Exerting control on this parameter leads us to consider the game from an optimal control problem’s perspective. Starting in the 50’s, optimal control was the key to studying variational problems. The main contribution in this field was made by (Pontryagin, 1964) when he theorized the maximum principle theorem [
In this paper, we study and formulate an asymmetric game as in [
The structure of this paper is presented as follows: In section 0 we present our formulation of the vaccination game inspired by [
We consider a game similar to the one in [
Player 2 V N V Player 1 V N V ( ( I 1 − c 1 , I 2 − c 2 ) ( I 1 − c 1 , I 2 ) ( I 1 , I 2 − c 2 ) ( I 1 − q L 1 , I 2 − q L 2 ) ) (1)
where I i is the income of Player i. This game has a bimatrix payoff, s.t. we can write the sub-matrices for each player in the game as follows:
G 1 = [ I 1 − c 1 I 1 − c 1 I 1 I 1 − q L 1 ] , G 2 = [ I 2 − c 2 I 2 I 2 − c 2 I 2 − q L 2 ]
If we consider this game in mixed strategies, denoted by ( x ,1 − x ) and ( y ,1 − y ) with x , y ∈ [ 0,1 ] being the probabilities of Player 1, respectively Player 2, to vaccinate, we write the expected payoff functions for each player as follows:
E 1 ( x ,1 − x ) = [ x ,1 − x ] T G 1 [ y 1 − y ] , E 2 ( y ,1 − y ) = [ y ,1 − y ] T G 2 T [ x 1 − x ] , (2)
which imply further:
E 1 ( x , 1 − x ) = x q L 1 − x c 1 − x y q L 1 + I 1 − q L 1 + y q L 1 E 2 ( y , 1 − y ) = y q L 2 − y c 2 − y x q L 2 + I 2 − q L 2 + x q L 2 (3)
To solve the game in (1) we use the reaction curves method so we rewrite the expected payoffs in (3) as linear functions of their corresponding variables s.t. for E 1 we have:
E 1 ( x , 1 − x ) = x ( L 1 q − c 1 − y L 1 q ) + ( I 1 − L 1 q + y L 1 q ) (4)
E 2 ( y , 1 − y ) = y ( L 2 q − x L 2 q − c 2 ) + ( I 2 − L 2 q + x L 2 q ) (5)
To maximize E 1 and E 2 in (4) we need to find their stationary points:
d E 1 d x = L 1 q − y L 1 q − c 1 , with y ∈ [ 0,1 ] d E 2 d y = L 2 q − x L 2 q − c 2 , with x ∈ [ 0,1 ] (6)
The derivative of E 1 depend on the parameter y; therefore, the sign of d E 1 d x depends on y to determine the growth in E 1 ,
d E 1 d x = { > 0 , if y > ( 1 − c 1 L 1 q ) & y ∈ [ 0 , 1 ] = 0 , if y = ( 1 − c 1 L 1 q ) < 0 , if y < ( 1 − c 1 L 1 q ) & y ∈ [ 0 , 1 ] (7)
Since E 1 is a linear function of y, then max x ∈ [ 0 , 1 ] E 1 is achieved for
x * = { 0 , 1 ≥ y > 1 − c 1 L 1 q any x * x ∈ [ 0 , 1 ] , y = 1 − c 1 L 1 q 1 , 0 ≤ y < 1 − c 1 L 1 q (8)
But y = 1 − c 1 L 1 q ∈ [ 0 , 1 ] implies that L 1 q ≥ c 1 ≥ 0 .
Same analysis can be made for E 2 and we get that max y ∈ [ 0 , 1 ] E 2 is achieved for:
y * = { 0 , 1 ≥ x > 1 − c 2 L 2 q any y * ∈ [ 0 , 1 ] , x = 1 − c 2 L 2 q 1 , 0 ≤ x < 1 − c 2 L 2 q (9)
But x = 1 − c 2 L 2 q ∈ [ 0 , 1 ] then by follow the same analysis as above, we get: L 2 q ≥ c 2 ≥ 0 .
Thus, we have an equilibrium when L 1 q ≥ c 1 ≥ 0 & L 2 q ≥ c 2 ≥ 0 . The Nash equilibrium in mixed strategies can be now calculated from finding all points of x and y so that the equilibria are (
( x 1 * , y 1 * ) = ( x 1 * = 1 , y 1 * = 0 ) ⇒ ( x 1 * = 1 , ( 1 − x 1 * ) = 0 , ( y 1 * = 0 , ( 1 − y 1 * ) = 1 ) ) (10)
( x 2 * , y 2 * ) = ( x 2 * = 0 , y 2 * = 1 ) ⇒ ( x 2 * = 0 , ( 1 − x 2 * ) = 1 , ( y 2 * = 1 , ( 1 − y 2 * ) = 0 ) ) (11)
( x 3 * , y 3 * ) = ( 1 − c 1 L 1 q , 1 − c 2 L 2 q ) ⇒ ( ( 1 − c 1 L 1 q , c 1 L 1 q ) , ( 1 − c 2 L 2 q , c 2 L 2 q ) ) (12)
Let us think now of our game as a game with two types of players, differentiated by income and vaccination costs, each with two pure strategies: i ∈ { V , N V } . Then we think of the mixed strategy x = ( x V , x N V ) ∈ [ 0 , 1 ] 2 , with x V + x N V = 1 , as the fraction of population of type 1, who chooses pure strategy i. Similarly,
y = ( y V , y N V ) ∈ [ 0 , 1 ] 2 , with y V + y N V = 1 , represents the fraction of population of type 2 who chooses pure strategy j ∈ { V , N V } .
It is known (see [
d x i d t = x i [ ( G 1 y ) i − x T G 1 y ] ; i = { V , N V } . d y j d t = y j [ ( G 2 x ) j − y T G 2 x ] ; j = { V , N V } . (13)
To simplify our study of the replicator dynamics associated with our game, we first agree to denote the pure strategies V, NV by indexes 1, respectively 2, for each player. Then we rescale (as in [
Definition 2.1 1) A game ( A ′ , B ′ ) is a rescaling of a bimatrix game ( A , B ) , denoted by ( A , B ) ~ ( A ′ , B ′ ) , if there exist constants e j , d i and α > 0 , β > 0 so that
a ′ i j = α a i j + e j and b ′ j i = β b j i + d i , i = 1 , 2 ¯ , j = 1 , 2 ¯ . (14)
2) If ( A ′ , B ′ ) = ( C , − C t ) , then the bimatrix game ( A , B ) is called a g-zero-sum game (with γ < 0 ) if there exist suitable C i j , f j , h i so that
a i j = C i j + f j and b j i = γ C j i + h j (15)
It is known that Nash equilibrium points of a rescaled game are the same as those of the original bimatrix game. We show below that our bimatrix game can be rescaled to a g-zero-sum-game.
Proposition 1 The bimatrix game ( G 1 , G 2 ) in (1) is a g-zero-sum-game for any γ : = − 1 .
Proof. We show first that there exists ( G ^ 1 , G ^ 2 ) , a rescaling of ( G 1 , G 2 ) as defined in (14), which simplifies the players’ matrices as an intermediary step. Then we show that ( G ^ 1 , G ^ 2 ) is a g-zero-sum-game using (15). In (14) let us take α : = 1 , β : = 1 . Then we take e 1 : = c 1 − I 1 and e 2 : = q L 1 − I 1 ; respectively we take: d 1 : = c 2 − I 2 and d 2 : = q L 2 − I 2 . So we have that
( G 1 , G 2 ) ~ ( G ^ 1 , G ^ 2 ) where G ^ 1 = [ 0 − c 1 + L 1 q 2 c 1 0 ] and G ^ 2 = [ 0 − c 2 + L 2 q 1 c 2 0 ]
In this way the relations (14) can be written as:
g ^ 1 i j = g 1 i j + e j ⇒ g 1 i j = g ^ 1 i j − e j and g ^ 2 j i = g 2 j i + d i ⇒ g 2 j i = g ^ 2 j i − d i , i , j ∈ 1 , 2 (16)
We now show that the bimatrix game ( G ^ 1 , G ^ 2 ) ~ ( C , − C t ) , i.e., that there exist C i j , f j , h i so that
g ^ 1 i j = C i j + f j and g ^ 2 j i = γ C j i + h i
We solve this linear system and we find the following:
C = [ − c 2 − L 2 q γ 2 − c 1 γ c 2 γ − c 1 γ − c 2 − L 2 q γ − c 1 − L 2 q γ 0 ] and f 1 : = c 2 − L 2 q γ 2 + c 1 γ ,
f 2 : = c 1 − L 2 q γ 2 − c 1 + L 1 q + c 1 γ − c 2 γ , h 1 = c 1 + c 2 − L 2 q γ , h 2 = 0
For γ = − 1 we have:
C = [ − ( c 2 − L 2 q ) + c 1 − c 2 + c 1 + ( c 2 − L 2 q ) c 1 − L 2 q 0 ] and f 1 : = c 2 − L 2 q − c 1 ,
f 2 : = ( c 1 − L 2 q ) − c 1 + L 1 q − c 1 + c 2 , h 1 = c 1 − ( c 2 − L 2 q ) , h 2 = 0
Using these values in (15) of Definition 0.1, the bimatrix game ( G ^ 1 , G ^ 2 ) is a (−1)-zero-sum-game:
( G ^ 1 , G ^ 2 ) ~ ( C , − C t ) = [ ( c 1 − c 2 + L 2 q , − c 1 + c 2 − L 2 q ) ( c 1 − L 2 q , − c 1 + L 2 q ) ( c 1 − L 2 q , − c 1 + L 2 q ) ( 0,0 ) ]
Using now (16) and (14) we can conclude that ( G 1 , G 2 ) is a g-zero-sum-game, since we found C i j , f ˜ j , h ˜ i and γ = − 1 so that
g 1 i j = C i j + f ˜ j and g 2 j i = γ C j i + h ˜ i ,
where
f ˜ j : = f j − e j , j = 1 , 2 and h ˜ = h i − d i , i = 1 , 2
Hence the original bimatrix game (1) is a (−1)-zero-sum-game:
( G 1 , G 2 ) ~ ( C , − C t ) ⇔ ( G 1 , G 2 ) ~ [ ( c 1 − c 2 + L 2 q , − c 1 + c 2 − L 2 q ) ( c 1 − L 2 q , − c 1 + L 2 q ) ( c 1 − L 2 q , − c 1 + L 2 q ) ( 0,0 ) ]
Nash-Pareto is a relaxation of the idea of evolutionarily stable strategy (ESS) for asymmetric games so that we can include mixed strategies. Suppose that our two subpopulations (of player of type 1, respectively of player of type 2) are in a state ( p , q ) ∈ S 2 × S 2 where S 2 : = { w ∈ [ 0 , 1 ] 2 | w 1 + w 2 = 1 } . This state will not be stable in an evolutionary sense if there exists a neighboring state, say ( x , y ) , such that both types can increase their mean payoff by deviating to ( x , y ) . We then define the following:
Definition 2.2 Let ( p , q ) be a state for our population in an asymmetric game with payoff matrices ( G 1 , G 2 ) . Then ( p , q ) is a Nash-Pareto pair for this game if the following two conditions hold:
1) p ⋅ G 1 q ≥ x ⋅ G 1 q and q ⋅ G 2 p ≥ y ⋅ G 2 p , ∀ ( x , y ) ∈ S 2 × S 2
2) For all states ( x , y ) ∈ S 2 × S 2 for which equality holds in condition a) above, we have,
if x ⋅ G 1 y > p ⋅ G 1 y ⇒ y ⋅ G 2 x < q ⋅ G 2 x , and
if y ⋅ G 2 x > q ⋅ G 2 x ⇒ x ⋅ G 1 y < p ⋅ G 1 y .
The following result is known (see [
Theorem 2.1 If the pair ( x , y ) is in the interior of the set S 2 × S 2 , then ( x , y ) is a Nash-Pareto pair of the bimatrix ( G 1 , G 2 ) iff ( G 1 , G 2 ) is a rescaled zero-sum-game. Moreover, the Nash-Pareto point is stable in the replicator dynamics (13).
Using Proposition 1 and Theorem 2.1 above we have that ( x 3 * , y 3 * ) ∈ S 2 × S 2 is indeed a Nash-Pareto point of our bimatrix game and is stable for the dynamics (13).
Let ( x , y ) ∈ [ 0,1 ] × [ 0,1 ] be a set of mixed vaccinating strategies for the players in our game so that x ≠ x 3 * and/or y ≠ y 3 * . Then, the reduced replicator dynamics associated with the dynamics (13) and to the asymmetric bimatrix game is given by (see [
d x d t = x ( 1 − x ) ( P 2 − ( P 1 + P 2 ) y ) ; d y d t = y ( 1 − y ) ( P 4 − ( P 3 + P 4 ) x ) (17)
where:
P 1 = [ x 3 * − x , − x 3 * + x ] G 1 [ y 1 − y ] = ( x − x 3 * ) ( L 1 q y − L 1 q + c 1 )
P 2 = [ x − x 3 * x 3 * − x ] G 1 [ y 3 * 1 − y 3 * ] = ( x − x 3 * ) ( − L 1 q y 3 * + L 1 q − c 1 )
P 3 = [ y 3 * − y − y 3 * + y ] G 2 [ x 1 − x ] = ( y − y 3 * ) ( L 2 q x − L 2 q )
P 4 = [ y − y 3 * y 3 * − y ] G 2 [ x 3 * 1 − x 3 * ] = ( y − y 3 * ) ( − L 2 q x 3 * + L 2 q )
Putting all computations together, we have the following explicit RD:
d x d t = x ( 1 − x ) L 1 q ( x − x 3 * ) ( 1 − y 3 * − c 1 L 1 q − y ( y − y 3 * ) ) ; (18)
d y d t = y ( 1 − y ) L 2 q ( y − y 3 * ) ( 1 − x 3 − x ( x − x 3 * ) )
It is immediate to see that all three Nash equilibria found in (10) are critical points of the RD (18).
According to the analysis in [
We formulate the problem of controlling cost in the system (18) as an optimal control problem that is considered by Yosida in [
Ω : min Φ ( x , u ) = ∫ 0 T L ( t , x ( t ) , u ( t ) ) d t + l 0 ( x ( 0 ) ) + l 1 ( x ( T ) ) , (19)
where L ( x ( t ) , u ( t ) ) is the Lagrangian function defined as L : [ 0, T ] × R n × R m → R , and x ∈ R n , u ∈ R m ; the function l ( x ( 0 ) , x ( T ) ) is a lower-semi continuous function defined as:
l ( x ( 0 ) , x ( T ) ) = { l 0 ( x ( 0 ) ) + l 1 ( x ( T ) ) if x ( 0 ) ∈ C 0 & x ( T ) ∈ C 1 + ∞ otherwise (20)
And the set ( x , u ) ∈ A C ( [ 0, T ] ; R n ) × M ( 0, T ; R m ) . Problem W is formulated subject to the following conditions:
x ( t ) , u ( t ) satisfiyanO . E . Dsystem u ( t ) ∈ U ( t ) a . e . t ∈ ( 0, T ) x ( 0 ) ∈ C 0 , x ( T ) ∈ C 1 , (21)
where C 0 = { x ( 0 ) } ⊂ R n and C 2 = R n . The control u ( t ) ∈ U ( t ) ⊂ R m , a . e . t ∈ ( 0, T ) [
In our case, we define the control u : [ 0, T ] → [ 1,2 ] such that u ( t ) = [ u 1 ( t ) , u 2 ( t ) ] , t ∈ [ 0 , T ] . The control u ( t ) ∈ [ 1,2 ] will reduce the cost of vaccination in our game (1) and in our RD system (18) as follows:
c 1 : = 2 c 1 1 + u 1 ( t ) and c 2 : = 2 c 2 1 + u 2 ( t ) .
Thus, we study the following problem (for details see [
Ω * : min Φ ( x , y , u ) = − x ( T ) − y (T)
{ thesystem ( 18 ) and u ( t ) ∈ [ 1,2 ] , a . e . t ∈ ( 0, T ) ( x ( 0 ) , y ( 0 ) ) ∈ C 0 ( x ( T ) , y ( T ) ) ∈ C 2 , a . e . t ∈ ( 0, T ) , (22)
where n = m = 2 , L ≡ 0 , l 0 ( x ( 0 ) , y ( 0 ) ) = 0 , C 0 = { x 3 * , y 3 * } , C 1 = R 2 and U ( t ) = [ 1 , 2 ] .
The proof of existence and uniqueness of the optimal solution ( x * , y * , u * ) for the problem Ω * can be found in [
In all our simulations below, we consider 1 ≥ q ≥ m a x { 2 c 1 ,2 c 2 } so that ( x 3 * , y 3 * ) ∈ [ 0,1 ] 2 . Also, we fix below the values of L 1 = L 2 = 0.5 to be the same.
In
We see that after T = 6 time periods, we get that for initial values of costs: c 1 = 0.1 , c 2 = 0.2 , the groups probabilities of vaccinating have changed from the starting Nash-Pareto pair of ( x 3 * , y 3 * ) = ( 0.56 % , 0.77 % ) to the pair ( x 3 * ( 6 ) , y 3 * ( 6 ) ) = ( 0.8 % ,0.762 % ) . Though group 2’s probability of vaccinating has slightly decreased, group 1’s has increased. This is an advantage when we look at these values from the point of view of the expected coverage in the population, which is roughly estimated as (without considering time lags between vaccination and vaccine uptake):
C o v ( 0 ) : = ϵ 1 x 3 * + ϵ 2 y 3 * at t = 0, for Nash-Paretovalues
and
C o v ( t ) : = ϵ 1 x 3 * ( t ) + ϵ 2 y 3 * ( t ) at any t ∈ [ 0, T ] ,
where ϵ i is the fraction of population made up of players of type i for i ∈ { 1,2 } .
For an illustration, we look at the case where ϵ 1 = ϵ 2 = 0.5 . In this case, we present in
The comparison of coverage levels can, in fact, be conducted for any pair of ( c 1 , c 2 ) in our analysis. The most desired optimally controlled states are those where the control leads to better coverage than the Nash-Pareto case. Moreover, our analysis can be easily expanded to two more cases: 1) players of type 1 are a majority (recall these are players with income level I 1 ; since I 1 < I 2 , then we have a population where the lower income group is a majority); 2) players of type 1 are a minority.
We present the difference these proportions make on the discussion of our results from the perspective of the overall vaccine coverage. First, let us consider that ϵ 1 = 0.7 and ϵ 2 = 0.3 . In this case, we see (
present our case when ϵ 1 = 0.3 and ϵ 2 = 0.7 . We see from
In this paper we presented a 2-player asymmetric bimatrix game with two pure strategies, vaccinating or non-vaccinating, against a potentially infectious disease. We transformed and associated with our game a replicator dynamics system whose only mix strategy state is a Nash-Pareto stable pair. This means that over time, this mixed strategy of vaccination is likely to endure. From a public health perspective, the time-stability of the mixed Nash-Pareto pair is not desirable, as it is hoped that vaccine coverage can be increased in a population.
We showed that by introducing an exogenous control to decrease the perceived costs of vaccinating to all individuals in the population, an increase in the expected vaccine coverage can be achieved. We also showed that the net growth in the expected coverage depends on the population mix of players of type 1 and type 2, which is to be expected, as their vaccinating strategies are different.
Our game can be easily generalized in at least two directions: first, we can
consider more than 2 groups in our population makeup and we can consider these groups differentiated not by income, but by age, cost ( c i ) and loss ( L i ). Second, we can introduce differing mechanisms for cost control. Our work here presents the basics of how such a multiplayer vaccination game can be analyzed and controlled, with the scope of gaining net increases in overall expected vaccine coverage levels.
The first author acknowledges the support of the National Sciences and Engineering Research Council (NSERC) of Canada through the Discovery Grant #400684.
Cojocaru, M.G. and Jaber, A.S. (2018) Optimal Control of a Vaccinating Game toward Increasing Overall Coverage. Journal of Applied Mathematics and Physics, 6, 754-769. https://doi.org/10.4236/jamp.2018.64067