Optimal Power Control in Cooperative Relay Networks Based on a Differential Game
Optimal Power Control in Cooperative Relay Networks Based on a Differential Game
ETRI Journal. 2014. Feb, 36(2): 280-285
Copyright © 2014, Electronics and Telecommunications Research Institute(ETRI)
  • Received : May 02, 2013
  • Accepted : August 06, 2013
  • Published : February 01, 2014
Export by style
Cited by
About the Authors
Haitao Xu
Xianwei Zhou

In this paper, the optimal power control problem in a cooperative relay network is investigated and a new power control scheme is proposed based on a non-cooperative differential game. Optimal power allocated to each node for a relay is formulated using the Nash equilibrium in this paper, considering both the throughput and energy efficiency together. It is proved that the non-cooperative differential game algorithm is applicable and the optimal power level can be achieved.
I. Introduction
Cooperative communications have been proposed for an enhancement in the performance of wireless networks [1] and to achieve cooperative diversity for improving network coverage and capacity [2] . A relay-cooperation transmission can be seen as a kind of cooperative communication, in which a relay station helps to forward data between mobile stations and base stations. In a relay-cooperation transmission, an increase in the power level of one node always results in more interference in the other nodes, caused by the simultaneous transmissions on nearby links [3] . Meanwhile, a higher power level causes more energy to be expended. For the above reasons, it is essential to design an enabling power control approach to alleviate co-channel interference in cooperative relay networks [4] .
Generally, power control is considered for maximizing throughput or minimizing energy consumption [5] . The power control for relay-cooperation transmissions is challenging owing to the fact that the power control strategies of relay nodes influence each other and decisions are made dynamically under competition. Each relay node can competitively allocate the power level in a dynamic manner such that their individual target system performance can be achieved. The throughput maximization and energy consumption minimization are intertwined, and the power control scheme should therefore consider these two fundamentally conflicting objectives concurrently.
A few works have been conducted on power control in cooperative relay networks [6] - [9] . In [6] , sub-carrier and power allocations are considered to obtain a better BER performance. In [7] , the power control is considered to increase the energy efficiency combing selective-relay cooperative communications. Game theory can be used to solve resource allocation problems of systems with conflicting components, and can be used in cooperative relay networks for settling the power control problems of relay nodes. Game theory has recently received increasing interest in the context of cooperative relay networks [3] , [8] , [9] . The problem of power control becomes more challenging when the relay nodes are non-cooperative, which motivates the use of game theory in research on ascertaining the importance of power control. In [3] , the authors consider the multi-user power control problem in Gaussian frequency-flat interference relay channels using a game-theoretic framework. In [8] , the energy efficiency of each user is maximized based on game theory and is measured in bits/joule. A game-theory-based decentralized relay selection scheme and a power allocation protocol were introduced in [9] to achieve a significant reduction in power consumption and an improvement in payoff.
The novelty of our work is in considering a differential game-based power control scheme to increase network throughput and minimize energy consumption. Differential games or continuous-time infinite dynamic games are used to study a class of decision problems, under which the evolution of the state is described by a differential equation and the players act throughout a time interval. Through a comparison with existing algorithms, a different optimal objective and a new non-cooperative differential-game-based power control scheme to obtain the Nash equilibrium for power control is proposed in this paper. Our model will take the objective of minimizing the total transmission cost with a tradeoff between network throughput and energy consumption. Relay nodes act as rational, selfish players competing in a differential game to obtain the optimal transmit power and minimize their own transmission cost.
The rest of this paper is organized as follows. The system model and problem statement are given in section II. Section III presents non-cooperative game solutions to the power control problem. Section IV provides the simulation results, and section V offers some concluding remarks regarding our proposal.
II. System Model and Problem Statement
We consider the uplink of a relay-assisted network with K users (game players) denoted by K ≡ {1, 2,⋯, K }. Each user is assigned a given channel (assumed to be orthogonal, typically in the frequency domain). A decode-and-forward protocol is considered in this model, in which the signals have to be decoded at the relay before being forwarded. User i can serve as a relay node for other users. Let D ( i ) denote the set of these users. Let pi ( s ) denote the direct transmission power of user i by time s , and pi,j ( s ) denote the transmission power in relaying the message of user j . Let
P R_max i ( s )
denote the maximum allowable power for a relay, and we then have
p i,j ( s )= ϖ i,j P R_max i ( s )
€ and
∑ j∈D( i ) ϖ i,j ≤1
. The transmission energy of player i during time interval [0, T ] is a predetermined positive constant.
Q iT = 0 T ( p i ( s )   + jD( i ) p ( s ) i,j ) dt, iK.
Equation (1) can be transformed into a differential game equation with the following initial and terminal boundary conditions:
Q i ( s ) = p i ( s )    + jD( i ) K p ( s ) i,j , s[ 0,T ], iK,
where an overdot denotes a derivative of a function with respect to time.
Users access a wireless system through the air interface, which is a common resource; since the air interface is a shared medium, each user’s transmission is a source of interference for others [10] . The signal-to-interference ratio (SIR) is a measure of the quality of signal reception for a wireless user. Thus, there exists a minimum acceptable SIR for each user. Assuming the minimum transmission power of user i is
p i ( s ) ¯
, all users are assigned the same bandwidth W , and user i splits the mi ( mi ∈[0,1]) fraction of its bandwidth for relaying [1] , as shown in Fig. 1 . The additional rate cost Ri ( s ) by time s can then be expressed as
R i ( s )= α i [ ( 1 m i )W ( log( 1+ p i ( s ) )log( 1+ p i ( s ) ¯ ) ) +            log( I i ( s ) )+ m i W jD( i ) log( 1+p ( s ) i,j ) ],
where f ( z ) = ( z b ) + = max{0, z b }, if z b ≤ 0 and f ( z )=0, and α i is a nonnegative parameter. The interference from neighboring relays is denoted by Ii ( s ) and can be expressed as
I i ( s )= ∑ j=1,j≠i N h i,j p j ( s ) +ω( s )
PPT Slide
Lager Image
System model.
Constrained by limited resources, the decoding set D ( i ) is time varying. Let xi ( s ) denote the stock, that is, the power of user i by time s , which is the sum of the transmission power and relaying power, affected by decoding set D ( i ). The evolution of this stock is governed by the following differential equation:
x i ( s ) = Q i ( s )   g i x i ( s ), iK, s[ 0,T ],
where gi is the time-varying parameter of D ( i ).
All relay users have to decode the signals before forwarding them, and thus users have a storage cost for relaying, which is constructed based on the evolution of xi ( s ). Let Si ( xi ( s )) denote the storage cost, and we have the following formula [11] :
S i ( x i ( s ) )= β i [ x i ( s ) p i ( s ) ], β>0 ,
where βi is a nonnegative parameter, which means the unit cost of energy consumed, and is interpreted as the number of information bits received per joule of energy consumed.
As a game player, each user seeks to minimize its payments of discounted sum of the increasing transmission cost. Let Ci ( pi ( s )) denote the payments of the discounted sum of the increasing transmission cost and pi ( s ) be the control of the differential game. The optimization problem of user i can thus be expressed as
min p i ( s ) C i ( p i ( s ) )= 0 T [ Q i ( s ) + R i ( s )+ S i ​ ( x i ( s ) ) ] e λs dt,
where λ is the common discount rate, which is applied to find the discount, subtracted from a future value to find the value before the game starts. The discount rate should be considered when deciding whether to spend resources on or give more resources for a transmission.
We are then led to a differential game, G G (K, { pi },{ Ci }), in which the following hold:
  • 1) The players ofGare users K ≡ {1, 2,⋯,K} .
  • 2) The strategy of playeriispi(s), which denotes the allocated power, and the game’s strategy profile isP(s) = (p1(s),p2(s),...,pk(s)).
  • 3) The players’ utilities are the objective of the minimum problem in (6).
Generally, the payments (utilities) of defined game G are not linear, which is different from the original normal game form.
III. Non-cooperative Optimal Power Control
In this section, we consider the model of a finite horizon, a series of dynamic optimization programs, and one feedback Nash equilibrium problem. The dynamic optimization program technique was developed by Bellman [12] and is given in Theorem 1.
Theorem 1: Bellman’s dynamic programming. A set of controls, u ( s ) = ϕ ( s, x ), constitutes an optimal solution to control (6) if there exist continuously differentiable functions, V ( s , x ), defined [ t 0 , T Rm R and satisfying the following Bellman equation,
V s (s,x)= min u { g[s,x,u]+ V x (s,x)f[s,x,u] }                  ={ g[s,x, ϕ (s,x)]+ V x (s,x)f[s,x, ϕ (s,x)] },
V( T,x )=q( x ).
In this case, time is presented by s , the state is denoted by x , and Vi ( s, x ) is the utility function (cost) of player i in time interval [ t 0 , T ] For the optimization problem of (6), value function Vi ( s , x ) can be represented as follows:
V i ( s,x )= 0 T [ Q i ( s,x( s ) ) + R i ( s,x( s ) )+ S i ( x( s ) ) ] e λs dt.
Performing the indicated maximization for the optimization problem indicated above, we can obtain the Nash equilibrium for users, that is,
p i ( s )= φ i (s)= α i ( 1 m i ) log 2 ( e ) β i V x i ( s,x ) e λs 1 1,  iK,   s[ 0,T ].
Proposition 1. System (7), (8) provides the solution.
V i ( s,x )=[ A i ( s )x+ B i (s) ] e λs ,  iK
with { A 1 ( s ), A 2 ( s ), A 3 ( s ),…, AK ( s )} satisfies the following equations:
A i ( s )=( θ β i λ+ g i ) e ( λ+ g i )( tT ) + β i λ+ g i ,
A i ( T )=θ,
and { B 1 ( s ), B 2 ( s ), B 3 ( s ),…, BK ( s )} is given by
B i ( s )= e λs [ 0 s f i ( t ) e λt dt + B i 0 ].
The proof of Proposition 1 is given in the Appendix. In (14), fi ( t ) is suppressed by { αi }, { mi }, and { βi }, which can be found in the Appendix.
Based on the above proposition, (6) can be solved through a non-cooperative differential game, as follows:
   Procedure. Optimal power  p i ( s ) assigned to user i            _ ¯      1) Choose objective user i, and start with a time interval,    [ t 0 ,T], and initial parameters for optimal power control:    2) Construct the minimum payments problem of user i:    3) Calculate the Nash equilibrium of the optimization    problem from equation (13). Control the power level of    node i based on the Nash equilibrium.                                       _
Proposition 2. The Nash equilibrium obtained from (13) varies with a change in parameter Ai ( s ) and has the same changing trend.
The result in Proposition 2 is straightforward, based on Proposition 1.
IV. Performance Evaluation
As Ai ( s ) has an obvious impact on the Nash equilibrium, we first consider the variation of Ai ( s ) with time and λ in this simulation, which is shown in Fig. 2 . It is easy to find that Ai ( s ) is proportional to the time vector.
In Fig. 2(a) , Ai ( s ) increases with time and varies from 0 to 5. As can be seen from (10), Nash equilibrium φi ( s ) is a function of Ai ( s ), the variation of which will significantly reflect on φi ( s ), which means the optimal power assigned to each user will be modified by Ai ( s ). On the other hand, Ai ( s ) is the coefficient of x in (11). The system solution Vi ( t , x ) in (11) will also be influenced by Ai ( s ). It can be seen from Fig. 2(b) that Ai ( s ) is inversely proportional to discount rate λ .
PPT Slide
Lager Image
Variation of Ai(s) with (a) time and (b) discount rate.
We can easy find the optimal power assigned to users through the differential power control game algorithm, which is shown in Fig. 3(a) . It can be seen from Fig. 3(a) that the Nash equilibrium obtained has the same variation trend as Ai ( s ), which is consistent with Proposition 2. Furthermore, we can find that the optimal power is proportional to the discount rate (shown in Fig. 3(b) ), which means we can assign more power for a transmission with a higher discount rate.
PPT Slide
Lager Image
Variation of pi(s) with (a) time and (b) discount rate.
V. Conclusion
In this paper, a non-cooperative differential-game based power-control approach for a cooperative relay network was proposed, which is distributed in each node and acts independently to achieve coordinate power control of the network resources. It was proved that a differential game can be used in cooperative relay networks for power control problems. Minimum payments of each node can be achieved, and the optimal power level assigned to relay nodes can be obtained.
Proof of Proposition 1.
Assume that Vi ( s , x ) is a polynomial that can be expressed as follows:
V i ( s,x )=[ A i ( s )x+ B i (s) ] e λs .
Using (15), system (7), (8) can be expressed as
[ λ A i ( s ) A i ( s ) ]x+[ λ B i ( s ) B i ( s ) ]= Q i ( s ) + R i ( s )+ β i [ x p i ]+ A i ( s )( Q i ( s ) g i x ),
A i ( T )x+ B i (T)=θ ( x i ( T ) x i ¯ ) + .
We then have
A i ( s )=( λ+ g i ) A i ( s ) β i ,
A i ( T )=θ,
B i ( s )=λ B i ( s )( 1+ A i ( s ) ) Q i ( s ) + R i ( s ) β i p i ,
B i (T)=θ x i ¯ .
Plugging equations into (20), we have
B i ( s )=λ B i ( s )( 1+ A i ( s ) )( α i ¯ β i ¯ A i ( s ) 1 + jD( i ) K ϖ i,j P R_max i ( s ) )+ α i [ ( 1 m i )W( log( α i ¯ β i ¯ A i ( s ) ) log( 1+ p i ( s ) ¯ ) ) + + m i W jD( i ) log( 1+ ϖ i,j P R_max i ( s ) ) log( I i ( s ) ) ] β i ( α i ¯ β i ¯ A i ( s ) 1 )=λ B i ( s )+ f i ( s ),
α i ¯
β i ¯
, for ∀ i K , are constraints involving the model parameters, { α 1 , α 2 , α 3 , …, αK }, { m 1 , m 2 , m 3 ,…, mK }, { β 1 , β 2 , β 3 ,…, βK } , and
f i ( s )=( 1+ A i ( s ) )( α i ¯ β i ¯ A i ( s ) 1+ jD( i ) K ϖ i,j P R_max i ( s ) ) β i ( α i ¯ β i ¯ A i ( s ) 1 ) + α i [ ( 1 m i )W ( log( α i ¯ β i ¯ A i ( s ) )log( 1+ p i ( s ) ¯ ) ) + log( I i ( s ) )+ m i W jD( i ) log( 1+ ϖ i,j P R_max i ( s ) ) ].
Solving (18) and (19), we have
A i ( s )=( θ β i λ+ g i ) e ( λ+ g i )( tT ) + β i λ+ g i .
Bi ( s ) can be solved as
B i ( s )= e λs [ 0 s f i ( t ) e λt dt + B i 0 ]
B i 0 =θ x ¯ i 0 T F i ( t ) e λt dt .
Hence, Proposition 1 follows. ⧠
This paper is supported by the National High Technology Research and Development Program of China (863 Program) (No. 2012AA121604).
Haitao Xu received his BS degree from Zhong Shan University, China, in 2007, and his MS degree at the University of Bristol in 2009. He is currently studying for a PhD at the University of Science and Technology Beijing, China. His research interests include mathematical models, management algorithms, self-organized networks, game theory and next-generation networks.
Xianwei Zhou received his BS degree from the Department of Mathematics at Southwest Teachers’ University in 1986 and his MS degree from the Department of System Science and Mathematics from Zhengzhou University in 1992, his PhD from the Department of Transportation Engineering of Southwest Jiao-tong University, China in 1999. He was engaged in post doctor study at School of Electronic and Information Engineering of Beijing Jiao-tong University, China, from 1999 to 2000. He is currently a professor at the Department of Communication Engineering, School of Computer & Communication Engineering, University of Science and Technology Beijing, China. His research interests include communication network, security next-generation networks, mobile IPv6, scheduling theory and game theory.
Zhang Z. 2008 “A Cooperation Strategy Based on Nash Bargaining Solution in Cooperative Relay Networks,” IEEE Trans. Veh. Technol. 57 (4) 2570 - 2577    DOI : 10.1109/TVT.2007.912960
Wang D. , Wang X. , Cai X. 2011 “Optimal Power Control for Multi-User Relay Networks over Fading Channels,” IEEE Trans. Wireless Commun. 10 (1) 199 - 207    DOI : 10.1109/TWC.2010.102210.100160
Qian L. P. 2013 “Pareto Optimal Power Control via Bisection Searching in Wireless Networks,” IEEE Commun. Lett. 17 (4) 709 - 712    DOI : 10.1109/LCOMM.2013.022213.130051
Shi Y. 2009 “A Game-Theoretic Approach for Distributed Power Control in Interference Relay Channels,” IEEE Trans. Wireless Commun. 8 (6) 3151 - 3161    DOI : 10.1109/TWC.2009.080831
Song Y.-K. , Kim D. 2010 “Convergence and Performance of Distributed Power Control Algorithms for Cooperative Relaying in Cellular Uplink Networks,” IEEE Trans. Veh. Technol. 59 (9) 4645 - 4651    DOI : 10.1109/TVT.2010.2066590
2009 “Cooperative Subcarrier and Power Allocation for a Two-Hop Decode-and-Forward OFCMD Based Relay Network,” IEEE Trans. Wireless Commun. 8 (9) 4797 - 4805    DOI : 10.1109/TWC.2009.081496
Zhou Z. 2008 “Energy-Efficient Cooperative Communication Based on Power Control and Selective Single-Relay in Wireless Sensor Networks,” IEEE Trans. Wireless Commun. 7 (8) 3066 - 3078    DOI : 10.1109/TWC.2008.061097
Zappone A. , Buzzi S. , Jorswieck E. 2011 “Energy-Efficient Power Control and Receiver Design in Relay-Assisted DS/CDMA Wireless Networks via Game Theory,” IEEE Commun. Lett. 15 (7) 701 - 703    DOI : 10.1109/LCOMM.2011.060111.110236
Khayatian H. , Saadat R. , Abouei J. 2013 “Coalition-Based Approaches for Joint Power Control and Relay Selection in Cooperative Networks,” IEEE Trans. Veh. Technol. 62 (2) 835 - 842    DOI : 10.1109/TVT.2012.2222681
Saraydar C.U. 2002 “Efficient Power Control via Pricing in Wireless Data Networks,” IEEE Trans. Commun. 50 (2) 291 - 303    DOI : 10.1109/26.983324
Miao X.-N. , Zhou X.-W. , Wu H.-Y. 2010 “A Cooperative Differential Game Model Based on Transmission Rate in Wireless Networks,” Operations Research Lett. 38 (4) 292 - 295    DOI : 10.1016/j.orl.2010.04.007
Yeung D.W.K. , Petrosjan L.A. 2006 “Cooperative Stochastic Differential Games,” Springer Series in Operations Research and Financial Engineering