In this video, we will study the cooperative differential games where the main question is of how to allocate the maximum joint payoff of players among them. Let's consider a case of a joint venture. For example, we have a set of economic agents investing into the joint venture. The question is of how to allocate the revenue of the joint venture among the investors. Let's consider the classical differential game model called the renewable resource extraction game. In here, we will consider three companies extracting wood in one area. Let's suppose that the one company thinks of how much resource or how much wood with the other companies would extract. Then, if we think of why not to extract more resources than the others extract, then I will be able to get more revenue. Of course, the other companies can do in the same way, they can think of why cannot I extract resources faster? Then as a result all resources would be extracted. Of course, this situation is not beneficial for all of the companies. Let's suppose that they make an agreement on strategies. So, they decide of how much resources each company would extract, so that the joint revenue of all companies would be maximized. Of course, for some of the companies or, for example, for the company one this would not be an optimal solution. For example, it would mean that for this company the revenues would be very small. Then how can we make this cooperative agreement profitable for all companies? We should allocate the maximum joint revenue or the maximum joint payoff among the companies. That's what we're going to do in this video. In order to do that, we need to construct a differential game model in characteristic function form. For that, we need to define two things, the first thing is to define of what strategies of all players maximize their joint payoff. The second thing is of how to allocate the maximum joint payoff of all players along the game. The first thing we can do by solving the following optimization problem, where the functional here is a sum of payoffs of all players. Payoff of each player in here, is defined on the interval [t0,T], and of course it depends on the strategy of player i and on the state of the game, the function x(t). By the state of the game, we understand the number of the renewable resources in the system or in this area. It is defined by the system of differential equations or by the motion equation presented on the slide below. Where the right-hand side of this differential equation, depends on the rates of extraction for each player or for strategies of each player, functions ui(t,x). Here, we also consider strategies to be the functions of t and x. So, the strategies are defined for each time instant and for each state or for each number of resources in the system. Also, the right-hand side depends on the current number of renewable resources in the system. So, if there are very few trees in the area than the number of the trees that would grow in one or two years will be smaller. So, if we extract wood very fast, then the number of the trees or the size of the forest that will grow in several years would be smaller. So, how can we define the strategies of players or the rates of the wood extraction for each company so that the joint revenue would be maximum? In order to do that, we need to use the same approach that we use in an optimal control problem, we need to use the Bellman equation. It is important to notice that the problem that we considered is not the game, it is an optimal control problem because the functional that we need to maximize is only one functional, but also it depends on the three functions, u1(t,x), u2(t,x) and u3(t,x). But we can say that, these functions are actually one function u(t,x). So we need to find the function or the vector function that maximizes this functional. On the first step, we need to define the Bellman function in the game, and it is defined in the same way as it was defined in the optimal control problem part and the corresponding Bellman equation is presented on the slide below. In here, we also need to try to find the Bellman function or the solution of the Bellman equation, when the Bellman function has some specific form. So here we suppose that the Bellman function is equal to the function A(t) multiplied by the square root of x plus C(t). Then by substituting this form of Bellman function to the Bellman equation and by maximizing the right-hand side of the Bellman equation we can define the optimal control or the optimal strategies of all players or the optimal resource extraction rates for the companies and they are presented on the slide. But the functions Ai(t) are not known. We can find them from the system of differential equations presented below. By solving this system of differential equations and by substituting the values for the function A(t) and C(t) into the optimal control formula and into the Bellman equation, we can define the optimal control and the corresponding maximum payoff. Then by substituting the formula of the optimal control into the motion equation, we can define the corresponding optimal trajectory or the optimal number of the resources in the system as a function of t. The corresponding results are presented on the slide. On the right hand side, you can see the optimal controls of the optimal strategies along the corresponding optimal trajectory. On the left-hand side, you can see the corresponding optimal trajectory. The next thing we need to do is do to define of how can we allocate the maximum joint payoff of the players among them. In order to do that, we also need to define the notion of the characteristic function for this game. For this class of games, we will use a characteristic function based on the Nash equilibrium. As you remember before, we use the maximin approach. But in here we use the Nash equilibrium, because the way to calculate the characteristic function using Nash equilibrium is much easier. So, for each coalition S we need to define the non-cooperative games with N-S+1 players. So, we'll suppose that all players from coalition S are acting as one player, try to maximize their joint payoff and the other players from the set N-S are acting as individuals. So, what we need to do is, we need to define the Nash equilibrium between the players acting as one from coalition S and the other players. We will do that using the approach that we already studied in the previous part, and we will define the Bellman equation for this game, where the Bellman function for the coalition S has the form presented on the slide and the Bellman function for the players from not coalition S is also present on the slide. Then, by substituting this Bellman function into the corresponding system of Bellman equations, we can derive the system of differential equations for the unknown functions As, Ai, Cs and Ci. It is important that in here, we do not need the form of the optimal strategies or Nash equilibrium strategies, because we only need to define the values of the function Vs(t,x). Because the Vs(t,x) according to this approach, is equal to the joint payoff of players from coalition S, when all of them are acting as one player and the others are using the Nash equilibrium strategies. As it turns out in this particular approach we can have the formula for the characteristic function for an arbitrary coalition S. As you can see on the slide, the system of differential equations for the functions As and Cs is defined for any coalition S. That is why this approach is more easier to implement in the differential games. So, using the values of characteristic function, we need to define the set of imputations. As the values of characteristic functions are defined for each initial state and time of the game or for any t, initial time and for any x which is the initial state or the initial conditions for the system differential equations, then in the same way we can define the set of imputations for any initial time and initial state but along the cooperative trajectory. So we suppose, that at the beginning of the game players use strategies corresponding to cooperative behavior and choose the corresponding trajectory. Then the question is of how to allocate the joint payoff along this trajectory. In order to do that, we need to define the set of imputations along this trajectory. That is what is presented on the slide. The set of imputations along their cooperative trajectory in our resource extraction game is presented in the slide. Here at any time instant t, ksi1 should be more equal to the value of characteristic function of player 1, ksi2 in any time instant, should be more equal to the value of characteristic function of the player 2, the same for the player 3 and the sum should be equal to the maximum joint payoff or the value of the characteristic function for all players. On this slide you can see the result of numerical simulations, where the imputation set for any possible t is presented. On this slide you can see a list of references, where you could find more information on how to define the cooperative differential games and what are the possible approaches to define a characteristic function in this class of games.