*By Sebastian Graves and Thomas Sargent*

This notebook computes optimal contracts for the three examples that lead off chapter 21 of
**Recursive Macroeconomic Theory, Fourth edition** by Lars Ljungqvist and Thomas Sargent.

The examples illustrate different sorts of tradeoffs between insurance and incentives that emerge under different limits on enforcement and information.

In each of the three economies, a planner or money-lender designs an efficient contract to supply insurance to a risk-averse consumer who receives an exogenous random stream of a non-storable endowment.

The only way that the consumer to smooth consumption across states and time is to interact with the planner.

The three models differ in the constraints that they impose on the planner.

These constraints express the planner's limited ability either to enforce a contract or to observe the consumer's endowment

Each of the examples uses a version of what we have nicknamed **dynamic programming squared**

In a dynamic programming squared problem, a value function from one Bellman equation is an argument of another Bellman equation.

In the examples below, a planner or money lender's value function will have as an argument the value of a villager that satisfies a Bellman equation

Imagine a village with a large number of ex ante identical households. Each household has preferences over consumption streams that are ordered by $$ E_{-1}\sum_{t=0}^\infty \beta^t u(c_t), $$ where $u(c)$ is an increasing, strictly concave, and twice continuously differentiable function, $\beta \in (0,1)$ is a discount factor, and $E_{-1}$ is the mathematical expectation not conditioning on any information available at time $0$ or later.

Each household receives a stochastic endowment stream $\{y_t\}_{t=0}^\infty$, where for each $t \geq 0$, $y_t$ is independently and identically distributed according to the discrete probability distribution ${\rm Prob} (y_t = \overline y_s) = \Pi_s,$ where $s \in \{1, 2, \ldots ,S\}\equiv {\bf S}$ and $\overline y_{s+1}>\overline y_s$.

The consumption good is not storable.

At time $t \geq 1$, the household has received a history of endowments $h_t = (y_t, y_{t-1}, \ldots, y_0).$

Endowment processes are distributed independently and identically both across time and across households.

In this setting, if there were a competitive equilibrium with complete markets, at date $0$ households would trade history- and date-contingent claims.

Since households are ex ante identical, each household would consume the per capita endowment in every period, and its lifetime utility would be

$$ v_{\rm pool} = \sum_{t=0}^\infty \beta^t \, u\!\left(\sum_{s=1}^S \Pi_s \overline y_s\right) = {1 \over 1-\beta}\, u\!\left(\sum_{s=1}^S \Pi_s \overline y_s\right) . $$

Households would thus insure away all risks from their individual endowment processes.

But the incentive constraints that we are about to specify make this allocation unattainable.

For each specification of incentive constraints, we shall solve a planning problem for an efficient allocation that respects those constraints.

Following a tradition started by
Edward Green (1987) [*Lending and the Smoothing of Uninsurable
Income*, in Edward C. Prescott and Neil Wallace, editors, **Contractual Arrangements for
Intertemporal Trade**, Minnesota Studies in Macroeconomics series, Vol.
1, Minneapolis: University of Minnesota Press, pp. 3--25], we assume that a *moneylender* or *planner* is
the only person in the village who has access to
a risk-free loan market outside the village.

The moneylender can borrow or lend at a constant one-period risk-free gross interest rate $R=\beta^{-1}$.

Households cannot borrow or lend with each other, and can trade only with the moneylender.

Furthermore, we assume that the moneylender is committed to honor his promises.

We will study three distinct environments in which there are three alternative types of incentive constraints.

**Enviroment a.** Both the money lender and the household observe the household's history of endowments at each time $t$.
Although the moneylender can commit to honor a
contract, households cannot commit and at any time are
free to walk away from an arrangement
with the moneylender
and live in perpetual autarky thereafter. They must be induced not to do so
by the structure of
the contract.
This is a model of *one-sided commitment* in which the
contract must be *self-enforcing*. That is, it must be structured to induce the household to prefer to
conform to it.

**Environment b.** Households *can* make commitments and enter
into enduring and binding contracts with the moneylender,
but they have private
information about their own incomes. The moneylender
can see neither their income nor their consumption. Instead,
exchanges between the moneylender and a household must
be based on the household's own reports about income
realizations. An incentive-compatible contract induces
a household to report its income truthfully.

**Environment c.** The environment is the same as b except that now households have access to a storage technology that
cannot be observed by the moneylender.
Households can store nonnegative amounts of goods at a risk-free
gross return of $R$ equal to the interest rate that
the moneylender faces in the outside credit market.
Since the moneylender can both borrow and lend at the interest
rate $R$ outside of the village,
the private storage technology does not change the economy's
aggregate resource constraint, but it does affect the set of
incentive-compatible contracts between the moneylender and the
households.

When we compute efficient allocations for each of these three environments, we find that the dynamics of the implied consumption allocations differ dramatically.

We shall see that the dynamics of consumption outcomes evidently differ substantially across the three environments, increasing monotonically and then flattening out in environment a, stochastically heading south in environment b, and stochastically heading north in environment c. These sample path properties will reflect how the optimal contracts cope with the three different frictions that we have put into the environment.

Chapter 21 of RMT4 explains why sample paths of consumption differ so much across these three settings.

For all three environments discussed, consumers have a utility function:

$$u(c) = - \gamma^{-1} \exp(-\gamma c)$$

We set $\gamma = 0.7$, and the discount factor, $\beta$ to 0.8.

The consumers receive an iid endowment that can take any integer in the range $[\bar y_1,...,\bar y_{5}] = [6,...,10]$.

The probability of each realisation is $\Pi_s = \frac{1-\lambda}{1-\lambda^{5}}\lambda^{s-1}$ with $\lambda = 0.4$.

As mentioned above, an interesting benchmark case is a complete markets environment.

Because all households are *ex ante* identical, in a complete markets economy each household would consumer the per capita endowment in every period, and its lifetime utility would be:

$$ v_{pool} = \frac{1}{1-\beta} u \left( \sum_{s=1}^S \Pi_s \bar y_s \right) = \frac{u(c_{pool})}{1-\beta} $$

Later we will compare the consumption paths for each enviroment to that which would occur in the complete markets environment.

In each environment, we compute allocations for the situation in which the planner or money lender just breaks even.

The first environment is one in which the planner is able to commit, but households are not.

At any time households are free to walk away from an arrangement with the planner, and live in perpetual autarky thereafter.

RMT4 shows how this problem can be written in a recursive form.

Equations 21.3.4 to 21.3.8 in RMT4 express the planners's problem as:

\begin{align} &P(v) = \max_{c_s,w_s} \sum_{s=1}^S \Pi_s \left[ (\bar y_s - c_s) + \beta P(w_s) \right] \\ &\text{subject to} \\ &\sum_{s=1}^S \Pi_s \left[ u(c_s) + \beta w_s \right] \geq v \\ &u(c_s) + \beta w_s \geq u(\bar y_s) + \beta v_{aut} \text{ , s = 1,...,S} \\ &c_s \in [c_{min},c_{max}] \\ &w_s \in [v_{aut},\bar v] \end{align}

where $w_s$ is the promised value with which the consumer will enter the next period, given that $y = \bar y_s$ this period.

The first constraint is a promise keeping constraint, while the second set of constraints are participation constraints. $[c_{min},c_{max}]$ is a bounded set, while $\bar v$ just needs to be a very large number.

The value of autarky to the households is:

$$ v_{aut} = \frac{1}{1-\beta} \sum_{s=1}^S \Pi_s u(\bar y_s) $$

Below we solve the moneylender's problem in this environment by approximating $P(v)$ using Chebyshev polynomials.

In [1]:

```
import numpy as np
from scipy.optimize import minimize, fsolve
from scipy.interpolate import UnivariateSpline
import matplotlib.pyplot as plt
import numpy.polynomial.chebyshev as cheb
%matplotlib inline
```

In [2]:

```
# Parameter values
gamma = 0.7
beta = 0.8
lamb = 0.4
S = 5
y_grid = np.linspace(6,5+S,S)
prob_grid = np.zeros(S)
for i in range(S):
prob_grid[i] = (1 - lamb)/(1-lamb**S)*lamb**(i)
# Utility function
u = lambda c: -gamma**(-1)*np.exp(-gamma*c)
u_inv = lambda u: np.log(-gamma*u)/(-gamma)
# Calculate complete markets consumption
c_pool = np.dot(prob_grid,y_grid)
# Calculate value of autarky
v_aut = 1/(1-beta)*np.dot(prob_grid, u(y_grid))
```

In [3]:

```
# Functions used in each environment
# Nodes and basis matrix for Chebyshev approximation
def Cheb_basis(order,lb,ub):
# Calculate roots of Chebyshev polynomial
k = np.linspace(order, 1, order)
roots = np.cos((2*k - 1)*np.pi/(2*order))
# Scale to approximation space
s = lb + (roots - -1)/2*(ub-lb)
# Create basis matrix
Phi = cheb.chebvander(roots, order-1)
return s, Phi
# Value Function Iteration
def Bellman_Iterations(s, Phi, P_fun, x_store, coeff, tolc=1e-6, bnds=None, cons=(), max_iters=100):
global x, c
c = coeff
order = Phi.shape[1]
iters = 0
diff = 1
while diff > tolc:
# 1. Maximization, given value function guess
P_iter = np.zeros(order)
for i in range(order):
x = s[i]
res = minimize(P_fun, x_store[i], method = 'SLSQP', bounds = bnds, constraints=cons, tol=1e-15)
x_store[i] = res.x
P_iter[i] = -P_fun(res.x)
# 2. Bellman updating of Value Function coefficients
c1 = np.linalg.solve(Phi, P_iter)
# 3. Compute distance and update
diff = max(abs(c1 - c))
print(diff)
c = np.copy(c1)
iters = iters + 1
if iters >= max_iters:
print('Convergence failed after {} iterations'.format(iters))
break
if diff < tolc:
print('Convergence achieved after {} iterations'.format(iters))
return c
```

In [4]:

```
# Value Function Approximation
# Set bounds and approximation order
v_min = v_aut
v_max = -0.065
c_min = 0
c_max = 50
order = 70
# Calculate nodes and basis matrix
s, Phi = Cheb_basis(order, v_min, v_max)
# Bounds for Maximisation
lb = np.concatenate([np.ones(S)*c_min, np.ones(S)*v_min], axis=0)
ub = np.concatenate([np.ones(S)*c_max, np.ones(S)*v_max], axis=0)
# Initialize Value Function coefficients and goess for c,w
y = (c_pool - u_inv(s*(1-beta)))/(1-beta)
c = np.linalg.solve(Phi, y)
x_init = np.concatenate([np.ones(S)*c_min, np.ones(S)*v_min], axis=0)
# Function to minimize and constraints
def P_fun(x):
scale = -1 + 2*(x[S:2*S] - v_min)/(v_max - v_min)
P = np.dot(cheb.chebvander(scale,order-1),c)
P_fun = - prob_grid.dot((y_grid - x[0:S]) + beta*P)
return P_fun
def cons12(y):
global x
return prob_grid.dot(u(y[0:S]) + beta*y[S:2*S]) - x
cons1 = ({'type': 'ineq', 'fun': lambda y: u(y[0:S]) + beta*y[S:2*S] - u(y_grid) - beta*v_aut},
{'type': 'ineq', 'fun': cons12})
bnds1 = np.concatenate([lb.reshape(2*S, 1), ub.reshape(2*S, 1)], axis = 1)
# Bellman Iterations
NBell = 5
tolc = 1e-6
diff = 1
iters = 1
x_store = {}
for i in range(order):
x_store[i] = x_init
c = Bellman_Iterations(s, Phi, P_fun, x_store, c, bnds=bnds1, cons=cons1)
```

In [5]:

```
# Time Series Simulation
T = 100
np.random.seed(2)
y_series = np.random.choice(y_grid,size = T,p = prob_grid)
c_series = np.zeros(T)
w_series = np.zeros(T)
resid_series = np.zeros(T)
pval_series = np.zeros(T)
# Initialize v such that P(v) = 0
v_find = lambda v: cheb.chebvander(-1 + 2*(v - v_min)/(v_max - v_min),order-1).dot(c)
x = fsolve(v_find,v_max)
res = minimize(P_fun,x_init,method = 'SLSQP',bounds = bnds1,constraints = cons1,tol=1e-15)
c_series[0] = res.x[np.where(y_grid == y_series[0])[0][0]]
w_series[0] = res.x[S + np.where(y_grid == y_series[0])[0][0]]
# Simulate
for t in range(1,T):
x = w_series[t-1]
res = minimize(P_fun, x_init,method = 'SLSQP',bounds = bnds1, constraints = cons1, tol=1e-15)
c_series[t] = res.x[np.where(y_grid == y_series[t])[0][0]]
w_series[t] = res.x[S + np.where(y_grid == y_series[t])[0][0]]
plt.plot(c_series, label = 'Environment (a)')
plt.plot(np.ones(T)*c_pool, label = 'Complete Markets')
plt.ylabel('Consumption')
plt.xlabel('Time')
plt.legend(loc = 'best');
plt.title('Environment (a)');
```

The above simulation is equivalent to Figure 21.2.1.a in RMT.

The discussion in RMT4 confirms that the household's consumption ratchets upwards over time.

The consumption level is constant after the first time that the household receives the highest possible endowment.

The second environment is one in which households *can* make commitments to enter into binding contracts with the planner, but they have private information about their incomes.

Consequently, incentive compatability constraints are required to ensure that households truthfully report their incomes.

Equations 21.5.1 to 21.5.5 in RMT4 express the planners's problem.

\begin{align} &P(v) = \max_{b_s,w_s} \sum_{s=1}^S \Pi_s \left[ -b_s + \beta P(w_s) \right] \\ &\text{s.t.} \\ &\sum_{s=1}^S \Pi_s \left[ u(\bar y_s + b_s) + \beta w_s \right] = v \\ & C_{s,k} \equiv u(\bar y_s + b_s) + \beta w_s - [ u(\bar y_s + b_k) + \beta w_k ] \geq 0 \hspace{2mm} \forall \hspace{2mm} s,k \in S \times S\\ &b_s \in [a - \bar y_s,\infty ] \\ &w_s \in [- \infty, v_{max}] \end{align}

Here $b_s$ is the transfer that the moneylender gives to a household who reports income $y_s$ if their promised value was $v$.

The promise keeping constraint remains, while the participation constraint has been replaced by a large set of incentive compatibility constraints.

RMT4 shows that we can discard many of the incentive compatibility constraints.

In solving the model below, we keep only the local upward and downward incentive compatibility constraints.

In [6]:

```
# Set bounds and approximation order
b_min = -20
b_max = 20
w_min = -150;
w_max = -0.04;
v_min = -150;
v_max = -0.04;
v_pool = u(c_pool)/(1-beta)
order = 70
# Calculate nodes and basis matrix
s, Phi = Cheb_basis(order,v_min,v_max)
# Bounds for Maximisation
lb = np.concatenate([np.ones(S)*b_min,np.ones(S)*w_min], axis=0)
ub = np.concatenate([np.ones(S)*b_max,np.ones(S)*w_max], axis=0)
# For initial guess, use upper bound given in RMT:
cbar = np.zeros(order)
upper = np.zeros(order)
for i in range(order):
cbar[i] = u_inv((1-beta)*s[i])
upper[i] = np.dot(prob_grid,(y_grid - cbar[i])/(1-beta))
c = np.linalg.solve(Phi,upper)
# Function to minimize and constraints
def P_fun2(x):
scale = -1 + 2*(x[S:2*S] - v_min)/(v_max - v_min)
P = np.dot(cheb.chebvander(scale,order-1),c)
P_fun = - prob_grid.dot(-x[0:S] + beta*P)
return P_fun
def cons23(y):
global x
return prob_grid.dot(u(y_grid + y[0:S]) + beta*y[S:2*S]) - x
cons2 = ({'type': 'ineq', 'fun': lambda x: u(y_grid[1:S] + x[1:S]) + beta*x[S+1:2*S] - u(y_grid[1:S] + x[0:S-1]) - beta*x[S:2*S-1]},
{'type': 'ineq', 'fun': lambda x: u(y_grid[0:S-1] + x[0:S-1]) + beta*x[S:2*S-1] - u(y_grid[0:S-1] + x[1:S]) - beta*x[S+1:2*S]},
{'type': 'eq', 'fun': cons23})
bnds2 = np.concatenate([lb.reshape(2*S,1),ub.reshape(2*S,1)], axis = 1)
x_store = {}
for i in range(order):
x_store[i] = np.concatenate([np.zeros(S),np.ones(S)*s[i]], axis=0)
c = Bellman_Iterations(s, Phi, P_fun2, x_store, c, tolc, bnds=bnds2, cons=cons2)
```

In [7]:

```
# Time Series Simulation
T = 800
np.random.seed(2)
y_series = np.random.choice(y_grid,size = T+1, p = prob_grid)
c_series = np.zeros(T)
w_series = np.zeros(T)
# Initialize v such that P(v) = 0
v_find = lambda v: cheb.chebvander(-1 + 2*(v - v_min)/(v_max - v_min),order-1).dot(c)
x = fsolve(v_find,v_aut)
x_init = np.concatenate([np.zeros(S),np.ones(S)*x],axis=0)
res = minimize(P_fun2,x_init,method = 'SLSQP',bounds = bnds2, constraints = cons2,tol=1e-10)
c_series[0] = y_series[0] + res.x[np.where(y_grid == y_series[0])[0][0]]
w_series[0] = res.x[S + np.where(y_grid == y_series[0])[0][0]]
x_init = res.x
# Simulate
for t in range(1,T):
x = w_series[t-1]
res = minimize(P_fun2,x_init,method = 'SLSQP',bounds = bnds2,constraints = cons2,tol=1e-10)
c_series[t] = y_series[t] + res.x[np.where(y_grid == y_series[t])[0][0]]
w_series[t] = res.x[S + np.where(y_grid == y_series[t])[0][0]]
x_init = res.x
# Plot
plt.plot(c_series, label = 'Environment (b)')
plt.plot(np.ones(T)*c_pool, label = 'Complete Markets')
plt.ylabel('Consumption')
plt.xlabel('Time')
plt.title('Environment (b)')
plt.legend(loc = 'best');
```

This simulation reported in the graph above confirms that in environment **b** the incentive compatibility constraints induce the planner to introduce a downward tilt into consumption paths.

The third environment is the same as in (b), except for the additional assumption that households have access to a storage technology.

A household can store nonnegative amounts that cannot be observed by the planner.

The text of RMT4 chaper 21 shows that the solution to this problem is the same as in an economy in which each household can lend *or borrow* at the risk-free gross interest rate R, subject to the natural debt limit.

Thus, the planner enables the household to relax the no-borrowing constraint implied by the restriction that it can store only nonnegative amounts

We can find the natural debt limit by iterating forward on the households budget constraint:

\begin{equation} c + k' = y + Rk \end{equation} This iteration gives: \begin{equation} k = \frac{1}{R} \sum_{j=0}^\infty \frac{c - y}{R^j} \end{equation}

Imposing non-negativity on consumption:

\begin{equation} k \geq - \frac{1}{R} \sum_{j=0}^\infty \frac{y}{R^j} \end{equation}

Finally, the natural debt limit is found by choosing the lowest possible value of the endowment, so that for any possible endowment stream the household can always pay back its debts:

\begin{equation} k \geq - \frac{1}{R} \sum_{j=0}^\infty \frac{\bar y_{min}}{R^j} = - \frac{\bar y_{min}}{R-1} \equiv \phi \end{equation}

A recursive presentation of the household's problem is then: \begin{align} &V(k,y) = \max_{c,k'} u(c) + \beta E [V(k',y')] \\ &\text{s.t.} \\ &c + k' = y + Rk \\ & k' \geq \phi \end{align}

As income is iid, we can re-write the household's problem with only one state.

Define a = k + y.

Then \begin{align} &V(a) = \max_{c,k'} u(c) + \beta E [V(Rk' + y')] \\ &\text{subject to} \\ &c + k' = a \\ & k' \geq \phi \end{align}

Below we solve this latter problem using Value Function Iteration, again with Chebyshev polynomials.

In [8]:

```
# Update parameter values
# Set bounds and approximation order
R = 1/beta
k_min = - y_grid[0]/(R - 1)
k_max = 100
a_min = R*k_min + min(y_grid)
a_max = R*k_max + max(y_grid)
order = 150
# Calculate nodes and basis matrix
s, Phi = Cheb_basis(order,a_min,a_max)
# Create bounds
bnds3 = np.array([[k_min,k_max]])
# Value function
def P_fun3(kprime):
global x,c
# Function to minimize
scale = -1 + 2*(R*kprime + y_grid - a_min)/(a_max - a_min)
P_fun = -(u(x - kprime) + beta * prob_grid.dot(cheb.chebval(scale, c)))
return P_fun
# Initialize guess and VF coefficients
c = np.zeros(order)
x_store = {}
for i in range(order):
x_store[i] = k_min
c = Bellman_Iterations(s, Phi, P_fun3, x_store, c, bnds=bnds3)
```

In [9]:

```
# Time Series Simulation
T = 800
np.random.seed(2)
y_series = np.random.choice(y_grid, size = T+1, p = prob_grid)
a_series = np.zeros(T+1)
c_series = np.zeros(T)
# Initialise at v_aut
def k_find(k):
scale = -1 + 2 * (R * k + y_grid - a_min)/(a_max - a_min)
return prob_grid.dot(cheb.chebval(scale,c)) - v_aut
k0 = fsolve(k_find,0)
a_series[0] = k0 + y_series[0]
# Simulate
for t in range(T):
x = a_series[t]
res = minimize(P_fun3, k_min,method='SLSQP', bounds=bnds3,tol=1e-15)
c_series[t] = a_series[t] - res.x
a_series[t+1] = R * res.x + y_series[t+1]
# Plot
plt.plot(c_series, label = 'Environment (c)')
plt.plot(np.ones(T)*c_pool, label = 'Complete Markets')
plt.ylabel('Consumption')
plt.xlabel('Time')
plt.title('Environment (c)')
plt.legend(loc = 'best')
```

Out[9]:

Notice that the introduction of a storage technology for the household means that the consumption path now has an upward trend.

This occurs because our parameter values satisfy $\beta R = 1$.