Welcome back! This is the second notebook of Riding the wave: Convection problems, the third module of "Practical Numerical Methods with Python".
The first notebook of this module discussed conservation laws and developed the non-linear traffic equation. We learned about the effect of the wave speed on the stability of the numerical method, and on the CFL number. We also realized that the forward-time/backward-space difference scheme really has many limitations: it cannot deal with wave speeds that move in more than one direction. It is also first-order accurate in space and time, which often is just not good enough. This notebook will introduce some new numerical schemes for conservation laws, continuing with the traffic-flow problem as motivation.
Let's explore the behavior of different numerical schemes for a moving shock wave. In the context of the traffic-flow model of the previous notebook, imagine a very busy road and a red light at x=4. Cars accumulate quickly in the front, where we have the maximum allowed density of cars between x=3 and x=4, and there is an incoming traffic of 50% the maximum allowed density (ρ=0.5ρmax).
Mathematically, this is:
ρ(x,0)={0.5ρmax0≤x<3ρmax3≤x≤4Let's find out what the initial condition looks like.
import numpy
from matplotlib import pyplot
%matplotlib inline
# Set the font family and size to use for Matplotlib figures.
pyplot.rcParams['font.family'] = 'serif'
pyplot.rcParams['font.size'] = 16
def rho_red_light(x, rho_max):
"""
Computes the "red light" initial condition with shock.
Parameters
----------
x : numpy.ndaray
Locations on the road as a 1D array of floats.
rho_max : float
The maximum traffic density allowed.
Returns
-------
rho : numpy.ndarray
The initial car density along the road
as a 1D array of floats.
"""
rho = rho_max * numpy.ones_like(x)
mask = numpy.where(x < 3.0)
rho[mask] = 0.5 * rho_max
return rho
# Set parameters.
nx = 81 # number of locations on the road
L = 4.0 # length of the road
dx = L / (nx - 1) # distance between two consecutive locations
nt = 40 # number of time steps to compute
rho_max = 10.0 # maximum taffic density allowed
u_max = 1.0 # maximum speed traffic
# Get the road locations.
x = numpy.linspace(0.0, L, num=nx)
# Compute the initial traffic density.
rho0 = rho_red_light(x, rho_max)
# Plot the initial traffic density.
fig = pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel(r'$x$')
pyplot.ylabel(r'$\rho$')
pyplot.grid()
line = pyplot.plot(x, rho0,
color='C0', linestyle='-', linewidth=2)[0]
pyplot.xlim(0.0, L)
pyplot.ylim(4.0, 11.0)
pyplot.tight_layout()
The question we would like to answer is: How will cars accumulate at the red light?
We will solve this problem using different numerical schemes, to see how they perform. These schemes are:
Before we do any coding, let's think about the equation a little bit. The wave speed uwave is −1 for ρ=ρmax and ρ≤ρmax/2, making all velocities negative. We should see a solution moving left, maintaining the shock geometry.
Now to some coding! First, let's define some useful functions and prepare to make some nice animations later.
def flux(rho, u_max, rho_max):
"""
Computes the traffic flux F = V * rho.
Parameters
----------
rho : numpy.ndarray
Traffic density along the road as a 1D array of floats.
u_max : float
Maximum speed allowed on the road.
rho_max : float
Maximum car density allowed on the road.
Returns
-------
F : numpy.ndarray
The traffic flux along the road as a 1D array of floats.
"""
F = rho * u_max * (1.0 - rho / rho_max)
return F
Before we investigate different schemes, let's create the function to update the Matplotlib figure during the animation.
from matplotlib import animation
from IPython.display import HTML
def update_plot(n, rho_hist):
"""
Update the line y-data of the Matplotlib figure.
Parameters
----------
n : integer
The time-step index.
rho_hist : list of numpy.ndarray objects
The history of the numerical solution.
"""
fig.suptitle('Time step {:0>2}'.format(n))
line.set_ydata(rho_hist[n])
Recall the conservation law for vehicle traffic, resulting in the following equation for the traffic density:
∂ρ∂t+∂F∂x=0F is the traffic flux, which in the linear traffic-speed model is given by:
F=ρumax(1−ρρmax)In the time variable, the natural choice for discretization is always a forward-difference formula; time invariably moves forward!
∂ρ∂t≈1Δt(ρn+1i−ρni)As is usual, the discrete locations on the 1D spatial grid are denoted by indices i and the discrete time instants are denoted by indices n.
In a convection problem, using first-order discretization in space leads to excessive numerical diffusion (as you probably observed in Lesson 1 of Module 2). The simplest approach to get second-order accuracy in space is to use a central difference:
∂F∂x≈12Δx(Fi+1−Fi−1)But combining these two choices for time and space discretization in the convection equation has catastrophic results! The "forward-time, central scheme" (FTCS) is unstable. (Go on: try it; you know you want to!)
The Lax-Friedrichs scheme was proposed by Lax (1954) as a clever trick to stabilize the forward-time, central scheme. The idea was to replace the solution value at ρni by the average of the values at the neighboring grid points. If we do that replacement, we get the following discretized equation:
ρn+1i−12(ρni+1+ρni−1)Δt=−Fni+1−Fni−12ΔxTake a careful look: the difference formula no longer uses the value at ρni to obtain ρn+1i. The stencil of the Lax-Friedrichs scheme is slightly different than that for the forward-time, central scheme.
This numerical discretization is stable. Unfortunately, substituting ρni by the average of its neighbors introduces a first-order error. Nice try, Lax!
To implement the scheme in code, we need to isolate the value at the next time step, ρn+1i, so we can write a time-stepping loop:
ρn+1i=12(ρni+1+ρni−1)−Δt2Δx(Fni+1−Fni−1)The function below implements Lax-Friedrichs for our traffic model. All the schemes in this notebook are wrapped in their own functions to help with displaying animations of the results. This is also good practice for developing modular, reusable code.
In order to display animations, we're going to hold the results of each time step in the variable rho
, a 2D array. The resulting array rho_n
has nt
rows and nx
columns.
def lax_friedrichs(rho0, nt, dt, dx, bc_values, *args):
"""
Computes the traffic density on the road
at a certain time given the initial traffic density.
Integration using Lax-Friedrichs scheme.
Parameters
----------
rho0 : numpy.ndarray
The initial traffic density along the road
as a 1D array of floats.
nt : integer
The number of time steps to compute.
dt : float
The time-step size to integrate.
dx : float
The distance between two consecutive locations.
bc_values : 2-tuple of floats
The value of the density at the first and last locations.
args : list or tuple
Positional arguments to be passed to the flux function.
Returns
-------
rho_hist : list of numpy.ndarray objects
The history of the car density along the road.
"""
rho_hist = [rho0.copy()]
rho = rho0.copy()
for n in range(nt):
# Compute the flux.
F = flux(rho, *args)
# Advance in time using Lax-Friedrichs scheme.
rho[1:-1] = (0.5 * (rho[:-2] + rho[2:]) -
dt / (2.0 * dx) * (F[2:] - F[:-2]))
# Set the value at the first location.
rho[0] = bc_values[0]
# Set the value at the last location.
rho[-1] = bc_values[1]
# Record the time-step solution.
rho_hist.append(rho.copy())
return rho_hist
We are now all set to run! First, let's try with CFL=1
# Set the time-step size based on CFL limit.
sigma = 1.0
dt = sigma * dx / u_max # time-step size
# Compute the traffic density at all time steps.
rho_hist = lax_friedrichs(rho0, nt, dt, dx, (rho0[0], rho0[-1]),
u_max, rho_max)
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
frames=nt, fargs=(rho_hist,),
interval=100)
# Display the video.
HTML(anim.to_html5_video())
Would the solution improve if we use smaller time steps? Let's check that!
# Set the time-step size based on CFL limit.
sigma = 0.5
dt = sigma * dx / u_max # time-step size
# Compute the traffic density at all time steps.
rho_hist = lax_friedrichs(rho0, nt, dt, dx, (rho0[0], rho0[-1]),
u_max, rho_max)
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
frames=nt, fargs=(rho_hist,),
interval=100)
# Display the video.
HTML(anim.to_html5_video())
Notice the strange "staircase" behavior on the leading edge of the wave? You may be interested to learn more about this: a feature typical of what is sometimes called "odd-even decoupling." Last year we published a collection of lessons in Computational Fluid Dynamics, called CFD Python, where we discuss odd-even decoupling.
The Lax-Friedrichs method uses a clever trick to stabilize the central difference in space for convection, but loses an order of accuracy in doing so. First-order methods are just not good enough for convection problems, especially when you have sharp gradients (shocks).
The Lax-Wendroff (1960) method was the first scheme ever to achieve second-order accuracy in both space and time. It is therefore a landmark in the history of computational fluid dynamics.
To develop the Lax-Wendroff scheme, we need to do a bit of work. Sit down, grab a notebook and grit your teeth. We want you to follow this derivation in your own hand. It's good for you! Start with the Taylor series expansion (in the time variable) about ρn+1:
ρn+1=ρn+∂ρn∂tΔt+(Δt)22∂2ρn∂t2+…For the conservation law with F=F(ρ), and using our beloved chain rule, we can write:
∂ρ∂t=−∂F∂x=−∂F∂ρ∂ρ∂x=−J∂ρ∂xwhere
J=∂F∂ρ=umax(1−2ρρmax)is the Jacobian for the traffic model. Next, we can do a little trickery:
∂F∂t=∂F∂ρ∂ρ∂t=J∂ρ∂t=−J∂F∂xIn the last step above, we used the differential equation of the traffic model to replace the time derivative by a spatial derivative. These equivalences imply that
∂2ρ∂t2=∂∂x(J∂F∂x)Let's use all this in the Taylor expansion:
ρn+1=ρn−∂Fn∂xΔt+(Δt)22∂∂x(J∂Fn∂x)+…We can now reorganize this and discretize the spatial derivatives with central differences to get the following discrete equation:
ρn+1i−ρniΔt=−Fni+1−Fni−12Δx+Δt2((J∂F∂x)ni+12−(J∂F∂x)ni−12Δx)Now, approximate the rightmost term (inside the parenthesis) in the above equation as follows: Jni+12(Fni+1−FniΔx)−Jni−12(Fni−Fni−1Δx)Δx
Then evaluate the Jacobian at the midpoints by using averages of the points on either side:
12Δx(Jni+1+Jni)(Fni+1−Fni)−12Δx(Jni+Jni−1)(Fni−Fni−1)Δx.Our equation now reads:
ρn+1i−ρniΔt=−Fni+1−Fni−12Δx+⋯+Δt4Δx2((Jni+1+Jni)(Fni+1−Fni)−(Jni+Jni−1)(Fni−Fni−1))Solving for ρn+1i:
ρn+1i=ρni−Δt2Δx(Fni+1−Fni−1)+⋯+(Δt)24(Δx)2[(Jni+1+Jni)(Fni+1−Fni)−(Jni+Jni−1)(Fni−Fni−1)]with
Jni=∂F∂ρ=umax(1−2ρniρmax).Lax-Wendroff is a little bit long. Remember that you can use \ slashes to split up a statement across several lines. This can help make code easier to parse (and also easier to debug!).
def jacobian(rho, u_max, rho_max):
"""
Computes the Jacobian for our traffic model.
Parameters
----------
rho : numpy.ndarray
Traffic density along the road as a 1D array of floats.
u_max : float
Maximum speed allowed on the road.
rho_max : float
Maximum car density allowed on the road.
Returns
-------
J : numpy.ndarray
The Jacobian as a 1D array of floats.
"""
J = u_max * (1.0 - 2.0 * rho / rho_max)
return J
def lax_wendroff(rho0, nt, dt, dx, bc_values, *args):
"""
Computes the traffic density on the road
at a certain time given the initial traffic density.
Integration using Lax-Wendroff scheme.
Parameters
----------
rho0 : numpy.ndarray
The initial traffic density along the road
as a 1D array of floats.
nt : integer
The number of time steps to compute.
dt : float
The time-step size to integrate.
dx : float
The distance between two consecutive locations.
bc_values : 2-tuple of floats
The value of the density at the first and last locations.
args : list or tuple
Positional arguments to be passed to the
flux and Jacobien functions.
Returns
-------
rho_hist : list of numpy.ndarray objects
The history of the car density along the road.
"""
rho_hist = [rho0.copy()]
rho = rho0.copy()
for n in range(nt):
# Compute the flux.
F = flux(rho, *args)
# Compute the Jacobian.
J = jacobian(rho, *args)
# Advance in time using Lax-Wendroff scheme.
rho[1:-1] = (rho[1:-1] -
dt / (2.0 * dx) * (F[2:] - F[:-2]) +
dt**2 / (4.0 * dx**2) *
((J[1:-1] + J[2:]) * (F[2:] - F[1:-1]) -
(J[:-2] + J[1:-1]) * (F[1:-1] - F[:-2])))
# Set the value at the first location.
rho[0] = bc_values[0]
# Set the value at the last location.
rho[-1] = bc_values[1]
# Record the time-step solution.
rho_hist.append(rho.copy())
return rho_hist
Now that's we've defined a function for the Lax-Wendroff scheme, we can use the same procedure as above to animate and view our results.
# Set the time-step size based on CFL limit.
sigma = 1.0
dt = sigma * dx / u_max # time-step size
# Compute the traffic density at all time steps.
rho_hist = lax_wendroff(rho0, nt, dt, dx, (rho0[0], rho0[-1]),
u_max, rho_max)
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
frames=nt, fargs=(rho_hist,),
interval=100)
# Display the video.
HTML(anim.to_html5_video())
Interesting! The Lax-Wendroff method captures the sharpness of the shock much better than the Lax-Friedrichs scheme, but there is a new problem: a strange wiggle appears right at the tail of the shock. This is typical of many second-order methods: they introduce numerical oscillations where the solution is not smooth. Bummer.
How do the oscillations at the shock front vary with changes to the CFL condition? You might think that the solution will improve if you make the time step smaller ... let's see.
# Set the time-step size based on CFL limit.
sigma = 0.5
dt = sigma * dx / u_max # time-step size
# Compute the traffic density at all time steps.
rho_hist = lax_wendroff(rho0, nt, dt, dx, (rho0[0], rho0[-1]),
u_max, rho_max)
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
frames=nt, fargs=(rho_hist,),
interval=100)
# Display the video.
HTML(anim.to_html5_video())
Eek! The numerical oscillations got worse. Double bummer!
Why do we observe oscillations with second-order methods? This is a question of fundamental importance!
The numerical oscillations that you observed with the Lax-Wendroff method on the traffic model can become severe in some problems. But actually the main drawback of the Lax-Wendroff method is having to calculate the Jacobian in every time step. With more complicated equations (like the Euler equations), calculating the Jacobian is a large computational expense.
Robert W. MacCormack introduced the first version of his now-famous method at the 1969 AIAA Hypervelocity Impact Conference, held in Cincinnati, Ohio, but the paper did not at first catch the attention of the aeronautics community. The next year, however, he presented at the 2nd International Conference on Numerical Methods in Fluid Dynamics at Berkeley. His paper there (MacCormack, 1971) was a landslide. MacCormack got a promotion and continued to work on applications of his method to the compressible Navier-Stokes equations. In 1973, NASA gave him the prestigious H. Julian Allen award for his work.
The MacCormack scheme is a two-step method, in which the first step is called a predictor and the second step is called a corrector. It achieves second-order accuracy in both space and time. One version is as follows:
ρ∗i=ρni−ΔtΔx(Fni+1−Fni) (predictor)If you look closely, it appears like the first step is a forward-time/forward-space scheme, and the second step is like a forward-time/backward-space scheme (these can also be reversed), averaged with the first result. What is so cool about this? You can compute problems with left-running waves and right-running waves, and the MacCormack scheme gives you a stable method (subject to the CFL condition). Nice! Let's try it.
def maccormack(rho0, nt, dt, dx, bc_values, *args):
"""
Computes the traffic density on the road
at a certain time given the initial traffic density.
Integration using MacCormack scheme.
Parameters
----------
rho0 : numpy.ndarray
The initial traffic density along the road
as a 1D array of floats.
nt : integer
The number of time steps to compute.
dt : float
The time-step size to integrate.
dx : float
The distance between two consecutive locations.
bc_values : 2-tuple of floats
The value of the density at the first and last locations.
args : list or tuple
Positional arguments to be passed to the flux function.
Returns
-------
rho_hist : list of numpy.ndarray objects
The history of the car density along the road.
"""
rho_hist = [rho0.copy()]
rho = rho0.copy()
rho_star = rho.copy()
for n in range(nt):
# Compute the flux.
F = flux(rho, *args)
# Predictor step of the MacCormack scheme.
rho_star[1:-1] = (rho[1:-1] -
dt / dx * (F[2:] - F[1:-1]))
# Compute the flux.
F = flux(rho_star, *args)
# Corrector step of the MacCormack scheme.
rho[1:-1] = 0.5 * (rho[1:-1] + rho_star[1:-1] -
dt / dx * (F[1:-1] - F[:-2]))
# Set the value at the first location.
rho[0] = bc_values[0]
# Set the value at the last location.
rho[-1] = bc_values[1]
# Record the time-step solution.
rho_hist.append(rho.copy())
return rho_hist
# Set the time-step size based on CFL limit.
sigma = 1.0
dt = sigma * dx / u_max # time-step size
# Compute the traffic density at all time steps.
rho_hist = maccormack(rho0, nt, dt, dx, (rho0[0], rho0[-1]),
u_max, rho_max)
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
frames=nt, fargs=(rho_hist,),
interval=100)
# Display the video.
HTML(anim.to_html5_video())
Once again, we ask: how does the CFL number affect the errors? Which one gives better results? You just have to try it.
# Set the time-step size based on CFL limit.
sigma = 0.5
dt = sigma * dx / u_max # time-step size
# Compute the traffic density at all time steps.
rho_hist = maccormack(rho0, nt, dt, dx, (rho0[0], rho0[-1]),
u_max, rho_max)
# Create an animation of the traffic density.
anim = animation.FuncAnimation(fig, update_plot,
frames=nt, fargs=(rho_hist,),
interval=100)
# Display the video.
HTML(anim.to_html5_video())
You can also obtain a MacCormack scheme by reversing the predictor and corrector steps. For shocks, the best resolution will occur when the difference in the predictor step is in the direction of propagation. Try it out! Was our choice here the ideal one? In which case is the shock better resolved?
In the red light problem, ρ≥ρmax/2, making the wave speed negative at all points . You might be wondering why we introduced these new methods; couldn't we have just used a forward-time/forward-space scheme? But, what if ρin<ρmax/2? Now, a whole region has negative wave speeds and forward-time/backward-space is unstable.
How do Lax-Friedrichs, Lax-Wendroff and MacCormack behave in this case? Try it out!
As you decrease ρin, what happens to the velocity of the shock? Why do you think that happens?
Peter D. Lax (1954), "Weak solutions of nonlinear hyperbolic equations and their numerical computation," Commun. Pure and Appl. Math., Vol. 7, pp. 159–193.
Peter D. Lax and Burton Wendroff (1960), "Systems of conservation laws," Commun. Pure and Appl. Math., Vol. 13, pp. 217–237.
R. W. MacCormack (1969), "The effect of viscosity in hypervelocity impact cratering," AIAA paper 69-354. Reprinted on Journal of Spacecraft and Rockets, Vol. 40, pp. 757–763 (2003). Also on Frontiers of Computational Fluid Dynamics, edited by D. A. Caughey, M. M. Hafez (2002), chapter 2: read on Google Books.
R. W. MacCormack (1971), "Numerical solution of the interaction of a shock wave with a laminar boundary layer," Proceedings of the 2nd Int. Conf. on Numerical Methods in Fluid Dynamics, Lecture Notes in Physics, Vol. 8, Springer, Berlin, pp. 151–163.
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, 'r').read())