# Generating C Code for the Scalar Wave Equation in Cartesian Coordinates

## Authors: Zach Etienne & Thiago Assumpção
### Formatting improvements courtesy Brandon Clark

## This module generates the C Code for the Scalarwave in Cartesian coordinates and sets up either monochromatic plane wave or spherical Gaussian [Initial Data](https://en.wikipedia.org/wiki/Initial_value_problem).

**Notebook Status:**  Validated

**Validation Notes:** This tutorial notebook has been confirmed to be self-consistent with its corresponding NRPy+ module, as documented below ([right-hand-side expressions](#code_validation1); [initial data expressions](#code_validation2)). In addition, all expressions have been validated against a trusted code (the [original SENR/NRPy+ code](https://bitbucket.org/zach_etienne/nrpy)).

### NRPy+ Source Code for this module:
* [ScalarWave/ScalarWave_RHSs.py](../edit/ScalarWave/ScalarWave_RHSs.py)
* [ScalarWave/InitialData.py](../edit/ScalarWave/InitialData.py)

## Introduction:
### Problem Statement

We wish to numerically solve the scalar wave equation as an [initial value problem](https://en.wikipedia.org/wiki/Initial_value_problem) in Cartesian coordinates:
$$\partial_t^2 u = c^2 \nabla^2 u \text{,}$$
where $u$ (the amplitude of the wave) is a function of time and space: $u = u(t,x,y,...)$ (spatial dimension as-yet unspecified) and $c$ is the wave speed, subject to some initial condition

$$u(0,x,y,...) = f(x,y,...)$$

and suitable spatial boundary conditions.

As described in the next section, we will find it quite useful to define
$$v(t,x,y,...) = \partial_t u(t,x,y,...).$$

In this way, the second-order PDE is reduced to a set of two coupled first-order PDEs

\begin{align}
\partial_t u &= v \\
\partial_t v &= c^2 \nabla^2 u.
\end{align}

We will use NRPy+ to generate efficient C codes capable of generating both initial data $u(0,x,y,...) = f(x,y,...)$; $v(0,x,y,...)=g(x,y,...)$, as well as finite-difference expressions for the right-hand sides of the above expressions. These expressions are needed within the *Method of Lines* to "integrate" the solution forward in time.

### The Method of Lines

Once we have initial data, we "evolve it forward in time", using the [Method of Lines](https://reference.wolfram.com/language/tutorial/NDSolveMethodOfLines.html). In short, the Method of Lines enables us to handle
1. the **spatial derivatives** of an initial value problem PDE using **standard finite difference approaches**, and
2. the **temporal derivatives** of an initial value problem PDE using **standard strategies for solving ordinary differential equations (ODEs)**, so long as the initial value problem PDE can be written in the form
$$\partial_t \vec{f} = \mathbf{M}\ \vec{f},$$
where $\mathbf{M}$ is an $N\times N$ matrix filled with differential operators that act on the $N$-element column vector $\vec{f}$. $\mathbf{M}$ may not contain $t$ or time derivatives explicitly; only *spatial* partial derivatives are allowed to appear inside $\mathbf{M}$. The scalar wave equation as written in the [previous module](Tutorial-ScalarWave.ipynb)
\begin{equation}
\partial_t
\begin{bmatrix}
u \\
v
\end{bmatrix}=
\begin{bmatrix}
0 & 1 \\
c^2 \nabla^2  & 0
\end{bmatrix}
\begin{bmatrix}
u \\
v
\end{bmatrix}
\end{equation}
satisfies this requirement.

Thus we can treat the spatial derivatives $\nabla^2 u$ of the scalar wave equation  using **standard finite-difference approaches**, and the temporal derivatives $\partial_t u$ and $\partial_t v$ using **standard approaches for solving ODEs**. In [the next module](Tutorial-Start_to_Finish-ScalarWave.ipynb), we will apply the highly robust [explicit Runge-Kutta fourth-order scheme](https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods) (RK4), used widely for numerically solving ODEs, to "march" (integrate) the solution vector $\vec{f}$ forward in time from its initial value ("initial data").

### Basic Algorithm

The basic algorithm for solving the scalar wave equation [initial value problem](https://en.wikipedia.org/wiki/Initial_value_problem), based on the Method of Lines (see section above) is outlined below, with NRPy+-based components highlighted in green. We will review how NRPy+ generates these core components in this module.

1. Allocate memory for gridfunctions, including temporary storage for the RK4 time integration.
1. Set gridfunction values to initial data.
1. Evolve the system forward in time using RK4 time integration. At each RK4 substep, do the following:
    1. Evaluate scalar wave RHS expressions.
    1. Apply boundary conditions.

**We refer to the right-hand side of the equation $\partial_t \vec{f} = \mathbf{M}\ \vec{f}$ as the RHS. In this case, we refer to the $\mathbf{M}\ \vec{f}$ as the "scalar wave RHSs".** In the following sections we will

1. Use NRPy+ to cast the scalar wave RHS expressions -- in finite difference form -- into highly efficient C code,
    1. first in one spatial dimension with fourth-order finite differences,
    1. and then in three spatial dimensions with tenth-order finite differences.
1. Use NRPy+ to generate monochromatic plane-wave initial data for the scalar wave equation, where the wave propagates in an arbitrary direction.

As for the $\nabla^2 u$ term, spatial derivatives are handled in NRPy+ via [finite differencing](https://en.wikipedia.org/wiki/Finite_difference).

We will sample the solution $\{u,v\}$ at discrete, uniformly-sampled points in space and time. For simplicity, let's assume that we consider the wave equation in one spatial dimension. Then the solution at any sampled point in space and time is given by
$$u^n_i = u(t_n,x_i) = u(t_0 + n \Delta t, x_0 + i \Delta x),$$
where $\Delta t$ and $\Delta x$ represent the temporal and spatial resolution, respectively. $v^n_i$ is sampled at the same points in space and time.

# Table of Contents
$$\label{toc}$$

1. [Step 1](#initializenrpy): Initialize core NRPy+ modules
1. [Step 2](#rhss1d): Scalar Wave RHSs in One Spatial Dimension, Fourth-Order Finite Differencing
    1. [Step 2.a](#ccode1d): C-code output example: Scalar wave RHSs with 4th order finite difference stencils
1. [Step 3](#rhss3d): Scalar Wave RHSs in Three Spatial Dimensions
    1. [Step 3.a](#code_validation1): Code Validation against `ScalarWave.ScalarWave_RHSs` NRPy+ module
    1. [Step 3.b](#ccode3d): C-code output example: Scalar wave RHSs with 10th order finite difference stencils and SIMD enabled
1. [Step 4](#id): Setting up Initial Data for the Scalar Wave Equation
    1. [Step 4.a](#planewave): The Monochromatic Plane-Wave Solution
    1. [Step 4.b](#sphericalgaussian): The Spherical Gaussian Solution (*Courtesy Thiago Assumpção*)
1. [Step 5](#code_validation2): Code Validation against `ScalarWave.InitialData` NRPy+ module
1. [Step 6](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Initialize core NRPy+ modules \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

Let's start by importing all the needed modules from NRPy+:

```python
# Step P1: Import needed NRPy+ core modules:
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import grid as gri               # NRPy+: Functions having to do with numerical grids
import finite_difference as fin  # NRPy+: Finite difference C code generation module
from outputC import lhrh         # NRPy+: Core C code output module
import sympy as sp               # SymPy: The Python computer algebra package upon which NRPy+ depends
```

# Step 2: Scalar Wave RHSs in One Spatial Dimension \[Back to [top](#toc)\]
$$\label{rhss1d}$$

To minimize complication, we will first restrict ourselves to solving the wave equation in one spatial dimension, so
\begin{align}
\partial_t u &= v \\
\partial_t v &= c^2 \nabla^2 u \\
 &= c^2 \partial_x^2 u.
\end{align}

We will construct SymPy expressions of the right-hand sides of $u$ and $v$ using [NRPy+ finite-difference notation](Tutorial-Finite_Difference_Derivatives.ipynb) to represent the derivative, so that finite-difference C-code kernels can be easily constructed.

Extension of this operator to higher spatial dimensions when using NRPy+ is straightforward, as we will see below.

```python
# Step P2: Define the C parameter wavespeed. The `wavespeed`
#          variable is a proper SymPy variable, so it can be
#          used in below expressions. In the C code, it acts
#          just like a usual parameter, whose value is
#          specified in the parameter file.
thismodule = "ScalarWave"
wavespeed = par.Cparameters("REAL",thismodule,"wavespeed", 1.0)

# Step 1: Set the spatial dimension parameter, and then read
#         the parameter as DIM.
par.set_parval_from_str("grid::DIM", 1)
DIM = par.parval_from_str("grid::DIM")

# Step 2: Register gridfunctions that are needed as input
#         to the scalar wave RHS expressions.
uu, vv = gri.register_gridfunctions("EVOL", ["uu", "vv"])

# Step 3: Declare the rank-2 indexed expression \partial_{ij} u,
#         which is symmetric about interchange of indices i and j
#         Derivative variables like these must have an underscore
#         in them, so the finite difference module can parse the
#         variable name properly.
uu_dDD = ixp.declarerank2("uu_dDD", "sym01")

# Step 4: Define right-hand sides for the evolution.
uu_rhs = vv
vv_rhs = 0
for i in range(DIM):
    vv_rhs += wavespeed*wavespeed*uu_dDD[i][i]

vv_rhs = sp.simplify(vv_rhs)
```

## Step 2.a: C-code output example: Scalar wave RHSs with 4th order finite difference stencils \[Back to [top](#toc)\]
$$\label{ccode1d}$$

As was discussed in [the finite difference section of the tutorial](Tutorial-Finite_Difference_Derivatives.ipynb), NRPy+ approximates derivatives using [finite-difference methods](https://en.wikipedia.org/wiki/Finite_difference),  the second-order derivative $\partial_x^2$ accurate to fourth-order in uniform grid spacing $\Delta x$ (from fitting the unique 4th-degree polynomial to 5 sample points of $u$) is given by
\begin{equation}
\left[\partial_x^2 u(t,x)\right]_j = \frac{1}{(\Delta x)^2}
\left(
-\frac{1}{12} \left(u_{j+2} + u_{j-2}\right)
+ \frac{4}{3}  \left(u_{j+1} + u_{j-1}\right)
- \frac{5}{2} u_j \right)
+ \mathcal{O}\left((\Delta x)^4\right).
\end{equation}

```python
# Step 5: Set the finite differencing order to 4.
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER", 4)

# Step 6: Generate C code for scalarwave evolution equations,
#         print output to the screen (standard out, or stdout).
fin.FD_outputC("stdout",
               [lhrh(lhs=gri.gfaccess("rhs_gfs", "uu"), rhs=uu_rhs),
                lhrh(lhs=gri.gfaccess("rhs_gfs", "vv"), rhs=vv_rhs)])
```

    {
       /*
        * NRPy+ Finite Difference Code Generation, Step 1 of 2: Read from main memory and compute finite difference stencils:
        */
       /*
        *  Original SymPy expression:
        *  "const double uu_dDD00 = invdx0**2*(-5*uu/2 + 4*uu_i0m1/3 - uu_i0m2/12 + 4*uu_i0p1/3 - uu_i0p2/12)"
        */
       const double uu_i0m2 = in_gfs[IDX2(UUGF, i0-2)];
       const double uu_i0m1 = in_gfs[IDX2(UUGF, i0-1)];
       const double uu = in_gfs[IDX2(UUGF, i0)];
       const double uu_i0p1 = in_gfs[IDX2(UUGF, i0+1)];
       const double uu_i0p2 = in_gfs[IDX2(UUGF, i0+2)];
       const double vv = in_gfs[IDX2(VVGF, i0)];
       const double FDPart1_Rational_5_2 = 5.0/2.0;
       const double FDPart1_Rational_1_12 = 1.0/12.0;
       const double FDPart1_Rational_4_3 = 4.0/3.0;
       const double uu_dDD00 = ((invdx0)*(invdx0))*(FDPart1_Rational_1_12*(-uu_i0m2 - uu_i0p2) + FDPart1_Rational_4_3*(uu_i0m1 + uu_i0p1) - FDPart1_Rational_5_2*uu);
       /*
        * NRPy+ Finite Difference Code Generation, Step 2 of 2: Evaluate SymPy expressions and write to main memory:
        */
       /*
        *  Original SymPy expressions:
        *  "[rhs_gfs[IDX2(UUGF, i0)] = vv,
        *    rhs_gfs[IDX2(VVGF, i0)] = uu_dDD00*wavespeed**2]"
        */
       rhs_gfs[IDX2(UUGF, i0)] = vv;
       rhs_gfs[IDX2(VVGF, i0)] = uu_dDD00*((wavespeed)*(wavespeed));
    }

**Success!** Notice that indeed NRPy+ was able to compute the spatial derivative operator,
\begin{equation}
\left[\partial_x^2 u(t,x)\right]_j \approx \frac{1}{(\Delta x)^2}
\left(
-\frac{1}{12} \left(u_{j+2} + u_{j-2}\right)
+ \frac{4}{3}  \left(u_{j+1} + u_{j-1}\right)
- \frac{5}{2} u_j \right),
\end{equation}
correctly (easier to read in the "Original SymPy expressions" comment block at the top of the C output.

As NRPy+ is designed to generate codes in arbitrary coordinate systems, instead of sticking with Cartesian notation for 3D coordinates, $x,y,z$, we instead adopt $x_0,x_1,x_2$ for our coordinate labels. Thus you will notice the appearance of `invdx0`$=1/\Delta x_0$, where $\Delta x_0$ as the (uniform) grid spacing in the zeroth, or $x_0$ direction. In this case $x_0$ represents the $x$ direction.

# Step 3: Scalar Wave RHSs in Three Spatial Dimensions \[Back to [top](#toc)\]
$$\label{rhss3d}$$

Let's next repeat the same process, only this time for the scalar wave equation in **3 spatial dimensions** (3D).

```python
# Step 1: Define the C parameter wavespeed. The `wavespeed`
#         variable is a proper SymPy variable, so it can be
#         used in below expressions. In the C code, it acts
#         just like a usual parameter, whose value is
#         specified in the parameter file.
wavespeed = par.Cparameters("REAL", thismodule, "wavespeed", 1.0)

# Step 2: Set the spatial dimension parameter
#         to *FOUR* this time, and then read
#         the parameter as DIM.
par.set_parval_from_str("grid::DIM", 3)
DIM = par.parval_from_str("grid::DIM")

# Step 3a: Reset gridfunctions registered in 1D case above,
#          to avoid NRPy+ throwing an error about double-
#          registering gridfunctions, which is not allowed.
gri.glb_gridfcs_list = []

# Step 3b: Register gridfunctions that are needed as input
#          to the scalar wave RHS expressions.
uu, vv = gri.register_gridfunctions("EVOL", ["uu", "vv"])

# Step 4: Declare the rank-2 indexed expression \partial_{ij} u,
#         which is symmetric about interchange of indices i and j
#         Derivative variables like these must have an underscore
#         in them, so the finite difference module can parse the
#         variable name properly.
uu_dDD = ixp.declarerank2("uu_dDD", "sym01")

# Step 5: Define right-hand sides for the evolution.
uu_rhs = vv
vv_rhs = 0
for i in range(DIM):
    vv_rhs += wavespeed*wavespeed*uu_dDD[i][i]

# Step 6: Simplify the expression for c^2 \nabla^2 u (a.k.a., vv_rhs):
vv_rhs = sp.simplify(vv_rhs)
```

## Step 3.a:  Validate SymPy expressions against `ScalarWave.ScalarWave_RHSs` NRPy+ module \[Back to [top](#toc)\]
$$\label{code_validation1}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the RHSs of the three-spatial-dimension Scalar Wave equation (i.e., `uu_rhs` and `vv_rhs`) between

1. this tutorial and
2. the [NRPy+ ScalarWave.ScalarWave_RHSs](../edit/ScalarWave/ScalarWave_RHSs.py) module.

```python
# Step 10: We already have SymPy expressions for uu_rhs and vv_rhs in
#          terms of other SymPy variables. Even if we reset the list
#          of NRPy+ gridfunctions, these *SymPy* expressions for
#          uu_rhs and vv_rhs *will remain unaffected*.
#
#          Here, we will use the above-defined uu_rhs and vv_rhs to
#          validate against the same expressions in the
#          ScalarWave/ScalarWave_RHSs.py module,
#          to ensure consistency between this tutorial
#          (historically speaking, the tutorial was written first)
#          and the ScalarWave_RHSs.py module itself.
#
# Reset the list of gridfunctions, as registering a gridfunction
#   twice will spawn an error.
gri.glb_gridfcs_list = []

# Step 11: Call the ScalarWave_RHSs() function from within the
#         ScalarWave/ScalarWave_RHSs.py module,
#         which should do exactly the same as in Steps 1-10 above.
import ScalarWave.ScalarWave_RHSs as swrhs
swrhs.ScalarWave_RHSs()

# Step 12: Consistency check between the tutorial notebook above
#         and the ScalarWave_RHSs() function from within the
#         ScalarWave/ScalarWave_RHSs.py module.
print("Consistency check between ScalarWave tutorial and NRPy+ module:")
print("uu_rhs - swrhs.uu_rhs = "+str(sp.simplify(uu_rhs - swrhs.uu_rhs))+"\t\t (should be zero)")
print("vv_rhs - swrhs.vv_rhs = "+str(sp.simplify(vv_rhs - swrhs.vv_rhs))+"\t\t (should be zero)")
```

    Consistency check between ScalarWave tutorial and NRPy+ module:
    uu_rhs - swrhs.uu_rhs = 0		 (should be zero)
    vv_rhs - swrhs.vv_rhs = 0		 (should be zero)

## Step 3.b: C-code output example: Scalar wave RHSs with 10th order finite difference stencils and SIMD enabled \[Back to [top](#toc)\]
$$\label{ccode3d}$$

Next we'll output the above expressions as Ccode, using the [NRPy+ finite-differencing C code kernel generation infrastructure](Tutorial-Finite_Difference_Derivatives.ipynb). This code will represent spatial derivatives as 10th-order finite differences and output the C code with [SIMD](https://en.wikipedia.org/wiki/SIMD) enabled. ([Common-subexpression elimination](https://en.wikipedia.org/wiki/Common_subexpression_elimination) is enabled by default.)

```python
# Step 7: Set the finite differencing order to 10.
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER", 10)

# Step 8: Generate C code for scalarwave evolution equations,
#         print output to the screen (standard out, or stdout).
fin.FD_outputC("stdout",
               [lhrh(lhs=gri.gfaccess("rhs_gfs","uu"),rhs=uu_rhs),
                lhrh(lhs=gri.gfaccess("rhs_gfs","vv"),rhs=vv_rhs)], params="enable_SIMD=True")
```

    {
       /*
        * NRPy+ Finite Difference Code Generation, Step 1 of 2: Read from main memory and compute finite difference stencils:
        */
       /*
        *  Original SymPy expressions:
        *  "[const REAL_SIMD_ARRAY uu_dDD00 = invdx0**2*(-5269*uu/1800 + 5*uu_i0m1_i1_i2/3 - 5*uu_i0m2_i1_i2/21 + 5*uu_i0m3_i1_i2/126 - 5*uu_i0m4_i1_i2/1008 + uu_i0m5_i1_i2/3150 + 5*uu_i0p1_i1_i2/3 - 5*uu_i0p2_i1_i2/21 + 5*uu_i0p3_i1_i2/126 - 5*uu_i0p4_i1_i2/1008 + uu_i0p5_i1_i2/3150),
        *    const REAL_SIMD_ARRAY uu_dDD11 = invdx1**2*(-5269*uu/1800 + 5*uu_i0_i1m1_i2/3 - 5*uu_i0_i1m2_i2/21 + 5*uu_i0_i1m3_i2/126 - 5*uu_i0_i1m4_i2/1008 + uu_i0_i1m5_i2/3150 + 5*uu_i0_i1p1_i2/3 - 5*uu_i0_i1p2_i2/21 + 5*uu_i0_i1p3_i2/126 - 5*uu_i0_i1p4_i2/1008 + uu_i0_i1p5_i2/3150),
        *    const REAL_SIMD_ARRAY uu_dDD22 = invdx2**2*(-5269*uu/1800 + 5*uu_i0_i1_i2m1/3 - 5*uu_i0_i1_i2m2/21 + 5*uu_i0_i1_i2m3/126 - 5*uu_i0_i1_i2m4/1008 + uu_i0_i1_i2m5/3150 + 5*uu_i0_i1_i2p1/3 - 5*uu_i0_i1_i2p2/21 + 5*uu_i0_i1_i2p3/126 - 5*uu_i0_i1_i2p4/1008 + uu_i0_i1_i2p5/3150)]"
        */
       const REAL_SIMD_ARRAY uu_i0_i1_i2m5 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2-5)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2m4 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2-4)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2m3 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2-3)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2m2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2-2)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2m1 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2-1)]);
       const REAL_SIMD_ARRAY uu_i0_i1m5_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1-5,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1m4_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1-4,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1m3_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1-3,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1m2_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1-2,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1m1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1-1,i2)]);
       const REAL_SIMD_ARRAY uu_i0m5_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0-5,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0m4_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0-4,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0m3_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0-3,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0m2_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0-2,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0m1_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0-1,i1,i2)]);
       const REAL_SIMD_ARRAY uu = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0p1_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0+1,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0p2_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0+2,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0p3_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0+3,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0p4_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0+4,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0p5_i1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0+5,i1,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1p1_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1+1,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1p2_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1+2,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1p3_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1+3,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1p4_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1+4,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1p5_i2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1+5,i2)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2p1 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2+1)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2p2 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2+2)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2p3 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2+3)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2p4 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2+4)]);
       const REAL_SIMD_ARRAY uu_i0_i1_i2p5 = ReadSIMD(&in_gfs[IDX4(UUGF, i0,i1,i2+5)]);
       const REAL_SIMD_ARRAY vv = ReadSIMD(&in_gfs[IDX4(VVGF, i0,i1,i2)]);
       const double tmpFDPart1_NegativeOne_ = -1.0;
       const REAL_SIMD_ARRAY FDPart1_NegativeOne_ = ConstSIMD(tmpFDPart1_NegativeOne_);

       const double tmpFDPart1_Rational_1_3150 = 1.0/3150.0;
       const REAL_SIMD_ARRAY FDPart1_Rational_1_3150 = ConstSIMD(tmpFDPart1_Rational_1_3150);

       const double tmpFDPart1_Rational_5269_1800 = 5269.0/1800.0;
       const REAL_SIMD_ARRAY FDPart1_Rational_5269_1800 = ConstSIMD(tmpFDPart1_Rational_5269_1800);

       const double tmpFDPart1_Rational_5_1008 = 5.0/1008.0;
       const REAL_SIMD_ARRAY FDPart1_Rational_5_1008 = ConstSIMD(tmpFDPart1_Rational_5_1008);

       const double tmpFDPart1_Rational_5_126 = 5.0/126.0;
       const REAL_SIMD_ARRAY FDPart1_Rational_5_126 = ConstSIMD(tmpFDPart1_Rational_5_126);

       const double tmpFDPart1_Rational_5_21 = 5.0/21.0;
       const REAL_SIMD_ARRAY FDPart1_Rational_5_21 = ConstSIMD(tmpFDPart1_Rational_5_21);

       const double tmpFDPart1_Rational_5_3 = 5.0/3.0;
       const REAL_SIMD_ARRAY FDPart1_Rational_5_3 = ConstSIMD(tmpFDPart1_Rational_5_3);

       const REAL_SIMD_ARRAY FDPart1_0 = MulSIMD(FDPart1_Rational_5269_1800, uu);
       const REAL_SIMD_ARRAY uu_dDD00 = MulSIMD(MulSIMD(invdx0, invdx0), FusedMulAddSIMD(FDPart1_Rational_5_126, AddSIMD(uu_i0m3_i1_i2, uu_i0p3_i1_i2), FusedMulAddSIMD(FDPart1_Rational_5_3, AddSIMD(uu_i0m1_i1_i2, uu_i0p1_i1_i2), FusedMulSubSIMD(FDPart1_Rational_1_3150, AddSIMD(uu_i0m5_i1_i2, uu_i0p5_i1_i2), FusedMulAddSIMD(FDPart1_Rational_5_1008, AddSIMD(uu_i0m4_i1_i2, uu_i0p4_i1_i2), FusedMulAddSIMD(FDPart1_Rational_5_21, AddSIMD(uu_i0m2_i1_i2, uu_i0p2_i1_i2), FDPart1_0))))));
       const REAL_SIMD_ARRAY uu_dDD11 = MulSIMD(MulSIMD(invdx1, invdx1), FusedMulAddSIMD(FDPart1_Rational_5_126, AddSIMD(uu_i0_i1m3_i2, uu_i0_i1p3_i2), FusedMulAddSIMD(FDPart1_Rational_5_3, AddSIMD(uu_i0_i1m1_i2, uu_i0_i1p1_i2), FusedMulSubSIMD(FDPart1_Rational_1_3150, AddSIMD(uu_i0_i1m5_i2, uu_i0_i1p5_i2), FusedMulAddSIMD(FDPart1_Rational_5_1008, AddSIMD(uu_i0_i1m4_i2, uu_i0_i1p4_i2), FusedMulAddSIMD(FDPart1_Rational_5_21, AddSIMD(uu_i0_i1m2_i2, uu_i0_i1p2_i2), FDPart1_0))))));
       const REAL_SIMD_ARRAY uu_dDD22 = MulSIMD(MulSIMD(invdx2, invdx2), FusedMulAddSIMD(FDPart1_Rational_5_126, AddSIMD(uu_i0_i1_i2m3, uu_i0_i1_i2p3), FusedMulAddSIMD(FDPart1_Rational_5_3, AddSIMD(uu_i0_i1_i2m1, uu_i0_i1_i2p1), FusedMulSubSIMD(FDPart1_Rational_1_3150, AddSIMD(uu_i0_i1_i2m5, uu_i0_i1_i2p5), FusedMulAddSIMD(FDPart1_Rational_5_1008, AddSIMD(uu_i0_i1_i2m4, uu_i0_i1_i2p4), FusedMulAddSIMD(FDPart1_Rational_5_21, AddSIMD(uu_i0_i1_i2m2, uu_i0_i1_i2p2), FDPart1_0))))));
       /*
        * NRPy+ Finite Difference Code Generation, Step 2 of 2: Evaluate SymPy expressions and write to main memory:
        */
       /*
        *  Original SymPy expressions:
        *  "[const REAL_SIMD_ARRAY __RHS_exp_0 = vv,
        *    const REAL_SIMD_ARRAY __RHS_exp_1 = wavespeed**2*(uu_dDD00 + uu_dDD11 + uu_dDD22)]"
        */
       const REAL_SIMD_ARRAY __RHS_exp_0 = vv;
       const REAL_SIMD_ARRAY __RHS_exp_1 = MulSIMD(MulSIMD(wavespeed, wavespeed), AddSIMD(uu_dDD00, AddSIMD(uu_dDD11, uu_dDD22)));
       WriteSIMD(&rhs_gfs[IDX4(UUGF, i0, i1, i2)], __RHS_exp_0);
       WriteSIMD(&rhs_gfs[IDX4(VVGF, i0, i1, i2)], __RHS_exp_1);
    }

# Step 4: Setting up Initial Data for the Scalar Wave Equation \[Back to [top](#toc)\]
$$\label{id}$$

## Step 4.a: The Monochromatic Plane-Wave Solution \[Back to [top](#toc)\]
$$\label{planewave}$$

The solution to the scalar wave equation for a monochromatic (single-wavelength) wave traveling in the $\hat{k}$ direction is
$$u(\vec{x},t) = f(\hat{k}\cdot\vec{x} - c t),$$
where $\hat{k}$ is a unit vector. We choose $f(\hat{k}\cdot\vec{x} - c t)$ to take the form
$$
f(\hat{k}\cdot\vec{x} - c t) = \sin\left(\hat{k}\cdot\vec{x} - c t\right) + 2,
$$
where we add the $+2$ to ensure that the exact solution never crosses through zero. In places where the exact solution passes through zero, the relative error (i.e., the most common error measure used to check that the numerical solution converges to the exact solution) is undefined. Also, $f(\hat{k}\cdot\vec{x} - c t)$ plus a constant is still a solution to the wave equation.

```python
# Step 1: Set parameters defined in other modules
xx = gri.xx # Sets the Cartesian coordinates xx[0]=x; xx[1]=y; xx[2]=z

# Step 2: Declare free parameters intrinsic to these initial data
time = par.Cparameters("REAL", thismodule, "time",0.0)
kk = par.Cparameters("REAL", thismodule, ["kk0", "kk1", "kk2"],[1.0,1.0,1.0])

# Step 3: Normalize the k vector
kk_norm = sp.sqrt(kk[0]**2 + kk[1]**2 + kk[2]**2)

# Step 4: Compute k.x
dot_product = sp.sympify(0)
for i in range(DIM):
    dot_product += xx[i]*kk[i]
dot_product /= kk_norm

# Step 5: Set initial data for uu and vv, where vv_ID = \partial_t uu_ID.
uu_ID_PlaneWave = sp.sin(dot_product - wavespeed*time)+2
vv_ID_PlaneWave = sp.diff(uu_ID_PlaneWave, time)
```

Next we verify that $f(\hat{k}\cdot\vec{x} - c t)$ satisfies the wave equation, by computing
$$\left(c^2 \nabla^2 - \partial_t^2 \right)\ f\left(\hat{k}\cdot\vec{x} - c t\right),$$
and confirming the result is exactly zero.

```python
sp.simplify(wavespeed**2*(sp.diff(uu_ID_PlaneWave,xx[0],2) +
                          sp.diff(uu_ID_PlaneWave,xx[1],2) +
                          sp.diff(uu_ID_PlaneWave,xx[2],2))
            - sp.diff(uu_ID_PlaneWave,time,2))
```

$\displaystyle 0$

## Step 4.b: The Spherical Gaussian Solution \[Back to [top](#toc)\]
$$\label{sphericalgaussian}$$

Here we will implement the spherical Gaussian solution, consists of ingoing and outgoing wave fronts:
\begin{align}
u(r,t) &= u_{\rm out}(r,t) + u_{\rm in}(r,t) + 1,\ \ \text{where}\\
u_{\rm out}(r,t) &=\frac{r-ct}{r} \exp\left[\frac{-(r-ct)^2}{2 \sigma^2}\right] \\
u_{\rm in}(r,t) &=\frac{r+ct}{r} \exp\left[\frac{-(r+ct)^2}{2 \sigma^2}\right] \\
\end{align}
where $c$ is the wavespeed, and $\sigma$ is the width of the Gaussian (i.e., the "standard deviation").

```python
# Step 1: Set parameters defined in other modules
xx = gri.xx # Sets the Cartesian coordinates xx[0]=x; xx[1]=y; xx[2]=z

# Step 2: Declare free parameters intrinsic to these initial data
time  = par.Cparameters("REAL", thismodule, "time",0.0)
sigma = par.Cparameters("REAL", thismodule, "sigma",3.0)

# Step 4: Compute r
r = sp.sympify(0)
for i in range(DIM):
    r += xx[i]**2
r = sp.sqrt(r)

# Step 5: Set initial data for uu and vv, where vv_ID = \partial_t uu_ID.
uu_ID_SphericalGaussianOUT = +(r - wavespeed*time)/r * sp.exp( -(r - wavespeed*time)**2 / (2*sigma**2) )
uu_ID_SphericalGaussianIN  = +(r + wavespeed*time)/r * sp.exp( -(r + wavespeed*time)**2 / (2*sigma**2) )
uu_ID_SphericalGaussian = uu_ID_SphericalGaussianOUT + uu_ID_SphericalGaussianIN + sp.sympify(1)
vv_ID_SphericalGaussian = sp.diff(uu_ID_SphericalGaussian, time)
```

Since the wave equation is linear, both the leftgoing and rightgoing waves must satisfy the wave equation, which implies that their sum also satisfies the wave equation.

Next we verify that $u(r,t)$ satisfies the wave equation, by computing
$$\left(c^2 \nabla^2 - \partial_t^2 \right)\left\{u_{\rm R}(r,t)\right\},$$

and

$$\left(c^2 \nabla^2 - \partial_t^2 \right)\left\{u_{\rm L}(r,t)\right\},$$

are separately zero. We do this because SymPy has difficulty simplifying the combined expression.

```python
print(sp.simplify(wavespeed**2*(sp.diff(uu_ID_SphericalGaussianOUT,xx[0],2) +
                                sp.diff(uu_ID_SphericalGaussianOUT,xx[1],2) +
                                sp.diff(uu_ID_SphericalGaussianOUT,xx[2],2))
                - sp.diff(uu_ID_SphericalGaussianOUT,time,2)) )

print(sp.simplify(wavespeed**2*(sp.diff(uu_ID_SphericalGaussianIN,xx[0],2) +
                                sp.diff(uu_ID_SphericalGaussianIN,xx[1],2) +
                                sp.diff(uu_ID_SphericalGaussianIN,xx[2],2))
                    - sp.diff(uu_ID_SphericalGaussianIN,time,2)))
```

    0
    0

# Step 5: Code Validation against `ScalarWave.InitialData` NRPy+ module  \[Back to [top](#toc)\]
$$\label{code_validation2}$$

As a code validation check, we will verify agreement in the SymPy expressions for plane-wave initial data for the Scalar Wave equation between
1. this tutorial and
2. the NRPy+ [ScalarWave.InitialData](../edit/ScalarWave/InitialData.py) module.

```python
# We just defined SymPy expressions for uu_ID and vv_ID in
# terms of other SymPy variables. Here, we will use the
# above-defined uu_ID and vv_ID to validate against the
# same expressions in the ScalarWave/InitialData.py
# module, to ensure consistency between this tutorial
# (historically speaking, the tutorial was written first)
# and the PlaneWave ID module itself.
#
# Step 6: Call the InitialData(Type="PlaneWave") function from within the
#         ScalarWave/InitialData.py module,
#         which should do exactly the same as in Steps 1-5 above.
import ScalarWave.InitialData as swid
swid.InitialData(Type="PlaneWave")

# Step 7: Consistency check between the tutorial notebook above
#         and the PlaneWave option from within the
#         ScalarWave/InitialData.py module.
print("Consistency check between ScalarWave tutorial and NRPy+ module: PlaneWave Case")
if sp.simplify(uu_ID_PlaneWave - swid.uu_ID) != 0:
    print("TEST FAILED: uu_ID_PlaneWave - swid.uu_ID = "+str(sp.simplify(uu_ID_PlaneWave - swid.uu_ID))+"\t\t (should be zero)")
    sys.exit(1)
if sp.simplify(vv_ID_PlaneWave - swid.vv_ID) != 0:
    print("TEST FAILED: vv_ID_PlaneWave - swid.vv_ID = "+str(sp.simplify(vv_ID_PlaneWave - swid.vv_ID))+"\t\t (should be zero)")
    sys.exit(1)
print("TESTS PASSED!")

# Step 8: Consistency check between the tutorial notebook above
#         and the SphericalGaussian option from within the
#         ScalarWave/InitialData.py module.
swid.InitialData(Type="SphericalGaussian")
print("Consistency check between ScalarWave tutorial and NRPy+ module: SphericalGaussian Case")
if sp.simplify(uu_ID_SphericalGaussian - swid.uu_ID) != 0:
    print("TEST FAILED: uu_ID_SphericalGaussian - swid.uu_ID = "+str(sp.simplify(uu_ID_SphericalGaussian - swid.uu_ID))+"\t\t (should be zero)")
    sys.exit(1)
if sp.simplify(vv_ID_SphericalGaussian - swid.vv_ID) != 0:
    print("TEST FAILED: vv_ID_SphericalGaussian - swid.vv_ID = "+str(sp.simplify(vv_ID_SphericalGaussian - swid.vv_ID))+"\t\t (should be zero)")
    sys.exit(1)
print("TESTS PASSED!")
```

    Consistency check between ScalarWave tutorial and NRPy+ module: PlaneWave Case
    TESTS PASSED!
    Consistency check between ScalarWave tutorial and NRPy+ module: SphericalGaussian Case
    TESTS PASSED!

# Step 6: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-ScalarWave.pdf](Tutorial-ScalarWave.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-ScalarWave")
```

    Created Tutorial-ScalarWave.tex, and compiled LaTeX file to PDF file
        Tutorial-ScalarWave.pdf


```python
from sympy import *
x = symbols('x')
f = Function('f')
dsolve(Derivative(f(x), x, x) + 9*f(x), f(x))

```

$\displaystyle f{\left(x \right)} = C_{1} \sin{\left(3 x \right)} + C_{2} \cos{\left(3 x \right)}$

```python
th, bta, r0 = symbols('th bta r0')
r1 = Function('r1')
#print(dsolve(Eq(Derivative(r1(th),th),sqrt(1/bta**2-r1(th)**2)),r1(th)))
```

    Eq(r1(th), Piecewise((sin(C1 + th)/sqrt(bta**2), bta**(-2) > 0), (nan, True)))

```python
r2 = Function('r2')
C1 = symbols('C1')
eq1 = sin(C1+th)/bta*r2(th)+cos(C1+th)/bta*Derivative(r2(th),th)
eq2 = (-1)*sin(th+C1)**2*sin(th)**2/bta**5
print(dsolve(Eq(eq1,eq2),r2(th)))
```

```python
C2 = symbols('C2')
#eq = Eq(r2(th), (C2 - r0**2*Integral(sqrt(tan(C1 + th)**2 + 1)*sin(th)**2/cos(C1 + th), th)/bta)/sqrt(tan(C1 + th)**2 + 1))
```

```python
from random import random
import numpy as np
from matplotlib import pyplot
C1 = 0
C2 = random()*2*np.pi
beta = random()*10**(-3)
alpha = random()*beta**3.65
rad = np.arange(0,np.pi,0.01)
r0 = 1/beta*np.sin(rad+C1)+alpha*C2*np.cos(rad+C1)
r0 = r0+alpha/beta**4*(np.sin(3*rad+C1)+4*np.sin(rad+C1)*np.log(abs(np.cos(rad+C1))))
r0 = r0+alpha/beta**4*(np.sin(rad-C1)*(5-4*np.log(abs(np.cos(rad+C1)))))
r0 = r0-4*rad*np.cos(rad+C1)*(1+2*np.cos(2*C1))
pyplot.polar(rad,1/beta*np.sin(rad+C1))
pyplot.polar(rad,r0)
```

```python
a, b, r1, r2, t = symbols('a b r1 r2 t')
series(1/(b+a*(r1+b*r2)**2*sin(t)**2)**2,b)
```

$\displaystyle \frac{1}{a^{2} r_{1}^{4} \sin^{4}{\left(t \right)}} + \frac{b \left(- \frac{4 r_{2}}{r_{1}} - \frac{2}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)}{a^{2} r_{1}^{4} \sin^{4}{\left(t \right)}} + \frac{b^{2} \left(3 \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)^{2} - \frac{2 r_{2}^{2}}{r_{1}^{2}}\right)}{a^{2} r_{1}^{4} \sin^{4}{\left(t \right)}} + \frac{b^{3} \left(- 4 \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)^{3} + \frac{6 r_{2}^{2} \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)}{r_{1}^{2}}\right)}{a^{2} r_{1}^{4} \sin^{4}{\left(t \right)}} + \frac{b^{4} \left(5 \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)^{4} - \frac{12 r_{2}^{2} \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)^{2}}{r_{1}^{2}} + \frac{3 r_{2}^{4}}{r_{1}^{4}}\right)}{a^{2} r_{1}^{4} \sin^{4}{\left(t \right)}} + \frac{b^{5} \left(- 6 \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)^{5} + \frac{20 r_{2}^{2} \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)^{3}}{r_{1}^{2}} - \frac{12 r_{2}^{4} \left(\frac{2 r_{2}}{r_{1}} + \frac{1}{a r_{1}^{2} \sin^{2}{\left(t \right)}}\right)}{r_{1}^{4}}\right)}{a^{2} r_{1}^{4} \sin^{4}{\left(t \right)}} + O\left(b^{6}\right)$

```python
r1 = Function('r1')
print(dsolve(Eq(Derivative(r1(t),t),sqrt(1/a**2/r1(t)**4/sin(t)**4-r1(t)**2)),r1(t)))
```

```python
print(dsolve(Eq(Derivative(r1(t),t),1/a/r1(t)**2/sin(t)**2),r1(t)))
```

    [Eq(r1(t), (C1 - 3*cos(t)/(a*sin(t)))**(1/3)), Eq(r1(t), (-3**(1/3) - 3**(5/6)*I)*(C1 - cos(t)/(a*sin(t)))**(1/3)/2), Eq(r1(t), (-3**(1/3) + 3**(5/6)*I)*(C1 - cos(t)/(a*sin(t)))**(1/3)/2)]

```python
from random import random
import numpy as np
from matplotlib import pyplot
alpha = random()*10**(-6)
rad = np.arange(0,5*np.pi/4,0.01)
r0 = (3/alpha)**(1/3)*(1-1/np.tan(rad))**(1/3)
pyplot.polar(rad,r0)
```

```python
import numpy as np
from matplotlib import pyplot
x = np.arange(0.01,100.0,0.01)
y1 = x*np.log(x)
y2 = x
y3 = x**2
pyplot.plot(x,y1)
pyplot.plot(x,y2)
pyplot.plot(x,y3)
pyplot.show()
```

```python

```


# Design of a Cold Weather Fuel

The venerable alcohol stove has been invaluable camping accessory for generations. They are simple, reliable, and in a pinch, can be made from aluminum soda cans.

Alcohol stoves are typically fueled with denatured alcohol. Denatured alcohol, sometimes called methylated spirits, is a generally a mixture of ethanol and other alcohols and compounds designed to make it unfit for human consumption. An MSDS description of one [manufacturer's product](https://www.korellis.com/wordpress/wp-content/uploads/2016/05/Alcohol-Denatured.pdf) describes a roughly fifity/fifty mixture of ethanol and methanol.

The problem with alcohol stoves is they can be difficult to light in below freezing weather. The purpose of this notebook is to design of an alternative cold weather fuel that could be mixed from other materials commonly available from hardware or home improvement stores.

## Data

The following data was collected for potential fuels commonly available at hardware and home improvement stores. The data consists of price (\$/gal.) and parameters to predict vapor pressure using the Antoine equation,

\begin{align}
\log_{10}P^{vap}_{s}(T) & = A_s - \frac{B_s}{T + C_s}
\end{align}

where the subscript $s$ refers to species, temperature $T$ is in units of degrees Celcius, and pressure $P$ is in units of mmHg. The additional information for molecular weight and specific gravity will be needed to present the final results in volume fraction.

```python
data = {
    'ethanol'          : {'MW':  46.07, 'SG': 0.791, 'A': 8.04494, 'B': 1554.3,  'C': 222.65},
    'methanol'         : {'MW':  32.04, 'SG': 0.791, 'A': 7.89750, 'B': 1474.08, 'C': 229.13},
    'isopropyl alcohol': {'MW':  60.10, 'SG': 0.785, 'A': 8.11778, 'B': 1580.92, 'C': 219.61},
    'acetone'          : {'MW':  58.08, 'SG': 0.787, 'A': 7.02447, 'B': 1161.0,  'C': 224.0},
    'xylene'           : {'MW': 106.16, 'SG': 0.870, 'A': 6.99052, 'B': 1453.43, 'C': 215.31},
    'toluene'          : {'MW':  92.14, 'SG': 0.865, 'A': 6.95464, 'B': 1344.8,  'C': 219.48},
}
```

## Denatured Alcohol

The first step is to determine the vapor pressure of denatured alcohol over a typical range of operating temperatures. For this we assume denatured alcohol is a 40/60 (mole fraction) mixture of ethanol and methanol.

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

def Pvap(T, s):
    return 10**(data[s]['A'] - data[s]['B']/(T + data[s]['C']))

def Pvap_denatured(T):
    return 0.4*Pvap(T, 'ethanol') + 0.6*Pvap(T, 'methanol')

T = np.linspace(0, 40, 200)

plt.plot(T, Pvap_denatured(T))
plt.title('Vapor Pressure of denatured alcohol')
plt.xlabel('temperature / °C')
plt.ylabel('pressure / mmHg')
plt.grid()
print("Vapor Pressure at 0C =", round(Pvap_denatured(0),1), "mmHg")
```

## Cold Weather Product Requirements

We seek a cold weather fuel with increased vapor pressure at 0°C and lower, and also provides safe and normal operation of the alcohol stove at higher operating temperatures.

For this purpose, we seek a mixture of commonly available liquids with a vapor pressure of at least 22 mmHg at the lowest possible temperature, and no greater than the vapor pressure of denatured alcohol at temperatures 30°C and above.

```python
for s in data.keys():
    plt.plot(T, Pvap(T,s))
plt.plot(T, Pvap_denatured(T), 'k', lw=3)
plt.legend(list(data.keys()) + ['denatured alcohol'])
plt.title('Vapor Pressure of selected compounds')
plt.xlabel('temperature / °C')
plt.ylabel('pressure / mmHg')
plt.grid()
```

## Optimization Model

The first optimization model is to create a mixture that maximizes the vapor pressure at -10°C while having a vapor pressure less than or equal to denatured alcohol at 30°C and above.

The decision variables in the optimization model correspond to $x_s$, the mole fraction of each species $s \in S$ from the set of available species $S$. By definition, the mole fractions must satisfy

\begin{align}
x_s & \geq 0 & \forall s\in S \\
\sum_{s\in S} x_s & = 1
\end{align}

The objective is to maximize the vapor pressure at low temperatures, say -10°C, while maintaing a vapor pressure less than or equal to denatured alcohol at 30°C. Using Raoult's law for ideal mixtures,

\begin{align}
\max_{x_s} \sum_{s\in S} x_s P^{vap}_s(-10°C) \\
\end{align}
subject to
\begin{align}
\sum_{s\in S} x_s P^{vap}_s(30°C) & \leq P^{vap}_{denatured\ alcohol}(30°C) \\
\end{align}

This optimization model is implemented in Pyomo in the following cell.

```python
import pyomo.environ as pyomo

m = pyomo.ConcreteModel()

S = data.keys()
m.x = pyomo.Var(S, domain=pyomo.NonNegativeReals)

def Pmix(T):
    return sum(m.x[s]*Pvap(T,s) for s in S)

m.obj = pyomo.Objective(expr = Pmix(-10), sense=pyomo.maximize)

m.cons = pyomo.ConstraintList()

m.cons.add(sum(m.x[s] for s in S)==1)
m.cons.add(Pmix(30) <= Pvap_denatured(30))
m.cons.add(Pmix(40) <= Pvap_denatured(40))

solver = pyomo.SolverFactory('glpk')
solver.solve(m)

print("Vapor Pressure at -10°C =", m.obj(), "mmHg")

T = np.linspace(-10,40,200)
plt.plot(T, Pvap_denatured(T), 'k', lw=3)
plt.plot(T, [Pmix(T)() for T in T], 'r', lw=3)
plt.legend(['denatured alcohol'] + ['cold weather blend'])
plt.title('Vapor Pressure of selected compounds')
plt.xlabel('temperature / °C')
plt.ylabel('pressure / mmHg')
plt.grid()
```

## Display Composition

```python
import pandas as pd

s = data.keys()
results = pd.DataFrame.from_dict(data).T
for s in S:
    results.loc[s,'mole fraction'] = m.x[s]()

MW = sum(m.x[s]()*data[s]['MW'] for s in S)
for s in S:
    results.loc[s,'mass fraction'] = m.x[s]()*data[s]['MW']/MW

vol = sum(m.x[s]()*data[s]['MW']/data[s]['SG'] for s in S)
for s in S:
    results.loc[s,'vol fraction'] = m.x[s]()*data[s]['MW']/data[s]['SG']/vol

results
```

      MW
      SG
      A
      B
      C
      mole fraction
      mass fraction
      vol fraction

      ethanol
      46.07
      0.791
      8.04494
      1554.30
      222.65
      0.000000
      0.0000
      0.000000

      methanol
      32.04
      0.791
      7.89750
      1474.08
      229.13
      0.000000
      0.0000
      0.000000

      isopropyl alcohol
      60.10
      0.785
      8.11778
      1580.92
      219.61
      0.000000
      0.0000
      0.000000

      acetone
      58.08
      0.787
      7.02447
      1161.00
      224.00
      0.428164
      0.2906
      0.311695

      xylene
      106.16
      0.870
      6.99052
      1453.43
      215.31
      0.571836
      0.7094
      0.688305

      toluene
      92.14
      0.865
      6.95464
      1344.80
      219.48
      0.000000
      0.0000
      0.000000

```python

```

```python

```

```python

```


# Lecture 11: Numerical optimization

[Download on GitHub](https://github.com/NumEconCopenhagen/lectures-2022)

[](https://mybinder.org/v2/gh/NumEconCopenhagen/lectures-2022/master?urlpath=lab/tree/11/Numerical_optimization.ipynb)

1. [Introduction](#Introduction)
2. [Example: The rosenbrock function](#Example:-The-rosenbrock-function)
3. [Sombrero function: Local minima and multistart](#Sombrero-function:-Local-minima-and-multistart)
4. [Constraints](#Constraints)
5. [Interpolation](#Interpolation)
6. [Dynamic optimization problems](#Dynamic-optimization-problems)
7. [Summary](#Summary)

You will learn to solve non-convex multi-dimensional optimization problems using numerical optimization with multistart and nesting (**scipy.optimize**). You will learn simple function approximation using linear interpolation (**scipy.interp**).

**Links:**

1. **scipy.optimize:** [overview](https://docs.scipy.org/doc/scipy/reference/optimize.html) + [tutorial](https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html)
2. **scipy.interp:** [overview](https://docs.scipy.org/doc/scipy/reference/interpolate.html) + [tutorial](https://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html)

**Useful note:** [Numerical Optimization in MATLAB](http://www.google.com/url?q=http%3A%2F%2Fweb.econ.ku.dk%2Fmunk-nielsen%2Fnotes%2FnoteOptimization.pdf&sa=D&sntz=1&usg=AFQjCNHX4tHx2_YsNaIt5FB5MBU5cfcS8g) (by Anders Munk-Nielsen)

```python
from types import SimpleNamespace
import numpy as np
import scipy as sp
from scipy import linalg
from scipy import optimize
from scipy import interpolate
import sympy as sm

%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
from matplotlib import cm
from mpl_toolkits.mplot3d import Axes3D
```

# 1. Introduction

All **optimization problems** are characterized by:

1. Control vector (choices), $\boldsymbol{x} \in \mathbb{R}^k$
2. Objective function (payoff) to minimize, $f:\mathbb{R}^k \rightarrow \mathbb{R}$ (differentiable or not)
3. Constraints, i.e. $\boldsymbol{x}  \in C \subseteq \mathbb{R}^k$ (linear or non-linear interdependence)

Note that $f$ might also take other inputs (parameters or a dataset), but these are fixed, and therefore not variables we optimize over.

**Maximization** is just **minimization** of $-f$.

All **optimizers** (minimizers) follow the structure:

1. Make initial guess
2. Evaluate the function (and perhaps gradients)
3. Check for convergence
4. Update guess and return to step 2

**Convergence:** "Small" change in function value since last iteration (or "zero" gradient).

**Characteristics** of optimizers:

1. Use gradients or not.
2. Allow for specifying bounds.
3. Allow for specifying general constraints.

**Gradients** provide useful information, but can be costly to compute (using analytical formula or numerically).

**Penalty terms** can (sometimes) instead be used to enforce bounds and constraints.

**Optimizers** you should know:

1. **Nelder-Mead:**
 * **Pro:** Robust (to e.g. noise in objective function) and does not require derivatives.
 * **Con:** Slow convergence. No bounds or constraints.
2. **Newton-CG:**
 * **Pro:** Require few iterations. Very precise with analytical hessian for smooth functions.
 * **Con:** Costly computation of hessian. No bounds or constraints.
3. **BFGS:** (like newton, but with smart computation of hessian)
  * **Pro:** Require few function evaluations.
  * **Con:** No bounds or constraints.
4. **L-BFGS-B:** Like BFGS, but allows for bounds.
5. **SLSQP:**
  * **Pro:** Bounds and constraints in multiple dimensions.
  * **Con:** Not as efficient as BFGS.

## 1.1 Gradient based optimizers

Let us look at the idea behind gradient based optimizers.

**One dimensional intuition:** Consider the second-order Taylor approximation around $x_n$:

$$
f_T(x) = f_T(x_n + \Delta x) \approx f(x_n)+ f^{\prime}(x_n) \Delta x + \frac{1}{2} f^{\prime\prime}(x_n) (\Delta x)^2
$$

Find the minimum wrt. to $\Delta x$ by solving the FOC:

$$
0 = \frac{d}{d\Delta x} f_T(x) = f^{\prime}(x_n) + f^{\prime\prime}(x_n) \Delta x \Leftrightarrow \Delta x = -\frac{f^{\prime}(x_n)}{f^{\prime\prime}(x_n)}
$$

**Algorithm:** `minimize_newton()`

1. Choose tolerance $\epsilon>0$, guess on $\boldsymbol{x}_0$, compute $f(\boldsymbol{x}_0)$, and set $n=1$.
2. Compute $\nabla f(\boldsymbol{x}_{n-1})$ (gradient/jacobian) and $\boldsymbol{H}f(\boldsymbol{x}_{n-1})$ (hessian).
3. Compute new guess

  $$
  \boldsymbol{x}_{n} = \boldsymbol{x}_{n-1} - [\boldsymbol{H}f(\boldsymbol{x}_{n-1})]^{-1} \nabla f(\boldsymbol{x}_{n-1})
  $$

3. If $|f(\boldsymbol{x}_n)-f(\boldsymbol{x}_{n-1})| < \epsilon$ then stop.
5. Set $n = n + 1$ and return to step 2.

```python
def minimize_newton(f,x0,jac,hess,max_iter=500,tol=1e-8):
    """ minimize function with Newtons' algorithm

    Args:

        f (callable): function
        x0 (np.ndarray): initial values
        jac (callable): jacobian
        hess (callable): hessian
        max_iter (int): maximum number of iterations
        tol (float): tolerance

    Returns:

        x (np.ndarray): minimum
        n (int): number of iterations used

    """

    # step 1: initialize
    x = x0
    fx = f(x0)
    n = 1

    # step 2-5: iteration
    while n < max_iter:

        x_prev = x
        fx_prev = fx

        # step 2: evaluate gradient and hessian
        jacx = jac(x_prev)
        hessx = hess(x_prev)

        # step 3: update x
        inv_hessx = linalg.inv(hessx)
        x = x_prev - inv_hessx@jacx

        # step 4: check convergence
        fx = f(x)
        if abs(fx-fx_prev) < tol:
            break

        # step 5: increment n
        n += 1

    return x,n
```

**Algorithm:** `minimize_gradient_descent()`

1. Choose tolerance $\epsilon>0$, potential step sizes, $ \boldsymbol{\alpha} = [\alpha_0,\alpha_1,\dots,\alpha_\#]$, guess on $\boldsymbol{x}_0$, compute $f(\boldsymbol{x}_0)$, and set $n=1$.
2. Compute $\nabla f(\boldsymbol{x}_{n-1})$.
3. Find good step size:

  $$
  \alpha^{\ast} = \arg \min_{\alpha \in \boldsymbol{\alpha}}  f(\boldsymbol{x}_{n-1} - \alpha \nabla f(\boldsymbol{x}_{n-1}))
  $$

4. Compute new guess:

  $$
  \boldsymbol{x}_{n} = \boldsymbol{x}_{n-1} - \alpha^{\ast} \nabla f(\boldsymbol{x}_{n-1})
  $$

5. If $|f(\boldsymbol{x}_n)-f(\boldsymbol{x}_{n-1})| < \epsilon$ then stop.
6. Set $n = n + 1$ and return to step 2.

```python
def minimize_gradient_descent(f,x0,jac,alphas=[0.01,0.05,0.1,0.25,0.5,1],max_iter=500,tol=1e-8):
    """ minimize function with gradient descent

    Args:

        f (callable): function
        x0 (np.ndarray): initial values
        jac (callable): jacobian
        alpha (list): potential step sizes
        max_iter (int): maximum number of iterations
        tol (float): tolerance

    Returns:

        x (np.ndarray): minimum
        n (int): number of iterations used

    """

    # step 1: initialize
    x = x0
    fx = f(x0)
    n = 1

    # step 2-6: iteration
    while n < max_iter:

        x_prev = x
        fx_prev = fx

        # step 2: evaluate gradient
        jacx = jac(x)

        # step 3: find good step size (line search)
        fx_ast = np.inf
        alpha_ast = np.nan
        for alpha in alphas:
            x = x_prev - alpha*jacx
            fx = f(x)
            if fx < fx_ast:
                fx_ast = fx
                alpha_ast = alpha

        # step 4: update guess
        x = x_prev - alpha_ast*jacx

        # step 5: check convergence
        fx = f(x)
        if abs(fx-fx_prev) < tol:
            break

        # d. update i
        n += 1

    return x,n
```

**Many generalizations:**

1. Use both Hessian and line search
2. Stop line search when improvement found
3. Limit attention to a "trust-region"

etc. etc. etc. etc.

# 2. Example: The rosenbrock function

Consider the **rosenbrock function**:

$$
f(\boldsymbol{x}) = f(x_1,x_2) =0.5(1-x_{1})^{2}+(x_{2}-x_{1}^{2})^{2}
$$

with **jacobian** (gradient)

$$
\nabla f(\boldsymbol{x})=\begin{bmatrix}\frac{\partial f}{\partial x_{1}}\\
\frac{\partial f}{\partial x_{2}}
\end{bmatrix}=\begin{bmatrix}-(1-x_{1})-4x_{1}(x_{2}-x_{1}^{2})\\
2(x_{2}-x_{1}^{2})
\end{bmatrix}
$$

and **hessian**:

$$
\boldsymbol{H}f(\boldsymbol{x})=\begin{bmatrix}\frac{\partial f}{\partial x_{1}x_{1}} & \frac{\partial f}{\partial x_{1}x_{2}}\\
\frac{\partial f}{\partial x_{1}x_{2}} & \frac{\partial f}{\partial x_{2}x_{2}}
\end{bmatrix}=\begin{bmatrix}1-4x_{2}+12x_{1}^{2} & -4x_{1}\\
-4x_{1} & 2
\end{bmatrix}
$$

**Note:** Minimum is at $(1,1)$ where $f(1,1)=0$.

**Check jacobian and hessian:**

```python
x1 = sm.symbols('x_1')
x2 = sm.symbols('x_2')
f = 0.5*(1.0-x1)**2 + (x2-x1**2)**2
```

```python
Df = sm.Matrix([sm.diff(f,i) for i in [x1,x2]])
Df
```

```python
Hf = sm.Matrix([[sm.diff(f,i,j) for j in [x1,x2]] for i in [x1,x2]])
Hf
```

**Implementation:**

```python
def _rosen(x1,x2):
    return 0.5*(1.0-x1)**2+(x2-x1**2)**2
def rosen(x):
    return _rosen(x[0],x[1])
def rosen_jac(x):
    return np.array([-(1.0-x[0])-4*x[0]*(x[1]-x[0]**2),2*(x[1]-x[0]**2)])
def rosen_hess(x):
    return np.array([[1-4*x[1]+12*x[0]**2,-4*x[0]],[-4*x[0],2]])
```

**3D Plot:**

```python
# a. grids
x1_vec = np.linspace(-2,2,500)
x2_vec = np.linspace(-2,2,500)
x1_grid,x2_grid = np.meshgrid(x1_vec,x2_vec,indexing='ij')
rosen_grid = _rosen(x1_grid,x2_grid)

# b. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.plot_surface(x1_grid,x2_grid,rosen_grid,cmap=cm.jet)

# c. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$u$')

# d. invert xaxis
ax.invert_xaxis()

# e. add colorbar
fig.colorbar(cs);
```

**Contour plot:**

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
levels = [1e-6,5*1e-6,1e-5,5*1e-5,1e-4,5*1e-4,1e-3,5*1e-3,1e-2,5*1e-2,1,2,4,6,8,12,16,20]
cs = ax.contour(x1_grid,x2_grid,rosen_grid,levels=levels,cmap=cm.jet)
fig.colorbar(cs);
```

**Newton:**

```python
x0 = np.array([5,4])
x,n = minimize_newton(rosen,x0,rosen_jac,rosen_hess)
print(n,x,rosen(x))
```

**Gradient descent:**

```python
x0 = np.array([5,4])
x,n = minimize_gradient_descent(rosen,x0,rosen_jac,alphas=[0.01,0.05,0.1,0.25,0.5,1])
print(n,x,rosen(x))
```

**Questions:** Any ideas for getting the gradient descent optimizer to converge faster?

## 2.1 Scipy minimizers

**Preperation I:** Function for collecting infomation while running optimizing:

```python
# complicated -> not necessary to understand it
def collect(x):

    # globals used to keep track across iterations
    global evals # set evals = 0 before calling optimizer
    global x0
    global x1s
    global x2s
    global fs

    # a. initialize list
    if evals == 0:
        x1s = [x0[0]]
        x2s = [x0[1]]
        fs = [rosen(x0)]

    # b. append trial values
    x1s.append(x[0])
    x2s.append(x[1])
    fs.append(rosen(x))

    # c. increment number of evaluations
    evals += 1

```

**Preperation II:** Function plotting the collected information:

```python
# complicated -> not necessary to understand it
def contour():

    global evals
    global x1s
    global x2s
    global fs

    # a. contour plot
    fig = plt.figure(figsize=(10,4))
    ax = fig.add_subplot(1,2,1)
    levels = [1e-6,5*1e-6,1e-5,5*1e-5,1e-4,5*1e-4,1e-3,5*1e-3,1e-2,5*1e-2,1,2,4,6,8,12,16,20]
    cs = ax.contour(x1_grid,x2_grid,rosen_grid,levels=levels,cmap=cm.jet)
    fig.colorbar(cs)
    ax.plot(x1s,x2s,'-o',ms=4,color='black')
    ax.set_xlabel('$x_1$')
    ax.set_ylabel('$x_2$')

    # b. function value
    ax = fig.add_subplot(1,2,2)
    ax.plot(np.arange(evals+1),fs,'-o',ms=4,color='black')
    ax.set_xlabel('iteration')
    ax.set_ylabel('function value')

```

**Nelder-Mead**

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,
                           method='Nelder-Mead',
                           callback=collect, # call collect() before each iteration
                           options={'disp':True}) # display the results
contour()
```

> **Note:** Does not require a gradient. Slow convergence close to target.
>
> **Iterations:** How many steps the algorithm has taken.
>
> **Function evaluations:** Will be higher than iterations. Used to compute next step.

We can also **print the information on results:**

```python
print(result)
```

We can also acess specific information of the result object:

```python
result.nit
```

**Newton** (with analytical hessian)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,hess=rosen_hess,
                           method='Newton-CG',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Smoother and faster.

**Newton** (with numerical hessian computed by scipy)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='Newton-CG',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Same as above, but gradient evaluations instead of hessian evaluations.

**BFGS** (with analytical gradient)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='BFGS',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Non-smooth, but fast. Very low number of function evaluations.

**BFGS** (with numerical gradient computed by scipy)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0, # no jac= specified
                           method='BFGS',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Same as above, but more function evaluations.

**L-BFGS-B** (with analytical gradient)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='L-BFGS-B',
                           bounds=((-3,3),(-3,3)),
                           callback=collect,
                           options={'disp':True})
contour()
```

**SLSQP**

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='SLSQP',
                           bounds=((-2,2),(-2,2)),
                           callback=collect,
                           options={'disp':True})
contour()
```

## 2.2 Controling the optimizers

> **Note:** See the settings for each optimizer in the [documention](https://docs.scipy.org/doc/scipy/reference/optimize.html).

We can lower the **tolerance**:

```python
evals = 0
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,
                           method='BFGS',
                           callback=collect,
                           options={'disp':True,'gtol':1e-8}) # note this
contour()
```

We can change the **maximum number of iterations**:

```python
evals = 0
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,
                           method='BFGS',
                           callback=collect,
                           options={'disp':True,'maxiter':5}) # note this and warning
contour()
```

**Question:** Can we make the program stop if the maximum number of iterations is too low?

# 3. Sombrero function: Local minima and multistart

Consider the **sombrero** function

$$
f(x_1,x_2) = g\Big(\sqrt{x_1^2 + x_2^2}\Big)
$$

where

$$
g(r) = -\frac{\sin(r)}{r+10^{-4}} + 10^{-4}r^2
$$

The **global minimum** of this function is (0,0). But the function also have (infinitely many) **local minima**. How to avoid these?

```python
def _sombrero(x1,x2):
    r = np.sqrt(x1**2 + x2**2)
    return -np.sin(r)/(r+1e-4) + 1e-4*r**2

sombrero = lambda x: _sombrero(x[0],x[1])
```

## 3.1 3D plot

```python
# a. grids
x1_vec = np.linspace(-15,15,500)
x2_vec = np.linspace(-15,15,500)
x1_grid_sombrero,x2_grid_sombrero = np.meshgrid(x1_vec,x2_vec,indexing='ij')
sombrero_grid = _sombrero(x1_grid_sombrero,x2_grid_sombrero)

# b. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.plot_surface(x1_grid_sombrero,x2_grid_sombrero,sombrero_grid,cmap=cm.jet)

# c. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$u$')

# d. invert xaxis
ax.invert_xaxis()

# e. colorbar
fig.colorbar(cs);
```

## 3.2 Multi-start - BFGS

**Multi-start:** Draw many random starting values:

```python
np.random.seed(1986)
x0s = -15 + 30*np.random.uniform(size=(5000,2)) # in [-15,15]
xs = np.empty((5000,2))
fs = np.empty(5000)
```

Try to solve with **BFGS** starting from each of these:

```python
fopt = np.inf
xopt = np.nan
for i,x0 in enumerate(x0s):

    # a. optimize
    result = optimize.minimize(sombrero,x0,method='BFGS')
    xs[i,:] = result.x
    f = result.fun

    # b. print first 10 or if better than seen yet
    if i < 10 or f < fopt: # plot 10 first or if improving
        if f < fopt:
            fopt = f
            xopt = xs[i,:]

        print(f'{i:4d}: x0 = ({x0[0]:6.2f},{x0[1]:6.2f})',end='')
        print(f' -> converged at ({xs[i][0]:6.2f},{xs[i][1]:6.2f}) with f = {f:.14f}')

# best solution
print(f'\nbest solution:\n x = ({xopt[0]:6.2f},{xopt[1]:6.2f}) -> f = {fopt:.14f}')
```

The solver, wrongly, **converges to many of the local minima**:

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(xs[:,0],xs[:,1]);
```

## 3.3 Multi-start - Nelder-Mead

Try to solve with **Nelder-Mead** starting from each of these:

```python
fopt = np.inf
xopt = np.nan
for i,x0 in enumerate(x0s):

    # a. optimize
    result = optimize.minimize(sombrero,x0,method='Nelder-Mead')
    xs[i,:] = result.x
    f = result.fun

    # b. print first 10 or if better than seen yet
    if i < 10 or f < fopt: # plot 10 first or if improving
        if f < fopt:
            fopt = f
            xopt = xs[i,:]
        print(f'{i:4d}: x0 = ({x0[0]:6.2f},{x0[1]:6.2f})',end='')
        print(f' -> converged at ({xs[i][0]:6.2f},{xs[i][1]:6.2f}) with f = {f:.12f}')

# best solution
print(f'\nbest solution:\n x = ({xopt[0]:6.2f},{xopt[1]:6.2f}) -> f = {fopt:.12f}')
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(xs[:,0],xs[:,1]);
```

## 3.4 Is there a better solution than multi-start?

**In short:** No.

**Potential improvement:** Use information from previous run to determine, where to look next. Fundamental trade-off between:

1. **Exploitation.** Focus on areas where previous evaluations returned low function values.
2. **Exploration.** Focus on completely new areas.

**Heuristic:** If the same optimum is obtained for many starting values, this is a good sign for it being the global optimum.

**Further discussion**: [Benchmarking Global Optimizers](https://fguvenendotcom.files.wordpress.com/2019/09/agk2019-september-nber-submit.pdf) ([code](https://github.com/serdarozkan/TikTak#tiktak))

# 4. Constraints

## 4.1 In general

Consider the **constrained problem**:

$$
\min_{x_1,x_2,x_3,x_4} x_1x_4(x_1+x_2+x_3) + x_3
$$

subject to

$$
\begin{aligned}
x_1x_2x_3x_4 &\geq 25 \\
x_1^2+x_2^2+x_3^2+x_4^2 &= 40 \\
1 \leq x_1,x_2,x_3,x_4 &\leq 5
\end{aligned}
$$

Define **objective** and **constraints**:

```python
def _objective(x1,x2,x3,x4):
    return x1*x4*(x1+x2+x3)+x3

def objective(x):
    return _objective(x[0],x[1],x[2],x[3])

def ineq_constraint(x):
    return x[0]*x[1]*x[2]*x[3]-25.0 # violated if negative

def eq_constraint(x):
    sum_eq = 40.0
    for i in range(4):
        sum_eq = sum_eq - x[i]**2
    return sum_eq # must equal zero
```

```python
# a. setup
bound = (1.0,5.0)
bounds = (bound, bound, bound, bound)
ineq_con = {'type': 'ineq', 'fun': ineq_constraint}
eq_con = {'type': 'eq', 'fun': eq_constraint}

# b. call optimizer
x0 = (40**(1/8),40**(1/8),40**(1/8),40**(1/8)) # fit the equality constraint
result = optimize.minimize(objective,x0,
                             method='SLSQP',
                             bounds=bounds,
                             constraints=[ineq_con,eq_con],
                             options={'disp':True})

print('\nx = ',result.x)
```

**Alternative:** Extend the **objective function with a penalty term**, where guesses outside the allowed bounds and constraints are projected into the allowed region, but a (large) penalty term is added to discourage this. Solve this problem with an unconstrained solver.

## 4.2 Economic application

Consider the following **consumption-saving problem**:

$$
\begin{aligned}
V(a_0) &= \max_{c_1,c_2,c_3} \frac{c_{1}^{1-\rho}}{1-\rho} + \beta \frac{c_{2}^{1-\rho}}{1-\rho} + \beta^2\frac{c_{3}^{1-\rho}}{1-\rho} + \beta^2\nu\frac{(a_{3}+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
&\text{s.t.}&\\
m_1 &= (1+r)a_0 + y_1\\
a_1 &= m_1-c_1\\
m_2 &= (1+r)a_1 + y_2\\
a_2 &= m_2-c_2\\
m_3 &= (1+r)a_2 + y_3\\
a_3 &= m_3-c_3\\
c_1,c_2,c_3 &\geq 0\\
a_1,a_2,a_3 &\geq 0\\
\end{aligned}
$$

where

* $m_t$ is cash-on-hand in period $t\in\{1,2,\dots,T\}$
* $c_t$ is consumption $t$
* $a_t$ is end-of-period assets and income in period $t$
* ${y_t}$ is income in period $t$
* $\beta > 0$ is the discount factor
* $r > -1$ is the interest rate
* $\rho > 1$ is the CRRA coefficient
* $\nu > 0 $ is the strength of the bequest motive
* $\kappa > 0$ is the degree of luxuriousness in the bequest motive
* $a_t\geq0$ is a no-borrowing constraint.

**Guide to solve such problem:**

1. Setup parameters
2. Formulate objective function
3. Determine how to handle constraints
4. Call optimizer

**Parameters:**

```python
par = SimpleNamespace()
par.a0 = 0.5
par.beta = 0.94
par.r = 0.04
par.rho = 8
par.kappa = 0.5
par.nu = 0.1
par.y = [1,2,4]
par.T = 3
```

**Objetive function:**

```python
def obj(c,par,full_return=False):

    # objective function with penalty term

    # a. allocate
    a = np.zeros(par.T) # end-of-period assets
    m = np.zeros(par.T) # cash-on-hand
    cb = np.zeros(par.T) # bounded

    # b. bound consumption and penalty
    penalty = 0.0
    for t in range(par.T):

        # i. lagged assets
        a_lag = a[t-1] if t > 0 else par.a0

        # ii. cash-on-hand
        m[t] = (1+par.r)*a_lag + par.y[t]

        # ii. bounded consumption
        if c[t] < 0:
            penalty += 10_000*np.abs(c[t]-0.0)
            cb[t] = 0
        elif c[t] > m[t]:
            penalty += 10_000*np.abs(c[t]-m[t])
            cb[t] = m[t]
        else:
            cb[t] = c[t]

        # d. end-of-period assets
        a[t] = m[t] - cb[t]

    # c. utility
    utility = 0.0

    # i. consumption
    for t in range(par.T):
        utility += par.beta**t*(cb[t]**(1-par.rho))/(1-par.rho)

    # ii. bequest
    utility += par.beta**(par.T-1)*par.nu*(a[-1]+par.kappa)**(1-par.rho)/(1-par.rho)

    # d. return negative utility + penalty
    if full_return:
        return utility,m,a
    else:
        return -utility + penalty
```

**Solve:**

```python
def solve(par):

    # a. initial geuss
    x0 = [par.a0/par.T,par.a0/par.T,par.a0/par.T]

    # b. solve
    results = optimize.minimize(obj,x0,args=(par,),method='nelder-mead')
    assert results.success
    print(f'solved in {results.nit} iteratoons [{results.nfev} function evaluations]')

    # c. details
    c = results.x
    utility,m,a = obj(c,par,full_return=True)
    print(f't = 0: a = {par.a0:.4f}')
    for t in range(par.T):
        print(f't = {t+1}: y = {par.y[t]:.4f}, m = {m[t]:.4f}, c = {c[t]:.4f}, a = {a[t]:.4f}')
    print(f'utility = {utility:.8f}')
```

```python
solve(par)
```

**What happens if the income path is reversed?**

```python
par.y = list(reversed(par.y))
solve(par)
```

**Question I:** Could we easily allowing for borrowing, i.e.

$$
\begin{aligned}
V(a_0) &= \max_{c_1,c_2,c_3} \frac{c_{1}^{1-\rho}}{1-\rho} + \beta \frac{c_{2}^{1-\rho}}{1-\rho} + \beta^2\frac{c_{3}^{1-\rho}}{1-\rho} + \beta^2\nu\frac{(a_{3}+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
&\text{s.t.}&\\
m_1 &= (1+r)a_0 + y_1\\
a_1 &= m_1-c_1\\
m_2 &= (1+r)a_1 + y_2\\
a_2 &= m_2-c_2\\
m_3 &= (1+r)a_2 + y_3\\
a_3 &= m_3-c_3\\
c_1,c_2,c_3 &\geq 0\\
a_1,a_2 &\geq -\lambda\\
a_3 &\geq 0
\end{aligned}
$$

**Question II:** Could we easily extend the problem to more periods?

$$
\begin{aligned}
V(a_0) &= \max_{c_1,c_2,\dots c_T} \sum_{t=1}^T \beta^{t-1} \frac{c_{t}^{1-\rho}}{1-\rho} + \beta^{T+1}\nu\frac{(a_{T}+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
&\text{s.t.}&\\
m_t &= (1+r)a_{t-1} + y_t\\
c_t &\geq 0\\
a_t &\geq 0
\end{aligned}
$$

**Follow-up question:** What is the problem for $T \rightarrow \infty$?

# 5. Interpolation

**Intermezzo:** To consider dynamic optimization problems, we need to think about interpolation.

**Inputs:**

1. Sorted vector of known points (grid vector), $G = \{G_i\}_{i=0}^{n-1}$
2. Vector of known values (at these points), $F = \{F_i = f(G_i)\}_{i=0}^{n-1}$
3. A new point, `x`

**Algorithm:** `linear_interpolate()`
1. Determine `i`  such that

$$
G_i \leq x < G_{i+1}
$$

2. Compute interpolated value by

$$
y =  F_{i} + \frac{F_{i+1}-F_{i}}{G_{i+1}-G_{i}}(x-G_{i})
$$

**Extrapolation:**

1. Below where $x < G_1$:

$$
y =  F_{0} + \frac{F_{1}-F_{0}}{G_{1}-G_{0}}(x-G_{0})
$$

2. Above where $x > G_{n-2}$:

$$
y =  F_{n-2} + \frac{F_{n-1}-F_{n-2}}{G_{n-1}-G_{n-2}}(x-G_{n-2})
$$

```python
def linear_interpolate(G,F,x):
    """ linear interpolation (and extrapolation)

    Args:

        G (np.ndarray): known points
        F (np.ndarray): known values
        x (float): point to be interpolated

    Returns:

        y (float): intepolated value

    """

    assert len(G) == len(F)
    n = len(G)

    # a. find index in known points
    if x < G[1]: # exprapolation below
        i = 0
    elif x > G[-2]: # extrapolation above
        i = n-2
    else: # true interpolation

        # search
        i = 0
        while x >= G[i+1] and i < n-1:
            i += 1

        assert x >= G[i]
        assert x < G[i+1]

    # b. interpolate
    diff_G = G[i+1]-G[i]
    diff_F = F[i+1]-F[i]
    slope = diff_F/diff_G
    y = F[i] + slope*(x-G[i])

    return y
```

## 5.1 Example

Consider the following function and known points:

```python
f = lambda x: (x-3)**3 - 3*x**2 + 5*x

G = np.linspace(-5,10,6)
F = f(G)
```

**Simple test:**

```python
for x in [-2.3,4.1,7.5,9.1]:
    true = f(x)
    y = linear_interpolate(G,F,x)
    print(f'x = {x:4.1f} -> true = {true:6.1f}, interpolated = {y:6.1f}')
```

**Scipy.interpolate:** Use the *RegularGridInterpolator*

```python
# a. construct interpolation function
interp_func = interpolate.RegularGridInterpolator([G],F,
                                                  bounds_error=False,
                                                  fill_value=None)

# bounds_error=False and fill_value=None allow for extrapolation

# b. interpolate
grid = np.linspace(-7,12,500)
interp_values = interp_func(grid)

# c. evaluate true values
true_values = f(grid)

# d. plot true and interpolated values
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(G,F,'o',label='known points')
ax.plot(grid,true_values,'-',lw=1,label='true function')
ax.plot(grid,interp_values,'-',lw=1,label='interpolated values')
ax.legend(loc='lower right',facecolor='white',frameon=True);
```

**Note:**

1. Linear interpolation works best when the function does not curve too much.
2. Extrapolation is much worse than interpolation.

**Multiple dimensions:** Same principle, ``interpolate.RegularGridInterpolator([G1,G2,G3],F)``.

# 6. Dynamic optimization problems

The following subject is hard. But also extremely useful. *If you master this, you can solve (almost) all economic models you meet on your way in life*.

## 6.1 Problem formulation

Consider a **household** living in two periods.

In the **second period** it gets utility from **consuming** and **leaving a bequest** (warm glow),

$$
\begin{aligned}
v_{2}(m_{2})&= \max_{c_{2}}\frac{c_{2}^{1-\rho}}{1-\rho}+\nu\frac{(a_2+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
a_2 &= m_2-c_2 \\
a_2 &\geq 0
\end{aligned}
$$

where

* $m_2$ is cash-on-hand
* $c_2$ is consumption
* $a_2$ is end-of-period assets
* $\rho > 1$ is the risk aversion coefficient
* $\nu > 0 $ is the strength of the bequest motive
* $\kappa > 0$ is the degree of luxuriousness in the bequest motive
* $a_2\geq0$ ensures the household *cannot* die in debt

The **value function** $v(m_2)$ measures the household's value of having $m_2$ at the beginning of period 2.

```python
def utility(c,par):
    return c**(1-par.rho)/(1-par.rho)

def bequest(m,c,par):
    return par.nu*(m-c+par.kappa)**(1-par.rho)/(1-par.rho)

def v2(c2,m2,par):
    return utility(c2,par) + bequest(m2,c2,par)
```

In the **first period**, the household gets utility from consuming and takes into account that it will also live in the next-period, where it receives a stochastic income,

$$
\begin{aligned}
v_1(m_1)&=\max_{c_1}\frac{c_{1}^{1-\rho}}{1-\rho}+\beta\mathbb{E}_{1}\left[v_2(m_2)\right]\\&\text{s.t.}&\\
a_1&=m_1-c_1\\
m_2&= (1+r)(m_1-c_1)+y_2 \\
y_{2}&= \begin{cases}
1-\Delta & \text{with prob. }0.5\\
1+\Delta & \text{with prob. }0.5
\end{cases}\\
a_1&\geq0
\end{aligned}
$$

where

* $m_1$ is cash-on-hand in period 1
* $c_1$ is consumption in period 1
* $a_1$ is end-of-period assets in period 1
* $\beta > 0$ is the discount factor
* $\mathbb{E}_1$ is the expectation operator conditional on information in period 1
* $y_2$ is income in period 2
* $\Delta \in (0,1)$ is the level of income risk (mean-preserving)
* $r$ is the interest rate
* $a_1\geq0$ ensures the household *cannot* borrow

```python
def v1(c1,m1,par,v2_interp):

    # a. v2 value, if low income
    m2_low = (1+par.r)*(m1-c1) + 1-par.Delta
    v2_low = v2_interp([m2_low])[0]

    # b. v2 value, if high income
    m2_high = (1+par.r)*(m1-c1) + 1+par.Delta
    v2_high = v2_interp([m2_high])[0]

    # c. expected v2 value
    prob_low = 0.5
    prob_high = 0.05
    expected_v2 = prob_low*v2_low + prob_high*v2_high

    # d. total value
    return utility(c1,par) + par.beta*expected_v2
```

## 6.2 Solve household problem

Choose **parameters**:

```python
par = SimpleNamespace()
par.rho = 8
par.kappa = 0.5
par.nu = 0.1
par.r = 0.04
par.beta = 0.94
par.Delta = 0.5
```

**Solve second period:**

```python
def solve_period_2(par):

    # a. grids
    m2_vec = np.linspace(1e-4,5,500)
    v2_vec = np.empty(500)
    c2_vec = np.empty(500)

    # b. solve for each m2 in grid
    for i,m2 in enumerate(m2_vec):

        # i. objective
        obj = lambda x: -v2(x[0],m2,par)

        # ii. initial value (consume half)
        x0 = m2/2

        # iii. optimizer
        result = optimize.minimize(obj,[x0],method='L-BFGS-B',bounds=((1e-8,m2),))

        # iv. save
        v2_vec[i] = -result.fun
        c2_vec[i] = result.x

    return m2_vec,v2_vec,c2_vec

# solve
m2_vec,v2_vec,c2_vec = solve_period_2(par)

# illustration
fig = plt.figure(figsize=(10,4))
ax = fig.add_subplot(1,2,1)
ax.plot(m2_vec,c2_vec)
ax.set_xlabel('$m_2$')
ax.set_ylabel('$c_2$')
ax.set_title('consumption function in period 2')

ax = fig.add_subplot(1,2,2)
ax.plot(m2_vec,v2_vec)
ax.set_xlabel('$m_2$')
ax.set_ylabel('$v_2$')
ax.set_title('value function in period 2')
ax.set_ylim([-40,1]);
```

**Note:** We now solve for the consumption function, rather than a specific optimum.

**Question:** Why is there a kink in the consumption function?

**Construct interpolator:**

```python
v2_interp = interpolate.RegularGridInterpolator([m2_vec], v2_vec,
                                                bounds_error=False,fill_value=None)
```

**Solve first period:**

```python
def solve_period_1(par,v2_interp):

    # a. grids
    m1_vec = np.linspace(1e-8,4,100)
    v1_vec = np.empty(100)
    c1_vec = np.empty(100)

    # b. solve for each m1 in grid
    for i,m1 in enumerate(m1_vec):

        # i. objective
        obj = lambda x: -v1(x[0],m1,par,v2_interp)

        # ii. initial guess (consume half)
        x0 = m1/2

        # iii. optimize
        result = optimize.minimize(obj,[x0],method='L-BFGS-B',bounds=((1e-12,m1),))

        # iv. save
        v1_vec[i] = -result.fun
        c1_vec[i] = result.x[0]

    return m1_vec,v1_vec,c1_vec

# solve
m1_vec,v1_vec,c1_vec = solve_period_1(par,v2_interp)

# illustrate
fig = plt.figure(figsize=(10,4))
ax = fig.add_subplot(1,2,1)
ax.plot(m1_vec,c1_vec)
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('consumption function in period 1')

ax = fig.add_subplot(1,2,2)
ax.plot(m1_vec,v1_vec)
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('value function in period 1')
ax.set_ylim([-40,1]);
```

**Summary:** We can summarize what we have done in a single function doing:

1. Solve period 2 (i.e. find $v_2(m_2)$ og $c_2(m_2)$)
2. Construct interpolator of $v_2(m_2)$
3. Solve period 1 (i.e. find $v_1(m_1)$ og $c_1(m_1)$)

```python
def solve(par):

    # a. solve period 2
    m2_vec,v2_vec,c2_vec = solve_period_2(par)

    # b. construct interpolator
    v2_interp = interpolate.RegularGridInterpolator([m2_vec], v2_vec,
        bounds_error=False,fill_value=None)

    # b. solve period 1
    m1_vec,v1_vec,c1_vec = solve_period_1(par,v2_interp)

    return m1_vec,c1_vec,m2_vec,c2_vec
```

**Plot consumption function for various level of income risk**, i.e varios $\Delta$

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

_Delta = par.Delta
for Delta in [0.05,0.15,0.25]:
    par.Delta = Delta
    m1_vec,c1_vec,m2_vec,c2_vec = solve(par)
    ax.plot(m1_vec,c1_vec,label=f'$\Delta = {Delta}$')

# reset
par.Delta = _Delta
ax.legend(loc='lower right',facecolor='white',frameon=True)
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('value function in period 1')
ax.set_xlim([0,2])
ax.set_ylim([0,1.5]);
```

**Main takeaway:** The household lower its consumption when risk increases (such as in a recession). This is called **precautionary saving**.

## 6.3 Simulation

**Step 1:** Solve and construct interpolators:

```python
Delta = 0.5
m1_vec,c1_vec,m2_vec,c2_vec = solve(par)

c1_interp = interpolate.RegularGridInterpolator([m1_vec], c1_vec,
                                                bounds_error=False,fill_value=None)

c2_interp = interpolate.RegularGridInterpolator([m2_vec], c2_vec,
                                                bounds_error=False,fill_value=None)
```

**Step 2:** Draw initail distribution of $m_1$ and simulate forward

```python
# a. draw initial m1
simN = 10000
sim_m1 = np.fmax(np.random.normal(1,0.1,size=simN),0) # "randomly" chosen distribution

# b. period 1
sim_c1 = c1_interp(sim_m1)
sim_a1 = sim_m1-sim_c1

# c. transition to period 2 with random draw
sim_m2 = (1+par.r)*sim_a1+np.random.choice([0.5,1.5],p=[0.5,0.5])

# d. period 2
sim_c2 = c2_interp(sim_m2)
```

**Step 3:** Plot distributions

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.hist(sim_c1,bins=100,label='period 1')
ax.hist(sim_c2,bins=100,label='period 2')

ax.legend(loc='lower right',facecolor='white',frameon=True)
ax.set_xlabel('$c_t$')
ax.set_ylabel('freq.')
ax.set_title('consumption');
```

**Conclusion:** You can now solve models with complex heterogeneity and uncertainty, and simulate the implied dynamics. By introducing various policies you can quantify their effect not just for the average, but for the full distribution.

# 7. Summary

**This lecture:**

1. Solving multidimensional optimization problems with and without gradients (and hessians)
2. Using multistart to alleviate problems with local minima (due to non-convexities)
3. Using penalty terms to solve constrained optimization problems
4. Linear interpolation between known points
5. Solving dynamic optimization problems backwards period-by-period

**Dynamic optimization:** Extremely useful technique. Can handle multiple periods, multiple states and choices, more shocks etc. You can solve general equilibrium models where the households solve such problems.

**Need more dynamic optimization?** [Mini-Course on Dynamic Programming](https://github.com/NumEconCopenhagen/ConsumptionSavingNotebooks/tree/master/00.%20DynamicProgramming#mini-course-in-dynamic-programming)

**Next lecture:** Canonical Economic Models.


# Expectations

## On scale

+ Given the parameters, constants, the period profit is around 2.2. Given that δ = 0.95, the value function should then be

$$ {2.2 \over (1- δ)} \approx 44$$

## On elasticities

+ As product becomes more inelastic, profit goes down. while prices go up
+ As product becomes more elastic, profit goes up, while prices go down

## On variance

+ With a risk neutral firm, it's not clear that a higher variance would reduce value
+ Higher variance should imply more variation in prices over time. Optimal price should vary with variance (fixing expected elasticity)

```python
%matplotlib inline

import matplotlib.pyplot as plt
import dill
import numpy as np
import pandas as pd
file_n = "2019-4-12medium_prod_vfi_dict.dill"
with open('../data/' + file_n, 'rb') as file:
    data_d = dill.load(file)

import sys
sys.path.append('../')
import src

lambdas = src.generate_simplex_3dims(n_per_dim=data_d['n_of_lambdas_per_dim'])
price_grid = np.linspace(data_d['min_price'], data_d['max_price'])

policy = data_d['policy']
valueF = data_d['valueF']

lambdas_ext = src.generate_simplex_3dims(n_per_dim=30)
lambdas_ext.shape
```

    (465, 3)

```python
from src import const
from typing import Callable

def period_profit(p: np.ndarray, lambdas: np.ndarray,
                  betas_transition=const.betas_transition):
    """
    Correct expected period return profit. See ReadMe for derivation

    p: level prices, NOT log
    """
    constant_part = (p-const.c) * np.e ** const.α * np.e ** ((const.σ_ɛ ** 2) / 2)
    summation = np.dot(np.e**(betas_transition*np.log(p[:, np.newaxis])), lambdas)

    return constant_part*summation

def myopic_price(lambdas: np.ndarray, betas_transition=const.betas_transition):
    """
    Given a lambda point, spits out optimal myopic price
    """
    #Elasticity implied by lambdas
    elasticity = np.dot(lambdas, betas_transition) #-2.2
    assert (elasticity < -1.0).all()
    return const.c / (1 + (1/elasticity))

def v0(lambdas_except_last: np.ndarray) -> Callable:
    """

    :param lambdas_except_last: D-1, then augmented
    :return:
    """
    full_lambdas = np.array(list(lambdas_except_last) + [1 - lambdas_except_last.sum()])
    optimal_price: float = myopic_price(full_lambdas)

    #Dirty trick because period_profit takes a vector price, not scalar
    prices = np.array([optimal_price, optimal_price+0.01])
    return period_profit(prices, full_lambdas)[0]

med_lambda = np.array([0., 0.1])

v0(med_lambda)
```

    2.896051097787312

```python
#Numbers taken from aguirragabiria_julia.ipynb

#Lambda1 become more relevant: which means demand becomes more elastic
#As product becomes more elastic, profit goes up
lambdas_selected = np.array([[0.0, 0.5, 0.5],
                    [0.1, 0.5, 0.4],
                    [0.2, 0.5, 0.3],
                    [0.3, 0.5, 0.2],
                    [0.4, 0.5, 0.1],
                    [0.5, 0.4, 0.1],
                    [0.6, 0.3, 0.1],
                    [0.7, 0.2, 0.1],
                    [0.8, 0.1, 0.1],
                    [0.9, 0.1, 0.0],
                    [1.0, 0.0, 0.0]])

valuef_values = valueF(lambdas_selected[:, 0:2])

fig, ax = plt.subplots(1, 1)
ax.plot(np.linspace(0, 1, num=11), valuef_values)

# Plot goes UP!
```

```python

```

## Analyze Value function as lambdas change value

Betas are given by : [-3. , -2.5, -2. ]

+ Thus, the first lambda is associated with -3 (more elastic)
+ The third one is associated with -2 (less elastic, more inelastic)

```python
lambdas_selected = np.array([[0.0, 0.5, 0.5],
                    [0.1, 0.5, 0.4],
                    [0.2, 0.5, 0.3],
                    [0.3, 0.5, 0.2],
                    [0.4, 0.5, 0.1],
                    [0.5, 0.4, 0.1],
                    [0.6, 0.3, 0.1],
                    [0.7, 0.2, 0.1],
                    [0.8, 0.1, 0.1],
                    [0.9, 0.1, 0.0],
                    [1.0, 0.0, 0.0]])

lambda1 = lambdas_selected[:, 0]
valuef_values = valueF(lambdas_selected[:, 0:2])

fig, ax = plt.subplots(1, 1)
ax.plot(lambda1, valuef_values)
ax.set_title("""Value function as the first lambda goes from 0 to 1
                \n (Demand becomes more elastic)
                """, size=16)
```

```python
lambdas_selected2 = np.array([[1., 0.],
                            [0.9, 0.1],
                            [0.8, 0.2],
                            [0.7, 0.3],
                            [0.6, 0.4],
                             [0.5, 0.5],
                             [0.4, 0.6],
                             [0.3, 0.7],
                             [0.2, 0.8],
                             [0.1, 0.9],
                             [0., 1.]
                            ])

lambda2 = lambdas_selected2[:, 1]
valuef_values = valueF(lambdas_selected2)

fig, ax = plt.subplots(1, 1)
ax.plot(lambda2, valuef_values)
ax.set_title("""Value function as the second lambda goes from 0 to 1
                \n (Demand becomes more inelastic)
                """, size=16)
```

## Analysis for myopic price and period profit

```python
from src import const

def period_profit(p, lambdas, betas_transition=const.betas_transition):
    """
    Not the right expected profit (expected value doesn't make epsilon go away)
    but it should be close enough
    """
    E_β = src.exp_b_from_lambdas(lambdas, betas_transition)
    logq = const.α + E_β*np.log(p)
    return (p-const.c)*np.e**logq

def myopic_price(lambdas: np.ndarray, betas_transition=const.betas_transition):
    """
    Given a lambda point, spits out optimal myopic price
    """
    #Elasticity implied by lambdas
    elasticity = np.dot(lambdas, betas_transition) #-2.2
    assert (elasticity < -1.0).all()
    return const.c / (1 + (1/elasticity))

def optimized_period_profit(lambdas):
    opt_p = myopic_price(lambdas)
    E_β = src.exp_b_from_lambdas(lambdas, const.betas_transition)
    logq = const.α + E_β*np.log(opt_p)

    return (opt_p-const.c)*np.e**logq
```

```python
lambdas_selected_3d = np.array([[0., 0.5, 0.5],
                            [0.1, 0.5, 0.4],
                            [0.2, 0.5, 0.2],
                            [0.3, 0.5, 0.2],
                            [0.4, 0.5, 0.1],
                             [0.5, 0.4, 0.1],
                             [0.6, 0.3, 0.1],
                             [0.7, 0.2, 0.1],
                             [0.8, 0.1, 0.1],
                             [0.9, 0.1, 0.],
                             [1., 0., 0.]
                            ])

lambda1 = lambdas_selected_3d[:, 0]
profit_values = optimized_period_profit(lambdas_selected_3d)

fig, ax = plt.subplots(1, 1)
ax.plot(lambda1, profit_values)
ax.set_title("Optimized period profit as lambda 1 goes from 0 to 1", size=16)
#from -2.2 to -3: As product becomes more elastic, prices go down. Profit goes up?
```

```python
E_β = src.exp_b_from_lambdas(lambdas_selected_3d, const.betas_transition)
```

```python
E_β
```

    array([-2.25, -2.35, -2.25, -2.55, -2.65, -2.7 , -2.75, -2.8 , -2.85,
           -2.95, -3.  ])

```python
const.betas_transition
```

    array([-3. , -2.5, -2. ])

```python
# Price goes down from  -2.2 to -3 (as product becomes more elastic)

lambda1 = lambdas_selected_3d[:, 0]
prices = myopic_price(lambdas_selected_3d, const.betas_transition)
fig, ax = plt.subplots(1, 1)
ax.plot(lambda1, prices)

ax.set_title("""
Prices over lambda1
\n
(as demand becomes more elastic)
""", size=15)
```

## Mathematical formulas for optimized price and profit

```python
from sympy import *
init_printing(use_unicode=True)
```

```python
p, c, α, β = symbols('p c α β', real=True)

logq = α + β*log(p)
q = exp(logq)
profit = (p-c)*q

optimal_price = solve(simplify(diff(profit, p)), p)[0]
optimal_price
```

```python
optimized_profit = (optimal_price - c)*exp(α + β*log(optimal_price))
simplify(diff(optimized_profit, β))
```

```python

def optimized_period_profit_over_elasticities(elasticities):
    opt_p = const.c*elasticities /(elasticities+1)
    logq = const.α + elasticities*np.log(opt_p)

    return (opt_p-const.c)*np.e**logq

def deriv_opt_profit_wrt_beta(β, α=1, c=0.5):
    num1 = c*np.exp(α + β*np.log( c*β /(β+1)  ))
    num2 = np.log(c*β /(β+1))
    num = num1*num2
    den = β + 1
    return -num/den

elasticities = np.linspace(-2.0, -5.0)
derivatives = deriv_opt_profit_wrt_beta(elasticities)
fig, [ax1, ax2] = plt.subplots(2, 1, figsize=(8, 10))
ax1.plot(elasticities, derivatives)
ax1.set_title("""
Derivative of Profit wrt elasticity. As product become more inelastic, price go up.\n
The derivative is always negative. So as product becomes more inelastic, value goes down""")

profits = optimized_period_profit_over_elasticities(elasticities)
ax2.plot(elasticities, profits)
ax2.set_title("""Profit wrt elasticity. As product becomes more inelastic, profit goes down.
              As product becomes more elastic, profit goes up.""")

fig.tight_layout()
```

```python
profit2 = (p-c)*exp(log(q))
mr = diff(profit, )
```

```python
def price_over_elast(elasticity):
    return const.c / (1 + (1/elasticity))
```

```python
elasticities = np.linspace(-2.0, -5.0)
prices = price_over_elast(elasticities)
fig, ax = plt.subplots(1, 1)
ax.plot(elasticities, prices)
ax.set_title("""
Prices. As product become more inelastic, price go up.\n
As product becomes more elastic, prices go down""")
```