# Lecture 14:  Solutions to Ordinary Differential Equations and Viscoelasticity
----

## Reading and Reference

* Advanced engineering Mathematics, E. Kreyszig, John wiley and Sons, 2010
* Numerical Recipes, W. Press, Cambridge University Press, 1986
* M. De Graef and M. McHenry, Structure of Materials, Cambridge University Press, 2nd ed.
* C. Hammond, The Basics of Crystallography and Diffraction, Oxford Science Publications, 4th ed.
* The content in the Introduction is modified from an online lecture from [David Balmer](http://www.homepages.ed.ac.uk/johnc/teaching/fluidmechanics4/2003-04/visco/index.html).

## What to Learn?

* The definition of an integral transform
* The algorithm for computing the discrete Fourier transform
* How diffraction patterns can be used to create phase contrast images in electron microscopy
* Learn how to solve ODEs using a CAS such as `Sympy`
* Use `SciPy`'s functionality to solve an ODE
* Apply these methods to problems of viscoelasticity

## What to Do?

* Compute the Fourier transform of different aperture functions.
* Select different regions of a Fourier transform to reconstruct a simulated TEM image and an image of your choosing.

### Introduction
----

The spring obeys Hooke's law:

$$\sigma=M\epsilon $$

where $\sigma$ is the stress, $M$ is the modulus of elasticity in the linear regime, and $\epsilon$ is the elastic strain.

The dashpot obeys:

$$\sigma = \eta \frac{d \epsilon}{dt}$$

where $\sigma$, and $\epsilon$ are defined as above and $\eta$ is related to the viscous response.

Some materials exhibit time dependent behavior under loading.  In polymers this behavior originates from chain elasticity due to coiling and uncoiling and non-reversible motion of the chains relative to one another.  The elastic and viscous behaviors are typically modeled using the spring and [dashpot](https://en.wikipedia.org/wiki/Dashpot).

In this lecture we will discuss the Maxwell model and the Kelvin model of viscoelastic materials.  Combinations of springs and dashpots can provide insight into materials behavior.  Combining many such Maxwell and Kelvin models can give insight into the spectrum of dynamic mechanical behavior seen in polymer systems.

### Maxwell Model

### Kelvin Model

To give you some concept for what we will discuss, this section covers a short derivation for the spring and dashpot in series.  Our first assumption is that for a spring and a dashpot in series that the stress is equally carried by each member in the load train and that the strain is a sum of strains in each element.  This is stated as:

$$ \epsilon_{total} = \epsilon_s + \epsilon_d $$

Taking the time derivative for each term produces:

$$ \frac{d\epsilon_{\mathrm{total}}}{dt} = \frac{d\epsilon_s}{dt} + \frac{d\epsilon_d}{dt} $$

We assume that the spring response is instantaneous so that we can write:

$$\frac{d\sigma}{dt} = M\frac{d\epsilon}{dt}  $$

Using the condition that the stresses are the same reduces the number of variables by one.  Using the condition that the strains add up to the total strain results in:

$$ \frac{d \epsilon_{total}(t)}{dt} = \frac{1}{M} \frac{d \sigma(t)}{dt} + \frac{\sigma(t)}{\eta}$$

Don't forget that both stress and strain are functions of time.

#### First Order ODE: DC Circuit

A simple DC circuit includes a switch (initially open), an inductor (L=2) and a resistor (R=6) in series.  The DC voltage is 100V.  What is the current in the circuit after the switch is closed?  We know that:

$$ E_L + E_R = 100V $$

and that

$$ E_L = L \frac{dI(t)}{dt} $$

and

$$ E_R = RI(t) $$

(You will see a strong correlation between these problems and viscoelastic problems later in this lecture.)  Typically, one would solve this problem by separation of variables.  In `sympy` there is a function called `dsolve()` that can help us with this.  Let us import `sympy` into the current namespace in a way that helps us keep track of what functions are coming from `sympy`:

```python
%matplotlib notebook
import sympy as sp
```

```python
# can also run quietly using:
#sp.init_session(quiet=True)

# set up some common symbols and report back to the user.
sp.init_session()
```

We create some additional symbols that are a bit easier to remember in this problem

```python
current, voltage = sp.symbols('current voltage', cls=Function)
```

We assign to the variable `equationToSolve` the equation we are going to pass to `dsolve()`.  At this point I have to say that Mathematica is really way more advanced than `sympy`.  If you find yourself really doing some heavy computing you may want to default to numerical methods if you are desperate to stay with Python or consider switching to Mathematica to do symbolic calculations.  I'm really caught between two worlds on this one.

```python
# This creates a relation between two objects.
?Eq
```

```python
equationToSolve = Eq(2*current(t).diff(t) + 9*current(t), 100)
equationToSolve
```

Here we solve the equation.

```python
solutionToEquation = dsolve(equationToSolve, current(t))
solutionToEquation
```

Unfortunately, when we use `dsolve()` the integration constants are not automatically created for us.

```python
var('C1 C2')
```

We substitute the initial condition and then solve for C1.  Afterwards we substitute C1 back into the general solution and plot the result.

```python
particularSolution = solutionToEquation.subs([(current(t),0),(t,0)])
particularSolution
```

```python
solutionSet = solveset(particularSolution,C1)
solutionList = [a for a in solutionSet]
solutionSet
```

```python
plot(solutionToEquation.subs(C1,-100).rhs,(t,0,2));
```

This time with no substitutions.

```python
L, R, t, V0 = symbols('L R t V0', positive=True)
A, V = symbols('A V', cls=Function)

rLCircuit = A(t).diff(t) + (R/L)*A(t) - V(t)
rLCircuit

solution = dsolve(rLCircuit, A(t))
solution
```

```python
particularSolution = dsolve(rLCircuit, A(t)).subs([(A(t),0),(t,0),(V(t),V0)]).doit()
particularSolution
```

`solveset` returns a set object.  When the set is a `FiniteSet` we can convert to a list and de-reference the list as I've done here.  Not sure this is good practice, in general.  So - I would only do this in an interactive session when I can inspect the return value.  I would not write library code like this.

```python
const1 = -L*V0/R
```

```python
simplify(solution.subs([(C1,const1),(V(t),V0)]).doit())
```

#### First Order ODE: Skydiving

This example is taking from Arfken's book.  Example problem 7.2.1.  The model describes a falling skydiver under a parachute.  We are trying to find the velocity profile as a function of time as well as the terminal velocity.  The equation of motion is:

$$ m \dot{v} = m g - b v^2 $$

where v is $v(t)$, a function of time.  Later in the problem we will provide values for $m$, $g$, and $b$ to find a particular solution.

```python
#from sympy import *
#init_session()
```

```python
mass, g, b, t = symbols('mass g b t', real=True)
v = symbols('v', cls=Function)

paraChute = mass*v(t).diff(t)-mass*g+b*(v(t))**2
paraChute
```

```python
solution = dsolve(paraChute,v(t),hint='lie_group')
solution
```

```python
particularSolution = solution.subs([(v(t),0),(t,0)])
particularSolution
```

```python
C1 = symbols('C1')

const = solve(particularSolution, C1)
const
```

```python
final_solution = (solution.subs(C1,const[1]).rhs).expand().simplify()
final_solution
```

We need to think a bit about our identities.

```python
1/tanh(t - I*pi/2).expand(complex=True)
```

```python
1/tanh(t - I*pi/2).expand(complex=True).simplify()
```

Now we can substitute and plot the solution.

```python
#%matplotlib notebook

plot(final_solution.subs([(g,9.8),(mass,70000),(b,700000)]),(t,0,2))
```

Numpy and SciPy give us the ability to do this work numerically, too.

```python
#%matplotlib notebook

from scipy.integrate import odeint  # for integrate.odeint
from pylab import *  # for plotting commands

def deriv(velocity,time):
    mass = 70000.
    drag = 700000.
    gravity = 9.8
    dvdt = gravity-(drag/mass)*velocity**2
    return array(dvdt)

time = linspace(0.0,1.0,1000)
yinit = array([0.0])  # initial values
y = odeint(deriv,yinit,time)

figure()
plot(time, y[:,0])
xlabel('t')
ylabel('y')
show()
```

#### DIY:  Nonseparable Exact ODE

Solve the following differential equation and plot the family of solutions for different constants:

$$
\frac{d f(x)}{dx} + \frac{1 + f(x)}{x} = 0
$$

```python
# your dsolve code goes here!
```

```python
# it is good practice to put your solution back into the differential equation to check the results.
```

```python
# your plotting code goes here!
```

#### RL Circuit with Forcing

Solve Example 7.2.5 from Arfken.  Assume that $I(t=0) = 0$.  Find the solution first, then allow $V(t) = V0$ (a constant).  While we won't do it here, you could let $V(t)$ be a function like $V0 \sin(\omega t)$.

```python
#%matplotlib notebook
#from sympy import *
#init_printing()
```

```python
L, R, t = symbols('L R t')
A, V = symbols('A V', cls=Function)
```

```python
rLCircuit = L*A(t).diff(t) + R*A(t) - V(t)
rLCircuit
```

```python
sp.symbols('V0')
```

```python
rLCircuit_constantV = rLCircuit.subs(V(t),V0)
rLCircuit_constantV
```

```python
solution = dsolve(rLCircuit_constantV, A(t))
solution
```

```python
solution_with_ics = solution.subs([(A(t),0),(t,0)])
solution_with_ics
```

```python
sp.symbols('C1')
const = sp.solve(solution_with_ics.rhs,C1)
const
```

```python
particularSolution = (solution.rhs).subs(C1, ((L/R)*sp.log(-V0))).simplify()
particularSolution
```

Redo this with interact?

```python
sp.plot(particularSolution.subs([(R,20),(V0,10),(L,2)]),(t,0,10));
```

#### Using `dsolve` in an Organized Way

```python
#import sympy as sp
#sp.init_printing()
```

The `dsolve` documentation can be found [here](http://docs.sympy.org/dev/modules/solvers/ode.html) and covers the types of equations that can be solved currently.  In a simple case the function `dsolve` is called/used by supplying the differental equation of interest and the function for which the solution is desired.  The results are then used in subsequent calculations to determine the constants in the problem.  We will use the examples below to reinforce this simple view of differential equation solving.

My recommendation for developing your solutions is as follows:

* express the equation/model you are interested in solving in the most general format
* substitute any known functional relationships that can eliminate unknown terms or derivatives
* call `dsolve` to find the general solution for your model
* substitute initial conditions into the results to determine the constants
* substitute the constants into the general solution to get your particular solution
* functionalize the results and visualize the results in a plot (if appropriate)
* use `interact` to permit exploration of free parameters and continue to develop an intuititve understanding of your model

```python
from sympy import Function, dsolve, Eq, Derivative, sin, cos, symbols
from sympy.abc import x
f = Function('f')
```

```python
modelToSolve = sp.Eq(Derivative(f(x), x, x) + 9*f(x),0)
modelToSolve
```

```python
generalSolution = dsolve(modelToSolve, f(x))
generalSolution
```

There are other ways to specify the derivative.

```python
dsolve(sp.diff(f(x),x,2) + 9*f(x), f(x))
```

#### Models of Viscoelastic Behavior:  Maxwell Model - Constant Stress

The physical experiment is one where we apply a constant stress to the Maxwell model.  The model responds by setting the stress in every element as constant and the strain is the sum of the individual strains.  The thought experiment is as follows:

* A stress is applied to the series (chain) of elements at rest
* The spring responds by Hooke's law instantaneously
* The dashpot is unresponsive at $t=0$
* From $t>0$ the spring's strain is fixed but the dashpot begins to strain
* There is no limit on how far the dashpot can extend
* A good exercise is to sketch a diagram of these steps

We begin by setting up our environment.  We import `interact` and `fixed` to help us plot and visualize the results of our computation.

```python
#%matplotlib notebook
from ipywidgets import interact, fixed
#import sympy as sp
```

```python
#sp.init_session()
sp.var('omega sigma epsilon eta M sigma0 C1 C2')
```

The Maxwell model is defined as follows:

```python
maxwellModel = Eq(epsilon(t).diff(t), sigma(t).diff(t)/M + sigma(t)/eta)
maxwellModel
```

First we will work out the solution for a constant applied stress, $\sigma_0$.  We start with the general solution and make the substitution that $\sigma(t)$ is constant by changing the symbol from a function of `t` to $\sigma_0$.

```python
generalSolution = dsolve(maxwellModel,epsilon(t))
generalSolution
```

```python
constantStressSolution = dsolve(maxwellModel,epsilon(t)).subs(sigma(t),sigma0).doit()
constantStressSolution
```

We know that at $t=0$ in this system, Hooke's law defines the strain at $t=0$.

```python
constantStressSolution.subs([(epsilon(t),sigma0/M),(t,0)])
```

```python
solutionToPlot = constantStressSolution.subs(C1,sigma0/M).rhs
solutionToPlot
```

Now we can plot the solution with appropriate substitutions for the two parameters and one constant.  We extract the RHS and use `subs` and `ipywidgets` to interactively plot the solution.  Alternatively you could `lambdafy` the solution and use `numpy` to develop the interactive plot (you might be more satisfied with the results).

```python
def plotSolution(eta0,M0):
    plot(solutionToPlot.subs([(eta,eta0),(M,M0),(sigma0,100)]),
         (t,0,5),
         #ylim=(0,1000),
         xlabel='time',
         ylabel=r'$\epsilon(t)$'
        )

interact(plotSolution,eta0=(1,100,1),M0=(1,100,1));
```

The major takeawy here is that strain rises linearly with time.

#### DIY:  Models of Viscoelastic Behavior:  Maxwell Model - Constant Strain

Because the strain is constant the time rate of change is zero and we can make ths substitution right from the start.

```python
%matplotlib inline
from sympy import *
init_session()
```

```python
var('epsilon, t, sigma, M, eta, epsilon0, sigma0')
```

We still have our maxwellModel defined from a previous computation:

```python
maxwellModel = Eq(epsilon(t).diff(t), sigma(t).diff(t)/M + sigma(t)/eta)
maxwellModel
```

```python
constantStrainModel = maxwellModel.subs(diff(epsilon(t),t),0)
constantStrainModel
```

```python
solutionToConstantStrainModel1 = dsolve(constantStrainModel.rhs,sigma(t))
solutionToConstantStrainModel1
```

```python
dsolve(maxwellModel,sigma(t)).subs(epsilon(t),0).doit()
```

```python
solutionToConstantStrainModel2 = dsolve(maxwellModel,sigma(t))
solution = solutionToConstantStrainModel2.subs(epsilon(t),0).doit()
solution
```

```python
var('C1')
```

```python
solution.subs([(t,0),(sigma(0),sigma0)])
```

Now we make the final substitutions and get our final equation:

```python
solution.subs([(C1,sigma0)])
```

What does this say?  We apply an initial stress at time zero.  The material strains to whatever is predicted by Hooke's law.  And then, without changing the strain again through the experiment, the stress drops exponentially.

```python
solutionToPlot = solution.subs([(C1,sigma0)]).rhs
solutionToPlot
```

Functionalize and plot the results.

```python
from ipywidgets import *

def plotSolution(M0, eta0):
    plot(solutionToPlot.subs([(M,M0),(eta,eta0),(sigma0,100)]),
         (t,0,5),
         ylim=(0,100),
         xlabel='time',
         ylabel=r'$\sigma(t)$'
        )

interact(plotSolution,M0=(1,100,1),eta0=(1,100,1));
```

#### Models of Viscoelastic Behavior:  Kelvin Model - Constant Stress

Following similar logic as above, except this time the strains are all taken to be the same and the stresses are additive.

```python
%matplotlib inline
from sympy import *
init_session()
```

```python
var('epsilon, t, sigma, M, eta, epsilon0, sigma0')
```

```python
kelvinModel = Eq(sigma(t),M*epsilon(t)+eta*epsilon(t).diff(t))
kelvinModel
```

In this problem we will let stress be a constant for all time:

$$ \sigma(t) = \sigma_0 $$

Making the substitution:

```python
kelvinModelConstantStress = kelvinModel.subs(sigma(t),sigma0)
kelvinModelConstantStress
```

```python
kelvinModelSolution = dsolve(kelvinModelConstantStress,epsilon(t))
kelvinModelSolution
```

```python
equationToSolveForC = kelvinModelSolution.subs([(epsilon(t),0),(t,0)])
equationToSolveForC.doit()
```

```python
var('C1')
```

```python
solve(equationToSolveForC.rhs,C1)
```

```python
kelvinModelSolution.subs(C1,eta/M*log(-sigma0)).collect(sigma0,)
```

Plot this for yourself!  Switch to Numpy plotting for performance improvements.

In this section I pose two important differential equation forms that result from the decomposition of Fick's Law.  As stated above, the steps to follow are:

* express the equation/model you are interested in solving in the most general format
* substitute any known functional relationships that can eliminate unknown terms or derivatives
* call `dsolve` to find the general solution for your model
* substitute initial conditions into the results to determine the constants
* substitute the constants into the general solution to get your particular solution
* functionalize the results and visualize the results in a plot (if appropriate)
* use `interact` to permit exploration of free parameters and continue to develop an intuititve understanding of your model

I'll help you get started by writing down the first model equation to solve:

```python
#%matplotlib notebook
import sympy as sp
import numpy as np
from ipywidgets import interact, fixed
sp.init_printing()
```

```python
x, t = sp.symbols('x t')
k, alpha = sp.symbols('k alpha', positive=True)
```

This is the first ODE of interest:

$$
\frac{d x(t)}{dt } \frac{1}{\alpha x(t)} + k = 0
$$

Solve this ODE with boundary conditions: $x(t) = 1$ at $t = 0$ and $x(t) = 0$ as $t \rightarrow \infty$.

```python
firstModelToSolve = x(t).diff(t)/(x(t)*alpha) + k
functionWeAreLookingFor = x(t)
```

```python
firstModelToSolve, functionWeAreLookingFor
```

```python
hintList = sp.classify_ode(firstModelToSolve)
hintList
```

```python
solutionList = [sp.dsolve(firstModelToSolve, functionWeAreLookingFor, hint=hint) for hint in hintList]
solutionList
```

```python
solutionToFirstModel = sp.dsolve(firstModelToSolve, functionWeAreLookingFor, hint=hintList[2])
solutionToFirstModel
```

```python
sp.var('C1'), solutionToFirstModel.subs([(C1,1)])
sp.plot(sp.exp(-2*t),(t,0,10));
```

This is the other ODE of interest:

$$
\frac{d^2 x(t)}{dt^2} \frac{1}{x(t)} + k = 0
$$

Solve this ODE with boundary conditions:

$$
x(t) = 0
$$

at $t = 0$ and $t = 2\pi$

```python
# 0

%matplotlib notebook
import sympy as sp
import numpy as np
from ipywidgets import interact, fixed
sp.init_printing()
x, t = sp.symbols('x t')
k, alpha = sp.symbols('k alpha', positive=True)

sp.diff(sp.diff(x(t))) == x(t).diff(t,2) == x(t).diff(t).diff(t)
```

```python
# 1

(x*k).subs(k,2).subs(x,3) == (x*k).subs([(k,2),(x,3)])
```

```python
# 2

polyEquation = sp.Eq(3*x**2 - 2,8)
sp.solve(polyEquation,x)
```

```python
secondModelToSolve = x(t).diff(t,2)/x(t) + k**2
sp.var('C1 C2')
solutionToSecondModel = sp.dsolve(secondModelToSolve).subs(C2,0)
solutionToSecondModel
```

### Homework
----

#### Solve the Maxwell model for a time varying stress.

Siolve the Maxwell model for a time varying stress.  Solve in `sympy` and plot with `numpy` (you'll get better plotting performance.  You will be graded on your solution as well as your visualization and written response.  Choose a forcing function such as:

$$\sigma(t) = \sigma_0 \sin(\omega t)$$

for this problem.


#### Nguyễn Tiến Dũng

KSTN - Toán Tin - K62

Đại học Bách khoa Hà Nội

20170062

##### Câu 1:

**a.** Không gian trạng thái $$\mathbb{I}_Y = \{-1, 1\}$$

Xét $I_i \in \{\{0\}, \{1;2\}\}$

Ta có:

$$
P(Y_{n+1} = i_{n+1}|Y_{n} = i_n, Y_{n-1} = i_{n-1},...,Y_0 = i_0)
= P(X_{n+1} \in \{I_{n+1}\}|X_n \in \{I_n\},...,X_0 \in \{I_0\})
= \sum P(X_{n+1} = j_{n+1}|X_{n} = j_n, X_{n-1} = j_{n-1},...,X_0 = j_0)
= \sum P(X_{n+1} = j_{n+1}|X_{n} = j_n)
$$

Mặt khác, ta có:

$$
P(Y_{n+1} = i_{n+1}|Y_{n} = i_n) = P(X_{n+1} \in \{I_{n+1}\}|X_n \in \{I_n\})
= \sum P(X_{n+1} = j_{n+1}|X_{n} = j_n)
$$

Từ đó chứng tỏ $(Y_n)$ là một xích Markov

**b.** Ma trận xác suất chuyển.

Ta có
$$
p_{-1, 1}^Y = P(X_n{n+1} = 1|X_n = 0) + P(X_{n+1} = 2|X_n = 0) = \frac{4}{7}
$$

Tương tự như vậy, ma trận xác suất chuyển là
$$
P_Y = \begin{bmatrix}
\frac{3}{7}&\frac{4}{7}\\
0.07x + \frac{1}{11}(1-x) &  0.77x + \frac{10}{11}(1-x)
\end{bmatrix}
$$

##### Câu 2:

\begin{equation}
P = \begin{bmatrix}
-1.164 & -0.622 & 0.993 \\
1.586 & 0.670 & 0.973 \\
1&1&1
\end{bmatrix}\begin{bmatrix}
-0.181 & 0& 0 \\
0 & -0.039 & 0 \\
0&0&0.990
\end{bmatrix}\begin{bmatrix}
-0.184&0.983&-0.774\\
-0.373&-1.313&1.648\\
0.577&0.330&0.125
\end{bmatrix}
\end{equation}

**a.** Từ đó dễ dàng xác định được $P^n$.

**b.** Ta có vector phân phối tại thời điểm $n$ là:

$$
\lambda_n = \lambda*P^n = [-0.119x - 0.130y + 0.568z, 0.633x-0.457y+0.325z, -0.498x+0.382y+0.123z]
$$

với $x = (-0.181)^n, y = (-0.039)^n, z = (0.990^n)$

##### Câu 3:

*Ảnh được đặt trong thư mục assignment_img trong trường hợp lỗi hiển thị*

**a.**

Đồ thị của xích Markov như sau:

Từ đó ta thấy

- Lớp liên thông: $\{1, 2\}, \{4\}$
- Xích Markov không tối giản do không có trạng thái nào đến được `2`

**b.**

Đồ thị của xích Markov như sau:

Từ đó thấy:

- Các lớp liên thông: $\{1\}, \{2\}, \{3,4\}$
- Không tối giản do `0` không liên thông với trạng thái nào

##### Câu 4:

*Ý tưởng*

Chương trình sử dụng `BFS` để tìm đường đi giữa 2 đỉnh trong một đồ thị, tức là nếu tồn tại đường đi trên đồ thị từ đỉnh `source` đến `target` thì trạng thái `source` $\rightarrow$ `target`

*Data structure*

Ta sử dụng `adjacency list` để biểu diễn đồ thị, do đó cần chuyển từ ma trận xác suất chuyển thành `adjacency list`.

Không cần quan tâm đến giá trị các ô trong ma trận, chỉ cần khác 0 là có đường đi

```python
def convert_to_adjacency(state, P):
    adjagecy_vector = {i:[] for i in state}
    for i in range(len(P)):
        for j in range(len(P)):
            if P[i][j] != 0:
                adjagecy_vector[state[i]].append(state[j])
    return adjagecy_vector
```

*Thuật toán*

Sử dụng `BFS`:

```python
def is_connected(state, P, source, target):
    vector = convert_to_adjacency(state, P)
    visit_status = {i: False for i in state}
    queue = []
    queue.append(source)
    while queue != []:
        current_state = queue[0]
        queue.pop(0)
        visit_status[current_state] = True
        for s in vector[current_state]:
            if visit_status[s] == False:
                queue.append(s)
        if target in queue:
            return True
    return False
```

```python
import processviz as pvz
import numpy as np

G = pvz.MarkovChain()
G.from_file()
```

Kiểm tra trạng thái `A` có đến được trạng thái `B` hay không

```python
self.is_connected(source, target)
```

Ví dụ, đối với xích Markov `G`, để kiểm tra trạng thái `i` có thể đến được trạng thái `j` hay không, ta làm như sau:

```python
G.is_connected(i, j)
```

```python
for i in G.state:
    for j in G.state:
        print(i, '->', j, ': ', G.is_connected(i, j))
```

    Rich -> Rich :  True
    Rich -> Neutral :  True
    Rich -> Poor :  True
    Rich -> Loser :  True
    Neutral -> Rich :  True
    Neutral -> Neutral :  True
    Neutral -> Poor :  True
    Neutral -> Loser :  True
    Poor -> Rich :  True
    Poor -> Neutral :  True
    Poor -> Poor :  True
    Poor -> Loser :  True
    Loser -> Rich :  True
    Loser -> Neutral :  True
    Loser -> Poor :  True
    Loser -> Loser :  True

##### Câu 5:

Ma trận xác suất chuyển $T$ được gọi là ma trận xác suất chuyển chính quy (ergodic) nếu tồn tại $n$ sao cho $T^n$ có tất cả các phần tử đều khác 0. Xích Markov được biểu diễn bởi $T$ được gọi là xích Markov chính quy.

Xét xích `Markov` $(X_n)_{n\ge0}$ có ma trận xác suất chuyển $P$.

---
##### Các kiến thức bổ sung:

**Định nghĩa 1:** *Trạng thái $i$ được gọi là `recurrent` nếu $\sum_{n = 1}^{\infty}p_{ii}^{(n)} = \infty$*

**Định nghĩa 2:** *Trạng thái $i$ có `period` $d > 1$ nếu $d$ là số lớn nhất thỏa mãn $p_{ii}^{(n)} > 0$ với mọi $n: d|n$ và $p_{ii}^{(n)} = 0$ trong các trường hợp còn lại $\rightarrow$ `periodic`

**Định nghĩa 3:** *Trạng thái $i$ không có chu kì được gọi là `aperiodic`*

**Định lí 1:** Xích `Markov` hữu hạn là tối giản thì các trạng thái là `recurrent`.

*Chứng minh*

Giả sử tồn tại trạng thái $i$ là `transient`. Khi đó tồn tại $n^*$ là số tự nhiên lớn nhất sao cho $p_{ii}^{n^*} > 0$. Ta chứng minh tồn tại số lớn hơn thoả mãn tính chất này.

Thật vậy, xét $p_{ii}^n$. Do $P$ là `irreducible`, ta đặt $n_{ij}: p_{ij}^{n_{ij}} > 0$. Ta có:

$$
p_{ij}^{n} \ge p_{ii_1}^{m_1}p_{i_1i_2}^{m_2}...p_{i_ni}^{m_{n+1}}, n = \sum_{k =1}^{n+1}m_{k}
$$

Khi đó, với mỗi $i_k \rightarrow i_{k+1}$, ta có thể lấy đoạn đường đi qua các đỉnh con $i_{k1}, i_{k2},..,i_{kn}, i_{k+1}$, mỗi đoạn này là tồn tại do xích là `irreducible`.

Quá trình này tiếp tục cho đến khi $n > n^*$, điều này trái với giả thiết ban đầu.

Điều vô lí này chứng tỏ các trạng thái của $P$ là `recurrent`

**Bổ đề:** *Giả sử $i$ là `aperiodic`, ta chứng minh tồn tại $n_i$ để $\forall n > n_i, p_{ii}^{(n)} > 0$.*

*Chứng minh*

Hiển nhiên dễ thấy $i$ là `recurrent`.

Xét $T = \{n|p_{ii}^{(n)}>0\}$. Do $i$ là `aperiodic` nên dễ thấy $d = GCD(n, n \in T) = 1$. Lấy $n_1, n_2 \in T$ sao cho $GCD(n_1, n_2) = 1$.

Khi đó, theo định lí [Sylvestor - Chicken McNugget theorem](https://artofproblemsolving.com/wiki/index.php/Chicken_McNugget_Theorem), ta có:

$$
\forall n > (n_1-1)(n_2-1) \\\Rightarrow p_{ii}^{(n)} > 0
$$

Ta có điều cần chứng minh.

---
*Chứng minh*

Ta chứng minh nếu $(X_n)$ là `irreducible` và tồn tại một trạng thái là `aperiodic` thì xích $(X_n)$ chính quy.

Thật vậy, giả sử $i$ là `aperiodic`. Ta chứng minh $\forall j \in I$, $j$ là `aperiodic`.

Theo bổ đề trên, hiển nhiên tồn tại $n_1$ sao cho $p_{ii}^{n}>0, n>n_1$. Do $P$ là `irreducible` nên tồn tại $n_{ij}, n_{ji}$ sao cho $p_{ij}^{n_{ij}}>0, p_{ji}^{n_{ji}}>0$.

Có:
$$
p_{jj}^{n_{ij} + n + n_{ji}} \ge p_{ij}^{n_{ij}}p_{ii}^{n}p_{ji}^{n_{ji}} > 0
$$

Lại có:
$$
p_{jj}^{n_{ij} + n+1 + n_{ji}} \ge p_{ij}^{n_{ij}}p_{ii}^{n+1}p_{ji}^{n_{ji}} > 0
$$

Vậy chứng tỏ $j$ là `aperiodic`.

Tiếp theo ta chứng minh tồn tại $N$ để $p_{ij}^N>0$.

Đặt $q = \underset{n_i}{max}\{p_{ii}^{(n)}>0, n>n_i\}$. Khi đó, ta chứng minh tồn tại $t_{ij}$ sao cho $p_{ij}^{(t)} > 0, \forall t > t_{ij}$.

Có:
$$
\left\{\begin{matrix}
p_{ii}^{(q)} > 0\\
p_{ij}^{(t)} \ge p_{ij}^{n_{ij}}p_{jj}^{t - n_{ij}} > 0, \forall t > t_{ij}
\end{matrix}\right.
$$

Ta hoàn toàn có thể chọn được $t_{ij}$ thỏa mãn điều trên bằng cách chọn $t_{ij} - n_{ij} > q$

Đặt $N = \underset{i, j \in I}{max}\{t_{ij}\}$

Dễ thấy rằng $p_{ij}^{(t)} > 0, \forall t>N$

Vậy ta có $(X_n)_{n \ge 0}$ là chính quy.

---

**Remark:** Em mới chỉ tìm được một số điều kiện để chứng minh $P$ là chính quy

Có điều kiện cần $P$ chính quy $\rightarrow$ phương trình $\pi P = \pi$ có nghiệm duy nhất, thì cũng suy ra được nếu $\pi(P - I) = 0$ không có nghiệm duy nhất thì $P$ không chính quy, nhưng mà em cũng không chắc được đó là tất cả các trường hợp cần tìm để $P$ chính quy, ví dụ như nếu rank(P - I) = len(P) nhưng mà không có trạng thái nào là `aperiodic`, hoặc sau khi luỹ thừa lên vài lần nó bắt đầu có hẳn 1 cột bằng 0 (transient).

Tạm thời trong bài này em sẽ kiểm tra 2 điều kiện trước, bao gồm `rank(P - I) = len(P)` và `P` là `irreducible + có một self-loop`

---

**Kiểm tra điều kiện:**

1. Self-loop
2. Rank(P - I) == len(P)

```python
G.is_regular()
```

    False

*Đồ thị minh hoạ cho trường hợp xích Markov không chính quy*

Đồ thị này có `period(i) = 2`, không thoả mãn tính `aperiodic`

```python
G.generate_state_graph(10)
G.generate_graph(4)
G.is_regular()
```

```python

```


Határozzuk meg az alábbi ábrán látható tartó súlypontvonalának eltolódását leíró $v\left(x\right)$ függvényt végeselemes módszer használatával, síkbeli egyenes gerendalemek alkalmazásával.
Vizsgáljuk meg a végeselemes megoldással kapott hajlítónyomatéki igénybevétel hibáját az egyes szakaszokon.
Határozzuk meg az $x = a/2$ keresztmetszetben a hajlítónyomatéki igénybevétel nagyságát 2, illetve 3 síkbeli egyenes gerendaelem alkalmazásával.
A tartók két különböző átmérőjű ($d_1 = 2d$, illetve $d_2 = d$) kör keresztmetszetű tartókból
vannak összeépítve.
A tartók anyaga lineárisan rugalmas, homogén, izotrop. A $d_1$ átmérőjű rész rugalmassági modulusza $E$, míg a $d_2$ átmérővel rendelkező részé $4E$.

```python
import sympy as sp
from sympy import Poly
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt

sp.init_printing()
```

```python
p, xi, L, I, E = sp.symbols("p, xi L I E")
a = sp.IndexedBase('a')
i = sp.Idx('i', (0, 5))
v1, v2, v3 = sp.symbols("v_1 v_2 v_3")
fi1, fi2, fi3 = sp.symbols("phi_1 phi_2 phi_3")
```

```python
ue = [v1,fi1,v2,fi2,v3,fi3]
ve = sp.Sum(a[i]*xi**i,(i,0,5)).doit()
ve
```

```python
fie = sp.diff(ve,xi)*2/L
fie
```

Csomóponti feltételrendszer megoldás

```python
cspfmego = sp.solve([ve.subs([(xi,-1)])-v1,ve.subs([(xi,0)])-v2,ve.subs([(xi,1)])-v3,
                 fie.subs([(xi,-1)])-fi1,fie.subs([(xi,0)])-fi2,fie.subs([(xi,1)])-fi3],[a[0],a[1],a[2],a[3],a[4],a[5]])
cspfmego
```

```python
Nv = sp.Matrix([(ve.subs(cspfmego).expand()).coeff(ue[i]) for i in range(6)])
Nv
```

```python
Nfi=sp.diff(Nv,xi)*(2/L)
sp.simplify((1/L)**(-1)*Nfi)
```

```python
Bv=sp.diff(Nv,xi,2)*(2/L)**2
sp.simplify((1/L**2)**(-1)*Bv)
```

```python
KeBEAM1D = I*E*sp.integrate(Bv*(Bv.T),(xi,-1,1))*L/2
(I*E/(35*L**3))**(-1)*KeBEAM1D
```

```python
Fp = p*sp.integrate(Nv,(xi,-1,1))*L/2
(p*L/60)**(-1)*Fp
```

```python
def Bvfgv(xi,L):
    return np.multiply(1/L**2,np.array([
        60*xi**3 - 24*xi**2 - 30*xi + 8,
        L*(10*xi**3 - 6*xi**2 - 3*xi + 1),
        48*xi**2 - 16,
        8*L*xi*(5*xi**2 - 3),
        -60*xi**3 - 24*xi**2 + 30*xi + 8,
        L*(10*xi**3 + 6*xi**2 - 3*xi - 1)]))
```

```python
def Nvfgv(xi,L):
    return np.array([
        3*xi**5/4 - xi**4/2 - 5*xi**3/4 + xi**2,
        L*xi**5/8 - L*xi**4/8 - L*xi**3/8 + L*xi**2/8,
        xi**4 - 2*xi**2 + 1,
        L*xi**5/2 - L*xi**3 + L*xi/2,
        -3*xi**5/4 - xi**4/2 + 5*xi**3/4 + xi**2,
        L*xi**5/8 + L*xi**4/8 - L*xi**3/8 - L*xi**2/8])
```

```python
def dddNvfgv(xi,L):
    return np.multiply(1/L**3,np.array([
        6*L*(30*xi**2 - 8*xi - 5),
        L**2*(30*xi**2 - 12*xi - 3),
        96*L*xi,
        L**2*(120*xi**2 - 24),
        6*L*(-30*xi**2 - 8*xi + 5),
        L**2*(30*xi**2 + 12*xi - 3)]))
```

```python
def Nfifgv(xi,L):
    return np.multiply(1/L,np.array([
        xi*(15*xi**3 - 8*xi**2 - 15*xi + 8)/2,
        L*xi*(5*xi**3 - 4*xi**2 - 3*xi + 2)/4,
        8*xi*(xi**2 - 1),
        L*(5*xi**4 - 6*xi**2 + 1),
        xi*(-15*xi**3 - 8*xi**2 + 15*xi + 8)/2,
        L*xi*(5*xi**3 + 4*xi**2 - 3*xi - 2)/4]))
```

```python
def KeBEAM1D(I,E,L):
    return np.multiply(I*E/(35*L**3), np.array([
        [5092, 1138*L, -3584, 1920*L, -1508, 242*L],
        [1138*L, 332*L**2, -896*L, 320*L**2, -242*L, 38*L**2],
        [-3584, -896*L, 7168, 0, -3584, 896*L],
        [1920*L, 320*L**2, 0, 1280*L**2, -1920*L, 320*L**2],
        [-1508, -242*L, -3584, -1920*L, 5092, -1138*L],
        [242*L, 38*L**2, 896*L, 320*L**2, -1138*L, 332*L**2]]))
```

```python
def Fp(p,L):
    return np.multiply(L*p/60,np.array([
        14,
        L,
        32,
        0,
        14,
        -L]))
```

```python
a=800e-3
b=400e-3
d=20e-3
E=50e9
nu=0.3
F1=2500
M1=-500
p0=-5000
```

```python
I1 = (2*d)**4*np.pi/64
I2 = (d)**4*np.pi/64
E1 = E
E2 = 4*E
Ke1 = KeBEAM1D(I1,E1,a)
Ke2 = KeBEAM1D(I2,E2,b)
```

```python
elemSZF = np.array([[1,2,3,4,5,6],[5,6,7,8,9,10]]) - 1
```

```python
KG=np.zeros((10,10))
```

```python
eind=0
KG[np.ix_(elemSZF[eind],elemSZF[eind])]+=Ke1
```

```python
eind=1
KG[np.ix_(elemSZF[eind],elemSZF[eind])]+=Ke2
```

```python
FG=np.zeros(10)
FG[elemSZF[0]] += Fp(p0,a)
FG[elemSZF[0,-2]] += F1
FG[elemSZF[1,-1]] += M1
FG
```

    array([ -933.33333333,   -53.33333333, -2133.33333333,     0.        ,
            1566.66666667,    53.33333333,     0.        ,     0.        ,
               0.        ,  -500.        ])

```python
fixSZF=np.array([1,2,9])-1
```

```python
szabadSZF=[i for i in range(0,10) if i not in fixSZF]
szabadSZF
```

```python
KGkond = KG[np.ix_(szabadSZF,szabadSZF)]
```

```python
FGkond = FG[np.ix_(szabadSZF)]
```

```python
Umego = np.linalg.solve(KGkond,FGkond)
Umego
```

    array([ 0.00381972,  0.02015963,  0.01358122,  0.02122066,  0.01252019,
           -0.03289202, -0.0933709 ])

```python
UG = np.zeros(10)
UG[np.ix_(szabadSZF)] += Umego
UG
```

    array([ 0.        ,  0.        ,  0.00381972,  0.02015963,  0.01358122,
            0.02122066,  0.01252019, -0.03289202,  0.        , -0.0933709 ])

```python
FR = np.dot(KG,UG)-FG
FR
```

    array([  1.25000000e+03,  -2.00000000e+02,   1.36424205e-12,
             6.36646291e-12,  -2.27373675e-12,   9.00968189e-12,
             0.00000000e+00,   0.00000000e+00,   2.50000000e+02,
             4.54747351e-13])

# Eredmények ábrázolása

```python
xiLista = np.linspace(-1,1,num = 20)
cspxKRD=[[0,a],[a,a+b]]
cspxKRD
```

```python
x1Lista=[(cspxKRD[0][1]-cspxKRD[0][0])/2*(xi+1) + cspxKRD[0][0] for xi in xiLista]
x2Lista=[(cspxKRD[1][1]-cspxKRD[1][0])/2*(xi+1) + cspxKRD[1][0] for xi in xiLista]
xLista = np.concatenate((x1Lista,x2Lista))
```

```python
v1Lista = [np.dot(Nvfgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
v2Lista = [np.dot(Nvfgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
vLista = np.concatenate((v1Lista,v2Lista))
```

```python
figv = plt.figure(num = 1, figsize=(16/2.54,10/2.54))
axv = figv.add_subplot(111)

axv.plot(xLista,vLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$v \, \left[\mathrm{m}\right]$")

plt.grid()
plt.legend()
plt.show()
```

```python
fi1Lista = [np.dot(Nfifgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
fi2Lista = [np.dot(Nfifgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
fiLista = np.concatenate((fi1Lista,fi2Lista))
```

```python
figfi = plt.figure(num = 2, figsize=(16/2.54,10/2.54))
axfi = figfi.add_subplot(111)

axfi.plot(xLista,fiLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$\varphi \, \left[\mathrm{rad}\right]$")

plt.grid()
plt.legend()
plt.show()
```

```python
Mh1Lista = [-I1*E1*np.dot(Bvfgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
Mh2Lista = [-I2*E2*np.dot(Bvfgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
MhLista = np.concatenate((Mh1Lista,Mh2Lista))
```

```python
figMh = plt.figure(num = 3, figsize=(16/2.54,10/2.54))
axMh = figMh.add_subplot(111)

axMh.plot(xLista,MhLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$M_h \, \left[\mathrm{Nm}\right]$")

plt.grid()
plt.legend()
plt.show()
```

```python
V1Lista = [I1*E1*np.dot(dddNvfgv(xi,a),UG[elemSZF[0]]) for xi in xiLista]
V2Lista = [I2*E2*np.dot(dddNvfgv(xi,b),UG[elemSZF[1]]) for xi in xiLista]
VLista = np.concatenate((V1Lista,V2Lista))
```

```python
figV = plt.figure(num = 4, figsize=(16/2.54,10/2.54))
axV = figV.add_subplot(111)

axV.plot(xLista,VLista)

plt.xlabel(r"$x \, \left[\mathrm{m}\right]$")
plt.ylabel(r"$V \, \left[\mathrm{N}\right]$")

plt.grid()
plt.legend()
plt.show()
```


## **Viscoelastic wave equation implementation on a staggered grid**

This is a first attempt at implementing the viscoelastic wave equation as described in [1]. See also the FDELMODC implementation by Jan Thorbecke [2].

In the following example, a three dimensional toy problem will be introduced consisting of a single Ricker source located at (100, 50, 35) in a 200 m $\times$ 100 m $\times$ 100 *m* domain.

```python
# Required imports:
import numpy as np
import sympy as sp

from devito import *
from examples.seismic.source import RickerSource, TimeAxis
from examples.seismic import ModelViscoelastic, plot_image
```

The model domain is now constructed. It consists of an upper layer of water, 50 m in depth, and a lower rock layer separated by a 4 m thick sediment layer.

```python
# Domain size:
extent = (200., 100., 100.) # 200 x 100 x 100 m domain
h = 1.0 # Desired grid spacing
shape = (int(extent[0]/h+1), int(extent[1]/h+1), int(extent[2]/h+1))

# Model physical parameters:
vp = np.zeros(shape)
qp = np.zeros(shape)
vs = np.zeros(shape)
qs = np.zeros(shape)
rho = np.zeros(shape)

# Set up three horizontally separated layers:
vp[:,:,:int(0.5*shape[2])+1] = 1.52
qp[:,:,:int(0.5*shape[2])+1] = 10000.
vs[:,:,:int(0.5*shape[2])+1] = 0.
qs[:,:,:int(0.5*shape[2])+1] = 0.
rho[:,:,:int(0.5*shape[2])+1] = 1.05

vp[:,:,int(0.5*shape[2])+1:int(0.5*shape[2])+1+int(4/h)] = 1.6
qp[:,:,int(0.5*shape[2])+1:int(0.5*shape[2])+1+int(4/h)] = 40.
vs[:,:,int(0.5*shape[2])+1:int(0.5*shape[2])+1+int(4/h)] = 0.4
qs[:,:,int(0.5*shape[2])+1:int(0.5*shape[2])+1+int(4/h)] = 30.
rho[:,:,int(0.5*shape[2])+1:int(0.5*shape[2])+1+int(4/h)] = 1.3

vp[:,:,int(0.5*shape[2])+1+int(4/h):] = 2.2
qp[:,:,int(0.5*shape[2])+1+int(4/h):] = 100.
vs[:,:,int(0.5*shape[2])+1+int(4/h):] = 1.2
qs[:,:,int(0.5*shape[2])+1+int(4/h):] = 70.
rho[:,:,int(0.5*shape[2])+1+int(4/h):] = 2.
```

Now create a Devito vsicoelastic model generating an appropriate computational grid along with absorbing boundary layers:

```python
# Create model
origin = (0, 0, 0)
spacing = (h, h, h)
so = 4 # FD space order (Note that the time order is by default 1).
nbl = 20 # Number of absorbing boundary layers cells
model = ModelViscoelastic(space_order=so, vp=vp, qp=qp, vs=vs, qs=qs,
                          b=1/rho, origin=origin, shape=shape, spacing=spacing,
                          nbl=nbl)
```

    Operator `initdamp` ran in 0.01 s
    Operator `pad_lam` ran in 0.01 s
    Operator `pad_mu` ran in 0.01 s
    Operator `pad_b` ran in 0.02 s
    Operator `pad_qp` ran in 0.03 s
    Operator `pad_qs` ran in 0.02 s

```python
# As pointed out in Thorbecke's implementation and documentation, the viscoelastic wave euqation is
# not always stable with the standard elastic CFL condition. We enforce a smaller critical dt here
# to ensure the stability.
model.dt_scale = .9
```

The source frequency is now set along with the required model parameters:

```python
# Source freq. in MHz (note that the source is defined below):
f0 = 0.12

# Thorbecke's parameter notation
l = model.lam
mu = model.mu
ro = model.b

k = 1.0/(l + 2*mu)
pi = l + 2*mu

t_s = (sp.sqrt(1.+1./model.qp**2)-1./model.qp)/f0
t_ep = 1./(f0**2*t_s)
t_es = (1.+f0*model.qs*t_s)/(f0*model.qs-f0**2*t_s)
```

```python
# Time step in ms and time range:
t0, tn = 0., 30.
dt = model.critical_dt
time_range = TimeAxis(start=t0, stop=tn, step=dt)
```

Generate Devito time functions for the velocity, stress and memory variables appearing in the viscoelastic model equations. By default, the initial data of each field will be set to zero.

```python
# PDE fn's:
x, y, z = model.grid.dimensions
damp = model.damp

# Staggered grid setup:

# Velocity:
v = VectorTimeFunction(name="v", grid=model.grid, time_order=1, space_order=so)

# Stress:
tau = TensorTimeFunction(name='t', grid=model.grid, space_order=so, time_order=1)

# Memory variable:
r = TensorTimeFunction(name='r', grid=model.grid, space_order=so, time_order=1)

s = model.grid.stepping_dim.spacing # Symbolic representation of the model grid spacing
```

And now the source and PDE's are constructed:

```python
# Source
src = RickerSource(name='src', grid=model.grid, f0=f0, time_range=time_range)
src.coordinates.data[:] = np.array([100., 50., 35.])

# The source injection term
src_xx = src.inject(field=tau[0, 0].forward, expr=src*s)
src_yy = src.inject(field=tau[1, 1].forward, expr=src*s)
src_zz = src.inject(field=tau[2, 2].forward, expr=src*s)

# Particle velocity
pde_v = v.dt - ro * div(tau)
u_v = Eq(v.forward, model.damp * solve(pde_v, v.forward))
# Strain
e = grad(v.forward) + grad(v.forward).T

# Stress equations
pde_tau = tau.dt - r.forward - l * t_ep / t_s * diag(div(v.forward)) -  mu * t_es / t_s * e
u_t = Eq(tau.forward, model.damp * solve(pde_tau, tau.forward))

# Memory variable equations:
pde_r = r.dt + 1 / t_s * (r + l * (t_ep/t_s-1) * diag(div(v.forward)) + mu * (t_es/t_s-1) * e)
u_r = Eq(r.forward, damp * solve(pde_r, r.forward))
```

We now create and then run the operator:

```python
# Create the operator:
op = Operator([u_v, u_r, u_t] + src_xx + src_yy + src_zz,
               subs=model.spacing_map)
```

```python
#NBVAL_IGNORE_OUTPUT

# Execute the operator:
op(dt=dt)
```

    Operator `Kernel` ran in 51.57 s

    PerformanceSummary([(PerfKey(name='section0', rank=None),
                         PerfEntry(time=0.022220999999999998, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[])),
                        (PerfKey(name='section1', rank=None),
                         PerfEntry(time=51.52180899999998, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[])),
                        (PerfKey(name='section2', rank=None),
                         PerfEntry(time=0.019150000000000007, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[]))])

Before plotting some results, let us first look at the shape of the data stored in one of our time functions:

```python
v[0].data.shape
```

    (2, 241, 141, 141)

Since our functions are first order in time, the time dimension is of length 2. The spatial extent of the data includes the absorbing boundary layers in each dimension (i.e. each spatial dimension is padded by 20 grid points to the left and to the right).

The total number of instances in time considered is obtained from:

```python
time_range.num
```

    150

Hence 223 time steps were executed. Thus the final time step will be stored in index given by:

```python
np.mod(time_range.num,2)
```

    0

Now, let us plot some 2D slices of the fields `vx` and `szz` at the final time step:

```python
#NBVAL_SKIP

# Mid-points:
mid_x = int(0.5*(v[0].data.shape[1]-1))+1
mid_y = int(0.5*(v[0].data.shape[2]-1))+1

# Plot some selected results:

plot_image(v[0].data[1, :, mid_y, :], cmap="seismic")
plot_image(v[0].data[1, mid_x, :, :], cmap="seismic")

plot_image(tau[2, 2].data[1, :, mid_y, :], cmap="seismic")
plot_image(tau[2, 2].data[1, mid_x, :, :], cmap="seismic")
```

```python
#NBVAL_IGNORE_OUTPUT

assert np.isclose(norm(v[0]), 0.102959, atol=1e-4, rtol=0)
```

# References

[1] Johan O. A. Roberston, *et.al.* (1994). "Viscoelatic finite-difference modeling" GEOPHYSICS, 59(9), 1444-1456.

[2] https://janth.home.xs4all.nl/Software/fdelmodcManual.pdf


```python
import sympy as sym
```

```python
x = sym.Symbol('x')
xd = sym.Symbol('xd')
xdd = sym.Symbol('xdd')

C = sym.Symbol('C') # Coriolis Force
l = sym.Symbol('lambda')# Inertia of the robot
Dd = sym.Symbol('Dd') # Damping of the virtual enviorment
Kd =  sym.Symbol('Kd') # Spring of the virtual enviroment
Fext =  sym.Symbol('Fext')

```

    xdd

```python
Fext = delta*xdd + (Dd + C)*xd + Kd*x
```

Define a storage function

```python
H =  1/2*xd*delta*xd + 1/2*x*Kd*x
Hd = xd*delta*xdd + 1/2*xd*deltad*xd + x*Kd*xd

xddn =  (Fext -(Dd+C)*xd -Kd*x)*delta**-1

Hd = xd*(Fext -(Dd+C)*xd -Kd*x) + 1/2*xd*deltad*xd + x*Kd*xd

Hd = xd*Fext - xd*(Dd)*xd + xd*(deltad - 2*C)*xd -xd*Kd*x + x*Kd*xd
```

```python
Hd = xd*Fext - xd*(Dd)*xd
```

```python
H = Ht - H
```


This example is kindly contributed by [FreddyBaudine](https://github.com/FreddyBaudine) for reproducing [pygae/galgebra#15](https://github.com/pygae/galgebra/issues/15) with modifications by [utensil](https://github.com/utensil).

```python
import sys
from galgebra.printer import  Format, xpdf, Fmt
Format()
from sympy import symbols, Rational, latex
from galgebra.ga import Ga
```

```python
# st4: Spacetime Algebra I
st4coords = (t,x,y,z) = symbols('t x y z', real=True)
st4 = Ga('e_t e_x e_y e_z', g=[1,-1,-1,-1], coords=st4coords)
(e_t,e_x,e_y,e_z) = st4.mv()
(e__t,e__x,e__y,e__z) = st4.mvr(norm = False)
```

```python
Format(Fmode = True, Dmode = True)
```

```python
(grad,rgrad) = st4.grads()
```

```python
s = st4.mv('s','scalar',f = True)
a = st4.mv('a','vector',f = True)
b = st4.mv('b','vector',f = True)
B = st4.mv('B','bivector',f = True)
C = st4.mv('C',3)
I = st4.mv('I','pseudo')
S = st4.mv('S','spinor',f = True)
M = st4.mv('M','mv',f = True)

M1inv = M.grade(1).inv()
M3inv = M.grade(3).inv()
```

```python
a.Fmt(1,r'\mathbf{a}')
```

\begin{equation*}\mathbf{a} = a^{t}  \boldsymbol{e}_{t} + a^{x}  \boldsymbol{e}_{x} + a^{y}  \boldsymbol{e}_{y} + a^{z}  \boldsymbol{e}_{z}\end{equation*}

```python
(a*a).Fmt(2,r'\mathbf{a} \mathbf{a}')
```

\begin{equation*}\mathbf{a} \mathbf{a} = {a^{t} }^{2} - {a^{x} }^{2} - {a^{y} }^{2} - {a^{z} }^{2}\end{equation*}

```python
a.inv().Fmt(3,r'\mathbf{a}^{-1}')
```

\begin{equation*}\mathbf{a}^{-1} =  \begin{aligned}[t]  & \frac{a^{t} }{{a^{t} }^{2} - {a^{x} }^{2} - {a^{y} }^{2} - {a^{z} }^{2}} \boldsymbol{e}_{t} \\  &  + \frac{a^{x} }{{a^{t} }^{2} - {a^{x} }^{2} - {a^{y} }^{2} - {a^{z} }^{2}} \boldsymbol{e}_{x} \\  &  + \frac{a^{y} }{{a^{t} }^{2} - {a^{x} }^{2} - {a^{y} }^{2} - {a^{z} }^{2}} \boldsymbol{e}_{y} \\  &  + \frac{a^{z} }{{a^{t} }^{2} - {a^{x} }^{2} - {a^{y} }^{2} - {a^{z} }^{2}} \boldsymbol{e}_{z}  \end{aligned} \end{equation*}

```python
(M.grade(1) * M.grade(1)).\
 Fmt(1,r'\langle \mathbf{M} \rangle _1 \langle \mathbf{M} \rangle _1')
```

\begin{equation*}\langle \mathbf{M} \rangle _1 \langle \mathbf{M} \rangle _1 = {M^{t} }^{2} - {M^{x} }^{2} - {M^{y} }^{2} - {M^{z} }^{2}\end{equation*}

```python
M1inv.Fmt(3,r'\langle \mathbf{M} \rangle _1 ^{-1}')
```

\begin{equation*}\langle \mathbf{M} \rangle _1 ^{-1} =  \begin{aligned}[t]  & \frac{M^{t} }{{M^{t} }^{2} - {M^{x} }^{2} - {M^{y} }^{2} - {M^{z} }^{2}} \boldsymbol{e}_{t} \\  &  + \frac{M^{x} }{{M^{t} }^{2} - {M^{x} }^{2} - {M^{y} }^{2} - {M^{z} }^{2}} \boldsymbol{e}_{x} \\  &  + \frac{M^{y} }{{M^{t} }^{2} - {M^{x} }^{2} - {M^{y} }^{2} - {M^{z} }^{2}} \boldsymbol{e}_{y} \\  &  + \frac{M^{z} }{{M^{t} }^{2} - {M^{x} }^{2} - {M^{y} }^{2} - {M^{z} }^{2}} \boldsymbol{e}_{z}  \end{aligned} \end{equation*}

```python
M.grade(3).Fmt(2,r'\langle \mathbf{M} \rangle _3')
```

\begin{equation*}\langle \mathbf{M} \rangle _3 = M^{txy}  \boldsymbol{e}_{t}\wedge \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} + M^{txz}  \boldsymbol{e}_{t}\wedge \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} + M^{tyz}  \boldsymbol{e}_{t}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} + M^{xyz}  \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}\end{equation*}

```python
(M.grade(3) * M.grade(3)).\
 Fmt(1,r'\langle \mathbf{M} \rangle _3 \langle \mathbf{M} \rangle _3')
```

\begin{equation*}\langle \mathbf{M} \rangle _3 \langle \mathbf{M} \rangle _3 = - {M^{txy} }^{2} - {M^{txz} }^{2} - {M^{tyz} }^{2} + {M^{xyz} }^{2}\end{equation*}

```python
M3inv.Fmt(3,r'\langle \mathbf{M} \rangle _3^{-1}')
```

\begin{equation*}\langle \mathbf{M} \rangle _3^{-1} =  \begin{aligned}[t]  & - \frac{M^{txy} }{{M^{txy} }^{2} + {M^{txz} }^{2} + {M^{tyz} }^{2} - {M^{xyz} }^{2}} \boldsymbol{e}_{t}\wedge \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y} \\  &  - \frac{M^{txz} }{{M^{txy} }^{2} + {M^{txz} }^{2} + {M^{tyz} }^{2} - {M^{xyz} }^{2}} \boldsymbol{e}_{t}\wedge \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{z} \\  &  - \frac{M^{tyz} }{{M^{txy} }^{2} + {M^{txz} }^{2} + {M^{tyz} }^{2} - {M^{xyz} }^{2}} \boldsymbol{e}_{t}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z} \\  &  - \frac{M^{xyz} }{{M^{txy} }^{2} + {M^{txz} }^{2} + {M^{tyz} }^{2} - {M^{xyz} }^{2}} \boldsymbol{e}_{x}\wedge \boldsymbol{e}_{y}\wedge \boldsymbol{e}_{z}  \end{aligned} \end{equation*}

```python

```


# Problem Set 2: Classification

To run and solve this assignment, one must have a working IPython Notebook installation. The easiest way to set it up for both Windows and Linux is to install [Anaconda](https://www.continuum.io/downloads). Then save this file to your computer (use "Raw" link on gist\github), run Anaconda and choose this file in Anaconda's file explorer. Use `Python 3` version. Below statements assume that you have already followed these instructions. If you are new to Python or its scientific library, Numpy, there are some nice tutorials [here](https://www.learnpython.org/) and [here](http://www.scipy-lectures.org/).

To run code in a cell or to render [Markdown](https://en.wikipedia.org/wiki/Markdown)+[LaTeX](https://en.wikipedia.org/wiki/LaTeX) press `Ctr+Enter` or `[>|]`(like "play") button above. To edit any code or text cell [double]click on its content. To change cell type, choose "Markdown" or "Code" in the drop-down menu above.

If a certain output is given for some cells, that means that you are expected to get similar results in order to receive full points (small deviations are fine). For some parts we have already written the code for you. You should read it closely and understand what it does.

Total: 100 points.

### 1. Logistic Regression

In this part of the exercise, you will build a logistic regression model to predict whether a student
gets admitted into a university.

Suppose that you are the administrator of a university department and you want to determine
each applicant’s chance of admission based on their results on two exams. You have historical
data from previous applicants in *ex2data1.txt* that you can use as a training set for logistic regression. For each
training example, you have the applicant’s scores on two exams and the admissions decision.

Your task is to build a classification model that estimates an applicant’s probability of admission based on the scores from those two exams. This outline and code framework will guide you through the exercise.

**1\.1 Implementation**

```python
import sys
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
print('Tested with:')
print('Python', sys.version)
print({x.__name__: x.__version__ for x in [np, matplotlib]})
```

    Tested with:
    Python 3.6.6 |Anaconda custom (64-bit)| (default, Jun 28 2018, 17:14:51)
    [GCC 7.2.0]
    {'numpy': '1.15.1', 'matplotlib': '2.2.2'}

**1.1.1 Visualizing the data**

Before starting to implement any learning algorithm, it is always good to visualize the data if possible. This first part of the code will load the data and display it on a 2-dimensional plot by calling the function plotData. The axes are the two exam scores, and the positive and negative examples are shown with different markers.

```python
################################################################################
# Try to fit your code and comments into 80 charecters because
# - it is guaranteed to look as intened on any screen size
# - it encourages you to write "flater" logic that is easier to reason about
# - it encourages you to decompose logic into comprehansible blocks.
#
# Try to avoid reassinging/mutating variables because when you encounter an
# unexplainable error (and you will) it is easier to have the whole history
# of values to reason about.
```

```python
# it is good to isolate logical parts to avoid variables leaking into the
# global scope and messing up your logic later in weird ways

def read_classification_csv_data(fn, add_ones=False):
    # read comma separated data
    data = np.loadtxt(fn, delimiter=',')
    X_, y_ = data[:, :-1], data[:, -1, None]  # a fast way to keep last dim

    # printing statistics of data before working with it might have saved
    # hundreds hours of of my time, do not repeat my errors :)
    print(X_.shape, X_.min(), X_.max(), X_.dtype)
    print(y_.shape, y_.min(), y_.max(), y_.dtype)
    # aha, y is float! this is not what we expected
    # what might go wrong with further y == 0 checks?
    # A: floating point equality comparison, that's what!

    # insert the column of 1's into the "X" matrix (for bias)
    X = np.insert(X_, X_.shape[1], 1, axis=1) if add_ones else X_
    y = y_.astype(np.int32)
    return X, y

X_data, y_data = read_classification_csv_data('ex2data1.txt', add_ones=True)
print(X_data.shape, X_data.min(), X_data.max(), X_data.dtype)
print(y_data.shape, y_data.min(), y_data.max(), y_data.dtype)
```

    (100, 2) 30.05882244669796 99.82785779692128 float64
    (100, 1) 0.0 1.0 float64
    (100, 3) 1.0 99.82785779692128 float64
    (100, 1) 0 1 int32

```python
# how does the *X[y.ravel()==1, :2].T trick work?
# https://docs.python.org/3/tutorial/controlflow.html#unpacking-argument-lists

def plot_data(X, y, labels, markers, xlabel, ylabel, figsize=(10, 6), ax=None):
    if figsize is not None:
        plt.figure(figsize=figsize)

    ax = ax or plt.gca()
    for label_id, (label, marker) in enumerate(zip(labels, markers)):
        ax.plot(*X[y.ravel()==label_id, :2].T, marker, label=label)
    ax.set_xlabel(xlabel)
    ax.set_ylabel(ylabel)
    plt.legend()
    ax.grid(True)

student_plotting_spec = {
    'X': X_data,
    'y': y_data,
    'xlabel': 'Exam 1 score',
    'ylabel': 'Exam 2 score',
    'labels': ['Not admitted', 'Admitted'],
    'markers': ['yo', 'k+'],
    'figsize': (10, 6)
}

plot_data(**student_plotting_spec)
plt.show()
```

**1.1.2 [5pts] Sigmoid function**

Before you start with the actual cost function, recall that the logistic regression hypothesis is defined as:

$h_\theta(x) = g(\theta^Tx)$

where function g is the sigmoid function. The sigmoid function is defined as:

$g(z) = \dfrac{1}{1+e^{-z}}$

Your first step is to implement/find a sigmoid function so it can be called by the rest of your program. Your code should also work with vectors and matrices. For a matrix, your function should perform the sigmoid function on every element.

When you are finished, (a) plot the sigmoid function, and (b) test the function with a scalar, a vector, and a matrix. For scalar large positive values of x, the sigmoid should be close to 1, while for scalar large negative values, the sigmoid should be close to 0. Evaluating sigmoid(0) should give you exactly 0.5.

```python
# check out scipy.special for great variaty of vectorized functions
# remember that sigmoid is the inverse of logit function
# maybe worth checking out scipy.special.logit first

from scipy.special import expit

sigmoid = expit

def check_that_sigmoid_f(f):
    # don't use np.arange with float step because it works as
    # val_{i+1} = val_i + step while val_i < end
    # what might do wrong with float precision?
    x_test = np.linspace(-10, 10, 50)
    sigm_test = f(x_test)
    plt.plot(x_test, sigm_test)
    plt.title("Sigmoid function")
    plt.grid(True)
    plt.show()

    # why should analytical_diff almost== finite_diff for sigmoid?
    analytical_diff = sigm_test*(1-sigm_test)
    finite_step = x_test[1]-x_test[0]
    finite_diff = np.diff(sigm_test) / finite_step
    print(x_test.shape, finite_diff.shape)
    plt.plot(x_test[:-1]+finite_step/2, finite_diff)
    plt.plot(x_test, analytical_diff)
    plt.title("Numerical (finite difference) derivative of 1d sigmoid")
    plt.grid(True)
    plt.show()

check_that_sigmoid_f(sigmoid)
```

**1.1.3 [15pts] Cost function and gradient**

Now you will implement the cost function and gradient for logistic regression. Complete the code
in the functions *hyposesis_function* and *binary_logistic_loss* below to return the value of the hypothesis function and the cost, respectively. Recall that the cost function in logistic regression is

$j(\theta) \ = \ \frac{1}{m} \ \sum_{i=1}^{m} \ [ \ -y^{(i)} log(h_\theta(x^{(i)})) \ - \ (1 - y^{(i)})log(1-h_\theta(x^{(i)})) \ ]$

and the gradient of the cost is a vector of the same length as $\theta$ where the $j^{th}$ element (for $j = 0, 1,...,n$) is defined as follows:

$\frac{\partial J(\theta)}{\partial \theta_{j}} \ = \ \frac{1}{m} \ \sum_{i=1}^{m} \ (h_\theta(x^{(i)})-y^{(i)}) x_j^{(i)}$

where $m$ is the number of points and $n$ is the number of features. Note that while this gradient looks identical to the linear regression gradient, the formula is
actually different because linear and logistic regression have different definitions of $h_\theta(x)$.

What is the value of loss function for $\theta = \bar 0$ regardless of input? Make sure your code also outputs this value.

```python
# we are trying to fit a function that would return a
# "probability of "

# hyposesis_function describes parametric family of functions that we are
# going to pick our "best fitting function" from. It is parameterized by
# real-valued vector theta, i.e. we are going to pick
#    h_best = argmin_{h \in H} logistic_loss_h(x, y, h)
# but because there exist a bijection between theta's and h's it is
# eqvivalent to choosing
#    theta_best = argmin_{theta \in H} logistic_loss_theta(x, y, theta)

def hyposesis_function(x, theta):
    return sigmoid(np.dot(x, theta))

def binary_logistic_loss(y, y_pred):
    """
    Arguments (np arrays of shape):

        x : [m, n] ground truth data
        y : [m, 1] ground truth prediction
        h : [m, n] -> [m, 1] our guess for a prediction function

    """
    assert y_pred.shape == y.shape

    # or weird sign stuff happens! like -1*y != -y
    y, y_pred = y.astype(np.float64), y_pred.astype(np.float64)

    neg_log_likelihoods = -y*np.log(y_pred) - (1-y)*np.log(1-y_pred)
    return np.mean(neg_log_likelihoods)

def logistic_loss_theta_grad(x, y, h, theta):
    y_pred = h(x, theta)
    point_wise_grads = (y_pred - y)*x
    grad = np.mean(point_wise_grads, axis=0)[:, None]
    assert grad.shape == theta.shape
    return grad

def logistic_loss_theta(x, y, h, theta):
    return binary_logistic_loss(y, h(x, theta))

```

```python
# Check that with theta as zeros, cost is about 0.693:
theta_init = np.zeros((X_data.shape[1], 1))
print(logistic_loss_theta(X_data, y_data, hyposesis_function, theta_init))
print(logistic_loss_theta_grad(X_data, y_data, hyposesis_function, theta_init))
```

    0.6931471805599453
    [[-12.00921659]
     [-11.26284221]
     [ -0.1       ]]

**1.1.4 Learning parameters using *fmin***

In the previous assignment, you found the optimal parameters of a linear regression model by
implementing gradient descent. You wrote a cost function and calculated its gradient, then took
a gradient descent step accordingly. This time, instead of taking gradient descent steps, you will
use a scipy.optimize built-in function called *fmin*.

The final $\theta$ value will then be used to plot the
decision boundary on the training data, as seen in the figure below.

```python
import climin
from functools import partial
```

```python
def optimize(theta_init, loss, loss_grad, max_iter=10000, print_every=1000, optimizer_fn=None, show=False):
    theta = theta_init.copy()
    opt_args = (theta, loss_grad)

    if optimizer_fn is None:
        optimizer_fn = partial(climin.GradientDescent, step_rate=1e-3, momentum=0.999)

    optimizer = optimizer_fn(*opt_args)

    loss_curve = []
    for opt_info in optimizer:
        n_iter = opt_info['n_iter']
        f_value = loss(theta)
        loss_curve.append(f_value)

        if print_every != 0 and n_iter % print_every == 0:
            print(n_iter, f_value)

        if n_iter == max_iter:
            break

    if show:
        plt.plot(loss_curve)
        plt.show()

    return theta, f_value
```

```python
theta_init = np.zeros((3, 1))
loss = partial(logistic_loss_theta, X_data, y_data, hyposesis_function)
loss_grad = partial(logistic_loss_theta_grad, X_data, y_data, hyposesis_function)
theta, best_cost = optimize(theta_init, loss, loss_grad, show=True)
print(best_cost)
```

```python
# Plotting the decision boundary: two points, draw a line between
# Decision boundary occurs when h = 0, or when
# theta_0*x1 + theta_1*x2 + theta_2 = 0
# y=mx+b is replaced by x2 = (-1/theta1)(theta2 + theta0*x1)

line_xs = np.array([np.min(X_data[:,0]), np.max(X_data[:,0])])
line_ys = (-1./theta[1])*(theta[2] + theta[0]*line_xs)
plot_data(**student_plotting_spec)
plt.plot(line_xs, line_ys, 'b-', lw=10, alpha=0.2, label='Decision Boundary')
plt.legend()
plt.show()
```

**1.1.5 [15pts] Evaluating logistic regression**

After learning the parameters, you can use the model to predict whether a particular student will
be admitted.

(a) [5 pts] Show that for a student with an Exam 1 score of 45 and an Exam 2 score of 85, you should
expect to see an admission probability of 0.776.

Another way to evaluate the quality of the parameters we have found is to see how well the
learned model predicts on our training set.

(b) [10 pts] In this part, your task is to complete the code in
*makePrediction*. The predict function will produce “1” or “0” predictions given a dataset and a learned
parameter vector $\theta$. After you have completed the code, the script below will proceed to report the
training accuracy of your classifier by computing the percentage of examples it got correct. You
should also see a Training Accuracy of 89.0.

```python
# For a student with an Exam 1 score of 45 and an Exam 2 score of 85,
# you should expect to see an admission probability of 0.776.
check_data = np.array([[45., 85., 1]])
print(check_data.shape)
print(hyposesis_function(check_data, theta))
```

    (1, 3)
    [[0.78755263]]

```python
def predict(x, theta):
    return hyposesis_function(x, theta) >= 0.5

def accuracy(x, y, theta):
    return np.mean(predict(x, theta) == y)

print(accuracy(X_data, y_data, theta))
```

    0.9

### 2. Regularized logistic regression

In this part of the exercise, you will implement regularized logistic regression to predict whether microchips from a fabrication plant pass quality assurance (QA). During QA, each microchip goes through various tests to ensure it is functioning correctly. Suppose you are the product manager of the factory and you have the test results for some microchips on two different tests. From these two tests, you would like to determine whether the microchips should be accepted or rejected. To help you make the decision, you have a dataset of test results on past microchips in *ex2data2.txt*, from which you can build a logistic regression model.

**2.1 Visualizing the data**

Similar to the previous parts of this exercise, plotData is used to generate the figure below,
where the axes are the two test scores, and the positive (y = 1, accepted) and negative (y = 0,
rejected) examples are shown with different markers.

The figure below shows that our dataset cannot be separated into positive and negative examples by a
straight line. Therefore, a straightforward application of logistic regression will not perform well on this dataset since logistic regression will only be able to find a linear decision boundary.

```python
X_data_, y_data = read_classification_csv_data('ex2data2.txt')
X_data = X_data_ - X_data_.mean(axis=0)[None, :]
print(X_data.shape, X_data.min(), X_data.max(), X_data.dtype)
print(y_data.shape, y_data.min(), y_data.max(), y_data.dtype)
```

    (118, 2) -0.83007 1.1089 float64
    (118, 1) 0.0 1.0 float64
    (118, 2) -0.9528415593220338 1.0161210915254237 float64
    (118, 1) 0 1 int32

```python
chip_plotting_spec = {
    'X': X_data,
    'y': y_data,
    'xlabel': 'Microchip Test 1 Result',
    'ylabel': 'Microchip Test 2 Result',
    'labels': ['rejected', 'accepted'],
    'markers': ['yo', 'k+'],
    'figsize': (6, 6)
}

plot_data(**chip_plotting_spec)
plt.show()
```

**2.2 Nonlinear feature mapping**

One way to fit the data better is to create more features from each data point. In *mapFeature* below, we will map the features into all polynomial terms of $x_1$ and $x_2$ up to the
sixth power as follows:

\begin{equation}
mapFeature(x) \ = \
\begin{bmatrix}
    1 \\
    x_1 \\
    x_2 \\
    x_1^2 \\
    x_1x_2 \\
    x_2^2 \\
    x_1^3 \\
    \vdots \\
    x_1x_2^5 \\
    x_2^6 \\
\end{bmatrix}
\end{equation}

As a result of this mapping, our vector of two features (the scores
on two QA tests) has been transformed into a 28-dimensional
vector. A logistic regression classifier trained on this
higher-dimension feature vector will have a more complex
decision boundary and will appear nonlinear when drawn in our
2-dimensional plot.
While the feature mapping allows us to build a more expressive
classifier, it is also more susceptible to overfitting. In the next parts
of the exercise, you will implement regularized logistic regression
to fit the data and also see for yourself how regularization can help combat the overfitting problem.

Either finite dimentional (or even infinite-dimentional, as you would see in the SVM leacture and the corresponding home assingment) feature mappings are usually denoted by $\Phi$ and therefore our hyposesis is now that the Bernoulli probability of chip matfunctioning might be described as

$$ p_i = \sigma(\Phi(x_i)^T \theta)$$

```python
from itertools import combinations_with_replacement

def polynomial_feature_map(X_data, degree=20, show_me_ur_powers=False):
    assert len(X_data.shape) == 2
    group_size = X_data.shape[1]
    # hm.. how to get all groups of size `group_size` of ints
    # such that their sum <= dergee?
    comb_iterator = combinations_with_replacement(range(degree+1), group_size)
    not_quite_powers = np.array(list(comb_iterator))
    powers_bad_order = not_quite_powers.copy()
    powers_bad_order[:, 1] -= not_quite_powers[:, 0]
    rising_power_idx = np.argsort(powers_bad_order.sum(axis=1))
    powers = powers_bad_order[rising_power_idx]
    if show_me_ur_powers is True:
        print(powers.T)
        print('total power per monomial', powers.sum(axis=1))
    X_with_powers = np.power(X_data[:, :, None], powers.T[None])
    # tu tu power rangers (with replacement)
    X_poly = np.prod(X_with_powers, axis=1)
    return X_poly

X_pf = polynomial_feature_map(X_data, show_me_ur_powers=True)
print(X_pf.shape)
```

    [[ 0  0  1  0  2  1  0  2  3  1  4  2  1  3  0  5  3  0  2  4  1  0  6  2
       5  4  3  1  4  0  1  2  6  7  3  5  2  3  4  6  8  5  7  0  1  4  7  3
       6  9  8  0  2  1  5  9  3  5  7  8  4 10  6  1  0  2  0  9  1  4  5  2
      10  7  8  6 11  3  7  0  1  9 12  4  6  2  8 11 10  3  5  6 13  1  5  7
       9  8  4 12  2 10  0 11  3  0 12 11 14 10  9  7  6  8 13  4  2  3  1  5
       5  4  9  0 15 14  7  1 10  8  3  2 11  6 12 13  8  3 11  2 14  9 12 13
      10 16  4  0  6  5  1 15  7 12  5  2 13 11  6 15  3 10  7  0  1 16 14  8
       4  9 17  5 12  2  4  8 14 13 16 11  9  6 15  0  1 10  3 18  7 17 15 13
      14 17 19 18 16  2 12  3  1  4  5  0  7  8  6  9 10 11 11 18  3  1 17 10
       4 16 13  5  0 19 15 12  6  9  7 14  2  8 20]
     [ 0  1  0  2  0  1  3  1  0  2  0  2  3  1  4  0  2  5  3  1  4  6  0  4
       1  2  3  5  3  7  6  5  1  0  4  2  6  5  4  2  0  3  1  8  7  5  2  6
       3  0  1  9  7  8  4  1  7  5  3  2  6  0  4  9 10  8 11  2 10  7  6  9
       1  4  3  5  0  8  5 12 11  3  0  8  6 10  4  1  2  9  7  7  0 12  8  6
       4  5  9  1 11  3 13  2 10 14  2  3  0  4  5  7  8  6  1 10 12 11 13  9
      10 11  6 15  0  1  8 14  5  7 12 13  4  9  3  2  8 13  5 14  2  7  4  3
       6  0 12 16 10 11 15  1  9  5 12 15  4  6 11  2 14  7 10 17 16  1  3  9
      13  8  0 13  6 16 14 10  4  5  2  7  9 12  3 18 17  8 15  0 11  1  4  6
       5  2  0  1  3 17  7 16 18 15 14 19 12 11 13 10  9  8  9  2 17 19  3 10
      16  4  7 15 20  1  5  8 14 11 13  6 18 12  0]]
    total power per monomial [ 0  1  1  2  2  2  3  3  3  3  4  4  4  4  4  5  5  5  5  5  5  6  6  6
      6  6  6  6  7  7  7  7  7  7  7  7  8  8  8  8  8  8  8  8  8  9  9  9
      9  9  9  9  9  9  9 10 10 10 10 10 10 10 10 10 10 10 11 11 11 11 11 11
     11 11 11 11 11 11 12 12 12 12 12 12 12 12 12 12 12 12 12 13 13 13 13 13
     13 13 13 13 13 13 13 13 13 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14
     15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 16 16 16 16 16 16 16 16
     16 16 16 16 16 16 16 16 16 17 17 17 17 17 17 17 17 17 17 17 17 17 17 17
     17 17 17 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 19 19
     19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 20 20 20 20 20 20
     20 20 20 20 20 20 20 20 20 20 20 20 20 20 20]
    (118, 231)

**2.3 Cost function and gradient**

Now you will implement code to compute the cost function and gradient for regularized logistic
regression. Recall that the regularized cost function in logistic regression is:

$j(\theta) \ = \ [ \ \frac{1}{m} \ \sum_{i=1}^{m} \ [ \ -y^{(i)} log(h_\theta(x^{(i)})) \ - \ (1 - y^{(i)})log(1-h_\theta(x^{(i)})) \ ] \ ] \ + \frac{\lambda}{2m} \sum_{j=2}^{n} \theta_j^2 $

Note that you should not regularize the parameter $\theta_0$ (Why not? Think about why that would be a bad idea).

The gradient of the cost function is a vector where the j element is defined as follows (you should understand how to obtain this expression):

$\frac{\partial J(\theta)}{\partial \theta_{0}} \ = \ \frac{1}{m} \ \sum_{i=1}^{m} \ (h_\theta(x^{(i)})-y^{(i)}) x_j^{(i)} \quad \quad \quad \quad \quad \quad$ for $\quad j=0$

$\frac{\partial J(\theta)}{\partial \theta_{j}} \ = \ (\frac{1}{m} \ \sum_{i=1}^{m} \ (h_\theta(x^{(i)})-y^{(i)}) x_j^{(i)}) + \frac{\lambda}{m}\theta_j \quad \quad \quad$ for $\quad j \ge 1$

**2.3.1 [10pts] Implementing regularized logistic regression**

Re-implement computeCost with regularization.

```python
# Cost function, default lambda (regularization) 0
def logistic_loss_theta_w_reg(x, y, h, theta, lambda_=0.0):
    m = x.shape[0]
    reg_term = np.sum(theta[1:]**2)/(2*m)
    loss = logistic_loss_theta(x, y, h, theta)
    total_loss = loss + lambda_ * reg_term
    return total_loss

def logistic_loss_theta_w_reg_grad(x, y, h, theta, lambda_=0.0):
    m = x.shape[0]
    grad = logistic_loss_theta_grad(x, y, h, theta)
    reg_term_grad = lambda_ * theta / m
    reg_term_grad[0] = 0
    return grad + reg_term_grad
```

Once you are done, you will call your cost function using the initial value of
θ (initialized to all zeros). You should see that the cost is about 0.693.

```python
theta_init = np.zeros((X_pf.shape[1], 1))
print(logistic_loss_theta_w_reg(X_pf, y_data, hyposesis_function, theta_init))
print(logistic_loss_theta_w_reg_grad(X_pf, y_data, hyposesis_function, theta_init))

loss = partial(logistic_loss_theta_w_reg, X_pf, y_data, hyposesis_function)
loss_grad = partial(logistic_loss_theta_w_reg_grad, X_pf, y_data, hyposesis_function)
theta, best_cost = optimize(theta_init, loss, loss_grad, max_iter=10000, print_every=0, show=True)
print('best loss', best_cost)
print('best acc', accuracy(X_pf, y_data, theta))
```

**2.3.2 [15pts] Learning parameters using *minimize***

You will use *optimize.minimize* to learn the optimal parameters $\theta$. If you
have completed the cost and gradient for regularized logistic regression correctly, you should be able to learn the parameters
$\theta$ using *minimize*. Implement the function *optimizeRegularizedTheta* below.

**2.4 Plotting the decision boundary**

To help you visualize the model learned by this classifier, we have provided the function
*plotBoundary* which plots the (non-linear) decision boundary that separates the
positive and negative examples.

```python
def plot_boundary(theta, ax=None):
    """
    Function to plot the decision boundary for arbitrary theta, X, y, lambda value
    Inside of this function is feature mapping, and the minimization routine.
    It works by making a grid of x1 ("xvals") and x2 ("yvals") points,
    And for each, computing whether the hypothesis classifies that point as
    True or False. Then, a contour is drawn with a built-in pyplot function.
    """
    ax = ax or plt.gca()
    x_range = np.linspace(-1,1.5,50)
    y_range = np.linspace(-1,1.5,50)
    xx, yy = np.meshgrid(x_range, y_range)
    X_fake = np.stack([xx, yy]).reshape(2, -1).T
    X_fake_fm = polynomial_feature_map(X_fake)
    y_pred_fake = hyposesis_function(X_fake_fm, theta)
    return ax.contour( x_range, y_range, y_pred_fake.reshape(50, 50).T, [0.5])
```

**2.4.1 [10pts] Plot Decision Boundaries**

(a) [4 pts] Use *plotBoundary* to obtain four subplots of the decision boundary for the following values of the regularization parameter: $\lambda \ = \ 0, 1, 10, 100$

(b) [2 pts] Comment on which plots are overfitting and which plots are underfitting.

(c) [2 pts] Which is the model with the highest bias? The highest variance?

(d) [2 pts] What is another way to detect overfitting?

```python
# (a) Build a figure showing contours for various values of regularization parameter, lambda

np.random.seed(2)
train_idx_mask = np.random.rand(X_pf.shape[0]) < 0.3
X_pf_train, y_train = X_pf[train_idx_mask], y_data[train_idx_mask]
X_pf_test, y_test = X_pf[~train_idx_mask], y_data[~train_idx_mask]
print([x.shape for x in (X_pf_train, y_train, X_pf_test, y_test)])

def silent_optimize_w_lambda(lambda_):
    theta_init = np.zeros((X_pf.shape[1], 1))
    data = (X_pf_train, y_train, hyposesis_function)
    loss = partial(logistic_loss_theta_w_reg, *data, lambda_=lambda_)
    loss_grad = partial(logistic_loss_theta_w_reg_grad, *data, lambda_=lambda_)
    optimizer_fn = partial(climin.GradientDescent, step_rate=1e-4, momentum=0.999)
    theta, final_loss = optimize(theta_init, loss, loss_grad, optimizer_fn=optimizer_fn, max_iter=1000, print_every=0, show=False)
    return theta, final_loss

thetas = []
plt.figure(figsize=(12,10))

# wow, I mutates an object used in the scope of another function (plot_data)
# don't do that! it is really hard to debug later
chip_plotting_spec['figsize'] = None

for id_, lambda_ in enumerate([0, 1, 5, 10]):
    ax = plt.subplot(2, 2, id_+1)
    theta, final_loss = silent_optimize_w_lambda(lambda_)
    thetas.append(theta)
    cnt = plot_boundary(theta, ax)
    plot_data(**chip_plotting_spec)
    cnt_fmt = {0.5: 'Lambda = %d' % lambda_}
    ax.clabel(cnt, inline=1, fontsize=15, fmt=cnt_fmt)
    acc = accuracy(X_pf_test, y_test, theta)
    ax.set_title("Decision Boundary, Accuracy = %.2f, Loss=%.2f" % (acc, final_loss))
    ax.plot(*X_data[train_idx_mask].T, 'r.', alpha=0.3)

plt.show()

ax = None
for th_id, theta in enumerate(thetas):
    ax = plt.subplot(2, 2, th_id+1, sharey=ax)
    ax.plot(theta)

plt.show()
```

### 3. Written part

These problems are extremely important preparation for the exam. Submit solutions to each problem by filling the markdown cells below.

**3.1 [10pts]** Maximum likelihood for Logistic Regression

Showing all steps, derive the LR cost function using maximum likelihood. Assume that
the probability of y given x is described by:

$P(\ y=1 \; \vert \; x \ ; \ \theta \ ) = h_{\theta}(x)$

$P(\ y=0 \; \vert \; x \ ; \ \theta \ ) = 1 - h_{\theta}(x)$

**First we write down the likelihood of the data given the parameters, which is:**

**$L(\theta) = \prod_{i=1}^{m}{h(x^i)}^{y^i}(1-h(x^i))^{1-y^i}$**

**Then we take log of both sides to get:**

**$ln L(\theta) = \sum_{i=1}^{m}{h(x^i)y^i+(1-h(x^i))(1-y^i)}$**

**which is the same (up to a constant multiplier) as the logistic regression cost.**

**3.2 [10pts]** Logistic Regression Classification with Label Noise

Suppose you are building a logistic regression classifier for images of dogs, represented by a feature vector x, into one of two categories $y \in \{0,1\}$, where 0 is “terrier” and 1 is “husky.” You decide to use the logistic regression model $p(y = 1 \ \vert \ x) = h_{\theta}(x)=\sigma(\theta^Tx).$ You collected an image dataset **D**$\ = \{x^{(i)},t^{(i)}\}$, however, you were very tired and made
some mistakes in assigning labels $t^{(i)}.$ You estimate that you were correct in about $\tau$ fraction of all cases.

(a) Write down the equation for the posterior probability $p(t = 1 \ \vert \ x)$ of the label being 1 for some point x, in terms of the probability of the true class, $p(y = 1 \ \vert \ x).$

(b) Derive the modified cost function in terms of $\ \theta, x^{(i)},t^{(i)}$ and $\tau$.

**(a) Using the sum rule, **

**$p(t=1 \ \vert \ x) \ = \ p(t=1,y=1 \ \vert \ x) \ + \ p(t=1,y=0 \ \vert \ x)$**

**$ \quad \quad \quad \quad \ = \ p(t=1 \ \vert \ y=1,x)p(y=1 \ \vert \ x) \ + \ p(t=1 \ \vert \ y=0,x)p(y=0 \ \vert \ x)$**

**$ \quad \quad \quad \quad \ = \ \tau \ p(y=1 \ \vert \ x) \ + \ (1-\tau)(1-p(y=1 \ \vert \ x))$**

**Here we used the fact that $p(t=1 \ \vert \ y=1,x)$ is the probability of the label being correct and $p(t=1 \ \vert \ y=0,x)$ is the probability of the label being incorrect.**

**(b) Substituting the expression for $p(t=1 \ \vert \ x)$  from (a) gives the final cost**

**$-ln \ p(D \ \vert \ \theta) \ = \ -\sum_{i=1}^{m} t^{(i)} \ ln \ [\ \tau \ p(y=1 \ \vert \ x) \ + \ (1-\tau)(1-p(y=1 \ \vert \ x)) \ ]$**

**$\quad \quad \quad \quad \quad \quad + \ (1-t^{(i)}) \ ln \ (1 - [ \ \tau \ p(y=1 \ \vert \ x) + (1- \tau)(1-p(y=1 \ \vert \ x)) \ ])$**

**$\quad \quad \quad \quad \ = \ -\sum_{i=1}^{m} t^{(i)} \ ln \ [\ \tau \ \sigma(\theta^Tx^{(i)}) \ + \ (1-\tau)(1-\sigma(\theta^Tx^{(i)})) \ ]$**

**$\quad \quad \quad \quad \quad \quad + \ (1-t^{(i)}) \ ln \ (1 - [ \ \tau \ \sigma(\theta^Tx^{(i)}) + (1- \tau)(1-\sigma(\theta^Tx^{(i)})) \ ])$**

**This is okay, but we can also simplify further:**

**$-ln \ p(D \ \vert \ \theta) \ = \ -\sum_{i=1}^{m} t^{(i)} \ ln \ [\ \sigma(\theta^Tx^{(i)})(2 \tau - 1) - \tau + 1 \ ]$**

**$\quad \quad \quad \quad \quad \quad + \ (1-t^{(i)}) \ ln \ (-\sigma(\theta^Tx^{(i)})(2 \tau -1) + \tau) $**

**3.3 [10pts] Cross-entropy loss for multiclass classification**

This problem asks you to derive the cross-entropy loss for a multiclass classification problem using maximum likelihood.
Consider the multiclass classification problem in which each input is assigned to one of $K$ mutually exclusive classes. The binary target variables $y_k$ ∈ {0, 1} have a "one-hot" coding scheme, where the value is 1 for the indicated class and 0 for all others. Assume that we can interpret the network outputs as $h_k(x,\theta) = p(y_k = 1|x)$, or the probability of the kth class.

Show that the maximum likelihood estimate of the parameters $\theta$ can be obtained by minimizing the multiclass *cross-entropy* loss function
<p>
$L(\theta)= - \frac{1}{N}\sum_{i=1}^{N} \sum_{k=1}^{K} y_{ik} \log(h_k(x_i,\theta))$
</p>
<p>
where $N$ is the number of examples $\{x_i,y_i\}$. </p>

**For a single example $x_i,y_i$, the log-likelihood function can be written as:**

**$\log P(y_{i}\ \vert \ x_i, \theta) = \log \prod_{k=1}^Kh_k(x_i, \theta)^{y_{ik}} =\sum_{k=1}^Ky_{ik} \log h_k(x_i,\theta)$**

**Due to the fact that $y_i$ is one-hot. Then the maximum likelihood solution maximizes**

**$\sum_{i=1}^N \log(P(y_{i}\ \vert \ x_i, \theta)) = \sum_{i=1}^N \sum_{k=1}^K y_{ik} \log(h_k(x_i, \theta))$**

**which is equivalent to minimizing $L(\theta)$. The constant factor $\frac{1}{N}$ does not change the solution.**

### Fix for Windows users

So here's an alternative implementation of `optimize()` that uses scipy instead of climin, if you find it difficult to install this package for some reason. It requires little more work because of the way this function passes values (gradients).

In addition to replacing optimize function with the one given below, you should also

1. add `theta = theta.reshape((-1, 1))` on top of all functions hyposesis_function, logistic_loss_theta_grad, logistic_loss_theta
2. Remove `optimizer_fn` argument from all calls to optimize() such as in 2.4.1
3. Add `ax.set_ylim((-1, 1))` in the 2.4.1(e)
4. And add apply `.ravel()` before returning from your the gradient function

```python
import scipy.optimize

def optimize(theta_init, loss, loss_grad, max_iter=10000, print_every=1000, optimizer_fn=None, show=False):
    theta = theta_init.copy()
    opt_args = {'x0': theta_init, 'fun': loss, 'jac': loss_grad, 'options': {'maxiter': max_iter}}

    loss_curve = []
    def scipy_callback(theta):
        f_value = loss(theta)
        loss_curve.append(f_value)

    if optimizer_fn is None:
        optimizer_fn = partial(scipy.optimize.minimize, method='BFGS', callback=scipy_callback)

    opt_result = optimizer_fn(**opt_args)

    if show:
        plt.plot(loss_curve)
        plt.show()

    return opt_result['x'].reshape((-1, 1)), opt_result['fun']
```


# Enforce conformal 3-metric $\det{\bar{\gamma}_{ij}}=\det{\hat{\gamma}_{ij}}$ constraint (Eq. 53 of [Ruchlin, Etienne, and Baumgarte (2018)](https://arxiv.org/abs/1712.07658))

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

[comment]: <> (Abstract: TODO)

**Notebook Status:**  Validated

**Validation Notes:** This tutorial notebook has been confirmed to be self-consistent with its corresponding NRPy+ module, as documented [below](#code_validation). In addition, its output has been

### NRPy+ Source Code for this module: [BSSN/Enforce_Detgammabar_Constraint.py](../edit/BSSN/Enforce_Detgammabar_Constraint.py)

## Introduction:
[Brown](https://arxiv.org/abs/0902.3652)'s covariant Lagrangian formulation of BSSN, which we adopt, requires that $\partial_t \bar{\gamma} = 0$, where $\bar{\gamma}=\det \bar{\gamma}_{ij}$. Further, all initial data we choose satisfies $\bar{\gamma}=\hat{\gamma}$.

However, numerical errors will cause $\bar{\gamma}$ to deviate from a constant in time. This actually disrupts the hyperbolicity of the PDEs, so to cure this, we adjust $\bar{\gamma}_{ij}$ at the end of each Runge-Kutta timestep, so that its determinant satisfies $\bar{\gamma}=\hat{\gamma}$ at all times. We adopt the following, rather standard prescription (Eq. 53 of [Ruchlin, Etienne, and Baumgarte (2018)](https://arxiv.org/abs/1712.07658)):

$$
\bar{\gamma}_{ij} \to \left(\frac{\hat{\gamma}}{\bar{\gamma}}\right)^{1/3} \bar{\gamma}_{ij}.
$$

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows:

1. [Step 1](#initializenrpy): Initialize needed NRPy+ modules
1. [Step 2](#enforcegammaconstraint): Enforce the $\det{\bar{\gamma}_{ij}}=\det{\hat{\gamma}_{ij}}$ constraint
1. [Step 3](#code_validation): Code Validation against `BSSN.Enforce_Detgammabar_Constraint` NRPy+ module
1. [Step 4](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Initialize needed NRPy+ modules \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

```python
# Step P1: import all needed modules from NRPy+:
from outputC import *             # NRPy+: Core C code output module
import finite_difference as fin   # NRPy+: Finite difference C code generation module
import grid as gri                # NRPy+: Functions having to do with numerical grids
import indexedexp as ixp          # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import reference_metric as rfm    # NRPy+: Reference metric support
import cmdline_helper as cmd      # NRPy+: Multi-platform Python command-line interface
import sympy as sp                # SymPy, Python's core symbolic algebra package
import BSSN.BSSN_quantities as Bq # NRPy+: BSSN quantities
import os,shutil,sys              # Standard Python modules for multiplatform OS-level functions

# Set spatial dimension (must be 3 for BSSN)
DIM = 3
par.set_parval_from_str("grid::DIM",DIM)

# Then we set the coordinate system for the numerical grid
par.set_parval_from_str("reference_metric::CoordSystem","SinhSpherical")
rfm.reference_metric() # Create ReU, ReDD needed for rescaling B-L initial data, generating BSSN RHSs, etc.
```

# Step 2: Enforce the $\det{\bar{\gamma}_{ij}}=\det{\hat{\gamma}_{ij}}$ constraint \[Back to [top](#toc)\]
$$\label{enforcegammaconstraint}$$

Recall that we wish to make the replacement:
$$
\bar{\gamma}_{ij} \to \left(\frac{\hat{\gamma}}{\bar{\gamma}}\right)^{1/3} \bar{\gamma}_{ij}.
$$
Notice the expression on the right is guaranteed to have determinant equal to $\hat{\gamma}$.

$\bar{\gamma}_{ij}$ is not a gridfunction, so we must rewrite the above in terms of $h_{ij}$:
\begin{align}
\left(\frac{\hat{\gamma}}{\bar{\gamma}}\right)^{1/3} \bar{\gamma}_{ij} &= \bar{\gamma}'_{ij}  \\
&= \hat{\gamma}_{ij} + \varepsilon'_{ij} \\
&= \hat{\gamma}_{ij} + \text{Re[i][j]} h'_{ij} \\
\implies h'_{ij} &= \left[\left(\frac{\hat{\gamma}}{\bar{\gamma}}\right)^{1/3} \bar{\gamma}_{ij} - \hat{\gamma}_{ij}\right] / \text{Re[i][j]} \\
&= \left(\frac{\hat{\gamma}}{\bar{\gamma}}\right)^{1/3} \frac{\bar{\gamma}_{ij}}{\text{Re[i][j]}} - \delta_{ij}\\
&= \left(\frac{\hat{\gamma}}{\bar{\gamma}}\right)^{1/3} \frac{\hat{\gamma}_{ij} + \text{Re[i][j]} h_{ij}}{\text{Re[i][j]}} - \delta_{ij}\\
&= \left(\frac{\hat{\gamma}}{\bar{\gamma}}\right)^{1/3} \left(\delta_{ij} + h_{ij}\right) - \delta_{ij}
\end{align}

Upon inspection, when expressing $\hat{\gamma}$ SymPy generates expressions like `(xx0)^{4/3} = pow(xx0, 4./3.)`, which can yield $\text{NaN}$s when `xx0 < 0` (i.e., in the `xx0` ghost zones). To prevent this, we know that $\hat{\gamma}\ge 0$ for all reasonable coordinate systems, so we make the replacement $\hat{\gamma}\to |\hat{\gamma}|$ below:

```python
# We will need the h_{ij} quantities defined within BSSN_RHSs
#    below when we enforce the gammahat=gammabar constraint
# Step 1: All barred quantities are defined in terms of BSSN rescaled gridfunctions,
#         which we declare here in case they haven't yet been declared elsewhere.

Bq.declare_BSSN_gridfunctions_if_not_declared_already()
hDD = Bq.hDD
Bq.BSSN_basic_tensors()
gammabarDD = Bq.gammabarDD

# First define the Kronecker delta:
KroneckerDeltaDD = ixp.zerorank2()
for i in range(DIM):
    KroneckerDeltaDD[i][i] = sp.sympify(1)

# The detgammabar in BSSN_RHSs is set to detgammahat when BSSN_RHSs::detgbarOverdetghat_equals_one=True (default),
#    so we manually compute it here:
dummygammabarUU, detgammabar = ixp.symm_matrix_inverter3x3(gammabarDD)

# Next apply the constraint enforcement equation above.
hprimeDD = ixp.zerorank2()
for i in range(DIM):
    for j in range(DIM):
        # Using nrpyAbs here, as it directly translates to fabs() without additional SymPy processing.
        #    This acts to simplify the final expression somewhat.
        hprimeDD[i][j] = \
        (nrpyAbs(rfm.detgammahat)/detgammabar)**(sp.Rational(1,3)) * (KroneckerDeltaDD[i][j] + hDD[i][j]) \
        - KroneckerDeltaDD[i][j]
```

# Step 3: Code Validation against `BSSN.Enforce_Detgammabar_Constraint` NRPy+ module \[Back to [top](#toc)\]
$$\label{code_validation}$$

Here, as a code validation check, we verify agreement in the C code output between

1. this tutorial and
2. the NRPy+ [BSSN.Enforce_Detgammabar_Constraint](../edit/BSSN/Enforce_Detgammabar_Constraint.py) module.

```python
##########
# Step 1: Generate enforce_detgammabar_constraint() using functions in this tutorial notebook:

Ccodesdir = os.path.join("enforce_detgammabar_constraint")
# First remove C code output directory if it exists
# Courtesy https://stackoverflow.com/questions/303200/how-do-i-remove-delete-a-folder-that-is-not-empty
shutil.rmtree(Ccodesdir, ignore_errors=True)
# Then create a fresh directory
cmd.mkdir(Ccodesdir)

enforce_detg_constraint_vars = [lhrh(lhs=gri.gfaccess("in_gfs","hDD00"),rhs=hprimeDD[0][0]),
                                lhrh(lhs=gri.gfaccess("in_gfs","hDD01"),rhs=hprimeDD[0][1]),
                                lhrh(lhs=gri.gfaccess("in_gfs","hDD02"),rhs=hprimeDD[0][2]),
                                lhrh(lhs=gri.gfaccess("in_gfs","hDD11"),rhs=hprimeDD[1][1]),
                                lhrh(lhs=gri.gfaccess("in_gfs","hDD12"),rhs=hprimeDD[1][2]),
                                lhrh(lhs=gri.gfaccess("in_gfs","hDD22"),rhs=hprimeDD[2][2]) ]

enforce_gammadet_string = fin.FD_outputC("returnstring",enforce_detg_constraint_vars,
                                         params="outCverbose=False,preindent=1,includebraces=False")

desc = "Enforce det(gammabar) = det(gammahat) constraint."
name = "enforce_detgammabar_constraint"
outCfunction(
    outfile=os.path.join(Ccodesdir, name + ".h-validation"), desc=desc, name=name,
    params="const rfm_struct *restrict rfmstruct,const paramstruct *restrict params, REAL *restrict in_gfs",
    body=enforce_gammadet_string.replace("IDX4", "IDX4S"),
    loopopts="AllPoints,Enable_rfm_precompute")

##########
# Step 2: Generate enforce_detgammabar_constraint() using functions in BSSN.Enforce_Detgammabar_Constraint

gri.glb_gridfcs_list = []

import BSSN.Enforce_Detgammabar_Constraint as EGC
EGC.output_Enforce_Detgammabar_Constraint_Ccode(outdir=Ccodesdir,
                                                exprs=EGC.Enforce_Detgammabar_Constraint_symb_expressions())

import filecmp
for file in [os.path.join(Ccodesdir,"enforce_detgammabar_constraint.h")]:
    if filecmp.cmp(file,file+"-validation") == False:
        print("VALIDATION TEST FAILED on file: "+file+".")
        sys.exit(1)
    else:
        print("Validation test PASSED on file: "+file)
##########
```

    Output C function enforce_detgammabar_constraint() to file enforce_detgammabar_constraint/enforce_detgammabar_constraint.h-validation
    Output C function enforce_detgammabar_constraint() to file enforce_detgammabar_constraint/enforce_detgammabar_constraint.h
    Validation test PASSED on file: enforce_detgammabar_constraint/enforce_detgammabar_constraint.h

# Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-BSSN-Enforcing_Determinant_gammabar_equals_gammahat_Constraint.pdf](Tutorial-BSSN-Enforcing_Determinant_gammabar_equals_gammahat_Constraint.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx --log-level='WARN' Tutorial-BSSN-Enforcing_Determinant_gammabar_equals_gammahat_Constraint.ipynb
!pdflatex -interaction=batchmode Tutorial-BSSN-Enforcing_Determinant_gammabar_equals_gammahat_Constraint.tex
!pdflatex -interaction=batchmode Tutorial-BSSN-Enforcing_Determinant_gammabar_equals_gammahat_Constraint.tex
!pdflatex -interaction=batchmode Tutorial-BSSN-Enforcing_Determinant_gammabar_equals_gammahat_Constraint.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode