Text provided under a Creative Commons Attribution license, CC-BY.  All code is made available under the FSF-approved BSD-3 license.  (c) Lorena A. Barba, Gilbert F. Forsyth 2017. Thanks to NSF for support via CAREER award #1149784.
[@LorenaABarba](https://twitter.com/LorenaABarba)

[@ggruszczynski](https://github.com/ggruszczynski)

```python
import timeit
import numpy as np
import matplotlib.pyplot as plt    #and the useful plotting library
```

Diffusion part 1: the fundamental solution
-----
***

The one-dimensional diffusion equation is:

$$\frac{\partial u}{\partial t}= \nu \frac{\partial^2 u}{\partial x^2}$$

The first thing you should notice is that —unlike the previous two simple equations we have studied— this equation has a second-order derivative. We first need to learn what to do with it!

### Discretizing $\frac{\partial ^2 u}{\partial x^2}$

The second-order derivative can be represented geometrically as the line tangent to the curve given by the first derivative.  We will discretize the second-order derivative with a Central Difference scheme: a combination of Forward Difference and Backward Difference of the first derivative.  Consider the Taylor expansion of $u_{i+1}$ and $u_{i-1}$ around $u_i$:

$u_{i+1} = u_i + \Delta x \frac{\partial u}{\partial x}\bigg|_i + \frac{\Delta x^2}{2} \frac{\partial ^2 u}{\partial x^2}\bigg|_i + \frac{\Delta x^3}{3!} \frac{\partial ^3 u}{\partial x^3}\bigg|_i + O(\Delta x^4)$

$u_{i-1} = u_i - \Delta x \frac{\partial u}{\partial x}\bigg|_i + \frac{\Delta x^2}{2} \frac{\partial ^2 u}{\partial x^2}\bigg|_i - \frac{\Delta x^3}{3!} \frac{\partial ^3 u}{\partial x^3}\bigg|_i + O(\Delta x^4)$

If we add these two expansions, you can see that the odd-numbered derivative terms will cancel each other out.  If we neglect any terms of $O(\Delta x^4)$ or higher (and really, those are very small), then we can rearrange the sum of these two expansions to solve for our second-derivative.

$u_{i+1} + u_{i-1} = 2u_i+\Delta x^2 \frac{\partial ^2 u}{\partial x^2}\bigg|_i + O(\Delta x^4)$

Then rearrange to solve for $\frac{\partial ^2 u}{\partial x^2}\bigg|_i$ and the result is:

$$\frac{\partial ^2 u}{\partial x^2}=\frac{u_{i+1}-2u_{i}+u_{i-1}}{\Delta x^2} + O(\Delta x^4)$$

### Discretizing both $\frac{\partial u}{\partial t}$ and $\frac{\partial ^2 u}{\partial x^2}$

We can now write the discretized version of the diffusion equation in 1D:

$$\frac{u_{i}^{n+1}-u_{i}^{n}}{\Delta t}=\nu\frac{u_{i+1}^{n}-2u_{i}^{n}+u_{i-1}^{n}}{\Delta x^2}$$

As before, we notice that once we have an initial condition, the only unknown is $u_{i}^{n+1}$, so we re-arrange the equation solving for our unknown:

$$u_{i}^{n+1}=u_{i}^{n}+\underbrace{\frac{\nu\Delta t}{\Delta x^2}}_{\beta}(u_{i+1}^{n}-2u_{i}^{n}+u_{i-1}^{n})$$

$$u_{i}^{n+1}=\beta u_{i-1}^{n} + u_{i}^{n}(1- 2 \beta) +\beta u_{i+1}^{n}$$

The above discrete equation allows us to write a program to advance a solution in time. But we need an initial condition. Let's continue using our favorite: the hat function. So, at $t=0$, $u=2$ in the interval $0.5\le x\le 1$ and $u=1$ everywhere else. We are ready to number-crunch!

```python
# %matplotlib inline

nx = 128
domain_length = 64
dx = domain_length / (nx-1)
xspace = np.linspace(0, domain_length, nx)

nt = 200                # the number of timesteps we want to calculate
nu = 5                  # the value of viscosity
sigma = .2              # sigma is a parameter, we'll learn more about it later
dt = sigma * dx**2 / nu # dt is defined using sigma ... more later!

u_IC = 0*np.ones(nx)                  # numpy function ones()
u_IC[int((nx-1)/4):int(nx/2 + 1)] = 1 # setting u = 2 between 0.5 and 1 as per our I.C.s

plt.plot(xspace, u_IC)
```

```python
import timeit

def calc_diffusion_FD_naive(IC,nx,nt,nu,dt):
    u = IC.copy()
    un = IC.copy() #our placeholder array, un, to advance the solution in time
    beta = nu * dt / dx**2

    for n in range(nt):  #iterate through time
        un = u.copy() #copy the existing values of u into un
        for i in range(0, nx):
            # this is slow (index operations & branching)
            if i == nx-1:
#                 u[i] = TODO... # periodic BC
            else:
#                 u[i] = TODO...
    return u

def calc_diffusion_FD(IC,nx,nt,nu,dt):
    u = IC.copy()
    un = IC.copy() #our placeholder array, un, to advance the solution in time
    beta = nu * dt / dx**2

    c_ind = np.arange(0, nx)
    l_ind = np.roll(c_ind, -1)
    r_ind = np.roll(c_ind, 1)

    for n in range(nt):  #iterate through time
        un = u.copy() # copy the existing values of u into un

#         lap_u = # TODO... un[...] - 2 * un[...] + un[...]  # periodic BC
#         u = un + beta* lap_u
    return u

starttime = timeit.default_timer()
u_FD = calc_diffusion_FD(u_IC,nx,nt,nu,dt)
print("The time difference is :", timeit.default_timer() - starttime)
plt.plot(xspace, u_FD)
```

# The convolution - part I

<https://numpy.org/doc/stable/reference/generated/numpy.convolve.html>

<https://en.wikipedia.org/wiki/Convolution>

The discrete convolution operation is known as:

$$
(a * v)[n]= \sum^{\infty}_{m=-\infty} a[m]v[n-m]
$$

Notice, that single step of the explicit algorithm implemented before can be expressed as convolution with a $[\beta,1-2 \beta,\beta]$ filter.
To compute more time steps, one have to convolve many times.

### Task

Solve the diffusion equation by convolving the initial condition with the filter in each iteration.

```python
def calc_diffusion_iterate_convolutions(IC,nx,nt,nu,dt):
    u = IC.copy()
    un = IC.copy() #our placeholder array, un, to advance the solution in time
    beta = nu * dt / dx**2
    filtr = np.array([beta,1-2*beta,beta])
    for n in range(nt):  # iterate through time
        un = u.copy()    # copy the existing values of u into un
        u = # TODO... https://numpy.org/doc/stable/reference/generated/numpy.convolve.html
    return u

u_iter_conv = calc_diffusion_iterate_convolutions(u_IC,nx,nt,nu,dt)
plt.plot(xspace, u_iter_conv)
```

# The convolution - part II

The fundamental solution of the heat equation is the Gaussian function (impulse responce).

Consider "diffusion" of a single particle.
The probability of finding a particle after T time steps follows the normal (a.k.a Gaussian) distribution.
To compute it, one can convolve the initial position with a Gaussian.
The result will be equivalent with repeated convolutions with small filter.

This means that convolving with a Gaussian tells us the solution to the diffusion equation after
a fixed amount of time. This is the same as low pass filtering an image. So smoothing, low pass
filtering, diffusion, all mean the same thing.

### Task

Solve the diffusion equation by convolving the initial condition with the Gaussian.

```python

def calc_diffusion_single_convolution(IC,x,nt,nu,dt):
    u = IC.copy()
    def get_gaussian(x, alfa, t):
        g = -(x-domain_length/2.)**2
        g /=(4*alfa*t)
        g = np.exp(g)
        g /= np.sqrt(4*np.pi*alfa*t)
        g *= domain_length/(nx-1) # normalize --> sum(g)=1
        return g

    time_spot = dt*nt
    fundamental_solution = get_gaussian(x, nu, time_spot)

    u = # TODO... https://numpy.org/doc/stable/reference/generated/numpy.convolve.html
    # plt.plot(x, fundamental_solution, marker='v', linestyle="", markevery=5)
    return u, fundamental_solution

u_single_conv, fs = calc_diffusion_single_convolution(u_IC,xspace,nt,nu,dt)
plt.plot(xspace, u_single_conv)

```

```python
# Now plot the solutions obtained using 3 different approaches on the same plot

plt.rcParams.update({'font.size': 16})
figure, axis = plt.subplots(1, 1, figsize=(10, 8))
plt.subplots_adjust(hspace=1)
axis.set_title('Diffusion')
axis.plot(xspace, u_FD, label=r'$u_{FD}$', linewidth="3")
axis.plot(xspace, u_iter_conv, label=r'$u_{multiple \; convolutions}$',  marker='o', linestyle="", markevery=1)
axis.plot(xspace, u_single_conv, label=r'$u_{convolution}$', marker='x', linestyle="", markevery=1)
axis.set_xlabel('x')
axis.set_ylabel('Concentration')
axis.legend(loc="upper right")
```

# Analytical solution: Advection - Diffusion of a Gaussian Hill

In case of an isotropic diffusion, the analytical solution describing evolution of a Gaussian Hill can be expressed as

$$
C(\boldsymbol{x}, t)=\frac{\left(2\pi\sigma_{0}^{2}\right)^{D/2} }{\left(2\pi(\sigma_{0}^{2} + 2 k t)\right)^{D/2}}
C_0 \exp \left(-\frac{\left(\boldsymbol{x}-\boldsymbol{x}_{0}-\boldsymbol{u} t\right)^{2}}{2\left(\sigma_{0}^{2}+ 2 k t\right)}\right)
$$

where:

* $C_0$ - initial concentration,
* $D$ - number of dimensions,
* $t$ - time,
* $k$ - conductivity,
* $\boldsymbol{u}$ - velocity of advection
* $\sigma_{0}$ the initial variance of the distribution.

## Task

1) Implement the `GaussianHillAnal` class.
It shall have a method `get_concentration_ND(self, X, t)`, which will return the concentration at given time and space.

2) Benchmark the FD code against analytical solution.

```python
from sympy.matrices import Matrix
import numpy as np
import sympy as sp

class GaussianHillAnal:
    def __init__(self, C0, X0, Sigma2_0, k, U, D):
        """
        :param C0: initial concentration
        :param X0: initial position of the hill's centre = Matrix([x0, y0])
        :param U:  velocity = Matrix([ux, uy])
        :param Sigma2_0: initial width of the Gaussian Hill
        :param k: conductivity
        :param dimenions: number of dimensions
        """
        self.C0 = C0
        self.X0 = X0
        self.U = U
        self.Sigma2_0 = Sigma2_0
        self.k = k
        self.dim = D

    def get_concentration_ND(self, X, t):
        decay = 2.*self.k*t
        L = X - self.X0 - self.U*t
        C = self.C0
        C *= pow(2. * np.pi * self.Sigma2_0, self.dim / 2.)
        C /= pow(2. * np.pi * (self.Sigma2_0 + decay), self.dim / 2.)
        C *= sp.exp(-(L.dot(L)) / (2.*(self.Sigma2_0 + decay)))
        return C

```

```python
time_0    = dt*nt/2     # initial contidion for FD
time_spot = dt*nt       # time to be simulated (by FD and analytically)

X0 = Matrix([domain_length/2.]) # center of the hill
C0 = 1.                 # concentration
variance = 30           # initial variance
reference_level = 0

T_0 = np.zeros(nx)
T_anal = np.zeros(nx)

gha = GaussianHillAnal(C0, X0, variance, nu, Matrix([0]), D=1)

for i in range(nx):
    T_0[i] = reference_level + gha.get_concentration_ND(Matrix([xspace[i]]), time_0)
    T_anal[i] = reference_level + gha.get_concentration_ND(Matrix([xspace[i]]), time_spot)

T_FD = calc_diffusion_FD(T_0,nx,nt,nu,dt)
T_single_conv, fs = calc_diffusion_single_convolution(T_0,xspace,nt,nu,dt)

plt.rcParams.update({'font.size': 16})
figure, axis = plt.subplots(1, 1, figsize=(8, 6))
plt.subplots_adjust(hspace=1)
axis.set_title('Diffusion of a Gaussian Hill')
axis.plot(xspace, T_0, label=r'$T_{0}$')
axis.plot(xspace, T_anal, label=r'$T_{anal}$')
axis.plot(xspace, T_FD, label=r'$T_{FD}$', marker='x', linestyle="", markevery=1)
axis.plot(xspace, T_single_conv, label=r'$T_{conv}$', marker='v', linestyle="", markevery=1)
axis.set_xlabel('x')
axis.set_ylabel('Concentration')
axis.legend(loc="upper right")

```

## Questions:

* How do you find the FD solution compared to analytical one? Experiment with different dx, dt.

* How would you asses that your mesh is fine enought in a real CFD simulation (without analytical solution)?

## Answers

* TODO ...

# Learn More

Inspiration

<http://www.cs.umd.edu/~djacobs/CMSC828seg/Diffusion.pdf>

<https://web.math.ucsb.edu/~helena/teaching/math124b/heat.pdf>

You should have completed Steps [1](./01_Step_1.ipynb) and [2](./02_Step_2.ipynb) before continuing. This Jupyter notebook continues the presentation of the **12 steps to Navier–Stokes**, the practical module taught in the interactive CFD class of [Prof. Lorena Barba](http://lorenabarba.com).

For a careful walk-through of the discretization of the diffusion equation with finite differences (and all steps from 1 to 4), watch **Video Lesson 4** by Prof. Barba on YouTube.

```python
from IPython.display import YouTubeVideo
YouTubeVideo('y2WaK7_iMRI')
```


# Job Shop Scheduling

**Implementation Note:** The following cell specifies the solver to used in the subsequent calculations. Some of these problems can become quite larger, and therefore the `gurobi` solver has been set as a default. If you don't have the `gurobi` solver then adjust the code to use the `glpk` solver, but know the calculations may take longer (and the benchmark problem will not solve at all). If you do have the `gurobi` solver, edit the location of the executable to match the location on your computer.

```python
from pyomo.environ import *
from pyomo.gdp import *

#solver = SolverFactory('glpk')
solver = SolverFactory('gurobi', executable='/usr/local/bin/gurobi.sh')
```

## Contents

* [Background](#Background)
* [Job Shop Example](#JobShopExample)
* [Task Decomposition](#TaskDecomposition)
* [Model Formulation](#ModelFormulation)
* [Pyomo Implementation](#PyomoImplementation)
* [Displaying a Solution](#DisplayingSolution)
* [Visualzing Results using Gantt Charts](#Visualization)
* [Appication to Scheduling of Batch Processes](#BatchProcesses)
    * [Single Product Strategies](#SingleProduct)
    * [Overlapping Tasks](#OverlappingTasks)
    * [Unit Cleanout](#UnitCleanout)
    * [Zero-Wait Policy](#ZeroWait)
* [Benchmark Problem LA19](#Benchmark)

## Background

A job shop consists of a set of distinct machines that process jobs. Each job is a series of tasks that require use of particular machines for known durations, and which must be completed in specified order. The job shop scheduling problem is to schedule the jobs on the machines to minimize the time necessary to process all jobs (i.e, the makespan) or some other metric of productivity. Job shop scheduling is one of the classic problems in Operations Research.

Data consists of two tables. The first table is decomposition of the jobs into a series of tasks. Each task lists a job name, name of the required machine, and task duration. The second table list task pairs where the first task must be completed before the second task can be started. This formulation is quite general, but can also specify situations with no feasible solutions.

## Job Shop Example

The following example of a job shop is from from Christelle Gueret, Christian Prins, Marc Sevaux, "Applications of Optimization with Xpress-MP," Dash Optimization, 2000.

In this example, there are three printed paper products that must pass through color printing presses in a particular order. The given data consists of a flowsheet showing the order in which each job passes through the color presses

and a table of data showing, in minutes, the amount of time each job requires on each machine.

| Machine | Color | Paper 1 | Paper 2 | Paper 3 |
| :-----: | :---: | :-----: | :-----: | :-----: |
| 1       | Blue  | 45      | 20      | 12      |
| 2       | Green | -       | 10      | 17      |
| 3       | Yellow| 10      | 34      | 28      |

What is the minimum amount of time (i.e, what is the makespan) for this set of jobs?

## Task Decomposition

The first step in the analysis is to decompose the process into a series of tasks. Each task is a (job,machine) pair. Some tasks cannot start until a prerequisite task is completed.

| Task (Job,Machine) | Duration | Prerequisite Task |
| :----------------: | :------: | :---------------: |
| (Paper 1, Blue)    | 45       | -                 |
| (Paper 1, Yellow)  | 10       | (Paper 1,Blue)    |
| (Paper 2, Blue)    | 20       | (Paper 2, Green)  |
| (Paper 2, Green)   | 10       | -                 |
| (Paper 2, Yellow)  | 34       | (Paper 2, Blue)   |
| (Paper 3, Blue)    | 12       | (Paper 3, Yellow) |
| (Paper 3, Green)   | 17       | (Paper 3, Blue)   |
| (Paper 3, Yellow)  | 28       | -                 |

We convert this to a JSON style representation where tasks are denoted by (Job,Machine) tuples in Python. The task data is stored in a Python dictionary indexed by (Job,Machine) tuples. The task data conists of a dictionary with duration ('dur') and (Job,Machine) pair for any prerequisite task.

```python
TASKS = {
    ('Paper_1','Blue')   : {'dur': 45, 'prec': None},
    ('Paper_1','Yellow') : {'dur': 10, 'prec': ('Paper_1','Blue')},
    ('Paper_2','Blue')   : {'dur': 20, 'prec': ('Paper_2','Green')},
    ('Paper_2','Green')  : {'dur': 10, 'prec': None},
    ('Paper_2','Yellow') : {'dur': 34, 'prec': ('Paper_2','Blue')},
    ('Paper_3','Blue')   : {'dur': 12, 'prec': ('Paper_3','Yellow')},
    ('Paper_3','Green')  : {'dur': 17, 'prec': ('Paper_3','Blue')},
    ('Paper_3','Yellow') : {'dur': 28, 'prec': None},
}
```

## Model Formulation

Each task is represented as an ordered pair $(j,m)$ where $j$ is a job, and $m$ is a machine.

| Parameter | Description |
| :-------- | :-----------|
| $\text{dur}_{j,m}$ | Duration of task $(j,m)$ |
| $\text{prec}_{j,m}$ | A task $(k,n) = \text{Prec}_{j,m}$ that must be completed before task $(j,m)$|

| Decision Variables | Description |
| :-------- | :-----------|
| $\text{makespan}$  | Completion of all jobs |
| $\text{start}_{j,m}$ | Start time for task $(j,m)$ |
| $y_{j,k,m}$ | boolean variable for tasks $(i,m)$ and $(j,m)$ on machine $m$ where $j < k$ |

Upper and lower bounds on the start and completion of task $(j,m)$

\begin{align}
\text{start}_{j,m} & \geq 0\\
\text{start}_{j,m}+\text{Dur}_{j,m} & \leq \text{makespan}
\end{align}

Satisfying prerequisite tasks

\begin{align}
\text{start}_{k,n}+\text{Dur}_{k,n}\leq\text{start}_{j,m}\ \ \ \ \text{for } (k,n) =\text{Prec}_{j,m}
\end{align}

Disjunctive Constraints

If $M$ is big enough, then satisfying
\begin{align}
\text{start}_{j,m}+\text{Dur}_{j,m} & \leq \text{start}_{k,m}+M(1 - y_{j,k,m})\\
\text{start}_{k,m}+\text{Dur}_{k,m} & \leq \text{start}_{j,m}+My_{j,k,m}
\end{align}

avoids conflicts for use of the same machine.

## Pyomo Implementation

The job shop scheduling problem is implemented below in Pyomo. The implementation consists of of a function JobShop(TASKS) that accepts a dictionary of tanks and returns a pandas dataframe containing an optimal schedule of tasks. An optional argument to JobShop allows one to specify a solver.

```python
def JobShop(TASKS):

    model = ConcreteModel()

    model.TASKS    = Set(initialize=TASKS.keys(), dimen=2)
    model.JOBS     = Set(initialize=set([j for (j,m) in TASKS.keys()]))
    model.MACHINES = Set(initialize=set([m for (j,m) in TASKS.keys()]))
    model.TASKORDER = Set(initialize = model.TASKS * model.TASKS, dimen=4,
        filter = lambda model,j,m,k,n: (k,n) == TASKS[(j,m)]['prec'])
    model.DISJUNCTIONS = Set(initialize=model.JOBS * model.JOBS * model.MACHINES, dimen=3,
        filter = lambda model,j,k,m: j < k and (j,m) in model.TASKS and (k,m) in model.TASKS)

    t_max = sum([TASKS[(j,m)]['dur'] for (j,m) in TASKS.keys()])

    model.makespan = Var(bounds=(0, t_max))
    model.start = Var(model.TASKS, bounds=(0, t_max))

    model.obj = Objective(expr = model.makespan, sense = minimize)

    model.fini = Constraint(model.TASKS, rule=lambda model,j,m:
        model.start[j,m] + TASKS[(j,m)]['dur'] <= model.makespan)

    model.prec = Constraint(model.TASKORDER, rule=lambda model,j,m,k,n:
        model.start[k,n] + TASKS[(k,n)]['dur'] <= model.start[j,m])

    model.disj = Disjunction(model.DISJUNCTIONS, rule=lambda model,j,k,m:
        [model.start[j,m] + TASKS[(j,m)]['dur'] <= model.start[k,m],
         model.start[k,m] + TASKS[(k,m)]['dur'] <= model.start[j,m]])

    TransformationFactory('gdp.chull').apply_to(model)
    solver.solve(model, tee=True)

    results = [{'Job': j,
                'Machine': m,
                'Start': model.start[j, m](),
                'Duration': TASKS[(j, m)]['dur'],
                'Finish': model.start[j, m]() + TASKS[(j, m)]['dur']}
               for j,m in model.TASKS]
    return results

results = JobShop(TASKS)
results
```

    Academic license - for non-commercial use only
    Optimize a model with 77 rows, 52 columns and 181 nonzeros
    Variable types: 38 continuous, 14 integer (14 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 2e+02]
      Objective range  [1e+00, 1e+00]
      Bounds range     [1e+00, 2e+02]
      RHS range        [1e+00, 4e+01]
    Found heuristic solution: objective 149.0000000
    Presolve removed 25 rows and 24 columns
    Presolve time: 0.00s
    Presolved: 52 rows, 28 columns, 144 nonzeros
    Variable types: 21 continuous, 7 integer (7 binary)

    Root relaxation: objective 8.500000e+01, 15 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0   85.00000    0    4  149.00000   85.00000  43.0%     -    0s
    H    0     0                      97.0000000   85.00000  12.4%     -    0s
         0     0     cutoff    0        97.00000   97.00000  0.00%     -    0s

    Cutting planes:
      Implied bound: 2

    Explored 1 nodes (30 simplex iterations) in 0.01 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 2: 97 149

    Optimal solution found (tolerance 1.00e-04)
    Best objective 9.700000000000e+01, best bound 9.700000000000e+01, gap 0.0000%
    Freed default Gurobi environment

    [{'Job': 'Paper_3',
      'Machine': 'Yellow',
      'Start': 0.0,
      'Duration': 28,
      'Finish': 28.0},
     {'Job': 'Paper_2',
      'Machine': 'Yellow',
      'Start': 30.0,
      'Duration': 34,
      'Finish': 64.0},
     {'Job': 'Paper_3',
      'Machine': 'Green',
      'Start': 42.0,
      'Duration': 17,
      'Finish': 59.0},
     {'Job': 'Paper_2',
      'Machine': 'Green',
      'Start': 0.0,
      'Duration': 10,
      'Finish': 10.0},
     {'Job': 'Paper_1',
      'Machine': 'Yellow',
      'Start': 87.0,
      'Duration': 10,
      'Finish': 97.0},
     {'Job': 'Paper_2',
      'Machine': 'Blue',
      'Start': 10.0,
      'Duration': 20,
      'Finish': 30.0},
     {'Job': 'Paper_3',
      'Machine': 'Blue',
      'Start': 30.0,
      'Duration': 12,
      'Finish': 42.0},
     {'Job': 'Paper_1',
      'Machine': 'Blue',
      'Start': 42.0,
      'Duration': 45,
      'Finish': 87.0}]

## Printing Schedules

```python
import pandas as pd

schedule = pd.DataFrame(results)

print('\nSchedule by Job')
print(schedule.sort_values(by=['Job','Start']).set_index(['Job', 'Machine']))

print('\nSchedule by Machine')
print(schedule.sort_values(by=['Machine','Start']).set_index(['Machine', 'Job']))
```

    Schedule by Job
                     Duration  Finish  Start
    Job     Machine
    Paper_1 Blue           45    87.0   42.0
            Yellow         10    97.0   87.0
    Paper_2 Green          10    10.0    0.0
            Blue           20    30.0   10.0
            Yellow         34    64.0   30.0
    Paper_3 Yellow         28    28.0    0.0
            Blue           12    42.0   30.0
            Green          17    59.0   42.0

    Schedule by Machine
                     Duration  Finish  Start
    Machine Job
    Blue    Paper_2        20    30.0   10.0
            Paper_3        12    42.0   30.0
            Paper_1        45    87.0   42.0
    Green   Paper_2        10    10.0    0.0
            Paper_3        17    59.0   42.0
    Yellow  Paper_3        28    28.0    0.0
            Paper_2        34    64.0   30.0
            Paper_1        10    97.0   87.0

## Visualizing Results with Gantt Charts

```python
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib as mpl
import pandas as pd

def Visualize(results):

    schedule = pd.DataFrame(results)
    JOBS = list(schedule['Job'].unique())
    MACHINES = list(schedule['Machine'].unique())
    makespan = schedule['Finish'].max()

    schedule.sort_values(by=['Job','Start'])
    schedule.set_index(['Job', 'Machine'], inplace=True)

    plt.figure(figsize=(12, 5 + (len(JOBS)+len(MACHINES))/4))
    plt.subplot(2,1,1)

    jdx = 0
    for j in sorted(JOBS):
        jdx += 1
        mdx = 0
        for m in MACHINES:
            mdx += 1
            c = mpl.cm.Dark2.colors[mdx%7]
            if (j,m) in schedule.index:
                plt.plot([schedule.loc[(j,m),'Start'],schedule.loc[(j,m),'Finish']],
                     [jdx,jdx],color = c,alpha=1.0,lw=25,solid_capstyle='butt')
                plt.text((schedule.loc[(j,m),'Start'] + schedule.loc[(j,m),'Finish'])/2.0,jdx,
                    m, color='white', weight='bold',
                    horizontalalignment='center', verticalalignment='center')

    plt.ylim(0.5,jdx+0.5)
    plt.title('Job Schedule')
    plt.gca().set_yticks(range(1,1+len(JOBS)))
    plt.gca().set_yticklabels(sorted(JOBS))
    plt.plot([makespan,makespan],plt.ylim(),'r--')
    plt.text(makespan,plt.ylim()[0]-0.2,str(round(makespan,2)),
            horizontalalignment='center', verticalalignment='top')
    plt.xlabel('Time')
    plt.ylabel('Jobs')
    plt.grid()

    plt.subplot(2,1,2)
    mdx = 0
    for m in sorted(MACHINES):
        mdx += 1
        jdx = 0
        for j in JOBS:
            jdx += 1
            c = mpl.cm.Dark2.colors[jdx%7]
            if (j,m) in schedule.index:
                plt.plot([schedule.loc[(j,m),'Start'],schedule.loc[(j,m),'Finish']],
                     [mdx,mdx],color = c,alpha=1.0,lw=25,solid_capstyle='butt')
                plt.text((schedule.loc[(j,m),'Start'] + schedule.loc[(j,m),'Finish'])/2.0,mdx,
                    j, color='white', weight='bold',
                    horizontalalignment='center', verticalalignment='center')

    plt.ylim(0.5,mdx+0.5)
    plt.title('Machine Schedule')
    plt.gca().set_yticks(range(1,1+len(MACHINES)))
    plt.gca().set_yticklabels(sorted(MACHINES))
    plt.plot([makespan,makespan],plt.ylim(),'r--')
    plt.text(makespan,plt.ylim()[0]-0.2,str(round(makespan,2)),
            horizontalalignment='center', verticalalignment='top')
    plt.xlabel('Time')
    plt.ylabel('Machines')
    plt.grid()

    plt.tight_layout()

Visualize(results)
```

## Application to Scheduling of Batch Processes

We will now turn our attention to the application of the job shop scheduling problem to the short term scheduling of batch processes. We illustrate these techniques using an example from Dunn (2013).

| Process | Mixer | Reactor | Separator | Packaging |
| :-----: | :---: | :-----: | :-------: | :-------: |
| A       | 1.0   | 5.0     | 4.0       | 1.5       |
| B       | -     | -       | 4.5       | 1.0       |
| C       | -     | 3.0     | 5.0       | 1.5       |

## Single Product Strategies

Before going further, we create a function to streamline the generation of the TASKS dictionary.

```python
def Recipe(jobs,machines,durations):
    TASKS = {}
    for j in jobs:
        prec = (None,None)
        for m,d in zip(machines,durations):
            task = (j,m)
            if prec == (None,None):
                TASKS.update({(j,m): {'dur': d, 'prec': None}})
            else:
                TASKS.update({(j,m): {'dur': d, 'prec': prec}})
            prec = task
    return TASKS

RecipeA = Recipe('A',['Mixer','Reactor','Separator','Packaging'],[1,5,4,1.5])
RecipeB = Recipe('B',['Separator','Packaging'],[4.5,1])
RecipeC = Recipe('C',['Separator','Reactor','Packaging'],[5,3,1.5])

Visualize(JobShop(RecipeA))
```

```python
Visualize(JobShop(RecipeB))
```

```python
Visualize(JobShop(RecipeC))
```

## Overlapping Tasks

Let's now consider an optimal scheduling problem where we are wish to make two batches of Product A.

```python
TASKS = Recipe(['A1','A2'],['Mixer','Reactor','Separator','Packaging'],[1,5,4,1.5])
results = JobShop(TASKS)
Visualize(results)
print("Makespan =", max([task['Finish'] for task in results]))
```

Earlier we found it tood 11.5 hours to produce one batch of product A.  As we see here, we can produce a second batch with only 5.0 additional hours because some of the tasks overlap.  The overlapping of tasks is the key to gaining efficiency in batch processing facilities.

Let's next consider production of a single batch each of products A, B, and C.

```python
TASKS = RecipeA
TASKS.update(RecipeB)
TASKS.update(RecipeC)

results = JobShop(TASKS)
Visualize(results)
print("Makespan =", max([task['Finish'] for task in results]))
```

The individual production of A, B, and C required 11.5, 5.5, and 9.5 hours, respectively, for a total of 25.5 hours. As we see here, by scheduling the production simultaneously, we can get all three batches done in just 15 hours.

As we see below, each additional set of three products takes an additionl 13 hours.  So there is considerable efficiency gained by scheduling over longer intervals whenever possible.

```python
TASKS = Recipe(['A1','A2'],['Mixer','Reactor','Separator','Packaging'],[1,5,4,1.5])
TASKS.update(Recipe(['B1','B2'],['Separator','Packaging'],[4.5,1]))
TASKS.update(Recipe(['C1','C2'],['Separator','Reactor','Packaging'],[5,3,1.5]))

results = JobShop(TASKS)
Visualize(results)
print("Makespan =", max([task['Finish'] for task in results]))
```

## Unit Cleanout

A common feature in batch unit operations is a requirement that equipment be cleaned prior to reuse.

In most cases the time needed for clean out would be equipment and product specific. Bur for the purposes of illustration, we implement this policy with a single non-negative parameter $t_{clean} \geq 0$ which, if specified, requires a period no less than $t_{clean}$ between the finish of one task and the start of another on every piece of equipment.

This is implemented by modifying the usual disjunctive constraints to avoid machine conflicts, i.e.,

\begin{align}
\text{start}_{j,m}+\text{Dur}_{j,m} & \leq \text{start}_{k,m}+M(1 - y_{j,k,m})\\
\text{start}_{k,m}+\text{Dur}_{k,m} & \leq \text{start}_{j,m}+My_{j,k,m}
\end{align}

to read

\begin{align}
\text{start}_{j,m}+\text{Dur}_{j,m} + t_{clean} & \leq \text{start}_{k,m}+M(1 - y_{j,k,m})\\
\text{start}_{k,m}+\text{Dur}_{k,m} + t_{clean} & \leq \text{start}_{j,m}+My_{j,k,m}
\end{align}

for sufficiently large $M$.

```python
def JobShop(TASKS, tclean=0):

    model = ConcreteModel()

    model.TASKS    = Set(initialize=TASKS.keys(), dimen=2)
    model.JOBS     = Set(initialize=set([j for (j,m) in TASKS.keys()]))
    model.MACHINES = Set(initialize=set([m for (j,m) in TASKS.keys()]))
    model.TASKORDER = Set(initialize = model.TASKS * model.TASKS, dimen=4,
        filter = lambda model,j,m,k,n: (k,n) == TASKS[(j,m)]['prec'])
    model.DISJUNCTIONS = Set(initialize=model.JOBS * model.JOBS * model.MACHINES, dimen=3,
        filter = lambda model,j,k,m: j < k and (j,m) in model.TASKS and (k,m) in model.TASKS)

    t_max = sum([TASKS[(j,m)]['dur'] for (j,m) in TASKS.keys()])

    model.makespan = Var(bounds=(0, t_max))
    model.start = Var(model.TASKS, bounds=(0, t_max))

    model.obj = Objective(expr = model.makespan, sense = minimize)

    model.fini = Constraint(model.TASKS, rule=lambda model,j,m:
        model.start[j,m] + TASKS[(j,m)]['dur'] <= model.makespan)

    model.prec = Constraint(model.TASKORDER, rule=lambda model,j,m,k,n:
        model.start[k,n] + TASKS[(k,n)]['dur'] <= model.start[j,m])

    model.disj = Disjunction(model.DISJUNCTIONS, rule=lambda model,j,k,m:
        [model.start[j,m] + TASKS[(j,m)]['dur'] + tclean <= model.start[k,m],
         model.start[k,m] + TASKS[(k,m)]['dur'] + tclean <= model.start[j,m]])

    TransformationFactory('gdp.chull').apply_to(model)
    solver.solve(model, tee=True)

    results = [{'Job': j,
                'Machine': m,
                'Start': model.start[j, m](),
                'Duration': TASKS[(j, m)]['dur'],
                'Finish': model.start[j, m]() + TASKS[(j, m)]['dur']}
               for j,m in model.TASKS]
    return results

results = JobShop(TASKS, tclean=0.5)
Visualize(results)
print("Makespan =", max([task['Finish'] for task in results]))
```

## Zero Wait Policy

One of the issues in the use of job shop scheduling for batch processing are situations where there it isn't possible to store intermediate materials. If there is no way to store intermediates, either in the processing equipment or in external vessels, then a **zero-wait** policy may be appropriate.

A zero-wait policy requires subsequent processing machines to be available immediately upon completion of any task. To implement this policy, the usual precident sequencing constraint of a job shop scheduling problem, i.e.,

\begin{align*}
\text{start}_{k,n}+\text{Dur}_{k,n} \leq \text{start}_{j,m}\ \ \ \ \text{for } (k,n) =\text{Prec}_{j,m}
\end{align*}

is changed to

\begin{align*}
\text{start}_{k,n}+\text{Dur}_{k,n} = \text{start}_{j,m}\ \ \ \ \text{for } (k,n) =\text{Prec}_{j,m}\text{ and ZW is True}
\end{align*}

if the zero-wait policy is in effect.

While this could be implemented on an equipment or product specific basis, here we add an optional ZW flag to the JobShop function that, by default, is set to False.

```python
def JobShop(TASKS, tclean=0, ZW=False):

    model = ConcreteModel()

    model.TASKS    = Set(initialize=TASKS.keys(), dimen=2)
    model.JOBS     = Set(initialize=set([j for (j,m) in TASKS.keys()]))
    model.MACHINES = Set(initialize=set([m for (j,m) in TASKS.keys()]))
    model.TASKORDER = Set(initialize = model.TASKS * model.TASKS, dimen=4,
        filter = lambda model,j,m,k,n: (k,n) == TASKS[(j,m)]['prec'])
    model.DISJUNCTIONS = Set(initialize=model.JOBS * model.JOBS * model.MACHINES, dimen=3,
        filter = lambda model,j,k,m: j < k and (j,m) in model.TASKS and (k,m) in model.TASKS)

    t_max = sum([TASKS[(j,m)]['dur'] for (j,m) in TASKS.keys()])

    model.makespan = Var(bounds=(0, t_max))
    model.start = Var(model.TASKS, bounds=(0, t_max))

    model.obj = Objective(expr = model.makespan, sense = minimize)

    model.fini = Constraint(model.TASKS, rule=lambda model,j,m:
        model.start[j,m] + TASKS[(j,m)]['dur'] <= model.makespan)

    if ZW:
        model.prec = Constraint(model.TASKORDER, rule=lambda model,j,m,k,n:
            model.start[k,n] + TASKS[(k,n)]['dur'] == model.start[j,m])
    else:
        model.prec = Constraint(model.TASKORDER, rule=lambda model,j,m,k,n:
            model.start[k,n] + TASKS[(k,n)]['dur'] <= model.start[j,m])

    model.disj = Disjunction(model.DISJUNCTIONS, rule=lambda model,j,k,m:
        [model.start[j,m] + TASKS[(j,m)]['dur'] + tclean <= model.start[k,m],
         model.start[k,m] + TASKS[(k,m)]['dur'] + tclean <= model.start[j,m]])

    TransformationFactory('gdp.chull').apply_to(model)
    solver.solve(model, tee=True)

    results = [{'Job': j,
                'Machine': m,
                'Start': model.start[j, m](),
                'Duration': TASKS[(j, m)]['dur'],
                'Finish': model.start[j, m]() + TASKS[(j, m)]['dur']}
               for j,m in model.TASKS]
    return results

results = JobShop(TASKS, tclean=0.5, ZW=True)
Visualize(results)
print("Makespan =", max([task['Finish'] for task in results]))
```

## Benchmark Problems

The file `jobshop1.txt` (available [here](http://people.brunel.ac.uk/~mastjjb/jeb/orlib/files/jobshop1.txt)) is a well known collection of 82 benchmark problems for job shop scheduling. The data format for each example consists of a single line for each job.  The data on each line is a sequence of (machine number, time) pairs showing the order in which machines process each job.

LA19 is a benchmark problem for job shop scheduling introduced by Lawrence in 1984, and a solution presented by Cook and Applegate in 1991. The following cell may take many minutes to hours to run, depending on the choice of solver and hardware.

```python
data = """
2  44  3   5  5  58  4  97  0   9  7  84  8  77  9  96  1  58  6  89
4  15  7  31  1  87  8  57  0  77  3  85  2  81  5  39  9  73  6  21
9  82  6  22  4  10  3  70  1  49  0  40  8  34  2  48  7  80  5  71
1  91  2  17  7  62  5  75  8  47  4  11  3   7  6  72  9  35  0  55
6  71  1  90  3  75  0  64  2  94  8  15  4  12  7  67  9  20  5  50
7  70  5  93  8  77  2  29  4  58  6  93  3  68  1  57  9   7  0  52
6  87  1  63  4  26  5   6  2  82  3  27  7  56  8  48  9  36  0  95
0  36  5  15  8  41  9  78  3  76  6  84  4  30  7  76  2  36  1   8
5  88  2  81  3  13  6  82  4  54  7  13  8  29  9  40  1  78  0  75
9  88  4  54  6  64  7  32  0  52  2   6  8  54  5  82  3   6  1  26
"""

TASKS = {}
prec = ''

lines = data.splitlines()
job= 0
for line in lines[1:]:
    j = "J{0:1d}".format(job)
    nums = line.split()
    prec = ''
    for m,dur in zip(nums[::2],nums[1::2]):
        task = (j,'M{0:s}'.format(m))
        if prec:
            TASKS[task] = {'dur':int(dur), 'prec':prec}
        else:
            TASKS[task] = {'dur':int(dur), 'prec':None}
        prec = task
    job += 1

Visualize(JobShop(TASKS))
```

### Recalculate Benchmark Problem with a Zero-Wait Policy

The following calculation is quite intensive and will take several minutes to finish with the `gurobi` solver.

```python
Visualize(JobShop(TASKS, ZW=True))
```

```python

```


## **Normal Distribution**

 - The mean, mode, and median of the distribution are equal
 - We only need to use the mean and standard deviation to explain the entire distribution
 - Unimodel distribution, only one peak
 - The mean, mode and median are all equal
 - The curve is symmetric at the center (i.e. around the mean, μ), Exactly half of the values are to the left of center and exactly half the values are to the right.
 - The total area under the curve is 1.

> Standard Normal Distribution also know as z-Distribution:
 - It has Mean = 0, Variance = 1 denoted by N(0,1)

**Empirical rule or 68–95–99.7 rule**

 - For an approximately normal data set, the values within one standard deviation of the mean account for about 68% of the  data; while within two standard deviations account for about 95%; and within three standard deviations account for about 99.7%.

**Chebyshev's Theorem /Inequality**
- The Theorem applies to any distribution, even if that distribution isn’t normal. This formula is useful to get results for standard deviations more than 1.
- Let X be a continuous random variable of a arbitrary probability distribution with finite expected value(mean) μ and finite non-zero variance $σ^2$. Then for positive integer k > 0, at least $(1−\frac{1}{k^2})$ of the data lie within k standard deviations of the mean.
\begin{align} Pr(\mu-k\sigma <= {X} <= \mu+k\sigma) >= 1-\frac{1}{k^2} \end{align}

**Markov's Inequality**
 - If X is a nonnegative random variable and $a > 0$, then the probability that X is at least a is at most the expectation of X divided by a
\begin{align} Pr(a\mu <= X) <= \frac{1}{a} \end{align}

### Normal distribution approximation (central limit theorem)
 - The central limit theorem states that if you have a population with mean $\mu$ and standard deviation $\sigma$ and take sufficiently large random samples(>= 30) from the population with replacement, then the distribution of the sample means will be approximately normally distributed i.e. mean of the sample means equals the mean of the original population and its variance is population variance divided by sample size. In simple words if we take more samples, especially large ones, the graph of the sample means will look more like a normal distribution.
 - Traditionally, we would gather samples to represent the random variable. Each sample has its own mean. Now if we start repeating the experiment and start gathering more samples and start calculating the mean of each of the sample then the samples mean will have its own probability distribution and this distribution will converge towards the normal distribution as we start increasing the number of samples.

 - If $ X_{1},X_{2},\ldots ,X_{n}$ are n independent and identically distributed random variables with same arbitrary distribution of mean $\mu$, and finite variance $\sigma^{2}$
  - sampling distribution mean ${\bar {X}}={\frac {1}{n}}\sum_{i=1}^{n}X_{i} \sim\text{N}(\mu,\sigma^2/n)$
  - i.e. distribution of sample mean $\bar{X}$ converges on a normal distribution,
  - mean of $\bar{X} = \mu$ and
  - variance of $\bar{X} = \sigma^{2}/n$, or standard error, s.e. = $\sigma/\sqrt {n}$
- the distribution of z-value of sample mean, $Z={\frac {{\overline {X}}-\mu }{\sigma/{\sqrt {n}}}} \sim\text{N}(0,1)$ converges on a standard normal distribution (mean = 0, variance = 1)
- sampling distribution variance ${S^{2}={\frac {1}{n-1}}\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}$
- the distribution of t-value, $t={\frac {{\overline {X}}-\mu }{S/{\sqrt {n}}}} \sim{t}_{n-1}$ does not follow normal curve (CLT) instead it follows a Student's t-distribution with $n-1$ degrees of freedom

### Student-t Distribution of sample mean

> **One Sample t-test**: Let $x_1, \cdots, x_n$ be the numbers observed in a sample from a continuously distributed population with expected value $\mu$. The sample mean and sample variance are given by:

\begin{aligned}
\bar {x}&={\frac {x_{1}+\cdots +x_{n}}{n}}
\\s^{2}&=\frac {1}{n-1}\sum _{i=1}^{n}(x_{i}-{\bar{x}})^{2}
\\t &= \frac{\bar{x} - \mu}{s/\sqrt{n}} \sim t_{n-1}
\end{aligned}

> This resulting t-value follows t-distribution with n-1 degrees of freedom when samples consist of independent identically distributed observations from a normally distributed population.

  \begin{aligned}
  \\t &= \frac{\bar{X} - \mu}{s/\sqrt{n}} =
  \frac{\frac{\bar{X} - \mu}{\sigma/\sqrt{n}}} {\sqrt{\frac{\frac {1}{n-1}\sum _{i=1}^{n}(X_{i}-{\bar{X}})^{2}}{\sigma^{2}}}} \sim t_{n-1}
  \end{aligned}

> If X follows a normal distribution with mean $\mu$ and variance $\sigma^{2}$:
  - The numerator follows standard normal distribution ∼ N(μ=0, σ=1)
  - The denominator follow scaled chi-squared $\frac{1}{n−1}\,χ^2_{n−1}$
  - Numerator and denominator should be independent
  Under these conditons the  t-statistic ∼ t(df=n−1)

## Normality test - Graphical methods

Normal Distribution

Uniform Distribution (Non - Normal)

Right Skewed Distribution (Non - Normal)

* In normal distribution, the points on a normal QQ Plot follow a straight line, whereas other distributions deviate strongly.
* The uniform distribution has too many observations in both extremities (very high and very low values).
* The exponential distribution has too many observations on the lower values, but too little in the higher values.

### 1. **Histogram**
 - The histogram is a great way to quickly visualize the distribution of a single variable.
 - Histogram of the sample data should be approximately a bell-shaped to resemble the data is normal distributed
 - Histogram of a sample from a normal distribution – it looks fairly symmetric, unimodal and bell shaped

```
import pandas as pd
from scipy.stats import norm
import matplotlib.pyplot as plt
import numpy as np
#my_data = pd.Series([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 8, 6, 4, 2])
#my_data.hist()
my_data = norm.rvs(loc=5, scale=3, size=10)
plt.hist(my_data)
plt.show()
```

```
import numpy as np
from scipy.stats import norm
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
n=300 # Sample size
data=np.random.uniform(low=1,high=10,size=n) # Sample
#data=np.random.normal(loc=0,scale=10,size=n)
bin_value=int(sqrt(len(data))) #choosing the number of bins to draw histogram
sns.distplot(data,bins=bin_value);
plt.xlabel("Sample Data",size=15);
plt.ylabel("Density",size=15);
plt.title("Histogram",size=20);
plt.show()
```

### 2. **Quantile-Quantile plot or QQ plot**
 - Null Hypopthesis H0: Sample data comes from Normal distribution
 - Plots of quantiles from two data set (sample and normal) against each other,
 - If the two data sets come from a common distribution, the points should approximately lie on a 45 degree angle (diagonal line y = x)
 - If the line is skewed to the left or right, it means that you do not have normally distributed data.
 - Standardized and sorted data set is plotted against the the expected z-score values of the corresponding quantiles from standard normal distribution
 - Steps:
    1. Order the items from smallest to largest.
    2. Draw a normal distribution curve. Divide the curve into n+1 segments. We have 9 values, so divide the curve into 10 equally-sized areas. For this example, each segment is 10% of the area (because 100% / 10 = 10%).

    3. Find the z-value (cut-off point) for each segment in Step 3. These segments are areas, so refer to a z-table (or use software) to get a z-value for each segment.
> The z-values are: 10% = -1.28, 20% = -0.84, 30% = -0.52, 40% = -0.25, 50% = 0, 60% = 0.25, 70% = 0.52, 80% = 0.84, 90% = 1.28, 100% = 3.0
    4. Plot your data set values (Step 1) against your normal distribution cut-off points (Step 3).

A Q–Q plot is used to compare the shapes of distributions, providing a graphical view of how properties such as location, scale, and skewness are similar or different in the two distributions. The use of Q–Q plots to compare two samples of data can be viewed as a non-parametric approach to comparing their underlying distributions. A Q–Q plot is generally a more powerful approach to do this than the common technique of comparing histograms of the two samples. Q–Q plots are also commonly used to compare a data set to a theoretical model by plotting their quantiles against each other. For normality test we compare our ordered sample data with the quantiles obtained from the standard normal distribution. Thus, Q–Q plot help us to identify substantive departures from normality. The resulting image look close to a straight line if the data are approximately normally distributed. Deviations from a straight line suggest departures from normality. The main step in constructing a Q–Q plot is calculating or estimating the quantiles to be plotted. To know how the quantiles are generated in scipy.stats.probplot() see this notebook.

Note: The normal probability plot is a special case of the Q–Q probability plot for a normal distribution.

```
# QQ Plot Implementation
import statsmodels.api as sm
from scipy.stats import norm
import pylab
import numpy as np
my_data = norm.rvs(size=1000)
# Create the random variables with mean 5, and sd 3
# my_data = stats.norm.rvs(loc=5, scale=3, size=10)
# sm.qqplot(my_data, loc = 4, scale = 3, line='s')
sm.qqplot(my_data, line='45')
pylab.show()
```

### 3. **Normal probability plot (Normal plot)**
 - Null Hypopthesis H0: Sample data comes from Normal distribution
 - The normal probability plot is a special case of QQ probability plot for a normal distribution
 - Standardized and sorted data set is plotted against the the zscore values
 - If the null hypothesis is true, the plotted points should approximately lie on a straight line.
 - If the line is skewed to the left or right, it means that you do not have normally distributed data.
 - Steps:
    1. Arrange your x-values in ascending order.
    2. Calculate $f_{i} = (i-0.375)/(n+0.25)$, where i is the position of the data value in the ordered list and n is the number of observations.
    3. Find the z-score for each $f_{i}$
    4. Plot your x-values on the horizontal axis and the corresponding z-score on the vertical axis.

This method is not preferred if the sample size is small.
If the sample size is sufficiently large most statistical significance tests may detect even trivial departures from the null hypothesis (i.e., although there may be some statistically significant effect, it may be too small to be of any practical significance); thus, additional investigation of the effect size is typically advisable, like a Q–Q plot. So, if we have sufficiently large sample, we can opt this method instead of statistical significance tests. For more information check this link and this link.
As it needs the visual inspection, if we don't have enough sample, it is trickier to make conclusions.

```
# Normal Plot Implementation
import statsmodels.api as sm
from scipy.stats import probplot
import pylab
import numpy as np
my_data = norm.rvs(size=1000)
# Create the random variables with mean 5, and sd 3
#my_data = stats.norm.rvs(loc=5, scale=3, size=10)
probplot(my_data, dist="norm", plot=plt)
pylab.show()
```

### 4. **Boxplot**
 - If data comes from a normal distribution, the box will be symmetrical with the mean and median in the center.
 - If the data meets the assumption of normality, there should also be few outliers.

Normal (left), Uniform (middle) and exponential (right) boxplots

```
# Boxplot Implementation
import pandas as pd
my_data = pd.Series([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 8, 6, 4, 2])
my_data.plot(kind = 'box')
plt.show()

```

## Frequentist tests for Normality (goodness-of-fit measure of departure from normality)

### 1. Jarque–Bera test
  - This test only works for a large enough number of data samples (>2000)
  - Null hypothesis is a joint hypothesis of the skewness being zero and the kurtosis being zero for the distribution to be normal.
    - Skew is a quantification of how much a distribution is pushed left or right, a measure of asymmetry in the distribution.
    - Kurtosis is a measure of whether the data are heavy-tailed or light-tailed relative to a normal distribution.
  - The test statistic is always nonnegative. If it is far from zero, indicates data do not have a normal distribution. For a sample of size $n$
    - s.e. of skewness is roughly $\sqrt{6/n}$
    - s.e of kurtosis is roughly $\sqrt{24/n}$
    - JB statistic follows chi-squared distribution with 2 degree of freedom, $JB = n (\frac{\text(skew)^2}{6}+\frac{\text(kurt)^2}{24}) \sim \chi^2(2)$

**Interpretation**

 * p-value ≤ 0.05, then we reject the null hypothesis i.e. we assume the distribution is not normal/gaussian.
 * p-value > 0.05, then we fail to reject the null hypothesis i.e. we assume the distribution is normal/gaussian.

```
from scipy.stats import jarque_bera, norm
import numpy as np
my_data = norm.rvs(size=1000)
#my_data = np.random.normal(loc=20, scale=5, size=150)
stat, p_value = jarque_bera(my_data)
print(stat, p_value)
alpha = 0.05
if p_value > alpha:
	print('Data appears Gaussian/Normal (fail to reject H0)')
else:
	print('Data does not appear Gaussian/Normal (reject H0)')
```

    1.2006239293609624 0.5486404529502638
    Data appears Gaussian/Normal (fail to reject H0)

### 2. D'Agostino's Pearson K-squared test,
  - Uses the fact that if the dataset is a normal distribution, skewness coefficient & kurtosis coefficient should be zero.
  - It is more accurate than the Jarque-Barre test.
  - Skewness test determines whether the skewness of the sample data is statistically different from zero, $z_{s} = skew/s.e.$
  - Kurtosis test determines whether the kurtosis of the sample data is statistically different from zero, $z_{k} = kurt/s.e.$
  - When the data is normally distributed the D’Agostino-Pearson Omnibus test statistic has a chi-square distribution with 2 degrees of freedom
    \begin{align}\text{DP-statistic} = z^2_k+z^2_s \sim \chi^{2}(2)\end{align}
  - This test should generally not be used for data sets with less than 20 elements

**Interpretation**

* p-value ≤ 0.05, then we reject the null hypothesis i.e. we assume the distribution of our variable is not normal/gaussian.
* p-value > 0.05, then we fail to reject the null hypothesis i.e. we assume the distribution of our variable is normal/gaussian.

```
from scipy.stats import normaltest, norm
import numpy as np
my_data = norm.rvs(size=1000)
#my_data = np.random.normal(loc=20, scale=5, size=150)
stat, p_value = normaltest(my_data)
print(stat, p_value)
alpha = 0.05
if p_value > alpha:
	print('Data appears Gaussian/Normal (fail to reject H0)')
else:
	print('Data does not appear Gaussian/Normal (reject H0)')
```

    0.9260787139035734 0.6293678620201191
    Data appears Gaussian/Normal (fail to reject H0)

### 3. Shapiro–Wilk test (S-W test)
 - Null hypothesis H0 is population is normally distributed and the mean and variance do not need to be specified in advance.
    - H0 : X ∼ N(μ, σ)
    - H1 : X ≁ N(μ, σ)
 - The Shapiro-Wilk test is for testing normality of sample from a normal distribution with unknown mean μ and unknown SD σ.
 - Shapiro Wilk test is the most powerful test for testing for a normal distribution when comparing with Anderson–Darling , Kolmogorov–Smirnov, Lilliefors tests. It cannot be used for testing against other distributions.
 - The test has a bias by sample size. The larger the sample, the more likely you’ll get a statistically significant result.
 - For Shapiro–Wilk test samples of at least 20 elements are recommended and is known not to work well in samples with many identical values.
 - Steps
    1. Rearrange the data in ascending order so that $x_{1} ≤ x_{2} … ≤ x_{n}$
    2. Calculate SS as follows:
      - $SS = \sum_{i=1}^{n}(x_{i}-{\bar{x}})^{2}$
    3. If n is even, let m =  n/2, while if n is odd let m = (n–1)/2
    4. Calculate b as follows, taking the $a_{i}$ weights from Shapiro-Wilk Table (based on the value of n). Note that if n is odd, the median data value is not used in the calculation of b.
      \begin{align}b = \sum_{i=1}^{m}a_{i}(x_{n+1-i}-x_{i})\end{align} where $x_{i}$ are the ordered random sample values, $a_{i}$ are constants generated from the covariances, variances and means of the sample (size n) from a normally distributed sample
    5. Calculate the test statistic $W = b^2 ⁄ SS$, small values indicates sample is not normally distributed
    6. Check the p-value for the value closest to W-statistic from Shapiro-Wilk Table

**Interpretation**

 * p-value ≤ 0.05, then we reject the null hypothesis i.e. we assume the distribution is not normal/gaussian.
 * p-value > 0.05, then we fail to reject the null hypothesis i.e. we assume the distribution is normal/gaussian.

```
from scipy.stats import shapiro, norm
import numpy as np
my_data = norm.rvs(size=1000)
#my_data = np.random.normal(loc=20, scale=5, size=150)
stat, p_value = shapiro(my_data)
print(stat, p_value)
alpha = 0.05
if p_value > alpha:
	print('Data appears Gaussian/Normal (fail to reject H0)')
else:
	print('Data does not appear Gaussian/Normal (reject H0)')
```

    0.9990300536155701 0.8881983160972595
    Data appears Gaussian/Normal (fail to reject H0)

### 4. Kolmogorov–Smirnov test (K-S test)
 - Null hypothesis (H0) assumes the data comes from the specified hypothetical distribution.
    - H0 : X ∼ N(μ, σ) with specified μ, σ
    - H1 : X ≁ N(μ, σ)
 - The Kolmogorov-Smirnov (K-S) test is a non-parametric test based on the empirical distribution function (ECDF)
 - It can be used for small samples and does not require that data frequencies be larger than 5
 - In general samples are standardized and compared with a standard normal distribution(i.e. with specified mean=0 and variance=1)
 - The K-S test is distribution free in the sense that the critical values do not depend on the specific distribution being tested
 - The test computes the distances between the known hypothetical probability distribution(e.g. normal distribution) to the distribution of sample data (empirical distribution function) and and defines the test statistic as the supremum(maximum) of the set of those distances.
 - The KS test is well-known but it has not much power and also sensitive to outliers. It can be used for other distribution.
 - Steps:
    1. Create an EDF(Empirical Distribution Function) for your sample data
    2. Specify CDF of parent distribution (i.e. one that you want to compare your EDF to)
    3. Graph the two distributions together
    4. Measure the greatest vertical distance between the two graphs
    5. Calculate the test statistic
       - F0(x) = the cdf of the hypothesized distribution(Normal)
       - Fdata(x) = the empirical distribution function of your observed data
       - K-S test statistic measures the largest distance between the EDF Fdata(x) and the theoretical CDF function(Normal) F0(x). The Test statistic follows a Kolmogorov distribution
    6. Find the critical value in the KS table
    7. Compare to the critical value.

**Interpretation**

  * If KS statistic > critical value, the null hypothesis is rejected i.e. distribution is not normal,
  * If the observed data perfectly follow a normal distribution, the value of the KS statistic will be 0
  * p-value ≤ 0.05, then we reject the null hypothesis i.e. we assume the distribution is not normal/gaussian.
  * p-value > 0.05, then we fail to reject the null hypothesis i.e. we assume the distribution is normal/gaussian.

```
from scipy.stats import kstest, norm
import numpy as np
my_data = norm.rvs(size=1000)
#my_data = np.random.normal(loc=20, scale=5, size=150)
#stat, p_value = stats.kstest(my_data, 'norm', args=(20, 5))
stat, p_value = kstest(my_data, 'norm')
print(stat, p_value)
alpha = 0.05
if p_value > alpha:
	print('Data appears Gaussian/Normal (fail to reject H0)')
else:
	print('Data does not appear Gaussian/Normal (reject H0)')
```

    0.03799672544124369 0.1086113103005732
    Data appears Gaussian/Normal (fail to reject H0)

### 5. Lilliefors test (or  K-S D test)
  - Null hypothesis H0: The data comes from a specified distribution(normal) based on the empirical distribution function for sample data
 - An improvement on the Kolomogorov-Smirnov (K-S) test. Correcting the K-S for small values at the tails of probability distributions
 - The difference between KS test and Lilliefors test is, it is accepted that the mean and variance of the population distribution are estimated rather than pre-specified by the user.
 - Instead of comparing the standardized data with the standard normal distribution as in Kolmogorov-Smirnov test, we will compare the sample data by the normal distribution with the estimated mean and estimated variance.
 - In Lilliefors test, first population mean and standard deviation are estimated from the sample data and then Kolmogorov-Smirnov test is applied based on the table of critical values in the Lilliefors Test Table
 - It is weaker test than Shapiro Wilk test, Anderson-Darling or Cramer-von Mises

**Interpretation**

 * p-value ≤ 0.05, then we reject the null hypothesis i.e. we assume the distribution is not normal/gaussian.
 * p-value > 0.05, then we fail to reject the null hypothesis i.e. we assume the distribution is normal/gaussian.

```
from scipy.stats import norm
import numpy as np
from statsmodels.stats.diagnostic import lilliefors
my_data = norm.rvs(size=500)
#my_data = np.random.normal(loc=20, scale=5, size=150)
stat, p_value = lilliefors(my_data)
alpha = 0.05
if p_value > alpha:
	print('Data appears Gaussian/Normal (fail to reject H0)')
else:
	print('Data does not appear Gaussian/Normal (reject H0)')
```

    Data appears Gaussian/Normal (fail to reject H0)

### 6. Pearson's chi-squared test (using contingency table)
 - Null hypothesis H0: data is sampled from a normal distribution
 - It is a measure of goodness of fit which is the sum of differences between observed and expected outcome frequencies for each bin and divided by the expected outcome frequency
 \begin{align} \sum_{i=1}^n {\frac{(O_i - E_i)}{E_i}^2}\sim \chi^2(n-1)\end{align}
    - For bin(i), $Oi$ = observed count, $Ei$ = expected count, degree of freedom = number of categories minus 1 = n-1
    - In contingency table, degree of freedom = (number of columns-1) + (number of rows-1)
 - A very small chi square test statistic(<= critical value) means that your observed data fits your expected data extremely well, it means there is a high correlation between your two sets of data.
 -  This test can be used only when data is categorical and randomly sampled, expected value for the number of sample observations for each level is greater than 5
 - The chi square test can only be used for data put into classes(bins). If variable is continuous, it should be binned data to make a frequency table or histogram before performing the test
 - The Chi-Square Test for Normality is not as powerful as other more specific tests (like Lilliefors)

**Interpretation**

 * p-value ≤ 0.05, then we reject the null hypothesis i.e. we assume the distribution of our variable is not normal/gaussian.
 * p-value > 0.05, then we fail to reject the null hypothesis i.e. we assume the distribution of our variable is normal/gaussian.

```
from scipy.stats import chisquare, norm
import numpy as np
my_data = norm.rvs(size=500)
#my_data = np.random.normal(loc=20, scale=5, size=150)
stat, p_value = chisquare(my_data)
alpha = 0.05
if p_value > alpha:
	print('Data appears Gaussian/Normal (fail to reject H0)')
else:
	print('Data does not appear Gaussian/Normal (reject H0)')
```

    Data appears Gaussian/Normal (fail to reject H0)

### 7. Anderson–Darling test (A-D test)
  - Null hypothesis H0: The data comes from a specified distribution(normal) based on the empirical distribution function for sample data
  - Anderson–Darling test is not quite as good as the Shapiro–Wilk test, but is better than other tests
  - When are comparing several distributions, we choose the distribution that gives the largest p-value; this will be the closest match to sample data.
  - It hypothesizes the data does arise from normal distribution, the cumulative distribution function (CDF) of the data assumed to follow a uniform distribution.
  - The Anderson-Darling test makes use of the specific distribution in calculating critical values.
  - The Anderson-Darling test is an alternative to the chi-square and Kolmogorov-Smirnov goodness-of-fit tests.
  - We can use the Anderson-Darling statistic to compare how well a data set fits different distributions. Kolmogorov-Smirnov test taken the maximum difference between the EDF curves, in Anderson-Darling test will consider all the differences.
  - Overall Anderson-Darling test is more powerful than Kolmogorov-Smirnov test because of the more detailed comparison it does.

```
from scipy.stats import anderson, norm
import numpy as np
my_data = norm.rvs(size=1000)
#my_data = np.random.normal(loc=20, scale=5, size=150)
result = anderson(my_data, 'norm')
print('Statistic: %.3f' % result.statistic)
for i in range(len(result.critical_values)):
	sl, cv = result.significance_level[i], result.critical_values[i]
	if result.statistic < result.critical_values[i]:
		print('%.3f: %.3f, data appears normal (fail to reject H0)' % (sl, cv))
	else:
		print('%.3f: %.3f, data does not appear normal (reject H0)' % (sl, cv))
```

    Statistic: 0.277
    15.000: 0.574, data appears normal (fail to reject H0)
    10.000: 0.653, data appears normal (fail to reject H0)
    5.000: 0.784, data appears normal (fail to reject H0)
    2.500: 0.914, data appears normal (fail to reject H0)
    1.000: 1.088, data appears normal (fail to reject H0)

### 8. Cramér–von Mises criterion (CvM or Watson test)

 - In KS test only the maximum distance is considered as test statistic. But, In Cramér–von Mises test, the test statistic will be based on all the deviations and it is sum of squares of the deviations.
 - Empirical evidences suggest that the Cramér–von Mises test is usually more powerful than the Kolmogorov–Smirnov test for a broad class of alternative hypothesis.
 - But both the KS and CvM statistics are insensitive when the differences between the curves is most prominent near the beginning or end of the distributions. This is because, by construction, the EDFs converge to 0.0 and 1.0 at the ends and any deviations must be small.

```
#!pip install scikit-gof
from scipy.stats import norm
import numpy as np
my_data = norm.rvs(size=500)
#my_data = np.random.normal(loc=20, scale=5, size=150)
from skgof import cvm_test
stat, p_value=cvm_test(my_data,'norm')
alpha = 0.05
if p_value > alpha:
	print('Data appears Gaussian/Normal (fail to reject H0)')
else:
	print('Data does not appear Gaussian/Normal (reject H0)')
```

    Data appears Gaussian/Normal (fail to reject H0)

### 9. Martinez-Iglewicz Test
  - This test for normality is based on the median and a robust estimator of dispersion.This test is very powerful for heavy-tailed symmetric distributions.

### Notes

For what criteria we should look while choosing the normality test?

1. Parameters of the normal distribution which you want to compare with the sample data.
2. Sample size
3. Which characteristic of the normal distribution you want to test with
4. Power

All these are discussed above whenever needed.

Ex1:

 - If you are not interested in the parameters of the normal and just want to simply determine if the distribution is normal or not S-W test, A-D test, Lilliefors test for testing normality with normal distribution unknown mean and variance is preferred.

 - If you are interested to compare with in specific parameters(Ex: In general we expect the error distribution ~ N(0,σ)) one can opt the KS test,AD test.

Ex2:

 - SW test was preferred for small samples.
 - AD test was preferred if we have enough sample size.
 - The asymptotic power of KS test is 1.
 - If we have sufficiently large sample QQ plot is preferred.

Ex3:

 - If we observe the the distribution of sample has the heavier tail, AD best will preferred.
 - If we observe the the distribution of sample was skewed(i.e., The empirical distribution is summarized through its skewness and kurtosis statistics and compared to the skewness and kurtosis of the normal distribution.) tests based on skewness and kurtosis(Jarque–Bera test) are preferred.
 - For the distributions that have slightly or definitely higher kurtosis than the normal, the skewness-kurtosis based tests are more powerful than the other types of test.

> The results of a test for normality should not only report a p-value but they should be accompanied by a careful interpretation of the probability plot and skewness and kurtosis statistics for a complete diagnosis.


# An application of Binomial transform

In this notebook we apply the [Binomial transform][on-oeis] to the Fibonacci numbers in order to substain the main [notebook][recurrence-notebook] where these numbers are studied in depth.

[on-oeis]:https://oeis.org/wiki/Binomial_transform
[recurrence-notebook]:http://nbviewer.jupyter.org/github/massimo-nocentini/PhD/blob/master/courses/paa/sympy-notebook/recurrences-unfolding.ipynb?flush_cache=true

```python
import sympy
from sympy import *
from sympy.abc import x, n, z, t, k

init_printing() # for nice printing, a-la' TeX
```

```python
%run "sums.py"

# duplicated code, put it into "sums.py"
def expand_sum_in_eq(eq_term):
    lhs, rhs = eq_term.lhs, eq_term.rhs
    return Eq(lhs, expand_Sum(rhs))

f = IndexedBase('f')
fibs = {f[i]:fibonacci(i) for i in range(100)}
```

```python
transforming_matrix = Matrix([
        [1,0,0,0,0,0,0,0,0],
        [0,1,0,0,0,0,0,0,0],
        [1,1,1,0,0,0,0,0,0],
        [-1,0,1,1,0,0,0,0,0],
        [1,0,0,1,1,0,0,0,0],
        [-1,0,0,0,1,1,0,0,0],
        [1,0,0,0,0,1,1,0,0],
        [-1,0,0,0,0,0,1,1,0],
        [1,0,0,0,0,0,0,1,1]])

transforming_matrix
```

```python
transforming_matrix**(-1)
```

```python
def gf(t): return t + 1/(1+t)

gf(t).series(n=10)
```

```python
def h(t): return t*(1+2*t+t**2)/(1+t+t**2)

(gf(t)*h(t)**2).series(n=10)
```

```python
pascal_matrix = Matrix([
        [1,0,0,0,0,0,0,0,0],
        [1,1,0,0,0,0,0,0,0],
        [1,2,1,0,0,0,0,0,0],
        [1,3,3,1,0,0,0,0,0],
        [1,4,6,4,1,0,0,0,0],
        [1,5,10,10,5,1,0,0,0],
        [1,6,15,20,15,6,1,0,0],
        [1,7,21,35,35,21,7,1,0],
        [1,8,28,56,70,56,28,8,1]])
pascal_matrix
```

```python
catalan_matrix = Matrix([
        [1,0,0,0,0,0,0,0,0],
        [1,1,0,0,0,0,0,0,0],
        [2,2,1,0,0,0,0,0,0],
        [5,5,3,1,0,0,0,0,0],
        [14,14,9,4,1,0,0,0,0],
        [42,42,28,14,5,1,0,0,0],
        [132,132,90,48,20,6,1,0,0],
        [429,429,297,165,75,27,7,1,0],
        [1430,1430,1001,572,275,110,35,8,1]])
catalan_matrix
```

```python
catalan_inverse_matrix = Matrix([
        [1,0,0,0,0,0,0,0,0],
        [1,0,0,0,0,0,0,0,0],
        [0,1,0,0,0,0,0,0,0],
        [0,-1,1,0,0,0,0,0,0],
        [0,0,-2,1,0,0,0,0,0],
        [0,0,1,-3,1,0,0,0,0],
        [0,0,0,3,-4,1,0,0,0],
        [0,0,0,-1,6,-5,1,0,0],
        [0,0,0,0,-4,10,-6,1,0]])
catalan_inverse_matrix
```

```python
odd_transformed_matrix = pascal_matrix * transforming_matrix
odd_transformed_matrix
```

```python
transforming_matrix * pascal_matrix
```

```python
(catalan_matrix**(-1) )*odd_transformed_matrix
```

```python
catalan_inverse_matrix * odd_transformed_matrix
```

```python
fib_matrix = Matrix([fibonacci(i) for i in range(9)])
fib_matrix_sym = Matrix([f[i] for i in range(9)])

fib_matrix, fib_matrix_sym
```

```python
a = Wild('a')
std_prod = transforming_matrix * fib_matrix
sym_prod = transforming_matrix * fib_matrix_sym

std_prod, sym_prod, sym_prod.subs({f[0]:0}), sym_prod.subs({f[0]:0}).replace(f[a]+f[a+1],f[a+2])
```

```python
rhs=pascal_matrix * transforming_matrix * fib_matrix
rhs_sym = pascal_matrix * sym_prod
rhs, rhs_sym
```

```python
a_range = range(1,18,2)
lhs=Matrix([fibonacci(i) for i in a_range]) - Matrix([1 for i in range(9)])
lhs_sym=Matrix([f[i] for i in a_range]) - Matrix([1 for i in range(9)])
lhs, lhs_sym
```

```python
Eq(lhs,rhs)
```

```python
eq_sym = Eq(lhs_sym, rhs_sym)
eq_sym
```

```python
f_minus1_vector = Matrix([f[-1] for i in range(9)])
one_plus_eq = eq_sym#.subs(-Integer(1),f[-1])
one_plus_eq = Eq(one_plus_eq.lhs + f_minus1_vector, one_plus_eq.rhs + f_minus1_vector)
one_plus_eq = Eq(one_plus_eq.lhs.subs(f[-1],1), one_plus_eq.rhs)
Eq(one_plus_eq.lhs, one_plus_eq.rhs.subs(fibs))
```

```python
fib0_term = f[0]
eq_sym.subs(fib0_term, fibs[fib0_term])
```

Since coefficient in the triangle on the rhs are a part of Pascal triangle, namely [A104712][oeis], the following is a generalization:
$$
    f_{2n+1} - 1 = \sum_{k=1}^{n}{{{n+1}\choose{k+1}}f_{k}}
$$

[oeis]:http://oeis.org/A104712

```python
gen_odd_fibs = Eq(f[2*n+1]-1, Sum(binomial(n+1, k+1)*f[k], (k,1,n)))
Eq(gen_odd_fibs, Sum(binomial(n+1, n-k)*f[k], (k,1,n)))
```

```python
expand_sum_in_eq(gen_odd_fibs.subs(n, 8))
```

```python
eq_sym.subs(fibs)
```

```python
eq_17 = Eq(f[17],f[-1] + rhs_sym[-1])
eq_18_shift = Eq(f[n], f[n-18]+8*f[n-17]+36*f[n-16]+84*f[n-15]+126*f[n-14]+126*f[n-13]+84*f[n-12]+36*f[n-11]+9*f[n-10]+f[n-9])
eq_17, eq_18_shift
```

```python
[eq_18_shift.subs(n,i).lhs.subs(fibs) - eq_18_shift.subs(n,i).rhs.subs(fibs) for i in range(18,32)]
```

again, fibonacci numbers, [A000045][oeis].

[oeis]:http://oeis.org/A000045

```python
from itertools import accumulate

to_accumulate = rhs_sym + ones(9,1)*f[-1]
even_rhs = Matrix(list(accumulate(to_accumulate, lambda folded, current_row: Add(folded, current_row) )))

even_lhs = Matrix([f[i] for i in range(2,19,2)])

even_fibs_matrix_eq = Eq(even_lhs, even_rhs)
even_fibs_matrix_eq
```

```python
even_transformed_matrix = Matrix([
        [1,0,0,0,0,0,0,0,0],
        [2,1,0,0,0,0,0,0,0],
        [4,4,1,0,0,0,0,0,0],
        [7,10,5,1,0,0,0,0,0],
        [11,20,15,6,1,0,0,0,0],
        [16,35,35,21,7,1,0,0,0],
        [22,56,70,56,28,8,1,0,0],
        [29,84,126,126,84,36,9,1,0],
        [37,120,210,252,210,120,45,10,1]])
even_transformed_matrix
```

```python
even_transforming_matrix = (pascal_matrix**(-1))*even_transformed_matrix
even_transforming_matrix
```

```python
(catalan_matrix**(-1) )*even_transformed_matrix
```

```python
catalan_inverse_matrix * even_transformed_matrix
```

```python
even_transforming_matrix * fib_matrix_sym
```

```python
even_vector_eq_sym = Eq(even_lhs - Matrix(list(range(1,10))),
                        pascal_matrix * even_transforming_matrix * fib_matrix_sym)
even_vector_eq_sym
```

```python
even_vector_eq_sym.subs(fib0_term, fibs[fib0_term])
```

Since coefficient in the triangle on the rhs are a part of Pascal triangle, namely [A104713][oeis], the following is a generalization:
$$
    f_{2n} - n = \sum_{k=1}^{n-1}{{{n+1}\choose{k+2}}f_{k}}
$$

[oeis]:http://oeis.org/A104713

```python
gen_even_fibs = Eq(f[2*n]-n, Sum(binomial(n+1, k+2)*f[k], (k,1,n-1)))
Eq(gen_even_fibs, Sum(binomial(n+1, n-k-1)*f[k], (k,1,n-1)))
```

```python
expand_sum_in_eq(gen_even_fibs.subs(n, 9))
```

```python
even_fibs_matrix_eq_minus1_appear = even_fibs_matrix_eq.subs(fibs)
Eq(even_fibs_matrix_eq.lhs, even_fibs_matrix_eq_minus1_appear, evaluate=False)
```

summands on the rhs form a known sequence [A054452][oeis].

[oeis]:http://oeis.org/A054452

```python
list(accumulate([fibonacci(2*i+1)-1 for i in range(21)]))
```

```python
def n_gf(t): return t/(1-t)**2

n_gf(t).series(n=20)
```

```python
def odd_fib_gf(t): return t**2/((1-t)**2*(1-3*t+t**2))

odd_fib_gf(t).series(n=20)
```

```python
composite_odd_fibs_gf = n_gf(t)+odd_fib_gf(t)
composite_odd_fibs_gf.factor(), composite_odd_fibs_gf.series(n=20)
```

```python
def odd_integers_gf(t): return ((n_gf(t)+n_gf(-t))/2).simplify()

odd_integers_gf(t).series(n=20)
```

```python
# here is the error: we should use the generating function of F(2n+1) instead of F(n) as done here!
def fib_gf(t): return t/(1-t-t**2)

fib_gf(odd_integers_gf(t)).series(n=20)
```

```python
def even_fibs_gf(t): return n_gf(t) + fib_gf(t)/(1-t)

even_fibs_gf(t).series(n=10)
```

```python
even_fibs_matrix_eq_minus1_appear.subs(f[-1],1)
```

```python
eq_17 = Eq(f[n], 8*f[n-17]+29*f[n-16]+84*f[n-15]+126*f[n-14]+126*f[n-13]+84*f[n-12]+36*f[n-11]+9*f[n-10]+f[n-9])
eq_17
```

```python
[eq_17.subs(n,i).lhs.subs(fibs) - eq_17.subs(n,i).rhs.subs(fibs) for i in range(17,31)]
```

which is known as [A022091][oeis].

[oeis]:http://oeis.org/A022091

---

This work by Massimo Nocentini is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://github.com/massimo-nocentini/recurrences-unfolding.


Exercise material of the MSc-level course **Numerical Methods in Geotechnical Engineering**.
Held at Technische Universität Bergakademie Freiberg.

Comments to:

*Prof. Dr. Thomas Nagel
Chair of Soil Mechanics and Foundation Engineering
Geotechnical Institute
Technische Universität Bergakademie Freiberg.*

https://tu-freiberg.de/en/soilmechanics

# Exercise on the least squares method

You're given a CPT data set consisting of cone resistance $q_\text{c}$ and frictional resistance $f_\text{s}$ measurements. Your first task is to check the results by plotting them and to compute as well as plot the friction ratio $R_\text{f}$.

```python
import numpy as np #numerical methods
#import sympy as sp #symbolic operations
import matplotlib.pyplot as plt #plotting
import pandas as pd

#Some plot settings
import plot_functions.plot_settings
%run plot_functions/CPT_plots.ipynb
```

```python
CPT_data = pd.read_csv('Data/CPT_2015.csv')
```

```python
CPT_data
```

      Teufe
      qc
      fs

      0
      [m]
      [MN/m²]
      [MN/m²]

      1
      0.0000000000
      0.0000000000
      0.0000000000

      2
      0.0200000000
      0.0000000000
      0.0000000000

      3
      0.0400000000
      0.0672000000
      0.0000000000

      4
      0.0600000000
      0.1176000000
      0.0002000000

      ...
      ...
      ...
      ...

      1287
      25.7200000000
      4.8557000000
      0.0518000000

      1288
      25.7400000000
      4.8280000000
      0.0497000000

      1289
      25.7600000000
      4.8296000000
      0.0497000000

      1290
      25.7800000000
      4.8479000000
      0.0497000000

      1291
      25.8000000000
      4.9254000000
      0.0497000000

1292 rows × 3 columns

```python
z = CPT_data.Teufe[1:].to_numpy(dtype=float)
qc = CPT_data.qc[1:].to_numpy(dtype=float)
fs = CPT_data.fs[1:].to_numpy(dtype=float)
with np.errstate(divide='ignore',invalid='ignore'):
    Rf = np.divide(fs,qc)
```

```python
plot_CPT_raw(z,qc,fs,Rf)
```

You're also given data on the relationship between $q_\text{c}$ and $I_\text{D}$. The data is valid only for $z > 2$ m as well as above the phreatic surface. Your task is now to find a functional relationship $I_\text{D}(q_\text{c})$ in order to estimate the $I_\text{D}$ distribution in the above soil profile investigated with cone prenetration testing.

```python
ID_data = pd.read_csv('Data/qc_ID.csv')
```

```python
ID_data
```

      qc
      ID

      0
      MPa
      NaN

      1
      3
      0.025327

      2
      6
      0.189061

      3
      9
      0.407843

      4
      12
      0.361879

      ...
      ...
      ...

      67
      11
      0.510890

      68
      15
      0.538081

      69
      19
      0.634794

      70
      23
      0.719363

      71
      27
      0.761487

72 rows × 2 columns

```python
qc_ID = ID_data.qc[1:].to_numpy(dtype=float)
ID = ID_data.ID[1:].to_numpy(dtype=float)
```

```python
plot_ID_raw(qc_ID, ID)
```

The profile appears to be a logarithmic function (you may also employ a polynomial formulation), i.e. we choose an Ansatz

$$
    I_\text{D}^\text{mod} = a + b \log q_\text{c}
$$

The first task at hand is now to identify the free parameters $a$ and $b$.

For that purpose, we would like to minimize the overall deviation between the model $I_\text{D}^\text{mod}(q_\text{c})$ and the experimental data $(I_\text{D},\, q_\text{c})$.

To measure this deviation, we introduce the error or residual

$$
    r = I_\text{D}^\text{mod} - I_\text{D}
$$

and seek the minimum of the squared error functional. In other words, we use the least squares approach:

\begin{align}
    a, b &= \text{arg}\,\text{min} \int \limits_{q_\text{c}^\text{min}}^{q_\text{c}^\text{max}} \frac{1}{2} r(q)^2\,\text{d} q
    \\
    &= \text{arg}\,\text{min} \sum \limits_{i=1}^{n_\text{exp}} \frac{(I_\text{D}^\text{mod} - I_\text{D})_i^2}{2}
\end{align}

We have two unknowns and require two equations. The necessary conditions for finding the minimum are given by:

\begin{align}
    0 &= \frac{\partial}{\partial a} \sum \limits_{i=1}^{n_\text{exp}} \frac{(I_\text{D}^\text{mod} - I_\text{D})_i^2}{2}
    \\
    0 &= \frac{\partial}{\partial b} \sum \limits_{i=1}^{n_\text{exp}} \frac{(I_\text{D}^\text{mod} - I_\text{D})_i^2}{2}
\end{align}

This can be reformulated as

\begin{align}
    0 &= \sum \limits_{i=1}^{n_\text{exp}} \left. \frac{\partial I_\text{D}^\text{mod}}{\partial a} \right|_i (I_\text{D}^\text{mod} - I_\text{D})_i
    \\
    0 &= \sum \limits_{i=1}^{n_\text{exp}} \left. \frac{\partial I_\text{D}^\text{mod}}{\partial b} \right|_i (I_\text{D}^\text{mod} - I_\text{D})_i
\end{align}

The necessary partial derivatives are given by

\begin{align}
    \frac{\partial I_\text{D}^\text{mod}}{\partial a} &= 1
    \\
    \frac{\partial I_\text{D}^\text{mod}}{\partial b} &= \log q_\text{c}
\end{align}

Thus, we arrive at the following set of equations to solve for $a$ and $b$:

\begin{align}
    0 &= \sum \limits_{i=1}^{n_\text{exp}} (a + b \log q_\text{c} - I_\text{D})_i
    \\
    0 &= \sum \limits_{i=1}^{n_\text{exp}} \log q_{\text{c},i} (a + b \log q_\text{c} - I_\text{D})_i
\end{align}

This results in the linear equation system $A x = \text{RHS}$:

\begin{align}
\begin{pmatrix}
n_\text{exp} & \sum \limits_{i=1}^{n_\text{exp}} \log q_{\text{c},i}
\\
\sum \limits_{i=1}^{n_\text{exp}} \log q_{\text{c},i} & \sum \limits_{i=1}^{n_\text{exp}} (\log q_{\text{c},i})^2
\end{pmatrix}
\begin{pmatrix}
a
\\
b
\end{pmatrix}
=
\begin{pmatrix}
\sum \limits_{i=1}^{n_\text{exp}} (I_\text{D})_i
\\
\sum \limits_{i=1}^{n_\text{exp}} (\log q_\text{c} I_\text{D})_i
\end{pmatrix}
\end{align}

Let's construct this system of equations and solve it:

```python
RHS = np.array([ID.sum(),(np.log(qc_ID)*ID).sum()])
```

```python
A = np.array([[len(ID),np.log(qc_ID).sum()],[np.log(qc_ID).sum(),(np.log(qc_ID)*np.log(qc_ID)).sum()]])
```

```python
x = np.linalg.solve(A,RHS)
x
```

    array([-0.29305064,  0.30846968])

As we see, $a\approx -0.293$ and $b \approx 0.308$. We put this into the model and then check whether it matches the data. For that purpose, we can now also compare model and data by plotting one over the other and compare them to an ideal line.

```python
ID_mod = lambda qc: x[0] + x[1] * np.log(qc)
```

```python
plot_ID_model(qc_ID, ID)
```

The visual inspection looks good.

We can quantify the coefficient of determination $R^2$. For that purpose we calculate the mean of the experimental data:

$$
    \bar{I}_\text{D} = \frac{1}{n_\text{exp}} \sum \limits_{i=1}^{n_\text{exp}} I_{\text{D},i}
$$

We can now estimate the variance of the experimental data:

$$
    \sigma^2_\text{exp} = \frac{1}{n_\text{exp}-1} \sum \limits_{i=1}^{n_\text{exp}} \left(I_{\text{D},i} - \bar{I}_\text{D} \right)^2
$$

We can compute a similar measure for the deviation between experimental data and model (the residual sum of squares):

$$
    \text{RSS} = \frac{1}{n_\text{exp}-1} \sum \limits_{i=1}^{n_\text{exp}} \left(I_{\text{D},i} - I_\text{D}^\text{mod} \right)^2
$$

We can now calculate the coefficient of determination:

$$
    R^2 = 1 - \frac{\text{RSS}}{\sigma^2_\text{exp}}
$$

Should the modelled values always match the experimental ones exactly, we'd arrive at $R^2=1$. If the model simply predicts the mean (so-called baseline model), we arrive at $R^2 = 0$. The more the model explains part of the variability of the data, the more $R^2$ approaches 1. If the model predictions are worse than the baseline model, $R^2$ is negative.

```python
R_squared = 1. - np.sum((ID - ID_mod(qc_ID))**2) / np.sum((ID - ID.mean())**2)
print("The model has a coefficient of determination of R² = %.4f." %R_squared)
```

    The model has a coefficient of determination of R² = 0.8565.

Now that we've established a model for the relationship between $I_\text{D}$ and $q_\text{c}$ as well as quantified it's reliability, we can come back to the task at hand and calculate the $I_\text{D}$ profile from the CPT data.

```python
with np.errstate(divide='ignore',invalid='ignore'):
    plot_prediction(z,qc,fs,np.where(z>0.))
```

We observe a wide range of negative data. This is because the model was applied outside of the parameter range for which it is applicable. The model is calibrated only for $3\,\text{MPa} \leq q_\text{c} \leq 30\,\text{MPa}$ and is valid only in depths below $2\,\text{m}$. We therefore restrict its application to this range:

```python

```

```python
admissible_depths = np.where(z > 2.)
admissible_qcs = np.where(qc > 3.)

admissible_data_points = np.intersect1d(admissible_depths,admissible_qcs)
```

Now we can plot only those data points that are within the admissible model range:

```python
plot_prediction(z,qc,fs,admissible_data_points)
```

## Remarks

The idea of this exercise was to demonstrate a possible use case for and the workflow for a least squares analysis. The application of the $I_\text{D}$ correlation to explain the $q_\text{c}$ data is of course greatly simplified. There are other potential causes of an increase in $q_\text{c}$ that are not explained by this model.

```python

```


# Linear VMM (Vessel Maneuvering Model) for wPCC using brix coefficients

# Purpose
Implementing according to:
Matusiak, Jerzy. Dynamics of a Rigid Ship. Aalto University, 2017. https://aaltodoc.aalto.fi:443/handle/123456789/24408.

# Methodology
Define the problem using SymPy

# Setup

```python
# %load imports.py
%matplotlib inline
%load_ext autoreload
%autoreload 2
%config Completer.use_jedi = False  ## (To fix autocomplete)

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)

import numpy as np
import os
import matplotlib.pyplot as plt

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp

## Local packages:
from src.data import mdl
from src import symbols
from src import prime_system
from src.symbols import *
from src.linear_vmm_equations import *

import src.linear_vmm_equations as eq
import src.models.linear_vmm as model
from src.visualization.plot import track_plot
import src.nonlinear_vmm_equations as nonlinear_vmm_equations

```

## Load test

```python
df_runs = mdl.runs()
```

```python
#id=22773
#id=22616
#id=22774
id=22770

df, units, meta_data = mdl.load(id=id, dir_path='../data/processed/kalman')
df['U'] = np.sqrt(df['u']**2 + df['v']**2)
df['beta'] = -np.arctan2(df['v'],df['u'])
df = df.iloc[100:-100].copy()
df.index = df.index.total_seconds()
df.index-=df.index[0]
df.sort_index(inplace=True)
```

```python
df.head()
```

```python
meta_data['rho']=1000
meta_data['mass'] = meta_data['Volume']*meta_data['rho']
meta_data.dropna()
```

```python
from src.visualization.plot import track_plot
fig,ax=plt.subplots()
#fig.set_size_inches(10,10)
track_plot(df=df, lpp=meta_data.lpp, x_dataset='x0', y_dataset='y0',  psi_dataset='psi', beam=meta_data.beam, ax=ax);
```

# Ship parameters

```python
T_ = (meta_data.TA + meta_data.TF)/2
L_ = meta_data.lpp
m_ = meta_data.mass
rho_ = meta_data.rho
B_ = meta_data.beam
CB_ = m_/(T_*B_*L_*rho_)
I_z_ = m_*meta_data.KZZ**2

ship_parameters = {
        'T' : T_,
        'L' : L_,
        'CB' :CB_,
        'B' : B_,
        'rho' : rho_,
        'x_G' : 0,  # motions are expressed at CG
        'm' : m_,
        'I_z': I_z_,
        'volume':meta_data.Volume,
    }

ps = prime_system.PrimeSystem(**ship_parameters)
ship_parameters_prime = ps.prime(ship_parameters)
```

# Brix parameters

```python
def calculate_prime(row, ship_parameters):
    return run(function=row['brix_lambda'], inputs=ship_parameters)

mask = df_parameters['brix_lambda'].notnull()
df_parameters.loc[mask,'prime'] = df_parameters.loc[mask].apply(calculate_prime, ship_parameters=ship_parameters, axis=1)
df_parameters.loc['Ydelta','prime'] = 0.005  # Just guessing
df_parameters.loc['Ndelta','prime'] = -df_parameters.loc['Ydelta','prime']/2  # Just guessing
df_parameters['prime'].fillna(0, inplace=True)
```

```python
df_parameters.head()
```

```python
ship_parameters
```

```python
interesting = [
    'u',
    'v',
    'r',
    'U',
    'x0',
    'y0',
    'psi',
    'delta',
    'beta',
]
df_ = df[interesting].copy()
df_ = ps.prime(df_, U=df['U'])
```

```python
df_.head()
```

```python
t = df_.index
t_prime = ps._prime(t, unit='time', U=df['U'])

fig,ax=plt.subplots()
ax.plot(t, t_prime)
ax.set_xlabel('$t$')
ax.set_ylabel("$t'$");

```

## Prime simulation

```python
interesting = [
    'u',
    'v',
    'r',
    'U',
    'x0',
    'y0',
    'psi',
    'delta',
    'beta',
]
df_ = df[interesting].copy()
df_ = ps.prime(df_, U=df['U'])
df_.index=t_prime

#t = df_.index
#control = df_[['delta']]
#control = {'delta':0,}

parameters = dict(df_parameters['prime'])

df_0 = df_.iloc[0]
y0 = {
    'u' : df_0['u'],
    'v' : df_0['v'],
    'r' : df_0['r'],
    'x0' : df_0['x0'],
    'y0' : df_0['y0'],
    'psi' : df_0['psi'],
    }

#solution = model.simulate(y0=y0, t=t, df_parameters=df_parameters, df_ship_parameters=df_ship_parameters, control=control, rtol=1e-3, atol=1e-3)
result = model.simulator.simulate(df_=df_, parameters=parameters, ship_parameters=ship_parameters_prime,
                                  control_keys=['delta'])
result.plot_compare()
```

```python
result.result
```

### Go back to SI

```python
t_prime = result.result.index
t = df.index
fig,ax=plt.subplots()
ax.plot(t,t_prime)
```

```python
U_ = t_prime/t*ship_parameters['L']
fig,ax=plt.subplots()
ax.plot(t,U_)
df.plot(y='U', ax=ax)
```

```python
#result_SI = ps.unprime(result.result, U=df['U'].values)
result_SI = ps.unprime(result.result, U=U_)
result_SI.index = df.index
```

```python
fig,ax = plt.subplots()
track_plot(df=df, lpp=ship_parameters['L'], beam=ship_parameters['B'],ax=ax, label='model test')
track_plot(df=result_SI, lpp=ship_parameters['L'], beam=ship_parameters['B'],ax=ax, label='simulation', color='green')
ax.legend()

for key in result_SI:
    fig,ax = plt.subplots()
    df.plot(y=key, label='model test', ax=ax)
    result_SI.plot(y=key, label='simulation', ax=ax)
    ax.set_ylabel(key)
```

### Simulation with SI states and prime parameters

```python
interesting = [
    'u',
    'v',
    'r',
    'U',
    'x0',
    'y0',
    'psi',
    'delta',
    'beta',
]
df_ = df[interesting].copy()
parameters = dict(df_parameters['prime'])

df_0 = df_.iloc[0]
y0 = {
    'u' : df_0['u'],
    'v' : df_0['v'],
    'r' : df_0['r'],
    'x0' : df_0['x0'],
    'y0' : df_0['y0'],
    'psi' : df_0['psi'],
    }

result = model.simulator.simulate(df_=df_, parameters=parameters, ship_parameters=ship_parameters,
                                  control_keys=['delta'], primed_parameters=True,prime_system=ps)
```

```python
result.plot_compare()
```

## Compare simulation approaches

```python
fig,ax = plt.subplots()
track_plot(df=result.result, lpp=ship_parameters['L'], beam=ship_parameters['B'],ax=ax, label='primed parameters')
track_plot(df=result_SI, lpp=ship_parameters['L'], beam=ship_parameters['B'],ax=ax, label='simulation', color='green')
ax.legend()

for key in result_SI:
    fig,ax = plt.subplots()
    result.result.plot(y=key, label='primed parameters', ax=ax)
    result_SI.plot(y=key, label='simulation', ax=ax)
    ax.set_ylabel(key)
```

```python

```


- - - -
# Mechpy Tutorials
a mechanical engineering toolbox

source code - https://github.com/nagordon/mechpy
documentation - https://nagordon.github.io/mechpy/

- - - -

Neal Gordon
2017-02-20

- - - -

## Composites

This a collection of sample codes for various calcuations useful for composites mechanics

## Python Initilaization with module imports

```python
cd ..
```

    Y:\github\mechpy

```python
# setup
import numpy as np
import sympy as sp
import scipy
from pprint import pprint
sp.init_printing(use_latex='mathjax')

import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (12, 8)  # (width, height)
plt.rcParams['font.size'] = 14
plt.rcParams['legend.fontsize'] = 16
from matplotlib import patches

#get_ipython().magic('matplotlib')  # seperate window
get_ipython().magic('matplotlib inline') # inline plotting
```

## mechpy Composite Mechanics

```python
from mechpy.math import T3rot, T6rot
from mechpy.composites import my_laminate_with_loading, material_plots, laminate_gen, \
plot_single_max_failure_loads
```

```python
import importlib
importlib.reload
```

    <function importlib.reload>

```python
from mechpy.math import T2rot
T2rot(45)
```

    array([[ 0.525, -0.851],
           [ 0.851,  0.525]])

```python
print(T6rot(45,45,45))
```

    [[ 0.25   0.25   0.5   -0.354 -0.354  0.25 ]
     [ 0.021  0.729  0.25   0.427 -0.073 -0.125]
     [ 0.729  0.021  0.25  -0.073  0.427 -0.125]
     [-0.25  -0.25   0.5    0.354  0.354  0.75 ]
     [ 0.854 -0.146 -0.707  0.354 -0.354  0.354]
     [-0.146  0.854 -0.707 -0.354  0.354  0.354]]

```python
help(laminate_gen)
```

    Help on function laminate_gen in module mechpy.composites:

    laminate_gen(lamthk=1.5, symang=[45, 0, 90], plyratio=2.0, matrixlayers=False, balancedsymmetric=True)
        ## function created to quickly create laminates based on given parameters
        lamthk=1.5    # total #thickness of laminate
        symang = [45,0,90, 30]  #symmertic ply angle
        plyratio=2.0  # lamina/matrix ratio
        matrixlayers=False  # add matrix layers between lamina plys
        nonsym=False    # symmetric
        mat = material type, as in different plies, matrix layer, uni tapes, etc
        #ply ratio can be used to vary the ratio of thickness between a matrix ply
             and lamina ply. if the same thickness is desired, plyratio = 1,
             if lamina is 2x as thick as matrix plyratio = 2

```python
%matplotlib inline
```

```python
material_plots()
```

```python
my_laminate_with_loading()
```

```python
plot_single_max_failure_loads(mymat='E-Glass Epoxy fabric M10E-3783', mylayup=[0,45,45,0] )
```

```python

```

```python

```

```python

```


# The Implicit Crank-Nicolson Difference Equation for the Heat Equation
## The Heat Equation
The Heat Equation is the first order in time ($t$) and second order in space ($x$) Partial Differential Equation:
\begin{equation}  \frac{\partial u}{\partial t} = \frac{\partial^2 u}{\partial x^2},\end{equation}
The equation describes heat transfer on a domain
\begin{equation}\Omega = \{ t \geq, 0\leq x \leq 1\}. \end{equation}
with an initial condition at time $t=0$ for all $x$ and boundary condition on the left ($x=0$) and right side ($x=1$).

## Crank-Nicolson Difference method
This note book will illustrate the Crank-Nicolson Difference method for the Heat Equation with the __initial conditions__
\begin{equation} u(x,0)=x^2, \ \ 0 \leq x \leq 1, \end{equation}
and __boundary condition__
\begin{equation} u(0,t)=t,  u(1,t)=2-exp(-t). \end{equation}

```python
# LIBRARY
# vector manipulation
import numpy as np
# math functions
import math

# THIS IS FOR PLOTTING
%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import warnings
warnings.filterwarnings("ignore")

```

## Discete Grid
The region $\Omega$ is discretised into a uniform mesh $\Omega_h$. In the space $x$ direction into $N$ steps giving a stepsize of
\begin{equation} h=\frac{1-0}{N},\end{equation}
resulting in
\begin{equation}x[i]=0+ih,\end{equation}
and into $N_t$ steps in the time $t$ direction giving a stepsize of
\begin{equation}k=\frac{1-0}{N}\end{equation}
resulting in
\begin{equation}t[i]=0+ik.\end{equation}
The Figure below shows the discrete grid points for $N=10$ and $N_t=15$, the red dots are the unknown values, the green dots are the known boundary conditions and the blue dots are the known initial conditions of the Heat Equation.

```python
N=10
Nt=100
h=1/N
k=1/Nt
r=k/(h*h)
time_steps=50
time=np.arange(0,(time_steps+.5)*k,k)
x=np.arange(0,1.0001,h)
X, Y = np.meshgrid(x, time)
fig = plt.figure(figsize=(8,6))
plt.plot(X,Y,'ro');
plt.plot(x,0*x,'bo',label='Initial Condition');
plt.plot(np.ones(time_steps+1),time,'go',label='Boundary Condition');
plt.plot(x,0*x,'bo');
plt.plot(0*time,time,'go');
plt.xlim((-0.02,1.02))
plt.xlabel('x')
plt.ylabel('time (ms)')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.title(r'Discrete Grid $\Omega_h,$ h= %s, k=%s'%(h,k),fontsize=24,y=1.08)
plt.show();
```

## Discrete Initial and Boundary Conditions

The discrete initial conditions are
\begin{equation}w[i,0]=x[i]\times x[i], \ \ 0 \leq x[i] \leq 1\end{equation}
and the discrete boundary conditions are
\begin{equation}w[0,j]=t[j],  w[10,j]=2-math.exp(t[j]), \end{equation}
where $w[i,j]$ is the numerical approximation of $U(x[i],t[j])$.

The Figure below plots the inital and boundary conditions for $t[0]=0.$

```python
w=np.zeros((N+1,time_steps+1))
b=np.zeros(N-1)
# Initial Condition
for i in range (1,N):
    w[i,0]=x[i]*x[i]

# Boundary Condition
for j in range (0,time_steps+1):
    w[0,j]=time[j]
    w[N,j]=2-math.exp(-time[j])

fig = plt.figure(figsize=(8,4))
plt.plot(x,w[:,0],'o:',label='Initial Condition')
plt.plot(x[[0,N]],w[[0,N],0],'go',label='Boundary Condition t[0]=0')
plt.xlim([-0.1,1.1])
plt.ylim([-0.1,1.1])
plt.title('Intitial and Boundary Condition',fontsize=24)
plt.xlabel('x')
plt.ylabel('w')
plt.legend(loc='best')
plt.show()
```

## The Implicit Crank-Nicolson Difference Equation
The implicit Crank-Nicolson difference equation of the Heat Equation is
\begin{equation}
\frac{w_{ij+1}-w_{ij}}{k}=\frac{1}{2}\left(\frac{w_{i+1j+1}-2w_{ij+1}+w_{i-1j+1}}{h^2}+\frac{w_{i+1j}-2w_{ij}+w_{i-1j}}{h^2}\right)
\end{equation}
rearranging the equation we get
\begin{equation}
-rw_{i-1j+1}+(2+2r)w_{ij+1}-rw_{i+1j+1}=rw_{i-1j}+(2-2r)w_{ij}+rw_{i+1j}
\end{equation}
for $i=1,...9$ where $r=\frac{k}{h^2}$.

This gives the formula for the unknown term $w_{ij+1}$ at the $(ij+1)$ mesh points
in terms of $x[i]$ along the jth time row.

Hence we can calculate the unknown pivotal values of $w$ along the first row of $j=1$ in terms of the known boundary conditions.

This can be written in matrix form
\begin{equation} A\mathbf{w}_{j+1}=B\mathbf{w}_{j} +\mathbf{b}_{j}+\mathbf{b}_{j+1} \end{equation}
for which $A$ is a $9\times9$ matrix:
\begin{equation}
\left(\begin{array}{cccc cccc}
2+2r&-r& 0&0&0 &0&0&0\\
-r&2+2r&-r&0&0&0 &0&0&0\\
0&-r&2+2r &-r&0&0& 0&0&0\\
0&0&-r&2+2r &-r&0&0& 0&0\\
0&0&0&-r&2+2r &-r&0&0& 0\\
0&0&0&0&-r&2+2r &-r&0&0\\
0&0&0&0&0&-r&2+2r &-r&0\\
0&0&0&0&0&0&-r&2+2r&-r\\
0&0&0&0&0&0&0&-r&2+2r\\
\end{array}\right)
\left(\begin{array}{c}
w_{1j+1}\\
w_{2j+1}\\
w_{3j+1}\\
w_{4j+1}\\
w_{5j+1}\\
w_{6j+1}\\
w_{7j+1}\\
w_{8j+1}\\
w_{9j+1}\\
\end{array}\right)=
\left(\begin{array}{cccc cccc}
2-2r&r& 0&0&0 &0&0&0\\
r&2-2r&r&0&0&0 &0&0&0\\
0&r&2-2r &r&0&0& 0&0&0\\
0&0&r&2-2r &r&0&0& 0&0\\
0&0&0&r&2-2r &r&0&0& 0\\
0&0&0&0&r&2-2r &r&0&0\\
0&0&0&0&0&r&2-2r &r&0\\
0&0&0&0&0&0&r&2-2r&r\\
0&0&0&0&0&0&0&r&2-2r\\
\end{array}\right)
\left(\begin{array}{c}
w_{1j}\\
w_{2j}\\
w_{3j}\\
w_{4j}\\
w_{5j}\\
w_{6j}\\
w_{7j}\\
w_{8j}\\
w_{9j}\\
\end{array}\right)+
\left(\begin{array}{c}
rw_{0j}\\
0\\
0\\
0\\
0\\
0\\
0\\
0\\
rw_{10j}\\
\end{array}\right)+
\left(\begin{array}{c}
rw_{0j+1}\\
0\\
0\\
0\\
0\\
0\\
0\\
0\\
rw_{10j+1}\\
\end{array}\right).
\end{equation}
It is assumed that the boundary values $w_{0j}$ and $w_{10j}$ are known for $j=1,2,...$, and $w_{i0}$ for $i=0,...,10$ is the initial condition.

The Figure below shows the matrix $A$ and its inverse $A^{-1}$ in color plot form for $r$.

```python
A=np.zeros((N-1,N-1))
B=np.zeros((N-1,N-1))
for i in range (0,N-1):
    A[i,i]=2+2*r
    B[i,i]=2-2*r

for i in range (0,N-2):
    A[i+1,i]=-r
    A[i,i+1]=-r
    B[i+1,i]=r
    B[i,i+1]=r

Ainv=np.linalg.inv(A)
fig = plt.figure(figsize=(12,4));
plt.subplot(121)
plt.imshow(A,interpolation='none');
plt.xticks(np.arange(N-1), np.arange(1,N-0.9,1));
plt.yticks(np.arange(N-1), np.arange(1,N-0.9,1));
clb=plt.colorbar();
clb.set_label('Matrix elements values');
plt.title('Matrix A r=%s'%(np.round(r,3)),fontsize=24)

plt.subplot(122)
plt.imshow(B,interpolation='none');
plt.xticks(np.arange(N-1), np.arange(1,N-0.9,1));
plt.yticks(np.arange(N-1), np.arange(1,N-0.9,1));
clb=plt.colorbar();
clb.set_label('Matrix elements values');
plt.title(r'Matrix $B$ r=%s'%(np.round(r,3)),fontsize=24)
fig.tight_layout()
plt.show();
```

## Results
To numerically approximate the solution at $t[1]$ the matrix equation becomes
\begin{equation} \mathbf{w}_{1}=A^{-1}(B\mathbf{w}_{0} +\mathbf{b}_{1}+\mathbf{b}_{0})\end{equation}
where all the right hand side is known.
To approximate solution at time $t[2]$ we use the matrix equation
\begin{equation} \mathbf{w}_{2}=A^{-1}(B\mathbf{w}_{1}+\mathbf{b}_{2} +\mathbf{b}_{1}). \end{equation}
Each set of numerical solutions $w[i,j]$ for all $i$ at the previous time step is used to approximate the solution $w[i,j+1]$.
The left and right plot below show the numerical approximation $w[i,j]$ of the Heat Equation using the BTCS method for $x[i]$ for $i=0,...,10$ and time steps $t[j]$ for $j=1,...,15$.

```python
for j in range (1,time_steps+1):
    b[0]=r*w[0,j-1]+r*w[0,j]
    b[N-2]=r*w[N,j-1]+r*w[N,j]
    v=np.dot(B,w[1:(N),j-1])
    w[1:(N),j]=np.dot(Ainv,v+b)

fig = plt.figure(figsize=(10,6));
plt.imshow(w.transpose(), aspect='auto')
plt.xticks(np.arange(len(x)), x)
plt.yticks(np.arange(len(time)), time)
plt.xlabel('x')
plt.ylabel('time')
clb=plt.colorbar()
clb.set_label('Temperature (w)')
plt.suptitle('Numerical Solution of the  Heat Equation r=%s'%(np.round(r,3)),fontsize=24,y=1.08)
fig.tight_layout()
plt.show()
```

```python

```

```python

```

Chapter 08 - Heat Equations/804_Heat Equation-CN-Different_Initial_Boundary.ipynb