# Introduction to TensorFlow
# Notebook - Automatic Differentiation

*Disclosure: This notebook is an adaptation of Toronto's Neural Networks and Deep Learning Course (CSC421) tutorial material.*

## Setup of python packages
Google Colaboratory comes with a preinstalled Python runtime

```
# Show used python version
import sys
print(f"Used python version: {sys.version}")
```

    Used python version: 3.6.9 (default, Apr 18 2020, 01:56:04)
    [GCC 8.4.0]

Install any python package via pip within the virtual machine provided by colab using

    !pip install <package>

Note: each time the runtime is reset, you have to reinstall all required packages.

```
# Install symbolic python package
!pip install sympy
```

    Requirement already satisfied: sympy in /usr/local/lib/python3.6/dist-packages (1.1.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy) (1.1.0)

```
# Import the installed package
import sympy as sp
```

## Approaches for computing derivatives

* **Numeric differentiation:** Approximating derivatives by finite differences:
$$
\frac{\partial}{\partial x_i} f(x_1, \dots, x_N) = \lim_{h \to 0} \frac{f(x_1, \dots, x_i + h, \dots, x_N) - f(x_1, \dots, x_i - h, \dots, x_N)}{2h}
$$

* **Symbolic differentiation:** automatic manipulation of mathematical expressions to get derivatives
    - Takes a math expression (e.g. sigmoid) and returns a math expression: $$\sigma(x) = \frac{1}{e^{-x} + 1} \rightarrow \frac{d\sigma(x)}{dx} = \frac{e^{-x}}{(e^{-x} + 1)^2} = \sigma(x)(1 - \sigma(x)$$
    - Used in SymPy or Mathematica

```
# Use SymPy for symbolic differentiation
# E.g. sigmoid of convolution of two inputs
x_1, x_2, w_1, w_2, b_1 = sp.symbols('x_1, x_2, w_1, w_2, b_1', real=True)
g = 1 / (1 + sp.exp(-(w_1 * x_1 + w_2 * x_2 + b_1)))

print(f'Mathematical Expression: $$g(x) = {sp.latex(g)}$$')
print(f'Partial Derivatives:')
for var in [x_1, x_2]:
    print(f'$$ \\frac{{\\partial g}}{{\\partial {str(var)} }} = {sp.latex(sp.simplify(g.diff(var)))} $$')
```

    Mathematical Expression: $$g(x) = \frac{1}{e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}} + 1}$$
    Partial Derivatives:
    $$ \frac{\partial g}{\partial x_1 } = \frac{w_{1} e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}}}{\left(e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}} + 1\right)^{2}} $$
    $$ \frac{\partial g}{\partial x_2 } = \frac{w_{2} e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}}}{\left(e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}} + 1\right)^{2}} $$

If posted to a text cell, the code snippet generates the following latex expressions.

Mathematical Expression: $$g(x) = \frac{1}{e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}} + 1}$$
Partial Derivatives:
$$ \frac{\partial g}{\partial x_1 } = \frac{w_{1} e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}}}{\left(e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}} + 1\right)^{2}} $$
$$ \frac{\partial g}{\partial x_2 } = \frac{w_{2} e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}}}{\left(e^{- b_{1} - w_{1} x_{1} - w_{2} x_{2}} + 1\right)^{2}} $$

* **Automatic differentiation:** Takes code that computes a function and returns code that computes the derivative of that function.
    - Reverse Mode AD: A method to get exact derivatives efficiently, by storing information as you go forward that you can reuse as you go backwards
    - The goal isn't to obtain closed-form solutions, but to be able to wirte a program that efficiently computes the derivatives (Backpropagation)

## Autograd

* [Autograd](https://github.com/HIPS/autograd) is a Python package for automatic differentiation.
* There are a lot of great [examples](https://github.com/HIPS/autograd/tree/master/examples) provided with the source code

### What can Autograd do?

From the Autograd Github repository:

* Autograd can automatically differentiate native Python and Numpy code.
* It can handle a large subset of Python's features, including loops, conditional statements (if/else), recursion and closures
* It can also compute higher-order derivatives
* It uses reverse-mode differentiation (a.k.a. backpropagation) so it can efficiently take gradients of scalar-valued functions with respect to array-valued arguments.

### Autograd vs Deep Learning Frameworks

Many Deep Learning packages implement automatic differentiation using _domain-specific languages_ within Python. Older versions, such as TensorFlow 1.X, required you to _explicitly_ construct a computation graph; Autograd constructs a computation graph _implicitly_, by tracking the sequence of operations that have been performed during the execution of a program.

Note: There is no direct GPU support for Autograd. If you're interested in automatic differentiation with support for hardware accelerators have a look at [JAX](https://github.com/google/jax). It is a successor that provides Just-in-Time compilation like PyTorch or TensorFlow.

## Autograd Basic Usage

Autograd wraps the NumPy package providing an almost identical API to the NumPy functionality, but performs additional bookkeeping in the background to build the computation graph.

```
# Install autograd
!pip install autograd
```

    Requirement already satisfied: autograd in /usr/local/lib/python3.6/dist-packages (1.3)
    Requirement already satisfied: numpy>=1.12 in /usr/local/lib/python3.6/dist-packages (from autograd) (1.18.4)
    Requirement already satisfied: future>=0.15.2 in /usr/local/lib/python3.6/dist-packages (from autograd) (0.16.0)

```
# Import autograd
import autograd.numpy as np # Import thinly-wrapped NumPy
from autograd import grad   # The only function of Autograd, you need to call
```

```
# Define a function like normal, using Python and Numpy
def tanh(x):
    y = np.exp(-x)
    return (1.0 - y) / (1.0 + y)

# Create a *function* that computes the gradient of tanh
grad_tanh = grad(tanh)  # autograd.grad takes a function as input

# Evaluate the gradient at x = 1.0
print(f'Autograd gradient:  {grad_tanh(1.0)}')

# Compare to numeric gradient computed using finite differences
h = 0.0001
print(f'Finite differences: {(tanh(1.0001) - tanh(0.9999)) / (2*h)}')
```

    Autograd gradient:  0.39322386648296376
    Finite differences: 0.39322386636453377

## Autograd vs Manual Gradients via Staged Computation

In this example, we will see how the computation of a function can be written as a composition of simpler functions. This provides a scalable strategy for computing gradients using the chain rule.

Say we want to write a function to compute the gradient of the *sigmoid function*:
$$
\sigma(x) = \frac{1}{e^{-x} + 1}
$$
We can write $\sigma(x)$ as a composition of several elementary functions, as $\sigma(x) = s(c(b(a(x))))$, where:

$$
a(x) = -x
$$

$$
b(a) = e^a
$$

$$
c(b) = 1 + b
$$

$$
s(c) = \frac{1}{c}
$$

Here, we have "staged" the computation such that it contains several intermediate variables, each of which are basic expressions for which we can easily compute the local gradients.

The input to this function is $x$, and the final output is represented by $s$. We wish compute the gradient of $d$ with respect to $x$, $\frac{\partial s}{\partial x}$. In order to make use of our intermediate computations, we can use the chain rule as follows:
$$
\frac{\partial s}{\partial x} = \frac{\partial s}{\partial c} \frac{\partial c}{\partial b} \frac{\partial b}{\partial a} \frac{\partial a}{\partial x}
$$

```
def grad_sigmoid_manual(x):
    """Implements the gradient of the logistic sigmoid function
    $\sigma(x) = 1 / (1 + e^{-x})$ using staged computation
    """
    # Forward pass, keeping track of intermediate values for use in the
    # backward pass
    a = -x         # -x in denominator
    b = np.exp(a)  # e^{-x} in denominator
    c = 1 + b      # 1 + e^{-x} in denominator
    s = 1.0 / c    # Final result, 1.0 / (1 + e^{-x})

    # Backward pass
    dsdc = (-1.0 / (c**2))
    dsdb = dsdc * 1
    dsda = dsdb * np.exp(a)
    dsdx = dsda * (-1)

    return dsdx

def sigmoid(x):
    y = 1.0 / (1.0 + np.exp(-x))
    return y

# Instead of writing grad_sigmoid_manual manually, we can use
# Autograd's grad function:
grad_sigmoid_automatic = grad(sigmoid)

# Compare the results of manual and automatic gradient functions:
print(f'Autograd gradient:  {grad_sigmoid_automatic(2.0)}')
print(f'Manual gradient:    {grad_sigmoid_manual(2.0)}')
```

    Autograd gradient:  0.1049935854035065
    Manual gradient:    0.1049935854035065

# Example

## Linear Regression with Autograd

The next section of the notebook shows an example of using Autograd in the context of **1-D linear regression** by gradient descent.

We try to fit a model to a function $y = wx + b$

### Review

We are given a set of data points $\{ (x_1, t_1), (x_2, t_2), \dots, (x_N, t_N) \}$, where each point $(x_i, t_i)$ consists of an *input value* $x_i$ and a *target value* $t_i$.

The **model** we use is:
$$
y_i = wx_i + b
$$

We want each predicted value $y_i$ to be close to the ground truth value $t_i$. In linear regression, we use squared error to quantify the disagreement between $y_i$ and $t_i$. The **loss function** for a single example is:
$$
\mathcal{L}(y_i,t_i) = \frac{1}{2} (y_i - t_i)^2
$$

The **cost function** is the loss averaged over all the training examples:
$$
\mathcal{C}(w,b) = \frac{1}{N} \sum_{i=1}^N \mathcal{L}(y_i, t_i) = \frac{1}{N} \sum_{i=1}^N \frac{1}{2} \left(wx_i + b - t_i \right)^2
$$

```
import autograd.numpy as np # Import wrapped NumPy from Autograd
from autograd import grad # To compute gradients

import matplotlib.pyplot as plt # Most common plotting / data visualization tool in Python
```

## Generate Synthetic Data

We generate a synthetic dataset $\{ (x_i, t_i) \}$ by first taking the $x_i$ to be linearly spaced in the range $[0, 1]$ and generating the corresponding value of $t_i$ using the following equation (where $w = 2$ and $b=0.5$):
$$
t_i = 2 x_i + 0.5 + \epsilon
$$

Here, $\epsilon \sim \mathcal{N}(0, 0.01)$ (that is, $\epsilon$ is drawn from a Gaussian distribution with mean 0 and variance 0.01). This introduces some random fluctuation in the data, to mimic real data that has an underlying regularity, but for which individual observations are corrupted by random noise.

```
# In our synthetic data, we have w = 2 and b = 0.5
N = 100 # Number of training data points
x = np.random.uniform(size=(N,))
eps = np.random.normal(size=(len(x),), scale=0.1)
t = 2.0 * x + 0.5 + eps
plt.plot(x, t, 'r.')
```

```
# Initialize random parameters
w = np.random.normal(0, 1)
b = np.random.normal(0, 1)
params = { 'w': w, 'b': b } # One option: aggregate parameters in a dictionary

def cost(params):
    y = params['w'] * x + params['b']
    return (1 / N) * np.sum(0.5 * np.square(y - t))

# Find the gradient of the cost function using Autograd
grad_cost = grad(cost)

num_epochs = 2000  # Number of epochs of training
alpha = 0.025       # Learning rate

for i in range(num_epochs):
    # Evaluate the gradient of the current parameters stored in params
    cost_params = grad_cost(params)

    # Gradient Descent step
    # Update parameters w and b
    params['w'] = params['w'] - alpha * cost_params['w']
    params['b'] = params['b'] - alpha * cost_params['b']

print(params)
```

    {'w': 1.926380013972517, 'b': 0.5288309549899681}

```
# Plot the training data again, together with the line defined by y = wx + b
# where w and b are our final learned parameters
plt.plot(x, t, 'r.')
plt.plot([0, 1], [params['b'], params['w'] + params['b']], 'b-')
```


```python
total = 0
for i in range(10):
    total = total + i
total
```

    45

```python
for i in range(3):
    print(i)
```

    0
    1
    2

$$\int_0^{+\infty} x^2 dx$$

$$ z = \frac{x}{y} $$

$$ P(A \mid B) = \frac{P(B \mid A) , P(A)}{P(B)} $$

$$ f(x) = x^2 $$

$ \sqrt 3 $

```python
def proportion(a, b):
    """
    We can use triple " to describe what our function does.

    Here for example: we're creating a function to calculate
    the proportion of a of a + b
    """
    return a / (a + b)
```

```python
proportion?
```

```python
proportion(20, 21)
```

    0.4878048780487805

```python
import sympy as sym
```

```python
x = sym.symbols('x')
```

```python
x-x
```

    0

```python
a, b = sym.symbols('a, b')
```

```python
expr = (a + b) ** 2
expr
```

    (a + b)**2

```python
sym.init_printing()
```

```python
expr
```

```python
expr.expand()
```

```python
sym.latex(expr.expand())
```

    'a^{2} + 2 a b + b^{2}'

```python
a, b, c, x = sym.symbols('a, b, c, x')
```

```python
sym.solveset(a * x ** 2 + b * x + c, x)
```

```python
sym.solveset(x ** 2 + 1, x, domain=sym.S.Reals)
```

```python
sym.limit(1/x, x, 0, dir="+")
```

```python
sym.diff(x ** 2 - sym.cos(x), x)
```

```python
sym.integrate(x ** 2 - sym.cos(x), x)
```

```python
sym.integrate(x ** 2 - sym.cos(x), (x, 0, 5))
```

```python
%matplotlib inline
```

```python
import matplotlib.pyplot as plt
import numpy as np

x = np.arange(20)
y = x**2

plt.plot(x, y)
```

```python
expr = x ** 2
p = sym.plot(expr);
```

```python
p.save("x_squared.pdf");
```

$$ A = \begin{pmatrix} 5 & 6 & 2 \\4 & 7 & 19 \\0 & 3 & 12 \end{pmatrix} $$
$$ B = \begin{pmatrix} 14 & -2 & 12\\ 4 & 4 & 5\\5 & 5 & 1\end{pmatrix} $$

import numpy as np

```python
A = np.matrix([[5, 6, 2],
               [4, 7, 19],
               [0, 3, 12]])
```

```python
A
```

    matrix([[ 5,  6,  2],
            [ 4,  7, 19],
            [ 0,  3, 12]])

```python
B = np.matrix([[14, -2, 12],
               [4, 4, 5],
               [5, 5, 1]])
```

```python
B
```

    matrix([[14, -2, 12],
            [ 4,  4,  5],
            [ 5,  5,  1]])

```python
5 * A
```

    matrix([[25, 30, 10],
            [20, 35, 95],
            [ 0, 15, 60]])

```python
A ** 3
```

    matrix([[ 557, 1284, 3356],
            [ 760, 2305, 6994],
            [ 288, 1074, 3519]])

```python
A + B
```

    matrix([[19,  4, 14],
            [ 8, 11, 24],
            [ 5,  8, 13]])

```python
A - B
```

    matrix([[ -9,   8, -10],
            [  0,   3,  14],
            [ -5,  -2,  11]])

```python
A * B
```

    matrix([[104,  24,  92],
            [179, 115, 102],
            [ 72,  72,  27]])

$$ A = \begin{pmatrix} 5 & 6 & 2 \\4 & 7 & 19 \\0 & 3 &12 \end{pmatrix} $$
$$ b = \begin{pmatrix} -1 \\2 \\1 \end{pmatrix} $$

```python
A = np.matrix([[5, 6, 2],
               [4, 7, 19],
               [0, 3, 12]])
b = np.matrix([[-1], [2], [1]])
```

```python
x = np.linalg.solve(A, b)
x
```

    matrix([[ 0.45736434],
            [-0.62790698],
            [ 0.24031008]])

```python
A * x
```

    matrix([[-1.],
            [ 2.],
            [ 1.]])

```python
Ainv = np.linalg.inv(A)
Ainv
```

    matrix([[-0.20930233,  0.51162791, -0.7751938 ],
            [ 0.37209302, -0.46511628,  0.6744186 ],
            [-0.09302326,  0.11627907, -0.08527132]])

```python
A * Ainv
```

    matrix([[  1.00000000e+00,   4.99600361e-16,  -4.71844785e-16],
            [  1.38777878e-16,   1.00000000e+00,  -2.63677968e-16],
            [  1.11022302e-16,   1.11022302e-16,   1.00000000e+00]])

```python
np.linalg.det(A)
```

```python
import pandas as pd
```

```python
df = pd.read_excel("data/goldbach.xlsx")
```

```python
df.head()
```

      N
      a
      b

      0
      4
      2
      2

      1
      6
      3
      3

      2
      8
      3
      5

      3
      10
      3
      7

      4
      10
      5
      5

```python
df.tail()
```

      N
      a
      b

      2591
      500
      151
      349

      2592
      500
      163
      337

      2593
      500
      193
      307

      2594
      500
      223
      277

      2595
      500
      229
      271

```python
df.describe()
```

      N
      a
      b

      count
      2596.000000
      2596.000000
      2596.000000

      mean
      311.466872
      72.297766
      239.169106

      std
      127.817425
      58.087254
      111.180401

      min
      4.000000
      2.000000
      2.000000

      25%
      214.000000
      23.000000
      157.000000

      50%
      329.000000
      59.000000
      239.000000

      75%
      420.000000
      107.000000
      317.000000

      max
      500.000000
      241.000000
      491.000000

```python
df[df['N'] == 322]
```

      N
      a
      b

      1250
      322
      5
      317

      1251
      322
      11
      311

      1252
      322
      29
      293

      1253
      322
      41
      281

      1254
      322
      53
      269

      1255
      322
      59
      263

      1256
      322
      71
      251

      1257
      322
      83
      239

      1258
      322
      89
      233

      1259
      322
      131
      191

      1260
      322
      149
      173

```python
s = df['N'].value_counts()
s
```

    420    30
    480    29
    462    28
    450    27
    390    27
    456    24
    468    24
    330    24
    486    23
    474    23
    498    23
    360    22
    492    22
    378    22
    414    21
    444    21
    300    21
    438    21
    426    21
    396    21
    324    20
    408    20
    354    20
    490    19
    336    19
    294    19
    210    19
    270    19
    432    19
    384    19
           ..
    58      4
    80      4
    88      4
    92      4
    152     4
    42      4
    26      3
    30      3
    24      3
    40      3
    22      3
    98      3
    52      3
    56      3
    62      3
    128     3
    44      3
    38      2
    18      2
    10      2
    14      2
    16      2
    20      2
    68      2
    28      2
    32      2
    12      1
    8       1
    6       1
    4       1
    Name: N, dtype: int64

```python
df = pd.DataFrame(s.rename('counts'))
df.head()
```

      counts

      420
      30

      480
      29

      462
      28

      450
      27

      390
      27

```python
df["normalised_counts"] = df['counts'] / df.index
```

```python
df.head()
```

      counts
      normalised_counts

      420
      30
      0.071429

      480
      29
      0.060417

      462
      28
      0.060606

      450
      27
      0.060000

      390
      27
      0.069231

```python
df.describe()
```

      counts
      normalised_counts

      count
      249.000000
      249.000000

      mean
      10.425703
      0.051769

      std
      5.897948
      0.029921

      min
      1.000000
      0.017588

      25%
      6.000000
      0.030973

      50%
      9.000000
      0.044776

      75%
      13.000000
      0.062500

      max
      30.000000
      0.250000

```python
%matplotlib inline
```

```python
df['counts'].hist();
```

```python
df['normalised_counts'].hist();
```

```python
df.to_csv("data/goldbach_counts.csv")
```

```python
import numpy as np
from sympy import *
from __future__ import division

init_printing()
```

```python
x = Symbol("x")
k = Symbol("k")
y = Symbol("y")

b = Symbol("b")

t = sin(2*pi*x*(k**2))/ (4*(pi**2)*(k**5)) + (x**2) / (2*k)

t
```

```python
t1 = diff(t,x)

t1
```

```python
t2 = diff(t1,x)

t2
```

```python
def sumatoria(a, termino, N=100):
    return sum(termino.subs(x,a).subs(k,b+1) for b in range(N))

def f(a):
    return sumatoria(a,t)

def f1(a):
    return sumatoria(a,t1)

def f2(a):
    return sumatoria(a,t2)
```

```python
plot(f(y), (y,0,1))

show()
```

```python
plot(f1(y), (y,0,1))

show()
```

```python
plot(f2(y), (y,0,1))

show()
```

```python
def CalcularyGraficar(t):
    t1 = diff(t,x)
    t2 = diff(t1,x)
    def f(a):
        return sumatoria(a,t)

    def f1(a):
        return sumatoria(a,t1)

    def f2(a):
        return sumatoria(a,t2)

    plot(f(y), (y,0,1))
    plot(f1(y), (y,0,1))
    plot(f2(y), (y,0,1))

    show()
```

```python
t = sin(2*pi*x*(k**2))/ (4*(pi**2)*(k**3)) + (x**2) / (2*k)

CalcularyGraficar(t)
```

```python
plot(f(y), (y,0,0.0001))

show()
```

```python
t = sin(2*pi*x*(k**2))/ (4*(pi**2)*(k**5))

CalcularyGraficar(t)
```

```python
t = sin(2*pi*x*(k**2))/ (4*(pi**2)*(k)) + (x**2) / (2*k)

CalcularyGraficar(t)
```

```python
t = sin(2*pi*x*(k**2))/ (4*(pi**2)*(k**10)) + (x**2) / (2*k)

CalcularyGraficar(t)
```

```python
%matplotlib inline
import numpy as np
import scipy as sp
import scipy.linalg as la
import matplotlib.pyplot as plt
```

```python
import scipy.integrate as sint

A = np.array([[1, -20],
              [3, 4]], dtype=np.double)

def yprime(y, t):
    return sp.dot(A, y)

t = np.linspace(0, 1, 51)
yzero = np.array([1, 0])

# Solve equation using odeint
y = sint.odeint(yprime, yzero, t) # Calls the Fortran library odeint
plt.plot(y[:, 0], y[:, 1]) # Blue line is y(t) for t in [0, 1]

plt.xlim([-4, 9])
plt.ylim([-3, 5])

# Solve for 10 t values using the exponential
tvals = np.linspace(0, 1, 11)

for tval in tvals:
    yval = sp.dot(la.expm(A*tval), yzero)
    plt.plot(yval[0], yval[1], 'rs') # Red squares are evaluated via the matrix exponential
```

```python
import networkx as nx

Adj = np.array([[0, 1, 1, 0, 0, 0],
                [1, 0, 1, 1, 1, 0],
                [1, 1, 0, 0, 0, 0],
                [0, 1, 0, 0, 0, 1],
                [0, 1, 0, 0, 0, 0],
                [0, 0, 0, 1, 0, 0]])

G = nx.from_numpy_matrix(Adj)
nx.draw(G, node_color='y', node_size=1000)
```

```python
centralities = np.diag(la.expm(np.array(Adj, dtype=np.double)))
nodeorder = np.argsort(centralities)[::-1]

print(np.array([nodeorder, centralities[nodeorder]]))

# Note: This is already built into networkx using the following command
# print nx.communicability_centrality_exp(G)
```

    [[ 1.          2.          0.          3.          4.          5.        ]
     [ 4.44723536  2.86427609  2.86427609  2.36018456  1.71615913  1.59432922]]

```python

```


# Clustering Exercises
This jupyter notebook provides some exercises with regard to clustering.
## Distance
A clustering algorithm often uses some notion of distance between datapoints to divide the datapoints in clusters. Therefore as a first exercise, we will define some definition of distance. An often used distance measure is the $L^p$ norm. The $L^p$ norm is often defined as:

\begin{align}
\left\| \textbf{x} \right\|_p := \left( \sum_{i=1}^{n} \lvert x_i \rvert^p \right)^{1/p}
\end{align}

In this definition, the most often chosen values for $p$ are 1, 2 and $\infty$.
- The $L^1$ norm is ofthen called the Taxicab norm or the Manhattan norm: see [Taxicab geometry](https://en.wikipedia.org/wiki/Taxicab_geometry).
- The $L^2$ norm is the same as the Euclidian distance.
- The $L^\infty$ norm is the same as the max(.) operator.

Now to practice some basic python and numpy, implement the euclidian distance using only simple matrix operations, in  `./code/utils.py`

```python
import numpy as np

import sys
sys.path.append('code/')

import utils

x = np.asarray([1,2,3])[:,np.newaxis]
y = np.asarray([7,8,9])[:,np.newaxis]

L2 = utils.euclidian_distance(x,y)
#print(L2)
```

## K-means
In this section we will try to implement a simple k-means clustering algorithm. To start we will make a dataset with *three* gaussian distributions.

```python
import numpy as np
import matplotlib.pyplot as plt

import sys
sys.path.append('code/')

import utils

def generate_k_gaussian_distributions(k, dimensions=2, number_of_datapoints=150, min_value=0, max_value=20, min_sigma=1, max_sigma=2):
        # Allocate memory
        dataset = np.zeros((k*number_of_datapoints, dimensions))
        labels = np.zeros((k*number_of_datapoints, 1))

        # Fill dataset and labels with d dimensional data
        for i in range(1,k+1):
            x = np.random.randint(min_value, max_value)
            y = np.random.randint(min_value, max_value)
            sigma = utils.random_float(min_sigma, max_sigma)
            dataset[(i-1)*number_of_datapoints:i*number_of_datapoints] = np.random.normal((x,y), scale=sigma, size=(number_of_datapoints, dimensions))
            labels[(i-1)*number_of_datapoints:i*number_of_datapoints] = (i-1)

        # Return the data and labels
        return dataset, labels

# Seed at 10 does not give to much overlap between samples
# Thus is a nice starting point for this exercise
np.random.seed(10)
X, y = generate_k_gaussian_distributions(3)

plt.plot(X[:,0], X[:,1], 'r.')
```

For this exercise we are going to assume that we already found out how many cluster we need. Namely, $k=3$. Implement the fitting of the k-means algorithm in `./code/kmeans.py` and implement the prediction of the targets. Try and implement with basic matrix operations, speed in this case is not a priority. Some hints are given.

```python
import sys
sys.path.append('code/')

import kmeans

from sklearn.metrics import adjusted_rand_score
import matplotlib.pyplot as plt

k=3
alpha=0.01

# set a convenient value for alpha, alpha is a cut off value
# e.g. shutdown the algorithm if the means are not changing anymore
# alpha = ?
mu = kmeans.fit(X, k, alpha)
y_predict = kmeans.predict(X, mu)

# Try to evaluate the performance of y_predict
# mind that the classes of y_predict are assigned randomly
# e.g. the classes of y and y_predict might be different
# but the clusters can still be correct

# Try to plot the clusters to evaluate

```

## Ideas for additional topics
### Hierarchical clustering


# HEWL S-SAD Merging Statistics

Merging statistics are a useful means to assess data quality in crystallography. However, each statistic has inherent shortcomings. For example, R-merge will appear inflated if the multiplicity is high, and the Pearson correlation coefficients used for $CC_{1/2}$ are very sensitive to outliers.

Most scaling and merging programs output multiple merging statistics to get around these shortcomings. However, one can imagine that it could also be useful to customize certain parameters, such as how many resolution bins are used. Or, perhaps a better statistic will be developed that is worth implementing.

In this notebook, ``reciprocalspaceship`` is used to compute half-dataset correlation coefficients ($CC_{1/2}$ and $CC_{anom}$) for a dataset collected from a tetragonal hen egg-white lysozyme (HEWL) crystal at 6550 eV.
These data are unmerged, but were scaled in AIMLESS.
They contain sufficient sulfur anomalous signal to determine a solution by the SAD method.
This illustrates the use of ``rs`` to implement a merging routine, create a custom analysis, and could be useful as a template for other exploratory crystallographic data analyses.
As an example, we will compare half-dataset correlations computed using both Pearson and Spearman correlation coefficients.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_context("notebook", font_scale=1.3)
import numpy as np
```

```python
import reciprocalspaceship as rs
```

```python
print(rs.__version__)
```

    0.9.9

---
### Load scaled, unmerged data

This data has been scaled in AIMLESS. The data includes the image number and the scaled **I** and **SIGI** values.

```python
hewl = rs.read_mtz("data/HEWL_unmerged.mtz")
```

```python
hewl.head()
```

      BATCH
      I
      SIGI
      PARTIAL

      H
      K
      L

      0
      0
      4
      137
      696.5212
      87.83294
      False

      4
      520
      710.6812
      88.107025
      False

      4
      856
      672.05634
      87.75671
      False

      4
      1239
      642.47485
      87.90302
      False

      4
      2160
      655.71783
      87.74394
      False

```python
print(f"Number of observed reflections: {len(hewl)}")
```

    Number of observed reflections: 816804

---
### Merging with Inverse-Variance Weights

Since the input data are unmerged, we will implement the inverse-variance weighting scheme used by AIMLESS to merge the observations.
The weighted average is a better estimator of the true mean than the raw average, and this weighting scheme corresponds to the maximum likelihood estimator of the true mean if we assume that the observations are normally-distributed about the true mean.

The merged intensity for each reflection, $I_h$, can be determined from the observed intensities, $I_{h,i}$, and error estimates, $\sigma_{h,i}$, as follows:

\begin{equation}
I_h = \frac{\sum_{i}w_{h,i} I_{h,i}}{\sum_{i} w_{h,i}}
\end{equation}

where the weight for each observation, $w_{h,i}$ is given by:

\begin{equation}
w_{h,i} = \frac{1}{(\sigma_{h,i})^2}
\end{equation}

The updated estimate of the uncertainty, $\sigma_{h}$, is given by:

\begin{equation}
\sigma_{h} = \sqrt{\frac{1}{\sum_{i} w_{h,i}}}
\end{equation}

Let's start by implementing the above equations in a function that will compute the merged $I_h$ and $\sigma_h$. We will use the [Pandas groupby](https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html) methods to apply this function across the unique Miller indices in the ``DataSet``. If we group Friedel pairs together, we will refer to the quantity as **IMEAN**, and if we keep the Friedel pairs separate we will refer to the quantities as **I(+)**  and **I(-)**.

```python
def merge(dataset, anomalous=False):
    """
    Merge dataset using inverse-variance weights.

    Parameters
    ----------
    dataset : rs.DataSet
        DataSet to be merged containing scaled I and SIGI
    anomalous : bool
        If True, I(+) and I(-) will be reported. If False,
        IMEAN will be reported

    Returns
    -------
    rs.DataSet
        Merged DataSet object
    """
    ds = dataset.hkl_to_asu(anomalous=anomalous)
    ds["w"]  = ds['SIGI']**-2
    ds["wI"] = ds["I"] * ds["w"]
    g = ds.groupby(["H", "K", "L"])

    result = g[["w", "wI"]].sum()
    result["I"] = result["wI"] / result["w"]
    result["SIGI"] = np.sqrt(1 / result["w"])
    result = result.loc[:, ["I", "SIGI"]]
    result.merged = True

    if anomalous:
        result = result.unstack_anomalous()

    return result
```

Using `anomalous=False`, this function can be used to compute **IMEAN** and **SIGIMEAN** by including both Friedel pairs:

```python
result1 = merge(hewl, anomalous=False)
```

```python
result1.sample(5)
```

      I
      SIGI

      H
      K
      L

      23
      22
      9
      1349.8007
      22.878157

      37
      19
      0
      260.0996
      9.075029

      17
      11
      15
      146.02324
      2.895325

      35
      1
      14
      9.629998
      3.902024

      15
      5
      18
      7.837115
      1.039951

Using `anomalous=True`, this function can be used to compute **I(+)**, **SIGI(+)**, **I(-)**, and **SIGI(-)** by separating Friedel pairs:

```python
result2 = merge(hewl, anomalous=True)
```

```python
result2.sample(5)
```

      I(+)
      SIGI(+)
      SIGI(-)
      I(-)

      H
      K
      L

      20
      4
      7
      790.00696
      13.43943
      13.45224
      819.43054

      21
      15
      16
      51.543583
      2.840802
      2.675156
      50.629883

      35
      8
      12
      56.64869
      2.762924
      2.779431
      52.87201

      16
      16
      15
      517.4729
      11.513449
      11.513449
      517.4729

      33
      23
      3
      101.954094
      3.753756
      3.605523
      89.131714

A variant of the above function is implemented in `rs.algorithms`, and we will use that implementation in the next section for computing merging statistics. This function computes **IMEAN**, **I(+)**, **I(-)**, and associated uncertainties for each unique Miller index.

```python
result3 = rs.algorithms.merge(hewl)
```

```python
result3.sample(5)
```

      IMEAN
      SIGIMEAN
      I(+)
      SIGI(+)
      I(-)
      SIGI(-)
      N(+)
      N(-)

      H
      K
      L

      38
      10
      4
      454.0382
      11.617793
      416.8525
      21.370535
      469.6386
      13.841894
      8
      20

      17
      9
      15
      101.308815
      2.1820927
      100.45327
      3.0412343
      102.21658
      3.1326878
      32
      31

      38
      7
      8
      4.571434
      0.9255639
      4.8481956
      1.2750458
      4.2631493
      1.3457003
      20
      20

      11
      5
      5
      3991.629
      47.690647
      4061.3477
      66.316765
      3916.9563
      68.63234
      60
      56

      27
      18
      11
      14.8991
      1.0327507
      10.999734
      1.4392322
      19.038132
      1.4828024
      20
      20

---
### Merging with 2-fold Cross-Validation

To compute correlation coefficients we will repeatedly split our data into half-datasets. We will do this by randomly splitting the data using the image number. These half-datasets will be merged independently and used to determine uncertainties in the correlation coefficients. We will first write a method to randomly split our data, and then we will then write a method that automates the sampling and merging of multiple half-datasets.

```python
def sample_halfdatasets(data):
    """Randomly split DataSet into two equal halves by BATCH"""
    batch = data.BATCH.unique().to_numpy(dtype=int)
    np.random.shuffle(batch)
    halfbatch1, halfbatch2 = np.array_split(batch, 2)
    half1 = data.loc[data.BATCH.isin(halfbatch1)]
    half2 = data.loc[data.BATCH.isin(halfbatch2)]
    return half1, half2
```

```python
def merge_dataset(dataset, nsamples):
    """
    Merge DataSet using inverse-variance weighting scheme. This represents the
    maximum-likelihood estimator of the mean of the observed intensities assuming
    they are independent and normally distributed with the same mean.

    Sample means across half-datasets can be used to compute the merging statistics CC1/2 and CCanom.
    """
    dataset = dataset.copy()
    samples = []
    for n in range(nsamples):
        half1, half2 = sample_halfdatasets(dataset)
        mergedhalf1 = rs.algorithms.merge(half1)
        mergedhalf2 = rs.algorithms.merge(half2)
        result = mergedhalf1.merge(mergedhalf2, left_index=True, right_index=True, suffixes=(1, 2))
        result["sample"] = n
        samples.append(result)
    return rs.concat(samples).sort_index()
```

---
### Merge HEWL data

We will now merge the HEWL data, repeatedly sampling across half-datasets in order to assess the distribution of correlation coefficients.

```python
# This cell takes a few minutes with nsamples=15
merged = merge_dataset(hewl, 15)
```

```python
merged
```

      IMEAN1
      SIGIMEAN1
      I(+)1
      SIGI(+)1
      I(-)1
      SIGI(-)1
      N(+)1
      N(-)1
      IMEAN2
      SIGIMEAN2
      I(+)2
      SIGI(+)2
      I(-)2
      SIGI(-)2
      N(+)2
      N(-)2
      sample

      H
      K
      L

      0
      0
      4
      657.99817
      43.882294
      657.99817
      43.882294
      657.99817
      43.882294
      4
      4
      662.40204
      25.35386
      662.40204
      25.353859
      662.40204
      25.353859
      12
      12
      0

      4
      645.79663
      43.884262
      645.79663
      43.884266
      645.79663
      43.884266
      4
      4
      666.4745
      25.35348
      666.4745
      25.35348
      666.4745
      25.35348
      12
      12
      1

      4
      662.0198
      25.348951
      662.0198
      25.348951
      662.0198
      25.348951
      12
      12
      659.13995
      43.907776
      659.13995
      43.907776
      659.13995
      43.907776
      4
      4
      2

      4
      660.4694
      33.18889
      660.4694
      33.18889
      660.4694
      33.18889
      7
      7
      661.946
      29.271538
      661.946
      29.271538
      661.946
      29.271538
      9
      9
      3

      4
      666.3023
      24.359608
      666.3023
      24.35961
      666.3023
      24.35961
      13
      13
      639.66833
      50.654987
      639.66833
      50.654987
      639.66833
      50.654987
      3
      3
      4

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      45
      10
      2
      30.170218
      7.3511124
      30.170218
      7.3511124
      NaN
      NaN
      1
      0
      15.318142
      3.9687188
      15.318142
      3.9687188
      NaN
      NaN
      3
      0
      9

      2
      7.669444
      6.6297607
      7.669444
      6.6297607
      NaN
      NaN
      1
      0
      22.89468
      4.1084785
      22.89468
      4.1084785
      NaN
      NaN
      3
      0
      10

      2
      11.665072
      5.2794247
      11.665072
      5.2794247
      NaN
      NaN
      2
      0
      24.119883
      4.656634
      24.119883
      4.656634
      NaN
      NaN
      2
      0
      11

      2
      7.669444
      6.6297607
      7.669444
      6.6297607
      NaN
      NaN
      1
      0
      22.89468
      4.1084785
      22.89468
      4.1084785
      NaN
      NaN
      3
      0
      12

      2
      17.961912
      4.288131
      17.961912
      4.288131
      NaN
      NaN
      3
      0
      20.064919
      6.0180674
      20.064919
      6.0180674
      NaN
      NaN
      1
      0
      13

187364 rows × 17 columns

---
### Compute $CC_{1/2}$ and $CC_{anom}$

We will first assign each reflection to a resolution bin and then we will compute the correlation coefficients

```python
merged, labels = merged.assign_resolution_bins(bins=15)
```

```python
groupby1 = merged.groupby(["sample", "bin"])[["IMEAN1", "IMEAN2"]]
pearson1 = groupby1.corr(method="pearson").unstack().loc[:, ("IMEAN1", "IMEAN2")]
pearson1.name = "Pearson"
spearman1 = groupby1.corr(method="spearman").unstack().loc[:, ("IMEAN1", "IMEAN2")]
spearman1.name = "Spearman"
results1 = rs.concat([pearson1, spearman1], axis=1)
results1 = results1.groupby("bin").agg(["mean", "std"])
```

```python
results1.head()
```

      Pearson
      Spearman

      mean
      std
      mean
      std

      bin

      0
      0.997334
      0.000316
      0.998895
      0.000063

      1
      0.997958
      0.000360
      0.999363
      0.000066

      2
      0.998971
      0.000306
      0.999678
      0.000023

      3
      0.999297
      0.000102
      0.999658
      0.000024

      4
      0.999368
      0.000147
      0.999687
      0.000022

```python
plt.figure(figsize=(8, 4))
plt.errorbar(results1.index, results1[("Pearson", "mean")],
             yerr=results1[("Pearson", "std")],
             color="#1b9e77",
             label=r"$CC_{1/2}$ (Pearson)")
plt.errorbar(results1.index, results1[("Spearman", "mean")],
             yerr=results1[("Spearman", "std")],
             color="#d95f02",
             label=r"$CC_{1/2}$ (Spearman)")
plt.xticks(results1.index, labels, rotation=45, ha='right', rotation_mode='anchor')
plt.ylabel("Correlation Coefficient")
plt.xlabel(r"Resolution Bin ($\AA$)")
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.grid(axis="y", linestyle='--')
plt.show()
```

It is important to note the scale on the y-axis -- this dataset is edge-limited, and as such the $CC_{1/2}$ is very high across all resolution bins. The Spearman CC appears higher across all resolution bins except at high resolution, and overall has a lower standard deviation among samples.
This is consistent with our expectation that Spearman CCs are a more robust estimator of correlation than Pearson CCs.

Let's now repeat this for the anomalous data, computing $CC_{anom}$:

```python
merged["ANOM1"] = merged["I(+)1"] - merged["I(-)1"]
merged["ANOM2"] = merged["I(+)2"] - merged["I(-)2"]

# Similar to CChalf, but we will only look at acentric reflections
groupby2 = merged.acentrics.groupby(["sample", "bin"])[["ANOM1", "ANOM2"]]
pearson2 = groupby2.corr(method="pearson").unstack().loc[:, ("ANOM1", "ANOM2")]
pearson2.name = "Pearson"
spearman2 = groupby2.corr(method="spearman").unstack().loc[:, ("ANOM1", "ANOM2")]
spearman2.name = "Spearman"
results2 = rs.concat([pearson2, spearman2], axis=1)
results2 = results2.groupby("bin").agg(["mean", "std"])
```

```python
results2.head()
```

      Pearson
      Spearman

      mean
      std
      mean
      std

      bin

      0
      0.422196
      0.037998
      0.497923
      0.029704

      1
      0.233349
      0.061975
      0.328263
      0.023917

      2
      0.357182
      0.051746
      0.444657
      0.029250

      3
      0.484710
      0.045454
      0.543314
      0.031731

      4
      0.579058
      0.023952
      0.611739
      0.020141

```python
plt.figure(figsize=(8, 4))
plt.errorbar(results2.index, results2[("Pearson", "mean")],
             yerr=results2[("Pearson", "std")],
             color="#1b9e77",
             label=r"$CC_{anom}$ (Pearson)")
plt.errorbar(results2.index, results2[("Spearman", "mean")],
             yerr=results2[("Spearman", "std")],
             color="#d95f02",
             label=r"$CC_{anom}$ (Spearman)")
plt.xticks(results2.index, labels, rotation=45, ha='right', rotation_mode='anchor')
plt.ylabel("Correlation Coefficient")
plt.xlabel(r"Resolution Bin ($\AA$)")
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.grid(axis="y", linestyle='--')
plt.show()
```

There is significant anomalous signal across all but the highest resolution bins. The Spearman CCs have smaller error bars than the corresponding Pearson CCs and the Spearman CCs are also higher in most bins. These differences highlight the influence of outlier measurements on the different correlation coefficients.

---
### Summary

We have used `reciprocalspaceship` to merge a dataset using inverse-variance weights. As part of this analysis, we performed repeated 2-fold cross-validation to compute Pearson and Spearman correlation coefficients and associated uncertainties. This relatively simple procedure to obtain uncertainty estimates for correlation coefficients is seldom done when analyzing merging quality. However, we can see that the standard deviation for computed correlation coefficients can be nearly $\pm0.1$ for quantities such as $CC_{anom}$. This is worth keeping in mind when analyzing SAD experiments because this dataset is very high quality (edge-limited) when one considers the $CC_{1/2}$. Putting these two correlation coefficients on the same axes emphasizes this point:

```python
plt.figure(figsize=(9, 6))
plt.errorbar(results1.index, results1[("Pearson", "mean")],
             yerr=results1[("Pearson", "std")],
             color='#1b9e77',
             label=r"$CC_{1/2}$ (Pearson)")
plt.errorbar(results1.index, results1[("Spearman", "mean")],
             yerr=results1[("Spearman", "std")],
             color='#d95f02',
             label=r"$CC_{1/2}$ (Spearman)")
plt.errorbar(results2.index, results2[("Pearson", "mean")],
             yerr=results2[("Pearson", "std")],
             color='#1b9e77',
             linestyle="--",
             label=r"$CC_{anom}$ (Pearson)")
plt.errorbar(results2.index, results2[("Spearman", "mean")],
             yerr=results2[("Spearman", "std")],
             color='#d95f02',
             linestyle="--",
             label=r"$CC_{anom}$ (Spearman)")
plt.xticks(results1.index, labels, rotation=45, ha='right', rotation_mode='anchor')
plt.ylabel(r"Correlation Coefficient")
plt.xlabel(r"Resolution Bin ($\AA$)")
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.grid(axis="y", linestyle='--')
plt.tight_layout()
```

Even a simple change to the procedure for computing merging statistics, such as substituting a Spearman correlation coefficient for a Pearson one, can alter the apparent quality of a dataset. By lowering the barrier to implementing new analyses, we hope that `reciprocalspaceship` can encourage the development of more robust indicators of crystallographic data quality.


# Prewhitening in Python
A notebook on how to prewhiten your data in Python.

Implementation as described in:

Representational fMRI analysis: an introductory tutorial. Alexander Walther, Naveed Ejaz, Nikolaus Kriegeskorte, Jörn Diedrichsen. [Link](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.4570&rep=rep1&type=pdf).

```python
import numpy as np
from scipy.linalg import sqrtm
import matplotlib.pyplot as plt
%matplotlib inline
```

Let's generate some correlated data:

```python
C = np.array([[1, .9], [.9, 1]])
x = np.random.multivariate_normal([0, 0], C, 10000)

plt.scatter(x[:, 0], x[:, 1])
plt.title('Original data')
plt.show()
```

Now, let's prewhiten the data as follows:

\begin{align}
X_{prewhitened} = (X'X)^{-1/2}X
\end{align}

```python
D = sqrtm(np.linalg.inv(np.cov(x.T)))  # D = prewhitening matrix
xw = D.dot(x.T).T
plt.title('Prewhitened')
plt.scatter(xw[:, 0], xw[:, 1])
plt.show()
```

Now, the covariance-matrix of X should (approximately) be the identity-matrix ($I$):

```python
np.cov(xw.T)
```

    array([[  1.00000000e+00,   2.53914523e-15],
           [  2.53914523e-15,   1.00000000e+00]])


# Chapter 8 Matrices

行列に関してはコメントは少なめです。(~~正直、SympyのMatrix使うぐらいなら、Numpy使ったほうが効率良さそうだよね〜.~~)

```python
from sympy import *
init_printing(use_unicode=True)
```

`Sympy`で行列を作るには、`Matrix`オブジェクトを使う. たとえば

```python
Matrix([
[1, -1],
[3, 4],
[0, 2] ]) #行ベクトルの組み合わせであることを明示するために[]カッコを余分に付けている。
```

とできる. 列ベクトルは

```python
Matrix([1,2,3]) #([])はベクトル
```

#### 行列の積

```python
M = Matrix([
[1, 2, 3],
[3, 2, 1] ])
```

```python
type(M)
```

    sympy.matrices.dense.MutableDenseMatrix

```python
N = Matrix([0, 1, 1])
```

```python
M*N
```

**注意**: `Matrix`オブジェクトは`mutable`.

## 8.1　基本的な演算

### 8.1.1 形(Shape)

```python
M = Matrix([
 [1, 2, 3],
 [-2, 0, 4]
])
```

```python
M
```

```python
M.shape #Numpy同様()はいらない.
```

(行, 列)

### 8.1.2 行と列を参照する

```python
M.row(0) #1行 0から始まる.
```

```python
M.col(-1) #3列(負で最終列を参照)
```

### 8.1.2 行と列の削除 / 挿入

#### 行 / 列の削除

```python
M.col_del(0) # 一列目を削除
```

```python
M
```

```python
M.row_del(1) # 二行目を削除
```

```python
M
```

#### 行 /　列の追加

```python
M
```

```python
M = M.row_insert(1, Matrix([[0, 4]])) #2行目に(0, 4)成分を追加
```

```python
M
```

```python
M = M.col_insert(0, Matrix([1, -2])) #列ベクトルを追加するのでカッコは一組
```

```python
M
```

## 8.2 基本的な方法

```python
M = Matrix([
    [1, 3],
    [-2,3]
])
```

```python
N = Matrix([
    [0,3],
    [0,7]
])
```

```python
M + N #足し算
```

```python
M*N #行列の積
```

```python
3*M #定数倍
```

```python
M**2 #べき乗
```

```python
M**-1 #逆行列
```

```python
N**-1 #行列式がゼロで存在しない.
```

```python
M = Matrix([
    [1, 2, 3],
    [4, 5, 6]
])
```

```python
M
```

```python
M.T #転置行列
```

## 8.3 行列の構成

### 8.3.1 単位行列

```python
eye(3)
```

```python
eye(4)
```

### 8.3.2 ゼロ行列

```python
zeros(2,3)
```

```python
zeros(4)
```

### 8.3.3 すべての成分が1の行列

```python
ones(2,3)
```

### 8.3.4 対角行列

```python
diag(1, 2, 3)
```

```python
diag(-1, ones(2, 2), Matrix([5, 7, 5])) #複数組み合わせる
```

## 8.4 高等的な扱い

### 8.4.1 行列式

```python
M = Matrix([
    [1, 0, 1],
    [2, -1, 3],
    [4, 3, 2]
])
```

```python
M
```

```python
M.det()
```

### 8.4.2 簡約化

```python
M = Matrix([
    [1, 0, 1, 3],
    [2, 3, 4, 7],
    [-1, -3, -3, -4]
])
```

```python
M.rref()
```

第一引数は簡約化した行列、第二引数はピボット列の添字リスト. この行列のランクは2.

### 8.4.3 ヌル空間

```python
M = Matrix([
    [1, 2, 3, 0, 0],
    [4, 10, 0, 0, 1]
])
```

```python
M.nullspace()
```

--->M*x=0なる方程式の解. ヌル空間の次元は3.

### 8.4.4 列空間

```python
M = Matrix([
    [1, 1, 2],
    [2, 1, 3],
    [3, 1, 4]
])
```

```python
M.columnspace()
```

### 8.4.5 固有値、固有ベクトル、対角化

```python
M = Matrix([
    [3, -2, 4, -2],
    [5, 3, -3, -2],
    [5, -2, 2, -2],
    [5, -2, -3 ,3]
])
```

```python
M.eigenvals()
```

--->固有値 -2 と 3 は多重度1, 固有値5は多重度 3

```python
M.eigenvects()
```

--->固有値とともに固有ベクトルも表示される. 常にコレを使ってもいいが、固有ベクトルの計算は時間がかかるので、固有値だけ欲しければeigenvals()を使うのが吉.

```python
P, D = M.diagonalize()
```

```python
P #直交行列
```

```python
D #対角行列
```

```python
P*D*P**-1
```

これはMに等しい.

#### 固有方程式

```python
lamda = symbols('lamda') #lambdaは予約語
```

```python
p = M.charpoly(lamda)
```

```python
factor(p) #Mの固有方程式
```


```python
import numpy as np
import sympy
from matplotlib import pyplot
%matplotlib inline
from sympy.utilities.lambdify import lambdify
```

```python
#Part A
```

```python
#Conditions
Vmax = 80 #km/h
L = 11 #km
rhomax = 250 #cars/km
nx = 51 #spatial discretization
dt = 0.001 #h
dx = L/(nx-1)

```

```python
x = np.linspace(0,L,nx)
rho0 = np.ones(nx)*10
rho0[10:20] = 50
```

```python
T1 = 6/60.0
nt = int(T1/dt)
print('Number of steps',nt)
```

    Number of steps 100

```python
v = np.ones(nx)

for i in range(nx):

    v[i] = Vmax*(1 - (rho0[i]/rhomax))

```

```python
vminimum = np.min(v)
vmin0 = vminimum/3.6
vmin0
print('Minimum velocity at t=0 (m/s): {:.2f}'.format(vmin0))
```

    Minimum velocity at t=0 (m/s): 17.78

```python
rho = rho0.copy()
v = np.ones(nx)
F = np.ones(nx)
for n in range(1,nt):
    F[1:]=Vmax*rho[1:]*(1-rho[1:]/rhomax)
    F[:-1]=Vmax*rho[:-1]*(1-rho[:-1]/rhomax)
    rho[1:]=rho[1:]-dt/dx*(F[1:]-F[:-1])
    v=F/rho
    if n==50:
        vmean3 = np.mean(v)
        vms3 = vmean3/3.6
        print('Average velocity at t=3 min (m/s): {:.2f}'.format(vms3))

vmin = np.min(v)
vmsmin = vmin/3.6
print('Minimum velocity at t=6 min (m/s): {:.2f}'.format(vmsmin))
```

    Average velocity at t=3 min (m/s): 20.62
    Minimum velocity at t=6 min (m/s): 18.74

```python
vmsmin
```

    18.744756132701628

```python
# Plot the solution after nt time steps
# along with the initial conditions.
pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel('x')
pyplot.ylabel('rho0,rho,v')
pyplot.grid()
pyplot.plot(x, rho0, label='Rho0',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, rho, label='Rho',
            color='C1', linestyle='-', linewidth=2)
pyplot.plot(x, v, label='v',
            color='C2', linestyle='-', linewidth=2)
pyplot.legend(loc='center right')

```

```python
pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel('x')
pyplot.ylabel('F')
pyplot.grid()
pyplot.plot(x, F, label='Flux',
            color='C3', linestyle='-', linewidth=2)
pyplot.legend(loc='upper left')
```

```python
#Part B
```

```python
#Conditions
Vmax = 136 #km/h
L = 11 #km
rhomax = 250 #cars/km
nx = 51 #spatial discretization
dt = 0.001 #h
dx = L/(nx-1)

```

```python
x = np.linspace(0,L,nx)
rho0 = np.ones(nx)*20
rho0[10:20] = 50
```

```python
T1 = 3/60.0
T1
nt = int(T1/dt)
print('Number of steps',nt)
```

    Number of steps 50

```python
v = np.ones(nx)

for i in range(nx):

    v[i] = Vmax*(1 - (rho0[i]/rhomax))
```

```python
vminimum = np.min(v)
vmin0 = vminimum/3.6
vmin0
print('Minimum velocity at t=0 (m/s): {:.2f}'.format(vmin0))
```

    Minimum velocity at t=0 (m/s): 30.22

```python
rho = rho0.copy()
v = np.ones(nx)
F = np.ones(nx)
for n in range(1,nt):
    F[1:]=Vmax*rho[1:]*(1-rho[1:]/rhomax)
    F[:-1]=Vmax*rho[:-1]*(1-rho[:-1]/rhomax)
    rho[1:]=rho[1:]-dt/dx*(F[1:]-F[:-1])
    v=F/rho

vmean3 = np.mean(v)
vms3 = vmean3/3.6
print('Average velocity at t=3 min (m/s): {:.2f}'.format(vms3))

vmin = np.min(v)
vmsmin = vmin/3.6
print('Minimum velocity at t=3 min (m/s): {:.2f}'.format(vmsmin))
```

    Average velocity at t=3 min (m/s): 33.88
    Minimum velocity at t=3 min (m/s): 30.83

```python
# Plot the solution after nt time steps
# along with the initial conditions.
pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel('x')
pyplot.ylabel('rho0,rho,v')
pyplot.grid()
pyplot.plot(x, rho0, label='Rho0',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, rho, label='Rho',
            color='C1', linestyle='-', linewidth=2)
pyplot.plot(x, v, label='v',
            color='C2', linestyle='-', linewidth=2)
pyplot.legend(loc='center right')
```

```python
pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel('x')
pyplot.ylabel('F')
pyplot.grid()
pyplot.plot(x, F, label='Flux',
            color='C3', linestyle='-', linewidth=2)
pyplot.legend(loc='upper left')
```


# What is the % odorant vapor in the air stream, as a function of odorant properties, jar dimensions, and air flow rate?

A cynlindrical jar with cross-sectional area $A$ and height $h$ contains a volume $V_{sol}$ of odorant solution (diluted in a solvent with zero vapor pressure), and the remaining volume, above the solution, is called $V_{head}$.

The flow rate of air into the jar, in mol/s, is $r_{air,in}$.  The gas flow out of the jar consists of the air component $r_{air, out}$ and the odorant component $r_{odor, out}$, also in mol/s.

The interface between the solution and the jar headspace has odorant molecules leaving with rate $r_{evap}$ and molecules condensing at rate $r_{condense}$, also in mol/s.

Therefore, the amount of odorant $n_o$ (in moles) in the jar headspace evolves according to:

\begin{equation}
\tag{1}\frac{dn_{odor}}{dt} = r_{evap} - r_{condense} - r_{odor, out}
\end{equation}

And the amount of air $n_{air}$ in the jar headspace evolves according to:

\begin{equation}
\tag{2}\frac{dn_{air}}{dt} = r_{air,in} - r_{air, out}
\end{equation}

Assuming that mixing with odorant vapor in the jar is instantaneous. Then the ratio of odorant efflux out of the jar to air efflux out of the jar is equal to the ratio of odor to air in the headspace of the jar at that moment.

\begin{equation}
\tag{3}\frac{n_{odor}}{n_{air}} = \frac{r_{odor,out}}{r_{air, out}}
\end{equation}

If evaporation is slow compared to the flow rate, we can assume that:

\begin{equation}
\tag{4}r_{air,in} = r_{air,out} + r_{odor, out}
\end{equation}

Combining equations 3 and 4 gives:

$$r_{odor,out} = r_{air,out}\frac{n_{odor}}{n_{air}} = (r_{air,in} - r_{odor, out})\frac{n_{odor}}{n_{air}}$$

$$r_{odor,out}(1+\frac{n_{odor}}{n_{air}}) = r_{air,in}\frac{n_{odor}}{n_{air}}$$

\begin{equation}
\tag{5}r_{odor,out} = \frac{r_{air,in}n_{odor}}{n_{air}+n_{odor}}
\end{equation}

And lastly, if both odorant and air are ideal gases, and air flow does not substantially change pressure in the jar, then the combined air and gas molecules fill the headspace at room temperature and ambient pressure according to:

\begin{equation}
\tag{6}n_{odor} + n_{air} = \frac{P_{room}V_{head}}{RT_{room}} \sim \frac{V_{head}}{22.4 Liters}
\end{equation}

Substituting (6) into (5) gives:

\begin{equation}
\tag{7}r_{odor, out} = \frac{r_{air,in}n_{odor}RT_{room}}{P_{room}V_{head}}
\end{equation}

The evaporation rate is assumed to be indepedent of time, depending (through a function $F_{evap}$ obtained empirically and described later) only upon the partial pressure $P^*_o$ of the liquid odorant, and the surface area $A$ of the liquid-vapor interface, where the partial pressure is equal to the mole fraction $f_o$ of the odorant in the liquid times its intrinsic vapor pressure $P_o$ (at this temperature), according to Raoult's law.

\begin{equation}
\tag{8}r_{evap} = F_{evap}(P^*_{odor})A
\end{equation}
\begin{equation}
\tag{9}P^*_{odor} = P_{odor}f_{odor}
\end{equation}

Meanwhile, the condensation rate depends upon the molar concentration C_o of odorant vapor in the headspace and the headspace volume according to a constant $k_{condense}$:

\begin{equation}
\tag{10}r_{condense} = k_{condense}AC_{odor}
\end{equation}
\begin{equation}
\tag{11}C_{odor} = n_{odor}/V_{head}
\end{equation}

Substituting (7-11) back into equation (1) gives a first order, linear differential equation in $n_{odor}$:

$$\frac{dn_{odor}}{dt} = F_{evap}(P_{odor},f_{odor})A - \frac{k_{condense}An_{odor}}{V_{head}} - \frac{r_{air,in}n_{odor}RT_{room}}{P_{room}V_{head}}$$

\begin{equation}
\tag{12}\frac{dn_{odor}}{dt} = F_{evap}(P_{odor},f_{odor})A - n_{odor}(\frac{1}{V_{head}}(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}}))
\end{equation}

We can define:
\begin{equation}
\tag{13}u = \frac{1}{V_{head}}(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}})
\end{equation}
and if the headspace volume changes very slowly, we can assume that this is independent of time.

The integrating factor is:
\begin{equation}
e^{\int{u}dt} = e^{ut}\tag{14}
\end{equation}

The canonical solution to (12) is thus:

\begin{equation}
\tag{15}n_{odor}(t) = e^{-ut}(\int{e^{ut}F_{evap}(P_{odor},f_{odor})Adt} + constant)
\end{equation}

Integrating gives:

\begin{equation}
n_{odor}(t) = \frac{F_{evap}(P_{odor},f_{odor})A}{u} + constant*e^{-ut}\tag{16}
\end{equation}

At steady state flow ($t = \infty$), the number of odorant molecules in the headspace is proportional to the evaporation function $F_{evap}$, which depends on the vapor pressure of the odorant, its mole fraction in solution, and the surface area of the liquid-vapor interface). The number of odorant molecules in the headspace is inversely proportional to $u$, which includes a weighted sum of the condensation rate constant and the air inflow rate.

\begin{equation}
\tag{17}
n_{odor}(t=\infty) = \frac{F_{evap}(P_{odor},f_{odor})A}{u}
\end{equation}

At the starting time ($t=0$), we will assume that the jar is already in equilibrium, and the number of odorant molecules in the headspace is determined by the odorant's partial pressure:

\begin{equation}
\tag{18}
n_{odor}(t=0) = \frac{P_{odor}f_{odor}V_{head}}{RT_{room}} = \frac{F_{evap}(P_{odor},f_{odor})A}{u} + constant
\end{equation}

So the constant is equal to:

\begin{equation}
\tag{19}
constant = \frac{P_{odor}f_{odor}V_{head}}{RT_{room}} - \frac{F_{evap}(P_{odor},f_{odor})A}{u}
\end{equation}

Substituting back into (16) gives:

\begin{equation}
n_{odor}(t) = \frac{F_{evap}(P_{odor},f_{odor})A}{u} + (\frac{P_{odor}f_{odor}V_{head}}{RT_{room}} - \frac{F_{evap}(P_{odor},f_{odor})A}{u})*e^{-ut}\tag{20}
\end{equation}

And substituting (13) into (20) gives:

\begin{equation}
n_{odor}(t) = \frac{V_{head}F_{evap}(P_{odor},f_{odor})A}{(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}})} + (\frac{P_{odor}f_{odor}V_{head}}{RT_{room}} - \frac{V_{head}F_{evap}(P_{odor},f_{odor})A}{(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}})})*e^{-(\frac{1}{V_{head}}(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}}))t}\tag{21}
\end{equation}

Subsituting (21) into (5) gives:

\begin{equation}
\frac{r_{odor,out}}{r_{air,in}} = \frac{\frac{V_{head}F_{evap}(P_{odor},f_{odor})A}{(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}})} + (\frac{P_{odor}f_{odor}V_{head}}{RT_{room}} - \frac{V_{head}F_{evap}(P_{odor},f_{odor})A}{(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}})})*e^{-(\frac{1}{V_{head}}(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}}))t}}{\frac{P_{room}V_{head}}{RT_{room}}}
\end{equation}

\begin{equation}
\tag{22}
\frac{r_{odor,out}}{r_{air,in}} = \frac{F_{evap}(P_{odor},f_{odor})}{(\frac{k_{condense}P_{room}}{RT_{room}} + \frac{r_{air,in}}{A})} + (\frac{P_{odor}f_{odor}}{P_{room}} - \frac{F_{evap}(P_{odor},f_{odor})}{(\frac{k_{condense}P_{room}}{RT_{room}} + \frac{r_{air,in}}{A})})*e^{-(\frac{1}{V_{head}}(k_{condense}A + \frac{r_{air,in}RT_{room}}{P_{room}}))t}
\end{equation}

The evaporation rate $F_{evap}$ depends on $P_{odor}$, $f_{odor}$, and $A$. According to *Mackay, D., & van Wesenbeeck, I. (2014). Correlation of Chemical Evaporation Rate with Vapor Pressure. Environmental Science & Technology, 48(17), 10259–10263.
doi:10.1021/es5029074* the pure odorant evaporation rate per unit area has the empirical form:
\begin{equation}
\tag{23}
F^*_{evap}(P_{odor}) = e^{1.0243 ln(\frac{P_{odor}}{P_{unity}} - 15.08)} * F_{unity}
\end{equation}
where $P_{unity} = 1 Pa$ and $F_{unity}= 1\frac{mol}{m^2s}$

Assuming linear dependence on mole fraction in solution:
\begin{equation}
\tag{25}
F_{evap}(P_{odor},f_{odor}) = F^*_{evap}(P_{odor})f_{odor} = f_{odor}e^{1.0243 ln(\frac{P_{odor}}{P_{unity}} - 15.08)} * F_{unity}
\end{equation}

At equilibrium, the condensation rate is equal to the evaporation rate, resulting in a $n_{odor}(t=0)$ given by the partial pressure $P^*_{odor} = P_{odor}f_{odor}$.  Setting (8) and (10) equal and subsituting (18) gives:
\begin{equation}
F_{evap}(P_{odor},f_{odor})A = k_{condense}A\frac{n_{odor}(t=0)}{V_{head}}
\end{equation}

\begin{equation}
\tag{26}
F^*_{evap}(P_{odor})f_{odor}A = k_{condense}A\frac{P_{odor}f_{odor}V_{head}}{V_{head}RT_{room}}
\end{equation}

\begin{equation}
\tag{27}
k_{condense} = \frac{RT_{room}F^*_{evap}(P_{odor})}{P_{odor}}
\end{equation}

Substituting (25) and (26) into (22) gives:
\begin{equation}
\frac{r_{odor,out}}{r_{air,in}}(t) = \frac{F^*_{evap}(P_{odor})f_{odor}}{(\frac{\frac{RT_{room}F^*_{evap}(P_{odor})}{P_{odor}}P_{room}}{RT_{room}} + \frac{r_{air,in}}{A})} + (\frac{P_{odor}f_{odor}}{P_{room}} - \frac{F^*_{evap}(P_{odor})f_{odor}}{(\frac{\frac{RT_{room}F^*_{evap}(P_{odor})}{P_{odor}}P_{room}}{RT_{room}} + \frac{r_{air,in}}{A})})e^{-(\frac{1}{V_{head}}(\frac{RT_{room}F^*_{evap}(P_{odor})}{P_{odor}}A + \frac{r_{air,in}RT_{room}}{P_{room}}))t}
\end{equation}

\begin{equation}
\frac{r_{odor,out}}{r_{air,in}}(t) = f_{odor}(\frac{F^*_{evap}(P_{odor})}{(\frac{F^*_{evap}(P_{odor})}{P_{odor}}P_{room} + \frac{r_{air,in}}{A})} + (\frac{P_{odor}}{P_{room}} - \frac{F^*_{evap}(P_{odor})}{(\frac{F^*_{evap}(P_{odor})}{P_{odor}}P_{room} + \frac{r_{air,in}}{A})})e^{-(\frac{RT_{room}}{V_{head}}(\frac{F^*_{evap}(P_{odor})}{P_{odor}}A + \frac{r_{air,in}}{P_{room}}))t})
\end{equation}

\begin{equation}
\tag{28}
\frac{r_{odor,out}}{r_{air,in}}(t) = f_{odor}(\frac{1}{(\frac{P_{room}}{P_{odor}} + \frac{r_{air,in}}{AF^*_{evap}(P_{odor})})} + (\frac{P_{odor}}{P_{room}} - \frac{1}{(\frac{P_{room}}{P_{odor}} + \frac{r_{air,in}}{AF^*_{evap}(P_{odor})})})e^{-(\frac{RT_{room}}{V_{head}}(\frac{F^*_{evap}(P_{odor})}{P_{odor}}A + \frac{r_{air,in}}{P_{room}}))t})
\end{equation}

This equation, describing the molar fraction of odorant in the air stream, has the functional form:

\begin{equation}
\tag{29}
\frac{r_{odor,out}}{r_{air,in}}(t) = f_{odor}(a + (b-a)e^{-ct})
\end{equation}

where $f_{odor}$ is the mole fraction of the odorant in solution, $f_{odor}a$ describes the steady-state fraction in the exiting vapor at $t=0$, $f_{odor}b$ describes the steady-state fraction in the exiting vapor at $t=\infty$, and $\frac{1}{c}$ is the time constant.

In the vapor pressure regime of most odorants (0.1 - 10 mm Hg), the ratio $\frac{F^*_{evap}(P_{odor})}{P_{odor}}$ does not differ more than $\sim5\%$ from $k_{evap} \sim3.2*10^{-7} \frac{mol}{m^2sPa}$, i.e. $F^*_{evap}$ is linear in $P_{odor}$. This means that the time constant $\frac{1}{c}$ is largely independent of the odorant in question.

Plugging in numbers:

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import quantities as pq
from quantities.constants.statisticalmechanics import R
from IPython.display import Markdown

sns.set(font_scale=1.5)
```

```python
jar_diameter = 6 * pq.cm
jar_height = 5 * pq.cm
height_filled = 1*pq.cm
A = np.pi * (jar_diameter / 2)**2
V_head = A * (jar_height - height_filled)

f_odor = 0.001 # A 0.1% solution
P_room = 1*pq.atm # 1 atmosphere
P_odor = 10.5 * pq.mmHg # Hexanal vapor pressure
T_room = (22 + 273.15)*pq.Kelvin
r_air_in = (1.0*pq.L/pq.min)*P_room/(R*T_room) # 1 L/min convered to mol/s

def F_star_evap(vp):
    # Units of Pascals
    vp = vp.rescale(pq.Pa)
    # Strip units for logarithm
    vp /= vp.units
    # Evaporation rate
    er = np.exp(1.0243*np.log(vp) - 15.08)
    if isinstance(er, np.ndarray):
        er[vp==0] = 0
    # Attach units
    er *= pq.mol / (pq.m**2 * pq.s)
    return er

t = np.linspace(0,5,1000) * pq.s # 1000 ms
# Use the form:
# ratio = f_odor*(a + (b - a)exp(-ct))
a = (1/(P_room/P_odor + r_air_in/(A*F_star_evap(P_odor)))).rescale(pq.dimensionless)
b = (P_odor/P_room).rescale(pq.dimensionless)
c = ((R*T_room/V_head)*(A*F_star_evap(P_odor)/P_odor + r_air_in/P_room)).rescale(1/pq.s)
ratio = f_odor*(a + (b - a)*np.exp(-c*t))

Markdown(r"a = %.3g<br>b= %.3g<br>c = %.3g($s^{-1}$)" % (a, b, c))
```

a = 0.0017
b= 0.0138
c = 0.168($s^{-1}$)

Here is the decay curve for the fraction of odorant (by mole) in the vapor leaving the jar over time:

```python
plt.plot(t, ratio)
plt.xlabel('Time (s)')
plt.ylabel('Volume fraction odorant\nin air stream')
plt.ylim(0, ratio.max()*1.1);
```

```python
vps = (np.logspace(-1,1,100) * pq.mmHg).rescale(pq.Pa)
ers = np.zeros(vps.shape)
for i, vp in enumerate(vps):
    ers[i] = F_star_evap(vp)
plt.scatter(vps, ers/vps)
plt.xscale('log')
plt.ylim(3e-7,3.5e-7)
plt.xlabel(r'Vapor pressure ($Pa$)')
plt.ylabel(r'Evaporation rate ($\frac{mol}{m^2s}$)');
```

```python
r_air_ins_volume = np.logspace(-4,2,100)*pq.L/pq.min
r_air_ins_mole = (r_air_ins_volume*P_room/(R*T_room)).rescale(pq.mol/pq.s)
k_evap = 3.2e-7*pq.mol/(pq.m**2*pq.s*pq.Pa)
cs = ((R*T_room/V_head)*(A*k_evap + r_air_ins_mole/P_room)).rescale(1/pq.s)
plt.scatter(r_air_ins_volume, 1/cs)
plt.xlabel(r'Air flow rate ($\frac{L}{min}$)')
plt.ylabel(r'Odorant depletion time constant ($s$)')
plt.xscale('log')
plt.yscale('log')
plt.xlim(1e-4,1e2)
plt.ylim(1e-2,1e2);
```

Using the linear form of $F^*_{evap}$, the dependence on $P_{odor}$ of the ratio between the initial and steady-state odorant enrichments disappears:

\begin{equation}
\tag{30}
\frac{b}{a} =
\frac{\frac{P_{odor}}{P_{room}}}{\frac{1}{(\frac{P_{room}}{P_{odor}} + \frac{r_{air,in}}{AF^*_{evap}(P_{odor})})}} =
\frac{\frac{P_{odor}}{P_{room}}}{\frac{1}{(\frac{P_{room}}{P_{odor}} + \frac{r_{air,in}}{Ak_{evap}P_{odor}})}} =
\frac{P_{room}+\frac{r_{air,in}}{Ak_{evap}}}{P_{room}} =
1+\frac{r_{air,in}}{Ak_{evap}P_{room}}
\end{equation}
The only way to keep this fraction close to 1 is to have a low air-flow rate $r_{air,in}$ or a large solution-vapor interface surface area $A$, i.e. a wide jar.

```python
b_a_ratios = 1 + (r_air_ins_mole/(A*k_evap*P_room)).rescale(pq.dimensionless)
plt.scatter(r_air_ins_volume, b_a_ratios)
plt.xlabel(r'Air flow rate ($\frac{L}{min}$)')
plt.ylabel(r'Odorant enrichment ratio $\frac{initial}{steadystate}$')
plt.xscale('log')
plt.yscale('log')
plt.xlim(1e-4,1e2)
plt.ylim(0.9,1e3);
```

In the interval between stimuli, if air flow through the jar is turned off, the odorant concentration increases again towards its intitial concentration. Here we assume that the pressure change inside the sealed jar due to the odorant realizing its partial pressure is small.

\begin{equation}
\tag{31}
\frac{dn_{odor}}{dt} = r_{evap} - r_{condense} = f_{odor}F*_{evap}(P_{odor})A - \frac{RT_{room}F^*_{evap}(P_{odor})An_{odor}}{P_{odor}V_{head}}
\end{equation}

We can define:
\begin{equation}
\tag{32}u^* = \frac{RT_{room}F^*_{evap}(P_{odor})A}{P_{odor}V_{head}}
\end{equation}

with a solution just like (21), except without the term for air influx:

\begin{equation}
\tag{32}
n_{odor}(t) = \frac{V_{head}F_{evap}(P_{odor},f_{odor})A}{k_{condense}A} + (\frac{P_{odor}f_{odor}V_{head}}{RT_{room}} - \frac{V_{head}F_{evap}(P_{odor},f_{odor})A}{k_{condense}A})*e^{-(\frac{1}{V_{head}}(k_{condense}A))t}
\end{equation}

\begin{equation}
n_{odor}(t) = e^{-ut}(\int{e^{ut}f_{odor}F^*_{evap}(P_{odor})Adt} + constant)
\end{equation}

\begin{equation}
n_{odor}(t) = \frac{f_{odor}F^*_{evap}(P_{odor})A}{u} + constant*e^{-ut}
\end{equation}

\begin{equation}
n_{odor}(t) = \frac{f_{odor}F^*_{evap}(P_{odor})A}{\frac{RT_{room}F^*_{evap}(P_{odor})A}{P_{odor}V_{head}}} + constant*e^{-ut}
\end{equation}

\begin{equation}
\tag{33}n_{odor}(t) = \frac{f_{odor}P_{odor}V_{head}}{RT_{room}} + constant*e^{-\frac{RT_{room}F^*_{evap}(P_{odor})A}{P_{odor}V_{head}}t}
\end{equation}

The value of the constant term depends on how depleted the odorant in the headspace is before the airflow is turned off.  Without less of generality we can simply define it as:
\begin{equation}
\tag{34}
constant = n_{odor}(t=0) - \frac{f_{odor}P_{odor}V_{head}}{RT_{room}}
\end{equation}

giving us:
\begin{equation}
\tag{35}n_{odor}(t) = \frac{f_{odor}P_{odor}V_{head}}{RT_{room}} + (n_{odor}(t=0) - \frac{f_{odor}P_{odor}V_{head}}{RT_{room}})*e^{-\frac{RT_{room}F^*_{evap}(P_{odor})A}{P_{odor}V_{head}}t}
\end{equation}

Therefore, a scenario of a stimulus of duration $T_on$ followed by an interval of duration $T_off$ exhibits the following behavior: The quantity of odorant in the jar headspace is initially $\frac{f_{odor}P_{odor}V_{head}}{RT_{room}}$, which follows trivially from Raoult's law and the ideal gas law.  Once the air flow is turned on, it declines by a factor of $\frac{b}{a} = 1+\frac{r_{air,in}}{Ak_{evap}P_{room}}$ with inverse time constant $c = \frac{RT_{room}}{V_{head}}(\frac{F^*_{evap}(P_{odor})}{P_{odor}}A + \frac{r_{air,in}}{P_{room}})) \sim \frac{RT_{room}}{V_{head}}(k_{evap}A + \frac{r_{air,in}}{P_{room}}))$. When the air is turned off, it returns to its initial value with inverse time constant $c = \frac{RT_{room}F^*_{evap}(P_{odor})A}{P_{odor}V_{head}} \sim \frac{RT_{room}k_{evap}A}{V_{head}}$.

```python
dt = 0.001
T_on = 2
T_off = 8
n_cycles = 10
t = np.arange(0,(T_on+T_off)*n_cycles,dt) * pq.s
t_on = int(T_on / dt)
t_off = int(T_off / dt)
ratios = np.zeros(t.shape)
# Use the form:
# ratio = f_odor*(a + (b - a)exp(-ct))
a = (1/(P_room/P_odor + r_air_in/(A*F_star_evap(P_odor)))).rescale(pq.dimensionless)
b = (P_odor/P_room).rescale(pq.dimensionless)
c_decay = ((R*T_room/V_head)*(A*F_star_evap(P_odor)/P_odor + r_air_in/P_room)).rescale(1/pq.s)
c_recover = ((R*T_room/V_head)*(A*F_star_evap(P_odor)/P_odor)).rescale(1/pq.s)
ratios[0:t_on] = f_odor*(a + (b - a)*np.exp(-c_decay*t[:t_on]))
ratios[t_on:t_on+t_off] = f_odor*(b + (ratios[t_on-1]/f_odor - b)*np.exp(-c_recover*t[:t_off]))
for cycle in range(1,n_cycles):
    ratios[cycle*t_on+cycle*t_off:(cycle+1)*t_on+cycle*t_off] = f_odor*(a + (ratios[cycle*t_on+cycle*t_off-1]/f_odor - a)*np.exp(-c_decay*t[:t_on]))
    ratios[(cycle+1)*t_on+cycle*t_off:(cycle+1)*t_on+(cycle+1)*t_off] = f_odor*(b + (ratios[(cycle+1)*t_on+cycle*t_off-1]/f_odor - b)*np.exp(-c_recover*t[:t_off]))

```

Repeated stimuli (with on time T_on and off time T_off will have an odorant mole fraction in the vapor that changes as follows):

```python
plt.plot(t, ratios)
plt.xlabel('Time ($s$)')
plt.ylabel('Mole fraction of odorant\nin headspace vapor');
plt.ylim(0, ratios[0]*1.1);
```

```python
print("Depletion time constant is %.3g s" % (1/c_decay))
print("Recovery time constant is %.3g s" % (1/c_recover))
```

    Depletion time constant is 5.95 s
    Recovery time constant is 48.4 s


## probablility density functions
\begin{align}
\mbox{Gaussian} \\
\ g(y, \mu, \sigma) = \frac{1}{\sigma \sqrt{2 \pi}}  \exp{\frac{-(y - \mu)^2}{2 \sigma^2}} \\
\end{align}
## adaptable parameters
\begin{align}
\Delta \mu = \alpha_{\mu} (r - b{\mu}) \frac{y - \mu}{\sigma^2} \\
\\
\Delta \sigma = \alpha_{\sigma} (r - b{\sigma}) \frac{(y - \mu)^2 - \sigma^2}{\sigma^3} \\
\end{align}

```python
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np

def g(y,u,s):
    c = 1 / (np.sqrt(2*np.pi) * s)
    return c * np.exp((-(y-u)**2)/(2*s**2))
```

```python
n_points = 1000
u = 6
s = 1
x = np.linspace(3, 9, n_points)
y = g(x,u,s)
plt.plot(x, y)
```

    <IPython.core.display.Javascript object>

    [<matplotlib.lines.Line2D at 0x114065e10>]

```python
s - 0.9
s_hi = u + s
s_lo = u - s
print('number >= %0.2f'%cutoff, sum(np.random.normal(u,s, n_points) <= s_hi))
print('number >= %0.2f'%cutoff, sum(np.random.normal(u,s, n_points) <= s_lo))
```

    number >= 7.00 805
    number >= 7.00 163

```python

```

```python

```


# Chapter 5 Simplifications

```python
from sympy import *
x, y, z = symbols('x, y, z')
init_printing(use_unicode=True)
```

## 5.1 単純化

`sympy`のどんな数式も`simplify()`で簡単な形にできる！:

```python
simplify(sin(x)**2 + cos(x)**2)
```

```python
simplify((x**3 + x**2 - x - 1) / (x**2 + 2*x + 1))
```

```python
simplify(gamma(x) / gamma(x-2)) #ガンマ関数(特殊関数)
```

#### 注意点:その1

```python
simplify(x**2 + 2*x + 1)
```

---> **因数分解できない!!!** 因数分解は`factor()`関数を使う:

```python
factor(x**2 + 2*x + 1)
```

#### 注意点:その2

`simplify()`は遅い！

#### 解決策

- `simplify()`は「ある程度」簡単な形にまでしか変形できないので、確実に式を簡単にしたいなら、その用途に応じた適切な関数を使うべき!

- インタラクティブシェルで`simplify`の挙動を見てから**個別の関数**(以下) を使って簡単にしよう.

## 5.2 多項式 / 有理式

### 5.2.1 `expand`関数

多項式を展開し、必要ならば項をキャンセルする.

```python
expand((x + 1)**2)
```

```python
expand((x + 2)*(x - 3))
```

「式を展開する」ことで「式が簡単になる」ことがある。

```python
expand((x + 1)*(x - 2) - (x - 1)*x) #式がキャンセルし合う
```

### 5.2.2 `factor`関数

数式を可能な限り因数分解する

```python
factor(x**3 - x**2 + x - 1)
```

```python
factor(x**2*z + 4*x*y*z + 4*y**2*z)
```

```python
factor_list(x**2*z + 4*x*y*z + 4*y**2*z) #(変数or定数, べき)
```

#### 三角関数程度の式なら、関数`factor`, `expand`で対応可能

```python
expand((cos(x) + sin(x))**2)
```

```python
factor(cos(x)**2 + 2*cos(x)*sin(x) + sin(x)**2)
```

### 5.2.3 `collect`関数

特定の変数でまとめたり、特定次の係数を取り出す.

```python
expr = x*y + x -3 + 2*x**2 - z*x**2 + x**3
```

```python
expr
```

```python
collected_expr = collect(expr, x) #xでまとめる.
```

```python
collected_expr
```

さらに以下のようにcoeffメソッドで特定次を取り出せる.

```python
collected_expr.coeff(x, 2) #xの2次だけ取り出す.
```

### 5.2.4 `cancel`関数

有理式を簡単にする

```python
cancel((x**2 + 2*x + 1) / (x**2 + x))
```

```python
expr = 1/x + (2*x/2 - 2) /(x - 4)
```

```python
expr
```

```python
cancel(expr) #分母を通分する
```

```python
factor(expr) #factorも同じような操作をする.
```

```python
expr = (x*y**2 - 2*x*y*z + x*z**2 + y**2 - 2*y*z + z**2) / (x**2 - 1)
```

```python
expr
```

```python
cancel(expr)
```

```python
factor(expr) #factorも同じような変形をする.
```

**コメント**

式を単にキャンセルさせてシンプルにさせたいときは、`factor()`より`cancel()`のほうが効率的

### 5.2.5 `apart`関数

有理式(分数)を部分分数分解する

```python
x = symbols('x')
expr = (4*x**3 + 21*x**2 + 10*x + 12) / (x**4 + 5*x**3 + 5*x**2 + 4*x)
```

```python
expr
```

```python
apart(expr)
```

## 5.3 三角関数

**コメント**: 逆三角関数は頭に"a"を付ける: acos, asin, atan, etc...

```python
acos(x)
```

```python
cos(acos(x))
```

```python
asin(1)
```

### 5.3.1 `trigsimp`関数

三角関数の表式を、公式を用いて可能な限りシンプルな形にする.

```python
trigsimp(sin(x)**2 + cos(x)**2)
```

```python
trigsimp(sin(x)**4 - 2*cos(x)**2*sin(x)**2 + cos(x)**4)
```

```python
trigsimp(sin(x)*tan(x)/sec(x))
```

```python
trigsimp(cosh(x)**2-sinh(x)**2)
```

### 5.3.2 `expand_trig`関数

三角関数の式を展開する。 `trigsimp`と`expand_trig`は完全に逆の操作をする

```python
expand_trig(sin(x + y))
```

```python
expand_trig(tan(2*x))
```

## 5.4 べき乗

```python
x, y = symbols('x y', positive=True) #変数が正であると仮定
```

```python
a, b = symbols('a, b', real = True) #変数が実数であると仮定
```

```python
z, t, c = symbols('z t c')
```

**コメント**: `sqrt(x)`と`x**Rational(1,2)`, `x**0.5`, `x**(1/2)`は同じ

```python
sqrt(x)
```

```python
x**Rational(1,2)
```

```python
x**(0.5)
```

```python
x**(1/2)
```

### 5.4.1 `powsimp` 関数

冪が変数(`Sympy`シンボル)のときに限り、シンプルな形にする

```python
powsimp(x**a*x**b) #これ以上簡単にできない.
```

```python
powsimp(x**a*y**a)
```

変数の仮定にかかわらず実行させたいとき:

```python
powsimp(t**c*z**c)
```

を

```python
powsimp(t**c*z**c, force=True)
```

とする. `t` もしくは `z` が負になっても強制的にこの変形は行われる.

```python
(z*t)**2 #冪が整数、有理数, 2のとき.
```

```python
sqrt(x*y) #同じ
```

**注意** このような式に対しては`powsimp`は使えない:

```python
powsimp(z**2*t**2) #指数が整数
```

```python
sqrt(x*y)
```

--->冪が変数のときに`powsimp`で簡単にできる.

### 5.4.2 `expand_power_expr`関数, `expand_power_base`関数

べき乗を展開する. `powsimp`関数と逆の操作

```python
expand_power_exp(x**(a + b))
```

```python
expand_power_base((x*y)**a)
```

**注意** これも`powsimp()`と同様で、変形できないときは元の式を返す:

```python
expand_power_base((z*t)**c)
```

`t*z`が正という条件を`symbols`でつけていれば展開できるが、
今回のようにそうと限らないときは展開してくれない. 強制的に行うには

```python
expand_power_base((z*t)**c, force=True)
```

とする. また冪が数のときは

```python
x**2*x**3
```

```python
expand_power_exp(x**5)
```

のように変形できない。

### 5.4.3 `powdenest`関数

べき乗のべき乗を展開

```python
(x**a)**b #カッコを外して展開
```

```python
powdenest((x**a)**b)
```

```python
powdenest((z**a)**b)
```

```python
powdenest((z**a)**b, force=True)
```

## 5.5 指数関数、対数関数

```python
ln(x) #ln(x)とlog(x)は同じ.
```

```python
log(x)
```

```python
x, y = symbols('x y', positive=True)
```

```python
n = symbols('n', real=True)
```

### 5.5.1 `expand_log`関数

対数関数を展開する

```python
expand_log(log(x*y))
```

```python
expand_log(log(x/y))
```

```python
expand_log(log(x**2))
```

```python
expand_log(log(x**n))
```

```python
expand_log(log(z*t))
```

**注意** これまでと同様にして、正でない変数は展開できないので、そのときは`Force=True`オプションを付ける。

```python
expand_log(log(z**2))
```

```python
expand_log(log(z**2), force=True)
```

### 5.5.2 `logcombine`関数

対数関数をシンプルにする.

```python
logcombine(log(x) + log(y)) #対数関数を簡単にする
```

```python
logcombine(n*log(x))
```

```python
logcombine(n*log(z))
```

```python
logcombine(n*log(z), force=True)
```

## 5.6 特殊関数

```python
x, y, z = symbols('x y z')
```

```python
k, m, n = symbols('k m n')
```

### 5.6.1 階乗

```python
factorial(n)
```

```python
factorial(10)
```

### 5.6.2 組み合わせ (Combination)

```python
binomial(n, k) #nCk
```

```python
combsimp(factorial(n) / factorial(n - 3)) #シンプルにする
```

```python
combsimp(binomial(n + 1, k + 1) / binomial(n, k))
```

### 5.6.3 ガンマ関数

```python
gamma(z)
```

```python
combsimp(gamma(x)*gamma(1 - x)) #ガンマ関数にも使える
```

### 5.6.4 一般化された超幾何関数

```python
hyper([1, 2], [3], z)
```

### 5.6.5 関数を別の関数で書き換える

```python
tan(x).rewrite(sin) #tanをsinで書き換える
```

```python
factorial(x).rewrite(gamma) #階乗をガンマ関数で書き換える
```

### 5.6.6 特殊関数をいくつかの恒等式で書き換える

```python
expand_func(gamma(x + 3))
```

次は[Chapter6 Calculus](https://hiroyuki827.github.io/SymPy_tutorial/Chapter6_Calculus.html)へ!


```python
from numpy import array
from sympy import symbols, Eq, Matrix, diff, Derivative, simplify, factor, expand, latex, init_printing, collect
init_printing()
from IPython.display import display, Math
```

## Define the interpolation functions

```python
x1, y1, x2, y2, x3, y3, x4, y4 = symbols('x_1, y_1, x_2, y_2, x_3, y_3, x_4, y_4')
r, s = symbols('r, s')

# Define the interpolation functions
h1 = factor(1/4*(1+r)*(1+s))
h2 = factor(1/4*(1-r)*(1+s))
h3 = factor(1/4*(1-r)*(1-s))
h4 = factor(1/4*(1+r)*(1-s))

display(Math('h_1 = ' + latex(h1)))
display(Math('h_2 = ' + latex(h2)))
display(Math('h_2 = ' + latex(h3)))
display(Math('h_2 = ' + latex(h4)))
```

$\displaystyle h_1 = 0.25 \left(r + 1\right) \left(s + 1\right)$

$\displaystyle h_2 = - 0.25 \left(r - 1\right) \left(s + 1\right)$

$\displaystyle h_2 = 0.25 \left(r - 1\right) \left(s - 1\right)$

$\displaystyle h_2 = - 0.25 \left(r + 1\right) \left(s - 1\right)$

# Relate the (x, y) Local Coordinate System to the (r, s) Natural Coordinate System

```python
# Relate the (x, y) coordinate system to the (r, s) coordinate system
x = h1*x1 + h2*x2 + h3*x3 + h4*x4
y = h1*y1 + h2*y2 + h3*y3 + h4*y4

display(Math('x = ' + latex(x)))
display(Math('y = ' + latex(y)))
```

$\displaystyle x = 0.25 x_{1} \left(r + 1\right) \left(s + 1\right) - 0.25 x_{2} \left(r - 1\right) \left(s + 1\right) + 0.25 x_{3} \left(r - 1\right) \left(s - 1\right) - 0.25 x_{4} \left(r + 1\right) \left(s - 1\right)$

$\displaystyle y = 0.25 y_{1} \left(r + 1\right) \left(s + 1\right) - 0.25 y_{2} \left(r - 1\right) \left(s + 1\right) + 0.25 y_{3} \left(r - 1\right) \left(s - 1\right) - 0.25 y_{4} \left(r + 1\right) \left(s - 1\right)$

# Find the Jacobian Matrix
The Jacobian matrix converts derivatives with respect to x and y into derivatives with respect to r and s

$J = \begin{bmatrix} \frac{dx}{dr} & \frac{dy}{dr} \\ \frac{dx}{ds} & \frac{dy}{ds} \end{bmatrix}$

```python
# Calculate the Jacobian matrix
J = Matrix([[diff(x, r), diff(y, r)],
            [diff(x, s), diff(y, s)]])

display(Math('J = ' + latex(factor(J))))
```

$\displaystyle J = \left[\begin{matrix}0.25 x_{1} \left(s + 1\right) - 0.25 x_{2} \left(s + 1\right) + 0.25 x_{3} \left(s - 1\right) - 0.25 x_{4} \left(s - 1\right) & 0.25 y_{1} \left(s + 1\right) - 0.25 y_{2} \left(s + 1\right) + 0.25 y_{3} \left(s - 1\right) - 0.25 y_{4} \left(s - 1\right)\\0.25 x_{1} \left(r + 1\right) - 0.25 x_{2} \left(r - 1\right) + 0.25 x_{3} \left(r - 1\right) - 0.25 x_{4} \left(r + 1\right) & 0.25 y_{1} \left(r + 1\right) - 0.25 y_{2} \left(r - 1\right) + 0.25 y_{3} \left(r - 1\right) - 0.25 y_{4} \left(r + 1\right)\end{matrix}\right]$

```python
B_kappa = Matrix([[0,      0,      -diff(h1, r), 0,      0,      -diff(h2, r), 0,      0,      -diff(h2, r), 0,      0,      -diff(h4, r)],
                  [0, diff(h1, s),       0,      0, diff(h2, s),       0,      0, diff(h3, s),       0,      0, diff(h4, s),       0     ],
                  [0, diff(h1, r), -diff(h1, s), 0, diff(h2, r), -diff(h2, s), 0, diff(h3, r), -diff(h3, s), 0, diff(h4, r), -diff(h4, s)]])

display(Math('B_\kappa = J^{-1}(1/4)' + latex(B_kappa*4)))
```

$\displaystyle B_\kappa = J^{-1}(1/4)\left[\begin{array}{cccccccccccc}0 & 0 & - 1.0 s - 1.0 & 0 & 0 & s + 1.0 & 0 & 0 & s + 1.0 & 0 & 0 & s - 1.0\\0 & r + 1.0 & 0 & 0 & 1.0 - 1.0 r & 0 & 0 & r - 1.0 & 0 & 0 & - 1.0 r - 1.0 & 0\\0 & s + 1.0 & - 1.0 r - 1.0 & 0 & - 1.0 s - 1.0 & r - 1.0 & 0 & s - 1.0 & 1.0 - 1.0 r & 0 & 1.0 - 1.0 s & r + 1.0\end{array}\right]$

```python
dH = Matrix([[diff(h1, r), diff(h2, r), diff(h3, r), diff(h4, r)],
             [diff(h1, s), diff(h2, s), diff(h3, s), diff(h4, s)]])

B_m = Matrix([[diff(h1, r),      0,      diff(h2, r),      0,      diff(h3, r),      0,      diff(h4, r),      0     ],
              [     0,      diff(h1, s),      0,      diff(h2, s),      0,      diff(h3, s),      0,      diff(h4, s)],
              [diff(h1, s), diff(h1, r), diff(h2, s), diff(h2, r), diff(h3, s), diff(h3, r), diff(h4, s), diff(h4, r)]])

display(Math('dH = (1/4)' + latex(dH*4)))
display(Math('B_m = J^{-1}(1/4)' + latex(B_m*4)))
print(B_m*4)
```

$\displaystyle dH = (1/4)\left[\begin{matrix}s + 1.0 & - 1.0 s - 1.0 & s - 1.0 & 1.0 - 1.0 s\\r + 1.0 & 1.0 - 1.0 r & r - 1.0 & - 1.0 r - 1.0\end{matrix}\right]$

$\displaystyle B_m = J^{-1}(1/4)\left[\begin{matrix}s + 1.0 & 0 & - 1.0 s - 1.0 & 0 & s - 1.0 & 0 & 1.0 - 1.0 s & 0\\0 & r + 1.0 & 0 & 1.0 - 1.0 r & 0 & r - 1.0 & 0 & - 1.0 r - 1.0\\r + 1.0 & s + 1.0 & 1.0 - 1.0 r & - 1.0 s - 1.0 & r - 1.0 & s - 1.0 & - 1.0 r - 1.0 & 1.0 - 1.0 s\end{matrix}\right]$

    Matrix([[s + 1.0, 0, -1.0*s - 1.0, 0, s - 1.0, 0, 1.0 - 1.0*s, 0], [0, r + 1.0, 0, 1.0 - 1.0*r, 0, r - 1.0, 0, -1.0*r - 1.0], [r + 1.0, s + 1.0, 1.0 - 1.0*r, -1.0*s - 1.0, r - 1.0, s - 1.0, -1.0*r - 1.0, 1.0 - 1.0*s]])

```python
r, s = symbols('r, s')
h = symbols('h')
x1, x2, x3, x4 = symbols('x1, x2, x3, x4')
y1, y2, y3, y4 = symbols('y1, y2, y3, y4')
w1, w2, w3, w4 = symbols('w1, w2, w3, w4')
theta_x1, theta_x2, theta_x3, theta_x4 = symbols('theta_x1, theta_x2, theta_x3, theta_x4')
theta_y1, theta_y2, theta_y3, theta_y4 = symbols('theta_y1, theta_y2, theta_y3, theta_y4')

gamma_rz = (1 + s)*((w1 - w2)/2 + (x1 - x2)/4*(theta_y1 + theta_y2) - (y1 - y2)/4*(theta_x1 + theta_x2)) \
         + (1 - s)*((w4 - w3)/2 + (x4 - x3)/4*(theta_y4 + theta_y3) - (y4 - y3)/4*(theta_x4 + theta_x3))

gamma_sz = (1 + r)*((w1 - w4)/2 + (x1 - x4)/4*(theta_y1 + theta_y4) - (y1 - y4)/4*(theta_x1 + theta_x4)) \
         + (1 - r)*((w2 - w3)/2 + (x2 - x3)/4*(theta_y2 + theta_y3) - (y2 - y3)/4*(theta_x2 + theta_x3))

display(1/4*collect(expand(gamma_rz*4), [w1, theta_x1, theta_y1, w2, theta_x2, theta_y2, w3, theta_x3, theta_y3, w4, theta_x4, theta_y4]))
print('')
display(1/4*collect(expand(gamma_sz*4), [w1, theta_x1, theta_y1, w2, theta_x2, theta_y2, w3, theta_x3, theta_y3, w4, theta_x4, theta_y4]))
```

# References

The following references were used in the formulation of this element. The primary reference was the first one because it had a section directly relating to isoparametric general plate bending elements, but all three were used. The first reference is a free download from MIT's website.

1. "Finite Element Procedures, 2nd Edition", Klaus-Jurgen Bathe
2. "Finite Element Analysis Fundamentals", Richard H. Gallagher
3. "A First Course in the Finite Element Method, 4th Edition", Daryl L. Logan

# Write the Equations that Map the (x, y) Coordinates to the (r, s) Coordinates

For any invertible matrix $M = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$, the inverse can be found as follows:

```python
a, b, c, d = symbols('a, b, c, d')

M = Matrix([[a, b],
           [c, d]])

display(Math('M^{-1} = ' + latex(M.inv())))
```

Recognizing that the denominator in each term of this inverse is the determinant of $M$, the inverse can be rewritten as:

$M^{-1} = \frac{1}{|M|}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$

Applying this to the Jacobian, we can express its the inverse as:

$J^{-1} = \frac{1}{|J|} \begin{bmatrix} \frac{dy}{ds} & \frac{-dy}{dr} \\ \frac{-dx}{ds} & \frac{dx}{dr} \end{bmatrix}$

We can relate the derivatives of the (r, s) coordinate system to the derivatives of the (x, y) coordinate system using the Jacobian operator:

$\begin{bmatrix} \frac{d}{dr} \\ \frac{d}{ds} \end{bmatrix} = \begin{bmatrix} \frac{dx}{dr} & \frac{dy}{dr} \\ \frac{dx}{ds} & \frac{dy}{ds}\end{bmatrix} \begin{bmatrix} \frac{d}{dx} \\ \frac{d}{dy} \end{bmatrix} = J \begin{bmatrix} \frac{d}{dx} \\ \frac{d}{dy} \end{bmatrix}$

Rearranging the equation gives us the formula for any derivative of the (x, y) coordinate system in terms of (r, s):

$\begin{bmatrix} \frac{d}{dx} \\ \frac{d}{dy} \end{bmatrix} = \frac{1}{|J|} \begin{bmatrix} \frac{dy}{ds} & \frac{-dy}{dr} \\ \frac{-dx}{ds} & \frac{dx}{dr} \end{bmatrix} \begin{bmatrix} \frac{d}{dr} \\ \frac{d}{ds} \end{bmatrix}$

Finally, multiplying through, we can obtain equations for the converting derivatives of any function with respect to x or y to derivatives with respect to r and s, respectively:

$\frac{d}{dx} = \frac{1}{|J|} (\frac{dy}{ds} \frac{d}{dr} + \frac{-dy}{dr} \frac{d}{ds})$

$\frac{d}{dy} = \frac{1}{|J|} (\frac{-dx}{ds} \frac{d}{dr} + \frac{dx}{dr} \frac{d}{ds})$

The bending strains vary through the thickness of the plate. Using the curvatures of the plate they can be expressed as:

$\begin{bmatrix} \epsilon_{xx} \\ \epsilon_{yy} \\ \gamma_{xy} \end{bmatrix} = -z \begin{bmatrix} \frac{d \beta_{x}}{dx} \\ \frac{d \beta_{y}}{dy} \\ \frac{d \beta_{x}}{dy} + \frac{d\beta_{y}}{dx} \end{bmatrix} = \frac{-z}{|J|} \begin{bmatrix} \frac{dy}{ds} \frac{d\beta_x}{dr} + \frac{-dy}{dr} \frac{d\beta_x}{ds} \\ \frac{-dx}{ds} \frac{d\beta_y}{dr} + \frac{dx}{dr} \frac{d\beta_y}{ds} \\ (\frac{-dx}{ds} \frac{d\beta_x}{dr} + \frac{dx}{dr} \frac{d\beta_x}{ds}) + (\frac{dy}{ds} \frac{d\beta_y}{dr} + \frac{-dy}{dr} \frac{d\beta_y}{ds})\end{bmatrix}$

```python

```

```python
w_zi, theta_xi, theta_yi = symbols('w_{zi}, \\theta_{xi}, \\theta_{yi}')
w_zj, theta_xj, theta_yj = symbols('w_{zj}, \\theta_{xj}, \\theta_{yj}')
w_zm, theta_xm, theta_ym = symbols('w_{zm}, \\theta_{xm}, \\theta_{ym}')
w_zn, theta_xn, theta_yn = symbols('w_{zn}, \\theta_{xn}, \\theta_{yn}')

d = Matrix([w_zi, theta_xi, theta_yi,
            w_zj, theta_xj, theta_yj,
            w_zm, theta_xm, theta_ym,
            w_zn, theta_xn, theta_yn])
d
```

```python
w = Matrix([h1, 0, 0, h2, 0, 0, h3, 0, 0, h4, 0, 0]).T*d
beta_x = -1*Matrix([0, 0, h1, 0, 0, h2, 0, 0, h3, 0, 0, h4]).T*d
beta_y = Matrix([0, h1, 0, 0, h2, 0, 0, h3, 0, 0, h4, 0]).T*d
display(w, beta_x, beta_y)
```

```python
beta_x_dx = 1/J.det()*(diff(y, s)*diff(beta_x, r) - diff(y, r)*diff(beta_x, s))
beta_x_dy = 1/J.det()*(-diff(x, s)*diff(beta_x, r) + diff(x, r)*diff(beta_x, s))
beta_y_dx = 1/J.det()*(diff(y, s)*diff(beta_y, r) - diff(y, r)*diff(beta_y, s))
beta_y_dy = 1/J.det()*(-diff(x, s)*diff(beta_y, r) + diff(x, r)*diff(beta_y, s))
```

```python
B_kappa = Matrix([[0, 0,           diff(-h1, r), 0, 0,           diff(-h2, r), 0, 0,           diff(-h3, r), 0, 0,           diff(-h4, r)],
                  [0, diff(h1, s), 0,            0, diff(h2, s), 0,            0, diff(h3, s), 0,            0, diff(h4, s), 0],
                  [0, diff(h1, r), diff(-h1, s), 0, diff(h2, r), diff(-h2, s), 0, diff(h3, r), diff(-h3, s), 0, diff(h4, r), diff(-h4, s)]])
B_kappa
```

```python
x =
gr = diff(x, r)
```


```python
from IPython.display import display, Latex
import sympy as sy

u, t, d, r = sy.symbols('u t d r')
sy.init_printing(use_unicode=True)

B = (u / 2) * (((r ** 2) * sy.sin(t))/((r ** 2 + d ** 2 - 2 * r * d * sy.cos(t))**(3/2)))

int1 = sy.Integral(B, t)
display(Latex(f"$${sy.latex(B)}={sy.latex(int1.doit())}={sy.latex(sy.integrate(B, t))}$$"))

```

$$\frac{r^{2} u \sin{\left(t \right)}}{2 \left(d^{2} - 2 d r \cos{\left(t \right)} + r^{2}\right)^{1.5}}=\frac{r^{2} u \left(\begin{cases} - \frac{\cos{\left(t \right)}}{\left(r^{2}\right)^{1.5}} & \text{for}\: d = 0 \\- \frac{\cos{\left(t \right)}}{\left(d^{2}\right)^{1.5}} & \text{for}\: r = 0 \\- \frac{1.0}{d r \left(d^{2} - 2 d r \cos{\left(t \right)} + r^{2}\right)^{0.5}} & \text{otherwise} \end{cases}\right)}{2}=\frac{r^{2} u \left(\begin{cases} - \frac{\cos{\left(t \right)}}{\left(r^{2}\right)^{1.5}} & \text{for}\: d = 0 \\- \frac{\cos{\left(t \right)}}{\left(d^{2}\right)^{1.5}} & \text{for}\: r = 0 \\- \frac{1.0}{d r \left(d^{2} - 2 d r \cos{\left(t \right)} + r^{2}\right)^{0.5}} & \text{otherwise} \end{cases}\right)}{2}$$