# Tutorial rápido de Python para Matemáticos

© Ricardo Miranda Martins, 2022 - http://www.ime.unicamp.br/~rmiranda/

## Índice

1. [Introdução](1-intro.html)
2. [Python é uma boa calculadora!](2-calculadora.html) [(código fonte)](2-calculadora.ipynb)
3. [Resolvendo equações](3-resolvendo-eqs.html)  [(código fonte)](3-resolvendo-eqs.ipynb)
4. [Gráficos](4-graficos.html)  [(código fonte)](4-graficos.ipynb)
5. **[Sistemas lineares e matrizes](5-lineares-e-matrizes.html)**  [(código fonte)](5-lineares-e-matrizes.ipynb)
6. [Limites, derivadas e integrais](6-limites-derivadas-integrais.html)  [(código fonte)](6-limites-derivadas-integrais.ipynb)
7. [Equações diferenciais](7-equacoes-diferenciais.html)  [(código fonte)](7-equacoes-diferenciais.ipynb)

# Resolvendo sistemas lineares

Resolver sistemas lineares é o ganha pão de matemáticos. Aliás, você sabe por quê dedicamos tanto tempo ensinando alunos a resolverem sistemas lineares? Simples: é que não sabemos direito resolver os não-lineares.

Existem basicamente duas formas do Python resolver sistemas lineares: pelo comando tradicional ```solve``` do SymPy ou pelo comando específico para sistemas lineares, o ```linsolve```. Essa segunda forma é mais rápida se o sistema for muito grande. No entanto, como você já deve imaginar, resolver sistemas lineares usando o NumPy é ainda mais rápido do que usando o SymPy.

Para sistemas muito grandes (muito mais que 1000 equações/variáveis), existem outros pacotes mais eficientes - se for seu caso, imagino que você não esteja aprendendo Python aqui nesse tutorial.

A rotina abaixo resolve o sistema

$$\left\{\begin{array}{lcl}
x+y-2z&=&1,\\
x+y-z&=&3,\\
3x+y-z&=&3
\end{array}\right.$$
usando o ```linsolve``` do SymPy.

```python
import sympy as sp
x, y, z = sp.symbols('x y z')

eq1=sp.Eq(x+y-2*z,1)
eq2=sp.Eq(x+y-z,3)
eq3=sp.Eq(3*x+y-z,3)

sp.linsolve((eq1,eq2,eq3),(x,y,z))
```

$\displaystyle \left\{\left( 0, \  5, \  2\right)\right\}$

Se quisermos a solução num outro formato, podemos usar o ```solve```:

```python
sp.solve((eq1,eq2,eq3),(x,y,z))
```

    {x: 0, y: 5, z: 2}

## Um exercício legal para testar nossos conhecimentos

Sejam $P=(\alpha_1,\beta_1,\gamma_1)$, $Q=(\alpha_2,\beta_2,\gamma_2)$ e $R=(\alpha_3,\beta_3,\gamma_3)$ pontos não-colineares em $\mathbb R^3.$ Encontre a equação do plano $\pi$, da forma $ax+by+cz=d$, que contém estes três pontos.

Primeiro vamos a uma solução "algébrica", levando em conta a equação do plano. Vamos usar os valores $P=(0,0,1)$, $Q=(3,1,0)$ e $R=(0,2,2)$ no exemplo abaixo.

```python
import sympy as sp

# define as variaveis e constantes que vamos usar
x, y, z = sp.symbols('x y z')
a, b, c, d = sp.symbols('a b c d')

# equacao do plano na forma geral
eq = sp.Eq(a*x+b*y+c*z,d)

# definindo os pontos
P=sp.Array([0,0,1])
Q=sp.Array([3,1,0])
R=sp.Array([0,2,2])

# substituindo x,y,z na equacao do plano pelos pontos
eq1=eq.subs([(x,P[0]),(y,P[1]),(z,P[2])])
eq2=eq.subs([(x,Q[0]),(y,Q[1]),(z,Q[2])])
eq3=eq.subs([(x,R[0]),(y,R[1]),(z,R[2])])

# as equacoes acima resultam num sistema nas variaveis a, b, c, d.
# resolvendo o sistema:
sol=sp.linsolve((eq1,eq2,eq3),(a,b,c,d))

#armazenando as solucoes
(A,B,C,D)=tuple(*sol)

# finalmente, exibindo a equação do plano, substituindo a,b,c,d pelos
# valores que encontramos
eq.subs([(a,A),(b,B),(c,C),(d,D)])

```

$\displaystyle \frac{d x}{2} - \frac{d y}{2} + d z = d$

A solução acima fica na dependência de uma constante, no caso $d$. Pode-se substituir qualquer valor para $d$.

Agora vamos a uma solução um pouco mais geométrica. Vamos obter o vetor normal do plano como sendo o produto vetorial $n=u\times v$, onde $u=PQ$ e $v=PR$. A seguir, obtemos a equação do plano utilizando um dos pontos por onde ele passa. A solução é mais longa, porém é bem mais elegante e evita o aparecimento de constantes indesejadas.

```python
# importando a biblioteca para calculos simbolicos
import sympy as sp

# vamos usar a biblioteca NumPy para calcular o produto vetorial e tambem
# o produto interno
import numpy as np

# define as variaveis e constantes que vamos usar
x, y, z = sp.symbols('x y z')
a, b, c, d = sp.symbols('a b c d')

# equacao do plano na forma geral
eq = sp.Eq(a*x+b*y+c*z,d)

# definindo os pontos
P=sp.Array([0,0,1])
Q=sp.Array([3,1,0])
R=sp.Array([0,2,2])

# criando os vetores
u=Q-P
v=R-P

# obtendo o vetor normal
n=np.cross(u,v)
(a,b,c)=n

# termo nao-homogeneo
d=np.dot(n,P)

# exibindo a equação do plano
sp.Eq(a*x+b*y+c*z,d)
```

$\displaystyle 3 x - 3 y + 6 z = 6$

# Matrizes

Claro que poderíamos ter mesclado os capítulos sobre matrizes e sistemas lineares. Só deixamos separado para poder explorar um pouco melhor as propriedades desses retângulos cheios de números que nós adoramos.

Aliás, você sabe de onde vem a regra estranha de multiplicar matrizes? Vem da composição de operadores lineares. O Python trabalha com matrizes muito bem. Só devemos ter um pouco de cuidado com a notação.

Vamos fixar a notação e trabalhar abaixo com as matrizes $$A=\begin{pmatrix}1&2\\ -3&1\end{pmatrix},$$ $$B=\begin{pmatrix}2\\ 4\end{pmatrix}$$ e
$$C=\begin{pmatrix}2&5\\1&1\end{pmatrix}$$
em nossos exemplos.

```python
import sympy as sp

A=sp.Matrix([[1,2],[-3,1]])
B=sp.Matrix([[2],[4]])
C=sp.Matrix([[2,5],[4,10]])
```

```python
A
```

$\displaystyle \left[\begin{matrix}1 & 2\\-3 & 1\end{matrix}\right]$

```python
B
```

$\displaystyle \left[\begin{matrix}2\\4\end{matrix}\right]$

```python
C
```

$\displaystyle \left[\begin{matrix}2 & 5\\4 & 10\end{matrix}\right]$

Algumas matrizes especiais podem ser criadas de modo mais simples. Por exemplo, a identidade $n\times n$ pode ser criada com o comando ```eye(n)```.

```python
sp.eye(2)
```

$\displaystyle \left[\begin{matrix}1 & 0\\0 & 1\end{matrix}\right]$

Já a matriz nula $m\times n$ pode ser criada com o comando ```zeros(m,n)```:

```python
sp.zeros(2,2)
```

$\displaystyle \left[\begin{matrix}0 & 0\\0 & 0\end{matrix}\right]$

Podemos também criar de modo muito fácil uma matriz $m\times n$ só com 1's com o comando ```ones(m,n)```:

```python
sp.ones(2,2)
```

$\displaystyle \left[\begin{matrix}1 & 1\\1 & 1\end{matrix}\right]$

O produto de matrizes é feito com o mesmo símbolo usual: para obter o produto de A por B, o comando é ```A*B```:

```python
A*C
```

$\displaystyle \left[\begin{matrix}10 & 25\\-2 & -5\end{matrix}\right]$

```python
A*B
```

$\displaystyle \left[\begin{matrix}10\\-2\end{matrix}\right]$

Lembre-se que nem sempre dá para multiplicar duas matrizes - só podemos multiplicar uma matriz $m\times n$ por uma $n\times r$. Se tentarmos com matrizes de dimensões "estranhas", vai dar erro:

```python
B*A
```

Lembre-se também que o produto de matrizes não é comutativo, ou seja, $A\cdot C$ pode ser diferente de $C\cdot A$, ou seja, $AC-CA$ pode ser diferente de zero:

```python
A*C-C*A
```

$\displaystyle \left[\begin{matrix}23 & 16\\24 & -23\end{matrix}\right]$

Se $A$ é uma matriz quadrada, dizemos que ela é invertível se existe outra matriz $D$ tal que $AD=DA=I_{2\times 2}$. Essa matriz $D$ é chamada de matriz inversa de $A$ e denotada por $D=A^{-1}$. Nem toda matriz admite inversa, mas quando a inversa existe ela pode ser calculada no Python com o comando ```A**(-1)```:

```python
A**(-1)
```

$\displaystyle \left[\begin{matrix}\frac{1}{7} & - \frac{2}{7}\\\frac{3}{7} & \frac{1}{7}\end{matrix}\right]$

Uma propriedade das matrizes que tem inversa é que ela tem o determinante diferente de zero. O determinante é uma "propriedade" da matriz, então podemos "resgatar" o determinante de $A$ com o sufixo ```A.det()```:

```python
A.det()
```

$\displaystyle 7$

A mesma lógica é seguida se quisermos os autovalores ou autovetores da matriz:

```python
A.eigenvals()
```

    {1 - sqrt(6)*I: 1, 1 + sqrt(6)*I: 1}

```python
A.eigenvects()
```

    [(1 - sqrt(6)*I,
      1,
      [Matrix([
       [sqrt(6)*I/3],
       [          1]])]),
     (1 + sqrt(6)*I,
      1,
      [Matrix([
       [-sqrt(6)*I/3],
       [           1]])])]

Se você achou a saída do comando acima um pouco feia, talvez prefira usar o ```pprint``` (pretty print):

```python
sp.init_printing(use_unicode=True)
A.eigenvects()
```

Agora sim, dá pra entender né? A saída é composta pelo autovalor, sua multiplicidade, e o(s) autovetor(es) associado(s). Vamos fazer um exemplo $3\times 3$ que tem um autovalor de multiplicidade 2 para ilustrar melhor isso:

```python
M=sp.Matrix([[3,0,0],[0,3,0],[0,1,1]])
M.eigenvects()
```

Determinar a solução do sistema linear $AX=0$ é o mesmo que calcular o "núcleo" da matriz $A$. Isso pode ser feito com o sufixo ```nullspace()```. Vamos fazer isso para as matrizes $A$ e $C$ para comparar os resultados:

```python
A.nullspace()
```

```python
C.nullspace()
```

Note que o resultado acima nos permite concluir que a matriz $C$ tem um autovalor nulo, e o autovetor associado é justamente a resposta anterior, que é o gerador do núcleo de $C$. Quer conferir? Então toma:

```python
C.eigenvects()
```

Também podemos obter os vetores que são geradores da imagem de $A$, ou seja, se considerarmos o operador $T_A:\mathbb R^2\rightarrow\mathbb R^2$, $T_A(v)\mapsto Av$, então o sufixo ```columnspace()``` nos dará os vetores que são geradores da imagem desse operador:

```python
A.columnspace()
```

```python
C.columnspace()
```

Uma coisa que sempre fazemos com matrizes é calcular a sua forma de Jordan, ou, quando possível, diagonalizá-la. Isso pode ser feito com o comando ```diagonalize()```:

```python
A.diagonalize()
```

Esse comando retorna duas matrizes: a primeira delas é a matriz mudança de base; a segunda é a forma diagonal da matriz. Se quisermos usar notações para essas matrizes, vamos denotar por $Q$ a matriz mudança de base e $D$ a matriz diagonal:

```python
Q,D = A.diagonalize()
```

Da teoria, sabemos que $Q\cdot D\cdot Q^{-1}=A$. Em geral, isso só dá certo de usarmos o ```simplify``` no produto matricial do lado esquerdo:

```python
sp.simplify(Q*D*(Q**(-1)))
```

## Um exercício interessante: reconhecimento de cônicas

Considere uma equação quadrática da forma $$ax^2+bxy+cy^2+dx+ey+f=0.$$

Você já deve saber que a representação dos pontos $(x,y)$ que satisfaem essa equação tem uma classificação bem detalhada: basicamente, com exceção de alguns casos degenerados, teremos ou uma elipse ou uma hipérbole ou uma parábola. São as famosas "cônicas", curvas que são obtidas na interseção de um plano com um cone duplo.

O procedimento mais comum usado para decidir qual dos casos temos, e para escrever uma equação mais "simpática" para essa cônica é usar autovalores e autovetores. Vamos fazer isso no exemplo abaixo.

Você pode trocar a equação que deve continuar funcionando (cuidado com algumas mudanças de coordenadas e com divisões por zero que podem aparecer no processo ao alterar os coeficientes).

A ideia do "reconhecimento de cônicas" é razoavelmente simples, mas os cálculos são bem chatos de fazer na mão.

A equação $$ax^2+bxy+cy^2+dx+ey+f=0$$ pode ser reescrita em forma matricial como
$$\begin{pmatrix}x&y
\end{pmatrix}\begin{pmatrix}a&b/2\\ b/2&c\end{pmatrix}\begin{pmatrix}x\\y
\end{pmatrix} +\begin{pmatrix}d&e
\end{pmatrix}\begin{pmatrix}x\\y
\end{pmatrix}+\begin{pmatrix}f\end{pmatrix}=0.$$

Como a matriz $\begin{pmatrix}f\end{pmatrix}$ é $1\times 1$, em geral não iremos usar a notação matricial e escrever somente $f$:

$$\begin{pmatrix}x&y
\end{pmatrix}\begin{pmatrix}a&b/2\\ b/2&c\end{pmatrix}\begin{pmatrix}x\\y
\end{pmatrix} +\begin{pmatrix}d&e
\end{pmatrix}\begin{pmatrix}x\\y
\end{pmatrix}+f=0.$$

Denote por $$A=\begin{pmatrix}a&b/2\\ b/2&c\end{pmatrix}.$$ Essa matriz satisfaz $A=A^t$, então pelo Teorema Espectral provado por Cauchy em 1829, existem matrizes $Q$ e $D$, sendo $D$ uma matriz diagonal, de modo que $$QD Q^{-1}=A,$$ o que é equivalente a $$D=Q^{-1}AQ.$$

Podemos construir as matrizes $Q$ e $D$ assim: a matriz $D$ é a matriz diagonal com os autovalores de $A$ e a matriz $Q$ é uma matriz cujas colunas são autovetores normais (de norma 1) de $A$, satifazendo $Q^t=Q^{-1}$. Após calculá-las, realizamos a mudança de coordenadas

$$\begin{pmatrix}x\\y\end{pmatrix}=Q\begin{pmatrix}u\\v\end{pmatrix}.$$

Substituindo na equação da cônica, temos
$$\left(Q\begin{pmatrix}u\\v\end{pmatrix}\right)^t \begin{pmatrix}a&b/2\\ b/2&c\end{pmatrix}Q\begin{pmatrix}u\\v\end{pmatrix} +\begin{pmatrix}d&e
\end{pmatrix}Q\begin{pmatrix}u\\v\end{pmatrix}+f=0,$$
ou seja,
$$\begin{pmatrix}u&v
\end{pmatrix}Q^t AQ\begin{pmatrix}u\\v\end{pmatrix} +\begin{pmatrix}d&e
\end{pmatrix}Q\begin{pmatrix}u\\v\end{pmatrix}+f=0.$$

Usando que $Q^tAQ=D$ a equação acima fica
$$\begin{pmatrix}u&v
\end{pmatrix}D\begin{pmatrix}u\\v\end{pmatrix} +\begin{pmatrix}d&e
\end{pmatrix}Q\begin{pmatrix}u\\v\end{pmatrix}+f=0.$$

Se $$D=\begin{pmatrix}\lambda&0\\0&\mu\end{pmatrix},$$ onde $\lambda,\mu$ são os autovalores de $A$, então escrevendo novamente a equação da cônica temos
$$\lambda u^2+\mu v^2+\alpha u+\beta v+f=0,$$ para alguns valores de $\alpha,\beta$. O próximo passo é completar quadrados para deixar a equação mais simples. Aqui é preciso tomar cuidado, e a teoria não pode ser tão geral, pois algumas condições degeneradas podem ocorrer ($\lambda$ ou $\mu$ serem zero, por exemplo). Vamos supor que $\lambda\neq 0$, $\mu\neq 0$ e seguir com as contas.

Reescrevemos a equação anterior como
$$\lambda\left( u^2+\dfrac{\alpha u}{\lambda}\right)+\mu \left(v^2+\dfrac{\beta v}{\mu}\right)+f=0.$$

Completando quadrados, temos
$$\lambda\left( u^2+\dfrac{\alpha u}{\lambda}\right)+\mu \left(v^2+\dfrac{\beta v}{\mu}\right)+f=0$$
e daí
$$\lambda\left( u^2+\dfrac{\alpha u}{\lambda}+ \dfrac{\alpha^2}{4\lambda^2} \right)+\mu \left(v^2+\dfrac{\beta v}{\mu}+\dfrac{\beta^2}{4\mu^2}\right)+f-\dfrac{\alpha^2}{4\lambda}-\dfrac{\beta^2}{4\mu}=0.$$

Agrupando tudo, ficamos com a equação
$$\lambda\left( u+\dfrac{\alpha}{2\lambda} \right)^2+\mu \left(v+\dfrac{\beta}{2\mu}\right)^2+\tilde f=0,$$ onde $\tilde f=f-\dfrac{\alpha^2}{4\lambda}-\dfrac{\beta^2}{4\mu}$.

Fazendo uma nova mudança de coordenadas para $w=u+\dfrac{\alpha}{2\lambda}$, $z=v+\dfrac{\beta}{2\mu}$ ficamos com a equacão $$\lambda w^2+\mu z^2+\tilde f=0.$$

Conhecendo os valores de $\lambda,\mu,\tilde f$ poderemos reconhecer a cônica! Vamos agora fazer os cálculos de tudo isso no Python, e depois fazer alguns gráficos.

```python
a=1
b=2
c=-3
d=1
e=2
f=-4
x, y = sp.symbols('x y')
sp.Eq(a*x**2+b*x*y+c*y**2+d*x+e*y+f,0)
```

Vantagens do computador: vamos usar o Python pra pegar uma dica do que esperar lá no fim, plotando os pontos que satisfazem essa equação. O código abaixo é uma versão simplificada do nosso código para plotar curvas de nível (no caso, a curva de nível 0).

```python
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
delta = 0.01
x = np.arange(-4.0, 4.0, delta)
y = np.arange(-4.0, 4.0, delta)
x, y = np.meshgrid(x, y)
z = a*x**2+b*x*y+c*y**2+d*x+e*y+f

fig = plt.figure()
ax = fig.add_subplot()
ax.spines['left'].set_position('center')
ax.spines['bottom'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')

plt.contour(x, y, z,0)
```

Humm.. me parece que teremos uma hipérbole. Isso significa que podemos esperar um autovalor positivo e um negativo. Vamos começar as contas definindo a matriz A.

Aqui vamos usar um truque: queremos levar todos esses cálculos de forma algébrica, sem aproximações numéricas. Só que na definição da matriz $A$, temos um $b/2$: se digitarmos somente $b/2$, o resultado vai ser em ponto flutuante. Então vamos forçar o SymPy a usar o $b/2$ como sendo um número racional, com o comando ```Rational(b,2)```.

```python
A=sp.Matrix([[a,sp.Rational(b, 2)],[sp.Rational(b, 2),c]])
```

Agora vamos pedir ao Python para diagonalizar a matriz $A$, salvar a matriz diagonal como $D$ e a matriz mudança de base como $Q$:

```python
Q,D = A.diagonalize()
```

Conferindo como estão as matrizes:

```python
Q,D
```

Para que a nossa estratégia de mudança de base para reconhecimento de cônica funcione, precisamos pegar a matriz de mudança de base como sendo uma matriz cujas colunas formam uma base ortonormal de $\mathbb R^2$. No momento temos somente uma base. Vamos então escolher outra matriz $Q$ normalizando as colunas da matriz atual. Isso pode ser feito com o comando abaixo:

```python
n1=sp.simplify(sp.sqrt(Q[0]**2+Q[2]**2))
n2=sp.simplify(sp.sqrt(Q[1]**2+Q[3]**2))
Q2=sp.simplify(sp.Matrix([[Q[0]/n1,Q[1]/n2],[Q[2]/n1,Q[3]/n2]]))
Q=Q2
Q
```

Note que agora a matriz $Q$ satisfaz o que queríamos: $Q^t=Q^{-1}$..

```python
sp.simplify(Q**(-1)-Q.transpose())
```

... e ainda vale $Q^{-1}AQ=D$:

```python
sp.simplify(Q**(-1)*A*Q)
```

Vamos agora ver como ficou a cônica após a primeira mudança de coordenada que fizemos:

```python
# aplicando a mudança de coordenadas; z2 será a nova equacao

x, y = sp.symbols('x y')
z2=sp.simplify(
    (sp.Matrix([[x,y]])*(Q**(-1)*A*Q)*sp.Matrix([[x],[y]]))+(sp.Matrix([[d,e]]))*Q*(sp.Matrix([[x],[y]]))+sp.Matrix([[f]])
)[0]
```

```python
sp.init_printing(use_unicode=False)
```

```python
# exibindo a nova equacao, e como vamos usá-la para plotar, passando tudo
# pra ponto flutuante. você vai precisar copiar o resultado desse comando e
# colar na próxima linha de código, no z = ....
z3=z2.evalf()
repr(z3)
```

    '1.23606797749979*x**2 + 1.43275483056186*x - 3.23606797749979*y**2 + 1.71674505838791*y - 4.0'

```python
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
delta = 0.01
x = np.arange(-4.0, 4.0, delta)
y = np.arange(-4.0, 4.0, delta)
x, y = np.meshgrid(x, y)
z3 = 1.23606797749979*x**2 + 1.43275483056186*x - 3.23606797749979*y**2 + 1.71674505838791*y - 4.0

fig = plt.figure()
ax = fig.add_subplot()
ax.spines['left'].set_position('center')
ax.spines['bottom'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')

plt.contour(x, y, z3,0)
```

Olha só, já conseguimos desentortar a hipérbole!! Agora só precisamos da segunda mudança de coordenadas para poder centralizá-la. Vamos precisar calcular $\alpha$, $\beta$ e os autovalores $\lambda$ e $\mu$.

```python
lam = D[0]
mu = D[3]
```

Note que $\alpha$ é como chamamos o coeficiente de $x$ na expressão pós-primeira mudança de coordenadas. Essa expressão pra gente está guardada na variável ```z2```. Como vamos recuperá-la? Bom, uma possibilidade é derivar essa expressão em $x$ e depois fazer todas as outras variáveis iguais a zero (pense um pouco nessa estratégia).

```python
x, y = sp.symbols('x y')
termolinear=sp.simplify((sp.Matrix([[d,e]]))*Q*(sp.Matrix([[x],[y]])))[0]
alfa=sp.diff(termolinear,x)
beta=sp.diff(termolinear,y)
```

Agora já podemos plotar o gráfico final:

```python
# definindo quem é f-til
ftil=f-(alfa**2)/(4*lam)-beta**2/(4*mu)

# resetando as variáveis e escrevendo a equacao final
x, y = sp.symbols('x y')
print((lam*x**2 + mu*y**2 +ftil).evalf())

# plotando
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
delta = 0.01
x = np.arange(-4.0, 4.0, delta)
y = np.arange(-4.0, 4.0, delta)
x, y = np.meshgrid(x, y)
z4 = 1.23606797749979*x**2 - 3.23606797749979*y**2 - 4.1875

fig = plt.figure()
ax = fig.add_subplot()
ax.spines['left'].set_position('center')
ax.spines['bottom'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')

plt.contour(x, y, z4,0)
```

Por fim, vamos plotar todas as curvas num único sistema de eixos, para você ver o que aconteceu. A curva em vermelho é a original, a curva em azul é após a mudança de coordenadas do tipo rotação, e a curva preta é após a segunda mudança de coordenadas, de translação.

```python
# plotando
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
delta = 0.01
x = np.arange(-4.0, 4.0, delta)
y = np.arange(-4.0, 4.0, delta)
x, y = np.meshgrid(x, y)

z = a*x**2+b*x*y+c*y**2+d*x+e*y+f
z3 = 1.23606797749979*x**2 + 1.43275483056186*x - 3.23606797749979*y**2 + 1.71674505838791*y - 4.0
z4 = 1.23606797749979*x**2 - 3.23606797749979*y**2 - 4.1875

fig = plt.figure()
ax = fig.add_subplot()
ax.spines['left'].set_position('center')
ax.spines['bottom'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')

plt.contour(x, y, z,0,colors=['red'])
plt.contour(x, y, z3,0,colors=['blue'])
plt.contour(x, y, z4,0,colors=['black'])
plt.show()
```

E a equação final é:

```python
x, y = sp.symbols('x y')
sp.Eq(lam*x**2+mu*y**2+ftil,0)
```


# The binomial distribution

Copyright 2016 Allen Downey

MIT License: http://opensource.org/licenses/MIT

```python
from __future__ import print_function, division

%matplotlib inline
%precision 6

import matplotlib.pyplot as plt

import numpy as np
```

```python
from inspect import getsourcelines

def show_code(func):
    lines, _ = getsourcelines(func)
    for line in lines:
        print(line, end='')
```

## Pmf

Here's a Pmf class that represents a Probability Mass Function, implemented using a Python dictionary that maps from possible outcomes to their probabilities.

```python
from distribution import Pmf
show_code(Pmf)
```

    class Pmf:

        def __init__(self, d=None):
            """Initializes the distribution.

            d: map from values to probabilities
            """
            self.d = {} if d is None else d

        def items(self):
            """Returns a sequence of (value, prob) pairs."""
            return self.d.items()

        def __repr__(self):
            """Returns a string representation of the object."""
            cls = self.__class__.__name__
            return '%s(%s)' % (cls, repr(self.d))

        def __getitem__(self, value):
            """Looks up the probability of a value."""
            return self.d.get(value, 0)

        def __setitem__(self, value, prob):
            """Sets the probability associated with a value."""
            self.d[value] = prob

        def __add__(self, other):
            """Computes the Pmf of the sum of values drawn from self and other.

            other: another Pmf or a scalar

            returns: new Pmf
            """
            if other == 0:
                return self

            pmf = Pmf()
            for v1, p1 in self.items():
                for v2, p2 in other.items():
                    pmf[v1 + v2] += p1 * p2
            return pmf

        __radd__ = __add__

        def total(self):
            """Returns the total of the probabilities."""
            return sum(self.d.values())

        def normalize(self):
            """Normalizes this PMF so the sum of all probs is 1.

            Args:
                fraction: what the total should be after normalization

            Returns: the total probability before normalizing
            """
            total = self.total()
            for x in self.d:
                self.d[x] /= total
            return total

        def mean(self):
            """Computes the mean of a PMF."""
            return sum(p * x for x, p in self.items())

        def var(self, mu=None):
            """Computes the variance of a PMF.

            mu: the point around which the variance is computed;
                    if omitted, computes the mean
            """
            if mu is None:
                mu = self.mean()

            return sum(p * (x - mu) ** 2 for x, p in self.items())

        def expect(self, func):
            """Computes the expectation of a given function, E[f(x)]

            func: function
            """
            return sum(p * func(x) for x, p in self.items())

        def display(self):
            """Displays the values and probabilities."""
            for value, prob in self.items():
                print(value, prob)

        def plot_pmf(self, **options):
            """Plots the values and probabilities."""
            xs, ps = zip(*sorted(self.items()))
            plt.plot(xs, ps, **options)

## The infamous biased coin

I'll create a Pmf that represents a biased coin, which has a 60% chance of landing heads and a 40% chance of landing tails.

If you are bothered by the fact that it is physically very difficult to bias a coin toss in this way, imagine a 10-sided die with `H` on 6 sides and `T` on 4 sides.

```python
coin = Pmf(dict(H=0.6, T=0.4))
coin.display()
```

    H 0.6
    T 0.4

We can use the `+` operator to compute the possible outcomes of two coin tosses and their probabilities.

```python
twice = coin + coin
twice.display()
```

    HH 0.36
    TT 0.16000000000000003
    TH 0.24
    HT 0.24

And similarly, the possible outcomes of three coin tosses.

```python
thrice = sum([coin]*3)
thrice.display()
```

    TTT 0.06400000000000002
    HTT 0.096
    THT 0.096
    HHH 0.216
    HHT 0.144
    HTH 0.144
    TTH 0.09600000000000002
    THH 0.144

Notice that the outcomes take the order of the tosses into account, so `HT` is considered a different outcome from `TH`.

If we don't care about the order and we only care about the number of heads and tails, we can loop through the outcomes and count the number of heads.

```python
from collections import Counter

for val, prob in sorted(thrice.items()):
    heads = val.count('H')
    print(heads, prob)
```

    3 0.216
    2 0.144
    2 0.144
    1 0.096
    2 0.144
    1 0.096
    1 0.09600000000000002
    0 0.06400000000000002

And we can make a new Pmf that maps from the total number of heads to the probability of that total.

```python
def make_pmf_heads(coin, n):
    coins = sum([coin]*n)
    pmf = Pmf()
    for val, prob in coins.items():
        heads = val.count('H')
        pmf[heads] += prob
    return pmf
```

Here's what it looks like:

```python
pmf_heads = make_pmf_heads(coin, 3)
pmf_heads.display()
```

    0 0.06400000000000002
    1 0.28800000000000003
    2 0.43199999999999994
    3 0.216

**Exerise:** Create `pmf_heads` for a few different values of `n` and plot them using the `plot_pmf` method.

```python
# Solution

for n in [5, 10, 15]:
    make_pmf_heads(coin, n).plot_pmf()
```

**Exerise:** Run the following example and see how long it takes.  Try it out with a few values of `n` and see how the run time depends on `n`.

```python
n = 15
%time make_pmf_heads(coin, n).plot_pmf()
```

This way of computing `pmf_heads` is not very efficient.  For `n` tosses, there are $2^n$ possible outcomes, and for large values of `n`, that is not tractable.

In the next section we will figure out a better way.

## The symbolic version

Looking at numerical output doesn't tell us much about how to generalize from this example.  We can learn more by replacing the numbers with symbols.

Here's a version of the Pmf where the probability of heads is the symbol `p`:

```python
from sympy import symbols

p = symbols('p')
sym_coin = Pmf(dict(H=p, T=1-p))
sym_coin.display()
```

    H p
    T -p + 1

Now we can see the distribution of the number of heads, which I'll call `k`, after a few tosses:

```python
make_pmf_heads(sym_coin, 2).display()
```

    0 (-p + 1)**2
    1 2*p*(-p + 1)
    2 p**2

```python
make_pmf_heads(sym_coin, 3).display()
```

    0 (-p + 1)**3
    1 3*p*(-p + 1)**2
    2 3*p**2*(-p + 1)
    3 p**3

```python
make_pmf_heads(sym_coin, 4).display()
```

    0 (-p + 1)**4
    1 4*p*(-p + 1)**3
    2 6*p**2*(-p + 1)**2
    3 4*p**3*(-p + 1)
    4 p**4

The general pattern is that probability of `k` heads after `n` tosses is the product of three terms

* The probability of `k` heads, which is `p**k`.
* The probability of `n-k` tails, which is `(1-p)**(n-k)`.
* An integer coefficient.

You might already know that the coefficient is the "binomial coefficient", which is written $n \choose k$ and pronounced "n choose k".  But pretend for a moment that you don't know that and let's figure it out.

## The binomial coefficient

To make the pattern easier to see, I'll create a fair coin where `p = 1-p = 1/2`

```python
fair_coin = Pmf(dict(H=p, T=p))
fair_coin.display()
```

    H p
    T p

Now the probability of all outcomes is `p**n`

```python
thrice = sum([fair_coin]*3)
thrice.display()
```

    TTT p**3
    HTT p**3
    THT p**3
    HHH p**3
    HHT p**3
    HTH p**3
    TTH p**3
    THH p**3

So when we count the number of heads, it is easier to see the coefficients.

```python
pmf_heads = make_pmf_heads(fair_coin, 3)
pmf_heads.display()
```

    0 p**3
    1 3*p**3
    2 3*p**3
    3 p**3

And even easier if we divide through by `p**n`

```python
for val, prob in pmf_heads.items():
    print(val, prob / p**3)
```

    0 1
    1 3
    2 3
    3 1

We can assemble the code from the previous cells into a function that prints the coefficients for a given value of `n`:

```python
def coefficients(n):
    fair_coin = Pmf(dict(H=p, T=p))
    pmf_heads = make_pmf_heads(fair_coin, n)
    for val, prob in pmf_heads.items():
        print(prob / p**n, end=' ')
    print()
```

Here are the coefficients for `n=3`

```python
coefficients(3)
```

    1 3 3 1

And here they are for `n` in the range from `1` to `9`

```python
for n in range(1, 10):
    coefficients(n)
```

    1 1
    1 2 1
    1 3 3 1
    1 4 6 4 1
    1 5 10 10 5 1
    1 6 15 20 15 6 1
    1 7 21 35 35 21 7 1
    1 8 28 56 70 56 28 8 1
    1 9 36 84 126 126 84 36 9 1

Now we can look for patterns.

* If we flip the coin `n` times, the coefficient associated with getting `k` heads is `1` if `k` is `0` or `n`.

* Otherwise, the `(n, k)` coefficient is the sum of two coefficients from the previous row: `(n-1, k)` and `(n-1, k-1)`

We can use these observations to compute the binomial coefficient recursively:

```python
def binomial_coefficient(n, k):
    if k==0 or k==n:
        return 1
    return binomial_coefficient(n-1, k) + binomial_coefficient(n-1, k-1)
```

And it yields the same results.

```python
binomial_coefficient(9, 5)
```

    126

Here are the results for `n` from 1 to 9 again.

```python
for n in range(1, 10):
    for k in range(n+1):
        print(binomial_coefficient(n, k), end=' ')
    print()
```

    1 1
    1 2 1
    1 3 3 1
    1 4 6 4 1
    1 5 10 10 5 1
    1 6 15 20 15 6 1
    1 7 21 35 35 21 7 1
    1 8 28 56 70 56 28 8 1
    1 9 36 84 126 126 84 36 9 1

So far so good.

**Exercise:**  SciPy provides a "special" function called `binom` that you can import from `scipy.special`.  Test it to confirm that it is consistent with our function.

Note that `scipy.special.binom` returns a float, so for large values of `n` it is only approximate.

```python
# Solution

from scipy.special import binom
binom(9, 5)
```

    126.000000

**Exercise:** The recursive implementation of `binomial_coefficient` is inefficient for large values of `n` because it computes the same intermediate results many times.  You can speed it up (a lot!) by memoizing previously computed results.

Write a version called `fast_binom` that caches results in a dictionary.

```python
# Solution

def fast_binom(n, k, cache={}):
    if k==0 or k==n:
        return 1
    try:
        return cache[n, k]
    except KeyError:
        res = fast_binom(n-1, k) + fast_binom(n-1, k-1)
        cache[n, k] = res
        return res

for n in range(1, 10):
    for k in range(n+1):
        print(binomial_coefficient(n, k), end=' ')
    print()
```

    1 1
    1 2 1
    1 3 3 1
    1 4 6 4 1
    1 5 10 10 5 1
    1 6 15 20 15 6 1
    1 7 21 35 35 21 7 1
    1 8 28 56 70 56 28 8 1
    1 9 36 84 126 126 84 36 9 1

## The Binomial PMF

Bringing it all together, we have

$PMF(k; n,p) = {n \choose k} p^k (1-p)^{n-k}$

This equation can be interpreted at two levels:

* If you want to know the probability of `k` heads in `n` tosses, where the probability of heads in each toss is `p`, you can use the formula on the right to compute it.

* More abstractly, this equation states that the formula on the right is the PMF of `k` with the parameters `n` and `p`.

To make the difference between these interpretations clear, let's look at two functions:

    1.  `eval_binomial_pmf` evaluates this formula for given values of `k`, `n`, and `p`.
    2.  `make_binomial_pmf` evaluates the formula for a range of values of `k`, and returns the resulting `Pmf` object.

First, here's `eval_binomial_pmf`:

```python
from scipy.special import binom

def eval_binomial_pmf(k, n, p):
    return binom(n, k) * p**k * (1-p)**(n-k)
```

We can use it to compute probabilities for each value of `k` directly:

```python
n = 3
p = 0.6

for k in range(n+1):
    print(k, eval_binomial_pmf(k, n, p))
```

    0 0.064
    1 0.288
    2 0.432
    3 0.216

And here's the corresponding `pmf_heads` for comparison

```python
coin = Pmf(dict(H=p, T=1-p))
make_pmf_heads(coin, n).display()
```

    0 0.06400000000000002
    1 0.28800000000000003
    2 0.43199999999999994
    3 0.216

They are the same, at least within floating point error.

Now here's `make_binomial_pmf`

```python
def make_binomial_pmf(n, p):
    pmf = Pmf()
    for k in range(n+1):
        pmf[k] = eval_binomial_pmf(k, n, p)
    return pmf
```

We can use it to make a Pmf that contains the possible values of `k` and their probabilities:

```python
pmf = make_binomial_pmf(n, p)
pmf.display()
```

    0 0.064
    1 0.288
    2 0.432
    3 0.216

Here's what the distribution of `k` looks like for given values of `n` and `p`:

```python
pmf.plot_pmf()
plt.xlabel('k');
```

If we hold `p` constant, we can see how the distribution of `k` changes as `n` increases

```python
p = 0.6
for n in [5, 10, 15]:
    make_binomial_pmf(n, p).plot_pmf(label=n)
plt.legend();
```

**Exercise:** Keeping `n=10`, plot the distribution of `k` for a few different values of `p`.

```python
# Solution

n = 10
for p in [0.2, 0.5, 0.8]:
    make_binomial_pmf(n, p).plot_pmf(label=p)
plt.legend();
```

The Pmf objects we just created represent differnet distributions of `k` based on different values of the parameters `n` and `p`.

As `make_binomial_pmf` demonstrates, if you give me `n` and `p`, I can compute a distribution of `k`.

Speaking casually, people sometimes refer to this equation as the PMF of "the" binomial distribution:

$PMF(k; n,p) = {n \choose k} p^k (1-p)^{n-k}$

More precisely, it is a "family" of distributions, where the parameters `n` and `p` specify a particular member of the family.

**Exercise:**  Suppose you toss a fair coin 10 times.  What is the probability of getting 5 heads?

If you run this experiment many times, what is the mean number of heads you expect?  What is the variance in the number of heads?

```python
# Solution

n = 10
p = 0.5
pmf = make_binomial_pmf(n, p)
print(pmf[5])
print(pmf.mean())
print(pmf.var())
```

    0.24609375
    5.0
    2.5

**Exercise:**  Suppose you toss a fair coin 10 times.  What is the probability of getting fewer than 5 heads?

If you run this experiment many times, what is the median number of heads you expect?  What is the interquartile range (IQR) in the number of heads?

Hint: You might want to make a CDF.

```python
# Solution

from distribution import compute_cumprobs, Cdf

xs, ps = compute_cumprobs(pmf.d)
cdf = Cdf(xs, ps)
print(cdf[4])

low, median, high = cdf.values([0.25, 0.5, 0.75])
print(median, high-low)
```

    0.376953125
    5 2

```python

```


# 17 PDEs: Waves

(See *Computational Physics* Ch 21 and *Computational Modeling* Ch 6.5.)

## Background: waves on a string

Assume a 1D string of length $L$ with mass density per unit length $\rho$ along the $x$ direction. It is held under constant tension $T$ (force per unit length). Ignore frictional forces and the tension is so high that we can ignore sagging due to gravity.

### 1D wave equation
The string is displaced in the $y$ direction from its rest position, i.e., the displacement $y(x, t)$ is a function of space $x$ and time $t$.

For small relative displacements $y(x, t)/L \ll 1$ and therefore small slopes $\partial y/\partial x$ we can describe $y(x, t)$ with a *linear* equation of motion:

Newton's second law applied to short elements of the string with length $\Delta x$ and mass $\Delta m = \rho \Delta x$: the left hand side contains the *restoring force* that opposes the displacement, the right hand side is the acceleration of the string element:

\begin{align}
\sum F_{y}(x) &= \Delta m\, a(x, t)\\
T \sin\theta(x+\Delta x) - T \sin\theta(x) &= \rho \Delta x \frac{\partial^2 y(x, t)}{\partial t^2}
\end{align}

The angle $\theta$ measures by how much the string is bent away from the resting configuration.

Because we assume small relative displacements, the angles are small ($\theta \ll 1$) and we can make the small angle approximation

$$
\sin\theta \approx \tan\theta = \frac{\partial y}{\partial x}
$$

and hence

\begin{align}
T \left.\frac{\partial y}{\partial x}\right|_{x+\Delta x} - T \left.\frac{\partial y}{\partial x}\right|_{x}  &= \rho \Delta x \frac{\partial^2 y(x, t)}{\partial t^2}\\
\frac{T \left.\frac{\partial y}{\partial x}\right|_{x+\Delta x} - T \left.\frac{\partial y}{\partial x}\right|_{x}}{\Delta x} &= \rho \frac{\partial^2 y}{\partial t^2}
\end{align}

or in the limit $\Delta x \rightarrow 0$ a linear hyperbolic PDE results:

\begin{gather}
\frac{\partial^2 y(x, t)}{\partial x^2} = \frac{1}{c^2} \frac{\partial^2 y(x, t)}{\partial t^2}, \quad c = \sqrt{\frac{T}{\rho}}
\end{gather}

where $c$ has the dimension of a velocity. This is the (linear) **wave equation**.

### General solution: waves

General solutions are propagating waves:

If $f(x)$ is a solution at $t=0$ then

$$
y_{\mp}(x, t) = f(x \mp ct)
$$

are also solutions at later $t > 0$.

Because of linearity, any linear combination is also a solution, so the most general solution contains both right and left propagating waves

$$
y(x, t) = A f(x - ct) + B g(x + ct)
$$

(If $f$ and/or $g$ are present depends on the initial conditions.)

In three dimensions the wave equation is

$$
\boldsymbol{\nabla}^2 y(\mathbf{x}, t) - \frac{1}{c^2} \frac{\partial^2 y(\mathbf{x}, t)}{\partial t^2} = 0\
$$

### Boundary and initial conditions

* The boundary conditions could be that the ends are fixed

  $$y(0, t) = y(L, t) = 0$$

* The *initial condition* is a shape for the string, e.g., a Gaussian at the center

  $$
  y(x, t=0) = g(x) = y_0 \frac{1}{\sqrt{2\pi\sigma}} \exp\left[-\frac{(x - x_0)^2}{2\sigma^2}\right]
  $$

  at time 0.
* Because the wave equation is *second order in time* we need a second initial condition, for instance, the string is released from rest:

  $$
  \frac{\partial y(x, t=0)}{\partial t} = 0
  $$

  (The derivative, i.e., the initial displacement velocity is provided.)

### Analytical solution
Solve (as always) with *separation of variables*.

$$
y(x, t) = X(x) T(t)
$$

and this yields the general solution (with boundary conditions of fixed string ends and initial condition of zero velocity) as a superposition of normal modes

$$
y(x, t) = \sum_{n=0}^{+\infty} B_n \sin k_n x\, \cos\omega_n t,
\quad \omega_n = ck_n,\ k_n = n \frac{2\pi}{L} = n k_0.
$$

(The angular frequency $\omega$ and the wave vector $k$ are determined from the boundary conditions.)

The coefficients $B_n$ are obtained from the initial shape:

$$
y(x, t=0) = \sum_{n=0}^{+\infty} B_n \sin n k_0 x = g(x)
$$

In principle one can use the fact that $\int_0^L dx \sin m k_0 x \, \sin n k_0 x = \pi \delta_{mn}$ (orthogonality) to calculate the coefficients:

\begin{align}
\int_0^L dx \sin m k_0 x \sum_{n=0}^{+\infty} B_n \sin n k_0 x &= \int_0^L dx \sin(m k_0 x) \, g(x)\\
\pi \sum_{n=0}^{+\infty} B_n \delta_{mn} &= \dots \\
B_m &= \pi^{-1} \dots
\end{align}

(but the analytical solution is ugly and I cannot be bothered to put it down here.)

## Numerical solution

1. discretize wave equation
2. time stepping: leap frog algorithm (iterate)

Use the central difference approximation for the second order derivatives:

\begin{align}
\frac{\partial^2 y}{\partial t^2} &\approx \frac{y(x, t+\Delta t) + y(x, t-\Delta t) - 2y(x, t)}{\Delta t ^2} = \frac{y_{i, j+1} + y_{i, j-1} - 2y_{i,j}}{\Delta t^2}\\
\frac{\partial^2 y}{\partial x^2} &\approx \frac{y(x+\Delta x, t) + y(x-\Delta x, t) - 2y(x, t)}{\Delta x ^2} = \frac{y_{i+1, j} + y_{i-1, j} - 2y_{i,j}}{\Delta x^2}
\end{align}

and substitute into the wave equation to yield the *discretized* wave equation:

$$
\frac{y_{i+1, j} + y_{i-1, j} - 2y_{i,j}}{\Delta x^2} = \frac{1}{c^2} \frac{y_{i, j+1} + y_{i, j-1} - 2y_{i,j}}{\Delta t^2}
$$

Re-arrange so that the future terms $j+1$ can be calculated from the present $j$ and past $j-1$ terms:

$$
y_{i,j+1} = 2(1 - \beta^2)y_{i,j} - y_{i, j-1} + \beta^2 (y_{i+1,j} + y_{i-1,j}), \quad
\beta := \frac{c}{\Delta x/\Delta t}
$$

This is the time stepping algorithm for the wave equation.

## Numerical implementation

```python
# if you have plotting problems, try
# %matplotlib inline
%matplotlib notebook

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

plt.style.use('ggplot')
```

### Student version

```python
L = 0.5  # m
Nx = 50
Nt = 100

Dx = L/Nx
Dt = 1e-4  # s

rho = 1.5e-2   # kg/m
tension = 150  # N

c = np.sqrt(tension/rho)

# TODO: calculate beta
beta = c*Dt/Dx
beta2 = beta**2

print("c = {0} m/s".format(c))
print("Dx = {0} m,  Dt = {1} s, Dx/Dt = {2} m/s".format(Dx, Dt, Dx/Dt))
print("beta = {}".format(beta))

X = np.linspace(0, L, Nx+1)  # need N+1!

def gaussian(x, y0=0.05, x0=L/2, sigma=0.1*L):
    return y0/np.sqrt(2*np.pi*sigma) * np.exp(-(x-x0)**2/(2*sigma**2))

# displacements at j-1, j, j+1
y0 = np.zeros_like(X)
y1 = np.zeros_like(y0)
y2 = np.zeros_like(y0)

# save array
y_t = np.zeros((Nt+1, Nx+1))

# boundary conditions
y0[0] = y0[-1] = y1[0] = y1[-1] = 0
y2[:] = y0

# initial conditions: velocity 0, i.e. no difference between y0 and y1
y0[1:-1] = y1[1:-1] = gaussian(X)[1:-1]

# save initial
t_index = 0
y_t[t_index, :] = y0
t_index += 1
y_t[t_index, :] = y1

for jt in range(2, Nt):
    y2[1:-1] = 2*(1-beta2)*y1[1:-1] - y0[1:-1] + beta2*(y1[2:] + y1[:-2])
    y0[:], y1[:] = y1, y2

    t_index += 1
    y_t[t_index, :] = y2
    print("Iteration {0:5d}".format(jt), end="\r")
else:
    print("Completed {0:5d} iterations: t={1} s".format(jt, jt*Dt))

```

    c = 100.0 m/s
    Dx = 0.01 m,  Dt = 0.0001 s, Dx/Dt = 100.0 m/s
    beta = 1.0
    Completed    99 iterations: t=0.0099 s

### Fancy version
Package as a function and can use `step` to only save every `step` time steps.

```python
def wave(L=0.5, Nx=50, Dt=1e-4, Nt=100, step=1,
        rho=1.5e-2, tension=150.):

    Dx = L/Nx

    #rho = 1.5e-2   # kg/m
    #tension = 150  # N

    c = np.sqrt(tension/rho)

    beta = c*Dt/Dx
    beta2 = beta**2

    print("c = {0} m/s".format(c))
    print("Dx = {0} m,  Dt = {1} s, Dx/Dt = {2} m/s".format(Dx, Dt, Dx/Dt))
    print("beta = {}".format(beta))

    X = np.linspace(0, L, Nx+1)  # need N+1!

    def gaussian(x, y0=0.05, x0=L/2, sigma=0.1*L):
        return y0/np.sqrt(2*np.pi*sigma) * np.exp(-(x-x0)**2/(2*sigma**2))

    # displacements at j-1, j, j+1
    y0 = np.zeros_like(X)
    y1 = np.zeros_like(y0)
    y2 = np.zeros_like(y0)

    # save array
    y_t = np.zeros((int(np.ceil(Nt/step)) + 1, Nx+1))

    # boundary conditions
    y0[0] = y0[-1] = y1[0] = y1[-1] = 0
    y2[:] = y0

    # initial conditions: velocity 0, i.e. no difference between y0 and y1
    y0[1:-1] = y1[1:-1] = gaussian(X)[1:-1]

    # save initial
    t_index = 0
    y_t[t_index, :] = y0
    if step == 1:
        t_index += 1
        y_t[t_index, :] = y1

    for jt in range(2, Nt):
        y2[1:-1] = 2*(1-beta2)*y1[1:-1] - y0[1:-1] + beta2*(y1[2:] + y1[:-2])
        y0[:], y1[:] = y1, y2

        if jt % step == 0 or jt == Nt-1:
            t_index += 1
            y_t[t_index, :] = y2
            print("Iteration {0:5d}".format(jt), end="\r")
    else:
        print("Completed {0:5d} iterations: t={1} s".format(jt, jt*Dt))

    return y_t, X, Dx, Dt, step

```

```python
y_t, X, Dx, Dt, step = wave()
```

    c = 100.0 m/s
    Dx = 0.01 m,  Dt = 0.0001 s, Dx/Dt = 100.0 m/s
    beta = 1.0
    Completed    99 iterations: t=0.0099 s

### 1D plot
Plot the output in the save array `y_t`. Vary the time steps that you look at with `y_t[start:end]`.

We indicate time by color changing.

```python
ax = plt.subplot(111)
ax.set_prop_cycle("color", [plt.cm.viridis(i) for i in np.linspace(0, 1, len(y_t))])
ax.plot(X, y_t[:40].T, alpha=0.5);
```

    <IPython.core.display.Javascript object>

### 3D plot
(Uses functions from previous lessons.)

```python
def plot_y(y_t, Dx, Dt, step=1):
    X, Y = np.meshgrid(range(y_t.shape[0]), range(y_t.shape[1]))
    Z = y_t.T[Y, X]
    fig = plt.figure()
    ax = fig.add_subplot(111, projection="3d")
    ax.plot_wireframe(Y*Dx, X*Dt*step, Z)
    ax.set_ylabel(r"time $t$ (s)")
    ax.set_xlabel(r"position $x$ (m)")
    ax.set_zlabel(r"displacement $y$ (m)")
    fig.tight_layout()
    return ax

def plot_surf(y_t, Dt, Dx, step=1, filename=None, offset=-1, zlabel=r'displacement',
             elevation=40, azimuth=20, cmap=plt.cm.coolwarm):
    """Plot y_t as a 3D plot with contour plot underneath.

    Arguments
    ---------
    y_t : 2D array
          displacement y(t, x)
    filename : string or None, optional (default: None)
          If `None` then show the figure and return the axes object.
          If a string is given (like "contour.png") it will only plot
          to the filename and close the figure but return the filename.
    offset : float, optional (default: 20)
          position the 2D contour plot by offset along the Z direction
          under the minimum Z value
    zlabel : string, optional
          label for the Z axis and color scale bar
    elevation : float, optional
          choose elevation for initial viewpoint
    azimuth : float, optional
          chooze azimuth angle for initial viewpoint
    """

    t = np.arange(y_t.shape[0])
    x = np.arange(y_t.shape[1])
    T, X = np.meshgrid(t, x)
    Y = y_t.T[X, T]

    fig = plt.figure()
    ax = fig.add_subplot(111, projection='3d')
    surf = ax.plot_surface(X*Dx, T*Dt*step, Y, cmap=cmap, rstride=2, cstride=2, alpha=1)
    cset = ax.contourf(X*Dx, T*Dt*step, Y, 20, zdir='z', offset=offset+Y.min(), cmap=cmap)

    ax.set_xlabel('x')
    ax.set_ylabel('t')
    ax.set_zlabel(zlabel)
    ax.set_zlim(offset + Y.min(), Y.max())

    ax.view_init(elev=elevation, azim=azimuth)

    cb = fig.colorbar(surf, shrink=0.5, aspect=5)
    cb.set_label(zlabel)

    if filename:
        fig.savefig(filename)
        plt.close(fig)
        return filename
    else:
        return ax
```

```python
plot_y(y_t, Dx, Dt)
```

    <IPython.core.display.Javascript object>

    <matplotlib.axes._subplots.Axes3DSubplot at 0x10e0aafd0>

```python
plot_surf(y_t, Dt, Dx, offset=-0.04, cmap=plt.cm.coolwarm)
```

    <IPython.core.display.Javascript object>

    <matplotlib.axes._subplots.Axes3DSubplot at 0x10f212cc0>

## von Neumann stability analysis: Courant condition

Assume that the solutions of the discretized equation can be written as normal modes

$$
y_{m,j} = \xi(k)^j e^{ikm\Delta x}, \quad t=j\Delta t,\ x=m\Delta x
$$

The time stepping algorith is stable if

$$
|\xi(k)| < 1
$$

Insert normal modes into the discretized equation

$$
y_{i,j+1} = 2(1 - \beta^2)y_{i,j} - y_{i, j-1} + \beta^2 (y_{i+1,j} + y_{i-1,j}), \quad
\beta := \frac{c}{\Delta x/\Delta t}
$$

and simplify (use $1-\cos x = 2\sin^2\frac{x}{2}$):

$$
\xi^2 - 2(1-2\beta^2 s^2)\xi + 1 = 0, \quad s=\sin(k\Delta x/2)
$$

The characteristic equation has roots

$$
\xi_{\pm} = 1 - 2\beta^2 s^2 \pm \sqrt{(1-2\beta^2 s^2)^2 - 1}.
$$

It has one root for

$$
\left|1-2\beta^2 s^2\right| = 1,
$$

i.e., for

$$
\beta s = 1
$$

We have two real roots for

$$
\left|1-2\beta^2 s^2\right| < 1 \\
\beta s > 1
$$

but one of the roots is always $|\xi| > 1$ and hence these solutions will diverge and not be stable.

For

$$
\left|1-2\beta^2 s^2\right| ≥ 1 \\
\beta s ≤ 1
$$

the roots will be *complex conjugates of each other*

$$
\xi_\pm = 1 - 2\beta^2s^2 \pm i\sqrt{1-(1-2\beta^2s^2)^2}
$$

and the *magnitude*

$$
|\xi_{\pm}|^2 =  (1 - 2\beta^2s^2)^2 - (1-(1-2\beta^2s^2)^2) = 1
$$

is unity: Thus the solutions will not grow and will be *stable* for

$$
\beta s ≤ 1\\
\frac{c}{\frac{\Delta x}{\Delta t}} \sin\frac{k \Delta x}{2} ≤ 1
$$

Assuming the "worst case" for the $\sin$ factor (namely, 1), the **condition for stability** is

$$
c ≤ \frac{\Delta x}{\Delta t}
$$

or

$$
\beta ≤ 1.
$$

This is also known as the **Courant condition**. When written as

$$
\Delta t ≤ \frac{\Delta x}{c}
$$

it means that the time step $\Delta t$ (for a given $\Delta x$) must be *smaller than the time that the  wave takes to travel one grid step*.

## Simplified simulation
The above example used real numbers for a cello string. Below is bare-bones where we just set $\beta$.

```python
L = 10.0
Nx = 100
Nt = 100
step = 1

Dx = L/Nx

beta2 = 1.0

X = np.linspace(0, L, Nx+1)  # need N+1!

def gaussian(x):
    return np.exp(-(x-5)**2)

# displacements at j-1, j, j+1
y0 = np.zeros_like(X)
y1 = np.zeros_like(y0)
y2 = np.zeros_like(y0)

# save array
y_t = np.zeros((int(np.ceil(Nt/step)) + 1, Nx+1))

# boundary conditions
y0[0] = y0[-1] = y1[0] = y1[-1] = 0
y2[:] = y0

# initial conditions: velocity 0, i.e. no difference between y0 and y1
y0[:] = y1[:] = gaussian(X)

# save initial
t_index = 0
y_t[t_index, :] = y0
if step == 1:
    t_index += 1
    y_t[t_index, :] = y1

for jt in range(2, Nt):
    y2[1:-1] = 2*(1-beta2)*y1[1:-1] - y0[1:-1] + beta2*(y1[2:] + y1[:-2])
    y0[:], y1[:] = y1, y2

    if jt % step == 0 or jt == Nt-1:
        t_index += 1
        y_t[t_index, :] = y2
        print("Iteration {0:5d}".format(jt), end="\r")
else:
    print("Completed {0:5d} iterations: t={1} s".format(jt, jt*Dt))

```

    Completed    99 iterations: t=0.0099 s

```python
%matplotlib inline
```

```python
ax = plt.subplot(111)
ax.set_prop_cycle("color", [plt.cm.viridis_r(i) for i in np.linspace(0, 1, len(y_t))])
ax.plot(X, y_t.T);
```

```python

```


```python
import numpy as np
import sympy as sy
sy.init_printing()
```

#  Similarity

If $A = PBP^{-1}$, we say $A$ is similar to $B$, decomposing $A$ into $PBP^{-1}$ is also called a similarity transformation.

If $n\times n$ matrices $A$ and $B$ are similar, they have the same eigenvalues.

The diagnoalization, which we will explain,  is a process of finding similar matrices.

#  Diagonalizable Matrix

Let $A$ be an $n\times n$ matrix. If there exists an $n\times n$ invertible matrix $P$ and a diagonal matrix $D$, such that

$$
A=PDP^{-1}
$$

then matrix $A$ is called a diagonalizable matrix.

And futher, the columns of $P$ are linearly independent eigenvectors of $A$, and its corresponding eigenvalues are on the diagonal of $D$. In other words, $A$ is diagonalizable if and only if the dimension of eigenspace basis is $n$.

It's easy to show why this equation holds.

Let

$$
P = \left[\begin{array}{llll}
{v}_{1} & {v}_{2} & \cdots & {v}_{n}
\end{array}\right]\\
$$

$$
D = \left[\begin{array}{cccc}
\lambda_{1} & 0 & \cdots & 0 \\
0 & \lambda_{2} & \cdots & 0 \\
\vdots & \vdots & & \vdots \\
0 & 0 & \cdots & \lambda_{n}
\end{array}\right]
$$

where $v_i, i \in (1, 2, ...n)$ is an eigenvector of $A$, $\lambda_i, i \in (1, 2, ...n)$ is an eigenvalue of $A$.

$$
AP = A\left[\begin{array}{llll}
{v}_{1} & {v}_{2} & \cdots & {v}_{n}
\end{array}\right]=\left[\begin{array}{llll}
A {v}_{1} & A {v}_{2} & \cdots & A {v}_{n}
\end{array}\right]
$$

$$P D=P\left[\begin{array}{cccc}
\lambda_{1} & 0 & \cdots & 0 \\
0 & \lambda_{2} & \cdots & 0 \\
\vdots & \vdots & & \vdots \\
0 & 0 & \cdots & \lambda_{n}
\end{array}\right]=\left[\begin{array}{lllll}
\lambda_{1} {v}_{1} & \lambda_{2} {v}_{2} & \cdots & \lambda_{n} {v}_{n}
\end{array}\right]$$

We know that $A{v}_i = \lambda_i{v}_i$, i.e.

$$
AP = PD
$$

Since $P$ has all independent eigenvectors, then

$$
A = PDP^{-1}
$$

#  Diagonalizing a Matrix

Consider a matrix

$$A=\left[\begin{array}{rrr}
1 & 3 & 3 \\
-3 & -5 & -3 \\
3 & 3 & 1
\end{array}\right]$$

diagonalize the matrix $A$.

Following these steps:

1. Find the eigenvalues of $A$
2. Find the eigenvectors of $A$
3. Construct $P$.
4. Construct $D$ from the corresponding columns of $P$.

```python
A = sy.Matrix([[1,3,3], [-3, -5, -3], [3,3,1]])
eig = sy.matrices.MatrixEigen.eigenvects(A)
eig
```

Construct $P$

```python
P = sy.zeros(3, 3)
P[:, 0] = eig[0][2][0]
P[:, 1] = eig[0][2][1]
P[:, 2] = eig[1][2][0]
P
```

Construct $D$

```python
D = sy.diag(eig[0][0], eig[0][0], eig[1][0])
D
```

We can verify if $PDP^{-1}=A$ holds:

```python
P * D * P.inv() == A
```

Of course we don't need to go through this process seperately. There is ```diagonalize``` method in SymPy.

```python
P, D = A.diagonalize()
```

```python
P
```

```python
D
```

Sometimes we just want to test if a matrix is diagonalizable, then use ```is_diagonalizable``` in SymPy.

```python
A.is_diagonalizable()
```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Strang-Book-Linear_Algebra_With_Python-Jupyter-Notebooks/Chapter 13 - Diagonalization.ipynb


#Introduction

Linear algebra is a field of mathematics that is widely used in various disciplines. Linear algebra plays an important role in data science and machine
learning. A solid understanding of linear algebra concepts can enhance the
understanding of many data science and machine learning algorithms. This
chapter introduces basic concepts for data science and includes vector spaces,
orthogonality, eigenvalues, matrix decomposition and further expanded to include linear regression and principal component analysis where linear algebra
plays a central role for solving data science problems. More advanced concepts
and applications of linear algebra can be found in many references $[1, 2, 3, 4]$.

#Elements of Linear Algebra

##Linear Spaces

###Linear Combinations

```python
import numpy as np
v = np.array([[3.3],[11]])
w = np.array([[-2],[-40]])
a = 1.5
b = 4.9
u = a*v + b*w
u
```

    array([[  -4.85],
           [-179.5 ]])

```python
import numpy as np
x = np.array([[4, 8, 1],
              [2, 11, 0],
              [1, 7.4, 0.2]])
y = ([3.65, 1.55, 3.42])
result = np.linalg.solve(x, y)
result
```

    array([-2.19148936,  0.5393617 ,  8.10106383])

###Linear Independence and Dimension

```python
import sympy
import numpy as np
matrix = np.array([[2,1,7],[2,9,11]])
_, inds = sympy.Matrix(matrix).T.rref()
print(inds)
```

    (0, 1)

This means vetor 0 and vector 1 are linearly independent.

```python
matrix = np.array([[0,1,0,0],[0,0,1,0],[0,1,1,0],[1,0,0,1]])
_, inds = sympy.Matrix(matrix).T.rref()
print(inds)
```

    (0, 1, 3)

This means vetor 0, vector 1, and vector 3 are lineatly independent while vector 2 is linearly dependent.

##Orthogonality

Inner product

```python
a = np.array([1.6,2.5,3.9])
b = np.array([4,1,11])
np.inner(a, b)
```

    51.8

Norm

```python
from numpy import linalg as LA
c = np.array([[ 1.3, -7.2, 12.1],
              [-1, 0, 4]])
LA.norm(c)
```

    14.728883189162714

Orthogonality

```python
v1 = np.array([1,-2, 4])
v2 = np.array([2, 5, 2])
dot_product = np.dot(v1,v2)
if dot_product == 0:
  print('v1 and v2 are orthorgonal')
else: print('v1 and v2 are not orthorgonal')
```

    v1 and v2 are orthorgonal

```python
n1 = LA.norm(v1)
n2 = LA.norm(v2)
if n1 == 1 and n2 == 1:
  print('v1 and v2 are orthornormal')
else: print('v1 and v2 are not orthornormal')

```

    v1 and v2 are not orthornormal

##Gram-Schmidt Process

```python
import numpy as np
def gs(X):
    Q, R = np.linalg.qr(X)
    return Q
```

```python
X = np.array([[3,-1,0],[1.8,11.3,-7.5], [4,13/4,-7/3]])
gs(X)
```

    array([[-0.56453245,  0.40891852, -0.71699983],
           [-0.33871947, -0.90691722, -0.25053997],
           [-0.75270993,  0.10142386,  0.65049286]])

##Eigenvalues and Eigenvectors

```python
import numpy as np
from numpy.linalg import eig
a = np.array([[2.1, -5/2, 11.4],
              [1, 3, 5],
              [2.4, 3.5, 7.4]])
u,v=eig(a)
print('E-value:', u)
print('E-vector', v)
```

    E-value: [12.05944603 -1.69840975  2.13896373]
    E-vector [[-0.63272391 -0.94998594  0.88313257]
     [-0.42654701 -0.10917553 -0.45885676]
     [-0.64631115  0.29258746 -0.09760805]]

#Linear Regression

##QR Decomposition

```python
import numpy as np
from numpy.linalg import qr
m = np.array([[1/2, -2.8, 5/3],
              [2.5, 3, 9],
              [8.3, 4, -5.2]])
q, r = qr(m)
print('Q:', q)
print('R:', r)

n = np.dot(q, r)
print('QR:', n)
```

    Q: [[-0.0575855   0.87080492 -0.4882445 ]
     [-0.28792749 -0.48276155 -0.82706653]
     [-0.95591928  0.09295198  0.27852874]]
    R: [[-8.6827415  -4.5262202   2.28345698]
     [ 0.         -3.51473053 -3.3768627 ]
     [ 0.          0.         -9.70568907]]
    QR: [[ 0.5        -2.8         1.66666667]
     [ 2.5         3.          9.        ]
     [ 8.3         4.         -5.2       ]]

##Least-squares Problems

Use direct inverse method

```python
import numpy as np
from scipy import optimize
import matplotlib.pyplot as plt
plt.style.use('seaborn-poster')
x = np.linspace(0, 10, 500)
y = 1/2 + x * np.random.random(len(x))
A = np.vstack([x, np.ones(len(x))]).T
y = y[:, np.newaxis]
lst_sqr = np.dot((np.dot(np.linalg.inv(np.dot(A.T,A)),A.T)),y)
print(lst_sqr)
```

    [[0.5248105 ]
     [0.48146998]]

Use the pseudoinverse

```python
pinv = np.linalg.pinv(A)
lst_sqr = pinv.dot(y)
print(lst_sqr)
```

    [[0.5248105 ]
     [0.48146998]]

Use numpy.linalg.lstsq

```python
lst_sqr = np.linalg.lstsq(A, y, rcond=None)[0]
print(lst_sqr)
```

    [[0.5248105 ]
     [0.48146998]]

Use optimize.curve_fit from scipy

```python
x = np.linspace(0, 10, 500)
y = 1/2 + x * np.random.random(len(x))
def func(x, a, b):
    y = a*x + b
    return y
lst_sqr = optimize.curve_fit(func, xdata = x, ydata = y)[0]
print(lst_sqr)
```

    [0.55132487 0.30482114]

## Linear Regression

```python
import numpy as np
from sklearn.linear_model import LinearRegression
x = np.array([5.3, 15.2, 25.8, 35.4, 45.5, 54.9]).reshape((-1, 1))
y = np.array([4.7, 20.4, 31/2, 33.2, 22, 38.6])
model = LinearRegression().fit(x, y)
r_sq = model.score(x, y)
print('coefficient of determination:', r_sq)
```

    coefficient of determination: 0.7007120636613271

```python
print('intercept:', model.intercept_)
```

    intercept: 5.763991287587402

```python
print('slope:', model.coef_)
```

    slope: [0.54813867]

```python
y_pred = model.predict(x)
print('predicted response:', y_pred, sep='\n')
```

    predicted response:
    [ 8.66912625 14.09569911 19.90596904 25.1681003  30.70430089 35.85680441]

#Principal Component Analysis

##Singular Value Decomposition

```python
from numpy import array
from scipy.linalg import svd

A = array([[3, -2, 5],
           [1, 0, -3],
           [4, 6, -1]])

print('Matrix A:')
print(A)
U, sigma, VT = svd(A)

print('The m × m orthogonal matrix:')
print(U)

print('The m × n diagonal matrix:')
print(sigma)

print('The n × n orthogonal matrix:')
print(VT)
```

    Matrix A:
    [[ 3 -2  5]
     [ 1  0 -3]
     [ 4  6 -1]]
    The m × m orthogonal matrix:
    [[-0.38248394  0.86430926  0.32661221]
     [ 0.23115377 -0.25273925  0.93951626]
     [ 0.89458033  0.43484753 -0.10311964]]
    The m × n diagonal matrix:
    [7.54629306 6.24447219 2.24945061]
    The n × n orthogonal matrix:
    [[ 0.35275906  0.81264401 -0.46386502]
     [ 0.6533104   0.14099937  0.74384454]
     [ 0.66988548 -0.56544574 -0.48116998]]

##Principal Component Analysis

Covariance Matrix

```python
A = array([[3, -2, 5],
           [1, 0, -3],
           [4, 6, -1]])
covMatrix = np.cov(A,bias=True)
print('Covariance matrix of A:')
print(covMatrix)
```

    Covariance matrix of A:
    [[ 8.66666667 -2.66666667 -7.66666667]
     [-2.66666667  2.88888889  4.33333333]
     [-7.66666667  4.33333333  8.66666667]]

Principal Component Analysis

```python
import numpy as np
from numpy.linalg import eig

X = np.random.randint(10,50,100).reshape(20,5)
X_meaned = X - np.mean(X , axis = 0)
covMatrix = np.cov(X_meaned, rowvar = False)
val,vec = eig(covMatrix)

s_index = np.argsort(val)[::-1]
s_val = val[s_index]
s_vec = vec[:,s_index]

n_components = 8
vec_sub = s_vec[:,0:n_components]

#Transform the data
X_reduced = np.dot(vec_sub.transpose(), X_meaned.transpose()).transpose()
X_reduced
```

    array([[-14.2097599 ,  11.1638481 ,   0.28399481,   6.72774374,
             -6.5223102 ],
           [ 25.06735619,   0.65710691,   6.85236627,  -0.17476694,
             -6.8806175 ],
           [-21.61411193, -13.0377964 , -12.16006542,   5.05948329,
              5.30216639],
           [-27.78196313,  -9.86254508,  -6.96332075,   2.28424895,
             -1.19146972],
           [ -0.22967199,  24.5661952 ,   2.66240043,   2.63708048,
              5.03380919],
           [ 11.15926987,  17.71804406,   2.51916061,   5.85025062,
             -5.6923201 ],
           [ 11.5482325 ,  -9.11311102,   9.98129819, -10.96725047,
             11.84124727],
           [  5.57296606,  -7.13463359, -25.32274739,   2.06609699,
             -0.24750938],
           [-17.22161772, -11.97678995,  13.20560437, -14.20500239,
            -11.55139844],
           [-13.62600723,  17.70093422,   8.90913338,   2.01548555,
              3.03422397],
           [-26.51289728,  -8.97206962,   4.14445894,   7.02884273,
             -2.02967354],
           [ -4.03180426,  25.35080447, -18.62772438,  -7.32506778,
              0.4062433 ],
           [ 14.2056229 ,  -6.05889182,  11.24736988,  -5.4258927 ,
             -6.71408721],
           [ 20.30238914, -14.09296297,  -7.57112907,   6.59989966,
             -5.98775578],
           [-11.4490343 , -18.98788569,  10.6311743 ,   1.03120515,
              6.07676389],
           [ 11.12612414,   3.55634731,  19.45836326,  11.10664498,
             -1.5839167 ],
           [ 27.74171335, -14.62724333, -18.83519665,   1.72806277,
              0.85015228],
           [ -3.39768961,   8.03689297,  -7.86477459,  -2.56432008,
              2.22848377],
           [  0.40503237,   6.32555287,  -4.50101459, -16.90395115,
              0.59424993],
           [ 12.94585082,  -1.21179662,  11.95064838,   3.43120659,
             13.03371858]])

Total Variance

```python
B = covMatrix
total_var_matr = B.trace()
print('Total variance of A:')
print(total_var_matr)
```

    Total variance of A:
    716.65


```python
import matplotlib.pyplot as plt
import numpy as np
# from scipy import stats
%matplotlib inline
```

## Heart curve
http://mathworld.wolfram.com/HeartCurve.html
$$\begin{align}
x&=16sin^3(t)\\
y&=13cos(t)-5cos(2t)-2cos(3t)-cos(4t)
\end{align}$$

```python
def heart():
    # http://mathworld.wolfram.com/HeartCurve.html
    t = np.linspace(0, 2 * np.pi, 360)
    # print(t)
    x = 16 * np.sin(t) ** 3
    y = 13 * np.cos(t) - 5 * np.cos(2 * t) - 2 * np.cos(3 * t) - np.cos(4 * t)
    plt.plot(x, y)
    plt.plot(x + 10, y)
    plt.plot(x + 20, y)
    plt.show()

def heart2():
    # http://mathworld.wolfram.com/HeartCurve.html
    t = np.linspace(0, 2 * np.pi, 360)
    # print(t)
    x = 16 * np.sin(t) ** 3
    y = 13 * np.cos(t) - 5 * np.cos(2 * t) - 2 * np.cos(3 * t) - np.cos(4 * t)
    r = np.linspace(.1, 1, 10)
    plt.text(0, 0, "I \u2764 Python")

    for n in r:
        # plt.plot(x + n, y)
        # plt.plot(x * n, y)
        # plt.plot(x * n / 5, y)
        plt.plot(x * n / 5, y * n / 5)
    plt.show()
```

```python
heart()
```

```python
heart2()
```

```python

```


```python
import numpy as np
import matplotlib.pyplot as plt
from sympy import Symbol, integrate
%matplotlib notebook
```

### Smooth local paths
We will use cubic spirals to generate smooth local paths. Without loss of generality, as $\theta$ smoothly changes from 0 to 1, we impose a condition on the curvature as follows

$\kappa = f'(x) = K(x(1-x))^n $

This ensures curvature vanishes at the beginning and end of the path. Integrating, the yaw changes as
$\theta = \int_0^x f'(x')dx'$

With $n = 1$ we get a cubic spiral, $n=2$ we get a quintic spiral and so on. Let us use the sympy package to find the family of spirals

1. Declare $x$ a Symbol

2. You want to find Integral of $f'(x)$

3. You can choose $K$ so that all coefficients are integers

Verify if $\theta(0) = 0$ and $\theta(1) = 1$

```python
K =  30 #choose for cubic/quintic
n =  2 #choose for cubic/ quintic
x =  Symbol('x') #declare as Symbol
print(integrate(K*(x*(1-x))**n, x)) # complete the expression
```

    6*x**5 - 15*x**4 + 10*x**3

```python
x = np.linspace(0, 1, num=100)
thetas = -2*x**3 + 3*x**2
plt.figure()
plt.plot(x, thetas,'.',label="$\\theta=-2x^3 + 3x^2$")
thetas = 6*x**5 - 15*x**4 + 10*x**3
plt.plot(x, thetas,'.',label="$\\theta=6x^5 - 15x^4 + 10x^3$")
plt.legend()
plt.show()
```

    <IPython.core.display.Javascript object>

```python
#write function to compute a cubic spiral
#input can be any theta_i and theta_f (not just 0 and 1)
def cubic_spiral(theta_i, theta_f, n=10):
    x = np.linspace(0, 1, num=n)
    #-2*x**3 + 3*x**2
    return (theta_f-theta_i)*(-2*x**3 + 3*x**2) + theta_i

def quintic_spiral(theta_i, theta_f, n=10):
    x = np.linspace(0, 1, num=n)
    #6*x**5 - 15*x**4 + 10*x**3
    return (theta_f-theta_i)*(6*x**5 - 15*x**4 + 10*x**3) + theta_i
```

### Plotting
Plot cubic, quintic spirals along with how $\theta$ will change from $\pi/2$ to $0$ when moving in a circular arc. Remember circular arc is when  $\omega $ is constant

```python
num_pts = 100
plt.figure()
plt.plot(np.pi/2*(1-np.linspace(0,1,num_pts)), label='Circular')
plt.plot(cubic_spiral(np.pi/2, 0, num_pts), label='Cubic')
plt.plot(quintic_spiral(np.pi/2, 0, num_pts),label='Quintic')
plt.grid()
plt.legend()
plt.show()
```

    <IPython.core.display.Javascript object>

## Trajectory

Using the spirals, convert them to trajectories $\{(x_i,y_i,\theta_i)\}$. Remember the unicycle model

$dx = v\cos \theta dt$

$dy = v\sin \theta dt$

$\theta$ is given by the spiral functions you just wrote. Use cumsum() in numpy to calculate {(x_i, y_i)}

What happens when you change $v$?

```python
num_pts = 50
v = 1
dt = 0.1

#cubic
theta = cubic_spiral(np.pi/2, np.pi, num_pts)
x = np.cumsum(v*np.cos(theta)*dt)
y = np.cumsum(v*np.sin(theta)*dt)

#Quintic
theta = quintic_spiral(np.pi/2, np.pi, num_pts+2)
xq = np.cumsum(v*np.cos(theta)*dt)
yq = np.cumsum(v*np.sin(theta)*dt)

#Circular
theta = np.pi/2*(1+np.linspace(0,1,num_pts-2))
xc = np.cumsum(v*np.cos(theta)*dt)
yc = np.cumsum(v*np.sin(theta)*dt)

# plot trajectories for circular/ cubic/ quintic
plt.figure()
plt.plot(xc, yc, label='Circular')
plt.plot(x, y, label='Cubic')
plt.plot(xq, yq, label='Quintic')
plt.legend()
plt.grid()
```

    <IPython.core.display.Javascript object>

## Symmetric poses

We have been doing only examples with $|\theta_i - \theta_f| = \pi/2$.

What about other orientation changes? Given below is an array of terminal angles (they are in degrees!). Start from 0 deg and plot the family of trajectories

```python
dt = 0.1
thetas = np.deg2rad([15, 30, 45, 60, 90, 120, 150, 180]) #convert to radians
plt.figure()
for tf in thetas:
    t = cubic_spiral(0, tf,50)
    x = np.cumsum(np.cos(t)*dt)
    y = np.cumsum(np.sin(t)*dt)
    plt.plot(x, y)

# On the same plot, move from 180 to 180 - theta
thetas = np.pi - np.deg2rad([15, 30, 45, 60, 90, 120, 150, 180])
for tf in thetas:
    t = cubic_spiral(np.pi, tf, 50)
    x = np.cumsum(np.cos(t)*dt)
    y = np.cumsum(np.sin(t)*dt)
    plt.plot(x, y)
plt.grid()
plt.show()
```

    <IPython.core.display.Javascript object>

Modify your code to print the following for the positive terminal angles $\{\theta_f\}$
1. Final x, y position in corresponding trajectory: $x_f, y_f$
2. $\frac{y_f}{x_f}$ and $\tan \frac{\theta_f}{2}$

What do you notice?
What happens when $v$ is doubled?

```python
dt = 0.05
v = 2.0
thetas = np.deg2rad([15, 30, 45, 60, 90, 120, 150, 180]) #convert to radians
plt.figure()
for tf in thetas:
    t = cubic_spiral(0, tf,100)
    x = np.cumsum(v*np.cos(t)*dt)
    y = np.cumsum(v*np.sin(t)*dt)
    plt.plot(x,y)
    print(f"tf:{np.rad2deg(tf):0.1f} xf:{x[-1]:0.3f} yf:{y[-1]:0.3f} yf/xf:{y[-1]/x[-1]:0.3f} tan(theta/2):{np.tan(tf/2):0.3f}")
plt.show()

```

    <IPython.core.display.Javascript object>

    tf:15.0 xf:9.873 yf:1.300 yf/xf:0.132 tan(theta/2):0.132
    tf:30.0 xf:9.497 yf:2.545 yf/xf:0.268 tan(theta/2):0.268
    tf:45.0 xf:8.892 yf:3.683 yf/xf:0.414 tan(theta/2):0.414
    tf:60.0 xf:8.087 yf:4.669 yf/xf:0.577 tan(theta/2):0.577
    tf:90.0 xf:6.041 yf:6.041 yf/xf:1.000 tan(theta/2):1.000
    tf:120.0 xf:3.743 yf:6.484 yf/xf:1.732 tan(theta/2):1.732
    tf:150.0 xf:1.610 yf:6.010 yf/xf:3.732 tan(theta/2):3.732
    tf:180.0 xf:-0.000 yf:4.812 yf/xf:-7880729543884461.000 tan(theta/2):16331239353195370.000

These are called *symmetric poses*. With this spiral-fitting approach, only symmetric poses can be reached.

In order to move between any 2 arbitrary poses, you will have to find an intermediate pose that is pair-wise symmetric to the start and the end pose.

What should be the intermediate pose? There are infinite possibilities. We would have to formulate it as an optimization problem. As they say, that has to be left for another time!

```python

```


```python
#  logistic regression, a type of regression that predicts a probability
# of an outcome given one or more independent variables.
# This in turn can be used for classification, which is predicting categories
# rather than real numbers as we did with linear regression.
```

```python
# The logistic function is an S-shaped curve (also known as a sigmoid curve) that,
# for a given set of input variables, produces an output variable between 0 and 1.
# Because the output variable is between 0 and 1 it can be used to represent a probability.

```

```python
# The logistic function in Python for one independent variable

import math

def predict_probability(x, b0, b1):
    p = 1.0 / (1.0 + math.exp(-(b0 + b1 * x)))
    return p

```

```python
from sympy import *
from matplotlib import *

b0, b1, x = symbols('b0 b1 x')

p = 1.0 / (1.0 + exp(-(b0 + b1 * x)))

p = p.subs(b0,-2.823)
p = p.subs(b1, 0.620)
print(p)

plot(p)
```

```python
# Using a plain logistic regression SciPy

import pandas as pd
from sklearn.linear_model import LogisticRegression

# Load the data
df = pd.read_csv('https://bit.ly/33ebs2R', delimiter=",")

# Extract input variables (all rows, all columns but last column)
X = df.values[:, :-1]

# Extract output column (all rows, last column)
Y = df.values[:, -1]

# Perform logistic regression
# Turn off penalty
model = LogisticRegression()
model.fit(X, Y)

# print beta1
print(model.coef_.flatten())

# print beta0
print(model.intercept_.flatten())

```

    [0.33467796]
    [-1.15144356]

```python
# Calculating the joint likelihood of observing all the points for a given logistic regression

import math
import pandas as pd

patient_data = pd.read_csv('https://bit.ly/33ebs2R', delimiter=",").itertuples()

b0 = -3.17576395
b1 = 0.69267212

def logistic_function(x):
    p = 1.0 / (1.0 + math.exp(-(b0 + b1 * x)))
    return p

# Calculate the joint likelihood
joint_likelihood = 1.0

for p in patient_data:
    if p.y == 1.0:
        joint_likelihood *= logistic_function(p.x)
    elif p.y == 0.0:
        joint_likelihood *= (1.0 - logistic_function(p.x))

print(joint_likelihood)
```

    4.7911180221699105e-05

```python
# Using Gradient Descent on Logistic Regression

points = list(pd.read_csv("https://tinyurl.com/y2cocoo7").itertuples())

b1, b0, i, n = symbols('b1 b0 i n')
x, y = symbols('x y', cls=Function)
joint_likelihood = Sum(log((1.0 / (1.0 + exp(-(b0 + b1 * x(i))))) ** y(i) \
	* (1.0 - (1.0 / (1.0 + exp(-(b0 + b1 * x(i)))))) ** (1 - y(i))), (i, 0, n))

# Partial derivative for m, with points substituted
d_b1 = diff(joint_likelihood, b1) \
		   .subs(n, len(points) - 1).doit() \
		   .replace(x, lambda i: points[i].x) \
		   .replace(y, lambda i: points[i].y)

# Partial derivative for m, with points substituted
d_b0 = diff(joint_likelihood, b0) \
		   .subs(n, len(points) - 1).doit() \
		   .replace(x, lambda i: points[i].x) \
		   .replace(y, lambda i: points[i].y)

# compile using lambdify for faster computation
d_b1 = lambdify([b1, b0], d_b1)
d_b0 = lambdify([b1, b0], d_b0)

# Perform Gradient Descent
b1 = 0.01
b0 = 0.01
L = .01

for j in range(10_000):
    b1 += d_b1(b1, b0) * L
    b0 += d_b0(b1, b0) * L

print(b1, b0)
```

    0.692669307537082 -3.175751550409825

```python
import pandas as pd
from sklearn.linear_model import LogisticRegression

employee_data = pd.read_csv("https://tinyurl.com/y6r7qjrp")

# grab independent variable columns
inputs = employee_data.iloc[:, :-1]

# grab dependent "did_quit" variable column
output = employee_data.iloc[:, -1]

# build logistic regression
fit = LogisticRegression().fit(inputs, output)

# Print coefficients:
print("COEFFICIENTS: {0}".format(fit.coef_.flatten()))
print("INTERCEPT: {0}".format(fit.intercept_.flatten()))

# Interact and test with new employee data
def predict_employee_will_stay(sex, age, promotions, years_employed):
    prediction = fit.predict([[sex, age, promotions, years_employed]])
    probabilities = fit.predict_proba([[sex, age, promotions, years_employed]])
    if prediction == [[1]]:
        return "WILL LEAVE: {0}".format(probabilities)
    else:
        return "WILL STAY: {0}".format(probabilities)

# Test a prediction
while True:
    n = input("Predict employee will stay or leave {sex},{age},{promotions},{years employed}: ")
    (sex, age, promotions, years_employed) = n.split(",")
    print(predict_employee_will_stay(int(sex), int(age), int(promotions), int(years_employed)))
```

```python
# You will notice that in the coefficients for sex, age, promotions,
# and years employed are displayed in that order. By the weight of the coefficients,
# you can see that sex and age play very little role in the prediction (they both have a weight near 0).
# However promotions and years_employed have significant weights of -2.504 and .97.
# Here’s a secret with this toy dataset:
#     I fabricated it so that an employee quits if they do not get a promotion roughly every 2 years.
#     Sure enough my logistic regression picked up this pattern and you can try it out with other employees as well.
```

```python

```


# Changing Fourier Conventions

The ``powerbox`` package allows for arbitrary Fourier conventions. Since (continuous) Fourier Transforms can be defined using different definitions of the frequency term, and varying normalisations, we allow any consistent combination of these, using the same parameterisation that Mathematica uses, i.e.:

$$ F(k) = \left(\frac{|b|}{(2\pi)^{1-a}}\right)^{n/2} \int f(r) e^{-i b\mathbf{k}\cdot\mathbf{r}} d^n\mathbf{r} $$

for the forward-transform and

$$ f(r) = \left(\frac{|b|}{(2\pi)^{1+a}}\right)^{n/2} \int F(k) e^{+i b\mathbf{k}\cdot\mathbf{r}} d^n \mathbf{k} $$

for its inverse. Here $n$ is the number of dimensions in the transform, and $a$ and $b$ are free to be any real number. Within ``powerbox``, $b$ is taken to be positive.

The most common choice of parameters is $(a,b) = (0,2\pi)$, which are the parameters that for example ``numpy`` uses. These are the defaults in the ``fft`` and ``ifft`` implemented in ``powerbox.dft``.

In cosmology (a field which ``powerbox`` was written in the context of), a more usual choice is $(a,b)=(1,1)$, and so these are the defaults within the ``PowerBox`` classes.

In this notebook we provide some examples on how to deal with these conventions.

Note that in the DFT/FFT, the $x_i$ and $k_i$ are related. Given a set of $N$ values of $x_i$, for which $x_N = (N-1)L/N x_0$ (i.e. each consecutive value is separated by $\Delta x = L/N$), and whose starting point is irrelevant (but see the following section on phases), the FFT returns values of $k_i$:

\begin{equation}
    k_i = \frac{2\pi}{b} \frac{i}{L}, \ \ i \in (-N/2, N/2).
\end{equation}

Similarly, performing an inverse FFT on a function over a set of $N$ $k$-values, whose separation is $\Delta k = L_k/N$, yields the following values of $x'_i$:

\begin{equation}
    x'_i = \frac{2\pi}{b} \frac{i}{L_k} \equiv i\Delta x, \ \ i \in (-N/2, N/2).
\end{equation}

If one performs a round-trip transform (forwards then inverse), the resulting $x'_i$ are equivalent to the input $x_i$, modulo a potential shift. The FFT algorithm itself does not depend on the actual values of the original $x_i$, only their spacing and number, and so without applying any phasing, the $x'_i$ can be assumed to be equivalent to $x_i$.

## Doing the DFT right.

In many fields, we are concerned primarily with the *continuous* FT, as defined above. However, typically we must evaluate this numerically, and therefore use a DFT or FFT. While the conversion between the two is simple, it is all too easy to forget which factors to normalise by to get back the analogue of the continuous transform.

That's why in ``powerbox`` we provide some fast fourier transform functions that do all the hard work for you. They not only normalise everything correctly to produce the continuous transform, they also return the associated fourier-dual co-ordinates. And they do all this for arbitrary conventions, as defined above. And they work for any number of dimensions.

Let's take a look at an example, using a simple Gaussian field in 2D:

$$ f(x) = e^{-\pi r^2}, $$

where $r^2 = x^2 + y^2.$

The Fourier transform of this field, using the standard mathematical convention is:

$$ \int e^{-\pi r^2} e^{-2\pi i k\cdot x} d^2x = e^{-\pi k^2}, $$

where $k^2 = k_x^2 + k_y^2$.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

from powerbox.dft import fft,ifft, fftshift
from powerbox.powerbox import _magnitude_grid
import powerbox as pbox
```

```python
pbox.__version__
```

    '0.6.0'

```python
# Parameters of the field
L = 10.
N = 512
dx = L/N

x = np.arange(-L/2,L/2,dx)[:N] # The 1D field grid
r = _magnitude_grid(x,dim=2)   # The magnitude of the co-ordinates on a 2D grid
field = np.exp(-np.pi*r**2)    # Create the field

# Generate the k-space field, the 1D k-space grid, and the 2D magnitude grid.
k_field, k, rk = fft(field,L=L,          # Pass the field to transform, and its size
                     ret_cubegrid=True   # Tell it to return the grid of magnitudes.
                    )

# Plot the field minus the analytic result
plt.imshow(k_field.real-np.exp(-np.pi*rk**2),extent=(k.min(),k.max(),k.min(),k.max()))
plt.colorbar();
```

We can now of course do the inverse transform, to ensure that we return the original:

```python
x_field, x_, rx = ifft(k_field, L = L,   # Note we can pass L=L, or Lk as the extent of the k-space grid.
                       ret_cubegrid=True)

plt.imshow(np.abs(x_field)-field,extent=(x.min(),x.max(),x.min(),x.max()))
plt.title("Residual Between Input and\n Forward+Inverse Transform", fontsize=14)
plt.colorbar()
plt.show();
```

We can also check that the xgrid returned is the same as the input xgrid:

```python
x_ -x
```

    array([[0., 0., 0., ..., 0., 0., 0.],
           [0., 0., 0., ..., 0., 0., 0.]])

### Changing the convention

Suppose we instead required the transform

$$ \int e^{-\pi r^2} e^{-i \nu \cdot x} d^2x = e^{-\nu^2/4\pi}. $$

This is the same transform but with the Fourier-convention $(a,b) = (1,1)$. We would do this like:

```python
# Generate the k-space field, the 1D k-space grid, and the 2D magnitude grid.
k_field, k, rk = fft(field,L=L,          # Pass the field to transform, and its size
                     ret_cubegrid=True,   # Tell it to return the grid of magnitudes.
                     a=1,b=1             # SET THE FOURIER CONVENTION
                    )

# Plot the field minus the analytic result
plt.imshow(np.abs(k_field)-np.exp(-1./(4*np.pi)*rk**2),extent=(k.min(),k.max(),k.min(),k.max()))
plt.colorbar();
```

Again, specifying the inverse transform with these conventions gives back the original:

```python
x_field, x_, rx = ifft(k_field, L = L,   # Note we can pass L=L, or Lk as the extent of the k-space grid.
                       ret_cubegrid=True,
                       a=1,b=1
                      )

plt.imshow(np.abs(x_field)-field,extent=(x.min(),x.max(),x.min(),x.max()))
plt.colorbar()
plt.show();
```

### Mixing up conventions

It may be that sometimes the forward and inverse transforms in a certain problem will have different conventions. Say the forward transform has parameters $(a,b)$, and the inverse has parameters $(a',b')$. Then first taking the forward transform, and *then* inverting it (in $n$-dimensions) would yield:

$$ \left( \frac{b'}{b(2\pi)^{a'-a}}\right)^{n/2} f\left(\frac{b'r}{b}\right), $$

and doing it the other way would yield:

$$ \left( \frac{b}{b'(2\pi)^{a'-a}}\right)^{n/2} F\left(\frac{bk}{b'}\right). $$

The ``fft`` and ``ifft`` functions handle these easily. For example, if $(a,b) = (0,2\pi)$ and $(a',b') = (0,1)$, then the 2D forward-then-inverse transform should be

$$ f(r/(2\pi))/ 2\pi, $$

and the inverse-then-forward should be

$$ 2\pi F(2\pi k). $$

```python
# Do the forward transform
k_field,k,rk = fft(field,L=L,a=0,b=2*np.pi, ret_cubegrid=True)

# Do the inverse transform, ensuring the boxsize is correct
mod_field,modx,modr = ifft(k_field,Lk=-2*k.min(),a=0,b=1, ret_cubegrid=True)

mod_field, bins = pbox.angular_average(mod_field.real, modr, 300)

plt.plot(bins,mod_field, label="Numerical",lw=3,ls='--')
plt.plot(bins,np.exp(-np.pi*(bins/(2*np.pi))**2)/(2*np.pi),label="Analytic")
plt.legend()
plt.yscale('log')
plt.xscale('log')
plt.ylim(1e-7,3)
plt.show()
```

### FFT Phases

One thing that can sometimes be frustrating about the FFT/IFFT is that it assumes that the input function begins at $x=0$ (or $k=0$). While the magnitude of the output is insensitive to this choice, the phase of the output is. This can result in the FFT of a Gaussian, for example, exhibiting highly oscillatory behaviour in its real and imaginary components.

Since ``powerbox.dft`` is all about being able to simply compute what you would "expect" for a continuous function, we give a simple option for specifying what your actual $x$-co-ordinates are. In fact, by default, ``powerbox`` assumes that the _centre_ of the array is the zero co-ordinate, which is more likely in usual analytic setups.

To specify a different co-ordinate system, just specify ``left_edge`` in ``fft`` or ``ifft``. For instance, ``left_edge=0`` will perform the usual ``fft``. The default is ``left_edge = -L/2``. You can specify a list of values, one for each transformed axis. The value can be any real value -- we don't merely cycle the input before transforming, we rather apply the Fourier Shift theorem. However, do note that applying a value other than $-L/2$ will mean that a round-trip transform cannot be an identity operation.

Remember that both ``fft`` and ``ifft`` assume that the central value is zero by default, and they both have outputs which conform to this standard, so that ``x == ifft(fft(x))``. The standard (``numpy``) behaviour can be obtained by using ``x == fftshift(ifft(fft(x, left_edge=0)))``.

```python
print(np.allclose(field, ifft(fft(field, L=L)[0], L=L)[0]))
print(np.allclose(field, fftshift(ifft(fft(field, L=L, left_edge=0)[0], L=L)[0])))

```

    True
    True

```python
print(np.allclose(field, fft(ifft(field, L=L)[0], L=L)[0]))
print(np.allclose(field, fftshift(fft(ifft(field, L=L, left_edge=0)[0], L=L)[0])))

```

    True
    True

## Using Different Conventions in Powerbox

These fourier-transform wrappers are used inside powerbox to do the heavy lifting. That means that one can pass a power spectrum which has been defined with arbitrary conventions, and receive a fully consistent box back.

Let's say, for example, that the fourier convention in your field was to use $(a,b)=(0,1)$, so that the power spectrum of a 2D field, $\delta_x$ was given by

$$P(k) =  \frac{1}{2\pi} \int \delta_x e^{-ikx} d^2x. $$

We now wish to create a realisation with a power spectrum following these conventions. Let's say the power spectrum is $P(k) = 0.1k^{-3}$.

```python
pb = pbox.PowerBox(
    N=512,dim=2,pk = lambda k : 0.1*k**-3.,
    a=0, b=1,        # Set the Fourier convention
    boxlength=50.0   # Has units of inverse k
)

plt.imshow(pb.delta_x(),extent=(0,50,0,50))
plt.colorbar()
plt.show()
```

When we check the power spectrum, we also have to remember to set the Fourier convention accordingly:

```python
power, kbins = pbox.get_power(pb.delta_x(), pb.boxlength, a= 0,b =1)

plt.plot(kbins,power,label="Numerical")
plt.plot(kbins,0.1*kbins**-3.,label="Analytic")
plt.legend()
plt.xscale('log')
plt.yscale('log')
plt.show()
```

```python

```


# Multi-component phase equilibrium

We can use Raolt's Law to capture the behavior of a liquid-vapor mixture of propane and n-butane, at a mixture pressure of 2 atm.

For each component $i$,
\begin{equation}
x_{f,i} P_{\text{sat}, i} = x_i P \;,
\end{equation}
where $x_i$ is the mole fraction of component $i$ in the gas phase, $x_{f,i}$ is the mole fraction of component $i$ in the liquid phase, $P$ is the mixture pressure, and $P_{\text{sat}, i}$ is the saturation pressure of component $i$ at the mixture temperature.

First, given a mixture with equal mole fractions of propane and n-butane in the liquid phase (0.5 and 0.5), find the equilibrium temperature and the gas-phase mole fractions:

```python
import matplotlib.pyplot as plt
%matplotlib inline

# these are mostly for making the saved figures nicer
import matplotlib_inline.backend_inline
matplotlib_inline.backend_inline.set_matplotlib_formats('pdf', 'png')
plt.rcParams['figure.dpi']= 150
plt.rcParams['savefig.dpi'] = 150

import numpy as np
from scipy.optimize import root
from CoolProp.CoolProp import PropsSI

from pint import UnitRegistry
ureg = UnitRegistry()
Q_ = ureg.Quantity
```

Specify the known data:

```python
pressure = Q_(2, 'atm')
components = ['propane', 'butane']

mole_fraction_liquid = np.zeros(2)
mole_fraction_liquid[0] = 0.5
mole_fraction_liquid[1] = 1.0 - mole_fraction_liquid[0]
```

We then write a function for the system of equations we need to solve to find our unknowns: the equilibrium temperature, and the two mole fractions of the components in the vapor phase. So, we need three equations:

$$
x_1 + x_2 = 1 \\
x_{f,1} P_{\text{sat}, 1} = x_1 P \\
x_{f,2} P_{\text{sat}, 2} = x_2 P \;,
$$

which come from conservation of mass and Raoult's Law.

We'll use the `root()` function, which finds the values of our unknowns that make our equations equal to zero (i.e., the roots). To use this, we need to make our equations all equal zero, such that the correct values of the unknowns satisfy them:

$$
x_1 + x_2 - 1 = 0 \\
x_{f,1} P_{\text{sat}, 1} - x_1 P = 0 \\
x_{f,2} P_{\text{sat}, 2} - x_2 P = 0
$$

Now, we can define the function, then solve for the roots:

```python
def equilibrium(xvals, pressure, components, mole_fraction_liquid):
    '''Function for finding equilibrium temperature and vapor mole fractions.

    xvals[0]: temperature (K)
    xvals[1:]: vapor mole fractions
    '''
    temp = xvals[0]
    mole_fraction_gas = [xvals[1], xvals[2]]

    pressure_sat = np.zeros(2)
    pressure_sat[0] = PropsSI('P', 'T', temp, 'Q', 1.0, components[0])
    pressure_sat[1] = PropsSI('P', 'T', temp, 'Q', 1.0, components[1])

    return [
        (mole_fraction_liquid[0] * pressure_sat[0] -
         mole_fraction_gas[0] * pressure
         ),
        (mole_fraction_liquid[1] * pressure_sat[1] -
         mole_fraction_gas[1] * pressure
         ),
        mole_fraction_gas[0] + mole_fraction_gas[1] - 1.0
    ]
```

```python
sol = root(
    equilibrium, [250, 0.5, 0.5],
    args=(pressure.to('Pa').magnitude, components, mole_fraction_liquid,)
    )

print(f'Equilibrium temperature: {sol.x[0]: .2f} K')
print(f'Gas mole fraction of {components[0]}: {sol.x[1]: .3f}')
print(f'Gas mole fraction of {components[1]}: {sol.x[2]: .3f}')
```

    Equilibrium temperature:  262.48 K
    Gas mole fraction of propane:  0.833
    Gas mole fraction of butane:  0.167

We see that though the we have equal moles of the components in the liquid phase, the mole fraction of propane is over 80% in the vapor phase.

We can use the same approach to show the relationship between liquid and gas mole fraction for the two components:

```python
# since we just have two components and the mole
# fractions must sum to 1.0, we can just create
# a single array for the first component
mole_fractions_liquid = np.linspace(0, 1, 51)

temps = np.zeros(len(mole_fractions_liquid))
mole_fractions_gas = np.zeros((len(mole_fractions_liquid), 2))

for idx, x in enumerate(mole_fractions_liquid):
    sol = root(
        equilibrium, [250, 0.5, 0.5],
        args=(pressure.to('Pa').magnitude, components, [x, 1.0 - x],)
        )
    temps[idx] = sol.x[0]
    mole_fractions_gas[idx] = [sol.x[1], sol.x[2]]

fig, ax = plt.subplots(1, 2)

ax[0].plot(mole_fractions_liquid, mole_fractions_gas[:,0])
ax[0].set_xlabel(f'Mole fraction of {components[0]} in liquid')
ax[0].set_ylabel(f'Mole fraction of {components[0]} in gas')
ax[0].grid(True)

ax[1].plot(1.0 - mole_fractions_liquid, mole_fractions_gas[:,1])
ax[1].set_xlabel(f'Mole fraction of {components[1]} in liquid')
ax[1].set_ylabel(f'Mole fraction of {components[1]} in gas')
ax[1].grid(True)

plt.tight_layout()
plt.show()
```

The line between the liquid and gas mole fractions is the *equilibrium line*.

Interestingly, we see that the gas phase is richer in propane than the liquid phase, and the leaner in butane.

We can also plot the temperature against the propane mole fractions in the liquid and gas phases:

```python
plt.plot(mole_fractions_liquid, temps, '--', label='bubble point line')
plt.plot(mole_fractions_gas[:,0], temps, label='dew point line')

# fill space between the lines
plt.fill_betweenx(
    temps, mole_fractions_liquid, mole_fractions_gas[:,0],
    color='gray', alpha=0.2, label='liquid & vapor'
    )

plt.xlabel('Mole fraction of propane in liquid and gas')
plt.ylabel('Temperature (K)')

# text box properties
props = dict(boxstyle='round', facecolor='white', alpha=0.6)

plt.text(0.1, 255, 'single-phase liquid', bbox=props)
plt.text(0.7, 275, 'single-phase vapor', bbox=props)

plt.grid(True)
plt.legend()
plt.show()
```

The line formed by the temperature and liquid mole fraction is the *bubble point line* and the line of temperature and vapor mole fraction is the *dew point line*.

The region between the lines shows the temperatures where the liquid and vapor phases can coexist; below the bubble point line, the mixture will always be in a liquid phase, and above the dew point line, the mixture will always in a vapor phase.

This plot can be used to examine the behavior of a two-phase mixture undergoing a heating process. For example, consider a mixture that starts at 250 K and a liquid mole fraction of propane of 0.6, as it is heated. Note that the total mole fraction of propane, 0.6, should remain constant through this process.

One useful quantity to calculate is the quality of mixture, the ratio of the moles of vapor to the total moles, can be found here using the information about propane:

$$
Q = \frac{n_g}{n} = \frac{n_g}{n_g + n_f} \\
Q = \frac{z_1 - x_1}{x_{f,1} - x_1} \;,
$$

where $z_1$ is the total mole fraction of propane ($n_1 / n$), which remains constant through this process.

1. Initially the mixture is entirely liquid, and the liquid mole fractions remain constant as the mixture is heated.
2. With increasing temperature, eventually we reach the bubble point line, at a temperature of about 259 K. The first bubbles appear, with a propane mole fraction of 0.89 in the vapor phase found via the *tie line* (the horizontal line connecting to the dew point line at this temperature).
3. As the mixture is heated more and the temperature increases, it forms a two-phase mixture. The liquid and vapor mole fractions of propane are found via the horizontal tie lines connecting to the bubble and dew point lines. For example, at about 265 K, the propane mole fractions are about 0.44 and 0.79 in the liquid and vapor phases, respectively. At this state, the quality is about 0.45.
4. Finally, with more heating, the temperature continues to rise until the mixture reaches the dew point line, where the quality is 1 and the mole fraction of propane in the vapor phase is 0.6. This occurs at about 274 K, and the last liquid droplet will have a propane mole fraction of about 0.24.
5. Further heating will bring the mixture fully into the vapor phase, with a propane mole fraction of 0.6.

It also shows the *temperature glide*, which is the difference between the temperatures at the dew line and bubble line for a given composition.

The calculations for these values follow:

```python
print('Point 2:')
print(f'Temperature: {temps[30]: .1f} K')
total_mole_fraction_propane = mole_fractions_liquid[30]
print(f'Mole fraction of propane in liquid phase: {mole_fractions_liquid[30]: .3f}')
print(f'Mole fraction of propane in the vapor phase: {mole_fractions_gas[30,0]: .3f}')
```

    Point 2:
    Temperature:  258.9 K
    Mole fraction of propane in liquid phase:  0.600
    Mole fraction of propane in the vapor phase:  0.885

```python
print('Point 3:')
print(f'Temperature: {temps[22]: .1f} K')
print(f'Mole fraction of propane in liquid phase: {mole_fractions_liquid[22]: .3f}')
print(f'Mole fraction of propane in the vapor phase: {mole_fractions_gas[22,0]: .3f}')

quality_propane = (
    (total_mole_fraction_propane - mole_fractions_liquid[22]) /
    (mole_fractions_gas[22,0] - mole_fractions_liquid[22])
    )
print(f'Quality of propane: {quality_propane: .3f}')
```

    Point 3:
    Temperature:  264.9 K
    Mole fraction of propane in liquid phase:  0.440
    Mole fraction of propane in the vapor phase:  0.794
    Quality of propane:  0.452

```python
print('Point 4 (approximate):')
print(f'Temperature: {temps[12]: .1f} K')
print(f'Mole fraction of propane in liquid phase: {mole_fractions_liquid[12]: .3f}')
print(f'Mole fraction of propane in the vapor phase: {mole_fractions_gas[12,0]: .3f}')
```

    Point 4 (approximate):
    Temperature:  274.7 K
    Mole fraction of propane in liquid phase:  0.240
    Mole fraction of propane in the vapor phase:  0.589


```python
%pylab inline
```

    Populating the interactive namespace from numpy and matplotlib

```python
from sympy.parsing.sympy_parser import parse_expr
from sympy import Matrix, symbols, expand
```

```python
xs = np.linspace(-1,1,32)
ys = np.linspace(-1,1,32)
X, Y = np.meshgrid(xs, ys)
fig = plt.figure(figsize=(6,6))
ax = fig.add_subplot(111)
ax.set_aspect(1)
mat = np.array([[.87,-.2],[.20,2]])
X, Y = zip(*[np.matmul(mat, np.array([x, y])) for x, y in zip(X.ravel(), Y.ravel())])
plot(X,Y,'o',c='purple',markersize=4);
axhline(0, lw=1, c='black')
axvline(0, lw=1, c='black')
xlim(-2,2)
ylim(-2,2)
plt.axis("on")
#xlim(-2,2)
#ylim(-2,2)
```

```python
mat = np.array([[.5,0],[0,2]])
```

```python
xs, ys = zip(*[np.matmul(mat, np.array([x, y])) for x, y in zip(X.ravel(), Y.ravel())])
plot(xs,ys,'o',c='purple',markersize=4);
```

```python
np.matmul(mat, np.array([1,0]))
```

    array([2, 2])

```python
np.dot(mat, np.array([1,1]))
```

    array([4, 4])

```python
def interpolate(M,N,alpha): return (1-alpha) * M + alpha * N
```

```python
interpolate(mat, np.eye(2), 0.5)
```

    array([[0.75, 0.  ],
           [0.  , 1.5 ]])

```python
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(16,10))

angle = 1/2*np.pi
mat = np.array([[np.cos(angle), np.sin(angle)],[ -np.sin(angle), np.cos(angle)]])
xs = np.linspace(-1,1,32) #np.array([1,-1])
ys = np.linspace(-1,1,32) #np.array([1,-1])
X, Y = np.meshgrid(xs, ys)
N = 6
for n in range(0, N+1):
    ax = fig.add_subplot(N+1, 1, n+1)
    ax.set_aspect(1)
    #plt.axis([0, 4, 0, 4])

    xs, ys = zip(*[np.matmul( interpolate(np.eye(2), mat, n/N), np.array([x, y])) for x, y in zip(X.ravel(), Y.ravel())])

    plot(xs,ys,'o',c='purple',markersize=4);
    axhline(0, lw=1, c='black')
    axvline(0, lw=1, c='black')
    plt.axis("off")

plt.tight_layout(pad=0)

plt.show()
```

```python

```


# ルンゲ・クッタ法

```python
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
sns.set('notebook', 'whitegrid', 'dark', font_scale=2, rc={"lines.linewidth": 2, 'grid.linestyle': '-'})
```

## オイラー法

$x'=f(t,x)$の初期値問題に対して漸化式
$$
    x_{n+1} = x_n + h f(t_n, x_n)
$$
によって近似解を計算する

```python
def Euler(t, x, f, h):
    return x + h * f(t, x)
```

$x'=f(t,x)$の右辺の定義

```python
def func(t,x):
    return x
```

```python
a, b = 0.0, 1.0
N = 10
h = (b-a)/N
t = a
x = 1.0
X = np.zeros(N+1)
X[0] = x
for n in range(N):
    x = Euler(t, x, func, h)
    X[n+1] = x
    t = a + (n + 1) * h
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,256), np.exp(np.linspace(a,b,256)), '--k')
ax.plot(np.linspace(a,b,N+1), X, 'o-k')
ax.set_xlabel("$t$")
ax.set_ylabel("$x$")
# plt.savefig("euler1.pdf", bbox_inches="tight")
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,N+1), np.abs(X-np.exp(np.linspace(a,b,N+1))), 'ok')
ax.set_xlabel("$t$")
ax.set_ylabel("error")
# plt.savefig("euler1_error.pdf", bbox_inches="tight")
```

```python
a, b = 0.0, 1.0
N = 100
h = (b-a)/N
t = a
x = 1.0
X = np.zeros(N+1)
X[0] = x
for n in range(N):
    x = Euler(t, x, func, h)
    X[n+1] = x
    t = a + (n + 1) * h
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,256), np.exp(np.linspace(a,b,256)), '--k')
ax.plot(np.linspace(a,b,N+1), X, '-k')
ax.set_xlabel("$t$")
ax.set_ylabel("$x$")
# plt.savefig("euler2.pdf", bbox_inches="tight")
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,N+1), np.abs(X-np.exp(np.linspace(a,b,N+1))), '.-k')
ax.set_xlabel("$t$")
ax.set_ylabel("error")
# plt.savefig("euler2_error.pdf", bbox_inches="tight")
```

## ホイン法
\begin{align}
    k_1 &= f(t_n, x_n),\\
    k_2 &= f(t_n + h, x_n + h k_1),\\
    x_{n+1} &= x_n + \frac{h}{2} (k_1 + k_2)
\end{align}

```python
def Heun(t, x, f, h):
    k1 = f(t, x)
    k2 = f(t + h, x + h * k1)
    return x + 0.5 * h * (k1 + k2)
```

```python
a, b = 0.0, 1.0
N = 10
h = (b-a)/N
t = a
x = 1.0
X2 = np.zeros(N+1)
X2[0] = x
for n in range(N):
    x = Heun(t, x, func, h)
    X2[n+1] = x
    t = a + (n + 1) * h
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,256), np.exp(np.linspace(a,b,256)), '--k')
ax.plot(np.linspace(a,b,N+1), X2, 'o-k')
ax.set_xlabel("$t$")
ax.set_ylabel("$x$")
# plt.savefig("heun1.pdf", bbox_inches="tight")
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,N+1), np.abs(X2-np.exp(np.linspace(a,b,N+1))), 'ok')
ax.set_xlabel("$t$")
ax.set_ylabel("error")
# plt.savefig("heun1_error.pdf", bbox_inches="tight")
```

## 古典的ルンゲ・クッタ法
$$
\begin{aligned}
k_1 &= f(t_n, x_n),\\
k_2 &= f(t_n + \tfrac{h}{2}, x_n + \tfrac{h}{2}k_1),\\
k_3 &= f(t_n + \tfrac{h}{2}, x_n + \tfrac{h}{2}k_2),\\
k_4 &= f(t_n + h, x_n + h k_3),\\
x_{n+1} &= x_n + \frac{h}{6}(k_1 + 2 k_2 + 2 k_3 + k_4)
\end{aligned}
$$

```python
def RK4(t, x, f, h):
    k1 = f(t, x)
    k2 = f(t+0.5*h, x+0.5*h*k1)
    k3 = f(t+0.5*h, x+0.5*h*k2)
    k4 = f(t+h, x+h*k3)
    return x + h * (k1 + 2 * k2 + 2 * k3 + k4) / 6
```

```python
a, b = 0.0, 1.0
N = 10
h = (b-a)/N
t = a
x = 1.0
X4 = np.zeros(N+1)
X4[0] = x
for n in range(N):
    x = RK4(t, x, func, h)
    X4[n+1] = x
    t = a + (n + 1) * h
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,256), np.exp(np.linspace(a,b,256)), '--k')
ax.plot(np.linspace(a,b,N+1), X4, 'o-k')
ax.set_xlabel("$t$")
ax.set_ylabel("$x$")
# plt.savefig("rk1.pdf", bbox_inches="tight")
```

```python
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.linspace(a,b,N+1), np.abs(X4-np.exp(np.linspace(a,b,N+1))), 'ok')
ax.set_xlabel("$t$")
ax.set_ylabel("error")
# plt.savefig("rk1_error.pdf", bbox_inches="tight")
```