## Smith-Hutton Problem

We want to solve the following PDE:

\begin{equation}
    u \frac{\partial \phi}{\partial x} + v \frac{\partial \phi}{\partial y} = \Gamma (\phi_{xx} + \phi_{yy})
\end{equation}

The independen variables, $x$ and $y$, and the diffusive constant, $\Gamma$, are used as input values for the NN, and the solution (i.e. $\phi(x,y)$) is the output. In order to find the solution, at each step the NN outputs are derived w.r.t the inputs to obtain the first order derivatives. The same is done for the gradients to obtain the second order derivatives. Then, a loss function that matches the PDE is built and the weights are updated accordingly. A loss function that matches the PDE is then used for optimization. If the loss function goes to zero, we can assume that our NN is indeed the solution to our PDE. We will try to find a general solution for different values of $\Gamma$, so we can then obtain a solution for different diffusivities without requiring new computations. The geometry of the problem is as follows:

```
# autoreload nangs
%reload_ext autoreload
%autoreload 2
%matplotlib inline
```

```
#imports
import math
import numpy as np
import matplotlib.pyplot as plt
import torch
```

First we define our PDE and set the values for training.

```
from nangs.pde import PDE
from nangs.bocos import PeriodicBoco, DirichletBoco, NeumannBoco
from nangs.solutions import MLP

class MyPDE(PDE):
    def __init__(self, inputs=None, outputs=None, order=2):
        super().__init__(inputs, outputs, order=order)
    def computePDELoss(self, grads, inputs, outputs, params):
        dpdx, dpdy = grads['p']['x'], grads['p']['y']
        d2pdx2, d2pdy2 = grads['2p']['xx'], grads['2p']['yy']
        x, y = inputs['x'], inputs['y']
        u, v = 2*y*(1-x**2), -2*x*(1-y**2)
        return [u*dpdx + v*dpdy - inputs['g']*(d2pdx2 + d2pdy2)]

# instanciate pde
pde = MyPDE(inputs=['x', 'y', 'g'], outputs=['p'])

# define input values
x = np.linspace(-1,1,60)
y = np.linspace(0,1,30)
g = np.array([0.1, 0.001, 0.000001])

pde.setValues({'x': x[1:-1], 'y': y[1:-1], 'g': g})
```

Boundary conditions.

```
# alpha parameter
a = 10

# left and rigth b.c
x1 = np.array([-1, 1])
p = np.zeros(2*len(y)*len(g))
for i in range(2*len(y)):
    for _g in g:
        p[i] = 1. - math.tanh(a)
boco = DirichletBoco('left_right', {'x': x1, 'y': y, 'g': g}, {'p': p})
pde.addBoco(boco)

# top b.c
p = np.zeros(len(x)*len(g))
for i in range(len(x)):
    for _g in g:
        p[i] = 1. - math.tanh(a)
boco = DirichletBoco('top', {'x': x, 'y': y[-1:], 'g': g}, {'p': p})
pde.addBoco(boco)

# bottom b.c
x1, x2, p1 = [], [], []
for i in range(len(x)):
    if x[i] < 0:
        x1.append(x[i])
    else:
        x2.append(x[i])
for _g in g:
    for i in range(len(x)):
        if x[i] < 0:
            p1.append(1.+math.tanh(a*(2*x[i]+1)))
x1, x2, p1 = np.array(x1), np.array(x2), np.array(p1)

# bottom left b.c
boco = DirichletBoco('bottom_left', {'x': x1, 'y': y[:1], 'g': g}, {'p': p1})
pde.addBoco(boco)

# bottom right b.c
boco = NeumannBoco('bottom_right', {'x': x2, 'y': y[:1], 'g': g}, grads={'p': 'y'})
pde.addBoco(boco)
```

```
# visualize inlet conditions (same ALPHA for every Gamma)
n, m = len(g), len(x1)
for i in range(n):
    plt.plot(x1,p1[i*m:(i+1)*m], label="ALPHA: {}".format(a))
plt.legend()
plt.grid()
plt.xlabel("x")
plt.ylabel("p    ", rotation=np.pi/2)
plt.show()
```

Now we define a topology for our solution and set the training parameters. Then we can find a solution for our PDE.

```
# define solution topology
mlp = MLP(pde.n_inputs, pde.n_outputs, 5, 2048, activations="sigmoid")

```

```
# find the solution
optimizer = torch.optim.Adam(mlp.parameters(), lr=3e-4)
pde.compile(mlp, optimizer)

hist = pde.solve(epochs=100)
```

Epoch 1/100 Losses 1.74857
 PDE  [ 0.00000 ]
 left_right 0.17609
 top 0.17609
 bottom_left 1.39639
 bottom_right 0.00000
Epoch 2/100 Losses 1.53763
 PDE  [ 0.00002 ]
 left_right 0.10797
 top 0.10735
 bottom_left 1.32227
 bottom_right 0.00002
Epoch 3/100 Losses 1.18091
 PDE  [ 0.07721 ]
 left_right 0.11919
 top 0.02221
 bottom_left 0.95166
 bottom_right 0.01064
Epoch 4/100 Losses 0.27065
 PDE  [ 0.11633 ]
 left_right 0.00431
 top 0.00191
 bottom_left 0.13392
 bottom_right 0.01419
Epoch 5/100 Losses 0.22910
 PDE  [ 0.09427 ]
 left_right 0.00646
 top 0.00211
 bottom_left 0.11348
 bottom_right 0.01278
Epoch 6/100 Losses 0.21152
 PDE  [ 0.08723 ]
 left_right 0.00692
 top 0.00246
 bottom_left 0.10287
 bottom_right 0.01205
Epoch 7/100 Losses 0.19816
 PDE  [ 0.08354 ]
 left_right 0.00608
 top 0.00207
 bottom_left 0.09557
 bottom_right 0.01091
Epoch 8/100 Losses 0.17400
 PDE  [ 0.07899 ]
 left_right 0.00443
 top 0.00164
 bottom_left 0.08044
 bottom_right 0.00850
Epoch 9/100 Losses 0.17238
 PDE  [ 0.07972 ]
 left_right 0.00361
 top 0.00201
 bottom_left 0.07739
 bottom_right 0.00965
Epoch 10/100 Losses 0.15546
 PDE  [ 0.07467 ]
 left_right 0.00260
 top 0.00174
 bottom_left 0.06837
 bottom_right 0.00808
Epoch 11/100 Losses 0.17068
 PDE  [ 0.08062 ]
 left_right 0.00328
 top 0.00308
 bottom_left 0.07346
 bottom_right 0.01024
Epoch 12/100 Losses 0.15366
 PDE  [ 0.07636 ]
 left_right 0.00225
 top 0.00180
 bottom_left 0.06412
 bottom_right 0.00913
Epoch 13/100 Losses 0.14492
 PDE  [ 0.07202 ]
 left_right 0.00271
 top 0.00204
 bottom_left 0.06037
 bottom_right 0.00778
Epoch 14/100 Losses 0.14476
 PDE  [ 0.07228 ]
 left_right 0.00205
 top 0.00170
 bottom_left 0.06062
 bottom_right 0.00812
Epoch 15/100 Losses 0.14939
 PDE  [ 0.07404 ]
 left_right 0.00264
 top 0.00248
 bottom_left 0.06335
 bottom_right 0.00689
Epoch 16/100 Losses 0.15705
 PDE  [ 0.07633 ]
 left_right 0.00376
 top 0.00401
 bottom_left 0.06538
 bottom_right 0.00757
Epoch 17/100 Losses 0.14215
 PDE  [ 0.07523 ]
 left_right 0.00171
 top 0.00128
 bottom_left 0.05701
 bottom_right 0.00692
Epoch 18/100 Losses 0.14484
 PDE  [ 0.07327 ]
 left_right 0.00365
 top 0.00316
 bottom_left 0.05834
 bottom_right 0.00643
Epoch 19/100 Losses 0.13695
 PDE  [ 0.07115 ]
 left_right 0.00229
 top 0.00200
 bottom_left 0.05596
 bottom_right 0.00555
Epoch 20/100 Losses 0.13587
 PDE  [ 0.06948 ]
 left_right 0.00226
 top 0.00223
 bottom_left 0.05603
 bottom_right 0.00586
Epoch 21/100 Losses 0.13578
 PDE  [ 0.06897 ]
 left_right 0.00284
 top 0.00281
 bottom_left 0.05639
 bottom_right 0.00477
Epoch 22/100 Losses 0.13968
 PDE  [ 0.06789 ]
 left_right 0.00349
 top 0.00342
 bottom_left 0.06048
 bottom_right 0.00439
Epoch 23/100 Losses 0.13244
 PDE  [ 0.06959 ]
 left_right 0.00219
 top 0.00212
 bottom_left 0.05291
 bottom_right 0.00564
Epoch 24/100 Losses 0.13348
 PDE  [ 0.06673 ]
 left_right 0.00279
 top 0.00243
 bottom_left 0.05729
 bottom_right 0.00424
Epoch 25/100 Losses 0.13544
 PDE  [ 0.07019 ]
 left_right 0.00264
 top 0.00226
 bottom_left 0.05603
 bottom_right 0.00433
Epoch 26/100 Losses 0.12952
 PDE  [ 0.06729 ]
 left_right 0.00246
 top 0.00197
 bottom_left 0.05377
 bottom_right 0.00403
Epoch 27/100 Losses 0.13127
 PDE  [ 0.06903 ]
 left_right 0.00198
 top 0.00138
 bottom_left 0.05421
 bottom_right 0.00467
Epoch 28/100 Losses 0.13033
 PDE  [ 0.06992 ]
 left_right 0.00247
 top 0.00201
 bottom_left 0.05258
 bottom_right 0.00335
Epoch 29/100 Losses 0.12731
 PDE  [ 0.06524 ]
 left_right 0.00221
 top 0.00178
 bottom_left 0.05484
 bottom_right 0.00324
Epoch 30/100 Losses 0.13155
 PDE  [ 0.06814 ]
 left_right 0.00241
 top 0.00188
 bottom_left 0.05492
 bottom_right 0.00419
Epoch 31/100 Losses 0.13087
 PDE  [ 0.07068 ]
 left_right 0.00225
 top 0.00218
 bottom_left 0.05222
 bottom_right 0.00354
Epoch 32/100 Losses 0.12955
 PDE  [ 0.06838 ]
 left_right 0.00223
 top 0.00244
 bottom_left 0.05280
 bottom_right 0.00370
Epoch 33/100 Losses 0.13003
 PDE  [ 0.06916 ]
 left_right 0.00222
 top 0.00179
 bottom_left 0.05205
 bottom_right 0.00480
Epoch 34/100 Losses 0.13530
 PDE  [ 0.06861 ]
 left_right 0.00325
 top 0.00291
 bottom_left 0.05682
 bottom_right 0.00370
Epoch 35/100 Losses 0.13015
 PDE  [ 0.06599 ]
 left_right 0.00310
 top 0.00261
 bottom_left 0.05439
 bottom_right 0.00406
Epoch 36/100 Losses 0.12388
 PDE  [ 0.06754 ]
 left_right 0.00204
 top 0.00196
 bottom_left 0.04945
 bottom_right 0.00289
Epoch 37/100 Losses 0.13289
 PDE  [ 0.07000 ]
 left_right 0.00313
 top 0.00292
 bottom_left 0.05352
 bottom_right 0.00333
Epoch 38/100 Losses 0.12289
 PDE  [ 0.06731 ]
 left_right 0.00169
 top 0.00171
 bottom_left 0.04915
 bottom_right 0.00303
Epoch 39/100 Losses 0.12576
 PDE  [ 0.06839 ]
 left_right 0.00182
 top 0.00142
 bottom_left 0.05107
 bottom_right 0.00307
Epoch 40/100 Losses 0.12970
 PDE  [ 0.06843 ]
 left_right 0.00259
 top 0.00216
 bottom_left 0.05273
 bottom_right 0.00379
Epoch 41/100 Losses 0.12234
 PDE  [ 0.06627 ]
 left_right 0.00229
 top 0.00229
 bottom_left 0.04796
 bottom_right 0.00352
Epoch 42/100 Losses 0.12675
 PDE  [ 0.06626 ]
 left_right 0.00280
 top 0.00255
 bottom_left 0.05135
 bottom_right 0.00379
Epoch 43/100 Losses 0.13132
 PDE  [ 0.06663 ]
 left_right 0.00356
 top 0.00292
 bottom_left 0.05498
 bottom_right 0.00324
Epoch 44/100 Losses 0.12155
 PDE  [ 0.06590 ]
 left_right 0.00187
 top 0.00164
 bottom_left 0.04987
 bottom_right 0.00227
Epoch 45/100 Losses 0.12750
 PDE  [ 0.06728 ]
 left_right 0.00272
 top 0.00247
 bottom_left 0.05218
 bottom_right 0.00286
Epoch 46/100 Losses 0.13458
 PDE  [ 0.06907 ]
 left_right 0.00384
 top 0.00301
 bottom_left 0.05404
 bottom_right 0.00461
Epoch 47/100 Losses 0.12766
 PDE  [ 0.06799 ]
 left_right 0.00239
 top 0.00239
 bottom_left 0.05244
 bottom_right 0.00246
Epoch 48/100 Losses 0.12188
 PDE  [ 0.06444 ]
 left_right 0.00277
 top 0.00253
 bottom_left 0.04973
 bottom_right 0.00241
Epoch 49/100 Losses 0.12893
 PDE  [ 0.06784 ]
 left_right 0.00284
 top 0.00225
 bottom_left 0.05224
 bottom_right 0.00376
Epoch 50/100 Losses 0.12908
 PDE  [ 0.06610 ]
 left_right 0.00307
 top 0.00275
 bottom_left 0.05422
 bottom_right 0.00294
Epoch 51/100 Losses 0.12096
 PDE  [ 0.06376 ]
 left_right 0.00216
 top 0.00192
 bottom_left 0.05038
 bottom_right 0.00274
Epoch 52/100 Losses 0.12196
 PDE  [ 0.06359 ]
 left_right 0.00243
 top 0.00201
 bottom_left 0.04987
 bottom_right 0.00405
Epoch 53/100 Losses 0.13192
 PDE  [ 0.06848 ]
 left_right 0.00260
 top 0.00253
 bottom_left 0.05434
 bottom_right 0.00396
Epoch 54/100 Losses 0.16856
 PDE  [ 0.08072 ]
 left_right 0.00531
 top 0.00328
 bottom_left 0.07086
 bottom_right 0.00840
Epoch 55/100 Losses 0.12874
 PDE  [ 0.07063 ]
 left_right 0.00264
 top 0.00215
 bottom_left 0.05081
 bottom_right 0.00252
Epoch 56/100 Losses 0.12288
 PDE  [ 0.06662 ]
 left_right 0.00250
 top 0.00223
 bottom_left 0.04954
 bottom_right 0.00199
Epoch 57/100 Losses 0.12257
 PDE  [ 0.06552 ]
 left_right 0.00224
 top 0.00217
 bottom_left 0.05050
 bottom_right 0.00214
Epoch 58/100 Losses 0.11430
 PDE  [ 0.06137 ]
 left_right 0.00227
 top 0.00227
 bottom_left 0.04636
 bottom_right 0.00203
Epoch 59/100 Losses 0.12720
 PDE  [ 0.06334 ]
 left_right 0.00312
 top 0.00305
 bottom_left 0.05450
 bottom_right 0.00318
Epoch 60/100 Losses 0.11407
 PDE  [ 0.06224 ]
 left_right 0.00146
 top 0.00141
 bottom_left 0.04729
 bottom_right 0.00166
Epoch 61/100 Losses 0.12031
 PDE  [ 0.06289 ]
 left_right 0.00282
 top 0.00268
 bottom_left 0.04960
 bottom_right 0.00233
Epoch 62/100 Losses 0.11662
 PDE  [ 0.06538 ]
 left_right 0.00142
 top 0.00141
 bottom_left 0.04628
 bottom_right 0.00213
Epoch 63/100 Losses 0.15194
 PDE  [ 0.07402 ]
 left_right 0.00490
 top 0.00265
 bottom_left 0.06215
 bottom_right 0.00822
Epoch 64/100 Losses 0.12962
 PDE  [ 0.06579 ]
 left_right 0.00271
 top 0.00178
 bottom_left 0.05645
 bottom_right 0.00289
Epoch 65/100 Losses 0.12276
 PDE  [ 0.06271 ]
 left_right 0.00211
 top 0.00190
 bottom_left 0.05311
 bottom_right 0.00293
Epoch 66/100 Losses 0.11654
 PDE  [ 0.06004 ]
 left_right 0.00212
 top 0.00177
 bottom_left 0.05017
 bottom_right 0.00244
Epoch 67/100 Losses 0.12352
 PDE  [ 0.06459 ]
 left_right 0.00260
 top 0.00265
 bottom_left 0.05112
 bottom_right 0.00256
Epoch 68/100 Losses 0.11195
 PDE  [ 0.05921 ]
 left_right 0.00187
 top 0.00182
 bottom_left 0.04644
 bottom_right 0.00260
Epoch 69/100 Losses 0.11794
 PDE  [ 0.06280 ]
 left_right 0.00200
 top 0.00199
 bottom_left 0.04839
 bottom_right 0.00277
Epoch 70/100 Losses 0.11000
 PDE  [ 0.06096 ]
 left_right 0.00141
 top 0.00138
 bottom_left 0.04389
 bottom_right 0.00236
Epoch 71/100 Losses 0.12551
 PDE  [ 0.06837 ]
 left_right 0.00220
 top 0.00221
 bottom_left 0.04910
 bottom_right 0.00363
Epoch 72/100 Losses 0.12046
 PDE  [ 0.06288 ]
 left_right 0.00207
 top 0.00253
 bottom_left 0.05074
 bottom_right 0.00224
Epoch 73/100 Losses 0.11589
 PDE  [ 0.06680 ]
 left_right 0.00163
 top 0.00175
 bottom_left 0.04261
 bottom_right 0.00311
Epoch 74/100 Losses 0.12043
 PDE  [ 0.05997 ]
 left_right 0.00296
 top 0.00318
 bottom_left 0.05114
 bottom_right 0.00319
Epoch 75/100 Losses 0.10814
 PDE  [ 0.06013 ]
 left_right 0.00156
 top 0.00170
 bottom_left 0.04243
 bottom_right 0.00231
Epoch 76/100 Losses 0.11096
 PDE  [ 0.06045 ]
 left_right 0.00165
 top 0.00149
 bottom_left 0.04449
 bottom_right 0.00287
Epoch 77/100 Losses 0.11994
 PDE  [ 0.06725 ]
 left_right 0.00174
 top 0.00154
 bottom_left 0.04556
 bottom_right 0.00386
Epoch 78/100 Losses 0.09825
 PDE  [ 0.05492 ]
 left_right 0.00160
 top 0.00140
 bottom_left 0.03853
 bottom_right 0.00180
Epoch 79/100 Losses 0.09225
 PDE  [ 0.05207 ]
 left_right 0.00184
 top 0.00188
 bottom_left 0.03413
 bottom_right 0.00234
Epoch 80/100 Losses 0.12090
 PDE  [ 0.06361 ]
 left_right 0.00356
 top 0.00263
 bottom_left 0.04681
 bottom_right 0.00429
Epoch 81/100 Losses 0.09951
 PDE  [ 0.05642 ]
 left_right 0.00157
 top 0.00148
 bottom_left 0.03676
 bottom_right 0.00327
Epoch 82/100 Losses 0.09454
 PDE  [ 0.05062 ]
 left_right 0.00176
 top 0.00178
 bottom_left 0.03781
 bottom_right 0.00257
Epoch 83/100 Losses 0.08943
 PDE  [ 0.04603 ]
 left_right 0.00164
 top 0.00170
 bottom_left 0.03739
 bottom_right 0.00267
Epoch 84/100 Losses 0.09474
 PDE  [ 0.05216 ]
 left_right 0.00171
 top 0.00186
 bottom_left 0.03610
 bottom_right 0.00291
Epoch 85/100 Losses 0.07872
 PDE  [ 0.04239 ]
 left_right 0.00149
 top 0.00155
 bottom_left 0.03063
 bottom_right 0.00266
Epoch 86/100 Losses 0.07725
 PDE  [ 0.04267 ]
 left_right 0.00105
 top 0.00133
 bottom_left 0.03022
 bottom_right 0.00197
Epoch 87/100 Losses 0.07231
 PDE  [ 0.03877 ]
 left_right 0.00122
 top 0.00135
 bottom_left 0.02865
 bottom_right 0.00232
Epoch 88/100 Losses 0.12123
 PDE  [ 0.05568 ]
 left_right 0.00323
 top 0.00347
 bottom_left 0.05419
 bottom_right 0.00465
Epoch 89/100 Losses 0.08028
 PDE  [ 0.04085 ]
 left_right 0.00189
 top 0.00203
 bottom_left 0.03265
 bottom_right 0.00285
Epoch 90/100 Losses 0.06538
 PDE  [ 0.03533 ]
 left_right 0.00144
 top 0.00146
 bottom_left 0.02505
 bottom_right 0.00210
Epoch 91/100 Losses 0.08675
 PDE  [ 0.04878 ]
 left_right 0.00157
 top 0.00192
 bottom_left 0.03090
 bottom_right 0.00358
Epoch 92/100 Losses 0.15345
 PDE  [ 0.05939 ]
 left_right 0.00264
 top 0.00285
 bottom_left 0.08194
 bottom_right 0.00662
Epoch 93/100 Losses 0.09246
 PDE  [ 0.04496 ]
 left_right 0.00135
 top 0.00146
 bottom_left 0.04170
 bottom_right 0.00300
Epoch 94/100 Losses 0.10083
 PDE  [ 0.04824 ]
 left_right 0.00251
 top 0.00243
 bottom_left 0.04378
 bottom_right 0.00388
Epoch 95/100 Losses 0.07658
 PDE  [ 0.04166 ]
 left_right 0.00114
 top 0.00139
 bottom_left 0.02960
 bottom_right 0.00278
Epoch 96/100 Losses 0.12424
 PDE  [ 0.05905 ]
 left_right 0.00426
 top 0.00458
 bottom_left 0.04938
 bottom_right 0.00698
Epoch 97/100 Losses 0.09279
 PDE  [ 0.04957 ]
 left_right 0.00183
 top 0.00184
 bottom_left 0.03622
 bottom_right 0.00333
Epoch 98/100 Losses 0.10118
 PDE  [ 0.04941 ]
 left_right 0.00334
 top 0.00341
 bottom_left 0.04068
 bottom_right 0.00434
Epoch 99/100 Losses 0.07435
 PDE  [ 0.03765 ]
 left_right 0.00130
 top 0.00128
 bottom_left 0.03109
 bottom_right 0.00304
Epoch 100/100 Losses 0.05319
 PDE  [ 0.02662 ]
 left_right 0.00080
 top 0.00075
 bottom_left 0.02292
 bottom_right 0.00209

    /opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3257: RuntimeWarning: Mean of empty slice.
      out=out, **kwargs)
    /opt/conda/lib/python3.7/site-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars
      ret = ret.dtype.type(ret / rcount)

```
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5))
ax1.plot(hist['train_loss'], label="train_loss")
ax1.grid(True)
ax1.legend()
ax1.set_yscale("log")
ax1.set_ylim([1e-2,1])
for boco in pde.bocos:
    ax2.plot(hist['bocos'][boco.name], label=boco.name)
ax2.legend()
ax2.grid(True)
ax2.set_yscale("log")
ax2.set_ylim([1e-4,1])
plt.show()
```

Finally, we can evaluate our solution.

```
# evaluate the solution
#pde.load_state_dict('best.pth')

x = np.linspace(-1,1,50)
y = np.linspace(0,1,25)
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(20,60))
g = np.array([0.1])
pde.evaluate({'x': x, 'y': y, 'g': g})
p1 = pde.outputs['p']
ax1.imshow(p1.reshape((len(y),len(x))), vmin=p1.min(), vmax=p1.max(), origin='lower',
           extent=[x.min(), x.max(), y.min(), y.max()])
ax1.set_title(f"$\\Gamma$ = 0.1")
g = np.array([0.001])
pde.evaluate({'x': x, 'y': y, 'g': g})
p2 = pde.outputs['p']
ax2.imshow(p2.reshape((len(y),len(x))), vmin=p2.min(), vmax=p2.max(), origin='lower',
           extent=[x.min(), x.max(), y.min(), y.max()])
ax2.set_title(f"$\\Gamma$ = 0.001")
g = np.array([0.000001])
pde.evaluate({'x': x, 'y': y, 'g': g})
p3 = pde.outputs['p']
ax3.imshow(p3.reshape((len(y),len(x))), vmin=p3.min(), vmax=p3.max(), origin='lower',
           extent=[x.min(), x.max(), y.min(), y.max()])
ax3.set_title(f"$\\Gamma$ = 0.000001")
#plt.colorbar()
plt.show()
```

```
# experimental results
exp = {
    'x': [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0],
    'g_0.1': [1.989,1.402,1.146,0.946,0.775,0.621,0.480,0.349,0.227,0.111,0.0],
    'g_0.001': [2.0000,1.9990,1.9997,1.985,1.841,0.951,0.154,0.001,0.0,0.0,0.0],
    'g_0.000001': [2.0000,2.0,2.0,1.999,1.964,1.0,0.036,0.001,0.0,0.0,0.0]
}

x = np.linspace(-1,1,200)
y = np.array([0])
pde.evaluate({'x': x, 'y': y, 'g': np.array([0.1])})
p = pde.outputs['p']
plt.plot(x,p, label=f"$\\Gamma$ = 0.1")
pde.evaluate({'x': x, 'y': y, 'g': np.array([0.001])})
p = pde.outputs['p']
plt.plot(x,p, label=f"$\\Gamma$ = 0.001")
pde.evaluate({'x': x, 'y': y, 'g': np.array([0.000001])})
p = pde.outputs['p']
plt.plot(x,p, label=f"$\\Gamma$ = 0.000001")

x1, p1, gamma = [], [], 0.1
for i in range(len(x)):
    if x[i] < 0:
        x1.append(x[i])
        p1.append(1.+math.tanh(a*(2*x[i]+1)))
x1, p1 = np.array(x1), np.array(p1)
plt.plot(x1,p1)

plt.plot(exp['x'], exp['g_0.1'], '.k')
plt.plot(exp['x'], exp['g_0.001'], '.k')
plt.plot(exp['x'], exp['g_0.000001'], '.k', label="experimental")
plt.grid()
plt.xlabel('x (y = 0)')
plt.ylabel('$\\phi$       ', rotation=np.pi/2)
plt.legend()
plt.show()
```

```

```


# 非線型Leslieモデル

\begin{equation}
    f(x, r) = \begin{pmatrix}
        r (x_1 + x_2) e^{-0.1(x_1 + x_2)} \\ x_1
    \end{pmatrix},\quad x = (x_1, x_2)
\end{equation}

```python
import numpy as np
import pathfollowing as pf
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
sns.set('poster', 'whitegrid', 'dark', rc={"lines.linewidth": 2, 'grid.linestyle': '-'})
```

```python
def func(x, a):
    return np.array([a[0] * (x[0] + x[1]) * np.exp(-0.1*(x[0]+x[1])) , x[0]])

def dfdx(x,a):
    return np.array([[ a[0]*np.exp(-0.1*(x[0]+x[1]))*(1.0 - 0.1*(x[0]+x[1])),a[0]*np.exp(-0.1*(x[0]+x[1]))*(1.0 - 0.1*(x[0]+x[1]))], [1.0, 0.0]])

def dfda(x,a):
    return np.array([(x[0] + x[1]) * np.exp(-0.1*(x[0]+x[1])), 0.0])
```

安定な3周期点をみつける

```python
x = np.array([1.0,1.0])
a = np.array([10.05])
for m in range(100):
    x = func(x, a)
    if m > 90:
        print(x)
```

    [36.05632191  6.20482028]
    [ 6.20482028 36.05632191]
    [6.20482028 6.20482028]
    [36.05632191  6.20482028]
    [ 6.20482028 36.05632191]
    [6.20482028 6.20482028]
    [36.05632191  6.20482028]
    [ 6.20482028 36.05632191]
    [6.20482028 6.20482028]

```python
print(x)
```

    [6.20482028 6.20482028]

3周期点の追跡

```python
a=np.array([10.05])
bd,bp,lp,pd=pf.pathfollow(x, a, func, dfdx, dfda,nmax=300, h=0.05, epsr=1.0e-10, epsb=1.0e-10, amin=10.0,amax=20.0,problem='map', quiet=True, period=3)
```

    # parameter arrived at boundary

周期倍分岐点が検出されている

```python
bd[pd[0]]
```

    {'TY': 'P',
     'x': array([3.65039653, 3.65039653]),
     'a': array([14.10442894]),
     'v': array([-0.29740519, -0.29740519,  0.90724876]),
     'udim': 1,
     'phi': array([0.76603973, 0.64279322, 0.        ])}

周期6の周期点の枝に切り替える

```python
v2 = pf.calcSwitchingVectorPD(bd[pd[0]], func, dfdx, dfda, period=6)
x2=bd[pd[0]]['x']
a2=bd[pd[0]]['a']
bd2,bp2,lp2, pd2=pf.pathfollow(x2, a2, func, dfdx, dfda, w=v2, nmax=400, h=0.025, epsr=1.0e-10, epsb=1.0e-10, amin=10.0,amax=20.0,problem='map', quiet=True,period=6)
```

    # parameter arrived at boundary

周期倍分岐点が検出されている

```python
bd2[pd2[0]]
```

    {'TY': 'P',
     'x': array([4.78940195, 6.76800969]),
     'a': array([15.87450444]),
     'v': array([0.1617148 , 0.72556074, 0.66888709]),
     'udim': 1,
     'phi': array([0.47744832, 0.87865984, 0.        ])}

周期12の周期点の枝に切り替える

```python
v4 = pf.calcSwitchingVectorPD(bd2[pd2[0]], func, dfdx, dfda, period=12)
x4=bd2[pd2[0]]['x']
a4=bd2[pd2[0]]['a']
bd4,bp4,lp4, pd4=pf.pathfollow(x4, a4, func, dfdx, dfda, w=v4, nmax=800, h=0.0125, epsr=1.0e-10, amin=10.0,amax=20.0,epsb=1.0e-12, problem='map', quiet=True,period=12)
```

    # parameter arrived at boundary

```python
print(bd4[pd4[0]])
```

    {'TY': 'P', 'x': array([4.29483468, 5.3470061 ]), 'a': array([16.31060259]), 'v': array([-0.338017  , -0.77613659,  0.53231241]), 'udim': 1, 'phi': array([0.93722738, 0.34871886, 0.        ])}

周期24

```python
v8 = pf.calcSwitchingVectorPD(bd4[pd4[0]], func, dfdx, dfda, period=24)
x8=bd4[pd4[0]]['x']
a8=bd4[pd4[0]]['a']
bd8,bp8,lp8, pd8=pf.pathfollow(x8, a8, func, dfdx, dfda, w=v8, nmax=1600, h=0.00625, epsr=1.0e-10, amin=10.0,amax=20.0,epsb=1.0e-12, problem='map', quiet=True,period=24)
```

    # parameter arrived at boundary

```python
print(bd[pd[0]]['a'], bd2[pd2[0]]['a'], bd4[pd4[0]]['a'], bd8[pd8[0]]['a'])
```

    [14.10442894] [15.87450444] [16.31060259] [16.40625687]

```python
bd_r = np.array([bd[m]['a'][0] for m in range(len(bd))])
bd_x = np.array([bd[m]['x'][0] for m in range(len(bd))])
bd_r2 = np.array([bd2[m]['a'][0] for m in range(len(bd2))])
bd_x2 = np.array([bd2[m]['x'][0] for m in range(len(bd2))])
bd_r4 = np.array([bd4[m]['a'][0] for m in range(len(bd4))])
bd_x4 = np.array([bd4[m]['x'][0] for m in range(len(bd4))])
bd_r8 = np.array([bd8[m]['a'][0] for m in range(len(bd8))])
bd_x8 = np.array([bd8[m]['x'][0] for m in range(len(bd8))])
```

```python
fig = plt.figure(figsize=(8, 5))
ax = fig.add_subplot(111)
ax.set_xlim(10,20)
ax.set_xlabel("$r$")
ax.set_ylabel("$x_1$")
ax.plot(bd_r ,bd_x, '-k')
ax.plot(bd_r2,bd_x2, '-k')
ax.plot(bd_r4,bd_x4, '-k')
ax.plot(bd_r8,bd_x8, '-k')
# plt.savefig("bd_leslie.pdf", bbox_inches='tight')
```


# Survial Models

## Time to event data

Often, we are interested in determining how some variable or intervention affects the time to occurence of a certain outcome. For example, a researcher may be interested in how a particular drug affects how long a person will survive with a particular disease. We have already seen how the Bayesian paradigm is able to model binary events. Now, we wish to go a step further and examine the waiting time for the occurence of such events.

Recall that the Bayesian paradigm consists of specifying a probability model for the observed data $D$, given an unknown vector of parameters leading to a likelihood function $L(\theta|D)$. We then assume that the parameters are random and follow some prior distribution $\pi(\theta)$. Inference for $\theta$ is carried out by way of the posterior distribution obtained by Bayes' theorem:

$$
\pi(\theta|D)\propto L(\theta|D)\pi(\theta).
$$

In survival analysis, the likelihood is somewhat more involved in its formulation. To be able to properly discuss the likelihood specification in survival models, however, we must first discuss several key quantities in survival analysis. As discussed our main outcome of interest is time, which has support $0\leq t< \infty$. We denote our time as with a random variable $T$.

## Important quantities in Survival Analysis

If the random variable $T$ is the time to the event we are studying, we can make probability statements about the lifetime of individuals. For example, we can calculate the probability of surviving past a particular time $t$ (i.e. $P(T>t)$). This probability, known as the survival function is the primary object of interest in survival analysis. The survival function is denoted by

$$S(t) = P(T > t) = 1 - F(t),$$

where $F$ is the [CDF](https://en.wikipedia.org/wiki/Cumulative_distribution_function) of $T$.  It is mathematically convenient to express the survival function in terms of the [hazard rate](https://en.wikipedia.org/wiki/Survival_analysis#Hazard_function_and_cumulative_hazard_function), $h(t)$.  The hazard rate is the instantaneous probability that the event occurs at time $t$ given that it has not yet occured.  That is,

$$\begin{align*}
h(t)
    & = \lim_{\Delta t \to 0} \frac{P(t < T < t + \Delta t\ |\ T > t)}{\Delta t} \\
    & = \frac{f(t)}{S(t)}
\end{align*}$$

Furthermore, we have that

$$S(t) = \exp\left(-\int_0^s h(s)\ ds\right).$$

This representation of the survival function shows that the cumulative hazard function

$$H(t) = \int_0^t h(s)\ ds$$

is an important quantity in survival analysis, since we may consicesly write $S(t) = \exp(-H(t)).$

An important, but subtle, point in survival analysis is [censoring](https://en.wikipedia.org/wiki/Survival_analysis#Censoring). In many survival studies, many data points are `right censored` or rather survival times are usually only known for a portion for those under study where as the remainder are only known to exceed certain values. Specifically, an observation is said to be right censored at $c$ if the exact value of the observation is not known but only that it is greater than or equal to $c$.

### Data notation

Suppose that there are $n$ subjects under study and that associated with the $i^{th}$ individual is a survival time $t_i^*$ and a fixed censoring time $c_i$. The exact survival time $t_i$ will be observed only if $t_i^* \leq c_i$. Hence our data is of the form $(y_i,\delta_i)$ where
$$
\begin{equation}
t_i = min(t_i^*, c_i)
\end{equation}
$$
and
$$
\begin{equation}
\delta_i= I(t_i^*\leq c_i)
\end{equation}
$$

In our dataset, $\delta_i$ is given by `df.event` and

```python
%matplotlib inline
import numpy as np
import pandas as pd
import pymc3 as pm
import arviz as az
from pymc3 import (Gamma, Poisson, Normal, Model, sample, forestplot,
                   NUTS, Metropolis, find_MAP, starting, traceplot)
import seaborn as sns
import matplotlib.pyplot as plt
from statsmodels import datasets
from theano import tensor as tt
from io import StringIO
import warnings
warnings.simplefilter('ignore')

sns.set_style("ticks")

SEEDS = 2234323, 172354266
np.random.seed(SEEDS)
```

### Mastectomy Data

In this tutorial, we will use an oncological dataset.
In this dataset, each row represents observations from a woman diagnosed with breast cancer that underwent a mastectomy.

- `time` ($t_i$) represents the time (in months) post-surgery that the woman was observed.
- `event` indicates whether or not the woman died during the observation period (i.e. $\delta_i$).
- `metastized` represents whether the cancer had metastized prior to surgery.

We will analyze the relationship between survival time post-mastectomy and whether or not the cancer had metastized.

```python
df = datasets.get_rdataset('mastectomy', 'HSAUR', cache=True).data
df.event = df.event.astype(np.int64)
df.metastized = (df.metastized == 'yes').astype(np.int64)
n_patients = df.shape[0]
patients = np.arange(n_patients)
```

```python
fig, ax = plt.subplots(figsize=(14, 10))

blue, _, green = sns.color_palette()[:3]

ax.hlines(patients[df.metastized.values == 0], 0, df[df.metastized.values == 0].time,
          color=green, label='Not metastized')

ax.hlines(patients[df.metastized.values == 1], 0, df[df.metastized.values == 1].time,
          color=blue, label='Metastized')

ax.scatter(df[df.event.values == 1].time, patients[df.event.values == 1],
           color='k', zorder=10, label='Death')

ax.set_xlim(left=0)
ax.set_xlabel('Months since mastectomy')
ax.set_yticks([])
ax.set_ylabel('Subject')

ax.set_ylim(-0.25, n_patients + 0.25)

ax.legend(loc='center right');
```

As an initial look into survival, we plot the non-parametric Kaplan Meier estimates of the survival via the package `lifelines`

```python
from lifelines import KaplanMeierFitter
kmf = KaplanMeierFitter()
```

```python
kmf.fit(df.time, event_observed=df.event)
```

    <lifelines.KaplanMeierFitter:"KM_estimate", fitted with 44 total observations, 18 right-censored observations>

The Kaplan-Meier plot shows the dynamics of the groups as events occur, accounting for both events and the population at risk during each time interval (accounting for dropout, etc.).

The survival probability is calculated at each interval by dividing the number of individuals surviving by the number at risk. Individuals who are lost are **censored**. The probability of surviving to any selected time interval is estimated from the cumulative probability of surviving all the previous intervals (i.e., their product). Note that the probability for any particular interval isn’t very accurate due to the small number of events, however the overall probability of surviving to each point is more accurate.

Things to note:

- Vertical axis is the estimated probability of survival for a **hypothetical cohort**, not the actual percent surviving.
- Precision of estimates depends on # observations. Hence, precision decreases from left to right.
- Curves often give the impression that events happen more frequently in the early intervals, due to the high survival rate and relatively large sample.

```python
time = df.time
event = df.event
ix0 = df.metastized==0
ix1 = df.metastized==1

kmf.fit(time[ix0], event[ix0], label='no Metastasis')
ax = kmf.plot(color=green)

kmf.fit(time[ix1], event[ix1], label='Metasasis')
kmf.plot(ax=ax, ls='dashed', color=blue, figsize=(14,10))
ax.set_xlabel('Time (months)')
ax.set_ylabel('Survival');
```

# Model Specification

We wish to relate a **treatment effect** to the time-to-event outcome. There are several ways to do this, but two main paradigms are pervasive in the literature:

1. Proportional Hazards
2. Accelerated Failure Time

Here, we will focus on the first of these models.

There are a number of probability models one could use for time-to-event data but the simplest is the **Exponential distribution**:

$$Pr(x \mid \lambda) = \lambda \exp( -\lambda x)  $$

The corresponding hazard function is constant:

$$h(x \mid \lambda) = \lambda$$

Alternatively, **Weibull distribution** is a very flexible model that is popular with many researchers.

$$Pr(x \mid \alpha, \beta) = \frac{\alpha x^{\alpha - 1} \exp(-(\frac{x}{\beta})^{\alpha})}{\beta^\alpha}$$

Its flexibility comes from the fact that it has a hazard rate which is either monotone increasing, decreasing, or constant.

$$h(x \mid \alpha, \beta) = \frac{\alpha}{\beta^{\alpha}} x^{\alpha -1}$$

The scale parameter $\beta$ is often fixed to 1, giving a simplified form:

$$h(x \mid \alpha) = \alpha x^{\alpha -1}$$

## Proportional Hazards Model

We want to specify a model that relates the group effect to the time outcome. Recall above that the hazard quantifies the instantaneous risk of disease at a given time point.

The easiest survival parameter to model is the hazard rate. The major approach to modeling the effects of covariates on survival is to model the conditional hazard rate as a function of the covariates. Two general classes of models have been used to relate covariate effects to survival: the family of **multiplicative hazard models** and the family of **additive hazard models**. We will focus on the former of these here.

For the family of multiplicative hazard rate models the conditional hazard rate of an individual with covariate vector $X$ is a product of a baseline hazard rate $h_0(t)$ and a non-negative function of the covariates,
$c(\beta^T X)$, that is, $h (t | z) =h_0(t)c(\beta^T X)$.

   - $h_0(x)$ may have a specified parametric form or it may be left as an arbitrary nonnegative function.
   - Any nonnegative function can be used for the link function $c()$. Most applications use the Cox (1972) model with $c(\beta^T X) = \exp(\beta^T X)$ which is chosen for its simplicity
   - Key Feature: when all the covariates are fixed at time 0, the hazard rates of two individuals with distinct
   values of X are proportional.

Using the exponential link, we have
$$
{\displaystyle h (t|X_{i})=h _{0}(t)\exp(\beta^T X_{i} )}
$$

You can see that the model is formulated in such a fashion that the group effect has a multiplicative effect on the hazard. Typically, semi-parametric models which don't have explicit representations of $h_0(t)$ are preferred, however for the purposes of this tutorial, we will assume a particular form for this baseline hazard.

The Weibull proportional hazards model represents the hazard as

$$
{\displaystyle h (t|X_{i})=\alpha\lambda t^{\alpha-1}\exp(\beta^TX_{i})}
$$

## Likelihood Specification

Due to right censoring, we need to adjust our likelihood to account for observations where we know that an individual survived up to a particular point. If a person is known to have survived at least to time $t$, then their likelihood contribution is simply the survival function evaluated at the censoring time. We typically express this with an indicator variable $(\delta_i)$ for each observation denoting whether or not the individual was right censored or observed.

First, recall that the Weibull proportional hazards model models the hazard as a function of the covariates. The probability density function is related to the hazard by:

$$
\begin{equation}
f(t) = h(t)S(t)
\end{equation}
$$

The likelihood specification for an individual is therefore given by:
$$
\begin{eqnarray}
L_i &=& [f(t|X)]^{\delta_i}[S(t|X)]^{1-\delta_i}\\
&=& [h(t|X)S(t|X)]^{\delta_i}[S(t|X)]^{1-\delta_i}\\
&=& [\alpha\lambda t^{\alpha-1}\exp(\beta^T X)exp(-\lambda t^{\alpha}\exp(\beta^T X))]^{\delta_i}[exp(-\lambda t^{\alpha}\exp(\beta^T X))]^{1-\delta_i}\\
&=&[\alpha\lambda t^{\alpha-1}\exp(\beta^T X-\lambda t^{\alpha}\exp(\beta^T X))]^{\delta_i}[exp(-\lambda t^{\alpha}\exp(\beta^T X))]^{1-\delta_i}\\
\end{eqnarray}
$$

### Specification in PyMC3

We've formulated the mathematical form of our likelihood - we now need to implement it. PyMC3 makes this really easy to do with the `DenistyDist` class. This class allows you to specify custom distributions by specifying the log-probability function that you want.

Continuing from before:

$$
\begin{eqnarray}
log L_i &=& \log[[\alpha\lambda t^{\alpha-1}\exp(\beta^T X-\lambda t^{\alpha}\exp(\beta^T X))]^{\delta_i}[exp(-\lambda t^{\alpha}\exp(\beta^T X))]^{1-\delta_i}]\\
&=& \delta_i \log[\alpha\lambda t^{\alpha-1}\exp(\beta^T X-\lambda t^{\alpha}\exp(\beta^T X))]+(1-\delta_i)(-\lambda t^{\alpha}\exp(\beta^T X))\\
&=&\delta_i(\log\alpha+\log\lambda+(\alpha-1)\log t + \beta^T X) - \lambda t^{\alpha}\exp(\beta^T X)
\end{eqnarray}
$$

Implemented in PyMC3, this is:

```python
def logp(event):
    return (event*(tt.log(α) + tt.log(λ)
                   + tt.log(tt.exp(β*np.array(metast)))
                   + (α-1)*tt.log(np.array(time)))
            - (λ*tt.exp(β*np.array(metast))
                   * np.array(time)**α))

survival = pm.DensityDist('survival', logp, observed={'event':event})
```

### Prior Distribution specification:

The next step is to supply priors for he parameters of the survival model. Looking at the parameters to estimate, we have

1. Weibull baseline hazard parameters: $\alpha$,$\lambda>0$
    - A natural prior for these parameters is the gamma distribution.

```python
from pymc3 import Gamma

params = (5, 1), (1, 3), (5, 5), (0.5, 0.5), (10, 1)

fig, axes = plt.subplots(1, len(params), figsize=(14, 4), sharey=True)
for ax, (alpha, beta) in zip(axes, params):
    sns.distplot(Gamma.dist(alpha, beta).random(size=1000), ax=ax, kde=False)
    ax.set_xlim(0, 20)
    ax.set_title(r'$\alpha={0}, \beta={1}$'.format(alpha, beta));
```

 - Alternatively, one might specify a half-Cauchy distribution:

```python
sns.distplot(pm.HalfCauchy.dist(1).random(size=1000), kde=False)
```

2. Group effect: $-\infty<\beta<\infty$

For the effect of metastasis, the most natural prior is the Normal distribution.

```python
sns.distplot(pm.Normal.dist(0,100).random(size=3000), kde=False)
```

### Implementation in PyMC3

Putting all the pieces together, we can fit the model using the pymc framework as so:

```python
metast = df.metastized.values
```

```python
n_iterations = 1000
n_tune = 1000
```

```python
with Model() as weibull_model:

    # Weibull shape parameter
    α = pm.HalfCauchy("α", 1)

    # Effects of metastisis
    λ = pm.HalfCauchy("λ", 1)

    # Weibull scales
    β = pm.Normal('β', 0, 100)

    linear = tt.exp(β*metast)

    # Weibull survival likelihood, accounting for censoring
    def logp(event):
        return event*(tt.log(α) + tt.log(λ) + tt.log(linear)
                      + (α-1)*tt.log(np.array(time))) - (λ*linear * np.array(time)**α)

    survival = pm.DensityDist('survival', logp, observed={'event':event})

    weibull_trace = pm.sample(n_iterations, tune=n_tune, cores=2)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (2 chains in 2 jobs)
    NUTS: [β, λ, α]
    Sampling 2 chains, 0 divergences: 100%|██████████| 4000/4000 [00:06<00:00, 653.42draws/s]
    The acceptance probability does not match the target. It is 0.8970896030434324, but should be close to 0.8. Try to increase the number of tuning steps.

We can view the posterior samples and assess convergence of the Markov Chain:

```python
az.plot_trace(weibull_trace, var_names=['α', 'λ', 'β']);
```

Since we have fit a proportional hazards model, the interpretation of the group effect is straightforward (at least with respect to the hazard). We need only to exponentiate the $\beta$ posterior samples for the effect size and corresponding credible intervals.

```python
hr_samps = np.exp(weibull_trace['β'])
print('Hazard ratio (metastisized vs not metastisized) at 5 years: {0:.2f} {1}'.format(hr_samps.mean(),
                                                            np.percentile(hr_samps, [2.5, 97.5]).round(2)))
```

    Hazard ratio (metastisized vs not metastisized) at 5 years: 2.35 [0.86 5.76]

> Interpretation: This means that the hazard for the metastasis group is bigger
than that for the group with no metastasis. Thus, metastasis has a negative effect on survival.

### Posterior Survival quantites

Given that we have posterior samples for each of the parameters, it is straightforward to obtain posterior survival curves, hazard rates, etc, with corresponding credible intervals.

```python
## Plot survival curves
n_intervals = 120
interval_bounds = np.arange(0, n_intervals + 1, 1)
```

```python
#baseline_lam = np.exp(weibull_trace['β'][n_burn:,0])
baseline_lam = np.repeat(1., n_iterations*2)
metast_lam = np.exp(weibull_trace['β'])
λ_trace = weibull_trace['λ']
α_trace = weibull_trace['α']
```

```python
def plot_with_hpd(x, lam, f, ax, linestyle='-', color=None, label=None, alpha=0.05, debug=False, **kwargs):

    m = np.median(f(lam, **kwargs), axis=0)

    percentiles = 100 * np.array([alpha / 2., 1. - alpha / 2.])
    hpd = np.percentile(f(lam, **kwargs), percentiles, axis=0)

    if debug:
        import pdb;pdb.set_trace()
    ax.fill_between(x, hpd[0], hpd[1], color=color, alpha=0.15)
    ax.step(x, m, color=color, linestyle=linestyle, label=label);
```

```python
def cum_weibull_hazard(lam, α=α_trace ,λ=λ_trace, t=n_intervals):
    return (λ_trace.reshape(-1,1)*lam.reshape(-1,1) * np.arange(t)) ** α.reshape(-1,1)

def weibull_survival(lam,α=α_trace, λ=λ_trace, t=n_intervals):
    return np.exp(-cum_weibull_hazard(lam, α, λ, t))

def cum_hazard(hazard, n=n_intervals):
    return (np.arange(n) * hazard.reshape(-1,1)).cumsum(axis=-1)

def survival(hazard):
    return np.exp(-cum_hazard(hazard))
```

```python
fig, (hazard_ax, surv_ax) = plt.subplots(ncols=2, sharex=True, sharey=False, figsize=(16, 6))

plot_with_hpd(interval_bounds[:-1], baseline_lam, cum_weibull_hazard,
              hazard_ax, color=green, linestyle=':', label='No Metastasis')
plot_with_hpd(interval_bounds[:-1], metast_lam, cum_weibull_hazard,
              hazard_ax, color=blue, linestyle='--', label='Metastisized')

hazard_ax.set_xlim(0, n_intervals);
hazard_ax.set_xlabel('Followup time');

hazard_ax.set_ylabel(r'Cumulative hazard $\Lambda(t)$');

hazard_ax.legend(loc=2);

plot_with_hpd(interval_bounds[:-1], baseline_lam, weibull_survival,
              surv_ax, color=green, linestyle=':')
plot_with_hpd(interval_bounds[:-1], metast_lam, weibull_survival,
              surv_ax, color=blue, linestyle='--')

surv_ax.set_xlim(0, n_intervals);
surv_ax.set_xlabel('Followup time');

surv_ax.set_ylabel('Survival function $S(t)$');

fig.suptitle('Bayesian survival model');
```

Looking at our KM estimate from earlier, we see that the posterior survival curves follow closely to the non-parametric K-M estimates.

```python
## Hazard ratio comparing 2 groups at a given time point
def weibull_hazard(lam, t,α=α_trace, λ=λ_trace):
    return (α.reshape(-1,1) * λ.reshape(-1,1) * lam.reshape(-1,1) * t**(α.reshape(-1,1) - 1))
```

We can of course, obtain our hazard ratio from the posterior samples:

```python
hazard_ratio = weibull_hazard(metast_lam, 60)/weibull_hazard(baseline_lam, 60)
print('Hazard ratio (metastisized vs notmetastisized) at 5 years: {0:.2f} [{1:.2f}, {2:.2f}]'.format(hazard_ratio.mean(),
                                                                     *np.percentile(hazard_ratio, [2.5, 97.5])))
```

    Hazard ratio (metastisized vs notmetastisized) at 5 years: 2.35 [0.86, 5.76]

Other quantities of interest include quantities such as the time at which survival is 50%

```python
baseline_survival = weibull_survival(baseline_lam)
metast_survival = weibull_survival(metast_lam)

np.where(baseline_survival.mean(axis=0)<=0.5)[0][0]
```

    81

```python
np.where(metast_survival.mean(axis=0)<=0.5)[0][0]
```

    40

Or relatedly, the survival to some specific time period.

```python
baseline_survival_5 = baseline_survival[:, 60]
metast_survival_5 = metast_survival[:, 60]
print('Baseline 5-year median survival: {0:.2f} {1}'.format(np.median(baseline_survival),
                                                        np.percentile(baseline_survival, [2.5, 97.5]).round(2)))

print('Metastasis group median 5-year survival: {0:.2f} {1}'.format(np.median(metast_survival_5),
                                                        np.percentile(metast_survival_5, [2.5, 97.5]).round(2)))
```

    Baseline 5-year median survival: 0.62 [0.15 0.98]
    Metastasis group median 5-year survival: 0.35 [0.11 0.83]

## Exercise: Heart transplant survival

Try your hand at building a survival model using PyMC3. Here is a small dataset based on 69 patients receiving heart transplants. Taken from "The Statistical Analysis of Failure Time Data" by Kalbfleisch and Prentice, Appendix I, pages 230-232 from stalib data depository. http://lib.stat.cmu.edu/datasets/stanford

Data columns:
- Age at transplant in years
- Survival Status 1=dead 0=alive
- Survival Time after transplant in days

Estimate the relationship between age at transplant and survival probability.

```python
data_str = """
41 1 5
40 1 16
54 1 16
29 1 17
55 1 28
52 1 30
40 1 39
35 0 39
56 1 43
36 1 45
42 1 51
50 1 53
42 1 58
52 1 61
61 1 66
45 1 68
49 1 68
53 1 72
47 1 72
64 1 77
51 1 78
53 1 80
54 1 81
56 1 90
53 1 96
48 1 100
28 0 109
46 1 110
23 0 131
47 1 153
43 1 165
26 0 180
52 1 186
47 1 188
51 1 207
51 1 219
47 0 265
48 1 285
19 1 285
49 1 308
42 1 334
44 0 340
47 1 342
54 0 370
48 0 397
52 0 445
46 0 482
48 0 515
52 0 545
48 1 583
26 0 596
47 0 630
47 0 670
50 1 675
58 1 733
32 0 841
44 1 852
41 0 915
38 0 941
45 1 979
48 1 995
43 1 1032
36 0 1141
45 0 1321
53 1 1386
48 0 1407
40 0 1571
48 0 1586
33 0 1799
"""

transplant_data = pd.read_table(StringIO(data_str), sep='\s+', names=['transplant_age', 'death', 'time'])
transplant_data.head()
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>transplant_age</th>
      <th>death</th>
      <th>time</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>41</td>
      <td>1</td>
      <td>5</td>
    </tr>
    <tr>
      <th>1</th>
      <td>40</td>
      <td>1</td>
      <td>16</td>
    </tr>
    <tr>
      <th>2</th>
      <td>54</td>
      <td>1</td>
      <td>16</td>
    </tr>
    <tr>
      <th>3</th>
      <td>29</td>
      <td>1</td>
      <td>17</td>
    </tr>
    <tr>
      <th>4</th>
      <td>55</td>
      <td>1</td>
      <td>28</td>
    </tr>
  </tbody>
</table>
</div>

```python
# Write your answer here
```

---
# References

Ibrahim, Joseph G., Ming‐Hui Chen, and Debajyoti Sinha. Bayesian survival analysis. John Wiley & Sons, Ltd, 2005.

Klein, John P., Moeschberger, Melvin L. Survival Analysis: Techniques for Censored and Truncated Data Second Edition. Springer, 2003.

[Survival Distributions, Hazard Functions, Cumulative Hazards](https://web.stanford.edu/~lutian/coursepdf/unit1.pdf)


# Big Linear Set of Quaternions

Generate a few quaternions using a class found in Q_tool_devo.

```python
%%capture
%matplotlib inline
import numpy as np
import sympy as sp
import matplotlib.pyplot as plt

# To get equations the look like, well, equations, use the following.
from sympy.interactive import printing
printing.init_printing(use_latex=True)
from IPython.display import display

# Tools for manipulating quaternions.
import Q_tools as qt;
```

The class is call QHArray().

```python
qha = qt.QHArray()

for q_step in qha.range(q_start=qt.QH([0, 0, 0, 0]), q_delta=qt.QH([1, 0.1, 0.2, 0.3]), n_steps=10):
    print(q_step)
```

    (0, 0, 0, 0) Q
    (1, 0.1, 0.2, 0.3) Q+1dQ
    (2, 0.2, 0.4, 0.6) Q+2dQ
    (3, 0.30000000000000004, 0.6000000000000001, 0.8999999999999999) Q+3dQ
    (4, 0.4, 0.8, 1.2) Q+4dQ
    (5, 0.5, 1.0, 1.5) Q+5dQ
    (6, 0.6, 1.2, 1.8) Q+6dQ
    (7, 0.7, 1.4, 2.1) Q+7dQ
    (8, 0.7999999999999999, 1.5999999999999999, 2.4) Q+8dQ
    (9, 0.8999999999999999, 1.7999999999999998, 2.6999999999999997) Q+9dQ
    (10, 0.9999999999999999, 1.9999999999999998, 2.9999999999999996) Q+10dQ

Write out 10k to disk.

```python
with open('/tmp/10k.data', 'w') as datafile:
    for q_step in qha.range(q_start=qt.QH([0, 0, 0, 0]), q_delta=qt.QH([1, 0.1, 0.2, 0.3]), n_steps=10000):
        datafile.write("{}, {}, {}, {}\n".format(q_step.t, q_step.x, q_step.y, q_step.z))
```

```python
! wc -l /tmp/10k.data
! tail -4 /tmp/10k.data
```

       10001 /tmp/10k.data
    9997, 999.7000000001588, 1999.4000000003175, 2999.100000000358
    9998, 999.8000000001588, 1999.6000000003176, 2999.400000000358
    9999, 999.9000000001588, 1999.8000000003176, 2999.700000000358
    10000, 1000.0000000001588, 2000.0000000003176, 3000.0000000003583

Bingo, bingo, we can make a large number of quaternions.

```python

```


## Doubling formula unrolling

```python
from pyecsca.ec.point import Point, InfinityPoint
from pyecsca.ec.mod import Mod, SymbolicMod
from pyecsca.ec.model import ShortWeierstrassModel, EdwardsModel, MontgomeryModel, TwistedEdwardsModel
from pyecsca.ec.coordinates import AffineCoordinateModel
from pyecsca.ec.context import local, DefaultContext
from pyecsca.ec.formula import LadderFormula, DifferentialAdditionFormula, DoublingFormula
from pyecsca.misc.cfg import getconfig
from sympy import symbols, Mod as SympyMod, Eq
import warnings
```

```python
def remove_mod(expression):
    if isinstance(expression, SympyMod):
        return remove_mod(expression.args[0])
    if not expression.args:
        return expression
    args = []
    for arg in expression.args:
        if isinstance(arg, SympyMod):
            arg = remove_mod(arg.args[0])
        else:
            arg = remove_mod(arg)
        args.append(arg)
    return expression.func(*args)
```

```python
# Whether to display formulas in LaTeX or as raw text
#display_mode = "latex"
display_mode = "raw"
```

## Short Weierstrass

```python
model = ShortWeierstrassModel()
acoords = AffineCoordinateModel(model)
a, b = symbols("a b")
p = 53  # This number is unused, but pyecsca needs some prime to be the modulus even for symbolic exec
getconfig().ec.unsatisfied_formula_assumption_action = "warning"
for coord_name, coords in model.coordinates.items():
    svars = {var:SymbolicMod(symbols(var), p) for var in coords.variables}
    pt = Point(coords, **svars)
    params = {"a": SymbolicMod(a, p), "b": SymbolicMod(b, p)}
    for param in coords.parameters:
        params[param] = SymbolicMod(symbols(param), p)
    print(f"###### {coord_name} ######")
    for name, formula in coords.formulas.items():
        if not isinstance(formula, DoublingFormula):
            continue
        print(f"--- {name} ---")
        with warnings.catch_warnings(record=True) as w:
            try:
                with local(DefaultContext()) as ctx:
                    res = formula(p, pt, **params)[0]
                action, _ = ctx.actions.get_by_index([0])
                for name, values in action.intermediates.items():
                    # The `values` here contains all the values assigned to the variable during the execution of the formula.
                    # Output all of the `values` to get all intermediate values.
                    value = values[-1]
                    if name in formula.outputs:  # Remove this test if you want the output of all intermediates.
                        if display_mode == "latex":
                            display(Eq(symbols(name), remove_mod(value.value.x)))
                        elif display_mode == "raw":
                            print(f"{name} =", remove_mod(value.value.x))
            except Exception as e:
                print(e)
        if w:
            print(w[0].message)
        print()

```

## Edwards

```python
model = EdwardsModel()
acoords = AffineCoordinateModel(model)
c, d = symbols("c d")
p = 53  # This number is unused, but pyecsca needs some prime to be the modulus even for symbolic exec
getconfig().ec.unsatisfied_formula_assumption_action = "warning"
for coord_name, coords in model.coordinates.items():
    svars = {var:SymbolicMod(symbols(var), p) for var in coords.variables}
    pt = Point(coords, **svars)
    params = {"c": SymbolicMod(c, p), "d": SymbolicMod(d, p)}
    for param in coords.parameters:
        params[param] = SymbolicMod(symbols(param), p)
    print(f"###### {coord_name} ######")
    for name, formula in coords.formulas.items():
        if not isinstance(formula, DoublingFormula):
            continue
        print(f"--- {name} ---")
        with warnings.catch_warnings(record=True) as w:
            try:
                with local(DefaultContext()) as ctx:
                    res = formula(p, pt, **params)[0]
                action, _ = ctx.actions.get_by_index([0])
                for name, values in action.intermediates.items():
                    # The `values` here contains all the values assigned to the variable during the execution of the formula.
                    # Output all of the `values` to get all intermediate values.
                    value = values[-1]
                    if name in formula.outputs:  # Remove this test if you want the output of all intermediates.
                        if display_mode == "latex":
                            display(Eq(symbols(name), remove_mod(value.value.x)))
                        elif display_mode == "raw":
                            print(f"{name} =", remove_mod(value.value.x))
            except Exception as e:
                print(e)
        if w:
            print(w[0].message)
        print()
```

## Montgomery

```python
model = MontgomeryModel()
acoords = AffineCoordinateModel(model)
a, b = symbols("a b")
p = 53  # This number is unused, but pyecsca needs some prime to be the modulus even for symbolic exec
getconfig().ec.unsatisfied_formula_assumption_action = "warning"
for coord_name, coords in model.coordinates.items():
    svars = {var:SymbolicMod(symbols(var), p) for var in coords.variables}
    pt = Point(coords, **svars)
    params = {"a": SymbolicMod(a, p), "b": SymbolicMod(b, p)}
    for param in coords.parameters:
        params[param] = SymbolicMod(symbols(param), p)
    print(f"###### {coord_name} ######")
    for name, formula in coords.formulas.items():
        if not isinstance(formula, DoublingFormula):
            continue
        print(f"--- {name} ---")
        with warnings.catch_warnings(record=True) as w:
            try:
                with local(DefaultContext()) as ctx:
                    res = formula(p, pt, **params)[0]
                action, _ = ctx.actions.get_by_index([0])
                for name, values in action.intermediates.items():
                    # The `values` here contains all the values assigned to the variable during the execution of the formula.
                    # Output all of the `values` to get all intermediate values.
                    value = values[-1]
                    if name in formula.outputs:  # Remove this test if you want the output of all intermediates.
                        if display_mode == "latex":
                            display(Eq(symbols(name), remove_mod(value.value.x)))
                        elif display_mode == "raw":
                            print(f"{name} =", remove_mod(value.value.x))
            except Exception as e:
                print(e)
        if w:
            print(w[0].message)
        print()
```

## Twisted Edwards

```python
model = TwistedEdwardsModel()
acoords = AffineCoordinateModel(model)
a, d = symbols("a d")
p = 53  # This number is unused, but pyecsca needs some prime to be the modulus even for symbolic exec
getconfig().ec.unsatisfied_formula_assumption_action = "warning"
for coord_name, coords in model.coordinates.items():
    svars = {var:SymbolicMod(symbols(var), p) for var in coords.variables}
    pt = Point(coords, **svars)
    params = {"a": SymbolicMod(a, p), "d": SymbolicMod(d, p)}
    for param in coords.parameters:
        params[param] = SymbolicMod(symbols(param), p)
    print(f"###### {coord_name} ######")
    for name, formula in coords.formulas.items():
        if not isinstance(formula, DoublingFormula):
            continue
        print(f"--- {name} ---")
        with warnings.catch_warnings(record=True) as w:
            try:
                with local(DefaultContext()) as ctx:
                    res = formula(p, pt, **params)[0]
                action, _ = ctx.actions.get_by_index([0])
                for name, values in action.intermediates.items():
                    # The `values` here contains all the values assigned to the variable during the execution of the formula.
                    # Output all of the `values` to get all intermediate values.
                    value = values[-1]
                    if name in formula.outputs:  # Remove this test if you want the output of all intermediates.
                        if display_mode == "latex":
                            display(Eq(symbols(name), remove_mod(value.value.x)))
                        elif display_mode == "raw":
                            print(f"{name} =", remove_mod(value.value.x))
            except Exception as e:
                print(e)
        if w:
            print(w[0].message)
        print()
```


# short notebook to show how to get the mc info or recalculate the energy distribution

```python
import numpy as np
import matplotlib.pyplot as plt

from ctapipe.utils import get_dataset_path
from eventio import SimTelFile
from lstchain import mc
```

```python
simtel_file = get_dataset_path('gamma_test_large.simtel.gz')
```

```python
source = SimTelFile(simtel_file)
```

```python
source.mc_run_headers[0]
```

    {'shower_prog_id': 1,
     'shower_prog_vers': 6990,
     'shower_prog_start': 1467892800,
     'detector_prog_id': 1,
     'detector_prog_vers': 1462392225,
     'detector_prog_start': 1467924457,
     'obsheight': 2150.0,
     'n_showers': 20000,
     'n_use': 20,
     'core_pos_mode': 1,
     'core_range': array([   0., 3000.], dtype=float32),
     'alt_range': array([1.2217305, 1.2217305], dtype=float32),
     'az_range': array([0., 0.], dtype=float32),
     'diffuse': 1,
     'viewcone': array([ 0., 10.], dtype=float32),
     'E_range': array([3.0e-03, 3.3e+02], dtype=float32),
     'spectral_index': -2.0,
     'B_total': 23.11772346496582,
     'B_inclination': -0.39641156792640686,
     'B_declination': 0.0,
     'injection_height': -1.0,
     'atmosphere': 26,
     'corsika_iact_options': 187,
     'corsika_low_E_model': 2,
     'corsika_high_E_model': 3,
     'corsika_bunchsize': 5.0,
     'corsika_wlen_min': 240.0,
     'corsika_wlen_max': 700.0,
     'corsika_low_E_detail': 0,
     'corsika_high_E_detail': 303}

```python
emin, emax = source.mc_run_headers[0]['E_range']
spectral_index = source.mc_run_headers[0]['spectral_index']
num_showers = source.mc_run_headers[0]['n_showers']
num_use = source.mc_run_headers[0]['n_use']
num_events = num_showers * num_use
```

---
## Compute the energy distribution

\begin{align}
N_{tot} = \int_{E_{min}}^{E_{max}} \frac{dN}{dE} dE = \int_{E_{min}}^{E_{max}} E^{\gamma} dE
\end{align}

If $\gamma = 1$:
$$
N_{tot} = \ln(E_{max}/E_{min})
$$

Else:
$$
N_{tot} = \frac{E_{max}^{\gamma+1} - E_{min}^{\gamma+1}}{\gamma + 1}
$$

```python
bins, y = mc.power_law_integrated_distribution(emin, emax, num_events, spectral_index)
```

---

## Getting the true simulated energies

```python
energies = []
with SimTelFile(simtel_file) as f:
    for i, event in enumerate(f.iter_mc_events()):
        energies.append(event['mc_shower']['energy'])
```

## Comparing both

```python
e_mean = (bins[:-1] + bins[1:])/2.

plt.figure(figsize=(15,10))
plt.hist(energies,
         bins=bins,
#          histtype='step',
         alpha=0.4,
         label='histogram of mc events energies',
        )
plt.xscale('log')
plt.xlabel(r'$E \,/\, \mathrm{TeV}$')
plt.tight_layout()
plt.yscale('log')

plt.plot(e_mean, y)

# plt.bar(e_mean, y, width=bins[:-1]-bins[1:], log=True,
#         color='red', alpha=0.5,
#         label='Histogram computed from mc info')

plt.step(e_mean, y, where='mid',
         linewidth=3,
         color='black',
         label='Histogram computed from mc info'
        )

plt.plot(e_mean, y)

plt.legend(fontsize=15);

```

```python

```


```
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
import sympy
#from astropy.visualization import astropy_mpl_style, quantity_support
#from google.colab import drive
#drive.mount('/content/drive')
c=299792458
```

Una matriz en python y algunas operaciones. Con esto uds van a construir un cuaderno para calcular eventos espacio-tiempo en 2 sistema de referencia.

**En la siguiente celda**

```
vx = input('Ingrese la velocidad para el boost en x')
vy = input('Ingrese la velocidad para el boost en y')
vz = input('Ingrese la velocidad para el boost en z')
vx = float(vx)
betax=vx/c
vy = float(vy)
betay=vy/c
vz = float(vz)
betaz=vz/c
v2=vx**2+vy**2+vz**2
beta2=betax**2+betay**2+betaz**2
if v2>c**2:
  print('Este valor de velocidad no es posible, está por encima de c, los cálculos NO SON CORRECTOS')

```

    Ingrese la velocidad para el boost en x5000
    Ingrese la velocidad para el boost en y4000
    Ingrese la velocidad para el boost en z3000

```
#Ahora se calcula directamente la T. de Lorentz con la forma general
Mgeneral=[[gamma, -gamma*betax, -gamma*betay, -gamma*betaz],[-gamma*betax, 1+(gamma-1)*betax**2/beta2, (gamma-1)*betax*betay/beta2,(gamma-1)*betax*betaz/beta2],[-gamma*betay, (gamma-1)*betay*betax/beta2 ,1+(gamma-1)*betay**2/beta2, (gamma-1)*betay*betaz/beta2],[-gamma*betaz, (gamma-1)*betaz*betax/beta2, (gamma-1)*betay*betaz/beta2, 1+(gamma-1)*betaz**2/beta2]]
print(Mgeneral)
```

    [[1.1547005383792517, -1.9258332015464707e-05, -1.5406665612371767e-05, -1.1554999209278823e-05], [-1.9258332015464707e-05, 1.0773502691896257, 0.06188021535170067, 0.04641016151377551], [-1.5406665612371767e-05, 0.06188021535170068, 1.0495041722813605, 0.037128129211020405], [-1.1554999209278823e-05, 0.04641016151377551, 0.037128129211020405, 1.0278460969082652]]