#  CS109B Data Science 2: Advanced Topics in Data Science
## Lecture 3 - Coding Environment Setup and review of `statsmodels`
###  Notebook B

**Harvard University**

**Spring 2021**

**Instructors:** Mark Glickman, Pavlos Protopapas, and Chris Tanner

**Additional Instructor:** Eleni Kaxiras

*Content:* Eleni Kaxiras and Will Claybaugh

---

```python
## RUN THIS CELL TO PROPERLY HIGHLIGHT THE EXERCISES
import requests
from IPython.core.display import HTML
styles = requests.get("https://raw.githubusercontent.com/Harvard-IACS/2019-CS109B/master/content/styles/cs109.css").text
HTML(styles)
```

```python
import numpy as np
from scipy.interpolate import interp1d
import matplotlib.pyplot as plt
import pandas as pd

%matplotlib inline
```

## Learning Goals

By the end of this lab, you should be able to:
* use `np.linalg.vander`
* use the weird R-style formulas in `statsmodels`
* practice least-squares regression in `statsmodels`

### Basis Functions

In our models we can use various types of functions as basis functions. Strictly speaking, in linear algebra where a basis for a subspace S of $\mathbb{R}^n$ is a set of vectors that spans S and is linearly independent. As a reminder, a set of vectors $\textbf{v}_1, \textbf{v}_2, ..., \textbf{v}_k$ are considered linearly independent if they cannot be written as a linear combination of each other, such that, if: $c_1\textbf{v}_1+c_2\textbf{v}_2+ ...+ c_k\textbf{v}_k = \textbf{0}$ then $c_1,c_2,...,c_k$ are all zero.

In data science where we have lots of imperfect data (with errors), as well as imperfect computers (with round-off errors), when we substitute their values into the matrices we almost always get column degeneracy, meaning, some of our columns become linear combinations of each other. Especially so if we use the monomial basis and go beyond ~5,6 degree of the polynomial.

Examples are:

- Monomials such as $x,x^2,x^4,x^5$
- Sigmoid/ReLU functions (neural networks)
- Fourier functions
- Wavelets
- Splines

The matrix produced when we substitute the values of our data into the basis functions is called the *design matrix*.

### Linear/Polynomial Regression

We will use the `diabetes` dataset.

Variables are:
- subject:   subject ID number
- age:       age diagnosed with diabetes
- acidity:   a measure of acidity called base deficit
Response:
- y:         natural log of serum C-peptide concentration

*Original source is Sockett et al. (1987) mentioned in Hastie and Tibshirani's book
"Generalized Additive Models".*

Reading the data in Pandas:

```python
diab = pd.read_csv("data/diabetes.csv")
diab.head()
```

      subject
      age
      acidity
      y

      0
      1
      5.2
      -8.1
      4.8

      1
      2
      8.8
      -16.1
      4.1

      2
      3
      10.5
      -0.9
      5.2

      3
      4
      10.6
      -7.8
      5.5

      4
      5
      10.4
      -29.0
      5.0

### Create the design matrix for a fictitious dataset
Let's keep just the `age` feature and create some columns of our own. Let's see how good this matrix is before we create the design matrix.

```python
diab_age = diab[['age']].copy()
diab_age['age2'] = diab_age.apply(lambda row: row['age']**2, axis=1)
diab_age['random'] = np.random.normal(0,1,len(diab_age))
diab_age['same'] = diab_age['age']
diab_age.head()
```

      age
      age2
      random
      same

      0
      5.2
      27.04
      0.983557
      5.2

      1
      8.8
      77.44
      -1.167022
      8.8

      2
      10.5
      110.25
      1.167812
      10.5

      3
      10.6
      112.36
      1.667912
      10.6

      4
      10.4
      108.16
      1.616005
      10.4

```python
A = diab_age.to_numpy(copy=True)
A[:5]
```

    array([[  5.2       ,  27.04      ,   0.98355687,   5.2       ],
           [  8.8       ,  77.44      ,  -1.16702237,   8.8       ],
           [ 10.5       , 110.25      ,   1.16781157,  10.5       ],
           [ 10.6       , 112.36      ,   1.66791156,  10.6       ],
           [ 10.4       , 108.16      ,   1.61600545,  10.4       ]])

Let's check if the columns of A are linearly independent by using some linear algebra methods from `numpy.linalg` and `sympy`.

```python
from numpy.linalg import matrix_rank
matrix_rank(A)
```

    3

```python
# check out which rows are linearly independent
import sympy
_, inds = sympy.Matrix(A).T.rref()
inds
```

    (0, 1, 2)

```python
np.linalg.cond(A)
```

    7.913850862345827e+16

### Create the design matrix for `age` using a polynomial basis
Let's keep just the `age` feature again and create the design matrix using a polynomial of degree `n`. First we will use the basic `numpy` formula `vander()`.

```python
vand = np.vander(diab_age.age, 2, increasing=True)
vand[:3], vand.shape
```

    (array([[ 1. ,  5.2],
            [ 1. ,  8.8],
            [ 1. , 10.5]]),
     (43, 2))

```python
## To our point why the Vandermonde matrix is usually ill-conditioned,
## find the condition number of this matrix
np.linalg.cond(vand), matrix_rank(vand)
```

    (24.709088478926237, 2)

Exercise: Vandermonde matrix

Change the degree of the polynomial and comment on what happens to the condition and rank of the matrix.

```python
vand = np.vander(diab_age.age, 8, increasing=True)
vand[:3], vand.shape
```

    (array([[1.00000000e+00, 5.20000000e+00, 2.70400000e+01, 1.40608000e+02,
             7.31161600e+02, 3.80204032e+03, 1.97706097e+04, 1.02807170e+05],
            [1.00000000e+00, 8.80000000e+00, 7.74400000e+01, 6.81472000e+02,
             5.99695360e+03, 5.27731917e+04, 4.64404087e+05, 4.08675596e+06],
            [1.00000000e+00, 1.05000000e+01, 1.10250000e+02, 1.15762500e+03,
             1.21550625e+04, 1.27628156e+05, 1.34009564e+06, 1.40710042e+07]]),
     (43, 8))

```python
## To our point why the Vandermonde matrix is usually ill-conditioned,
## find the condition number of this matrix
np.linalg.cond(vand), matrix_rank(vand)
```

    (4207821284.7228603, 8)

### Linear/Polynomial regression with statsmodels.

As you remember from 109a, we have two tools for Linear Regression:
- `statsmodels` [https://www.statsmodels.org/stable/regression.html](https://www.statsmodels.org/stable/regression.html), and
- `sklearn`[https://scikit-learn.org/stable/index.html](https://scikit-learn.org/stable/index.html)

Previously, in this notebook, we worked from a vector of target values and a design matrix we built ourself. In 109a we used e.g. `sklearn`'s PolynomialFeatures to build the matrix. Now we will look at `statsmodels` which allows users to fit statistical models using R-style **formulas**. They build the target value and design matrix for you.

**Note:** Categorical features (e.g. let's say we had a categorical feature called Region, are designated by `C(Region)`), polynomial features (e.g. age) are entered as `np.power(age, n)` where `n` is the degree of the polynomial **OR** `np.vander(age, n, increasing=True)`.

```
# Example: if our target variable is 'Lottery', while 'Region' is a categorical predictor and all the others are numerical:
df = dta.data[['Lottery', 'Literacy', 'Wealth', 'Region']]

formula='Lottery ~ Literacy + Wealth + C(Region) + Literacy * Wealth'
```

For more on these formulas see:

- https://www.statsmodels.org/stable/examples/notebooks/generated/formulas.html
- https://patsy.readthedocs.io/en/latest/overview.html

```python
import statsmodels.formula.api as smf

model1 = smf.ols('y ~ age', data=diab)
fit1_lm = model1.fit()
```

Let's build a dataframe to predict values on (sometimes this is just the test or validation set). Very useful for making pretty plots of the model predictions - predict for TONS of values, not just whatever's in the training set.

```python
x_pred = np.linspace(0.5,20,100)

predict_df = pd.DataFrame(data={"age":x_pred})
predict_df.head()
```

      age

      0
      0.500000

      1
      0.696970

      2
      0.893939

      3
      1.090909

      4
      1.287879

Use `get_prediction(<data>).summary_frame()` to get the model's prediction (and error bars!)

```python
prediction_output = fit1_lm.get_prediction(predict_df).summary_frame()
prediction_output.head()
```

      mean
      mean_se
      mean_ci_lower
      mean_ci_upper
      obs_ci_lower
      obs_ci_upper

      0
      4.037574
      0.233301
      3.566412
      4.508736
      2.650278
      5.424870

      1
      4.053940
      0.228886
      3.591696
      4.516184
      2.669647
      5.438232

      2
      4.070305
      0.224489
      3.616940
      4.523670
      2.688952
      5.451658

      3
      4.086671
      0.220113
      3.642143
      4.531198
      2.708193
      5.465149

      4
      4.103036
      0.215759
      3.667302
      4.538770
      2.727369
      5.478703

Plot the data, the fitted model, the **confidence intervals**, and the *prediction intervals*. For more on how `statsmodels` calculates these intervals see: https://www.statsmodels.org/stable/_modules/statsmodels/regression/_prediction.html

```python
ax1 = diab.plot.scatter(x='age',y='y',c='brown',title="Diabetes data with least-squares linear fit")
ax1.set_xlabel("Age at Diagnosis")
ax1.set_ylabel("Log C-Peptide Concentration")

ax1.plot(predict_df.age, prediction_output['mean'],color="green")
ax1.plot(predict_df.age, prediction_output['mean_ci_lower'], color="blue",linestyle="dashed")
ax1.plot(predict_df.age, prediction_output['mean_ci_upper'], color="blue",linestyle="dashed")
ax1.plot(predict_df.age, prediction_output['obs_ci_lower'], color="green",linestyle="dashdot")
ax1.plot(predict_df.age, prediction_output['obs_ci_upper'], color="green",linestyle="dashdot");
```

Breakout Room Exercise

- Fit a 3rd degree polynomial model to predict `y` using only `age` and
- Plot the model and its confidence intervals.
- Change the degree of your polynomial and see what happens to the fitted curve.
- Does our model have an intercept? *Note*: we can discover the existence or not of an intercept in our model by running:
```
model_name.params
```

```python
# your answer here

model2 = smf.ols(formula = 'y ~ np.vander(age, 4, increasing=True)-1', data=diab, missing='drop')
fit2_lm = model2.fit()
```

```python
x_pred = np.linspace(0.5,20,100)

predict_df = pd.DataFrame(data={"age":x_pred})
predict_df.head()
```

      age

      0
      0.500000

      1
      0.696970

      2
      0.893939

      3
      1.090909

      4
      1.287879

```python
prediction_output = fit2_lm.get_prediction(predict_df).summary_frame()
prediction_output.head()
```

      mean
      mean_se
      mean_ci_lower
      mean_ci_upper
      obs_ci_lower
      obs_ci_upper

      0
      3.057129
      0.405478
      2.236972
      3.877285
      1.599098
      4.515159

      1
      3.173396
      0.370403
      2.424185
      3.922608
      1.754061
      4.592732

      2
      3.285013
      0.338413
      2.600508
      3.969517
      1.898743
      4.671282

      3
      3.392068
      0.309541
      2.765962
      4.018174
      2.033685
      4.750451

      4
      3.494653
      0.283823
      2.920567
      4.068739
      2.159449
      4.829857

```python
ax2 = diab.plot.scatter(x='age',y='y',c='brown',title="Diabetes data with least-squares linear fit")
ax2.set_xlabel("Age at Diagnosis")
ax2.set_ylabel("Log C-Peptide Concentration")

ax2.plot(predict_df.age, prediction_output['mean'],color="green")
ax2.plot(predict_df.age, prediction_output['mean_ci_lower'], color="blue",linestyle="dashed")
ax2.plot(predict_df.age, prediction_output['mean_ci_upper'], color="blue",linestyle="dashed")
ax2.plot(predict_df.age, prediction_output['obs_ci_lower'], color="green",linestyle="dashdot")
ax2.plot(predict_df.age, prediction_output['obs_ci_upper'], color="green",linestyle="dashdot");
```

```python
# solution

```

Vandermonde matrix in formulas

It's easier to build higher order polynomials using `np.vandrer()`.

```python
formula = "y ~ np.vander(age, 6, increasing=True) -1"
fit3_lm = smf.ols(formula=formula, data=diab).fit()
```

```python
fit3_lm.params
```

    np.vander(age, 6, increasing=True)[0]    3.147753
    np.vander(age, 6, increasing=True)[1]    0.130248
    np.vander(age, 6, increasing=True)[2]    0.136034
    np.vander(age, 6, increasing=True)[3]   -0.028749
    np.vander(age, 6, increasing=True)[4]    0.002064
    np.vander(age, 6, increasing=True)[5]   -0.000050
    dtype: float64

```python
## To our point why the Vandermonde matrix is usually ill-conditioned,
# find the condition number of this matrix
np.linalg.cond(np.vander(predict_df.age, 6, increasing=True))
```

    11539278.6863067

```python
# solution
poly_predictions = fit3_lm.get_prediction(predict_df).summary_frame()
poly_predictions.head()
```

      mean
      mean_se
      mean_ci_lower
      mean_ci_upper
      obs_ci_lower
      obs_ci_upper

      0
      3.243420
      0.734767
      1.754641
      4.732198
      1.308481
      5.178358

      1
      3.295358
      0.568205
      2.144066
      4.446650
      1.606277
      4.984440

      2
      3.353648
      0.438495
      2.465173
      4.242123
      1.831507
      4.875789

      3
      3.417256
      0.347322
      2.713515
      4.120997
      1.995013
      4.839499

      4
      3.485217
      0.295584
      2.886306
      4.084128
      2.111820
      4.858614

```python
# solution
x_pred = np.linspace(0.5,15,100)
predict_df = pd.DataFrame(data={"age":x_pred})

ax2 = diab.plot.scatter(x='age',y='y',c='Red',title="Diabetes data with least-squares cubic fit")
ax2.set_xlabel("Age at Diagnosis")
ax2.set_ylabel("Log C-Peptide Concentration")

ax2.plot(predict_df.age, poly_predictions['mean'],color="green")
ax2.plot(predict_df.age, poly_predictions['mean_ci_lower'], color="blue",linestyle="dashed", label='confidence interval')
ax2.plot(predict_df.age, poly_predictions['mean_ci_upper'], color="blue",linestyle="dashed");
ax2.legend();
```

Discussion

#### QR decomposition (**Beyond the scope of this class**)

As you know, to find the parameters of our model, we may try to solve the so-called *normal equations*, which, written in matrix form, are:

\begin{equation}
    (\boldsymbol{A^T}\cdot \boldsymbol{A}) \cdot \boldsymbol{b} = \boldsymbol{A} \cdot \boldsymbol{y}
\end{equation}

The direct solution is $\hat{\boldsymbol{b}}=(\boldsymbol{A}^T\cdot \boldsymbol{A})^{-1}\cdot \boldsymbol{A}^T \cdot \boldsymbol{y}$

Solving the least-squares problem directly via the normal equations is susceptible to roundoff error when the condition of the matrix $\boldsymbol{A}$ is large. An alternative technique involves QR decomposition (details in any good linear algebra book). `statsmodels` lets you use this technique via a parameter in the `.fit`:
```
   .fit(method='qr')
```

#### Let's try with QR now

```python
formula = "y ~ np.vander(age, 6, increasing=True) -1"
fit3_lm = smf.ols(formula=formula, data=diab).fit(method='qr')
```

```python
fit3_lm.params
```

    np.vander(age, 6, increasing=True)[0]   NaN
    np.vander(age, 6, increasing=True)[1]   NaN
    np.vander(age, 6, increasing=True)[2]   NaN
    np.vander(age, 6, increasing=True)[3]   NaN
    np.vander(age, 6, increasing=True)[4]   NaN
    np.vander(age, 6, increasing=True)[5]   NaN
    dtype: float64

```python
# solution
poly_predictions = fit3_lm.get_prediction(predict_df).summary_frame()
poly_predictions.head()
```

      mean
      mean_se
      mean_ci_lower
      mean_ci_upper
      obs_ci_lower
      obs_ci_upper

      0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      1
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      3
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      4
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

```python
# solution
ax2 = diab.plot.scatter(x='age',y='y',c='Red',title="Diabetes data with least-squares cubic fit")
ax2.set_xlabel("Age at Diagnosis")
ax2.set_ylabel("Log C-Peptide Concentration")

ax2.plot(predict_df.age, poly_predictions['mean'],color="green")
ax2.plot(predict_df.age, poly_predictions['mean_ci_lower'], color="blue",linestyle="dashed", label='confidence interval')
ax2.plot(predict_df.age, poly_predictions['mean_ci_upper'], color="blue",linestyle="dashed");
ax2.legend();
```

```python

```

```python

```


```python
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
#sns.set_theme(style="whitegrid")
sns.set(rc={'figure.figsize':(16,13)})
```

```python
# Absolute error
def AE(y, y_hat, t, T):

    resid = np.abs(y - y_hat)

    return resid

#############################################

# Temporal decay loss
def EP_td(y, y_hat, t, T, losstype="loss", weight=1):

    decay = ((T-t)/T)
    resid = np.abs(y - y_hat)

    if losstype=="loss":
        EP = resid**decay

    if losstype=="penalty":
        EP = resid+ weight*(resid**decay)

    return EP

# Progressive temporal decay loss
def EP_prog_td(y, y_hat, t, T, losstype="loss", weight=1):

    decay = 1/t

    resid = np.abs(y - y_hat)

    if losstype=="loss":
        EP = resid*decay
    if losstype=="penalty":
        EP = resid+ weight*(resid*decay)

    return EP
```

### Previous formulations

\begin{equation}MAE_{PtD} = \frac{1}{N}\sum_{i=1}^{N}\frac{1}{T}\sum_{t=1}^{T_i}\mid y_{t}^i - \hat{y}_{t}^i\mid ^{\frac{T_i-t}{T_i}}\end{equation}

and

\begin{equation}MAE_{MtD} = \frac{1}{N}\sum_{i=1}^{N}\frac{1}{T}\sum_{t=1}^{T_i}\frac{\mid y_{t}^i - \hat{y}_{t}^i\mid}{t} \end{equation}

```python
"""
Missing the "balance" weight to adjust accuracy versus earliness,
and missing a more flexible rate of decay form
"""
```

## Useful formulas for exponentials:

Radioactive decay example:
\begin{equation}
X_{t+1} = \frac{X_{t}}{2}
\end{equation}
Half-life = 1 year

Bounded exponential growth (Logistic equation):

\begin{equation}
X_{t+1} = rX_{t}\left(1-\frac{X_{t}}{max(X)}\right)
\end{equation}

```python

```

# New loss formulation

Basic components:

\begin{equation}
MAE + \beta f(MAE,\alpha)
\end{equation}

Where $\beta$ is the weighting parameter of the earliness component, versus the $\gamma$ parameter that weight normal accuracy. $\alpha$ acts as a decay-rate parameter specifying to which degree the temporal decay should prioritize the earliest events in a sequence/trace.

The earliness/penalty component consist of:

\begin{equation}
f(MAE,\alpha) = \frac{1}{N}\sum_{i=1}^{N}\frac{1}{T}\sum_{t=1}^{T_i}\frac{\mid y_{t}^i - \hat{y}_{t}^i\mid}{\alpha t}
\end{equation}

and thus the full loss becomes:

\begin{equation}
MAE_{td}: \frac{1}{N}\sum_{i=1}^{N}\frac{1}{T}\sum_{t=1}^{T_i}\gamma\mid y_{t}^i - \hat{y}_{t}^i\mid + \beta \frac{\mid y_{t}^i - \hat{y}_{t}^i\mid}{\alpha t}
\end{equation}

However, the first batches of experiments show that adding the accuracy component only leads to inferior performance compared to baseline MAE. Current implementation of MAE_td is thus as simple as:

\begin{equation}
MAE_{td}: \frac{1}{N}\sum_{i=1}^{N}\frac{1}{T}\sum_{t=1}^{T_i}\frac{\mid y_{t}^i - \hat{y}_{t}^i\mid}{\alpha t}
\end{equation}

```python
# Temporal decay loss
def EP_td(y, y_hat, t, T, losstype="penalty", alpha=1, beta=1, gamma=1):

    resid = np.abs(y - y_hat)

    EP = gamma*resid + beta*(resid/t*alpha)

    return EP
```

```python
def generate_obs(metric, T = 10, alpha = 1, beta = 1, gamma=1, loss_comp="loss"):
    y = np.linspace(start=250,stop=50, num=10,endpoint=True)
    y_hat = np.linspace(start=250,stop=50, num=10,endpoint=True)+50
    t = np.array([1,2,3,4,5,6,7,8,9,10])

    #placeholder
    resid = [0]*len(t)

    if metric == "AE":
        resid = AE(y, y_hat, t, T)

    if metric == "EP_td":
        resid = EP_td(y, y_hat, t, T, losstype=loss_comp, alpha=alpha, beta=beta, gamma=gamma)

    #if metric == "EP_prog_td":
    #    resid = EP_prog_td(y, y_hat, t, T, losstype=loss_comp, weight=weight_1)

    df = pd.DataFrame({"y":y,
                      "y_hat":y_hat,
                      "t":t,
                      "resid":resid,
                      "loss":[metric]*T,
                       "alpha":[alpha]*T,
                       "beta":[beta]*T,
                       "gamma":[gamma]*T})
    return df

"""
Test the data generator with no loss function
"""
generate_obs("EP_td",
             T = 10,
             alpha = 1,
             beta = 1,
             gamma=1,
             loss_comp="loss")

```

      y
      y_hat
      t
      resid
      loss
      alpha
      beta
      gamma

      0
      250.000000
      300.000000
      1
      100.000000
      EP_td
      1
      1
      1

      1
      227.777778
      277.777778
      2
      75.000000
      EP_td
      1
      1
      1

      2
      205.555556
      255.555556
      3
      66.666667
      EP_td
      1
      1
      1

      3
      183.333333
      233.333333
      4
      62.500000
      EP_td
      1
      1
      1

      4
      161.111111
      211.111111
      5
      60.000000
      EP_td
      1
      1
      1

      5
      138.888889
      188.888889
      6
      58.333333
      EP_td
      1
      1
      1

      6
      116.666667
      166.666667
      7
      57.142857
      EP_td
      1
      1
      1

      7
      94.444444
      144.444444
      8
      56.250000
      EP_td
      1
      1
      1

      8
      72.222222
      122.222222
      9
      55.555556
      EP_td
      1
      1
      1

      9
      50.000000
      100.000000
      10
      55.000000
      EP_td
      1
      1
      1

```python
metrics_of_interest = ["AE","EP_td"] #,"EP_prog_td"

#accuracy/earliness balance
betas = [1] #[0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75]
gammas = [1] #[0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75]

#decay rate parameter
alphas = [0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75]

data = []

for balance1 in betas:

    for balance2 in gammas:

        for rate in alphas:

            for metric in metrics_of_interest:

                obs = generate_obs(metric, beta = balance1, gamma=balance2, alpha = rate)

                data.append(obs)

residuals = pd.concat(data)
residuals.head()
```

      y
      y_hat
      t
      resid
      loss
      alpha
      beta
      gamma

      0
      250.000000
      300.000000
      1
      50.0
      AE
      0.0
      1
      1

      1
      227.777778
      277.777778
      2
      50.0
      AE
      0.0
      1
      1

      2
      205.555556
      255.555556
      3
      50.0
      AE
      0.0
      1
      1

      3
      183.333333
      233.333333
      4
      50.0
      AE
      0.0
      1
      1

      4
      161.111111
      211.111111
      5
      50.0
      AE
      0.0
      1
      1

```python
g = sns.FacetGrid(residuals, col="alpha", col_wrap=4, height=4, size=3,  hue="loss", aspect=1) #
g.map(sns.barplot, 't','resid')
plt.savefig("loss_functions.png")
plt.show()
```

## Multiple settings

```python
metrics_of_interest = ["EP_td"] #,"EP_prog_td"

#accuracy/earliness balance
betas = [1] #[0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75]
gammas = [0, 0.5, 1] #[0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75]

#decay rate parameter
alphas = [0, 0.75, 1.0, 1.75]

data = []

for balance1 in betas:

    for balance2 in gammas:

        for rate in alphas:

            for metric in metrics_of_interest:

                obs = generate_obs(metric, beta = balance1, gamma=balance2, alpha = rate)

                data.append(obs)

residuals = pd.concat(data)
residuals#.head(10)
```

      y
      y_hat
      t
      resid
      loss
      alpha
      beta
      gamma

      0
      250.000000
      300.000000
      1
      0.000000
      EP_td
      0.00
      1
      0.0

      1
      227.777778
      277.777778
      2
      0.000000
      EP_td
      0.00
      1
      0.0

      2
      205.555556
      255.555556
      3
      0.000000
      EP_td
      0.00
      1
      0.0

      3
      183.333333
      233.333333
      4
      0.000000
      EP_td
      0.00
      1
      0.0

      4
      161.111111
      211.111111
      5
      0.000000
      EP_td
      0.00
      1
      0.0

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      5
      138.888889
      188.888889
      6
      64.583333
      EP_td
      1.75
      1
      1.0

      6
      116.666667
      166.666667
      7
      62.500000
      EP_td
      1.75
      1
      1.0

      7
      94.444444
      144.444444
      8
      60.937500
      EP_td
      1.75
      1
      1.0

      8
      72.222222
      122.222222
      9
      59.722222
      EP_td
      1.75
      1
      1.0

      9
      50.000000
      100.000000
      10
      58.750000
      EP_td
      1.75
      1
      1.0

120 rows × 8 columns

```python
g = sns.FacetGrid(residuals, col="alpha", row="gamma", hue="loss")
g.map(sns.barplot, 't','resid')
```

```python
metrics_of_interest = ["EP_td"] #,"EP_prog_td"

balance_weights = np.linspace(start=0.5,stop=2, num=4,endpoint=True) #[ 0.5, 1.0, 1.5]
rates = np.linspace(start=0.5, stop=2, num=4, endpoint=True) #[1.0, 1.25, 1.5]

data = []

for rate in rates:

    for balance in balance_weights:

        for metric in metrics_of_interest:

            obs = generate_obs(metric, balance = balance, rate = rate)

            data.append(obs)

residuals = pd.concat(data)
residuals.head(20)
```

      y
      y_hat
      t
      resid
      loss
      balance
      rate

      0
      250.000000
      300.000000
      1
      62.500000
      EP_td
      0.5
      0.5

      1
      227.777778
      277.777778
      2
      56.250000
      EP_td
      0.5
      0.5

      2
      205.555556
      255.555556
      3
      54.166667
      EP_td
      0.5
      0.5

      3
      183.333333
      233.333333
      4
      53.125000
      EP_td
      0.5
      0.5

      4
      161.111111
      211.111111
      5
      52.500000
      EP_td
      0.5
      0.5

      5
      138.888889
      188.888889
      6
      52.083333
      EP_td
      0.5
      0.5

      6
      116.666667
      166.666667
      7
      51.785714
      EP_td
      0.5
      0.5

      7
      94.444444
      144.444444
      8
      51.562500
      EP_td
      0.5
      0.5

      8
      72.222222
      122.222222
      9
      51.388889
      EP_td
      0.5
      0.5

      9
      50.000000
      100.000000
      10
      51.250000
      EP_td
      0.5
      0.5

      0
      250.000000
      300.000000
      1
      75.000000
      EP_td
      1.0
      0.5

      1
      227.777778
      277.777778
      2
      62.500000
      EP_td
      1.0
      0.5

      2
      205.555556
      255.555556
      3
      58.333333
      EP_td
      1.0
      0.5

      3
      183.333333
      233.333333
      4
      56.250000
      EP_td
      1.0
      0.5

      4
      161.111111
      211.111111
      5
      55.000000
      EP_td
      1.0
      0.5

      5
      138.888889
      188.888889
      6
      54.166667
      EP_td
      1.0
      0.5

      6
      116.666667
      166.666667
      7
      53.571429
      EP_td
      1.0
      0.5

      7
      94.444444
      144.444444
      8
      53.125000
      EP_td
      1.0
      0.5

      8
      72.222222
      122.222222
      9
      52.777778
      EP_td
      1.0
      0.5

      9
      50.000000
      100.000000
      10
      52.500000
      EP_td
      1.0
      0.5

```python
g = sns.FacetGrid(residuals, col="balance",row="rate", hue="loss")
g.map(sns.barplot, 't','resid')
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

analysis/.ipynb_checkpoints/Temporal loss parameterized version-checkpoint.ipynb


#### _Speech Processing Labs 2021: SIGNALS 1: More on Interpreting the DFT (Extension)_

```python
## Run this first!
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

import cmath
from math import floor
from matplotlib.animation import FuncAnimation
from IPython.display import HTML
plt.style.use('ggplot')

from dspMisc import *
```

# More on Interpreting the  DFT

     This notebook is extension material:  This notebook goes through DFT output frequencies and leakage in more detail than is strictly necessary for this course.  It's perfectly fine to skip it for now.

### Learning Outcomes
* Understand how sampling rate effects the DFT output
* Understand what the DFT leakage is.

### Need to know
* Topic Videos: Fourier Analysis, Frequency Domain
* [Digital Signals: Sampling sinusoids](./signals-1-2-sampling-sinusoids.ipynb)
* [The Discrete Fourier Transform](./signals-1-3-discrete-fourier-transform-in-detail.ipynb)

Equation alert: If you're viewing this on github, please note that the equation rendering is not always perfect. You should view the notebooks through a jupyter notebook server for an accurate view.

## A Very Quick Recap of the DFT

The [previous notebook](./signals-1-3-discrete-fourier-transform-in-detail.ipynb) went through the mechanics of the Discrete Fourier Transform (DFT).  To summarize, the DFT input and output are broadly:

* **Input:**  $N$ amplitude samples over time
    * $x[n]$, for $n=0..N-1$ (i.e. a time series of $N$ samples)

* **Output:** the dot product (i.e., the similiarity) between the input and $N$ sinusoids with different frequencies
    * DFT[k] $= Me^{-j\phi}$, i.e. a complex number (in polar form) with **magnitude** $M$ and **phase** angle $\phi$
    * The $N$ DFT outputs represent $N$ equally space frequencies between 0 and the sampling rate.

The outputs are calculated using the following formula for $k=0,...N-1$.

$$
\begin{align}
DFT[k] &= \sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi nk}{N}} \\
&= \sum_{n=0}^{N-1} x[n]\big[\cos\big(\frac{2\pi nk}{N} \big) - j \sin\big(\frac{2\pi nk}{N} \big) \big]
\end{align}
$$

You can think DFT[k] as a **phasor**, which looks like an analogue clockhand (i.e. a vector) ticking (i.e., rotating) around a clockface (i.e. a circle), where the length of the hand is the **peak amplitude** of that wave, and how fast it goes around the clock is it's frequency.  Each of these DFT[k] 'clocks' corresponds to a sinusoid of a specific frequency.

Each DFT[k] output essentially tells us whether the input signal has a sinusoidal component that matches the $k$th DFT phasor frequency. So, we talk about the DFT outputs as providing the **frequency response** of the input signal.

Since the the DFT outputs are complex numbers, we can talk about them in terms of magnitude and phase angle: The **magnitude** of DFT[k] tells us how much we'd weight the $k$-th phasor if we were to try to reconstruct the original input by adding all the DFT phasors together.  The **phase angle** of DFT[k] tells use whether we need to shift that wave along the time axis.

## The DFT Frequency Response:  Which Frequencies?

In [the first notebook on interpreting the DFT](./signals-1-1-interpreting-the-discrete-fourier-transform.ipynb) we saw that for input of length $N$, the DFT **output analysis frequencies** are $N$ evenly space points between 0 Hz and the sampling rate. So, how can we see this from the DFT equation?

We can first note that DFT[0] (corresponding to a 0 Hz, i.e., a phasor stuck at one point) is the average of the input sequence. This tells us  the amplitude of the waveform (i.e. whether it's centered above or below 0 in amplitude).  This is often referred to as the DC component ('Direct Current') in electrical engineering texts.

Now, we can work out all the other output frequencies by noticing that DFT[1] represents a phasor that takes N equal steps to make one complete one full circle (clockwise starting from (1,0)).  So, $e^{-j 2\pi n/N}$ in the equation represents the $n$th step around the circle.

Let's call the **sampling rate** $f_s$ (samples/second).  We can then figure out the frequency of represented by DFT[1] by figuring out the time it takes to make one cycle (i.e., the period), which is the time it takes to make $N$ steps.

* The time between each sample (i.e., the **sampling time**) is $t_s = 1/f_s$  (seconds)

* So, $N$ samples takes $t_s \times N$  (seconds x samples = seconds)

* And it will take the $k=1$ phasor $T = t_s \times N$  (seconds) to make 1 complete cycle
    * This is the **period** or **wavelength** of the phasor

* Thus, the **frequency** of the $k=1$ phasor is $f_{min} = 1/T = 1/(t_s N) $ (cycles/second)
    * i.e., $f_{min} = f_s/N$

So, the minimum frequency that we can analyse in an application of the DFT $f_{min}$ depends on the input size $N$ and the sampling rate $f_s$.

From there we can see that DFT[k] represents a phasor that completes the circle $k$ times faster than the one corresponding to DFT[1]. That is,

    The frequency associated with DFT[k] is:
    $kf_{min}$ (cycles/second) = $kf_s/N$

Since  $k$ = 0,...,$N-1$, this is the same as saying we take taking N evenly space points between 0 Hz and the sampling rate, $f_s$, which is the shortcut we took in [the first notebook on the DFT](./signals-1-1-interpreting-the-discrete-fourier-transform.ipynb).

Thinking about this in terms of sampling rates and aliasing  explains why you get the mirroring effect in the DFT outputs: Once you get to half the sampling rate, your samples are too far apart (in time) to capture the actual frequency of the sinusoid, as we can't capture 2 points per cycle. Sinusoids of those higher frequencies become indistinguishable from their lower (mirror) counterpart.  So in analyzing what frequency components are in an input signal we only consider the first $N/2$ DFT outputs (corresponding to 0 to $f_s/2$ Hz, i.e. the Nyquist Frequency)

So, the important thing to remember is that the DFT outputs depend on:

* The **number of samples** in the input sequence, $N$
* The **sampling rate**, $f_s$ samples/second

### Exercise
Assume we have a sampling rate of $f_s = 32$ samples/second, and an input length of $N=16$.

* What's the frequency associated with DFT[1]?
* What's the frequency associated with DFT[5]?

### Notes

```python

```

## Leakage

One of the main things to remember about the DFT is that you're calculating the correlation between the input and phasors with specific frequencies.  If your input exactly matches one of those phasor frequencies the magnitude response will show a positive magnitude for that phasor and zero for everything else.  However, if the input frequency falls between output frequencies, then you'll see **spectral leakage**.  The DFT outputs close to the input frequency will also get positive magnitudes, with the DFT output closest to the input frequency getting the highest magnitude.

The following code gives an example

```python
## input size
N=64

## sampling rate
f_s = 64

freq1 = 4.5 ## In between DFT output frequencies
freq2 = 20 ## One of the DFT outputs
#freq2 = 6

amplitude1 = 1
amplitude2 = 0.5

x1, time_steps = gen_sinusoid(frequency=freq1, phase=0, amplitude=amplitude1, sample_rate=f_s, seq_length=N, gen_function=np.cos)
x2, time_steps = gen_sinusoid(frequency=freq2, phase=np.pi/2, amplitude=amplitude2, sample_rate=f_s, seq_length=N, gen_function=np.cos)
x_compound = x1  + x2

## Plot the input
fig, timedom = plt.subplots(figsize=(16, 4))
timedom.scatter(time_steps, x_compound, color='magenta')
timedom.plot(time_steps, x_compound, color='magenta')
timedom.set_xlabel("time (s)")
timedom.set_ylabel("Amplitude")
timedom.set_title("Leakage example (time domain)")

```

```python
## Do the DFT on the compound waveform as above:
mags, phases = get_dft_mag_phase(x_compound, seq_len=N)
dft_freqs = get_dft_freqs_all(sample_rate=f_s, seq_len=N)

## Plot the magnitudes
fig, fdom = plt.subplots(figsize=(16, 4))

## Just plot the first N/2 frequencies since we know that they are the mirrored for k>N/2
fdom.scatter(dft_freqs[:round(N/2)], mags[:round(N/2)])
fdom.set_xlabel("Frequency (Hz)")
fdom.set_ylabel("Magnitude")
fdom.set_title("Leakage example: Magnitude response")
#print(mags)

```

### Leakage as the normalized sinc function

Leakage makes the DFT harder to interpret. However, we can derive the shape that leakage will have from the the DFT equation and some algebra about rectangular functions.  It turns out that leakage for a particular frequency follows the normalized **sinc** function:

$$
\begin{align}
X(m) &= \Big|\frac{AN}{2} \cdot \mathop{sinc}(c-m)\Big|\\
&= \Big|\frac{AN}{2} \cdot \frac{\sin(\pi(c-m))}{2\pi(c-m)}\Big|\\
\end{align}
$$

Where $A$ is the peak amplitude of the input, $N$ is the input sequence length, $c$ is the number of cycles completed in the input sequence time.  If $c$ is a whole number we just get the usual DFT response (i.e. a single spike at the corresponding frequency), but if $c$ is not a whole number, we get a spread across output frequency bins.

The sinc function is a bit hard to think about from just the equation, but it's easy to recognize when plotted (as below)

Let's check whether the sinc function matches what we get in the DFT. First we write a function to evaluate the leakage function in between our DFT outputs.

```python
## Calculate the approximated leakage as the sinc function
def calc_leakage(freq,  sample_rate, seqlen, amplitude=1):
    sequence_time = (1/sample_rate)*seqlen

    ## number of cycles in input for component
    c = freq * sequence_time
    print("c=", c)

    ## Interpolate between DFT ouput indices
    ms = np.arange(0, seqlen, 0.1)

    ## Approximated response - we could actually just return a function here, but
    ## let's just keep things concrete for now.
    leakage = np.abs(amplitude * seqlen * 0.5 * np.sinc((c-ms)))

    return leakage, ms * (sample_rate/seqlen)

```

Now let's plot the leakage predicted for our two input components separately (top) and added together (bottom)

```python
## Calculate the leakage function for our know input wave components
leakage1, ms = calc_leakage(freq1, f_s, N, amplitude=amplitude1)
leakage2, ms = calc_leakage(freq2, f_s, N, amplitude=amplitude2)

## Plot the magnitude response and the leakage function for each of our input components
fig, fdom = plt.subplots(figsize=(16, 4))
fdom.set(xlim=(-1, N/2), ylim=(-1, N))
fdom.scatter(dft_freqs, mags)

fdom.plot(ms, leakage1)
fdom.plot(ms, leakage2)
fdom.set_xlabel("Frequency (Hz)")
fdom.set_ylabel("Magnitude")
fdom.set_title("Leakage function for each input component frequency")

## Plot the magnitude response and the sum of the leakage functions
fig, fdom = plt.subplots(figsize=(16, 4))
fdom.set(xlim=(-1, N/2), ylim=(-1, N))
fdom.scatter(dft_freqs, mags)
fdom.plot(ms, leakage1 + leakage2, color='C5')
fdom.set_xlabel("Frequency (Hz)")
fdom.set_ylabel("Magnitude")
fdom.set_title("Sum of leakage functions for input components")

## It fits, though not perfectly!
```

In the top figure, you should see that peaks (**main lobes**) of each leakage function are aligned with our input component frequencies. The peaks are at the same points as the DFT outputs when the sinusoidal component has a frequency matching the DFT output frequency (i.e. 12 Hz).  Otherwise we see the spread of leakage around the input component frequency (i.e. around 4.5 Hz).

You'll also notice that our DFT magnitudes points don't always line up perfectly with our sinc curves.  Part of this is because the leakage function is an _approximation_.  Nevertheless, it's a very good approximation!

### Exercise

* What happens if the frequencies of the two components making the compound waveform are very close together?
    * e.g. make `f_in2=6`
* What if one of the components has a relatively small amplitude?
    * e.g. change `amplitude` of the second input to 0.5

### Notes

```python

```

### Shaping the lobes
The leakage sinc function has a peak around a specific frequency. If we want our DFT to be better able to distinguish between close frequencies, we need that peak, the **main lobe** to be narrower.  We also want the other peaks, the **side lobes** to be flatter.  We can achieve this using different **windowing methods** on our input.  This is why you see 'Hanning' as the default option for window method in the advanced spectrogram settings in praat.

We'll see some more examples of this when we look at different types of filters later.  But for now the main thing to observe is that leakage might give you the impression that specific frequency components are present in your waveform, when what's actually happening is that your waveform has frequencies that don't match the DFT phasors.  Another thing that can happen is that the peak for a low amplitude component gets subsumed into the main lobe of a nearby frequency component.  This might make you miss frequency components in your input!

```python

```

## Extra:

Here's the composition and decomposition for the compound waveform one you saw in Notebook 1!

```python
N=1028
f_s = 1028
f_in1 = 8 ## In between DFT output frequencies
f_in2 = 20  ## One of the DFT outputs
f_in3 = 36  ## One of the DFT outputs

x1, time_steps = gen_sinusoid(frequency=f_in1, phase=0, amplitude=0.5, sample_rate=f_s, seq_length=N, gen_function=np.cos)
x2, time_steps = gen_sinusoid(frequency=f_in2, phase=np.pi/2, amplitude=1, sample_rate=f_s, seq_length=N, gen_function=np.cos)
x3, time_steps = gen_sinusoid(frequency=f_in3, phase=0, amplitude=0.3, sample_rate=f_s, seq_length=N, gen_function=np.cos)

x_compound = x1  + x2 + x3

## Plot the input
fig, timedom = plt.subplots(figsize=(16, 4))
#timedom.scatter(time_steps, x_compound, color='magenta')
#timedom.set(xlim=(0, 0.2))
timedom.plot(time_steps, x_compound, color='magenta')
```

```python
## Plot the input

fig = plt.figure(figsize=(15,15))
gs = fig.add_gridspec(3,2)

ymax=2
timedom = fig.add_subplot(gs[0, 0])
timedom.set(xlim=(-0.1, 1), ylim=(-ymax,ymax))
timedom.plot(time_steps, x_compound, color='magenta')
timedom.set_title("waveform made from adding 3 sine waves")

s1 = fig.add_subplot(gs[0, 1])
s1.set(xlim=(-0.1, 1), ylim=(-ymax,ymax))
s1.plot(time_steps, x1)
s1.set_title("component 1: 8 Hz")

s2 = fig.add_subplot(gs[1, 1])
s2.set(xlim=(-0.1, 1), ylim=(-ymax,ymax))
s2.plot(time_steps, x2)
s2.set_title("component 2: 20 Hz")

s3 = fig.add_subplot(gs[2, 1])
s3.set(xlim=(-0.1, 1), ylim=(-ymax,ymax))
s3.plot(time_steps, x3)
s3.set_title("component 3: 36 Hz")

#fig.savefig("../fig/compound_waveform.png")
```

```python
## Do the DFT on the compound waveform as above:
mags, phases = get_dft_mag_phase(x_compound, seq_len=N)
dft_freqs = get_dft_freqs_all(sample_rate=f_s, seq_len=N)

## Plot the magnitudes
fig,fdom = plt.subplots(figsize=(16, 4))
fdom.set(xlim=(-1, N/2))
## Just plot the first N/2 frequencies since we know that they are the mirrored for k>N/2
fdom.scatter(dft_freqs[:round(N/2)], mags[:round(N/2)])
fdom.set_xlabel("Frequency (Hz)")
fdom.set_ylabel("Magnitude")
fdom.set_title("Magnitude Response")
#print(mags)

```

```python

```

```python

```

```python

```

```python

```


# FFM234, Klassisk fysik och vektorfält - Veckans tal

**Tobias Wenger och Christian Forssén, Chalmers**

Date: **Aug 10, 2019**

## Uppgift 6.6.6

Beräkna normalytintegralen av

$$
\begin{equation}
\vec{F}= F_0 \frac{a^2}{(x^2+y^2+z^2)^{3/2}} \left[ x \hat{x}  + y \hat{y}
+ \left( z + \frac{z}{a} \frac{(x^2+y^2+z^2)^{3/2}}{a^2} \right) \hat{z} \right ]
\label{_auto1} \tag{1}
\end{equation}
$$

över ytan $S_1:\ x^2+y^2=(z-3a)^2$, $0\leq z\leq 3a$. $F_0$ och $a$ är konstanter.

**Hint.**
* Ytan motsvarar en kon utan bottenplatta.

* Skriv om fältet i sfäriska koordinater.

* Den första termen av $\vec{F}$ motsvarar en punktkälla belägen i origo.

  * Ytintegralen för denna term kan räknas ut genom att beräkna vilken rymdvinkel som konen upptar sedd från punktkällan.

  * Alternativt kan man sluta ytan genom en halvsfär som inkluderar punktkällan inne i volymen.

  * ... eller en yta som gör att punktkällan hamnar utanför.

* Den andra termen av $\vec{F}$ termen är en rymdkälla med en källtäthet, $\nabla \cdot \vec{F}_2$ som är enkel att räkna ut.

  * Här blir det enkelt att använda Gauss sats. Men kom ihåg att sluta ytan på lämpligt sätt.

**Answer.**
$\int_{S_1} \vec{F} \cdot \mbox{d}\vec{S} = 11 \pi F_0 a^2$

**Solution.**
Det vi ska räkna ut är normalytintegralen över $S_1$, dvs

$$
\begin{equation}
\int_{S_1}\ d\vec{S}\cdot \vec{F}
\label{_auto2} \tag{2}
\end{equation}
$$

Detta kan så klart lösas genom att hitta en normalvektor till ytan, sedan hitta en parametrisering till konen och utföra den resulterande integralen. Man ska dock ha i ryggraden att man kan använda Gauss sats istället, det finns ingen garanti att det blir enklare men det är ofta värt att tänka på det.

Vi använder oss därför av den generella lösningsstrategin för Gauss sats:

1. Bestäm utseendet på ytan $S$ och rita en tydlig figur.

2. Undersök fältet $\vec{F}$. Singulariteter? Beräkna $\nabla \cdot \vec{F}$.

3. Slut ytan $S$. Undvik singulariteter inuti den inneslutna volymen.

4. Teckna Gauss sats och beräkna integralen.

5. Har normalen rätt riktning?

Vi känner igen ytan $S_1$ som en kon. Här kan man tänka på konen som sammansatt av många cirklar vars radie bestäms av var i $z-$led man befinner sig. I $z=3a$ ser man att radien på cirkeln blir noll, detta är således spetsen av konen. I $z=0$ ser vi att cirkeln har radien $3a$. Vi börjar med att rita upp ytan $S_1$, den visas i figur [fig:cone](#fig:cone).

Ytan $S_1$ uppritad.

* Undersök fältet $\vec{F}$. Singulariteter? Beräkna $\nabla \cdot \vec{F}$.

Ska man använda Gauss sats så är det viktigt att man behandlar singulära delar av fältet korrekt, dvs identifiera *källtermer* som är inneslutna innanför ytan. När man separerat de singulära delarna av fältet så kan man sen beräkna divergensen och utföra volymsintegralen. Vi skriver fältet $\vec{F}$ som två olika fält

$$
\begin{equation}
\vec{F}=\vec{F_1}+\vec{F_2}
\label{_auto3} \tag{3}
\end{equation}
$$

där

$$
\begin{equation}
\vec{F_1}= F_0 \frac{a^2}{(x^2+y^2+z^2)^{3/2}} \left( x,y,z \right ) = F_0 a^2 \frac{\hat{r}}{r^2}
\label{_auto4} \tag{4}
\end{equation}
$$

$$
\begin{equation}
\vec{F_2}= F_0 \frac{z}{a}\hat{z}
\label{_auto5} \tag{5}
\end{equation}
$$

För att skriva om fältet $\vec{F}_1$ till sfäriska kordinater har vi använt oss av att

$$
\begin{equation}
x\hat{x}+y\hat{y}+z\hat{z} = r\hat{r}
\label{_auto6} \tag{6}
\end{equation}
$$

$$
\begin{equation}
\sqrt{x^2+y^2+z^2}=r
\label{_auto7} \tag{7}
\end{equation}
$$

Vi känner nu igen fältet $\vec{F}_1$ som fältet från en punktkälla i origo. Styrkan hos punktkällan är $4\pi F_0 a^2$.

Fältet $\vec{F}_2$ motsvarar en rymdkälla med konstant källtäthet

$$
\begin{equation}
\nabla \cdot \vec{F_2} = F_0 \partial_z \frac{z}{a} = \frac{F_0}{a}.
\label{_auto8} \tag{8}
\end{equation}
$$

* Slut ytan $S$. Undvik singulariteter inuti den inneslutna volymen.

Gauss sats lyder

$$
\begin{equation}
\int_{S}\ \vec{F} \cdot d\vec{S} = \int_{V}\ \nabla \cdot \vec{F} dV
\label{_auto9} \tag{9}
\end{equation}
$$

och det är viktigt att tänka på att $S=\partial V$, dvs ytan $S$ är randen till volymen $V$. Det betyder att $S$ måste vara en sluten yta (hur kan den annars vara randen till en volym?). Det betyder att för att kunna använda Gauss sats för denna uppgiften måste vi sluta konen. Det mest uppenbara sättet att sluta ytan är att lägga till en bottenplatta till konen. \footnote{Vi kan såklart sluta yan med något annat än en bottenplatta om vi vill. Tex kunde vi ta en halsvsfär och Gauss sats fungerar. Men för att det ska vara lönt att använda Gauss sats är det viktigt att den ytan vi lägger till är enklare att integrera över än originalytan.}

Bottenplattan motsvarar alltså cirkeln som konen ligger på i $x,y$ planet. Vi kallar denna ytan $S_2$ och skriver den som $S_2:\ x^2+y^2\leq (3a)^2$.

Vi har dock ett litet problem nu: Punktkällan ligger mitt i bottenplattan, dvs på ytan. Hur ska man tänka nu? Vi kommer att illustrera *tre* olika sätt att hantera denna punktkälla.
1. Slut istället ytan på ett annat sätt så att punktkällan hamnar helt utanför.

2. Slut istället ytan på ett annat sätt så att punktkällan hamnar helt innanför.

3. Beräkna bidraget från punktkällan genom att räkna ut rymdvinkeln som konen upptar.

Alla tre metoderna kommer givetvis att ge samma svar.

För att följa alt. 1 inför vi en halvsfär upp i det övre halvplanet med centrum i origo. Halvsfärens radie kallar vi $\epsilon$ och låter denna vara liten så att halvsfärens yta inte skär konens mantelyta. Denna nya yta kallar vi $S^+_\epsilon$. Tillsammans med $S_1$ och den modifierade bottenplattan $S_{2-\epsilon}$ bildas en sluten yta som *inte* kommer att innesluta punktkällan i volymen.

För att följa alt. 2 så sluter vi t.ex. ytan med en halvsfär ned i det nedre halvplanet med centrum i origo. Här kan vi låta radien vara lika med $3a$ så att vi direkt får en sluten yta av $S_1 + S^-_{3a}$ som omsluter punktkällan, där $S^-_{3a}$ är den införda halvsfärens yta.

För att följa alt. 3 handlar det bara om att beräkna hur stor rymdvinkel som konen upptar sedd från punktkällan. Se mer nedan.

* Teckna Gauss sats och beräkna integralen.

Vi börjar med det oproblematiska bidraget från $\vec{F}_2$ med den sökta ytintegralen

$$
\begin{equation}
I_2 \equiv \int_{S_1} \vec{F}_2 \cdot d\vec{S}
\label{_auto10} \tag{10}
\end{equation}
$$

Vi använder den slutna ytan $S_1+S_2$ som vi kan använda i Gauss sats

$$
\begin{equation}
\int_{S_1+S_2} \vec{F}_2 \cdot d\vec{S} = \int_V \nabla \cdot \vec{F}_2 dV
\label{_auto11} \tag{11}
\end{equation}
$$

Vi bollar över integralen över $S_2$ till andra sidan av likheten och får

$$
\begin{equation}
\label{eq:GaussTot} \tag{12}
\int_{S_1} \vec{F}_2 \cdot d\vec{S} = \int_V \nabla \cdot \vec{F}_2 dV - \int_{S_2} \vec{F}_2 \cdot d\vec{S}
\end{equation}
$$

där den ytintgral som uppgiften ber oss beräkna nu står i vänsterledet. Vi ser att det var viktigt att sluta ytan då vi nu har en extra term att dra bort från värdet av volymsintegralen. Nyttan med Gauss sats är som störst när divergensen är simpel och normalytintegralen över bottenplattan (eller vilken annan del man slutit sin yta med) är simpel.

Divergensen för $\vec{F}_2$ räknade vi ut ovan och vi får

$$
\begin{equation}
\int_V \nabla \cdot \vec{F_2} dV = \frac{F_0}{a} \int_V dV = F_0 \pi (3a)^2
\label{_auto12} \tag{13}
\end{equation}
$$

där vi i det sista steget använt att volymen av en kon är $V=\pi r^2 h/3$ där $r$ är basradien och $h$ är höjden.

Vi behöver nu ytintegralen över bottenlattan kvar enligt ekv. ([eq:GaussTot](#eq:GaussTot)). Det är viktigt att notera att bottenplattans normalvektor pekar ut från den inneslutna volymen, dvs nedåt. Vi ska alltså beräkna

$$
\begin{equation}
\int_{S_2} \vec{F_2} \cdot d\vec{S} = -\int_{S_2} \vec{F_2} \cdot \hat{z} dS= -\int_{S_2} F_0\frac{z}{a}\hat{z} \cdot \hat{z} dS
\label{_auto13} \tag{14}
\end{equation}
$$

eftersom $z=0$ på $S_2$ så integralen över $\vec{F}_2$ på bottenplattan är noll. Vi har alltså att

$$
\begin{equation}
I_2 = F_0 \pi (3a)^2
\label{_auto14} \tag{15}
\end{equation}
$$

Återstår integralen över $\vec{F}_1$. Enligt alt. 1 blir

$$
\begin{equation}
\label{eq:GaussTot2} \tag{16}
I_1 = \int_{S_1} \vec{F}_1 \cdot d\vec{S} = \int_{V_1} \nabla \cdot \vec{F}_1 dV - \int_{S_{2-\epsilon}} \vec{F}_1 \cdot d\vec{S} - \int_{S^+_{\epsilon}} \vec{F}_1 \cdot d\vec{S}.
\end{equation}
$$

Volymsintegralen görs här över en volym som inte inkluderar punktkällan. Den blir därför noll (divergensen $\nabla \cdot \vec{F}_1=0$ i hela denna volym). Ytintegralen över bottenplattan $S_{2-\epsilon}$ blir också noll eftersom fältets riktning $\hat{r}$ är vinkelrät mot ytans normalriktning. På denna yta ligger ju $\hat{r}$ alltid i $xy$ planet. Slutligen har vi integralen över halvsfären. Normalen måste här vara riktad i negativ $\hat{r}$-led för att peka ut från volymen

$$
\begin{equation}
\int_{S^+_{\epsilon}} \vec{F_1} \cdot d\vec{S} = \int_{S^+_{\epsilon}} F_0 a^2 \frac{\hat{r}}{\epsilon^2} \cdot (-\hat{r}) \epsilon^2 \sin\theta \mbox{d}\theta \mbox{d}\phi= -F_0 a^2 2\pi
\label{_auto15} \tag{17}
\end{equation}
$$

Slutligen får vi alltså $I_1 = 0 - 0 - (-F_0 a^2 2\pi ) = F_0 a^2 2\pi$.

Med alt. 2 har vi att

$$
\begin{equation}
\label{eq:GaussTot3} \tag{18}
I_1 = \int_{S_1} \vec{F}_1 \cdot d\vec{S} = \int_{V_2} \nabla \cdot \vec{F}_1 dV - \int_{S^-_{3a}} \vec{F}_1 \cdot d\vec{S}.
\end{equation}
$$

Nu kommer volymen $V_2$ att innesluta punktkällan och bidraget från denna term blir lika med punktkällans styrka: $F_0 a^2 4\pi$. Ytintegralen över $S^-_{3a}$ göra på precis samma sätt som ovan, men vi skall nu notera att normalriktningen pekar i positiv $\hat{r}$-led för att peka ut från volymen. Bidraget blir $F_0 a^2 2\pi$. Slutligen ger alltså denna metod att $I_1 = F_0 a^2 4\pi - F_0 a^2 2\pi = F_0 a^2 2\pi$, precis som alt. 1 ovan.

Enligt alt. 3 hade vi direkt kunnat räkna ut ytintegralen över konens mantelyta med Cederwall ekv. 6.7. Sett från punktkällan i origo upptar konens mantelyta en rymdvinkel $2\pi$. Det totala bidraget blir därför $(2\pi)/(4\pi) = 1/2$ av punktkällans styrka. Dvs vi får $I_1 = (F_0 a^2 4\pi)/2 = F_0 a^2 2\pi$. Givetvis samma resultat som med alt. 1 och 2.

Nu kombinerar vi ihop våra delresultat

$$
\begin{equation}
\int_{S_1} \vec{F} \cdot d\vec{S} = I_1 + I_2 = F_0 a^2 2\pi + F_0 \pi (3a)^2 = F_0 \pi 11 a^2
\label{_auto16} \tag{19}
\end{equation}
$$

* Har normalen rätt riktning?

Kontrollera gärna en gång till att ytnormalerna i de ytintegraler som ger nollskilda bidrag har rätt bidrag. Detta gäller i synnerhet de två halvsfärer som vi införde för att räkna ut $I_1$ enligt alt. 1 och alt. 2.

Svaret på uppgiften är alltså $11\pi F_0 a^2$.

Kontrollera också dimensionen på svaret. Vårt fält har dimensionen $[F_0]$ och ytintegralen skall därför ha dimensionen $[F_0]$*area, vilket verkar stämma.

### Remarks

Uppgiften illustrerar olika sätt att hantera en singularitet som hamnar på randen till en volym.

Det är värt att notera att den här uppgiften innehåller ett icke-standard moment som utgörs av att punktkällan ligger på den bottenyta som är naturlig att sluta ytan med.

Notera att vi i slutändan fick precis halva styrkan
av punktkällan som bidrag oavsett vilken metod vi använde för att hantera den. Det betyder att det kanske enklaste sättet att se detta på är att punktkällan strålar
halva sitt bidrag in mot konen $S_1$ i detta fallet, dvs man tänker som i
metod 3.


# Jupyter notebook test
This is a test for the integration of a jupyter notebook in github pages

\begin{equation}
 (k,u) \longmapsto J(k,u)
\end{equation}

```python
import numpy as np
from matplotlib import pyplot as plt
import scipy
import matplotlib as mpl
from matplotlib import cm
plt.style.use('seaborn')
mpl.rcParams['image.cmap'] = u'viridis'
plt.rc('font', **{'family': 'serif', 'serif': ['Computer Modern']})
```

We are first going to define the function that we are going to torture

```python
def branin_2d(X, switch=False):
        """ Scaled branin function:
        global minimizers are
        [0.124, 0.818], [0.54277, 0.1513], [0.96133, 0.16466]
        """
        X = np.atleast_2d(X)
        y, x = X[:, 1], X[:, 0]
        if switch:
            x, y = y, x
        x2 = 15 * y
        x1 = 15 * x - 5
        quad = (x2 - (5.1 / (4 * np.pi**2)) * x1**2 + (5 / np.pi) * x1 - 6)**2
        cosi = (10 - (10 / np.pi * 8)) * np.cos(x1) - 44.81
        return (quad + cosi) / (51.95) + 2.0

ngrid = 200
X_, Y_ = np.linspace(0, 1, ngrid), np.linspace(0, 1, 2 * ngrid)
xx, yy = np.meshgrid(X_, Y_, indexing = 'ij')
pair_of_points = np.array([xx, yy]).T.reshape(-1, 2, order = 'F')

```

```python
p = .99
out_t = branin_2d(pair_of_points).reshape(ngrid, ngrid * 2)
kstar_t = out_t.argmin(0)
Jstar_t = out_t.min(0)
rho_t = (out_t / Jstar_t[np.newaxis, :])
alpha_p_t = np.quantile(rho_t, p, axis=1)
```

```python
plt.figure(figsize=(8,10))
plt.subplot(2, 1, 1)
plt.contourf(xx, yy, out_t)
plt.plot(X_[kstar_t], Y_, 'o', color='white')
plt.xlabel(r'$k$')
plt.ylabel(r'$u$')
plt.subplot(2, 1, 2)
plt.plot(X_, alpha_p_t)
# plt.plot(Y_, Jstar_t)     #
plt.xlim((0, 1))
plt.title(r'$\alpha_p$ estimated: {}'.format(alpha_p_t.min()))
plt.tight_layout()
plt.show()
```

```python

```

```python

```


```python
# Método para resolver las energías y eigenfunciones de un sistema cuántico numéricamente por Teoría de Pertubaciones
# Modelado Molecular 2
# By: José Manuel Casillas Martín     22-oct-2017
import numpy as np
from sympy import *
from sympy.physics.qho_1d import E_n, psi_n
from sympy.physics.hydrogen import E_nl, R_nl
from sympy import init_printing; init_printing(use_latex = 'mathjax')
from scipy import integrate
from scipy.constants import hbar, m_e, m_p, e
from mpmath import spherharm
from numpy import inf, array
import numpy as np
import matplotlib.pyplot as plt
import traitlets
from IPython.display import display
from ipywidgets import Layout, Box, Text, Dropdown, Label, IntRangeSlider, IntSlider, RadioButtons
```

Teoría de Perturbaciones

Consiste en resolver un sistema perturbado(se conoce la solución al no perturbado), y donde el interés es conocer la contribución de la parte perturbada $H'$ al nuevo sistema total.

$$ H = H^{0}+H'$$

La resolución adecuada del problema, depende en gran parte, de una correcta elección de $H'$.

```python
form_item_layout = Layout(display='flex',flex_flow='row',justify_content='space-between')

PType=Dropdown(options=['Particle in a one-dimensional box', 'Harmonic oscilator', 'Hydrogen atom (Helium correction)'])
Pert=Text()
Rang=IntRangeSlider(min=0, max=20, step=1, disabled=False, continuous_update=False, orientation='horizontal',\
                    readout=True, readout_format='d')
M=Text()
Correc=Dropdown(options=['1', '2'])
hbarra=Dropdown(options=[1, 1.0545718e-34])

form_items = [
    Box([Label(value='Problem'),PType], layout=form_item_layout),
    Box([Label(value='Perturbation'),Pert], layout=form_item_layout),
    Box([Label(value='Correction order'),Correc], layout=form_item_layout),
    Box([Label(value='n Range'),Rang], layout=form_item_layout),
    Box([Label(value='Mass'),M], layout=form_item_layout),
    Box([Label(value='Hbar'),hbarra], layout=form_item_layout),]

form = Box(form_items, layout=Layout(display='flex',flex_flow='column',border='solid 2px',align_items='stretch',width='40%'))
form
```

En esta caja interactiva llena los datos del problema que deseas resolver.

# Nota 1:
Es recomendable usar unidades atómicas de Hartree para eficientar los cálculos. 1 u.a. (energía)= 27.211eV.

# Nota 2:
Para la partícula en una caja unidimensional es recomendable que n sea mayor a 1.

## Nota 3:
Para la correción a la energía del átomo de Helio sólo es necesario seleccionar el problema, automáticamente se calcula la correción a primer orden y no se corrigen las funciones de onda.

```python
Problem=PType.value
form_item_layout = Layout(display='flex',flex_flow='row',justify_content='space-between')

L=Text()
W=Text()
atomic_number=RadioButtons(options=['1 (Show Hydrogen energies)','2 (Correct Helium first energy)'],disabled=False)

if Problem=='Particle in a one-dimensional box':
    form_items = [Box([Label(value='Large of box'),L], layout=form_item_layout)]

if Problem=='Harmonic oscilator':
    form_items = [Box([Label(value='Angular Fr'),W], layout=form_item_layout)]

if Problem=='Hydrogen atom (Helium correction)':
    form_items = [Box([Label(value='Atomic number'),atomic_number], layout=form_item_layout)]

form = Box(form_items, layout=Layout(display='flex',flex_flow='column',border='solid 2px',align_items='stretch',width='40%'))
form
```

```python
# Variables que se utilizarán
# x=variable de integracion, l=largo del pozo, m=masa del electrón, w=frecuencia angular
# n=número cuántico principal, Z=Número atómico, q=número cuántico angular(l)
var('x theta phi')
var('r1 r2', real=True)
var('l m hbar w n Z', positive=True, real=True)

# Perturbación
if Pert.value!='':
    H_p=sympify(Pert.value)
    h_p=eval(Pert.value)
else:
    H_p=0
    h_p=0

# Constantes
h=hbarra.value
a0=5.2917721067e-11
if M.value!='':
    mass=float(eval(M.value))
else:
    mass=1

# Energías y funciones que se desea corregir
n_inf=min(Rang.value)
n_sup=max(Rang.value)

if Problem=='Particle in a one-dimensional box':
    if L.value=='':
        large=1
    else:
        large=float(eval(L.value))
    omega=0
    # Energías del pozo de potencial infinito
    k=n*pi/l
    En=hbar**2*k**2/(2*m)

    # Funciones de onda del pozo de potencial infinito
    Psin=sqrt(2/l)*sin(n*pi*x/l)

    # Límites del pozo definido de 0 a l para sympy
    li_sympy=0
    ls_sympy=l
    # Mismo limites para scipy
    li_scipy=0
    ls_scipy=large

if Problem=='Harmonic oscilator':
    large=0
    if W.value=='':
        omega=1
    else:
        omega=float(eval(W.value))
    # Energías del oscilador armónico cuántico
    En=E_n(n,w)

    # Funciones de onda del oscilador armónico cuántico
    Psin=psi_n(n,x,m,w)

     # Límites del pozo definido de -oo a oo para sympy
    li_sympy=-oo
    ls_sympy=oo
    # Límites del pozo definido de -oo a oo para scipy
    li_scipy=-inf
    ls_scipy=inf

if Problem=='Hydrogen atom (Helium correction)':
    if atomic_number.value=='1 (Show Hydrogen energies)':
        z=1
    if atomic_number.value=='2 (Correct Helium first energy)':
        z=2
    large=0
    omega=0

    # Energías del átomo hidrogenoide
    En=z*E_nl(n,z)

    # Funciones de onda del átomo de hidrógeno
    # Número cuántico l=0
    q=0 # La variable l ya esta siendo utilizada para el largo de la caja por ello se sustituyo por q
    Psin=(R_nl(n,q,r1,z)*R_nl(n,q,r2,z))

     # Límites del átomo de hidrógeno de 0 a oo para sympy
    li_sympy=0
    ls_sympy=oo
    # Límites del átomo de hidrógeno de 0 a oo para scipy
    li_scipy=0
    ls_scipy=inf
```

Para sistemas no degenerados, la corrección a la energía a primer orden se calcula como

$$ E_{n}^{(1)} = \int\psi_{n}^{(0)*} H' \psi_{n}^{(0)}d\tau$$

** Tarea 1 : Programar esta ecuación si conoces $H^{0}$ y sus soluciones. **

```python
def correcion_1st_order_Energy(E_n,Psi_n,H_p,li,ls):
    E1_n=Integral(Psi_n*(H_p)*Psi_n,(x,li,ls)).doit()
    return(E_n+E1_n)
```

```python
# Correción de la energía a primer orden
E=[]
Eev=[]
Ec1=[]
if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
    for i in range(n_inf,n_sup+1):
        E.append(En.subs({n:i}))
        Eev.append(E[i-n_inf].subs({m:mass, l:large, hbar:h}).evalf())
        Ec1.append(correcion_1st_order_Energy(En.subs({n:i}),Psin.subs({n:i}),H_p,li_sympy,ls_sympy))
if Problem=='Hydrogen atom (Helium correction)':
    for i in range(n_inf,n_sup+1):
        E.append(En.subs({n:i}))
        Eev.append(E[i-n_inf])
    if z==2:
        integral_1=Integral(Integral((16*z**6*r1*r2**2*exp(-2*z*(r1+r2))),(r2,0,r1)),(r1,0,oo)).doit()
        integral_2=Integral(Integral((16*z**6*r1**2*r2*exp(-2*z*(r1+r2))),(r2,r1,oo)),(r1,0,oo)).doit()
        integral_total=(integral_1+integral_2)
        Ec1.append(E[0]+integral_total)
```

Y la corrección a la función de onda, también a primer orden, se obtiene como:

 $$ \psi_{n}^{(1)} = \sum_{m\neq n} \frac{\langle\psi_{m}^{(0)} | H' | \psi_{n}^{(0)} \rangle}{E_{n}^{(0)} - E_{m}^{(0)}} \psi_{m}^{(0)}$$

**Tarea 2: Programar esta ecuación si conoces $H^{0}$ y sus soluciones. **

```python
# Correción de las funciones a primer orden
if Pert.value!='':
    if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
        Psi_c=[]
        integrals=np.zeros((n_sup+1,n_sup+1))
        for i in range(n_inf,n_sup+1):
            a=0
            for j in range(n_inf,n_sup+1):
                if i!=j:
                    integ= lambda x: eval(str(Psin.subs({n:j})*(h_p)*Psin.subs({n:i}))).subs({m:mass,l:large,w:omega,hbar:h})
                    integrals[i,j]=integrate.quad(integ,li_scipy,ls_scipy)[0]
                    cte=integrals[i,j]/(En.subs({n:i,m:mass,l:large})-En.subs({n:j,m:mass,l:large})).evalf()
                    a=a+cte*Psin.subs({n:j})
            Psi_c.append(Psin.subs({n:i})+a)
```

**Tarea 3: Investigue las soluciones a segundo orden y también programe las soluciones. **

Y la corrección a la energía a segundo orden, se obtiene como:

 $$  E_{n}^{(2)} = \sum_{m\neq n} \frac{|\langle\psi_{m}^{(0)} | H' | \psi_{n}^{(0)} \rangle|^{2}}{E_{n}^{(0)} - E_{m}^{(0)}} $$

```python
# Correción a la energía a segundo orden
if Pert.value!='':
    if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
        if Correc.value=='2':
            Ec2=[]
            for i in range(n_inf,n_sup+1):
                a=0
                for j in range(n_inf,n_sup+1):
                    if i!=j:
                        cte=((integrals[i,j])**2)/(En.subs({n:i,m:mass,l:large,hbar:h})-En.subs({n:j,m:mass,l:large,hbar:h})).evalf()
                        a=a+cte
                Ec2.append(Ec1[i-n_inf]+a)
```

**A continuación se muestran algunos de los resultados al problema resuelto**

Las energías sin perturbación son:

```python
E
```

$$\left [ \frac{\hbar w}{2}, \quad \frac{3 w}{2} \hbar, \quad \frac{5 w}{2} \hbar, \quad \frac{7 w}{2} \hbar\right ]$$

La correción a primer orden de las energías son:

```python
Ec1
```

$$\left [ \frac{\hbar w}{2} + \frac{\hbar}{2 m w}, \quad \frac{3 w}{2} \hbar + \frac{3 \hbar}{2 m w}, \quad \frac{5 w}{2} \hbar + \frac{5 \hbar}{2 m w}, \quad \frac{7 w}{2} \hbar + \frac{7 \hbar}{2 m w}\right ]$$

Si seleccionaste en los parámetros iniciales una correción a segundo orden entonces...

Las correciones a la energía a segundo orden son:

```python
Ec2
```

$$\left [ \frac{\hbar w}{2} - \frac{1}{w} 7.11189250474996 \cdot 10^{33} + \frac{\hbar}{2 m w}, \quad \frac{3 w}{2} \hbar - \frac{1}{w} 1.18531541745832 \cdot 10^{34} + \frac{3 \hbar}{2 m w}, \quad \frac{5 w}{2} \hbar - \frac{1}{w} 2.37063083491664 \cdot 10^{33} + \frac{5 \hbar}{2 m w}, \quad \frac{7 w}{2} \hbar + \frac{1}{w} 2.13356775142498 \cdot 10^{34} + \frac{7 \hbar}{2 m w}\right ]$$

Ahora vamos con la función de onda $(\psi)$

```python
form_item_layout = Layout(
    display='flex',
    flex_flow='row',
    justify_content='space-between')

Graph=IntSlider(min=n_inf, max=n_sup, step=1, disabled=False, continuous_update=False, orientation='horizontal',\
                readout=True, readout_format='d')

form_items = [
    Box([Label(value='What function do you want to see?'),
         Graph], layout=form_item_layout)]

form = Box(form_items, layout=Layout(
    display='flex',
    flex_flow='column',
    border='solid 2px',
    align_items='stretch',
    width='40%'))
form
```

La función de onda original es:

```python
Psin.subs({n:Graph.value})
```

$$\frac{\sqrt{2} m^{\frac{3}{4}} w^{\frac{3}{4}} x}{\hbar^{\frac{3}{4}} \sqrt[4]{\pi}} e^{- \frac{m w x^{2}}{2 \hbar}}$$

La correción a primer orden a la función de onda (utilizando todas las funciones en el rango seleccionado) es:

```python
if Problem=='Particle in a one-dimensional box' or Problem=='Harmonic oscilator':
    if Pert.value!='':
            Psi_c[Graph.value-n_inf]
    if Pert.value=='':
        print('No se ingreso ninguna perturbación')
if Problem=='Hydrogen atom (Helium correction)':
    print('Este programa no corrige las fucniones de un átomo hidrogenoide')
```

Vamos a graficarlas para verlas mejor...

La función de onda original es:

```python
if Problem=='Particle in a one-dimensional box':
    plot(eval(str(Psin)).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),xlim=(li_scipy,ls_scipy),\
         title='$\psi_{%d}$'%Graph.value)
if Problem=='Harmonic oscilator':
    plot(eval(str(Psin)).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),xlim=(-10*h/(mass*omega),10*h/(mass*omega)),\
         title='$\psi_{%d}$'%Graph.value)
if Problem=='Hydrogen atom (Helium correction)':
    print('Densidad de probabilidad para un electrón')
    plot(eval(str((4*pi*x**2*R_nl(Graph.value,q,x,z)**2))),xlim=(0,10),ylim=(0,20/Graph.value), title='$\psi_{%ds}$'%Graph.value)
    print('Tome en cuenta que debido a la dificultad para seleccionar los límites de la gráfica se muestran bien los primeros\n\
3 estados. A partir de ahí visualizar la gráfica se complica.')
```

La corrección a la función de onda es:

```python
if Problem=='Particle in a one-dimensional box':
    if Pert.value!='':
        plot(eval(str(Psi_c[Graph.value-n_inf])).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),\
             xlim=(li_scipy,ls_scipy),title='$\psi_{%d}$'%Graph.value)
    if Pert.value=='':
        print('No se ingreso ninguna perturbación')
if Problem=='Harmonic oscilator':
    if Pert.value!='':
        plot(eval(str(Psi_c[Graph.value-n_inf])).subs({n:Graph.value,m:mass,l:large,w:omega,hbar:h}),\
             xlim=(-10*h/(mass*omega),10*h/(mass*omega)),title='$\psi_{%d}$' %Graph.value)
    if Pert.value=='':
        print('No se ingreso ninguna perturbación')
if Problem=='Hydrogen atom (Helium correction)':
    print('Este programa no corrige las fucniones de un átomo hidrogenoide')
```

**Tarea 4.  Resolver el átomo de helio aplicando los programas anteriores.**

Para resolver el átomo de helio se utilizaron los conceptos, que sirvieron como base para las primeras tareas. Sin embargo, en el Apéndice 1 viene con mayor detalles las consideraciones tomadas para resolver el problema.

## Apéndice 1

Para el cálculo a las correciones del átomo de Helio se tomó en cuenta lo siguiente...

La función de onda del átomo de Helio puede ser representada como:

$$ \psi_{nlm} = \psi(r1)_{nlm} \psi(r2)_{nlm}$$

Donde, para el estado fundamental:

$$ \psi(r_{1}.r_{2})_{100} = \frac{Z^{3}}{\pi a_{0}^{3}} e^{\frac{-Z}{a_{0}}(r_{1}+r_{2})}$$

Y la perturbación sería el término de repulsión entre los dos electrones, es decir:

$$ H'= \frac{e^{2}}{r_{12}}=\frac{e^{2}}{|r_{1}-r_{2}|}$$

Finalmente la correción a primer orden de la energía sería:

$$ E^{1}= \langle\psi_{n}^{(0)} | H' | \psi_{n}^{(0)} \rangle =\frac{Z^{6}e^{2}}{\pi^{2} a_{0}^{6}} \int_{0}^{2\pi}\int_{0}^{2\pi}\int_{0}^{\pi}\int_{0}^{\pi}\int_{0}^{\infty}\int_{0}^{\infty} \frac{e^{\frac{-2Z}{a_{0}}(r_{1}+r_{2})}}{r_{12}} r_{1}^{2}r_{2}^{2}sen{\theta_{1}}sen{\theta_{2}} dr_{2} dr_{1} d\theta_{2} d\theta_{1} d\phi_{2} d\phi_{1}$$

Se utiliza una expansión del termino de repulsión con los armónicos esféricos y se integra la parte angular. Una vez hecho eso, la integral queda expresada de la siguiente manera:

$$ E^{1}= \frac{16Z^{6}e^{2}}{a_{0}^{6}} \left[\int_{0}^{\infty} r_{1}^{2} e^{\frac{-2Z}{a_{0}}r_{1}} \left(\int_{0}^{r_{1}} \frac{r_{2}^{2}}{r_{1}} e^{\frac{-2Z}{a_{0}}r_{2}} dr_{2}+\int_{r_{1}}^{\infty}r_{2} e^{\frac{-2Z}{a_{0}}r_{2}}dr_{2}\right) dr_{1} \right]$$

**Tarea 5: Método variacional-perturbativo. **

Este método nos permite estimar de forma precisa $E^{(2)}$ y correcciones perturbativas de la energía de órdenes más elevados para el estado fundamental del sistema, sin evaluar sumas infinitas. Ver ecuación 9.38 del libro.

```python

```

**Tarea 6. Revisar sección 9.7. **

Inicialmente a mano, y sengunda instancia favor de intentar programar sección del problema, i.e. integral de Coulomb  e integral de intercambio.

```python

```


```python
import os, sys
import h5py
import numpy as np
from scipy.io import loadmat
import cv2
import matplotlib
%matplotlib inline
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from numpy import matrix as mat
from sympy import *
from numpy import linalg as la
```

```python
def getFx(para, frame):   # para中为(13*frame)的一套参数，frame传进来是为了确定循环次数
    #先写出参数表达式，ABDCDEF六个点的齐次坐标
    K = Matrix([[1149.67569986785, 0.0, 508.848621645943],
                     [0.0, 1147.59161666764, 508.064917088557],
                     [0.0, 0.0, 1.0]])
    r11, r12, r13, r14, r21, r22, r23, r24, r31, r32, r33 = symbols('r11 r12 r13 r14 r21 r22 r23 r24 r31 r32 r33')
    Rt = Matrix([[r11, r12, r13, r14], [r21, r22, r23, r24], [r31, r32, r33, 1]])
    a, b, c, th, al = symbols('a b c th al')
    ua, va, wa, ub, vb, wb, uc, vc, wc, ud, vd, wd, ue, ve, we, uf, vf, wf = symbols('ua va wa ub vb wb uc vc wc ud vd wd ue ve we uf vf wf')
    f = Symbol('f')
    XA = Matrix([[-a * c * cos(th) * cos(al)], [c-a * c * sin(th)], [-a * c *  cos(th) * sin(al)], [1]])
    XB = Matrix([[0], [c], [0], [1]])
    XC = Matrix([[a * c * cos(th) * cos(al)], [c+a * c * sin(th)], [a * c * cos(th) * sin(al)], [1]])
    XD = Matrix([[-b * c], [0], [0], [1]])
    XE = Matrix([[0], [0], [0], [1]])
    XF = Matrix([[b * c], [0], [0], [1]])
    ua, va, wa = K[0,:] * (Rt * XA), K[1,:] * (Rt * XA), K[2,:] * (Rt * XA)
    ub, vb, wb = K[0,:] * (Rt * XB), K[1,:] * (Rt * XB), K[2,:] * (Rt * XB)
    uc, vc, wc = K[0,:] * (Rt * XC), K[1,:] * (Rt * XC), K[2,:] * (Rt * XC)
    ud, vd, wd = K[0,:] * (Rt * XD), K[1,:] * (Rt * XD), K[2,:] * (Rt * XD)
    ue, ve, we = K[0,:] * (Rt * XE), K[1,:] * (Rt * XE), K[2,:] * (Rt * XE)
    uf, vf, wf = K[0,:] * (Rt * XF), K[1,:] * (Rt * XF), K[2,:] * (Rt * XF)
    #根据每一帧的循环，提取出Rt的参数，K是公用的，代入参数写出3D坐标，并计算出u/w，v/w
    #写成f的形式，即按照六个点分块，每块里面有M帧
    getfx = mat(np.zeros((6*frame*2,1)))
    for i in range(6):
        for j in range(frame):
            if i == 0 :
                f = Matrix([ua/wa, va/wa])
            elif i == 1 :
                f = Matrix([ub/wb, vb/wb])
            elif i == 2 :
                f = Matrix([uc/wc, vc/wc])
            elif i == 3 :
                f = Matrix([ud/wd, vd/wd])
            elif i == 4 :
                f = Matrix([ue/we, ve/we])
            else:
                f = Matrix([uf/wf, vf/wf])
            f_value = f.subs({r11:para[13*j], r12:para[13*j+1], r13:para[13*j+2], r14:para[13*j+3],
                        r21:para[13*j+4], r22:para[13*j+5], r23:para[13*j+6], r24:para[13*j+7],
                        r31:para[13*j+8], r32:para[13*j+9], r33:para[13*j+10], th:para[13*j+11],
                        al:para[13*j+12], a:0.35, b:0.25, c:-0.095})
            getfx[i*frame*2+j*2] = f_value[0]
            getfx[i*frame*2+j*2+1] = f_value[1]
    #返回getfx值，2*frame*6 by 1
    return getfx
```

```python
def getJacobian(point, frame, para):
    # 用参数表示K，R矩阵
    focalx, focaly, px, py = symbols('focalx focaly px py')
    r11, r12, r13, r14, r21, r22, r23, r24, r31, r32, r33 = symbols('r11 r12 r13 r14 r21 r22 r23 r24 r31 r32 r33')
    Rt = Matrix([[r11, r12, r13, r14], [r21, r22, r23, r24], [r31, r32, r33, 1]])
    K = Matrix([[focalx, 0, px], [0, focaly, py], [0, 0, 1]])
    # KRt = K * Rt
    # 用参数表示ABCDEF六个点坐标
    a, b, c, th, al = symbols('a b c th al')
    ua, va, wa, ub, vb, wb, uc, vc, wc, ud, vd, wd, ue, ve, we, uf, vf, wf = symbols('ua va wa ub vb wb uc vc wc ud vd wd ue ve we uf vf wf')
    f = Symbol('f')
    if point == 0 :
        XA = Matrix([[-a * c * cos(th) * cos(al)], [c-a * c * sin(th)], [-a * c *  cos(th) * sin(al)], [1]])
        ua, va, wa = K[0,:] * (Rt * XA), K[1,:] * (Rt * XA), K[2,:] * (Rt * XA)
        f = Matrix([ua/wa, va/wa])
    elif point == 1 :
        XB = Matrix([[0], [c], [0], [1]])
        ub, vb, wb = K[0,:] * (Rt * XB), K[1,:] * (Rt * XB), K[2,:] * (Rt * XB)
        f = Matrix([ub/wb, vb/wb])
    elif point == 2 :
        XC = Matrix([[a * c * cos(th) * cos(al)], [c+a * c * sin(th)], [a * c * cos(th) * sin(al)], [1]])
        uc, vc, wc = K[0,:] * (Rt * XC), K[1,:] * (Rt * XC), K[2,:] * (Rt * XC)
        f = Matrix([uc/wc, vc/wc])
    elif point == 3 :
        XD = Matrix([[-b * c], [0], [0], [1]])
        ud, vd, wd = K[0,:] * (Rt * XD), K[1,:] * (Rt * XD), K[2,:] * (Rt * XD)
        f = Matrix([ud/wd, vd/wd])
    elif point == 4 :
        XE = Matrix([[0], [0], [0], [1]])
        ue, ve, we = K[0,:] * (Rt * XE), K[1,:] * (Rt * XE), K[2,:] * (Rt * XE)
        f = Matrix([ue/we, ve/we])
    elif point == 5:
        XF = Matrix([[b * c], [0], [0], [1]])
        uf, vf, wf = K[0,:] * (Rt * XF), K[1,:] * (Rt * XF), K[2,:] * (Rt * XF)
        f = Matrix([uf/wf, vf/wf])
    args = Matrix([r11, r12, r13, r14, r21, r22, r23, r24, r31, r32, r33, th, al, a, b, c])
    f_X1 = f[0,:].jacobian(args)
    f_X2 = f[1,:].jacobian(args)
    JA = Matrix([f_X1, f_X2])   # 2 by 16 matrix
    JA_value = JA.subs({focalx:1149.676, focaly:1147.592, px:508.849, py:508.065, r11:para[13*frame], r12:para[13*frame+1],
                                 r13:para[13*frame+2], r14:para[13*frame+3], r21:para[13*frame+4], r22:para[13*frame+5],
                                 r23:para[13*frame+6], r24:para[13*frame+7], r31:para[13*frame+8], r32:para[13*frame+9],
                                 r33:para[13*frame+10], th:para[13*frame+11], al:para[13*frame+12], a:0.35, b:0.25, c:-0.095})
    #JA_value = JA_value.subs({f:1149.68})
    return JA_value
```

```python
def getJ(para, frame):
    getj = mat(np.zeros((6*frame*2, 13*frame)))
    for m in range(6):
        for n in range(frame):
            JA_value = getJacobian(m, n, para)
            #print(JA_value)
            getj[2*(m*frame+n):2*(m*frame+n+1), 13*n:13*n+13] = JA_value[:, 0:13]
            #getj[2*(m*frame+n):2*(m*frame+n+1), -3:] = JA_value[:, -3:]
    return getj
```

```python
def getE(getfx, frame, k, xkp):
    E = mat(np.zeros((6*frame*2,1)))
    for i in range(6):
        for j in range(frame):
            if i==0 :
                E[(i*frame+j)*2] = getfx[i*frame*2+j*2] - xkp[i*3+k,0]
                E[(i*frame+j)*2+1] = getfx[i*frame*2+j*2+1] - xkp[i*3+k, 1]
            elif i==1 :
                E[(i*frame+j)*2] = getfx[i*frame*2+j*2] - xkp[i*3+k,0]
                E[(i*frame+j)*2+1] = getfx[i*frame*2+j*2+1] - xkp[i*3+k, 1]
            elif i==2 :
                E[(i*frame+j)*2] = getfx[i*frame*2+j*2] - xkp[i*3+k,0]
                E[(i*frame+j)*2+1] = getfx[i*frame*2+j*2+1] - xkp[i*3+k, 1]
            elif i==3 :
                E[(i*frame+j)*2] = getfx[i*frame*2+j*2] - xkp[i*3+k,0]
                E[(i*frame+j)*2+1] = getfx[i*frame*2+j*2+1] - xkp[i*3+k, 1]
            elif i==4 :
                E[(i*frame+j)*2] = getfx[i*frame*2+j*2] - xkp[i*3+k,0]
                E[(i*frame+j)*2+1] = getfx[i*frame*2+j*2+1] - xkp[i*3+k, 1]
            elif i==5 :
                E[(i*frame+j)*2] = getfx[i*frame*2+j*2] - xkp[i*3+k,0]
                E[(i*frame+j)*2+1] = getfx[i*frame*2+j*2+1] - xkp[i*3+k, 1]
    return E
```

```python
def LM_opti(frame, ith, x2d, x_para, u=1, v=2, step_max=500):
    J = mat(np.zeros((6*frame*2, 13*frame)))
    E = mat(np.zeros((6*frame*2,1)))   # E = f(X) - b ;
    E_temp = mat(np.zeros((6*frame*2,1)))   # E_temp compare with E in L-M
    x_k = mat(x_para.copy())   #parameter initialization
    step = 0   # iteration steps
    mse_last = 0   # mse value after iteration each time

    # L-M Algorithm obtain optimal parameters
    while(step < step_max):
        step += 1
        mse, mse_temp = 0, 0

        # generate Jacobian Matrix and calculate E
        getfx = mat(np.zeros((6*frame*2,1)))
        getfx = getFx(x_k, frame)
        E = getE(getfx, frame, ith, x2d)   #单帧模式，第几帧，GT

        for i in range(6*frame*2):
            mse += E[i]**2
        mse /= 6*frame*2

        # get new J
        J = mat(np.zeros((6*frame*2, 13*frame)))
        J = getJ(x_k, frame)

        # delta X = ...
        #print(J.T * J)
        dx = mat(np.zeros((13*frame,1)))
        LM = u * mat(np.eye(13*frame))
        dx = -(J.T * J + LM).I * J.T * E
        x_k_temp = x_k.copy()
        x_k_temp += dx

        #R的更新不能简单赋值
        #get R meet R.T*R=I
        #U * D * V.T = R --> R+ = U * V.T
        '''
        R_old = mat([[x_k_temp[0,0], x_k_temp[1,0], x_k_temp[2,0]],
                     [x_k_temp[4,0], x_k_temp[5,0], x_k_temp[6,0]],
                     [x_k_temp[8,0], x_k_temp[9,0], x_k_temp[10,0]]])
        U, sigma, VT = la.svd(R_old)
        R_new = U * VT
        x_k_temp[0,0], x_k_temp[1,0], x_k_temp[2,0] = R_new[0,0], R_new[0,1], R_new[0,2]
        x_k_temp[4,0], x_k_temp[5,0], x_k_temp[6,0] = R_new[1,0], R_new[1,1], R_new[1,2]
        x_k_temp[8,0], x_k_temp[9,0], x_k_temp[10,0] = R_new[2,0], R_new[2,1], R_new[2,2]
        '''
        ###########

        # calculate E_temp with x_k_temp
        # copy from E with x_k
        getfx_temp = mat(np.zeros((6*frame*2,1)))
        getfx_temp = getFx(x_k_temp, frame)
        E_temp = getE(getfx_temp, frame, ith, x2d)

        for i in range(6*frame*2):
            mse_temp += E_temp[i]**2
        mse_temp /= 6*frame*2

        # segma value to choose optimization model
        segma = (mse - mse_temp)/((dx.T * (u * dx - J.T * E))[0,0])

        # calculate new u
        if segma > 0:
            s = 1.0/3.0
            v = 2
            x_k = x_k_temp
            mse = mse_temp
            u = u * max(s, 1-pow(2*segma,3))
            u = u[0,0]
        else:
            u = u * v
            v = v * 2
            x_k = x_k_temp

        print("step = %d, abs(mse-mse_last) = %.8f" %(step, abs(mse-mse_last)))
        if abs(mse-mse_last)<0.000001:
            break
        mse_last = mse

    print("step = ", step)
    print("mse = ", mse_last)
    #print("parameter = ", x_k)

    return x_k
```

```python
frame = 3
m = loadmat("valid.mat")

# camera intrinsic matrix
K = m["annot"][0][0][4]
K_cam = K[0][0].tolist()

# key point 3D groundtruth
gt = m["annot"][0][0][3]
img_gt = np.zeros((frame, 3, 17))
img1_gt = gt[135]   # array 3 by 17
img2_gt = gt[136]   # array 3 by 17
img3_gt = gt[137]   # array 3 by 17
img_gt[0] = img1_gt
img_gt[1] = img2_gt
img_gt[2] = img3_gt
#print(img_gt)

# key point 2D groundtruth
kp = np.zeros((frame, 17, 2))
for m in range(frame):
    for i in range(17):
        u = K_cam[0] * mat([img_gt[m][0][i], img_gt[m][1][i], img_gt[m][2][i]]).T
        v = K_cam[1] * mat([img_gt[m][0][i], img_gt[m][1][i], img_gt[m][2][i]]).T
        w = K_cam[2] * mat([img_gt[m][0][i], img_gt[m][1][i], img_gt[m][2][i]]).T
        kp[m][i][0] = u/w
        kp[m][i][1] = v/w
#print(kp)

# show image
img1 = cv2.imread("S9_Posing_1.55011271_000676.jpg")
img2 = cv2.imread("S9_Posing_1.55011271_000681.jpg")
img3 = cv2.imread("S9_Posing_1.55011271_000686.jpg")
plt.figure(figsize = (15, 45)) # 图像窗口名称
plt.subplot(131)
plt.imshow(img1[:,:,[2,1,0]])
plt.subplot(132)
plt.imshow(img2[:,:,[2,1,0]])
plt.subplot(133)
plt.imshow(img3[:,:,[2,1,0]])
plt.axis('on') # 关掉坐标轴为 off
plt.title('image') # 图像题目
plt.show()

# visualize key points
"""
txt = ['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17']
plt.figure()
plt.subplot(131)
img_kp = plt.scatter(kp[0][:,0], kp[0][:,1], s = 80, c = 'g', marker = 'X')
for i in range(17):
    plt.annotate(txt[i], xy = (kp[0][i,0], kp[0][i,1]), xytext = (kp[0][i,0]+0.1, kp[0][i,1]+0.1))
plt.subplot(132)
img_kp = plt.scatter(kp[1][:,0], kp[1][:,1], s = 80, c = 'g', marker = 'X')
for i in range(17):
    plt.annotate(txt[i], xy = (kp[1][i,0], kp[1][i,1]), xytext = (kp[1][i,0]+0.1, kp[1][i,1]+0.1))
plt.subplot(133)
img_kp = plt.scatter(kp[2][:,0], kp[2][:,1], s = 80, c = 'g', marker = 'X')
for i in range(17):
    plt.annotate(txt[i], xy = (kp[2][i,0], kp[2][i,1]), xytext = (kp[2][i,0]+0.1, kp[2][i,1]+0.1)) # 这里xy是需要标记的坐标，xytext是对应的标签坐标
plt.axis('on') # 关掉坐标轴为 off
plt.title('keypoint of image') # 图像题目
"""

# visualize torso joints A-F
plt.figure(figsize = (5,15))
plt.subplot(311)
for i in {0,1,4,8,11,14}:
    img_kp = plt.scatter(kp[0][i,0], kp[0][i,1], s = 80, c = 'g', marker = 'X')
plt.subplot(312)
for i in {0,1,4,8,11,14}:
    img_kp = plt.scatter(kp[1][i,0], kp[1][i,1], s = 80, c = 'g', marker = 'X')
plt.subplot(313)
for i in {0,1,4,8,11,14}:
    img_kp = plt.scatter(kp[2][i,0], kp[2][i,1], s = 80, c = 'g', marker = 'X')
plt.axis('on') # 关掉坐标轴为 off
plt.title('torso keypoint of image') # 图像题目
plt.show()

# save 2D keypoint coordinate
x2d = np.zeros((6 * frame,2))   #[AAABBBCCCDDDEEEFFF].T
for i in range(6):
    for j in range(frame):
        if i==0 :
            x2d[i*frame+j, :] = kp[j][14, :]
        elif i==1 :
            x2d[i*frame+j, :] = kp[j][8, :]
        elif i==2 :
            x2d[i*frame+j, :] = kp[j][11, :]
        elif i==3 :
            x2d[i*frame+j, :] = kp[j][4, :]
        elif i==4 :
            x2d[i*frame+j, :] = kp[j][0, :]
        elif i==5 :
            x2d[i*frame+j, :] = kp[j][1, :]
print(x2d)
```

```python
# parameter initialization for first frame
x_para = np.zeros((13,1))
x_para[0] = -1   # r11
x_para[5] = 1   # r22
x_para[10] = -1   # r33
x_para[3] = 0.0047   #tx(cal)
x_para[7] = -0.0997   #ty(cal)
x_para[11] = 0   # th
x_para[12] = 0   #al
#x_para[-3] = 0.35   # a
#x_para[-2] = 0.25   # b
#x_para[-1] = -0.095  # c(cal)
getfx_ini = getFx(x_para, 1)
print(getfx_ini)
```

    [[476.02538041]
     [284.62882932]
     [514.25209744]
     [284.62882932]
     [552.47881446]
     [284.62882932]
     [486.94729956]
     [393.65003291]
     [514.25209744]
     [393.65003291]
     [541.55689531]
     [393.65003291]]

```python
# visualize first frame after initilization
plt.figure()
img_kp = plt.scatter(kp[0][14,0], kp[0][14,1], s = 80, c = 'g', marker = 'X')
img_kp = plt.scatter(getfx_ini[0,0], getfx_ini[1,0], s = 80, c = 'r', marker = 'X')
img_kp = plt.scatter(kp[0][8,0], kp[0][8,1], s = 80, c = 'g', marker = 'X')
img_kp = plt.scatter(getfx_ini[2,0], getfx_ini[3,0], s = 80, c = 'r', marker = 'X')
img_kp = plt.scatter(kp[0][11,0], kp[0][11,1], s = 80, c = 'g', marker = 'X')
img_kp = plt.scatter(getfx_ini[4,0], getfx_ini[5,0], s = 80, c = 'r', marker = 'X')
img_kp = plt.scatter(kp[0][4,0], kp[0][4,1], s = 80, c = 'g', marker = 'X')
img_kp = plt.scatter(getfx_ini[6,0], getfx_ini[7,0], s = 80, c = 'r', marker = 'X')
img_kp = plt.scatter(kp[0][0,0], kp[0][0,1], s = 80, c = 'g', marker = 'X')
img_kp = plt.scatter(getfx_ini[8,0], getfx_ini[9,0], s = 80, c = 'r', marker = 'X')
img_kp = plt.scatter(kp[0][1,0], kp[0][1,1], s = 80, c = 'g', marker = 'X')
img_kp = plt.scatter(getfx_ini[10,0], getfx_ini[11,0], s = 80, c = 'r', marker = 'X')
plt.axis('on') # 关掉坐标轴为 off
plt.title('image_kp_ABCDEF_original_opt') # 图像题目
plt.show()
```

```python
# optimization for each frame
X_para = np.zeros((13*frame,1))
#X_para[-3] = 0.35   # a
#X_para[-2] = 0.25   # b
#X_para[-1] = -0.095  # c(cal)
para_data = np.zeros((frame,13))
x_k_para = x_para.copy()
for i in range(frame):
    x_k = LM_opti(1, i, x2d, x_k_para)   #单帧优化，第几帧，GT，初始化参数

    # R matrix adjustment

    R_old = mat([[x_k[0,0], x_k[1,0], x_k[2,0]],
             [x_k[4,0], x_k[5,0], x_k[6,0]],
             [x_k[8,0], x_k[9,0], x_k[10,0]]])
    U, sigma, VT = la.svd(R_old)
    R_new = U * VT
    x_k[0,0], x_k[1,0], x_k[2,0] = R_new[0,0], R_new[0,1], R_new[0,2]
    x_k[4,0], x_k[5,0], x_k[6,0] = R_new[1,0], R_new[1,1], R_new[1,2]
    x_k[8,0], x_k[9,0], x_k[10,0] = R_new[2,0], R_new[2,1], R_new[2,2]

    print()
    print("For the No.", i+1, " frame : x_k = ", x_k)
    para_data[i] = mat(x_k).T
    # for next frame initilization
    x_k_para = x_k
    X_para[13*i:13*i+13] = x_k[0:13,0]
print(para_data)
print(X_para)
```

    step = 1, abs(mse-mse_last) = 9.22249205
    step = 2, abs(mse-mse_last) = 0.02378165
    step = 3, abs(mse-mse_last) = 0.00072360
    step = 4, abs(mse-mse_last) = 0.00000500
    step = 5, abs(mse-mse_last) = 0.00000281
    step = 6, abs(mse-mse_last) = 0.00000229
    step = 7, abs(mse-mse_last) = 0.00000187
    step = 8, abs(mse-mse_last) = 0.00000154
    step = 9, abs(mse-mse_last) = 0.00000126
    step = 10, abs(mse-mse_last) = 0.00000104
    step = 11, abs(mse-mse_last) = 0.00000085
    step =  11
    mse =  [[9.19797099]]

    For the No. 1  frame : x_k =  [[-0.99777882]
     [-0.06650087]
     [-0.00388072]
     [ 0.00648089]
     [-0.06657683]
     [ 0.99358229]
     [ 0.09144262]
     [-0.09969034]
     [-0.0022252 ]
     [ 0.09149788]
     [-0.99580278]
     [ 0.07770556]
     [ 0.05833333]]
    step = 1, abs(mse-mse_last) = 9.94782781
    step = 2, abs(mse-mse_last) = 0.02746635
    step = 3, abs(mse-mse_last) = 0.00177855
    step = 4, abs(mse-mse_last) = 0.00027871
    step = 5, abs(mse-mse_last) = 0.00020889
    step = 6, abs(mse-mse_last) = 0.00016999
    step = 7, abs(mse-mse_last) = 0.00013906
    step = 8, abs(mse-mse_last) = 0.00011305
    step = 9, abs(mse-mse_last) = 0.00009120
    step = 10, abs(mse-mse_last) = 0.00007303
    step = 11, abs(mse-mse_last) = 0.00005807
    step = 12, abs(mse-mse_last) = 0.00004587
    step = 13, abs(mse-mse_last) = 0.00003602
    step = 14, abs(mse-mse_last) = 0.00002814
    step = 15, abs(mse-mse_last) = 0.00002187
    step = 16, abs(mse-mse_last) = 0.00001692
    step = 17, abs(mse-mse_last) = 0.00001303
    step = 18, abs(mse-mse_last) = 0.00001000
    step = 19, abs(mse-mse_last) = 0.00000765
    step = 20, abs(mse-mse_last) = 0.00000583
    step = 21, abs(mse-mse_last) = 0.00000443
    step = 22, abs(mse-mse_last) = 0.00000336
    step = 23, abs(mse-mse_last) = 0.00000254
    step = 24, abs(mse-mse_last) = 0.00000191
    step = 25, abs(mse-mse_last) = 0.00000144
    step = 26, abs(mse-mse_last) = 0.00000108
    step = 27, abs(mse-mse_last) = 0.00000081
    step =  27
    mse =  [[9.91725084]]

    For the No. 2  frame : x_k =  [[-9.94832017e-01]
     [-6.34123174e-02]
     [-7.92977700e-02]
     [ 2.83009081e-04]
     [-7.57785120e-02]
     [ 9.83513479e-01]
     [ 1.64191515e-01]
     [-9.97721359e-02]
     [ 6.75786612e-02]
     [ 1.69352043e-01]
     [-9.83235989e-01]
     [ 1.05507098e-01]
     [ 5.96611604e-01]]
    step = 1, abs(mse-mse_last) = 12.45541207
    step = 2, abs(mse-mse_last) = 1.93660396
    step = 3, abs(mse-mse_last) = 0.17921099
    step = 4, abs(mse-mse_last) = 0.00122374
    step = 5, abs(mse-mse_last) = 0.00079439
    step = 6, abs(mse-mse_last) = 0.00054167
    step = 7, abs(mse-mse_last) = 0.00037458
    step = 8, abs(mse-mse_last) = 0.00025998
    step = 9, abs(mse-mse_last) = 0.00018026
    step = 10, abs(mse-mse_last) = 0.00012464
    step = 11, abs(mse-mse_last) = 0.00008589
    step = 12, abs(mse-mse_last) = 0.00005898
    step = 13, abs(mse-mse_last) = 0.00004037
    step = 14, abs(mse-mse_last) = 0.00002755
    step = 15, abs(mse-mse_last) = 0.00001874
    step = 16, abs(mse-mse_last) = 0.00001272
    step = 17, abs(mse-mse_last) = 0.00000861
    step = 18, abs(mse-mse_last) = 0.00000581
    step = 19, abs(mse-mse_last) = 0.00000391
    step = 20, abs(mse-mse_last) = 0.00000263
    step = 21, abs(mse-mse_last) = 0.00000177
    step = 22, abs(mse-mse_last) = 0.00000118
    step = 23, abs(mse-mse_last) = 0.00000079
    step =  23
    mse =  [[10.33582969]]

    For the No. 3  frame : x_k =  [[-0.99022431]
     [-0.09286013]
     [-0.10408076]
     [-0.00734224]
     [-0.1147367 ]
     [ 0.96658829]
     [ 0.22922166]
     [-0.10002233]
     [ 0.07931769]
     [ 0.23892275]
     [-0.96779369]
     [ 0.01941385]
     [ 1.19945973]]
    [[-9.97778821e-01 -6.65008667e-02 -3.88072259e-03  6.48089423e-03
      -6.65768262e-02  9.93582293e-01  9.14426244e-02 -9.96903414e-02
      -2.22519653e-03  9.14978801e-02 -9.95802785e-01  7.77055571e-02
       5.83333333e-02]
     [-9.94832017e-01 -6.34123174e-02 -7.92977700e-02  2.83009081e-04
      -7.57785120e-02  9.83513479e-01  1.64191515e-01 -9.97721359e-02
       6.75786612e-02  1.69352043e-01 -9.83235989e-01  1.05507098e-01
       5.96611604e-01]
     [-9.90224315e-01 -9.28601256e-02 -1.04080755e-01 -7.34224120e-03
      -1.14736704e-01  9.66588287e-01  2.29221665e-01 -1.00022327e-01
       7.93176866e-02  2.38922749e-01 -9.67793689e-01  1.94138534e-02
       1.19945973e+00]]
    [[-9.97778821e-01]
     [-6.65008667e-02]
     [-3.88072259e-03]
     [ 6.48089423e-03]
     [-6.65768262e-02]
     [ 9.93582293e-01]
     [ 9.14426244e-02]
     [-9.96903414e-02]
     [-2.22519653e-03]
     [ 9.14978801e-02]
     [-9.95802785e-01]
     [ 7.77055571e-02]
     [ 5.83333333e-02]
     [-9.94832017e-01]
     [-6.34123174e-02]
     [-7.92977700e-02]
     [ 2.83009081e-04]
     [-7.57785120e-02]
     [ 9.83513479e-01]
     [ 1.64191515e-01]
     [-9.97721359e-02]
     [ 6.75786612e-02]
     [ 1.69352043e-01]
     [-9.83235989e-01]
     [ 1.05507098e-01]
     [ 5.96611604e-01]
     [-9.90224315e-01]
     [-9.28601256e-02]
     [-1.04080755e-01]
     [-7.34224120e-03]
     [-1.14736704e-01]
     [ 9.66588287e-01]
     [ 2.29221665e-01]
     [-1.00022327e-01]
     [ 7.93176866e-02]
     [ 2.38922749e-01]
     [-9.67793689e-01]
     [ 1.94138534e-02]
     [ 1.19945973e+00]]

```python
def visual2D(kp, getfx):
    plt.figure()
    img_kp = plt.scatter(kp[14,0], kp[14,1], s = 80, c = 'g', marker = 'X')
    img_kp = plt.scatter(getfx[0,0], getfx[1,0], s = 80, c = 'r', marker = 'X')
    img_kp = plt.scatter(kp[8,0], kp[8,1], s = 80, c = 'g', marker = 'X')
    img_kp = plt.scatter(getfx[2,0], getfx[3,0], s = 80, c = 'r', marker = 'X')
    img_kp = plt.scatter(kp[11,0], kp[11,1], s = 80, c = 'g', marker = 'X')
    img_kp = plt.scatter(getfx[4,0], getfx[5,0], s = 80, c = 'r', marker = 'X')
    img_kp = plt.scatter(kp[4,0], kp[4,1], s = 80, c = 'g', marker = 'X')
    img_kp = plt.scatter(getfx[6,0], getfx[7,0], s = 80, c = 'r', marker = 'X')
    img_kp = plt.scatter(kp[0,0], kp[0,1], s = 80, c = 'g', marker = 'X')
    img_kp = plt.scatter(getfx[8,0], getfx[9,0], s = 80, c = 'r', marker = 'X')
    img_kp = plt.scatter(kp[1,0], kp[1,1], s = 80, c = 'g', marker = 'X')
    img_kp = plt.scatter(getfx[10,0], getfx[11,0], s = 80, c = 'r', marker = 'X')
    plt.axis('on') # 关掉坐标轴为 off
    plt.title('image_kp_ABCDEF_original_opt') # 图像题目
    plt.show()
```

```python
visual = np.zeros((13,1))
for i in range(frame):
    visual = mat(para_data[i]).T
    getfx_visual = getFx(visual, 1)
    visual2D(kp[i], getfx_visual)
```

```python
def visual3D(img_gt, para):
    K = Matrix([[1149.67569986785, 0.0, 508.848621645943],
                 [0.0, 1147.59161666764, 508.064917088557],
                 [0.0, 0.0, 1.0]])
    r11, r12, r13, r14, r21, r22, r23, r24, r31, r32, r33 = symbols('r11 r12 r13 r14 r21 r22 r23 r24 r31 r32 r33')
    Rt = Matrix([[r11, r12, r13, r14], [r21, r22, r23, r24], [r31, r32, r33, 1]])
    a, b, c, th, al = symbols('a b c th al')
    ua, va, wa, ub, vb, wb, uc, vc, wc, ud, vd, wd, ue, ve, we, uf, vf, wf = symbols('ua va wa ub vb wb uc vc wc ud vd wd ue ve we uf vf wf')
    XA = Matrix([[-a * c * cos(th) * cos(al)], [c-a * c * sin(th)], [-a * c *  cos(th) * sin(al)], [1]])
    XB = Matrix([[0], [c], [0], [1]])
    XC = Matrix([[a * c * cos(th) * cos(al)], [c+a * c * sin(th)], [a * c * cos(th) * sin(al)], [1]])
    XD = Matrix([[-b * c], [0], [0], [1]])
    XE = Matrix([[0], [0], [0], [1]])
    XF = Matrix([[b * c], [0], [0], [1]])
    A3D = Rt * XA
    B3D = Rt * XB
    C3D = Rt * XC
    D3D = Rt * XD
    E3D = Rt * XE
    F3D = Rt * XF
    j = 0
    #A3D_value = A3D.subs({r11:para[13*j], r12:para[13*j+1], r13:para[13*j+2], r14:para[13*j+3],
    #            r21:para[13*j+4], r22:para[13*j+5], r23:para[13*j+6], r24:para[13*j+7],
    #            r31:para[13*j+8], r32:para[13*j+9], r33:para[13*j+10], th:para[13*j+11],
    #            al:para[13*j+12], a:para[-3], b:para[-2], c:para[-1]})
    Y = mat([img_gt[0][0], img_gt[1][0], img_gt[2][0]]).T
    s = Y[2,0]
    print(s)
    #A3D_value = getValue(A3D, para) * s
    A3D_value = A3D.subs({r11:para[0], r12:para[1], r13:para[2], r14:para[3],
        r21:para[4], r22:para[5], r23:para[6], r24:para[7],
        r31:para[8], r32:para[9], r33:para[10], th:para[11],
        al:para[12], a:0.35, b:0.25, c:-0.095}) * s
    B3D_value = B3D.subs({r11:para[0], r12:para[1], r13:para[2], r14:para[3],
        r21:para[4], r22:para[5], r23:para[6], r24:para[7],
        r31:para[8], r32:para[9], r33:para[10], th:para[11],
        al:para[12], a:0.35, b:0.25, c:-0.095}) * s
    C3D_value = C3D.subs({r11:para[0], r12:para[1], r13:para[2], r14:para[3],
        r21:para[4], r22:para[5], r23:para[6], r24:para[7],
        r31:para[8], r32:para[9], r33:para[10], th:para[11],
        al:para[12], a:0.35, b:0.25, c:-0.095}) * s
    D3D_value = D3D.subs({r11:para[0], r12:para[1], r13:para[2], r14:para[3],
        r21:para[4], r22:para[5], r23:para[6], r24:para[7],
        r31:para[8], r32:para[9], r33:para[10], th:para[11],
        al:para[12], a:0.35, b:0.25, c:-0.095}) * s
    E3D_value = E3D.subs({r11:para[0], r12:para[1], r13:para[2], r14:para[3],
        r21:para[4], r22:para[5], r23:para[6], r24:para[7],
        r31:para[8], r32:para[9], r33:para[10], th:para[11],
        al:para[12], a:0.35, b:0.25, c:-0.095}) * s
    F3D_value = F3D.subs({r11:para[0], r12:para[1], r13:para[2], r14:para[3],
        r21:para[4], r22:para[5], r23:para[6], r24:para[7],
        r31:para[8], r32:para[9], r33:para[10], th:para[11],
        al:para[12], a:0.35, b:0.25, c:-0.095}) * s
    print(A3D_value,B3D_value,C3D_value,D3D_value,E3D_value,F3D_value)

    X1 = mat([img_gt[0][14], img_gt[1][14], img_gt[2][14]]).T
    X2 = mat([img_gt[0][8], img_gt[1][8], img_gt[2][8]]).T
    X3 = mat([img_gt[0][11], img_gt[1][11], img_gt[2][11]]).T
    X4 = mat([img_gt[0][4], img_gt[1][4], img_gt[2][4]]).T
    X5 = mat([img_gt[0][0], img_gt[1][0], img_gt[2][0]]).T
    X6 = mat([img_gt[0][1], img_gt[1][1], img_gt[2][1]]).T
    print(X1, X2, X3, X4, X5, X6)

    # 绘制散点图
    fig = plt.figure()
    ax = Axes3D(fig)
    ax.scatter(X1[0], X1[1], X1[2], s = 80, c = 'g', marker = 'X')
    ax.scatter(X2[0], X2[1], X2[2], s = 80, c = 'g', marker = 'X')
    ax.scatter(X3[0], X3[1], X3[2], s = 80, c = 'g', marker = 'X')
    ax.scatter(X4[0], X4[1], X4[2], s = 80, c = 'g', marker = 'X')
    ax.scatter(X5[0], X5[1], X5[2], s = 80, c = 'g', marker = 'X')
    ax.scatter(X6[0], X6[1], X6[2], s = 80, c = 'g', marker = 'X')
    ax.scatter(A3D_value[0], A3D_value[1], A3D_value[2], s = 80, c = 'r', marker = 'X')
    ax.scatter(B3D_value[0], B3D_value[1], B3D_value[2], s = 80, c = 'r', marker = 'X')
    ax.scatter(C3D_value[0], C3D_value[1], C3D_value[2], s = 80, c = 'r', marker = 'X')
    ax.scatter(D3D_value[0], D3D_value[1], D3D_value[2], s = 80, c = 'r', marker = 'X')
    ax.scatter(E3D_value[0], E3D_value[1], E3D_value[2], s = 80, c = 'r', marker = 'X')
    ax.scatter(F3D_value[0], F3D_value[1], F3D_value[2], s = 80, c = 'r', marker = 'X')

    ax.set_zlabel('Z', fontdict={'size': 15, 'color': 'red'})
    ax.set_ylabel('Y', fontdict={'size': 15, 'color': 'red'})
    ax.set_xlabel('X', fontdict={'size': 15, 'color': 'red'})
    plt.show()

    # 计算3D error
    d1 = A3D_value-X1
    e1 = sum(map(lambda x:x**2, d1))
    d2 = B3D_value-X2
    e2 = sum(map(lambda x:x**2, d2))
    d3 = C3D_value-X3
    e3 = sum(map(lambda x:x**2, d3))
    d4 = D3D_value-X4
    e4 = sum(map(lambda x:x**2, d4))
    d5 = E3D_value-X5
    e5 = sum(map(lambda x:x**2, d5))
    d6 = F3D_value-X6
    e6 = sum(map(lambda x:x**2, d6))
    error = (e1**(0.5)+e2**(0.5)+e3**(0.5)+e4**(0.5)+e5**(0.5)+e6**(0.5))/6
    print(error)
    print(d1,d2,d3,d4,d5,d6)
```

```python
visual3d = np.zeros((13,1))
for i in range(frame):
    visual3d = mat(para_data[i]).T
    visual3D(img_gt[i], visual3d)
```

```python

```

annot/BA and LM --- mF6P (m=3) #2 fix a:b:c.ipynb