# The problem

The product development department at Dream-Inc is suggesting a software upgrade to a cleaning robot, claiming to decrease its failure rate. To quantify this claim, a test was conducted. Software
version (“treatment”), run-id (“id”), and outcome (“failure”) was recorded and provided to you. You are tasked to help senior management with guidance on an informed decision.

# Tasks

Specifically, you are asked to answer the following questions:
1. Is software version B decreasing the error rate. If yes, by how much?
2. Please quantify your confidence in statement (1)?
3. Each failed run costs Dream-Inc $2.32, the software update \$43k. Which software version should be used?

Please communicate your answers in a short presentation. Feel free to prepare back-up-slides for relevant technical details and interesting observations.

# Solution

First things first: the libraries we need.

```python
# Manipulation of dataframes
import pandas as pd

# Matrix/vector calculations
import numpy as np

# Normal distribution
import scipy
from scipy.stats import norm

# Charts
import matplotlib.pyplot as plt
%matplotlib inline
```

Then we check versions. This is useful whenever we need to consult documentation.

```python
print(f'Pandas version: {pd.__version__}')
print(f'Numpy version: {np.__version__}')
print(f'Scipy version: {scipy.__version__}')
```

    Pandas version: 1.1.5
    Numpy version: 1.19.5
    Scipy version: 1.4.1

Mount my Google Drive account so I can access the datasets hosted in `DATA_PATH`.

```python
from google.colab import drive
drive.mount('/content/drive')

DATA_PATH = 'drive/MyDrive/data'
```

    Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).

Load the data. Notice that, because I already know the datasets, I can instruct Pandas to cast the `treatment` feature as strings instead of floats, which is appropriate because this feature is indeed categorical, not numerical.

```python
labels = pd.read_csv(f'{DATA_PATH}/label.csv', sep='|', dtype={'treatment': str})
outcomes = pd.read_csv(f'{DATA_PATH}/outcome.csv', sep='|')
```

# Checking NA (AKA blanks)

First we implement an utility function to help us spot NA in the datasets:

```python
def check_na(df):
  for (column_name, data) in df.iteritems():
    n_blanks = len(data) - len(data.dropna())
    if n_blanks > 0:
      print(f'Column "{column_name}" has {n_blanks} NA')
```

Then we run it against both datasets:

```python
check_na(outcomes)
check_na(labels)
```

    Column "failure" has 3 NA

Now that we know where are those NA, we can safely drop them:

```python
outcomes = outcomes.dropna()
```

## Cast types

```python
print(labels.dtypes)
print(outcomes.dtypes)
```

    id           object
    treatment    object
    dtype: object
    id          object
    failure    float64
    dtype: object

```python
dataset = outcomes.merge(labels, on='id')
```

```python
dataset.head()
```

      id
      failure
      treatment

      0
      5d6c2b66-27b8-4c31-8670-c9e0cc1c9a3d
      0.0
      1.4

      1
      d628fd57-c4cd-4bfd-855e-59da588a3925
      0.0
      1.4

      2
      e185f40d-e88d-41bd-a2fd-2cd0fbaf7ce1
      0.0
      1.5

      3
      0351b627-06b4-451e-8bbc-842e318ec84b
      1.0
      1.4

      4
      1ab70e83-0947-4ef7-9a36-2367c0c68f5a
      0.0
      1.4

```python
id_counts = dataset.id.value_counts()
id_counts[id_counts > 1]
```

    aa9eed90-8ba9-4d14-aeae-24f23069c739    4
    Name: id, dtype: int64

```python
dataset.failure.value_counts()
```

     0.0      2201
     1.0        59
     100.0       1
    -1.0         1
    Name: failure, dtype: int64

```python
dataset.treatment.value_counts()
```

    1.4    1507
    1.5     755
    Name: treatment, dtype: int64

## Drop noise on `failure`

```python
from math import isclose

@np.vectorize
def not_noise(failure):
  return isclose(failure, 0) or isclose(failure, 1)

df = dataset[not_noise(dataset.failure)]
```

```python
df.failure.value_counts()
```

    0.0    2201
    1.0      59
    Name: failure, dtype: int64

## Convert `failure` to `success` (and boolean)

```python
@np.vectorize
def success(failure):
  return not bool(failure)

df = pd.DataFrame(data={
    'id': dataset.id,
    'success': success(dataset.failure),
    'version': dataset.treatment
})
```

```python
df.head()
```

      id
      success
      version

      0
      5d6c2b66-27b8-4c31-8670-c9e0cc1c9a3d
      True
      1.4

      1
      d628fd57-c4cd-4bfd-855e-59da588a3925
      True
      1.4

      2
      e185f40d-e88d-41bd-a2fd-2cd0fbaf7ce1
      True
      1.5

      3
      0351b627-06b4-451e-8bbc-842e318ec84b
      False
      1.4

      4
      1ab70e83-0947-4ef7-9a36-2367c0c68f5a
      True
      1.4

```python
df.dtypes
```

    id         object
    success      bool
    version    object
    dtype: object

## Assure independent events and samples

Remove duplicates of any cleaning robot (ID).

```python
df = df.drop_duplicates(subset='id')
```

```python
id_duplicates = not len(set(df.id)) == len(df.id)

print(f'There are duplicate ID in dataset: {id_duplicates}')
```

    There are duplicate ID in dataset: False

## Statistical test

The appropriate test for this problem is the two-proportions hypothesis test, which is implemented by the `proportions_ztest` module in the `statsmodels` package.

### Hypotheses

$
\begin{align}
H_0&: p_B - p_A = 0 \\
H_a&: p_B - p_A > 0
\end{align}
$

Without any reason to argue in favor of type-I or type-II errors, we **choose** the standard **relevance level** $\alpha = 5\%$, which implies **confidence level** of 95%.

```python
𝛼 = 0.05
```

Moreover, the **inequality** above implies we must use a single-tail test. In this case the critical statistics $z_\alpha$ is:

```python
z_alpha = norm.ppf(1-𝛼)
print(f'Critical value, z_alpha = {z_alpha}')
```

    Critical value, z_alpha = 1.6448536269514722

### Proportions

Now we need to calculate success proportion for each sample, as well as for both samples together (joint sample). We do this in Pandas by filtering the dataframe based on the feature `version`:

```python
version_A = df[df.version == '1.4']
version_B = df[df.version == '1.5']
```

```python
n_success_A = len(version_A[version_A.success]) # Amount of successes for 1.4
n_A = len(version_A) # Sample size for 1.4
p_A = n_success_A / n_A # Success rate (or proportion) for 1.4

n_success_B = len(version_B[version_B.success])
n_B = len(version_B)
p_B = n_success_B / n_B

# Joint successes, sample size and success proportion
n_success = n_success_A + n_success_B
n = n_A + n_B
p = n_success / n
```

```python
#1.4: "p-hat-A": x1 = 43 and n1 = 1505 → (43/1505 = 0.0286)
#1.5: "p-hat-B": x2 = 16 and n2 = 753  → (16/753   = 0.0212)
```

```python
print(n_success_A)
print(n_A-n_success_A)
print(n_A)
print(n_success_B)
print(n_B-n_success_B)
print(n_B)
print(n_success)
print(n-n_success)
print(n)
```

    1461
    44
    1505
    737
    17
    754
    2198
    61
    2259

```python
print(f'p_A = {p_A}')
print(f'p_B = {p_B}')
print(f'p = {p}')

print(f'{1 - p_A}')
print(f'{1 - p_B}')
print(f'{p_B-p_A}')

```

    p_A = 0.9707641196013289
    p_B = 0.9774535809018567
    p = 0.9729969012837538
    0.029235880398671088
    0.022546419098143256
    0.0066894613005278325

```python
# Alex, this is a quite complex technique and I used it just to simplify the
# calculation for the amount of decimal places to ensure only one significative
# figure in joint error for success proportion, below. Do not even bother to
# understand it. I can explain it to you after your interview.
def compose(*functions):
  def composition(x):
    ans = x
    for f in reversed(functions):
      ans = f(ans)
    return ans

  return composition
```

```python
stdev_A = np.sqrt(p_A*(1-p_A))
stdev_B = np.sqrt(p_B*(1-p_B))

standard_error_A = stdev_A / np.sqrt(n_A)
standard_error_B = stdev_B / np.sqrt(n_B)
```

```python
# Joint standard error on joint success proportion
joint_standard_error = np.sqrt(stdev_A**2/n_A + stdev_B**2/n_B)

# Joint error is the standard joint error scaled by the choosed confidence level (two-tail)
joint_error = norm.ppf(0.975) * joint_standard_error

# The next lines truncate the joint error to just one significative figure.
decimal_places = compose(int, np.abs, np.floor, np.log10)
dp = decimal_places(joint_error)
truncated_joint_error = round(joint_error, dp)
print(f'Joint error: {truncated_joint_error}')

bottom_limit = (p_A - p_B) - truncated_joint_error
top_limit = (p_A - p_B) + truncated_joint_error
print(f'{round(bottom_limit, dp)} < p_A - p_B < {round(top_limit, dp)}')
```

    Joint error: 0.01
    -0.02 < p_A - p_B < 0.0

Notice that, according to confidence interval, we might have $\hat p_A > \hat p_B$ as well as $\hat p_A < \hat p_B$.

### Checking assumptions

The assumptions under which the proportion test is valid are:
1. Independent samples.
1. $n_Ap_A > 5$ and $n_A(1-p_A) > 5$
1. $n_Bp_B > 5$ and $n_B(1-p_B) > 5$

Concerning assumption 1, we argue that, as long the same cleaning robot is not present in both samples, and there are no repetitions of the same ID within each sample and in both samples, then the samples are independent, as are each event within each sample. However, it remains to check if there are any correlation among cleaning robots and the area they clean.

Concerning assumptions 2 and 3, we can compute and check explicitly:

```python
print(f'Condition (2) is satisfied: {n_A * p_A > 5 and n_A * (1-p_A) > 5}')
print(f'Condition (3) is satisfied: {n_B * p_B > 5 and n_B * (1-p_B) > 5}')
```

    Condition (2) is satisfied: True
    Condition (3) is satisfied: True

### The test

The function `proportions_ztest` implements the test for proportion. So we need to import it:

```python
from statsmodels.stats.proportion import proportions_ztest
```

```python
z_stat, p_value = proportions_ztest(
  [n_success_B, n_success_A],
  [n_B, n_A],
  alternative='larger'
)

print(f'Sample A (version 1.4): {n_success_A} successes in {n_A} tries.')
print(f'Sample B (version 1.5): {n_success_B} successes in {n_B} tries.')
print(f'Significance level (𝛼): {𝛼}')
print(f'z-statistic: {z_stat}')
print(f'p-value: {p_value}')
```

    Sample A (version 1.4): 1461 successes in 1505 tries.
    Sample B (version 1.5): 737 successes in 754 tries.
    Significance level (𝛼): 0.05
    z-statistic: 0.9249631316745907
    p-value: 0.1774925427443519

### Interpretation

The (absolute value of) $z$ statistics is smaller than $z_\alpha$, which means the result is not too extreme to be meaningful. In statistical language: the data we have does not provide enough evidence to reject the null hypothesis that $p_B - p_A = 0$. This can also be checked with $p$-value, which is greater than the relevance level $\alpha = 5\%$. Indeed, the $p$-value means there are 18% chance that we observe what we observed, given the two samples come from the same population or, equivalently, that null hypothesis is true. In other words, the difference $p_A - p_B$ is not statistically meaningful. In practice it means there is no advantage (or disadvantage) to upgrade software from version 1.4 to 1.5, as it does not provide any meaningul difference in the long run.

### Graphical interpretation

First we define the probability density functions for the sampling proportion, to be reused. Note that, according to central limit theorem, the sampling proportion $\bar P \sim N\left(p, \sqrt{p(1-p)/n}\right)$. i.e., it follows gaussian distribution with mean equal to proportion $p$ and standard deviation $\sqrt{p(1-p)/n}$, where $n$ is the sample size.

```python
# The probability density function for sample A (version 1.4)
@np.vectorize
def pdf_A(x):
  return norm.pdf(x, loc=p_A, scale=standard_error_A)

# The probability density function for sample B (version 1.5)
@np.vectorize
def pdf_B(x):
  return norm.pdf(x, loc=p_B, scale=standard_error_B)
```

```python
fig, ax = plt.subplots()
plt.xlabel('P')
plt.ylabel('Probability density')

# Define the range of the chart
xs = np.linspace(0.95, 1, 100)

# Plot probability density functions
ax.plot(xs, pdf_A(xs), label='1.4')
ax.plot(xs, pdf_B(xs), label='1.5')

# Add vertical line and label for p_A
plt.vlines(p_A, 0, pdf_A(p_A), linestyles='dotted')
plt.text(p_A, 50, '$\hat p_A$', ha='right', fontsize=15)

# Add vertical line and label for p_B
plt.vlines(p_B, 0, pdf_B(p_B), linestyles='dotted')
plt.text(p_B, 50, '$\hat p_B$', ha='left', fontsize=15)

# Add legend
ax.legend();
```

As we can see above, the overlap between probability density functions is not negligible. This is a graphical evidence of the hypothesis test result.

The following chart presents a simplified view of the above result, for didatic purposes only:

```python
fig, ax = plt.subplots()
plt.xlabel('P')
plt.ylabel('Probability density')

# Define the range of the chart
xs = np.linspace(0.95, 1, 100)

# Plot probability density functions
ax.plot(xs, pdf_A(xs), label='1.4')

error = norm.ppf(1-𝛼) * standard_error_A
xs = np.linspace(0.95, p_A+error, 100)
ys = pdf_A(xs)
plt.fill_between(xs, ys, alpha=0.2)

# Add vertical line and label for p_A
plt.vlines(p_A, 0, pdf_A(p_A), linestyles='dotted')
plt.text(p_A, 50, '$\hat p_A$', ha='right', fontsize=15)

# Add vertical line and label for p_B
plt.vlines(p_B, 0, pdf_B(p_B), linestyles='dotted')
plt.text(p_B, 50, '$\hat p_B$', ha='left', fontsize=15)

plt.show()
```

In picture above, were $\hat p_B$ a population parameter, its value would be almost beyond the critical $P$ (the end of shaded area). This means obtaining $\hat p_A$ as far from $\hat p_B$ as seen above is just above 5% (relevance level): the blank area.

### Hand calculation

```python
p = (n_success_A + n_success_B) / (n_A + n_B)
p
```

    0.9729969012837538

```python
z_statistic = (p_B - p_A) / np.sqrt(p*(1-p)*(1/n_A + 1/n_B))
z_statistic
```

    0.9249631316745907

```python
p_value = norm.sf(z_statistic) # sf = Survival function. Check documentation: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bernoulli.html
p_value
```

    0.1774925427443519

### Resampling

The framework used to test the hypothesis does not impose any restriction to the difference in samples size. In any case we can perform the test with two samples of the same size. For this we only need to undersample sample A, which is greater than B.

```python
undersample_A = pd.DataFrame(data={
    'success': np.random.choice(version_A.success, size=len(version_B))
})
```

```python
n_success_A_undersample = len(undersample_A[undersample_A.success])
n_A_undersample = len(undersample_A)

z_stat, p_value = proportions_ztest(
  [n_success_A_undersample, n_success_B],
  [n_A_undersample, n_B],
  alternative='smaller'
)

print(f'Sample A (version 1.4): {n_success_A_undersample} successes in {n_A_undersample} tries.')
print(f'Sample B (version 1.5): {n_success_B} successes in {n_B} tries.')
print(f'Significance level (𝛼): {𝛼}')
print(f'z-statistic: {z_stat}')
print(f'p-value: {p_value}')
```

    Sample A (version 1.4): 727 successes in 754 tries.
    Sample B (version 1.5): 737 successes in 754 tries.
    Significance level (𝛼): 0.05
    z-statistic: -1.5300435561304269
    p-value: 0.06300297401539187

Conclusion is the same as before: $p$-value is greater than significance level $\alpha$, which means the observed result is quite probable, given both samples came from the same population. In other words, there is no distinction between success rate for samples A and B.

Equivalent analysis can be done with z-statistic: it is smaller (absolute value) than critical value ($\sim 1.6$).

### Statistical power

Cohen's d metric is a measure of effect size:

```python
cohen_effect_size = (p_B - p_A) / joint_standard_error
cohen_effect_size
```

    0.9646755334156003

The value ~0.4 for Cohen measure means we are facing a small to medium effect size.

```python
from statsmodels.stats.power import zt_ind_solve_power
```

```python
n_B
```

    754

```python
zt_ind_solve_power(effect_size=cohen_effect_size, alpha=𝛼, nobs1=n_A, ratio=n_B/n_A, alternative='larger')
```

    1.0

The above result is the _statistical power_ for the test. It means the probability of correctly reject the null hypothesis is nearly 100%, supposing the alternative hypothesis is true. That means we can safely state the conclusion (not rejecting the null hypothesis) of the above test is right.

The statistical power is also $1-\beta$, where $\beta$ is the probability of type II errors. In words, there are nearly 0% chance of not reject the null hypothesis, in the case there is an effect (difference between $p_A$ and $p_B$).

# Answers

## Question 1
_Is software version B decreasing the error rate. If yes, by how much?_

**Answer**: no. Software version B does not improve (decrease) error rate in the long run.

## Question 2

_Please quantify your confidence in statement (1)._

**Answer**: the _confidence level_ for the assertion that $p_A = p_B$ is 95%. That means that 95% of the samples of size 2259 extracted from the population of all (potentially infinite) cleaning sessions, shall provide success rate within range $-0.04 < p_A - p_B < 0.02$.

## Question 3

_Each failed run costs Dream-Inc $2.32, the software update \$43k. Which software version should be used?_

**Answer**: because there is no statistically significant difference on success (or failure) rates between versions 1.4 and 1.5, the recommendation is to **not** upgrade. The average loss per run is $0.06.

```python
average_loss_per_event = 2.32 * (1 - p) + 0 * p
average_loss_per_event
```

    0.06264718902169113

# Entendendo populações, amostras e o teorema do limite central

```python
# População
mu = 3
sigma = 0.6
```

```python
xs = np.linspace(-5, 10, 100)
ys = norm.pdf(xs, loc=mu, scale=sigma)
```

```python
plt.plot(xs, ys)
```

```python
sample = norm.rvs(loc=mu, scale=sigma, size=50)
```

```python
sample
```

    array([2.34218594, 3.18938007, 3.48093747, 2.69121637, 2.37496776,
           3.35682762, 2.36820321, 2.5742666 , 3.07940155, 2.49122252,
           3.73539842, 2.32378797, 1.83966285, 3.13395997, 3.42902073,
           2.7351111 , 2.5610694 , 3.69529849, 2.49882521, 2.89458062,
           3.46258769, 2.35097785, 3.34439153, 3.59492184, 2.48752196,
           2.10139792, 2.71787866, 3.96212302, 3.41561638, 2.68291001,
           3.7381028 , 2.03798938, 3.33481453, 3.03913568, 3.13727912,
           3.61996843, 3.2258806 , 2.84588698, 2.83397885, 2.19659772,
           2.82929554, 2.67515128, 2.49067552, 2.81686632, 3.3638592 ,
           2.5182478 , 3.22539881, 3.20178911, 2.96633692, 2.98597646])

```python
x_bar = sample.mean()
x_bar
```

    2.919977634882829

```python
s = sample_stdev = sample.std(ddof=1)
s
```

    0.5044678722839728

$\bar x = \hat\mu$

$s = \hat\sigma$

$\hat p$

```python
from scipy.stats import bernoulli
```

```python
# População
p = 0.7
```

```python
xs = [0, 1]
ys = bernoulli.pmf(xs, p=p)
```

```python
plt.bar(xs, ys)
```

mu = 0 * 0.3 + 1 * 0.7

mu = 0 * (1-p) + 1 * p

mu = p

```python
mu = p
mu
```

    0.7

```python
stdev = np.sqrt(p*(1-p))
stdev
```

    0.45825756949558405

```python
sample = bernoulli.rvs(p, size=30)
sample
```

    array([0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1,
           0, 1, 1, 1, 1, 0, 1, 0])

```python
sample_size_I_choose = 30000

sample_means = []

for n in range(1000):
  sample_mean = bernoulli.rvs(p, size=sample_size_I_choose).mean()
  sample_means.append(sample_mean)
```

```python
df = pd.DataFrame(data={'means': sample_means})
df.head()
```

      means

      0
      0.696700

      1
      0.696267

      2
      0.703367

      3
      0.700067

      4
      0.699500

```python
standard_error = stdev / np.sqrt(sample_size_I_choose)

df.hist(bins=20)
xs = np.linspace(0.68, 0.72, 100)
ys = norm.pdf(xs, loc=p, scale=standard_error) # fator de normalização obtido na "raça"
plt.plot(xs, ys)
```

```python
# standard error is the standard deviation of the mean of means
standard_error = stdev / np.sqrt(sample_size_I_choose)
standard_error
```

    0.0026457513110645908

```python
standard_error*3
```

    0.007937253933193772

```python
print(p - standard_error*3)
print(p + standard_error*3)

```

    0.6920627460668062
    0.7079372539331937

```python
# Teorema do limite central
# Distribuição normal com:
mu = p
sigma = standard_error
```

```python
# 1.5
v_esperado_B = 2.32 * (1 - p_B) + 0 * p_B
v_esperado_B
```

    0.05230769230769235

```python
# 1.4
v_esperado_A = 2.32 * (1 - p_A) + 0 * p_A
v_esperado_A
```

    0.06782724252491693

```python
v = v_esperado_A - v_esperado_B
v
```

    0.015519550217224579

$\langle f \rangle = \sum_x P(x) f(x)$

$\langle\text{prejuízo}\rangle = \sum_x P(x) \text{prejuízo}(x)$

```python
43e3/v
```

    2770698.8539059516

$X \sim N(\mu, \sigma) \Rightarrow \bar X \sim N\left(\mu, \frac{\sigma}{\sqrt{n}}\right)$

$P$ is a random variable which follows the Bernoulli distribution with parameter $p$ (probability of success):

$P \sim \text{Bernoulli}(p) \Rightarrow \bar P \sim N\left(p, \frac{\sqrt{p(1-p)}}{\sqrt{n}}\right)$

## Simulation 2

```python
from scipy.stats import bernoulli
```

```python
p1 = 0.971
p2 = 0.977
```

```python
var1 = p1 * (1-p1)
var2 = p2 * (1-p2)
```

```python
n1 = 30000 # Sample size I've choosen
n2 = 20000
```

```python
polled_std = np.sqrt(var1/n1 + var2/n2)
polled_std
```

    0.0014360304082202907

```python
sample_means = []

for n in range(1000):
  sample1 = bernoulli.rvs(p1, size=n1)
  sample2 = bernoulli.rvs(p2, size=n2)

  hat_p1 = sample1.mean()
  hat_p2 = sample2.mean()

  sample_means.append(hat_p2 - hat_p1)

df = pd.DataFrame(data={'means': sample_means})
```

```python
df.hist(bins=20)

xs = np.linspace(0, 0.01, 100)
ys = norm.pdf(xs, loc=df.means.mean(), scale=polled_std)
plt.plot(xs, ys)
```

```python

```


# <center> AcM HW03 </center>
<center> Yuanxing Cheng, Xavier, 程远星, 810925466 </center>

## Question 1
$\newcommand{\ffrac}{\displaystyle \frac}$

($\text{a}$)

$\begin{align}
S \cdot A_{[25]} &= \$200000 \times 0.06143 = \$12286
\end{align}$

($\text{b}$)

$\begin{align}
S \cdot A_{[25]:\overline{30}|}^{\;1} &= S\left( A_{[25]} - {_{30|}}A_{[25]}\right) \\
&= S\left( A_{[25]} - \upsilon^{30} \cdot {_{30}}p_{[25]} \cdot A_{[25]+30}\right) \\
&= S\left( A_{[25]}-\left( \frac{1} {1+i} \right)^{30} \cdot \frac{l_{[25]+30}} {l_{[25]}} \cdot A_{55}\right)\\
&= \$100000 \times \left( 0.06143 -\left( \frac{1} {1+5\%} \right)^{30} \cdot \frac{97846.20} {99865.69} \times 0.23524\right)\\
&= \$810.144
\end{align}$

($\text{c}$)

$\begin{align}
S\cdot A_{[25]:\overline{20}|}&= S\left(A_{[25]:\overline{20}|}^{\;1}+A_{[25]:\overline{20}|}^{\;\;\;\;\;1}\right) \\
&= S\left( A_{[25]} - \upsilon^{20} \cdot {_{20}}p_{[25]} \cdot A_{[25]+20} + \upsilon^{20} \cdot {_{20}}p_{[25]} \right) \\
&= S\left( A_{[25]} - \left( \frac{1} {1+i} \right)^{20} \cdot \frac{l_{[25]+20}} {l_{[25]}} \cdot A_{45} + \left( \frac{1} {1+i} \right)^{20} \cdot \frac{l_{[25]+20}} {l_{[25]}} \right) \\
&= \$500000 \times \left( 0.06143 - \left( \frac{1} {1+5\%} \right)^{20} \cdot \frac{99033.94} {99865.69} \cdot 0.15161 + \left( \frac{1} {1+5\%} \right)^{20} \cdot \frac{99033.94} {99865.69} \right) \\
&= \$198258.08
\end{align}$

($\text{d}$)

$\begin{align}
S_1\cdot A_{[25]:\overline{10}|}^{\;1} + S_2\cdot {_{10|}}A_{[25]}&= S_1 \left( A_{[25]}- {_{10|}}A_{[25]} \right) + S_2\cdot \upsilon^{10} \cdot {_{10}}p_{25} \cdot A_{[25]+10} \\
&= S_1 \left( A_{[25]}-\left( \frac{1} {1+i} \right)^{10} \cdot \frac{l_{[25]+10}} {l_{[25]}} \cdot A_{35} \right) +  S_2\cdot\left( \frac{1} {1+i} \right) ^{10} \cdot \frac{l_{[25]+10}} {l_{[25]}} \cdot A_{35} \\
&= \$ 60000 \left( 0.06143 -\left( \frac{1} {1+5\%} \right)^{10} \cdot \frac{99556.75} {99865.69} \cdot 0.09653 \right) \\
&\;\;\;\;+ \$ 30000 \cdot\left( \frac{1} {1+5\%} \right) ^{10} \cdot \frac{99556.75} {99865.69} \cdot 0.09653 \\
&= \$1913.4684
\end{align} $

($\text{e}$)

$\begin{align}
S\cdot \sqrt{ {^{2}}A_{[25]} - \left( A_{[25]}\right)^2} &= \$200000 \cdot \sqrt{0.00783 - 0.06143^2} \\
&= \$12737.90422
\end{align}$

$$\\[2em]$$

## Question 2
($\text{a}$)

- If he dies in age $40.06$, then $T_{[40]} = 40.06 - 40 = 0.06$. And $K_{[40]}^{(12)} = \ffrac{1} {12}\left \lfloor 12 \times 0.06 \right \rfloor = 0$. The PV of the death benefit is

$$\begin{align}
\text{PV} &= S \cdot \frac{1} {\left( 1 + i\right)^{(1/12)+0}}\\
&= \$50000 \times \frac{1} {\left( 1 + 4.5\%\right)^{(1/12)}} \\
&= \$ 49816.93
\end{align}$$

- If he dies in age $55.98$, then $T_{[40]} = 55.98 - 40 = 15.98$. And $K_{[40]}^{(12)} = \ffrac{1} {12}\left \lfloor 12 \times 15.98 \right \rfloor = 15\ffrac{11} {12}$. The PV of the death benefit is

$$\begin{align}
\text{PV} &= S \cdot \frac{1} {\left( 1 + i\right)^{(1/12)+(15+11/12)}}\\
&= \$50000 \times \frac{1} {\left( 1 + 4.5\%\right)^{16}} \\
&= \$ 24723.47
\end{align}$$

- If he dies in age $97.43$, then $T_{[40]} = 97.43 - 40 = 57.43$. And $K_{[40]}^{(12)} = \ffrac{1} {12}\left \lfloor 12 \times 57.43 \right \rfloor = 57\ffrac{5} {12}$. The PV of the death benefit is

$$\begin{align}
\text{PV} &= S \cdot \frac{1} {\left( 1 + i\right)^{(1/12)+(57+5/12)}}\\
&= \$50000 \times \frac{1} {\left( 1 + 4.5\%\right)^{57.5}} \\
&= \$ 3979.09
\end{align}$$

($\text{b}$)

First we solve the inequlity for $K_{[40]}^{(12)}$, and we denote the power on the discount factor as $L$.

$$\text{PV} = \$50000 \times \frac{1} {\left( 1 + 4.5\%\right)^{L}} > \$20000,$$

so that $L = \ffrac{1} {12} + K_{[40]}^{(12)}< 20.81680$. Then we need to know the equivalent requirement for $T_{[40]}$.

$$K_{[40]}^{(12)} = \ffrac{1} {12}\left \lfloor 12 \times \left( T_{[40]}\right) \right \rfloor < 20.81680 - \frac{1} {12}$$

Solveing this we have the restriction required that is $T_{[40]} < 20 \ffrac{9} {12}$. So that we have

$$\begin{align}
\Pr\left[ \text{PV}>\$20000 \right] &= \Pr\left[ T_{[40]} < 20 \frac{9} {12}  \right] \\
&= 1 - {_{20 \frac{9} {12}}}p_{[40]} \\
&\stackrel{\text{UDD}}= 1 - {_{20}}p_{[40]} \cdot \left( 1 - \frac{9} {12} q_{[40]+20} \right) \\
&= 1 - \frac{l_{[40]+20}} {l_{[40]}} \times \left( 1 - \frac{9} {12} \times \frac{l_{[40]+20} - l_{[40]+21}} {l_{[40]+20}} \right) \\
&= 1 - \frac{96634.14} {99327.82} \times \left( 1 - \frac{9} {12} \times \frac{96634.14 - 96305.75} {96634.14} \right) \\
&\approx 0.029599
\end{align}$$

$$\\[50em]$$

$$\\[1em]$$
## Question 3
($\text{a}$)

$$\begin{align}
{_{10|5}}q_{65} &= {_{10}}p_{65} - {_{15}}p_{65} \\
&= \sqrt{\frac{1} {10}} - \sqrt{\frac{1} {15}} \\
&= 0.058029
\end{align}$$

($\text{b}$)

Since the payment could only happen in two times, that is when she is $85$ years old, or $100$ years old, there're only three possible value for $Z$.
1. Dies before age $85$, $Z = \$0$
2. Dies after age $85$ before $100$, $Z = \ffrac{\$500000} {(1.08)^{20}} = \$107274.10$
3. Dies after age $100$, $Z = \ffrac{\$500000} {(1.08)^{20}} + \ffrac{\$1000000} {(1.08)^{35}} = \$174908.65$

And for each of $Z$ value we have$$\DeclareMathOperator*{\Pr}{Pr}$$
1. $\Pr[Z=0] = {_{20}}q_{65} = 1 - {_{20}}p_{65} = 1 - \sqrt{\ffrac{1} {20}}\approx 0.7764$
2. $\Pr[Z = 107274.10] = {_{20|15}}q_{65} = {_{20}}p_{65} - {_{35}}p_{65} = \sqrt{\ffrac{1} {20}} - \sqrt{\ffrac{1} {35}} \approx 0.0546$
3. $\Pr[Z = 174908.65] = {_{35}}p_{65} = \sqrt{\ffrac{1} {35}} \approx 0.1690$

So to sum up, we have the following table

$$\begin{array}{c:ccc}\hline
z & 0 & 107274.10 & 174908.65 \\ \hline
\Pr[Z = z] & 0.7764 & 0.0546 & 0.1690\\ \hline
\end{array}$$

($\text{c}$)

Easily, we can find the probability that the PV of the benefits is less than $\$100000$ from the table is under the condition that $Z = \$0$, so that the value is $0.7764$.

($\text{d}$)

$$\begin{align}
\text{EPV} &= \sum_{z} z \cdot \Pr[Z = z] \\
&= 0 \times 0.7764 + 107274.10 \times 0.0546 + 174908.65 \times 0.1690 \\
&= 35416.72602
\end{align}$$

($\text{e}$)

$$\begin{align}
\sigma_Z &= \sqrt{ \sum_{z} \Pr[Z = z] \cdot \left( z - \text{EPV} \right)^2} \\
&= \sqrt{ 0.7764 \times \left( 0 - 35416.72 \right)^2 + 0.0546 \times \left( 107274.10 - 35416.72 \right)^2 + 0.1690 \times \left( 174908.65 - 35416.72 \right)^2}\\
&= 67410.68299
\end{align}$$

($\text{f}$)

$$\\[42em]$$

$$\\[1em]$$
## Question 4

($\text{a}$)

$$\begin{align}
\text{EPV} &= S \cdot A_{30:\overline{3}|}^{\, 1} \\
&= S \cdot \sum_{k=0}^{3-1} \upsilon^{k+1} \cdot {_{k|}}q_{30} \\
&= S \cdot \left( \upsilon^{1} \cdot \frac{l_{30} - l_{31}} {l_{30}} + \upsilon^{2} \cdot \frac{l_{31} - l_{32}} {l_{30}} + \upsilon^{3} \cdot \frac{l_{32} - l_{33}} {l_{30}} \right) \\
&= S \cdot \left( \left( \frac{1} {1+i} \right)^{1} \cdot \frac{d_{30}} {l_{30}} + \left( \frac{1} {1+i} \right)^{2} \cdot \frac{d_{31}} {l_{30}}+\left( \frac{1} {1+i} \right)^{3} \cdot \frac{d_{32}} {l_{30}} \right) \\
&= \$300000 \cdot \left( \left( \frac{1} {1+10\%} \right)^{1} \cdot \frac{34.78} {10000.00} + \left( \frac{1} {1+10\%} \right)^{2} \cdot \frac{38.10} {10000.00} + \left( \frac{1} {1+10\%} \right)^{3} \cdot \frac{41.76} {10000.00} \right) \\
&= \$ 2834.42
\end{align}$$

($\text{b}$)

1. Using the $\text{UDD}$ method, since $1+i = \left( 1 + \frac{i^{(12)}} {12}\right)^{12}$,
$$i^{(12)} = \left(\sqrt[12]{1+i}-1\right) \times 12 = \left(\sqrt[12]{1+10\%}-1\right) \times 12 \approx 9.57\% \\[1em]
\text{EPV} = S \cdot A^{(12)}{_{30:\overline{3}|}^{\, 1}} \stackrel{\text{UDD}} = S \cdot \frac{i} {i^{(12)}} A_{30:\overline{3}|}^{\, 1} = \$ 300000 \times \frac{10\%} {9.57\%}\times \frac{\$ 2834.42} {\$300000} =  \$2962.096$$

2. Using the $\text{CA}$ method
$$\text{EPV} = S \cdot A^{(12)} {_{30:\overline{3}|}^{\, 1}} \stackrel{\text{CA}}= S \cdot \left(1+i\,\right)^{\frac{12-1} {2\times12}}\; A_{30:\overline{3}|} = \$300000 \times \left(1+10\%\,\right)^{\frac{12-1} {2\times12}}\;\times \frac{\$ 2834.42} {\$300000} =  \$2960.983$$

($\text{c}$)

1. Using the $\text{UDD}$ method
$$\text{EPV} = S \cdot \bar{A}_{30:\overline{3}|}^{\, 1} \stackrel{\text{UDD}}= S \cdot \frac{i} {\delta} A_{30:\overline{3}|}^{\, 1} = \$ 300000 \times \frac{10\%} {\log(1+10\%)}\times \frac{\$ 2834.42} {\$300000} =  \$2973.890$$

2. Using the $\text{CA}$ method
$$\text{EPV} = S \cdot \bar{A}_{30:\overline{3}|}^{\, 1} \stackrel{\text{CA}}= S \cdot \left(1+i\,\right)^{0.5} A_{30:\overline{3}|}^{\, 1} = \$300000 \times \left(1+10\%\,\right)^{0.5}\;\times \frac{\$ 2834.42} {\$300000} =  \$2972.765$$

($\text{d}$)

$$
\begin{align}
A_{32} = \upsilon \cdot q_{32} + \upsilon \cdot p_{32} \cdot A_{33} = \frac{1} {1+i} \cdot \frac{d_{32}} {l_{32}} + \frac{1} {1+i} \cdot \frac{l_{33}} {l_{32}} \cdot A_{33} = 0.5469836898589647 \\[0.9em]
A_{31} = \upsilon \cdot q_{31} + \upsilon \cdot p_{31} \cdot A_{32} = \frac{1} {1+i} \cdot \frac{d_{31}} {l_{31}} + \frac{1} {1+i} \cdot \frac{l_{32}} {l_{31}} \cdot A_{32} = 0.4988324599568871 \\[0.9em]
A_{30} = \upsilon \cdot q_{30} + \upsilon \cdot p_{30} \cdot A_{31} = \frac{1} {1+i} \cdot \frac{d_{30}} {l_{30}} + \frac{1} {1+i} \cdot \frac{l_{31}} {l_{30}} \cdot A_{31} = 0.4550686551465064
\end{align}$$

($\text{e}$)

$$
\begin{align}
\text{EPV} &= S \cdot A_{32:\overline{2}|} \\
&= S \cdot \left( A_{32:\overline{2}|}^{\, 1} + {_{2}}E_{32} \right) \\
&= S \cdot \left( A_{32} - {_{2}}E_{32} \cdot A_{32+2} + \upsilon^{2} \cdot {_{2}}p_{32} \right) \\
&= S \cdot \left( \upsilon \cdot q_{32} + \upsilon^2 {_{1|}}q_{32} + \upsilon^{2} \cdot {_{2}}p_{32} \right) \\
&= S \cdot \left( \upsilon \cdot q_{32} + \upsilon^2 \left( p_{32} - {_{2}}p_{32}  \right) + \upsilon^{2} \cdot {_{2}}p_{32} \right) \\
&= S \cdot \left( \upsilon \cdot q_{32} + \upsilon^2 \cdot p_{32} \right) \\
&= S \cdot \left( \frac{1} {1+i} \cdot \frac{l_{32} - l_{33}} {l_{32}} + \left( \frac{1} {1+i} \right)^2 \cdot \frac{l_{33}} {l_{32}} \right) \\
&= \$25000 \cdot \left( \frac{1} {1+10\%} \cdot \frac{9927.12 - 9885.35} {9839.55} + \left( \frac{1} {1+10\%} \right)^2 \cdot \frac{9885.35} {9927.12} \right) \\
&= \$20669.82765435818
\end{align}
$$

($\text{f}$)

Before continuing, we need to separately calculate the $A_{32:\overline{2}|}^{\, 1}$.

$$\begin{align}
A_{32:\overline{2}|}^{\, 1} &= A_{32} - {_{2}}E_{32} A_{34} \\
&= \upsilon \cdot q_{32} + \upsilon^2 \left( p_{32} - {_{2}}p_{32}  \right) \\
&= \frac{1} {1+i} \cdot \frac{l_{32} - l_{33}} {l_{32}} + \left( \frac{1} {1+i} \right)^2 \cdot \frac{l_{33} - l_{34}} {l_{32}}\\
&= \frac{1} {1+10\%} \cdot \frac{9927.12 - 9885.35} {9927.12} + \left( \frac{1} {1+10\%} \right)^2 \cdot \frac{9885.35 - 9839.55} {9927.12} \\
&= 0.007637979645241294
\end{align}
$$

And the ${_{2}}E_{32} = \upsilon^{2} \cdot {_{2}}p_{32} = 0.8191559590427263$

1. Using the $\text{UDD}$ method, since $1+i = \left( 1 + \frac{i^{(4)}} {4}\right)^{4}$, $\\[1em]$
$$
i^{(4)} = \left(\sqrt[4]{1+i}-1\right) \times 4 = \left(\sqrt[4]{1+10\%}-1\right) \times 4 \approx 9.645475633778045\% \\[1em]
\begin{align}
\text{EPV} &= S \cdot A^{(4)}{_{32:\overline{2}\displaystyle{|}}} \\[0.8em]
&\stackrel{\text{UDD}} = S \cdot \left( \frac{i} {i^{(4)}}A_{32:\overline{2} \displaystyle{|}}^{\, 1} + {_{2}}E_{32} \right)\\[0.8em]
&= \$ 25000 \times \left( \frac{10\%} {9.66\%}\times 0.007637979645241294 +  0.8191559590427263 \right) \\
&= \$20676.866912934158
\end{align}\\[1em]
$$

2. Using the $\text{CA}$ method, $\\[1em]$
$$
\begin{align}
\text{EPV} &= S \cdot A^{(4)} {_{32:\overline{2}\displaystyle{|}}} \\[0.8em]
&\stackrel{\text{CA}} = S \cdot \left( \left(1+i\,\right)^{ \frac{4-1} {2 \times 4} }\; A_{32:\overline{2}\displaystyle{|}}^{\, 1} + {_{2}}E_{32} \right) \\[0.8em]
&= \$25000 \times \left( \left(1+10\%\,\right)^{\ffrac{4-1} {2\times4}} \; \times 0.007637979645241294 +  0.8191559590427263 \right) \\
&= \$20676.796683123583
\end{align}\\[1em]
$$

($\text{g}$)

1. Using the $\text{UDD}$ method, $\\[1em]$
$$
\begin{align}
\text{EPV} &= S \cdot \bar{A}_{32:\overline{2}\displaystyle{|}} \\
&\stackrel{\text{UDD}}= S \cdot \left( \frac{i} {\delta} A_{32:\overline{2}\displaystyle{|}}^{\, 1} + {_{2}}E_{32} \right) \\
&= \$ 25000 \times \left(  \frac{10\%} {\log(1+10\%)}\times 0.007637979645241294 +  0.8191559590427263 \right) \\
&= \$20679.244302793024
\end{align}\\[1em]
$$

2. Using the $\text{CA}$ method, $\\[1em]$
$$
\begin{align}
\text{EPV} &= S \cdot \bar{A}_{32:\overline{2}\displaystyle{|}} \\
&\stackrel{\text{CA}}= S \cdot \left( \left(1+i\,\right)^{0.5} A_{32:\overline{2}\displaystyle{|}}^{\, 1} + {_{2}}E_{32} \right) \\
&= \$25000 \times \left( \left(1+10\%\,\right)^{0.5}\;\times 0.007637979645241294 +  0.8191559590427263 \right) \\
&= \$20679.168491919972
\end{align}\\[1em]
$$

$$\\[42em]$$

$$\\[2em]$$

$$\\[1em]$$
## Question 5

($\text{a}$)

$${_{t}}p_{x} = \exp \left\{ - \int_{0}^{t} \mu_{x+s} \; \mathrm{d}s \right\} = \exp \left\{ -\int_{0}^{t} 0.03 \; \mathrm{d}s \right\} = e^{-0.03 t}$$

($\text{b}$)

$${_{20|10}}q_{50} = {_{20}}p_{50} - {_{30}}p_{50} = e^{-0.03 \times 20} - e^{-0.03 \times 30} = 0.14224$$

($\text{c}$)

$$\begin{align}
\text{EPV} &= S \cdot \bar{A}_{50:\overline{15}|}^{\, 1} \\
&= S \cdot \left( \int_{0}^{15} e^{-\delta t} \cdot {_{t}}p_{50} \cdot \mu_{x+t} \; \mathrm{d}t \right) \\
&= S \cdot \left( \int_{0}^{15} 1.07^{-t} \cdot e^{-0.03 t} \cdot 0.03 \; \mathrm{d}t \right) \\
&= 0.03 \times S \cdot \left( \int_{0}^{15}  \left( 1.07 \cdot e^{0.03}\right)^{-t} \; \mathrm{d}t \right) \\
&= 0.03 \times 100000 \cdot \left( - \left. \frac{ \left( 1.07 \cdot e^{0.03} \right)^{-t}} {\log \left( 1.07 \cdot e^{0.03}\right)}\right|_{0}^{15} \right) \\
&= 23619.85
\end{align}$$


# 결합, 주변, 조건부 확률 밀도 함수

두 개 이상의 확률 변수가 서로 관계를 가지며 존재하는 경우를 생각해 보자. 예를 들어 학교에 있는 학생의 키와 몸무게를 측정하는 경우 한 명의 학생 $\omega$에 대해 두 개의 자료 ($x$, $y$)가 한 쌍으로 나오게 된다. 이렇게 취득한 자료를 확률 변수 $X$와 $Y$로 볼 때, 이들의 확률 분포를 한번에 묘사하기 위한 확률 분포를 **결합 확률 분포(joint probability distribution)**라고 한다.

결합 확률 분포도 단일 연속 확률 변수의 경우와 마찬가지로 누적 확률 분포 함수(cumulative probability function)와 확률 밀도 함수(probability density function)를 통해 서술된다.

## 연속 확률 변수의 결합 누적  확률 분포 함수

두 확률 변수 $X$, $Y$에 대한 누적 확률 분포 함수 $F_{XY}(x, y) $는 다음과 같이 정의한다.

$$ F_{XY}(x, y) = P(\{ X < x \} \cap \{ Y < y \}) = P(X < x, Y < y) $$

만약 구간의 끝을 나타내는 두 독립 변수 $x$, $y$중 하나가 무한대 값을 가지는 경우에는 해당 변수의 값은 어떤 값을 가져도 상관없으므로 남은 하나의 변수에 대한 누적 확률 분포 함수로 줄어든다. 이를 **주변 확률 분포(marginal probability distribution)**이라고 한다.

$$ F_X(x)=F_{XY}(x, \infty) $$

$$ F_Y(x)=F_{XY}(\infty, y) $$

누적 확률 분포 함수 $F_{XY}(x, y) $는 다음과 같은 특성을 가진다.

$$ F_{XY}(\infty, \infty)=1 $$

$$ F_{XY}(-\infty, y)=F_{XY}(x,-\infty)=0 $$

## 연속 확률 변수의 결합 확률 밀도 함수

단일 확률 변수의 경우처럼 누적 결합 확률 분포 함수를 미분하여 결합 확률 밀도 함수를 정의할 수 있다. 다만 이 경우에는 독립 변수가 2개이므로 각각에 대해 모두 편미분(partial differentication)해야 한다.

$$ f_{XY} = \dfrac{\partial^2 F_{XY}(x, y)}{\partial x \partial y} $$

결합 확률 밀도 함수를 특정 구간에 대해 적분하면 해당 구간에 대한 확률이 된다.

$$  \int_{x_1}^{x_2} \int_{y_1}^{y_2} f_{XY}(x,y)dxdy = P\big(\{ x_1 \leq X \leq x_2, \; y_1 \leq Y \leq y_2 \}\big)  $$

따라서 결합 확률 밀도 함수를 모든 변수에 대해 $-\infty$에서 $\infty$ 까지 적분하면 값이 1이 된다.

$$  \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f_{XY}(x,y)dxdy=1 $$

연속 확률 변수의 결합 확률 밀도 함수는 2차원 함수가 된다. 아래는 다변수 정규 분포의 결합 확률 밀도의 예를 그린 것이다.

```python
mu = [0, 0]
cov = [[2, -1],[-1, 18]]
rv = sp.stats.multivariate_normal(mu, cov)
xx = np.linspace(-5, 5, 100)
yy = np.linspace(-5, 5, 100)
XX, YY = np.meshgrid(xx, yy)
ZZ = rv.pdf(np.dstack([XX, YY]))
plt.contour(XX, YY, ZZ)
plt.xlabel("x")
plt.ylabel("y")
plt.title("Joint Probability Density")
plt.axis("equal")
plt.show()
```

동일한 결합 확률 밀도 함수를 3차원으로 그리면 아래와 같다.

```python
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = Axes3D(fig)
ax.contour(XX, YY, ZZ)
ax.set_xlabel("x")
ax.set_ylabel("y")
ax.set_title("Joint Probability Density")
ax.view_init(50, -80)
plt.show()
```

## 이산 확률 변수의 결합 확률 질량 함수

다변수 이산 확률 변수에 대해서는 결합 확률 질량 함수를 바로 구할 수 있다. 결합 확률 질량 함수는 모든 확률 변수의 값이 특정 숫자가 될 확률을 듯한다.

$$ f_{XY}(x,y) = P(X=x,Y=y) $$

결합 확률 질량 함수는 댜음과 같이 2차원 표의 형태가 된다.

```python
pmf = np.array([[0, 0, 0, 0, 1, 1],
                [0, 0, 1, 2, 1, 0],
                [0, 1, 3, 3, 1, 0],
                [0, 1, 2, 1, 0, 0],
                [1, 1, 0, 0, 0, 0]])
pmf = pmf/pmf.sum()
pmf
```

    array([[ 0.  ,  0.  ,  0.  ,  0.  ,  0.05,  0.05],
           [ 0.  ,  0.  ,  0.05,  0.1 ,  0.05,  0.  ],
           [ 0.  ,  0.05,  0.15,  0.15,  0.05,  0.  ],
           [ 0.  ,  0.05,  0.1 ,  0.05,  0.  ,  0.  ],
           [ 0.05,  0.05,  0.  ,  0.  ,  0.  ,  0.  ]])

```python
sns.heatmap(pmf)
plt.xlabel("x")
plt.ylabel("y")
plt.title("Joint Probability Mass Function")
plt.show()
```

## 주변 확률 밀도 함수

**주변 확률 밀도 함수(marginal probability density function)**는 다변수 확률 밀도 함수를 특정한 하나의 변수에 대해 평균한 값을 말한다. 따라서 결합 확률 밀도 함수를 특정 변수에 대해서만 적분하여 구한다.

평균 연산(적분)으로 인해 차원이 한 개 줄어들기 때문에 2차원 확률 변수의 주변 확률 밀도 함수는 1차원 함수가 된다.

$$
\begin{align}%\label{}
\nonumber  f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x,y)dy \\
\nonumber  f_Y(y) = \int_{-\infty}^{\infty} f_{XY}(x,y)dx
\end{align}
$$

이산 확률 변수의 경우에는 **주변 확률 질량 함수(marginal probability mass function)** 를 다음과 같이 특정한 하나의 변수에 대해 총합을 구하여 정의한다

$$
\begin{align}%\label{}
\nonumber  f_X(x) = \sum_{y_j} f_{XY}(x,y_j) \\
\nonumber  f_Y(y) = \sum_{x_i} f_{XY}(x_i,y) \\
\end{align}
$$

위에서 예로 든 이산 확률 변수의 경우에 주변 확률 질량 함수를 계산하면 다음과 같다.

```python
pmf
```

    array([[ 0.  ,  0.  ,  0.  ,  0.  ,  0.05,  0.05],
           [ 0.  ,  0.  ,  0.05,  0.1 ,  0.05,  0.  ],
           [ 0.  ,  0.05,  0.15,  0.15,  0.05,  0.  ],
           [ 0.  ,  0.05,  0.1 ,  0.05,  0.  ,  0.  ],
           [ 0.05,  0.05,  0.  ,  0.  ,  0.  ,  0.  ]])

```python
pmf_marginal_x = pmf.sum(axis=0)
pmf_marginal_x
```

    array([ 0.05,  0.15,  0.3 ,  0.3 ,  0.15,  0.05])

```python
pmf_marginal_y = pmf.sum(axis=1)
pmf_marginal_y[:, np.newaxis]
```

    array([[ 0.1],
           [ 0.2],
           [ 0.4],
           [ 0.2],
           [ 0.1]])

위에서 예로 든 연속 확률 변수의 경우에 주변 확률 밀도 함수를 계산하면 다음과 같다.

```python
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.contour(XX, YY, ZZ, levels=np.linspace(0, 0.04, 30), alpha=0.3)
ax.plot(yy, ZZ.mean(axis=1), zdir='x', lw=3)
ax.plot(xx, ZZ.mean(axis=0), zdir='y', lw=3)
ax.set_xlabel("x")
ax.set_ylabel("y")
ax.set_title("Marginal Probability Density")
ax.view_init(50, -80)
plt.show()
```

## 조건부 확률 밀도 함수

조건부 확률 밀도 함수(conditional probability density function)는 다변수 확률 변수 중 하나의 값이 특정 값으로 고정되어 상수가 되어 버린 경우, 나머지 변수에 대한 확률 밀도 함수를 말한다.

$$ f_{X \mid Y}(x \mid y_0) = \dfrac{f_{XY}(x, y=y_0)}{f_{Y}(y_0)} $$

$$ f_{Y \mid X}(y \mid x_0) = \dfrac{f_{XY}(x, y=y_0)}{f_{X}(x_0)} $$

주변 확률 밀도 함수와 마찬가지로 차원이 감소하지만 평균 연산(적분)으로 변수가 없어진 주변 확률 밀도 함수와는 값이 달라진다.

이산 확률 변수의 경우에는 조건부 확률 질량 함수를 사용하여 정의한다.

위에서 예로 든 이산 확률 변수의 경우에 조건부 확률 질량 함수를 계산하면 다음과 같다.

```python
pmf
```

    array([[ 0.  ,  0.  ,  0.  ,  0.  ,  0.05,  0.05],
           [ 0.  ,  0.  ,  0.05,  0.1 ,  0.05,  0.  ],
           [ 0.  ,  0.05,  0.15,  0.15,  0.05,  0.  ],
           [ 0.  ,  0.05,  0.1 ,  0.05,  0.  ,  0.  ],
           [ 0.05,  0.05,  0.  ,  0.  ,  0.  ,  0.  ]])

```python
cond_y0 = pmf[0, :]/pmf_marginal_y[0]
cond_y0
```

    array([ 0. ,  0. ,  0. ,  0. ,  0.5,  0.5])

```python
cond_y1 = pmf[1, :]/pmf_marginal_y[1]
cond_y1
```

    array([ 0.  ,  0.  ,  0.25,  0.5 ,  0.25,  0.  ])

11. 기초 확률론4 - 상관 관계/01. 결합, 주변, 조건부 확률 밀도 함수.ipynb


# Exercises

## Problem 11: Define nodes and elements

Consider a domain $\Omega =[0,2]$ divided into the three elements
$[0,1]$, $[1,1.2]$, and $[1.2,2]$.

For P1 and P2 elements, set up the list of coordinates and nodes
(`nodes`) and the numbers of the nodes that belong to each element
(`elements`) in two cases: 1) nodes and elements numbered from left to
right, and 2) nodes and elements numbered from right to left.

**Solution.**
We can write up figure sketches and the data structure in code:

```python
# P1 elements
# Left to right numbering
"""
elements: |--0--|--1--|--2--|
nodes:    0     1     2     3
"""

nodes    = [0, 1, 1.2, 2]
elements = [[0,1], [1,2], [2,3]]

# Right to left numbering
"""
elements: |--2--|--1--|--0--|
nodes:    3     2     1     0
"""

nodes    = [2, 1.2, 1, 0]
elements = [[1,0], [2,1], [3,2]]

# P2 elements

# Left to right numbering
"""
elements: |--0--|--1--|--2--|
nodes:    0  1  2  3  4  5  6
"""

nodes = [0, 0.5, 1, 1.1, 1.6, 2]
elements = [[0,1,2], [2,3,4], [4,5,6]]

# Right to left numbering
"""
elements: |--2--|--1--|--0--|
nodes:    6  5  4  3  2  1  0
"""

nodes = [2, 1.6, 1.2, 1.1, 1, 0.5, 0]
elements = [[2,1,0], [4,3,2], [6,5,4]]
```

Filename: `fe_numberings1`.

## Problem 12: Define vertices, cells, and dof maps

Repeat [Problem 11: Define nodes and elements](#fem:approx:fe:exer:mesh1), but define the
data structures `vertices`, `cells`, and `dof_map` instead of
`nodes` and `elements`.

**Solution.**
Written in Python, the solution becomes

```python
# P1 elements
# Left to right numbering
"""
elements: |--0--|--1--|--2--|
vertices: 0     1     2     3
dofs:     0     1     2     3
"""
# elements:   0   1   2
# vertices: 0   1   2   3

vertices = [0, 1, 1.2, 2]
cells    = [[0,1], [1,2], [2,3]]
dof_map  = [[0,1], [1,2], [2,3]]

# Right to left numbering
"""
elements: |--2--|--1--|--0--|
vertices: 3     2     1     0
dofs:     3     2     1     0
"""

vertices = [2, 1.2, 1, 0]
cells    = [[1,0], [2,1], [3,2]]
dof_map  = [[1,0], [2,1], [3,2]]

# P2 elements

# Left to right numbering
# elements:   0   1   2
"""
elements: |--0--|--1--|--2--|
vertices: 0     1     2     3
dofs:     0  1  2  3  4  5  6
"""

vertices = [0, 1, 1.2, 2]
cells    = [[0,1], [1,2], [2,3]]
dof_map  = [[0,1,2], [2,3,4], [4,5,6]]

# Right to left numbering
# elements:   2   1   0
"""
elements: |--2--|--1--|--0--|
vertices: 3     2     1     0
dofs:     6  5  4  3  2  1  0
"""

vertices = [2, 1.2, 1, 0]
cells    = [[1,0], [2,1], [3,2]]
dof_map  = [[2,1,0], [4,3,2], [6,5,4]]
```

Filename: `fe_numberings2`.

## Problem 13: Construct matrix sparsity patterns

[Problem 11: Define nodes and elements](#fem:approx:fe:exer:mesh1) describes a element mesh
with a total of five elements, but with two different element and
node orderings. For each of the two orderings,
make a $5\times 5$ matrix and fill in the entries that will be nonzero.

**Hint.**
A matrix entry $(i,j)$ is nonzero if $i$ and $j$ are nodes in the
same element.

**Solution.**
If we create an empty matrix, we can run through all elements and
then over all local node pairs and mark that the corresponding
entry $(i,j)$ in the global matrix is a nonzero entry.
The `elements` data structure is sufficient. Below is a program
that fills matrix entries with an `X` and prints the matrix sparsity
pattern.

```python

def sparsity_pattern(elements, N_n):
    import numpy as np
    matrix = np.zeros((N_n, N_n), dtype=str)
    matrix[:,:] = '0'
    for e in elements:
        for i in e:
            for j in e:
                matrix[i,j] = 'X'
    matrix = matrix.tolist()
    matrix = '\n'.join([' '.join([matrix[i][j]
                                  for j in range(len(matrix[i]))])
                        for i in range(len(matrix))])
    return matrix

print('\nP1 elements, left-to-right numbering')
N_n = 4
elements = [[0,1], [1,2], [2,3]]
print((sparsity_pattern(elements, N_n)))

print('\nP1 elements, right-to-left numbering')
elements = [[1,0], [2,1], [3,2]]
print((sparsity_pattern(elements, N_n)))

print('\nP2 elements, left-to-right numbering')
N_n = 7
elements = [[0,1,2], [2,3,4], [4,5,6]]
print((sparsity_pattern(elements, N_n)))

print('\nP1 elements, right-to-left numbering')
elements = [[2,1,0], [4,3,2], [6,5,4]]
print((sparsity_pattern(elements, N_n)))
```

The output becomes

        P1 elements, left-to-right numbering
        X X 0 0
        X X X 0
        0 X X X
        0 0 X X

        P1 elements, right-to-left numbering
        X X 0 0
        X X X 0
        0 X X X
        0 0 X X

        P2 elements, left-to-right numbering
        X X X 0 0 0 0
        X X X 0 0 0 0
        X X X X X 0 0
        0 0 X X X 0 0
        0 0 X X X X X
        0 0 0 0 X X X
        0 0 0 0 X X X

        P1 elements, right-to-left numbering
        X X X 0 0 0 0
        X X X 0 0 0 0
        X X X X X 0 0
        0 0 X X X 0 0
        0 0 X X X X X
        0 0 0 0 X X X
        0 0 0 0 X X X

Filename: `fe_sparsity_pattern`.

## Problem 14: Perform symbolic finite element computations

Perform symbolic calculations to find formulas for the coefficient
matrix and right-hand side when approximating $f(x) = \sin (x)$ on
$\Omega=[0, \pi]$ by two P1 elements of size $\pi/2$.  Solve the
system and compare $u(\pi/2)$ with the exact value 1.

**Solution.**
Here are suitable `sympy` commands:

```python
import sympy as sym
# Mesh: |--------|-------|
#       0      pi/2      pi
#
# Basis functions:
#
#   phi_0   phi_1   phi_2
#     \      /\      /
#      \    /  \    /
#       \  /    \  /
#        \/      \/
#     |-------|-------|
#     0      pi/2     pi

x = sym.Symbol('x')
A = sym.zeros(3,3)
f = sym.sin

phi_0 = 1 - (2*x)/sym.pi
phi_1l = 2*x/sym.pi          # left part of phi_1
phi_1r = 2 - (2*x)/sym.pi    # right part of phi_1
phi_2 = x/(sym.pi/2) - 1
node_0 = 0
node_1 = sym.pi/2
node_2 = sym.pi

# Diagonal terms
A[0,0] = sym.integrate(phi_0**2,  (x, node_0, node_1))
A[1,1] = sym.integrate(phi_1l**2, (x, node_0, node_1)) + \
         sym.integrate(phi_1r**2, (x, node_1, node_2))
A[2,2] = sym.integrate(phi_2**2,  (x, node_1, node_2))

# Off-diagonal terms
A[0,1] = sym.integrate(phi_0*phi_1l, (x, node_0, node_1))
A[1,0] = A[0,1]

A[1,2] = sym.integrate(phi_1r*phi_2, (x, node_1, node_2))
A[2,1] = A[1,2]

print(('A:\n', A))  # Can compare with general matrix, h=pi/2

b = sym.zeros(3,1)

b[0] = sym.integrate(phi_0*f(x),  (x, node_0, node_1))
b[1] = sym.integrate(phi_1l*f(x), (x, node_0, node_1)) + \
       sym.integrate(phi_1r*f(x), (x, node_1, node_2))
b[2] = sym.integrate(phi_2*f(x),  (x, node_1, node_2))

print(('b:\n', b))

c = A.LUsolve(b)
print(('c:\n', c))

for i in range(len(c)):
    print(('c[%d]=%g' % (i, c[i].evalf())))
print(('u(pi/2)=%g' % c[1]))

# For reports
print((sym.latex(A)))
print((sym.latex(b)))
print((sym.latex(c)))
```

Running the program, we get the matrix system

$$
\left[\begin{matrix}\frac{\pi}{6} & \frac{\pi}{12} & 0\\\frac{\pi}{12} & \frac{\pi}{3} & \frac{\pi}{12}\\0 & \frac{\pi}{12} & \frac{\pi}{6}\end{matrix}\right]
\left[\begin{matrix}\frac{1}{\pi} \left(- \frac{24}{\pi} + 8\right)\\\frac{-28 + \frac{168}{\pi}}{7 \pi}\\\frac{1}{\pi} \left(- \frac{24}{\pi} + 8\right)\end{matrix}\right]
=
\left[\begin{matrix}- \frac{2}{\pi} + 1\\\frac{4}{\pi}\\- \frac{2}{\pi} + 1\end{matrix}\right]
$$

The solution at the midpoint is $1.15847$, i.e., 16% error.

Filename: `fe_sin_P1`.

## Problem 15: Approximate a steep function by P1 and P2 elements

Given

$$
f(x) = \tanh(s(x-\frac{1}{2}))
$$

use the Galerkin or least squares method with finite elements to find
an approximate function $u(x)$. Choose $s=20$ and try
$N_e=4,8,16$ P1 elements and
$N_e=2,4,8$ P2 elements.
Integrate $f{\varphi}_i$ numerically.

**Hint.**
You can automate the computations by calling the `approximate` method
in the `fe_approx1D_numint` module.

**Solution.**
The set of calls to `approximate` becomes

```python
from fe_approx1D_numint import approximate
from sympy import tanh, Symbol
x = Symbol('x')

steepness = 20
arg = steepness*(x-0.5)
approximate(tanh(arg), symbolic=False, numint='GaussLegendre2',
            d=1, N_e=4, filename='fe_p1_tanh_4e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre2',
            d=1, N_e=8, filename='fe_p1_tanh_8e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre2',
            d=1, N_e=16, filename='fe_p1_tanh_16e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre3',
            d=2, N_e=2, filename='fe_p2_tanh_2e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre3',
            d=2, N_e=4, filename='fe_p2_tanh_4e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre3',
            d=2, N_e=8, filename='fe_p2_tanh_8e')
```

Filename: `fe_tanh_P1P2`.

## Problem 16: Approximate a steep function by P3 and P4 elements

**a)**
Solve [Problem 15: Approximate a steep function by P1 and P2 elements](#fem:approx:exer:tanh:P1P2) using $N_e=1,2,4$ P3 and P4
elements.

**Solution.**
We can easily adopt the code from [Problem 15: Approximate a steep function by P1 and P2 elements](#fem:approx:exer:tanh:P1P2):

```python
from fe_approx1D_numint import approximate, u_glob
from sympy import tanh, Symbol, lambdify
x = Symbol('x')

steepness = 20
arg = steepness*(x-0.5)

approximate(tanh(arg), symbolic=False, numint='GaussLegendre4',
            d=3, N_e=1, filename='fe_p3_tanh_1e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre4',
            d=3, N_e=2, filename='fe_p3_tanh_2e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre4',
            d=3, N_e=4, filename='fe_p3_tanh_4e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre5',
            d=4, N_e=1, filename='fe_p4_tanh_1e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre5',
            d=4, N_e=2, filename='fe_p4_tanh_2e')
approximate(tanh(arg), symbolic=False, numint='GaussLegendre5',
            d=4, N_e=4, filename='fe_p4_tanh_4e')
```

**b)**
How will an interpolation method work in
this case with the same number of nodes?

**Solution.**
The coefficients arising from the interpolation method are trivial to compute
since $c_i=f(x_i)$, where $x_i$ are the global nodes. The function
`u_glob` in the `fe_approx1D_numint` module can be used to compute
appropriate arrays for plotting the resulting finite element function.
We create plots where the finite element approximation is shown along
with $f(x)$ and the interpolation points.
Since `u_glob` requires the `vertices`, `cells`, and `dof_map` data
structures, we must compute these for the values of number of
elements ($N_e$) and the polynomial degree ($d$).

```python
# Interpolation method
import numpy as np
import matplotlib.pyplot as plt
f = lambdify([x], tanh(arg), modules='numpy')

# Compute exact f on a fine mesh
x_fine = np.linspace(0, 1, 101)
f_fine = f(x_fine)

for d in 3, 4:
    for N_e in 1, 2, 4:
        h = 1.0/N_e  # element length
        vertices = [i*h for i in range(N_e+1)]
        cells = [[e, e+1] for e in range(N_e)]
        dof_map = [[d*e + i for i in range(d+1)] for e in range(N_e)]
        N_n = d*N_e + 1  # Number of nodes
        x_nodes = np.linspace(0, 1, N_n)  # Node coordinates
        U = f(x_nodes)  # Interpolation method samples node values
        x, u, _ = u_glob(U, vertices, cells, dof_map,
                         resolution_per_element=51)
        plt.figure()
        plt.plot(x, u, '-', x_fine, f_fine, '--',
                 x_nodes, U, 'bo')
        plt.legend(['%d P%d elements' % (N_e, d),
                    'exact', 'interpolation points'],
                   loc='upper left')
        plt.savefig('tmp_%d_P%d.pdf' % (N_e, d))
        plt.savefig('tmp_%d_P%d.png' % (N_e, d))
plt.show()
```

Filename: `fe_tanh_P3P4`.

## Exercise 17: Investigate the approximation error in finite elements

The theory ([109](#fem:approx:fe:error:theorem)) from the section [Computing the error of the approximation](#fem:approx:fe:error) predicts that the error in the P$d$
approximation of a function should behave as $h^{d+1}$, where $h$ is
the length of the element. Use experiments to verify this asymptotic
behavior (i.e., for small enough $h$).  Choose three examples:
$f(x)=Ae^{-\omega x}$ on $[0,3/\omega]$, $f(x) = A\sin (\omega x)$ on
$\Omega=[0, 2\pi/\omega]$ for constant $A$ and $\omega$, and
$f(x)=\sqrt{x}$ on $[0,1]$.

**Hint 1.**
Run a series of experiments: $(h_i,E_i)$, $i=0,\ldots,m$, where $E_i$
is the $L^2$ norm of the error corresponding to element length $h_i$.
Assume an error model $E=Ch^r$ and compute $r$ from two successive
experiments:

$$
r_i = \ln (E_{i+1}/E_i)/\ln (h_{i+1}/h_i),\quad i=0,\ldots,m-1{\thinspace .}
$$

Hopefully, the sequence $r_0,\ldots,r_{m-1}$ converges to the true
$r$, and $r_{m-1}$ can be taken as an approximation to $r$.
Run such experiments for different $d$ for the different $f(x)$ functions.

**Hint 2.**
The `approximate` function in `fe_approx1D_numint.py` is handy for
calculating the numerical solution. This function returns the
finite element solution as the coefficients $\left\{ {c}_i \right\}_{i\in{\mathcal{I}_s}}$.
To compute $u$, use `u_glob` from the same module.
Use the Trapezoidal rule to integrate the $L^2$ error:

```python
xc, u = u_glob(c, vertices, cells, dof_map)
e = f_func(xc) - u
L2_error = 0
e2 = e**2
for i in range(len(xc)-1):
    L2_error += 0.5*(e2[i+1] + e2[i])*(xc[i+1] - xc[i])
L2_error = np.sqrt(L2_error)
```

The reason for this Trapezoidal integration is
that `u_glob` returns coordinates `xc` and corresponding `u` values
where some of the coordinates (the cell vertices) coincides, because
the solution is computed in one element at a time, using all local
nodes. Also note that there are many coordinates in $xc$ per cell
such that we can accurately compute the error inside each cell.

**Solution.**
Here is an appropriate program:

```python
from fe_approx1D_numint import approximate, mesh_uniform, u_glob
from sympy import sqrt, exp, sin, Symbol, lambdify, simplify
import numpy as np
from math import log

x = Symbol('x')
A = 1
w = 1

cases = {'sqrt': {'f': sqrt(x), 'Omega': [0,1]},
         'exp': {'f': A*exp(-w*x), 'Omega': [0, 3.0/w]},
         'sin': {'f': A*sin(w*x), 'Omega': [0, 2*np.pi/w]}}

results = {}
d_values = [1, 2, 3, 4]

for case in cases:
    f = cases[case]['f']
    f_func = lambdify([x], f, modules='numpy')
    Omega = cases[case]['Omega']
    results[case] = {}
    for d in d_values:
        results[case][d] = {'E': [], 'h': [], 'r': []}
        for N_e in [4, 8, 16, 32, 64, 128]:
            try:
                c = approximate(
                    f, symbolic=False,
                    numint='GaussLegendre%d' % (d+1),
                    d=d, N_e=N_e, Omega=Omega,
                    filename='tmp_%s_d%d_e%d' % (case, d, N_e))
            except np.linalg.linalg.LinAlgError as e:
                print((str(e)))
                continue
            vertices, cells, dof_map = mesh_uniform(
                N_e, d, Omega, symbolic=False)
            xc, u, _ = u_glob(c, vertices, cells, dof_map, 51)
            e = f_func(xc) - u
            # Trapezoidal integration of the L2 error over the
            # xc/u patches
            e2 = e**2
            L2_error = 0
            for i in range(len(xc)-1):
                L2_error += 0.5*(e2[i+1] + e2[i])*(xc[i+1] - xc[i])
            L2_error = np.sqrt(L2_error)
            h = (Omega[1] - Omega[0])/float(N_e)
            results[case][d]['E'].append(L2_error)
            results[case][d]['h'].append(h)
        # Compute rates
        h = results[case][d]['h']
        E = results[case][d]['E']
        for i in range(len(h)-1):
            r = log(E[i+1]/E[i])/log(h[i+1]/h[i])
            results[case][d]['r'].append(round(r, 2))

print(results)
for case in results:
    for d in sorted(results[case]):
        print(('case=%s d=%d, r: %s' % \
              (case, d, results[case][d]['r'])))
```

The output becomes

        case=sqrt d=1, r: [1.0, 1.0, 1.0, 1.0, 1.0]
        case=sqrt d=2, r: [1.0, 1.0, 1.0, 1.0, 1.0]
        case=sqrt d=3, r: [1.0, 1.0, 1.0, 1.0, 1.0]
        case=sqrt d=4, r: [1.0, 1.0, 1.0, 1.0, 1.0]
        case=exp d=1, r: [2.01, 2.01, 2.0, 2.0, 2.0]
        case=exp d=2, r: [2.81, 2.89, 2.94, 2.97, 2.98]
        case=exp d=3, r: [3.98, 4.0, 4.0, 4.0, 4.0]
        case=exp d=4, r: [4.87, 4.93, 4.96, 4.98, 4.99]
        case=sin d=1, r: [2.15, 2.06, 2.02, 2.0, 2.0]
        case=sin d=2, r: [2.68, 2.83, 2.93, 2.97, 2.99]
        case=sin d=3, r: [4.06, 4.04, 4.01, 4.0, 4.0]
        case=sin d=4, r: [4.79, 4.9, 4.96, 4.98, 4.99]

showing that the convergence rate stabilizes quite quickly at $N_e=128$
cells. While the theory predicts the rate as $d+1$, this is only
fulfilled for the exponential and sine functions, while the square root
functions gives a rate 1 regardless of $d$. The reason is that the
estimate ([109](#fem:approx:fe:error:theorem)) contains the integral of
the derivatives of $f$ over $[0,1]$. For $f=\sqrt{x}$, we
have $f'=\frac{1}{2} x^{-1/2}$, $f''=-\frac{1}{4}x^{-3/2}$, and all integrals
of $f''$ and higher derivatives are infinite on $[0,L]$. Our experiments
show that the method still converges, but $f$ is not smooth enough that
higher-order elements give superior convergence rates.

Filename: `Pd_approx_error`.

## Problem 18: Approximate a step function by finite elements

Approximate the step function

$$
f(x) = \left\lbrace\begin{array}{ll}
0 & \mbox{ if } 0\leq x < {1/2},\\
1 & \mbox{ if } {1/2} \leq x \geq {1/2}
\end{array}\right.
$$

by 2, 4, 8, and 16 elements and  P1, P2, P3, and P4. Compare approximations visually.

**Hint.**
This $f$ can also be expressed in terms of the Heaviside function $H(x)$:
$f(x) = H(x-{1/2})$.
Therefore, $f$ can be defined by

```python
f = sym.Heaviside(x - sym.Rational(1,2))
```

making the `approximate` function in the
`fe_approx1D.py` module an obvious candidate to solve the
problem. However, `sympy` does not handle symbolic integration
with this particular integrand, and the `approximate` function faces a problem
when converting `f` to a Python function (for plotting) since
`Heaviside` is not an available function in `numpy`.

An alternative is to perform hand calculations. This is an instructive
task, but in practice only feasible for few elements and P1 and P2 elements.
It is better to copy the functions `element_matrix`, `element_vector`,
`assemble`, and `approximate` from the `fe_approx1D_numint.py` file
and edit these functions such that they can compute approximations
with `f` given as a Python function and not a symbolic expression.
Also assume that `phi` computed by the `basis` function is a Python
callable function. Remove all instances of the `symbolic` variable
and associated code.

**Solution.**
The modifications of `element_matrix`, `element_vector`,
`assemble`, and `approximate` from the `fe_approx1D_numint.py` file
are listed below.

```python
from fe_approx1D_numint import mesh_uniform, u_glob
from fe_approx1D import basis
import numpy as np

def element_matrix(phi, Omega_e, numint):
    n = len(phi)
    A_e = np.zeros((n, n))
    h = Omega_e[1] - Omega_e[0]
    detJ = h/2  # dx/dX
    for r in range(n):
        for s in range(r, n):
            for j in range(len(numint[0])):
                Xj, wj = numint[0][j], numint[1][j]
                A_e[r,s] += phi[r](Xj)*phi[s](Xj)*detJ*wj
            A_e[s,r] = A_e[r,s]
    return A_e

def element_vector(f, phi, Omega_e, numint):
    n = len(phi)
    b_e = np.zeros(n)
    h = Omega_e[1] - Omega_e[0]
    detJ = h/2
    for r in range(n):
        for j in range(len(numint[0])):
            Xj, wj = numint[0][j], numint[1][j]
            xj = (Omega_e[0] + Omega_e[1])/2 + h/2*Xj  # mapping
            b_e[r] += f(xj)*phi[r](Xj)*detJ*wj
    return b_e

def assemble(vertices, cells, dof_map, phi, f, numint):
    N_n = len(list(set(np.array(dof_map).ravel())))
    N_e = len(cells)
    A = np.zeros((N_n, N_n))
    b = np.zeros(N_n)
    for e in range(N_e):
        Omega_e = [vertices[cells[e][0]], vertices[cells[e][1]]]
        A_e = element_matrix(phi[e], Omega_e, numint)
        b_e = element_vector(f, phi[e], Omega_e, numint)
        #print('element', e)
        #print(b_e)
        for r in range(len(dof_map[e])):
            for s in range(len(dof_map[e])):
                A[dof_map[e][r],dof_map[e][s]] += A_e[r,s]
            b[dof_map[e][r]] += b_e[r]
    return A, b

def approximate(f, d, N_e, numint, Omega=[0,1], filename='tmp'):
    """
    Compute the finite element approximation, using Lagrange
    elements of degree d, to a Python functionn f on a domain
    Omega. N_e is the number of elements.
    numint is the name of the numerical integration rule
    (Trapezoidal, Simpson, GaussLegendre2, GaussLegendre3,
    GaussLegendre4, etc.). numint=None implies exact
    integration.
    """
    from math import sqrt
    numint_name = numint  # save name
    if numint == 'Trapezoidal':
        numint = [[-1, 1], [1, 1]]
    elif numint == 'Simpson':
        numint = [[-1, 0, 1], [1./3, 4./3, 1./3]]
    elif numint == 'Midpoint':
        numint = [[0], [2]]
    elif numint == 'GaussLegendre2':
        numint = [[-1/sqrt(3), 1/sqrt(3)], [1, 1]]
    elif numint == 'GaussLegendre3':
        numint = [[-sqrt(3./5), 0, sqrt(3./5)],
                  [5./9, 8./9, 5./9]]
    elif numint == 'GaussLegendre4':
        numint = [[-0.86113631, -0.33998104,  0.33998104,
                   0.86113631],
                  [ 0.34785485,  0.65214515,  0.65214515,
                    0.34785485]]
    elif numint == 'GaussLegendre5':
        numint = [[-0.90617985, -0.53846931, -0.        ,
                   0.53846931,  0.90617985],
                  [ 0.23692689,  0.47862867,  0.56888889,
                    0.47862867,  0.23692689]]
    elif numint is not None:
        print(('Numerical rule %s is not supported '\
              'for numerical computing' % numint))
        sys.exit(1)

    vertices, cells, dof_map = mesh_uniform(N_e, d, Omega)

    # phi is a list where phi[e] holds the basis in cell no e
    # (this is required by assemble, which can work with
    # meshes with different types of elements).
    # len(dof_map[e]) is the number of nodes in cell e,
    # and the degree of the polynomial is len(dof_map[e])-1
    phi = [basis(len(dof_map[e])-1) for e in range(N_e)]

    A, b = assemble(vertices, cells, dof_map, phi, f,
                    numint=numint)

    print(('cells:', cells))
    print(('vertices:', vertices))
    print(('dof_map:', dof_map))
    print(('A:\n', A))
    print(('b:\n', b))
    c = np.linalg.solve(A, b)
    print(('c:\n', c))

    if filename is not None:
        title = 'P%d, N_e=%d' % (d, N_e)
        title += ', integration: %s' % numint_name
        x_u, u, _ = u_glob(np.asarray(c), vertices, cells, dof_map,
                           resolution_per_element=51)
        x_f = np.linspace(Omega[0], Omega[1], 10001) # mesh for f
        import scitools.std as plt
        plt.plot(x_u, u, '-',
                 x_f, f(x_f), '--')
        plt.legend(['u', 'f'])
        plt.title(title)
        plt.savefig(filename + '.pdf')
        plt.savefig(filename + '.png')
    return c
```

With a purely numerical version of the `approximate` function, we can
easily investigate the suggested approximations in this exercise:

```python
def exercise():
    def f(x):
        if isinstance(x, (float,int)):
            return 0 if x < 0.5 else 1
        elif isinstance(x, np.ndarray):
            return np.where(x < 0.5, 0, 1)

    N_e_values = [2, 4, 8, 16]
    for d in 1, 2, 3, 4:
        for N_e in N_e_values:
            approximate(f, numint='GaussLegendre%d' % (d+1),
                        d=d, N_e=N_e,
                        filename='fe_Heaviside_P%d_%de' % (d, N_e))
        for ext in 'pdf', 'png':
            cmd = 'doconce combine_images '
            cmd += ext + ' -2 '
            cmd += ' '.join(['fe_Heaviside_P%d_%de' % (d, N_e)
                             for N_e in N_e_values])
            cmd += ' fe_Heaviside_P%d' % d
            print(cmd)
            os.system(cmd)
```

Running this function reveals that even finite elements
(and not only sines, as demonstrated in [Exercise 8: Fourier series as a least squares approximation](#fem:approx:exer:Fourier))
give oscillations around a discontinuity.

**Remarks.**
It is of extreme importance to use a Gauss-Legendre numerical integration
rule that matches the degree of polynomials in the basis.
Using a rule with fewer points may lead to very strange results.

Filename: `fe_Heaviside_P1P2`.

## Exercise 19: 2D approximation with orthogonal functions

**a)**
Assume we have basis functions ${\varphi}_i(x,y)$ in 2D that are
orthogonal such that $({\varphi}_i,{\varphi}_j)=0$ when $i\neq j$.  The
function `least_squares` in the file [`approx2D.py`](${fem_src}/fe_approx2D.py) will then spend much time on computing
off-diagonal terms in the coefficient matrix that we know are zero.
To speed up the computations, make a version `least_squares_orth` that
utilizes the orthogonality among the basis functions.

**Solution.**
We 1) remove the `j` loop in the `least_squares` function and set
`j = i`,
2) make `A` a vector (i.e., $(N+1, 1)$ matrix as `b` and `c`),
3) solve for `c[i,0]` as soon as `A[i,0]` and `b[i,0]` are computed.

```python
import sympy as sym
import mpmath

def least_squares_orth(f, psi, Omega, symbolic=True,
                       print_latex=False):
    """
    Given a function f(x,y) on a rectangular domain
    Omega=[[xmin,xmax],[ymin,ymax]],
    return the best approximation to f(x,y) in the space V
    spanned by the functions in the list psi.
    This function assumes that psi are orthogonal on Omega.
    """
    # Modification of least_squares function: drop the j loop,
    # set j=i, compute c on the fly in the i loop.

    N = len(psi) - 1
    # Note that A, b, c becmes (N+1)x(N+1), use 1st column
    A = sym.zeros(N+1)
    b = sym.zeros(N+1)
    c = sym.zeros(N+1)
    x, y = sym.symbols('x y')
    print(('...evaluating matrix...', A.shape, b.shape, c.shape))
    for i in range(N+1):
        j = i
        print(('(%d,%d)' % (i, j)))

        integrand = psi[i]*psi[j]
        if symbolic:
            I = sym.integrate(integrand,
                             (x, Omega[0][0], Omega[0][1]),
                             (y, Omega[1][0], Omega[1][1]))
        if not symbolic or isinstance(I, sym.Integral):
            # Could not integrate symbolically, use numerical int.
            print(('numerical integration of', integrand))
            integrand = sym.lambdify([x,y], integrand, 'mpmath')
            I = mpmath.quad(integrand,
                            [Omega[0][0], Omega[0][1]],
                            [Omega[1][0], Omega[1][1]])
        A[i,0] = I

        integrand = psi[i]*f
        if symbolic:
            I = sym.integrate(integrand,
                             (x, Omega[0][0], Omega[0][1]),
                             (y, Omega[1][0], Omega[1][1]))
        if not symbolic or isinstance(I, sym.Integral):
            # Could not integrate symbolically, use numerical int.
            print(('numerical integration of', integrand))
            integrand = sym.lambdify([x,y], integrand, 'mpmath')
            I = mpmath.quad(integrand,
                            [Omega[0][0], Omega[0][1]],
                            [Omega[1][0], Omega[1][1]])
        b[i,0] = I
        c[i,0] = b[i,0]/A[i,0]
    print()
    print(('A:\n', A, '\nb:\n', b))
    c = [c[i,0] for i in range(c.shape[0])]  # make list
    print(('coeff:', c))

    # c is a sympy Matrix object, numbers are in c[i,0]
    u = sum(c[i]*psi[i] for i in range(len(psi)))
    print(('approximation:', u))
    print(('f:', sym.expand(f)))
    if print_latex:
        print((sym.latex(A, mode='plain')))
        print((sym.latex(b, mode='plain')))
        print((sym.latex(c, mode='plain')))
    return u, c
```

**b)**
Apply the function to approximate

$$
f(x,y) = x(1-x)y(1-y)e^{-x-y}
$$

on $\Omega = [0,1]\times [0,1]$ via basis functions

$$
{\varphi}_i(x,y) = \sin ((p+1)\pi x)\sin((q+1)\pi y),\quad i=q(N_x+1) + p,
$$

where $p=0,\ldots,N_x$ and $q=0,\ldots,N_y$.

**Hint.**
Get ideas from the function `least_squares_orth` in
the section [Orthogonal basis functions](#fem:approx:global:orth) and
file [`approx1D.py`](${fem_src}/fe_approx1D.py).

**Solution.**
A function for computing the basis functions may look like this:

```python
def sine_basis(Nx, Ny):
    """
    Compute basis sin((p+1)*pi*x)*sin((q+1)*pi*y),
    p=0,...,Nx, q=0,...,Ny.
    """
    x, y = sym.symbols('x y')
    psi = []
    for q in range(0, Ny+1):
        for p in range(0, Nx+1):
            r = sym.sin((p+1)*sym.pi*x)*sym.sin((q+1)*sym.pi*y)
            psi.append(r)
    return psi
```

Application of this basis to approximate the given function is coded in
the following function:

```python
def demo(N):
    """
    Find the approximation of f by the least squares method.
    The basis is sin((p+1)*pi*x)sin((q+1)*pi*y) where
    0<p<=N, p<q<=N.
    """
    x, y = sym.symbols('x y')
    f = x*(1-x)*y*(1-y)*sym.exp(-x-y)

    psi = sine_basis(N, N)

    Omega = [[0,1], [0,1]]
    u, c  = least_squares_orth(f, psi, Omega, symbolic=False)
    from approx2D import comparison_plot
    comparison_plot(f, u, Omega, title='N=%d' % N)
    print(c)

if __name__=='__main__':
    #test_least_squares_orth()
    demo(N=2)
```

A lesson learned is that `symbolic=False` is important, otherwise `sympy`
consumes a lot of CPU time on trying to integrate symbolically.

The figure below shows the error in the approximation for $N=0$ (left)
and $N=2$ (right). The coefficients for $N=2$ decay rapidly:

        [0.025, 0.0047, 0.0014, 0.0047, 0.0009, 0.0003, 0.0014, 0.0003,
         8.2e-5]

**c)**
Make a unit test for the `least_squares_orth` function.

**Solution.**
Let us use the basis in b), fix the coefficients of some function
$f$, and check that the computed approximation, with the
same basis, has the same coefficients (this test employs the principle
that if $f\in V$, then $u=f$).

```python
def test_least_squares_orth():
    # Use sine functions
    x, y = sym.symbols('x y')
    N = 2  # (N+1)**2 = 9 basis functions
    psi = sine_basis(N, N)
    f_coeff = [0]*len(psi)
    f_coeff[3] = 2
    f_coeff[4] = 3
    f = sum(f_coeff[i]*psi[i] for i in range(len(psi)))
    # Check that u exactly reproduces f
    u, c = least_squares_orth(f, psi, Omega=[[0,1], [0,1]],
                              symbolic=False)
    import numpy as np
    diff = np.abs(np.array(c) - np.array(f_coeff)).max()
    print(('diff:', diff))
    tol = 1E-15
    assert diff < tol
```

Filename: `approx2D_ls_orth`.

## Exercise 20: Use the Trapezoidal rule and P1 elements

Consider the approximation of some $f(x)$ on an interval $\Omega$ using
the least squares or Galerkin methods with P1 elements. Derive
the element matrix and vector using the
Trapezoidal rule ([117](#fem:approx:fe:numint1:trapez)) for calculating
integrals on the reference element. Assemble the contributions, assuming
a uniform cell partitioning, and show that the resulting linear system
has the form $c_i=f(x_{i})$ for $i\in{\mathcal{I}_s}$.

**Solution.**
The Trapezoidal rule for integrals on $[-1,1]$
is given by ([117](#fem:approx:fe:numint1:trapez)).
The expressions for the entries in the element matrix
are given by ([82](#fem:approx:fe:mapping:Ae)) in
the section [Mapping to a reference element](#fem:approx:fe:mapping):

$$
\begin{align*} \tilde A^{(e)}_{r,s} &=
\int_{-1}^1 {\tilde{\varphi}}_r(X){\tilde{\varphi}}_s(X)\det J\,{\, \mathrm{d}X}\\
&\approx \frac{h}{2}({\tilde{\varphi}}_r(-1){\tilde{\varphi}}_s(-1)
+ {\tilde{\varphi}}_r(1){\tilde{\varphi}}_s(1)){\thinspace .}
\end{align*}
$$

We know that if ${\tilde{\varphi}}_r(\pm 1)$ is 0 or 1, so evaluating
the formula above for $r,s=0,1$ gives

$$
\tilde A^{(e)} = \frac{h}{2}\left(\begin{array}{cc}
1 & 0\\
0 & 1
\end{array}\right){\thinspace .}
$$

As usual, $h$ is the length of the element in physical coordinates.

The element vector in the reference element is given by
([83](#fem:approx:fe:mapping:be)):

$$
\begin{align*}
\tilde b^{(e)}_{r} &=  \int_{-1}^1 f(x(X)){\tilde{\varphi}}_r(X)\det J\,{\, \mathrm{d}X}\\
&\approx \frac{h}{2}(f(x(-1)){\tilde{\varphi}}_r(-1)
+ f(x(1)){\tilde{\varphi}}_r(1)){\thinspace .}
\end{align*}
$$

Evaluating the formula for $r=0,1$ leads to

$$
\tilde b^{(e)} = \frac{h}{2}\left(\begin{array}{c}
f(x_L)\\
f(x_R)
\end{array}\right),
$$

where $x_L$ and $x_R$ are the $x$ coordinates of the local points
$X=-1$ and $X=1$, respectively.

With a uniform mesh with nodes $x_{i}=ih$, the element matrix and
vectors assemble to a coefficient matrix

$$
\frac{h}{2}\hbox{diag}(1, 2, \ldots, 2, 1),
$$

and right-hand side vector

$$
\frac{h}{2}(f(x_{0}), 2f(x_{1}), \ldots, 2f(x_{N_n-1}),
f(x_{N_n})){\thinspace .}
$$

The factors $h/2$ and $2$ cancel, so we are left with the solution of
the system as

$$
c_i = f(x_{i}){\thinspace .}
$$

Filename: `fe_P1_trapez`.

## Exercise 21: Compare P1 elements and interpolation

We shall approximate the function

$$
f(x) = 1 + \epsilon\sin (2\pi nx),\quad x\in \Omega = [0,1],
$$

where $n\in\mathbb{Z}$ and $\epsilon \geq 0$.

**a)**
Plot $f(x)$ for $n=1,2,3$ and find the wavelength of the function.

**b)**
We want to use $N_P$ elements per wavelength. Show that the number
of elements is then $nN_P$.

**c)**
The critical quantity for accuracy is the number of elements per
wavelength, not the element size in itself. It therefore suffices
to study an $f$ with just one wavelength in $\Omega = [0,1]$.
Set $\epsilon = 0.5$.

Run the least squares or projection/Galerkin method for
$N_P=2,4,8,16,32$. Compute the error $E=||u-f||_{L^2}$.

**Hint 1.**
Use the `fe_approx1D_numint` module to compute $u$ and use
the technique from the section [Computing the error of the approximation](#fem:approx:fe:error) to
compute the norm of the error.

**Hint 2.**
Read up on the Nyquist–Shannon sampling theorem.

**d)**
Repeat the set of experiments in the above point, but
use interpolation/collocation based on the node points to
compute $u(x)$ (recall that $c_i$ is now simply $f(x_{i})$).
Compute the error $E=||u-f||_{L^2}$.
Which method seems to be most accurate?

Filename: `fe_P1_vs_interp`.

## Exercise 22: Implement 3D computations with global basis functions

Extend the [`approx2D.py`](${fem_src}/approx2D.py) code to 3D
by applying ideas from the section [Extension to 3D](#fem:approx:3D:global).
Construct some 3D problem to make a test function for the
implementation.

**Hint.**
Drop symbolic integration since it is in general too slow for 3D problems.
Also use `scipy.integrate.nquad` instead of `mpmath.quad`
for numerical integration, since it is much faster.

**Solution.**
We take a copy of `approx2D.py` and drop the `comparison_plot` function since
plotting in 3D is much more complicated (could make a special version with
curves through lines in the 3D domain, for instance).
Furthermore, we remove the lines with symbolic integration and replace
the calls to `mpmath.quad` by calls to
`scipy.integrate.nquad`. The resulting function becomes

```python
import sympy as sym
import numpy as np
import scipy.integrate

def least_squares(f, psi, Omega):
    """
    Given a function f(x,y,z) on a rectangular domain
    Omega=[[xmin,xmax],[ymin,ymax],[zmin,zmax]],
    return the best approximation to f in the space V
    spanned by the functions in the list psi.
    f and psi are symbolic (sympy) expressions, but will
    be converted to numeric functions for faster integration.
    """
    N = len(psi) - 1
    A = np.zeros((N+1, N+1))
    b = np.zeros(N+1)
    x, y, z = sym.symbols('x y z')
    f = sym.lambdify([x, y, z], f, modules='numpy')
    psi_sym = psi[:]  # take a copy, needed for forming u later
    psi = [sym.lambdify([x, y, z], psi[i]) for i in range(len(psi))]

    print('...evaluating matrix...')
    for i in range(N+1):
        for j in range(i, N+1):
            print(('(%d,%d)' % (i, j)))

            integrand = lambda x, y, z: psi[i](x,y,z)*psi[j](x,y,z)
            I, err = scipy.integrate.nquad(
                integrand,
                [[Omega[0][0], Omega[0][1]],
                 [Omega[1][0], Omega[1][1]],
                 [Omega[2][0], Omega[2][1]]])
            A[i,j] = A[j,i] = I
        integrand = lambda x, y, z: psi[i](x,y,z)*f(x,y,z)
        I, err = scipy.integrate.nquad(
            integrand,
            [[Omega[0][0], Omega[0][1]],
             [Omega[1][0], Omega[1][1]],
             [Omega[2][0], Omega[2][1]]])
        b[i] = I
    print()
    c = np.linalg.solve(A, b)
    if N <= 10:
        print(('A:\n', A, '\nb:\n', b))
        print(('coeff:', c))
    u = sum(c[i]*psi_sym[i] for i in range(len(psi_sym)))
    print(('approximation:', u))
    return u, c
```

As test example, we can use the basis

$$
{\psi}_{p,q,r} = \sin((p+1)\pi x)\sin((q+1)\pi y)\sin((r+1)\pi z),
$$

for $p=1,\ldots,N_x$, $q=1,\ldots,N_y$, $r=1,\ldots,N_z$.
We choose $f$ as some prescribed combination of these functions and
check that the computed $u$ is exactly equal to $f$.

```python
def sine_basis(Nx, Ny, Nz):
    """
    Compute basis sin((p+1)*pi*x)*sin((q+1)*pi*y)*sin((r+1)*pi*z),
    p=0,...,Nx, q=0,...,Ny, r=0,...,Nz.
    """
    x, y, z = sym.symbols('x y z')
    psi = []
    for r in range(0, Nz+1):
        for q in range(0, Ny+1):
            for p in range(0, Nx+1):
                s = sym.sin((p+1)*sym.pi*x)*\
                    sym.sin((q+1)*sym.pi*y)*sym.sin((r+1)*sym.pi*z)
                psi.append(s)
    return psi

def test_least_squares():
    # Use sine functions
    x, y, z = sym.symbols('x y z')
    N = 1  # (N+1)**3 = 8 basis functions
    psi = sine_basis(N, N, N)
    f_coeff = [0]*len(psi)
    f_coeff[3] = 2
    f_coeff[4] = 3
    f = sum(f_coeff[i]*psi[i] for i in range(len(psi)))
    # Check that u exactly reproduces f
    u, c = least_squares(f, psi, Omega=[[0,1], [0,1], [0,1]])
    diff = np.abs(np.array(c) - np.array(f_coeff)).max()
    print(('diff:', diff))
    tol = 1E-15
    assert diff < tol
```

Filename: `approx3D`.

## Exercise 23: Use Simpson's rule and P2 elements

Redo [Exercise 20: Use the Trapezoidal rule and P1 elements](#fem:approx:fe:exer:1D:trapez), but use P2
elements and Simpson's rule based on sampling the integrands at
the nodes in the reference cell.

**Solution.**
Simpson's rule for integrals on $[-1,1]$
is given by ([118](#fem:approx:fe:numint1:Simpson)).
The expressions for the entries in the element matrix
are given by ([82](#fem:approx:fe:mapping:Ae)):

$$
\begin{align*} \tilde A^{(e)}_{r,s} &=
\int_{-1}^1 {\tilde{\varphi}}_r(X){\tilde{\varphi}}_s(X)\det J\,{\, \mathrm{d}X}\\
&\approx \frac{1}{3}\frac{h}{2}({\tilde{\varphi}}_r(-1){\tilde{\varphi}}_s(-1)
+ 4{\tilde{\varphi}}_r(0){\tilde{\varphi}}_s(0)
+ {\tilde{\varphi}}_r(1){\tilde{\varphi}}_s(1)){\thinspace .}
\end{align*}
$$

The expressions for ${\tilde{\varphi}}_r(X)$ are given by
([84](#fem:approx:fe:mapping:P1:phi0))-([85](#fem:approx:fe:mapping:P1:phi1)).
Evaluating the formula for $r,s=0,1,2$ gives the element matrix

$$
\tilde A^{(e)} = \frac{h}{6}\left(\begin{array}{ccc}
1 & 0 & 0\\
0 & 4 & 0\\
0 & 0 & 1
\end{array}\right){\thinspace .}
$$

As usual, $h$ is the length of the element in physical coordinates.

The element vector in the reference element is given by
([83](#fem:approx:fe:mapping:be)):

$$
\begin{align*}
\tilde b^{(e)}_{r} &=  \int_{-1}^1 f(x(X)){\tilde{\varphi}}_r(X)\det J\,{\, \mathrm{d}X}\\
&\approx \frac{1}{3}\frac{h}{2}(f(x(-1)){\tilde{\varphi}}_r(-1)
+ 4f(x(0)){\tilde{\varphi}}_r(0)
+ f(x(1)){\tilde{\varphi}}_r(1)){\thinspace .}
\end{align*}
$$

Evaluating the formula for $r=0,1,2$ leads to

$$
\tilde b^{(e)} = \frac{h}{2}\left(\begin{array}{c}
f(x_L)\\
4f(x_c)
f(x_R)
\end{array}\right),
$$

where $x_L$, $x_c$, and $x_R$ are the $x$ coordinates of the local points
$X=-1$, $X=0$, and $X=1$, respectively. These correspond to the nodes
in the element.

With a uniform mesh with nodes $x_{i}=ih$, the element matrix and
vectors assemble to a coefficient matrix

$$
\frac{h}{6}\hbox{diag}(1, 4, 2, 4, 2, 4, \ldots, 2, 4, 1),
$$

and right-hand side vector

$$
\frac{h}{6}(f(x_{0}), 4f(x_{1}), 2f(x_{2}),
4f(x_{3}), 2f(x_{4}), \ldots, 2f(x_{N_n-2}),
4f(x_{N_n-1}), f(x_{N_n})){\thinspace .}
$$

The factors $h/6$, $2$ and $4$ all cancel, so we are left with the solution of
the system as

$$
c_i = f(x_{i}){\thinspace .}
$$

Filename: `fe_P2_simpson`.

## Exercise 24: Make a 3D code for Lagrange elements of arbitrary order

Extend the code from the section [Refined code with curve plotting](#fem:approx:fenics:2D:2) to 3D.


```python
### Running in Google Colab? You'll want to uncomment and run these cell once each time you start this notebook.

"""
!wget https://raw.githubusercontent.com/psheehan/CIERA-HS-Program/master/BonusProblems/Module2/HRstars.csv
"""
```

*Modified from the CIERA IDEAS program python notes (https://github.com/IDEASpy/IDEASpy) and from notes by G. Besla, P. Pinto, E. Hayati (https://github.com/gurtina/P105A_2019)*

## Hertzsprung-Russell Diagram

In this activity, you are going to make one of the most important tools in Astronomy, a Hertzprung-Russell (H-R) Diagram. H-R Diagrams plot the temperature of a star against how bright a star is (i.e. it's "luminosity"). They are tremendously useful for studying many aspects of stars because, as it turns out, different types of stars fall in specific places on an H-R Diagram.

The file "HRstars.csv" contains the temperature (in Kelvin; first column) and the luminosity (in solar luminosity; second column) of a set of stars. Use numpy loadtxt to read the data file and extract the appropriate data as separate variables.

```python
##Import Modules

##Read data from file. Use delimiter = ','
data = np.loadtxt('', delimiter=',')  ##fill in the blank

temperature = data[]
luminosity = data[]

```

Now that you've loaded the data, make a plot showing temperature vs. luminosity for the stars in this dataset. Make sure to label your axes properly!

```python
##Import Modules

##Plot the data
fig, ax = plt.subplots(nrows=1, ncols=1)

##Make title and x, y labels.
##X label: Temperature (K), Y label: Luminosity (Lsun)
##Title: HR Diagram

```

Now, we're going to make some adjustments to your plot. H-R diagrams are traditionally plotted with temperature increasing from right to left. Also, you might notice that a lot of the data points are crowded around zero. This is because the luminosities and temperatures of stars span many "orders of magnitude" (i.e. factors of 10). One way to nicely deal with this is to change the axes to a log scale. Make a new version of your above plot, but now reverse the x-axis and make both axes have a logarithmic scale:

```python

```

Now lets add some color! Hotter stars are bluer and colder stars are redder, so lets use a color map called 'RdYlBu' for the colors of the points. (_Hint_: The ax.scatter function allows you to change the colors of the points based on an array of input values. If the color map goes from red at small values to blue at large values, what array should we give the scatter function to map colors properly?)

```python
cm = plt.cm.get_cmap('RdYlBu')  ##This is the color map for the stars

```

Finally, the size of a star depends on two properties, it's temperature and size. Hotter stars are brighter, but it also turns out bigger stars are brighter because they have more surface area to emit light from. Let's incorporate information about the size of stars in our plot. Use this equation

\begin{equation}
\frac{L}{L_{\odot}}=(\frac{R}{R_{\odot}})^2(\frac{T}{T_{\odot}})^4
\end{equation}

with

\begin{equation} T_{\odot} = 5780 K \end{equation}

to approximate the sizes of the points representing the stars. (_Hint_: The scatter function also allows you to specify the size of points based on an array of input values.)

```python
##Approximate the sizes of the stars with the equation above and multiply them by 15.
Tsun=5780.
sstar=15*   ##Fill in the blank

```

What do you notice about the stars in your H-R Diagram? The features you see in this diagram correspond to different types of stars, as you can see in the H-R Diagram shown below (_Note_: Double click on this cell to edit and remove the "\<!--" and "\-->" below to see the image). Each of these different types of stars correspond to interesting physics that happens at different points in the lifetime of stars. The "main sequence" is where stars live during the vast majority of their life, while they are fusing Hydrogen as a source of fuel. "Giant" stars are what you get when stars near the end of their lives. Though the exact physics is too complicated for this simple notebook (though do feel free to ask an instructor, if interested!), in short, stars near the end of their lives start burning a huge amount of fuel in a short amount of time, causing them to swell up and get very large and bright. But because they are so large, their surfaces also cool down some. Finally, white dwarfs are what is left over when a giant star can no longer hold on to most of this material at large radii when the star is swollen, and so is left with a super hot core. But because they are small (about the size of Earth!), they aren't very bright.

<!--

-->

```python

```