# Lambda School Data Science Module 142
## Sampling, Confidence Intervals, and Hypothesis Testing

## Prepare - examine other available hypothesis tests

If you had to pick a single hypothesis test in your toolbox, t-test would probably be the best choice - but the good news is you don't have to pick just one! Here's some of the others to be aware of:

```
import numpy as np
from scipy.stats import chisquare  # One-way chi square test

# Chi square can take any crosstab/table and test the independence of rows/cols
# The null hypothesis is that the rows/cols are independent -> low chi square
# The alternative is that there is a dependence -> high chi square
# Be aware! Chi square does *not* tell you direction/causation

ind_obs = np.array([[1, 1], [2, 2]]).T
print(ind_obs)
print(chisquare(ind_obs, axis=None))

dep_obs = np.array([[16, 18, 16, 14, 12, 12], [32, 24, 16, 28, 20, 24]]).T
print(dep_obs)
print(chisquare(dep_obs, axis=None))
```

```
# Distribution tests:
# We often assume that something is normal, but it can be important to *check*

# For example, later on with predictive modeling, a typical assumption is that
# residuals (prediction errors) are normal - checking is a good diagnostic

from scipy.stats import normaltest
# Poisson models arrival times and is related to the binomial (coinflip)
sample = np.random.poisson(5, 1000)
print(normaltest(sample))  # Pretty clearly not normal
```

```
# Kruskal-Wallis H-test - compare the median rank between 2+ groups
# Can be applied to ranking decisions/outcomes/recommendations
# The underlying math comes from chi-square distribution, and is best for n>5
from scipy.stats import kruskal

x1 = [1, 3, 5, 7, 9]
y1 = [2, 4, 6, 8, 10]
print(kruskal(x1, y1))  # x1 is a little better, but not "significantly" so

x2 = [1, 1, 1]
y2 = [2, 2, 2]
z = [2, 2]  # Hey, a third group, and of different size!
print(kruskal(x2, y2, z))  # x clearly dominates
```

And there's many more! `scipy.stats` is fairly comprehensive, though there are even more available if you delve into the extended world of statistics packages. As tests get increasingly obscure and specialized, the importance of knowing them by heart becomes small - but being able to look them up and figure them out when they *are* relevant is still important.

## T-test Assumptions

<https://statistics.laerd.com/statistical-guides/independent-t-test-statistical-guide.php>

- Independence of means

Are the means of our voting data independent (do not affect the outcome of one another)?

The best way to increase thel likelihood of our means being independent is to randomly sample (which we did not do).

```
from scipy.stats import ttest_ind

?ttest_ind
```

- "Homogeneity" of Variance?

Is the magnitude of the variance between the two roughly the same?

I think we're OK on this one for the voting data, although it probably could be better, one party was larger than the other.

If we suspect this to be a problem then we can use Welch's T-test

```
?ttest_ind
```

- "Dependent Variable" (sample means) are Distributed Normally

<https://stats.stackexchange.com/questions/9573/t-test-for-non-normal-when-n50>

Lots of statistical tests depend on normal distributions. We can test for normality using Scipy as was shown above.

This assumption is often assumed even if the assumption is a weak one. If you strongly suspect that things are not normally distributed, you can transform your data to get it looking more normal and then run your test. This problem typically goes away for large sample sizes (yay Central Limit Theorem) and is often why you don't hear it brought up. People declare the assumption to be satisfied either way.

## Central Limit Theorem

```
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

sample_means = []
for x in range(0,3000):
  coinflips = np.random.binomial(n=1, p=.5, size=30)
  one_sample = coinflips
  sample_means.append(coinflips.mean())

print(len(sample_means))
print(sample_means)
```

```
df = pd.DataFrame({'a': one_sample})
df.head()
```

```
df.a.hist()
```

```
ax = plt.hist(sample_means, bins=30)
plt.title('Distribution of 3000 sample means \n (of 30 coinflips each)');
```

What does the Central Limit Theorem State? That no matter the initial distribution of the population, the distribution of sample means taken will approximate a normal distribution as $n \rightarrow \infty$.

This has very important implications for hypothesis testing and is precisely the reason why the t-distribution begins to approximate the normal distribution as our sample size increases.

## Standard Error of the Mean

What does it mean to "estimate"? the Population mean?

```
import numpy as np
import pandas as pd

lambda_heights = np.random.uniform(4,6.5, size=2000)
print(len(lambda_heights))
lambda_heights
```

```
print("Population Mean:", lambda_heights.mean())
print("Population Standard Deviation:", lambda_heights.std())
```

```
population = pd.DataFrame({'heights': lambda_heights})
print(population.shape)
population.head()
```

```
sample = population.sample(100)
print(sample.shape)
sample.head()
```

```
print("Sample Mean 1:", sample['heights'].mean())
```

```
sample = population.sample(100)
print(sample.shape)
sample.head()
```

```
print("Sample Mean 2:", sample['heights'].mean())
```

## Build and Interpret a Confidence Interval

```
def confidence_interval(data, confidence=0.95):
  """
  Calculate a confidence interval around a sample mean for given data.
  Using t-distribution and two-tailed test, default 95% confidence.

  Arguments:
    data - iterable (list or numpy array) of sample observations
    confidence - level of confidence for the interval

  Returns:
    tuple of (mean, lower bound, upper bound)
  """
  data = np.array(data)
  mean = np.mean(data)
  n = len(data)
  stderr = stats.sem(data)
  interval = stderr * stats.t.ppf((1 + confidence) / 2.0, n - 1)
  return (mean, mean - interval, mean + interval)
```

```
coinflips_100 = np.random.binomial(n=1, p=.5, size=100)

sample_std = np.std(coinflips_100)
print("sample standard deviation:", sample_std)
sample_size = len(coinflips_100)
print("sample size:", sample_size)
```

```
standard_error = sample_std / (sample_size**(.5))

print("standard error:", standard_error)
```

```
from scipy import stats

stderr = stats.sem(coinflips_100, ddof=0)
stderr
```

###What confidence level do we want our confidence interval to represent?

```
t = stats.t.ppf(.975 , sample_size-1)
```

```
sample_mean = coinflips_100.mean()
```

```
confidence_interval = (sample_mean - t*stderr, sample_mean + t*stderr)

margin_of_error = t*stderr

print("Sample Mean", sample_mean)
print("Margin of Error:", margin_of_error)
print("Confidence Interval:", confidence_interval)
```

```
confidence_interval[0]
```

```
confidence_interval[1]
```

## Graphically Represent a Confidence Interval

```
import seaborn as sns

sns.kdeplot(coinflips_100)
plt.axvline(x=confidence_interval[0], color='red')
plt.axvline(x=confidence_interval[1], color='red')
plt.axvline(x=sample_mean, color='k');
```

## Relationship between Confidence Intervals and T-tests

Confidence Interval == Bounds of statistical significance for our t-test

A sample mean that falls inside of our confidence interval will "FAIL TO REJECT" our null hypothesis

A sample mean that falls outside of our confidence interval will "REJECT" our null hypothesis

```
from scipy.stats import t, ttest_1samp
```

```
import numpy as np

coinflip_means = []
for x in range(0,100):
  coinflips = np.random.binomial(n=1, p=.5, size=30)
  coinflip_means.append(coinflips.mean())

print(coinflip_means)
```

```
# Sample Size
n = len(coinflip_means)
# Degrees of Freedom
dof = n-1
# The Mean of Means:
mean = np.mean(coinflip_means)
# Sample Standard Deviation
sample_std = np.std(coinflip_means, ddof=1)
# Standard Error
std_err = sample_std/n**.5

CI = t.interval(.95, dof, loc=mean, scale=std_err)
print("95% Confidence Interval: ", CI)
```

```
'''You can roll your own CI calculation pretty easily.
The only thing that's a little bit challenging
is understanding the t stat lookup'''

# 95% confidence interval
t_stat = t.ppf(.975, dof)
print("t Statistic:", t_stat)

CI = (mean-(t_stat*std_err), mean+(t_stat*std_err))
print("Confidence Interval", CI)
```

    t Statistic: 1.9842169515086827
    Confidence Interval (0.48189276007256693, 0.5181072399274331)

A null hypothesis that's just inside of our confidence interval == fail to reject

```
ttest_1samp(coinflip_means, .49)
```

A null hypothesis that's just outside of our confidence interval == reject

```
ttest_1samp(coinflip_means, .4818927)
```

## Run a $\chi^{2}$ Test "by hand" (Using Numpy)

```
df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/adult.csv', na_values=" ?")
print(df.shape)
df.head()
```

```
df.describe()
```

```
df.describe(exclude='number')
```

```
cut_points = [0, 9, 19, 29, 39, 49, 1000]
label_names = ['0-9', '10-19', '20-29', '30-39', '40-49', '50+']
df['hours_per_week_categories'] = pd.cut(df['hours-per-week'], cut_points, labels=label_names)

df.head()
```

```
df['sex'].value_counts()
```

```
df['hours_per_week_categories'].value_counts()
```

```
df = df.sort_values(by='hours_per_week_categories', ascending=True)

df.head()
```

```
contingency_table = pd.crosstab(df['sex'], df['hours_per_week_categories'], margins=True)

contingency_table
```

```
femalecount = contingency_table.iloc[0][0:6].values
femalecount
```

```
malecount = contingency_table.iloc[1][0:6].values
malecount
```

```
import matplotlib.pyplot as plt
import seaborn as sns

#Plots the bar chart
fig = plt.figure(figsize=(10, 5))
sns.set(font_scale=1.8)
categories = ["0-9","10-19","20-29","30-39","40-49","50+"]
p1 = plt.bar(categories, malecount, 0.55, color='#d62728')
p2 = plt.bar(categories, femalecount, 0.55, bottom=malecount)
plt.legend((p2[0], p1[0]), ('Female', 'Male'))
plt.xlabel('Hours per Week Worked')
plt.ylabel('Count')
plt.show()
```

## Expected Value Calculation
\begin{align}
expected_{i,j} =\frac{(row_{i} \text{total})(column_{j} \text{total}) }{(\text{total observations})}
\end{align}

```
# Get Row Sums
row_sums = contingency_table.iloc[0:2, 6].values
col_sums = contingency_table.iloc[2, 0:6].values

print(row_sums)
print(col_sums)
```

```
total = contingency_table.loc['All','All']
total
```

    32561

```
expected = []
for i in range(len(row_sums)):
  expected_row = []
  for column in col_sums:
    expected_val = column*row_sums[i]/total
    expected_row.append(expected_val)
  expected.append(expected_row)

expected = np.array(expected)
print(expected.shape)
print(expected)
```

```
observed = pd.crosstab(df['sex'], df['hours_per_week_categories']).values
print(observed.shape)
observed
```

## Chi-Squared Statistic with Numpy

\begin{align}
\chi^2 = \sum \frac{(observed_{i}-expected_{i})^2}{(expected_{i})}
\end{align}

For the $observed$ values we will just use a version of our contingency table without the margins as a numpy array. In this way, if our observed values array and our expected values array are the same shape, then we can subtract them and divide them directly which makes the calculations a lot cleaner. No for loops!

```
chi_squared = ((observed - expected)**2/(expected)).sum()
print(f"Chi-Squared: {chi_squared}")
```

```
# Calculate Degrees of Freedom
dof = (len(row_sums)-1)*(len(col_sums)-1)
print(f"Degrees of Freedom: {dof}")
```

## Run a $\chi^{2}$ Test using Scipy

```
chi_squared, p_value, dof, expected = stats.chi2_contingency(observed)

print(f"Chi-Squared: {chi_squared}")
print(f"P-value: {p_value}")
print(f"Degrees of Freedom: {dof}")
print("Expected: \n", np.array(expected))
```

Null Hypothesis: Hours worked per week bins is **independent** of sex.

Due to a p-value of 0, we REJECT the null hypothesis that hours worked per week and sex are independent, and conclude that there is an association between hours worked per week and sex.

## Assignment - Build a confidence interval

A confidence interval refers to a neighborhood around some point estimate, the size of which is determined by the desired p-value. For instance, we might say that 52% of Americans prefer tacos to burritos, with a 95% confidence interval of +/- 5%.

52% (0.52) is the point estimate, and +/- 5% (the interval $[0.47, 0.57]$) is the confidence interval. "95% confidence" means a p-value $\leq 1 - 0.95 = 0.05$.

In this case, the confidence interval includes $0.5$ - which is the natural null hypothesis (that half of Americans prefer tacos and half burritos, thus there is no clear favorite). So in this case, we could use the confidence interval to report that we've failed to reject the null hypothesis.

But providing the full analysis with a confidence interval, including a graphical representation of it, can be a helpful and powerful way to tell your story. Done well, it is also more intuitive to a layperson than simply saying "fail to reject the null hypothesis" - it shows that in fact the data does *not* give a single clear result (the point estimate) but a whole range of possibilities.

How is a confidence interval built, and how should it be interpreted? It does *not* mean that 95% of the data lies in that interval - instead, the frequentist interpretation is "if we were to repeat this experiment 100 times, we would expect the average result to lie in this interval ~95 times."

For a 95% confidence interval and a normal(-ish) distribution, you can simply remember that +/-2 standard deviations contains 95% of the probability mass, and so the 95% confidence interval based on a given sample is centered at the mean (point estimate) and has a range of +/- 2 (or technically 1.96) standard deviations.

Different distributions/assumptions (90% confidence, 99% confidence) will require different math, but the overall process and interpretation (with a frequentist approach) will be the same.

Your assignment - using the data from the prior module ([congressional voting records](https://archive.ics.uci.edu/ml/datasets/Congressional+Voting+Records)):

### Confidence Intervals:
1. Generate and numerically represent a confidence interval
2. Graphically (with a plot) represent the confidence interval
3. Interpret the confidence interval - what does it tell you about the data and its distribution?

### Chi-squared tests:
4. Take a dataset that we have used in the past in class that has **categorical** variables. Pick two of those categorical variables and run a chi-squared tests on that data
  - By hand using Numpy
  - In a single line using Scipy

Stretch goals:

1. Write a summary of your findings, mixing prose and math/code/results. *Note* - yes, this is by definition a political topic. It is challenging but important to keep your writing voice *neutral* and stick to the facts of the data. Data science often involves considering controversial issues, so it's important to be sensitive about them (especially if you want to publish).
2. Apply the techniques you learned today to your project data or other data of your choice, and write/discuss your findings here.
3. Refactor your code so it is elegant, readable, and can be easily run for all issues.

```
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats
from scipy.stats import t, ttest_1samp
```

```
df1 = pd.read_csv('house-votes-84.data')
df1.columns = ['Class Name', 'Handicapped_Infants','Water_project_cost_sharing',
              'Adoption_of_the_budget_res',
              'Physician_fee_freeze','El_Salvador_aid', 'Religious_Groups_in_schools',
              'Anti_Satellite_test_ban','Aid_to_Nicaraguan_contras','Mx_missile',
              'Immigration','Synfuels_corporation_cutback','Education_spending',
              'Superfund_right_to_sue','Crime','Duty_free_exports','export_Admin_act_SA']
df1.replace({'y':  1, 'n': 0,'?': np.nan}, inplace=True)
df1.head()
```

```
dems = df1[df1['Class Name'] == 'democrat']
reps = df1[df1['Class Name'] == 'republican']

#Sample Mean and size

dem_crime = dems['Crime'].dropna()
sample_size =  len(dem_crime)

# Sample Standard Deviation and Standard Error

sample_std = np.std(dem_crime)
standard_e = sample_std / (sample_size**(0.5))

# T-stat, margin of error, confidence interval

t = stats.t.ppf(.975 , sample_size-1)

margin_of_error = t*standard_e

confidence_interval = (dem_crime.mean() - t*standard_e, dem_crime.mean() + t*standard_e)

print("Sample Mean", dem_crime.mean())
print("Margin of Error:", margin_of_error)
print("Confidence Interval:", confidence_interval)
```

    Sample Mean 0.35019455252918286
    Margin of Error: 0.05859842293726726
    Confidence Interval: (0.2915961295919156, 0.4087929754664501)

```
sns.kdeplot(dem_crime, color='orange')
plt.axvline(x=confidence_interval[0], color='blue')
plt.axvline(x=confidence_interval[1], color='blue')
plt.axvline(x=dem_crime.mean(), color='g');
```

Using the given data found in the 'Crime" column of dems, the 95% confidence interval was calculated to be:
(0.2915961295919156, 0.4087929754664501)

Any sample mean that falls within this interval will fail to reject the null hypothesis, whereas any sample mean that falls outside of it will reject the null hypothesis.

##Chi Squared Test

```
cut_points = [0, 19, 39, 59, 79, 99, 1000]
label_names = ['0-19', '20-39', '40-59', '60-79', '80-99', '100+']
df['age_cats'] = pd.cut(df['age'], cut_points, labels=label_names)

df.head()
```

      age
      workclass
      fnlwgt
      education
      education-num
      marital-status
      occupation
      relationship
      race
      sex
      capital-gain
      capital-loss
      hours-per-week
      country
      salary
      age_cats

      0
      39
      State-gov
      77516
      Bachelors
      13
      Never-married
      Adm-clerical
      Not-in-family
      White
      Male
      2174
      0
      40
      United-States
      <=50K
      20-39

      1
      50
      Self-emp-not-inc
      83311
      Bachelors
      13
      Married-civ-spouse
      Exec-managerial
      Husband
      White
      Male
      0
      0
      13
      United-States
      <=50K
      40-59

      2
      38
      Private
      215646
      HS-grad
      9
      Divorced
      Handlers-cleaners
      Not-in-family
      White
      Male
      0
      0
      40
      United-States
      <=50K
      20-39

      3
      53
      Private
      234721
      11th
      7
      Married-civ-spouse
      Handlers-cleaners
      Husband
      Black
      Male
      0
      0
      40
      United-States
      <=50K
      40-59

      4
      28
      Private
      338409
      Bachelors
      13
      Married-civ-spouse
      Prof-specialty
      Wife
      Black
      Female
      0
      0
      40
      Cuba
      <=50K
      20-39

```
df1 = pd.crosstab(df['salary'], df['age_cats'], margins = True)
df1
```

      age_cats
      0-19
      20-39
      40-59
      60-79
      80-99
      All

      salary

      <=50K
      1655
      13849
      7224
      1887
      105
      24720

      >50K
      2
      2818
      4369
      636
      16
      7841

      All
      11593
      1657
      16667
      2523
      121
      32561

```
row_sums = df1.iloc[0:2, 5].values
col_sums = df1.iloc[2, 0:5].values

total = df1.loc['All','All']

expected = []
for i in range(len(row_sums)):
  expected_row = []
  for column in col_sums:
    expected_val = column*row_sums[i]/total
    expected_row.append(expected_val)
  expected.append(expected_row)

expected = np.array(expected)
print(expected.shape)
print(expected)

observed = pd.crosstab(df['salary'], df['age_cats']).values
print(observed.shape)
observed
```

    (2, 5)
    [[ 8801.29480053  1257.97856331 12653.42710605  1915.43748656
         91.86204355]
     [ 2791.70519947   399.02143669  4013.57289395   607.56251344
         29.13795645]]
    (2, 5)

    array([[ 1655, 13849,  7224,  1887,   105],
           [    2,  2818,  4369,   636,    16]])

```
chi_squared = ((observed - expected)**2/(expected)).sum()
print(f"Chi-Squared: {chi_squared}")

dof = (len(row_sums)-1)*(len(col_sums)-1)
print(f"Degrees of Freedom: {dof}")
```

    Chi-Squared: 2172.824317596846
    Degrees of Freedom: 4

```
chi_squared, p_value, dof, expected = stats.chi2_contingency(observed)

print(f"Chi-Squared: {chi_squared}")
print(f"P-value: {p_value}")
print(f"Degrees of Freedom: {dof}")
print("Expected: \n", np.array(expected))
```

    Chi-Squared: 2172.824317596846
    P-value: 0.0
    Degrees of Freedom: 4
    Expected:
     [[ 1257.97856331 12653.42710605  8801.29480053  1915.43748656
         91.86204355]
     [  399.02143669  4013.57289395  2791.70519947   607.56251344
         29.13795645]]

## Resources

- [Interactive visualize the Chi-Squared test](https://homepage.divms.uiowa.edu/~mbognar/applets/chisq.html)
- [Calculation of Chi-Squared test statistic](https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test)
- [Visualization of a confidence interval generated by R code](https://commons.wikimedia.org/wiki/File:Confidence-interval.svg)
- [Expected value of a squared standard normal](https://math.stackexchange.com/questions/264061/expected-value-calculation-for-squared-normal-distribution) (it's 1 - which is why the expected value of a Chi-Squared with $n$ degrees of freedom is $n$, as it's the sum of $n$ squared standard normals)


# Heat transfer for pipes

```python
"""
importing the necessary libraries, do not modify
"""
%matplotlib inline

from IPython.display import clear_output

import schemdraw as schem
import schemdraw.elements as e

import matplotlib.pyplot as plt
import numpy as np
import math
import scipy.constants as sc

import sympy as sym

```

Fig. 1: Illustration of internal convection.

The above sketch illustrates the focus of this notebook: How to quantify the heat transfer between a pipe, in which a fluid flows, and its surroundings. The heat transfer from the outer surface of the pipe to the outer flow is to defined in the previous chapter, external convection. In the following, this notebook establishes the tools necessary to solve the internal convection problem.

## Entry flow and fully developed internal flow

Fig. 2: Pipe flow nomenclature.

### Python module

For internal flow, the module is loaded as:

```python
from Libraries import HT_internal_convection as intconv

```

As an example, consider the flow of water in a pipe of diameter $D=10$ cm, length $L=10$m. The water thermodynamic properties are estimated at $T_f=50^\circ$C. The bulk velocity is $U_m=2$m/s.

```python
from Libraries import thermodynamics as thermo
T_f = 50 #C
waterflow = thermo.Fluid('water',T_f,"C")

L_pipe = 10. #m
D_pipe = 0.1 #m
Um_pipe = 2 #m/s
```

```python
?intconv.PipeFlow
```

```python
pipe = intconv.PipeFlow(D= D_pipe, L=L_pipe,
                        rho=waterflow.rho, nu=waterflow.nu, Um=Um_pipe)
```

 Fig. 3. Friction factor in pipe flow as a function of Re and relative surface roughness.

A uniform flow entering a pipe (Fig. 2) first experiences streamwise variation of velocity to accommodate the wall boundary conditions. A boundary layer, of thickness $\delta$, forms on the wall and grows until its edge reaches the pipe centerline. This region is the hydrodynamic entrance region. Beyond that point, the flow becomes fully developed, which means that

 In the laminar regime, the velocity profile is only a function of $r$,

 In the turbulent regime, the mean velocity profile is only a function of $r$.

Friction drag or the force exerted by the flow onto the pipe wall governs the pressure gradient necessary to generate a desired flowrate. Calculation of the friction drag leads to the design of the mechanical force creating the pressure gradient. In fully developed (laminar or turbulent) regimes, the pressure gradient may be determined by

$$
-\frac{\Delta\overline{P}}{L}=f\,\frac{1}{D}\,\frac{\rho U_m^2}{2}
$$

where $D=2R$ and $L$ are the diameter and length of the pipe, respectively, and $f$ is the friction factor. The bulk velocity or average velocity is

$$
U_m=\frac{\dot{m}}{\rho A_c}
$$

where $\dot{m}$ is the mass flux
$$
\dot{m}=\int_0^{2\pi}\int_0^R\rho \overline{u}(r)\,r\,dr d\theta=2\pi\int_0^R\rho \overline{u}(r)\,r\,dr
$$
and $A_c=\pi R^2$
The Reynolds number of the flow is based on the bulk velocity and pipe diameter:

$$
Re_D=\frac{\rho U_mD}{\mu}=\frac{4\dot{m}}{\pi D\mu}
$$

The friction factor in the laminar regime is rigorously derived:
$$
f = \frac{64}{Re_D}
$$

and is valid up to the critical Reynolds number $Re_{D,c}$, which in most pipe is around 2,000. Be aware that in certain research facilities, the flow can remain laminar for Reynolds numbers up to 10,000. The Reynolds 2,000 is not absolute, universal property, but is the best guess from most engineering applications.

Beyond the critical Reynolds number, $f$ is a function of the roughness to diameter ratio $\varepsilon=e/D$ (e is typically the standard deviation of the roughness height) and the Reynolds number. A trustworthy empirical correlation is the Colebrook formula:

$$
\frac{1}{\sqrt{f}}=-2\log_{10}\left[\frac{\varepsilon}{3.7}+\frac{2.51}{Re_D\sqrt{f}}\right]
$$

which is solved below for a range of relative roughness $\varepsilon$.

Often there is a need to determine the pump or blower power $P$ necessary to move the flow at a prescribed pressure drop:

$$
P=\frac{\dot{m}}{\rho}\Delta p= \underbrace{(\Delta p)A_c}_\text{force}\cdot U_m
$$

### Example of functions
Going back to our library, let's explore how to determine some of the properties defined above:

Reynolds number:

```python
print("Re= %1.2e" %pipe.Re)
```

    Re= 3.61e+05

Mass flow rate:

```python
print("mass flowrate= %1.1f kg/s" %pipe.mdot)
```

    mass flowrate= 15.5 kg/s

Compute the friction factor:

```python
# pipe.f_turbulent()
pipe.f_laminar()
```

```python
print("f= %1.5f" %pipe.f)
```

    f= 0.00018

The mean pressure gradient is:

```python
print("-dP/dx= %1.0f Pa/m" %pipe.dPdx)
```

    -dP/dx= 35 Pa/m

## Heat transfer by internal convection

The temperature is expected to vary both in the streamwise direction and in the radial direction. To reduce the complexity of the problem, we define the mean temperature as:
$$
T_m=\frac{1}{\dot{m}C_p}\int_{A_c}\rho\,u\,C_p\, T\,dA_c
$$
where $\dot{m}$ is the mass flow rate, $rho$ and $C_p$ are the density and specific heat of the fluid and $A_c$ is the cross-sectional area of the pipe.
The local heat flux may be now expressed as:
$$
q_s''=h(T_s-T_m)
$$
where $h$ is the local convection heat transfer coefficient and $T_s$ is the surface temperature on the inner wall of the pipe. The variation of temperature in the fully developed flow can be shown to be governed by the following ODE:

$$
\frac{dT_m}{dx}=\frac{P}{\dot{m}C_p}h(T_s-T_m)
$$

where $P$ is the perimeter of the pipe.
If the local heat flux is maintained constant over the length of the pipe $L$, the total heat rate is

$$
q_\text{conv}=(PL)q_s''\, \text{$q_s''=$constant}
$$

and the streamwise distribution of the mean temperature is linear:
$$
T_m(x)=T_{m,i}+\frac{q_s''P}{\dot{m}C_p}x,\, \text{$q_s''=$constant}
$$
For the case of constant wall temperature $T_s$, the temperature distribution is the solution of the above ODE, thus of exponential nature. For practical applications, you most always need to compute the overall heat transfer and the outlet mean temperature $T_{m,o}$. The integration of the above ODE for $x=0$ to $x=L$ yields

$$
\frac{T_s-T_{m,o}}{T_s-T_{m,i}}=\exp\left(-\frac{PL}{\dot{m}C_p}\overline{h}\right),\, \text{$T_s=$constant}
$$

where
$$
\overline{h}=\frac{1}{L}\int_0^L h(x)dx
$$
If you must compute the mean temperature at $x$ an integration from $0$ to $x$ yields
T_mx_Ts_constant(T_s,T_mi,P,L,mdot,Cp,hbar,x)

$$
\frac{T_s-T_{m}(x)}{T_s-T_{m,i}}=\exp\left(-\frac{PL}{\dot{m}C_p}\overline{h}_x\right),\, \text{$T_s=$constant}
$$

where
$$
\overline{h}_x=\frac{1}{L}\int_0^x h(x')dx'
$$
The computation of the total heat transfer rate can be shown to write:

$$
q_\text{conv}=\overline{h}(PL)\Delta T_\text{lm},\, \text{$T_s=$constant}
$$

with the log mean temperature
log_mean_temperature(T_s,T_o,T_i)

$$
\Delta T_\text{lm}=\cfrac{T_{m,i}-T_{m,o}}{\ln\left(\cfrac{T_s-T_{m,o}}{T_s-T_{m,i}}\right)}
$$

In many problem, $T_s$ is not defined but the outside ambient temperature $T_\infty$, the thermal conductivity of the pipe is known. One needs to determine the total resistance of the system $R_\text{tot}$, which requires calculating the heat transfer coefficient of the forced or natural convection, occuring on the outside of the pipe, the radiation coefficient if needed, the thermal resistance due by conduction within the pipe, which may include multiple components in the presence of insulation for example, and the internal convection heat transfer coefficient (to be defined below). In such cases, the variation of temperature between inlet and outlet becomes:
T_mo_T_infty(T_infty,T_mi,P,L,mdot,Cp,R_tot)

$$
\frac{T_\infty-T_{m,o}}{T_\infty-T_{m,i}}=\exp\left(-\frac{1}{\dot{m}C_pR_\text{tot}}\right)
$$

and the total heat transfer rate is

$$
q=\frac{\Delta T_\text{lm}}{R_\text{tot}}
$$

The equations derived in this cell enable:

 The computation of the internal convection heat transfer coefficient if $T_{m,i}$ and $T_{m,o}$ are known.

 The computation of $T_{m,i}$ or $T_{m,o}$ if one is known and $\overline{h}$ is known

 The computation of the required mass flux to achieve given $T_{m,i}$ and $T_{m,o}$, albeit through an iterative process

## Correlations for convection heat transfer coefficients in internal pipe flows

Here we detailed only the correlations for fully developed flows. For laminar flows, the nusselt numbers are constant, thus the library HT_internal_convection provides directly $\overline{h}$:
laminar_isoflux()

$$
Nu=\frac{hD}{k}=4.36,\, \text{$q_s''=$constant}
$$

laminar_isothermal()

$$
Nu=\frac{hD}{k}=4.36,\, \text{$q_s''=$constant}
$$

```python
pipe.laminar_isoflux()
print("Nu= %1.2f for laminar isoflux" %pipe.Nu)
pipe.laminar_isothermal()
print("Nu= %1.2f for laminar isothermal" %pipe.Nu)
```

    Nu= 4.36 for laminar isoflux
    Nu= 3.66 for laminar isothermal

In turbulent flows, there is a choice of correlations:
Dittus_Boelter(Re,Pr,mode)

$$
Nu=\frac{hD}{k}=0.023Re^{4/5}Pr^n
$$

with mode being either 'cooling' or 'heating'

```python
pipe.Dittus_Boelter(mode='cooling',Pr=waterflow.Pr)
print("Nu= %1.0f for cooling" %pipe.Nu)
pipe.Dittus_Boelter(mode='heating',Pr=waterflow.Pr)
print("Nu= %1.0f for heating" %pipe.Nu)
```

    Nu= 940 for cooling
    Nu= 1067 for heating

Sieder_Tate(Re,Pr,mu,mu_s)

$$
Nu=\frac{hD}{k}=0.027Re^{4/5}Pr^{1/3}\left(\cfrac{\mu}{\mu_s}\right)^{0.14}
$$

```python
T_s = 75 #C
watersurface = thermo.Fluid('water',thermo.C2K(T_s))
pipe.Sieder_Tate(mu=waterflow.mu,mu_s=watersurface.mu,Pr=waterflow.Pr)
print("Nu= %1.0f" %pipe.Nu)
```

    Nu= 1213

Gnielinski(Re,Pr,f)

$$
Nu=\frac{hD}{k}=\frac{(f/8)(Re-1000)Pr}{1+12.7(f/8)^{1/2}(Pr^{2/3}-1)}
$$

```python
pipe.Gnielinski(f=pipe.f, Pr=waterflow.Pr)
print("Nu= %1.0f" %pipe.Nu)
```

    Nu= 26

Skupinski(Re,Pr)

$$
Nu=\frac{hD}{k}=4.82+0.0185\left(Re\,Pr\right)^{0.827},\, \text{$q_s''=$constant}
$$

```python
pipe.Skupinski(Pr=waterflow.Pr)
print("Nu= %1.0f" %pipe.Nu)
```

    Nu= 2089

Seban(Re,Pr)

$$
Nu=\frac{hD}{k}=5.0+0.025\left(Re\,Pr\right)^{0.8},\, \text{$T_s=$constant}
$$

```python
pipe.Seban(Pr=waterflow.Pr)
print("Nu= %1.0f" %pipe.Nu)
```

    Nu= 1931

## Natural convection around cylinder

Fig. 4: Illustration of the flow induced by natural convection around a cylinder. Insert shows the angular distribution of the local Nu.

In a fluid entirely at rest, a heated surface transfers its heat via pure conduction. Natural convection is the enhanced heat transfer between a body of fluid at rest (at infinity) and a heated surface through the creation of a convective flow driven by buoyancy forces. Fig. 4 illustrates a natural convection flow occuring around a cylinder. The fluid at the bottom of the cylinder $\theta=0$ becomes buoyant through heat transfer between the cylinder and the fluid and rises along the surface of the cylinder. This process creates two boundary layers that merge at $\theta = \pi$ to create a vertical jet-like flow, also called a plume. Plumes are characteristic flows of natural convection, i.e. they are found irrespective of the geometry of the heated object.

The library is called in the following way:

```python
from Libraries import HT_natural_convection as natconv
```

The non-dimensional numbers relevant to natural convection are:
the Grashof number
Grashof(g,beta,DT,D,nu)

$$
Gr = \frac{g\beta(\Delta T)D^3}{\nu^2}
$$

and the Rayleigh number
Rayleigh(g,beta,DT,D,nu,alpha)

$$
Ra = Gr.Pr= \frac{g\beta(\Delta T)D^3}{\nu\alpha}
$$

where $g$ is the gravity magnitude, $\beta$ is the volumetric thermal expansion coefficient at a given pressure $p$
$$
\beta = -\frac{1}{\rho}\left(\frac{\partial\rho}{\partial T}\right)_p
$$
$\Delta T$ is the absolute temperature difference between the heated surface temperature $T_s$ and the fluid temperature at infinity $T_\infty$, $\Delta T= \vert T_s-T_\infty\vert$, $D$ is the characteristic length of the system (here the diameter) and $\nu$ and $\alpha$ are the kinematic viscosity and the thermal diffusivity, both of dimensions $\text{m$^2$/s}$.
Note that for the ideal gas law
$$
p =\rho \frac{R}{M}T\text{ or } \rho = \frac{p}{\frac{R}{M}T}
$$
thus the expansion coefficient is

$$
\beta = \frac{1}{T}\text{ for an ideal gas, $T$ in K}
$$

For a liquid, $\beta$ must be interpolated from a table. All thermodynamics quantities involved are to be defined at the film temperature which is the arithmetic mean

$$
T_f=\frac{T_s+T_\infty}{2}
$$

```python
#air
T_infty = 10#C
T_s = 50#C
D = 0.1#m
T_f = (T_s+T_infty)/2
airflow = thermo.Fluid('air',T_f,"C")
Gr= natconv.Gr(beta=airflow.beta,D=D,DT=T_s-T_infty,nu=airflow.nu)
print('Natural convection Gr= %1.2e'%Gr)
Ra= natconv.Ra(alpha=airflow.alpha,beta=airflow.beta,D=D,DT=T_s-T_infty,nu=airflow.nu)
print('Natural convection Ra= %1.2e'%Ra)
```

    Natural convection Gr= 5.03e+06
    Natural convection Ra= 3.57e+06

The Grashof and Rayleigh number quantify the ratio of buoyancy to viscous forces. When they are large enough, a convective flow sets in and the heat transfer increases in comparison to pure conduction. The Nusselt number, ratio of convective to conduction heat transfer (i.e. $>1$ in the presence of a convection flow) is typically a power law of the Rayleigh number. In the case of the flow around a cylinder with isothermal surface temperature, there are two correlations:
Morgan(Ra)

$$
\overline{Nu}=\frac{\overline{h}D}{k}=C\,Ra^n
$$

Churchill-Chu(Ra,Pr)

$$
\overline{Nu}=\frac{\overline{h}D}{k}=\left[0.60+\frac{0.387Ra^{1/6}}{\left[1+\left(\frac{0.559}
{Pr}\right)^{9/16}\right]^{8/27}}
\right]^2
$$

Both are valid for $Ra\leq10^{12}$. The Nusselt is averaged over the perimeter of the cylinder to account for the angular variation of heat transfer discussed earlier. The heat transfer from natural convection from a heated cylinder of diameter $D$ and length $L$ is

$$
q=\overline{h}(\pi DL)(T_s-T_\infty)=\frac{1}{R_\text{th,conv}}(T_s-T_\infty)
$$

where $R_\text{th,conv}$ may computed with R_th_convection(h,A)

```python
airnatconv = natconv.HorizontalCylinder(correlation='Morgan',Ra=Ra)
print("Morgan correlation: Nu= %1.2f" %airnatconv.Nu)
airnatconv = natconv.HorizontalCylinder(correlation='Churchill-Chu',Ra=Ra,Pr=airflow.Pr)
print("Churchill-Chu correlation: Nu= %1.2f" %airnatconv.Nu)
```

    Morgan correlation: Nu= 20.87
    Churchill-Chu correlation: Nu= 20.91

```python
font = {'family' : 'serif',
        #'color'  : 'black',
        'weight' : 'normal',
        'size'   : 14,
        }

from matplotlib.ticker import FormatStrFormatter
plt.rc('font', **font)
N = 100
Ra = np.logspace(5,12,N)
Nu_Morgan = np.zeros(N)
Nu_ChurchillChu = np.zeros(N)
Pr = 1.0
for i in range(N):
    flow = natconv.HorizontalCylinder(correlation='Morgan',Ra=Ra[i])
    Nu_Morgan[i] = flow.Nu
    flow = natconv.HorizontalCylinder(correlation='Churchill-Chu',Ra=Ra[i],Pr=Pr)
    Nu_ChurchillChu[i] = flow.Nu

plt.loglog(Ra,Nu_Morgan, label = r"Morgan",lw = 2)
plt.loglog(Ra,Nu_ChurchillChu, label = r"Churchill-Chu", lw= 2)
plt.xlabel(r"$Ra$")
plt.ylabel(r"$Nu$")
plt.legend(loc=3, bbox_to_anchor=[0., 1.01], ncol=2, shadow=False, fancybox=True)
plt.show()
```

```python
plt.plot(Ra,np.abs(Nu_Morgan-Nu_ChurchillChu)/Nu_ChurchillChu,lw = 2)
plt.xlabel(r"$Ra$")
plt.ylabel(r"$\vert Nu_{M}-Nu_{CC}\vert/Nu_{CC}$")
plt.show()
```

## Assignment

 Read this entire notebook. Using the textbook, add restrictions and range of validity for the above correlations when applicable. Add the entry length Nu correlation for laminar flow

 Add a section on entrance flow

 How should the entrance flow region be treated in turbulent flows?

Solve 8.31, 8.36, 8.43

### 8.31

To cool a summer home without using a vapor-compression refrigeration cycle, air is routed through a plastic pipe ($k=0.15\text{ W/m.K}$, $D_i=0.15\text{ m}$, $D_o=0.17\text{ m}$) that is submerged in an adjoini
ng body of water. The water temperature is nominally at $T_\infty= 17^\circ\text{C}$, and a convection coefﬁcient of $h_o\approx 1500\text{ W/m$^2$. K}$ is maintained at the outer surface of the pipe.

If air from the home enters the pipe at a temperature of $T_{m,i}= 29^\circ\text{C}$ and a volumetric ﬂow rate of $\dot{\forall}_i= 0.025\text{ m$^3$/s}$, what pipe length $L$ is needed to provide a discharge temperature of $T_{m,o}=21^\circ\text{C}$? What is the fan power required
to move the air through this length of pipe if its inner surface is smooth?

#### Solution

The length of the pipe is the given by solving
$$
\frac{T_\infty-T_{m,o}}{T_\infty-T_{m,i}}=\exp\left(-\frac{1}{\dot{m}C_pR_\text{tot}}\right)
$$
for the target outlet temperature $T_{m,o}$. First, assuming 1D, steady convection on the outside of the pipe, we must solve for $R'_{tot}$. Since
$$
R_{tot}=\frac{R'_{tot}}{L}
$$
the pipe length is
$$
L=-\dot{m}C_pR'_\text{tot}\ln\frac{T_\infty-T_{m,o}}{T_\infty-T_{m,i}}
$$

```python
from Libraries import HT_thermal_resistance as res

Rp = []
Rp.append(res.Resistance("$R'_{conv,i}$","W/m"))
Rp.append(res.Resistance("$R'_{cond,pipe}$","W/m"))
Rp.append(res.Resistance("$R'_{conv,o}$","W/m"))

d = schem.Drawing()
d.add(e.DOT, label = r"$T_{m,i}$")
d.add(e.RES, d = 'right', label = Rp[0].name)
d.add(e.DOT, label = r"$T_{s,i}$")
R1 = d.add(e.RES, d = 'right', label = Rp[1].name)
d.add(e.DOT, label = r"$T_{s,o}$")
d.add(e.RES, d='right', label = Rp[2].name)
d.add(e.DOT, label="$T_\infty$")
L1 = d.add(e.LINE, toplabel = "$q'$", endpts = [[-2.25, 0], [-0.25, 0]])
d.labelI(L1, arrowofst = 0)
d.draw()
```

```python
from Libraries import thermodynamics as thermo
from Libraries import HT_internal_convection as intconv

k_pipe = 0.15 #W/m.K
Di = 0.15 #m
Do = 0.17 #m
T_infty = 17. #C
h_o = 1500 #W/m^2.K
T_mi = 29 #C
T_mo = 21 #C

Qdot = 0.025 #m^3/s
T_m = (T_mi + T_mo)/2

airi = thermo.Fluid('air',T_mi,"C")
airm = thermo.Fluid('air', T_m,"C")
airflow = intconv.PipeFlow(D=Di, L = 1., mdot = airi.rho*Qdot, nu = airm.nu, rho = airi.rho)
airflow.Dittus_Boelter(mode='cooling',Pr=airm.Pr)
print("Re=%.0f" %airflow.Re)
print("Nu=%.0f" %airflow.Nu)
hbar_i = airflow.Nu*airm.k/Di
print("hbar,i=%.2f W/m^2.K" %hbar_i)
Rp[0].convection(hbar_i,np.pi*Di)
Rp[1].cond_cylinder(k = k_pipe,ra=Di,rb=Do,L=1)
Rp[2].convection(400,A=np.pi*Do)

Rptot = 0
for i in range(3):
    Rptot += Rp[i].R

# def L_given_other_params(T_infty,T_mo,T_mi,mdot,Cp,Rptot):
#     return -mdot*Cp*Rptot*np.log((T_infty -T_mo)/(T_infty - T_mi))
# L = L_given_other_params(T_infty,T_mo,T_mi,airi.rho*Qdot,airm.Cp,Rptot)
L = intconv.L_given_other_params(T_infty,T_mo,T_mi,airi.rho*Qdot,airm.Cp,Rptot)
print("Length needed to achieve T_mo=%.0f C is %.1f m" %(T_mo,L))
```

    Re=13625
    Nu=42
    hbar,i=7.32 W/m^2.K
    Length needed to achieve T_mo=21 C is 13.8 m

```python
from Libraries import HT_natural_convection as natconv
T_f = (T_infty + T_m)/2
water = thermo.Fluid("water",T_f,"C")
Ra = natconv.Ra(beta=water.beta,DT=T_m - T_infty, D=Do,nu=water.nu,alpha = water.alpha)
print("Ra=%.2e" %(Ra))
waterconv = natconv.HorizontalCylinder("Churchill-Chu",Ra,water.Pr)
print("Nu=%.0f" %waterconv.Nu)
print("For natural convection, h_o=%.0f W/m^2.K" %(waterconv.Nu*water.k/Do))
# waterforced = extconv.CircularCylinder()
```

    Ra=5.94e+08
    Nu=124
    For natural convection, h_o=436 W/m^2.K

This little exercise demonstrates that natural convection does not achieve the cooling capacity assumed in the problem ($h_o=1500\mathrm{W}/\mathrm{m}^2.K$)

```python
from Libraries import HT_natural_convection as natconv
?natconv.HorizontalCylinder
```

### 8.36

Hot water at mean temperature $T_m=50\text{$^\circ$C}$ is routed from one building in which it is generated to an adjoining building in which it is used for space heating. Transfer between the buildings occurs in a steel pipe ($k=60\text{ W/m.K}$) of $100 \text{ mm}$ outside diameter and 8-mm wall thickness. During the winter, representative environmental conditions involve air at $T_\infty= -5^\circ \mathrm{C}$ and $V_\infty=3\text{ m/s}$ in cross ﬂow over the pipe.
Using the Churchill Bernstein and Dittus Boehler correlations, calculate the total heat transfer rate per unit length $q'$, the daily energy cost $Q'=q'\times 24\text{ h/d}$ per meter and the cost per day and per meter assuming an electricity cost of $\text{\$}0.05\text{/kW.h}$.

**FYI:** This is the Churchill-Bernstein correlation which you can call with the `from Libraries import HT_external_convection as extconv` `airflow=extconv.CircularCylinder('Churchill-Bernstein',Re,Pr)`
$$
Nu_D = \frac{hD}{k_f}=0.3+\frac{0.62Re_D^{1/2}Pr^{1/3}}{\left[1+\left(\frac{0.4}{Pr}\right)^{2/3}\right]^{1/4}}\left[1+\left(\frac{Re_D}{282,000}\right)^{5/8}\right]^{4/5}
$$

The heat transfer problem in any cross sectional area of the pipe is
$$
q' = \frac{T_m - T _\infty}{R'_{tot}}
$$

with

$$
R'_{tot}= R'_{conv,int} + R'_{cond,p}+R'_{conv,ext}
$$

We must find the convection coefficients $h_{int}$ and $h_{ext}$, using the appropriate correlations.

```python
Tm = 50 #C
Um = 0.5 #m/s

Di = 0.084 #m
Do = 0.1 #m
kp = 60 #W/m.K

T_infty = -5 #C
U_infty = 3 #m/s

from Libraries import HT_thermal_resistance as res
Rp = []
Rp.append(res.Resistance("$R'_{conv,int}","W/m"))
Rp.append(res.Resistance("$R'_{cond,p}","W/m"))
Rp.append(res.Resistance("$R'_{conv,ext}","W/m"))

# internal convection
from Libraries import thermodynamics as thermo
from Libraries import HT_internal_convection as intconv

water = thermo.Fluid('water',Tm,"C")
pipeflow = intconv.PipeFlow(D=Di,L=1,Um=Um,nu=water.nu)
print("Re_D_pipe= %.0f" %pipeflow.Re)
pipeflow.Dittus_Boelter(mode='cooling',Pr=water.Pr)
hint = pipeflow.Nu*water.k/Di
print("hint=%.1f W/m^2.K" %hint)
Rp[0].convection(h=hint,A=np.pi*Di)

# conduction
Rp[1].cond_cylinder(k=kp,ra=Di,rb=Do,L=1.)

# external convection

#guess for surface temperature at D=Do
T_so = 49.21 #C
T_f = (T_infty + T_so)/2
air = thermo.Fluid('air',T_f,"C")
Re_air = U_infty * Do/air.nu
# print(Re_air)
from Libraries import HT_external_convection as extconv
airflow = extconv.CircularCylinder('Churchill-Bernstein',Re_air,air.Pr)
hext = airflow.Nu*air.k/Do
print("hext=%.1f W/m^2.K" %hext)
Rp[2].convection(h=hext,A=np.pi*Do)

# total thermal resistance
Rptot = 0.
for i in range(3):
    Rptot += Rp[i].R

qp = (Tm - T_infty)/Rptot

print("Heat rate per unit length: %.0f W/m" %qp)

#New estimate of T_so
T_so = T_infty + qp*Rp[2].R
print("New T_so = %.2f C" %T_so)
```

```python
Tm = 50 #C
Um = 0.5 #m/s

Di = 0.084 #m
Do = 0.1 #m
kp = 60 #W/m.K

T_infty = -5 #C
U_infty = 3 #m/s

from Libraries import HT_thermal_resistance as res
Rp = []
Rp.append(res.Resistance("$R'_{conv,int}","W/m"))
Rp.append(res.Resistance("$R'_{cond,p}","W/m"))
Rp.append(res.Resistance("$R'_{conv,ext}","W/m"))

# internal convection
from Libraries import thermodynamics as thermo
from Libraries import HT_internal_convection as intconv

water = thermo.Fluid('water',Tm,"C")
pipeflow = intconv.PipeFlow(D=Di,L=1,Um=Um,nu=water.nu)
print("Re_D_pipe= %.0f" %pipeflow.Re)
pipeflow.Dittus_Boelter(mode='cooling',Pr=water.Pr)
hint = pipeflow.Nu*water.k/Di
print("hint=%.1f W/m^2.K" %hint)
Rp[0].convection(h=hint,A=np.pi*Di)

# conduction
Rp[1].cond_cylinder(k=kp,ra=Di,rb=Do,L=1.)

# external convection

# initial guess for surface temperature at D=Do
T_so = 0. #C
errT = np.inf
iteration = 0
while (errT > 1.0) and (iteration < 10):
    iteration += 1
    T_so_old = T_so
    T_f = (T_infty + T_so)/2
    air = thermo.Fluid('air',T_f,"C")
    Re_air = U_infty * Do/air.nu
    # print(Re_air)
    from Libraries import HT_external_convection as extconv
    airflow = extconv.CircularCylinder('Churchill-Bernstein',Re_air,air.Pr)
    hext = airflow.Nu*air.k/Do
    print("hext=%.1f W/m^2.K" %hext)
    Rp[2].convection(h=hext,A=np.pi*Do)

    # total thermal resistance
    Rptot = 0.
    for i in range(3):
        Rptot += Rp[i].R

    qp = (Tm - T_infty)/Rptot

    print("Heat rate per unit length: %.0f W/m" %qp)

    #New estimate of T_so
    T_so = T_infty + qp*Rp[2].R
    print("New T_so = %.2f C" %T_so)
    errT = abs(T_so - T_so_old)
    print("errT=%.3e" %errT)
```

```python
Qp = qp*1e-3*24
print("Daily energy loss: %.3f kW.h/d/m" %Qp)
Cp = Qp * 0.05
print("Cost: $%.3f /m.d " %Cp)
```

### 8.42

Atmospheric air enters a $10\text{ m}$-long, $150\text{ mm}$-diameter uninsulated heating duct at $60\text{$^\circ$C}$ and $0.04\text{ kg/s}$. The duct surface temperature is approximately constant at $Ts=15\text{$^\circ$C}$.

(a) What are the outlet air temperature, the heat rate q, and pressure drop $\Delta p$ for these conditions?

(b) To illustrate the tradeoff between heat transfer rate and pressure drop considerations, calculate $q$ and $\Delta p$ for diameters in the range from $0.1$ to $0.2\text{ m}$. In your analysis, maintain the total surface area,
$A_s=\pi DL$, at the value computed for part (a). Plot $q$, $\Delta p$, and $L$ as a function of the duct diameter.

```python

```

```python
Tm = 50 #C
Um = 0.5 #m/s

Di = 0.084 #m
Do = 0.1 #m
kp = 60 #W/m.K

T_infty = -5 #C
U_infty = 3 #m/s

from Libraries import HT_thermal_resistance as res
Rp = []
Rp.append(res.Resistance("$R'_{conv,int}","W/m"))
Rp.append(res.Resistance("$R'_{cond,p}","W/m"))
Rp.append(res.Resistance("$R'_{conv,ext}","W/m"))

# internal conduction
from Libraries import HT_internal_convection as intconv
water = thermo.Fluid('water',Tm,"C")

pipeflow = intconv.PipeFlow(D=Di,L=1,Um=Um,nu=water.nu)
print(pipeflow.Re,water.Pr)
pipeflow.Dittus_Boelter(mode='cooling',Pr=water.Pr)
print(pipeflow.Nu*water.k/Di)
Rp[0].convection(h=pipeflow.Nu*water.k/Di,A=np.pi*Di)

#conduction
Rp[1].cond_cylinder(k=kp,ra=Di,rb=Do)

# external convection
from Libraries import HT_external_convection as extconv
T_so = 49.2
T_fo = (T_infty + T_so)/2
air = thermo.Fluid('air',T_fo,"C")
Re_air = U_infty*Do/air.nu
airflow = extconv.CircularCylinder('Churchill-Bernstein',Re_air,air.Pr)
Rp[2].convection(airflow.Nu*air.k/Do,np.pi*Do)
print(airflow.Nu*air.k/Do)
Rptot = 0
for i in range(3):
    Rptot += Rp[i].R
    print(Rp[i].R)
qp = (Tm - T_infty)/Rptot
print(qp)
T_so_1 = T_infty + qp*Rp[2].R
print(T_so_1)
```


# Optimización media-varianza

La **teoría de portafolios** es una de los avances más importantes en las finanzas modernas e inversiones.
- Apareció por primera vez en un [artículo corto](https://www.math.ust.hk/~maykwok/courses/ma362/07F/markowitz_JF.pdf) llamado "Portfolio Selection" en la edición de Marzo de 1952 de "the Journal of Finance".
- Escrito por un desconocido estudiante de la Universidad de Chicago, llamado Harry Markowitz.
- Escrito corto (sólo 14 páginas), poco texto, fácil de entender, muchas gráficas y unas cuantas referencias.
- No se le prestó mucha atención hasta los 60s.

Finalmente, este trabajo se convirtió en una de las más grandes ideas en finanzas, y le dió a Markowitz el Premio Nobel casi 40 años después.
- Markowitz estaba incidentalmente interesado en los mercados de acciones e inversiones.
- Estaba más bien interesado en entender cómo las personas tomaban sus mejores decisiones cuando se enfrentaban con "trade-offs".
- Principio de conservación de la miseria. O, dirían los instructores de gimnasio: "no pain, no gain".
- Si queremos más de algo, tenemos que perder en algún otro lado.
- El estudio de este fenómeno era el que le atraía a Markowitz.

De manera que nadie se hace rico poniendo todo su dinero en la cuenta de ahorros. La única manera de esperar altos rendimientos es si se toma bastante riesgo. Sin embargo, riesgo significa también la posibilidad de perder, tanto como ganar.

Pero, ¿qué tanto riesgo es necesario?, y ¿hay alguna manera de minimizar el riesgo mientras se maximizan las ganancias?
- Markowitz básicamente cambió la manera en que los inversionistas pensamos acerca de esas preguntas.
- Alteró completamente la práctica de la administración de inversiones.
- Incluso el título de su artículo era innovador. Portafolio: una colección de activos en lugar de tener activos individuales.
- En ese tiempo, un portafolio se refería a una carpeta de piel.
- En el resto de este módulo, nos ocuparemos de la parte analítica de la teoría de portafolios, la cual puede ser resumida en dos frases:
 - No pain, no gain.
 - No ponga todo el blanquillo en una sola bolsa.

**Objetivos:**
- ¿Qué es la línea de asignación de capital?
- ¿Qué es el radio de Sharpe?
- ¿Cómo deberíamos asignar nuestro capital entre un activo riesgoso y un activo libre de riesgo?

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Línea de asignación de capital

### 1.1. Motivación

El proceso de construcción de un portafolio tiene entonces los siguientes dos pasos:
1. Escoger un portafolio de activos riesgosos.(el mejor)
2. Decidir qué tanto de tu riqueza invertirás en el portafolio y qué tanto invertirás en activos libres de riesgo.

Al paso 2 lo llamamos **decisión de asignación de activos**.

Preguntas importantes:
1. ¿Qué es el portafolio óptimo de activos riesgosos?
 - ¿Cuál es el mejor portafolio de activos riesgosos?
 - Es un portafolio eficiente en media-varianza.
2. ¿Qué es la distribución óptima de activos?
 - ¿Cómo deberíamos distribuir nuestra riqueza entre el portafolo riesgoso óptimo y el activo libre de riesgo?
 - Concepto de **línea de asignación de capital**.
 - Concepto de **radio de Sharpe**.

Dos suposiciones importantes:
- Funciones de utilidad media-varianza.
- Inversionista averso al riesgo.

La idea sorprendente que saldrá de este análisis, es que cualquiera que sea la actitud del inversionista de cara al riesgo, el mejor portafolio de activos riesgosos es idéntico para todos los inversionistas.

Lo que nos importará a cada uno de nosotros en particular, es simplemente la desición óptima de asignación de activos.
___

### 1.2. Línea de asignación de capital

Sean:
- $r_s$ el rendimiento del activo riesgoso,
- $r_f$ el rendimiento libre de riesgo, y
- $w$ la fracción invertida en el activo riesgoso.

 Realizar deducción de la línea de asignación de capital en el tablero.

**Tres doritos después...**

#### Línea de asignación de capital (LAC):
$E[r_p]$ se relaciona con $\sigma_p$ de manera afín. Es decir, mediante la ecuación de una recta:

$$E[r_p]=r_f+\frac{E[r_s-r_f]}{\sigma_s}\sigma_p.$$

- La pendiente de la LAC es el radio de Sharpe $\frac{E[r_s-r_f]}{\sigma_s}=\frac{E[r_s]-r_f}{\sigma_s}$,
- el cual nos dice qué tanto rendimiento obtenemos por unidad de riesgo asumido en la tenencia del activo (portafolio) riesgoso.

Ahora, la pregunta es, ¿dónde sobre esta línea queremos estar?
___

### 1.3. Resolviendo para la asignación óptima de capital

Recapitulando de la clase pasada, tenemos las curvas de indiferencia: **queremos estar en la curva de indiferencia más alta posible, que sea tangente a la LAC**.

 Ver en el tablero.

Analíticamente, el problema es

$$\max_{w} \quad E[U(r_p)]\equiv\max_{w} \quad E[r_p]-\frac{1}{2}\gamma\sigma_p^2,$$

donde los puntos $(\sigma_p,E[r_p])$ se restringen a estar en la LAC, esto es $E[r_p]=r_f+\frac{E[r_s-r_f]}{\sigma_s}\sigma_p$ y $\sigma_p=w\sigma_s$. Entonces el problema anterior se puede escribir de la siguiente manera:

$$\max_{w} \quad r_f+wE[r_s-r_f]-\frac{1}{2}\gamma w^2\sigma_s^2.$$

 Encontrar la $w$ que maximiza la anterior expresión en el tablero.

**Tres doritos después...**

La solución es entonces:

$$w^\ast=\frac{E[r_s-r_f]}{\gamma\sigma_s^2}.$$

De manera intuitiva:
- $w^\ast\propto E[r_s-r_f]$: a más exceso de rendimiento que se obtenga del activo riesgoso, más querremos invertir en él.
- $w^\ast\propto \frac{1}{\gamma}$: mientras más averso al riesgo seas, menos querrás invertir en el activo riesgoso.
- $w^\ast\propto \frac{1}{\sigma_s^2}$: mientras más riesgoso sea el activo, menos querrás invertir en él.
___

## 2. Ejemplo de asignación óptima de capital: acciones y billetes de EU

Pongamos algunos números con algunos datos, para ilustrar la derivación que acabamos de hacer.

En este caso, consideraremos:
- **Portafolio riesgoso**: mercado de acciones de EU (representados en algún índice de mercado como el S&P500).
- **Activo libre de riesgo**: billetes del departamento de tesorería de EU (T-bills).

Tenemos los siguientes datos:

$$E[r_{US}]=11.9\%,\quad \sigma_{US}=19.15\%, \quad r_f=1\%.$$

Recordamos que podemos escribir la expresión de la LAC como:

\begin{align}
E[r_p]&=r_f+\left[\frac{E[r_{US}-r_f]}{\sigma_{US}}\right]\sigma_p\\
      &=0.01+\text{S.R.}\sigma_p,
\end{align}

donde $\text{S.R}=\frac{0.119-0.01}{0.1915}\approx0.569$ es el radio de Sharpe (¿qué es lo que es esto?).

Grafiquemos la LAC con estos datos reales:

```python
# Importamos librerías que vamos a utilizar
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
```

```python
# Datos
Ers= 0.119
ss=0.1915
rf=0.01
# Radio de Sharpe para este activo
RS= (Ers-rf)/ss
# Vector de volatilidades del portafolio (sugerido: 0% a 50%)
sp=np.linspace(0,0.5,100)
# LAC
Erp= rf+RS*sp
```

```python
# Gráfica
plt.figure(figsize=(8,6))
plt.plot(sp,Erp,lw=2,label='LAC')
plt.plot(0,rf,'or',ms=10,label='risk free')
plt.plot(ss,Ers,'ob',ms=10,label='Portafolio riesgoso')
plt.xlabel('Volatilidad $\sigma_p$')
plt.ylabel('Rendimiento Esperado $E[r_p]$')
plt.grid()
plt.legend(loc='best')
plt.show()
```

Bueno, y ¿en qué punto de esta línea querríamos estar?
- Pues ya vimos que depende de tus preferencias.
- En particular, de tu actitud de cara al riesgo, medido por tu coeficiente de aversión al riesgo.

Solución al problema de asignación óptima de capital:

$$\max_{w} \quad E[U(r_p)]$$

$$w^\ast=\frac{E[r_s-r_f]}{\gamma\sigma_s^2}$$

Dado que ya tenemos datos, podemos intentar para varios coeficientes de aversión al riesgo:

```python
# importar pandas
import pandas as pd
```

```python
# Crear un DataFrame con los pesos, rendimiento
# esperado y volatilidad del portafolio óptimo
# entre los activos riesgoso y libre de riesgo
# cuyo índice sean los coeficientes de aversión
# al riesgo del 1 al 10 (enteros)
gamma=np.linspace(1,10,10)
tabla=pd.DataFrame(data={'$\gamma$': gamma,
                        '$w^*$':(Ers-rf)/(gamma*ss**2)})
tabla
```

      $\gamma$
      $w^*$

      0
      1.0
      2.972275

      1
      2.0
      1.486137

      2
      3.0
      0.990758

      3
      4.0
      0.743069

      4
      5.0
      0.594455

      5
      6.0
      0.495379

      6
      7.0
      0.424611

      7
      8.0
      0.371534

      8
      9.0
      0.330253

      9
      10.0
      0.297227

**Interpretacion**

entre mas averso al riesgo menos se va a invertir en portafolios riesgosos.

¿Cómo se interpreta $w^\ast>1$?
- Cuando $0<w^\ast<1$, entonces $0<1-w^\ast<1$. Lo cual implica posiciones largas en el mercado de activos y en el activo libre de riesgo.
- Por el contrario, cuando $w^\ast>1$, tenemos $1-w^\ast<0$. Lo anterior implica una posición corta en el activo libre de riesgo (suponiendo que se puede) y una posición larga (de más del 100%) en el mercado de activos: apalancamiento.

# Anuncios parroquiales.

## 1. Quiz la siguiente clase.

## 2. Tarea 5 segunda entrega para el viernes.

Created with Jupyter by Esteban Jiménez Rodríguez.


$$ \LaTeX \text{ command declarations here.}
\newcommand{\N}{\mathcal{N}}
\newcommand{\R}{\mathbb{R}}
\renewcommand{\vec}[1]{\mathbf{#1}}
\newcommand{\norm}[1]{\|#1\|_2}
\newcommand{\d}{\mathop{}\!\mathrm{d}}
\newcommand{\qed}{\qquad \mathbf{Q.E.D.}}
\newcommand{\vx}{\mathbf{x}}
\newcommand{\vy}{\mathbf{y}}
\newcommand{\vt}{\mathbf{t}}
\newcommand{\vb}{\mathbf{b}}
\newcommand{\vw}{\mathbf{w}}
\newcommand{\vm}{\mathbf{m}}
\newcommand{\I}{\mathbb{I}}
\newcommand{\th}{\text{th}}
$$

```python
%matplotlib inline
from Lec07 import *
```

# EECS 445:  Introduction to Machine Learning
## Lecture 07:  Naive Bayes

* Instructor:  **Jacob Abernethy** and **Jia Deng**
* Date:  September 28, 2016

*Lecture Exposition Credit:*  Benjamin Bray, Valli Chockalingam

## Outline
- Probabilistic Models
    - Generative Models
    - Discriminative Models
- Naive Bayes Classifiers
    - Independence Assumption
    - MLE and MAP Parameter Estimates

## Reading List

- Suggested:
    - **[PRML]**, §4.2: Probabilistic Generative Models
    - **[PRML]**, §4.3: Probabilistic Discriminative Models
    - **[MLAPP]**, §3.5: Naive Bayes Classifiers

> In this lecture, we will first talk about the some concepts of probabilistic models for classifiers, especially generative model and discriminant model. And we will introduce Naive Bayes classifier, which assumes independent features give label and is a classical classifier commonly used in spam email classification.

## Probablistic Models

### Probablistic Models: Generative Models

- **Generative model** learns *class-conditional * $P(X | Y)$ and label densities $P(Y)$ from training data

- Perform prediction using the **posterior** via Bayes' Rule. For some new data $\vx^{new}$
    $$
    \begin{align}
    y
    & = \underset{k \in \{1, \dots, K\}}{\arg \max} P(Y=k | X = \vx^{new} ) \\
    & = \underset{k \in \{1, \dots, K\}}{\arg \max} \frac{P(X = \vx^{new} | Y=k)P(Y=k)}{P(X = \vx^{new})} \\
    & = \boxed{\underset{k \in \{1, \dots, K\}}{\arg \max} P(X = \vx^{new} | Y=k)P(Y=k)}
    \end{align}
    $$
    of which the last equality holds because the denominator $P(X = \vx^{new})$ is independent of $k$.

- Basic idea of prediction is picking the label with largest posterior probability given its features $\vx^{new}$.

- Why is this model called **generative**?
    - We learned *class-conditional probabiliy* $P(X | Y)$ from training data.
    - $P(X | Y)$ is distribution of data $X$ given label $Y$
    - So given some label $Y$, could **generate**/sample new data $X$ from $P(X | Y)$.

- The *prior* $P(Y)$ encodes beliefs about popularity of each label

- By comparing the synthetic data and real data, we get a sense of how good our generative model is.

### Probablistic Models: Discriminative Models

- Conversely, a **discriminative model** learns posterior $P(Y | X)$ directly from training data.

- Goal: select a hypothesis to *discriminates* between class labels.

- The prediction for some new data $\vx^{new}$ is
    $$ y = \underset{k \in \{1, \dots, K\}}{\arg \max} P(Y=k | X = \vx^{new} ) \\$$

- Does not (necessarily) provide the ability to **generate** new random examples because unlike generative models, we have no idea what $P(X|Y)$ is.

- Allows us to focus purely on the classification task

- We will discuss the pros and cons of each model later.

### Probablistic Models: Discriminative Models—Property

- The discriminative approach will typically
    - make fewer generative assumptions about the data
        - However, reconstruction features from labels may require prior knowledge

## Naive Bayes Classifiers

> Follows the approach taken by **[MLAPP]**

### Naive Bayes:  Problem

- We will use **Naive Bayes** to solve the following classification problem:
    - **Categorical** feature vector $\vx = (x_1, x_2, \dots, x_D)$ with length $D$
        - Each feature $x_d \in \{1, \dots ,M \}$, $\forall d = 1, \dots, D$
    - Predict discrete class label $y \in \{1, 2, \dots, C \}$

- For example, in **Spam Mail Classification**,
    - Predict whether an email is `SPAM` ($y=1$) or `HAM` ($y=0$)
    - Use words / metadata in the email as features
    - For simplicity, we can use **bag-of-words** features,
        - Assume fixed vocabulary $V$ of size $|V| = D$
        - Feature $x_d$, for $d \in \{1, 2, \dots, D \}$, indicates the existence of $d\text{th}$ word in the email
        - Eg. $x_d = 1$ if $d\text{th}$ word is in the email; $x_d = 0$ otherwise
        - In this case $M=2$

### Naive Bayes:  Independence Assumption and Full model

- The essence of Naive Bayes is the **conditionally independence assumption**
    $$
    P(\vx | y = c) = \prod_{d=1}^D P(x_d | y=c)
    $$
    i.e., given the label, all features are independent.

- The **full generative** model of Naive Bayes is:
    $$
    \begin{align}
    y       &\sim \mathrm{Categorical}(\pi) \\
    x_d | y=c &\sim \mathrm{Categorical}(\theta_{cd}) \quad \forall\, d = 1,\dots,D
    \end{align}
    $$
    with parameters:
    - **Class priors** $\pi = (\pi_1, \dots, \pi_C) \in \Delta^C$,
        - i.e. $P(y = c) = \pi_c$, $\forall c = 1,\dots,C $
        - $\Delta^C$ is C-**simplex**. $\pi \in \Delta^C$ is saying that $\sum_{c=1}^C \pi_c = 1$ and $\pi_c \geq 0, \forall c=1,\dots,C$
    - **Class-conditional probabilities** $\theta_{cd} = (\theta_{cd1},  \dots, \theta_{cdM}) \in \Delta^M$
        - i.e. $P(x_d = m| y = c) = \theta_{cdm}$ for every $d = 1,\dots,D, m = 1, \dots, M, c = 1, \dots, C$

- Parameter $\pi$ and $\theta$ are learned from training data.

>**Remark**
> - **NOTE** in definition and derivation of this lecture, we assume a more general case $x_d \in \{1, \dots ,M \}$ of which $M>2$. But in spam email classification and the derivation in textbook, binary feature, i.e. $M=2$, is used. So don't get confused!

> - When $M=2$, $x_d | y=c$ is also Bernoulli distribution.

### Naive Bayes: Prediction

- Given the independence assumption and full model, for some new data $\vx^{\text{new}} = (x_1^{\text{new}}, \dots, x_D^{\text{new}})$ we will classify based on
    $$
    \begin{align}
    y
    &=\underset{c \in \{1,\dots,C\}}{\arg \max} P(y=c|\vx = \vx^{\text{new}}) \\
    &=\underset{c \in \{1,\dots,C\}}{\arg \max} P(\vx = \vx^{\text{new}} | y=c) P(y=c) \\
    &=\underset{c \in \{1,\dots,C\}}{\arg \max} P(y=c) \prod \nolimits_{d=1}^{D} P(x_d = x_d^{\text{new}} | y=c) \\
    &=\boxed{\underset{c \in \{1,\dots,C\}}{\arg \max} \pi_c \prod \nolimits_{d=1}^{D} \theta_{cdx_d^{\text{new}}}} \\
    \end{align}
    $$

- If we assume $x_d^{\text{new}} \in \{1,\dots,M \}, \forall d = 1,\dots,D$, we could also express the above expression equivalently using **indicator function**
    $$
    y = \underset{c \in \{1,\dots,C\}}{\arg \max} \pi_c \prod \nolimits_{d=1}^{D} \prod \nolimits_{m=1}^{M} \theta_{cdm}^{\mathbb{I}(m=x_d^{\text{new}})}
    $$

- So as long as we learned parameter $\pi$ and $\theta$, we could classify.

> **Remark**

> - Indicator function
    $$
    \mathbb{I}(m=x_d^{\text{new}}) =
    \begin{cases}
    1 & \text{ if } m=x_d^{\text{new}}\\
    0 & \text{ otherwise}
    \end{cases}
    $$

> - In inner product $\prod \nolimits_{m=1}^{M} \theta_{cdm}^{\mathbb{I}(m=x_d^{\text{new}})}$, only $\theta_{cdx_d^{\text{new}}}$ is multiplied and all the other multipliers are 1 due to the power of indicator function.

> - One thing to note is that the above classification criterion is the product of a series numbers smaller than 1 which will generate a rather small number. A better way is to take **logarithm** to transform product into summation and then compare.

### Naive Bayes:  Parameter Estimation

- **Goal:** Given training data $\mathcal{D} = \{ (\vec{x}_1, y_1), \dots, (\vec{x}_N, y_N) \}$, estimate **class-conditional probabilities** $\theta$ and **class priors** $\pi$.

- We will discuss the **MLE** and **MAP** parameter estimates.

### Naive Bayes:  Maximum Likelihood

- The **likelihood** for a single data case $(\vec{x}_n, y_n=c)$ is
    $$
    \begin{align}
    & P((\vec{x}_n, y_n) | \pi, \theta) \\
    &= P(y_n) \prod \nolimits_{d=1}^D P(x_{nd}|y_n) \\
    &= \prod \nolimits_{c=1}^C P(y_n=c)^{\I(y_n=c)} \cdot \prod \nolimits_{c=1}^C \prod \nolimits_{d=1}^D \prod \nolimits_{m=1}^M P(x_{nd}=m|y_n=c)^{\I(x_{nd}=m) \I(y_n=c)}\\
    &= \prod \nolimits_{c=1}^C \pi_c^{\I(y_n=c)} \cdot \prod \nolimits_{c=1}^C \prod \nolimits_{d=1}^D \prod \nolimits_{m=1}^M \theta_{cdm}^{\I(x_{nd}=m) \I(y_n=c)}\\
    \end{align}
    $$

- Therefore, the **log-likelihood** is
    $$
    \begin{split}
    & \log P((\vec{x}_n, y_n) | \pi, \theta) \\
    & = \sum \nolimits_{c=1}^C \I(y_n=c) \log \pi_c + \sum \nolimits_{c=1}^C \sum \nolimits_{d=1}^D \sum \nolimits_{m=1}^M \I(x_{nd}=m) \I(y_n=c) \log \theta_{cdm}
    \end{split}
    $$

- The **log-likelihood** for all training data $\mathcal{D} = \{ (\vec{x}_n, y_n) \}_{n=1}^N $ is
    $$
    \begin{align}
    & \log P(\mathcal{D}| \pi, \theta)\\
    &= \log \prod \nolimits_{n=1}^N P((\vec{x}_n, y_n) | \pi, \theta) = \sum \nolimits_{n=1}^N \log P((\vec{x}_n, y_n) | \pi, \theta) \\
    &= \boxed{\sum \nolimits_{n=1}^N \sum \nolimits_{c=1}^C \I(y_n=c) \log \pi_c + \sum \nolimits_{n=1}^N \sum \nolimits_{c=1}^C \sum \nolimits_{d=1}^D \sum \nolimits_{m=1}^M \I(x_{nd}=m) \I(y_n=c) \log \theta_{cdm}}
    \end{align}
    $$

### Naive Bayes:  Maximum Likelihood

- With the constraints $\sum_{c=1}^C \pi_c=1$ and $\sum_{m=1}^M \theta_{cdm}=1$, we could maximize log-likelihood function $\log P(\mathcal{D}| \pi, \theta)$ using *Lagrange multiplier*. (Derivation is in the notes!)

- By maximizing log-likelihood function, we could have maximum likelihood estimators:
    $$
    \hat{\pi}_c = \frac{N_c}{N} \quad \hat{\theta}_{cdm} = \frac{N_{cdm}}{N_c}
    $$
    and
    $$
    \hat{\pi} = (\hat{\pi}_1, \dots,\hat{\pi}_c, \dots,\hat{\pi}_C); \hat{\theta}_{cd} = (\hat{\theta}_{cd1}, \dots,\hat{\theta}_{cdm}, \dots,\hat{\theta}_{cdM})
    $$
    - $N = $ Number of examples in $\mathcal{D}$
    - $N_c = $ Number of examples in class $c$ in $\mathcal{D}$
    - $N_{cdm} = $ Number of examples in class $c$ with $x_d = m$ in $\mathcal{D}$

- Intuitive Interpretation
    - The class prior $\pi$ is obtained from the density of each class $\{1, \dots, C\}$ in $\mathcal{D}$
    - The class-conditional probability $\theta_{cd}$ is obtained from the density of $x_d \in \{1,\dots,M \}$ among all examples in class $c$

> **Remark**

> - Derivation of **maximum likelihood estimator** $\hat{\pi}_c$

>     - We have the following problem
        $$
        \begin{matrix}
        \left\{
        \begin{split}
        \max \quad &\log P(\mathcal{D}| \pi, \theta) \\
        \text{s.t.}  \quad &\sum \nolimits_{c=1}^C \pi_c=1
        \end{split} \right.
        &
        \overset{\text{equivalent to}}{\Longrightarrow}
        &
        \left \{
        \begin{split}
        \max \quad & \sum \nolimits_{n=1}^N \sum \nolimits_{c=1}^C \I(y_n=c) \log \pi_c \\
        \text{s.t.} \quad &\sum \nolimits_{c=1}^C \pi_c = 1
        \end{split}
        \right.
        \end{matrix}
        $$
        We drop the second term in $\log P(\mathcal{D}| \pi, \theta)$ because it doesn't depend on $\pi_c$
>     - The lagragian is
        $$
        L(\pi, \lambda) = \sum \nolimits_{n=1}^N \sum \nolimits_{c=1}^C \I(y_n=c) \log \pi_c - \lambda \sum \nolimits_{c=1}^C \pi_c -\lambda
        $$
>     - Setting partial derivative with respect to $\pi_c$ to 0, we have
        $$
        \frac{\partial L(\pi, \lambda)}{\partial \pi_c} = 0
        \quad \Rightarrow  \quad
        \sum \nolimits_{n=1}^N \I(y_n=c) \frac{1}{\pi_c} - \lambda = 0
        \quad \Rightarrow  \quad
        \pi_c = \frac{1}{\lambda} \sum \nolimits_{n=1}^N \I(y_n=c)
        $$
>     - Plug $\pi_c$ back into the constraint $\sum \nolimits_{c=1}^C \pi_c=1$, we have
        $$
        \frac{1}{\lambda} \sum \nolimits_{c=1}^C \sum \nolimits_{n=1}^N \I(y_n=c) = 1
        \quad \Rightarrow  \quad
        \lambda = \sum \nolimits_{c=1}^C \sum \nolimits_{n=1}^N \I(y_n=c)
        $$
>     - Plug $\lambda$ into $\pi_c = \frac{1}{\lambda} \sum \nolimits_{n=1}^N \I(y_n=c)$, we have
        $$
        \hat{\pi}_c = \frac{\sum \nolimits_{n=1}^N \I(y_n=c)}{ \sum \nolimits_{n=1}^N \sum \nolimits_{c=1}^C \I(y_n=c)} = \frac{N_c}{N}
        $$

> - Derivation of maximum likelihood estimator $\hat{\theta}_{cdm}$

>     - With the constraint $\sum_{m=1}^M \theta_{cdm}=1$, using similar approach, we could have
        $$
        \hat{\theta}_{cdm}
        = \frac{\sum \nolimits_{n=1}^N \I(x_{nd}=m) \I(y_n=c)}{\sum_{n=1}^N \sum_{m=1}^M \I(x_{nd}=m) \I(y_n=c)}
        = \frac{N_{cdm}}{N_c}
        $$

>    - Details are left as an exercise XD

### Naive Bayes:  Sparse Features

- **Problem:** When working with text, features are **sparse**:
    - In training, we only see a *small, small* fraction of words in the vocabulary
    - Moreover, we won't see all words exhibited across all classes

- This causes overfitting!
    - What if a word (e.g. "`subject:`") occurs in every training example of both classes?
    - Then if we encounter a new email without this word, our algorithm will crash.
    - What happens if that word never appears in testing?  (*Black Swan Paradox*)

### Naive Bayes:  Priors

- **Solution:** Place Dirichlet priors on $\pi$ and $\theta_{cd}$ to *smooth out* unknowns:
    $$
    \begin{align}
    \pi       &\sim \mathrm{Dirichlet}(\alpha_1, \dots, \alpha_C) \\
    \theta_{cd}  &\sim \mathrm{Dirichlet}(\beta_{cd1}, \dots, \beta_{cdM})  & &\quad  \forall\, c=1,\dots,C; d=1, \dots D \\
    y         &\sim \mathrm{Categorical}(\pi)                     \\
    x_d | y=c &\sim \mathrm{Categorical}(\theta_{cd}) & & \quad \forall\, d = 1,\dots,D
    \end{align}
    $$

- **Dirichlet distribution** $\pi \sim \mathrm{Dirichlet}(\alpha_1, \dots, \alpha_C)$ defines the distribution of C-simplex $\pi = (\pi_1, \dots, \pi_C)$ such that
    - $\pi_1, \dots, \pi_C \geq 0$
    - $\pi_1+ \dots + \pi_C = 1$
    - PDF $f(\pi_1, \dots, \pi_C ) = \frac{1}{B(\alpha)} \prod_{c=1}^C \pi_c^{\alpha_c-1}$, where the quantity $B(\alpha)$ is to normalize the distribution

- When $M=2$, $\theta_{cd}$ reduces to **Beta** distribution

### Naive Bayes:  MAP Estimate

- The **MAP parameter** estimates with priors
    $$
    \pi \sim \mathrm{Dirichlet}(\alpha_1, \dots, \alpha_C) \qquad \theta_{cd} \sim \mathrm{Dirichlet}(\beta_{cd1}, \dots, \beta_{cdM})
    $$
    are
    $$
    \hat{\pi}_c = \frac{N_c+\alpha_c-1}{N + \sum_{c'=1}^C (\alpha_{c'}-1)}
    \quad
    \hat{\theta}_{cdm} = \frac{N_{cdm}+\beta_{cdm}-1}{N_c + \sum_{m'=1}^M (\beta_{cdm'}-1)}
    $$

- Proof is in the notes!

- The Dirichlet $\alpha$ and $\beta_{cd}$ parameters turn out to be **pseudocounts**!
    - We assume we've seen $\alpha_c$ examples of class $c$ beforehand
    - and $\beta_{cdm}$ examples with $x_d = m$ in class $c$.

- The choice $\alpha_c = \beta_{cdm} = 1$ is referred to as **Laplace Smoothing**

### Naive Bayes:  Mean Estimate

- Note that the posterior of parameters still have **Dirichlet** distributions!
    $$
    \pi|\mathcal{D} \sim \mathrm{Dirichlet}(N_1+\alpha_1, \dots, N_C+\alpha_C) \qquad \theta_{cd}|\mathcal{D} \sim \mathrm{Dirichlet}(N_{cd1}+\beta_{cd1}, \dots, N_{cdM}+\beta_{cdM})
    $$
    Proof for $\pi|\mathcal{D}$ is in the notes! Proof for $\theta_{cd}|\mathcal{D}$ is left as an exercise.

- If we pick the **mean** of posteriors as parameter estimate, we could get a slightly different result:
    $$
    \bar{\pi}_c = \frac{N_c+\alpha_c}{N + \sum_{c'=1}^C \alpha_{c'}}
    \quad
    \bar{\theta}_{cdm} = \frac{N_{cdm}+\beta_{cdm}}{N_c + \sum_{m'=1}^M \beta_{cdm'}}
    $$
    $-1$ terms no longer exist!

- You might see this version of estimate in **[MLAPP]** and some online materials

- Advantage of using mean estimate
    - Since posteriors still have Dirichlet distribution, we could use posterior as the prior for the next learning phase!
    - This can be helpful in sequential learning!

> **Remark**

> - Posterior also has Dirichlet distribution!

>     - **Prior:**
        $$
        \pi \sim \mathrm{Dirichlet}(\alpha_1, \dots, \alpha_C)
        \qquad
        f(\pi) = \frac{1}{B(\alpha)} \prod \nolimits_{c=1}^C \pi_c^{\alpha_c-1}
        $$

>     - **Likelihood:**
        $$
        P(\mathcal{D} | \pi) = \prod \nolimits_{n=1}^N \prod \nolimits_{c=1}^C \pi_c^{\I(y_n=c)} = \prod \nolimits_{c=1}^C \pi_c^{\sum \nolimits_{n=1}^N \I(y_n=c)} = \prod \nolimits_{c=1}^C \pi_c^{N_c}
        $$

>     - **Posterior**
        $$
        \begin{split}
        f(\pi | \mathcal{D})
        &= \frac{P(\mathcal{D} | \pi) f(\pi)}{P(\mathcal{D})} = \frac{1}{P(\mathcal{D})} \prod \nolimits_{c=1}^C \pi_c^{N_c} \cdot \frac{1}{B(\alpha)} \prod \nolimits_{c=1}^C \pi_c^{\alpha_c-1} \\
        &= \frac{1}{P(\mathcal{D}) B(\alpha)} \prod \nolimits_{c=1}^C \pi_c^{N_c+\alpha_c-1} \\
        &= \frac{1}{B'(\alpha)} \prod \nolimits_{c=1}^C \pi_c^{N_c+\alpha_c-1}
        \end{split}
        $$
        of which $B'(\alpha) = P(\mathcal{D}) B(\alpha)$.

>     - From the expression of $f(\pi | \mathcal{D}) $, we could see posterior also has **Dirichlet** distribution
        $$
        \pi | \mathcal{D} \sim \mathrm{Dirichlet}(N_1+\alpha_1, \dots, N_C+\alpha_C)
        $$

> - Conjugate Prior

>     - If posterior and prior are in the same distribution family with respect to some likelihood, we could call this distribution **conjugate prior** for that likelihood.

>     - This is useful because we could take the posterior as the prior for next learning phase, which enables us to do sequential Bayesian learning.

>     - In our case, we have shown Dirichlet distribution is the conjugate prior of the multinomial distribution!

> - Derivation of **MAP estimate** ${\hat{\pi}_c}_{MAP}$

>     - MAP estimate is obtained by maximizing the posterior $f(\pi | \mathcal{D})$
        $$
        \begin{matrix}
        \left\{
        \begin{split}
        \max \quad & \prod \nolimits_{c=1}^C \pi_c^{N_c+\alpha_c-1}\\
        \text{s.t.}  \quad &\sum \nolimits_{c=1}^C \pi_c=1
        \end{split} \right.
        &
        \overset{\text{equivalent to}}{\Longrightarrow}
        &
        \left \{
        \begin{split}
        \max \quad & \sum \nolimits_{c=1}^C \left( N_c + \alpha_c -1 \right) \log \pi_c \\
        \text{s.t.} \quad &\sum \nolimits_{c=1}^C \pi_c = 1
        \end{split}
        \right.
        \end{matrix}
        $$

>     - Recall when deriving maximum likelihood estimator, we solved the following problem
        $$
        \left \{
        \begin{split}
        \max \quad & \sum \nolimits_{c=1}^C \sum \nolimits_{n=1}^N \I(y_n=c) \log \pi_c \\
        \text{s.t.} \quad &\sum \nolimits_{c=1}^C \pi_c = 1
        \end{split}
        \right.
        \longrightarrow
        {\hat{\pi}_c}_{MLE} = \frac{\sum \nolimits_{n=1}^N \I(y_n=c)}{ \sum \nolimits_{n=1}^N \sum \nolimits_{c=1}^C \I(y_n=c)} = \frac{N_c}{N}
        $$

>     - So the solution of our current problem can be easily read off
        $$
        \begin{split}
        {\hat{\pi}_c}_{MAP}
        &= \frac{N_c + \alpha_c -1 }{\sum \nolimits_{c'=1}^C \left(N_c + \alpha_c' -1 \right)} = \frac{N_c + \alpha_c -1}{N + \sum \nolimits_{c'=1}^C \left(\alpha_c' -1\right)}
        \end{split}
        $$

> - Derivation of **MAP estimator** $\hat{\theta}_{cdm}$ is left as an exercise! Approach is exactly the same as deriving $\hat{\pi}_c$.

### Naive Bayes:  Is Independence Justified?

- Naive Bayes assumes features contribute *independently* to the class label.

    - This is the *simplest possible* generative model... and an **extreme** assumption...

- This model is *naive* because we would never expect features to be independent!

    -  We are completely ignoring correlations between variables!

- It seems not to matter that independence is often false...
    - Naive Bayes performs surprisingly well on real-world data
    - Naive Bayes is often used as a baseline

## Summary of Classifiers
- **Logistic Regression**
    * Provides model for $P(y | \vx)$ using sigmoid
    * No explicit model for $P(\vx | y)$
- **Naive Bayes**
    * Provides a full model for $P(\vx | y )$ and $P(y)$
    * Assumes independence between features *conditioned on* target $y$
    * Typically requires discrete data (can generalize to continuous spaces)
    * ML estimates are pretty straightforward


# Lab 9: Hardware-efficient trial states for variational quantum eigensolvers

In this lab, we want to find the ground state energy of the following two-qubit Hamiltonian representing molecular $H_2$ at its equilibrium distance. Note that we have already mapped the fermionic operators onto qubits here.

```
h2_hamiltonian = (-1.0523732)  II +
                 (0.39793742)  IZ +
                 (-0.3979374)  ZI +
                 (-0.0112801)  ZZ +
                 (0.18093119)  XX
```

Note that these co-efficients are functions of the interatomic distance between the atoms. A term like `IZ` is shorthand notation for a tensor product of two-qubit operators -- the identity operator ($I$) on one qubit and pauli-Z operator ($Z$) on the other qubit.

### Installing necessary packages

Before we begin, you will need to install some prerequisites into your environment. Run the cell below to complete these installations. At the end, the cell outputs will be cleared.

```python
!pip install -U -r grading_tools/requirements.txt

from IPython.display import clear_output
clear_output()
```

    Requirement already satisfied, skipping upgrade: pybind11>=2.4 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from qiskit-aer==0.5.1->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (2.5.0)
    Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from requests>=2.19->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (2020.6.20)
    Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from requests>=2.19->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (3.0.4)
    Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from requests>=2.19->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (2.10)
    Requirement already satisfied, skipping upgrade: six>=1.5 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from python-dateutil>=2.8.0->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (1.15.0)
    Requirement already satisfied, skipping upgrade: cryptography>=1.3 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (2.9.2)
    Requirement already satisfied, skipping upgrade: ntlm-auth>=1.0.2 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (1.5.0)
    Requirement already satisfied, skipping upgrade: more-itertools in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from quandl->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (8.4.0)
    Requirement already satisfied, skipping upgrade: inflection>=0.3.1 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from quandl->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (0.5.0)
    Requirement already satisfied, skipping upgrade: pandas>=0.14 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from quandl->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (1.0.5)
    Requirement already satisfied, skipping upgrade: mpmath>=0.19 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from sympy>=1.3->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (1.1.0)
    Requirement already satisfied, skipping upgrade: decorator>=4.3.0 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from networkx>=2.2->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (4.4.2)
    Requirement already satisfied, skipping upgrade: joblib>=0.11 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (0.16.0)
    Requirement already satisfied, skipping upgrade: threadpoolctl>=2.0.0 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (2.1.0)
    Requirement already satisfied, skipping upgrade: pyrsistent>=0.14.0 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from jsonschema>=2.6->qiskit-terra==0.14.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (0.16.0)
    Requirement already satisfied, skipping upgrade: attrs>=17.4.0 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from jsonschema>=2.6->qiskit-terra==0.14.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (19.3.0)
    Requirement already satisfied, skipping upgrade: cffi!=1.11.3,>=1.8 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (1.14.0)
    Requirement already satisfied, skipping upgrade: pytz>=2017.2 in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from pandas>=0.14->quandl->qiskit-aqua==0.7.0->qiskit==0.19->-r grading_tools/requirements.txt (line 1)) (2020.1)
    Requirement already satisfied, skipping upgrade: pycparser in c:\users\hernan\anaconda3\envs\qiskit\lib\site-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7->-r grading_tools/requirements.txt (line 2)) (2.20)

# Hardware efficient trial states

We need to prepare a guess for the ground state of this Hamiltonian. For this exercise, we will construct a hardware efficient trial state of depth `d=2`, with CNOT (`CX`) gates for the entanglers. We will interleave each entangling layer with a set of $R_yR_z$ gates on each qubit (total 4 layers). For the first guess, we will use rotation angles of $\pi/2$ on all the $R_y$ and $R_z$ gates.

```python
from numpy import pi
from qiskit import QuantumCircuit, Aer, execute
from qiskit.visualization import plot_histogram
```

```python
def prepare_hets_circuit(depth, angle1, angle2):

    hets_circ = QuantumCircuit(depth)
    hets_circ.ry(angle1, 0)
    hets_circ.rz(angle1, 0)
    hets_circ.ry(angle1, 1)
    hets_circ.rz(angle1, 1)

    for ii in range(depth):
        hets_circ.cx(0,1)
        hets_circ.ry(angle2,0)
        hets_circ.rz(angle2,0)
        hets_circ.ry(angle2,1)
        hets_circ.rz(angle2,1)

    return hets_circ
```

```python
hets_circuit = prepare_hets_circuit(2, pi/2, pi/2)
hets_circuit.draw()
```

     ┌──────────┐┌──────────┐     ┌──────────┐┌──────────┐     ┌──────────┐»
q_0: ┤ RY(pi/2) ├┤ RZ(pi/2) ├──■──┤ RY(pi/2) ├┤ RZ(pi/2) ├──■──┤ RY(pi/2) ├»
     ├──────────┤├──────────┤┌─┴─┐├──────────┤├──────────┤┌─┴─┐├──────────┤»
q_1: ┤ RY(pi/2) ├┤ RZ(pi/2) ├┤ X ├┤ RY(pi/2) ├┤ RZ(pi/2) ├┤ X ├┤ RY(pi/2) ├»
     └──────────┘└──────────┘└───┘└──────────┘└──────────┘└───┘└──────────┘»
«     ┌──────────┐
«q_0: ┤ RZ(pi/2) ├
«     ├──────────┤
«q_1: ┤ RZ(pi/2) ├
«     └──────────┘

# Measuring expectation values

Next, we measure expectation values. **We will begin by measuring the ZZ expectation value, or $\langle ZZ \rangle$**. We will first create a copy of the `hets_circ` quantum circuit that we created above, and add measurements to it.

```python
def measure_zz_circuit(given_circuit):
    zz_meas = given_circuit.copy()
    zz_meas.measure_all()
    return zz_meas

zz_meas = measure_zz_circuit(hets_circuit)
zz_meas.draw()
```

        ┌──────────┐┌──────────┐     ┌──────────┐┌──────────┐     ┌──────────┐»
   q_0: ┤ RY(pi/2) ├┤ RZ(pi/2) ├──■──┤ RY(pi/2) ├┤ RZ(pi/2) ├──■──┤ RY(pi/2) ├»
        ├──────────┤├──────────┤┌─┴─┐├──────────┤├──────────┤┌─┴─┐├──────────┤»
   q_1: ┤ RY(pi/2) ├┤ RZ(pi/2) ├┤ X ├┤ RY(pi/2) ├┤ RZ(pi/2) ├┤ X ├┤ RY(pi/2) ├»
        └──────────┘└──────────┘└───┘└──────────┘└──────────┘└───┘└──────────┘»
meas_0: ══════════════════════════════════════════════════════════════════════»
                                                                              »
meas_1: ══════════════════════════════════════════════════════════════════════»
                                                                              »
«        ┌──────────┐ ░ ┌─┐
«   q_0: ┤ RZ(pi/2) ├─░─┤M├───
«        ├──────────┤ ░ └╥┘┌─┐
«   q_1: ┤ RZ(pi/2) ├─░──╫─┤M├
«        └──────────┘ ░  ║ └╥┘
«meas_0: ════════════════╩══╬═
«                           ║
«meas_1: ═══════════════════╩═
«

Next, let's execute this quantum circuit and see the measurement outcomes.

```python
simulator = Aer.get_backend('qasm_simulator')
```

```python
result = execute(zz_meas, backend = simulator, shots=10000).result()
counts = result.get_counts(zz_meas)

plot_histogram(counts)
```

We can analyze the counts and calculate the $\langle ZZ \rangle$ as follows:

```python
def measure_zz(given_circuit, num_shots = 10000):

    zz_meas = measure_zz_circuit(given_circuit)

    result = execute(zz_meas, backend = simulator, shots = num_shots).result()
    counts = result.get_counts(zz_meas)

    if '00' not in counts:
        counts['00'] = 0
    if '01' not in counts:
        counts['01'] = 0
    if '10' not in counts:
        counts['10'] = 0
    if '11' not in counts:
        counts['11'] = 0

    total_counts = counts['00'] + counts['11'] + counts['01'] + counts['10']
    zz = counts['00'] + counts['11'] - counts['01'] - counts['10']
    zz = zz / total_counts

    return zz
```

```python
zz = measure_zz(hets_circuit)
print("<ZZ> =", str(zz))
```

    <ZZ> = -0.0096

**What about $\langle ZI \rangle$ and  $\langle IZ \rangle$? Do these need new circuits?**

The answer is no, and they can be computed from the results above.

```python
def measure_zi(given_circuit, num_shots = 10000):

    zz_meas = measure_zz_circuit(given_circuit)

    result = execute(zz_meas, backend = simulator, shots = num_shots).result()
    counts = result.get_counts(zz_meas)

    if '00' not in counts:
        counts['00'] = 0
    if '01' not in counts:
        counts['01'] = 0
    if '10' not in counts:
        counts['10'] = 0
    if '11' not in counts:
        counts['11'] = 0

    total_counts = counts['00'] + counts['11'] + counts['01'] + counts['10']

    zi = counts['00'] - counts['11'] + counts['01'] - counts['10']
    zi = zi / total_counts

    return zi

def measure_iz(given_circuit, num_shots = 10000):

    zz_meas = measure_zz_circuit(given_circuit)

    result = execute(zz_meas, backend = simulator, shots = num_shots).result()
    counts = result.get_counts(zz_meas)

    if '00' not in counts:
        counts['00'] = 0
    if '01' not in counts:
        counts['01'] = 0
    if '10' not in counts:
        counts['10'] = 0
    if '11' not in counts:
        counts['11'] = 0

    total_counts = counts['00'] + counts['11'] + counts['01'] + counts['10']

    iz = counts['00'] - counts['11'] - counts['01'] + counts['10']
    iz = iz / total_counts

    return iz
```

```python
zi = measure_zi(hets_circuit)
print("<ZI> =", str(zi))

iz = measure_iz(hets_circuit)
print("<IZ> =", str(iz))
```

    <ZI> = -0.0128
    <IZ> = 0.0126

**Next, we measure $\langle XX \rangle$**

```python
def measure_xx_circuit(given_circuit):
    xx_meas = given_circuit.copy()

    ### WRITE YOUR CODE BETWEEN THESE LINES - START
    xx_meas.h([0,1])
    xx_meas.measure_all()
    ### WRITE YOUR CODE BETWEEN THESE LINES - END

    return xx_meas
```

```python
xx_meas = measure_xx_circuit(hets_circuit)
xx_meas.draw()
```

        ┌──────────┐┌──────────┐     ┌──────────┐┌──────────┐     ┌──────────┐»
   q_0: ┤ RY(pi/2) ├┤ RZ(pi/2) ├──■──┤ RY(pi/2) ├┤ RZ(pi/2) ├──■──┤ RY(pi/2) ├»
        ├──────────┤├──────────┤┌─┴─┐├──────────┤├──────────┤┌─┴─┐├──────────┤»
   q_1: ┤ RY(pi/2) ├┤ RZ(pi/2) ├┤ X ├┤ RY(pi/2) ├┤ RZ(pi/2) ├┤ X ├┤ RY(pi/2) ├»
        └──────────┘└──────────┘└───┘└──────────┘└──────────┘└───┘└──────────┘»
meas_0: ══════════════════════════════════════════════════════════════════════»
                                                                              »
meas_1: ══════════════════════════════════════════════════════════════════════»
                                                                              »
«        ┌──────────┐┌───┐ ░ ┌─┐
«   q_0: ┤ RZ(pi/2) ├┤ H ├─░─┤M├───
«        ├──────────┤├───┤ ░ └╥┘┌─┐
«   q_1: ┤ RZ(pi/2) ├┤ H ├─░──╫─┤M├
«        └──────────┘└───┘ ░  ║ └╥┘
«meas_0: ═════════════════════╩══╬═
«                                ║
«meas_1: ════════════════════════╩═
«

```python
def measure_xx(given_circuit, num_shots = 10000):

    xx_meas = measure_xx_circuit(given_circuit)

    result = execute(xx_meas, backend = simulator, shots = num_shots).result()
    counts = result.get_counts(xx_meas)

    if '00' not in counts:
        counts['00'] = 0
    if '01' not in counts:
        counts['01'] = 0
    if '10' not in counts:
        counts['10'] = 0
    if '11' not in counts:
        counts['11'] = 0

    total_counts = counts['00'] + counts['11'] + counts['01'] + counts['10']
    xx = counts['00'] + counts['11'] - counts['01'] - counts['10']
    xx = xx / total_counts

    return xx

xx = measure_xx(hets_circuit)
print("<XX> =", str(xx))
```

    <XX> = -1.0

### Now we evaluate the energy of the trial state

```python
def get_energy(given_circuit, num_shots = 10000):

    zz = measure_zz(given_circuit, num_shots = num_shots)
    iz = measure_iz(given_circuit, num_shots = num_shots)
    zi = measure_zi(given_circuit, num_shots = num_shots)
    xx = measure_xx(given_circuit, num_shots = num_shots)

    energy = (-1.0523732)*1 + (0.39793742)*iz + (-0.3979374)*zi + (-0.0112801)*zz + (0.18093119)*xx

    return energy
```

```python
energy = get_energy(hets_circuit)
print("The energy of the trial state is", str(energy))
```

    The energy of the trial state is -1.2380422891079996

# Computing gradients

The objective of the next set of exercises is to compute the next set of angles to use for the trial state preparation. Create two quantum circuits using `prepare_hets_circuit(depth=2, angle2 = pi/2)` for measuring expectation values with values for `angle1 = pi/2 + 0.1*pi/2` and `angle1 = pi/2 - 0.1*pi/2`, and measure the energy of both trial states.

```python
hets_circuit_plus = None
hets_circuit_minus = None

### WRITE YOUR CODE BETWEEN THESE LINES - START
hets_circuit_plus = prepare_hets_circuit(2, pi/2 + 0.1*pi/2, pi/2)
hets_circuit_minus = prepare_hets_circuit(2, pi/2 - 0.1*pi/2, pi/2)
### WRITE YOUR CODE BETWEEN THESE LINES - END

energy_plus = get_energy(hets_circuit_plus, num_shots=100000)
energy_minus = get_energy(hets_circuit_minus, num_shots=100000)

print(energy_plus, energy_minus)
```

    -1.1585050832636001 -1.2789501977138

As you can see, one of these is certainly lower energy than the other, and is also lower energy than the case when `angle1 = pi/2`. This is a suitable next point for our iteration of a variational eigensolver.

# Submitting your solutions for grading

Now, grade your solution by running the cell below after filling in your name and email address. Always provide the same name and email as the one you used during registration to ensure consistency.

```python
name = 'Hernan Amiune'
email = 'amiune@gmail.com'

### Do not change the lines below
from grading_tools import grade

#grade(answer=measure_xx_circuit(hets_circuit), name=name, email=email, labid='lab9', exerciseid='ex1')
grade(answer=hets_circuit_plus, name=name, email=email, labid='lab9', exerciseid='ex2')
grade(answer=hets_circuit_minus, name=name, email=email, labid='lab9', exerciseid='ex3')
```

    Grading...
    lab9/ex2 - 🎉 Correct
    🎊 Hurray! You have a new correct answer! Let's submit it.
    Submitting the answers for lab9...
    📝 Our records, so far, are:
    Correct answers: lab1:ex1, lab2:ex1, lab3:ex1, lab4:ex1, lab5:ex1, lab6:ex1a, lab6:ex1b, lab9:ex1, lab9:ex2
    Grading...
    lab9/ex3 - 🎉 Correct
    🎊 Hurray! You have a new correct answer! Let's submit it.
    Submitting the answers for lab9...
    📝 Our records, so far, are:
    Correct answers: lab1:ex1, lab2:ex1, lab3:ex1, lab4:ex1, lab5:ex1, lab6:ex1a, lab6:ex1b, lab9:ex1, lab9:ex2, lab9:ex3

**Help us improve our educational tools by submitting your code**

If you would like to help us learn how to improve our educational materials and offerings, you can opt in to send us a copy of your Jupyter notebook. By executing the cell below, you consent to sending us the code in your Jupyter notebook. All of the personal information will be anonymized.

```python
from IPython.display import display, Javascript;display(Javascript('IPython.notebook.save_checkpoint();'));
from grading_tools import send_code;send_code('ex1.ipynb')
```

## Bonus 1

While this is not graded, explore whether the decision above would be easy if your execution ran different numbers of shots. In particular, measure `energy_plus` and `energy_minus` again with `100`, `1000` and `10000` shots to explore how easy or difficult this decision gets with each one.

```python
energy_plus_100, energy_plus_1000, energy_plus_10000 = 0, 0, 0
energy_minus_100, energy_minus_1000, energy_minus_10000 = 0, 0, 0

### WRITE YOUR CODE BETWEEN THESE LINES - START

energy_plus_100 = get_energy(hets_circuit_plus, num_shots=100)
energy_minus_100 = get_energy(hets_circuit_minus, num_shots=100)

energy_plus_1000 = get_energy(hets_circuit_plus, num_shots=1000)
energy_minus_1000 = get_energy(hets_circuit_minus, num_shots=1000)

energy_plus_10000 = get_energy(hets_circuit_plus, num_shots=10000)
energy_minus_10000 = get_energy(hets_circuit_minus, num_shots=10000)

### WRITE YOUR CODE BETWEEN THESE LINES - END

print(energy_plus_100, energy_minus_100, "difference = ", energy_minus_100 - energy_plus_100)
print(energy_plus_1000, energy_minus_1000, "difference = ", energy_minus_1000 - energy_plus_1000)
print(energy_plus_10000, energy_minus_10000, "difference = ", energy_minus_10000 - energy_plus_10000)
```

    -1.1737489269999999 -1.2046815113999998 difference =  -0.030932584399999907
    -1.1545757563399999 -1.30252163908 difference =  -0.1479458827400002
    -1.154993323366 -1.2807243096939998 difference =  -0.12573098632799984

## Bonus 2

While this is not graded, diagonalize the Hamiltonian by writing down the matrices for the Pauli operators `I`, `X` and `Z`, and find the exact ground state energy.

```python
### WRITE YOUR CODE BETWEEN THESE LINES - START
import numpy as np

I = [[1,0],
     [0,1]]
X = [[0,1],
     [1,0]]
Z = [[1,0],
     [0,-1]]

I = np.array(I)
X = np.array(X)
Z = np.array(Z)

h2_hamiltonian = -1.0523732 * (I@I) + 0.39793742 * (I@Z) + -0.3979374 * (Z@I) + -0.0112801 * (Z@Z) + 0.18093119 * (X@X)

print(h2_hamiltonian)

#w, v = np.linalg.eigh(h2_hamiltonian)

### WRITE YOUR CODE BETWEEN THESE LINES - END
```

    [[-0.88272209  0.        ]
     [ 0.         -0.88272213]]

```python

```

```python

```