# Expt 1: Density Measurements and Uncertainties
### Objectives
- To learn the basics of good lab practice
- To learn how to estimate uncertainties
- To learn how to propagate uncertainties
- To learn how to compare measurement methods

### Equipment
- One set of five (5) unknown cylinders of the same composition
- One beaker
- One or two 50 mL graduated cylinders
- One vernier or digital caliper
- One micrometer
- One digital scale or beam balance
- One “mass hanging” assembly

### Safety
- Never drop metal cylinders into glass containers!
- Please read the section below on proper use of a graduated cylinder.

# Introduction
One very important and useful property of materials is the **density**, ρ, of that material. Density is defined to be the
amount of mass contained within a certain volume:

\begin{equation}
\rho \equiv \frac{m}{v}
\label{eq_density}
\end{equation}

This property can be used to determine the total mass of an object if we know its volume, or it can be used to
determine if an object will float in a liquid (if the liquid has a higher density than the object). The density of an
object is also a useful property to help identify it.

In this lab, your job will be to determine the density of a set of rods of a given composition. Measuring the mass
will be the easy part. **You will use three methods to determine the volume:** Direct measurement with scales,
measurement of the amount of water it displaces, and measurement of its equivalent weight of water. You will
tabulate your results and compare the first method with the second and third methods.

It will be to your benefit to be slow and careful while performing this lab.

# Theory

In this laboratory, there are a lot of calculations of the uncertainty in a derived quantity, based on the uncertainties in the raw quantities involved.  We can understand how to calculate these uncertainties using just a little bit of calculus!

Consider a function, $z = f(x,y)$.  We imagine that $x$ and $y$ are two independent raw quantities measured in the experiment, and $z$ is some derived quantity.  The chain-rule of differentiation, together with an assumption that the uncertainties are Gaussian in nature, lead to the following:

\begin{equation}
(\delta z)^2 = \left( \frac{\partial{f}}{\partial{x}} \right)^2 (\delta x)^2 +
 \left( \frac{\partial{f}}{\partial{y}} \right)^2 (\delta y)^2
     \label{eq_unc}
\end{equation}

Now, consider the function $z = \frac{x}{y}$.  This particular relationship will be very useful in this experiment!

The calculation of density in Equation $\eqref{eq_density}$ will require a determination of the mass and volume of each rod. The
final density will be an average of the computations for each method of measuring the volume of the cylinder. We
will need the uncertainty in the computed density. Using Equation $\eqref{eq_unc}$, we can show that:

\begin{equation}
\left(\frac{\delta \rho}{\rho}\right)^2 = \left(\frac{\delta m}{m}\right)^2 + \left(\frac{\delta V}{V}\right)^2
\label{eq_unc_den}
\end{equation}

A second extremely important concept in this experiment is the calculation of the weighted average of a series of numbers, each of which has an uncertainty associated with it.  For example, imagine that we make four measurements of the same quantity, $x$, and have an uncertainty, $\delta x$, for each of these four measurements.  The questions are:

(a) what is the $average$ of these four measurements?
(b) what is the $uncertainty$ in this average?

Normally, when we average a series of numbers, we simply add up the values and divide by the total number of values:

\begin{equation}
\bar{x} = \frac{1}{N} \sum_{i=1}^N x_i
\end{equation}

However, when each of the measurements has an uncertainty, it is not so simple.  The idea is that we want to attach more importance to measurements with $small$ uncertainties, and less importance to measurements with $large$ uncertainties.  This is called calculating a $weighted$ average.  For Gaussian uncertainties, it turns out that the way that we do this is to calculate the average as:

\begin{equation}
\bar{x} = \frac{\sum_{i=1}^N \frac{x_i}{\delta x_i^2}} {\sum_{i=1}^N \frac{1}{\delta x_i^2}}
\end{equation}

Note that if $\delta x_i$ is small, this term in the summation will be large, which is exactly what we want!

For the calculation of the uncertainty in $\bar{x}$, there are $TWO$ separate uncertainties that we must consider.  The first is the normal statistical uncertainty,
based on the fact that each of the individual measurements has an uncertainty.  This is
readily calculable using Equation $\eqref{eq_unc}$, which yields:

\begin{equation}
\delta \bar{x}_{statistical} = \frac{1}{N} \sqrt{\sum_{i=1}^N \delta x_i^2}
\end{equation}

The second part of the uncertainty in $\bar{x}$ is based on the notion that the values $x_i$ vary from one another (even though in a perfect world that would all be identical since they represent the same physical quantity).  This variation is known as a $systematic$ uncertainty.  The calculation of this uncerainty is straightfoward but tedious.  The result is:

\begin{equation}
\delta \bar{x}_{systematic} = \sqrt{ \frac{1}{N} \left(
\frac{\sum_{i=1}^N \frac{x_i^2}{\delta x_i^2}}{\sum_{i=1}^N \frac{1}{\delta x_i^2}} - \bar{x}^2
\right)}
\end{equation}

Example:  Consider the following measurements of the quantity, $x$

\begin{equation}
x_1 = 100.0 \pm 0.1 \\
x_2 = 102.0 \pm 0.2 \\
x_3 = 98.0 \pm 0.02 \\ \nonumber
\end{equation}

\begin{equation}
\sum_{i=1}^N \frac{x_i}{\delta x_i^2} = \frac{100}{0.1^2} + \frac{102}{0.2^2} + \frac{98}{0.02^2} = 257550.0 \nonumber
\end{equation}

\begin{equation}
\sum_{i=1}^N \frac{1}{\delta x_i^2} = \frac{1}{0.1^2} + \frac{1}{0.2^2} + \frac{1}{0.02^2} = 2625.0 \nonumber
\end{equation}

\begin{equation}
\bar{x} = \frac{257550.0}{2625.0} = 98.1143 \nonumber
\end{equation}

Notice the $weighted$ average is quite close to 98, because the uncertainty in that measured value is significantly smaller than the other uncertainties.

\begin{equation}
\delta \bar{x}_{statistical} = \frac{1}{N} \sqrt{\sum_{i=1}^N \delta x_i^2} = \frac{1}{3} \sqrt{(0.1)^2 + (0.2)^2 + (0.02)^2} = 0.0748  \nonumber
\end{equation}

\begin{equation}
\sum_{i=1}^N \frac{x_i^2}{\delta x_i^2} = \frac{100^2}{0.1^2} + \frac{102^2}{0.2^2} + \frac{98^2}{0.02^2} = 25270100.0 \nonumber
\end{equation}

\begin{equation}
\delta \bar{x}_{systematic} = \sqrt{ \frac{1}{2} \left( \frac{25270100.0}{2625} - (98.11)^2 \right) } = 0.3819 \nonumber
\end{equation}

Notice that the systematic uncertainty is about 5 times larger than the statistical uncertainty.  Thus, we would conclude that these three measurements do $NOT$ agree with one another, within statistical uncertainty!

Let's write a python function to do all of this!  It will take the $x$ and $\delta x$ measurements as numpy array inputs, and will return $\bar{x}$, $\delta \bar{x}_{statistical}$, and $\delta \bar{x}_{systematic}$.  We can use this throughout the course to calculate weighted averages and uncertainties!!!!!!

```python
import numpy as np

def weighted_average(x,deltax):
    # calculate the statistically weighted average of density
    sq_sum1 = 0.0
    sq_sum2 = 0.0
    sq_sum3 = 0.0
    sq_sum4 = 0.0

    for i in range(len(x)):
        sq_sum1 = sq_sum1 + x[i]/deltax[i]**2
        sq_sum2 = sq_sum2 + 1.0/deltax[i]**2
        sq_sum3 = sq_sum3 + x[i]**2/deltax[i]**2
        sq_sum4 = sq_sum4 + deltax[i]**2

    xbar = sq_sum1/sq_sum2
    delta_xbar_systematic = np.sqrt(1/(len(x)-1)*(np.abs(sq_sum3/sq_sum2 - xbar**2)))
    delta_xbar_statistical = 1.0/len(x)*np.sqrt(sq_sum4)

    return xbar,delta_xbar_statistical,delta_xbar_systematic

x = np.array([100.0,102.0,98.0])
deltax = np.array([0.1,0.2,0.02])

xbar,delta_xbar_statistical,delta_xbar_systematic = weighted_average(x,deltax)

print ("")
print ("Average Density = %0.4f +/- %0.4f (syst.) +/- %0.4f (stat.) g/cm^3" %
       (xbar,delta_xbar_systematic,delta_xbar_statistical))
```

    Average Density = 98.1143 +/- 0.3819 (syst.) +/- 0.0748 (stat.) g/cm^3

### Direct Measurement
A measurement of the height and radius of the cylinder allows you to compute the volume using the formula:

\begin{equation}
V = \pi r^2 h
\end{equation}

The propagated uncertainties can again be derived from equation $\eqref{eq_unc}$. If our uncertainty in
measuring the radius and height are $\delta r$ and $\delta h$, we first we compute the uncertainty in $r^2$:

\begin{equation}
\frac{\delta \left(r^2 \right)}{r^2} = 2 \frac{\delta r}{r}
\end{equation}

and then we compute the uncertainty in volume:

\begin{equation}
\left(\frac{\delta V}{V}\right)^2 = \left(\frac{\delta \left(r^2 \right)}{r^2} \right)^2 + \left(\frac{\delta h}{h}\right)^2
\end{equation}

or, substituting:

\begin{equation}
\left(\frac{\delta V}{V}\right)^2 = \left(2 \frac{\delta r}{r} \right)^2 + \left(\frac{\delta h}{h}\right)^2
\end{equation}

Notice that we can take the left side directly and use it to compute the uncertainty in density once we have the uncertainty in measuring the mass. Instead of re-computing it every time you need it, it might be useful to write this value down for each rod.

### Measurement of Displaced Water
This measurement gives a direct result with the uncertainty, $\delta V$, given by the uncertainty in measuring the markings
on the graduated cylinder.

### Measurement of Equivalent Weight of Water
This is probably the most difficult measurement to understand, but the easiest to compute with very good accuracy. If you simply drop a mass into the water and let it sit at the bottom of the beaker, the weight you will measure will simply be the weight of the water plus the weight of the mass. However, if the mass is *suspended* in the water, the tension in the string will take up some of that weight. How much?

Imagine that a block of Styrofoam were in the water. It would float. Why? Because it is *less* dense than water. Now imagine a block of gold in the water. It would sink. Why? Because it is *more* dense than water. So if we had something with *exactly* the same density as the water, it would neither float nor sink: it would remain stationary at any depth.

By using the tension in the string to support the cylinder, we are compensating for the higher density and forcing the cylinder to remain stationary---so it acts as if it were the same density as the water. When we measure the weight of the water plus the *suspended* mass, we should get the weight of the water *plus the weight of water which has the same volume as the cylinder*. Therefore, we find the volume of the cylinder by, first, finding the difference in mass with and without the cylinder:

\begin{equation}
m_{w} =  m_{2} - m_{1}
\end{equation}

The uncertainty in this determination comes from equation $\eqref{eq_unc}$:

\begin{equation}
\left(\delta m_{w} \right)^2 = \left(\delta m_{1} \right)^2 + \left(\delta m_{2} \right)^2
\end{equation}

Then we use equation $\eqref{eq_density}$ to determine the volume of the cylinder:

\begin{equation}
V = \frac{m_{w}}{\rho_{w}}
\end{equation}

where $ \rho_{w}$ is the density of the water. This has an uncertainty given by:

\begin{equation}
\left(\frac{\delta V}{V}\right)^2 = \left(\frac{\delta m_{w}}{m_{w}}\right)^2 + \left(\frac{\delta \rho_{w}}{\rho_{w}}\right)^2
\end{equation}

Then we can use equation $\eqref{eq_unc_den}$ to compute the uncertainty in measuring the density of the rod.

# Experimental Procedure
Once you’ve picked three different-sized rods made of the same material, the first item of business is to record the common data for each of the three experiments: the masses of each of the cylinders. Enter this in the cell immediately below this one, in the appropriate place.

Remember that, when estimating your measurement uncertainty, a good rule of thumb is to use plus-or-minus half of the smallest increment you can measure directly. Sometimes you can use a smaller value than this if you feel you can interpret the spaces between the markings more accurately. For digital measurements, however, this is not possible.

Note that you will record here the $absolute$ error in the mass measurement, and then we subsequently $calculate$ the fractional error in the mass, $\delta m /m$.

```python
# Raw Data
# Table 1: Mass Characteristics of Test Cylinders

# Import Numpy and Pandas with standard names
import pandas as pd
import numpy as np

# Create an empty numpy array to hold the raw data
raw_data_1 = np.empty((3,3))

# Set the cylinder number column identifiers for each cylinder
raw_data_1[0][0]=1
raw_data_1[1][0]=2
raw_data_1[2][0]=3

# Create a Pandas dataframe, and convert the cylinder number column to integer format
df1 = pd.DataFrame(raw_data_1, columns=["Cylinder Number",
                                        "Mass (g)",
                                        "𝛿m (g)"])
df1['Cylinder Number'] = df1['Cylinder Number'].astype(int)

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

df1['Mass (g)'] = [42.000,141,339.000]
df1['𝛿m (g)'] = [0.010,0.020,0.030]

###########################################

# Calculate 𝛿m/m, and display the dataframe
df1['𝛿m/m'] = df1['𝛿m (g)']/df1['Mass (g)']

from IPython.display import display
display(df1)
print ("Table 1: Mass Characteristics of Test Cylinders")
```

      Cylinder Number
      Mass (g)
      𝛿m (g)
      𝛿m/m

      0
      1
      42.0
      0.01
      0.000238

      1
      2
      141.0
      0.02
      0.000142

      2
      3
      339.0
      0.03
      0.000088

    Table 1: Mass Characteristics of Test Cylinders

### Direct Measurement
It is important that, whenever you use a tool to measure something, that tool reads zero when nothing is present (not verifying this means you may have a “Zero Error”---see your Lab Manual). Before measuring a cylinder with the calipers and micrometer, close the jaws of each completely and ensure that you read zero before you take a measurement. If it does not read zero, you will need to either fix the zero reading or compensate for it.

#### Using the Digital Calipers to Measure Length
First, verify that the digital calipers read in millimeters. You should see the “mm” symbol appear in the display. If it is reading in inches, press the blue “mm/inch” button one time.
Second, you must ensure that the calipers read zero when you close the jaws and put just a little bit of pressure on the thumb wheel. If the reading is not zero, hold the jaws closed with just a little pressure on the thumb wheel and (with the other hand) press the “ZERO” button one time. The display should now read zero.

You can now take measurements of the length of the rod as shown in Figure 1.

Notice that your accuracy is limited to two decimal places, so your height measurement will have an uncertainty of $\delta h$ = 0.005 mm. You should take three or four measurements by rotating the cylinder slightly between each measurement and report the average value.

#### Using the Digital Micrometer to Measure Diameter
Now measure the diameter of your cylinder with the micrometer. Once again, we first need to ensure we’re measuring in millimeters. If the display does not read “mm,” press and hold the “in/mm” button until it does. Second, we check to make sure that the micrometer reads zero when nothing is in place. To do this with the MARATHON CO 0030025 micrometer, close the gap on the micrometer by rotating the **big** wheel until it clicks. If the reading on the micrometer is not zero, press and **hold** the “SET” button until it does.

You can now take measurements of the diameter of the rod as shown in Figure 2. Notice that the rod is standing on the table while one hand holds the frame of the micrometer and the other hand rotates the big knob. ***Always use the big knob when changing the gap in the digital micrometer.*** This knob has a ratchet which, when properly used, always gives the same pressure to the measuring bar keeping all of your measurements consistent.

Because the accuracy is is 0.001 mm, your uncertainty in the diameter of the cylinder will be $\delta d$ = 0.0005 mm. What is the uncertainty in your radius? Using Equation $\eqref{eq_unc}$ we find that

\begin{equation}
\left(\frac{\delta r}{r}\right)^2 = \left(\frac{\delta d}{d}\right)^2
\end{equation}

\begin{equation}
\delta r = \delta d \left(\frac{r}{d}\right) = \delta d \left(\frac{r}{2r}\right)
\end{equation}

But the radius is half the diameter so

\begin{equation}
\delta r = \frac{\delta d}{2}
\end{equation}

as you might have expected.

Enter your raw data into the cell immediately below this one in the appropriate place!

```python
# Raw Data
# Table 2: Details of Direct Measurements of Cylinders

# Create an empty numpy array to hold the raw data
raw_data_2 = np.empty((3,10))

# Set the cylinder number column identifiers for each cylinder
raw_data_2[0][0]=1
raw_data_2[1][0]=2
raw_data_2[2][0]=3

# Create a Pandas dataframe, and convert the cylinder number column to integer format
df2 = pd.DataFrame(raw_data_2, columns=["Cylinder",
                                        "h (cm)",
                                        "𝛿h/h",
                                        "d (cm)",
                                        "r (cm)",
                                       "𝛿r/r",
                                        "V (cm^3)",
                                        "𝛿V/V",
                                        "ρ",
                                        "𝛿ρ"])
df2['Cylinder'] = df2['Cylinder'].astype(int)

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

# measurements of the height and diameter of each cylinder
df2['h (cm)'] = [1.000,1.500,2.000]
df2['d (cm)'] = [4.00,6.00,8.00]

# uncertainties in height and diameter
delta_h = 0.005
delta_d = 0.007

###########################################

# Calculate the various columns, and display the dataframe
df2['𝛿h/h'] = delta_h/df2['h (cm)']
df2['r (cm)'] = df2['d (cm)']/2.0
df2['𝛿r/r'] = delta_d/df2['d (cm)']
df2['V (cm^3)'] = np.pi*df2['r (cm)']**2*df2['h (cm)']

# From equation 6 in the theory section
df2['𝛿V/V'] = np.sqrt(4.0*df2['𝛿r/r']**2 + df2['𝛿h/h']**2)

df2['ρ'] = df1['Mass (g)']/df2['V (cm^3)']

# From equation 2 in the theory section
df2['𝛿ρ'] = df2['ρ']*np.sqrt(df1['𝛿m/m']**2 + df2['𝛿V/V']**2)

rho_bar,delta_rho_statistical,delta_rho_systematic = weighted_average(df2['ρ'],df2['𝛿ρ'])

display(df2)
print ("Table 2: Details of Direct Measurements of Cylinders")
print("")
print ("Average Density = %0.4f +/- %0.4f (syst.) +/- %0.4f (stat.) g/cm^3" %
       (rho_bar,delta_rho_systematic,delta_rho_statistical))

```

      Cylinder
      h (cm)
      𝛿h/h
      d (cm)
      r (cm)
      𝛿r/r
      V (cm^3)
      𝛿V/V
      ρ
      𝛿ρ

      0
      1
      1.0
      0.005000
      4.0
      2.0
      0.001750
      12.566371
      0.006103
      3.342254
      0.020414

      1
      2
      1.5
      0.003333
      6.0
      3.0
      0.001167
      42.411501
      0.004069
      3.324570
      0.013535

      2
      3
      2.0
      0.002500
      8.0
      4.0
      0.000875
      100.530965
      0.003052
      3.372095
      0.010295

    Table 2: Details of Direct Measurements of Cylinders

    Average Density = 3.3530 +/- 0.0153 (syst.) +/- 0.0089 (stat.) g/cm^3

### Measurements Using Water
Because the final measurement requires knowledge of the density of water, we should measure this now while the graduated cylinders are dry. **You will need to measure the mass of a known volume of water.** I will leave the details up to you, but remember that you will need to determine your uncertainty in this measurement, again using Equation $\eqref{eq_unc}$:

\begin{equation}
\left(\frac{\delta \rho_{w}}{\rho_{w}}\right)^2 = \left(\frac{\delta m_{w}}{m}\right)^2 + \left(\frac{\delta V}{V}\right)^2
\end{equation}

### Measurement of Displaced Water
The graduated cylinders measure liquid volume in milliliters. But this unit is the same as a cubic centimeter ($cm^3$). You will need to fill the graduated cylinder with just enough water to fully submerge the cylinder (about 25 mL of water should do). Measure this amount of water by reading the location of the bottom of the meniscus.

Care must be taken when using a graduated cylinder. They are tall and thin and tip easily. When they tip, they spill their contents and---quite often---break.

Even though we are only using water, general practice demands that you ***never*** set the graduated cylinder on a table and crouch down so that your eyes are level with the cylinder. If the graduated cylinder should tip, its contents could splash into your face. In a general chemistry laboratory, you might well be using liquids much less benign than water. The proper way to read a graduated cylinder is to hold it up from the top so that it hangs straight down at eye level.

You will need to measure the difference between the volume with $\left(V_{2} \right)$ and without $\left(V_{1} \right)$ the cylinder. **Care must be taken when placing the metal cylinder into the graduated cylinder.** The metal cylinder can easily break the graduated cylinder if you simply drop the metal into the glass. Even if you don’t break the graduated cylinder, you will splash water out of the cylinder which will make your measurement unusable. For this reason, **tilt the graduated cylinder as much as possible and allow the metal cylinder to gently slide down the glass wall.** Be certain that the metal cylinder is completely dry before putting it in the water and ensure that it is fully submerged before you take data. Record the difference in water volume readings. If your graduated cylinder is marked in 1 mL increments, then your uncertainty *in each measurement* is no more than $\delta V = 0.5 cm^3$, but could be a bit less if you are very careful.

Enter your raw data into the cell immediately below this one in the appropriate place!

```python
# Raw Data
# Table 3: Details of Volume Measurement of Cylinders

# Create an empty numpy array to hold the raw data
raw_data_3 = np.empty((3,8))

# Set the cylinder number column identifiers for each cylinder
raw_data_3[0][0]=1
raw_data_3[1][0]=2
raw_data_3[2][0]=3

# Create a Pandas dataframe, and convert the cylinder number column to integer format
df3 = pd.DataFrame(raw_data_3, columns=["Cylinder",
                                        "V1 (cm^3)",
                                        "V2 (cm^3)",
                                        "V (cm^3)",
                                        "𝛿V (cm^3)",
                                        "𝛿V/V",
                                        "ρ (g/cm^3)",
                                        "𝛿ρ (g/cm^3)"])
df3['Cylinder'] = df3['Cylinder'].astype(int)

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

# measurements of the volume with and without each cylinder
df3['V1 (cm^3)'] = [32.5,92.4,200.5]
df3['V2 (cm^3)'] = [20.0,50.0,100.0]

###########################################

# Calculate the various columns, and display the dataframe
delta_V = 0.5
df3['𝛿V (cm^3)'] = np.sqrt(2.0)*delta_V
df3['V (cm^3)'] =df3['V1 (cm^3)'] - df3['V2 (cm^3)']
df3['𝛿V/V'] = df3['𝛿V (cm^3)']/df3['V (cm^3)']

df3['ρ (g/cm^3)'] = df1['Mass (g)']/df3['V (cm^3)']

# From equation 2 in the theory section
df3['𝛿ρ (g/cm^3)'] = df3['ρ (g/cm^3)']*np.sqrt(df1['𝛿m/m']**2 + df3['𝛿V/V']**2)

rho_bar,delta_rho_statistical,delta_rho_systematic = weighted_average(df3['ρ (g/cm^3)'],df3['𝛿ρ (g/cm^3)'])

display(df3)
print ("Table 3: Details of Volume Measurement of Cylinders")
print("")
print ("Average Density = %0.4f +/- %0.4f (syst.) +/- %0.4f (stat.) g/cm^3" %
       (rho_bar,delta_rho_systematic,delta_rho_statistical))

```

      Cylinder
      V1 (cm^3)
      V2 (cm^3)
      V (cm^3)
      𝛿V (cm^3)
      𝛿V/V
      ρ (g/cm^3)
      𝛿ρ (g/cm^3)

      0
      1
      32.5
      20.0
      12.5
      0.707107
      0.056569
      3.360000
      0.190072

      1
      2
      92.4
      50.0
      42.4
      0.707107
      0.016677
      3.325472
      0.055461

      2
      3
      200.5
      100.0
      100.5
      0.707107
      0.007036
      3.373134
      0.023735

    Table 3: Details of Volume Measurement of Cylinders

    Average Density = 3.3657 +/- 0.0121 (syst.) +/- 0.0665 (stat.) g/cm^3

### Measurement of Equivalent Weight of Water
For your final measurement of the volume of the metal cylinders, you will need to fill a beaker with some water and place it on your scale. Note the mass of the water plus beaker for each measurement as $m_{1}$.

Next, tie a loop of wire around your completely dry metal cylinder and be sure that it can be dangled without dropping. Then, gently lower the cylinder into the water until it is completely submerged. **Do not let it touch the bottom of the beaker.** Use a clamp to hang the metal at this height so that the tension will not change in the wire. Measure the new mass, $m_{2}$, and use the information in the theory section to compute the volume of the cylinder.

Enter your raw data into the cell immediately below this one in the appropriate place!

```python
# Raw Data
# Table 4: Details of Displaced Water Measurement of Cylinders

# Create an empty numpy array to hold the raw data
raw_data_4 = np.empty((3,10))

# Set the cylinder number column identifiers for each cylinder
raw_data_4[0][0]=1
raw_data_4[1][0]=2
raw_data_4[2][0]=3

# Create a Pandas dataframe, and convert the cylinder number column to integer format
df4 = pd.DataFrame(raw_data_4, columns=["Cylinder",
                                        "m1 (g)",
                                        "m2 (g)",
                                        "mw (g)",
                                        "𝛿mw (g)",
                                        "𝛿mw/mw",
                                        "V (cm^3)",
                                        "𝛿V/V",
                                        "ρ (g/cm^3)",
                                        "𝛿ρ (g/cm^3)"])
df4['Cylinder'] = df4['Cylinder'].astype(int)

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

# mass of cylinders
df4['m1 (g)'] = [20.0,50.0,100.0]
df4['m2 (g)'] = [32.5,92.4,200.5]

# Density and uncertainty of water
density_water = 1.0
density_water_uncertainty = 0.002

# uncertainties in the weights
delta_m1 = 0.005
delta_m2 = 0.005

###########################################

# Calculate the various columns, and display the dataframe
df4['mw (g)'] =df4['m2 (g)'] - df4['m1 (g)']

df4['𝛿mw (g)'] = np.sqrt(delta_m1**2 + delta_m2**2)

df4['𝛿mw/mw'] = np.sqrt((delta_m1/df4['m1 (g)'])**2 + (delta_m2/df4['m2 (g)'])**2)

df4['V (cm^3)'] = df4['mw (g)']/density_water

df4['𝛿V/V'] = np.sqrt(df4['𝛿mw/mw']**2 + (density_water_uncertainty/density_water)**2)

df4['ρ (g/cm^3)'] = df1['Mass (g)']/df4['V (cm^3)']

#df4['𝛿V (cm^3)'] = df4['V (cm^3)']*(np.sqrt(df4['𝛿mw (g)']**2 + (df4['𝛿ρ (g/cm^3)']/df4['ρ (g/cm^3)'])**2))

# From equation 2 in the theory section
df4['𝛿ρ (g/cm^3)'] = df4['ρ (g/cm^3)']*np.sqrt(df4['𝛿mw/mw']**2 + df4['𝛿V/V']**2)

rho_bar,delta_rho_statistical,delta_rho_systematic = weighted_average(df4['ρ (g/cm^3)'],df4['𝛿ρ (g/cm^3)'])

display(df4)
print ("Table 4: Details of Displaced Water Measurement of Cylinders")
print("")
print ("Average Density = %0.4f +/- %0.4f (syst.) +/- %0.4f (stat.) g/cm^3" %
       (rho_bar,delta_rho_systematic,delta_rho_statistical))

```

      Cylinder
      m1 (g)
      m2 (g)
      mw (g)
      𝛿mw (g)
      𝛿mw/mw
      V (cm^3)
      𝛿V/V
      ρ (g/cm^3)
      𝛿ρ (g/cm^3)

      0
      1
      20.0
      32.5
      12.5
      0.007071
      0.000294
      12.5
      0.002021
      3.360000
      0.006863

      1
      2
      50.0
      92.4
      42.4
      0.007071
      0.000114
      42.4
      0.002003
      3.325472
      0.006672

      2
      3
      100.0
      200.5
      100.5
      0.007071
      0.000056
      100.5
      0.002001
      3.373134
      0.006752

    Table 4: Details of Displaced Water Measurement of Cylinders

    Average Density = 3.3526 +/- 0.0143 (syst.) +/- 0.0039 (stat.) g/cm^3

# Your Lab Report
### EXPERIMENTAL PROCEDURE Section
In one or two paragraphs, ***describe the method you used to measure the density of tap water***. **You will be graded on your ability to clearly describe what you did**. The description should be clear enough that someone else could reproduce your results by reading this section. **Although it is tempting to write “instructions,” you will be graded poorly if you do so.**

### DATA Section
What was your average measured density for the substance (you must include the uncertainty!)?

### CONCLUSION section
In a few paragraphs, answer the following questions. Do ***NOT*** use bullets, use full paragraphs.
* What substance do you think you have?
    * What is the accepted density of that material (CITE YOUR SOURCE)?
    * How does your measured density compare to the accepted density [use the percent error formula]?
    * Was density alone enough to identify the material? What other qualities did you use?
* How well do the three methods measure volume (compare the $ \delta V$ and $ \frac{\delta V}{V}$ for each)?
* Does each method give similar answers for the volume of each cylinder?
* How does the uncertainty in the volume measurement affect the uncertainty in the density measurement?
* What sources of error/uncertainty did you have? How could you have removed or reduced them?

### Appendix
- Include your signed and completed data sheet.
- Include a sample calculation for every computation you made.


```python
import torch
import torchvision
import torchvision.transforms as transforms
import numpy as np
import matplotlib.pyplot as plt
```

```python
def gradient_penalty(D, x_true, x_fake, derivative):
    if len(x_true.shape) == 2:
        alpha = torch.rand(x_true.size()[0],1, device=x_true.device)
    else:
        alpha = torch.rand(x_true.size()[0],1,1,1, device=x_true.device)

    alpha = alpha.expand(x_true.size())
    interpolates = alpha * x_true + (1-alpha) * x_fake
    interpolates = torch.autograd.Variable(interpolates,requires_grad=True)
    disc_interpolates = D(interpolates)
    gradients = torch.autograd.grad(outputs=disc_interpolates,
                                    inputs=interpolates,
                                    grad_outputs=torch.ones(disc_interpolates.size(), device=x_true.device),
                                    create_graph=True,
                                    retain_graph=True,
                                    only_inputs=True)[0]
    gradient_penalty = ((gradients.norm(2,dim=1)-derivative) ** 2).mean() * 10
    return gradient_penalty
```

```python
def sample_swissroll(N):
    r = torch.sqrt(torch.linspace(0, 1, N)) * 780 * (2*np.pi)/360
    dx = -torch.cos(r)*r*0.05 + torch.rand(N) * (0.05)
    dy = torch.sin(r)*r*0.05 + torch.rand(N) * (0.05)
    x = torch.stack([dx, dy]).t()
    return x

def sample_gmm(N, k):
    x = torch.randn(N, 2)
    cluster_size = N // k
    for i in range(k):
        rand_std = torch.rand(1,2)*2 + 0.5
        rand_mu = torch.rand(1,2)*24 - 12
        x[i*cluster_size:(i+1)*cluster_size] = x[i*cluster_size:(i+1)*cluster_size] * rand_std + rand_mu
    return x
```

# 1. Training GAN

Original GAN objective for $G$ and $D$:

\begin{equation}
\min_G \max_D \quad \mathbb{E}_{x \sim p_{data}(x)}[\log D(x)]+\mathbb{E}_{z \sim p_z{z}}[\log(1-D(G(z)))]
\end{equation}

Wassersten GAN objective for $D$:

\begin{equation}
\max_D \quad \mathbb{E}_{x \sim p_{data}(x)}[D(x)] - \mathbb{E}_{x \sim p_{fake}} [D(x)]
\end{equation}
subject to D is 1-Lipschitz.

```python
NUM_OF_POINTS = 2500
Z_DIM = 1
```

```python
x = sample_swissroll(NUM_OF_POINTS)
```

```python
plt.scatter(x[:, 0], x[:, 1], c="b", alpha=0.1)
plt.axis("equal")
```

```python
G = torch.nn.Sequential(
    torch.nn.Linear(Z_DIM, 32),
    torch.nn.ReLU(),
    torch.nn.Linear(32, 32),
    torch.nn.ReLU(),
    torch.nn.Linear(32, 2)
)

D = torch.nn.Sequential(
    torch.nn.Linear(2, 32),
    torch.nn.ReLU(),
    torch.nn.Linear(32, 32),
    torch.nn.ReLU(),
    torch.nn.Linear(32, 1)
)
```

```python
G
```

    Sequential(
      (0): Linear(in_features=1, out_features=32, bias=True)
      (1): ReLU()
      (2): Linear(in_features=32, out_features=32, bias=True)
      (3): ReLU()
      (4): Linear(in_features=32, out_features=2, bias=True)
    )

```python
D
```

    Sequential(
      (0): Linear(in_features=2, out_features=32, bias=True)
      (1): ReLU()
      (2): Linear(in_features=32, out_features=32, bias=True)
      (3): ReLU()
      (4): Linear(in_features=32, out_features=1, bias=True)
    )

```python
optim_g = torch.optim.Adam(lr=0.001, params=G.parameters())
optim_d = torch.optim.Adam(lr=0.001, params=D.parameters())
# most of the time you should set this to Adam optimizer.
# see: https://github.com/soumith/ganhacks by Soumith Chintala et al.
```

```python
EPOCH = 30
BATCH_SIZE = 50
LOOP_PER_EPOCH = NUM_OF_POINTS // BATCH_SIZE
WASSERSTEIN = False
```

### Initial state

```python
with torch.no_grad():
    z = torch.randn(NUM_OF_POINTS, Z_DIM)
    x_fake = G(z)
plt.scatter(x[:, 0], x[:, 1], c="b", alpha=0.1)
plt.scatter(x_fake[:, 0], x_fake[:, 1], c="r", alpha=0.1)
plt.axis("equal")
```

### Epoch loop

```python
D_loss_hist = []
G_loss_hist = []
```

```python
for e in range(EPOCH):
    D_running_loss = 0.0
    G_running_loss = 0.0
    # for shuffling data
    # e.g. [1, 2, 3, 4, 5] -> [3, 5, 1, 4, 2]
    R = torch.randperm(NUM_OF_POINTS)
    for i in range(LOOP_PER_EPOCH):
        #######################
        # TRAIN DISCRIMINATOR #
        #######################

        # first, generate some fake data.
        # we should not train G in the discriminator
        # training loop.
        with torch.no_grad():
            z = torch.randn(BATCH_SIZE, Z_DIM)
            x_fake = G(z)

        # draw some real data.
        x_real = x[R[i*BATCH_SIZE:(i+1)*BATCH_SIZE]]
        # estimate logits
        logit_real = D(x_real)
        logit_fake = D(x_fake)

        if WASSERSTEIN:
            # try to increase discrepancy by
            # decreasing logit_fake
            # increasing logit_real
            D_loss = (logit_fake-logit_real).mean()
            # with respect to:
            # D is 1-Lipschitz (i.e. grad(D, x) <= 1)
            D_loss += gradient_penalty(D, x_real, x_fake, derivative=1)
            # (though they actually use grad(D, x) == 1)
            # D approximates Wasserstein-1 distance.

        else:
            # estimate probabilities
            p_real = torch.sigmoid(logit_real)
            p_fake = torch.sigmoid(logit_fake)
            # compute loss
            D_loss = -(torch.log(p_real) + torch.log(1-p_fake)).mean()

        # compute gradients
        optim_d.zero_grad()
        D_loss.backward()
        # gradient descent step
        optim_d.step()
        # log
        D_running_loss += D_loss.item() / 1

        #######################
        #   TRAIN GENERATOR   #
        #######################

        # first, stop the unnecessary gradient computation
        # for the discriminator, as we are optimizing for G.
        for p in D.parameters():
            p.requires_grad = False

        z = torch.randn(BATCH_SIZE, Z_DIM)
        x_fake = G(z)
        logit_fake = D(x_fake)
        if WASSERSTEIN:
            # try to decrease the discrepancy by taking a step
            # in the opposite direction of the objective of D.
            # increase logit_fake
            G_loss = -(logit_fake).mean()
        else:
            p_fake = torch.sigmoid(logit_fake)
            # compute loss
            # G_loss = -(-torch.log(1-p_fake)).mean()

            # an alternative better behaved loss
            # treat generated samples as real, and try to
            # increase the realness probability of fake samples
            G_loss = -(torch.log(p_fake)).mean()

        # compute gradients
        optim_g.zero_grad()
        G_loss.backward()
        # gradient descent step
        optim_g.step()
        # log
        G_running_loss += G_loss.item()

        # now we can open the gradient valves for D.
        for p in D.parameters():
            p.requires_grad = True

    D_running_loss /= LOOP_PER_EPOCH
    D_loss_hist.append(D_running_loss)
    G_running_loss /= LOOP_PER_EPOCH
    G_loss_hist.append(G_running_loss)

    print("Epoch: %d - D loss: %.5f, G loss: %.5f" % (e+1, D_running_loss, G_running_loss))
```

    Epoch: 1 - D loss: 1.38930, G loss: 0.68919
    Epoch: 2 - D loss: 1.38442, G loss: 0.68205
    Epoch: 3 - D loss: 1.35015, G loss: 0.69693
    Epoch: 4 - D loss: 1.32094, G loss: 0.73094
    Epoch: 5 - D loss: 1.46332, G loss: 0.64218
    Epoch: 6 - D loss: 1.30806, G loss: 0.84517
    Epoch: 7 - D loss: 1.40926, G loss: 0.77699
    Epoch: 8 - D loss: 1.46597, G loss: 0.60766
    Epoch: 9 - D loss: 1.41963, G loss: 0.62354
    Epoch: 10 - D loss: 1.40186, G loss: 0.63901
    Epoch: 11 - D loss: 1.38199, G loss: 0.65958
    Epoch: 12 - D loss: 1.36882, G loss: 0.68004
    Epoch: 13 - D loss: 1.38330, G loss: 0.67819
    Epoch: 14 - D loss: 1.42056, G loss: 0.66917
    Epoch: 15 - D loss: 1.36481, G loss: 0.74993
    Epoch: 16 - D loss: 1.39842, G loss: 0.71262
    Epoch: 17 - D loss: 1.40873, G loss: 0.69368
    Epoch: 18 - D loss: 1.38818, G loss: 0.72105
    Epoch: 19 - D loss: 1.38459, G loss: 0.70675
    Epoch: 20 - D loss: 1.40012, G loss: 0.68353
    Epoch: 21 - D loss: 1.38568, G loss: 0.68601
    Epoch: 22 - D loss: 1.36583, G loss: 0.72039
    Epoch: 23 - D loss: 1.38259, G loss: 0.71805
    Epoch: 24 - D loss: 1.42233, G loss: 0.64885
    Epoch: 25 - D loss: 1.38567, G loss: 0.67891
    Epoch: 26 - D loss: 1.39547, G loss: 0.67198
    Epoch: 27 - D loss: 1.39440, G loss: 0.66765
    Epoch: 28 - D loss: 1.39411, G loss: 0.67703
    Epoch: 29 - D loss: 1.38550, G loss: 0.69583
    Epoch: 30 - D loss: 1.36093, G loss: 0.73137

```python
with torch.no_grad():
    z = torch.randn(NUM_OF_POINTS, Z_DIM)
    x_fake = G(z)
plt.scatter(x[:, 0], x[:, 1], c="b", alpha=0.1)
plt.scatter(x_fake[:, 0], x_fake[:, 1], c="r", alpha=0.1)
plt.axis("equal")
```

```python
plt.plot(D_loss_hist, label="D loss")
plt.plot(G_loss_hist, label="G loss")
plt.ylabel("Loss")
plt.xlabel("Epoch")
plt.legend()
```

# 2. Using GANs to enforce a specific distribution

This application is first proposed in Adversarial Autoencoders, but it is a general procedure and can be applied to many problems.

```python
EPOCH = 30
BATCH_SIZE = 128
WASSERSTEIN = False

trainset = torchvision.datasets.MNIST(root="./data", train=True, download=True, transform=transforms.ToTensor())
testset = torchvision.datasets.MNIST(root="./data", train=False, download=True, transform=transforms.ToTensor())
loader = torch.utils.data.DataLoader(trainset, batch_size=BATCH_SIZE, shuffle=True)
```

```python
x, y = iter(loader).next()
```

```python
canvas = torchvision.utils.make_grid(x, nrow=16)
plt.imshow(canvas.permute(1,2,0))
```

```python
encoder = torch.nn.Sequential(
    torch.nn.Linear(784, 128),
    torch.nn.ReLU(),
    torch.nn.Linear(128, 128),
    torch.nn.ReLU(),
    torch.nn.Linear(128, 2),
    torch.nn.Tanh()
)
decoder = torch.nn.Sequential(
    torch.nn.Linear(2, 128),
    torch.nn.ReLU(),
    torch.nn.Linear(128, 128),
    torch.nn.ReLU(),
    torch.nn.Linear(128, 784)
)
discriminator = torch.nn.Sequential(
    torch.nn.Linear(2, 128),
    torch.nn.ReLU(),
    torch.nn.Linear(128, 128),
    torch.nn.ReLU(),
    torch.nn.Linear(128, 1)
)
```

```python
encoder.to("cuda")
decoder.to("cuda")
discriminator.to("cuda")
```

    Sequential(
      (0): Linear(in_features=2, out_features=128, bias=True)
      (1): ReLU()
      (2): Linear(in_features=128, out_features=128, bias=True)
      (3): ReLU()
      (4): Linear(in_features=128, out_features=1, bias=True)
    )

```python
optim_g = torch.optim.Adam(lr=0.0001, params=[{"params": encoder.parameters()},
                                             {"params": decoder.parameters()}])
optim_d = torch.optim.Adam(lr=0.0001, params=discriminator.parameters())
```

### Initial state

```python
loader2 = torch.utils.data.DataLoader(testset, batch_size=1000, shuffle=True)
x, y = iter(loader2).next()
with torch.no_grad():
    h = encoder(x.reshape(x.shape[0], -1).to("cuda")).cpu()
h_real = sample_swissroll(1000)
plt.scatter(h_real[:, 0], h_real[:, 1], c="b", alpha=0.1)
plt.scatter(h[:, 0], h[:, 1], c="r", alpha=0.1)
```

```python
loader3 = torch.utils.data.DataLoader(testset, batch_size=16, shuffle=True)
x, y = iter(loader3).next()
with torch.no_grad():
    h = encoder(x.reshape(16, -1).to("cuda"))
    x_bar = decoder(h)
    x_bar = x_bar.reshape(16, 1, 28, 28).cpu()
x_all = torch.cat([x, x_bar])
canvas = torchvision.utils.make_grid(x_all, nrow=16)
plt.figure(figsize=(16, 4))
plt.imshow(canvas.permute(1, 2, 0))
```

```python
D_loss_hist = []
G_loss_hist = []
MSE_loss_hist = []
```

```python
for e in range(EPOCH):
    MSE_running_loss = 0.0
    D_running_loss = 0.0
    G_running_loss = 0.0
    for i, (x, y) in enumerate(loader):
        #######################
        #  TRAIN AUTOENCODER  #
        #######################
        x = x.reshape(x.shape[0], -1).to("cuda")
        h = encoder(x)
        x_bar = decoder(h)
        mse_loss = ((x-x_bar)**2).mean()

        optim_g.zero_grad()
        mse_loss.backward()
        optim_g.step()

        MSE_running_loss += mse_loss.item()

        #######################
        # TRAIN DISCRIMINATOR #
        #######################
        with torch.no_grad():
            h_fake = encoder(x)
        h_real = sample_swissroll(x.shape[0]).to("cuda")

        logit_real = discriminator(h_real)
        logit_fake = discriminator(h_fake)

        if WASSERSTEIN:
            D_loss = (logit_fake-logit_real).mean()
            D_loss += gradient_penalty(discriminator, h_real, h_fake, derivative=1)
        else:
            p_real = torch.sigmoid(logit_real)
            p_fake = torch.sigmoid(logit_fake)
            D_loss = -(torch.log(p_real) + torch.log(1-p_fake)).mean()

        D_loss *= 0.1
        optim_d.zero_grad()
        D_loss.backward()
        optim_d.step()
        D_running_loss += D_loss.item()

        #######################
        #   TRAIN GENERATOR   #
        #######################
        for p in discriminator.parameters():
            p.requires_grad = False

        h_fake = encoder(x)
        logit_fake = discriminator(h_fake)
        if WASSERSTEIN:
            G_loss = -(logit_fake).mean()
        else:
            p_fake = torch.sigmoid(logit_fake)
            G_loss = -(torch.log(p_fake)).mean()

        G_loss *= 0.1
        optim_g.zero_grad()
        G_loss.backward()
        optim_g.step()
        G_running_loss += G_loss.item()

        for p in discriminator.parameters():
            p.requires_grad = True

    D_running_loss /= (i+1)
    D_loss_hist.append(D_running_loss)
    G_running_loss /= (i+1)
    G_loss_hist.append(G_running_loss)
    MSE_running_loss /= (i+1)
    MSE_loss_hist.append(MSE_running_loss)

    print("Epoch: %d- MSE loss: %.5f, D loss: %.5f, G loss: %.5f" % (e+1, MSE_running_loss, D_running_loss, G_running_loss))
```

    Epoch: 1- MSE loss: 0.04349, D loss: 0.13772, G loss: 0.07001
    Epoch: 2- MSE loss: 0.04346, D loss: 0.13776, G loss: 0.07000
    Epoch: 3- MSE loss: 0.04340, D loss: 0.13778, G loss: 0.06999
    Epoch: 4- MSE loss: 0.04333, D loss: 0.13782, G loss: 0.06998
    Epoch: 5- MSE loss: 0.04326, D loss: 0.13784, G loss: 0.06997
    Epoch: 6- MSE loss: 0.04320, D loss: 0.13785, G loss: 0.06995
    Epoch: 7- MSE loss: 0.04314, D loss: 0.13787, G loss: 0.06994
    Epoch: 8- MSE loss: 0.04308, D loss: 0.13787, G loss: 0.06994
    Epoch: 9- MSE loss: 0.04303, D loss: 0.13790, G loss: 0.06992
    Epoch: 10- MSE loss: 0.04298, D loss: 0.13791, G loss: 0.06991
    Epoch: 11- MSE loss: 0.04292, D loss: 0.13795, G loss: 0.06991
    Epoch: 12- MSE loss: 0.04287, D loss: 0.13795, G loss: 0.06989
    Epoch: 13- MSE loss: 0.04282, D loss: 0.13796, G loss: 0.06988
    Epoch: 14- MSE loss: 0.04279, D loss: 0.13800, G loss: 0.06988
    Epoch: 15- MSE loss: 0.04272, D loss: 0.13800, G loss: 0.06986
    Epoch: 16- MSE loss: 0.04269, D loss: 0.13803, G loss: 0.06987
    Epoch: 17- MSE loss: 0.04263, D loss: 0.13804, G loss: 0.06984
    Epoch: 18- MSE loss: 0.04258, D loss: 0.13804, G loss: 0.06983
    Epoch: 19- MSE loss: 0.04255, D loss: 0.13805, G loss: 0.06983
    Epoch: 20- MSE loss: 0.04251, D loss: 0.13805, G loss: 0.06981
    Epoch: 21- MSE loss: 0.04245, D loss: 0.13808, G loss: 0.06982
    Epoch: 22- MSE loss: 0.04241, D loss: 0.13809, G loss: 0.06981
    Epoch: 23- MSE loss: 0.04239, D loss: 0.13812, G loss: 0.06979
    Epoch: 24- MSE loss: 0.04234, D loss: 0.13810, G loss: 0.06979
    Epoch: 25- MSE loss: 0.04230, D loss: 0.13812, G loss: 0.06980
    Epoch: 26- MSE loss: 0.04226, D loss: 0.13813, G loss: 0.06978
    Epoch: 27- MSE loss: 0.04223, D loss: 0.13814, G loss: 0.06979
    Epoch: 28- MSE loss: 0.04219, D loss: 0.13814, G loss: 0.06977
    Epoch: 29- MSE loss: 0.04213, D loss: 0.13813, G loss: 0.06977
    Epoch: 30- MSE loss: 0.04210, D loss: 0.13817, G loss: 0.06976

```python
loader2 = torch.utils.data.DataLoader(testset, batch_size=1000, shuffle=True)
x, y = iter(loader2).next()
with torch.no_grad():
    h = encoder(x.reshape(x.shape[0], -1).to("cuda")).cpu()
h_real = sample_swissroll(1000)
plt.scatter(h_real[:, 0], h_real[:, 1], c="b", alpha=0.1)
plt.scatter(h[:, 0], h[:, 1], c="r", alpha=0.1)
```

```python
loader3 = torch.utils.data.DataLoader(testset, batch_size=16, shuffle=True)
x, y = iter(loader3).next()
with torch.no_grad():
    h = encoder(x.reshape(16, -1).to("cuda"))
    x_bar = decoder(h)
    x_bar = x_bar.reshape(16, 1, 28, 28).cpu()
x_all = torch.cat([x, x_bar])
canvas = torchvision.utils.make_grid(x_all, nrow=16)
plt.figure(figsize=(16, 4))
plt.imshow(canvas.permute(1, 2, 0))
```

```python
plt.plot(D_loss_hist, label="D loss")
plt.plot(G_loss_hist, label="G loss")
plt.ylabel("Loss")
plt.xlabel("Epoch")
plt.legend()
```

```python

```

inzva x METU ImageLab Joint Program/Week 6 - GANs 2/GANs - Bonus.ipynb


# Characterization of Systems in the Spectral Domain

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Communications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Phase and Group Delay

The [phase and group delay](https://en.wikipedia.org/wiki/Group_delay_and_phase_delay) characterize the phase and delay properties of an LTI system. Both quantify the frequency dependent delay that is imprinted on a signal when passing through a system. In many applications the delay introduced by a system should be as small as possible or within reasonable limits.

### Phase Delay

For an LTI system with transfer function $H(j \omega)$ the phase delay is defined as follows

\begin{equation}
t_p(\omega) = - \frac{\varphi(j \omega)}{\omega}
\end{equation}

where $\varphi(j \omega) = \arg \{ H(j \omega) \}$ denotes the phase of the transfer function. The phase delay quantifies the delay of a single harmonic exponential signal $e^{j \omega t}$ with frequency $\omega$ when passing through the system. The negative sign in the definition of the phase delay results in a positive phase delay $t_p(\omega) > 0$ when a signal is delayed by a system. Note that the phase delay may not be defined for $\omega = 0$.

**Example**

As example, the phase delay $t_p(\omega)$ is computed for the [2nd order low-pass filter introduced before](../laplace_transform/network_analysis.ipynb#Example:-Second-Order-Low-Pass-Filter). First the transfer function $H(j \omega)$ is defined in `SymPy`

```python
%matplotlib inline
import sympy as sym
sym.init_printing()

L, R, C = sym.symbols('L R C', positive=True)
w = sym.symbols('omega', real=True)
s = sym.I * w

H = 1 / (C*L*s**2 + C*R*s + 1)
H
```

Now the phase delay $t_p(\omega)$ is computed

```python
phi = sym.arg(H)
tp = - phi/w
tp
```

and the result is visualized using the normalized values $R=1$, $L=0.5$ and $C=0.4$ for the elements of the low-pass filter

```python
RLC = {R: 1, L: sym.Rational('.5'), C: sym.Rational('.4')}
sym.plot(tp.subs(RLC), (w, -10, 10),
         xlabel='$\omega$', ylabel='$t_p(j \omega)$')
```

    <sympy.plotting.plot.Plot at 0x106658208>

### Group Delay

The group delay is defined as the derivative of the phase with respect to the frequency

\begin{equation}
t_g(\omega) = - \frac{d \varphi(j \omega)}{d \omega}
\end{equation}

The group delay quantifies the delay the amplitude envelope of a group of exponential signals observes when passing through a system. The negative sign in above definition results in a positive group delay for a system imposing a delay onto the input signal. Note that the [phase](https://en.wikipedia.org/wiki/Instantaneous_phase) $\varphi(j \omega)$ is in general only unique for $- \pi < \varphi(j \omega) \leq \pi$. If the phase exceeds this range it is wrapped back. For meaningful results it is required to unwrap the phase before computing the group delay.

**Example**

The group delay $t_g(\omega)$ of above 2nd order low-pass filter is computed and plotted for the normalized values

```python
tg = - sym.diff(phi, w)
sym.plot(tg.subs(RLC), (w, -10, 10),
         xlabel='$\omega$', ylabel='$t_g(j \omega)$')
```

    <sympy.plotting.plot.Plot at 0x11686aeb8>

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples*.


# Excitation probability

Try and follow Baragiola *et al.* in their conventions:
\begin{align}
    \operatorname{Var}[\xi_t^2]
    &=
    \frac{1}{\Omega^2}
    \\
    \operatorname{Var}[\xi_t]
    &=
    \sigma_T^2=\frac{2}{\Omega^2}
    \\
    \operatorname{Var}[\xi_\omega^2]
    &=
    \Omega^2
    \\
    \operatorname{Var}[\xi_\omega]
    &=
    \sigma_\omega^2=\frac{\Omega^2}{2}
\end{align}

\begin{align}
    \xi_{t,\mathrm{gau}}&=\sqrt[4]{\frac{\Omega^2}{2\pi}}\exp\left(\frac{\Omega^2(t-t_a)^2}{4}\right)
\end{align}

```python
from functools import partial
import pickle
import numpy as np
from scipy.integrate import quad

from tqdm import tqdm_notebook

import pysme.hierarchy as hier

# #from quantum_utilities.qubits import Id, sigm, sigz, zero
```

```python
# Define Lowering operator and Sigma Z

Id = np.eye(2, dtype=complex)

zero = np.zeros((2,2), dtype=complex)

sigz = np.array([[1, 0], [0, -1]], dtype=complex)

sigm = np.array([[0, 0], [1, 0]], dtype=complex)
```

```python
def xi_gau(t, Omega, ta):
    return np.power(Omega**2/(2*np.pi), 1/4) * np.exp(-Omega**2*(t - ta)**2/4)
```

```python
quad(lambda t: xi_gau(t, Omega=0.5, ta=0)**2, -3/0.5, 3/0.5)
```

```python
rs = np.arcsinh(np.sqrt(np.arange(5)))
```

```python
np.exp(rs)
```

    array([1.        , 2.41421356, 3.14626437, 3.73205081, 4.23606798])

```python
np.sinh(rs)**2
```

    array([0., 1., 2., 3., 4.])

```python
Omegas = np.power(2, np.linspace(-3, 3, 7))
Omegas
```

    array([0.125, 0.25 , 0.5  , 1.   , 2.   , 4.   , 8.   ])

```python
S = Id
L = sigm
H = zero
```

```python
rho0 = (Id - sigz)/2
```

```python
try:
    with open('2019-11-20-bandwidth-me-solns.pickle', 'xb') as f:
        n_maxs = [25]
        factories = {n_max: hier.HierarchyIntegratorFactory(2, n_max) for n_max in n_maxs}

        integrators = {}
        for r_idx in tqdm_notebook(range(len(rs)), desc='r'):
            r = rs[r_idx]
            for Omega_idx in tqdm_notebook(range(len(Omegas)), desc='Omega', leave=False):
                Omega = Omegas[Omega_idx]
                for n_max, factory in tqdm_notebook(factories.items(), desc='n_max', leave=False):
                    integrators[n_max,r_idx,Omega_idx] = factory.make_uncond_integrator(
                        partial(xi_gau, Omega=Omega, ta=0), S, L, H, r, mu=0)

        solns = {}
        for Omega_idx in tqdm_notebook(range(len(Omegas)), desc='Omega'):
            Omega = Omegas[-1-Omega_idx]
            times = np.linspace(-3/Omega, 3/Omega, 2**10+1)
            for r_idx in tqdm_notebook(range(len(rs)), desc='r', leave=False):
                r = rs[r_idx]
                for n_maxy in tqdm_notebook(n_maxs, desc='n_max', leave=False):
                    solns[n_max,r_idx,Omega_idx] = integrators[n_max,r_idx,Omega_idx].integrate(rho0, times)
        pickle.dump(solns, f)
except FileExistsError:
    print('Data already exist.')
```

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:7: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`
      import sys

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:9: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`
      if __name__ == '__main__':

    HBox(children=(FloatProgress(value=0.0, description='Omega', max=7.0, style=ProgressStyle(description_width='i…

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:11: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`
      # This is added back by InteractiveShellApp.init_path()

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:11: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`
      # This is added back by InteractiveShellApp.init_path()

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:9: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`
      if __name__ == '__main__':

    HBox(children=(FloatProgress(value=0.0, description='Omega', max=7.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='Omega', max=7.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='Omega', max=7.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='Omega', max=7.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:16: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`
      app.launch_new_instance()

    HBox(children=(FloatProgress(value=0.0, description='Omega', max=7.0, style=ProgressStyle(description_width='i…

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:19: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    /Users/azaghal/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:21: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='r', max=5.0, style=ProgressStyle(description_width='initi…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

    HBox(children=(FloatProgress(value=0.0, description='n_max', max=1.0, style=ProgressStyle(description_width='i…

```python
with open('2019-11-20-bandwidth-me-solns.pickle', 'rb') as f:
    solns = pickle.load(f)
```

```python
def rect(x, a, b):
    return np.where(x < a, 0, np.where(x < b, 1, 0))

def rho_from_ket(ket):
    return np.outer(ket, ket.conj())

def vac_rho(n):
    ket = np.zeros(n + 1, dtype=np.complex)
    ket[0] = 1
    return rho_from_ket(ket)
```

```python
solns.keys()
```

    dict_keys([(25, 0, 0), (25, 1, 0), (25, 2, 0), (25, 3, 0), (25, 4, 0), (25, 0, 1), (25, 1, 1), (25, 2, 1), (25, 3, 1), (25, 4, 1), (25, 0, 2), (25, 1, 2), (25, 2, 2), (25, 3, 2), (25, 4, 2), (25, 0, 3), (25, 1, 3), (25, 2, 3), (25, 3, 3), (25, 4, 3), (25, 0, 4), (25, 1, 4), (25, 2, 4), (25, 3, 4), (25, 4, 4), (25, 0, 5), (25, 1, 5), (25, 2, 5), (25, 3, 5), (25, 4, 5), (25, 0, 6), (25, 1, 6), (25, 2, 6), (25, 3, 6), (25, 4, 6)])

```python
for r_idx in range(len(rs)):
    plt.semilogx(Omegas, [(solns[25,r_idx,Omega_idx].get_expectations(sigz, vac_rho(25)).max() + 1)/2
                          for Omega_idx in range(len(Omegas))],
                 label='$r={:.2f}$'.format(rs[r_idx]))
plt.xlabel(r'$\Omega$')
plt.ylabel(r'$\mathrm{Pr}(e)$')
plt.legend()
plt.ylim(-0.05, 1.05e3)
plt.tight_layout()
```

```python

```


# Echantillonnage d’importance adaptatif en Population Monte-Carlo

## 1. Echantillonnage d’importance et optimalité au sens de la variance asymptotique

### T1
On suppose que $\int|h(x)|\pi(x)dx < \infty$, quel théorème peut-on utiliser pour prouver que $\tilde{\Pi}_{g,n}(h)$ converge avec probabilité 1 vers $\Pi(h)$?

En notant $Y_i = h(X_i) \frac{\pi(X_i)}{g(Xi)}$, on peut utiliser la Loi Forte des Grands Nombres pour montrer que $\frac{1}{n} \sum_{i=1}^n Y_i \xrightarrow[]{p.s.} \mathbb{E}_g(Y_i)$.

Puisque $\mathbb{E}_g(Y_i) = \int h(X_i) \frac{\pi(X_i)}{g(Xi)} g(x) dx$, cela revient exactement à montrer que $\tilde{\Pi}_{g,n}(h) \rightarrow \Pi(h)$ avec une probabilité 1.

### T2
Quel hypothèse faut-il ajouter pour obtenir la propriété: pour tout intervalle $I$,
$\mathbb{P} \left( \sqrt{n} \left( \tilde{\Pi}_{g,n}(h) - \Pi(h) \right) / \sqrt{\tilde{V}_g(h)} \in I \right) \rightarrow \int_I \frac{e^{-x^2/2}}{\sqrt{2\pi}} dx$,
avec $\tilde{V}_g(h) = \int \pi(x) h^2(x) \frac{\pi(x)}{g(x)} dx - \left[ \Pi(x) \right]^2$ ?

L'hypothèse à ajouter est que $Y_i$ soit de carré intégrable, pour que $\int \pi(x) h^2(x) \frac{\pi(x)}{g(x)} dx$ ait un sens.

### S1

On prend pour $\Pi$ la loi uniforme sur [0,1] et on pose $h(x)=x$. Nous allons d'abord illustrer graphiquement en fonction de $n,$ I'approximation de $\Pi(h)$ par $\tilde{\Pi}_{g, n}(h)$ où on a posé $g=\pi .$ Pour cela, tirer $m=2000$ variables aléatoires $\left(X_{i}\right)_{1 \leq i \leq m}$ iid suivant une loi uniforme et tracer les valeurs prises par $\tilde{\Pi}_{g, n}(h)$ pour $n=1,2, \ldots, m$ On pourra aussi représenter les bornes inférieures et supérieures d'un intervalle de confiance à $95 \%$ de $\Pi(h)$ pour ces mêmes valeurs de $n$.

Ce rapport contient uniquement le code pour la visualisation. Vous trouverez l'implémentation de tous les algorithmes dans `spellbook.py`.

Ici, nous utilisons l'intervalle de confiance t, car on suppose que pour les grands n la distribution de $\tilde{\Pi}_{g, n}(h)$ est normale et que pour les grands n la distribution de Student est proche de la normale. En général, dans cet exemple, l'intervalle joue un rôle plutôt esthétique et la position exacte de ses limites n'est pas très importante.

```python
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
warnings.filterwarnings("ignore")
sns.set()
%matplotlib inline

SEED = 666
```

```python
from scipy.stats import t
def visualize_simple_simulation(estimated_values, estimated, title, ylim):
    m = len(estimated)
    intervals = np.array([t.interval(0.95, n - 1, estimated[:n].mean(), estimated[:n].std()/n**0.5) for n in range(1, m+1)])
    plt.figure(figsize=(20, 10))
    plt.plot(estimated_values, label=r"$\tilde{\Pi}_{g, n}(h)$")
    plt.ylabel(r"$\tilde{\Pi}_{g, n}(h)$", size=16)
    plt.xlabel(r"$n$", size=16)
    plt.fill_between(np.arange(0, m, 1), intervals[:, 0], intervals[:, 1], color='green', alpha=.5, label=r"95% t-confint")
    plt.ylim(*ylim)
    plt.title(title, size=16)
    plt.legend(fontsize=16)
    plt.show()
```

```python
from spellbook import uniform_0_1, run_simple_simulation
np.random.seed(SEED)

def h(x):
    return x

M = 2000
estimated_values, estimated = run_simple_simulation(h, uniform_0_1, uniform_0_1, M)
visualize_simple_simulation(estimated_values, estimated, "S1", (0.4, 0.6))
```

Nous pouvons voir que notre estimateur approche la vraie valeur 0.5. L'intervalle de confiance diminue à chaque fois.

### S2

Reprendre la question précédente, en remplaçant la densité $g$ par $g:[0,1] \mapsto \mathbb{R}^{+}$ telle que $g(x)=\frac{5}{2} x^{3 / 2}$ pour $x \in[0,1] .$ Pour simuler une variable aléatoire $X$ suivant cette densité $g,$ on pourra utiliser (en la justifiant) la méthode suivante: tirer une variable aléatoire $U$ suivant une loi uniforme sur [0,1] et poser $X=U^{2 / 5}$. Vérifier expérimentalement qu'il vaut mieux utiliser la densité instrumentale définie sur [0,1] par $g(x)=\frac{5}{2} x^{3 / 2}$ plutôt que la densité instrumentale uniforme: $g(x)=1$ pour $x \in[0,1]$.

Comme $\tilde{G}^{-1}(x) = x^{2/5}$ sur $[0,1]$ est la fonction quantile de v.a. à densité $g$, la v.a. $X= \tilde{G}^{-1}(U) =U^{2 / 5}$ est à densité $g$.

```python
from spellbook import Distribution, g_polynomial
np.random.seed(SEED)

M = 2000
estimated_values, estimated = run_simple_simulation(h, g_polynomial, uniform_0_1, M)
visualize_simple_simulation(estimated_values, estimated, "S2", (0.4, 0.6))
```

Dans ce cas la convergence est beaucoup plus rapide, la taille limite de l'intervalle de confiance est moindre. Cela montre que la densité instrumentale $g(x)=\frac{5}{2} x^{3 / 2}$ est plus appropriée pour estimation.

### T3
En utilisant l’inégalité de Cauchy-Schwarz, montrer que $\int \pi(x) h^2(x) \frac{\pi(x)}{g(x)} dx \geq \left( \int \pi(x)|h(x)|dx \right)$

En décomposant $\pi(x)h(x)$ en $\frac{\pi(x)h(x)}{\sqrt{g(x)}} \cdot \sqrt{g(x)}$, l'inégalité de Cauchy-Schwarz donne :

$\left( \int |\pi(x)h(x)| dx \right)^2 \leq \int \frac{\pi^2(x)h^2(x)}{|g(x)|} dx \cdot \int |g(x)|dx $

Par positivité de $\pi$ et $g$ qui sont des densités de probabilité,

$\left( \int \pi(x)|h(x)| dx \right)^2 \leq \int \frac{\pi^2(x)h^2(x)}{g(x)} dx \cdot \int g(x)dx $

Et puisque $g$ est une densité de probabilité, $\int g(x) dx = 1$, ce qui aboutit à l'inégalité demandée.

### T4
En déduire que $\tilde{V}_g(h) \geq \tilde{V}_{\tilde{g}}(h)$, avec $\tilde{g} = \frac{|h(x)| \pi(x)}{\int |h(y)|\pi(y) dy}$

$\tilde{V}_{\tilde{g}}(h) = \left( \int \pi(x)|h(x)| dx \right)^2 - \left[ \Pi(x) \right]^2$

Donc d'après la question précédente, il vient directement que $\tilde{V}_{\tilde{g}}(h) \leq \tilde{V}_g(h)$.
D'après Cauchy-Schwarz, il y a égalité lorsque $\frac{\pi(x)h(x)}{\sqrt{g(x)}}$ et $\sqrt{g(x)}$ sont proportionnels, soit lorsque $g(x) = \lambda \pi(x)h(x)$, avec $\lambda \in \mathbb{R}$.

### S3

Simulation. En guise d'illustration de la question précédente, on prendra pour $\Pi$ la loi $\mathcal{N}(0,1)$ et on posera $h(x)=x .$ Ecrire la densité $\tilde{g} .$ Calculer la fonction de répartition de $\tilde{g} .$ En déduire une façon de simuler un échantillon $\left(X_{i}\right)_{1 \leq i \leq n}$ de variables aléatoires indépendantes et de même loi de densité $\tilde{g}$.

\begin{equation}
\begin{split}
\int|h(y)| \pi(y) d y & = \int|y|\frac{1}{\sqrt{2 \pi}} \exp \left(-\frac{-y^{2}}{2}\right) d y = 2\int_{0}^{+\infty}y\frac{1}{\sqrt{2 \pi}} \exp \left(-\frac{-y^{2}}{2}\right) d y \\
& = \frac{1}{\sqrt{2 \pi}} \int_{0}^{+\infty} \frac{e^{-\frac{u}{2}}}{2} d u = \sqrt{\frac{2}{\pi}}
\end{split}
\end{equation}

\begin{equation}
\tilde{g}(x)=\frac{|h(x)| \pi(x)}{\int|h(y)| \pi(y) d y} = \frac{1}{2}|x|\exp\left(-\frac{x^2}{2}\right)
\end{equation}

La fonction de répartition de $\tilde{g}$:

\begin{equation}
\tilde{G}(x) = \begin{cases}
   \frac{1}{2}\exp\left(-\frac{x^2}{2}\right), x \le 0
   \\
   1 -  \frac{1}{2}\exp\left(-\frac{x^2}{2}\right), x > 0
\end{cases}
\end{equation}

Pour faire un tirage de cette distribution nous allons aussi besoin la fonction réciproque de $\tilde{G}(x)$:

\begin{equation}
\tilde{G}^{-1}(x) = \begin{cases}
   -\sqrt{-2\ln(2x)}, 0 \le x \le \frac{1}{2}
   \\
   \sqrt{-2\ln(2-2x)}, \frac{1}{2} < x \le 1
\end{cases}
\end{equation}

La v.a. $\tilde{G}^{-1}(U)$ est donc de la loi $\tilde{G}$.

Vérifions que l'échantillonnage de $\tilde{g}$ fonctionne correctement :

```python
from spellbook import g_tilde
np.random.seed(SEED)

plt.figure(figsize=(10, 10))
x = np.linspace(-4, 4, 10000)
sns.distplot(g_tilde.sample(1000000), label="sampled")
plt.plot(x, g_tilde.pdf(x), label="true pdf")
plt.legend()
plt.show()
```

Vérifier par simulation l'inégalité de la question 4 en prenant pour $g$ la densité de la loi $\mathcal{N}(0,1)$.

```python
from spellbook import normal_standard
np.random.seed(SEED)

M = 100000
estimated_values_g_tilde, estimated_tilde  = run_simple_simulation(h, g_tilde, normal_standard, M)
visualize_simple_simulation(estimated_values_g_tilde, estimated_tilde, "S3 optimal density", (-0.05, 0.05))

M = 100000
estimated_values_normal_standard, estimated_normal = run_simple_simulation(h, normal_standard, normal_standard, M)
visualize_simple_simulation(estimated_values_normal_standard, estimated_normal, "S3 normal density", (-0.05, 0.05))
```

Ici l'éstimateur avec la densité instrumentale optimale a les plus courts intervales de confiance. Il semble que sa variance soit aussi la plus faible. Pour vérifier cela faisons un tirage de la grandeur $\sqrt{n}\left(\tilde{\Pi}_{g, n}(h)-\Pi(h)\right)$ suivante: pour $m=100$ tirons $\tilde{\Pi}_{g, m}(h)$ $10000$ fois.

```python
np.random.seed(SEED)

est_tilde = []
est_norm = []

M = 100
for i in range(10000):
    est_tilde.append(run_simple_simulation(h, g_tilde, normal_standard, M)[0][-1]*M**0.5)
    est_norm.append(run_simple_simulation(h, normal_standard, normal_standard, M)[0][-1]*M**0.5)
```

Visualisons les distributions de $\sqrt{n}\left(\tilde{\Pi}_{g, n}(h)-\Pi(h)\right)$ obtenues.

```python
plt.figure(figsize=(10, 10))
sns.distplot(est_tilde, bins=20, label="optimal density")
sns.distplot(est_norm, bins=20, label="normal density")
plt.legend()
plt.show()
```

Les valeurs de la variance estimées:

```python
print(np.array(est_tilde).var(), np.array(est_norm).var())
```

    0.6473688119286807 1.0003134790472354

Cela confirme l'inegalité $\tilde{V}_{g}(h) \geq \tilde{V}_{\tilde{g}}(h)$. Les variances sont presques aux variances des valeurs $h\left(X\right) \frac{\pi\left(X\right)}{g\left(X\right)}$. Ceci est cohérent avec TLC:

```python
print(estimated_tilde.var(), estimated_normal.var())
```

    0.6366377865777144 0.9961142087693273

## 2. Réduction systématique de la variance

### T6
Vérifier que $\frac{\sigma^2(F(\alpha))}{\sigma^2(\alpha)} = \displaystyle{\int} \frac{\pi^2(x) h^2(x)}{\sum_{i=1}^d \alpha_i g_i(x)} \left( \displaystyle{\sum_{i=1}^d} \frac{\alpha_i g_i(x)}{\sum_{j=1}^d \alpha_j g_j(x)} \int \frac{g_i(u) \pi^2(u) h^2(u)}{[\sum_{k=1}^d \alpha_k g_k(u)]^2} du \right)^{-1} dx$

$\frac{\sigma^2(F(\alpha))}{\sigma^2(\alpha)} = \displaystyle{\int} \pi^2(x) h^2(x) \left( \displaystyle{\sum_{i=1}^d} \alpha_i \int \frac{g_i(u) \pi^2(u) h^2(u)}{[\sum_{k=1}^d \alpha_k g_k(u)]^2} du \cdot g_i(x) \right)^{-1} dx$

Puis en multipliant "en haut et en bas" par $\sum_{i=1}^d \alpha_i g_i(x)$ on obtient le résultat attendu.

### T7
En déduire en utilisant une inégalité de convexité que $\sigma^2(F(\alpha)) \leq \sigma^2(\alpha)$.

Nous pensons qu'ici il faut utiliser l'inégalité de Jensen mais nous ne voyons pas comment l'appliquer.

### T8 (Pas trop sûr)
Montrer que $n^{-1}\sum_{i=1}^n \left( \frac{h(X_i^t)\pi(X_i^t)}{\sum_{s=1}^d \tilde{\alpha}_s^t g_s(X_i^t)} \right)^2 \mathbb{1}_{Z_i^t = k}$ converge avec une probabilité 1 vers $\tilde{\alpha}_k^t \int g_k(x) \frac{\pi^2(x) h^2(x)}{[\sum_{i=1}^d \tilde{\alpha}_i^t g_i(x)]^2}$

En notant $Y_i^t = \frac{h(X_i^t)\pi(X_i^t)}{\sum_{s=1}^d \tilde{\alpha}_s^t g_s(X_i^t)}$,
la Loi Forte des Grands Nombres donne :

$\frac{1}{n} \sum_{i=1}^n (Y_i^t)^2 \mathbb{1}_{Z_i^t = k} \xrightarrow[]{p.s.} \displaystyle{\int} \left( \frac{h(x)\pi(x)}{\sum_{s=1}^d \tilde{\alpha}_s^t g_s(x)} \right)^2 \; f_{(X_i^t \cap Z_i^t = k)}(x) dx$

Et $f_{(X_i^t \cap Z_i^t = k)}(x) = f_{(X_i^t \; | \; Z_i^t = k)}(x) \cdot \mathbb{P}(Z_i^t = k) = g_k(x) \cdot \tilde{\alpha}_k^t$, d'où le résultat attendu.

### T9
En déduire que $n^{-1}\sum_{i=1}^n \left( \frac{h(X_i^t)\pi(X_i^t)}{\sum_{s=1}^d \tilde{\alpha}_s^t g_s(X_i^t)} \right)^2$ converge avec probabilité 1 vers $\sigma^2(\tilde{\alpha}^t)$.

En reprenant les notations de la question précédente :

$n^{-1}\sum_{i=1}^n (Y_i^t)^2 = \displaystyle{\sum_{k=1}^d} \left[ n^{-1}\sum_{i=1}^n (Y_i^t)^2 \mathbb{1}_{Z_i^t = k} \right]$

Donc d'après la question précédente :

$n^{-1}\sum_{i=1}^n (Y_i^t)^2 \xrightarrow[]{p.s.} \displaystyle{\sum_{k=1}^d} \tilde{\alpha}_k^t \int g_k(x) \frac{\pi^2(x) h^2(x)}{(\sum_{s=1}^d \tilde{\alpha}_s^t g_s(x))^2} dx $

En permutant les sommes on obtient :

$n^{-1}\sum_{i=1}^n (Y_i^t)^2 \xrightarrow[]{p.s.} \int g_k(x) \frac{\pi^2(x) h^2(x)}{\sum_{s=1}^d \tilde{\alpha}_s^t g_s(x)} dx = \sigma^2(\tilde{\alpha}^t)$

### S4

On mettra en oeuvre l'algorithme sur un exemple où la loi $\Pi$ visée est une $\mathcal{N}(0,1), h(x)=x, d=3,$ et les densités $g_{1}$ et $g_{2}$ sont respectivement la densité d'une $\mathcal{N}(0,1),$ la densité $\tilde{g}$ définie dans la partie 1 et $g_{3}$ est une densité simulable quelconque.

Comme $g_3$ j'ai choisi la distribution de Laplace avec la densite $g_3(x)=\frac{1}{2} \exp (-|x|)$.

```python
from spellbook import laplace, AdaptiveSampling
np.random.seed(SEED)

advs = AdaptiveSampling(h, [normal_standard, g_tilde, laplace], normal_standard)
history = advs.fit()
```

    Algorithm converged on iteration 77

Visualisons quelques resultats.

```python
plt.figure(figsize=(10, 8))
plt.plot(history["variance_log"], label=r"variance of $h\left(X\right) \frac{\pi\left(X\right)}{g\left(X\right)}$")
plt.xlabel("number of iteration", size=16)
plt.ylabel("variance", size=16)
plt.legend(fontsize=16)
plt.show()
```

```python
plt.figure(figsize=(10, 8))
plt.plot(np.vstack(history["alphas_log"])[:, 0], label="alpha_1")
plt.plot(np.vstack(history["alphas_log"])[:, 1], label="alpha_2")
plt.plot(np.vstack(history["alphas_log"])[:, 2], label="alpha_3")
plt.xlabel("number of iteration", size=16)
plt.ylabel("alpha", size=16)
plt.legend(fontsize=16)
plt.show()
```

```python
print(*advs.alphas)
```

    0.0 1.0 0.0

Le résultat est totalement attendu: l'algorithme a choisi la distribution optimale parmi disponible. Cela prouve une fois de plus que l'on ne peut faire mieux que $\tilde{g}$.

### S5

On cherchera aussi des exemples numériques pour $\Pi$ où on mettra en évidence que les $\tilde{\alpha}_{i}^{t}$ peuvent converger vers des quantités différentes de 0 et 1.

Posons la loi $\Pi$ visée est une $U(0,1), h(x)=x, d=3,$ et les densités $g_{3}$ et $g_{1}$ sont respectivement la densité de S1, la densité de S2 et $g_{2}$ est une densité suivante: $g_{2}(x)=\frac{1}{\ln2}\frac{1}{x+1}$ sur [0,1].

```python
from spellbook import g_hyperbolic
np.random.seed(SEED)

advs = AdaptiveSampling(h, [g_polynomial, g_hyperbolic, uniform_0_1], uniform_0_1)
history = advs.fit(tolerance=1e-4)
```

    Algorithm converged on iteration 121

```python
plt.figure(figsize=(10, 8))
plt.plot(history["variance_log"], label=r"variance of $h\left(X\right) \frac{\pi\left(X\right)}{g\left(X\right)}$")
plt.xlabel("number of iteration", size=16)
plt.ylabel("variance", size=16)
plt.legend(fontsize=16)
plt.show()
```

```python
plt.figure(figsize=(10, 8))
plt.plot(np.vstack(history["alphas_log"])[:, 0], label="alpha_1")
plt.plot(np.vstack(history["alphas_log"])[:, 1], label="alpha_2")
plt.plot(np.vstack(history["alphas_log"])[:, 2], label="alpha_3")
plt.xlabel("number of iteration", size=16)
plt.ylabel("alpha", size=16)
plt.legend(fontsize=16)
plt.show()
```

```python
print(*advs.alphas)
```

    0.8274335098105728 0.0 0.17256649018942707

Nous voyons que s'il n'y a pas de distribution optimale entre les distributions disponibles, l'algorithme est capable de converger vers des coefficients plus divers.

### S6

On mettra en évidence aussi la décroissance systématique de la variance à chaque itération de l'algorithme sur un exemple simple que vous choisirez.

Pour cette dernière simulation nous allons reprendre la configuration de S5. Après chaque itération de l'algoritme on va répéter le processus de S3. Pour faire cela utilosons le mode débogage de l'algoritme: `debug = True`.

```python
np.random.seed(SEED)

advs = AdaptiveSampling(h, [g_polynomial, g_hyperbolic, uniform_0_1], uniform_0_1)
history = advs.fit(max_iter=20, debug=True)
```

    Maximal iteration (20) is reached

Visualisons les distributions.

```python
plt.figure(figsize=(10, 8))

for i in range(10):
    sns.distplot(history["global_estimation_log"][i], label=f"step {i+1}", hist=False)
plt.legend(fontsize=16)
plt.title(r"distribution of $\sqrt{n}\tilde{\Pi}_{g, n}(h)$", size=16)
plt.show()
```

On peut voir qu'à chaque étape la variance est réduite. Visualisons les variance elles-mêmes.

```python
plt.figure(figsize=(10, 8))
plt.plot([history["global_estimation_log"][i].var() for i in range(20)])
plt.xlabel("number of iteration", size=16)
plt.ylabel(r"variance of $\sqrt{n}\tilde{\Pi}_{g, n}(h)$", size=16)
plt.show()
```

Nous sommes enfin convaincus que l'algorithme proposé réduit la variance.