**TODO**:
- fix pyspice to do not only R sweeps but also other parameter sweeps. The issue appears to be in the `plot` module that read the return from ngspice server results.
    - need to experiment with ngspice server, since I have already confirmed R sweep just using ngspice alone
    - run simulation from pyspice step by step and grab results without `plot`
    - fix `plot`

- use what we have done
- find max power and efficincy with the SPICE
- add usage of standard resistor values from sweep

```python
#Library import statements

from skidl.pyspice import *
#can you say cheeky
import PySpice as pspice
#becouse it's written by a kiwi you know
import lcapy as kiwi

import sympy as sym
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import warnings

from IPython.display import YouTubeVideo, display

import traceback
```

    WARNING: KICAD_SYMBOL_DIR environment variable is missing, so the default KiCad symbol libraries won't be searched.

#notebook specific loading control statements
%matplotlib inline
#tool to log notebook internals
#https://github.com/jrjohansson/version_information
%load_ext version_information
%version_information skidl, PySpice,lcapy, sympy, numpy, matplotlib, pandas, scipy

# Maximum Power Transfer Theorem

The Maximum Power transfer theorem for DC states that when the Load is equal to the Thevenin resistance of the circuit($R_{L}=R_{th}$) the power delivered to the load is maximized which is proven via taking the derivative of $$P=i^2 R_L=(\dfrac{V_{th}}{R_{th}+R_L})^2 R_L$$ and setting it to zero and solving. The details of which are given by ALL ABOUT ELECTRONICS in his YT video [Maximum Power Transfer Theorem for DC Circuits (with Examples)](https://www.youtube.com/watch?v=RbII8o49Hvs). But suffice to say we have already done the work to find this in the last two sections of this chapter by finding the Thevenin and Norton values of the circuit or when permitted using the .tf to find the DC transfer function which also gives us the input resistance. But here we want to show how to just sweep the load resistance itself (or any resistor in the circuit) and then find the peak in the resulting data that will give us the maximum power while also measuring the efficiency of the load where then we can find the ideal optimal load by finding the intersection of the two curves. We can do this by again utilizing the machinery we have developed to easily find the Thevenin and Norton equivalent circuits for circuits with multiple sources. Further, we want to then translate that semi-arbitrary value to a value we can use in the real world where the values you can get for a resistor are finite. So, let’s start with a theoretical model and then work with Examples 2 and 3 from ALL ABOUT ELECTRONICS video and develop a tool to automatically do all this for us.

## Max Power Delivered vs Most Efficient Power Delivered

```python
efficiency, powerload, powersource, current, voltage, Rth, RL =sym.symbols(r"\eta, P_L, P_S, i, v, R_{th}, R_L")
efficiency, powerload, powersource, current, voltage, Rth, RL
```

For a DC Thevenin reduced circuit (one that contains only a Thevenin voltage source, Thevenin equivalent resistor, and equivalent load resistor) we know that the current will be

```python
ithev_eq=sym.Eq(current, voltage/(Rth+RL)); ithev_eq
```

and the power in the load will be

```python
powerload_eq=sym.Eq(powerload, current**2 * RL); powerload_eq
```

and thus the power the load receives reduces to

```python
powerload_eq=powerload_eq.subs({ithev_eq.lhs: ithev_eq.rhs}); powerload_eq
```

Using the first example from [Maximum Power Transfer Theorem for DC Circuits (with Examples)](https://www.youtube.com/watch?v=RbII8o49Hvs) we know and can find the Thevenin equivalent easily, and thus the max power that the source can supply. And so if we then plot the values of the power of the load vs the resistance of the load we get

```python
subs={Rth:8,  voltage:32}
```

```python
powerload_lam=sym.lambdify(RL, powerload_eq.rhs.subs(subs))

RL_sweep=np.linspace(0, 100)
plt.plot(RL_sweep, powerload_lam(RL_sweep), label=f'${powerload}$ vs ${Rth}$')
plt.plot(8, powerload_lam(subs[Rth]), 'ro', label=f'Max Load Power {powerload_lam(subs[Rth])} [watts] @ {subs[Rth]} [Ohms]')
plt.xlabel(f"${RL}$[Ohms]"); plt.ylabel(f"${powerload}$[Watts]")
plt.legend()
plt.grid()
plt.title('Load Power vs Load Resistance');
```

But this is not our efficient load since our Thevenin equivalent source of the voltage and current source given by this example can deliver a total power of 128 [watts] when the load is short-circuited. So, then what is the most efficient load. While efficiency is defined as. Where we are looking at the global sources and global loads to our circuit which we have made easy in this example by looking only at a Thevenin reduced DC circuit

```python
efficiency_eq=sym.Eq(efficiency, powerload/powersource); efficiency_eq
```

the power from the source with a completed Thevenin circuit with load is the sum of the power in the Thevenin and load resistances

```python
powersource_eq=sym.Eq(powersource, current**2 *Rth +current**2 *RL); powersource_eq
```

Substituting the expression for the current in the Thevenin circuit we have

```python
powersource_eq=sym.simplify(powersource_eq.subs({ithev_eq.lhs: ithev_eq.rhs})); powersource_eq
```

plotting this as a function of the load we can see how the load affects the power supplied on the Thevenin equivalent source

```python
powersource_lam=sym.lambdify(RL, powersource_eq.rhs.subs(subs))

plt.plot(RL_sweep, powersource_lam(RL_sweep), label=f'${powerload}$ vs ${Rth}$')
plt.plot(8, powersource_lam(subs[Rth]), 'ro', label=f'Source Power {powersource_lam(subs[Rth])} [watts] @ ${Rth}$ ({subs[Rth]} [Ohms])')
plt.xlabel(f"${RL}$[Ohms]"); plt.ylabel(f"${powerload}$[Watts]")
plt.legend()
plt.grid()
plt.title('Source Power vs Load Resistance');
```

so then we find for the case of a DC Thevenin circuit our efficiency is

```python
efficiency_eq=efficiency_eq.subs({powersource_eq.lhs:powersource_eq.rhs, powerload_eq.lhs:powerload_eq.rhs  }); efficiency_eq
```

So then we can compare the efficiency (which is only defined between 0 and 1) to the power transferred to the load normalized to the maximum power at the Thevenin resistance as follows

```python
powerload_norm_lam=sym.lambdify(RL, powerload_eq.rhs.subs(subs)/powerload_eq.rhs.subs(subs).subs({RL:subs[Rth]}))

efficiency_lam=sym.lambdify(RL, efficiency_eq.rhs.subs(subs))

plt.plot(RL_sweep, powerload_norm_lam(RL_sweep), label=f'Normalized ${powerload}$ vs ${RL}$')
plt.plot(subs[Rth], powerload_norm_lam(subs[Rth]), 'ro', label=f'Max Load Power {powerload_lam(subs[Rth])} [watts] @ {subs[Rth]} [Ohms]')

plt.plot(RL_sweep, efficiency_lam(RL_sweep), label=f'${efficiency}$ vs ${RL}$')

plt.plot(subs[Rth], efficiency_lam(subs[Rth]), 'r*',
         label=f'${efficiency}={efficiency_lam(subs[Rth])}$ at $\max[{powerload}]$')

plt.xlabel(f"${RL}$[Ohms]");
plt.ylabel(r'$\dfrac{'+f"{powerload}"+'}{\max['+f"{powerload}"+"]}$")

plt.legend()
plt.grid()
plt.title(f'Maxium Load Power Transfer & Efficiency vs ${RL}$');
```

So then if we need to optimize the DC maximum power to the load, we know it to be the load resistance equal the Thevenin resistance; but if we could sacrifice the maximum power to the load in order max out the efficiency of our total circuit while minimizing the loss of power delivered to load. We then can then find said load via the intersection of the two curves above thusly

```python
#%%writefile -a DC_1_Codes.py
#chapter 1 section 6 findIntersection function
#Assist function to find the intersection of two functions

#from https://glowingpython.blogspot.com/2011/05/hot-to-find-intersection-of-two.html
#load fslove from scipy's optimize module
from scipy.optimize import fsolve

#helper function to find the intersection of two functions with an initial guess
def findIntersection(fun1,fun2,x0):
    """
    Aid function to find the intersection point of two curves
    from: https://glowingpython.blogspot.com/2011/05/hot-to-find-intersection-of-two.html

    Args:
        func1(function or class): the first function whose curve is
            used to find the intersection of the two curves

        func2(function or class): the second function whose curve is
            used to find the intersection of the two curves

        x0 (float); initial guess of the intersection of the two functions

    Returns:
        Returns array of float that are the intersections of the two functions,
        this is not very robust and thus one should read `fsolve`'s documentation
        for caveats  of usage
    """
    return fsolve(lambda x : fun1(x) - fun2(x),x0)
```

```python
#find the find the intersection and round value to three digits and; get just the first intersection
optimal_point=np.around(findIntersection(powerload_norm_lam, efficiency_lam, subs[Rth]), 3)[0]
optimal_point
```

```python
plt.plot(RL_sweep, powerload_norm_lam(RL_sweep), label=f'${powerload}$ vs ${RL}$')
plt.plot(subs[Rth], powerload_norm_lam(subs[Rth]), 'ro', label=f'Max ${powerload}$ {powerload_lam(subs[Rth])} [watts] @ {subs[Rth]} [Ohms]')

plt.plot(RL_sweep, efficiency_lam(RL_sweep), label=f'${efficiency}$ vs ${RL}$')
plt.plot(optimal_point, efficiency_lam(optimal_point), 'go',
         label=f'Mutualy Optimal Point @${RL}={optimal_point}$[Ohm]; ${efficiency}={efficiency_lam(optimal_point)}$; ${powerload}={powerload_lam(optimal_point)}$ [watts]')

plt.xlabel(f"${RL}$[Ohms]")
plt.ylabel(r'$\dfrac{'+f"{powerload}"+'}{\max['+f"{powerload}"+"]}$")

plt.legend()
plt.grid()
plt.title(f'Maximum Load Power Transfer & Efficiency vs ${RL}$ with cross optimization');
```

Thus, by moving to the intersection of the Power to the load and the efficiency of the circuit at the load from the Maximum power to the load point. We have a reduction of the amount of power delivered to the load of

```python
powerload_at_mpow=powerload_lam(subs[Rth])
powerload_at_mp=powerload_lam(optimal_point)
f"{powerload_at_mpow}[watts] to {powerload_at_mp}[watts]; a {powerload_at_mp-powerload_at_mpow}[watt] change"
```

    '32.0[watts] to 24.0[watts]; a -8.0[watt] change'

And have gained an efficiency boost of

```python
efficiency_at_mpow=efficiency_lam(subs[Rth])
efficiency_at_mp=efficiency_lam(optimal_point)
f"{efficiency_at_mpow*100}% to {efficiency_at_mp*100}%; a {(efficiency_at_mp-efficiency_at_mpow)*100}% gain"
```

    '50.0% to 75.0%; a 25.0% gain'

So now the task at hand is to move beyond this ideal theoretical example to having SPICE find the values we need and then building a testbench class in python that will do the data analysis along with all the SPICE work we just showed automatically for nearly any DC circuit. Where AC circuits and circuits that rely on Transient circuit effects (think switch-mode power supplies) will have to be analyzed separately when we cross those bridges.

## Cannot move on due to the issue in pyspice that prevents sweeping anything but Current and Voltage Sources; see TODO above

## Example 2 from "Maximum Power Transfer Theorem for DC Circuits (with Examples)" @ ~4:47 min

```python
YouTubeVideo('RbII8o49Hvs', width=500, height=400, start=287)
```

```python
reset()
net_1=Net('N1'); net_2=Net('N2'); net_3=Net('N3'); net_4=Net('N4')

#voltage source bottom left
vs=V(dc_value=100@u_V); vs['p', 'n']+=net_1, gnd
#restors on the center leg
rleft=R(ref='left', value=4@u_Ohm); rleft[1, 2]+=net_1, net_2
rright=R(ref='right', value=4@u_Ohm); rright[1, 2]+=net_2, net_3

#vcvs and resistor on top leg
vcvs=E(voltage_gain=1)
#vcvs inputs; outputs
vcvs['ip', 'in']+=net_2, net_1; vcvs['op', 'on']+=net_1, net_4
rtop=R(ref='top', value=4@u_Ohm); rtop[1, 2]+=net_4, net_3

#load with dummy resistance
rload=R(ref='load', value=1@u_Ohm); rload[1, 2]+=net_3, gnd

circ=generate_netlist()
print(circ)
```

    .title
    V1 N1 0 100V
    Rleft N1 N2 4Ohm
    Rright N2 N3 4Ohm
    E1 N1 N4 N2 N1 1
    Rtop N4 N3 4Ohm
    Rload N3 0 1Ohm

    No errors or warnings found during netlist generation.

## Example 3 from "Maximum Power Transfer Theorem for DC Circuits (with Examples)" @ ~8:46 min

```python
YouTubeVideo('RbII8o49Hvs', width=500, height=400, start=526)
```

```python
reset()
net_1=Net('N1'); net_2=Net('N2'); net_3=Net('N3')

vs=V(dc_value=10@u_V); vs['p', 'n']+=net_1, gnd

#center T network
rleft=R(ref='left', value=30@u_Ohm); rleft[1, 2]+=net_1, net_2
rcenter=R(ref='center', value=30@u_Ohm); rcenter[1, 2]+=net_2, gnd
rright=R(ref='right', value=30@u_Ohm); rright[1, 2]+=net_2, net_3

#rvar with dummy resistance
rtest=R(ref='test', value=1@u_Ohm); rtest[1, 2]+=net_1, net_3

rload=R(ref='load', value=10@u_Ohm); rload[1, 2]+=net_3, gnd

circ=generate_netlist()
print(circ)
```

    .title
    V1 N1 0 10V
    Rleft N1 N2 30Ohm
    Rcenter N2 0 30Ohm
    Rright N2 N3 30Ohm
    Rtest N1 N3 1Ohm
    Rload N3 0 10Ohm

    No errors or warnings found during netlist generation.

## Citations:
[1] ALL ABOUT ELECTRONICS. "Maximum Power Transfer Theorem for DC Circuits (with Examples)," YouTube, May 20, 2017. [Video file]. Available: https://youtu.be/RbII8o49Hvs. [Accessed: Nov 30, 2020].

[2] @JustGlowing, “How to find the intersection of two functions,” The Glowing Python, 10-May-2011. [Online]. Available: https://glowingpython.blogspot.com/2011/05/hot-to-find-intersection-of-two.html. [Accessed: 20-Nov-2020].

```python

```


# Experiment 2: Airtrack I---Kinematics
### Objectives
- To learn good lab practice
- To learn how to estimate and propagate uncertainties
- To learn how position, velocity, and acceleration are related
- To learn how to use the Airtrack
- To learn how to use PASCO Capstone Data Acquisition and Analysis software

### Equipment
- One Airtrack, Blower, Cart, and Acoustic Sensor
- One wooden reflector
- One 2x4 “shim”
- One 30 cm plastic ruler
- One two-meter stick

### Safety
- **Do not attempt to push the carts faster than 1 m/s.**
- **Be careful placing the carts on the track.**
- **Do not damage the track.** Please place paper underneath the carts when they are resting on the track without the air turned on.
- **Do not slide the carts on the track without the air turned on.**

# Introduction

Before one can study the action of forces on objects, one must be able to describe motion. This description of motion is called $kinematics$. We will be learning how to use two tools to investigate the meanings of the three kinematic equations for motion in one dimension:
The definition of average velocity:

\begin{equation}
v_{avg} \equiv \frac{\Delta x}{\Delta t}
\label{eq:v_avg}
\end{equation}

The definition of average acceleration:

\begin{equation}
a_{avg} \equiv \frac{\Delta v}{\Delta t}
\label{eq:a_avg}
\end{equation}

And the position of an object undergoing constant acceleration:

\begin{equation}
x(t) = \frac{1}{2} a t^2 + v_o t + x_o
\label{eq:x_of_t}
\end{equation}

In this lab, your job will be to compare different measurements of velocity and acceleration for an air-cart experiencing both no acceleration and constant acceleration.

# Theory

### Measurement of Velocity and Acceleration
An acoustic sensor will measure the position of a cart on the airtrack at specific intervals of time. By use of the definition of velocity (Equation $\eqref{eq:v_avg}$), we can compute the average velocity between two measurements. Notice that if we plot the position versus time, the fraction is the same as the rise divided by the run-- or the slope of the graph. By plotting *x* vs. *t*, we then find the slope of the graph and determine the velocity.

This same technique works for determining acceleration. However, notice that the variables to plot are now *velocity* versus time.

### Analysis of Uncertainties
It will be important to know what the uncertainties are in our computed velocities and accelerations.

First, recall that a difference in position or time is a subtraction of two quantities:
\begin{equation}
\Delta x = x_2 - x_1 \nonumber
\end{equation}

Therefore, we can use (R1) to compute the uncertainty in the difference:
\begin{equation}
(\delta(\Delta x))^2 = (\delta x_2)^2 - (\delta x_1)^2 \nonumber
\end{equation}

However, in this case, the uncertainties $\delta x_1$ and $\delta x_2$ should be the same, so we can say that the uncertainty in our computed difference is:
\begin{equation}
\delta(\Delta x) = \sqrt{2} (\delta x) \nonumber
\end{equation}

where $\delta x$ is the uncertainty in a single position measurement. We can say the same for time differences:
\begin{equation}
\delta(\Delta t) = \sqrt{2} (\delta t) \nonumber
\end{equation}

where $\delta t$ is the uncertainty in a single time measurement. Notice that the uncertainty in the difference is larger than the uncertainty in each individual measurement.

Computing the uncertainty in velocity, $\delta v$, is a straightforward application of (R2). Looking at Equation $\eqref{eq:v_avg}$ we have:
\begin{equation}
\left(\frac{\delta v}{v}\right)^2 = \left(\frac{\delta (\Delta x)}{\Delta x}\right)^2 + \left(\frac{\delta (\Delta t)}{\Delta t}\right)^2 \nonumber
\end{equation}

This equation needs some work to become useful. Capstone will take data points at constant $\Delta t$. But because $\Delta x$ may be different for each measurement, we should try to remove it from our equation. We do this by using the definition of velocity to replace $\Delta x$ with $v \Delta t$:
\begin{equation}
\left(\frac{\delta v}{v}\right)^2 = \left(\frac{\delta (\Delta x)}{v \Delta t}\right)^2 + \left(\frac{\delta (\Delta t)}{\Delta t}\right)^2 \nonumber
\end{equation}

Factoring out $\Delta t$ so that we need only use it once:
\begin{equation}
\left(\frac{\delta v}{v}\right)^2 = \frac{1}{(\Delta t)^2} \left(\left(\frac{\delta (\Delta x)}{v}\right)^2 + \left(\delta (\Delta t)\right)^2 \right) \nonumber
\end{equation}

We may also multiply through by $v^2$:
\begin{equation}
\left(\delta v\right)^2 = \frac{1}{(\Delta t)^2} \left(\left(\delta (\Delta x)\right)^2 + v^2 \left(\delta (\Delta t)\right)^2 \right)
\label{eq:v_unc}
\end{equation}

Now we have an equation to find our uncertainty in velocity based on things we know. Notice that the uncertainty in the velocity *increases* with the velocity. Also notice that the uncertainty is *not zero* when the velocity is zero.
Does this make sense?

# Experimental Procedure

### Leveling the Airtrack
In order to study unaccelerated motion, we must ensure that gravity does not interfere. To do this, we must level the airtrack. The best method for leveling the airtrack is to use the cart itself to verify that it is not pulled one way or the other by gravity.

1. Turn on the blower (you should be able to feel the air coming out of the drilled holes) and place a cart at the 100 cm mark.
2. Place your fingers at the front and rear of the cart so that it remains at rest.
3. Release the cart without pushing it.

If the cart tends to move to one side, you will need to raise that side (or lower the other side). This can be accomplished by rotating the feet on the two-foot support. If this is still not sufficient, you may need to place a folded sheet of paper under one of the two supports until the cart favors neither side. It’s OK if the cart slowly oscillates.

### Configuring the Acoustic Sensor
Place a wooden reflector into the center top hole in the cart. You will now need to place the sensor so that it properly reads the position of the reflector.

1. Place the sensor at the opposite end from where the air hose is placed.
2. Place a cart at that end of the airtrack and set the sensor at the same height as the center of the wooden reflector.
3. Ensure that the sensor is centered over the track and pointed straight down the track.
4. Ensure that the top switch is in “cart” mode (*not* “stick figure” mode).

There are two plugs which must be inserted into the PASCO 850 interface. The yellow plug goes into Digital Input 1 and the black plug goes into Digital Input 2.

### Using PASCO Capstone DAA
Capstone will be used throughout the physics laboratories to collect data, so you should spend some time now learning how to use it. To set it up for the first time, the PASCO 850 interface must be turned on (press and hold the power button until it beeps and the green light is lit) and connected to the computer.

1. Start Capstone.
2. Click “Hardware Setup” on the left-side “Tools” palette. A picture of the PASCO 850 interface will appear.
3. Click on the Digital Input 1 port and select “Motion Sensor II.”
4. Click on “Properties” in the lower left. You will need to enter the proper speed of sound. The speed of sound is (331+0.6T) m/s where T is the air temperature in Celcius. Click “OK.”
5. Click on “Hardware Setup” to close the window.
6. In the right-side palette, drag “Graph” to the main window. A large graph will appear.

At the very bottom of the window, notice the sample rate (in Hz). This number tells you how many data points are taken every second. You will need this number later.

When you click “Record,” Capstone will start taking data on all the sensors attached to the 850 interface. When you click “Stop,” Capstone will stop taking data. Every Record/Stop sequence is considered a “run.” All of the data runs you gather are listed in the “Data Summary” tab on the left-side palette. Clicking “Data Summary” a second time will close that window.

### Testing the Acoustic Sensor using Capstone
A very quick test of the sensor would be to click “Record” and move your hand towards and away from the sensor a few times and then click “Stop.” By clicking `<Select Measurement>` on the y-axis and selecting “Position (m),” Capstone will plot a graph of that data. If this works, then your sensor works and it’s time to test the alignment with your cart.

Turn on the air for your airtrack and place a cart on the track. Push the cart towards the sensor and click “Record.” Let the cart bounce back and forth between the bumpers a few times before clicking “Stop.” The new data should have replaced the old data on the graph. (If not, then there are two ways to have the new data plotted: (1) In the “Data Summary” tab, drag the desired run icon from under “Position” and drop it onto the graph. (2) The toolbar on the menu itself has a rainbow-colored triangle on it. Click the arrow next to that icon and select the data run you would like to plot.)

The data should look something like this:

### Troubleshooting
If your data does not look like this, you will need to work on the alignment of your sensor with the reflector. Check the following things:

1. The sensor should be at the same height as the center of the reflector.
2. The sensor should be pointed directly at the reflector.
3. The reflector should be fully facing the sensor.
4. The sensor must be centered on the track (look down over the sensor and make sure the center of the gold circle is over the ridge-line of the airtrack).
5. The sensor must be pointed directly down the track.
6. The switch on the sensor must be set to “Cart.”
7. The green light should blink while data is being taken.

We will be using the PASCO Capstone software frequently in all the laboratory courses, so it is important that you become familiar with this software now. Once your data looks pretty good, perform the following functions:

1. In the “Data Summary” tab, double-click on the data tag and rename it to “Calibration Run #1.”
2. Print your graph in landscape mode (File -> Print, then click the “Preferences” button).
3. Have your instructor sign off on your graph.
4. Grab the right side of your graph and make some space for another display.
5. Drag “Meter” or “Digits” or “Table” display into the open area, click on that display’s `<Select Measurement>` button and select a data source, then click Record/Stop.
6. Click each button on the graph to see what it does.
7. Learn how to pan and zoom your graph.
8. Drag the legend around the graph.
9. Delete all your graphs/tables (select the display by clicking on it, then click on the “Delete Selected Display” icon in the upper Capstone toolbar).
10. Delete every data run and start over by clicking the arrow next to “Delete Last Run” in the bottom tool palette and selecting “Delete All Runs.”

```python
# Sensor test (Sawtooth plot)

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from sklearn import linear_model

reg = linear_model.LinearRegression()

# Reads the name of the csv file and gets the data
df = pd.read_csv("./Demo files/Sawtooth test.csv")

# Prints information about the file
#df.info()

# Sets the size of the graph
fig = plt.figure(figsize=(7,5), dpi= 100)

# Defines the x and y values
time = df.filter(['Time (s)'], axis=1).dropna()
position = df.filter(['Position (m)'], axis=1).dropna()

# Creates the base plot with titles
df.plot(x='Time (s)', y='Position (m)', kind ='scatter')
plt.ylabel('Position (m)')
plt.title('Sawtooth Graph')

# Gets the data values for x and y
x_data = time.values.reshape(-1, 1)
y_data = position.values.reshape(-1, 1)

# Takes the x and y values to make a line
reg.fit(x_data,y_data)
y_fit = x_data*reg.coef_ + reg.intercept_

# Add a dotted line to simulate the sawtooth shape
plt.plot(x_data,y_data,'b--',label='Linear Fit')

# Adds the legend to the plot
plt.legend()

# Displays the plot
plt.show()

print("Slope = (slope) m/s")
print("Y-int = (y-int) m")
```

### Acoustic Sensor Uncertainty
The acoustic sensor can only measure distance so well. How well? You’ll need to determine that. The best way to do this is to take many distance measurements at a fixed location and find the standard deviation of those measurements. Here is one method of doing this:

Turn the air off so that the cart doesn’t move. At three locations on your airtrack (near the sensor, at the 100 cm mark, and at the far end from the sensor):
1. $Carefully$ place the cart at its location on a sheet of paper and press “Record.”
2. After 30 seconds, press “Stop.”
3. Drag a “Table” from the right tool palette onto the main window.
4. In one column, click the `<Select Measurement>` button and select “Time (s).”
5. In the other column, click the `<Select Measurement>` button and select “Position (m).”
6. The cell above should show your last data run. If not, click it and select your last data run.
7. On the table, click the arrow next to the summation sign ($\Sigma$) and make sure “Mean” and “Standard Deviation” are selected. Then click the summation sign ($\Sigma$) itself.
8. Read the standard deviation off the table at the bottom. This is your $\delta x$.

If your standard deviation is greater than 0.001 m (1.0 mm) then you should try to fine-tune the alignment of your sensor. It is very possible to get less than one millimeter uncertainty at the far end of the track. By looking at the
displayed significant digits, your table also tells you what the uncertainty is for a single time measurement. Once you are done aligning your set-up, record your new worst (highest) position uncertainty (this will probably be at
your far position) as well as your uncertainty in time, and the uncertainties for any differences the computer would calculate between two adjacent points.

```python
# Raw Data 1

import pandas as pd
import numpy as np
import ipywidgets as widgets

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

uncertainty_x = 0.001
uncertainty_t = 0.002

uncertainty_Delta_x = 0.001
uncertainty_Delta_t = 0.003

###########################################

print("𝛿x =", uncertainty_x, " 𝛿t =", uncertainty_t)
print("")
print("𝛿(Δx) =", uncertainty_Delta_x, " 𝛿(Δt) =", uncertainty_Delta_t)
print("")
```

    𝛿x = 0.001  𝛿t = 0.002

    𝛿(Δx) = 0.001  𝛿(Δt) = 0.003

What is the difference in time between two $successive$ data points? (You can either look at your table or recall the Sample Rate from above).

```python
# Raw Data 2

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

Delta_t = 0.005

###########################################

print("Δt =", Delta_t)
```

    Δt = 0.005

### Unaccelerated Motion
This first part of the experiment is to compare the position of the cart as a function of time with its velocity and verify equation $\eqref{eq:v_avg}$:

1. Start fresh: delete ALL data sets and close all data windows.
2. Drag a “Graph” from the right tool palette to your data area. Select “Position (m)” for your y-axis.
3. Turn on the air and remove the paper from under the cart.
4. With your cart at the far end of the track, give it a good push towards the sensor.
5. Immediately after the cart bounces off the near bumper, press “Record.”
6. Immediately BEFORE the cart strikes the far bumper, press “Stop.”
7. Turn off the air.
8. Rename this run “Level Airtrack”.
9. Right-click on one of the data points and, in the window that pops up, un-check “Show Connecting Lines.”
10. Move the legend so that it does not cover any data points.
11. Type an appropriate title for this graph and print it in Landscape mode. You will need to make the grid lines show up clearly in your graph. To do this, right click on the plot and select “Plot Area Properties.”
12. Click on the “Position (m)” axis label and select “Velocity (m/s)” instead.
13. Repeat steps 9—11.

It is now up to you to draw the best-fit line for the Position vs. Time graph by hand and find its slope as well as draw the best-fit ***horizontal*** line for the Velocity vs. Time graph and find its height. Use the methods outlined in your Lab Manual for this.

```python
# Position vs. Time Graph

# Reads the name of the csv file and gets the data
df = pd.read_csv("./Demo files/Position vs Time test.csv")

# Prints information about the file
#df.info()

print("")

# Defines the x and y values
time = df.filter(['Time (s)'], axis=1).dropna()
position = df.filter(['Position (m)'], axis=1).dropna()

# Creates the base plot with titles
df.plot(x='Time (s)', y='Position (m)', kind ='scatter')
plt.ylabel('Position (m)')
plt.title('Position vs. Time')

# Gets the data values for x and y
x_data = time.values.reshape(-1, 1)
y_data = position.values.reshape(-1, 1)

# Takes the x and y values to make a trendline
reg.fit(x_data,y_data)
y_fit = x_data*reg.coef_ + reg.intercept_

# Adds the trendline to the plot
plt.plot(x_data,y_fit,'r--',label='Linear Fit')

# Adds the legend to the plot
plt.legend()

# Displays the plot
plt.show()

print("Slope = (slope) m/s")
print("Y-int = (y-int) m")
```

```python
# Velocity vs. Time Graph (unaccelerated)

# Reads the name of the csv file and gets the data
df = pd.read_csv("./Demo files/Unacc_Velocity  vs Time test.csv")

# Prints information about the file
#df.info()

# Sets the size of the graph
fig = plt.figure(figsize=(7,5), dpi= 100, facecolor='w', edgecolor='k')

# Defines the x and y values
time = df.filter(['Time (s)'], axis=1).dropna()
position = df.filter(['Velocity (m/s)'], axis=1).dropna()

# Creates the base plot with titles
df.plot(x='Time (s)', y='Velocity (m/s)', kind ='scatter')
plt.ylabel('Velocity (m/s)')
plt.title('Velocity  vs. Time')

# Gets the data values for x and y
x_data = time.values.reshape(-1, 1)
y_data = position.values.reshape(-1, 1)

# Takes the x and y values to make a trendline
reg.fit(x_data,y_data)
y_fit = x_data*reg.coef_ + reg.intercept_

# Adds the trendline to the plot
plt.plot(x_data,y_fit,'r--',label='Linear Fit')

# Adds the legend to the plot
plt.legend()

# Displays the plot
plt.show()

print("Slope = (slope) m/s")
print("Y-int = (y-int) m")
print("")
```

For the slope of your Position vs. Time graph (these values represent your uncertainty in **reading** the graph---i.e. how well can you locate a point on the graph “by eye?”):

```python
# Raw Data 3

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

uncertainty_x = 0.001
uncertainty_t = 0.002

uncertainty_Delta_x = 0.001
uncertainty_Delta_t = 0.003

###########################################

print("𝛿x =", uncertainty_x, " 𝛿t =", uncertainty_t)
print("")
print("𝛿(Δx) =", uncertainty_Delta_x, " 𝛿(Δt) =", uncertainty_Delta_t)
```

    𝛿x = 0.001  𝛿t = 0.002

    𝛿(Δx) = 0.001  𝛿(Δt) = 0.003

### Accelerated Motion
Now that you’ve gotten some data to investigate unaccelerated motion, you need to collect data on accelerated motion. To do this, you will need to allow gravity to provide the acceleration by tilting the track.

1. Measure the distance between the supports on the airtrack. This will be your base length.
2. Measure the thickness of a 2x4 “shim.” This will be your height change.
3. Place the 2x4 “shim” under the foot nearest the acoustic sensor.
4. You will need to re-align the acoustic sensor using the methods described earlier.
5. Re-measure the uncertainty in position at the end of the track furthest from the sensor and try to reduce that uncertainty below 1 mm. Record this uncertainty in your Data Sheet.

When you are ready to take data:

1. Remove everything except one graph from the data area and make sure the y-axis is set to “Acceleration ($m / s^2$).”
2. Turn on the air, remove the paper, and allow the cart to slide down the airtrack and bounce off the far bumper.
3. **START** taking data ***immediately after*** the cart strikes the bumper.
4. **STOP** taking data ***just before*** the cart strikes the bumper again.
5. Turn off the air.
6. Rename this data set as “Tilted Airtrack.”

This time, let’s allow the Capstone software to analyze our data:

1. Click the arrow beside the summation symbol ($\Sigma$) and make sure that “Mean” and “Standard Deviation” are selected. Then click the summation symbol ($\Sigma$).
2. Move the legend so that it does not obscure the data points.
3. Type in an appropriate title and print this graph in Landscape Mode.
4. Click the summation symbol ($\Sigma$) again to remove the fit.
5. Click the axis button labeled “Acceleration ($m / s^2$)” and select “Velocity (m/s)”
6. Click the arrow next to to the “Fit” button and select “Linear.” Then click the “Fit” button.
7. Move the legend and linear fit windows so that they do not obscure the data points.
8. Type in an appropriate title and print this graph in Landscape Mode.
9. Click the “Fit” button to remove the fit.
10. Click the axis button labeled “Velocity (m/s)” and select “Position (m).”
11. Click the arrow next to to the “Fit” button, unselect “Linear” and select “Quadratic.” Then click the “Fit” button.
12. Move the legend and quadratic fit windows so that they do not obscure the data points.
13. Type in an appropriate title and print this graph in Landscape Mode.

```python
# Acceleration vs Time Graph (Tilted Airtrack)

# Reads the name of the csv file and gets the data
df = pd.read_csv("./Demo files/Acceleration vs Time test.csv")

# Prints information about the file
#df.info()

print("")

# Defines the x and y values
time = df.filter(['Time (s)'], axis=1).dropna()
position = df.filter(['Acceleration (m/s^2)'], axis=1).dropna()

# Creates the base plot with titles
df.plot(x='Time (s)', y='Acceleration (m/s^2)', kind ='scatter')
plt.ylabel('Acceleration (m/s^2)')
plt.title('Acceleration vs Time')

# Gets the data values for x and y
x_data = time.values.reshape(-1, 1)
y_data = position.values.reshape(-1, 1)

# Takes the x and y values to make a line
reg.fit(x_data,y_data)
y_fit = x_data*reg.coef_ + reg.intercept_

# Add a dotted line to simulate the sawtooth shape
plt.plot(x_data,y_data,'b--',label='Linear Fit')

# Adds the legend to the plot
plt.legend()

# Displays the plot
plt.show()

print("Slope = (slope) m/s")
print("Y-int = (y-int) m")
print("")
```

```python
# Position vs. Time Graph (accelerated)

# Reads the name of the csv file and gets the data
df = pd.read_csv("./Demo files/Position vs Time test.csv")

# Prints information about the file
#df.info()

print("")

# Defines the x and y values
time = df.filter(['Time (s)'], axis=1).dropna()
position = df.filter(['Position (m)'], axis=1).dropna()

# Creates the base plot with titles
df.plot(x='Time (s)', y='Position (m)', kind ='scatter')
plt.ylabel('Position (m)')
plt.title('Position vs. Time')

# Gets the data values for x and y
x_data = time.values.reshape(-1, 1)
y_data = position.values.reshape(-1, 1)

# Takes the x and y values to make a trendline
reg.fit(x_data,y_data)
y_fit = x_data*reg.coef_ + reg.intercept_

# Adds the trendline to the plot
plt.plot(x_data,y_fit,'r--',label='Linear Fit')

# Adds the legend to the plot
plt.legend()

# Displays the plot
plt.show()

print("Slope = (slope) m/s")
print("Y-int = (y-int) m")
print("")
```

```python
# Velocity vs. Time Graph (Accelerated Motion)

# Reads the name of the csv file and gets the data
df = pd.read_csv("./Demo files/Velocity vs Time test.csv")

# Prints information about the file
#df.info()

print("")

# Defines the x and y values
time = df.filter(['Time (s)'], axis=1).dropna()
velocity = df.filter(['Velocity (m/s)'], axis=1).dropna()

# Creates the base plot with titles
df.plot(x='Time (s)', y='Velocity (m/s)', kind ='scatter')
plt.ylabel('Velocity (m/s)')
plt.title('Velocity vs. Time')

# Gets the data values for x and y
#x_data = time.values.reshape(-1, 1)
#y_data = position.values.reshape(-1, 1)

# Takes the x and y values to make a trendline
#reg.fit(x_data,y_data)
#y_fit = x_data*reg.coef_ + reg.intercept_

# Adds the trendline to the plot
#plt.plot(x_data,y_fit,'r--',label='Linear Fit')

# Adds the legend to the plot
#plt.legend()

# Displays the plot
plt.show()

print("Slope = (slope) m/s")
print("Y-int = (y-int) m")
print("")
```

```python
# Raw Data 4

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

# The distance between the two supports and the uncertainty in that distance
d = 25.0
uncertainty_d = 0.002

# The thickness of the 2x4 and the uncertainty in that thickness
h = 1.0
uncertainty_h = 0.003

# Your new uncertainty in cart position
uncertainty_x = 0.005

# The angle your track was tilted
theta = 25.0

###########################################

print("d =", d, " 𝛿d =", uncertainty_d)
print("")
print("h =", h, " 𝛿h =", uncertainty_h)
print("")
print("𝛿x =", uncertainty_x)
print("")
print("θ =", theta)
```

    d = 25.0  𝛿d = 0.002

    h = 1.0  𝛿h = 0.003

    𝛿x = 0.005

    θ = 25.0

***Print out all six graphs.***

# Your Lab Report

## EXPERIMENTAL PROCEDURE Section
You will need to write a few paragraphs to carefully describe how you set up your experiment. Include the following specific items:
- What did you do to ensure that your zero-acceleration and constant-acceleration experiments were free from unwanted sources of acceleration?
- How did you configure your measurement equipment so that you got the most accuracy?
- How did you compute the best-line fits and slopes for your ***x*** **vs.** ***t*** and ***v*** **vs.** ***t*** graphs?
- How did you set the track and determine the angle for the constant-acceleration experiment?
- How did you re-calibrate your measurement equipment after changing experiments?

**Remember:** This section is a description of what you did---not a set of instructions. Include a description of any problems you encountered and how you solved them.

## Data
Report the following data (with units and uncertainties):

- Velocity of cart without acceleration
- Angle that you tilted the airtrack
- Average Acceleration from the Acceleration vs. Time graph
- Average Acceleration calculated form the slope of the Velocity vs. Time graph
- Average Acceleration computed from the parabolic fit to your Position vs. Time graph

Include all six of your graphs in order: Sawtooth, two unaccelerated motion graphs, and three accelerated motion graphs.

## Results and Conclusions
**Unaccelerated Motion Experiment**
- Why does the time between peaks in the saw-tooth pattern increase? Will this affect your results?
- Compare the slope of the Position vs. Time graph to the average value of the velocities in the velocity vs. time graph. Use the % difference formula and cite your uncertainties. Do the two numbers overlap if you include your uncertainties?
- Using equation $\eqref{eq:v_unc}$, compute the uncertainty in the speed of the cart at the speed you measured.

**Accelerated Motion Experiment**
Create the following table in your report and fill in the entries:

```python
# Table 1: Percent Differences Between Different Methods of Computing Acceleration

# Create an empty numpy array to hold the raw data
raw_data_1 = np.empty((4,5), dtype = object)

# Set the column identifiers
raw_data_1[0][0]= 1
raw_data_1[1][0]= 2
raw_data_1[2][0]= 3
raw_data_1[3][0]= 4

# Create a Pandas dataframe, and convert the cylinder number column to integer format
df1 = pd.DataFrame(raw_data_1, columns=[" ",
                                        "Average Acceleration from Acceleration vs. Time graph",
                                        "Acceleration Calculated from Slope of Velocity vs. Time Graph",
                                        "Acceleration Computed from Parabolic Fit of Position vs. Time Graph",
                                       "The value of (g sin θ )"])

df1[' '] = ['Av. Acceleration from a vs. t graph',
                   'Acceleration from Slope of v vs. t Graph',
                   'Accelerationfrom Parabolic Fit of m vs. t Graph.',
                   'The value of (g sin θ )']

#### Enter Raw Data Here!!!!!!!!!!!!!! ####

# Replace the 0's with your percent differences for the table
df1['Average Acceleration from Acceleration vs. Time graph'] = ['N/A',0,0, 0]

df1['Acceleration Calculated from Slope of Velocity vs. Time Graph'] = [0,'N/A',0, 0]

df1['Acceleration Computed from Parabolic Fit of Position vs. Time Graph'] = [0,0,'N/A', 0]

df1['The value of (g sin θ )'] = [0,0,0, 'N/A']

###########################################

display(df1)
print ("Table 1: Percent Differences Between Different Methods of Computing Acceleration")
print("")
```

      Average Acceleration from Acceleration vs. Time graph
      Acceleration Calculated from Slope of Velocity vs. Time Graph
      Acceleration Computed from Parabolic Fit of Position vs. Time Graph
      The value of (g sin θ )

      0
      Av. Acceleration from a vs. t graph
      N/A
      0
      0
      0

      1
      Acceleration from Slope of v vs. t Graph
      0
      N/A
      0
      0

      2
      Accelerationfrom Parabolic Fit of m vs. t Graph.
      0
      0
      N/A
      0

      3
      The value of (g sin θ )
      0
      0
      0
      N/A

    Table 1: Percent Differences Between Different Methods of Computing Acceleration

To compare the “A” value to the acceleration, you will need to know that the A, B, and C values are the constants for the following fit to the data:

\begin{equation}
x(t) = At^2 + Bt + C \nonumber
\end{equation}

Compare this equation to equation $\eqref{eq:x_of_t}$ to determine the acceleration, *a*.

**Errors/Uncertainties**
Did you have any systematic errors (i.e. were there sources of acceleration in either experiment that you could not remove)?

How could you reduce your errors/uncertainties?

## Appendix
- Include your signed raw data sheet
- Include sample calculations for your best-line fit and slope for your unaccelerated motion graphs as well as the angle your track was set to for the accelerated motion.


## Imports

We use the SymPy library to work with mathematical aspects of symbolic provenance polynomials, such as multiplication, addition, substitution, etc.

```python
# !pip install sympy
```

```python
# PW-explorer
from PW_explorer.load_worlds import load_worlds
from PW_explorer.run_clingo import run_clingo
from PW_explorer.export import PWEExport
from PW_explorer.visualize import PWEVisualization
from PWE_NB_Extension.helper import ASPRules
# Other utilities
from copy import deepcopy
import pandas as pd
import sympy
from functools import reduce
```

```python
# Graph-Visualization
import networkx as nx
from nxpd import draw
import nxpd
from nxpd import nxpdParams
nxpdParams['show'] = 'ipynb'
```

```python
%load_ext PWE_NB_Extension
```

## Helpers

```python
def simple_3hop_viz(pw_rel_dfs, gv_md, prov=False):
    g = nx.MultiDiGraph()

    g.graph['rankdir'] = 'LR'
    hop_edges_color = 'black'
    thop_edges_color = 'blue'
    thop_df_name = 'thop_3' if prov else 'thop_2'

    if thop_df_name in pw_rel_dfs:
        for i, row in pw_rel_dfs[thop_df_name].iterrows():
            g.add_edge(row['x1'], row['x2'], color=thop_edges_color, constraint='false')
        for i, row in pw_rel_dfs['hop_3'].iterrows():
            g.add_edge(row['head'], row['tail'], color=hop_edges_color, style='dotted')
    else:
        for i, row in pw_rel_dfs['hop_3'].iterrows():
            g.add_edge(row['head'], row['tail'], color=hop_edges_color, label=row['hop_name'])
    return g
```

## Provenance Polynomials

#### Consider a database D of named hops:

```python
%%clingo --donot-display_input -lci hop_db_a --donot-run

% schema hop(head, tail, hop_name)
hop(a,a, p).
hop(a,b, q).
hop(b,a, r).
hop(b,c, s).
```

We can visualize it easily as follows:

```python
hop_db_a_asp_out, hop_db_a_md = run_clingo(hop_db_a)
hop_db_a_pw_rel_dfs, _, _ = load_worlds(hop_db_a_asp_out, hop_db_a_md, silent=True)
draw(simple_3hop_viz(hop_db_a_pw_rel_dfs, hop_db_a_md['graphviz']))
```

#### Now consider the traditional 3hop query:

```python
%%clingo --donot-display_input -lci thop_query --donot-run

thop(X,Y) :- hop(X,Z1, P1), hop(Z1,Z2, P2), hop(Z2,Y, P3).
```

On our hop database D, this yields the following output:

```python
%clingo -l hop_db_a thop_query --donot-display_input -exp thop_D
```

    Output:

1 Answer: 1
2 hop(a,a,p) hop(a,b,q) hop(b,a,r) hop(b,c,s) thop(a,a) thop(b,a) thop(a,b) thop(b,b) thop(a,c) thop(b,c)
3 SATISFIABLE
4
5 Models       : 1
6 Calls        : 1
7 Time         : 0.001s (Solving: 0.00s 1st Model: 0.00s Unsat: 0.00s)
8 CPU Time     : 0.001s

```python
thop_D['pw_rel_dfs'], _, thop_D['pw_objs'] = load_worlds(thop_D['asp_soln'], thop_D['meta_data'], silent=True)
draw(simple_3hop_viz(thop_D['pw_rel_dfs'], thop_D['meta_data']['graphviz']))
```

As we can see, we get 6 3hop outputs: (a,a), (a,b), (a,c), (b,a), (b,b), (b,c)

#### Now consider the provenance-enhanced version of the same query:

i.e. we 'record' the hops that build each of the 3hop

```python
%%clingo --donot-display_input --donot-run -lci thop_prov_query

%schema thop(start, dest, prov)
thop(X,Y, f(P1,P2,P3)) :- hop(X,Z1, P1), hop(Z1,Z2, P2), hop(Z2,Y, P3).
```

Note that the 3hop/2 query is a projection of this query, i.e., we can get the outputs using:

thop(X,Y) :- thop(X,Y,\_).

On running it with our database D, we get the following result:

```python
%clingo -l hop_db_a thop_prov_query --donot-display_input -exp thop_prov_D
```

    Output:

1 Answer: 1
2 hop(a,a,p) hop(a,b,q) hop(b,a,r) hop(b,c,s) thop(a,a,f(p,p,p)) thop(b,a,f(r,p,p)) thop(a,a,f(q,r,p)) thop(a,b,f(p,p,q)) thop(b,b,f(r,p,q)) thop(a,b,f(q,r,q)) thop(a,a,f(p,q,r)) thop(b,a,f(r,q,r)) thop(a,c,f(p,q,s)) thop(b,c,f(r,q,s))
3 SATISFIABLE
4
5 Models       : 1
6 Calls        : 1
7 Time         : 0.001s (Solving: 0.00s 1st Model: 0.00s Unsat: 0.00s)
8 CPU Time     : 0.001s

```python
rel = 'thop_3'

thop_prov_D['pw_rel_dfs'], thop_prov_D['rel_schemas'], thop_prov_D['pw_objs'] \
= load_worlds(thop_prov_D['asp_soln'], thop_prov_D['meta_data'], internal_facts_as_string=False)
```

    Number of Models: 1

```python
thop_prov_D['pw_rel_dfs'][rel]
```

      pw
      start
      dest
      prov

      0
      1
      a
      a
      [f, p, p, p]

      1
      1
      b
      a
      [f, r, p, p]

      2
      1
      a
      a
      [f, q, r, p]

      3
      1
      a
      b
      [f, p, p, q]

      4
      1
      b
      b
      [f, r, p, q]

      5
      1
      a
      b
      [f, q, r, q]

      6
      1
      a
      a
      [f, p, q, r]

      7
      1
      b
      a
      [f, r, q, r]

      8
      1
      a
      c
      [f, p, q, s]

      9
      1
      b
      c
      [f, r, q, s]

```python
thop_prov_D['pw_rel_dfs']['hop_3']
```

      pw
      head
      tail
      hop_name

      0
      1
      a
      a
      p

      1
      1
      a
      b
      q

      2
      1
      b
      a
      r

      3
      1
      b
      c
      s

The provenance information captured here (in the 'prov' column) can help us create the __"provenance polynomials"__

```python
# Helper Functions

def prov_mono(row):
    provs = sympy.symbols(row['prov'][1:])
    expr = reduce(lambda x, y: x*y, provs)
    return expr

def list_to_freq_list(l):
    if len(l) <= 0:
        return []
    l_ = [[l[0],1]]
    for i in range(1, len(l)):
        if l[i] == l_[-1][0]:
            l_[-1][1] += 1
        else:
            l_.append([l[i],1])
    return l_

def prov_mono_ord(row):
    f_list = list_to_freq_list(row['prov'][1:])
    f_list_exprs = list(map(lambda p: '{}^{}'.format(p[0],p[1]) if p[1]>1 else '{}'.format(p[0]), f_list))
    expr = "${}$".format(" ".join(f_list_exprs))
    return expr
```

First, we create the provenance monomials:

```python
thop_prov_D['pw_rel_dfs'][rel]['prov_mono_ord'] = thop_prov_D['pw_rel_dfs'][rel].apply(prov_mono_ord, axis=1)
thop_prov_D['pw_rel_dfs'][rel]
```

      pw
      start
      dest
      prov
      prov_mono_ord

      0
      1
      a
      a
      [f, p, p, p]
      $p^3$

      1
      1
      b
      a
      [f, r, p, p]
      $r p^2$

      2
      1
      a
      a
      [f, q, r, p]
      $q r p$

      3
      1
      a
      b
      [f, p, p, q]
      $p^2 q$

      4
      1
      b
      b
      [f, r, p, q]
      $r p q$

      5
      1
      a
      b
      [f, q, r, q]
      $q r q$

      6
      1
      a
      a
      [f, p, q, r]
      $p q r$

      7
      1
      b
      a
      [f, r, q, r]
      $r q r$

      8
      1
      a
      c
      [f, p, q, s]
      $p q s$

      9
      1
      b
      c
      [f, r, q, s]
      $r q s$

Note that the above provenance monomials (prov_mono_ord) are 'ordered' i.e. we preserve the order of the hops that make up the 3hop. In traditional provenance polynomials, order is not preserved, as shown below:

```python
thop_prov_D['pw_rel_dfs'][rel]['prov_mono'] = thop_prov_D['pw_rel_dfs'][rel].apply(prov_mono, axis=1)
thop_prov_D['pw_rel_dfs'][rel]['prov_mono_'] = thop_prov_D['pw_rel_dfs'][rel].apply(lambda row: '${}$'.format(sympy.latex(row['prov_mono'])), axis=1)
thop_prov_D['pw_rel_dfs'][rel][['pw', 'start', 'dest', 'prov', 'prov_mono_ord', 'prov_mono_']]
```

      pw
      start
      dest
      prov
      prov_mono_ord
      prov_mono_

      0
      1
      a
      a
      [f, p, p, p]
      $p^3$
      $p^{3}$

      1
      1
      b
      a
      [f, r, p, p]
      $r p^2$
      $p^{2} r$

      2
      1
      a
      a
      [f, q, r, p]
      $q r p$
      $p q r$

      3
      1
      a
      b
      [f, p, p, q]
      $p^2 q$
      $p^{2} q$

      4
      1
      b
      b
      [f, r, p, q]
      $r p q$
      $p q r$

      5
      1
      a
      b
      [f, q, r, q]
      $q r q$
      $q^{2} r$

      6
      1
      a
      a
      [f, p, q, r]
      $p q r$
      $p q r$

      7
      1
      b
      a
      [f, r, q, r]
      $r q r$
      $q r^{2}$

      8
      1
      a
      c
      [f, p, q, s]
      $p q s$
      $p q s$

      9
      1
      b
      c
      [f, r, q, s]
      $r q s$
      $q r s$

Now, we can form the _provenance polynomials_ for each of the 3hop solutions by using a group-by (start-dest pairs) on the monomials and then using a simple summation aggregation function:

```python
thop_prov_D_prov_poly = thop_prov_D['pw_rel_dfs'][rel].groupby(['start', 'dest']).agg(
    prov_poly=('prov_mono', lambda x: '$'+sympy.latex(sum(x))+'$'))
thop_prov_D_prov_poly.style.set_properties(subset=['prov_poly'], **{'width': '100px'})
```

        	        	        prov_poly

        start	        dest

                        a
                        a
                        $p^{3} + 2 p q r$

                                b
                        $p^{2} q + q^{2} r$

                                c
                        $p q s$

                        b
                        a
                        $p^{2} r + q r^{2}$

                                b
                        $p q r$

                                c
                        $q r s$

#### The provenance polynomials can be used for various use-cases.

For instance, say we want to know what happens when we slightly modify the database D, say delete the hop 'p' (i.e. hop(a,a)).

One way to do it would be to modify the database D:

(let's call this modified database Db)

```python
%%clingo --donot-display_input -lci hop_db_b --donot-run

% schema hop(head, tail, hop_name)

% hop(a,a, p). --> DELETE
hop(a,b, q).
hop(b,a, r).
hop(b,c, s).
```

And re-run the query:

```python
%clingo -l hop_db_b thop_query --donot-display_input -exp thop_Db
```

    Output:

1 Answer: 1
2 hop(a,b,q) hop(b,a,r) hop(b,c,s) thop(a,b) thop(b,a) thop(b,c)
3 SATISFIABLE
4
5 Models       : 1
6 Calls        : 1
7 Time         : 0.001s (Solving: 0.00s 1st Model: 0.00s Unsat: 0.00s)
8 CPU Time     : 0.001s

As we can see, there are 3 outputs: 3hop(a,b), 3hop(b,a), 3hop(b,c).

Or, we could delete some of the results in Datalog using the previous result, using a procedure like shown below:

```python
%%clingo -lci thop_remove_p --donot-run --donot-display_input

del_thop(X,Y, f(P1,P2,P3)) :- thop(X,Y, f(P1,P2,P3)), P1=p.
del_thop(X,Y, f(P1,P2,P3)) :- thop(X,Y, f(P1,P2,P3)), P2=p.
del_thop(X,Y, f(P1,P2,P3)) :- thop(X,Y, f(P1,P2,P3)), P3=p.

new_thop(X,Y, F) :- thop(X,Y, F), not del_thop(X,Y, F).
#show new_thop/3.
```

Here, the thop/3 logical atoms come from materializing our earlier results from trop_prov_D experiments into a database of logical atoms, as shown below:

```python
thop_prov_D_facts = PWEExport.export_as_asp_facts(thop_prov_D['pw_objs'], include_pw_ids=False)
ASPRules(thop_prov_D_facts)
```

 1 hop(a,a,p).
 2 hop(a,b,q).
 3 hop(b,a,r).
 4 hop(b,c,s).
 5 thop(a,a,f(p,p,p)).
 6 thop(b,a,f(r,p,p)).
 7 thop(a,a,f(q,r,p)).
 8 thop(a,b,f(p,p,q)).
 9 thop(b,b,f(r,p,q)).
10 thop(a,b,f(q,r,q)).
11 thop(a,a,f(p,q,r)).
12 thop(b,a,f(r,q,r)).
13 thop(a,c,f(p,q,s)).
14 thop(b,c,f(r,q,s)).

In this way, we don't need to re-run the 3hop query itself, just do further computations on the already available results.

Now we run the 'thop_remove_p' rules along with the set of logical facts above, to get the desired results:

```python
%clingo -l thop_remove_p thop_prov_D_facts --donot-display_input
```

    Output:

1 Answer: 1
2 new_thop(a,b,f(q,r,q)) new_thop(b,a,f(r,q,r)) new_thop(b,c,f(r,q,s))
3 SATISFIABLE
4
5 Models       : 1
6 Calls        : 1
7 Time         : 0.002s (Solving: 0.00s 1st Model: 0.00s Unsat: 0.00s)
8 CPU Time     : 0.002s

As we can see, this also gives us the same 3 results. However, even this requires re-running some sort of querying the provenance information in Datalog.

#### Using Provenance Polynomials
We can get the desired result without ever re-running the query, by simply substituting p=0 in the provenance monomials and polynomials:

Recall that the provenance monomials on the original database D looked as follows:

```python
thop_prov_D['pw_rel_dfs'][rel][['pw', 'start', 'dest', 'prov_mono_']]
```

      pw
      start
      dest
      prov_mono_

      0
      1
      a
      a
      $p^{3}$

      1
      1
      b
      a
      $p^{2} r$

      2
      1
      a
      a
      $p q r$

      3
      1
      a
      b
      $p^{2} q$

      4
      1
      b
      b
      $p q r$

      5
      1
      a
      b
      $q^{2} r$

      6
      1
      a
      a
      $p q r$

      7
      1
      b
      a
      $q r^{2}$

      8
      1
      a
      c
      $p q s$

      9
      1
      b
      c
      $q r s$

Now, we simply substitute p=0 in these provenance monomials:

```python
thop_prov_Db = deepcopy(thop_prov_D)
substitutions = {'p':0}

thop_prov_Db['pw_rel_dfs'][rel]['prov_mono'] \
= thop_prov_Db['pw_rel_dfs'][rel].apply(lambda row: row['prov_mono'].subs(substitutions).evalf(), axis=1)

thop_prov_Db['pw_rel_dfs'][rel]['prov_mono_'] \
= thop_prov_Db['pw_rel_dfs'][rel].apply(lambda row: '${}$'.format(sympy.latex(row['prov_mono'])), axis=1)

thop_prov_Db['pw_rel_dfs'][rel][['pw', 'start', 'dest', 'prov_mono_']]
```

      pw
      start
      dest
      prov_mono_

      0
      1
      a
      a
      $0$

      1
      1
      b
      a
      $0$

      2
      1
      a
      a
      $0$

      3
      1
      a
      b
      $0$

      4
      1
      b
      b
      $0$

      5
      1
      a
      b
      $q^{2} r$

      6
      1
      a
      a
      $0$

      7
      1
      b
      a
      $q r^{2}$

      8
      1
      a
      c
      $0$

      9
      1
      b
      c
      $q r s$

And removing the 0 monomials:

```python
drop_cond = thop_prov_Db['pw_rel_dfs'][rel].prov_mono == 0
thop_prov_Db['pw_rel_dfs'][rel].drop(thop_prov_Db['pw_rel_dfs'][rel][drop_cond].index, axis=0, inplace=True)
thop_prov_Db['pw_rel_dfs'][rel][['pw', 'start', 'dest', 'prov_mono_']]
```

      pw
      start
      dest
      prov_mono_

      5
      1
      a
      b
      $q^{2} r$

      7
      1
      b
      a
      $q r^{2}$

      9
      1
      b
      c
      $q r s$

As we can see, we get the results, along with the correct provenance information, without re-running the query or any other Datalog operation.

Like before, we can compute the provenance polynomials by grouping by the variables in the head (start, dest) and aggregating the provenance monomials by adding them:

```python
thop_prov_Db['pw_rel_dfs'][rel].groupby(['start', 'dest']).agg(prov_poly=('prov_mono',
                                                          lambda x: '$'+sympy.latex(sum(x))+'$'))
```

      prov_poly

      start
      dest

      a
      b
      $q^{2} r$

      b
      a
      $q r^{2}$

      c
      $q r s$

#### Provenance Polynomials have other interestings uses as well

For instance, say each hop had a corresponding 'work' value, i.e. amount of work required to make the hop.
Then, one might be interested in finding the most/minimum amount of work required for each 3hop/2 (start-dest pair). This can be done using __tropical semirings__ i.e. we treat multiplication as addition and addition as the max/min operator.

```python
%%clingo --donot-display_input -lci hop_db_a_work --donot-run

% schema hop_work(hop_name, work)
hop_work(p, 0).
hop_work(q, 5).
hop_work(r, 10).
hop_work(s, 15).
```

One way to compute the desired 3hops would be to re-run the entire operation with a Datalog program like the one below:

(say for the min operation)

```python
%%clingo --donot-display_input -l hop_db_a hop_db_a_work

thop(X,Y, WORK) :- hop(X,Z1,P1), hop(Z1,Z2,P2), hop(Z2,Y,P3),
    hop_work(P1,WP1), hop_work(P2,WP2), hop_work(P3,WP3), WORK=WP1+WP2+WP3.

thop_opt(X,Y, WORK) :- #min{W: thop(X,Y, W)} = WORK, thop(X,Y,_).

#show thop_opt/3.
```

    Output:

1 Answer: 1
2 thop_opt(a,a,0) thop_opt(a,b,5) thop_opt(a,c,20) thop_opt(b,a,10) thop_opt(b,b,15) thop_opt(b,c,30)
3 SATISFIABLE
4
5 Models       : 1
6 Calls        : 1
7 Time         : 0.002s (Solving: 0.00s 1st Model: 0.00s Unsat: 0.00s)
8 CPU Time     : 0.002s

Note that since we do not have any provenance information, we, for example, know that there exists a sequence of 3 hops that takes 0 work to get from a to a, but we don't know which of the 3 (we know from earlier that there are 3 ways to go from a to a in 3 hops) 3hop(a,a) sequences achieves this.

Or, using the provenance information from earlier, we can do the following:

```python
%%clingo --donot-display_input -l thop_prov_D_facts hop_db_a_work

thop(X,Y, f(P1,P2,P3), WORK) :- thop(X,Y, f(P1,P2,P3)),
    hop_work(P1,WP1), hop_work(P2,WP2), hop_work(P3,WP3), WORK=WP1+WP2+WP3.

thop_opt(X,Y, WORK) :- #min{W: thop(X,Y, _, W)} = WORK, thop(X,Y,_,_).
thop_opt(X,Y, PROV, WORK) :- thop_opt(X,Y, WORK), thop(X,Y,PROV,WORK).

#show thop_opt/4.
```

    Output:

1 Answer: 1
2 thop_opt(a,a,f(p,p,p),0) thop_opt(b,a,f(r,p,p),10) thop_opt(a,b,f(p,p,q),5) thop_opt(b,b,f(r,p,q),15) thop_opt(a,c,f(p,q,s),20) thop_opt(b,c,f(r,q,s),30)
3 SATISFIABLE
4
5 Models       : 1
6 Calls        : 1
7 Time         : 0.002s (Solving: 0.00s 1st Model: 0.00s Unsat: 0.00s)
8 CPU Time     : 0.002s

As we can see, the answers match exactly, and we never had to re-evaluate the original 3hop query on the database. In addition, we know the hops that make up this optimized 3hop.

Another way to do this is directly in Python using the provenance monomials as follows:

First, we import the work values for the hops into Python and make a Python dictionary to enable substitution into the provenance monomials.

```python
hop_db_a_work_asp_out, hop_db_a_work_md = run_clingo(hop_db_a_work)
hop_db_a_work_pw_rel_dfs, _, _ = load_worlds(hop_db_a_work_asp_out, hop_db_a_work_md, silent=True)
hop_work_map = {row['hop_name']: int(row['work']) for _, row in hop_db_a_work_pw_rel_dfs['hop_work_2'].iterrows()}
hop_work_map
```

    {'p': 0, 'q': 5, 'r': 10, 's': 15}

```python
def tropical_semiring_mono(row):
    provs = sympy.symbols(row['prov'][1:])
    expr = reduce(lambda x, y: x+y, provs)
    return expr
```

Now, we can build our tropical semiring expressions:

```python
thop_opt_prov_D = deepcopy(thop_prov_D)

thop_opt_prov_D['pw_rel_dfs'][rel]['prov_trop_mono'] = thop_opt_prov_D['pw_rel_dfs'][rel].apply(tropical_semiring_mono, axis=1)
thop_opt_prov_D['pw_rel_dfs'][rel]['prov_trop_mono_'] = thop_opt_prov_D['pw_rel_dfs'][rel].apply(lambda row: '${}$'.format(sympy.latex(row['prov_trop_mono'])), axis=1)
thop_opt_prov_D['pw_rel_dfs'][rel][['pw', 'start', 'dest', 'prov', 'prov_trop_mono_']]
```

      pw
      start
      dest
      prov
      prov_trop_mono_

      0
      1
      a
      a
      [f, p, p, p]
      $3 p$

      1
      1
      b
      a
      [f, r, p, p]
      $2 p + r$

      2
      1
      a
      a
      [f, q, r, p]
      $p + q + r$

      3
      1
      a
      b
      [f, p, p, q]
      $2 p + q$

      4
      1
      b
      b
      [f, r, p, q]
      $p + q + r$

      5
      1
      a
      b
      [f, q, r, q]
      $2 q + r$

      6
      1
      a
      a
      [f, p, q, r]
      $p + q + r$

      7
      1
      b
      a
      [f, r, q, r]
      $q + 2 r$

      8
      1
      a
      c
      [f, p, q, s]
      $p + q + s$

      9
      1
      b
      c
      [f, r, q, s]
      $q + r + s$

To get the optimal solutions, we can now aggregate based on the min/max value of the prov_expr_ after grouping by the start-dest pairs:

```python
thop_opt_prov_D_trop_poly = thop_opt_prov_D['pw_rel_dfs'][rel].groupby(['start', 'dest']).agg(
    prov_trop_poly_min=('prov_trop_mono', lambda x: '$'+sympy.latex(sympy.Min(*x))+'$'),
    prov_trop_poly_max=('prov_trop_mono', lambda x: '$'+sympy.latex(sympy.Max(*x))+'$'))

thop_opt_prov_D_trop_poly.style.set_properties(subset=['prov_trop_poly_min', 'prov_trop_poly_max'], **{'width': '150px'})
```

        	        	        prov_trop_poly_min	        prov_trop_poly_max

        start	        dest

                        a
                        a
                        $\min\left(3 p, p + q + r\right)$
                        $\max\left(3 p, p + q + r\right)$

                                b
                        $\min\left(2 p + q, 2 q + r\right)$
                        $\max\left(2 p + q, 2 q + r\right)$

                                c
                        $p + q + s$
                        $p + q + s$

                        b
                        a
                        $\min\left(2 p + r, q + 2 r\right)$
                        $\max\left(2 p + r, q + 2 r\right)$

                                b
                        $p + q + r$
                        $p + q + r$

                                c
                        $q + r + s$
                        $q + r + s$

Now, we can substitute the work values to get the numerical solution:

```python
thop_opt_prov_D['pw_rel_dfs'][rel]['prov_trop_mono'] \
= thop_opt_prov_D['pw_rel_dfs'][rel].apply(lambda row: row['prov_trop_mono'].subs(hop_work_map).evalf(), axis=1)

thop_opt_prov_D['pw_rel_dfs'][rel]['prov_trop_mono_'] \
= thop_opt_prov_D['pw_rel_dfs'][rel].apply(lambda row: '${}$'.format(sympy.latex(row['prov_trop_mono'])), axis=1)

thop_opt_prov_D['pw_rel_dfs'][rel][['pw', 'start', 'dest', 'prov_trop_mono_']]
```

      pw
      start
      dest
      prov_trop_mono_

      0
      1
      a
      a
      $0$

      1
      1
      b
      a
      $10.0$

      2
      1
      a
      a
      $15.0$

      3
      1
      a
      b
      $5.0$

      4
      1
      b
      b
      $15.0$

      5
      1
      a
      b
      $20.0$

      6
      1
      a
      a
      $15.0$

      7
      1
      b
      a
      $25.0$

      8
      1
      a
      c
      $20.0$

      9
      1
      b
      c
      $30.0$

```python
thop_opt_prov_D['pw_rel_dfs'][rel].groupby(['start', 'dest']).agg(prov_trop_poly_min=('prov_trop_mono',
                                                          lambda x: '$'+sympy.latex(min(x))+'$'),
                                                                 prov_trop_poly_max=('prov_trop_mono',
                                                          lambda x: '$'+sympy.latex(max(x))+'$'))
```

      prov_trop_poly_min
      prov_trop_poly_max

      start
      dest

      a
      a
      $0$
      $15.0$

      b
      $5.0$
      $20.0$

      c
      $20.0$
      $20.0$

      b
      a
      $10.0$
      $25.0$

      b
      $15.0$
      $15.0$

      c
      $30.0$
      $30.0$

As we can see, this matches the earlier solutions exactly, and required no further Datalog computation, just pure mathematical substitution and manipulation.

One can repeat the demonstrations above with minor modifications for other variants of the 3hop query such as:

```python
%%clingo --donot-display_input --donot-run

%schema thop(start, prov)
thop(X, f(P1,P2,P3)) :- e(X,Z1, P1), e(Z1,Z2, P2), e(Z2,Y, P3).

#show thop/2.
```

```python
%%clingo --donot-display_input --donot-run

%schema thop(dest, prov)
thop(Y, f(P1,P2,P3)) :- e(X,Z1, P1), e(Z1,Z2, P2), e(Z2,Y, P3).

#show thop/2.
```

```python
%%clingo --donot-display_input --donot-run

%schema thop(dest, prov)
thop(Z1, Z2, f(P1,P2,P3)) :- e(X,Z1, P1), e(Z1,Z2, P2), e(Z2,Y, P3).

#show thop/3.
```

```python
%%clingo --donot-display_input --donot-run

%schema thop(dest, prov)
thop(X, Z1, Z2, Y, f(P1,P2,P3)) :- e(X,Z1, P1), e(Z1,Z2, P2), e(Z2,Y, P3).

#show thop/5.
```

And so on......

#### However, provenance polynomials do have some limitations.

For instance, since they don't preserve order, one cannot tell, say (p,q,r) from (q,r,p), since they're both reduced to $pqr$.

```python

```


# Extended Kalman Smoother

```python
# %load imports.py
%load_ext autoreload
%autoreload 2
%reload_kedro
%config Completer.use_jedi = False  ## (To fix autocomplete)

import pandas as pd
from src.models.vmm import ModelSimulator
import matplotlib.pyplot as plt
import matplotlib
#matplotlib.rcParams["figure.figsize"] = (5,5)
plt.style.use('presentation')
from src.visualization.plot import track_plots, plot, captive_plot
import kedro
import numpy as np
import os.path
import anyconfig

from myst_nb import glue
from src.symbols import *
import src.symbols as symbols
from src.system_equations import *

from IPython.display import display, Math, Latex, Markdown
from sympy.physics.vector.printing import vpprint, vlatex

from src.models.regression import MotionRegression

from src.parameters import df_parameters
p = df_parameters["symbol"]

# Read configs:
conf_path = os.path.join("../../conf/base/")
runs_globals_path = os.path.join(
    conf_path,
    "runs_globals.yml",
)

runs_globals = anyconfig.load(runs_globals_path)
model_test_ids = runs_globals["model_test_ids"]

join_globals_path = os.path.join(
    conf_path,
    "join_globals.yml",
)

joins = runs_globals["joins"]
join_runs_dict = anyconfig.load(join_globals_path)

globals_path = os.path.join(
    conf_path,
    "globals.yml",
)
global_variables = anyconfig.load(globals_path)

vmm_names = global_variables["vmms"]
only_joined = global_variables[
    "only_joined"
]  # (regress/predict with only models from joined runs)S

vmms = {}
for vmm_name in vmm_names:
    vmms[vmm_name] = catalog.load(vmm_name)

```

    UsageError: Line magic function `%reload_kedro` not found.

```python
id = 22774
data = catalog.load(f"{ id }.data")
data_ek_smooth = catalog.load(f"{ id }.data_ek_smooth")
data_ek_filter = catalog.load(f"{ id }.data_ek_filter")
data['r1d'] = np.gradient(data['r'], data.index)
```

    2022-03-28 14:43:55,585 - kedro.io.data_catalog - INFO - Loading data from `22774.data` (CSVDataSet)...
    2022-03-28 14:43:55,819 - kedro.io.data_catalog - INFO - Loading data from `22774.data_ek_smooth` (CSVDataSet)...
    2022-03-28 14:43:55,869 - kedro.io.data_catalog - INFO - Loading data from `22774.data_ek_filter` (CSVDataSet)...

```python
dataframes = {
    'raw' : data,
    #'EKF' : data_ek_filter,
    'EKS' : data_ek_smooth,

}
fig = plot(dataframes, keys=['psi','r','r1d'], ncols=3, fig_size=matplotlib.rcParams["figure.figsize"]);
fig.axes[0].set_ylabel(r'$\psi$')
fig.axes[2].set_ylabel(r'$\dot{r}$')
fig.axes[2].set_xlabel('time');

```

```python

```

```python

```


```python
from IPython.core.display import HTML
HTML("""
<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: middle;
}
</style>
""")
```

<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: middle;
}
</style>

# *Circuitos Elétricos I - Semana 12.1*

```python
import matplotlib.pyplot as plt
import numpy as np
import sympy as sp
from utils import round_expr, symdisp, symplot

from sympy.polys.partfrac import apart

# temp workaround
import warnings
from matplotlib import MatplotlibDeprecationWarning
warnings.filterwarnings('ignore', category=MatplotlibDeprecationWarning)

plt.rcParams['figure.figsize'] = 6, 4
plt.rcParams['legend.fontsize'] = 13
plt.rcParams['lines.linewidth'] = 2
plt.rcParams['axes.grid'] = False
```

```python
# transformada de Laplace
def L(f,t,s):
    return sp.laplace_transform(f, t, s, noconds=True)

# transformada inversa de Laplace
def invL(F,s,t):
    return sp.re(sp.inverse_laplace_transform(F, s, t, noconds=True))

# funções para auxílio na expansão em frações parciais
def adjustCoeff(expr):
    coeff = expr.as_numer_denom()
    c0 = sp.poly(coeff[1].cancel()).coeffs()[0]

    return (coeff[0].cancel()/c0)/(coeff[1].cancel()/c0)

def partFrac(expr, Ndigits):
    expr = expr.cancel()
    expr = apart(adjustCoeff(expr), s, full=True).doit()

    return sp.N(expr, Ndigits)

sp.init_printing()
```

#### Definindo algumas variáveis simbólicas de interesse

```python
s     = sp.symbols('s')
a     = sp.symbols('a', real=True, positive=True)
omega, t = sp.symbols('omega, t', real=True)
infty = sp.oo
```

### Problema 1

O circuito da figura a seguir está em regime estacionário no momento em que chave é aberta. Sabe-se que $v(t)=\mathrm{12\;V}$.

a. Determine $I_0(s)$ e $I_1(s)$.\
b. Verifique a consistência das respostas do item a. com os teoremas do valor inicial e do valor final.\
c. Determine a função de transferência $H_0(s)$ entre $V(s)$ e $I_0(s)$.\
d. Determine a função de transferência $H_1(s)$ entre $V(s)$ e $I_1(s)$.\
e. Determine a função de transferência $H_c(s)$ entre $V(s)$ e $V_c(s)$.\
f. Determine a $v_c(t)$ para o caso em que a tensão aplicada $v(t)$ em $\mathrm{V}$ corresponde ao gráfico a seguir:

a. Determinando $I_0(s)$ e $I_1(s)$:

```python
I0, I1, s = sp.symbols('I0, I1, s')

# define os sistema de equações
eq1 = sp.Eq((-2*s**2)*I0 + (2*s**2 + 10*s + 250)*I1, 2.4*s + 12)
eq2 = sp.Eq((4*s+50)*I0 -2*s*I1, -2.4)

# resolve o sistema
soluc = sp.solve([eq1, eq2],[I0, I1], dict=True)
soluc

I0 = [sol[I0] for sol in soluc]
I1 = [sol[I1] for sol in soluc]

I0 = I0[0]
I1 = I1[0]

print('Correntes no domínio de Laplace: \n')
symdisp('I_0(s) =', I0, 'As')
symdisp('I_1(s) =', I1, 'As')
```

    Correntes no domínio de Laplace:

$\displaystyle I_0(s) =- \frac{150.0}{s^{3} + 35.0 s^{2} + 375.0 s + 3125.0}\;As$

$\displaystyle I_1(s) =\frac{6.0 s^{2} + 210.0 s + 750.0}{5.0 s^{3} + 175.0 s^{2} + 1875.0 s + 15625.0}\;As$

b. Checando a consistência das soluções

**Teorema do valor inicial (TVI)**

$$
f(0^+) = \lim_{t \to 0^+}f(t) = \lim_{s \to \infty}sF(s)
$$

```python
i0_0_tvi = sp.limit(s*I0, s, infty)
i1_0_tvi = sp.limit(s*I1, s, infty)

symdisp('i_0(0^+) = ', i0_0_tvi, ' A' )
symdisp('i_1(0^+) = ', i1_0_tvi, ' A' )
```

$\displaystyle i_0(0^+) = 0\; A$

$\displaystyle i_1(0^+) = 1.2\; A$

**Teorema do valor final (TVF)**

$$
f(\infty) = \lim_{t \to \infty}f(t) = \lim_{s \to 0}sF(s)
$$

```python
i0_inf_tvf = sp.limit(s*I0, s, 0)
i1_inf_tvf = sp.limit(s*I1, s, 0)

symdisp('i_0(\infty) = ', i0_inf_tvf, ' A' )
symdisp('i_1(\infty) = ', i1_inf_tvf, ' A' )
```

$\displaystyle i_0(\infty) = 0\; A$

$\displaystyle i_1(\infty) = 0\; A$

```python
C = 4e-3  # F

# Calculando Vc
Vc = (1/(s*C))*I1

Vc = Vc.simplify()

symdisp('V_c(s) =', adjustCoeff(Vc).simplify(), 'Vs')
```

$\displaystyle V_c(s) =\frac{300.0 s^{2} + 10500.0 s + 37500.0}{s \left(1.0 s^{3} + 35.0 s^{2} + 375.0 s + 3125.0\right)}\;Vs$

```python
 sp.limit(s*Vc, s, 0)
```

```python
np.roots([1, 35, 375, 3125, 0])
```

    array([-25. +0.j,  -5.+10.j,  -5.-10.j,   0. +0.j])

```python
partFrac(Vc, 4)
```

```python
vc = invL(partFrac(Vc, 4), s, t)

symdisp('v_c(t) = ', vc, ' V')
```

$\displaystyle v_c(t) = 12.0 \theta\left(t\right) + 3.0 e^{- 25.0 t} \theta\left(t\right) + 30.0 e^{- 5.0 t} \sin{\left(10.0 t \right)} \theta\left(t\right) - 15.0 e^{- 5.0 t} \cos{\left(10.0 t \right)} \theta\left(t\right)\; V$

```python
# plota funções no domínio do tempo
intervalo = np.arange(-1, 4, 0.01)
symplot(t, vc, intervalo, 'vc(t)')
```

c-e) Determinando $H_0(s)$, $H_0(s)$ e $H_c(s)$.

```python
V = 12/s

H0 = I0/V

symdisp('H_0(s) =', H0,)
```

$\displaystyle H_0(s) =- \frac{12.5 s}{s^{3} + 35.0 s^{2} + 375.0 s + 3125.0}\; $

```python
H1 = I1/V
H1 = adjustCoeff(H1)

symdisp('H_1(s) =', H1,)
```

$\displaystyle H_1(s) =\frac{0.1 s^{3} + 3.5 s^{2} + 12.5 s}{1.0 s^{3} + 35.0 s^{2} + 375.0 s + 3125.0}\; $

```python
Hc = Vc/V
Hc = adjustCoeff(Hc)

symdisp('H_c(s) =', Hc,)
```

$\displaystyle H_c(s) =\frac{25.0 s^{2} + 875.0 s + 3125.0}{1.0 s^{3} + 35.0 s^{2} + 375.0 s + 3125.0}\; $

f. $v_c(t)=?$

```python
v = 2*(sp.Heaviside(t)-sp.Heaviside(t-1)) - 1*(sp.Heaviside(t-1)-sp.Heaviside(t-2))

symdisp('v(t) =', v, ' V')

# plota funções no domínio do tempo
intervalo = np.arange(-1, 3, 0.01)
symplot(t, v, intervalo, 'v(t)')
```

```python
# determina Vc(s) via função de transferência
V = L(v, t, s)
Vc = Hc*V

symdisp('V_c(s) =', Vc.simplify(), ' Vs')
```

$\displaystyle V_c(s) =\frac{\left(25.0 s^{2} + 875.0 s + 3125.0\right) \left(2 e^{2 s} - 3 e^{s} + 1\right) e^{- 2 s}}{s \left(1.0 s^{3} + 35.0 s^{2} + 375.0 s + 3125.0\right)}\; Vs$

```python
# função auxiliar Va(s)
P = (25*s**2 + 875*s + 3125)/(s**4 + 35*s**3 + 375*s**2 + 3125*s)
P = partFrac(P, 10)

# encontra va(t)
p = invL(P, s, t)
p = p.expand()

symdisp('P(s) =', P , ' Vs')
symdisp('p(t) =', p, ' V')
```

$\displaystyle P(s) =\frac{-0.625 + 1.25 i}{s + 5.0 + 10.0 i} + \frac{-0.625 - 1.25 i}{s + 5.0 - 10.0 i} + \frac{0.25}{s + 25.0} + \frac{1}{s}\; Vs$

$\displaystyle p(t) =1.0 \theta\left(t\right) + 0.25 e^{- 25.0 t} \theta\left(t\right) + 2.5 e^{- 5.0 t} \sin{\left(10.0 t \right)} \theta\left(t\right) - 1.25 e^{- 5.0 t} \cos{\left(10.0 t \right)} \theta\left(t\right)\; V$

```python
vc = 2*p - 3*p.subs({t:sp.UnevaluatedExpr(t-1)}) + p.subs({t:sp.UnevaluatedExpr(t-2)})

symdisp('v_c(t) =', vc, ' V')
```

$\displaystyle v_c(t) =2.0 \theta\left(t\right) + 1.0 \theta\left(t - 2\right) + 0.25 \theta\left(t - 2\right) e^{- 25.0 \left(t - 2\right)} + 2.5 \theta\left(t - 2\right) e^{- 5.0 \left(t - 2\right)} \sin{\left(10.0 \left(t - 2\right) \right)} - 1.25 \theta\left(t - 2\right) \cos{\left(10.0 \left(t - 2\right) \right)} e^{- 5.0 \left(t - 2\right)} - 3 \left(1.0 \theta\left(t - 1\right) + 0.25 \theta\left(t - 1\right) e^{- 25.0 \left(t - 1\right)} + 2.5 \theta\left(t - 1\right) e^{- 5.0 \left(t - 1\right)} \sin{\left(10.0 \left(t - 1\right) \right)} - 1.25 \theta\left(t - 1\right) \cos{\left(10.0 \left(t - 1\right) \right)} e^{- 5.0 \left(t - 1\right)}\right) + 0.5 e^{- 25.0 t} \theta\left(t\right) + 5.0 e^{- 5.0 t} \sin{\left(10.0 t \right)} \theta\left(t\right) - 2.5 e^{- 5.0 t} \cos{\left(10.0 t \right)} \theta\left(t\right)\; V$

```python
# plota funções no domínio do tempo
intervalo = np.arange(-1, 3, 0.01)
symplot(t, [v, vc], intervalo, ['v(t)','vc(t)'])
```

Jupyter notebooks/Circuitos Eletricos I - Semana 12.1.ipynb


# Problema de selección de portafolio con preferencias media-varianza

En la clase pasada hablamos acerca de:
- preferencias,
- funciones de utilidad,
- la actitud de los inversionistas de cara al riesgo,
- la aversión al riesgo, entre otros.

Todas ellas son piezas que necesitamos para responder la pregunta de ¿cómo un inversionista toma la decisión óptima de selección de portafolio?

En esta clase al fin estamos listos para ensamblar estos conceptos y escribir el problema de selección de portafolios.

En el camino aprenderemos acerca del concepto de **utilidad esperada**, que nos permite trabajar con incertidumbre en el modelado económico (una de las ideas más importantes en economía). Esta idea tiene más de 60 años, y básicamente dice que los individuos, cuando están de cara a incertidumbre, maximizan el valor esperado de su utilidad (solo cierto si somos homo economicus).

Además del concepto de utilidad esperada, aprenderemos acerca de **preferencias media-varianza**. Es decir, supondremos que los inversionistas toman decisiones basados en un tipo particular de preferencias.

Con lo anterior, estableceremos el problema de selección de portafolios.

**Objetivos:**

- ¿Qué es utilidad esperada?
- ¿Qué son las preferencias media-varianza?
- Funciones de utilidad media-varianza.
- Enunciado y solución del problema básico de selección de portafolio.

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Utilidad esperada
- Básicamente combina las probabilidades de los resultados con cómo los inversionistas se sienten con dichos resultados.
- En otras palabras, la utilidad esperada multiplica la probabilidad de suceso de un evento con la utilidad que genera dicho evento.

Recordemos que las *funciones de utilidad* permiten a los inversionistas expresar cómo se sienten con los resultados, especialmente en los malos ratos.

Entonces la *utilidad esperada* es una herramienta que nos permite cuantificar cómo nos sentimos en nuestros malos momentos económicos, capturando el riesgo con la probabilidad de ocurrencia de dichos malos momentos.

Dado este marco de trabajo, cualquier decisión se puede escribir como la maximización de la utilidad esperada:
\begin{align}
\max_{\theta} & \quad E[U(W)], \\
\end{align}
mediante la escogencia de cierta variable $\theta$ (gastos, planes de ahorro, compra de activos, planes de producción, etc.).

Para nuestros propósitos, la variable de decisión serán los pesos o ponderaciones del portafolio.

Adicionalmente, en el contexto de la decisión de distribución de la riqueza entre activos, el problema de maximización tendrá comúnmente las siguientes restricciones:
- universo de inversión,
- posición en los activos dados.

**Ejemplo.**

Supongamos que un inversionista debe determinar la composición óptima de su portafolio, que contiene activos y bonos. Supongamos que son los únicos instrumentos disponibles.

Sean:
- $w_s$: peso o ponderación de activos en el portafolio,
- $w_b$: peso o ponderación de bonos en el portafolio,
- $r_s$: rendimiento de los activos, y
- $r_b$: rendimiento de los bonos.

De manera que podemos escribir el problema de selección de portafolios como la maximización de la utilidad esperade de nuestra riqueza futura, la cual dependerá de nuestros rendimientos:

\begin{align}
\max_{w_s,w_b} &\quad E[U(W)]\\
\text{s. a.}   &\quad W=W_0(1+w_sr_s+w_br_b)\\
               &\quad w_s+w_b=1
\end{align}

Preguntas:
- ¿Qué significan las restricciones?
- Ya que tenemos planteado este problema básico, ¿qué haría falta para empezar a resolverlo?
___

## 2. Preferencias media-varianza

### 2.1. Utilidad media-varianza

Entonces, ¿qué funciones de utilidad deberíamos de usar en este problema de selección de portafolios?

- La respuesta es: **preferencias media-varianza**.
- Éstas serán representadas en términos de funciones de utilidad como: **utilidad media-varianza**.

Usamos la *utilidad media-varianza* en el problema de selección de portafolios dado que ésta decribe el "trade-off" entre riesgo y rendimiento que enfrentan los inversionistas. La *utilidad media-varianza* está dada por la siguiente expresión:

$$U=E[r_p]-\frac{1}{2}\gamma\sigma_p^2,$$

donde
- $E[r_p]$ es el rendimiento esperado del portafolio,
- $\sigma_p^2$ es la varianza del portafolio, y
- $\gamma$ es el coeficiente de aversión al riesgo.

#### Intuición acerca de la función de utilidad media-varianza:
- Sólo se preocupa por medias :) y varianzas :(.
- Incrementa con: rendimiento esperado del portafolio.
- Decrece con: varianza del portafolio (riesgo).
- Malos tiempos: rendimientos son bajos y las volatilidades son altas.
- Conecta bastante bien con la teoría moderna de portafolios, la cual caracteriza los rendimientos con medias y varianzas únicamente.
- Criticada por su limitación: supone que los inversionistas sólo se preocupan por medias y varianzas.

### 2.2. Curvas de indiferencia

*¿Recuerdan las **curvas de nivel** que se ven en cálculo de varias variables?*
- Bien, acá nos servirán para representar la utilidad media-varianza gráficamente.
- En el contexto de utilidad media-varianza, las curvas de nivel se llaman **curvas de indiferencia**.

Dados ciertos niveles de utilidad $U_1>U_2>U_3$, las curvas de indiferencia relativas a estos niveles de utilidad, son los lugares geométricos en el espacio de rendimiento esperado vs. volatilidad representados por las siguientes expresiones

$$U_1=E[r_p]-\frac{1}{2}\gamma\sigma_p^2\Rightarrow E[r_p]=\frac{1}{2}\gamma\sigma_p^2+U_1,$$

$$U_2=E[r_p]-\frac{1}{2}\gamma\sigma_p^2\Rightarrow E[r_p]=\frac{1}{2}\gamma\sigma_p^2+U_2,$$

$$U_3=E[r_p]-\frac{1}{2}\gamma\sigma_p^2\Rightarrow E[r_p]=\frac{1}{2}\gamma\sigma_p^2+U_3.$$

**Gráficamente**

```python
# Importar numpy y pyplot
import numpy as np
import matplotlib.pyplot as plt
%matplotlib  inline
```

```python
# Coeficiente de aversión al riesgo (entre 1 y 10 comúnmente)
g= 9 #elegido en clase
# Niveles de utilidad 3, 2, 1
U1, U2, U3 = 0.15, 0.1, 0.05
# Vector de volatilidades (sugerido 1%-60%)
sp = np.linspace(0.01, 0.6, 100)
# Curvas de indiferencia
E1= 0.5*g*sp**2+U1
E2= 0.5*g*sp**2+U2
E3= 0.5*g*sp**2+U3
```

```python
# Gráfica
plt.figure(figsize=(8,6))
plt.plot(sp, E1, label='$U_1=0.15$')
plt.plot(sp, E2,label='$U_2=0.1$')
plt.plot(sp, E3,label='$U_3=0.05$')
plt.legend(loc=4)
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado($E[r_p]$)')
plt.grid()
```

Va en aumento hacia ariba y hacia la izquierda.

Bueno, ¿y porqué se llaman curvas de indiferencia?, ¿qué representa una cuva de indiferencia?

- Porque sobre una misma curva el nivel de utilidad es el mismo (es indiferente).
- Son todas las combinaciones de riesgo y rendimiento que producen un mismo nivel de utilidad.

Volviendo al problema de selección de portafolios, queremos la utilidad más alta.
- ¿Cuál de las anteriores curvas de indiferencia corresponde a la utilidad más alta?
- Intuitivamente, ¿porqué?
- Curvas de indiferencia para niveles de utilidad más altos, estarán...

Notamos además que las anteriores curvas de indiferencia son *paralelas* una con otra. Claro, las dibujamos con el mismo coeficiente de aversión al riesgo.

¿Cómo cambian estas curvas para coeficientes de aversión al riesgo más altos?

```python
# Coeficientes de aversión al riesgo (entre 1 y 10 comúnmente)
g1, g2, g3= 3,5,7
# Nivel de utilidad
U= 0.15
# Vector de volatilidades (sugerido 1%-60%)
sp_new= np.linspace(0.01,0.6,100)
# Curvas de indiferencia
E1= 0.5*g1*sp**2+U
E2= 0.5*g2*sp**2+U
E3= 0.5*g3*sp**2+U
```

```python
# Gráfica
plt.figure(figsize=(8,6))
plt.plot(sp, E1, label='$\gamma_1=3$')
plt.plot(sp, E2,label='$\gamma_2=5$')
plt.plot(sp, E3,label='$\gamma_3=7$')
plt.legend(loc=4)
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado($E[r_p]$)')
plt.grid()
```

¿Cómo interpretamos las anteriores gráficas?, ¿qué pasa con las personas más aversas al riesgo?
- Se puede ver de dos maneras: para un mismo nivel de rendimiento esperado, una persona más aversa al riesgo soporta un nivel menor de riesgo; equivalentemente, para un mismo nivel de riesgo, una persona más aversa al riesgo requerirá un nivel de rendimiento esperado más alto.

Con todo lo anterior, el problema de selección de portafolios se puede plantear como *"encontrar la curva de indeferencia más alta dado el conjunto de oportunidades de inversión y restricciones"*.

## 3. Problema de selección de portafolios: una ilustración

Ahora ilustraremos el problema de selección de portafolios con algunos datos.
- Por ahora solo queremos ilustrar gráficamente cómo se resuelve este problema. Trabajar en la intuición.
- En las siguientes dos clases nos enfocaremos en cómo resolverlo analíticamente.

Acá tenemos el rendimiento medio anual y la volatilidad para dos instrumentos usando datos de EU: instrumentos de deuda (bonos) y acciones. Supondremos que el inversionista solo puede invertir en estas dos clases de instrumentos.

```python
# Importamos pandas
import pandas as pd
```

```python
# Datos
data=pd.DataFrame(index=['Stocks', 'Bonds', 'CorrSB']
                  , columns=['Mean', 'Std'])
data['Mean']=[0.119, 0.59, 0.113]
data['Std']=[0.1915, 0.0833, None]
data
```

      Mean
      Std

      Stocks
      0.119
      0.1915

      Bonds
      0.590
      0.0833

      CorrSB
      0.113
      NaN

Entonces, ¿cuál es la distribución de riqueza óptima?, o más bien, ¿cuál es la composición óptima del portafolio para un inversionista dado su nivel de aversión al riesgo?

**Primero.** Recordamos que, para dos activos, podemos trazar la frontera de mínima varianza tomando todas las posibles combinaciones de los dos activos.

De nuevo, sean:
- $w_s=w$: peso o ponderación de activos en el portafolio,
- $w_b=1-w$: peso o ponderación de bonos en el portafolio,
- $r_s$: rendimiento de los activos, y
- $r_b$: rendimiento de los bonos.

Entonces

$$E[r_p]=wE[r_{s}]+(1-w)E[r_b]$$

$$\sigma_p^2=w^2\sigma_{s}^2+(1-w)^2\sigma_b^2+2w(1-w)\rho_{s,b}\sigma_s\sigma_b$$

```python
# Vector de w variando entre 0 y 1 con n pasos
w= np.linspace(0, 1, 101)
# Rendimientos esperados individuales
Es = data.loc['Stocks', 'Mean']
Eb = data.loc['Bonds', 'Mean']
# Volatilidades individuales
ss= data.loc['Stocks', 'Std']
sb= data.loc['Bonds', 'Std']
# Correlacion
rsb= data.loc['CorrSB', 'Mean']
Es, Eb, ss, sb, rsb
```

    (0.119, 0.59, 0.1915, 0.0833, 0.113)

```python
# Crear un DataFrame cuyas columnas sean rendimiento
# y volatilidad del portafolio para cada una de las w
# generadas
portafolios= pd.DataFrame(data={'Media': w*Es+(1-w)*Eb,
                               'Vol':(w**2*ss**2+(1-w)**2*sb**2+2*w*(1-w)*rsb*ss*sb)**0.5})
portafolios.head(),portafolios.tail()
```

    (     Media       Vol
     0  0.59000  0.083300
     1  0.58529  0.082705
     2  0.58058  0.082155
     3  0.57587  0.081650
     4  0.57116  0.081191,        Media       Vol
     96   0.13784  0.184246
     97   0.13313  0.186054
     98   0.12842  0.187866
     99   0.12371  0.189681
     100  0.11900  0.191500)

cuando $w_s$ es igual entonces $w_b$ es igual a $1$ y estamos invirtiendo todo en bonos

```python
# Gráfica
plt.figure(figsize=(6,4))
plt.plot(portafolios['Vol'], portafolios['Media'], label='Port')
plt.legend(loc=3)
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado($E[r_p]$)')
plt.grid()
```

**Segundo.** Graficamos en la misma ventana, curvas de indiferencia.

```python
# Niveles de utilidad 0.06, 0.07, 0.09
U4, U5, U6= 0.06, 0.07, 0.09
# Coeficiente de aversión al riesgo
g=3
# Curvas de indiferencia
E1= 0.5*g*sp**2+U4
E2= 0.5*g*sp**2+U5
E3= 0.5*g*sp**2+U6
```

```python
# Gráfica
# Gráfica
plt.figure(figsize=(8,8))
plt.plot(portafolios['Vol'], portafolios['Media'], label='Port')

plt.plot(portafolios['Vol'], E1, label='$U_4=0.15$')
plt.plot(portafolios['Vol'], E2, label='$U_5=0.1$')
plt.plot(portafolios['Vol'], E3, label='$U_6=0.05$')

plt.legend(loc=4)
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado($E[r_p]$)')
plt.grid()
plt.show()
```

**Tercero.** La elección óptima está dada por la curva de indiferencia para el nivel de utilidad más alto que es tangente a la frontera media-varianza.
- Claramente, esta selección depende del coeficiente de aversión al riesgo.

```python
# Gráfica con zoom
# Gráfica
plt.figure(figsize=(8,8))
plt.plot(portafolios['Vol'], portafolios['Media'], label='Port')

plt.plot(portafolios['Vol'], E1, label='$U_4=0.15$')
plt.plot(portafolios['Vol'], E2, label='$U_5=0.1$')
plt.plot(portafolios['Vol'], E3, label='$U_6=0.05$')

plt.legend(loc=4)
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado($E[r_p]$)')
plt.grid()
plt.axis([0.11, 0.15, 0.08, 0.11])
plt.show()
```

El portafolio que maximiza mi volátilidad es aquel que sea tangente a nuestra curva de mínima varianza

# Anuncios parroquiales

## 1. Quiz la siguiente clase.
## 2. Un par de artículos del WSJ y el NYT que discuten herramientas disponibles para la medición de su propia tolerancia al riesgo:
- [Artículo 1](https://www.nytimes.com/2016/02/13/your-money/as-stocks-fall-its-time-to-measure-your-risk-tolerance.html)
- [Artículo 2](https://www.wsj.com/articles/check-your-tolerance-for-investment-risk-now-before-markets-sag-1405619939)

Created with Jupyter by Esteban Jiménez Rodríguez.


# !!! D . R . A . F . T !!!

# Chromatic Adaptation

Adaptation is the process of change by which an organism becomes better suited to its environment.

In the context of colour science, it is defined as the process by which the state of the visual system is modified by previous and present exposure to stimuli that may have various luminance values, spectral distributions and angular subtenses. [1]

The three essential types of adaptation for human vision are:

*  Dark adaptation
*  Light adaptation
*  Chromatic adaptation

### Dark Adaptation

Dark adaptation happens when the luminance level decreases and as the result of the lack of illumination, the observer visual system adapts to the stimulus and the visual sensitivity increases.

When entering a dark area, initially, the [cone cells](http://en.wikipedia.org/wiki/Cone_cell) sensitivity increases to reach full adaptation after around 10 minutes, then the [rod cells](http://en.wikipedia.org/wiki/Rod_cell) sensitivity outperforms the cones and reach complete adaptation after 30 minutes. [2] [3] [4]

A notable effect of rod cells driving the visual system at low luminance level is that humans become colour-blind in those low lighting conditions.

### Light Adaptation

Light adaptation is similar to dark adaption but instead the visual sensitivity decreases with luminance level increase.

The adaptation process happening when entering a bright area is faster than dark adaptation and occurs in around 5 minutes. The rod cells first saturate as [rhodopsin](http://en.wikipedia.org/wiki/Rhodopsin) *photobleaches*, while the cone cells continue to adapt reaching peak sensitivity after 5-10 minutes. [4] [5] [6]

### Chromatic Adaptation

Chromatic adaptation is defined as the visual process whereby approximate compensation is made for changes in the colours of stimuli, especially in the case of changes in illuminants. [7]

Chromatic adaptation controls the independent sensitivity of the three cones cells type and is the most inportant adaptation mechanism in colour appearance. A white object viewed under different lighting conditions (daylight, tungsten or incandescent lighting) will retain its white appearance because the sensitivity of the cone cells is independently adjusted to compensate for the changes in energy level at the wavelengths ranges they are sensitive to.

Chromatic adaptation can be thought of as analogous to automatic white balancing feature of a digital still camera.[8]

## Johannes Von Kries Hypothesis

Chromatic adaptation models are based on physiological plausible representations of cone responses signals. All modern chromatic adaptation models build upon Von Kries (1902) hypothesis: [9]

> This can be conceived in the sense that the individual components present in the organ of vision are completely independent of one another and each is fatigued or adapted exclusively according to its own function (Translation by MacAdam 1970: 118).

The modern interpretation of Von Kries (1902) hypothesis, the *Von Kries Coefficient Law*, defines the following relationship: [10]

$$
\begin{equation}
\begin{vmatrix}
L_a\\
M_a\\
S_a
\end{vmatrix}=
\begin{vmatrix}
k_L&0.0&0.0\\
0.0&k_M&0.0\\
0.0&0.0&k_S
\end{vmatrix}
\begin{vmatrix}
L\\
M\\
S
\end{vmatrix}
\end{equation}
$$

or

$$
\begin{equation}
\begin{aligned}
L_a&\ =k_L\cdot L\\
M_a&\ =k_M\cdot M\\
S_a&\ =k_S\cdot S
\end{aligned}
\end{equation}
$$

with

$$
\begin{equation}
k_L=\cfrac{1}{L_{max}},\qquad k_M=\cfrac{1}{M_{max}},\qquad k_S=\cfrac{1}{S_{max}}
\end{equation}
$$

or

$$
\begin{equation}
k_L=\cfrac{1}{L_{white}},\qquad k_M=\cfrac{1}{M_{white}},\qquad k_S=\cfrac{1}{S_{white}}
\end{equation}
$$

where $L_a$, $M_a$ and $S_a$ are the post-adaptation cones signals and $k_L$, $k_M$ and $k_S$ are the coefficients used to scale the initial cone signals $L$, $M$ and $S$.

Given the above equations, the *Von Kries Coefficient Law* can be used to compute *corresponding colour stimuli* between two different viewing conditions.

The *CIE* defines *corresponding colour stimuli* as pairs of colour stimuli that have same colour appearance when one is seen in one set of adaptation conditions and the other is seen in a different set. [11]

Computing the *corresponding colour stimuli* is done as follows:

Post-adaptation signals are computed for the two viewing conditions, then, the first viewing condition post-adaptation signals are set equal to the second viewing condition post-adaptation signals, finally the model is reversed for the second viewing condition:

$$
\begin{equation}
\begin{vmatrix}
X_2\\
Y_2\\
Z_2
\end{vmatrix}=
\textbf{M}^{-1}
\begin{vmatrix}
L_{max2}&0.0&0.0\\
0.0&M_{max2}&0.0\\
0.0&0.0&S_{max2}
\end{vmatrix}
\begin{vmatrix}
k_L&0.0&0.0\\
0.0&k_M&0.0\\
0.0&0.0&k_S
\end{vmatrix}
\textbf{M}
\begin{vmatrix}
X\\
Y\\
Z
\end{vmatrix}
\end{equation}
$$

with

$$
\begin{equation}
L_2=\biggl(\cfrac{L_1}{L_{max1}}\biggr)L_{max2},\qquad M_2=\biggl(\cfrac{M_1}{M_{max1}}\biggr)M_{max2},\qquad S_2=\biggl(\cfrac{S_1}{S_{max1}}\biggr)S_{max2}
\end{equation}
$$

where $\textbf{M}$ is the *chromatic adaptation transform* matrix converting from *CIE XYZ* tristimulus values to relative cone responses $LMS$.

[Colour](https://github.com/colour-science/colour/) defines the following *chromatic adaptation transforms*:

```python
import colour

print(sorted(colour.CHROMATIC_ADAPTATION_TRANSFORMS))
```

    ['Bianco', 'Bianco PC', 'Bradford', 'CAT02', 'CAT02_BRILL_CAT', 'CMCCAT2000', 'CMCCAT97', 'Fairchild', 'Sharp', 'Von Kries', 'XYZ Scaling']

## Johannes Von Kries Chromatic Adaptation Transform

## Bibliography

1. ^ CIE. (n.d.). 17-18 adaptation. Retrieved September 07, 2014, from http://eilv.cie.co.at/term/18
2. ^ Wikipedia. (n.d.). Dark adaptation. Retrieved September 07, 2014, from http://en.wikipedia.org/wiki/Adaptation_(eye)#Dark_Adaptation
3. ^ Fairchild, M. D. (2013). Dark Adaptation. In *Color Appearance Models* (3rd ed., p. 985,3749). Wiley. ASIN:B00DAYO8E2
4. ^ Kalloniatis, M., & Luu, C. (n.d.). Light and Dark Adaptation. Retrieved September 07, 2014, from http://webvision.med.utah.edu/book/part-viii-gabac-receptors/light-and-dark-adaptation/
5. ^ Wikipedia. (n.d.). Light adaptation. Retrieved September 07, 2014, from http://en.wikipedia.org/wiki/Adaptation_(eye)#Light_Adaptation
6. ^ Fairchild, M. D. (2013). Light Adaptation. In *Color Appearance Models* (3rd ed., p. 1006,3728). Wiley. ASIN:B00DAYO8E2
7. ^ CIE. (n.d.). 17-140 chromatic adaptation. Retrieved September 07, 2014, from http://eilv.cie.co.at/term/140
8. ^ Fairchild, M. D. (2013). Chromatic Adaptation. In *Color Appearance Models* (3rd ed., p. 1029,3749). Wiley. ASIN:B00DAYO8E2
9. ^ Fairchild, M. D. (2013). VON KRIES MODEL. In *Color Appearance Models* (3rd ed., p. 4204). Wiley. ASIN:B00DAYO8E2
10. ^ Fairchild, M. D. (2013). Equations 9.1-9.11. In *Color Appearance Models* (3rd ed., pp. 4225–4262). Wiley. ASIN:B00DAYO8E2
11. ^ CIE. (n.d.). 17-260 corresponding colour stimuli. Retrieved September 07, 2014, from http://eilv.cie.co.at/term/260