Trusted Notebook" width="250 px" align="left">

## _*Quantum Process Tomography*_

The latest version of this notebook is available on https://github.com/QISKit/qiskit-tutorial.

***
### Contributors
Christopher J. Wood$^{1}$

1. IBM T.J. Watson Research Center, Yorktown Heights, NY, USA

## Introduction

In this notebook we demonstrate how to design and run experiments to perform quantum process tomography using QISKit, and demonstrate this using both simulators, and the IBM Quantum Experience. This is implemented similarly to the [quantum state tomography](state_tomography.ipynb) example notebook [here](state_tomography.ipynb).

The functions used in this notebook to implement state tomography may be imported from the **`tools.qcvv.tomography`** module.

#### Reconstruction Methods

State reconstruction may be done using a variety of methods. In this notebook, we implement two simple cases:
* Least squares fitting
* Fast maximum likelihood (*see [J Smolin, JM Gambetta, G Smith, Phys. Rev. Lett. 108, 070502](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.108.070502), open access: [arXiv:1106.5458](https://arxiv.org/abs/1106.5458)*).

Note that while the fast maximum likelihood method constrains the resulting channel to be positive, it does not enfore complete positivity of the final estimate.

```python
import sys
try:
    sys.path.append("../../") # go to parent dir
    import Qconfig
    print('Qconfig loaded from %s.' % Qconfig.__file__)
    qx_config = {
        "APItoken": Qconfig.APItoken,
        "url": Qconfig.config['url']}
except:
    qx_config = {
        "APItoken":"YOUR_TOKEN_HERE",
        "url":"https://quantumexperience.ng.bluemix.net/api"}
    print('Qconfig.py not found on Python path; Qconfig loaded from qx_config in above code.')
```

    Qconfig loaded from C:\Users\IBM_ADMIN\Anaconda3\envs\qiskitpip\lib\site-packages\Qconfig.py.

```python
import numpy as np

# importing the QISKit
from qiskit import QuantumCircuit, QuantumProgram

# import tomography libary
import qiskit.tools.qcvv.tomography as tomo

# useful additional packages
from qiskit.tools.visualization import plot_state
from qiskit.tools.qi.qi import *
```

## Building process tomography measurement circuits

To perform quantum process tomography of an n-qubit quantum operation, we must construct a family of circuits that implements a tomographically complete set of preparations and measurements of each of those qubits. The **`tools.qcvv.tomography`** module contains functions to generate these measurement circuits for general n-qubit systems.

#### Specifying the qubits, preparation basis, and measurement basis

A process tomography experiment is specified by a **`process_tomography_set`** object:
```python
tomo_set = tomo.process_tomography_set(qubits, meas_basis, prep_basis)
```
where
- `qubits` is a list of the qubit indexes within a register (eg. [0, 1] for the first 2 qubits)
- `meas_basis` is the basis to measure each qubit in. The default value is `meas_basis='Pauli'`.
- `prep_basis` is the basis to measure each qubit in. The default value is `prep_basis='SIC'`.

This contains information about which qubits we are constructing the quantum operation for, and what bases to prepare and measure these qubits in. The default preparation basis corresponds to preparing each qubit in each of the projectors of a symmetric informationally complete positive operator valued measurement (SIC-POVM):
$$\begin{align}
\left|S_0\right\rangle & = \left|0\right\rangle \\
\left|S_1\right\rangle & = \frac{1}{\sqrt{3}}\left|0\right\rangle
                            + \sqrt{\frac{2}{3}}\left|1\right\rangle \\
\left|S_2\right\rangle & = \frac{1}{\sqrt{3}}\left|0\right\rangle
                            + e^{-2i \pi/3}\sqrt{\frac{2}{3}}\left|1\right\rangle \\
\left|S_3\right\rangle & = \frac{1}{\sqrt{3}}\left|0\right\rangle
                            + e^{2i \pi/3}\sqrt{\frac{2}{3}}\left|1\right\rangle.
\end{align}$$
The default measurement basis corresponds to performing quantum state tomography on each input state by measuring each qubit in the $X$, $Y$, and $Z$ Pauli bases.

For $n$-qubits this gives a total of $4^n3^{n}$ circuits that must be run. Hence it is not recommended to perform process tomography for more than 2 or 3 qubits.

#### Adding preparation and measurement circuits to a Quantum Program

Once a process tomography set has been defined, the function to append the tomography circuits to a `QuantumProgram` is:
```python
tomo_labels = tomo.create_tomography_circuits(Q_program, 'name', qr, cr, process_tomo_set)
```
where
- `Q_program` is the quantum program containing the preparation circuit `'name'`.
- `'name'` is the name of the preparation circuit.
- `qr` is the quantum register in `Q_program` to be measured.
- `cr` is the classical register in `Q_program` to store the measurement results.
- `process_tomo_set` is a `process_tomography_set`.

The function output, `tomo_labels`, is a list of the strings for each of the tomography circuit names stored in the `Q_program`, which is used to execute the tomography experiments. The prepared and measured qubits will be `qr[j]`, with corresponding measurement outcomes stored in classical bits `cr[j]`, where `j` corresponds to each qubit in the process tomography set.

## Extracting process tomography measurement data

After building the process tomography circuits and executing them on an available backend we must process the output results. This is done by the function
```python
tomo_data = tomo.tomography_data(results, 'name', process_tomo_set)
```
where
- `results` is the Result object returned from execution of `process_tomo_set` circuits.
- `name` is the name of the original circuit being measured via process tomography.
- `process_tomo_set` is the tomography set used for generation of tomography data.

This returns a dictionary that stores the preparation basis, measurement basis, and measured counts, along with preparation and mesurement basis configuration corresponding to each outcome.

## Performing tomographic reconstruction

Finally, we may perform the fitting of the tomographic data to reconstruct a description of the process matrix. This is done by the function
```python
tomo.fit_tomography_data(tomo_data, method)
```
where
- `tomo_data` is the data returned from `process_tomography_data`.
- `method` is the reconstruction method to use. The default value is `method='wizard'`.

The return process matrix is a Choi matrix in the column vectorization convention. For a quantum process $\mathcal E$ this is defined as the matrix
$J(\mathcal E) = \sum_{i,j} |i\rangle\langle j | \otimes \mathcal{E}(|i\rangle\langle j |)$. By default, the trace of the returned Choi matrix will be 1, but this can be changed with optional arguments. We also note that the default reconstruction method constrains the process matrix to be a completely positive (CP) map (by fitting a positive-semidefinite Choi matrix); however, it *does not* constrain it to be a trace-preserving (TP) map.

### Reconstruction Methods
Currently only two methods are available.
- `method='wizard'` (default) uses the fast maximum liklihood reconstruction from [1] to return a postive semidefinite process matrix.
- `method='leastsq'` does not constrain the returned matrix to be positive-semidefinite.

Both of these methods are the same as for performing state tomography, as process tomography is implemented as state tomographic reconstruction of the Choi matrix.

# Process Tomography Examples

We now present two examples of 1- and 2-qubit quantum process tomography. We will consider reconstruction of a Hadamard gate and a CNOT gate.

### Setting up the Quantum Program

First we construct a quantum program with two circuits, one containing a single Hadamard gate on qubit 0, and one containing a single CNOT gate between qubits 0 and 1.

```python
Q_program = QuantumProgram()
Q_program.set_api(qx_config['APItoken'], qx_config['url']) # set the APIToken and API url

# Creating registers
qr = Q_program.create_quantum_register("qr", 2)
cr = Q_program.create_classical_register("cr", 2)

# hadamard on qubit-1 only
had = Q_program.create_circuit("had", [qr], [cr])
had.h(qr[1])

# CNOT gate with qubit 1 control, qubit 0 target (target for ibmqx4)
cnot = Q_program.create_circuit("cnot", [qr], [cr])
cnot.cx(qr[1], qr[0])
```

    C:\Users\IBM_ADMIN\Anaconda3\envs\qiskitpip\lib\site-packages\qiskit\_quantumprogram.py:707: DeprecationWarning: set_api() will be deprecated in upcoming versions (>0.5.0). Using the API object instead is recommended.
      "Using the API object instead is recommended.", DeprecationWarning)

    <qiskit.extensions.standard.cx.CnotGate at 0x77fb0b8>

## Process tomography of a Hadamard gate

We will first demonstrate a process tomography experiment for a single-qubit H gate.

### Visualization of the ideal process

The plots below show the Choi matrix for the Hadamard gate

```python
U_had = np.array([[1,1],[1,-1]])/np.sqrt(2)
# compute Choi-matrix from unitary
had_choi = outer(vectorize(U_had))
plot_state(had_choi)
```

### Simulation of process tomography experiment

Next, we build the process tomography circuits and execute them on the simulator backend

```python
# process tomography set for a quantum operation on qubit 1
had_tomo_set =  tomo.process_tomography_set([1])

# Generate process tomography preparation and measurement circuits
had_tomo_circuits = tomo.create_tomography_circuits(Q_program, 'had', qr, cr, had_tomo_set)
print('Tomography circuit labels for "had" circuit:')
for label in had_tomo_circuits:
    print(label)
```

    Tomography circuit labels for "had" circuit:
    had_prep_S0(1)_meas_X(1)
    had_prep_S0(1)_meas_Y(1)
    had_prep_S0(1)_meas_Z(1)
    had_prep_S1(1)_meas_X(1)
    had_prep_S1(1)_meas_Y(1)
    had_prep_S1(1)_meas_Z(1)
    had_prep_S2(1)_meas_X(1)
    had_prep_S2(1)_meas_Y(1)
    had_prep_S2(1)_meas_Z(1)
    had_prep_S3(1)_meas_X(1)
    had_prep_S3(1)_meas_Y(1)
    had_prep_S3(1)_meas_Z(1)

```python
backend = 'local_qasm_simulator'
shots = 1000
had_tomo_results = Q_program.execute(had_tomo_circuits, shots=shots, backend=backend)
```

Next we fit the tomographic data

```python
had_process_data = tomo.tomography_data(had_tomo_results, 'had', had_tomo_set)
had_choi_fit = tomo.fit_tomography_data(had_process_data, options={'trace':2})
print('Process Fidelity = ', state_fidelity(vectorize(U_had)/2, had_choi_fit))
plot_state(had_choi_fit)
```

### Running process tomography on a real device

Now we run tomography on the IBM Quantum Experience device.

```python
# Use the IBM Quantum Experience
backend = 'ibmqx4'
# Take 500 shots for each measurement basis
# Note: reduce this number for larger number of qubits
shots = 500
# set max credits
max_credits = 8
# Run the experiment
hadqx_tomo_results = Q_program.execute(had_tomo_circuits, backend=backend, shots=shots,
                                       max_credits=max_credits, timeout=240)
print(hadqx_tomo_results)
```

    COMPLETED

```python
hadqx_process_data = tomo.tomography_data(hadqx_tomo_results, 'had', had_tomo_set)
hadqx_choi_fit = tomo.fit_tomography_data(hadqx_process_data, options={'trace':2})
print('Process Fidelity = ', state_fidelity(vectorize(U_had)/2, hadqx_choi_fit))
plot_state(hadqx_choi_fit)
```

## Process tomography of a CNOT gate

We will demonstrate a process tomography experiment for a CNOT gate.

### Visualization of the ideal process

```python
#unitary matrix for CNOT with qubit 1 as control and qubit 0 as target.
U_cnot = np.array([[1,0,0,0],[0,1,0,0],[0,0,0,1],[0,0,1,0]])
# compute Choi-matrix from unitary
cnot_choi = outer(vectorize(U_cnot))
plot_state(cnot_choi)
```

### Simulation of process tomography experiment

Our test circuit simply consists of a quantum program of two qubits with a single gate, the CNOT gate. Note that we must include a two-qubit register to record measurement outcomes.

```python
# process tomography set for quantum operation on qubits 0 and 1
cnot_tomo_set = tomo.process_tomography_set([1, 0])

# Generate process tomography circuits
cnot_tomo_circuits = tomo.create_tomography_circuits(Q_program, 'cnot', qr, cr, cnot_tomo_set)
```

```python
backend = 'local_qasm_simulator'
shots = 1000
#cnot_tomo_results = Q_program.execute(cnot_tomo_circuits, backend=backend, shots=shots, timeout=300)
cnot_process_data = tomo.tomography_data(cnot_tomo_results, 'cnot', cnot_tomo_set)
```

Now we fit the data and plot the results

```python
cnot_choi_fit = tomo.fit_tomography_data(cnot_process_data, options={'trace':4})
print('Process Fidelity = ', state_fidelity(vectorize(U_cnot)/2, cnot_choi_fit/4))
plot_state(cnot_choi_fit)
```

### Running process tomography on a real device

We now demonstate running tomography on a real device. Note that 2-qubit process tomography requires evaluation of 144 circuits, so circuits may need to be submitted to the device in batches, and will require many credits for evaluation on the IBM Q Experience.

We will break our submission up into 8 batches of 18 circuits for submission

```python
# Use the IBM Quantum Experience
backend = 'ibmqx4'
# Take 500 shots for each measurement basis
# Note: reduce this number for larger number of qubits
shots = 500
# set max credits
max_credits = 8
# Set batch size and complete
batch_size = 18
status = Q_program.get_backend_status('ibmqx4')
if status['available'] == False or status['pending_jobs'] > 5:
    print('Warning: the selected backend appears to be busy or unavailable at present; consider choosing a different one if possible.')
for j in range(8):
    batch_circs = cnot_tomo_circuits[j*batch_size:(j+1)*batch_size]
    if j == 0:
        cnotqx_tomo_results = Q_program.execute(batch_circs, backend=backend, shots=shots,
                                                max_credits=max_credits, timeout=240)
    else:
        cnotqx_tomo_results += Q_program.execute(batch_circs, backend=backend, shots=shots,
                                                 max_credits=max_credits, timeout=240)
    msg = 'Batch %d/%d: %s' % (j+1, 8, cnotqx_tomo_results)
    print(msg)
```

    C:\Users\IBM_ADMIN\Anaconda3\envs\qiskitpip\lib\site-packages\qiskit\_quantumprogram.py:909: DeprecationWarning: get_backend_status() will be deprecated in upcoming versions (>0.5.0). Using qiskit.backends.get_backend_instance('name').status instead is recommended.
      "instead is recommended.", DeprecationWarning)

    Batch 1/8: COMPLETED
    Batch 2/8: COMPLETED
    Batch 3/8: COMPLETED
    Batch 4/8: COMPLETED
    Batch 5/8: COMPLETED
    Batch 6/8: COMPLETED
    Batch 7/8: COMPLETED

    WARNING:IBMQuantumExperience.IBMQuantumExperience:Got a 502 code response to https://quantumexperience.ng.bluemix.net/api/Jobs?access_token=meoQkBHgebsXjLkLw0U1vyjx509mb6OGBosFtAEPoxo1r6pyPlBRIW9p1Tm3Uu6U: 502 Bad Gateway: Registered endpoint failed to handle the request.

    Batch 8/8: COMPLETED

Finally, we process and fit the process tomography data

```python
cnotqx_process_data = tomo.tomography_data(cnotqx_tomo_results, 'cnot', cnot_tomo_set)
cnotqx_choi_fit = tomo.fit_tomography_data(cnotqx_process_data, options={'trace':4})
print('Process Fidelity = ', state_fidelity(vectorize(U_cnot)/2, cnotqx_choi_fit/4))
plot_state(cnotqx_choi_fit)
```

```python
%run "../../version.ipynb"
```

Version information

Please note that this tutorial is targeted to the stable version of the QISKit. The following versions of the packages are recommended:

Package	Version

QISKit	 0.4.10

IBMQuantumExperience	>= 1.8.26

numpy	>= 1.13, < 1.14

scipy	>= 0.19, < 0.20

matplotlib	>= 2.0, < 2.1

    Installed packages are as the following

    Python 3.6.2 :: Continuum Analytics, Inc.

    # packages in environment at C:\Users\IBM_ADMIN\Anaconda3\envs\QISKitPIP:
    #
    IBMQuantumExperience      1.9.0                     <pip>
    matplotlib                2.1.2                     <pip>
    numpy                     1.13.1                   py36_0
    numpy                     1.14.2                    <pip>
    numpydoc                  0.7.0            py36ha25429e_0
    qiskit                    0.4.14                    <pip>
    scipy                     1.0.1                     <pip>
    scipy                     0.19.1              np113py36_0

```python

```


```python
import IPython as IPy
IPy.display.IFrame('https://en.wikipedia.org/wiki/Conversion_between_quaternions_and_Euler_angles#Definition',width=800,height=480)
```

# Quaternion (JPL quaternion convention)
> ## $ q = [ q_0 \quad q_1  \quad q_2  \quad q_3]^T = [q_w  \quad q_x  \quad q_y  \quad q_z]^T $
> ## $ |q|^2 = q_{0}^{2} + q_{1}^{2} + q_{2}^{2} + q_{3}^{2} = q_{w}^{2} + q_{x}^{2} + q_{y}^{2} + q_{z}^{2} $

```python
import IPython as IPy
IPy.display.IFrame("https://en.wikipedia.org/wiki/Direction_cosine#Three-dimensional_Cartesian_coordinates",width=800,height=480)
```

# direction cosine & direction vector
> # direction cos(ine)
> ## a dirction vector where magnitude = 1
>> ### $
\alpha = cos(a) =
\frac{\vec{d} \cdot \hat{i}}{||\vec{d}||} \\
\quad \text{where, a is angle between axis of rotaion and i axix}\\
\beta = cos(b) =
\frac{\vec{d} \cdot \hat{j}}{||\vec{d}||} \\
\quad \text{where, b is angle between axis of rotaion and j axix}\\
\gamma = cos(c) =
\frac{\vec{d} \cdot \hat{k}}{||\vec{d}||} \\
\quad \text{where, c is angle between axis of rotaion and k axix}
$

```python
import sympy as sm
import sympy.vector
import sympy.algebras
import numpy as np
import matplotlib.pyplot as plt
%matplotlib widget

q0, q1, q2, q3 = sm.symbols('q_0 q_1 q_2 q_3')
a,b1,b2,b3 = sm.symbols('alpha beta1:4')
x,y,z = sm.symbols('x y z')
B = sm.vector.CoordSys3D('')

C = B.create_new('C','cylindrical')
S = B.create_new('S',transformation='spherical')
Q = B.orient_new_quaternion('Q',q0,q1,q2,q3)

# Direction Cosine
v = B.x*B.i + B.y*B.j + B.z*B.k
cx = v.dot(B.i)/sm.sqrt(v.dot(v)) # cos(v)_xaxis
cy = v.dot(B.j)/sm.sqrt(v.dot(v)) # cos(v)_yaxis
cz = v.dot(B.k)/sm.sqrt(v.dot(v)) # cos(v)_zaxis
# q
q0 = sm.cos(a/2)
q1 = sm.sin(a/2) * cx
q2 = sm.sin(a/2) * cy
q3 = sm.sin(a/2) * cz

q = q0**2 + q1**2 + q2**2 + q3**2
q.simplify()
```

$\displaystyle 1$

```python
sm.cos(sm.pi)

```

$\displaystyle -1$

```python
import sympy.algebras
a,b,c,d,e,f,g,h = sm.symbols('a b c d e f g h')
q0 = sm.algebras.Quaternion(1,1,1,1)
q1 = sm.algebras.Quaternion(a,b,c,d)
q2 = sm.algebras.Quaternion(e,f,g,h)
type(q1.adjoint())
```

    transpose

```python
q1.conjugate()
```

$\displaystyle a + - b i + - c j + - d k$

```python
QM = sm.Matrix([[a,-b,-c,-d],
                [b, a,-d, c],
                [c, d, a,-b],
                [d,-c, b, a]])
QM
```

$\displaystyle \left[\begin{matrix}a & - b & - c & - d\\b & a & - d & c\\c & d & a & - b\\d & - c & b & a\end{matrix}\right]$

```python
iM = sm.Matrix([[0,-1, 0, 0],
                [1, 0, 0, 0],
                [0, 0, 0,-1],
                [0, 0, 1, 0]])
jM = sm.Matrix([[0, 0,-1, 0],
                [0, 0, 0, 1],
                [1, 0, 0, 0],
                [0,-1, 0, 0]])
kM = sm.Matrix([[0, 0, 0,-1],
                [0, 0,-1, 0],
                [0, 1, 0, 0],
                [1, 0, 0, 0]])
iM*jM*kM
```

$\displaystyle \left[\begin{matrix}-1 & 0 & 0 & 0\\0 & -1 & 0 & 0\\0 & 0 & -1 & 0\\0 & 0 & 0 & -1\end{matrix}\right]$

```python
Q.base_vectors()
```

    (Q.i, Q.j, Q.k)

```python
q0.exp()
```

$\displaystyle e \cos{\left(\sqrt{3} \right)} + \frac{\sqrt{3} e \sin{\left(\sqrt{3} \right)}}{3} i + \frac{\sqrt{3} e \sin{\left(\sqrt{3} \right)}}{3} j + \frac{\sqrt{3} e \sin{\left(\sqrt{3} \right)}}{3} k$

```python
import sympy as sm
a = sm.MatrixSymbol('a',3,3)
b = sm.MatrixSymbol('b',3,3)
sm.BlockMatrix([[a,b],[b,a]]).as_explicit()
a.as_explicit()
```

$\displaystyle \left[\begin{matrix}a_{0, 0} & a_{0, 1} & a_{0, 2}\\a_{1, 0} & a_{1, 1} & a_{1, 2}\\a_{2, 0} & a_{2, 1} & a_{2, 2}\end{matrix}\right]$


# Cocco, Gomes, & Maenhout (2005)

 # "[Consumption and Portfolio Choice Over the Life Cycle](https://academic.oup.com/rfs/article-abstract/18/2/491/1599892)"

 - Notebook created by [Mateo Velásquez-Giraldo](https://github.com/Mv77) and [Matthew Zahn](https://sites.google.com/view/matthew-v-zahn/matthew-v-zahn).

This notebook uses the [Econ-ARK/HARK](https://github.com/econ-ark/hark) toolkit to describe the main results and reproduce the figures in the linked paper. The main HARK tool used here is the $\texttt{PortfolioConsumerType}$ class. For an introduction to this module, see the [HARK's Documentation](https://hark.readthedocs.io/en/latest/).

Our results using the HARK toolkit resemble those of the original paper, but we have not been able to replicate its results exactly for reasons that we have not been able to track down. We suspect there might be subtleties in the normalization or calibration that we have missed and that could be causing the discrepancies. We would be thankful if a reader (or the original authors) were able to discover the reason for the discrepancies, and could issue a pull request for a revision that reconciles them.

We thank Chris Carroll and Sylvain Catherine for comments and guidance.

```python
# This cell does some preliminary set up

# Packages
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# Import relevenat HARK tools
import HARK.ConsumptionSaving.ConsPortfolioModel as cpm

# This is a jupytext paired notebook that autogenerates BufferStockTheory.py
# which can be executed from a terminal command line via "ipython BufferStockTheory.py"
# But a terminal does not permit inline figures, so we need to test jupyter vs terminal
# Google "how can I check if code is executed in the ipython notebook"
from IPython import get_ipython # In case it was run from python instead of ipython

# If the ipython process contains 'terminal' assume not in a notebook
def in_ipynb():
    try:
        if 'terminal' in str(type(get_ipython())):
            return False
        else:
            return True
    except NameError:
        return False

# Determine whether to make the figures inline (for spyder or jupyter)
# vs whatever is the automatic setting that will apply if run from the terminal
if in_ipynb():
    # %matplotlib inline generates a syntax error when run from the shell
    # so do this instead
    get_ipython().run_line_magic('matplotlib', 'inline')
else:
    get_ipython().run_line_magic('matplotlib', 'auto')
```

### The base model

The authors' aim is to represent the life cycle of a consumer that is exposed to uninsurable labor income risk and how he chooses to allocate his savings between a risky and a safe asset, without the possibility to borrow or short-sell.

##### Dynamic problem

The problem of an agent $i$ of age $t$ in the base model is recursively represented as

\begin{split}
V_{i,t} =& \max_{0\leq C_{i,t} \leq X_{i,t}, \alpha_{i,t}\in[0,1]} U(C_{i,t}) + \delta p_t E_t\{ V_{i,t+1} (X_{i,t+1}) \}\\
&\text{s.t}\\
&X_{i,t+1} = Y_{i,t+1} + (X_{i,t} - C_{i,t})(\alpha_{i,t} R_{t+1} + (1-\alpha_{i,t})\bar{R}_f)
\end{split}

where $C_{i,t}$ is consumption, $\alpha_{i,t}$ is the share of savings allocated to the risky asset, $Y_{i,t}$ is labor income, and $X_{i,t}$ represents wealth. The utility function $U(\cdot)$ is assumed to be CRRA in the base model. Extensions beyond the baseline model include an additively separable bequest motive in the utility function. The discount factor is $\delta$ and $p_t$ is the probability of survival from $t$ to $t+1$. Death is certain at a maximum period $T$.

Note that the consumer cannot borrow or short-sell.

The control variables in the problem are $\{C_{it}, \alpha_{it}\}^T_{t=1}$ and the state variables are $\{t, X_{it}, v_{it} \}^T_{t=1}$. The agent solves for  policy rules as a function of the state variables—$C_{it}(X_{it}, v_{it})$ and $\alpha_{it}(X_{it}, v_{it})$.

#### Labor income

An important driver of the paper's results is the labor income process. It is specified as follows:

\begin{equation}
\log Y_{i,t} = f(t,Z_{i,t}) + v_{i,t} + \epsilon_{i,t}, \quad \text{for } t\leq K.
\end{equation}

where $K$ is the (exogenous) age of retirement, $Z_{i,t}$ are demographic characteristics, $\epsilon_{i,t}\sim \mathcal{N}(0,\sigma^2_\epsilon)$ is a transitory shock, and  $v_{i,t}$ is a permanent component following a random walk

\begin{equation}
v_{i,t} = v_{i,t-1} + u_{i,t} = v_{i,t-1} + \xi_t + \omega_{i,t}
\end{equation}

in which the innovation is decomposed into an aggregate ($\xi_t$) and an idiosyncratic component ($\omega_{i,t}$), both following mean-0 normal distributions.

Post-retirement income is a constant fraction $\lambda$ of permanent income in the last working year $K$.

A crucial aspect of the labor income process is that $f(\cdot,\cdot)$ is calibrated to match income profiles in the PSID, capturing the usual humped shape of income across lifetime.

#### Matching labor income in HARK

In HARK's consumption-saving models, the income process takes the form
\begin{equation}
    \ln Y_t = \ln P_t + \ln \theta_t
\end{equation}
where $P_t$ represents permanent income and $\ln \theta_t \sim N(0,\sigma_\theta)$ transitory shocks to income. Permanent income evolves according to
\begin{equation}
    \ln P_{t+1} = \ln \Gamma_{t+1} +  \ln \psi_{t+1} + \ln P_t
\end{equation}
where $\Gamma_{t+1}$ is a deterministic growth factor, and $\ln \psi_{t+1} \sim N(0,\sigma_\psi)$ a permanent income shock.

To represent the author's assumptions in HARK, we express both income processes as sums of deterministic components and i.i.d shocks
\begin{align}
\text{Cocco et. al} &\quad& \ln Y_{i,t} &=& f(t,Z_{i,t}) + v_{i,0} &+&\sum_{k=1}^t u_{i,k} &+&\qquad \varepsilon_{i,t} \\
\text{HARK}        &\quad& \ln Y_{i,t} &=& \ln P_{i,0} + \sum_{k=1}^t \ln \Gamma_k &+&\sum_{k=1}^t \ln \psi_{i,k} &+& \qquad \ln \theta_{i,t}.
\end{align}

These representations make evident the mapping that we use

|HARK | Cocco et. al |
| :---: | :-----------: |
| $\ln$ $P_{i,0}$ | $f(0,Z_{i,0})$ + $v_{i,0}$ |
| $\ln$ $\Gamma_{t+1}$| $f(t+1$, $Z_{i,t+1})$ - $f(t,Z_{i,t})$|
|$\ln$ $\psi_{i,k}$| $u_{i,k}$|
|$\ln$ $\theta_{i,t}$| $\varepsilon_{i,t}$|

and to achieve a retirement income that is equal to a fraction $\lambda$ of permanent income in the last working period $K$, we simply make $\Gamma_{K+1} = \lambda$ and $\Gamma_{t} = 1$ $\forall t>K+1$.

#### Assets and their returns

There are two assets available for consumers to allocate their savings.

- Bonds: paying a risk-free return $\bar{R}_f$.

- Stocks: paying a stochastic return $R_t = \bar{R}_f + \mu + \eta_t$, where the stochastic component $\eta_t \sim \mathcal{N}(0, \sigma^2_\eta)$ is allowed to be correlated with the aggregate labor income innovation $\xi_t$.

The borrowing and short-selling constraints ensure that agents cannot allocate negative dollars to either of these assets or borrow against future labor income or retirement wealth. Recall $\alpha_{i,t}$ is the proportion of the investor's savings that are invested in the risky asset. The model's constraints imply that $\alpha_{i,t}\in[0,1]$ and wealth is non-negative.

### Calibration—Summary

The paper defines and calibrates several parameters which can be broken down into the following categories:

__1. Preferences and life cycle__

 | Parameter | Description | Code | Value |
 |:---:| ---         | ---  | :---: |
 | $\delta$ | Time Preference Factor | $\texttt{DiscFac}$ | 0.96 |
 | $\gamma$ | Coefficient of Relative Risk Aversion| $\texttt{CRRA}$ | 10 |
 | $p_t$ | Survival Probability | $\texttt{LivPrb}$ | [0.6809,0.99845] |
 | $t_0$ | Starting Age | $\texttt{t_start}$ | 20 |
 | $t_r$ | Retirement Age | $\texttt{t_ret}$ | 65 |
 | $T$ | Maximum Age | $\texttt{t_end}$ | 100 |

__2. Income process and the finanical assets__

 | Parameter | Description | Code | Value|
 |:---:| ---         | ---  | :---: |
 | $f(t,Z_{i,t})$| Average income at each stage of life | $\texttt{det_income}$ | $ \exp($0.530339 + 0.16818 $t$+ (0.0323371/10) $t^2$ + (0.0019704/100) $t^3)$ |
 | $\lambda$ | Last Period Labor Income Share for Retirement | $\texttt{repl_fac}$ | 0.68212 |
 | $\log \Gamma$ | Permanent Income Growth Factor | $\texttt{PermGroFac}$ | $\{\log f_{t+1} - \log f_t\}^{t_r+1}_{t=20}$ |
 | $\mathsf{R}$ | Interest Factor | $\texttt{Rfree}$ | 1.02 |
 | $\mu$ | Average Stock Return | $\texttt{RiskyDstnFunc}$ \& $\texttt{RiskyDrawFunc}$ | 1.06 |
 | $\sigma_\eta$ | Std Dev of Stock Returns | $\texttt{RiskyDstnFunc}$ \& $\texttt{RiskyDrawFunc}$ | 0.157 |

__3. Shocks__

 | Parameter | Description | Code | Value |
 |:---:| ---         | ---  | :---: |
 | $\sigma_v$ | Std Dev of Log Permanent Shock| $\texttt{PermShkStd}$ | 0.102956 |
 | $\sigma_\epsilon$ | Std Dev of Log Transitory Shock| $\texttt{TranShkStd}$ | 0.27166 |

For reference, the authors' source Fortran code that includes these paramerization details is available on [Gomes' personal page](http://faculty.london.edu/fgomes/research.html). Code that solves the model is also available in [Julia](https://github.com/econ-ark/HARK/issues/114#issuecomment-371891418).

### Calibration in our implementation.

We replicate the previously defined parameter values and transition processes for state variables and shocks, and format them into the structures required by HARK.

This is done in the external file [Calibration/params.py](https://github.com/matthew-zahn/CGMPort/blob/master/CGMPort-Shared/Code/Python/Calibration/params.py), which we now import.

```python
# Calibrate the model in line with the information above
import sys,os
# The following code navigates to another directory where a python script with the parameters for the model is saved.
sys.path.append(os.path.realpath('Calibration/'))
# Loading the parameters from the Calibration/params.py script
from params import dict_portfolio, time_params, det_income, age_plot_params
```

All of the model's parameters are contained in the structure <tt>dict_portfolio<tt>, which can now be passed to HARK's <tt>PortfolioConsumerType<tt> to build and solve a representation of our problem.

```python
# Solve the model with the given parameters
agent = cpm.PortfolioConsumerType(**dict_portfolio)
agent.solve()
```

### A note on normalization

The problem as specified above makes the value function homogeneous with respect to permanent labor income. This is convenient as it allows for a re-statement of the problem in variables that are normalized by permanent income or its random components, eliminating a state variable.

The authors report (page 497) taking the normalization $v_{i,t} = 1$. This amounts to defining normalized variables $\tilde{\cdot}$ as the original variable divided by $e^{v_{i,t}-1}$. For instance:

\begin{equation}
\tilde{Y_{i,t}} = \frac{Y_{i,t}}{\exp(v_{i,t}-1)} = \frac{\exp (f(t,Z_{i,t}) + v_{i,t} + \varepsilon_{i,t})}{\exp(v_{i,t}-1)} = \exp( f(t,Z_{i,t}) + 1 + \varepsilon_{i,t} )
\end{equation}

These normalized variables have the convenient interpretation of the state that things would be in if, it weren't for permanent shocks. The author's depictions of policy functions are presented in terms of these normalized variables.

On the other hand, HARK normalizes the problem by total permanent income $P_t = \exp (f(t,Z_{i,t}) + v_{i,t})$ and its solution objects are therefore in terms of normalized variables $\hat{\cdot}$, defined as
\begin{equation}
\hat{X_{i,t}} = \frac{X_{i,t}}{P_{i,t}} = \frac{X_{i,t}}{\exp (f(t,Z_{i,t}) + v_{i,t})}.
\end{equation}

Therefore, to present our results in a way consistent with that of the original authors, we would use the following relationship
\begin{equation}
\tilde{X_{i,t}} = \hat{X_{i,t}} \times \exp (f(t,Z_{i,t})+1)
\end{equation}

However, our results are much more consistent with those of the original authors when we take the normalization $v_{i,t} = 0$, which also make sense since it makes the random-walk multiplicative part of permanent income $\exp{v_{i,t}}=1$. We therefore assume this is a typo, take $v_{i,t} = 0$, document this issue in the \textbf{Puzzles} section below, and use the relationship
\begin{equation}
\tilde{X_{i,t}} = \hat{X_{i,t}} \times \exp (f(t,Z_{i,t})).
\end{equation}

The next cell defines our normalization factor

```python
# Define a normalization factor
norm_factor = det_income*np.exp(0)
```

### Key Results

#### The optimal risky asset share

The plot below shows the policy function for the risky portfolio share as a function of wealth at different ages.

The optimal risky share is decreasing in wealth. The authors argue this is due to the fact that, at low levels of wealth, relatively safe human wealth represents a higher fraction of the consumer's wealth, so he shifts his tradeable wealth towards riskier alternatives.

Analyzing the policy rule by age also shows that the risky share increases from young to middle age, and decreases from middle to old age. This is consistent with the previous interpretation: shares trace the humped shape of labor earnings.

These estimates are different from what is produced in the original paper, which are also reproduced below. Generally, the policy functions do not share the same curvature, which leads to greater reductions in the optimal portfolio share at lower levels of wealth.

```python
# Plot portfolio rule
eevalgrid = np.linspace(0,300,100)
plt.figure()
# In the last period of life you consume everything
# so portfolio choice is irrelevant

# Ages
ages = age_plot_params
age_born = time_params['Age_born']
for a in ages:
    plt.plot(eevalgrid,
             agent.solution[a-age_born].ShareFuncAdj(eevalgrid/norm_factor[a-age_born]),
             label = 'Age = %i' %(a))
plt.xlabel('Wealth')
plt.ylabel('Risky portfolio share')
plt.title('Risky Portfolio Policy Function')
plt.legend()
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

We present the author's originally reported policy functions for comparison

#### Consumption behavior

The plot below shows the policy function for consumption as a function of wealth at different ages.

At all age levels consumption increases with wealth. The consumption function also appears to shift upwards as life progresses.

Our consumption policy functions again do not match those of the original paper, which are also reproduced below. Consumption also appears to increase with age in our policy functions that does not come through in the results presented in the paper.

```python
# Plot consumption function
plt.figure()
ages = age_plot_params
for a in ages:
    plt.plot(eevalgrid,
             agent.solution[a-age_born].cFuncAdj(eevalgrid/norm_factor[a-age_born])*norm_factor[a-age_born],
             label = 'Age = %i' %(a))
plt.xlabel('Wealth')
plt.ylabel('Consumption')
plt.title('Consumption Policy Function')
plt.legend()
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

We again present the author's originally reported policy functions for comparison

### Simulations

Using the policy functions obtained from solving the model we present a series of simulations to highlight features of the model.

We first run a few simulations to verify the quality of our calibration.

The figures below show simulated levels of permanent income and risky portfolio shares for 5 agents over their life spans. We can see the model generates a heterogeneous permanent income distribution. Interestingly, all of these agents tend to follow the same general pattern for investing in the risky asset. Early in life, all of their portfolios are invested in the risky asset. This declines as the agent ages and converges to approximately 35% once they reach retirement.

```python
# Set up simulation parameters

# Number of agents and periods in the simulation.
agent.AgentCount = 5 # Number of instances of the class to be simulated.
# Since agents can die, they are replaced by a new agent whenever they do.

# Number of periods to be simulated
agent.T_sim = 80

# Set up the variables we want to keep track of.
agent.track_vars = ['aNrmNow','cNrmNow', 'pLvlNow', 't_age', 'ShareNow','mNrmNow']

# Run the simulations
agent.initializeSim()
agent.simulate()

# Present diagnostic plots.
plt.figure()
plt.plot(agent.t_age_hist+time_params['Age_born'], agent.pLvlNow_hist,'.')
plt.xlabel('Age')
plt.ylabel('Permanent income')
plt.title('Simulated Income Paths')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)

plt.figure()
plt.plot(agent.t_age_hist+time_params['Age_born'], agent.ShareNow_hist,'.')
plt.xlabel('Age')
plt.ylabel('Risky share')
plt.title('Simulated Risky Portfolio Shares')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

#### The average life cycle patterns

We now increase the number of simulations to examine and compare the behavior of the mean values of variables of interest at different ages, conditional on survival. In each case we present the original plots from the paper for reference.

The plot below illustrates the average dynamics of permanent income, consumption, and market resources across all of the simulated agents. The plot follows the general pattern observed in the original paper. However, our results show that the agents are accumulating significantly more market resources.

```python
# Number of agents and periods in the simulation.
agent.AgentCount = 50 # Number of instances of the class to be simulated.
# Since agents can die, they are replaced by a new agent whenever they do.

# Number of periods to be simulated
agent.T_sim = 80*50

# Run the simulations
agent.initializeSim()
agent.simulate()

raw_data = {'Age': agent.t_age_hist.flatten()+time_params['Age_born'],
            'pIncome': agent.pLvlNow_hist.flatten(),
            'rShare': agent.ShareNow_hist.flatten(),
            'nrmM': agent.mNrmNow_hist.flatten(),
            'nrmC': agent.cNrmNow_hist.flatten()}

Data = pd.DataFrame(raw_data)
Data['Cons'] = Data.nrmC * Data.pIncome
Data['M'] = Data.nrmM * Data.pIncome

# Find the mean of each variable at every age
AgeMeans = Data.groupby(['Age']).mean().reset_index()
```

```python
plt.figure()
plt.plot(AgeMeans.Age, AgeMeans.pIncome,
         label = 'Income')
plt.plot(AgeMeans.Age, AgeMeans.M,
         label = 'Market resources')
plt.plot(AgeMeans.Age, AgeMeans.Cons,
         label = 'Consumption')
plt.legend()
plt.xlabel('Age')
plt.title('Variable Means Conditional on Survival')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

```python
# Find age percentiles
AgePC5 = Data.groupby(['Age']).quantile(0.05).reset_index()
AgePC95 = Data.groupby(['Age']).quantile(0.95).reset_index()

# plot till death - 1
age_1 = time_params['Age_death'] - time_params['Age_born']

plt.figure()
plt.ylim([0, 1.1])
plt.plot(AgeMeans.Age[:age_1], AgeMeans.rShare[:age_1], label = 'Mean')
plt.plot(AgePC5.Age[:age_1], AgePC5.rShare[:age_1], '--r', label='Perc. 5')
plt.plot(AgePC95.Age[:age_1], AgePC95.rShare[:age_1], '--g', label = 'Perc. 95')
plt.legend()

plt.xlabel('Age')
plt.ylabel('Risky Share')
plt.title('Risky Portfolio Share Mean Conditional on Survival')
plt.grid()

if not in_ipynb():
    plt.show(block=False)
    plt.pause(1)
else:
    plt.show(block=True)
```

### Other results in the original paper

#### The welfare implications of different allocation rules

The authors next conduct a welfare analysis of different allocation rules, including popular heuristics. The rules are presented in the next figure.

The utility cost of each policy in terms of constant consumption streams with respect to the authors calculated optimal policy function is reported in the next table.

Interestingly, the "no-income" column corresponds to the usual portfolio choice result of the optimal share being the quotient of excess returns and risk times relative risk aversion, disregarding labor income. The experiment shows this allocation produces substantial welfare losses.

#### Heterogeneity and sensitivity analysis

The authors also considered a number of extensions to the baseline model. These are summarized below along with their main conclusions.

- Labor income risk: Income risk may vary across employment sectors relative to the baseline model. The authors examine extreme cases for industries that have a large standard deviation and temporary income shocks. While some differences appear across sectors, the results are generally in line with the baseline model.
- Disastrous labor income shocks: The authors find that even a small probability of zero labor income lowers the optimal portfolio allocation in stocks, while the qualitative features of the baseline model are preserved.
- Uncertain retirement income: The authors consider two types of uncertainty for retirement income; it is stochastic and correlated with current stock market performance and allowing for disastrous labor income draws before retirement. The first extension has results essentially the same as the baseline case. The second leads to more conservative portfolio allocations but is broadly consistent with the baseline model.
- Endogenous borrowing constraints: The authors add borrowing to their model by building on credit-market imperfections. They find that the average investor borrows about \$5,000 and are in debt for most of their working life. The agents eventually pay off this debt and save for retirement. Relative to the benchmark model, the investor has put less of their money in their portfolio and arrive at retirement with substantially less wealth. These results are particularly pronounced at the lower end of the income distribution relative to the higher end. Additional details are available in the text.
- Bequest motive: The authors introduce a bequest motive into the agent's utility function (i.e., $b>0$). Young investors are more impatient and tend to save less for bequests. As the agent ages, savings increases and is strongest once the agent retires. This leads to effects on the agent's portfolio allocation. Taking a step-back however, these effects are not very large unless $b$ is large.
- Educational attainment: The authors generally find that savings are consistent across education groups. They note that for a given age, the importance of future income is increasing with education level. This implies that riskless asset holdings are larger for these households.
- Risk aversion and intertemporal substitution: Lowering the level of risk aversion in the model leads to changes in the optimal portfolio allocation and wealth accumulation. Less risk-averse investors accumulate less precautionary savings and invest more in risky assets.

### Conclusion

This article provides a dynamic model with accurate lifetime income profiles in which labor income increases risky asset holdings, as it is seen as a closer substitute of risk-free assets. It finds an optimal risky asset share that decreases in wealth and with age, after middle age. The model is also used to show that ignoring labor income for portfolio allocation can generate substantial welfare losses.

### Puzzles and Questions

- Table 4 says stock returns are $0.06$. They might mean that the equity premium $\mu$ is $0.06$.
- The authors report taking the normalization $v_{i,t} = 1$. However the ranges of their results seem more consistent with $v_{i,t} = 0$ so that $\exp (v_{i,t}) = 1$, which also makes more sense for interpretation.

### Robustness Analyses

Given the differences between our results and the original paper, we did a number of checks to ensure our model was behaving consistently with well-established theoretical results. Specifically we checked:
- For an infinitely lived agent with log normal returns, that their optimal portfolio allocation converges to the Campbell-Viceira (2002) approximation to the optimal portfolio share in Merton-Samuelson (1969) model.
- For an infinitely lived agent with no labor income that can only invest in a single risky asset, that their marginal propensity to consumer converges to the theoretical MPC of Merton-Samuelson (1969).
- For an agent facing no labor income risk, that their consumption patterns precisely match the results from a perfect foresight solution.

In all three cases, we verified that our HARK model holds up to these results. More details and specific results are available upon request.

As the HARK toolkit continues to develop, there are additional sensitivities that we can perform to further check the credibility of our results. Specifically, once human wealth is available in the $\texttt{PortfolioConsumerType}$ class, we can perform the following additional checks, which were kindly suggested by Professor Sylvain Catherine:
- Shut down the income risk and remove retirement income. The solution to this new problem are provided by Merton 1971. Basically, you capitalize future earnings as an endowment of risk free asset. Then the equity share should be such that Equity/(Wealth+NPV of Human capital) is the same as the equity share in Merton 1969.
- Adding back the permanent income risk and check if the equity share is consistent with Viceira 2001. Viceira tells you something like this: $\pi = \frac{\mu - r}{\gamma \sigma^2_s} + \left(\frac{\mu - r}{\gamma \sigma^2_s} - \beta_{HC} \right) \frac{HC}{W}$, where $\beta_{HC} = \frac{\text{Cov}(r_{HC},r_s)}{\text{Var}(r_s)}$. In the CGM problem it is easy to compute $\beta_{HC}$ because earnings follow a simple random walk. HC is the NPV of human capital, which you can approximate very well by discounting expected earnings by $r+\beta_{HC}*(rm-r)$.

### Bibliographic entry of the original article

```python
### Bibtex entry
'''
@article{10.1093/rfs/hhi017,
    author = {Cocco, João F. and Gomes, Francisco J. and Maenhout, Pascal J.},
    title = "{Consumption and Portfolio Choice over the Life Cycle}",
    journal = {The Review of Financial Studies},
    volume = {18},
    number = {2},
    pages = {491-533},
    year = {2005},
    month = {02},
    abstract = "{This article solves a realistically calibrated life cycle model of consumption and portfolio choice with non-tradable labor income and borrowing constraints. Since labor income substitutes for riskless asset holdings, the optimal share invested in equities is roughly decreasing over life. We compute a measure of the importance of human capital for investment behavior. We find that ignoring labor income generates large utility costs, while the cost of ignoring only its risk is an order of magnitude smaller, except when we allow for a disastrous labor income shock. Moreover, we study the implications of introducing endogenous borrowing constraints in this incomplete-markets setting.}",
    issn = {0893-9454},
    doi = {10.1093/rfs/hhi017},
    url = {https://doi.org/10.1093/rfs/hhi017},
    eprint = {http://oup.prod.sis.lan/rfs/article-pdf/18/2/491/24421441/hhi017.pdf},
}
'''
```

```python

```


```python
import numpy as np
import pandas as pd
import numpy.linalg as LA
```

# Solve Matrix Equation : $\large{Ax = b}$

## Example: 1 Modelling

```python
df = pd.read_csv('data/data1.csv')
df
```

      no.
      f
      x
      y
      z

      0
      1
      0.720482
      0.737451
      4.10
      0.8695

      1
      2
      0.812391
      0.884248
      3.65
      0.5750

      2
      3
      0.813530
      0.863871
      3.55
      0.5125

      3
      4
      0.532572
      0.872902
      3.85
      0.6520

      4
      5
      0.697305
      0.891665
      4.70
      0.5400

      5
      6
      0.733407
      0.880975
      0.70
      0.7745

      6
      7
      0.654187
      0.733362
      2.25
      0.7130

      7
      8
      0.709219
      0.839979
      -1.10
      0.9320

      8
      9
      0.750032
      0.917038
      -1.25
      0.7335

      9
      10
      0.713197
      0.915913
      0.90
      0.7345

      10
      11
      0.521182
      0.885547
      -0.15
      0.7685

      11
      12
      0.824820
      0.884369
      0.60
      0.7090

      12
      13
      0.770785
      0.873444
      1.25
      0.4890

      13
      14
      0.772823
      0.726831
      3.40
      0.5375

      14
      15
      0.600803
      0.638667
      1.80
      0.8985

      15
      16
      0.792857
      0.753639
      1.10
      0.5705

      16
      17
      0.800259
      0.819985
      0.80
      0.4965

      17
      18
      0.537606
      0.847556
      -0.25
      0.8340

      18
      19
      0.575679
      0.884614
      2.85
      0.7870

      19
      20
      0.821320
      0.863913
      0.75
      0.6245

      20
      21
      0.325209
      0.757005
      -0.20
      0.7630

      21
      22
      0.790763
      0.899136
      5.30
      0.4470

      22
      23
      0.803798
      0.897501
      3.45
      0.3915

      23
      24
      0.785717
      0.895834
      4.40
      0.4190

      24
      25
      0.754932
      0.808529
      3.80
      0.5265

      25
      26
      0.793982
      0.892413
      5.10
      0.4350

      26
      27
      0.789723
      0.881186
      3.40
      0.4320

      27
      28
      0.797951
      0.936163
      3.20
      0.5280

      28
      29
      0.796906
      0.934264
      2.80
      0.4445

      29
      30
      0.800149
      0.941757
      3.00
      0.4490

\begin{equation}
f=a+bx+cy+dz
\end{equation}

Normal equation are

$\sum_{}^{}{f}=na+b\sum{x}+c\sum{y}+d\sum{z}$

$\sum_{}^{}{xf}=a\sum{x}+b\sum{x^2}+c\sum{xy}+d\sum{xz}$

$\sum_{}^{}{yf}=a\sum{y}+b\sum{xy}+c\sum{y^2}+d\sum{yz}$

$\sum_{}^{}{zf}=a\sum{z}+b\sum{xz}+c\sum{zy}+d\sum{z^2}$

\begin{equation}
A=
\begin{pmatrix}
a11 & a12 & a13 &a14\\
a21 & a22 & a23 &a24\\
a31 & a32 & a33 &a34\\
a41 & a42 & a43 &a44\\
\end{pmatrix}
\end{equation}
\begin{equation}
b=
\begin{pmatrix}
b1 \\
b2 \\
b3 \\
b4\\
\end{pmatrix}
\end{equation}

```python
n=30
a11=n
a12=np.sum(df['x'])     #sum of x
a13=np.sum(df['y'])     #sum of y
a14=np.sum(df['z'])     #sum of z
a21=np.sum(df['x'])     #sum of x
a22=np.sum(df['x']**2)  #sum of x2
a23=np.sum(np.dot(df['x'],df['y']))   #sum of xy
a24=np.sum(np.dot(df['x'],df['z']))   #sum of xz
a31=np.sum(df['y'])                   #sum of y
a32=np.sum(np.dot(df['y'],df['x']))   #sum of xy
a33=np.sum(df['y']**2)                #sum of y2
a34=np.sum(np.dot(df['y'],df['z']))   #sum of yz
a41=np.sum(df['z'])                   #sum of z
a42=np.sum(np.dot(df['z'],df['x']))   #sum of xz
a43=np.sum(np.dot(df['z'],df['y']))   #sum of yz
a44=np.sum(df['z']**2)                #sum of z2
b1=np.sum(df['f'])                    #sum of f
b2=np.sum(np.dot(df['f'],df['x']))    #sum of fx
b3=np.sum(np.dot(df['f'],df['y']))    #sum of fy
b4=np.sum(np.dot(df['f'],df['z']))    #sum of fz
```

```python
A = np.array([[a11,a12,a13,a14],[a21,a22,a23,a24],[a31,a32,a33,a34],[a41,a42,a43,a44]])
b = np.array([b1,b2,b3,b4])
x = LA.solve(A,b)
x
```

    array([ 0.85507492,  0.14860913, -0.00131447, -0.41792768])

```python
print('\n a=',"{:10.3}".format(x[0]))
print('\n b=',"{:.3}".format(x[1]))
print('\n c=',"{:.3}".format(x[2]))
print('\n d=',"{:.3}".format(x[3]))
```

     a=      0.855

     b= 0.149

     c= -0.00131

     d= -0.418

```python
f_exp=[] # expected value of f
for i in range(n):
    f_exp.append(x[0]+x[1]*df.iloc[i,2]+x[2]*df.iloc[i,3]+x[3]*df.iloc[i,4])
f_exp
```

    [0.5958894003599622,
     0.7413760360958024,
     0.7645997139268501,
     0.7072466193451008,
     0.7557255547611699,
     0.6613906839806531,
     0.6631192534951209,
     0.5918408249814666,
     0.68644830702124,
     0.6830369792734646,
     0.6656950521790324,
     0.6894008781179288,
     0.7788669964212885,
     0.7339832421031306,
     0.5721125629994265,
     0.7271988454495809,
     0.7683794858588099,
     0.6328064581873758,
     0.6538812940618572,
     0.7214786253986405,
     0.6489567737789436,
     0.7949143877809594,
     0.8202981096648657,
     0.8073086989485949,
     0.7501958066178496,
     0.7991933185714644,
     0.8010132155323733,
     0.7693252220065355,
     0.8044656816412954,
     0.8034356104392159]

\begin{equation}
RMSE=\sqrt{\frac{1}{n}\sum{(f-\hat{f})^2}}\\
R^2=1-\frac{\sum{(f-\hat{f})^2}}{\sum{(f-\bar{f})^2}}
\end{equation}

```python
RMSE=np.sqrt(np.sum((df['f']-f_exp)**2)/n)
R_square=1-(np.sum(np.sum(np.sum((df['f']-f_exp)**2))/np.sum((df['f']-df['f'].mean())**2)))
```

```python
print('\n RMSE=',"{:.2}".format(RMSE))
print('\n R square=',"{:.2}".format(R_square))
```

     RMSE= 0.093

     R square= 0.36

## Example:2 Electric circuit

Calculate I1,I2,I3

Using Kirchhoff's laws in junction a,loop abdea,aegha

$I3=I1+I2$

$25.1I1+24+40.5I2=0$

$-18+40.5I2+78.25I3=0$

```python
A = np.array([[1,1,-1],[25.1,40.5,0],[0,40.5,78.25]]) #arranging I1,I2,I3
b = np.array([0,-24,18])
x = LA.solve(A,b)
x
```

    array([ 18.9867374, -12.3596817,   6.6270557])

```python
print('\n I1=',"{:.3}".format(x[0]))
print('\n I2=',"{:.3}".format(x[1]))
print('\n I3=',"{:.3}".format(x[2]))

```

     I1= 19.0

     I2= -12.4

     I3= 6.63

```python

```