```python
import numpy as np
import sympy as sp
import itertools
import matplotlib.pyplot as plt
%matplotlib notebook

T=P=L=B=rho=1
x=np.linspace(0,1,100)
y1=B*np.sin(1*np.pi*x/L)
y2=B*np.sin(2*np.pi*x/L)
y3=B*np.sin(3*np.pi*x/L)
y4=B*np.sin(4*np.pi*x/L)
y5=B*np.sin(5*np.pi*x/L)
plt.plot(x,y1,x,y2,x,y3,x,y4,x,y5)
plt.gca().legend(('n=1','n=2','n=3','n=4','n=5'))
plt.title("Y(n)")
plt.show()
plt.savefig('modals.png')
```

    <IPython.core.display.Javascript object>

```python

```


(c) Juan Gomez 2019. Thanks to Universidad EAFIT for support. This material is part of the course Introduction to Finite Element Analysis

# Elasticity in a notebook

## Introduction

This notebook sumarizes the boundary value problem (BVP) for the linearized theory of elasticity. It is assumed that the student is familiar with the fundamental concepts of stress and strains. After presenting the BVP in its stress and displacements forms the NB focuses in the principle of virtual work as an alternative representation of equlibrium. The NB concludes with a proposed homework or in-class activity which particularizes the general equations to a two-dimensional idealization. **After completing this notebook you should be able to:**

* Recognize the equations of equilibrium for a material point in an elastic medium in its stress and displacements forms.

* Recognize the equations of equlibrium for a material point in an elastic medium in its displacement form, together with conditions of the displacement field at the surface as an elliptic boundary value problem.

* Recognize the principle of virtual displacements as an alternative formulation of the equlibrium for a material point in an elastic medium.

* Recognize two-dimensional idealizations in the forms of plane strain and plane stress models.

## Equilibrium equations.

Consider the schematic representation of an elastic solid ocupying a volume $V$ bounded by the surface $S$. The volume and the surface are termed the **domain** and the **boundary** respectively. The outward vector normal to the boundary is $\widehat n$.

The solid is subjected to the action of external actions in the form of **surface tractions $t_i^{(\widehat n)}$** applied directly through the boundary and distant actions or **body forces $f_i$**. The prescribed surface tractions are applied over the part of the surface labeled $S_t$, while over the remaing part of the surface, $S_u = S - S_t$, there are prescribed displacements ${\overline u}_i$. The prescribed tractions and displacements are termed the **boundary conditions (BCs)** of the problem and for the resulting boundary value problem to be well possed $S_t$ and $S_u$ must satisfy:

\begin{align*}
S & =S_t\cup S_u\\
\varnothing & =S_t\cap S_u.
\end{align*}

**Questions:**

**For the problem shown in the figure below identify the tractions and displacements boundary conditions and specify the regions $S_t$ and $S_u$. Indicate the normal vector in each part of the surface.**

**¿ Accordying to the specified BCs is the problem well possed?**

### Governing equations

To find governing equations for the internal stresses $\sigma_{ij}$ appearing at a field point of position vector $\overrightarrow x$ we apply the laws of conservation of linear momentum and of moment of momentum over an arbitrary region of the solid. From the arbitrariness in the computational region it follows that he equilibrium equations for a field point $\overrightarrow x$ are:

\begin{align*}
& \sigma_{ij,j}+f_i=0\\
& \sigma_{ij}=\sigma_{ji}.
\end{align*}

for $\overrightarrow x$ in $V$.

There are two issues with the above equilibrium equations:

* They have infinite solutions of the form $\sigma_{ij}=\sigma_{ij}(\overrightarrow x)$.

* There are 9 unknowns but only 6 equations making the systen indeterminate.

To destroy the indeterminancy we must consider the kinematic changes in the problem and its connection to the stresses.

**Questions:**

**If the stress tensor at a point in a given reference system $x-y$ is given by the matrix**

$$
\sigma=\begin{bmatrix}\sigma_{xx}&\tau_{xy}\\\tau_{xy}&\sigma_{yy}\end{bmatrix}
$$

**find the tensor at the same point but expressed in a second coordinate system $x' - y'$ rotated by an angle $\theta$ with respect to the first one.**

## Kinematic description.

To describe the change in configuration at the **material point** level and the total change of configuration of the complete domain we represent local changes at a point in terms of the relative displacement $du_i$ as a linear transformation of differential fiber elements like:

$$
du_i=\varepsilon_{ij}dx_j+\omega_{ij}dx_j
$$

where $\varepsilon_{ij}$ and $\omega_{ij}$ are the strain tensor and the local rigid-body rotation vector respectively and given by:

$$
\varepsilon_{ij}=\frac12\left(\frac{\partial u_i}{\partial x_j}+\frac{\partial u_j}{\partial x_i}\right)
$$

and

$$
\omega_{ij}=\frac12\left(\frac{\partial u_i}{\partial x_j}-\frac{\partial u_j}{\partial x_i}\right).
$$

**Questions:**

**Using explicit notation write the particular forms of the relative displacements vector $du_i$, the strain tensor $\epsilon_{ij}$ and the rotation vector $\omega_{ij}$ for a two-dimensional idealization.**

## Stress-strain relationship (Hooke's law).

For a linear elastic material the strains can be related to the stresses in terms of material parameters $\mu$ and $\lambda$ as given by the following general form of Hooke's law:

$$
\sigma_{ij}=2\mu\varepsilon_{ij}+\lambda\varepsilon_{kk}\delta_{ij}.
$$

In this form $\mu$ and $\lambda$ are called Lamé parameters and they are material constants, completely defining the material response and changing from one material to the other.

In most engineering applictions it is more common to describe the material response in terms of Young's modulus ($E$) and shear modulus ($G\equiv\mu$). There are two alternative parameters that can be used in the description of a material, namely Poisson's ratio ($\nu$) relating direct normal strains to secondary transverse strains and the compressibility parameter ($\kappa$) describing the material response to purely volumetric changes. In fact from $E, \mu , \nu , \lambda$ and $\kappa$ there are only two material parameters which are linearly independent leaving the others as combinations from the two taken as the basis. See for instance Shames and Cozzarelli (1997).

## Equations of motion in terms of displacements.

To arrive at a solvable boundary value problem we use the strain-displacement relationship after using the definition of $\varepsilon_{ij}$ in terms of $u_i$ giving:

$$
\sigma_{ij}=\mu(u_{i,j}+u_{j,i})+\lambda u_{k,k}\delta_{ij}
$$

thus allowing us to write:

$$
\sigma_{ij,j}=\mu(u_{i,jj}+u_{j,ij})+\lambda u_{k,ki}.
$$

Substituting the above into the stress equations yields:

$$
\left(\lambda+\mu\right)u_{j,ij}+\mu u_{i,jj}+f_i=0
$$

which are the Navier equations governing the displacement field in an elastic solid. Note that the Navier equations involve equilibrium, kineamtic relationships and Hooke's law.

The boundary value problem is completed after considering the conditions at the boundary $S$ namely;

$$
t_i^{(\widehat n)}=\sigma_{ij}{\widehat n}_j
$$

for $\overset\rightharpoonup x\in S_t$ and

$$
u_i=\overline{u_i}
$$

for $\overset\rightharpoonup x\in S_u$.

**Questions:**

**Use explicit notation to write the particular form of the Navier equations in a two-dimensional idealization.**

## The prinicple of virtual work (or virtual displacements).

### Theorem

An elastic state $(\sigma_{ij},\varepsilon_{ij},u_i)$ is the unique solution to the boundary value problem given by:

$$
\sigma_{ij,j}+f_i=0
$$

for $\overset\rightharpoonup x\in V$ and the boundary conditions

$$
t_i^{(\widehat n)}=\sigma_{ij}{\widehat n}_j
$$

for $\overset\rightharpoonup x\in S_t$ and

$$
u_i=\overline{u_i}
$$

for $\overset\rightharpoonup x\in S_u$ and where $S_t$ and $S_u$ are such that

\begin{align*}
S & =S_t\cup S_u\\
\varnothing & =S_t\cap S_u
\end{align*}

If

$$
\int_{V(\overset\rightharpoonup x)}\sigma_{ij}\delta\varepsilon_{ij}\operatorname dV(\overset\rightharpoonup x)-\int_{V(\overset\rightharpoonup x)}f_i\delta u_i\operatorname d{V(\overset\rightharpoonup x)}-\int_{S_t}t_i^{(\widehat n)}\delta u_i\operatorname dS=0
$$

for a given arbitrary displacement field $\delta u_i$ such $\delta u_i=0$ in $S_u$.

### Proof:

To show the validity of the PVW write

$$
\int_{V(\overset\rightharpoonup x)}\left(\sigma_{ij,j}+f_i\right)\delta u_i\operatorname dV(\overset\rightharpoonup x)=0
$$

which is valid since $\delta u_i$ is arbitrary. Then expand and use integration by parts in the first integral to get:

$$
-\int_{V(\overset\rightharpoonup x)}\sigma_{ij}\delta u_{i,j}\operatorname dV(\overset\rightharpoonup x)+\int_{S_t}t_i^{(\widehat n)}\delta u_i\operatorname dS+\int_{V(\overset\rightharpoonup x)}f_i\delta u_i\operatorname d{V(\overset\rightharpoonup x)}=0
$$

and use the symmetry condition $\sigma_{ij} = \sigma_{ji}$ to arrive at:

$$
\int_{V(\overset\rightharpoonup x)}\sigma_{ij}\delta\varepsilon_{ij}\operatorname dV(\overset\rightharpoonup x)-\int_{S_t}t_i^{(\widehat n)}\delta u_i\operatorname dS-\int_{V(\overset\rightharpoonup x)}f_i\delta u_i\operatorname d{V(\overset\rightharpoonup x)}=0.
$$

**Questions:**

* **If the stress solutin for the wedge shown previously is given by:**

\begin{align*}
\sigma_{xx} & =SCot(\phi)\\
\sigma_{yy} & =-STan(\phi)\\
\tau_{xy} & =0.
\end{align*}

**write the particular form of the principle of virtual work.**
**Note: Propose an arbitrary function $\delta u_i$ such $\delta u_i = 0$ in $S_u$ and verify that the principle holds for the given stress solution.**

## Two-dimensional idealization

In two-dimensional idealizations (eg., plane strain and plane stress) the elastic state is defined in terms of the displacement vector $u^T=\begin{bmatrix}u&v\end{bmatrix}$ where $u$ and $v$ are the horizontal and vertical scalar components of the displacement. The kinematic relationships defining strains

$$
\varepsilon^T=\begin{bmatrix}\varepsilon_{xx}&\varepsilon_{yy}&\gamma_{xy}\end{bmatrix}
$$

in terms of displacements follows:

\begin{align*}
\varepsilon_{xx} & =\left(\frac{\partial u}{\partial x}\right)\\
\varepsilon_{yy} & =\left(\frac{\partial v}{\partial y}\right)\\
\gamma_{yy} & =\left(\frac{\partial u}{\partial y}+\frac{\partial v}{\partial x}\right)
\end{align*}

Similarly the stress tensor

$$
\sigma^T=\begin{bmatrix}\sigma_{xx}&\sigma_{yy}&\tau_{xy}\end{bmatrix}
$$

satisfies the equilibrium equations

\begin{align*}
& \frac{\partial\sigma_{xx}}{\partial x}+\frac{\displaystyle\partial\tau_{xy}}{\displaystyle\partial y}+f_x=0\\
& \frac{\displaystyle\partial\tau_{xy}}{\displaystyle\partial x}+\frac{\partial\sigma_{yy}}{\partial x}+f_y=0.\\
\end{align*}

On the other hand the constitutive relationship for plane stress can be written like

$$\left\{\sigma\right\}=\left[C\right]\left\{\varepsilon\right\}$$

where the constitutive matrix reads:

$C=\frac E{1-\nu^2}\begin{bmatrix}1&\nu&0\\\nu&1&0\\0&0&\frac{1-\nu}2\end{bmatrix}$.

## Class activity

The symmetric wedge of semi-angle $\phi$ and side $\ell$

is loaded by surface tractions of magnitude $S$ as shown in the figure. A very sensitive engineer assumes that the stress solution for this problem is given by:

\begin{align*}
\sigma_{xx} & =SCot(\phi)\\
\sigma_{yy} & =-STan(\phi)\\
\tau_{xy} & =0.
\end{align*}

* Verify that this is in fact the solution to the problem.

* Find the corresponding strain field using the inverse form of the stress-strain relationship:

\begin{align*}
\varepsilon_{xx} & =\frac1E\left(\sigma_{xx}-\nu\sigma_{yy}\right)\\
\varepsilon_{yy} & =\frac1E\left(\sigma_{yy}-\nu\sigma_{xx}\right)\\
\gamma_{xy} & =\frac{\tau_{xy}}\mu
\end{align*}

* Find the displacement field after integrating the displacement increment:

\begin{align*}
du & =\varepsilon_{xx}dx+\frac12\gamma_{xy}dy\\
dv & =\frac12\gamma_{xy}dx+\varepsilon_{yy}dy
\end{align*}

* For the case of a wedge with $\phi=45^0$ and $\ell = 1.0$ complete the missing parts in the code provided below and plot the displacement field if $S= 1.0$, $E=1.0$ and $\nu = 1/3$.

* Complete the missing parts in the code provided below and plot the internal strain energy $W=\int_{V(\overrightarrow x)}\frac12\sigma_{ij}\varepsilon_{ij}\operatorname dV(\overrightarrow x)$

## Notes:

* **For the stress field to be the solution it must satisfy the equations of equlibrium and the boundary conditions.**

* **To plot the displacement field use the plotter coded in previous notebooks.**

## Python solution

To visualize the solution use intrinsic Python interpolation tools like in previously developed examples. For that purpose create a mesh of triangular linear elements as shown in the figure. In this case the mesh is represented by the nodal and elemental files **Wnodes.txt** and **Weles.txt.** (These files must recide in memory.). Care must be taken with the location of the reference system in the mesh and that of the analytic solution.

### Import modules

```python
%matplotlib inline
import matplotlib.pyplot as plt
from matplotlib.tri import Triangulation, CubicTriInterpolator
import numpy as np
import sympy as sym
```

### Wedge solution

In the following function code the displacement and stress solution for a material point of coordinates $(x , y)$ in the wedge reference system:

```python
def cunia(x,y):
    """Computes the solution for self-equilibated wedge
       at a point (x , y).
    """
#

    ux   = 1.0
    uy   = 1.0
    sigx = 1.0
    sigy = 1.0
    return ux , uy , sigx , sigy
```

### Interpolation and visualization subroutines

Use the plotting functions **plot_disp()** and **plot_stress()** from **SolidsPy** to visualize the displacement and stress solution. Recall that these subroutines also use two auxiliary functions to handle the conversion from the mesh to Pyhton Triangularization objects.

```python
def plot_disp(UC, nodes, elements, Ngra , plt_type="contourf", levels=12,
               savefigs=False, title="Solution:" ):
    """Plots a 2D nodal displacement field using a triangulation.

    Parameters
    ----------
    UC : ndarray (float)
      Array with the displacements.
    nodes : ndarray (float)
      Array with number and nodes coordinates:
        `number coordX coordY BCX BCY`
    elements : ndarray (int)
      Array with the node number for the nodes that correspond to each
      element.

    """
    tri = mesh2tri(nodes, elements)
    tri_plot(tri, UC[:, 0] , Ngra, title=r'$u_x$',
             figtitle=title + "Horizontal displacement",
             levels=levels, plt_type=plt_type, savefigs=savefigs,
             filename="ux_sol.pdf" )
    tri_plot(tri, UC[:, 1],  Ngra , title=r'$u_y$',
             figtitle=title + "Vertical displacement",
             levels=levels, plt_type=plt_type, savefigs=savefigs,
             filename="uy_sol.pdf")
```

```python
def plot_stress(S_nodes, nodes, elements, Ngra ,  plt_type="contourf", levels=12,
               savefigs=False ):
    """Plots a 2 component stresses field using a triangulation.

    The stresses need to be computed at nodes first.

    Parameters
    ----------
    S_nodes : ndarray (float)
      Array with the nodal stresses.
    nodes : ndarray (float)
      Array with number and nodes coordinates:
        `number coordX coordY`
    elements : ndarray (int)
      Array with the node number for the nodes that correspond to each
      element.

    """
    tri = mesh2tri(nodes, elements)
    tri_plot(tri, S_nodes[:, 0], Ngra ,  title=r'$\sigma_{11}$',
             figtitle="Solution: sigma-xx stress",
             levels=levels, plt_type=plt_type, savefigs=savefigs,
             filename="sigmaxx_sol.pdf")
    tri_plot(tri, S_nodes[:, 1], Ngra ,  title=r'$\sigma_{22}$',
             figtitle="Solution: sigma-xy stress",
             levels=levels, plt_type=plt_type, savefigs=savefigs,
             filename="sigmaxy_sol.pdf")
```

```python
def mesh2tri(nodes, elements):
    """Generates a matplotlib.tri.Triangulation object from the mesh

    Parameters
    ----------
    nodes : ndarray (float)
      Array with number and nodes coordinates:
        `number coordX coordY BCX BCY`
    elements : ndarray (int)
      Array with the node number for the nodes that correspond to each
      element.

    Returns
    -------
    tri : Triangulation
        An unstructured triangular grid consisting of npoints points
        and ntri triangles.

    """
    x = nodes[:, 1]
    y = nodes[:, 2]
    triangs = []
    for el in elements:
        if el[1]==3:
            triangs.append(el[[3, 4, 5]])
            triangs.append(el[[5, 6, 3]])
        if el[1]==9:
            triangs.append(el[[3, 6, 8]])
            triangs.append(el[[6, 7, 8]])
            triangs.append(el[[6, 4, 7]])
            triangs.append(el[[7, 5, 8]])
        if el[1]==2:
            triangs.append(el[3:])

    tri = Triangulation(x, y, np.array(triangs))
#
    return tri
```

```python
def tri_plot(tri, field, Ngra ,  title="", figtitle="", levels=12, savefigs=False,
             plt_type="contourf" , filename="solution_plot.pdf"  ):

    plt.figure(Ngra)
    if plt_type=="pcolor":
        disp_plot = plt.tripcolor
    elif plt_type=="contourf":
        disp_plot = plt.tricontourf
    plt.figure(figtitle)
    disp_plot(tri, field, levels, shading="gouraud")
    plt.title(title)
    plt.colorbar(orientation='vertical')
    plt.axis("image")
    plt.grid()
```

### Main program

Complete the main program accordingly to read the mesh files and evaluate the solution at every nodal point from each element.

```python
nodes        = np.loadtxt('files/' + 'Wnodes.txt')
elements     = np.loadtxt('files/' +'Weles.txt')
nn =len(nodes[:,0])
#
coords=np.zeros([nn,2])
coords[:,0]=nodes[:,1]
coords[:,1]=nodes[:,2]
#
U   = np.zeros([nn , 2])
Sig = np.zeros([nn , 2])
```

```python
for i in range(0,nn):
    x = coords[i,0]
    y = coords[i,1]
    ux , uy , sx , sy = cunia(x , y)
    U[i , 0]   = ux
    U[i , 1]   = uy
    Sig[i , 0] = sx
    Sig[i , 1] = sy
```

### Plot the solution

```python
plot_disp(U, nodes, elements , 1 , plt_type="contourf", levels=12 )
#plot_stress(Sig, nodes, elements , 2 , savefigs = True)
```

### Glossary of terms

**Boundary value problem:** A set of partial differential equations specified over a given domain $V$ bounded by a surface or boundary $S$ where bundary conditions or prescribed characteritics of the solution are available.

**Material point:** Fundamental mathemtical abstraction in the continuum model and representing the equivalent of a particle in classical mechanics. This material point has no shape nor volume yet it experiences mechanical interactions.

**Tractions vector:** This is the fundamental description of forces introduced by Cauchy. In fact tractions represent forces per unit surface at a material point.

**Stress tensor:** The complete set of traction vectors associatted to three (two in plane problems) non co-lineal directions and completely defining the state of forces at the material point.

**Strain tensor:** This second order tensor describes the local changes in shape along the infinite directions emanating from a material point.

**Constitutive tensor:** Set of material parameters, transforming like a tensor, and fully describing the stress-strain response for a given material.

### References

* Timoshenko, S.P., and Goodier, J.N. (1976). Theory of Elasticity. International Student Edition. McGraw-Hill International.

* Love, A. E. H. (2013). A treatise on the mathematical theory of elasticity. Cambridge university press.

* Shames, I.H and Cozzarelli, F.A. (1997). Elastic and inelastic stress analysis. Taylor and Francis.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./nb_style.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>

```python

```


# Nested Logit Model: Compute Willingness To Pay Indicators

```python
import pandas as pd
import numpy as np
import biogeme.database as db
import biogeme.biogeme as bio
import biogeme.models as models
import biogeme.optimization as opt
import biogeme.results as res
from biogeme.expressions import Beta, DefineVariable, Derive
import seaborn as sns
import matplotlib.pyplot as plt
```

**Import Optima data**

```python
pandas = pd.read_csv("../../Data/6-Discrete Choice Models/optima.dat",sep='\t')
database = db.Database ("data/optima", pandas)
```

**Use collumn names as variables**

```python
globals().update(database.variables)
```

**Exclude some unwanted entries**

```python
exclude = (Choice == -1.)
database.remove(exclude)
```

**Define some dummy variables**

```python
male = (Gender == 1)
female = (Gender == 2)
unreportedGender = (Gender == -1)

fulltime = (OccupStat == 1)
notfulltime = (OccupStat != 1)
```

**Rescale some data**

```python
TimePT_scaled = TimePT / 200
TimeCar_scaled = TimeCar / 200
MarginalCostPT_scaled = MarginalCostPT / 10
CostCarCHF_scaled = CostCarCHF / 10
distance_km_scaled = distance_km / 5
```

**Compute normalizing weights for each alternative**

```python
sumWeight = database.data['Weight'].sum()
normalized_Weight = Weight * len(database.data['Weight']) / sumWeight
```

**Create parameters to be estimated**

```python
ASC_CAR              = Beta('ASC_CAR',0,None,None,0)
ASC_PT               = Beta('ASC_PT',0,None,None,1)
ASC_SM               = Beta('ASC_SM',0,None,None,0)
BETA_TIME_FULLTIME   = Beta('BETA_TIME_FULLTIME',0,None,None,0)
BETA_TIME_OTHER      = Beta('BETA_TIME_OTHER',0,None,None,0)
BETA_DIST_MALE       = Beta('BETA_DIST_MALE',0,None,None,0)
BETA_DIST_FEMALE     = Beta('BETA_DIST_FEMALE',0,None,None,0)
BETA_DIST_UNREPORTED = Beta('BETA_DIST_UNREPORTED',0,None,None,0)
BETA_COST            = Beta('BETA_COST',0,None,None,0)
```

**Define the utility functions**

\begin{align}
V_{PT} & = \beta_{PT} + \beta_{time_{fulltime}} X_{time_{PT}} X_{fulltime} + \beta_{time_{other}} X_{time_{PT}} X_{not\_fulltime} + \beta_{cost} X_{cost_{PT}} \\
V_{car} & = \beta_{car} + \beta_{time_{fulltime}} X_{time_{car}} X_{fulltime} + \beta_{time_{other}} X_{time_{car}} X_{not\_fulltime} + \beta_{cost} X_{cost_{car}} \\
V_{SM} & = \beta_{SM} + \beta_{male} X_{distance} X_{male} + \beta_{female} X_{distance} X_{female} + \beta_{unreported} X_{distance} X_{unreported}
\end{align}

```python
V_PT = ASC_PT + BETA_TIME_FULLTIME * TimePT_scaled * fulltime + \
       BETA_TIME_OTHER * TimePT_scaled * notfulltime + \
       BETA_COST * MarginalCostPT_scaled
V_CAR = ASC_CAR + \
        BETA_TIME_FULLTIME * TimeCar_scaled * fulltime + \
        BETA_TIME_OTHER * TimeCar_scaled * notfulltime + \
        BETA_COST * CostCarCHF_scaled
V_SM = ASC_SM + \
       BETA_DIST_MALE * distance_km_scaled * male + \
       BETA_DIST_FEMALE * distance_km_scaled * female + \
       BETA_DIST_UNREPORTED * distance_km_scaled * unreportedGender
```

**Associate utility functions with alternatives and associate availability of alternatives**

In this example all alternatives are available for each individual

```python
V = {0: V_PT,
     1: V_CAR,
     2: V_SM}

av = {0: 1,
      1: 1,
      2: 1}
```

**Define the nests**

1. Define the nests paramenters
2. List alternatives in nests

```python
MU_NO_CAR = Beta('MU_NO_CAR', 1.,1.,None,0)

CAR_NEST = 1., [1]
NO_CAR_NEST = MU_NO_CAR, [0, 2]

nests = CAR_NEST, NO_CAR_NEST
```

**Define the WTP**

```python
WTP_PT_TIME = Derive(V_PT,'TimePT') / Derive(V_PT,'MarginalCostPT')
WTP_CAR_TIME = Derive(V_CAR,'TimeCar') / Derive(V_CAR,'CostCarCHF')
```

**Define what we want to simulate**

```python
simulate = {'weight': normalized_Weight,
            'WTP PT time': WTP_PT_TIME,
            'WTP CAR time': WTP_CAR_TIME}
```

**Define the Biogeme object**

```python
biogeme = bio.BIOGEME(database, simulate, removeUnusedVariables=False)
biogeme.modelName = "optima_nested_logit_wtp"
```

**Retrieve the names of the variables we want to use. Then retrieve the results from the model that we estimated earlier**

```python
betas = biogeme.freeBetaNames

results = res.bioResults(pickleFile='optima_nested_logit.pickle')
betaValues = results.getBetaValues ()
```

**Perform the simulation**

```python
simulatedValues = biogeme.simulate(betaValues)
```

**Compute the WTP for the car and its confidence interval**

```python
wtpcar = (60 * simulatedValues['WTP CAR time'] * simulatedValues['weight']).mean()

b = results.getBetasForSensitivityAnalysis(betas,size=100)
left,right = biogeme.confidenceIntervals(b,0.9)
wtpcar_left = (60 * left['WTP CAR time'] * left['weight']).mean()
wtpcar_right = (60 * right['WTP CAR time'] * right['weight']).mean()
```

**Print the Time WTP for car**

```python
print(f"Average WTP for car: {wtpcar:.3g} CI:[{wtpcar_left:.3g},{wtpcar_right:.3g}]")
```

    Average WTP for car: 3.96 CI:[2.11,7.41]

**Analize all WTP values**

```python
print("Unique values: ", [f"{i:.3g}" for i in 60 * simulatedValues['WTP CAR time'].unique()])
```

    Unique values:  ['2.42', '6.69']

**Check this for groups within the population. Define a subgroup that automatically does this for us**

```python
def wtpForSubgroup(filter):
    size = filter.sum()
    sim = simulatedValues[filter]
    totalWeight = sim['weight'].sum()
    weight = sim['weight'] * size / totalWeight
    wtpcar = (60 * sim['WTP CAR time'] * weight ).mean()
    wtpcar_left = (60 * left[filter]['WTP CAR time'] * weight ).mean()
    wtpcar_right = (60 * right[filter]['WTP CAR time'] * weight ).mean()
    return wtpcar, wtpcar_left,wtpcar_right
```

**Filter: Full time workers**

```python
filter = database.data['OccupStat'] == 1
w,l,r = wtpForSubgroup(filter)
print(f"WTP car for workers: {w:.3g} CI:[{l:.3g},{r:.3g}]")
```

    WTP car for workers: 6.69 CI:[4.51,10.5]

**Filter: Females**

```python
filter = database.data['Gender'] == 2
w,l,r = wtpForSubgroup(filter)
print(f"WTP car for females: {w:.3g} CI:[{l:.3g},{r:.3g}]")
```

    WTP car for females: 3.17 CI:[1.41,6.51]

**Filter: Males**

```python
filter = database.data['Gender'] == 1
w,l,r = wtpForSubgroup(filter)
print(f"WTP car for males: {w:.3g} CI:[{l:.3g},{r:.3g}]")
```

    WTP car for males: 4.96 CI:[2.99,8.54]

**Plot the distribution of individuals and their WTP in the population**

```python
plt.hist(60*simulatedValues['WTP CAR time'],
         weights = simulatedValues['weight'])
plt.xlabel("WTP (CHF/hour)")
plt.ylabel("Individuals")
plt.show()
```

```python

```


## 2.1 Analyzing One Categorical Variable

In this lesson, we focus on a single categorical variable. For a high-level summary of a categorical variable, we can use the `.describe()` command. Note that the behavior of `.describe()` will change, depending on whether `pandas` thinks that the variable is quantitative or categorical, which is why it is important to cast categorical variables to the right type, as we discussed in the previous chapter.

```python
df_titanic["class"].describe()
```

To completely summarize a single categorical variable, we report the number of times each level appeared, or its **frequency**.

```python
class_counts = df_titanic["class"].value_counts()
class_counts
```

Notice that the levels are sorted in decreasing order of frequency by default. We can also report the levels in the order they appear in the data set...

```python
df_titanic["class"].value_counts(sort=False)
```

...or in alphabetical order, by sorting the index:

```python
class_counts.sort_index()
```

Note that this produces a _new_ `Series` with the index sorted. It does not sort the original `Series` `class_counts`. To sort the original series, we need to specify that the sorting should be done in place, just like we did for `.set_index()` in Chapter 1.

```python
class_counts.sort_index(inplace=True)
class_counts
```

Any other order would require selecting the levels manually, in the desired order.

```python
class_counts.loc[
    ["1st", "2nd", "3rd",
     "deck crew", "engineering crew", "victualling crew",
     "restaurant staff"]
]
```

This information can be visualized using a **bar chart**.

```python
class_counts.plot.bar()
```

Instead of reporting counts, we can also report proportions or probabilities, or the **relative frequencies**. We can calculate the relative frequencies by specifying `normalize=True` in `.value_counts()`.

```python
class_probs = df_titanic["class"].value_counts(normalize=True)
class_probs.sort_index()
```

This is equivalent to taking the counts and dividing by their sum.

```python
class_counts / class_counts.sum()
```

Notice that the relative frequencies add up to 1.0, by construction. We can report these relative frequencies using probability notation. For example:

$$ P(\text{1st class}) = 0.146806. $$

The complete collection of probabilities of all levels of a variable is called the **distribution** of that variable. So the code above calculates the distribution of "class" on the Titanic.

The bar chart for relative frequencies (i.e., probabilities) looks qualitatively the same as the bar chart for frequencies (i.e., counts). The only difference is that the scale on the $y$-axis is different.

```python
class_probs.sort_index().plot.bar()
```

In the next lesson, we will see why it is often more useful to plot the relative frequencies (i.e., probabilities) rather than the frequencies (i.e., counts).

### 2.1.1 Transforming Categorical Variables

A categorical variable can be transformed by mapping its levels to new levels. For example, we may only be interested in whether a person on the titanic was a passenger or a crew member. The variable `class` is too detailed. We can create a new variable, `type`, that is derived from the existing variable `class`. Observations with a `class` of "1st", "2nd", or "3rd" get a value of "passenger", while observations with a `class` of "deck crew", "engineering crew", or "deck crew" get a value of "crew".

```python
df_titanic["type"] = df_titanic["class"].map({
    "1st": "passenger",
    "2nd": "passenger",
    "3rd": "passenger",
    "victualling crew": "crew",
    "engineering crew": "crew",
    "deck crew": "crew"
})

df_titanic
```

Upon closer inspection of this `DataFrame`, we see that we accidentally left out the level "restaurant staff" in the input to `.map()`. Any levels that are unspecified will be mapped to the missing value `NaN`.

This suggests a more concise way to define the new variable `type`. We can specify only the levels for passengers in the mapping and then fill in the missing values afterwards.

```python
df_titanic["type"] = df_titanic["class"].map({
    "1st": "passenger",
    "2nd": "passenger",
    "3rd": "passenger"
})

# Replace all missing values by "crew"
df_titanic["type"].fillna("crew", inplace=True)

df_titanic
```

For more complex mappings, the `.map()` method also accepts a function. So the above

```python
def class_to_type(c):
  if c in ["1st", "2nd", "3rd"]:
    return "passenger"
  else:
    return "crew"

df_titanic["class"].map(class_to_type)
```

We can apply the techniques we learned above to calculate the _distribution_ of this new variable, which only has two levels.

```python
df_titanic["type"].value_counts(normalize=True)
```

### 2.1.2 Conditional Probabilities

What fraction of males were crew members? To answer questions like this, we have to filter the `DataFrame` to include only males.

The standard way to filter a `DataFrame` is to use a **boolean mask**. A boolean mask is simply a `Series` of booleans whose index matches the index of the `DataFrame`.

The easiest way to create a boolean mask is to use one of the standard comparison operators `==`, `<`, `>`, and `!=` on an existing column in the `DataFrame`. For example, the following code produces a boolean mask that is equal to `True` for the males on the Titanic and `False` otherwise.

```python
df_titanic["gender"] == "male"
```

Notice the subtle way the equality operator `==` is being used here! We are comparing an array with a string, i.e.,

$$\begin{align}
& \begin{bmatrix} \text{"male"} \\ \text{"male"} \\ \text{"male"} \\ \text{"female"} \\ \vdots \\ \text{"male"} \end{bmatrix} & \text{with} & & \text{"male"}.
\label{broadcast1}\tag{1}
\end{align}$$

In most programming languages, this comparison would make no sense. An array of strings is obviously not equal to a string $\eqref{broadcast1}$. However, `pandas` automatically applies the equality operator to _each_ element of the array. As a result, we get an entire array (i.e., a `Series`) of booleans.

$$\begin{align}
 \begin{bmatrix} \text{"male"} \\ \text{"male"} \\ \text{"male"} \\ \text{"female"} \\ \vdots \\ \text{"male"} \end{bmatrix} &== \text{"male"} &\Longrightarrow & &  \begin{bmatrix} \text{True} \\ \text{True} \\ \text{True} \\ \text{False} \\ \vdots \\ \text{True} \end{bmatrix}.
\label{broadcast2}\tag{2}
\end{align}$$

When an operation is applied to each element of an array, it is said to be **broadcast** over that array $\eqref{broadcast2}$.

Now, we can use the boolean mask as a filter on the `DataFrame` to extract the rows where the mask equals `True`.

```python
df_male = df_titanic[df_titanic["gender"] == "male"]
df_male
```

Note that every person in this new `DataFrame` is male. If you inspect the index, you will see that rows 4 and 5 are missing. That is because those passengers were female.

Finally, we can apply the methods we've learned in this lesson to `df_male` to answer the original question: "What fraction of males were crew members?"

```python
df_male["type"].value_counts(normalize=True)
```

It appears that about `50.4657%` of the males were crew members. We can notate this using _conditional probability_ notation:

$$ P(\text{crew} | \text{male}) = 0.504657. $$

The bar $|$ is read as "given". The information after the bar is the given information. In this case, we were interested in the probability a person was a crew member, "given" they were male. That is, after restricting to the male passengers (i.e., using a boolean mask as a filter), we want to know the relative frequency of crew members.

We can also filter on multiple criteria. For example, if we want to know the fraction of male _survivors_ who were crew members, we need to combine two boolean masks, one based on the column `gender` and another based on the column `survived`. The two masks can be combined using the logical operator `&`.

```python
(df_titanic["gender"] == "male") & (df_titanic["survived"] == 1)
```

Notice that the logical operator was _broadcast_ (that word again!) over the elements of the two `Series`. In other words, the logical operator was applied to each element, producing a `Series` of booleans.

Now we can use this new boolean mask to filter the `DataFrame`, just as we did before.

```python
df_male_survivors = df_titanic[(df_titanic["gender"] == "male") & (df_titanic["survived"] == 1)]
df_male_survivors
```

Besides $\verb|&|$, there are two other logical operators that can be used to modify and combine boolean masks.

- $\verb|&|$ means "and"
- $\verb|||$ means "or"
- $\verb|~|$ means "not"

Like $\verb|&|$, the operators $\verb|||$ and $\verb|~|$ are broadcast over the boolean masks.

### 2.1.3 Exercises

_Exercises 1-2 ask you to continue working with the Titanic data set explored in this lesson._

1\. What proportion of crew members were male? Express your answer using probability notation. How does this proportion differ from the proportion that was calculated in the lesson?

2\. What is the distribution of gender among passengers on the Titanic? What is the distribution of gender among passengers on the Titanic who survived? Express your answers using probability notation.

_Exercises 3-5 deal with the OKCupid data set, which consists of user profiles in the San Francisco Bay Area on the dating website OKCupid. This data set is available at the URL https://dlsun.github.io/pods/data/okcupid.csv._

3\. Make a visualization of the distribution of drinking status.

4\. Create a new variable that indicates whether the user has, likes, or dislikes cats. Visualize the distribution of this variable.

5\. If you were a heterosexual female interested in dating a non-smoker, how many options would you have in this data set?

TesterBook/02_Categorical_Data/.ipynb_checkpoints/2.1 Analyzing One Categorical Variable-checkpoint.ipynb

TesterBook/02_Categorical_Data/2.1 Analyzing One Categorical Variable.ipynb


# Neural Nets t3

```python
%matplotlib widget
#%matplotlib inline
%load_ext autoreload
%autoreload 2
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

```python
# import Importing_Notebooks
import numpy as np
from scipy import ndimage
import matplotlib.pyplot as plt
import dill
```

A network built of components which:
1. accept an ordered set of reals (we'll use `numpy.array`, and  call them vectors) at the input port and produce another at the output port - this is forward propagation. ${\displaystyle f\colon \mathbf {R} ^{n}\to \mathbf {R} ^{m}}$
1. accept an ordered set of reals at the output port, representing the gradient of the loss function at the output, and produce the gradient of the loss function at the input port - this is back propagation, aka backprop. ${\displaystyle b\colon \mathbf {R} ^{m}\to \mathbf {R} ^{n}}$
1. from the gradient of the loss function at the output, calculate the partial of the loss function w.r.t the internal parameters ${\displaystyle \frac{\partial E}{\partial w} }$
1. accept a scalar $\eta$ to control the adjustment of internal parameters. _Or is this effected by scaling the loss gradient before passing??_
1. update internal parameters ${\displaystyle w \leftarrow w - \eta \frac{\partial E}{\partial w} }$

```python
class Layer:
    def __init__(self):
        pass

    def __call__(self, x):
        """Computes response to input"""
        raise NotImplementedError

    def backprop(self, output_delE):
        """Uses output error gradient to adjust internal parameters, and returns gradient of error at input"""
        raise NotImplementedError
```

A network built of a cascade of layers:

```python
class Network:
    def __init__(self):
        self.layers = []
        self.eta = 0.1 #FIXME

    def extend(self, net):
        self.layers.append(net)

    def __call__(self, input):
        v = input
        for net in self.layers:
            v = net(v)
        return v

    def learn(self, facts):
        for (x, expected) in facts:
            y = self(x)
            e = y - expected
            loss = e.dot(e)/2.0
            egrad = e * self.eta
            for net in reversed(self.layers):
                egrad = net.backprop(egrad)
        return loss
```

## A Neural Net lab bench

```python
# from nnbench import NNBench
from matplotlib.widgets import Slider, Button, RadioButtons
```

```python
class NNBench:
    def __init__(self, net, ideal=lambda x:x):
        self.net = net
        self.ideal = ideal
        self.gc_protect = []
        self.seed = 3

    def checkpoint_net(self):
        self.net_checkpoint = dill.dumps(self.net)

    def rollback_net(self):
        self.net = dill.loads(self.net_checkpoint)

    def training_data_gen(self, n):
        """Generate n instances of labelled training data"""
        np.random.seed(self.seed)
        for i in range(n):
            v = np.random.randn(2)
            yield (v, self.ideal(v))

    def learn(self, n=100):
        return [self.net.learn([fact]) for fact in self.training_data_gen(n)]

    def learning_potential(self, n=100, eta=None):
        stash = dill.dumps(self.net)
        if eta is not None: # only change the net's eta if a value was passed to us
            self.net.eta = eta
        loss = self.net.learn(fact for fact in self.training_data_gen(n))
        self.net = dill.loads(stash)
        return -np.log(loss)

    def plot_learning(self, n):
        from matplotlib import pyplot as plt
        # self.losses = losses = [self.net.learn(fact for fact in self.training_data_gen(n))]
        losses = self.learn(n)
        plt.yscale('log')
        plt.plot(range(len(losses)),losses)
        plt.show(block=0)

    def knobs_plot_learning(self, n):
        pickled_net = dill.dumps(self.net)
        # from matplotlib import pyplot as plt
        fig, ax = plt.subplots()
        plt.subplots_adjust(left=0.25, bottom=0.25)
        a0 = 5
        f0 = 3

        ###
        losses = [self.net.learn([fact]) for fact in self.training_data_gen(n)]
        l, = plt.plot(range(len(losses)), losses, lw=2)
        ax.margins(x=0)
        plt.yscale('log')

        axcolor = 'lightgoldenrodyellow'
        axfreq = plt.axes([0.25, 0.1, 0.65, 0.03], facecolor=axcolor)
        axamp = plt.axes([0.25, 0.15, 0.65, 0.03], facecolor=axcolor)

        sfreq = Slider(axfreq, '⍺', 0, 1, valinit=self.net.eta)
        samp = Slider(axamp, 'Num', 1, 1000, valinit=100, valstep=1)

        filtfunc = [lambda x:x]

        big = max(losses)
        ax.set_title(f"maxloss:{big}")

        iax = plt.axes([0.025, 0.7, 0.15, 0.15])
        def make_iax_image():
            return np.concatenate([np.concatenate((l.M,np.array([l.b])),axis=0)
                                   for l in self.net.layers
                                  if hasattr(l, 'M')],axis=1)
        def update_iax(img=[iax.imshow(make_iax_image())]):
            img[0].remove()
            img[0] = iax.imshow(make_iax_image())

        def update(val,ax=ax,loc=[l]):
            n = int(samp.val)
            self.net = dill.loads(pickled_net)
            sfunc = lambda x: 2**(-1.005/(x+.005))
            self.net.eta = sfunc(sfreq.val)
            #sfreq.set_label("2.4e"%(self.net.eta,))
            losses = filtfunc[0]([self.net.learn([fact]) for fact in self.training_data_gen(n)])
            big = max(losses)
            ax.set_title(f"⍺={self.net.eta:1.3e},max loss:{big}")
            loc[0].remove()
            loc[0], = ax.plot(range(len(losses)), losses, lw=2,color='xkcd:blue')
            ax.set_xlim((0,len(losses)))
            ax.set_ylim((min(losses),big))
            update_iax()
            fig.canvas.draw_idle()

        sfreq.on_changed(update)
        samp.on_changed(update)

        resetax = plt.axes([0.8, 0.025, 0.1, 0.04])
        button = Button(resetax, 'Reset', color=axcolor, hovercolor='0.975')

        def reset(event):
            self.seed += 1
            update()
        button.on_clicked(reset)

        rax = plt.axes([0.025, 0.5, 0.15, 0.15], facecolor=axcolor)
        radio = RadioButtons(rax, ('raw', 'low pass', 'green'), active=0)

        def colorfunc(label):
            if label == "raw":
                filtfunc[0] = lambda x:x
            elif label == "low pass":
                filtfunc[0] = lambda x:ndimage.gaussian_filter(np.array(x),3)
            #l.set_color(label)
            #fig.canvas.draw_idle()
            update()
        radio.on_clicked(colorfunc)

        plt.show()
        #return 'gc protect:', update, reset, colorfunc,sfreq,samp, radio, button
        self.gc_protect.append((update, reset, colorfunc,sfreq,samp, radio, button))
```

## Useful Layers

### Identify

```python
class IdentityLayer(Layer):
    def __call__(self, x):
        return x

    def backprop(self, output_delE):
        return output_delE
```

### Affine
A layer that does an [affine transformation](https://mathworld.wolfram.com/AffineTransformation.html) aka affinity, which is the classic fully-connected layer with output offsets.

$$ \mathbf{M} \mathbf{x} + \mathbf{b} = \mathbf{y} $$
where
$$
\mathbf{x} = \sum_{j=1}^{n} x_j \mathbf{\hat{x}}_j \\
\mathbf{b} = \sum_{i=1}^{m} b_i \mathbf{\hat{y}}_i \\
\mathbf{y} = \sum_{i=1}^{m} y_i \mathbf{\hat{y}}_i
$$
and $\mathbf{M}$ can be written
$$
\begin{bmatrix}
    m_{1,1} & \dots & m_{1,n} \\
    \vdots & \ddots & \vdots \\
    m_{m,1} & \dots & m_{m,n}
\end{bmatrix} \\
$$

#### Error gradient back-propagation
$$
\begin{align}
 \frac{\partial loss}{\partial\mathbf{x}}
  = \frac{\partial loss}{\partial\mathbf{y}} \frac{\partial\mathbf{y}}{\partial\mathbf{x}}
  = \mathbf{M}\frac{\partial loss}{\partial\mathbf{y}}
\end{align}
$$
_SOLVE: Left-multiply or right-multiply?_

#### Parameter adjustment
$$
 \frac{\partial loss}{\partial\mathbf{M}}
 = \frac{\partial loss}{\partial\mathbf{y}} \frac{\partial\mathbf{y}}{\partial\mathbf{M}}
 = \frac{\partial loss}{\partial\mathbf{y}} \mathbf{x} \\
 \frac{\partial loss}{\partial\mathbf{b}}
 = \frac{\partial loss}{\partial\mathbf{y}} \frac{\partial\mathbf{y}}{\partial\mathbf{b}}
 = \frac{\partial loss}{\partial\mathbf{y}}
$$

```python
class AffinityLayer(Layer):
    """An affine transformation, which is the classic fully-connected layer with offsets"""
    def __init__(self, n, m):
        self.M = np.empty((m, n))
        self.b = np.empty(m)
        self.randomize()

    def randomize(self):
        self.M[:] = np.random.randn(*self.M.shape)
        self.b[:] = np.random.randn(*self.b.shape)

    def __call__(self, x):
        self.input = x
        self.output = self.M @ x + self.b
        return self.output

    def backprop(self, output_delE):
        input_delE = self.M @ output_delE
        self.M -= np.einsum('i,j', output_delE, self.input) # use np.outer?
        self.b -= output_delE
        return input_delE
```

### Map
Maps a scalar function on the inputs, for e.g. activation layers.

```python
class MapLayer(Layer):
    """Map a scalar function on the input taken element-wise"""
    def __init__(self, fun, dfundx):
        self.vfun = np.vectorize(fun)
        self.vdfundx = np.vectorize(dfundx)

    def __call__(self, x):
        self.input = x
        return self.vfun(x)

    def backprop(self, output_delE):
        input_delE = self.vdfundx(self.input) * output_delE
        return input_delE
```

___

## Tests

### One identity layer
See if the wheels turn:

```python
net = Network()
net.extend(IdentityLayer())
all(net(np.arange(3)) == np.arange(3))
```

    True

It does not learn, as expected:

```python
facts = [(np.arange(2*n, 2*n+2), np.arange(2*n+1, 2*n-1, -1)) for n in range(3)]
net.learn(facts)
```

    1.0

```python
net(np.arange(2,4))
```

    array([2, 3])

### One map layer

```python
net = Network()
net.extend(MapLayer(lambda x: x+1, lambda d: 1))
all(net(np.arange(3)) == np.arange(3)+1)
```

    True

It does not learn, as expected:

```python
net.learn(facts), all(net(np.arange(5)) == np.arange(5)+1), net(np.arange(2,4))
```

    (2.0, True, array([3, 4]))

### One affine layer

```python
net = Network()
net.extend(AffinityLayer(2,2))
```

```python
t = net.layers[0]
t.M, t.b
```

    (array([[-0.07555974,  0.50894321],
            [-0.79693707,  0.52115632]]),
     array([-0.30574839,  0.99669659]))

#### Can it learn the identity transformation?

```python
bench = NNBench(net)
bench.checkpoint_net()
bench.learning_potential()
```

    20.852776766237184

```python
bench.plot_learning(100)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

```python
bench.ideal = lambda v: np.array([v[1], v[0]])
bench.knobs_plot_learning(100)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

### Learn thru a map layer

This layer squares its input and divides by two:

```python
net = Network()
net.extend(AffinityLayer(2,2))

def dtanh(x):
    v = np.tanh(x)
    return (1+v)*(1-v)

net.extend(MapLayer(lambda x:x*x/2.0, lambda d:d))
#net.extend(MapLayer(np.tanh, dtanh))
bench = NNBench(net)
bench.checkpoint_net()
```

```python
net.layers[0].M, net.layers[0].b
```

    (array([[-0.32158469,  0.15113037],
            [-0.01862772,  0.48352879]]),
     array([0.76896516, 1.36624284]))

#### Can it learn difference squared?

```python
bench.ideal = lambda v: [(v[0]-v[1])**2,0]
#bench.ideal = lambda v: [(v[0]>0)*2-1,(v[0]>v[1])*2-1]
bench.learning_potential()
#bench.knobs_plot_learning(100)
```

    /opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:13: RuntimeWarning: overflow encountered in multiply
      del sys.path[0]
    /opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:19: RuntimeWarning: invalid value encountered in subtract
    /opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:20: RuntimeWarning: invalid value encountered in subtract

    nan

```python
bench.knobs_plot_learning(100)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    /opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:13: RuntimeWarning: overflow encountered in multiply
      del sys.path[0]
    /opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:19: RuntimeWarning: invalid value encountered in subtract
    /opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:20: RuntimeWarning: invalid value encountered in subtract

### add a RELU

```python
bench.net.layers = []
bench.net.extend(AffinityLayer(2,2))
leak = 0
bench.net.extend(MapLayer(lambda x: (x*(1+leak/2)+abs(x)*(1-leak/2))/2, lambda d: [leak,1][1 if d>0 else 0]))
bench.net.layers
```

    [<__main__.AffinityLayer at 0x7fbec396afd0>,
     <__main__.MapLayer at 0x7fbec3971110>]

### XOR

```python
net = Network()
net.extend(AffinityLayer(2,2))
```

```python
t = net.layers[0]
t.M, t.b
```

    (array([[ 1.14726479, -0.11022916],
            [ 0.38825041, -0.38712718]]),
     array([-0.58722031,  1.91082685]))

```python

```


```python
from collections import Counter
from typing import Tuple, List
import numpy as np
from networkx import MultiGraph
from networkx import nx
from sympy.combinatorics import Permutation
import matplotlib.pyplot as plt

# from SurfaceCodes.utilites import permlist_to_tuple

class SurfaceCodeGraph(MultiGraph):
    def __init__(self, sigma: Tuple[Tuple[int]], alpha: Tuple[Tuple[int]]):
        super().__init__()
        self.sigma = sigma  # should include singletons corresponding to fixed points
        self.alpha = alpha  # should include singletons corresponding to fixed points
        f = self.compute_phi()
        self.phi = self.permlist_to_tuple(f)
        self.node_info = self.build_node_info()  # print dictionary for [sigma, alpha, phi]

        self.code_graph = nx.MultiGraph()

        # Create black nodes for each cycle in sigma along with white nodes
        # representing "half edges" around the black nodes
        for cycle in self.sigma:
            self.code_graph.add_node(cycle, bipartite=1)
            for node in cycle:
                self.code_graph.add_node(node, bipartite=0)
                self.code_graph.add_edge(cycle, node)

        # Create black nodes for each cycle in phi along with white nodes
        # representing "half edges" around the black nodes
        for cycle in self.phi:
            self.code_graph.add_node(cycle, bipartite=1)
            for node in cycle:
                self.code_graph.add_edge(cycle, node)

        # Create nodes for each cycle in alpha then
        # glue the nodes corresponding to a the pairs
        for pair in self.alpha:
            self.code_graph.add_node(pair)
            self.code_graph = nx.contracted_nodes(self.code_graph, pair[0], pair[1], self_loops=True)
            # Now contract pair with pair[0] to make sure edges (white nodes) are labeled
            # by the pairs in alpha to keep track of the gluing from the previous step
            self.code_graph = nx.contracted_nodes(self.code_graph, pair, pair[0], self_loops=True)

    def permlist_to_tuple(self, perms):
        """
        convert list of lists to tuple of tuples in order to have two level iterables
        that are hashable for the dictionaries used later
        """
        return tuple(tuple(perm) for perm in perms)

    def compute_phi(self):
        """compute the list of lists full cyclic form of phi (faces of dessin [sigma, alpha, phi])"""
        s = Permutation(self.sigma)
        a = Permutation(self.alpha)
        f = ~(a * s)
        f = f.full_cyclic_form  # prints permutation as a list of lists including all singletons (fixed points)
        return f

    def build_node_info(self):
        count = -1

        self.sigma_dict = dict()
        for count, cycle in enumerate(self.sigma):
            self.sigma_dict[cycle] = count

        self.phi_dict = dict()
        for count, cycle in enumerate(self.phi, start=count + 1):
            self.phi_dict[cycle] = count

        self.alpha_dict = dict()
        for count, pair in enumerate(self.alpha, start=count + 1):
            self.alpha_dict[pair] = count
        return tuple([self.sigma_dict, self.alpha_dict, self.phi_dict])

    def boundary_1(self, edge):
        """
        compute boundary of a single edge given by a white node (cycle in alpha)
        """
#         if len(self.code_graph.neighbors(edge)) < 2:
#             boundary1 = []
        # else:
        boundary1 = Counter([x[1] for x in self.code_graph.edges(edge) if x[1] in self.sigma_dict])
        odd_boundaries = [x for x in boundary1 if boundary1[x] % 2]
        # [node for node in self.code_graph.neighbors(edge) if node in self.sigma_dict]
        return odd_boundaries

    def del_1(self, edges: List[Tuple[int]]):
        """
        boundary of a list of edges, i.e. an arbitrary 1-chain over Z/2Z
        """

        boundary_list = [self.boundary_1(edge) for edge in edges]
        a = Counter([y for x in boundary_list for y in x])
        boundary_list = [x[0] for x in a.items() if x[1] % 2 == 1]
        return boundary_list

    def boundary_2(self, face):
        """
        compute boundary of a single face node
        """
        # boundary2 = [node for node in self.code_graph.neighbors(face) if node in self.alpha_dict]
        boundary2 = Counter([x[1] for x in self.code_graph.edges(face) if x[1] in self.alpha_dict])
        odd_boundaries = [x for x in boundary2 if boundary2[x] % 2]
        return odd_boundaries

    def del_2(self, faces: List[Tuple[int]]):
        """
        boundary of a list of faces, i.e. an arbitrary 2-chain over Z/2Z
        """

        boundary_list = [self.boundary_2(face) for face in faces]
        a = Counter([y for x in boundary_list for y in x])
        boundary_list = [x[0] for x in a.items() if x[1] % 2 == 1]
        return boundary_list

    def coboundary_1(self, star):
        """
        compute coboundary of a single star
        """
        # coboundary = self.code_graph.neighbors(star)
        coboundary1 = Counter([x[1] for x in self.code_graph.edges(star)])
        odd_coboundaries = [x for x in coboundary1 if coboundary1[x] % 2]
        return odd_coboundaries

    def delta_1(self, stars: List[Tuple[int]]):
        """
        coboundary of a list of stars, i.e. an arbitrary 0-cochain over Z/2Z
        """
        coboundary_list = [self.coboundary_1(star) for star in stars]
        a = Counter([y for x in coboundary_list for y in x])
        coboundary_list = [x[0] for x in a.items() if x[1] % 2 == 1]
        return coboundary_list

    def coboundary_2(self, edge):
        """
        compute coboundary of a single edge given by a white node (cycle in alpha)
        """
        # coboundary2 = [node for node in self.code_graph.neighbors(edge) if node in self.phi_dict]
        coboundary2 = Counter([x[1] for x in self.code_graph.edges(edge) if x[1] in self.phi_dict])
        odd_coboundaries = [x for x in coboundary2 if coboundary2[x] % 2]
        return odd_coboundaries

    def delta_2(self, edges: List[Tuple[int]]):
        """
        coboundary of a list of edges, i.e. an arbitrary 1-cochain over Z/2Z
        given by a list of cycles in alpha
        """
        coboundary_list = [self.coboundary_2(edge) for edge in edges]
        a = Counter([y for x in coboundary_list for y in x])
        coboundary_list = [x[0] for x in a.items() if x[1] % 2 == 1]
        return coboundary_list

    def vertex_basis(self):
        self.v_basis_dict = dict()
        self.v_dict = dict()
        A = np.eye(len(self.sigma), dtype=np.uint8)
        for count, cycle in enumerate(self.sigma):
            self.v_dict[cycle] = count
            self.v_basis_dict[cycle] = A[count, :].T
        return (self.v_basis_dict)

    def edge_basis(self):
        self.e_basis_dict = dict()
        self.e_dict = dict()
        B = np.eye(len(self.alpha), dtype=np.uint8)
        for count, cycle in enumerate(self.alpha):
            self.e_dict[cycle] = count
            self.e_basis_dict[cycle] = B[count, :].T
        return (self.e_basis_dict)

    def face_basis(self):
        self.f_basis_dict = dict()
        self.f_dict = dict()
        C = np.eye(len(self.phi), dtype=np.uint8)
        for count, cycle in enumerate(self.phi):
            self.f_dict[cycle] = count
            self.f_basis_dict[cycle] = C[count, :].T
        return (self.f_basis_dict)

    def d_2(self):
        self.D2 = np.zeros(len(self.e_dict), dtype=np.uint8)
        for cycle in self.phi:
            bd = self.boundary_2(cycle)
            if bd != []:
                image = sum([self.e_basis_dict[edge] for edge in bd])
            else:
                image = np.zeros(len(self.e_dict))
            self.D2 = np.vstack((self.D2, image))
        self.D2 = np.array(self.D2[1:, :]).T
        return self.D2, self.D2.shape

    def d_1(self):
        self.D1 = np.zeros(len(self.v_dict), dtype=np.uint8)
        for cycle in self.alpha:
            bd = self.boundary_1(cycle)
            if bd != []:
                image = sum([self.v_basis_dict[vertex] for vertex in bd])
            else:
                bd = np.zeros(len(self.e_dict))
            self.D1 = np.vstack((self.D1, image))
        self.D1 = np.array(self.D1[1:, :]).T
        return self.D1, self.D1.shape

    def euler_characteristic(self):
        """
        Compute the Euler characteristic of the surface in which the graph is embedded
        """
        chi = len(self.phi) - len(self.alpha) + len(self.sigma)
        return (chi)

    def genus(self):
        """
        Compute the genus of the surface in which the graph is embedded
        """
        g = int(-(len(self.phi) - len(self.alpha) + len(self.sigma) - 2) / 2)
        return (g)

    def draw(self, node_type='', layout=''):
        """
        Draw graph with vertices, edges, and faces labeled by colored nodes and their integer indices
        corresponding to the qubit indices for the surface code
        """
        if node_type not in ['cycles', 'dict']:
            raise ValueError('node_type can be "cycles" or "dict"')
        elif layout == 'spring':
            pos = nx.spring_layout(self.code_graph)
        elif layout == 'spectral':
            pos = nx.spectral_layout(self.code_graph)
        elif layout == 'planar':
            pos = nx.planar_layout(self.code_graph)
        elif layout == 'shell':
            pos = nx.shell_layout(self.code_graph)
        elif layout == 'circular':
            pos = nx.circular_layout(self.code_graph)
        elif layout == 'spiral':
            pos = nx.spiral_layout(self.code_graph)
        elif layout == 'random':
            pos = nx.random_layout(self.code_graph)
        else:
            raise ValueError(
                "no layout defined: try one of these: " +
                "['spring','spectral','planar','shell','circular','spiral','random']")
            # white nodes
        nx.draw_networkx_nodes(self.code_graph, pos,
                               nodelist=list(self.alpha),
                               node_color='c',
                               node_size=500,
                               alpha=0.3)
        # vertex nodes
        nx.draw_networkx_nodes(self.code_graph, pos,
                               nodelist=list(self.sigma),
                               node_color='b',
                               node_size=500,
                               alpha=0.6)
        # face nodes
        nx.draw_networkx_nodes(self.code_graph, pos,
                               nodelist=list(self.phi),
                               node_color='r',
                               node_size=500,
                               alpha=0.6)
        # edges
        nx.draw_networkx_edges(self.code_graph, pos, width=1.0, alpha=0.5)

        labels = {}

        if node_type == 'cycles':
            '''
            label nodes the cycles of sigma, alpha, and phi
            '''
            for node in self.alpha_dict:
                # stuff = self.alpha_dict[node]
                labels[node] = f'$e$({node})'
            for node in self.sigma_dict:
                # something = self.sigma_dict[node]
                labels[node] = f'$v$({node})'
            for node in self.phi_dict:
                # something2 = self.phi_dict[node]
                labels[node] = f'$f$({node})'
            nx.draw_networkx_labels(self.code_graph, pos, labels, font_size=12)

        if node_type == 'dict':
            '''
            label nodes with v, e, f and indices given by node_dict corresponding to
            qubit indices of surface code
            '''

            for node in self.alpha_dict:
                # stuff = self.alpha_dict[node]
                labels[node] = f'$e$({self.alpha_dict[node]})'
            for node in self.sigma_dict:
                # something = self.sigma_dict[node]
                labels[node] = f'$v$({self.sigma_dict[node]})'
            for node in self.phi_dict:
                # something2 = self.phi_dict[node]
                labels[node] = f'$f$({self.phi_dict[node]})'
            nx.draw_networkx_labels(self.code_graph, pos, labels, font_size=12)

        # plt.axis('off')
        # plt.savefig("labels_and_colors.png") # save as png
        plt.show()  # display

```

```python
sigma = ((0,1,2), (3,4,5), (6,7,8,9))
alpha = ((0,3),(1,4),(2,6),(5,7),(8,9))
SCG = SurfaceCodeGraph(sigma, alpha)
```

```python
SCG.draw('dict', layout = 'spring')
```

```python
SCG.draw('cycles', layout = 'spring')
```

```python
SCG.node_info
```

    ({(0, 1, 2): 0, (3, 4, 5): 1, (6, 7, 8, 9): 2},
     {(0, 3): 5, (1, 4): 6, (2, 6): 7, (5, 7): 8, (8, 9): 9},
     {(0, 6, 8, 5, 1, 3, 7, 2, 4): 3, (9,): 4})

```python
SCG.genus()
```

    1

```python
SCG.face_basis()
```

    {(0, 6, 8, 5, 1, 3, 7, 2, 4): array([1, 0], dtype=uint8),
     (9,): array([0, 1], dtype=uint8)}

```python
SCG.vertex_basis()
```

    {(0, 1, 2): array([1, 0, 0], dtype=uint8),
     (3, 4, 5): array([0, 1, 0], dtype=uint8),
     (6, 7, 8, 9): array([0, 0, 1], dtype=uint8)}

```python
SCG.edge_basis()
```

    {(0, 3): array([1, 0, 0, 0, 0], dtype=uint8),
     (1, 4): array([0, 1, 0, 0, 0], dtype=uint8),
     (2, 6): array([0, 0, 1, 0, 0], dtype=uint8),
     (5, 7): array([0, 0, 0, 1, 0], dtype=uint8),
     (8, 9): array([0, 0, 0, 0, 1], dtype=uint8)}

```python
SCG.del_1([(0,3)])
```

    [(0, 1, 2), (3, 4, 5)]

```python
SCG.boundary_1((0,3))
```

    [(0, 1, 2), (3, 4, 5)]

```python
SCG.del_1([(1,4)])
```

    [(0, 1, 2), (3, 4, 5)]

```python
SCG.boundary_1((1,4))
```

    [(0, 1, 2), (3, 4, 5)]

```python
SCG.del_1([(2,6)])
```

    [(0, 1, 2), (6, 7, 8, 9)]

```python
SCG.boundary_1((2,6))
```

    [(0, 1, 2), (6, 7, 8, 9)]

```python
SCG.del_1([(5,7)])
```

    [(3, 4, 5), (6, 7, 8, 9)]

```python
SCG.boundary_1((5,7))
```

    [(3, 4, 5), (6, 7, 8, 9)]

```python
SCG.del_1([(8,9)])
```

    []

```python
SCG.boundary_1((8,9))
```

    []

```python
SCG.d_1()
```

    (array([[1, 1, 1, 0, 0],
            [1, 1, 0, 1, 1],
            [0, 0, 1, 1, 1]], dtype=uint8),
     (3, 5))

```python
SCG.d_2()
```

    (array([[0, 0],
            [0, 0],
            [0, 0],
            [0, 0],
            [1, 1]], dtype=uint8),
     (5, 2))

```python
SCG.D1
```

    array([[1, 1, 1, 0, 0],
           [1, 1, 0, 1, 1],
           [0, 0, 1, 1, 1]], dtype=uint8)

```python
SCG.D2
```

    array([[0, 0],
           [0, 0],
           [0, 0],
           [0, 0],
           [1, 1]], dtype=uint8)

```python
def rowSwap(A, i, j):
    temp = np.copy(A[i, :])
    A[i, :] = A[j, :]
    A[j, :] = temp

def colSwap(A, i, j):
    temp = np.copy(A[:, i])
    A[:, i] = A[:, j]
    A[:, j] = temp

def scaleCol(A, i, c):
    A[:, i] *= int(c) * np.ones(A.shape[0], dtype=np.int64)

def scaleRow(A, i, c):
    A[i, :] = np.array(A[i, :], dtype=np.float64) * c * np.ones(A.shape[1], dtype=np.float64)

def colCombine(A, addTo, scaleCol, scaleAmt):
    A[:, addTo] += scaleAmt * A[:, scaleCol]

def rowCombine(A, addTo, scaleRow, scaleAmt):
    A[addTo, :] += scaleAmt * A[scaleRow, :]
```

```python
def simultaneousReduce(A, B):
    if A.shape[1] != B.shape[0]:
        raise Exception("Matrices have the wrong shape.")

    numRows, numCols = A.shape

    i, j = 0, 0
    while True:
        if i >= numRows or j >= numCols:
            break

        if A[i, j] == 0:
            nonzeroCol = j
            while nonzeroCol < numCols and A[i, nonzeroCol] == 0:
                nonzeroCol += 1

            if nonzeroCol == numCols:
                i += 1
                continue

            colSwap(A, j, nonzeroCol)
            rowSwap(B, j, nonzeroCol)

        pivot = A[i, j]
        scaleCol(A, j, 1.0 / pivot)
        scaleRow(B, j, 1.0 / pivot)

        for otherCol in range(0, numCols):
            if otherCol == j:
                continue
            if A[i, otherCol] != 0:
                scaleAmt = -A[i, otherCol]
                colCombine(A, otherCol, j, scaleAmt)
                rowCombine(B, j, otherCol, -scaleAmt)

        i += 1;
        j += 1

    return A%2, B%2

def finishRowReducing(B):
    numRows, numCols = B.shape

    i, j = 0, 0
    while True:
        if i >= numRows or j >= numCols:
            break

        if B[i, j] == 0:
            nonzeroRow = i
            while nonzeroRow < numRows and B[nonzeroRow, j] == 0:
                nonzeroRow += 1

            if nonzeroRow == numRows:
                j += 1
                continue

            rowSwap(B, i, nonzeroRow)

        pivot = B[i, j]
        scaleRow(B, i, 1.0 / pivot)

        for otherRow in range(0, numRows):
            if otherRow == i:
                continue
            if B[otherRow, j] != 0:
                scaleAmt = -B[otherRow, j]
                rowCombine(B, otherRow, i, scaleAmt)

        i += 1;
        j += 1

    return B%2

def numPivotCols(A):
    z = np.zeros(A.shape[0])
    return [np.all(A[:, j] == z) for j in range(A.shape[1])].count(False)

def numPivotRows(A):
    z = np.zeros(A.shape[1])
    return [np.all(A[i, :] == z) for i in range(A.shape[0])].count(False)

def bettiNumber(d_k, d_kplus1):
    A, B = np.copy(d_k), np.copy(d_kplus1)
    simultaneousReduce(A, B)
    finishRowReducing(B)

    dimKChains = A.shape[1]
    print("dim 1-chains:",dimKChains)
    kernelDim = dimKChains - numPivotCols(A)
    print("dim ker d_1:",kernelDim)
    imageDim = numPivotRows(B)
    print("dim im d_2:",imageDim)

    return "dim homology:",kernelDim - imageDim

```

```python
simultaneousReduce(SCG.D1.astype('float64'), SCG.D2.astype('float64'))
```

    (array([[1., 0., 0., 0., 0.],
            [0., 1., 0., 0., 0.],
            [1., 1., 0., 0., 0.]]),
     array([[0., 0.],
            [1., 1.],
            [1., 1.],
            [0., 0.],
            [1., 1.]]))

```python
finishRowReducing(SCG.D2.astype('float64'))
```

    array([[1., 1.],
           [0., 0.],
           [0., 0.],
           [0., 0.],
           [0., 0.]])

```python
numPivotCols(SCG.D2.astype('float64'))
```

    2

```python
numPivotRows(SCG.D2.astype('float64'))
```

    1

```python
bettiNumber(SCG.D1.astype('float64'), SCG.D2.astype('float64'))
```

    dim 1-chains: 5
    dim ker d_1: 2
    dim im d_2: 1

    ('dim homology:', 1)


# Eigen-Portfolio Construction
### Francisco A. Ibanez

## Part 1: Data
The data set is composed of closing prices of 11 ETFs available for trading in US exchanges. We chose using these securities over single stocks due to their high liquidity and accessibility, making this exercise easy to replicate by retail investors. In terms of asset classes, the set covers Equities, Rates, Credit and Commodities. The data is pulled from Yahoo Finance, and all these funds are quoted in US Dollars.

Given that these prices are in total return space, we first must subtract the cash component from their return. For this, we use the daily risk-free rate published on **Ken French's website**. In days for which we do not have a risk-free rate, we use the most recent value for the missing entry.

Given the length of the ETF data we are working with, the data set consists of observations extending from 4/30/2007 to 11/30/2021. Given that these ETFs are passive and benchmarked against an index, we might be able to extend their length by using the corresponding benchmark to backfill the data set in a future iteration. For now, we are restricting ourselves to use what is available.

Additionally, and for the sake of simplicity, we are dropping from the set rows (dates) with missing observations. In future work these missing observations will be estimated through other methods such as Markov Chain Monte Carlo.

```python
import yfinance as yf
import pandas as pd
import pandas_datareader.data as web
from datetime import datetime
import numpy as np
import matplotlib.pyplot as plt
import itertools
import matplotlib.pyplot as plt
from matplotlib import cm
import matplotlib.ticker as mtick
import seaborn as sns

# ETF prices
meta = pd.read_csv('etfs.csv', index_col=0)
collect = []
for n in meta.index:
    ticker = yf.Ticker(n)
    px = ticker.history(period="max")['Close']
    px.name = n
    collect += [px]

prices = pd.concat(collect, axis=1)

# Risk free rate (Fama-French)
ff = web.DataReader(
    'F-F_Research_Data_Factors_daily',
    'famafrench',
    start='12/31/1999'
)
rf = ff[0]['RF'] * .01  # Originally expressed in percentage points

# Subtract the risk-free rate
total_return = prices.pct_change().dropna()
excess_return = total_return.sub(rf.reindex(total_return.index, method='ffill'), axis=0)

# Summary of the data set
data_summary = pd.DataFrame(index=meta.index)
data_summary['Name'] = meta['name']
data_summary['Asset class'] = meta['asset']
data_summary['Region'] = meta['region']
data_summary['Starting date'] = prices.apply(pd.Series.first_valid_index).sort_values()
data_summary['Ending date'] = prices.apply(pd.Series.last_valid_index).sort_values()
data_summary['Data points'] = prices.count()

data_summary
```

      Name
      Asset class
      Region
      Starting date
      Ending date
      Data points

      ticker

      IVV
      iShares Core S&P 500
      Equities
      United States
      2000-05-19
      2022-02-25
      5478

      EZU
      iShares MSCI Eurozone
      Equities
      Europe
      2000-07-31
      2022-02-25
      5429

      EWU
      iShares MSCI United Kingdom
      Equities
      Europe
      1996-03-18
      2022-02-25
      6532

      ILF
      iShares Latin America 40
      Equities
      Emerging Markets
      2001-10-26
      2022-02-25
      5119

      EWJ
      iShares MSCI Japan
      Equities
      Japan
      1996-03-18
      2022-02-25
      6532

      FXI
      iShares China Large-Cap
      Equities
      Emerging Markets
      2004-10-08
      2022-02-25
      4377

      IAU
      iShares Gold
      Commodities
      Precious Metals
      2005-01-28
      2022-02-25
      4300

      SLV
      iShares Silver
      Commodities
      Precious Metals
      2006-04-28
      2022-02-25
      3986

      IEF
      iShares 7-10 Year Treasury
      Rates
      United States
      2002-07-30
      2022-02-25
      4930

      TIP
      iShares TIPS
      Rates
      United States
      2003-12-05
      2022-02-25
      4588

      LQD
      iShares iBoxx $ Investment Grade
      Credit
      United States
      2002-07-30
      2022-02-25
      4930

```python
# excess_return.columns = [meta.loc[x, 'name'] for x in excess_return]
meta
excess_return.to_csv('C:\Users\franc\Google Drive\bin\cuadernos\retornos_etf.csv')

```

## Part 2: Portfolio Diversification Framework
### 2.1 Measuring Diversification
From the correlation matrix below, we can draw the following observations:
1. Correlation between the equity related ETF, both US and non-US centric, is very high
2. Correlation between the bond related ETF (Rates and Credit) is very high
3. Credit is decently correlated to Equities

```python
# Correlation matrix
R = excess_return.corr()
fig, ax = plt.subplots(figsize=(10, 10))
plt.title('Correlation Matrix')
sns.heatmap(R, linewidths=.5, cmap='coolwarm', vmax=1, vmin=-1, square=True, ax=ax, cbar_kws={'shrink': .5})
plt.show()
```

Despite the large and apparently diversified set of investable assets, it is natural to raise the question of how many bets this set contains, given the notorious correlation in the set. To shed some light on this question, we can diagonalize the risk model ($\Sigma$) by decomposing it into its eigen-values and eigen-vectors.

\begin{align}
\Sigma v & = \lambda v \\
\Sigma V & = V \Lambda \\
\Sigma & = V\Lambda V^{-1}
\end{align}

$$
\Lambda = \begin{bmatrix}
\lambda_{1} & 0 & \dots & 0 \\
0 & \lambda_{2} & \dots & 0 \\
\vdots & \vdots & \ddots & \vdots\\
0 & 0 & \dots & \lambda_{N} \\
\end{bmatrix}_{N \times N}  , \enspace
\mathbf{V} = \begin{bmatrix}
\mid & \mid &  & \mid \\
v_{1} & v_{2} & \dots & v_{N} \\
\mid & \mid &  & \mid \\
\end{bmatrix}_{N \times N}
$$

where the eigen-value $\lambda_{i}$ is the risk associated to the eigen-vector $v_{i}$. Because $trace(\Sigma) = trace(\Lambda)$, the percentage of "total risk" of this set of securities explained by the principal component $i$ will be given by:

$$
\frac{\lambda_{i}}{trace(\Lambda)}
$$

```python
# We diagonalize Sigma into its eigen-vector and eigen-value matrices
Sigma = excess_return.cov().values # Variance-Covariance matrix
Lambda, V = np.linalg.eig(Sigma)
order = Lambda.argsort()[::-1]
Lambda = Lambda[order]
V = V[:, order]

# Percentage of "total risk" explained by each principal component
pct_of_risk = np.divide(Lambda, np.sum(Lambda))
pct_of_risk = pd.Series(pct_of_risk, index=[f'PC {str(x + 1)}' for x in range(len(V))])
pct_of_risk.plot.bar(figsize=(12, 5), title='Percentage of Total Risk Explained by each Principal Components')
plt.show()
```

The results below indicate that about **60% of the total risk in this set is coming just from the first principal component, while the first four principal components already explain more than 90% of the total variance**. Despite sourcing risk from different asset classes and across different global regions, we (often) arrive to very concentrated investment portfolios. This is due to the common misconception in the investment industry that diversification is coming from variety in names and labels (i.e. different securities, different markets, different asset class) instead of having an homogeneous risk loadings on the statistical factors (whatever they are) driving the returns of this set. The aim of successful portfolio construction should be to provide the right linear transformation (i.e. weights vector) that reduces the concentration of portfolio risk, in factor terms, within a given set of constraints.

For this reason, Meucci (2009) provides a framework for portfolio diversification in factor terms by decomposing the risk model into its eigen-values and eigen-vectors and using the latter to project "label weights" ($w$) into "factor weights" ($\tilde{w}$).

\begin{align}
\sigma_{p} & = w^{T}\Sigma w \\
\sigma_{p} & = w^{T} V\Lambda V^{-1} w \\
\tilde{w} & = V^{-1} w
\end{align}

Using this "factor weights" vector, the author introduces a *diversification distribution vector* ($p_{n}$) which contains the percentage of portfolio risk ($\sigma_{p}$) that can be explained by each eigen-vector. The entries of the resulting a vector of elements are always positive (for positive semi-definite matrices) and add up to 1 (due to the normalization), reason for which these can be seen as probabilty masses.

\begin{align}
p_{n} & \equiv \frac{\tilde{w}^{2}_{n}\lambda_n}{w^{T}\Sigma w}   , \enspace n = 1, \dots , N.\\
\end{align}

Then, the author states that to achieve a well-diversified portfolio the probability masses $p_{n}$ must be approximately equal and thus the diversification distribution should be close to uniform. The dispersion of these probabilities can be summarized by the exponential of the Shannon entropy, which the authors call the *effective number of bets* $\eta (w)$.

\begin{align}
\eta (w) & \equiv \exp \left( -\sum^{N}_{n=1} p_{n} \ln{(p_{n})} \right)
\end{align}

Finally, diversification is said to reach its maximum when the *diversification distribution vector* is completely uniform, where $\eta (w) = n$. The opposite is true when $\eta(w)=1$.

### 2.2 Spectral Risk Parity
Meucci (2009) introduces the notion of maximizing the effective number of bets, $\eta(w)$, to achieve well balanced portfolios by computing

\begin{align}
\arg \max_{w} \eta(w)
\end{align}

which is formalized in Meucci et al. (2013) where the authors provide a closed form solution for it. In their derivation, the authors use non-normalized asset returns and calculate the covariance matrix as risk model. The caveat of this practice is that the resulting eigen-vectors (and consequently the resulting factor weights, $\tilde{w}$) will be heavily tilted towards high-variance assets, undermining diversification in a multi-asset framework, where different asset classes provide different levels of risk. Instead of using the covariance matrix ($\Sigma$) as risk model, we propose using the correlation matrix ($R$) instead, which would be equivalent to estimating the covariance matrix using standardized returns (dividing the observations by their sample volatility). We propose the following decomposition:

\begin{align}
S &= \begin{bmatrix}
\sigma_{1} & 0 & \dots & 0 \\
0 & \sigma_{2} & \dots & 0 \\
\vdots & \vdots & \ddots & \vdots\\
0 & 0 & \dots & \sigma_{N} \\
\end{bmatrix}_{N \times N}
\end{align}

which allows us to go from the covariance matrix to the correlation matrix, and perform the following change of basis:
\begin{align}
\Sigma &= S R S \\
\Sigma &= S E L E^{-1}S \\
\end{align}

where $E$ and $L$ are the eigen-vectors and eige-values of $R$, respectively. Consequently, the portfolio variance will be given by:

\begin{align}
w^{T} \Sigma w &= w^{T} S R S w \\
&= w^{T} S E L E^{-1} S w \\
\end{align}

Using the expression above we can arrive to a closed form solution, which is mathematically similar to and preserves the spirit of the results of Meucci et al. (2013), but works on a standardized basis instead (because of the use of the correlation matrix). The **Spectral Risk Parity** (SRP) weights can be obtained by equalizing the risk contribution coming from each eigen-vector (i.e. **Spectral Risk Contribution**), which is equivalent to:

$$
\ell_{j} \left( \sum^{N}_{i=1} w_{i}\varepsilon_{i, j} \sigma_{i}\right)^{2} = c \\
\forall j \in \{1, \dots, N\}
$$

where $\ell_{j}$ and $\varepsilon_{i, j}$ are entries corresponding to the $j$-th eigenvalue and eigen-vector, $w_{i}$ and $\sigma_{i}$ are the entries in the weights vector and the $S$ matrix corresponding to asset $i$, and $c$ is just a given constant. The choice of the constant will depend on the leverage and volatility targets specified in the investment policy, for which the weights can be normalized after the calculation.

In matrix terms, the expression above can be laid out as

\begin{align}
\theta &= \boldsymbol{1}_{N} \\[10pt]
\end{align}

where the non-normalized spectral risk contribution $\theta$ is defined as

\begin{align}
\theta &= \begin{bmatrix}
\ell_{1} \left( \sum^{N}_{i=1} w_{i}\varepsilon_{i, 1} \sigma_{1}\right)^{2} \\
\ell_{2} \left( \sum^{N}_{i=1} w_{i}\varepsilon_{i, 2} \sigma_{2}\right)^{2} \\
\vdots \\
\ell_{N} \left( \sum^{N}_{i=1} w_{i}\varepsilon_{i, N} \sigma_{N} \right)^{2}  \\
\end{bmatrix}
\end{align}

and $1_{N}$ is a N-by-1 vector of ones. The previous can be summarized in matrix form:

$$
\theta = diag\left(E^{T} S w\right) L E^{T} S w\\
$$

Finally, the Spectral Risk Parity portfolio target weights ($w_{SRP}$) will be a function of the target leverage or risk target of the portfolio:

\begin{align}
diag\left(E^{T} S w\right) L E^{T} S w  &= \boldsymbol{1}_{N}\\
L^{\frac{1}{2}}E^{T} S w &= \boldsymbol{1}_{N} \\
w_{SRP} &\propto S^{-1} E L^{-\frac{1}{2}} \boldsymbol{1}_{N}
\end{align}

### 2.3 Case Study: Diversification in Example Portfolios
Just to illustrate the framework above, we will calculate the effective number of bets for the SRP portfolio and three other naive portfolio construction methodologies:

- Equal-Weighted Portfolio ($w_{equally}$)
- Inverse Volatility ($w_{inversevol}$)
- Unconstrained Mean-Variance Optimization ($w_{mvo}$)

where

\begin{align}
w_{equally} & = 1_{N} n^{-1} \\
w_{inversevol} & \propto 1_{N} \sigma^{-1} \\
w_{mvo} & \propto \Sigma^{-1} \mu
\end{align}

```python
# Correlation matrix
Rho = excess_return.corr().values
S = np.diag(excess_return.std())

# We diagonalize Rho into its eigen-vector and eigen-value matrices
Lambda, V = np.linalg.eig(Rho)
order = Lambda.argsort()[::-1]
Lambda = np.diag(Lambda[order])
V = V[:, order]

# Spectral risk parity (SRP)
n = Lambda.shape[0]  # Number of elements
w_esrc = np.linalg.inv(S) @ V @ np.power(np.linalg.inv(Lambda), 1/2) @ np.ones(n).reshape(-1, 1)
w_esrc = w_esrc / w_esrc.sum()

# Benchmark methodologies
w_equally = np.array([1 / n] * n).reshape(-1, 1)
w_inversevol = (1 / np.diag(S)).reshape(-1, 1)
w_inversevol /= w_inversevol.sum()
Sigma = excess_return.cov().values
mu = excess_return.mean().values.reshape(-1, 1)
w_mvo = mu.T @ np.linalg.inv(Sigma)
w_mvo = (w_mvo / w_mvo.sum()).reshape(-1, 1)

# Calculate the diversification distribution vector (p_n) for each
p_equally = np.multiply(V.T @ S @ w_equally, Lambda @ V.T @ S @ w_equally) / (w_equally.T @ Sigma @ w_equally)
p_inversevol = np.multiply(V.T @ S @ w_inversevol, Lambda @ V.T @ S @ w_inversevol) / (w_inversevol.T @ Sigma @ w_inversevol)
p_mvo = np.multiply(V.T @ S @ w_mvo, Lambda @ V.T @ S @ w_mvo) / (w_mvo.T @ Sigma @ w_mvo)
p_esrc = (np.diag((V.T @ S @ w_esrc).flatten()) @ Lambda @ V.T @ S @ w_esrc) / (w_esrc.T @ S @ Rho @ S @ w_esrc)

# Finally we calculate the effective number of bets for each portfolio
enb_equally = np.exp(-np.sum(np.multiply(p_equally, np.log(p_equally))))
enb_inversevol = np.exp(-np.sum(np.multiply(p_inversevol, np.log(p_inversevol))))
enb_mvo = np.exp(-np.sum(np.multiply(p_mvo, np.log(p_mvo))))
enb_srp = np.exp(-np.sum(np.multiply(p_esrc, np.log(p_esrc))))

port_name = ['Equal-Weighted', 'Inverse Volatility', 'Mean-Variance Optimization', 'Spectral Risk Parity']
enb_all = dict(zip(port_name, [enb_equally, enb_inversevol, enb_mvo, enb_srp]))

# Let's visualize the results
theta = pd.DataFrame(np.concatenate([p_equally, p_inversevol, p_mvo, p_esrc], axis=1), columns=port_name, index=data_summary['Name'])
fig, axs = plt.subplots(nrows=2, ncols=2, figsize=(10, 10))

color_map = list(cm.get_cmap('tab20')(range(20))) + list(cm.get_cmap('Set3')(range(11)))
for i, loc in enumerate(itertools.product(range(2), range(2))):
    label = port_name[i]
    axs[loc].pie(
        theta[label],
        normalize=True,
        startangle=90,
        counterclock=False,
        wedgeprops=dict(width=0.5),
        colors=color_map
    )
    axs[loc].set_title(label, fontsize=12.5)
    eta = round(enb_all[label], 1)
    axs[loc].annotate(f'$\eta$ = {eta}', xy=(-0.225, 0), fontsize=15)

fig.legend([f'Eigen-vector {x + 1}' for x in range(n)], bbox_to_anchor=(1.25, .6), ncol=1, title='Risk contributed by eigenvectors')
fig.suptitle('Sepectral Risk Conbtribution and Concentration', fontsize=15)
plt.tight_layout()
plt.show()
```

Using the entire sample for the parameter estimation, we can observe that the equal-weighted and Inverse Volatility portfolio construction methodologies result in very concentrated portfolios, which risk is mostly drive by the first eigen-vector. The previous is summarized by the effective number of bets, $\eta(w)$, which does not exceed 2. In terms of interpretability, this is equivalent to say that, despite investing in 11 securities, those two portfolios are equivalent to hold less than two independent bets. The case of the unconstrained Mean-Variance Optimization is slightly better, by leveraging the marginal Sharpe ratio of each asset (i.e., taking the covariances into account) it is able to increase the effective number of bets to slightly above 3. Even though this might sound as good news, we must remember that this is a purely in-sample and illustrative exercise. As it was expected, the linear transformation provided by the Spectral Risk Parity can provide that maximum diversification case were $\eta(w)=N$.

As it can be seen below, both Mean-Variance Optimization and Spectral Risk Parity constructions are able to achieve higher diversification by introducing short-sells, which might not be always easily achievable in practice.

```python
weights = pd.DataFrame(np.concatenate([w_equally, w_inversevol, w_mvo, w_esrc], axis=1), columns=port_name, index=data_summary['Name'])

colors = zip(port_name, ['royalblue'])
fig, axs = plt.subplots(nrows=1, ncols=4, figsize=(12.5, 5), sharey=True)
for i, loc in enumerate(range(4)):
    label = port_name[i]
    axs[loc].barh(
        width=weights[label].values[::-1],
        y=range(weights.shape[0]),
        zorder=100,
        color='dodgerblue'
    )
    axs[loc].set_title(label, fontsize=12.5)
    axs[loc].xaxis.set_major_formatter(mtick.PercentFormatter(xmax=1, decimals=0))
    axs[loc].grid(ls='--', zorder=-100)
    axs[loc].axvline(0, color='black', zorder=150, lw=.75)

axs[2].set_yticks(range(weights.shape[0]))
axs[2].set_yticklabels(data_summary['Name'][::-1])
axs[2].set_ylim(-1, weights.shape[0])
fig.suptitle('Resulting Portfolio Weights', fontsize=15)
plt.tight_layout()
```

## Part 3: Implementation
### 3.1 Challenges in the Implementation of SRP
The implementation of SRP is not free to challenges. Meucci et al. (2013) noted that, for a given risk model $\Sigma$, there are $2^{N-1}$ weights vectors that can yield the same level of diversification where $\eta(w)=N$. This is due to the fact that the sign of each eigen-vector is irrelevant when projecting these back into weights space. They suggest trying different combinations of eigen-vector signs based on different desired portfolio results, such as maximum Sharpe ratio and global minimum variance. Similarly, and for the sake of simplicity, we will select the set of eigen-vector signs that yields the portfolio with minimum volatility within the sets of $2^{N-1}$ possible portfolios.

Furthermore we propose the formalization of the excercise as follows. The sets of portfolios $W_{N \times 2^{N-1}}$ that spans all the possible linear transformation that result in $\eta(w)=N$ is given by

\begin{align}
W \propto S^{-1}E L^{-\frac{1}{2}}J
\end{align}

where

\begin{align}
J = \begin{bmatrix}
\mid & \mid &  & \mid \\
j_{1} & j_{2} & \dots & j_{2^{N-1}} \\
\mid & \mid &  & \mid \\
\end{bmatrix}_{N \times 2^{N-1}}
\end{align}

and each $j_{i}$ is a vector with entries 1 and -1 that change the sign of each eigenvector, and together all $j$ span every possible combination of signs. Consequently, the portfolio variances will be given by the $2^{N-1}$ diagonal terms in

$$
W^{T}\Sigma W
$$

Finally, we choose the $n$-th $j$ vector that generates then lowest diagonal entry in the expression above.

Other challenges, that include numerical issues in estimation of the eigenvectors and their stability, will not be studied in this simple implementation but will be addressed in future iterations.

### 3.2 Simplified Backtest
#### 3.2.1 Specification
The case-study seen in 2.3 was a completely in-sample exercise that had the objective of illustrating how the diversification measures are estimated. Now we can conduct a simple empirical analysis to simulate the historical performance of the SRP portfolio construction approach against other benchmark methodologies shown in 2.3. The specification of the backtest is the following:
- **Start date**: 12/31/2009
- **End date**: 2021/11/30
- **Data frequency**: Daily
- **Rebalance frequency**: Monthly (end of month)
- **Estimation window**: 2-year
- **Volatility target (Ex-ante)**: 10%
- **Transaction costs**: 0bp

**Notes**
1. The estimation window is updated on each rebalance, so there is no forward-looking bias
2. The volatility target is in place to make the results of the different portfolio construction methodologies directly comparable
3. The classes and functions used in this exercise are stored in **utils.py** located in this folder. Please refer to that file if more specific details are needed.

```python
from utils import PortfolioRebalance, effective_bets

bt_start = datetime(2009, 12, 31)
bt_end = datetime(2021, 11, 30)
bt_window = pd.DateOffset(years=2)
bt_rebalance_freq = 'M'
rebalance_dates = pd.date_range(start=bt_start, end=bt_end, freq=bt_rebalance_freq)
vol_targeting = .1 * np.sqrt(1 / 252)
portfolio_labels = ['Equal-Weighted', 'Unconstrained Mean-Variance', 'Inverse Volatility', 'Spectral Risk Parity']

# Portfolio weights generation
weights_equalweights = pd.DataFrame(index=rebalance_dates, columns=excess_return.columns)
weights_inversevol = pd.DataFrame(index=rebalance_dates, columns=excess_return.columns)
weights_maxsharpe = pd.DataFrame(index=rebalance_dates, columns=excess_return.columns)
weights_spectral = pd.DataFrame(index=rebalance_dates, columns=excess_return.columns)
enb = pd.DataFrame(index=rebalance_dates, columns=portfolio_labels)
for dt in rebalance_dates:
    sample_t = excess_return.loc[dt - bt_window:dt]
    port = PortfolioRebalance(sample=sample_t, volatility_target=vol_targeting)
    # Equal-weight portfolio
    port.get_equalweight()
    weights_equalweights.loc[dt] = port.weights
    enb.loc[dt, portfolio_labels[0]] = effective_bets(weights=port.weights.values.reshape(-1, 1), cov=port.cov)

    # Maximum Sharpe ratio portfolio
    port.get_maxsharpe()
    weights_maxsharpe.loc[dt] = port.weights
    enb.loc[dt, portfolio_labels[1]] = effective_bets(weights=port.weights.values.reshape(-1, 1), cov=port.cov)

    # Inverse volatility portfolio
    port.get_inversevol()
    weights_inversevol.loc[dt] = port.weights
    enb.loc[dt, portfolio_labels[2]] = effective_bets(weights=port.weights.values.reshape(-1, 1), cov=port.cov)

    # Spectral risk parity portfolio
    port.get_spectral(criteria='MinVol')
    weights_spectral.loc[dt] = port.weights
    enb.loc[dt, portfolio_labels[3]] = effective_bets(weights=port.weights.values.reshape(-1, 1), cov=port.cov)

# Portfolio returns calculation
weekly_returns = excess_return.add(1).groupby(pd.Grouper(freq=bt_rebalance_freq)).prod().sub(1)
returns_equalweights = weights_equalweights.shift(1).mul(weekly_returns).reindex(rebalance_dates).sum(axis=1)
returns_maxsharpe = weights_maxsharpe.shift(1).mul(weekly_returns).reindex(rebalance_dates).sum(axis=1)
returns_inversevol = weights_inversevol.shift(1).mul(weekly_returns).reindex(rebalance_dates).sum(axis=1)
returns_spectral = weights_spectral.shift(1).mul(weekly_returns).reindex(rebalance_dates).sum(axis=1)

bt_results = pd.concat([returns_equalweights, returns_maxsharpe, returns_inversevol, returns_spectral], axis=1)
bt_results.columns = portfolio_labels
bt_results.iloc[0] = np.nan
```

#### 3.2.2 Results

```python
bt_results_levels = bt_results.fillna(0).add(1).cumprod()
bt_dd = bt_results_levels.div(bt_results_levels.expanding().max()).sub(1)

# Stats
mu_ws = bt_results.add(1).prod().sub(1).mul(12 / bt_results.shape[0])
sigma_ws = bt_results.std().mul(np.sqrt(12))
sharpe_ratios = mu_ws / sigma_ws
max_dd = bt_dd.min()
res = pd.concat([mu_ws, sigma_ws, sharpe_ratios, max_dd], axis=1).round(2)
res.columns = ['Mean Excess-Return (Ann.)', 'Volatility (Ann.)', 'Sharpe Ratio', 'Max. Drawdown']
res
```

      Mean Excess-Return (Ann.)
      Volatility (Ann.)
      Sharpe Ratio
      Max. Drawdown

      Equal-Weighted
      0.03
      0.09
      0.31
      -0.19

      Unconstrained Mean-Variance
      0.20
      0.11
      1.71
      -0.25

      Inverse Volatility
      0.07
      0.10
      0.74
      -0.17

      Spectral Risk Parity
      0.23
      0.10
      2.25
      -0.13

In the studied sample, Spectral Risk Parity was able to deliver the highest risk-adjusted return (given by the Sharpe ratio), while exhibiting the lowest maximum drawdown. As expected, the maximum drawdown delivered by the Unconstrained Mean-Variance were the maximum in the set of methodologies, given how noisy the ex-ante estimate of the expected return can be.

As it can be seen in the plots below, the delivered performance of the SRP construction methodology seems more stable over time. This also gets translated in drawdowns that rarely go beyond one standard deviation, as opposed to the other three methodologies. In terms of the effective number of bets, the portfolio construction methodologies that ignore the correlation structure (i.e., Equal-Weighted and Inverse Volatility) moves around 2, while the unconstrained Mean-Variance portfolios gave around between 3 and 6 effective independent bets over the sample. As expected, the SRP portfolio gave constant 11 independent bets during the exercise.

```python
# Time-series plots
fig, axs = plt.subplots(nrows=3, ncols=1, sharex=True, figsize=(10, 10))
axs[0].plot(bt_results_levels.mul(10).apply(np.log10), zorder=100)
axs[1].plot(bt_dd, zorder=100)
axs[2].plot(enb, zorder=100)
titles = ['Excess Return Levels (Log-Scale)', 'Rolling Drawdowns', 'Effective Number of Bets ($\eta$)']
axs[1].yaxis.set_major_formatter(mtick.PercentFormatter(1, decimals=0))
axs[2].set_yticks(range(1, excess_return.shape[1] + 1))
for i, ax in enumerate(axs):
    ax.set_title(titles[i], fontsize=12.5)
    ax.grid(ls='--', zorder=-100)
fig.legend(bt_results_levels.columns, bbox_to_anchor=(.85, 0), ncol=4)
fig.tight_layout()
```

```python
color_map = list(cm.get_cmap('tab20')(range(11)))
fig, axs = plt.subplots(nrows=2, ncols=2, sharex=True, figsize=(12.5, 7.5))
fig.suptitle('Simulated Portfolio Allocation')
weights_equalweights.plot.bar(stacked=True, ax=axs[0, 0], width=1, color=color_map, legend=False, title=bt_results.columns[0], zorder=100)
weights_maxsharpe.plot.bar(stacked=True, ax=axs[0, 1], width=1, color=color_map, legend=False, title=bt_results.columns[1], zorder=100)
weights_inversevol.plot.bar(stacked=True, ax=axs[1, 0], width=1, color=color_map, legend=False, title=bt_results.columns[2], zorder=100)
weights_spectral.plot.bar(stacked=True, ax=axs[1, 1], width=1, color=color_map, legend=False, title=bt_results.columns[3], zorder=100)
year_end = [dt for dt in rebalance_dates if dt.month == 12]

for row in axs:
    for ax in row:
        ax.grid(ls='--', zorder=-100)
        ax.set_xticks([i for i, dt in enumerate(rebalance_dates) if dt.month == 12])
        ax.set_xticklabels([dt.strftime('%Y') for dt in year_end])
        ax.yaxis.set_major_formatter(mtick.PercentFormatter(1, decimals=0))

fig.tight_layout()
fig.legend(data_summary.reindex(weights_equalweights.columns)['Name'], bbox_to_anchor=(.95, 0), ncol=4)
plt.show()
```

### 3.3 Next steps
To expand on and deepen this analysis, the suggested research outline is:
1. Work out a way of achieving a more reliable and stable spectral decomposition to provide better results by introducing eigen-vector truncation or regularization (this would also speed up the calculation time), which would solve the $2^{N-1}$ problem of $J$ presented in **3.1**
2. Introduce leverage and short-selling constraints to increase the real-life applications of the methodology
3. Increase the size of the investible universe
4. Expand the spectrum of risk by including elements in $\Sigma$ beyond investible securities, to achieve a more diversified portfolio
5. Study how I can use the previous item, so the framework can also suggest which uninvested securities can be added to the portfolio to improve diversification
6. Explore the Singular Value Decomposition alternative to the eigen-decomposition to incorporate the time-series dimensionality of the data
7. Provide a more close-to-reality backtest, including more assets, real-life constraints, and transaction costs

```python
RxW = weights_spectral.shift(1).mul(weekly_returns).reindex(rebalance_dates)
nav = returns_spectral.add(1).cumprod().mul(100)
pnl = RxW.mul(nav.shift(1), axis=0)

fig, axs = plt.subplots(nrows=1, ncols=1, sharex=True, figsize=(15, 10))
pnl[:'2013'].cumsum().plot.bar(stacked=True, ax=axs)
```

```python
RxW = weights_maxsharpe.shift(1).mul(weekly_returns).reindex(rebalance_dates)
nav = returns_maxsharpe.add(1).cumprod().mul(100)
pnl = RxW.mul(nav.shift(1), axis=0)

fig, axs = plt.subplots(nrows=1, ncols=1, sharex=True, figsize=(15, 10))
pnl[:'2013'].cumsum().plot.bar(stacked=True, ax=axs)
```