# Tutorial-IllinoisGRMHD: outer_boundaries.C

## Authors: Leo Werneck & Zach Etienne

**This module is currently under development**

## In this tutorial module we explain the outer boundary conditions imposed on the quantities evolved within `IllinoisGRMHD`

### Required and recommended citations:

* **(Required)** Etienne, Z. B., Paschalidis, V., Haas R., Mösta P., and Shapiro, S. L. IllinoisGRMHD: an open-source, user-friendly GRMHD code for dynamical spacetimes. Class. Quantum Grav. 32 (2015) 175009. ([arxiv:1501.07276](http://arxiv.org/abs/1501.07276)).
* **(Required)** Noble, S. C., Gammie, C. F., McKinney, J. C., Del Zanna, L. Primitive Variable Solvers for Conservative General Relativistic Magnetohydrodynamics. Astrophysical Journal, 641, 626 (2006) ([astro-ph/0512420](https://arxiv.org/abs/astro-ph/0512420)).
* **(Recommended)** Del Zanna, L., Bucciantini N., Londrillo, P. An efficient shock-capturing central-type scheme for multidimensional relativistic flows - II. Magnetohydrodynamics. A&A 400 (2) 397-413 (2003). DOI: 10.1051/0004-6361:20021641 ([astro-ph/0210618](https://arxiv.org/abs/astro-ph/0210618)).

# Table of Contents
$$\label{toc}$$

This module is organized as follows

0. [Step 0](#src_dir): **Source directory creation**
1. [Step 1](#introduction): **Introduction**
1. [Step 2](#outer_boundaries__c): **`outer_boundaries.C`**
    1. [Step 2.a](#outer_boundaries__amu): *The vector potential variables*
        1. [Step 2.a.i](#outer_boundaries__amu__linear_extrapolation): Defining the linear extrapolation operators
        1. [Step 2.a.ii](#outer_boundaries__amu__applying_bcs): Applying outer boundary conditions to $A_{\mu}$
    1. [Step 2.b](#outer_boundaries__hydro_vars): *The hydrodynamic variables*
        1. [Step 2.b.i](#outer_boundaries__hydro_vars__zero_deriv_outflow): Defining the zero derivative, outflow operators
        1. [Step 2.b.ii](#outer_boundaries__hydro_vars__applying_bcs): Applying boundary conditions to $\left\{P,\rho_{b},v^{i}\right\}$
    1. [Step 2.c](#outer_boundaries__conservatives): *The conservative variables*
1. [Step 3](#code_validation): **Code validation**
1. [Step 4](#latex_pdf_output): **Output this notebook to $\LaTeX$-formatted PDF file**

# Step 0: Source directory creation \[Back to [top](#toc)\]
$$\label{src_dir}$$

We will now use the [cmdline_helper.py NRPy+ module](Tutorial-Tutorial-cmdline_helper.ipynb) to create the source directory within the `IllinoisGRMHD` NRPy+ directory, if it does not exist yet.

```python
# Step 0: Creation of the IllinoisGRMHD source directory
# Step 0a: Add NRPy's directory to the path
# https://stackoverflow.com/questions/16780014/import-file-from-parent-directory
import os,sys
nrpy_dir_path = os.path.join("..","..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)

# Step 0b: Load up cmdline_helper and create the directory
import cmdline_helper as cmd
IGM_src_dir_path = os.path.join("..","src")
cmd.mkdir(IGM_src_dir_path)

# Step 0c: Create the output file path
outfile_path__outer_boundaries__C = os.path.join(IGM_src_dir_path,"outer_boundaries.C")
```

# Step 1: Introduction \[Back to [top](#toc)\]
$$\label{introduction}$$

# Step 2: `outer_boundaries.C` \[Back to [top](#toc)\]
$$\label{outer_boundaries__c}$$

The strategy used to set outer boundary for the primitives, $\left\{P,\rho_{b},v^{i}\right\}$, and for the scalar and vector potentials, $\left\{\left[\sqrt{\gamma}\Phi\right],A_{i}\right\}$, follows eqs. (39) and (40) of the [original release paper of IllinoisGRMHD](https://arxiv.org/pdf/1501.07276.pdf). For example, if we are trying to apply boundary condition along the $x$-direction, we would have

$$
\boxed{
E_{i+1}
=
\left\{
\begin{align}
E_{i}\ , &{\rm\ if\ } E\in\left\{P,\rho_{b},v^{y},v^{z}\right\},{\rm\ or\ } E=v^{x}\ {\rm and\ } v^{x}\geq0\\
0\ ,     &{\rm\ if\ } E=v^{x}\ {\rm and\ } v^{x}<0\\
2E_{i} - E_{i-1}\ , &{\rm\ if\ } E\in\left\{\left[\sqrt{\gamma}\Phi\right],A_{x},A_{y},A_{z}\right\}
\end{align}
\right.
}\ ,
$$

for the ghostzone points along the *positive* $x$-direction, and

$$
\boxed{
E_{i-1}
=
\left\{
\begin{align}
E_{i}\ , &{\rm\ if\ } E\in\left\{P,\rho_{b},v^{y},v^{z}\right\},{\rm\ or\ } E=v^{x}\ {\rm and\ } v^{x}\geq0\\
0\ ,     &{\rm\ if\ } E=v^{x}\ {\rm and\ } v^{x}<0\\
2E_{i} - E_{i+1}\ , &{\rm\ if\ } E\in\left\{\left[\sqrt{\gamma}\Phi\right],A_{x},A_{y},A_{z}\right\}
\end{align}
\right.
}\ ,
$$

for the ghostzone points along the *negative* $x$-direction.In this way, linear extrapolation outer boundary conditions are applied to the vector potential variables $\left\{\left[\sqrt{\gamma}\Phi\right],A_{i}\right\}$ and zero-derivative, outflow outer boundary conditions are applied to the hydrodynamic variables $\left\{P,\rho_{b},v^{i}\right\}$.

## Step 2.a: The vector potential variables \[Back to [top](#toc)\]
$$\label{outer_boundaries__amu}$$

### Step 2.a.i: Defining the linear extrapolation operators \[Back to [top](#toc)\]
$$\label{outer_boundaries__amu__linear_extrapolation}$$

We start by applying outer boundary conditions to $\left\{\left[\sqrt{\gamma}\Phi\right],A_{i}\right\}$. We follow the prescription described above:

$$
\boxed{
\begin{align}
\text{Positive direction: }E_{i+1} = 2E_{i} - E_{i-1}\ , &{\rm\ if\ } E\in\left\{\left[\sqrt{\gamma}\Phi\right],A_{x},A_{y},A_{z}\right\}\\
\text{Negative direction: }E_{i-1} = 2E_{i} - E_{i+1}\ , &{\rm\ if\ } E\in\left\{\left[\sqrt{\gamma}\Phi\right],A_{x},A_{y},A_{z}\right\}
\end{align}
}\ ,
$$

which uses a linear extrapolation outer boundary condition.

```python
%%writefile $outfile_path__outer_boundaries__C
/*******************************************************
 * Outer boundaries are handled as follows:
 * (-1) Update RHS quantities, leave RHS quantities zero on all outer ghostzones (including outer AMR refinement, processor, and outer boundaries)
 * ( 0) Let MoL update all evolution variables
 * ( 1) Apply outer boundary conditions (BCs) on A_{\mu}
 * ( 2) Compute B^i from A_i everywhere, synchronize B^i
 * ( 3) Call con2prim to get primitives on interior pts
 * ( 4) Apply outer BCs on {P,rho_b,vx,vy,vz}.
 * ( 5) (optional) set conservatives on outer boundary.
 *******************************************************/

#include "cctk.h"
#include <cstdio>
#include <cstdlib>
#include <cmath>
#include "cctk_Arguments.h"
#include "cctk_Parameters.h"

#include "IllinoisGRMHD_headers.h"
#include "IllinoisGRMHD_EoS_lowlevel_functs.C"
#include "inlined_functions.C"

#define IDX(i,j,k) CCTK_GFINDEX3D(cctkGH,(i),(j),(k))

#define XMAX_OB_LINEAR_EXTRAP(FUNC,imax) for(int k=0;k<cctk_lsh[2];k++) for(int j=0;j<cctk_lsh[1];j++) FUNC[IDX(imax,j,k)] = 2.0 * FUNC[IDX(imax-1,j,k)] - FUNC[IDX(imax-2,j,k)];
#define YMAX_OB_LINEAR_EXTRAP(FUNC,jmax) for(int k=0;k<cctk_lsh[2];k++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,jmax,k)] = 2.0 * FUNC[IDX(i,jmax-1,k)] - FUNC[IDX(i,jmax-2,k)];
#define ZMAX_OB_LINEAR_EXTRAP(FUNC,kmax) for(int j=0;j<cctk_lsh[1];j++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,j,kmax)] = 2.0 * FUNC[IDX(i,j,kmax-1)] - FUNC[IDX(i,j,kmax-2)];

#define XMIN_OB_LINEAR_EXTRAP(FUNC,imin) for(int k=0;k<cctk_lsh[2];k++) for(int j=0;j<cctk_lsh[1];j++) FUNC[IDX(imin,j,k)] = 2.0 * FUNC[IDX(imin+1,j,k)] - FUNC[IDX(imin+2,j,k)];
#define YMIN_OB_LINEAR_EXTRAP(FUNC,jmin) for(int k=0;k<cctk_lsh[2];k++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,jmin,k)] = 2.0 * FUNC[IDX(i,jmin+1,k)] - FUNC[IDX(i,jmin+2,k)];
#define ZMIN_OB_LINEAR_EXTRAP(FUNC,kmin) for(int j=0;j<cctk_lsh[1];j++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,j,kmin)] = 2.0 * FUNC[IDX(i,j,kmin+1)] - FUNC[IDX(i,j,kmin+2)];
```

    Writing ../src/outer_boundaries.C

### Step 2.a.ii: Applying outer boundary conditions to $A_{\mu}$ \[Back to [top](#toc)\]
$$\label{outer_boundaries__amu__applying_bcs}$$

Now we apply boundary conditions to $A_{\mu}$. The code below is pretty straightforward, but it is useful to understand the following `cctk` variables (refer to the **Cactus Variables** paragraph of section C1.6.2 of the [Einstein Toolkit UserGuide](https://einsteintoolkit.org/usersguide/UsersGuidech9.html#x13-81000C1.6) for further details):

1. `cctk_lsh[i]`: the number of *total* number of grid points along direction $x^{i}$, used *by each processor*.
2. `cctk_bbox[i]`: an array of integers that tell if the boundary gridpoints used by each processor are *internal* (i.e. artificial) or *physical* (i.e. actual boundary points). The variable follows the pattern:
    1. `cctk_bbox[0]`: **Direction**: $x$ | **Orientation**: $+$ | Returns $\color{red}{0}$ if the boundary is $\color{red}{\text{artificial}}$ and $\color{blue}{1}$ if it is $\color{blue}{\text{physical}}$
    1. `cctk_bbox[1]`: **Direction**: $x$ | **Orientation**: $-$ | Returns $\color{red}{0}$ if the boundary is $\color{red}{\text{artificial}}$ and $\color{blue}{1}$ if it is $\color{blue}{\text{physical}}$
    1. `cctk_bbox[2]`: **Direction**: $y$ | **Orientation**: $+$ | Returns $\color{red}{0}$ if the boundary is $\color{red}{\text{artificial}}$ and $\color{blue}{1}$ if it is $\color{blue}{\text{physical}}$
    1. `cctk_bbox[3]`: **Direction**: $y$ | **Orientation**: $-$ | Returns $\color{red}{0}$ if the boundary is $\color{red}{\text{artificial}}$ and $\color{blue}{1}$ if it is $\color{blue}{\text{physical}}$
    1. `cctk_bbox[4]`: **Direction**: $z$ | **Orientation**: $+$ | Returns $\color{red}{0}$ if the boundary is $\color{red}{\text{artificial}}$ and $\color{blue}{1}$ if it is $\color{blue}{\text{physical}}$
    1. `cctk_bbox[5]`: **Direction**: $z$ | **Orientation**: $-$ | Returns $\color{red}{0}$ if the boundary is $\color{red}{\text{artificial}}$ and $\color{blue}{1}$ if it is $\color{blue}{\text{physical}}$

```python
%%writefile -a $outfile_path__outer_boundaries__C

/*********************************************
 * Apply outer boundary conditions on A_{\mu}
 ********************************************/
extern "C" void IllinoisGRMHD_outer_boundaries_on_A_mu(CCTK_ARGUMENTS) {
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;

  if(CCTK_EQUALS(EM_BC,"frozen")) return;

  bool Symmetry_none=false; if(CCTK_EQUALS(Symmetry,"none")) Symmetry_none=true;

  int levelnumber = GetRefinementLevel(cctkGH);

  IllinoisGRMHD_convert_ADM_to_BSSN__enforce_detgtij_eq_1__and_compute_gtupij(cctkGH,cctk_lsh,  gxx,gxy,gxz,gyy,gyz,gzz,alp,
                                                                gtxx,gtxy,gtxz,gtyy,gtyz,gtzz,
                                                                gtupxx,gtupxy,gtupxz,gtupyy,gtupyz,gtupzz,
                                                                phi_bssn,psi_bssn,lapm1);

  // Don't apply approximate outer boundary conditions on initial data, which should be defined everywhere, or on levels != [coarsest level].
  if(cctk_iteration==0 || levelnumber!=0) return;

  if(cctk_nghostzones[0]!=cctk_nghostzones[1] || cctk_nghostzones[0]!=cctk_nghostzones[2])
    CCTK_VError(VERR_DEF_PARAMS,"ERROR: IllinoisGRMHD outer BC driver does not support unequal number of ghostzones in different directions!");
  for(int which_bdry_pt=0;which_bdry_pt<cctk_nghostzones[0];which_bdry_pt++) {
    int imax=cctk_lsh[0]-cctk_nghostzones[0]+which_bdry_pt; // for cctk_nghostzones==3, this goes {cctk_lsh-3,cctk_lsh-2,cctk_lsh-1}; outer bdry pt is at cctk_lsh-1
    int jmax=cctk_lsh[1]-cctk_nghostzones[1]+which_bdry_pt;
    int kmax=cctk_lsh[2]-cctk_nghostzones[2]+which_bdry_pt;

    int imin=cctk_nghostzones[0]-which_bdry_pt-1; // for cctk_nghostzones==3, this goes {2,1,0}
    int jmin=cctk_nghostzones[1]-which_bdry_pt-1;
    int kmin=cctk_nghostzones[2]-which_bdry_pt-1;

    if(cctk_bbox[1]) { XMAX_OB_LINEAR_EXTRAP(Ax,imax); XMAX_OB_LINEAR_EXTRAP(Ay,imax); XMAX_OB_LINEAR_EXTRAP(Az,imax); XMAX_OB_LINEAR_EXTRAP(psi6phi,imax); }
    if(cctk_bbox[3]) { YMAX_OB_LINEAR_EXTRAP(Ax,jmax); YMAX_OB_LINEAR_EXTRAP(Ay,jmax); YMAX_OB_LINEAR_EXTRAP(Az,jmax); YMAX_OB_LINEAR_EXTRAP(psi6phi,jmax); }
    if(cctk_bbox[5]) { ZMAX_OB_LINEAR_EXTRAP(Ax,kmax); ZMAX_OB_LINEAR_EXTRAP(Ay,kmax); ZMAX_OB_LINEAR_EXTRAP(Az,kmax); ZMAX_OB_LINEAR_EXTRAP(psi6phi,kmax); }

    if(cctk_bbox[0]) {                    XMIN_OB_LINEAR_EXTRAP(Ax,imin); XMIN_OB_LINEAR_EXTRAP(Ay,imin); XMIN_OB_LINEAR_EXTRAP(Az,imin); XMIN_OB_LINEAR_EXTRAP(psi6phi,imin); }
    if(cctk_bbox[2]) {                    YMIN_OB_LINEAR_EXTRAP(Ax,jmin); YMIN_OB_LINEAR_EXTRAP(Ay,jmin); YMIN_OB_LINEAR_EXTRAP(Az,jmin); YMIN_OB_LINEAR_EXTRAP(psi6phi,jmin); }
    if((cctk_bbox[4]) && Symmetry_none) { ZMIN_OB_LINEAR_EXTRAP(Ax,kmin); ZMIN_OB_LINEAR_EXTRAP(Ay,kmin); ZMIN_OB_LINEAR_EXTRAP(Az,kmin); ZMIN_OB_LINEAR_EXTRAP(psi6phi,kmin); }
  }
}
```

    Appending to ../src/outer_boundaries.C

## Step 2.b: The hydrodynamic variables \[Back to [top](#toc)\]
$$\label{outer_boundaries__hydro_vars}$$

### Step 2.b.i: Defining the zero derivative, outflow operators \[Back to [top](#toc)\]
$$\label{outer_boundaries__hydro_vars__zero_deriv_outflow}$$

We now apply outer boundary conditions to $\left\{P,\rho_{b},v^{i}\right\}$, imposing zero derivative, outflow boundary conditions. We follow the prescription described above:

$$
\boxed{
\begin{matrix}
\text{Positive direction: }E_{i+1}
=
\left\{
\begin{matrix}
E_{i}\ , &{\rm\ if\ } E\in\left\{P,\rho_{b},v^{y},v^{z}\right\},{\rm\ or\ } E=v^{x}\ {\rm and\ } v^{x}\geq0\\
0\ ,     &{\rm\ if\ } E=v^{x}\ {\rm and\ } v^{x}<0
\end{matrix}
\right.\\
\text{Negative direction: }E_{i-1}
=
\left\{
\begin{matrix}
E_{i}\ , &{\rm\ if\ } E\in\left\{P,\rho_{b},v^{y},v^{z}\right\},{\rm\ or\ } E=v^{x}\ {\rm and\ } v^{x}\geq0\\
0\ ,     &{\rm\ if\ } E=v^{x}\ {\rm and\ } v^{x}<0
\end{matrix}
\right.
\end{matrix}
}\ .
$$

```python
%%writefile -a $outfile_path__outer_boundaries__C

#define XMAX_OB_SIMPLE_COPY(FUNC,imax) for(int k=0;k<cctk_lsh[2];k++) for(int j=0;j<cctk_lsh[1];j++) FUNC[IDX(imax,j,k)] = FUNC[IDX(imax-1,j,k)];
#define YMAX_OB_SIMPLE_COPY(FUNC,jmax) for(int k=0;k<cctk_lsh[2];k++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,jmax,k)] = FUNC[IDX(i,jmax-1,k)];
#define ZMAX_OB_SIMPLE_COPY(FUNC,kmax) for(int j=0;j<cctk_lsh[1];j++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,j,kmax)] = FUNC[IDX(i,j,kmax-1)];

#define XMIN_OB_SIMPLE_COPY(FUNC,imin) for(int k=0;k<cctk_lsh[2];k++) for(int j=0;j<cctk_lsh[1];j++) FUNC[IDX(imin,j,k)] = FUNC[IDX(imin+1,j,k)];
#define YMIN_OB_SIMPLE_COPY(FUNC,jmin) for(int k=0;k<cctk_lsh[2];k++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,jmin,k)] = FUNC[IDX(i,jmin+1,k)];
#define ZMIN_OB_SIMPLE_COPY(FUNC,kmin) for(int j=0;j<cctk_lsh[1];j++) for(int i=0;i<cctk_lsh[0];i++) FUNC[IDX(i,j,kmin)] = FUNC[IDX(i,j,kmin+1)];

#define XMAX_INFLOW_CHECK(vx,imax) for(int k=0;k<cctk_lsh[2];k++) for(int j=0;j<cctk_lsh[1];j++) if(vx[IDX(imax,j,k)]<0.) vx[IDX(imax,j,k)]=0.;
#define YMAX_INFLOW_CHECK(vy,jmax) for(int k=0;k<cctk_lsh[2];k++) for(int i=0;i<cctk_lsh[0];i++) if(vy[IDX(i,jmax,k)]<0.) vy[IDX(i,jmax,k)]=0.;
#define ZMAX_INFLOW_CHECK(vz,kmax) for(int j=0;j<cctk_lsh[1];j++) for(int i=0;i<cctk_lsh[0];i++) if(vz[IDX(i,j,kmax)]<0.) vz[IDX(i,j,kmax)]=0.;

#define XMIN_INFLOW_CHECK(vx,imin) for(int k=0;k<cctk_lsh[2];k++) for(int j=0;j<cctk_lsh[1];j++) if(vx[IDX(imin,j,k)]>0.) vx[IDX(imin,j,k)]=0.;
#define YMIN_INFLOW_CHECK(vy,jmin) for(int k=0;k<cctk_lsh[2];k++) for(int i=0;i<cctk_lsh[0];i++) if(vy[IDX(i,jmin,k)]>0.) vy[IDX(i,jmin,k)]=0.;
#define ZMIN_INFLOW_CHECK(vz,kmin) for(int j=0;j<cctk_lsh[1];j++) for(int i=0;i<cctk_lsh[0];i++) if(vz[IDX(i,j,kmin)]>0.) vz[IDX(i,j,kmin)]=0.;
```

    Appending to ../src/outer_boundaries.C

### Step 2.b.ii: Applying boundary conditions to $\left\{P,\rho_{b},v^{i}\right\}$ \[Back to [top](#toc)\]
$$\label{outer_boundaries__hydro_vars__applying_bcs}$$

As with the previous case, applying the boundary conditions is a straightforward procedure. We refer the reader to the `cctk` quantities discussed in [Step 2.a.ii](#outer_boundaries__amu__applying_bcs), in case clarifications are needed.

```python
%%writefile -a $outfile_path__outer_boundaries__C

/*******************************************************
 * Apply outer boundary conditions on {P,rho_b,vx,vy,vz}
 * It is better to apply BCs on primitives than conservs,
 * because small errors in conservs can be greatly
 * amplified in con2prim, sometimes leading to unphysical
 * primitives & unnecessary fixes.
 *******************************************************/
extern "C" void IllinoisGRMHD_outer_boundaries_on_P_rho_b_vx_vy_vz(CCTK_ARGUMENTS) {
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;

  if(CCTK_EQUALS(Matter_BC,"frozen")) return;

  bool Symmetry_none=false; if(CCTK_EQUALS(Symmetry,"none")) Symmetry_none=true;

  int levelnumber = GetRefinementLevel(cctkGH);

  // Don't apply approximate outer boundary conditions on initial data, which should be defined everywhere, or on levels != [coarsest level].
  if(cctk_iteration==0 || levelnumber!=0) return;

  int ENABLE=1;

  IllinoisGRMHD_convert_ADM_to_BSSN__enforce_detgtij_eq_1__and_compute_gtupij(cctkGH,cctk_lsh,  gxx,gxy,gxz,gyy,gyz,gzz,alp,
                                                                gtxx,gtxy,gtxz,gtyy,gtyz,gtzz,
                                                                gtupxx,gtupxy,gtupxz,gtupyy,gtupyz,gtupzz,
                                                                phi_bssn,psi_bssn,lapm1);

  //if(levelnumber<=11110) {
  if(cctk_nghostzones[0]!=cctk_nghostzones[1] || cctk_nghostzones[0]!=cctk_nghostzones[2])
    CCTK_VError(VERR_DEF_PARAMS,"ERROR: IllinoisGRMHD outer BC driver does not support unequal number of ghostzones in different directions!");
  for(int which_bdry_pt=0;which_bdry_pt<cctk_nghostzones[0];which_bdry_pt++) {
    int imax=cctk_lsh[0]-cctk_nghostzones[0]+which_bdry_pt; // for cctk_nghostzones==3, this goes {cctk_lsh-3,cctk_lsh-2,cctk_lsh-1}; outer bdry pt is at cctk_lsh-1
    int jmax=cctk_lsh[1]-cctk_nghostzones[1]+which_bdry_pt;
    int kmax=cctk_lsh[2]-cctk_nghostzones[2]+which_bdry_pt;

    int imin=cctk_nghostzones[0]-which_bdry_pt-1; // for cctk_nghostzones==3, this goes {2,1,0}
    int jmin=cctk_nghostzones[1]-which_bdry_pt-1;
    int kmin=cctk_nghostzones[2]-which_bdry_pt-1;

    // Order here is for compatibility with old version of this code.
    /* XMIN & XMAX */
    // i=imax=outer boundary
    if(cctk_bbox[1]) { XMAX_OB_SIMPLE_COPY(P,imax); XMAX_OB_SIMPLE_COPY(rho_b,imax); XMAX_OB_SIMPLE_COPY(vx,imax); XMAX_OB_SIMPLE_COPY(vy,imax); XMAX_OB_SIMPLE_COPY(vz,imax); if(ENABLE) XMAX_INFLOW_CHECK(vx,imax); }
    // i=imin=outer boundary
    if(cctk_bbox[0]) {
      XMIN_OB_SIMPLE_COPY(P,imin); XMIN_OB_SIMPLE_COPY(rho_b,imin); XMIN_OB_SIMPLE_COPY(vx,imin); XMIN_OB_SIMPLE_COPY(vy,imin); XMIN_OB_SIMPLE_COPY(vz,imin); if(ENABLE) XMIN_INFLOW_CHECK(vx,imin); }

    /* YMIN & YMAX */
    // j=jmax=outer boundary
    if(cctk_bbox[3]) { YMAX_OB_SIMPLE_COPY(P,jmax); YMAX_OB_SIMPLE_COPY(rho_b,jmax); YMAX_OB_SIMPLE_COPY(vx,jmax); YMAX_OB_SIMPLE_COPY(vy,jmax); YMAX_OB_SIMPLE_COPY(vz,jmax); if(ENABLE) YMAX_INFLOW_CHECK(vy,jmax); }
    // j=jmin=outer boundary
    if(cctk_bbox[2]) {
      YMIN_OB_SIMPLE_COPY(P,jmin); YMIN_OB_SIMPLE_COPY(rho_b,jmin); YMIN_OB_SIMPLE_COPY(vx,jmin); YMIN_OB_SIMPLE_COPY(vy,jmin); YMIN_OB_SIMPLE_COPY(vz,jmin); if(ENABLE) YMIN_INFLOW_CHECK(vy,jmin); }

    /* ZMIN & ZMAX */
    // k=kmax=outer boundary
    if(cctk_bbox[5]) { ZMAX_OB_SIMPLE_COPY(P,kmax); ZMAX_OB_SIMPLE_COPY(rho_b,kmax); ZMAX_OB_SIMPLE_COPY(vx,kmax); ZMAX_OB_SIMPLE_COPY(vy,kmax); ZMAX_OB_SIMPLE_COPY(vz,kmax); if(ENABLE) ZMAX_INFLOW_CHECK(vz,kmax); }
    // k=kmin=outer boundary
    if((cctk_bbox[4]) && Symmetry_none) {
      ZMIN_OB_SIMPLE_COPY(P,kmin); ZMIN_OB_SIMPLE_COPY(rho_b,kmin); ZMIN_OB_SIMPLE_COPY(vx,kmin); ZMIN_OB_SIMPLE_COPY(vy,kmin); ZMIN_OB_SIMPLE_COPY(vz,kmin); if(ENABLE) ZMIN_INFLOW_CHECK(vz,kmin); }
  }

```

    Appending to ../src/outer_boundaries.C

## Step 2.c: The conservative variables \[Back to [top](#toc)\]
$$\label{outer_boundaries__conservatives}$$

After we have applied boundary conditions to our primitives (i.e. hydrodynamics) variables, we [make sure their values lie within the physical range and then recompute the conservatives](Tutorial-IllinoisGRMHD__apply_tau_floor__enforce_limits_on_primitives_and_recompute_conservs.ipynb). Notice that the boundary conditions are then not applied directly to the conservative variables. The reason why the code is structured in this way is because small variations in the values of the conservative variables can cause the conservative-to-primitive algorithm to fail.

```python
%%writefile -a $outfile_path__outer_boundaries__C

  /**********************************
   * Piecewise Polytropic EOS Patch *
   *   Setting up the EOS struct    *
   **********************************/
  /*
   * The short piece of code below takes care
   * of initializing the EOS parameters.
   * Please refer to the "inlined_functions.C"
   * source file for the documentation on the
   * function.
   */
  eos_struct eos;
  initialize_EOS_struct_from_input(eos);

#pragma omp parallel for
  for(int k=0;k<cctk_lsh[2];k++) for(int j=0;j<cctk_lsh[1];j++) for(int i=0;i<cctk_lsh[0];i++) {
        if(((cctk_bbox[0]) && i<cctk_nghostzones[0]) ||
           ((cctk_bbox[1]) && i>=cctk_lsh[0]-cctk_nghostzones[0]) ||
           ((cctk_bbox[2]) && j<cctk_nghostzones[1]) ||
           ((cctk_bbox[3]) && j>=cctk_lsh[1]-cctk_nghostzones[1]) ||
           ((cctk_bbox[4]) && k<cctk_nghostzones[2] && CCTK_EQUALS(Symmetry,"none")) ||
           ((cctk_bbox[5]) && k>=cctk_lsh[2]-cctk_nghostzones[2])) {
          int index = CCTK_GFINDEX3D(cctkGH,i,j,k);
          int ww;

          CCTK_REAL METRIC[NUMVARS_FOR_METRIC],dummy=-1e100; // Set dummy to insane value, to ensure it isn't being used.
          ww=0;
          //psi[index] = exp(phi[index]);
          METRIC[ww] = phi_bssn[index];ww++;
          METRIC[ww] = dummy;          ww++; // Don't need to set psi.
          METRIC[ww] = gtxx[index];    ww++;
          METRIC[ww] = gtxy[index];    ww++;
          METRIC[ww] = gtxz[index];    ww++;
          METRIC[ww] = gtyy[index];    ww++;
          METRIC[ww] = gtyz[index];    ww++;
          METRIC[ww] = gtzz[index];    ww++;
          METRIC[ww] = lapm1[index];   ww++;
          METRIC[ww] = betax[index];   ww++;
          METRIC[ww] = betay[index];   ww++;
          METRIC[ww] = betaz[index];   ww++;
          METRIC[ww] = gtupxx[index];  ww++;
          METRIC[ww] = gtupyy[index];  ww++;
          METRIC[ww] = gtupzz[index];  ww++;
          METRIC[ww] = gtupxy[index];  ww++;
          METRIC[ww] = gtupxz[index];  ww++;
          METRIC[ww] = gtupyz[index];  ww++;

          CCTK_REAL U[MAXNUMVARS];
          ww=0;
          U[ww] = rho_b[index]; ww++;
          U[ww] = P[index];     ww++;
          U[ww] = vx[index];    ww++;
          U[ww] = vy[index];    ww++;
          U[ww] = vz[index];    ww++;
          U[ww] = Bx[index];    ww++;
          U[ww] = By[index];    ww++;
          U[ww] = Bz[index];    ww++;

          struct output_stats stats;
          CCTK_REAL CONSERVS[NUM_CONSERVS],TUPMUNU[10],TDNMUNU[10];

          const int already_computed_physical_metric_and_inverse=0;
          CCTK_REAL g4dn[4][4],g4up[4][4];
          IllinoisGRMHD_enforce_limits_on_primitives_and_recompute_conservs(already_computed_physical_metric_and_inverse,U,stats,eos,METRIC,g4dn,g4up, TUPMUNU,TDNMUNU,CONSERVS);

          rho_b[index] = U[RHOB];
          P[index]     = U[PRESSURE];
          vx[index]    = U[VX];
          vy[index]    = U[VY];
          vz[index]    = U[VZ];

          rho_star[index]=CONSERVS[RHOSTAR];
          tau[index]     =CONSERVS[TAUENERGY];
          mhd_st_x[index]=CONSERVS[STILDEX];
          mhd_st_y[index]=CONSERVS[STILDEY];
          mhd_st_z[index]=CONSERVS[STILDEZ];

          if(update_Tmunu) {
            ww=0;
            eTtt[index] = TDNMUNU[ww]; ww++;
            eTtx[index] = TDNMUNU[ww]; ww++;
            eTty[index] = TDNMUNU[ww]; ww++;
            eTtz[index] = TDNMUNU[ww]; ww++;
            eTxx[index] = TDNMUNU[ww]; ww++;
            eTxy[index] = TDNMUNU[ww]; ww++;
            eTxz[index] = TDNMUNU[ww]; ww++;
            eTyy[index] = TDNMUNU[ww]; ww++;
            eTyz[index] = TDNMUNU[ww]; ww++;
            eTzz[index] = TDNMUNU[ww];
          }
          //if(i==5 && j==5 && k==5) CCTK_VInfo(CCTK_THORNSTRING,"%e %e %e %e",eTtt[index],eTtx[index],eTty[index],eTxy[index]);
          //CCTK_VInfo(CCTK_THORNSTRING,"YAY: "); for(ww=0;ww<10;ww++) CCTK_VInfo(CCTK_THORNSTRING,"%e ",TDNMUNU[ww]); CCTK_VInfo(CCTK_THORNSTRING,"");
        }
      }
}

```

    Appending to ../src/outer_boundaries.C

# Step 3: Code validation \[Back to [top](#toc)\]
$$\label{code_validation}$$

First we download the original `IllinoisGRMHD` source code and then compare it to the source code generated by this tutorial notebook.

```python
# Verify if the code generated by this tutorial module
# matches the original IllinoisGRMHD source code

# First download the original IllinoisGRMHD source code
import urllib
from os import path

original_IGM_file_url  = "https://bitbucket.org/zach_etienne/wvuthorns/raw/5611b2f0b17135538c9d9d17c7da062abe0401b6/IllinoisGRMHD/src/outer_boundaries.C"
original_IGM_file_name = "outer_boundaries-original.C"
original_IGM_file_path = os.path.join(IGM_src_dir_path,original_IGM_file_name)

# Then download the original IllinoisGRMHD source code
# We try it here in a couple of ways in an attempt to keep
# the code more portable
try:
    original_IGM_file_code = urllib.request.urlopen(original_IGM_file_url).read().decode("utf-8")
    # Write down the file the original IllinoisGRMHD source code
    with open(original_IGM_file_path,"w") as file:
        file.write(original_IGM_file_code)
except:
    try:
        original_IGM_file_code = urllib.urlopen(original_IGM_file_url).read().decode("utf-8")
        # Write down the file the original IllinoisGRMHD source code
        with open(original_IGM_file_path,"w") as file:
            file.write(original_IGM_file_code)
    except:
        # If all else fails, hope wget does the job
        !wget -O $original_IGM_file_path $original_IGM_file_url

# Perform validation
Validation__outer_boundaries__C  = !diff $original_IGM_file_path $outfile_path__outer_boundaries__C

if Validation__outer_boundaries__C == []:
    # If the validation passes, we do not need to store the original IGM source code file
    !rm $original_IGM_file_path
    print("Validation test for outer_boundaries.C: PASSED!")
else:
    # If the validation fails, we keep the original IGM source code file
    print("Validation test for outer_boundaries.C: FAILED!")
    # We also print out the difference between the code generated
    # in this tutorial module and the original IGM source code
    print("Diff:")
    for diff_line in Validation__outer_boundaries__C:
        print(diff_line)
```

    Validation test for outer_boundaries.C: FAILED!
    Diff:
    19a20
    > #include "IllinoisGRMHD_EoS_lowlevel_functs.C"
    31a33
    >
    73a76
    >
    91a95
    >
    155c159,170
    <   // FIXME: only for single gamma-law EOS.
    ---
    >
    >   /**********************************
    >    * Piecewise Polytropic EOS Patch *
    >    *   Setting up the EOS struct    *
    >    **********************************/
    >   /*
    >    * The short piece of code below takes care
    >    * of initializing the EOS parameters.
    >    * Please refer to the "inlined_functions.C"
    >    * source file for the documentation on the
    >    * function.
    >    */
    157,165c172,173
    <   eos.neos=neos;
    <   eos.K_poly=K_poly;
    <   eos.rho_tab[0]=rho_tab[0];
    <   eos.P_tab[0]=P_tab[0];
    <   eos.gamma_th=gamma_th;
    <   eos.eps_tab[0]=eps_tab[0];
    <   eos.k_tab[0]=k_tab[0];   eos.k_tab[1]=k_tab[1];
    <   eos.gamma_tab[0]=gamma_tab[0]; eos.gamma_tab[1]=gamma_tab[1];
    <
    ---
    >   initialize_EOS_struct_from_input(eos);
    >
    246a255
    >

# Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-IllinoisGRMHD__outer_boundaries.pdf](Tutorial-IllinoisGRMHD__outer_boundaries.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means).

```python
latex_nrpy_style_path = os.path.join(nrpy_dir_path,"latex_nrpy_style.tplx")
#!jupyter nbconvert --to latex --template $latex_nrpy_style_path --log-level='WARN' Tutorial-IllinoisGRMHD__outer_boundaries.ipynb
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__outer_boundaries.tex
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__outer_boundaries.tex
#!pdflatex -interaction=batchmode Tutorial-IllinoisGRMHD__outer_boundaries.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```


$$ \LaTeX \text{ command declarations here.}
\newcommand{\R}{\mathbb{R}}
\renewcommand{\vec}[1]{\mathbf{#1}}
\newcommand{\X}{\mathcal{X}}
\newcommand{\D}{\mathcal{D}}
\newcommand{\G}{\mathcal{G}}
\newcommand{\Parents}{\mathrm{Parents}}
\newcommand{\NonDesc}{\mathrm{NonDesc}}
\newcommand{\I}{\mathcal{I}}
$$

```python
from __future__ import division

# scientific
%matplotlib inline
from matplotlib import pyplot as plt;
import numpy as np;

# ipython
import IPython;

# python
import os;

#####################################################

# image processing
import PIL;

# trim and scale images
def trim(im, percent=100):
    print("trim:", percent);
    bg = PIL.Image.new(im.mode, im.size, im.getpixel((0,0)))
    diff = PIL.ImageChops.difference(im, bg)
    diff = PIL.ImageChops.add(diff, diff, 2.0, -100)
    bbox = diff.getbbox()
    if bbox:
        x = im.crop(bbox)
        return x.resize(((x.size[0]*percent)//100,
                         (x.size[1]*percent)//100),
                        PIL.Image.ANTIALIAS);

#####################################################

# daft (rendering PGMs)
import daft;

# set to FALSE to load PGMs from static images
RENDER_PGMS = False;

# decorator for pgm rendering
def pgm_render(pgm_func):
    def render_func(path, percent=100, render=None, *args, **kwargs):
        print("render_func:", percent);
        # render
        render = render if (render is not None) else RENDER_PGMS;

        if render:
            print("rendering");
            # render
            pgm = pgm_func(*args, **kwargs);
            pgm.render();
            pgm.figure.savefig(path, dpi=300);

            # trim
            img = trim(PIL.Image.open(path), percent);
            img.save(path, 'PNG');
        else:
            print("not rendering");

        # error
        if not os.path.isfile(path):
            raise("Error:  Graphical model image %s not found."
                  + "You may need to set RENDER_PGMS=True.");

        # display
        return IPython.display.Image(filename=path);#trim(PIL.Image.open(path), percent);

    return render_func;

######################################################
```

# EECS 445:  Machine Learning
## Lecture 15:  Exponential Families & Bayesian Networks
* Instructor:  **Jacob Abernethy**
* Date:  November 7, 2016

*Lecture Exposition Credit:*  Benjamin Bray & Valliappa Chockalingam

## References

- **[MLAPP]** Murphy, Kevin. [*Machine Learning:  A Probabilistic Perspective*](https://mitpress.mit.edu/books/machine-learning-0).  2012.
- **[Koller & Friedman 2009]** Koller, Daphne and Nir Friedman.  [*Probabilistic Graphical Models*](https://mitpress.mit.edu/books/probabilistic-graphical-models).  2009.
- **[Hero 2008]** Hero, Alfred O..  [*Statistical Methods for Signal Processing*](http://web.eecs.umich.edu/~hero/Preprints/main_564_08_new.pdf).  2008.
- **[Blei 2011]** Blei, David. [*Notes on Exponential Families*](https://www.cs.princeton.edu/courses/archive/fall11/cos597C/lectures/exponential-families.pdf).  2011.
- **[Jordan 2010]** Jordan, Michael I..  [*The Exponential Family:  Basics*](http://www.cs.berkeley.edu/~jordan/courses/260-spring10/other-readings/chapter8.pdf).  2008.

## Outline

- Exponential Families
    - Sufficient Statistics & Pitman-Koopman-Darmois Theorem
    - Mean and natural parameters
    - Maximum Likelihood estimation
- Probabilistic Graphical Models
    - Directed Models (Bayesian Networks)
    - Conditional Independence & Factorization
    - Examples

# Exponential Families

> Uses material from **[MLAPP]** §9.2 and **[Hero 2008]** §3.5, §4.4.2

### Exponential Family: Introduction

We have seen many distributions.
* Bernoulli
* Gaussian
* Exponential
* Gamma

Many of these belong to a more general class called the **exponential family**.

### Exponential Family:  Introduction

Why do we care?
* only family of distributions with finite-dimensional **sufficient statistics**
* only family of distributions for which **conjugate priors** exist
* makes the least set of assumptions subject to some user-chosen constraints (**Maximum Entropy**)
* core of generalized linear models and **variational inference**

### Sufficient Statistics

**Recall:** A **statistic** $T(\D)$ is a function of the observed data $\D$.
- Mean, $T(x_1, \dots, x_n) = \frac{1}{n}\sum_{k=1}^n x_k$
- Variance, maximum, mode, etc.

### Sufficient Statistics:  Definition

Suppose we have a model $P$ with parameters $\theta$.  Then,

> A statistic $T(\D)$ is **sufficient** for $\theta$ if no other statistic calculated from the same sample provides any additional information about the parameter.

That is, if $T(\D_1) = T(\D_2)$, our estimate of $\theta$ given $\D_1$ or $\D_2$ will be the same.
- Mathematically, $P(\theta | T(\D), \D) = P(\theta | T(\D))$ independently of $\D$

### Sufficient Statistics:  Example

Suppose $X \sim \mathcal{N}(\mu, \sigma^2)$ and we observe $\mathcal{D} = (x_1, \dots, x_n)$.  Let
- $\hat\mu$ be the sample mean
- $\hat{\sigma}^2$ be the sample variance

Then $T(\mathcal{D}) = (\hat\mu, \hat{\sigma}^2)$ is sufficient for $\theta=(\mu, \sigma^2)$.
- Two samples $\D_1$ and $\D_2$ with the same mean and variance give the same estimate of $\theta$

(we are sweeping some details under the rug)

### Exponential Family:  Definition

**INTUITION**: $p()$ has **exp family form** when density $p(x | \theta)$ can be written as $$\exp(\text{Linear combination of }\theta\text{ and features of }x)$$

**DEF:** $p(x | \theta)$ has **exponential family form** if:
$$
\begin{align}
p(x | \theta)
&= \frac{1}{Z(\theta)} h(x) \exp\left[ \eta(\theta)^T \phi(x) \right] \\
&= h(x) \exp\left[ \eta(\theta)^T \phi(x) - A(\theta) \right]
\end{align}
$$

- $Z(\theta)$ is the **partition function** for normalization
- $A(\theta) = \log Z(\theta)$ is the **log partition function**
- $\phi(x) \in \R^d$ is a vector of **sufficient statistics**
- $\eta(\theta)$ maps $\theta$ to a set of **natural parameters**
- $h(x)$ is a scaling constant, usually $h(x)=1$

### Example:  Bernoulli

The Bernoulli distribution can be written as
$$
\begin{align}
\mathrm{Ber}(x | \mu)
&= \mu^x (1-\mu)^{1-x} \\
&= \exp\left[ x \log \mu + (1-x) \log (1-\mu) \right] \\
&= \exp\left[ \eta(\mu)^T \phi(x) \right]
\end{align}
$$

where $\eta(\mu) = (\log\mu, \log(1-\mu))$ and $\phi(x) = (x, 1-x)$
- There is a linear dependence between features $\phi(x)$
- This representation is **overcomplete**
- $\eta$ is not uniquely determined

### Example:  Bernoulli

Instead, we can find a **minimal** parameterization:
$$
\begin{align}
\mathrm{Ber}(x | \mu)
&= (1-\mu) \exp\left[ x \log\frac{\mu}{1-\mu} \right]
\end{align}
$$

This gives **natural parameters** $\eta = \log \frac{\mu}{1-\mu}$.
- Now, $\eta$ is unique

### Other Examples

Exponential Family Distributions:
- Multivariate normal
- Exponential
- Dirichlet

Non-examples:
- Student t-distribution can't be written in exponential form
- Uniform distribution support depends on the parameters $\theta$

### Log-Partition Function

Derivatives of the **log-partition function** $A(\theta)$ yield **cumulants** of the sufficient statistics *(Exercise!)*
- $\nabla_\theta A(\theta) = E[\phi(x)]$
- $\nabla^2_\theta A(\theta) = \text{Cov}[ \phi(x) ]$

This guarantees that $A(\theta)$ is convex!
- Its Hessian is the covariance matrix of $X$, which is positive-definite.
- Later, this will guarantee a unique global maximum of the likelihood!

#### Proof of Convexity: First Derivative

$$
\begin{align}
\frac{dA}{d\theta}
&= \frac{d}{d\theta} \left[ \log \int exp(\theta\phi(x))h(x)dx \right] \\
&= \frac{\frac{d}{d\theta} \int exp(\theta\phi(x))h(x)dx)}{\int exp(\theta\phi(x))h(x)dx)} \\
&= \frac{\int \phi(x)exp(\theta\phi(x))h(x)dx}{exp(A(\theta))} \\
&= \int \phi(x) \exp[\theta\phi(x)-A(\theta)] h(x) dx \\
&= \int \phi(x) p(x) dx \\
&= E[\phi(x)]
\end{align}
$$

#### Proof of Convexity: Second Derivative

$$
\begin{align}
\frac{d^2A}{d\theta^2}
& = \int \phi(x)\exp[\theta \phi(x) - A(\theta)] h(x) (\phi(x) - A'(\theta)) dx \\
& = \int \phi(x) p(x) (\phi(x) - A'(\theta))dx \\
& = \int \phi^2(x) p(X) dx - A'(\theta) \int \phi(x)p(x)dx \\
& = E[\phi^2(x)] - E[\phi(x)]^2  \hspace{2em}   (\because A'(\theta) = E[\phi(x)])  \\
& = Var[\phi(x)]
\end{align}
$$

#### Proof of Convexity: Second Derivative

For multi-variate case, we have

$$ \frac{\partial^2A}{\partial\theta_i \partial\theta_j} = E[\phi_i(x)\phi_j(x)] - E[\phi_i(x)] E[\phi_j(x)]$$

and hence,
$$ \nabla^2A(\theta) = Cov[\phi(x)] $$

Since covariance is positive definite, we have $A(\theta)$ convex as required.

### Exponential Family:  Likelihood for a *Set* of Data

For data $\D = (x_1, \dots, x_N)$, the likelihood is
$$
p(\D|\theta)
= \left[ \prod_{k=1}^N h(x_k) \right] Z(\theta)^{-N} \exp\left[ \eta(\theta)^T \left(\sum_{k=1}^N \phi(x_k) \right) \right]
$$

The sufficient statistics are now $\phi(\D) = \sum_{k=1}^N \phi(x_k)$.
- **Bernoulli:**  $\phi = \# Heads$
- **Normal:** $\phi = [ \sum_k x_k, \sum_k x_k^2 ]$

### Exponential Family:  MLE

For natural parameters $\theta$ and data $\D = (x_1, \dots, x_N)$,
$$
\log p(\D|\theta) = \theta^T \phi(\D) - N A(\theta)
$$

Since $-A(\theta)$ is concave and $\theta^T\phi(\D)$ linear,
- the log-likelihood is concave
- under many conditions, there is a unique global maximum!
> Strict convexity of the log-partition function requires that we are working with a "regular exponential family". More on this can be found in [These Notes](http://pages.cs.wisc.edu/~jerryzhu/cs731/expfamily.pdf).

### Exponential Family:  MLE

To find the maximum, recall $\nabla_\theta A(\theta) = E_\theta[\phi(x)]$, so
\begin{align*}
\nabla_\theta \log p(\D | \theta) & =
\nabla_\theta(\theta^T \phi(\D) - N A(\theta)) \\
& = \phi(\D) - N E_\theta[\phi(X)] = 0
\end{align*}
Which gives
$$E_\theta[\phi(X)] = \frac{\phi(\D)}{N} = \frac{1}{N} \sum_{k=1}^N \phi(x_k)$$

At the MLE $\hat\theta_{MLE}$, the empirical average of sufficient statistics equals their expected value.
- this is called **moment matching**

### Exponential Family:  MLE for the Bernoulli

As an example, consider the Bernoulli distribution
- Sufficient statistic $N$, $\phi(\D) = \# Heads$

$$
\hat\mu_{MLE} = \frac{\# Heads}{N}
$$

### Bayes for Exponential Family

Exact Bayesian analysis is considerably simplified if the prior is **conjugate** to the likelihood.
- Simply, this means that prior $p(\theta)$ has the same form as the posterior $p(\theta|\mathcal{D})$.

This requires likelihood to have finite sufficient statistics
* Exponential family to the rescue!

**Note**: We will release some notes on cojugate priors + exponential families. It's hard to learn from slides and needs a bit more description.

### Likelihood for exponential family

Likelihood:
$$ p(\mathcal{D}|\theta) \propto g(\theta)^N \exp[\eta(\theta)^T s_N]\\
s_N = \sum_{i=1}^{N}\phi(x_i)$$

In terms of canonical parameters:
$$ p(\mathcal{D}|\eta) \propto \exp[N\eta^T \bar{s} -N A(\eta)] \\
\bar s = \frac{1}{N}s_N $$

### Conjugate prior for exponential family

* The prior and posterior for an exponential family involve two parameters, $\tau$ and $\nu$, initially set to $\tau_0, \nu_0$

$$ p(\theta| \nu_0, \tau_0) \propto g(\theta)^{\nu_0} \exp[\eta(\theta)^T \tau_0] $$

* Denote $ \tau_0 = \nu_0 \bar{\tau}_0$ to separate out the size of the **prior pseudo-data**, $\nu_0$ , from the mean of the sufficient statistics on this pseudo-data, $\tau_0$ . Hence,

$$ p(\theta| \nu_0, \bar \tau_0) \propto \exp[\nu_0\eta^T \bar \tau_0  - \nu_0 A(\eta)] $$

* Think of $\tau_0$ as a "guess" of the future sufficient statistics, and $\nu_0$ as the strength of this guess

### Prior:  Example

$$
\begin{align}
p(\theta| \nu_0, \tau_0)
&\propto (1-\theta)^{\nu_0} \exp[\tau_0\log(\frac{\theta}{1-\theta})] \\
&= \theta^{\tau_0}(1-\theta)^{\nu_0 - \tau_0}
\end{align}
$$

Define $\alpha = \tau_0 +1 $ and $\beta = \nu_0 - \tau_0 +1$ to see that this is a **beta distribution**.

### Posterior

Posterior:
$$ p(\theta|\mathcal{D}) = p(\theta|\nu_N, \tau_N) = p(\theta| \nu_0 +N, \tau_0 +s_N) $$

Note that we obtain **hyper-parameters** by adding. Hence,

$$ \begin{align}
p(\eta|\mathcal{D})
&\propto \exp[\eta^T (\nu_0 \bar\tau_0 + N \bar s) - (\nu_0 + N) A(\eta) ] \\
&= p(\eta|\nu_0 + N, \frac{\nu_0 \bar\tau_0 + N \bar s}{\nu_0 + N})
\end{align}$$
where $\bar s = \frac 1 N \sum_{i=1}^{N}\phi(x_i)$.

* *posterior hyper-parameters are a convex combination of the prior mean hyper-parameters and the average of the sufficient statistics.*

## Break time!

# Probabilistic Graphical Models

> Uses material from **[MLAPP]** §10.1, 10.2 and **[Koller & Friedman 2009]**.

> "I basically know of two principles for treating complicated systems in simple ways:  the first is the principle of modularity and the second is the principle of abstraction.  I am an apologist for computational probability in machine learning because I believe that probability theory implements these two principles in deep and intriguing ways — nameley through factorization and through averaging.  Exploiting these two mechanisms as fully as possible seems to me to be the way forward in machine learning"  – Michael Jordan (qtd. in MLAPP)

### Graphical Models:  Motivation

Suppose we observe multiple correlated variables $x=(x_1, \dots, x_n)$.
- Words in a document
- Pixels in an image

How can we compactly represent the **joint distribution** $p(x|\theta)$?
- How can we tractably *infer* one set of variables given another?
- How can we efficiently *learn* the parameters?

### Joint Probability Tables

One (bad) choice is to write down a **Joint Probability Table**.
- For $n$ binary variables, we must specify $2^n - 1$ probabilities!
- Expensive to store and manipulate
- Impossible to learn so many parameters
- Very hard to interpret!

> Can we be more concise?

### Motivating Example:  Coin Flips

What is the joint distribution of three independent coin flips?
- Explicitly specifying the JPT requires $2^3-1=7$ parameters.

Assuming independence, $P(X_1, X_2, X_3) = P(X_1)P(X_2)P(X_3)$
- Each marginal $P(X_k)$ only requires one parameter, the bias
- This gives a total of $3$ parameters, compared to $8$.

> Exploiting the **independence structure** of a joint distribution leads to more concise representations.

### Motivating Example:  Naive Bayes

In Naive Bayes, we assumed the features $X_1, \dots, X_N$ were independent given the class label $C$:
$$
P(x_1, \dots, x_N, c) = P(c) \prod_{k=1}^N P(x_k | c)
$$

This greatly simplified the learning procedure:
- Allowed us to look at each feature individually
- Only need to learn $O(CN)$ probabilities, for $C$ classes and $N$ features

### Conditional Independence

The key to efficiently representing large joint distributions is to make **conditional independence** assumptions of the form
$$
X \perp Y \mid Z
\iff
p(X,Y|Z) = p(X|Z)p(Y|Z)
$$

> Once $z$ is known, information about $x$ does not tell us any information about $y$ and vice versa.

An effective way to represent these assumptions is with a **graph**.

### Bayesian Networks:  Definition

A **Bayesian Network** $\mathcal{G}$ is a directed acyclic graph whose nodes represent random variables $X_1, \dots, X_n$.
- Let $\Parents_\G(X_k)$ denote the parents of $X_k$ in $\G$
- Let $\NonDesc_\G(X_k)$ denote the variables in $\G$ who are not descendants of $X_k$.

> Examples will come shortly...

### Bayesian Networks:  Local Independencies

Every Bayesian Network $\G$ encodes a set $\I_\ell(\G)$ of **local independence assumptions**:

> For each variable $X_k$, we have $(X_k \perp \NonDesc_\G(X_k) \mid \Parents_\G(X_k))$

Every node $X_k$ is conditionally independent of its nondescendants given its parents.

### Example:  Naive Bayes

The graphical model for Naive Bayes is shown below:
- $\Parents_\G(X_k) = \{ C \}$, $\NonDesc_\G(X_k) = \{ X_j \}_{j\neq k} \cup \{ C \}$
- Therefore $X_j \perp X_k \mid C$ for any $j \neq k$

```python
@pgm_render
def pgm_naive_bayes():
    pgm = daft.PGM([4,3], origin=[-2,0], node_unit=0.8, grid_unit=2.0);
    # nodes
    pgm.add_node(daft.Node("c", r"$C$", -0.25, 2));
    pgm.add_node(daft.Node("x1", r"$X_1$", -1, 1));
    pgm.add_node(daft.Node("x2", r"$X_2$", -0.5, 1));
    pgm.add_node(daft.Node("dots", r"$\cdots$",  0, 1, plot_params={ 'ec' : 'none' }));
    pgm.add_node(daft.Node("xN", r"$X_N$",  0.5, 1));

    # edges
    pgm.add_edge("c", "x1", head_length=0.08);
    pgm.add_edge("c", "x2", head_length=0.08);
    pgm.add_edge("c", "xN", head_length=0.08);

    return pgm;
```

```python
%%capture
pgm_naive_bayes("images/naive-bayes.png");
```

### Subtle Point:  Graphs & Distributions

A Bayesian network $\G$ over variables $X_1, \dots, X_N$ encodes a set of **conditional independencies**.
- Shows independence structure, nothing more.
- Does **not** tell us how to assign probabilities to a configuration $(x_1, \dots x_N)$ of the variables.

There are **many** distributions $P$ satisfying the independencies in $\G$.
- Many joint distributions share a common structure, which we exploit in algorithms.
- The distribution $P$ may satisfy other independencies **not** encoded in $\G$.

### Subtle Point:  Graphs & Distributions

If $P$ satisfies the independence assertions made by $\G$, we say that
- $\G$ is an **I-Map** for $P$
- or that $P$ **satisfies** $\G$.

Any distribution satisfying $\G$ shares common structure.
- We will exploit this structure in our algorithms
- This is what makes graphical models so **powerful**!

### Review:  Chain Rule for Probability

We can factorize any joint distribution via the **Chain Rule for Probability**:
$$
\begin{align}
P(X_1, \dots, X_N)
&= P(X_1) P(X_2, \dots, X_N | X_1) \\
&= P(X_1) P(X_2 | X_1) P(X_3, \dots, X_N | X_1, X_2) \\
&= \prod_{k=1}^N P(X_k | X_1, \dots, X_{k-1})
\end{align}
$$

> Here, the ordering of variables is arbitrary.  This works for any permutation.

### Bayesian Networks:  Topological Ordering

Every network $\G$ induces a **topological (partial) ordering** on its nodes:
> Parents assigned a lower index than their children

```python
@pgm_render
def pgm_topological_order():
    pgm = daft.PGM([4, 4], origin=[-4, 0])

    # Nodes
    pgm.add_node(daft.Node("x1", r"$1$", -3.5, 2))
    pgm.add_node(daft.Node("x2", r"$2$", -2.5, 1.3))
    pgm.add_node(daft.Node("x3", r"$3$", -2.5, 2.7))
    pgm.add_node(daft.Node("x4", r"$4$", -1.5, 1.6))
    pgm.add_node(daft.Node("x5", r"$5$", -1.5, 2.3))
    pgm.add_node(daft.Node("x6", r"$6$", -0.5, 1.3))
    pgm.add_node(daft.Node("x7", r"$7$", -0.5, 2.7))

    # Add in the edges.
    pgm.add_edge("x1", "x4", head_length=0.08)
    pgm.add_edge("x1", "x5", head_length=0.08)
    pgm.add_edge("x2", "x4", head_length=0.08)
    pgm.add_edge("x3", "x4", head_length=0.08)
    pgm.add_edge("x3", "x5", head_length=0.08)
    pgm.add_edge("x4", "x6", head_length=0.08)
    pgm.add_edge("x4", "x7", head_length=0.08)
    pgm.add_edge("x5", "x7", head_length=0.08)

    return pgm;
```

```python
%%capture
pgm_topological_order("images/topological-order.png")
```

### Factorization Theorem:  Statement

**Theorem:** *(Koller & Friedman 3.1)* If $\G$ is an I-map for $P$, then $P$ **factorizes** as follows:
$$
P(X_1, \dots, X_N) = \prod_{k=1}^N P(X_k \mid \Parents_\G(X_k))
$$

> Let's prove it together!

### Factorization Theorem:  Proof

First, apply the chain rule to any topological ordering:
$$
P(X_1, \dots, X_N) = \prod_{k=1}^N P(X_k \mid X_1, \dots, X_{k-1})
$$

Consider one of the factors $P(X_k \mid X_1, \dots, X_{k-1})$.

### Factorization Theorem:  Proof

Since our variables $X_1,\dots,X_N$ are in topological order,
- $\Parents_\G(X_k) \subseteq \{ X_1, \dots, X_{k-1} \}$
- None of $X_k$'s descendants can possibly lie in $\{ X_1, \dots, X_{k-1} \}$

Therefore, $\{ X_1, \dots, X_{k-1} \} = \Parents_\G(X_k) \cup \mathcal{Z}$
- for some $\mathcal{Z} \subseteq \NonDesc_\G(X_k)$.

### Factorization Theorem:  Proof

Recall the following property of conditional independence:
$$
( X \perp Y, W \mid Z ) \implies (X \perp Y \mid Z)
$$

Since $\G$ is an I-map for $P$ and $\mathcal{Z} \subseteq \NonDesc_\G(X_k)$, we have
$$\begin{align}
& (X_k \perp \NonDesc_\G(X_k) \mid \Parents_\G(X_k)) \\
\implies & (X_k \perp \mathcal{Z} \mid \Parents_\G(X_k))
\end{align}
$$

### Factorization Theorem:  Proof

We have just shown $(X_k \perp \mathcal{Z} \mid \Parents_\G(X_k))$, therefore
$$
P(X_k \mid X_1, \dots, X_{k-1}) = P(X_k \mid \Parents_\G(X_k))
$$

- Recall $\{ X_1, \dots, X_{k-1} \} = \Parents_\G(X_k) \cup \mathcal{Z}$.

> **Remember:**  $X_k$ is conditionally independent of its nondescendants given its parents!

### Factorization Theorem:  End of Proof

Applying this to every factor, we see that
$$
\begin{align}
P(X_1, \dots, X_N)
&= \prod_{k=1}^N P(X_k \mid X_1, \dots, X_{k-1}) \\
&= \prod_{k=1}^N P(X_k \mid \Parents_\G(X_k))
\end{align}
$$

### Factorization Theorem:  Consequences

We just proved that for any $P$ satisfying $\G$,
$$
P(X_1, \dots, X_N) = \prod_{k=1}^N P(X_k \mid \Parents_\G(X_k))
$$

It suffices to store **conditional probability tables** $P(X_k | \Parents_\G(X_k))$!
- Requires $O(N2^k)$ features if each node has $\leq k$ parents
- Substantially more compact than **JPTs** for $N$ large, $\G$ sparse
- We can also specify that a CPD is Gaussian, Dirichlet, etc.

### Example:  Fully Connected Graph

A **fully connected graph** makes no independence assumptions.
$$
P(A,B,C) = P(A) P(B|A) P(C|A,B)
$$

```python
@pgm_render
def pgm_fully_connected_a():
    pgm = daft.PGM([4, 4], origin=[0, 0])

    # nodes
    pgm.add_node(daft.Node("a", r"$A$", 2, 3.5))
    pgm.add_node(daft.Node("b", r"$B$", 1.3, 2.5))
    pgm.add_node(daft.Node("c", r"$C$", 2.7, 2.5))

    # add in the edges
    pgm.add_edge("a", "b", head_length=0.08)
    pgm.add_edge("a", "c", head_length=0.08)
    pgm.add_edge("b", "c", head_length=0.08)

    return pgm;
```

```python
%%capture
pgm_fully_connected_a("images/fully-connected-a.png")
```

### Example:  Fully Connected Graph

There are many possible fully connected graphs:
$$
\begin{align}
P(A,B,C)
&= P(A) P(B|A) P(C|A,B) \\
&= P(B) P(C|B) P(A|B,C)
\end{align}
$$

```python
@pgm_render
def pgm_fully_connected_b():
    pgm = daft.PGM([8, 4], origin=[0, 0])

    # nodes
    pgm.add_node(daft.Node("a1", r"$A$", 2, 3.5))
    pgm.add_node(daft.Node("b1", r"$B$", 1.5, 2.8))
    pgm.add_node(daft.Node("c1", r"$C$", 2.5, 2.8))

    # add in the edges
    pgm.add_edge("a1", "b1", head_length=0.08)
    pgm.add_edge("a1", "c1", head_length=0.08)
    pgm.add_edge("b1", "c1", head_length=0.08)

    # nodes
    pgm.add_node(daft.Node("a2", r"$A$", 4, 3.5))
    pgm.add_node(daft.Node("b2", r"$B$", 3.5, 2.8))
    pgm.add_node(daft.Node("c2", r"$C$", 4.5, 2.8))

    # add in the edges
    pgm.add_edge("b2", "c2", head_length=0.08)
    pgm.add_edge("b2", "a2", head_length=0.08)
    pgm.add_edge("c2", "a2", head_length=0.08)

    return pgm;
```

```python
%%capture
pgm_fully_connected_b("images/fully-connected-b.png")
```

### Bayesian Networks & Causality

The fully-connected example brings up a crucial point:

> Directed edges do **not** necessarily represent causality.

Bayesian networks encode **independence assumptions** only.
- This representation is not unique.

### Example:  Markov Chain

State at time $t$ depends only on state at time $t-1$.
$$
P(X_0, X_1, \dots, X_N) = P(X_0) \prod_{t=1}^N P(X_t \mid X_{t-1})
$$

```python
@pgm_render
def pgm_markov_chain():
    pgm = daft.PGM([6, 6], origin=[0, 0])

    # Nodes
    pgm.add_node(daft.Node("x1", r"$\mathbf{x}_1$", 2, 2.5))
    pgm.add_node(daft.Node("x2", r"$\mathbf{x}_2$", 3, 2.5))
    pgm.add_node(daft.Node("ellipsis", r" . . . ", 3.7, 2.5, offset=(0, 0), plot_params={"ec" : "none"}))
    pgm.add_node(daft.Node("ellipsis_end", r"", 3.7, 2.5, offset=(0, 0), plot_params={"ec" : "none"}))
    pgm.add_node(daft.Node("xN", r"$\mathbf{x}_N$", 4.5, 2.5))

    # Add in the edges.
    pgm.add_edge("x1", "x2", head_length=0.08)
    pgm.add_edge("x2", "ellipsis", head_length=0.08)
    pgm.add_edge("ellipsis_end", "xN", head_length=0.08)

    return pgm;
```

```python
%%capture
pgm_markov_chain("images/markov-chain.png")
```

### Example:  Hidden Markov Model

Noisy observations $X_k$ generated from hidden Markov chain $Y_k$.
$$
P(\vec{X}, \vec{Y}) = P(Y_1) P(X_1 \mid Y_1) \prod_{k=2}^N \left(P(Y_k \mid Y_{k-1}) P(X_k \mid Y_k)\right)
$$

```python
@pgm_render
def pgm_hmm():
    pgm = daft.PGM([7, 7], origin=[0, 0])

    # Nodes
    pgm.add_node(daft.Node("Y1", r"$Y_1$", 1, 3.5))
    pgm.add_node(daft.Node("Y2", r"$Y_2$", 2, 3.5))
    pgm.add_node(daft.Node("Y3", r"$\dots$", 3, 3.5, plot_params={'ec':'none'}))
    pgm.add_node(daft.Node("Y4", r"$Y_N$", 4, 3.5))

    pgm.add_node(daft.Node("x1", r"$X_1$", 1, 2.5, observed=True))
    pgm.add_node(daft.Node("x2", r"$X_2$", 2, 2.5, observed=True))
    pgm.add_node(daft.Node("x3", r"$\dots$", 3, 2.5, plot_params={'ec':'none'}))
    pgm.add_node(daft.Node("x4", r"$X_N$", 4, 2.5, observed=True))

    # Add in the edges.
    pgm.add_edge("Y1", "Y2", head_length=0.08)
    pgm.add_edge("Y2", "Y3", head_length=0.08)
    pgm.add_edge("Y3", "Y4", head_length=0.08)

    pgm.add_edge("Y1", "x1", head_length=0.08)
    pgm.add_edge("Y2", "x2", head_length=0.08)
    pgm.add_edge("Y4", "x4", head_length=0.08)

    return pgm;
```

```python
%%capture
pgm_hmm("images/hmm.png")
```

### Example:  Plate Notation

We can represent (conditionally) iid variables using **plate notation**.

```python
@pgm_render
def pgm_plate_example():
    pgm = daft.PGM([4,3], origin=[-2,0], node_unit=0.8, grid_unit=2.0);
    # nodes
    pgm.add_node(daft.Node("lambda", r"$\lambda$", -0.25, 2));

    pgm.add_node(daft.Node("t1", r"$\theta_1$", -1, 1.3));
    pgm.add_node(daft.Node("t2", r"$\theta_2$", -0.5, 1.3));
    pgm.add_node(daft.Node("dots1", r"$\cdots$",  0, 1.3, plot_params={ 'ec' : 'none' }));
    pgm.add_node(daft.Node("tN", r"$\theta_N$",  0.5, 1.3));

    pgm.add_node(daft.Node("x1", r"$X_1$", -1, 0.6));
    pgm.add_node(daft.Node("x2", r"$X_2$", -0.5, 0.6));
    pgm.add_node(daft.Node("dots2", r"$\cdots$",  0, 0.6, plot_params={ 'ec' : 'none' }));
    pgm.add_node(daft.Node("xN", r"$X_N$",  0.5, 0.6));

    pgm.add_node(daft.Node("LAMBDA", r"$\lambda$", 1.5, 2));
    pgm.add_node(daft.Node("THETA", r"$\theta_k$", 1.5,1.3));
    pgm.add_node(daft.Node("XX", r"$X_k$", 1.5,0.6));

    # edges
    pgm.add_edge("lambda", "t1", head_length=0.08);
    pgm.add_edge("lambda", "t2", head_length=0.08);
    pgm.add_edge("lambda", "tN", head_length=0.08);
    pgm.add_edge("t1", "x1", head_length=0.08);
    pgm.add_edge("t2", "x2", head_length=0.08);
    pgm.add_edge("tN", "xN", head_length=0.08);

    pgm.add_edge("LAMBDA", "THETA", head_length=0.08);
    pgm.add_edge("THETA", "XX", head_length=0.08);

    pgm.add_plate(daft.Plate([1.1,0.4,0.8,1.2], label=r"$\qquad\quad\; K$",
    shift=-0.1))

    return pgm;
```

```python
%%capture
pgm_plate_example("images/plate-example.png")
```


# Markdown Cell

## Sub heading this is big text

#### Smaller subheadings

This is just a _paragraph_ and this is **bolded**.

```python
# code cell
name = "Jonathan"
```

# Tips and Tricks

A markdown cell lets you do many _useful things_.

```python
import numpy as np

# don't do:
# from numpy import *
```

```python
max("a")
```

```python
np.max("a")
```

## Imports

```python
# %matplotlib inline
# %config InlineBackend.figure_format='retina'

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
from pivottablejs import pivot_ui
import sys
import warnings

warnings.filterwarnings("ignore")
```

# Keyboard shortcuts

Practice doing these a few times and try to force yourself to use them whenever you can.

## Help (h)
For help in the Notebook, `ESC` + `h`
(doesn't work in Lab)

## Select multiple cells

j, k, and arrow keys while holding shift selects multiple cells.

```python
first = 1
```

```python
second = 2
```

```python
third = 3
```

## Split a cell with -

Well, `ctrl + shift + -`, but remember it by horizontal.

```python

```

```python

```

```python

```

```python

```

```python

```

## Add a new cell (a)bove

## Add a new cell (b)elow

```python

```

## (d)(d)elete
## unD(z)o

## (c)opy cells

## (v)aste cells

# Pivot Tables w/ pandas

A library and example from: http://nicolas.kruchten.com/content/2015/09/jupyter_pivottablejs/

```python
canadian_politics = pd.read_csv("../data/mps2.csv")
```

```python
# recommend using .head()
canadian_politics.head(10)
```

```python
sns.distplot(canadian_politics["Age"].dropna());
```

```python
sns.set_context("poster", font_scale=1.3)
```

```python
fig, ax = plt.subplots(figsize=(12, 8))
sns.distplot(canadian_politics["Age"].dropna())
fig.tight_layout()
```

# Enhanced Pandas Dataframe Display

```python
# Province, Party, (deselect) Average, Age, Heatmap
```

```python
pivot_ui(canadian_politics)
```

```python
newdf = pd.read_clipboard(sep='\t')
newdf.fillna("")
```

```python

```

```python
canadian_politics['Age-bin'] = pd.cut(canadian_politics['Age'], [x for x in range(10, 100, 5)])
# pd.qcut# neat!
```

# Tab -- Your Friend

```python
import numpy as np
```

```python
from numpy.random import chisquare, choice
```

```python
np.random.chisquare()
```

```python
# pure tab right ↓ less useful
np.random.choice()
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## shift-tab

```python
# shift-tab right ↓ more useful
np.linspace(start=50, stop=100, endpoint=False)
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## shift-tab-tab

```python
np.linspace(start=50, end=120)
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## shift-tab-tab-tab

```python
np.linspace(start=50, stop=150, num=100, endpoint=False)
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## shift-tab-tab-tab-tab

```python
plt.plot(np.linspace(start, stop, num=50, ))
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## DO NOT TRY shift-tab-tab-tab-tab-tab

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## ?

```python
np.linspace?
```

```python
?np.linspace
```

## ??

(Lab can scroll if you click)

```python
np.linspace??
```

```python
!code ~/miniconda3/envs/dspy3/lib/python3.6/site-packages/numpy/core/function_base.py
```

# Random stuff

```python
import textwrap
def example_function():
    """Docstring for example function"""

    print(textwrap.dedent("""
    This is a multi-lined string
    that I want to write inside of a function.
    Notice what happens when I print this.
        And when something is indented more."""))

example_function()
```

```python
# python3.6+
```

```python
name
```

```python
f"{name}'s name is not Alex."
```

```python
age = 37
f"{age} plus 2 = {age + 2}"

# Note:
# f"""{example_dictionary["key"]}"""
```

```python

```

```python

```

```python

```

```python

```

## Inspect _everything_ and Find and Replace

```python
# But first find and replace
def silly_function(xval):
    """Takes a value and returns the value."""
    xval_sq = xval ** 2.0
    3 + 1
    xval_abs = np.sqrt(xval_sq)
    return xval_abs
```

```python
silly_function(-2,)
```

```python
silly_function?
```

```python
silly_function??
```

```python
!ls ../data/
```

```python
coal_years = !ls ../data/coal_prod_20*.csv
```

```python
coal_years
```

```python
from glob import glob
```

```python
for filename in glob("../data/coal_prod_20*.csv"):
    print(filename)
```

## Line numbers (lowercase "L")

Type lowercase "L" to have line numbers; shift-L for line numbers notebook-wide.

## Move blocks of code around

### Indent/dedent

    Cmd + [
    Cmd + ]

### Comment

    Cmd + /

```python
ex_dictionary = {}

# Indent/dedent/comment

for index in range(5):
    ex_dictionary["float_one"] = 1
    ex_dictionary["float_two"] = 2
    ex_dictionary["float_three"] = 3
    ex_dictionary["float_four"] = 4
```

```python
ex_dictionary
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## Multicursor magic

Hold down `option`, click and drag (for big cursor).

cmd + click == wherever you click.

Shift command P -- command palette Classic Notebook
Shift command C -- command palette in JupyterLab

Hide left bar (CMD + B)

Full screen CMD + Shift + D (but CTRL + CMD F for browser fullscreen)

Move cells around mouse

```python
example["one_better_neat"] = 1
example["two_better_neat"] = 2
example["three_better_neat"] = 3
example["four_better_neat"] = 4
```

## Monospace

You can also get `monospaced` fonts by indenting 4 spaces:

    mkdir toc
    cd toc

## Syntax Highlighting

Wrap with triple-backticks and language:

```bash
mkdir toc
cd toc
wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
```

```SQL
SELECT first_name,
       last_name,
       year_of_birth
FROM presidents
WHERE year_of_birth > 1800;
```

## Headings and LaTeX

With text and $\LaTeX$ support.

$$\begin{align}
        B'&=-\nabla \times E,\\
        E'&=\nabla \times B - 4\pi j
\end{align}$$

# Magics

 - `%` $\equiv$ inline magic
 - `%%` $\equiv$ cell magic

## Read through the documentation: https://ipython.readthedocs.io/en/stable/interactive/magics.html

```latex
%%latex

If you want to get crazier$\ldots$

\begin{equation}
\oint_S {E_n dA = \frac{1}{{\varepsilon _0 }}} Q_\textrm{inside}
\end{equation}
```

```python
%%python2
print "hi"
```

```bash
%%bash
wget http://www.ast.cam.ac.uk/%7Erfc/vpfit12.2.tar.gz
mkdir -p vpfit12
cd vpfit12
tar -xvzf ../vpfit12.2.tar.gz
```

## Scripting

```python
normal_argument = 12.4
second_argument = 98.4

arg_with_spaces = "the secret to life"
```

```bash
%%bash -s {normal_argument} {second_argument}
echo "This script knows the value of the argument: $1"
echo "It also has no trouble with the second argument: $2"
```

```bash
%%bash -s "$arg_with_spaces"
echo "This bash script knows $1."
```

```python
# %%R -i df -o df2
# df2 <-
```

```python
ls vpfit10/
```

```python
tailthing = "*.ipynb"
```

```python
tailthing
```

```python
!ls {tailthing}
```

```python
output = !ls
```

```python
output
```

## Need to set or change environment variables

```python
%env
```

## Danger zone

    _, _N and _iN

```python
!pwd
```

```python
a = 3
a
```

```python
print(canadian_politics.head().to_latex())
```

```python
5 * 83
```

```python
_
```

```python
3 + 7
```

```python
_
```

```python

```

```python
print(_81)
```

```python

```

```python
saved = _25
```

```python
saved
```

```python
%history
```

```python
%history -opf alex.txt
```

```python

```


# Ikeda $B_e$ assumtion.

```python
from rolldecayestimators import equations
```

    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 461 ('figure.figsize   : 5, 3   ## figure size in inches')
    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 462 ('figure.dpi       : 100        ## figure dots per inch')

# Purpose
The quadratic or cubic model can be expressed using the linearized equivalent damping ($B_e$) according to .:

```python
equations.B_e_equation
```

$\displaystyle B_{e} = B_{1} + \frac{8 B_{2} \omega_{0} \phi_{a}}{3 \pi}$

```python
equations.B_e_equation_cubic
```

$\displaystyle B_{e} = B_{1} + \frac{8 B_{2} \omega_{0} \phi_{a}}{3 \pi} + 0.75 B_{3} \omega_{0}^{2} \phi_{a}^{2}$

But I have some doubt about the validity of this, which will be investigated in this notebook.

# Methodology
A quadratic and cubic model from Simplified Ikeda will be used to calculate $B_e$. $B_e$ will also be obtained from Roll-decay simulations with these models, will the value be the same?

# WIP - improvements
(WORK IN PROGRESS)
Use this section only if the notebook is not final.

Notable TODOs:
* todo 1
* todo 2
* todo 3

## Results
Describe and comment the most important results.

# Suggested next steps
State suggested next steps, based on results obtained in this notebook.

# Setup

```python
# %load imports.py
"""
These is the standard setup for the notebooks.
"""

%matplotlib inline
%load_ext autoreload
%autoreload 2

#from jupyterthemes import jtplot
#jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)

import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
import os
import matplotlib.pyplot as plt
from collections import OrderedDict
#plt.style.use('paper')
from reports.paper_writing import save_fig

#import data
import copy
from mdldb.run import Run

from sklearn.pipeline import Pipeline
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from rolldecayestimators.direct_estimator_cubic import EstimatorQuadraticB, EstimatorCubic
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
import rolldecayestimators.equations as equations
import rolldecayestimators.lambdas as lambdas
from rolldecayestimators.substitute_dynamic_symbols import lambdify
import rolldecayestimators.symbols as symbols
import sympy as sp

from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex

from sklearn.metrics import r2_score
from src.data import database
from mdldb import tables
import shipflowmotionshelpers.shipflowmotionshelpers as helpers
import src.visualization.visualize as visualize

```

```python
from rolldecayestimators.simplified_ikeda_class import SimplifiedIkeda
import rolldecayestimators
from scipy.integrate import solve_ivp
```

```python
zeta_lambda = lambdify(sp.solve(equations.extinction_equation,symbols.zeta)[0])

def calculate_B_n(X_amplitudes):
    B_ns=[None,]
    for i in range(len(X_amplitudes)-1):

        row1 = X_amplitudes.iloc[i]
        row2 = X_amplitudes.iloc[i+1]
        t_ = row2.name - row1.name
        B_n = zeta_lambda(omega0=row1['omega0'],phi_0=row1['phi_a'], phi_a=row2['phi_a'],
                t=t_)
        B_ns.append(B_n)

    return B_ns
```

## Linear

```python
Math(vlatex(equations.roll_decay_equation_himeno_linear))
```

$\displaystyle A_{44} \ddot{\phi} + B_{1} \dot{\phi} + C_{1} \phi = 0$

```python
eq_acceleration_linear = sp.Eq(symbols.phi_dot_dot,
                        sp.solve(equations.roll_decay_equation_himeno_linear,symbols.phi_dot_dot)[0])
Math(vlatex(eq_acceleration_linear))
```

$\displaystyle \ddot{\phi} = - \frac{B_{1} \dot{\phi} + C_{1} \phi}{A_{44}}$

```python
accelaration_linear_lambda = lambdify(sp.solve(equations.roll_decay_equation_himeno_linear,symbols.phi_dot_dot)[0])
```

## Quadratic

```python
Math(vlatex(equations.roll_decay_equation_himeno_quadratic_b))
```

$\displaystyle A_{44} \ddot{\phi} + C_{1} \phi + \left(B_{1} + B_{2} \left|{\dot{\phi}}\right|\right) \dot{\phi} = 0$

```python
eq_acceleration_quadratic = sp.Eq(symbols.phi_dot_dot,
                        sp.solve(equations.roll_decay_equation_himeno_quadratic_b,symbols.phi_dot_dot)[0])

accelaration_quadratic_lambda = lambdify(sp.solve(equations.roll_decay_equation_himeno_quadratic_b,symbols.phi_dot_dot)[0])

Math(vlatex(eq_acceleration_quadratic))
```

$\displaystyle \ddot{\phi} = - \frac{B_{1} \dot{\phi} + B_{2} \left|{\dot{\phi}}\right| \dot{\phi} + C_{1} \phi}{A_{44}}$

```python
class RollDecayLinear():

    def __init__(self,A_44, B_1, C_1):
        self.parameters = {
            'A_44':A_44,
            'B_1':B_1,
            'C_1':C_1,
        }

    def time_step(self,t,states):

        phi = states[0]
        phi1d = states[1]
        phi2d = accelaration_linear_lambda(**self.parameters, phi=phi, phi1d=phi1d)

        d_states_dt = np.array([phi1d, phi2d])
        return d_states_dt

    def simulate(self,t,phi0=np.deg2rad(10),phi1d0=0):

        initial_state = [phi0,phi1d0]

        t_span = [t[0], t[-1]]

        result = solve_ivp(fun=simulation.time_step, t_span=t_span,  y0=initial_state, t_eval=t)
        assert result.success
        df_result = pd.DataFrame(index=result.t, data=result.y.T, columns = ['phi','phi1d'])
        return df_result

class RollDecayQuadratic(RollDecayLinear):

    def __init__(self,A_44, B_1, B_2, C_1):
        self.parameters = {
            'A_44':A_44,
            'B_1':B_1,
            'B_2':B_2,
            'C_1':C_1,
        }

    def time_step(self,t,states):

        phi = states[0]
        phi1d = states[1]
        phi2d = accelaration_quadratic_lambda(**self.parameters, phi=phi, phi1d=phi1d)

        d_states_dt = np.array([phi1d, phi2d])
        return d_states_dt
```

```python
N=100000
A_44 = 2.2
B_1 = 0.10
B_2 = 1.5
C_1 = 0.5

t = np.linspace(0,200,N)
phi0=np.deg2rad(10)
phi1d0 = 0
initial_state = [phi0,phi1d0]

simulations = {
    'linear':RollDecayLinear(A_44=A_44, B_1=B_1, C_1=C_1),
    'quadratic':RollDecayQuadratic(A_44=A_44, B_1=B_1, B_2=B_2, C_1=C_1),
}
```

```python
equations.C_equation_linear
```

$\displaystyle C = GM g m$

```python
A_44_eq = sp.Eq(symbols.A_44, equations.A44)
A_44_eq
```

$\displaystyle A_{44} = \frac{GM g m}{\omega_{0}^{2}}$

```python
eqs = [
    A_44_eq,
    equations.C_equation_linear,

]
omega0_eq = sp.Eq(symbols.omega0,sp.solve(eqs, symbols.omega0, symbols.GM)[1][0])
omega0_eq
```

$\displaystyle \omega_{0} = \sqrt{\frac{C}{A_{44}}}$

```python
omega0 = np.sqrt(C_1/A_44)
```

```python
t_span = [t[0], t[-1]]

results = {}
X_amplitudes = {}
for name,simulation in simulations.items():

    df_result = simulation.simulate(t=t, phi0=phi0, phi1d0=phi1d0)

    results[name]=df_result
    X_amplitudes[name]=rolldecayestimators.measure.calculate_amplitudes_and_damping(X=df_result)
```

```python
for name in results.keys():
    fig,ax=plt.subplots()
    df_result = results[name]
    amplitudes = X_amplitudes[name]
    df_result.plot(y='phi',ax=ax)
    amplitudes.plot(y='phi_a', ax=ax)
    ax.grid(True)
    ax.set_title(name)
```

```python
equations.extinction_equation
```

$\displaystyle \phi_{a} = \phi_{0}{\left(t \right)} e^{- \omega_{0} t \zeta}$

```python
sp.Eq(symbols.zeta,sp.solve(equations.extinction_equation,symbols.zeta)[0])
```

$\displaystyle \zeta = \frac{\log{\left(\frac{\phi_{0}{\left(t \right)}}{\phi_{a}} \right)}}{\omega_{0} t}$

```python
equations.B_e_equation
```

$\displaystyle B_{e} = B_{1} + \frac{8 B_{2} \omega_{0} \phi_{a}}{3 \pi}$

```python
for name in results.keys():
    amplitudes = X_amplitudes[name]
    amplitudes['B_n2'] = calculate_B_n(amplitudes)

    omega0=amplitudes['omega0']
    phi_a=amplitudes['phi_a']
    amplitudes['B_e'] = B_1 + B_2*8/(3*np.pi)*omega0*phi_a

    amplitudes['B_1/2omega0'] = B_1/(2*omega0*A_44)
    amplitudes['B_e/2omega0'] = amplitudes['B_e']/(2*omega0*A_44)

```

```python
for name in results.keys():
    amplitudes = X_amplitudes[name]
    fig,ax=plt.subplots()
    amplitudes.plot(x='phi_a', y='B_n2', style='-', ax=ax)
    amplitudes.plot(x='phi_a', y='B_1/2omega0', style='--', ax=ax)
    amplitudes.plot(x='phi_a', y='B_e/2omega0', style='--',ax=ax)

    y_lim = ax.get_xlim()
    #ax.set_ylim(0,y_lim[1])
    ax.set_title(name)
```

```python
def align_yaxis(ax1, v1, ax2, v2):
    """adjust ax2 ylimit so that v2 in ax2 is aligned to v1 in ax1"""
    _, y1 = ax1.transData.transform((0, v1))
    _, y2 = ax2.transData.transform((0, v2))
    inv = ax2.transData.inverted()
    _, dy = inv.transform((0, 0)) - inv.transform((0, y1-y2))
    miny, maxy = ax2.get_ylim()
    ax2.set_ylim(miny+dy, maxy+dy)
```

```python
from scipy.integrate import cumtrapz

df_results = results['linear'].copy()
phi = df_results['phi']
phi1d = df_results['phi1d']

df_results['B'] = phi1d*B_1

fig,ax=plt.subplots()
df_results.plot(y='phi', ax=ax, lw=2, alpha=1)
ax.legend(loc='upper left')
ax.set_ylabel('$\phi$ [rad]')

ax_damping = ax.twinx()
df_results.plot(y='B', style='r-', lw=2, ax=ax_damping)
ax_damping.set_ylabel('Damping B [Nm]')

align_yaxis(ax, 0, ax_damping, 0)
```

```python
df_results['E_kin'] = 1/2*A_44*phi1d**2
E_loss = cumtrapz(df_results['B'],x=phi)
E_loss = np.concatenate([[0],E_loss])
df_results['E_loss'] = E_loss
df_results['E_pot'] = C_1*phi**2/2
df_results['E_sys'] = df_results['E_kin'] + df_results['E_pot']
df_results['E_tot'] = df_results['E_loss'] + df_results['E_sys']
```

```python
#with plt.style.context('paper'):
fig,ax=plt.subplots()
#fig.set_size_inches(15,10)
#fig.set_dpi(300)
df_results.plot.area(y = ['E_kin','E_pot','E_loss'], label=[r'$E_{kin}$',r'$E_{pot}$',r'$E_{loss}$'], color=['r','g','b'], ax=ax)
ax.set_xlabel('Time [s]')
ax.set_ylabel('Energy [kNm]')

save_fig(fig, name='energy_transfer')
```

```python
E_loss2 = cumtrapz(phi1d,x=phi)
E_loss2 = np.concatenate([[0],E_loss2])
B_es = df_results['E_loss']/E_loss2

```

```python

```

```python
from scipy.integrate import cumtrapz

df_results = results['quadratic'].copy()
phi = df_results['phi']
phi1d = df_results['phi1d']

df_results['B'] = (B_1 + B_2*np.abs(phi1d))*phi1d

fig,ax=plt.subplots()
df_results.plot(y='phi', ax=ax, lw=2, alpha=1)
ax.legend(loc='upper left')
ax.set_ylabel('$\phi$ [rad]')

ax_damping = ax.twinx()
df_results.plot(y='B', style='r-', lw=2, ax=ax_damping)
ax_damping.set_ylabel('Damping B [Nm]')

align_yaxis(ax, 0, ax_damping, 0)
```

```python
df_results['E_kin'] = 1/2*A_44*phi1d**2
E_loss = cumtrapz(df_results['B'],x=phi)
E_loss = np.concatenate([[0],E_loss])
df_results['E_loss'] = E_loss
df_results['E_pot'] = C_1*phi**2/2
df_results['E_sys'] = df_results['E_kin'] + df_results['E_pot']
df_results['E_tot'] = df_results['E_loss'] + df_results['E_sys']
```

```python
E_loss2 = cumtrapz(phi1d,x=phi)
E_loss2 = np.concatenate([[1],E_loss2])

df_results['B_e'] = df_results['E_loss']/E_loss2
```

```python
fig,ax=plt.subplots()
df_results.plot(y='B_e', ax=ax)
```

```python
fig,ax=plt.subplots()
df_results.plot(y='B_e', ax=ax)
ax.set_xlim(3,200)
ax.set_ylim(0.16,0.20)
```

```python
amplitudes = X_amplitudes['quadratic']
df_results['phi_a'] = np.interp(df_results.index,amplitudes.index,amplitudes['phi_a'])
omega0 = np.sqrt(C_1/A_44)
df_results['B_e_formula'] = B_1 + B_2*8/(3*np.pi)*omega0*df_results['phi_a']

fig,ax=plt.subplots()
mask = df_results['phi_a']<0.11
df_results.loc[mask].plot(x='phi_a', y=['B_e','B_e_formula'],ax=ax)
```

```python
omega0 = np.sqrt(C_1/A_44)
df_results['zeta'] = df_results['B_e']/(2*omega0*A_44)
```

```python
fig,ax=plt.subplots()
mask = df_results['phi_a']<0.11
df_results.loc[mask].plot(x='phi_a', y='zeta',ax=ax)
```

```python
B_e = df_results['B_e'].iloc[-1]
B_e
```

    0.16615505838874878

```python
simulation = RollDecayLinear(A_44=A_44, B_1=B_e, C_1=C_1)
df_result = simulation.simulate(t=t, phi0=phi0, phi1d0=phi1d0)
```

```python
fig,ax=plt.subplots()
results['quadratic'].plot(y='phi', ax=ax, label='quadratic')
results['linear'].plot(y='phi', ax=ax, style='--', label='linear')
df_result.plot(y='phi', ax=ax, style='--', label='linearized')

```

```python
fig,ax=plt.subplots()

df_error = pd.DataFrame(index=df_result.index)
ref = results['quadratic']['phi']
df_error['linear'] = results['linear']['phi'] - ref
df_error['linearized'] = df_result['phi'] - ref

df_error.plot(ax=ax)
```

```python

```

```python
damping = equations.roll_decay_equation_himeno_quadratic_b.lhs.subs([(symbols.A_44,0),(symbols.C_1,0)])
Math(vlatex(damping))
```

$\displaystyle \left(B_{1} + B_{2} \left|{\dot{\phi}}\right|\right) \dot{\phi}$

```python
x = sp.symbols('x')
damping2 = symbols.B_1*x + symbols.B_2*x**2
damping2
```

$\displaystyle B_{1} x + B_{2} x^{2}$

```python
x = sp.symbols('x')
damping3 = symbols.B_1*x
damping3
```

$\displaystyle B_{1} x$

```python
s = sp.fourier_series(damping3, (x, -symbols.omega0/2*symbols.t, symbols.omega0/2*symbols.t))
s
```

$\displaystyle \frac{B_{1} \omega_{0} t \sin{\left(\frac{2 \pi x}{\omega_{0} t} \right)}}{\pi} - \frac{B_{1} \omega_{0} t \sin{\left(\frac{4 \pi x}{\omega_{0} t} \right)}}{2 \pi} + \frac{B_{1} \omega_{0} t \sin{\left(\frac{6 \pi x}{\omega_{0} t} \right)}}{3 \pi} + \ldots$

```python
s_truncate = s.truncate(n=2)
s_truncate
```

$\displaystyle \frac{B_{1} \omega_{0} t \sin{\left(\frac{2 \pi x}{\omega_{0} t} \right)}}{\pi} - \frac{B_{1} \omega_{0} t \sin{\left(\frac{4 \pi x}{\omega_{0} t} \right)}}{2 \pi}$

```python
sp.integrate(s_truncate,(symbols.t, -symbols.omega0/2*symbols.t, symbols.omega0/2*symbols.t))
```

$\displaystyle - \frac{B_{1} \omega_{0} \left(- \frac{\omega_{0}^{2} t^{2} \sin{\left(\frac{4 \pi x}{\omega_{0}^{2} t} \right)}}{8} - \frac{\pi t x \cos{\left(\frac{4 \pi x}{\omega_{0}^{2} t} \right)}}{2} - \frac{2 \pi^{2} x^{2} \operatorname{Si}{\left(\frac{4 \pi x}{\omega_{0}^{2} t} \right)}}{\omega_{0}^{2}}\right)}{\pi} + \frac{B_{1} \omega_{0} \left(\frac{\omega_{0}^{2} t^{2} \sin{\left(\frac{4 \pi x}{\omega_{0}^{2} t} \right)}}{8} + \frac{\pi t x \cos{\left(\frac{4 \pi x}{\omega_{0}^{2} t} \right)}}{2} + \frac{2 \pi^{2} x^{2} \operatorname{Si}{\left(\frac{4 \pi x}{\omega_{0}^{2} t} \right)}}{\omega_{0}^{2}}\right)}{\pi} + \frac{B_{1} \omega_{0} \left(- \frac{\omega_{0}^{2} t^{2} \sin{\left(\frac{8 \pi x}{\omega_{0}^{2} t} \right)}}{8} - \pi t x \cos{\left(\frac{8 \pi x}{\omega_{0}^{2} t} \right)} - \frac{8 \pi^{2} x^{2} \operatorname{Si}{\left(\frac{8 \pi x}{\omega_{0}^{2} t} \right)}}{\omega_{0}^{2}}\right)}{2 \pi} - \frac{B_{1} \omega_{0} \left(\frac{\omega_{0}^{2} t^{2} \sin{\left(\frac{8 \pi x}{\omega_{0}^{2} t} \right)}}{8} + \pi t x \cos{\left(\frac{8 \pi x}{\omega_{0}^{2} t} \right)} + \frac{8 \pi^{2} x^{2} \operatorname{Si}{\left(\frac{8 \pi x}{\omega_{0}^{2} t} \right)}}{\omega_{0}^{2}}\right)}{2 \pi}$

```python

```

```python
func = 2*x
s = sp.fourier_series(func, (x, -10, 10))
s
```

$\displaystyle \frac{40 \sin{\left(\frac{\pi x}{10} \right)}}{\pi} - \frac{20 \sin{\left(\frac{\pi x}{5} \right)}}{\pi} + \frac{40 \sin{\left(\frac{3 \pi x}{10} \right)}}{3 \pi} + \ldots$

```python
s_trunate = s.truncate(n=10)
s_lambda = lambdify(s_trunate)
func_lambda = lambdify(func)

x_ = np.linspace(-10,20,100)
y_ = s_lambda(x=x_)
y = func_lambda(x=x_)

fig,ax=plt.subplots()
ax.plot(x_,y, label='function')
ax.plot(x_,y_, label='fourier series n=1')
```

```python
sp.integrate(x,(x, 0, 1))
```

$\displaystyle \frac{1}{2}$

```python

```