Thermal Modeling lecture notes

1  Introduction (19/2)

1.1  Thermal effects: A historical perspective

1.1.1  Major milestones

1.1.1.1  Ancient times

1.1.1.2  Early modern period

1.1.1.3  Modern times

1.2  Why thermal modeling?

1.2.1  Heating: Good and Bad

1.2.2  Exposure/implant safety assessment

1.2.2.1  Exposure safety

1.2.2.2  Implant safety

1.2.3  Thermal therapies

1.2.3.1  Treatment planning

1.2.3.2  Novel therapies and devices

1.2.3.3  Bio-heat modeling

1.2.4  Self-heating in nanoscale devices

1.2.4.1  CBRAM device

1.2.4.2  Nanodevice design work flow

1.2.4.3  Electron-Phonon energy exchange

1.2.4.4  Test device structures

1.2.4.5  Influence of self-heating

2  Thermal Modeling (Bio) (26/2)

2.1  The Pennes Bioheat equation

2.1.1  Continuum level thermal modeling

2.1.2  Steady state

2.1.3  Boundary conditions in the PBE

2.1.4  Natural and forced convection

2.1.5  Heat radiation

2.1.6  Terms of the PBE

2.1.6.1  Diffusion

2.1.6.2  Exposure

2.1.6.3  Metabolism

2.1.6.4  Perfusion

2.1.7  High temperature limit

2.1.8  Linear regime: Considering only temperature increase

2.1.9  Power deposition

2.1.9.1  Exposure

2.1.9.2  RF Dosimetry

2.1.10  Beyond PBE

2.2  Application: Mobile Phone Exposure Safety

2.2.1  Averaged SAR and temperature increase

2.2.2  Debates and considerations

3  Thermal Modeling (Bio) (4/3)

3.1  The UTCI model

3.1.1  Definition of the Universal Thermal Climate Index

3.1.1.1  Advantages

3.1.1.2  Disadvantages

3.1.2  Tuning & Validation

3.1.3  Discretization

3.1.4  Coordinate system

3.2  Thermoregulation in UTCI

3.2.1  Perfusion

3.2.2  Heat exchange with environment

3.2.2.1  Convection

3.2.2.2  Radiation

3.2.2.3  Evaporation

3.2.2.4  Respiration

3.2.3  CNS response model

3.2.4  Sensation

3.2.5  Numerical implementation

3.2.5.1  Space

3.2.5.2  Time

3.2.5.3  Scheme

3.3  Application: MRI exposure safety (RF)

3.3.1  MRI RF safety

3.3.1.1  Scenarios

3.3.1.2  MRI heating

3.3.2  Thermal dose

3.3.3  Thermal damaga

3.3.4  CEM43

3.3.4.1  Concept

3.3.4.2  Definition

3.3.4.3  Tresholds

3.3.4.4  Advantages of dose-based limits

4  Numerical Methods (11/3)

4.1  Discretization of continuum models

4.1.1  Structured grids

4.1.2  Unstructured meshes

4.2  FDTD

4.2.1  Temporal discretization

4.2.2  Spatial discretization

4.2.3  Dealing with interfaces

4.2.4  Boundary conditions

4.2.5  Staircasing

4.2.6  Conformal corrections

4.2.7  Stability

4.2.7.1  CFL criterion

4.2.8  Other considerations

4.3  FEM

4.3.1  Pros and cons

4.3.2  Overview of the Galerkin method

4.4  Fast Poisson solvers

4.4.1  Green's functions

4.4.2  Green's function of the PBE

4.4.3  Overview of the Fast Poisson method

5  Thermal Therapy Treatment Planning (18/3)

5.1  Introduction

5.1.1  Hyperthermic oncology

5.1.2  HT energy delivery technology

5.1.3  Thermal ablation

5.1.4  Temperature modeling

5.1.5  Hyperthermic Treatment Planning (HTP)

5.1.5.1  Embedding in clinlcal flow

5.1.5.2  Steps in HTP

5.1.5.3  Motivation

5.1.5.4  Requirements

5.2  Imaging

5.2.1  Definition and objectives

5.2.2  Sim4Life framework

5.3  Personalized model generation

5.3.1  Implementation

5.3.2  Considerations

5.3.3  Surface extraction and discretization

5.3.3.1  Implementation

5.3.3.2  Considerations

5.4  EM modeling

5.4.1  Implementation

5.4.2  Considerations

5.5  Acoustic modeling

5.5.1  Implementation

5.5.2  Considerations

5.6  Thermal modeling

5.6.1  Implementation

5.6.2  Considerations

5.7  The use of HPC

5.8  Optimization

5.8.1  Objectives

5.8.2  Array steering optimization

5.8.2.1  Implementation

5.8.2.2  Considerations

5.9  FUS steering

5.10  Effect quantification

5.10.1  Implementation

5.10.2  Considerations

5.11  Clinical integration

5.11.1  Implementation

5.11.2  Considerations

5.12  Validation, Verification and Uncertainty Quantification

5.12.1  Verification

5.12.2  Validation

5.12.2.1  Local fields

5.12.2.2  In vivo heating

5.12.3  UQ

5.13  Limitations and need for further research

5.13.1  Tissue parameters

5.13.2  Dynamic tissue models

5.13.3  Closed-loop treatment control

5.13.4  Clinical integration

5.13.5  Effectiveness and outcome prediction

5.13.6  Validation and quality assurance

5.14  Conclusions

5.15  Exercise: Fast temperature increase predictions for phased arrays

6  Verification and Validation (25/3)

6.1  Background

6.1.1  Benefits of computational modeling in MedTech and computational life sciences

6.1.2  Computational modeling for regulatory submissions

6.1.3  Verification and validation standard (ASME, FDA)

6.1.4  Multi-physics

6.1.5  In vivo issues

6.1.6  Benchmarks

6.1.7  Context-of-use

6.1.8  Uncertainty and sensitivity analysis

6.2  V & V example: FUS Simulator: Solver Verification & Validation

6.2.1  Verification

6.2.1.1  Critical effects for FDTD implementation of linear acoustic pressure wave model

6.2.1.2  Method of Manufactured Solutions

6.2.2  Validation

6.2.2.1  Gamma method

6.2.2.2  UA-Gamma Combination

6.2.2.3  Measurement setup

6.2.2.4  Results

6.3  V & V example: MRI Induced Heating: Validation from EM Exposure to in vivo heating

6.3.1  Illustrating Example: MRI RF Exposure

6.3.2  MR RF Safety Background

6.3.2.1  Limits

6.3.2.2  Exposure

6.3.2.3  psSAR & T

6.3.2.4  T Distribution

6.3.2.5  pTx vs CP

6.3.3  Verification

6.3.3.1  Coil modeling with Huygens vs direct

6.3.4  Validation

6.3.4.1  Incident E/B-field

6.3.4.2  Local field - requirements

6.3.4.3  Local field - experimental setup

6.3.4.4  Local field - results

6.3.4.5  EM induced heating - setup

6.3.4.6  EM induced heating - results

6.3.4.7  In vivo heating

6.4  V & V example: EM-neuron interaction: Getting the Quantity-of-interest wrong

6.4.1  EM induced neuro-stimulation

6.4.1.1  Considerations

6.4.1.2  Modeling

6.4.1.3  Results

6.5  V & V example: Computational Human Phantoms

6.5.1  Verification

6.5.2  Validation

6.6  Aspects of Quality Assurance

6.7  Exercise: Method of Manufactured Solutions

7  Nanoscale Thermal Modeling I (1/4)

7.1  Learning goals

7.2  Summary

7.3  Outline

7.4  From temperature to phonon

7.4.1  Introduction

7.4.2  Heat capacity

7.4.3  Heat flow

7.4.3.1  Definition of heat flow

7.4.3.2  Calculation of heat flow

7.4.4  Thermal conductivity

7.5  Electron vs. phonon

7.5.1  Definitions

7.5.1.1  Electrons

7.5.1.2  Phonons

7.5.2  Bulk density-of-states (DOS)

7.5.2.1  Electrons

7.5.2.2  Phonons

7.5.3  Occupancy: Distribution function

7.5.3.1  Electrons

7.5.3.2  Phonons

7.5.4  Population

7.5.4.1  Electrons

7.5.4.2  Phonons

7.5.5  Current

7.5.5.1  Electrons

7.5.5.2  Phonons

7.5.6  Unoccupied state

7.5.6.1  Electrons

7.5.6.2  Phonons

7.6  Boltzmann Transport Equation (BTE)

7.6.1  Derivation to physical model

7.6.1.1  Motivation

7.6.1.2  Phonon equation of motion

7.6.1.3  Statement of BTE

7.6.1.4  Deriving temperature from the BTE

7.6.1.5  Linearized phonon BTE

7.6.1.6  Thermal current - heat flow

7.6.1.7  Thermal conductivity

7.6.2  Description of scattering mechanisms

7.6.2.1  Relaxation time approximation

7.6.2.2  Examples of scattering

7.6.3  Examples of thermal conductivity

7.6.3.1  Thermal conductivity of Si

7.6.3.2  Thermal conductivity of Si NW

7.6.3.3  Thermal conductivity of InAs

8  Nanoscale Thermal Modeling II: Phonon dispersion (8/4)

8.1  Learning goals

8.2  Summary

8.3  Outline

8.4  Phonon bandstructure

8.4.1  Why do we need this quantity?

8.4.1.1  Thermal conductivity recap

8.4.2  Ingredients for $\kappa$ calculation: Require phonon band structure

8.4.2.1  Phonon density-of-states

8.4.2.2  Phonon velocity

8.4.2.3  Relaxation time for scattering type (hard/not discussed)

8.4.3  Example (recap): Phonon dispersion in Si

8.4.3.1  Acoustic phonons

8.4.3.2  Optical phonons

8.4.3.3  Transverse branches

8.4.3.4  Longitudinal branches

8.4.3.5  Phonon velocity as a derivative of frequency w.r.t. momentum

8.4.4  Example: Other phonon dispersions

8.4.4.1  Polar semiconductor (InAs): LO/TO split

8.4.4.2  Graphene: ZA, ZO modes out-of-plane

8.5  Classical phonon bandstructure calculation: Newton/Hooke

8.5.1  Simple 1-D atomic chain

8.5.1.1  Atom oscillations by Newton's EoM and Hooke's law

8.5.1.2  Unit cell of atomic chain

8.5.1.3  Period boundary conditions

8.5.1.4  Result: LSE w/ two coupled EoM

8.5.2  Dynamical matrix of the system from LSE of coupled EoM

8.5.2.1  Eigenvalue problem w/ $\omega^2$ as eigenvalue

8.5.2.2  Analytic solution:  Characteristic polynomial

8.5.2.3  Only positive eigenvalues admissible

8.5.2.4  Obtain two branches for 1D: 1 optical, 1 acoustic

8.5.2.5  Optical branch oscillation mode: out-of-phase motion

8.5.2.6  Acoustic branch oscillation mode:  in-phase motion

8.5.2.7  Extending from 1D to 3D: 6 branches but similar to 1D

8.5.3  1D already allows extracting analytical parameters

8.5.3.1  DOS from phonon band structure

8.5.3.2  Injection velocity

8.5.4  Simplification by linearization

8.5.4.1  Phonon band structure

8.5.4.2  DOS

8.5.4.3  Injection velocity

8.6  Advanced phonon bandstructure calculation: VFF and DFT

8.6.1  Harmonic approximation

8.6.1.1  Energy of solids and the Hamiltonian

8.6.1.2  Phonon amplitude affects total energy $E$

8.6.1.3  Harmonic approximation of the total energy: 2nd order

8.6.1.4  Out-of-equilibrium: Force from integrating Hessian

8.6.1.5  The dynamical matrix: Eigenvalue problem as in 1D atomic chain

8.6.2  Valence-force-field (VFF) model

8.6.2.1  VFF equals tight-binding for electrons: An empirical method

8.6.2.2  The Keating model

8.6.2.3  Extended VFF model

8.6.3  Density functional theory (DFT) model

8.6.3.1  Schroedinger equation with $E_{xc}$

8.6.3.2  Charge density $\rho$

8.6.3.3  Exchange correlation energy functional $E_{xc}[\rho]$

8.6.3.4  Frozen phonon approach

8.6.3.5  Near-perfect fit to experiment w/o fitting parameters

8.6.3.6  More efficient: density functional perturbation theory

9  Phonon Quantum Transport (22/4)

9.1  Learning goals

9.2  Summary

9.3  Outline

9.4  Boundary conditions

9.4.1  Intro/Recap

9.4.1.1  Semi-classical approach vs. QM approach for phonon transport

9.4.1.2  Recap: 1D phonon model

9.4.1.3  Phonon EoM

9.4.1.4  Generalized dynamical equation

9.4.2  Periodic boundary conditions

9.4.2.1  Periodic boundary conditions: Infinite length

9.4.2.2  Simple phonon bandstructure

9.4.3  Closed boundary conditions

9.4.3.1  Closed boundary conditions: Finite length

9.4.3.2  Discrete phonon frequencies

9.4.4  Open boundary conditions

9.4.4.1  Open boundary conditions: Quasi-infinite length

9.5  Quantum transport approach

9.5.1  Definition of contacts/reservoirs

9.5.1.1  Division of simulation domain

9.5.1.2  Contact properties

9.5.1.3  Physical interpretation

9.5.2  Derivation of the Wave Function formalism

9.5.2.1  Connection to device

9.5.2.2  Hierarchy truncation

9.5.2.3  Variable elimination

9.5.2.4  Wave Function formalism

9.5.3  DOS and transmission function

9.5.3.1  Solution of the WF formalism

9.5.3.2  DOS and Transmission

9.5.3.3  Homogeneous system: Example

9.5.3.4  Inhomogeneous system: Example

9.5.3.5  Heat current

9.5.4  Non-equilibrium Green’s Function

9.5.4.1  Green’s Functions in general

9.5.4.2  Phonon Green’s Functions

10  Thermoelectric Power Generators (29/4)

10.1  Learning goals

10.2  Summary

10.3  Outline

10.4  Thermoelectricity: an introduction

10.4.1  How to recycle waste heat?

10.4.1.1  Energy Consumption in US

10.4.1.2  Waste heat

10.4.1.3  Thermoelectricity

10.4.2  ZT figure of merit

10.4.2.1  Thermoelectric generator

10.4.2.2  Thermoelectric efficiency

10.4.2.3  𝒁𝑻 definition and interpretation

10.4.2.4  Representative ZT examples

10.4.3  From bulk to nanostructures

10.4.3.1  From bulk to nanostructures: Theory historical perspective

10.4.3.2  Theory validation

10.4.3.3  Rough Si nanowires

10.5  Mathematical formulation

10.5.1  Current and conductance

10.5.1.1  Calculation of 𝒁𝑻

10.5.1.2  𝒁𝑻 ingredients

10.5.1.3  Current flows

10.5.1.4  Conductance definition

10.5.2  Electrical and thermal components

10.5.2.1  Electrical currents by electrons

10.5.2.2  Electrical current linearization

10.5.2.3  Thermal current from electrons

10.5.2.4  Simplification and generalization

10.5.2.5  Thermal current from phonons

10.5.2.6  Calculation of 𝒁𝑻

10.5.3  Simulation example

10.5.3.1  𝒁𝑻 summary

10.5.3.2  Example: Si nanowire

10.5.3.3  Electron/phonon transmission

10.5.3.4  𝑲𝟎, 𝑲𝟏, 𝑲𝟐, and 𝑳𝟐

10.5.3.5  𝒁𝑻 results

11  Numerics and Parallelization (6/5)

11.1  Learning goals

11.2  Summary

11.3  Outline

11.4  Basic numerical operations

11.4.1  BLAS/LAPACK libraries

11.4.1.1  Basic operations

11.4.1.2  BLAS

11.4.1.3  Eigenvalue (EV) problems

11.4.1.4  LAPACK

11.4.1.5  Calling BLAS/LAPACK

11.4.1.6  Vendor libraries

11.4.2  Parallelization strategies

11.4.2.1  Shared vs. distributed memory

11.4.2.2  Shared memory parallelization

11.4.2.3  OpenMP example: 𝑪 = 𝑨𝑩

11.4.2.4  OpenMP timing results

11.4.2.5  Distributed memory parallelization

11.4.2.6  MPI example: 𝑪 = 𝑨𝑩

11.4.2.7  MPI timing results

11.4.3  Graphics processing units (GPUs)

11.4.3.1  GPU architecture

11.4.3.2  GPU example 𝑪 = 𝑨𝑩

11.4.3.3  GPU timing results

11.5  Open boundary calculation

11.5.1  Generalized eigenvalue problem

11.5.1.1  Phonon ballistic transport

11.5.1.2  Benchmark structure

11.5.1.3  OBC formulation

11.5.1.4  GEPV and shift-and-invert

11.5.1.5  Timing standard solution

11.5.2  Contour integral techniques

11.5.2.1  Physical observation

11.5.2.2  Rayleigh-Ritz projection method

11.5.2.3  Beyn algorithm

11.5.2.4  Timing: OBC improvement

11.6  Solution of the Wave Function equation

11.6.1  SplitSolve algorithm

11.6.1.1  Porting WF to GPUs

11.6.1.2  SplitSolve Algorithm

11.6.1.3  Timing: SplitSolve

## Introduction (19/2)

- [x] ***processed***

### Thermal effects: A historical perspective

#### Major milestones

##### Ancient times
##### Early modern period
##### Modern times

### Why thermal modeling?
#### Heating: Good and Bad

#### Exposure/implant safety assessment
##### Exposure safety
##### Implant safety

#### Thermal therapies
##### Treatment planning
##### Novel therapies and devices
##### Bio-heat modeling

#### Self-heating in nanoscale devices
##### CBRAM device
##### Nanodevice design work flow
##### Electron-Phonon energy exchange
##### Test device structures
##### Influence of self-heating

## Thermal Modeling (Bio) (26/2)

- [x] ***processed***

### The Pennes Bioheat equation

#### Continuum level thermal modeling

#### Steady state

#### Boundary conditions in the PBE

#### Natural and forced convection

#### Heat radiation

#### Terms of the PBE
##### Diffusion
##### Exposure
##### Metabolism
##### Perfusion

#### High temperature limit

#### Linear regime: Considering only temperature increase

#### Power deposition
##### Exposure
##### RF Dosimetry

#### Beyond PBE

### Application: Mobile Phone Exposure Safety

#### Averaged SAR and temperature increase
#### Debates and considerations

## Thermal Modeling (Bio) (4/3)

- [X] ***processed***

### The UTCI model

#### Definition of the Universal Thermal Climate Index
##### Advantages
##### Disadvantages

#### Tuning & Validation

#### Discretization

#### Coordinate system

### Thermoregulation in UTCI

#### Perfusion

#### Heat exchange with environment
##### Convection
##### Radiation
##### Evaporation
##### Respiration

#### CNS response model

#### Sensation

#### Numerical implementation
##### Space
##### Time
##### Scheme

### Application: MRI exposure safety (RF)

#### MRI RF safety
##### Scenarios
##### MRI heating

#### Thermal dose

#### Thermal damaga

#### CEM43
##### Concept
##### Definition
##### Tresholds
##### Advantages of dose-based limits

## Numerical Methods (11/3)

- [X] ***processed***

### Discretization of continuum models
#### Structured grids

#### Unstructured meshes

### FDTD

#### Temporal discretization

#### Spatial discretization

#### Dealing with interfaces

#### Boundary conditions

#### Staircasing

#### Conformal corrections

#### Stability
##### CFL criterion

#### Other considerations

### FEM

#### Pros and cons
#### Overview of the Galerkin method

### Fast Poisson solvers

#### Green's functions

#### Green's function of the PBE

#### Overview of the Fast Poisson method

## Thermal Therapy Treatment Planning (18/3)

- [X] ***processed***

### Introduction

#### Hyperthermic oncology

#### HT energy delivery technology

#### Thermal ablation

#### Temperature modeling

#### Hyperthermic Treatment Planning (HTP)
##### Embedding in clinlcal flow
##### Steps in HTP
##### Motivation
##### Requirements
###### Determination of energy deposition
###### Temperature increase determination
###### Knowledge about tissue properties
###### Treatment optimization
###### Effect & outcome prediction
###### Clinical integration

### Imaging

#### Definition and objectives

#### Sim4Life framework

### Personalized model generation

#### Implementation

#### Considerations

#### Surface extraction and discretization
##### Implementation
##### Considerations

### EM modeling

#### Implementation

#### Considerations

### Acoustic modeling

#### Implementation

#### Considerations

### Thermal modeling

#### Implementation

#### Considerations

### The use of HPC

### Optimization

#### Objectives

#### Array steering optimization
##### Implementation
##### Considerations

### FUS steering

### Effect quantification

#### Implementation

#### Considerations

### Clinical integration

#### Implementation

#### Considerations

### Validation, Verification and Uncertainty Quantification

#### Verification

#### Validation
##### Local fields
##### In vivo heating

#### UQ

### Limitations and need for further research

#### Tissue parameters

#### Dynamic tissue models

#### Closed-loop treatment control

#### Clinical integration

#### Effectiveness and outcome prediction

#### Validation and quality assurance

### Conclusions

### Exercise: Fast temperature increase predictions for phased arrays

## Verification and Validation (25/3)

- [X] ***processed***

###  Background

#### Benefits of computational modeling in MedTech and computational life sciences

#### Computational modeling for regulatory submissions

#### Verification and validation standard (ASME, FDA)

#### Multi-physics

#### In vivo issues

#### Benchmarks

#### Context-of-use

#### Uncertainty and sensitivity analysis

### V & V example: FUS Simulator: Solver Verification & Validation
#### Verification
##### Critical effects for FDTD implementation of linear acoustic pressure wave model
##### Method of Manufactured Solutions

#### Validation
##### Gamma method
##### UA-Gamma Combination
##### Measurement setup
##### Results

### V & V example: MRI Induced Heating: Validation from EM Exposure to in vivo heating
#### Illustrating Example: MRI RF Exposure
#### MR RF Safety Background
##### Limits
##### Exposure
##### psSAR & T
##### T Distribution
##### pTx vs CP

#### Verification
##### Coil modeling with Huygens vs direct

#### Validation
##### Incident E/B-field
##### Local field - requirements
##### Local field - experimental setup
##### Local field - results

##### EM induced heating - setup
##### EM induced heating - results

##### In vivo heating

### V & V example: EM-neuron interaction: Getting the Quantity-of-interest wrong
#### EM induced neuro-stimulation
##### Considerations
##### Modeling
##### Results

### V & V example: Computational Human Phantoms
#### Verification
#### Validation

### Aspects of Quality Assurance

### Exercise: Method of Manufactured Solutions

## Nanoscale Thermal Modeling I (1/4)

- [X] ***processed***

### Learning goals
- [ ] the difference between a **temperature-** and a **phonon-based** modeling of heat
- [ ] when to use the laws of thermodynamics (lectures by Dr. Neufeld) and when to compute the phonon distribution function (this part)
- [ ] how to compute the **thermal conductivity** of systems and how it looks like in some materials
- [ ] what are the most relevant **scattering mechanisms** limiting the thermal conductivity of solids and how to describe them

### Summary
* Definition of thermal current / heat flow
* “Continuous vs. discrete” calculation of the temperature distribution in solid state-systems
* Introduction to the Boltzmann Transport Equation
* Extraction of the thermal conductivity

### Outline
* From temperature to phonons
    * When to use what
* Electron vs. phonon properties
    * Definition
    * Density-of-states
    * Occupancy
* Boltzmann Transport Equation (BTE)
    * Derivation of the physical model
    * Description of scattering mechanisms
    * Thermal conductivity examples

### From temperature to phonon

#### Introduction

* In solids, heat carried by **phonons**. **Exception**: In **metals, electrons** also contribute significantly
* **Phonons** == quantized lattice vibrations propagating as waves
* **Temperature T** == measure of atoms' deviation from equilibrium as they vibrate

#### Heat capacity

* characterizing physical property
* w/ $\Delta Q$ thermal energy, $\Delta T$ change of temperature, $V$ volume:
 $C_V = \frac{\Delta Q}{\Delta T}\Bigr|_{V=\text{const}}$
* **isochoric** == $(V=\text{const})$
* w/ $M$ mass: **specific heat capacity** $c_V = \frac{C_V}{M}$

#### Heat flow

##### Definition of heat flow

* **heat current** $J(c)$ flows from hot to cold in the form of **phonon flow**
* phonons move w/ average velocity $<\mathbf{v_x}>$ along $\mathbf{x}$
* phonons scatter at surfaces, defects, e/a other

##### Calculation of heat flow

We consider heat flow $\mathbf{J_x}$ through a plane $A$ at $x$:

and observe:

* at e/a point $\mathbf{x}$, thermal energy $\mathbf{Q(x)}$ has 50% chance to move L or R
* net heat flow $\mathbf{J(x)}$ is sum of contributions from both sides:
$\begin{align}
\mathbf{J(x)} & = \mathbf{J_{\text{L}\rightarrow\text{R}}(x-l_x) + J_{\text{R}\rightarrow\text{L}}(x+l_x)} \\
              & = \frac{1}{2} \cdot ((Q(x - l_x) - Q(x + l_x)) \cdot \left< v_x \right> \cdot \frac{1}{V}
\end{align}$

then using:

* $Q(x - l_x) - Q(x + l_x) \approx -2 \cdot l_x \cdot \frac{dQ(x)}{dx}$
* $Q(x) = f(T(x))$ and hence $\frac{dQ(x)}{dx} = \frac{dQ}{dT}\frac{dT}{dx}$
* for $\tau$ avg time btw two scattering events: mean free path along $x$ is $l_x = \left< v_x \right>\cdot\tau$
* heat capacity $C_V = \frac{dQ}{dT}$

we get:
$\begin{align}
J(x) & = - l_x \frac{dQ}{dx} \left< v_x \right> \frac{1}{V} \nonumber \\
J(x) & = - l_x \frac{dQ}{dT}\frac{dT}{dx} \left< v_x \right> \frac{1}{V} \nonumber \\
J(x) & = - \left< v_x \right>^2 \tau C_V \frac{dT}{dx} \frac{1}{V} \nonumber \\
J(x) & = - \frac{C_V}{V} {\left< v_x \right>}^2 \tau \frac{dT}{dx} \nonumber \\
J(x) & = - \kappa\frac{dT}{dx}
\end{align}$

where we defined $\kappa$ as the thermal conductivity:

$ \kappa = \frac{C_V}{V}{\left< v_x \right>}^2 \tau$

or in 3D w/ $\left< v_x \right> = \left< v_y \right> = \left< v_z \right>$ and $\mathbf{\left< v \right>}^2 = \left< v_x \right>^2 = \left< v_y \right>^2 = \left< v_z \right>^2$ and $\left< v \right> \tau = l_{\text{mfp}}$:

$ \kappa = \frac{C_V}{3V}{\left< v \right>} l_{\text{mfp}}$

#### Thermal conductivity

* **thermal conductivity** as defined above indicates how well a material conducts heat
* definition only valid if considered length $L \gg l_{\text{mfp}}$ the mean free path length: Macroscopic temperature $T$ unambigously defined
* $L \approx l_{\text{mfp}}$ requires **phonon population distribution**

### Electron vs. phonon

Property             | Electron                | Phonon
---------------------|-------------------------|-------
energy               | $E$                     | $\hbar\omega$
frequency            | $\frac{E}{\hbar}$       | $\omega$
momentum             | $k$                     | $q$
band structure       | $E(k) = \frac{\hbar^2 k^2}{2m^{*}}$ | $\omega(q)= \omega_{\text{opt}}, v_{\text{sound}}q$

#### Definitions

##### Electrons

##### Phonons

#### Bulk density-of-states (DOS)

##### Electrons

##### Phonons

#### Occupancy: Distribution function

##### Electrons

##### Phonons

#### Population

##### Electrons

##### Phonons

#### Current

##### Electrons

##### Phonons

#### Unoccupied state

##### Electrons

##### Phonons

### Boltzmann Transport Equation (BTE)

#### Derivation to physical model

##### Motivation

##### Phonon equation of motion

##### Statement of BTE

##### Deriving temperature from the BTE

##### Linearized phonon BTE

##### Thermal current - heat flow

##### Thermal conductivity

#### Description of scattering mechanisms

##### Relaxation time approximation

##### Examples of scattering

#### Examples of thermal conductivity

##### Thermal conductivity of Si

##### Thermal conductivity of Si NW

##### Thermal conductivity of InAs

## Nanoscale Thermal Modeling II: Phonon dispersion (8/4)

- [X] ***processed***

### Learning goals
- [ ] why the phonon bandstructure (dispersion) is a relevant quantity and what information can be extracted from it (density-of-states and velocity)
- [ ] how to compute the phonon dispersion from a simple one-dimensional atomic chain and what is the **dynamical matrix** (DM) of a system
- [ ] how to generally derive the DM of a solid from its **total energy** and the forces acting on each atom
- [ ] what is the difference between an empirical and quantum mechanical treatment

### Summary
* Calculation of phonon dispersion with 1-D model
* Extraction of DOS and phonon velocity
* Definition of Hessian and dynamical matrix
* Keating VFF model and its extension
* Phonon bandstructure computation with DFT

### Outline
* Phonon bandstructure
    * Why do we need this quantity?
* Classical phonon bandstructure calculation
    * Simple 1-D atomic chain
    * Dynamical matrix of the system
* Advanced phonon bandstructure calculation
    * Harmonic approximation Valence-force-field
    * (VFF) model
    * Density functional theory (DFT) model

### Phonon bandstructure

#### Why do we need this quantity?
##### Thermal conductivity recap

#### Ingredients for $\kappa$ calculation: Require phonon band structure
##### Phonon density-of-states
##### Phonon velocity
##### Relaxation time for scattering type (hard/not discussed)

#### Example (recap): Phonon dispersion in Si
##### Acoustic phonons
##### Optical phonons
##### Transverse branches
##### Longitudinal branches
##### Phonon velocity as a derivative of frequency w.r.t. momentum

#### Example: Other phonon dispersions
##### Polar semiconductor (InAs): LO/TO split
##### Graphene: ZA, ZO modes out-of-plane

### Classical phonon bandstructure calculation: Newton/Hooke

#### Simple 1-D atomic chain
##### Atom oscillations by Newton's EoM and Hooke's law
##### Unit cell of atomic chain
##### Period boundary conditions
##### Result: LSE w/ two coupled EoM

#### Dynamical matrix of the system from LSE of coupled EoM
##### Eigenvalue problem w/ $\omega^2$ as eigenvalue
##### Analytic solution:  Characteristic polynomial
##### Only positive eigenvalues admissible
##### Obtain two branches for 1D: 1 optical, 1 acoustic
##### Optical branch oscillation mode: out-of-phase motion
##### Acoustic branch oscillation mode:  in-phase motion
##### Extending from 1D to 3D: 6 branches but similar to 1D

#### 1D already allows extracting analytical parameters
##### DOS from phonon band structure
##### Injection velocity

#### Simplification by linearization
##### Phonon band structure
##### DOS
##### Injection velocity

### Advanced phonon bandstructure calculation: VFF and DFT

#### Harmonic approximation

##### Energy of solids and the Hamiltonian
##### Phonon amplitude affects total energy $E$
##### Harmonic approximation of the total energy: 2nd order
###### The force terms
###### The Hessian
##### Out-of-equilibrium: Force from integrating Hessian
##### The dynamical matrix: Eigenvalue problem as in 1D atomic chain
###### Acceleration from second derivative of phonon amplitude
###### Definition of phonon amplitude

#### Valence-force-field (VFF) model
##### VFF equals tight-binding for electrons: An empirical method
##### The Keating model
###### Bond stretching
###### Bond bending
###### Total energy in Keating model
###### Downside of Keating model: Poor fit to experiment
##### Extended VFF model
###### Model features
###### Cross bond stretching
###### Coplanar bond bending
###### Fit to experiment much improved over Keating model

#### Density functional theory (DFT) model
##### Schroedinger equation with $E_{xc}$
##### Charge density $\rho$
##### Exchange correlation energy functional $E_{xc}[\rho]$
###### LDA
###### GGA
###### HSE06
##### Frozen phonon approach
###### Total energy expression
###### Dynamical matrix formulation
###### Computational effort: 6N + 1 simulations
##### Near-perfect fit to experiment w/o fitting parameters
##### More efficient: density functional perturbation theory

## Phonon Quantum Transport (22/4)

- [X] ***processed***

### Learning goals
- [ ] what the different types of phonon **boundary conditions** are and what they physically represent
- [ ] how **semi-infinite contacts** must be attached to a system of interest to inject phonons on one side of the domain and collect them on the other
- [ ] how the **Wave Function (WF) formalism** allows to compute the phonon (quantum) transport properties of a structure driven out-of-equilibrium
- [ ] what the **Non-equilibrium Green’s Function (NEGF)** is and how it is related to WF

### Summary
* Boundary conditions for phonon calculations
* Phonon injection, reflection, and transmission
* Wave Function (WF) formalism for phonons
* Extraction of DOS, transmission, and heat current
* Phonon Non-equilibrium Green’s Functions

### Outline
* Boundary conditions
    * Periodic, closed, open
* Quantum transport approach
    * Definition of contacts/reservoirs
    * Derivation of the Wave Function formalism
    * DOS and transmission function
    * Non-equilibrium Green’s Function

### Boundary conditions

#### Intro/Recap
##### Semi-classical approach vs. QM approach for phonon transport
###### Semi-classical: Phonons as particles
###### QM: Phonons as waves

##### Recap: 1D phonon model
###### 1D atomic chain
###### Unit cell
###### Hooke's law

##### Phonon EoM
###### Ansatz

##### Generalized dynamical equation
###### General form
###### Matrix form

#### Periodic boundary conditions

##### Periodic boundary conditions: Infinite length
###### Eigenvalue problem

##### Simple phonon bandstructure
###### Acoustic branch: In phase
###### Optical branch: Out-of-phase
###### No phonon transport

#### Closed boundary conditions

##### Closed boundary conditions: Finite length
###### Acoustic branch: In phase
###### Optical branch: Out-of-phase
###### No phonon transport

##### Discrete phonon frequencies
###### Two discrete phonon frequency solutions

#### Open boundary conditions

##### Open boundary conditions: Quasi-infinite length
###### Neither periodic nor closed
###### Injection, Reflection, Transmission/Collection

### Quantum transport approach

#### Definition of contacts/reservoirs

##### Division of simulation domain
###### Injection
###### Reflection
###### Transmission

##### Contact properties
###### Semi-infiniteness
###### Linear combination of optical & acoustic phonon modes
###### Complex conjugate solution

##### Physical interpretation
###### Transmission-only assumption
###### Injection
###### Reflection

#### Derivation of the Wave Function formalism

##### Connection to device
###### Connecting cells
###### Infinite hierarchy

##### Hierarchy truncation
###### Using the injection/reflection coefficient formulation
###### Using semi-infiniteness

##### Variable elimination
###### Finding $\vec{q}_j^L$
###### Finding $\vec{b}_{Lj}$
###### Finding $\vec{a}_{Lj}$
###### Algorithm

##### Wave Function formalism
###### Dynamical matrix
###### Boundary self-energy
###### Injection vector
###### Matrix sizes

#### DOS and transmission function

##### Solution of the WF formalism
###### Formulation
###### Algorithm
###### Frequency- and position-resolved DOS
###### Transmission function

##### DOS and Transmission
###### DOS
###### Transmission function
###### Velocity ratio

##### Homogeneous system: Example
###### Contact band structure
###### DOS
###### Transmission function

##### Inhomogeneous system: Example
###### DOS
###### Transmission function

##### Heat current
###### Thermal current w/ temperature difference
###### Landauer-Büttiker formula
###### Bose-Einstein distribution function
###### Homogeneous chain carries more heat

#### Non-equilibrium Green’s Function

##### Green’s Functions in general
###### Operator
###### Kernel
###### Green's function formulation
###### Propagator interpretation
###### Correlation function interpretation
###### Recovery of original problem formulation

##### Phonon Green’s Functions
###### Retarded phonon Green's Function
###### Matrix formulation
###### Matrix size
###### Non-equilibrium (NEGF) formalism
###### Speed/effiecency: WF vs. NEGF
###### Other types of phonon Green's Functions
###### Advantages of phonon Green's Functions

## Thermoelectric Power Generators (29/4)

- [X] ***processed***

### Learning goals
- [ ] how the waste heat can be converted back to electricity with carefully designed structures
- [ ] how the efficiency of these thermoelectric generators can be evaluated and when they can be considered as good devices
- [ ] what the ZT figure of merit, the Seebeck coefficient, and the power factor are
- [ ] how nanostructuring has revolutionized the field of thermoelectricity since the beginning of the 90’s
- [ ] where the field currently stands

### Summary
* Energy consumption and waste heat
* Principle of thermoelectricity and basic device
* Voltage- and temperature-induced electrical and thermal currents flowing through a structure
* Calculation of the 𝑍𝑇 figure of merit

### Outline
* Thermoelectricity: an introduction
    * How to recycle waste heat?
    * ZT figure of merit
    * From bulk to nanostructures
* Mathematical formulation
    * Current and conductance
    * Electrical and thermal components
    * Simulation example

### Thermoelectricity: an introduction

#### How to recycle waste heat?
#####  Energy Consumption in US
###### Share of fossil fuels
###### Definition of quad
###### Share of energy wasted

#####  Waste heat
###### Waste heat key figures

#####  Thermoelectricity
###### Seebeck effect
###### Seeback coefficient
###### Thermoelectricity
###### Conversion efficiency

#### ZT figure of merit

#####  Thermoelectric generator
###### Nano-generators
###### Semiconductor thermoelectric generator principle of operation

#####  Thermoelectric efficiency
###### Carnot efficiency
###### Overall efficiency
###### $ZT$ vs. $\frac{\eta}{\eta_{\text{Carn}}}$ curve

#####  𝒁𝑻 definition and interpretation
###### ZT figure of merit
###### Seebeck coefficient
###### Electrical conductivity
###### Thermal conductivity
###### What is a a "good" thermoelectric material?

#####  Representative ZT examples
###### Sample ZT values
###### What is a "good" ZT value?
###### Problems with ZT values obtainable

#### From bulk to nanostructures
##### From bulk to nanostructures: Theory historical perspective
###### Key theory papers
###### Key ansatz

#####  Theory validation
###### Experimental confirmation of theory
###### Main results of validation

#####  Rough Si nanowires
###### Power factor
###### Thermal conductivity

### Mathematical formulation

#### Current and conductance

#####  Calculation of 𝒁𝑻
###### Ballistic limit assumption
###### Scattering: Phenomenological addition

#####  𝒁𝑻 ingredients
###### $I_{d,    \Delta V}$
###### $I_{d,    \Delta T}$
###### $I_{th,e, \Delta T}$
###### $I_{th,e, \Delta V}$
###### $I_{th,ph,\Delta T}$

#####  Current flows
###### Situation w/ $\Delta V$ applied
###### Situation w/ $\Delta T$ applied

#####  Conductance definition
###### Seebeck coefficient $[V/K$
###### Electrical conductance $[A/V, S]$
###### Electronic component of thermal conductance $[W/K]$
###### Phonon component of thermal conductance $[W/K]$

#### Electrical and thermal components

#####  Electrical currents by electrons
###### Return of Landauer-Büttiker
###### Return of Fermi-Dirac
###### Electron transmission function

#####  Electrical current linearization
###### Taylor expansion for small $\Delta V$
###### Taylor expansion for small $\Delta T$

#####  Thermal current from electrons
###### Once more Landauer-Büttiker with a twist
###### Thermal energy carried by a electron w/ energy $E$
###### Taylor expansion for small $\Delta V$
###### Taylor expansion for small $\Delta T$

#####  Simplification and generalization
###### Identity relation of derivatives of Fermi-Dirac
###### **K**ompact formulation of currents

#####  Thermal current from phonons
###### Thermal current from previous lecture L9
###### Taylor expansion for small $\Delta T$

#####  Calculation of 𝒁𝑻
###### Using simplifi**K**ation to express desired ingredients
###### Seebeck coeffcient
###### Electrical conductance
###### Electronic component of the thermal conductance
###### Phonon component of the thermal conductance

#### Simulation example

#####  𝒁𝑻 summary
###### Definition of ZT using $K_i$
###### Compromises needed: No ideal solution
###### Maximizing $K_1$
###### Low thermal conductivity vs. high electrical mobility

#####  Example: Si nanowire
###### Crystal axis <100>
###### Dimensions

#####  Electron/phonon transmission
###### Electron transmission function
###### Tight-binding model
###### Phonon transmission function
###### VFF model

#####  𝑲𝟎, 𝑲𝟏, 𝑲𝟐, and 𝑳𝟐
######  Characteristics of obtained values

#####  𝒁𝑻 results
###### Ideal assumptions, yet...

## Numerics and Parallelization (6/5)

- [X] ***processed***

### Learning goals
- [ ] how matrix multiplications or eigenvalue problems are handled under the hood in scripting languages such as Matlab or Python
- [ ] how to parallelize a scientific code with a shared or distributed memory strategy
- [ ] how graphics processing units (GPUs) can speed up a compute intensive application
- [ ] how the open boundary conditions and the resulting Wave Function equations can be efficiently solved with a dedicated algorithm

### Summary
* BLAS and LAPACK libraries
* Parallelization with OpenMP and MPI
* Matrix-matrix multiplication parallelization
* Acceleration of the Wave Function formalism with the Beyn and SplitSolve algorithms

### Outline
* Basic numerical operations
    * BLAS/LAPACK libraries Parallelization strategies
    * Graphics processing units (GPUs)
* Open boundary calculation
    * Generalized eigenvalue problem
    * Contour integral techniques
* Solution of the Wave Function equation
    * SplitSolve algorithm

### Basic numerical operations

#### BLAS/LAPACK libraries

##### Basic operations
###### Matlab
###### Numpy
###### C++

##### BLAS
###### Operands
###### Level 1/2/3

##### Eigenvalue (EV) problems
###### Typical algorithms

##### LAPACK
###### Operands
###### Examples

##### Calling BLAS/LAPACK
###### Headers
###### Drivers

##### Vendor libraries
###### Efficiency
###### Examples

#### Parallelization strategies

##### Shared vs. distributed memory

##### Shared memory parallelization
###### OpenMP

##### OpenMP example: 𝑪 = 𝑨𝑩

##### OpenMP timing results
###### ```OMP_NUM_THREADS```
###### Partitions

##### Distributed memory parallelization
###### SPMD
###### MPI

##### MPI example: 𝑪 = 𝑨𝑩

##### MPI timing results
###### Overhead

#### Graphics processing units (GPUs)

##### GPU architecture
###### Cores
###### Memory
###### Frequency

##### GPU example 𝑪 = 𝑨𝑩

##### GPU timing results
###### Multiple cores - 1 GPU
###### Speedup

### Open boundary calculation

#### Generalized eigenvalue problem

##### Phonon ballistic transport
###### WF phonon equation
###### OBC
###### Schrödinger equation w/ OBC
###### Direct solvers

##### Benchmark structure
###### Nanowire features
###### Simulation features

##### OBC formulation
###### Semi-infinite contacts
###### Injection, Reflection, Transmission
###### Non-linear eigenvalue problem

##### GEPV and shift-and-invert
###### Linearization of EVP
###### Shift-and-invert

##### Timing standard solution
###### 2 nodes for memory
###### 2 core OBC parallelization in 90s

#### Contour integral techniques
##### Physical observation
###### Neglecting certain states
###### Annulus

##### Rayleigh-Ritz projection method
###### Lower-dimensional subspace
###### Projector matrices

##### Beyn algorithm
###### Step 1: Projector $Q_i$
###### Step 2: SVD of $Q_0$
###### Step 3: Construction of intermediate M
###### Step 4: Solving reduced EVP
###### Quirks and features

##### Timing: OBC improvement
###### 2.35 speed-up of OBC

###  Solution of the Wave Function equation

#### SplitSolve algorithm
##### Porting WF to GPUs
###### Block tri-diagonal $A$
###### Sparse $Inj$
###### Spares $\Pi$
###### Sherman-Morrison-Woodbury on $A-BC$

##### SplitSolve Algorithm
###### Pre-processing
###### Post-processing
###### Key boost: OBC on CPU, SplitSolve on GPU in PARALLEL

##### Timing: SplitSolve
###### 4x speed-up, up to ~100x for larger systems

# References

Unless mentioned otherwise, all content and images from:
* ETHZ 227-0622-00L Thermal Modeling: Part I lecture notes by Dr. Esra Neufeld
* ETHZ 227-0622-00L Thermal Modeling: Part II lecture notes by Prof. Dr. Mathieu Luisier

```python

```


# Introduction To Causality

## Why Bother?

First and foremost, you might be wondering: what's in it for me? Here is what:

## Data Science is Not What it Used to Be (or it Finally Is)

Data Scientist has been labeled [The Sexiest Job of the 21st Century](https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century) by Harvard Business Review. This was no empty statement. For a decade now, Data Scientist has been at the spotlight. AI experts had [salaries that rivaled those of sports superstars](https://www.economist.com/business/2016/04/02/million-dollar-babies). In the search for fame and fortune, hundreds of young professionals entered into what seemed a frenetic golden rush to get the Data Science title as quickly as possible. Whole new industries sprang around the hype. Miraculous teaching methods could make you a Data Scientist without requiring you to look at a single math formula. Consulting specialists promised millions if your company could unlock the potential of data. AI or, Machine Learning, has been called the new electricity and data, the new oil.

Meanwhile, we kind of forgot about those that have been doing "old fashioned" science with data all along. During all this time, economists were trying to answer what is the true impact of education on one's earnings, biostatisticians were trying to understand if saturated fat led to higher chance of heart attack and psychologists were trying to understand if words of affirmation led indeed to a happier marriage. If we were to be completely honest, data scientist is not a recent field. We are simply aware of it just now due to the massive amount of free marketing the media has provided.

To use a Jim Collins analogy, think about pouring yourself an ice cold cup of your favorite beer. If you do this the right way, most of the cup will be beer but there will be a 1 finger thick layer of foam at the top. This cup is just like Data Science.

1. It's the beer. The statistical foundations, the scientific curiosity, the passion for difficult problems. All of this was proven very valuable throughout hundreds of years.
2. It's the foam. The fluffy stuff built on unrealistic expectations that will eventually go away.

This foam might come down crashing faster than you think. As The Economist puts it

> The same consultants who predict that AI will have a world-altering impact also report that real managers in real companies are finding AI hard to implement, and that enthusiasm for it is cooling. Svetlana Sicular of Gartner, a research firm, says that 2020 could be the year AI falls onto the downslope of her firm’s well-publicised “hype cycle”. Investors are beginning to wake up to bandwagon-jumping: a survey of European AI startups by MMC, a venture-capital fund, found that 40% did not seem to be using any AI at all.

In the midst of all this craze, what should we, as Data Scientists - or better yet, as "just" Scientists - do? As a starter, if you are smart, you will learn to ignore the foam. We are in it for the beer. Math and statistics has been useful since forever and it is unlikely it will stop now. Second, learn what makes your work valuable and useful, not the latest shining tool that no one figured out how to use.

Last but not least, remember that there are no shortcuts. Knowledge in Math and Statistics are valuable precisely because they are hard to acquire. If everyone could do it, excess supply would drive its price down. So **toughen up**! Learn them as well as you can. And heck, why not? have fun along the way as we embark on this quest only **for the Brave and True**.

## Answering a Different Kind of Question

The type of question Machine Learning is currently very good at answering is of the prediction kind. As Ajay Agrawal, Joshua Gans and Avi Goldfarb puts it in the book Prediction Machines, "the new wave of artificial intelligence does not actually bring us intelligence but instead a critical component of intelligence - prediction". We can do all sorts of wonderful things with machine learning. The only requirement is that we frame our problems as prediction ones. Want to translate from english to portuguese? Then build a ML model that predicts portuguese sentences when given english sentences. Want to recognize faces? Then build a ML model that predicts the presence of a face in a subsection of a picture. Want to build a self driving car? Then build one ML model to predict the direction of the wheel and the pressure on the brakes and accelerator when presented with images and sensors from the surroundings of a car.

However, ML is not a panacea. It can perform wonders under very strict boundaries and still fail miserably if the data it's using deviates a little from what the model is accustomed to. To give another example from Prediction Machines, "in many industries, low prices are associated with low sales. For example, in the hotel industry, prices are low outside the tourist season, and prices are high when demand is highest and hotels are full. Given that data, a naive prediction might suggest that increasing the price would lead to more rooms sold.”

ML is notoriously bad at this inverse causality type of problems. They require us to answer "what if" questions, what Economists call counterfactuals. What would happen if instead of this price I'm currently asking for my merchandise, I use another price? What would happen if instead of this low fat diet I'm in, I do a low sugar one? If you work in a bank, giving credit, you will have to figure out how changing the customer line changes your revenue. Or if you work at the local government, you might be asked to figure out how to make the schooling system better. Should you give tablets to every kid because the era of digital knowledge tells you to? Or should you build an old fashioned library?

At the heart of these questions there is a causal inquiry we wish to know the answer. Causal questions permeate everyday problems, like figuring out how to make sales go up, but they also play an important role on dilemmas that are very personal and dear to us: do I have to go to an expensive school to be successful in life (does education cause earnings)? Does imigration lowers my chances of getting a job (does imigration causes unemployment to go up)? Does money transfer to the poor lower the crime rate? It doesn't matter the field you are in, it is very likely you had or will have to answer some type of causal question. Unfortunately for ML, we can't rely on correlation type predictions to tackle them.

Answering this kind of question is tougher than most people appreciate. Your parents have probably repeated to you that "association is not causation", "association is not causation". But actually explaining why that is the case is a bit more involved. This is what this introduction to causal inference is all about. As for fhe rest of this book, it will be dedicated to **figuring how to make association be causation**.

## When Association IS Causation

Intuitively, we kind of know why association is not causation. If someone tells you that schools that give tablets to its students perform better than those who don't, you can quickly point out that it is probably the case that those schools with the tablets are richer. As such, they would do better than average even without the tablets. Because of this, we can't conclude that giving tablets to kids during classes will cause an increase in their academic performance. We can only say that tablets in school are associated with high academic performance.

```python
import pandas as pd
import numpy as np
from scipy.special import expit
import seaborn as sns
from matplotlib import pyplot as plt
from matplotlib import style

style.use("fivethirtyeight")

np.random.seed(123)
n = 100
tuition = np.random.normal(1000, 300, n).round()
tablet = np.random.binomial(1, expit((tuition - tuition.mean()) / tuition.std())).astype(bool)
enem_score = np.random.normal(200 - 50 * tablet + 0.7 * tuition, 200)
enem_score = (enem_score - enem_score.min()) / enem_score.max()
enem_score *= 1000

data = pd.DataFrame(dict(enem_score=enem_score, Tuition=tuition, Tablet=tablet))
```

```python
plt.figure(figsize=(6,8))
sns.boxplot(y="enem_score", x="Tablet", data=data).set_title('ENEM score by Tablet in Class')
plt.show()
```

To get beyond simple intuition, let's first establish some notation. This will be our common language to speak about causality. Think of it as the common tongue we will use to identify other brave and true causal warriors and that will compose our cry in the many battles to come.

Let's call \\(T_i\\) the treatment intake for unit i.

$
T_i=\begin{cases}
1 \ \text{if unit i received the treatment}\\
0 \ \text{otherwise}\\
\end{cases}
$

The treatment here doesn't need to be a medicine or anything from the medical field. Istead, it is just a term we will use to denote some intervention for which we want to know the effect. In our case, the treatment is giving tablets to students. As a side note, you might sometimes see \\(D\\) instead of \\(T\\) to denote the treatment.

Now, let's call \\(Y_i\\) the observed outcome variable for unit i.

$
Y_i
$

The outcome is our variable of interest. We want to know if the treatment has any influence in it. In our tablet example, it would be the academic performance.

Here is where things get interesting. The **fundamental problem of causal inference** is that we can never observe the same unit with and without treatment. It is as if we have two diverging roads and we can only know what lies ahead of the one we take. As in Robert Frost poem:

>Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;

To wrap our heads around this, we will talk a lot in term of **potential outcomes**. They are potential because they didn't actually happen. Instead they denote **what would have happened** in the case some treatment was taken. We sometimes call the potential outcome that happened, factual, and the one that didn't happen, counterfactual.

As for the notation, we use an additional subscript:

\\(Y_{0i}\\) is the potential outcome for unit i without the treatment.

\\(Y_{1i}\\) is the potential outcome for **the same** unit i with the treatment.

Sometimes you might see potential outcomes represented as functions \\(Y_i(t)\\), so beware. \\(Y_{0i}\\) could be \\(Y_i(0)\\) and \\(Y_{1i}\\) could be \\(Y_i(1)\\). Here, we will use the subscript notation most of the time.

Back to our example, \\(Y_{1i}\\) is the academic performance for student i if he or she has tablets at the classroom. Whether this is or not the case, it doesn't matter for \\(Y_{1i}\\). It is the same regardless. If student i gets the tablet, we can observe \\(Y_{1i}\\). If not, we can observe \\(Y_{0i}\\). Notice how in this last case, \\(Y_{1i}\\) is still defined, we just can't see it. In this case, it is a counterfactual potential outcome.

With potential outcomes, we can define the individual treatment effect:

 \\(Y_{1i} - Y_{0i}\\)

Of course, due to the fundamental problem of causal inference, we can never know the individual treatment effect because we only observe one of the potential outcomes. For the time being, let's focus on something easier than estimating the individual treatment effect. Instead, lets focus on the **average treatment effect**, which is defined as follows.

\\(ATE = E[Y_1 - Y_0]\\)

where, `E[...]` is the expected value. Another easier quantity to estimate is the **average treatment effect on the treated**:

\\(ATET = E[Y_1 - Y_0 | T=1]\\)

Now, I know we can't see both potential outcomes, but just for the sake of the argument, let's suppose we could. Pretend that the causal inference deity is pleased with the many statistical battles that we fought and has rewarded us with godlike powers to see the alternative potential outcomes. With that power, say we collect data on 4 schools. We know if they gave tablets to its students and their score on some annual academic test. Here, tablets are the treatment, so \\(T=1\\) if the school gives tablets to its kids. \\(Y\\) will be the test score.

```python
pd.DataFrame(dict(
    i= [1,2,3,4],
    y0=[500,600,800,700],
    y1=[450,600,600,750],
    t= [0,0,1,1],
    y= [500,600,600,750],
    te=[-50,0,-200,50],
))
```

      i
      y0
      y1
      t
      y
      te

      0
      1
      500
      450
      0
      500
      -50

      1
      2
      600
      600
      0
      600
      0

      2
      3
      800
      600
      1
      600
      -200

      3
      4
      700
      750
      1
      750
      50

The \\(ATE\\) here would be the mean of the last column, that is, of the treatment effect:

\\(ATE=(-50 + 0 - 200 + 50)/4 = -50\\)

This would mean that tablets reduced the academic performance of students, on average, by 50 points. The \\(ATET\\) here would be the mean of the last column when \\(T=1\\):

\\(ATET=(- 200 + 50)/2 = -75\\)

This is saying that, for the schools that were treated, the tablets reduced the academic performance of students, on average, by 75 points. Of course we can never know this. In reality, the table above would look like this:

```python
pd.DataFrame(dict(
    i= [1,2,3,4],
    y0=[500,600,np.nan,np.nan],
    y1=[np.nan,np.nan,600,750],
    t= [0,0,1,1],
    y= [500,600,600,750],
    te=[np.nan,np.nan,np.nan,np.nan],
))
```

      i
      y0
      y1
      t
      y
      te

      0
      1
      500.0
      NaN
      0
      500
      NaN

      1
      2
      600.0
      NaN
      0
      600
      NaN

      2
      3
      NaN
      600.0
      1
      600
      NaN

      3
      4
      NaN
      750.0
      1
      750
      NaN

This is surely not ideal, you might say, but can't I still take the mean of the treated and compare it to the mean of the untreated? In other words, can't I just do \\(ATE=(600+750)/2  - (500 + 600)/2  = 125\\)? Well, no! Notice how different the results are. That's because you've just committed the gravest sin of mistaking association for causation. To understand why, let's take a look into the main enemy of causal inference.

## Bias

Bias is what makes association different from causation. Fortunately, it too can be easily understood with our intuition. Let's recap our tablets in the classroom example. When confronted with the claim that schools that give tablets to their kids do better on tests score, we can rebut it by saying those schools will probably do better anyway, even without the tables. That is because they probably have more money than the other schools; hence they can pay better teachers, afford better classrooms, and so on. In other words, it is the case that treated schools (with tablets) are not comparable with untreated schools.

To say this in potential outcome notation is to say that \\(Y_0\\) of the treated is different from the \\(Y_0\\) of the untreated. Remember that the \\(Y_0\\) of the treated **is counterfactual**. We can't observe it, but we can reason about it. In this particular case, we can even leverage our understanding of how the world works to go even further. We can say that, probably, \\(Y_0\\) of the treated is bigger than \\(Y_0\\) of the untreated schools. That is because schools that can afford to give tablets to their kids can also afford other factors that contribute to better test scores. Let this sink in for a moment. It takes some time to get used to talking about potential outcomes. Read this paragraph again and make sure you understand it.

With this in mind, we can show with very simple math why is it the case that association is not causation. Association is measured by \\(E[Y|T=1] - E[Y|T=0]\\). In our example, this is the average test score for the schools with tablets minus the average test score for those without it. On the other hand, causation is measured by \\(E[Y_1 - Y_0]\\).

To see how they relate, let's take the association measurement and replace the observed outcomes with the potential outcomes. For the treated, the observed outcome is \\(Y_1\\). For the untreated, the observed outcome is \\(Y_0\\).

$
E[Y|T=1] - E[Y|T=0] = E[Y_1|T=1] - E[Y_0|T=0]
$

Now, let's add and subtract \\(E[Y_0|T=1]\\). This is a counterfactual outcome. It tells what would have been the outcome of the treated, had they not received the treatment.

$
E[Y|T=1] - E[Y|T=0] = E[Y_1|T=1] - E[Y_0|T=0] + E[Y_0|T=1] - E[Y_0|T=1]
$

Finally, we reorder the terms, merge some expectations, and lo and behold:

$
E[Y|T=1] - E[Y|T=0] = \underbrace{E[Y_1 - Y_0|T=1]}_{ATET} + \underbrace{\{ E[Y_0|T=1] - E[Y_0|T=0] \}}_{BIAS}
$

This simple piece of math encompasses all the problems we will encounter in causal questions. I cannot stress how important it is that you understand every aspect of it. If you're ever forced to tattoo something on your arm, this equation should be a good candidate for it. It's something to hold onto very dearly and really understand what is telling us, like some sacred text that can be interpreted 100 different ways. In fact, let's take a deeper look. Let's break it down into some of its implications. First, this equation tells why association is not causation. As we can see, association is equal to the treatment effect on the treated plus a bias term. **The bias is given by how the treated and control group differ before the treatment, that is, in case neither of them has received the treatment**. We can now say precisely why we are suspicious when someone tells us that tablets in the classroom boost academic performance. We think that, is this example, \\(E[Y_0|T=0] < E[Y_0|T=1]\\), that is, schools that can afford to give tablets to their kids are better than those that can't, **regardless of the tablets treatment**.

Why does this happen? We will talk more about that once we enter confounding, but for now you can think of bias arising because many things we can't control are changing together with the treatment. As a result, the treated and untreated schools don't differ only on the tablets. They also differ on the tuition cost, location, teachers...
For us to say that tablets in the classroom increase academic performance, we would need for schools with and without them to be, on average, similar to each other.

```python
plt.figure(figsize=(10,6))
sns.scatterplot(x="Tuition", y="enem_score", hue="Tablet", data=data, s=70).set_title('ENEM score by Tuition Cost')
plt.show()
```

Now that we understand the problem, let's look at the solution. We can also say what would be necessary to make association equal to causation. **If \\(E[Y_0|T=0] = E[Y_0|T=1]\\), then, association IS CAUSATION!** Understanding this is not just remembering the equation. There is a strong intuitive argument here. To say that \\(E[Y_0|T=0] = E[Y_0|T=1]\\) is to say that treatment and control group are comparable before the treatment. Or, in the case that the treated had not been treated, if we could observe its \\(Y_0\\), then its outcome would be the same as the untreated. Mathematically, the bias term would vanish:

$
E[Y|T=1] - E[Y|T=0] = E[Y_1 - Y_0|T=1]
$

Also, if the treated and the untreated only differ on the treatment itself, that is,  \\(E[Y_0|T=0] = E[Y_0|T=1]\\)

We have that the causal impact on the treated is the same as in the untreated (because they are very similar).

$
\begin{align}
E[Y_1 - Y_0|T=1] &= E[Y_1|T=1] - E[Y_0|T=1] \\
&= E[Y_1|T=1] - E[Y_0|T=0] \\
&= E[Y|T=1] - E[Y|T=0]
\end{align}
$

Hence, in this case, the **difference in means BECOMES the causal effect**:

$
E[Y|T=1] - E[Y|T=0] = ATE = ATET
$

Once again, this is so important that I think it is worth going over it again, now with pretty pictures. If we do a simple average comparison between the treatment and the untreated group, this is what we get (blue dots didn't receive the treatment, that is, the tablet):

Notice how the difference in outcomes between the two groups can have two causes:

1. The treatment effect. The increase in test score that is caused by giving kids tablets.
2. Other differences between the treatment and untreated that are NOT the treatment itself. In this case, treated and untreated differ in the sense that the treated have a much higher tuition price. Some of the difference in test scores can be due to the effect of tuition price on better education.

The true treatment effect can only be obtained if we had godlike powers to observe the potential outcome, like in the left figure below. The individual treatment effect is the difference between the unit's outcome and another theoretical outcome that the same unit would have in case it got the alternative treatment. These are the counterfactual outcomes and are denoted in light color.

In the right plot, we depicted what is the bias that we’ve talked about before. We get the bias if we set everyone to not receive the treatment. In this case, we are only left with the \\(T_0\\) potential outcome. Then, we see how the treated and untreated groups differ. If they do, it means that something other than the treatment is causing the treated and untreated to be different. This something is the bias and is what shadows the true treatment effect.

Now, contrast this with a hypothetical situation where there is no bias. Suppose that tablets are randomly assigned to schools. In this situation, rich and poor schools have the same chance of receiving the treatment. Treatment would be well distributed across all the tuition spectrum.

In this case, the difference in the outcome between treated and untreated IS the average causal effect. This happens because there is no other source of difference between treatment and untreated other than the treatment itself. All the differences we see must be attributed to it. Another way to say this is that there is no bias.

If we set everyone to not receive the treatment in such a way that we only observe the \\(Y_0\\)s, we would find no difference between the treated and untreated groups.

This is the herculean task causal inference is all about. It is about finding clever ways of removing bias, of making the treated and the untreated comparable so that all the difference we see between them is only the average treatment effect. Ultimately, causal inference is about figuring out how the world really works, stripped of all delusions and misinterpretations. And now that we understand this, we can move forward to mastering some of the most powerful methods to remove bias, the weapons of the Brave and True to identify the causal effect.

## Key Ideas

So far, we've seen that association is not causation. Most importantly, we've seen precisely why it isn't and how can we make association be causation. We've also introduced the potential outcome notation as a way to wrap our head around causal reasoning. With it, we saw statistics as two potential realities: one in which the treatment is given and another in which it is not. But, unfortunately, we can only measure one of them, and that is where the fundamental problem of causal inference lies.

Moving forward, we will see some of the basic techniques to estimate causal effect, starting with the golden standard of a randomised trial. I'll also review some statistical concepts as we go. I'll end with a quote often used in causal inference classes, taken from a kung-fu series:

> 'What happens in a man's life is already written. A man must move through life as his destiny wills.' -Caine
'Yes, yet each man is free to live as he chooses. Though they seem opposite, both are true.' -Old Man

## References

I like to think of this book as a tribute to Joshua Angrist, Alberto Abadie and Christopher Walters for their amazing Econometrics class. Most of the ideas here are taken from their classes at the American Economic Association. Watching them is what's keeping me sane during this tough year of 2020.
* [Cross-Section Econometrics](https://www.aeaweb.org/conference/cont-ed/2017-webcasts)
* [Mastering Mostly Harmless Econometrics](https://www.aeaweb.org/conference/cont-ed/2020-webcasts)

I'll also like to reference the amazing books from Angrist. They have shown me that Econometrics, or 'Metrics as they call it, is not only extremely useful but also profoundly fun.

* [Mostly Harmless Econometrics](https://www.mostlyharmlesseconometrics.com/)
* [Mastering 'Metrics](https://www.masteringmetrics.com/)

My final reference is Miguel Hernan and Jamie Robins' book. It has been my trustworthy companion in the most thorny causal questions I had to answer.

* [Causal Inference Book](https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/)

The beer analogy was taken from the awesome [Stock Series](https://jlcollinsnh.com/2012/04/15/stocks-part-1-theres-a-major-market-crash-coming-and-dr-lo-cant-save-you/), by JL Colins. This is an absolute must read for all of those wanting to learn how to productively invest their money.

## Contribute

Causal Inference for the Brave and True is an open-source material on causal inference, the statistics of science. It uses only free software, based in Python. Its goal is to be accessible monetarily and intellectually.
If you found this book valuable and you want to support it, please go to [Patreon](https://www.patreon.com/causal_inference_for_the_brave_and_true). If you are not ready to contribute financially, you can also help by fixing typos, suggesting edits or giving feedback on passages you didn't understand. Just go to the book's repository and [open an issue](https://github.com/matheusfacure/python-causality-handbook/issues). Finally, if you liked this content, please share it with others who might find it useful and give it a [star on GitHub](https://github.com/matheusfacure/python-causality-handbook/stargazers).


```python
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.font_manager as mfm
import os
import numpy as np
import scipy as sp
from datetime import timedelta
import yfinance as yf
```

```python
_CHN_FONT_ = None
_FONT_PROP_ = None
_CHN_EN_DICT_ = './data/locationDict.csv'
_CHN_LatLng_DICT_ = './data/lat_lng.csv'
# The province names in China
# Ignore Hong Kong, Macau, Taiwan and Tibet
# English name
names_province = ['Anhui', 'Beijing', 'Chongqing', 'Fujian', 'Gansu', 'Guangdong', 'Guangxi', 'Guizhou',
                  'Hainan', 'Hebei', 'Heilongjiang', 'Henan', 'Hubei', 'Hunan', 'Inner Mongolia', 'Jiangsu',
                  'Jiangxi', 'Jilin', 'Liaoning', 'Ningxia', 'Qinghai', 'Shaanxi', 'Shandong', 'Shanghai',
                  'Shanxi', 'Sichuan', 'Tianjin', 'Xinjiang', 'Yunnan', 'Zhejiang']
# Chinese name
names_province_cn = ['安徽省', '北京市', '重庆市', '福建省', '甘肃省', '广东省', '广西壮族自治区', '贵州省',
                     '海南省', '河北省', '黑龙江省', '河南省', '湖北省', '湖南省', '内蒙古自治区', '江苏省',
                     '江西省', '吉林省', '辽宁省', '宁夏回族自治区', '青海省', '陕西省', '山东省', '上海市',
                     '山西省', '四川省', '天津市', '新疆维吾尔自治区', '云南省', '浙江省']

# A one to one corrspondence with the names used by Pyecharts
pyecharts_province_dict = {'北京市':'北京', '天津市':'天津', '河北省':'河北', '山西省':'山西',
                           '内蒙古自治区':'内蒙古', '辽宁省':'辽宁', '吉林省':'吉林', '黑龙江省':'黑龙江',
                           '上海市':'上海', '江苏省':'江苏', '浙江省':'浙江', '安徽省':'安徽',
                           '福建省':'福建', '江西省':'江西', '山东省':'山东', '河南省':'河南',
                           '湖北省':'湖北','湖南省':'湖南', '广东省':'广东', '广西壮族自治区':'广西',
                           '海南省':'海南', '重庆市':'重庆', '四川省':'四川', '贵州省':'贵州',
                           '云南省':'云南', '西藏自治区':'西藏', '陕西省':'陕西', '甘肃省':'甘肃',
                           '青海省':'青海', '宁夏回族自治区':'宁夏', '新疆维吾尔自治区':'新疆',
                           '香港特别行政区':'香港', '澳门特别行政区':'澳门','台湾省':'台湾'}

# English names of the capital cities of provinces
provincial_capital_dict = {'Anhui':'Hefei', 'Fujian':'Fuzhou', 'Gansu':'Lanzhou','Guangdong':'Guangzhou','Guizhou':'Guiyang',
                           'Hainan':'Haikou', 'Hebei':'Shijiazhuang','Heilongjiang':'Harbin', 'Henan':'Zhengzhou',
                           'Hubei':'Wuhan', 'Hunan':'Changsha', 'Jiangsu':'Nanjing', 'Jiangxi':'Nanchang',
                           'Jilin':'Changchun', 'Liaoning':'Shenyang', 'Qinghai':'Xining', 'Shaanxi':'Xi’an',
                           'Shandong':'Jinan', 'Shanxi':'Taiyuan', 'Sichuan':'Chengdu', 'Yunnan':'Kunming',
                           'Zhejiang':'Hangzhou', 'Guangxi':'Nanning','Inner Mongolia':'Hohhot', 'Ningxia':'Yinchuan',
                           'Xinjiang':'Ürümqi', 'Tibet':'Lhasa', 'Beijing':'Beijing', 'Chongqing':'Chongqing',
                           'Shanghai':'Shanghai', 'Tianjin':'Tianjin'}

# Populations of provinces
# data source: http://www.chamiji.com/2019chinaprovincepopulation
# unit 10,000
provincial_population_dict = {'Anhui':6323.6, 'Fujian':3941, 'Gansu': 2637.26,'Guangdong':11346,'Guizhou':3600,
                           'Hainan':934.32, 'Hebei':7556.30,'Heilongjiang':3773.1, 'Henan':9605,
                           'Hubei':5917, 'Hunan':6898.8, 'Jiangsu':8050.7, 'Jiangxi':4647.6,
                           'Jilin':2704.06, 'Liaoning':4359.3, 'Qinghai':603.23, 'Shaanxi':3864.4,
                           'Shandong':10047.24, 'Shanxi':3718.34, 'Sichuan':8341, 'Yunnan':4800.5,
                           'Zhejiang':5737, 'Guangxi':4926,'Inner Mongolia':2534.0, 'Ningxia':688.11,
                           'Xinjiang':2486.76, 'Tibet':337.15, 'Beijing':2154.2, 'Chongqing': 3101.79,
                           'Shanghai':2423.78, 'Tianjin':1559.60}
# Populations of cities
# data source: http://www.chamiji.com/
# unit 10,000
city_population_dict = {'北京市':2154.2, '天津市':1559.60, '上海市':2423.78, '重庆市':3101.79,
                        '亳州市':523.7, '六安市':483.7, '合肥市':808.7, '安庆市':469.1, '宣城市':264.8, '宿州市':568.14, # 安徽省
                        '池州市':147.4, '淮北市':225.4, '淮南市':349.0, '滁州市':411.4, '芜湖市':374.8, '蚌埠市':339.20,
                        '铜陵市':162.9, '阜阳市':820.7, '马鞍山市':233.7, '黄山市':140.7,
                        '南京市':843.62, '南通市':731, '宿迁市':492.59, '常州市':472.9, '徐州市':880.20, '扬州市':453.1, # 江苏省
                        '无锡市':657.45, '泰州市':463.57, '淮安市':492.50, '盐城市':720, '苏州市':1072.17, '连云港市':452.0,
                        '镇江市':319.64,
                        '东营市':217.21, '临沂市':1062.4, '威海市':283, '德州市':581, '日照市':293.03, '枣庄市':392.73, # 山东省
                        '泰安市':564.0, '济南市':746.04, '济宁市':834.59, '淄博市':470.2, '滨州市':392.25, '潍坊市':937.3,
                        '烟台市':712.18, '聊城市':606.43, '菏泽市':876.5, '青岛市':939.48,
                        '丽水市':219.25, '台州市':613.90, '嘉兴市':472.6, '宁波市':820.2, '杭州市':980.6, '温州市':925, # 浙江省
                        '湖州市':301.09, '绍兴市':503.5, '舟山市':117.3, '衢州市':220.9, '金华市':560.4,
                        '三明市':258, '南平市':269, '厦门市':411, '宁德市':291, '泉州市':870, '漳州市':514.0, # 福建省
                        '福州市':774, '莆田市':290.0, '龙岩市':264,
                        '东莞市':834.25, '中山市':326, '云浮市':326, '佛山市':765.67, '广州市':1449.84, '惠州市':477.70, # 广东省
                        '揭阳市':608.6, '梅州市':437.43, '汕头市':560.82, '汕尾市':305.33, '江门市':456.17, '河源市':309.11,
                        '深圳市':1252.83, '清远市':386.0, '湛江市':730.5, '潮州市':265.08, '珠海市':176.54, '肇庆市':411.54,
                        '茂名市':620.41, '阳江市': 254.29, '韶关市':297.92,
                        '北海市':166.33, '南宁市':715.33, '崇左市':208.68, '来宾市':221.86, '柳州市':400.00, '桂林市':505.75, # 广西壮族自治区
                        '梧州市':303.7, '河池市':352.35, '玉林市': 581.08, '百色市':364.65, '贵港市':437.54, '贺州市':205.67,
                        '钦州市':328, '防城港市':94.02,
                        '万宁市':float("NaN"), '三亚市':76.42, '儋州市':90.57, '海口市':227.21, '三沙市':float("NaN"), '东方市':float("NaN"), # 海南省 ##########
                        '临高县':float("NaN"), '乐东黎族自治县':float("NaN"),'五指山市':float("NaN"),'保亭黎族苗族自治县':float("NaN"), '定安县':float("NaN"),'屯昌县':float("NaN"),
                        '文昌市':float("NaN"),'昌江黎族自治县':float("NaN"),'澄迈县':float("NaN"),'琼中黎族苗族自治县':float("NaN"),'琼海市':float("NaN"),'白沙黎族自治县':float("NaN"),
                        '陵水黎族自治县':float("NaN"),
                        '保定市':1046.92, '唐山市': 789.7, '廊坊市':467.8, '张家口市':443.3, '承德市':356.50, '沧州市':755.49, # 河北省
                        '石家庄市':1087.99, '秦皇岛市':311.08, '衡水市':446.04, '邢台市':735.16, '邯郸市':951.11,
                        '乌兰察布市':210.25, '乌海市':56.11, '兴安盟':160.42, '包头市':287.8, '呼伦贝尔市':252.92, '呼和浩特市':311.5, # 内蒙古自治区
                        '巴彦淖尔市':168.5, '赤峰市':431.5, '通辽市':312.87, '鄂尔多斯市':206.87, '锡林郭勒盟':105.16, '阿拉善盟':24.8,
                        '临汾市':450.03, '吕梁市':388.56, '大同市':345.60, '太原市':442.15, '忻州市':317.20, '晋中市':338.15, # 山西省
                        '晋城市':234.31, '朔州市':178.12, '运城市':535.97, '长治市':346.8, '阳泉市':141.44,
                        '三门峡市':227.29, '信阳市':647.41, '南阳市':1001.36, '周口市':867.78, '商丘市':732.53, '安阳市':517.6, # 河南省
                        '平顶山市':520.77, '开封市':456.49, '新乡市':579.41, '洛阳市':688.85, '济源市':73.27, '漯河市':266.53,
                        '濮阳市':360.94, '焦作市':359.07, '许昌市':443.74, '郑州市':1013.6, '驻马店市':619.02, '鹤壁市':162.73,
                        '娄底市':393.18, '岳阳市':579.71, '常德市':582.7, '张家界市':153.79, '怀化市':497.96, '株洲市':402.08, # 湖南省
                        '永州市':545.21, '湘潭市':286.5, '湘西土家族苗族自治州':264.95, '益阳市':441.38, '衡阳市':724.34, '邵阳市':737.05,
                        '郴州市':474.5, '长沙市':815.47,
                        '仙桃市':154.45, '十堰市':341.8, '咸宁市':253.51, '天门市':128.35, '孝感市':491.50, '宜昌市':413.56, # 湖北省 ##########
                        '恩施土家族苗族自治州':336.10, '武汉市':1089.29, '潜江市': float("NaN"), '神农架林区':7.68, '荆州市':564.17, '荆门市':290.15,
                        '襄阳市':565.4, '鄂州市':107.69, '随州市':221.05, '黄冈市':634.1, '黄石市':247.05,
                        '上饶市':678.34, '九江市':487.33, '南昌市':546.35, '吉安市':494.19, '宜春市':555.37, '抚州市':403.10, # 江西省 2018
                        '新余市':118.07, '景德镇市':166.49, '萍乡市':192.50, '赣州市':861.2, '鹰潭市':116.75,
                        '六盘水市':292.41, '安顺市':234.44, '毕节市':665.97, '贵阳市':480.20, '遵义市':624.83, '铜仁市':315.69, # 贵州省 2018
                        '黔东南苗族侗族自治州':352.37, '黔南布依族苗族自治州':327.1, '黔西南布依族苗族自治州':286,
                        '乐山市':327.21, '内江市':375.37, '凉山彝族自治州':521.29, '南充市':641.79, '宜宾市':453, '巴中市':331.67, # 四川省 2018
                        '广元市':266.00, '广安市':325.0, '德阳市':353.2, '成都市':1604.5, '攀枝花市':123.61, '泸州市':431.72,
                        '甘孜藏族自治州':118.63, '眉山市':297.48, '绵阳市':483.56, '自贡市':290.14, '资阳市':255.3, '达州市':568.95,
                        '遂宁市':323.59, '阿坝藏族羌族自治州':94.01, '雅安市':153.78,
                        '临沧市':252.60, '丽江市':129.0, '保山市':261.4, '大理白族自治州':361.88, '德宏傣族景颇族自治州':130.90, '怒江傈僳族自治州':54.7, # 云南省 2018
                        '文山壮族苗族自治州':363.6, '昆明市':678.3, '昭通市': 553.7, '普洱市':262.7, '曲靖市':612.2, '楚雄彝族自治州':274.40,
                        '玉溪市':238.1, '红河哈尼族彝族自治州':471.3, '西双版纳傣族自治州':118.0, '迪庆藏族自治州':41.2,
                        '咸阳市':437.6, '商洛市':238.13, '安康市':266.1, '宝鸡市':378.10, '延安市':226.31, '榆林市':340.33, # 陕西省
                        '汉中市':344.93, '渭南市':538.29, '西安市':953.44, '铜川市':83.34,
                        '临夏回族自治州':204.41, '兰州市':372.96, '嘉峪关市':24.98, '天水市':333.98, '定西市':280.84, '平凉市':211.28, # 甘肃省
                        '庆阳市':200.55, '张掖市':122.93, '武威市':182.53, '甘南藏族自治州':74.23, '白银市':172.93, '酒泉市':112.36,
                        '金昌市':46.92, '陇南市':287.42,
                        '果洛藏族自治州':20.57, '海东市':147.08, '海北藏族自治州':28.3, '海南藏族自治州':47.24, '海西蒙古族藏族自治州':51.52, '玉树藏族自治州':40.95, # 青海省
                        '西宁市':235.50, '黄南藏族自治州':27.42,
                        '乌鲁木齐市':222.61, '五家渠市':float("NaN"), '伊犁哈萨克自治州':461.71, '克孜勒苏柯尔克孜自治州':62.06, '克拉玛依市':44.28, '北屯市':float("NaN"), # 新疆维吾尔自治区 2018 ##########
                        '博尔塔拉蒙古自治州':47.54, '双河市':float("NaN"), '可克达拉市':float("NaN"), '吐鲁番市':63.73, '和田地区':252.28, '哈密市':56.11,
                        '喀什地区':464.97, '图木舒克市':float("NaN"), '塔城地区':15.2, '巴音郭楞蒙古自治州':127.93, '昆玉市':float("NaN"), '昌吉回族自治州':161,
                        '石河子市':float("NaN"), '铁门关市':float("NaN"), '阿克苏地区':254.6, '阿勒泰地区':67.16, '阿拉尔市':float("NaN"),
                        '中卫市':115.75, '吴忠市':140.37, '固原市':122.82, '石嘴山市':80.29, '银川市':222.54, # 宁夏回族自治区 2018
                        '吉林市':415.35, '四平市':320.4, '延边朝鲜族自治州':210.14, '松原市':275.41, '白城市':190.9, '白山市':119.5, # 吉林省 2018
                        '辽源市':117.94, '通化市':217.15, '长春市':748.9,
                        '丹东市':239.5, '大连市':698.75, '抚顺市':210.7, '朝阳市':295, '本溪市':147.63, '沈阳市':829.4, # 辽宁省 2018
                        '盘锦市':143.65, '营口市':243.8, '葫芦岛市':277.0, '辽阳市':183.7, '铁岭市':299.8, '锦州市':296.4,
                        '阜新市':186.2, '鞍山市':344.0,
                        '七台河市':78.6, '伊春市':115.9, '佳木斯市':234.5, '双鸭山市':142.3, '哈尔滨市':955.0, '大兴安岭地区':43.93, # 黑龙江省 2018
                        '大庆市':273.1, '牡丹江市':254.8, '绥化市':527.6, '鸡西市':175, '鹤岗市':100.9, '黑河市':160.5,
                        '齐齐哈尔市':533.7
                       }

# Reset the unit of populations to be 1 instead of 10,000
provincial_population_dict.update((x, y*1e4) for x, y in provincial_population_dict.items())
city_population_dict.update((x, y*1e4) for x, y in city_population_dict.items())

# Obtain the latitude and longitude of a province from the local dictionary
# Save as a dictionary
def get_province_latlng_dict():
    province_latlng_dict = {}
    chn_latlng = pd.read_csv(_CHN_LatLng_DICT_, encoding='utf-8')
    for key in provincial_capital_dict:
        lat = chn_latlng[chn_latlng.city == provincial_capital_dict[key]]['lat'].tolist()[0]
        lng = chn_latlng[chn_latlng.city == provincial_capital_dict[key]]['lng'].tolist()[0]
        province_latlng_dict[key] = (lat, lng)
    return province_latlng_dict

# Save as a dataframe
def get_province_latlng_df():
    data = pd.DataFrame.from_dict(province_latlng_dict, orient='index', columns = ['lat', 'lng'])
    data = data.reset_index()
    data = data.rename(columns={'index':'province_name'})
    #data_gephi = data.rename(columns={'province_name':'id'})
    #data_gephi.to_csv(r'./data/data_network_P2P_gephi_id.csv', index = False)
    return data

province_latlng_dict = get_province_latlng_dict()
province_latlng_df = get_province_latlng_df()
```

```python
# Settings on the Chinese font
def set_font(font_file):
    if not os.path.exists(font_file):
        print(font_file + " not found.  If you wish to display Chinese characters in plots, please use set_font() to set the path to the font file.")
    else:
        global _CHN_FONT_, _FONT_PROP_
        _CHN_FONT_ = font_file
        _FONT_PROP_ = mfm.FontProperties(fname=_CHN_FONT_)
    return

set_font('./STFANGSO.TTF')   # for displaying Chinese characters in plots

def use_chn():
    return _CHN_FONT_ is None

# Add English name for a province or a city
def add_en_location(df, tag = 'city'):
    '''Add province_name_en, and city_name_en'''
    chn_en = pd.read_csv(_CHN_EN_DICT_, encoding='utf-8')
    translation = dict([t for t in zip(chn_en['Chinese'], chn_en['English'])])
    if tag == 'province':
        df['province_name_en'] = df['province_name'].replace(translation)
    elif tag == 'city':
        df['province_name_en'] = df['province_name'].replace(translation)
        df['city_name_en'] = df['city_name'].replace(translation)
    elif tag == 'network':
        df['source_en'] = df['source'].replace(translation)
        df['target_en'] = df['target'].replace(translation)
    return df
```

```python
# Data loading
_Data_PATH_ = './data/parameters/'
_Data_PATH_X_ = './data/parameters_x/'
_Data_PATH_XX_ = './data/parameters_xx/'
_Data_PATH_M_ = './data/parameters_m/'
_Data_PATH_MX_ = './data/parameters_mx/'
_Data_PATH_MXX_ = './data/parameters_mxx/'
_City_PATH_ = './data/data_DXY_city_all.csv'
_Province_PATH_ = './data/data_DXY_province_all.csv'
_Province_Domestic_PATH_ = './data/data_DXY_province_all_domestic.csv'
_Network_P2P_PATH_ = './data/data_network_P2P.csv'

def load_DXY_raw():
    raw_city = pd.read_csv(_City_PATH_)
    raw_province = pd.read_csv(_Province_PATH_)
    raw_province_domestic = pd.read_csv(_Province_Domestic_PATH_)
    raw_city['update_date'] = pd.to_datetime(raw_city['update_date'])
    raw_city['update_date'] = raw_city['update_date'].dt.date
    raw_province['update_date'] = pd.to_datetime(raw_province['update_date'])
    raw_province['update_date'] = raw_province['update_date'].dt.date
    raw_province_domestic['update_date'] = pd.to_datetime(raw_province_domestic['update_date'])
    raw_province_domestic['update_date'] = raw_province_domestic['update_date'].dt.date
    return raw_city, raw_province, raw_province_domestic

def load_network_raw():
    # Do not distinguish between move_in and move_out
    raw = pd.read_csv(_Network_P2P_PATH_)
    raw['update_date'] = pd.to_datetime(raw['update_date'])
    raw['update_date'] = raw['update_date'].dt.date
    return raw

def load_ind_simulation_raw():
    df_parameters_list = []
    df_estimation_list = []
    for name in names_province:
        df_parameters_single = pd.read_csv(_Data_PATH_ + name + '_parameters.csv')
        df_estimation_single = pd.read_csv(_Data_PATH_ + name + '_estimation.csv')
        df_parameters_list.append(df_parameters_single)
        df_estimation_list.append(df_estimation_single)
    return df_parameters_list, df_estimation_list

def load_ind_simulation_raw_x():
    df_parameters_list = []
    df_estimation_list = []
    for name in names_province:
        df_parameters_single = pd.read_csv(_Data_PATH_X_ + name + '_parameters.csv')
        df_estimation_single = pd.read_csv(_Data_PATH_X_ + name + '_estimation.csv')
        df_parameters_list.append(df_parameters_single)
        df_estimation_list.append(df_estimation_single)
    return df_parameters_list, df_estimation_list

def load_ind_simulation_raw_xx():
    df_parameters_list = []
    df_estimation_list = []
    for name in names_province:
        df_parameters_single = pd.read_csv(_Data_PATH_XX_ + name + '_parameters.csv')
        df_estimation_single = pd.read_csv(_Data_PATH_XX_ + name + '_estimation.csv')
        df_parameters_list.append(df_parameters_single)
        df_estimation_list.append(df_estimation_single)
    return df_parameters_list, df_estimation_list
```

```python
# Some utility functions
# Get the list of dates from the start_date to the end_date
def daterange(start_date, end_date):
    for n in range(int ((end_date - start_date).days) + 1):
        yield start_date + timedelta(n)

# Transform the migration edgelist to the migration matrix
def matrix_P2P(data_single, m):
    mindex = np.zeros((m, m))
    for i, name_i in enumerate(names_province):
        for j, name_j in enumerate(names_province):
            if i == j:
                mindex[i][j] = 0
            else:
                temp = data_single[(data_single.source_en == name_i) & (data_single.target_en == name_j)].value.tolist()
                if temp == []:
                    mindex[i][j] = 0
                else:
                    mindex[i][j] = temp[0]
    return mindex

# Perform the operation for all migration data from the start_date to the end_date
def matrix_P2P_all(data_network_P2P, start_date, end_date):
    mindex_list = []
    m = len(names_province)
    for item in daterange(start_date, end_date):
        data_single = data_network_P2P[data_network_P2P.update_date == item]
        mindex = matrix_P2P(data_single, m)
        mindex_list.append(mindex)
    return mindex_list
```

```python
# The state names in the US (including D.C.)
# Ignore oversea territories: American Samoa, Guam, Northern Mariana Islands, Puerto Rico, and Virgin Islands

names_state = ['Alabama', 'Alaska', 'Arizona', 'Arkansas', 'California', 'Colorado', 'Connecticut', 'Delaware',
                 'District of Columbia',
                 'Florida', 'Georgia', 'Hawaii', 'Idaho', 'Illinois', 'Indiana', 'Iowa', 'Kansas',
                 'Kentucky', 'Louisiana', 'Maine', 'Maryland', 'Massachusetts', 'Michigan', 'Minnesota', 'Mississippi',
                 'Missouri', 'Montana', 'Nebraska', 'Nevada', 'New Hampshire', 'New Jersey', 'New Mexico', 'New York',
                 'North Carolina', 'North Dakota', 'Ohio', 'Oklahoma', 'Oregon', 'Pennsylvania', 'Rhode Island', 'South Carolina',
                 'South Dakota', 'Tennessee', 'Texas', 'Utah', 'Vermont', 'Virginia', 'Washington', 'West Virginia',
                 'Wisconsin', 'Wyoming']

# The abbreviations of state names
names_state_short_dict = {
    'Alabama': 'AL', 'Alaska':'AK', 'Arizona':'AZ', 'Arkansas':'AR', 'California':'CA', 'Colorado':'CO', 'Connecticut':'CT', 'Delaware':'DE',
    'Florida':'FL', 'Georgia':'GA', 'Hawaii':'HI', 'Idaho':'ID', 'Illinois':'IL', 'Indiana':'IN', 'Iowa':'IA', 'Kansas':'KS',
    'Kentucky':'KY', 'Louisiana':'LA', 'Maine':'ME', 'Maryland':'MD', 'Massachusetts':'MA', 'Michigan':'MI', 'Minnesota':'MN', 'Mississippi':'MS',
    'Missouri':'MO', 'Montana':'MT', 'Nebraska':'NE', 'Nevada':'NV', 'New Hampshire':'NH', 'New Jersey':'NJ', 'New Mexico':'NM', 'New York':'NY',
    'North Carolina':'NC', 'North Dakota':'ND', 'Ohio':'OH', 'Oklahoma':'OK', 'Oregon':'OR', 'Pennsylvania':'PA', 'Rhode Island':'RI', 'South Carolina':'SC',
    'South Dakota':'SD', 'Tennessee':'TN', 'Texas':'TX', 'Utah':'UT', 'Vermont':'VT', 'Virginia':'VA', 'Washington':'WA', 'West Virginia':'WV',
    'Wisconsin':'WI', 'Wyoming':'WY',
    'District of Columbia': 'DC', 'Guam': 'Guam', 'Northern Mariana Islands':'Northern Mariana Islands',
    'Puerto Rico': 'PR', 'Virgin Islands': 'VI'}

# The populations of states
state_population_dict = {
    'Alabama': 4903185, 'Alaska':731545, 'Arizona':7278717, 'Arkansas':3017825,
    'California':39512223, 'Colorado':5758736, 'Connecticut':3565287, 'Delaware':973764,
    'Florida':21477737, 'Georgia':10617423, 'Hawaii':1415872, 'Idaho':1787065,
    'Illinois':12671821, 'Indiana':6732219, 'Iowa':3155070, 'Kansas':2913314,
    'Kentucky':4467673, 'Louisiana':4648794, 'Maine':1344212, 'Maryland':6045680,
    'Massachusetts':6949503, 'Michigan':9986857, 'Minnesota':5639632, 'Mississippi':2976149,
    'Missouri':6137428, 'Montana':1068778, 'Nebraska':1934408, 'Nevada':3080156,
    'New Hampshire':1359711, 'New Jersey':8882190, 'New Mexico':2096829, 'New York':19453561,
    'North Carolina':10488084, 'North Dakota':762062, 'Ohio':11689100, 'Oklahoma':3956971,
    'Oregon':4217737, 'Pennsylvania':12801989, 'Rhode Island':1059361, 'South Carolina':5148714,
    'South Dakota':884659, 'Tennessee':6833174, 'Texas':28995881, 'Utah':3205958,
    'Vermont':623989, 'Virginia':8535519, 'Washington':7614893, 'West Virginia':1792147,
    'Wisconsin':5822434, 'Wyoming':578759, 'District of Columbia': 705749, 'Guam': 165718,
    'Puerto Rico': 3193694, 'Virgin Islands': 104914}
```

```python
# The PVI values of states
df_state_pvi = pd.DataFrame(columns = ['state_name', 'partisan', 'value'])
df_state_pvi.loc[len(df_state_pvi)] = ['Alabama', 'R', 14]
df_state_pvi.loc[len(df_state_pvi)] = ['Alaska', 'R', 9]
df_state_pvi.loc[len(df_state_pvi)] = ['Arizona', 'R', 5]
df_state_pvi.loc[len(df_state_pvi)] = ['Arkansas', 'R', 15]

df_state_pvi.loc[len(df_state_pvi)] = ['California', 'D', 12]
df_state_pvi.loc[len(df_state_pvi)] = ['Colorado', 'D', 1]
df_state_pvi.loc[len(df_state_pvi)] = ['Connecticut', 'D', 6]
df_state_pvi.loc[len(df_state_pvi)] = ['Delaware', 'D', 6]

df_state_pvi.loc[len(df_state_pvi)] = ['Florida', 'R', 2]
df_state_pvi.loc[len(df_state_pvi)] = ['Georgia', 'R', 5]
df_state_pvi.loc[len(df_state_pvi)] = ['Hawaii', 'D', 18]

df_state_pvi.loc[len(df_state_pvi)] = ['Idaho', 'R', 19]
df_state_pvi.loc[len(df_state_pvi)] = ['Illinois', 'D', 7]

df_state_pvi.loc[len(df_state_pvi)] = ['Indiana', 'R', 9]
df_state_pvi.loc[len(df_state_pvi)] = ['Iowa', 'R', 3]
df_state_pvi.loc[len(df_state_pvi)] = ['Kansas', 'R', 13]
df_state_pvi.loc[len(df_state_pvi)] = ['Kentucky', 'R', 15]
df_state_pvi.loc[len(df_state_pvi)] = ['Louisiana', 'R', 11]
df_state_pvi.loc[len(df_state_pvi)] = ['Maine', 'D', 3]
df_state_pvi.loc[len(df_state_pvi)] = ['Maryland', 'D', 12]
df_state_pvi.loc[len(df_state_pvi)] = ['Massachusetts', 'D', 12]
df_state_pvi.loc[len(df_state_pvi)] = ['Michigan', 'D', 1]
df_state_pvi.loc[len(df_state_pvi)] = ['Minnesota', 'D', 1]
df_state_pvi.loc[len(df_state_pvi)] = ['Mississippi', 'R', 9]
df_state_pvi.loc[len(df_state_pvi)] = ['Missouri', 'R', 9]
df_state_pvi.loc[len(df_state_pvi)] = ['Montana', 'R', 11]
df_state_pvi.loc[len(df_state_pvi)] = ['Nebraska', 'R', 14]
df_state_pvi.loc[len(df_state_pvi)] = ['Nevada', 'D', 1]
df_state_pvi.loc[len(df_state_pvi)] = ['New Hampshire', 'Even', 0]
df_state_pvi.loc[len(df_state_pvi)] = ['New Jersey', 'D', 7]
df_state_pvi.loc[len(df_state_pvi)] = ['New Mexico', 'D', 3]
df_state_pvi.loc[len(df_state_pvi)] = ['New York', 'D', 12]
df_state_pvi.loc[len(df_state_pvi)] = ['North Carolina', 'R', 3]
df_state_pvi.loc[len(df_state_pvi)] = ['North Dakota', 'R', 17]
df_state_pvi.loc[len(df_state_pvi)] = ['Ohio', 'R', 3]
df_state_pvi.loc[len(df_state_pvi)] = ['Oklahoma', 'R', 20]
df_state_pvi.loc[len(df_state_pvi)] = ['Oregon', 'D', 5]
df_state_pvi.loc[len(df_state_pvi)] = ['Pennsylvania', 'Even', 0]
df_state_pvi.loc[len(df_state_pvi)] = ['Rhode Island', 'D', 10]
df_state_pvi.loc[len(df_state_pvi)] = ['South Carolina', 'R', 8]
df_state_pvi.loc[len(df_state_pvi)] = ['South Dakota', 'R', 14]
df_state_pvi.loc[len(df_state_pvi)] = ['Tennessee', 'R', 14]
df_state_pvi.loc[len(df_state_pvi)] = ['Texas', 'R', 8]
df_state_pvi.loc[len(df_state_pvi)] = ['Utah', 'R', 20]
df_state_pvi.loc[len(df_state_pvi)] = ['Vermont', 'D', 15]
df_state_pvi.loc[len(df_state_pvi)] = ['Virginia', 'D', 1]
df_state_pvi.loc[len(df_state_pvi)] = ['Washington', 'D', 7]
df_state_pvi.loc[len(df_state_pvi)] = ['West Virginia', 'R', 19]
df_state_pvi.loc[len(df_state_pvi)] = ['Wisconsin', 'Even', 0]
df_state_pvi.loc[len(df_state_pvi)] = ['Wyoming', 'R', 25]
df_state_pvi.loc[len(df_state_pvi)] = ['District of Columbia', 'D', 43]
df_state_pvi.loc[len(df_state_pvi)] = ['Guam', 'None', 0]
df_state_pvi.loc[len(df_state_pvi)] = ['Northern Mariana Islands', 'None', 0]
df_state_pvi.loc[len(df_state_pvi)] = ['Puerto Rico', 'None', 0]
df_state_pvi.loc[len(df_state_pvi)] = ['Virgin Islands', 'None', 0]

def color_pvi(df_state_pvi, state):

    palette_D = plt.get_cmap('Blues')
    palette_R = plt.get_cmap('Reds')
    palette_Even = plt.get_cmap('Greens')
    palette_None = plt.get_cmap('Purples')

    D_max = max(df_state_pvi[df_state_pvi['partisan'] == 'D']['value'])
    D_min = min(df_state_pvi[df_state_pvi['partisan'] == 'D']['value'])
    R_max = max(df_state_pvi[df_state_pvi['partisan'] == 'R']['value'])
    R_min = min(df_state_pvi[df_state_pvi['partisan'] == 'R']['value'])

    partisan = df_state_pvi[df_state_pvi['state_name'] == state]['partisan'].tolist()[0]
    value = df_state_pvi[df_state_pvi['state_name'] == state]['value'].tolist()[0]
    if partisan == 'Even':
        return palette_Even(0.6)
    elif partisan == 'None':
        return palette_None(0.6)
    elif partisan == 'D':
        return palette_D((value - D_min)/2/(D_max - D_min) + 0.3)
    else:
        return palette_R((value - R_min)/2/(R_max - R_min) + 0.3)
```

```python
# Stock market data
# S & P 500 index
def load_SP_raw():
    '''
    This provides a way to lookinto the 'raw' data
    '''
    SP = yf.Ticker("^GSPC")
    raw = SP.history(period="max")
    raw = raw.reset_index()
    rename_dict = {'Date': 'update_date',
                   'Open': 'open',
                   'High': 'high',
                   'Low': 'low',
                   'Close': 'close',
                   'Volume': 'volume',
                   'Dividends': 'dividends',
                   'Stock Splits': 'stock splits'
                  }
    data = raw.rename(columns=rename_dict)
    data['update_date'] = pd.to_datetime(data['update_date'])  # original type of update_time after read_csv is 'str'
    data['update_date'] = data['update_date'].dt.date
    data = data.reset_index(drop=True)
    print('Data date range: ', data['update_date'].min(), 'to', data['update_date'].max())
    print('Number of rows in raw data: ', data.shape[0])
    return data
# Stock price of a certain cooperation
def load_STOCK_raw(name):
    '''
    This provides a way to lookinto the 'raw' data
    '''
    SP = yf.Ticker(name)
    raw = SP.history(period="max")
    raw = raw.reset_index()
    rename_dict = {'Date': 'update_date',
                   'Open': 'open',
                   'High': 'high',
                   'Low': 'low',
                   'Close': 'close',
                   'Volume': 'volume',
                   'Dividends': 'dividends',
                   'Stock Splits': 'stock splits'
                  }
    data = raw.rename(columns=rename_dict)
    data['update_date'] = pd.to_datetime(data['update_date'])  # original data type of update_time is 'str'
    data['update_date'] = data['update_date'].dt.date
    data = data.reset_index(drop=True)
    #print('Data date range: ', data['update_date'].min(), 'to', data['update_date'].max())
    #print('Number of rows in raw data: ', data.shape[0])
    return data
```

```python
# ODE systems
#%%latex
#Difference equations
#without immigration
#\begin{align}
#S(t + 1) - S(t) &= -\beta(t)S(t)\frac{I(t)}{N(t)}, \\
#E(t + 1) - E(t) &= \beta(t)S(t)\frac{I(t)}{N(t)} - \sigma(t)E(t), \\
#I(t + 1) - I(t) &= \sigma(t)E(t) - \gamma(t)I(t), \\
#R(t + 1) - R(t) &= \gamma(t)I(t)
#\end{align}
#Difference equations
#with immigration
#\begin{align}
#S_i(t + 1) - S_i(t) &= -\beta_i(t)S_i(t)\frac{I_i(t)}{N_i(t)} - \sum_{j, j \neq i}a_{ij}(t)S_i(t) + \sum_{j, j \neq i}a_{ji}(t)S_j(t) \\
#E_i(t + 1) - E_i(t) &= \beta_i(t)S_i(t)\frac{I_i(t)}{N_i(t)} - \sigma_i(t)E_i(t) - \sum_{j, j \neq i}a_{ij}(t)E_i(t) + \sum_{j, j \neq i}a_{ji}(t)E_j(t) \\
#I_i(t + 1) - I_i(t) &= \sigma_i(t)E_i(t) - \gamma_i(t)I_i(t) \\
#R_i(t + 1) - R_i(t) &= \gamma_i(t)I_i(t)
#\end{align}
```