# Demonstration of SMPS Calculations

The SMPS calculations require two main packages from ``atmPy`` - ``smps`` and ``dma``.  ``dma`` contains the ``DMA`` class and its children.  The children of ``DMA`` simply contain the definition of the dimensions of the DMA used in the current SMPS instance.  In this case, we use the definition of the NOAA wide DMA, which has the dimensions $r_i = 0.0312$, $r_o = 0.03613$ and $l = 0.34054$ where all units are in meters.  The ``SMPS`` object provides a set of utilities for taking the scan data, applying a transfer function and correcting the distribution for multiple charges to produce a size distribution.

The import of the ``sizedistribution`` package allows us to manipulate the output of the DMA ($dN/d\log{D_p}$) such that we can pull out other representations of the size distribution.  The remaining packages are simply used for data manipulation.

```python
from atmPy.instruments.DMA import smps
from atmPy.instruments.DMA import dma
from matplotlib import colors
import matplotlib.pyplot as plt
from numpy import meshgrid
import numpy as np
import pandas as pd
from matplotlib.dates import date2num
from matplotlib import dates
from atmPy import sizedistribution as sd
%matplotlib inline
```

The first thing we do in the analysis is we create a new SMPS object with the ``DMA`` instance we wish to use.  Here, we also set the initial directory to search for SMPS data.  When a new ``SMPS`` object is created, an open file dialog window will be produced and the user may select one or many files to analyze.  The file names will be stored in the ``SMPS`` attribute ``files``.

```python
hagis = smps.SMPS(dma.NoaaWide(),scan_folder="C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans")
```

## Determining the Lag

In order to properly analyze a scan, we must first align the data such that the particle concentrations are consistent with the conditions in the DMA.  Although conditions such as voltages and flows adjust to changes almost immediately, there will be a lag in the particle response due to isntrument residence time.  We can determine the lag either through a correlation or heuristically.  The SMPS provides a function ``getLag`` which takes an integer indicating the file index to select in the attribute ``files`` containing the array of files selected by the user.  The optional input, ``delta``, allows the user to offset the result of the correlation by some amount to provide a more reasonable estimate of the lag.

The ``SMPS::getLag`` method will produce two plots.  The first is the results from the attempted correlation and the second shows how the two scans align with the lag estimate, both the smoothed and raw data. This method will set the ``lag`` attribute in the ``SMPS`` instance which will be used in future calculations.  This attribute is directly accessible if the user wishes to adjust it.

```python
hagis.getLag(10, delta=10)
```

## Processing the Files

Once the user has pointed to the files they wish to use, they can begin processing of the files using the function ``SMPS::procFiles()``.  Each file is processed as follows:

1. The raw data concerning the conditions is truncated for both the up and down scans to the beginning and end of the respective scans.  The important parameters here are the values $t_{scan}$ and $t_{dwell}$ from the header of the files.  The range of the data from the upscan spans the indices 0 to $t_{scan}$ and the range for the down data is $t_{scan}+t_{dwell}$ to $2\times t_{scan}+t_{dwell}$.
2. The CN data is adjusted based on the lag.  This data is truncated for the up scan as $t_{lag}$ to $t_{lag} + t_{scan}$ where $t_{lag}$ is the lag time determined by the user (possibly with the function ``SMPS::getLag()``.  In the downward scan, the data array is reversed and the data is truncated to the range $t_{dwell}-t_{lag}$ to $t_{scan}+t_{dwell}-t_{lag}$.  In all cases, the CN concentration is calculated from the 1 second buffer and the CPC flow rate as $N_{1 s}/Q_{cpc}$.
3. The truncated [CN] is then smoothed using a Lowess smoothing function for both the up and down data.
4. Diameters for each of the corresponding [CN] are then calculated from the set point voltage (rather than the measured voltage).
5. The resulting diameters and smoothed [CN] are then run through a transfer function.  In this case, the transfer function is a simple full width half max based off of the mobility range of the current voltage.  This function allows us to produce a $d\log D_p$ for the ditribution.
6. The distribution is then corrected based on the algorithm described below.
7. The charge corrected distribution is then converted to a logarithmic distribution using the values from the FWHM function.
8. The resulting distribution is then interpolated onto a logarithmically distributed array that consists of bin ranging from 1 to 1000 nm.

```python
hagis.lag = 10
hagis.proc_files()
```

    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_08_20_41.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_08_31_11.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_08_41_41.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_08_52_11.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_09_02_41.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_09_13_12.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_09_23_42.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_09_34_12.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_09_44_42.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_09_55_12.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_10_05_43.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_10_16_13.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_10_26_43.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_10_37_13.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_10_47_43.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_10_58_14.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_11_08_44.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_11_19_14.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_11_29_44.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_11_40_14.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_11_50_45.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_12_01_15.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_12_11_45.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_12_22_15.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_12_32_45.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_12_43_16.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_12_53_46.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_13_04_16.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_13_14_46.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_13_25_16.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_13_35_47.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_13_46_17.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_13_56_47.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_14_07_17.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_14_17_47.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_14_28_18.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_14_38_48.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_14_49_18.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_14_59_48.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_15_10_18.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_15_20_49.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_15_31_19.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_15_41_49.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_15_52_19.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_16_02_49.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_16_13_20.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_16_23_50.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_16_34_20.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_16_44_50.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_16_55_20.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_17_05_51.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_17_16_21.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_17_26_51.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_17_37_21.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_17_47_51.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_17_58_22.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_18_08_52.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_18_19_22.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_18_29_52.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_18_40_22.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_18_50_53.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_19_01_23.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_19_11_53.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_19_22_23.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_19_32_53.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_19_43_24.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_19_53_54.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_20_04_24.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_20_14_54.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_20_25_24.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_20_35_55.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_20_46_25.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_20_56_55.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_21_07_25.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_21_17_55.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_21_28_26.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_21_38_56.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_21_49_26.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_21_59_56.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_22_10_26.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_22_20_57.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_22_31_27.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_22_41_57.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_22_52_27.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_23_02_57.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_23_13_28.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_23_23_58.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_23_34_28.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_23_44_58.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150315_23_55_28.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_00_05_59.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_00_16_29.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_00_26_59.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_00_37_29.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_00_47_59.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_00_58_30.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_01_09_00.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_01_19_30.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_01_30_00.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_01_40_30.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_01_51_01.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_02_01_31.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_02_12_01.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_02_22_31.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_02_33_01.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_02_43_32.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_02_54_02.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_03_04_32.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_03_15_02.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_03_25_32.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_03_36_03.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_03_46_33.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_03_57_03.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_04_07_33.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_04_18_03.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_04_28_34.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_04_39_04.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_04_49_34.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_05_00_04.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_05_10_34.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_05_21_05.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_05_31_35.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_05_42_05.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_05_52_35.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_06_03_05.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_06_13_36.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_06_24_06.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_06_34_36.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_06_45_06.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_06_55_36.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_07_06_07.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_07_16_37.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_07_27_07.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_07_37_37.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_07_48_07.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_07_58_38.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_08_09_08.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_08_19_38.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_08_30_08.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_08_40_38.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_08_51_09.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_09_01_39.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_09_12_09.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_09_22_39.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_09_33_09.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_09_43_40.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_09_54_10.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_04_40.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_15_10.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_25_40.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_36_11.txt
    Issue processing file C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_36_11.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_42_24.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_46_44.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_51_04.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_55_24.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_10_59_44.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_04_04.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_08_25.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_12_45.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_17_05.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_21_25.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_25_45.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_30_06.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_34_26.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_38_46.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_43_06.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_47_26.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_51_47.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_11_56_07.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_00_27.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_04_47.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_09_07.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_13_28.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_17_48.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_22_08.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_26_28.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_30_48.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_35_09.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_39_29.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_43_49.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_48_09.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_52_29.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_12_56_50.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_01_10.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_05_30.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_09_50.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_14_10.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_18_31.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_22_51.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_27_11.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_31_31.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_35_51.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_40_12.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_44_32.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_48_52.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_53_12.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_13_57_32.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_01_53.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_06_13.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_10_33.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_14_53.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_19_13.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_23_34.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_27_54.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_32_14.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_36_34.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_40_54.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_45_15.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_49_35.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_53_55.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_14_58_15.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_02_35.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_06_56.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_11_16.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_15_36.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_19_56.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_24_16.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_28_37.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_32_57.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_37_17.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_41_37.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_45_57.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_50_18.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_54_38.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_15_58_58.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_03_18.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_07_38.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_11_59.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_16_19.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_20_39.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_24_59.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_29_19.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_33_40.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_38_00.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_42_20.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_46_40.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_51_00.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_55_21.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_16_59_41.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_04_01.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_08_21.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_12_41.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_17_02.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_21_22.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_25_42.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_30_02.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_34_22.txt
    C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans/SCAN_20150316_17_38_43.txt

## Charge Correction

The SMPS scans mobilities which are a function of voltage.  In truth, at each voltage setpoint, the DMA allows a range of particle mobilities through the instrument.  This range is expressed by the DMA transfer function as

\begin{equation}
\Omega = 1/q_a\max{\left[q_a,q_s,\left[\frac{1}{2}\left(q_a+q_s\right)-\right]\right]}
\end{equation}

\begin{equation}
Z=\frac{n*e*Cc(D_p)}{3*\pi*\mu(T)*D_p}
\end{equation}

In any charge correction, we must assume that there are no particles beyond the topmost bin.  This allows us to make the assumption that all particles in that bin are singly charged.  In order to determine the total number of particles in the current bin, we can simply use Wiedensohler's equation for the charging efficiency for singly charged particles.  Starting at the topmost bin, we can calculate the total number of particles as

\begin{equation}
\frac{N_1(D_p,i)}{f_1}=N(D_p)
\end{equation}

where $N_1$ is the number of particles of size $D_p$ having 1 charge, $f_1$ is the charging efficiency for singly charged particles and $N(D_p)$ is the total number of particles diameter $D_p$.  Using the initial number of particles, we can then calculated the number of multiply charged particles in a similar fashion.

Once these numbers have been calculated, we can determine the location of the multiply charged particles (i.e. the diameter bin with which they have been identified).  To do

Finally, to get the total number of particles in the bin, we can apply the sum

\begin{equation}
N(D_p) = \frac{N_1(D_p)}{f_1}\sum_{i=0}^\inf{f_i}
\end{equation}

However, in each of these cases, only a finite number of particles may be available in each bin, so in the code, we will have to take the minimum of the following:

\begin{equation}
\delta{N(k)}=\min{\left(\frac{f_iN_1}{f_1},N_k\right)}
\end{equation}

where $\delta{N(k)}$ is the number of particles to remove from bin $k$ and $N_k$ is the number of particles in bin $k$.

##  Output

In the following, we take the results from the ``SMPS::procFiles()`` method and produce a color map of size distributions in $dN/d\log D_p$ space as a function of time.  The attribute ``date`` from the instance of ``SMPS`` is a set of ``DateTime`` for each scan based on the start time of the file and the scan time collected from the header.

```python
hagis.date

```

    [datetime.datetime(2015, 3, 15, 8, 20, 41),
     datetime.datetime(2015, 3, 15, 8, 25, 56),
     datetime.datetime(2015, 3, 15, 8, 31, 11),
     datetime.datetime(2015, 3, 15, 8, 36, 26),
     datetime.datetime(2015, 3, 15, 8, 41, 41),
     datetime.datetime(2015, 3, 15, 8, 46, 56),
     datetime.datetime(2015, 3, 15, 8, 52, 11),
     datetime.datetime(2015, 3, 15, 8, 57, 26),
     datetime.datetime(2015, 3, 15, 9, 2, 41),
     datetime.datetime(2015, 3, 15, 9, 7, 56),
     datetime.datetime(2015, 3, 15, 9, 13, 12),
     datetime.datetime(2015, 3, 15, 9, 18, 27),
     datetime.datetime(2015, 3, 15, 9, 23, 42),
     datetime.datetime(2015, 3, 15, 9, 28, 57),
     datetime.datetime(2015, 3, 15, 9, 34, 12),
     datetime.datetime(2015, 3, 15, 9, 39, 27),
     datetime.datetime(2015, 3, 15, 9, 44, 42),
     datetime.datetime(2015, 3, 15, 9, 49, 57),
     datetime.datetime(2015, 3, 15, 9, 55, 12),
     datetime.datetime(2015, 3, 15, 10, 0, 27),
     datetime.datetime(2015, 3, 15, 10, 5, 43),
     datetime.datetime(2015, 3, 15, 10, 10, 58),
     datetime.datetime(2015, 3, 15, 10, 16, 13),
     datetime.datetime(2015, 3, 15, 10, 21, 28),
     datetime.datetime(2015, 3, 15, 10, 26, 43),
     datetime.datetime(2015, 3, 15, 10, 31, 58),
     datetime.datetime(2015, 3, 15, 10, 37, 13),
     datetime.datetime(2015, 3, 15, 10, 42, 28),
     datetime.datetime(2015, 3, 15, 10, 47, 43),
     datetime.datetime(2015, 3, 15, 10, 52, 58),
     datetime.datetime(2015, 3, 15, 10, 58, 14),
     datetime.datetime(2015, 3, 15, 11, 3, 29),
     datetime.datetime(2015, 3, 15, 11, 8, 44),
     datetime.datetime(2015, 3, 15, 11, 13, 59),
     datetime.datetime(2015, 3, 15, 11, 19, 14),
     datetime.datetime(2015, 3, 15, 11, 24, 29),
     datetime.datetime(2015, 3, 15, 11, 29, 44),
     datetime.datetime(2015, 3, 15, 11, 34, 59),
     datetime.datetime(2015, 3, 15, 11, 40, 14),
     datetime.datetime(2015, 3, 15, 11, 45, 29),
     datetime.datetime(2015, 3, 15, 11, 50, 45),
     datetime.datetime(2015, 3, 15, 11, 56),
     datetime.datetime(2015, 3, 15, 12, 1, 15),
     datetime.datetime(2015, 3, 15, 12, 6, 30),
     datetime.datetime(2015, 3, 15, 12, 11, 45),
     datetime.datetime(2015, 3, 15, 12, 17),
     datetime.datetime(2015, 3, 15, 12, 22, 15),
     datetime.datetime(2015, 3, 15, 12, 27, 30),
     datetime.datetime(2015, 3, 15, 12, 32, 45),
     datetime.datetime(2015, 3, 15, 12, 38),
     datetime.datetime(2015, 3, 15, 12, 43, 16),
     datetime.datetime(2015, 3, 15, 12, 48, 31),
     datetime.datetime(2015, 3, 15, 12, 53, 46),
     datetime.datetime(2015, 3, 15, 12, 59, 1),
     datetime.datetime(2015, 3, 15, 13, 4, 16),
     datetime.datetime(2015, 3, 15, 13, 9, 31),
     datetime.datetime(2015, 3, 15, 13, 14, 46),
     datetime.datetime(2015, 3, 15, 13, 20, 1),
     datetime.datetime(2015, 3, 15, 13, 25, 16),
     datetime.datetime(2015, 3, 15, 13, 30, 31),
     datetime.datetime(2015, 3, 15, 13, 35, 47),
     datetime.datetime(2015, 3, 15, 13, 41, 2),
     datetime.datetime(2015, 3, 15, 13, 46, 17),
     datetime.datetime(2015, 3, 15, 13, 51, 32),
     datetime.datetime(2015, 3, 15, 13, 56, 47),
     datetime.datetime(2015, 3, 15, 14, 2, 2),
     datetime.datetime(2015, 3, 15, 14, 7, 17),
     datetime.datetime(2015, 3, 15, 14, 12, 32),
     datetime.datetime(2015, 3, 15, 14, 17, 47),
     datetime.datetime(2015, 3, 15, 14, 23, 2),
     datetime.datetime(2015, 3, 15, 14, 28, 18),
     datetime.datetime(2015, 3, 15, 14, 33, 33),
     datetime.datetime(2015, 3, 15, 14, 38, 48),
     datetime.datetime(2015, 3, 15, 14, 44, 3),
     datetime.datetime(2015, 3, 15, 14, 49, 18),
     datetime.datetime(2015, 3, 15, 14, 54, 33),
     datetime.datetime(2015, 3, 15, 14, 59, 48),
     datetime.datetime(2015, 3, 15, 15, 5, 3),
     datetime.datetime(2015, 3, 15, 15, 10, 18),
     datetime.datetime(2015, 3, 15, 15, 15, 33),
     datetime.datetime(2015, 3, 15, 15, 20, 49),
     datetime.datetime(2015, 3, 15, 15, 26, 4),
     datetime.datetime(2015, 3, 15, 15, 31, 19),
     datetime.datetime(2015, 3, 15, 15, 36, 34),
     datetime.datetime(2015, 3, 15, 15, 41, 49),
     datetime.datetime(2015, 3, 15, 15, 47, 4),
     datetime.datetime(2015, 3, 15, 15, 52, 19),
     datetime.datetime(2015, 3, 15, 15, 57, 34),
     datetime.datetime(2015, 3, 15, 16, 2, 49),
     datetime.datetime(2015, 3, 15, 16, 8, 4),
     datetime.datetime(2015, 3, 15, 16, 13, 20),
     datetime.datetime(2015, 3, 15, 16, 18, 35),
     datetime.datetime(2015, 3, 15, 16, 23, 50),
     datetime.datetime(2015, 3, 15, 16, 29, 5),
     datetime.datetime(2015, 3, 15, 16, 34, 20),
     datetime.datetime(2015, 3, 15, 16, 39, 35),
     datetime.datetime(2015, 3, 15, 16, 44, 50),
     datetime.datetime(2015, 3, 15, 16, 50, 5),
     datetime.datetime(2015, 3, 15, 16, 55, 20),
     datetime.datetime(2015, 3, 15, 17, 0, 35),
     datetime.datetime(2015, 3, 15, 17, 5, 51),
     datetime.datetime(2015, 3, 15, 17, 11, 6),
     datetime.datetime(2015, 3, 15, 17, 16, 21),
     datetime.datetime(2015, 3, 15, 17, 21, 36),
     datetime.datetime(2015, 3, 15, 17, 26, 51),
     datetime.datetime(2015, 3, 15, 17, 32, 6),
     datetime.datetime(2015, 3, 15, 17, 37, 21),
     datetime.datetime(2015, 3, 15, 17, 42, 36),
     datetime.datetime(2015, 3, 15, 17, 47, 51),
     datetime.datetime(2015, 3, 15, 17, 53, 6),
     datetime.datetime(2015, 3, 15, 17, 58, 22),
     datetime.datetime(2015, 3, 15, 18, 3, 37),
     datetime.datetime(2015, 3, 15, 18, 8, 52),
     datetime.datetime(2015, 3, 15, 18, 14, 7),
     datetime.datetime(2015, 3, 15, 18, 19, 22),
     datetime.datetime(2015, 3, 15, 18, 24, 37),
     datetime.datetime(2015, 3, 15, 18, 29, 52),
     datetime.datetime(2015, 3, 15, 18, 35, 7),
     datetime.datetime(2015, 3, 15, 18, 40, 22),
     datetime.datetime(2015, 3, 15, 18, 45, 37),
     datetime.datetime(2015, 3, 15, 18, 50, 53),
     datetime.datetime(2015, 3, 15, 18, 56, 8),
     datetime.datetime(2015, 3, 15, 19, 1, 23),
     datetime.datetime(2015, 3, 15, 19, 6, 38),
     datetime.datetime(2015, 3, 15, 19, 11, 53),
     datetime.datetime(2015, 3, 15, 19, 17, 8),
     datetime.datetime(2015, 3, 15, 19, 22, 23),
     datetime.datetime(2015, 3, 15, 19, 27, 38),
     datetime.datetime(2015, 3, 15, 19, 32, 53),
     datetime.datetime(2015, 3, 15, 19, 38, 8),
     datetime.datetime(2015, 3, 15, 19, 43, 24),
     datetime.datetime(2015, 3, 15, 19, 48, 39),
     datetime.datetime(2015, 3, 15, 19, 53, 54),
     datetime.datetime(2015, 3, 15, 19, 59, 9),
     datetime.datetime(2015, 3, 15, 20, 4, 24),
     datetime.datetime(2015, 3, 15, 20, 9, 39),
     datetime.datetime(2015, 3, 15, 20, 14, 54),
     datetime.datetime(2015, 3, 15, 20, 20, 9),
     datetime.datetime(2015, 3, 15, 20, 25, 24),
     datetime.datetime(2015, 3, 15, 20, 30, 39),
     datetime.datetime(2015, 3, 15, 20, 35, 55),
     datetime.datetime(2015, 3, 15, 20, 41, 10),
     datetime.datetime(2015, 3, 15, 20, 46, 25),
     datetime.datetime(2015, 3, 15, 20, 51, 40),
     datetime.datetime(2015, 3, 15, 20, 56, 55),
     datetime.datetime(2015, 3, 15, 21, 2, 10),
     datetime.datetime(2015, 3, 15, 21, 7, 25),
     datetime.datetime(2015, 3, 15, 21, 12, 40),
     datetime.datetime(2015, 3, 15, 21, 17, 55),
     datetime.datetime(2015, 3, 15, 21, 23, 10),
     datetime.datetime(2015, 3, 15, 21, 28, 26),
     datetime.datetime(2015, 3, 15, 21, 33, 41),
     datetime.datetime(2015, 3, 15, 21, 38, 56),
     datetime.datetime(2015, 3, 15, 21, 44, 11),
     datetime.datetime(2015, 3, 15, 21, 49, 26),
     datetime.datetime(2015, 3, 15, 21, 54, 41),
     datetime.datetime(2015, 3, 15, 21, 59, 56),
     datetime.datetime(2015, 3, 15, 22, 5, 11),
     datetime.datetime(2015, 3, 15, 22, 10, 26),
     datetime.datetime(2015, 3, 15, 22, 15, 41),
     datetime.datetime(2015, 3, 15, 22, 20, 57),
     datetime.datetime(2015, 3, 15, 22, 26, 12),
     datetime.datetime(2015, 3, 15, 22, 31, 27),
     datetime.datetime(2015, 3, 15, 22, 36, 42),
     datetime.datetime(2015, 3, 15, 22, 41, 57),
     datetime.datetime(2015, 3, 15, 22, 47, 12),
     datetime.datetime(2015, 3, 15, 22, 52, 27),
     datetime.datetime(2015, 3, 15, 22, 57, 42),
     datetime.datetime(2015, 3, 15, 23, 2, 57),
     datetime.datetime(2015, 3, 15, 23, 8, 12),
     datetime.datetime(2015, 3, 15, 23, 13, 28),
     datetime.datetime(2015, 3, 15, 23, 18, 43),
     datetime.datetime(2015, 3, 15, 23, 23, 58),
     datetime.datetime(2015, 3, 15, 23, 29, 13),
     datetime.datetime(2015, 3, 15, 23, 34, 28),
     datetime.datetime(2015, 3, 15, 23, 39, 43),
     datetime.datetime(2015, 3, 15, 23, 44, 58),
     datetime.datetime(2015, 3, 15, 23, 50, 13),
     datetime.datetime(2015, 3, 15, 23, 55, 28),
     datetime.datetime(2015, 3, 16, 0, 0, 43),
     datetime.datetime(2015, 3, 16, 0, 5, 59),
     datetime.datetime(2015, 3, 16, 0, 11, 14),
     datetime.datetime(2015, 3, 16, 0, 16, 29),
     datetime.datetime(2015, 3, 16, 0, 21, 44),
     datetime.datetime(2015, 3, 16, 0, 26, 59),
     datetime.datetime(2015, 3, 16, 0, 32, 14),
     datetime.datetime(2015, 3, 16, 0, 37, 29),
     datetime.datetime(2015, 3, 16, 0, 42, 44),
     datetime.datetime(2015, 3, 16, 0, 47, 59),
     datetime.datetime(2015, 3, 16, 0, 53, 14),
     datetime.datetime(2015, 3, 16, 0, 58, 30),
     datetime.datetime(2015, 3, 16, 1, 3, 45),
     datetime.datetime(2015, 3, 16, 1, 9),
     datetime.datetime(2015, 3, 16, 1, 14, 15),
     datetime.datetime(2015, 3, 16, 1, 19, 30),
     datetime.datetime(2015, 3, 16, 1, 24, 45),
     datetime.datetime(2015, 3, 16, 1, 30),
     datetime.datetime(2015, 3, 16, 1, 35, 15),
     datetime.datetime(2015, 3, 16, 1, 40, 30),
     datetime.datetime(2015, 3, 16, 1, 45, 45),
     datetime.datetime(2015, 3, 16, 1, 51, 1),
     datetime.datetime(2015, 3, 16, 1, 56, 16),
     datetime.datetime(2015, 3, 16, 2, 1, 31),
     datetime.datetime(2015, 3, 16, 2, 6, 46),
     datetime.datetime(2015, 3, 16, 2, 12, 1),
     datetime.datetime(2015, 3, 16, 2, 17, 16),
     datetime.datetime(2015, 3, 16, 2, 22, 31),
     datetime.datetime(2015, 3, 16, 2, 27, 46),
     datetime.datetime(2015, 3, 16, 2, 33, 1),
     datetime.datetime(2015, 3, 16, 2, 38, 16),
     datetime.datetime(2015, 3, 16, 2, 43, 32),
     datetime.datetime(2015, 3, 16, 2, 48, 47),
     datetime.datetime(2015, 3, 16, 2, 54, 2),
     datetime.datetime(2015, 3, 16, 2, 59, 17),
     datetime.datetime(2015, 3, 16, 3, 4, 32),
     datetime.datetime(2015, 3, 16, 3, 9, 47),
     datetime.datetime(2015, 3, 16, 3, 15, 2),
     datetime.datetime(2015, 3, 16, 3, 20, 17),
     datetime.datetime(2015, 3, 16, 3, 25, 32),
     datetime.datetime(2015, 3, 16, 3, 30, 47),
     datetime.datetime(2015, 3, 16, 3, 36, 3),
     datetime.datetime(2015, 3, 16, 3, 41, 18),
     datetime.datetime(2015, 3, 16, 3, 46, 33),
     datetime.datetime(2015, 3, 16, 3, 51, 48),
     datetime.datetime(2015, 3, 16, 3, 57, 3),
     datetime.datetime(2015, 3, 16, 4, 2, 18),
     datetime.datetime(2015, 3, 16, 4, 7, 33),
     datetime.datetime(2015, 3, 16, 4, 12, 48),
     datetime.datetime(2015, 3, 16, 4, 18, 3),
     datetime.datetime(2015, 3, 16, 4, 23, 18),
     datetime.datetime(2015, 3, 16, 4, 28, 34),
     datetime.datetime(2015, 3, 16, 4, 33, 49),
     datetime.datetime(2015, 3, 16, 4, 39, 4),
     datetime.datetime(2015, 3, 16, 4, 44, 19),
     datetime.datetime(2015, 3, 16, 4, 49, 34),
     datetime.datetime(2015, 3, 16, 4, 54, 49),
     datetime.datetime(2015, 3, 16, 5, 0, 4),
     datetime.datetime(2015, 3, 16, 5, 5, 19),
     datetime.datetime(2015, 3, 16, 5, 10, 34),
     datetime.datetime(2015, 3, 16, 5, 15, 49),
     datetime.datetime(2015, 3, 16, 5, 21, 5),
     datetime.datetime(2015, 3, 16, 5, 26, 20),
     datetime.datetime(2015, 3, 16, 5, 31, 35),
     datetime.datetime(2015, 3, 16, 5, 36, 50),
     datetime.datetime(2015, 3, 16, 5, 42, 5),
     datetime.datetime(2015, 3, 16, 5, 47, 20),
     datetime.datetime(2015, 3, 16, 5, 52, 35),
     datetime.datetime(2015, 3, 16, 5, 57, 50),
     datetime.datetime(2015, 3, 16, 6, 3, 5),
     datetime.datetime(2015, 3, 16, 6, 8, 20),
     datetime.datetime(2015, 3, 16, 6, 13, 36),
     datetime.datetime(2015, 3, 16, 6, 18, 51),
     datetime.datetime(2015, 3, 16, 6, 24, 6),
     datetime.datetime(2015, 3, 16, 6, 29, 21),
     datetime.datetime(2015, 3, 16, 6, 34, 36),
     datetime.datetime(2015, 3, 16, 6, 39, 51),
     datetime.datetime(2015, 3, 16, 6, 45, 6),
     datetime.datetime(2015, 3, 16, 6, 50, 21),
     datetime.datetime(2015, 3, 16, 6, 55, 36),
     datetime.datetime(2015, 3, 16, 7, 0, 51),
     datetime.datetime(2015, 3, 16, 7, 6, 7),
     datetime.datetime(2015, 3, 16, 7, 11, 22),
     datetime.datetime(2015, 3, 16, 7, 16, 37),
     datetime.datetime(2015, 3, 16, 7, 21, 52),
     datetime.datetime(2015, 3, 16, 7, 27, 7),
     datetime.datetime(2015, 3, 16, 7, 32, 22),
     datetime.datetime(2015, 3, 16, 7, 37, 37),
     datetime.datetime(2015, 3, 16, 7, 42, 52),
     datetime.datetime(2015, 3, 16, 7, 48, 7),
     datetime.datetime(2015, 3, 16, 7, 53, 22),
     datetime.datetime(2015, 3, 16, 7, 58, 38),
     datetime.datetime(2015, 3, 16, 8, 3, 53),
     datetime.datetime(2015, 3, 16, 8, 9, 8),
     datetime.datetime(2015, 3, 16, 8, 14, 23),
     datetime.datetime(2015, 3, 16, 8, 19, 38),
     datetime.datetime(2015, 3, 16, 8, 24, 53),
     datetime.datetime(2015, 3, 16, 8, 30, 8),
     datetime.datetime(2015, 3, 16, 8, 35, 23),
     datetime.datetime(2015, 3, 16, 8, 40, 38),
     datetime.datetime(2015, 3, 16, 8, 45, 53),
     datetime.datetime(2015, 3, 16, 8, 51, 9),
     datetime.datetime(2015, 3, 16, 8, 56, 24),
     datetime.datetime(2015, 3, 16, 9, 1, 39),
     datetime.datetime(2015, 3, 16, 9, 6, 54),
     datetime.datetime(2015, 3, 16, 9, 12, 9),
     datetime.datetime(2015, 3, 16, 9, 17, 24),
     datetime.datetime(2015, 3, 16, 9, 22, 39),
     datetime.datetime(2015, 3, 16, 9, 27, 54),
     datetime.datetime(2015, 3, 16, 9, 33, 9),
     datetime.datetime(2015, 3, 16, 9, 38, 24),
     datetime.datetime(2015, 3, 16, 9, 43, 40),
     datetime.datetime(2015, 3, 16, 9, 48, 55),
     datetime.datetime(2015, 3, 16, 9, 54, 10),
     datetime.datetime(2015, 3, 16, 9, 59, 25),
     datetime.datetime(2015, 3, 16, 10, 4, 40),
     datetime.datetime(2015, 3, 16, 10, 9, 55),
     datetime.datetime(2015, 3, 16, 10, 15, 10),
     datetime.datetime(2015, 3, 16, 10, 20, 25),
     datetime.datetime(2015, 3, 16, 10, 25, 40),
     datetime.datetime(2015, 3, 16, 10, 30, 55),
     datetime.datetime(2015, 3, 16, 10, 36, 11),
     datetime.datetime(2015, 3, 16, 10, 41, 26),
     datetime.datetime(2015, 3, 16, 10, 42, 24),
     datetime.datetime(2015, 3, 16, 10, 44, 34),
     datetime.datetime(2015, 3, 16, 10, 46, 44),
     datetime.datetime(2015, 3, 16, 10, 48, 54),
     datetime.datetime(2015, 3, 16, 10, 51, 4),
     datetime.datetime(2015, 3, 16, 10, 53, 14),
     datetime.datetime(2015, 3, 16, 10, 55, 24),
     datetime.datetime(2015, 3, 16, 10, 57, 34),
     datetime.datetime(2015, 3, 16, 10, 59, 44),
     datetime.datetime(2015, 3, 16, 11, 1, 54),
     datetime.datetime(2015, 3, 16, 11, 4, 4),
     datetime.datetime(2015, 3, 16, 11, 6, 14),
     datetime.datetime(2015, 3, 16, 11, 8, 25),
     datetime.datetime(2015, 3, 16, 11, 10, 35),
     datetime.datetime(2015, 3, 16, 11, 12, 45),
     datetime.datetime(2015, 3, 16, 11, 14, 55),
     datetime.datetime(2015, 3, 16, 11, 17, 5),
     datetime.datetime(2015, 3, 16, 11, 19, 15),
     datetime.datetime(2015, 3, 16, 11, 21, 25),
     datetime.datetime(2015, 3, 16, 11, 23, 35),
     datetime.datetime(2015, 3, 16, 11, 25, 45),
     datetime.datetime(2015, 3, 16, 11, 27, 55),
     datetime.datetime(2015, 3, 16, 11, 30, 6),
     datetime.datetime(2015, 3, 16, 11, 32, 16),
     datetime.datetime(2015, 3, 16, 11, 34, 26),
     datetime.datetime(2015, 3, 16, 11, 36, 36),
     datetime.datetime(2015, 3, 16, 11, 38, 46),
     datetime.datetime(2015, 3, 16, 11, 40, 56),
     datetime.datetime(2015, 3, 16, 11, 43, 6),
     datetime.datetime(2015, 3, 16, 11, 45, 16),
     datetime.datetime(2015, 3, 16, 11, 47, 26),
     datetime.datetime(2015, 3, 16, 11, 49, 36),
     datetime.datetime(2015, 3, 16, 11, 51, 47),
     datetime.datetime(2015, 3, 16, 11, 53, 57),
     datetime.datetime(2015, 3, 16, 11, 56, 7),
     datetime.datetime(2015, 3, 16, 11, 58, 17),
     datetime.datetime(2015, 3, 16, 12, 0, 27),
     datetime.datetime(2015, 3, 16, 12, 2, 37),
     datetime.datetime(2015, 3, 16, 12, 4, 47),
     datetime.datetime(2015, 3, 16, 12, 6, 57),
     datetime.datetime(2015, 3, 16, 12, 9, 7),
     datetime.datetime(2015, 3, 16, 12, 11, 17),
     datetime.datetime(2015, 3, 16, 12, 13, 28),
     datetime.datetime(2015, 3, 16, 12, 15, 38),
     datetime.datetime(2015, 3, 16, 12, 17, 48),
     datetime.datetime(2015, 3, 16, 12, 19, 58),
     datetime.datetime(2015, 3, 16, 12, 22, 8),
     datetime.datetime(2015, 3, 16, 12, 24, 18),
     datetime.datetime(2015, 3, 16, 12, 26, 28),
     datetime.datetime(2015, 3, 16, 12, 28, 38),
     datetime.datetime(2015, 3, 16, 12, 30, 48),
     datetime.datetime(2015, 3, 16, 12, 32, 58),
     datetime.datetime(2015, 3, 16, 12, 35, 9),
     datetime.datetime(2015, 3, 16, 12, 37, 19),
     datetime.datetime(2015, 3, 16, 12, 39, 29),
     datetime.datetime(2015, 3, 16, 12, 41, 39),
     datetime.datetime(2015, 3, 16, 12, 43, 49),
     datetime.datetime(2015, 3, 16, 12, 45, 59),
     datetime.datetime(2015, 3, 16, 12, 48, 9),
     datetime.datetime(2015, 3, 16, 12, 50, 19),
     datetime.datetime(2015, 3, 16, 12, 52, 29),
     datetime.datetime(2015, 3, 16, 12, 54, 39),
     datetime.datetime(2015, 3, 16, 12, 56, 50),
     datetime.datetime(2015, 3, 16, 12, 59),
     datetime.datetime(2015, 3, 16, 13, 1, 10),
     datetime.datetime(2015, 3, 16, 13, 3, 20),
     datetime.datetime(2015, 3, 16, 13, 5, 30),
     datetime.datetime(2015, 3, 16, 13, 7, 40),
     datetime.datetime(2015, 3, 16, 13, 9, 50),
     datetime.datetime(2015, 3, 16, 13, 12),
     datetime.datetime(2015, 3, 16, 13, 14, 10),
     datetime.datetime(2015, 3, 16, 13, 16, 20),
     datetime.datetime(2015, 3, 16, 13, 18, 31),
     datetime.datetime(2015, 3, 16, 13, 20, 41),
     datetime.datetime(2015, 3, 16, 13, 22, 51),
     datetime.datetime(2015, 3, 16, 13, 25, 1),
     datetime.datetime(2015, 3, 16, 13, 27, 11),
     datetime.datetime(2015, 3, 16, 13, 29, 21),
     datetime.datetime(2015, 3, 16, 13, 31, 31),
     datetime.datetime(2015, 3, 16, 13, 33, 41),
     datetime.datetime(2015, 3, 16, 13, 35, 51),
     datetime.datetime(2015, 3, 16, 13, 38, 1),
     datetime.datetime(2015, 3, 16, 13, 40, 12),
     datetime.datetime(2015, 3, 16, 13, 42, 22),
     datetime.datetime(2015, 3, 16, 13, 44, 32),
     datetime.datetime(2015, 3, 16, 13, 46, 42),
     datetime.datetime(2015, 3, 16, 13, 48, 52),
     datetime.datetime(2015, 3, 16, 13, 51, 2),
     datetime.datetime(2015, 3, 16, 13, 53, 12),
     datetime.datetime(2015, 3, 16, 13, 55, 22),
     datetime.datetime(2015, 3, 16, 13, 57, 32),
     datetime.datetime(2015, 3, 16, 13, 59, 42),
     datetime.datetime(2015, 3, 16, 14, 1, 53),
     datetime.datetime(2015, 3, 16, 14, 4, 3),
     datetime.datetime(2015, 3, 16, 14, 6, 13),
     datetime.datetime(2015, 3, 16, 14, 8, 23),
     datetime.datetime(2015, 3, 16, 14, 10, 33),
     datetime.datetime(2015, 3, 16, 14, 12, 43),
     datetime.datetime(2015, 3, 16, 14, 14, 53),
     datetime.datetime(2015, 3, 16, 14, 17, 3),
     datetime.datetime(2015, 3, 16, 14, 19, 13),
     datetime.datetime(2015, 3, 16, 14, 21, 23),
     datetime.datetime(2015, 3, 16, 14, 23, 34),
     datetime.datetime(2015, 3, 16, 14, 25, 44),
     datetime.datetime(2015, 3, 16, 14, 27, 54),
     datetime.datetime(2015, 3, 16, 14, 30, 4),
     datetime.datetime(2015, 3, 16, 14, 32, 14),
     datetime.datetime(2015, 3, 16, 14, 34, 24),
     datetime.datetime(2015, 3, 16, 14, 36, 34),
     datetime.datetime(2015, 3, 16, 14, 38, 44),
     datetime.datetime(2015, 3, 16, 14, 40, 54),
     datetime.datetime(2015, 3, 16, 14, 43, 4),
     datetime.datetime(2015, 3, 16, 14, 45, 15),
     datetime.datetime(2015, 3, 16, 14, 47, 25),
     datetime.datetime(2015, 3, 16, 14, 49, 35),
     datetime.datetime(2015, 3, 16, 14, 51, 45),
     datetime.datetime(2015, 3, 16, 14, 53, 55),
     datetime.datetime(2015, 3, 16, 14, 56, 5),
     datetime.datetime(2015, 3, 16, 14, 58, 15),
     datetime.datetime(2015, 3, 16, 15, 0, 25),
     datetime.datetime(2015, 3, 16, 15, 2, 35),
     datetime.datetime(2015, 3, 16, 15, 4, 45),
     datetime.datetime(2015, 3, 16, 15, 6, 56),
     datetime.datetime(2015, 3, 16, 15, 9, 6),
     datetime.datetime(2015, 3, 16, 15, 11, 16),
     datetime.datetime(2015, 3, 16, 15, 13, 26),
     datetime.datetime(2015, 3, 16, 15, 15, 36),
     datetime.datetime(2015, 3, 16, 15, 17, 46),
     datetime.datetime(2015, 3, 16, 15, 19, 56),
     datetime.datetime(2015, 3, 16, 15, 22, 6),
     datetime.datetime(2015, 3, 16, 15, 24, 16),
     datetime.datetime(2015, 3, 16, 15, 26, 26),
     datetime.datetime(2015, 3, 16, 15, 28, 37),
     datetime.datetime(2015, 3, 16, 15, 30, 47),
     datetime.datetime(2015, 3, 16, 15, 32, 57),
     datetime.datetime(2015, 3, 16, 15, 35, 7),
     datetime.datetime(2015, 3, 16, 15, 37, 17),
     datetime.datetime(2015, 3, 16, 15, 39, 27),
     datetime.datetime(2015, 3, 16, 15, 41, 37),
     datetime.datetime(2015, 3, 16, 15, 43, 47),
     datetime.datetime(2015, 3, 16, 15, 45, 57),
     datetime.datetime(2015, 3, 16, 15, 48, 7),
     datetime.datetime(2015, 3, 16, 15, 50, 18),
     datetime.datetime(2015, 3, 16, 15, 52, 28),
     datetime.datetime(2015, 3, 16, 15, 54, 38),
     datetime.datetime(2015, 3, 16, 15, 56, 48),
     datetime.datetime(2015, 3, 16, 15, 58, 58),
     datetime.datetime(2015, 3, 16, 16, 1, 8),
     datetime.datetime(2015, 3, 16, 16, 3, 18),
     datetime.datetime(2015, 3, 16, 16, 5, 28),
     datetime.datetime(2015, 3, 16, 16, 7, 38),
     datetime.datetime(2015, 3, 16, 16, 9, 48),
     datetime.datetime(2015, 3, 16, 16, 11, 59),
     datetime.datetime(2015, 3, 16, 16, 14, 9),
     datetime.datetime(2015, 3, 16, 16, 16, 19),
     datetime.datetime(2015, 3, 16, 16, 18, 29),
     datetime.datetime(2015, 3, 16, 16, 20, 39),
     datetime.datetime(2015, 3, 16, 16, 22, 49),
     datetime.datetime(2015, 3, 16, 16, 24, 59),
     datetime.datetime(2015, 3, 16, 16, 27, 9),
     datetime.datetime(2015, 3, 16, 16, 29, 19),
     datetime.datetime(2015, 3, 16, 16, 31, 29),
     datetime.datetime(2015, 3, 16, 16, 33, 40),
     datetime.datetime(2015, 3, 16, 16, 35, 50),
     datetime.datetime(2015, 3, 16, 16, 38),
     datetime.datetime(2015, 3, 16, 16, 40, 10),
     datetime.datetime(2015, 3, 16, 16, 42, 20),
     datetime.datetime(2015, 3, 16, 16, 44, 30),
     datetime.datetime(2015, 3, 16, 16, 46, 40),
     datetime.datetime(2015, 3, 16, 16, 48, 50),
     datetime.datetime(2015, 3, 16, 16, 51),
     datetime.datetime(2015, 3, 16, 16, 53, 10),
     datetime.datetime(2015, 3, 16, 16, 55, 21),
     datetime.datetime(2015, 3, 16, 16, 57, 31),
     datetime.datetime(2015, 3, 16, 16, 59, 41),
     datetime.datetime(2015, 3, 16, 17, 1, 51),
     datetime.datetime(2015, 3, 16, 17, 4, 1),
     datetime.datetime(2015, 3, 16, 17, 6, 11),
     datetime.datetime(2015, 3, 16, 17, 8, 21),
     datetime.datetime(2015, 3, 16, 17, 10, 31),
     datetime.datetime(2015, 3, 16, 17, 12, 41),
     datetime.datetime(2015, 3, 16, 17, 14, 51),
     datetime.datetime(2015, 3, 16, 17, 17, 2),
     datetime.datetime(2015, 3, 16, 17, 19, 12),
     datetime.datetime(2015, 3, 16, 17, 21, 22),
     datetime.datetime(2015, 3, 16, 17, 23, 32),
     datetime.datetime(2015, 3, 16, 17, 25, 42),
     datetime.datetime(2015, 3, 16, 17, 27, 52),
     datetime.datetime(2015, 3, 16, 17, 30, 2),
     datetime.datetime(2015, 3, 16, 17, 32, 12),
     datetime.datetime(2015, 3, 16, 17, 34, 22),
     datetime.datetime(2015, 3, 16, 17, 36, 32),
     datetime.datetime(2015, 3, 16, 17, 38, 43),
     datetime.datetime(2015, 3, 16, 17, 40, 53)]

```python
index = []
for i,e in enumerate(hagis.date):
    if e is None:
        index.append(i)

print(index)

if index:
    hagis.date = np.delete(hagis.date, index)
    hagis.dn_interp = np.delete(hagis.dn_interp,index, axis=0)
```

    []

```python
xfmt = dates.DateFormatter('%m/%d %H:%M')
xi = date2num(hagis.date)
XI, YI = meshgrid(xi, hagis.diam_interp)
#XI = dates.datetime.datetime.fromtimestamp(XI)
Z = hagis.dn_interp.transpose()
Z[np.where(Z <= 0)] = np.nan
```

```python
pmax = 1e6 # 10**np.ceil(np.log10(np.amax(Z[np.where(Z > 0)])))
pmin = 1 #10**np.floor(np.log10(np.amin(Z[np.where(Z > 0)])))
fig, ax = plt.subplots()
pc = ax.pcolor(XI, YI, Z, cmap=plt.cm.jet, norm=colors.LogNorm(pmin, pmax, clip=False), alpha=0.8)

plt.colorbar(pc)
plt.yscale('log')
plt.ylim(5, 1000)
ax.xaxis.set_major_formatter(xfmt)
fig.autofmt_xdate()
fig.tight_layout()
```

## Use of the SizeDistr Object

The ``sizeditribution`` package contains some classes and routines for ready manipulation of the data.  But first, we will need to convert the data of interest to a PANDAS data frame with the time as index.

```python
dataframe = pd.DataFrame(hagis.dn_interp)
dataframe.index = hagis.date
```

In addition, we will need to convert the bin centers produced by the ``SMPS`` object to bin edges.  To do this, we will make the simple assumption that the bin edges are just the halfway points between the centers.  For the edge cases, we will simply take the difference between the smallest bin center and the halfway point between the first and second bin centers and subtract this value from the smallest diameter.  Similarly, for the largest diameter, we will take the difference between the halfway point between the largest and second largest bin centers and the largest bin center and add it to the largest bin center.

```python
binedges = (hagis.diam_interp[1:]+hagis.diam_interp[:-1])/2
first = hagis.diam_interp[0] -(binedges[0]-hagis.diam_interp[0])
last = hagis.diam_interp[-1]+ (hagis.diam_interp[-1]-binedges[-1])
binedges = np.append([first],binedges)
binedges=np.append(binedges,[last])
sizeDistr = sd.SizeDist_TS(dataframe,binedges, 'dNdlogDp')
```

```python
f,a,b,c = sizeDistr.plot(vmax = pmax, vmin = pmin, norm='log', showMinorTickLabels=False, cmap=plt.cm.jet)
a.set_ylim((5,1000))
```

Once we have the corresponding ``SizeDistr`` object, we can now change the current distribution which is in $dN/d\log D_p$ space and change this to a surface area distribution in log space.  This will produce a new object that we will call ``sfSD``.

```python
sfSD = sizeDistr.convert2dSdlogDp()
```

```python
from imp import reload
reload(sd)

```

    <module 'atmPy.sizedistribution' from 'C:\\Users\\mrichardson\\Documents\\PythonScripts\\atm_py\\atmPy\\sizedistribution.py'>

```python
f,a,b,c = sfSD.plot(vmax = 1e10, vmin = 1e4, norm='log', showMinorTickLabels=False,removeTickLabels=['200','300','400',] ,cmap =plt.cm.jet)
a.set_ylim((5,1000))
```

To get an overall view, we can further manipulate the data to produce average distributions from the entire time series.

```python
avgAt = sizeDistr.average_overAllTime()
```

    C:\Anaconda3\lib\site-packages\numpy\core\_methods.py:59: RuntimeWarning: Mean of empty slice.
      warnings.warn("Mean of empty slice.", RuntimeWarning)

```python
f,a = avgAt.plot(norm='log')
# a.set_yscale('log')
```

```python
avgAtS = sfSD.average_overAllTime()
```

```python
f,a= avgAtS.plot(norm='log')
a.set_yscale('log')
```

In the previous analysis, it appears that we have a size distribution which centers around 60 nm in number and somewhere around 100 to 200 nm in surface.

```python

```


# [BSSN](http://www2.yukawa.kyoto-u.ac.jp/~yuichiro.sekiguchi/3+1.pdf) Hamiltonian and momentum constraint equations, in ***curvilinear*** coordinates, using a covariant reference metric approach: C code generation

## Authors: Ian Ruchlin & Zach Etienne
### Formatting improvements courtesy Brandon Clark

## This module constructs the BSSN Hamiltonian and momentum constraint equations as symbolic (SymPy) expressions, in terms of the core BSSN quantities $\left\{h_{i j},a_{i j},\phi, K, \lambda^{i}, \alpha, \mathcal{V}^i, \mathcal{B}^i\right\}$, as defined in [Ruchlin, Etienne, and Baumgarte (2018)](https://arxiv.org/abs/1712.07658) (see also [Baumgarte, Montero, Cordero-Carrión, and Müller (2012)](https://arxiv.org/abs/1211.6632).

### This module implements a generic curvilinear coordinate reference metric approach matching that of [Ruchlin, Etienne, and Baumgarte (2018)](https://arxiv.org/abs/1712.07658), which is an extension of the spherical coordinate reference metric approach of [Baumgarte, Montero, Cordero-Carrión, and Müller (2012)](https://arxiv.org/abs/1211.6632), which builds upon the covariant "Lagrangian" BSSN formalism of [Brown (2009)](https://arxiv.org/abs/0902.3652). *See also citations within each article.*

**Notebook Status:**  Validated

**Validation Notes:** All expressions generated in this module have been validated against a trusted code where applicable (the original NRPy+/SENR code, which itself was validated against [Baumgarte's code](https://arxiv.org/abs/1211.6632)).

### NRPy+ Source Code for this module: [BSSN/BSSN_constraints.py](../edit/BSSN/BSSN_constraints.py)

[comment]: <> (Introduction: TODO)

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#initializenrpy): Initialize needed Python/NRPy+ modules
1. [Step 2](#hamiltonianconstraint): Construct the Hamiltonian constraint $\mathcal{H}$.
1. [Step 3](#momentumconstraint): Construct the momentum constraint $\mathcal{M}^i$.
1. [Step 4](#code_validation): Code Validation against `BSSN.BSSN_constraints` NRPy+ module
1. [Step 5](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Initialize needed Python/NRPy+ modules \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

We start by loading the needed modules. Notably, this module depends on several quantities defined in the [BSSN/BSSN_quantities.py](../edit/BSSN/BSSN_quantities.py) Python code, documented in the NRPy+ [BSSN quantities](Tutorial-BSSN_quantities.ipynb). In [Step 2](#hamiltonianconstraint) we call functions within [BSSN.BSSN_quantities](../edit/BSSN/BSSN_quantities.py) to define quantities needed in this module.

```python
# Step 1: Initialize needed Python/NRPy+ modules
import sympy as sp               # SymPy, Python's core symbolic algebra package on which NRPy+ depends
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import grid as gri               # NRPy+: Functions having to do with numerical grids
import reference_metric as rfm   # NRPy+: Reference metric support
import BSSN.BSSN_quantities as Bq

# Step 1.a: Set spatial dimension (must be 3 for BSSN, as BSSN is
#           a 3+1-dimensional decomposition of the general
#           relativistic field equations)
DIM = 3

# Step 1.b: Given the chosen coordinate system, set up
#           corresponding reference metric and needed
#           reference metric quantities
# The following function call sets up the reference metric
#    and related quantities, including rescaling matrices ReDD,
#    ReU, and hatted quantities.
rfm.reference_metric()
```

# Step 2: $\mathcal{H}$, the Hamiltonian constraint \[Back to [top](#toc)\]
$$\label{hamiltonianconstraint}$$

Next we define the Hamiltonian constraint. Eq. 13 of [Baumgarte *et al.*](https://arxiv.org/pdf/1211.6632.pdf) yields:
$$
\mathcal{H} = {\underbrace {\textstyle \frac{2}{3} K^2}_{\rm Term\ 1}} -
{\underbrace {\textstyle \bar{A}_{ij} \bar{A}^{ij}}_{\rm Term\ 2}} +
{\underbrace {\textstyle e^{-4\phi} \left(\bar{R} - 8 \bar{D}^i \phi \bar{D}_i \phi - 8 \bar{D}^2 \phi\right)}_{\rm Term\ 3}}
$$

```python
# Step 2: The Hamiltonian constraint.
# First declare all needed variables
Bq.declare_BSSN_gridfunctions_if_not_declared_already() # Sets trK
Bq.BSSN_basic_tensors()   # Sets AbarDD
Bq.gammabar__inverse_and_derivs() # Sets gammabarUU
Bq.AbarUU_AbarUD_trAbar_AbarDD_dD() # Sets AbarUU and AbarDD_dD
Bq.RicciBar__gammabarDD_dHatD__DGammaUDD__DGammaU() # Sets RbarDD
Bq.phi_and_derivs() # Sets phi_dBarD & phi_dBarDD

# Term 1: 2/3 K^2
H = sp.Rational(2,3)*Bq.trK**2

# Term 2: -A_{ij} A^{ij}
for i in range(DIM):
    for j in range(DIM):
        H += -Bq.AbarDD[i][j]*Bq.AbarUU[i][j]

# Term 3a: trace(Rbar)
Rbartrace = sp.sympify(0)
for i in range(DIM):
    for j in range(DIM):
        Rbartrace += Bq.gammabarUU[i][j]*Bq.RbarDD[i][j]

# Term 3b: -8 \bar{\gamma}^{ij} \bar{D}_i \phi \bar{D}_j \phi = -8*phi_dBar_times_phi_dBar
# Term 3c: -8 \bar{\gamma}^{ij} \bar{D}_i \bar{D}_j \phi      = -8*phi_dBarDD_contraction
phi_dBar_times_phi_dBar = sp.sympify(0) # Term 3b
phi_dBarDD_contraction  = sp.sympify(0) # Term 3c
for i in range(DIM):
    for j in range(DIM):
        phi_dBar_times_phi_dBar += Bq.gammabarUU[i][j]*Bq.phi_dBarD[i]*Bq.phi_dBarD[j]
        phi_dBarDD_contraction  += Bq.gammabarUU[i][j]*Bq.phi_dBarDD[i][j]

# Add Term 3:
H += Bq.exp_m4phi*(Rbartrace - 8*(phi_dBar_times_phi_dBar + phi_dBarDD_contraction))
```

# Step 3: $\mathcal{M}^i$, the momentum constraint \[Back to [top](#toc)\]
$$\label{momentumconstraint}$$

***Courtesy Ian Ruchlin***

The following definition of the momentum constraint is a simplification of Eq. 47 or [Ruchlin, Etienne, & Baumgarte (2018)](https://arxiv.org/pdf/1712.07658.pdf), which itself was a corrected version of the momentum constraint presented in Eq. 14 of [Baumgarte *et al*](https://arxiv.org/pdf/1211.6632.pdf).

Start with the physical momentum constraint
$$
\mathcal{M}^{i} \equiv D_{j} \left ( K^{i j} - \gamma^{i j} K \right ) = 0 \; .
$$
Expanding and using metric compatibility with the physical covariant derivative $D_{i}$ yields
$$
\mathcal{M}^{i} = D_{j} K^{i j} - \gamma^{i j} \partial_{j} K \; .
$$
The physical extrinsic curvature $K_{i j}$ is related to the trace-free extrinsic curvature $A_{i j}$ by
$$
K_{i j} = A_{i j} + \frac{1}{3} \gamma_{i j} K \; .
$$
Thus,
$$
\mathcal{M}^{i} = D_{j} A^{i j} - \frac{2}{3} \gamma^{i j} \partial_{j} K \; .
$$
The physical metric $\gamma_{i j}$ is related to the conformal metric $\bar{\gamma}_{i j}$ by the conformal rescaling
$$
\gamma_{i j} = e^{4 \phi} \bar{\gamma}_{i j} \; ,
$$
and similarly for the trace-free extrinsic curvature
$$
A_{i j} = e^{4 \phi} \bar{A}_{i j} \; .
$$
It can be shown (Eq. (3.34) in Baumgarte & Shapiro (2010) with $\alpha = -4$ and $\psi = e^{\phi}$) that the physical and conformal covariant derivatives obey
$$
D_{j} A^{i j} = e^{-10 \phi} \bar{D}_{j} \left (e^{6 \phi} \bar{A}^{i j} \right ) \; .
$$
Then, the constraint becomes
$$
\mathcal{M}^i = e^{-4\phi} \left(
{\underbrace {\textstyle \bar{D}_j \bar{A}^{ij}}_{\rm Term\ 1}} +
{\underbrace {\textstyle 6 \bar{A}^{ij}\partial_j \phi}_{\rm Term\ 2}} -
{\underbrace {\textstyle \frac{2}{3} \bar{\gamma}^{ij}\partial_j K}_{\rm Term\ 3}}\right) \; .
$$

Let's first implement Terms 2 and 3:

```python
# Step 3: M^i, the momentum constraint

MU = ixp.zerorank1()

# Term 2: 6 A^{ij} \partial_j \phi:
for i in range(DIM):
    for j in range(DIM):
        MU[i] += 6*Bq.AbarUU[i][j]*Bq.phi_dD[j]

# Term 3: -2/3 \bar{\gamma}^{ij} K_{,j}
trK_dD = ixp.declarerank1("trK_dD") # Not defined in BSSN_RHSs; only trK_dupD is defined there.
for i in range(DIM):
    for j in range(DIM):
        MU[i] += -sp.Rational(2,3)*Bq.gammabarUU[i][j]*trK_dD[j]
```

Now, we turn our attention to Term 1. The covariant divergence involves upper indices in $\bar{A}^{i j}$, but it would be easier for us to finite difference the rescaled $\bar{A}_{i j}$. A simple application of the inverse conformal metric yields
$$
\bar{D}_{j} \bar{A}^{i j} = \bar{\gamma}^{i k} \bar{\gamma}^{j l} \bar{D}_{j} \bar{A}_{k l} \; .
$$
As usual, the covariant derivative is related to the ordinary derivative using the conformal Christoffel symbols
$$
\bar{D}_{k} \bar{A}_{i j} = \partial_{k} \bar{A}_{i j} - \bar{\Gamma}^{l}_{k i} \bar{A}_{l j} - \bar{\Gamma}^{l}_{k j} \bar{A}_{i l} \; .
$$
It is the ordinary derivative above that is approximated by finite difference. The BSSN formulation used here does not rely on spatial derivatives $\partial_{k} \bar{A}_{i j}$ in any of the right-hand-sides (except for the advection term, which uses the upwinded derivative), and so we must declare a new ordinary, centered stencil derivative field of rank 3.

```python
# First define aDD_dD:
aDD_dD = ixp.declarerank3("aDD_dD","sym01")

# Then evaluate the conformal covariant derivative \bar{D}_j \bar{A}_{lm}
AbarDD_dBarD = ixp.zerorank3()
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            AbarDD_dBarD[i][j][k] = Bq.AbarDD_dD[i][j][k]
            for l in range(DIM):
                AbarDD_dBarD[i][j][k] += -Bq.GammabarUDD[l][k][i]*Bq.AbarDD[l][j]
                AbarDD_dBarD[i][j][k] += -Bq.GammabarUDD[l][k][j]*Bq.AbarDD[i][l]

# Term 1: Contract twice with the metric to make \bar{D}_{j} \bar{A}^{ij}
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            for l in range(DIM):
                MU[i] += Bq.gammabarUU[i][k]*Bq.gammabarUU[j][l]*AbarDD_dBarD[k][l][j]

# Finally, we multiply by e^{-4 phi} and rescale the momentum constraint:
for i in range(DIM):
    MU[i] *= Bq.exp_m4phi / rfm.ReU[i]
```

# Step 4: Code Validation against `BSSN.BSSN_constraints` NRPy+ module \[Back to [top](#toc)\]
$$\label{code_validation}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the RHSs of the BSSN equations between
1. this tutorial and
2. the NRPy+ [BSSN.BSSN_constraints](../edit/BSSN/BSSN_constraints.py) module.

By default, we analyze these expressions in Spherical coordinates, though other coordinate systems may be chosen.

```python
# Step 4: Code Validation against BSSN.BSSN_constraints NRPy+ module

# We already have SymPy expressions for BSSN constraints
#         in terms of other SymPy variables. Even if we reset the
#         list of NRPy+ gridfunctions, these *SymPy* expressions for
#         BSSN constraint variables *will remain unaffected*.
#
#         Here, we will use the above-defined BSSN constraint expressions
#         to validate against the same expressions in the
#         BSSN/BSSN_constraints.py file, to ensure consistency between
#         this tutorial and the module itself.
#
# Reset the list of gridfunctions, as registering a gridfunction
#   twice (in the bssnrhs.BSSN_RHSs() call) will spawn an error.
gri.glb_gridfcs_list = []

# Call the BSSN_RHSs() function from within the
#          BSSN/BSSN_RHSs.py module,
#          which should do exactly the same as in Steps 1-16 above.
import BSSN.BSSN_constraints as bssncon
bssncon.BSSN_constraints()

print("Consistency check between BSSN_constraints tutorial and NRPy+ module: ALL SHOULD BE ZERO.")

print("H - bssncon.H = " + str(H - bssncon.H))
for i in range(DIM):
    print("MU["+str(i)+"] - bssncon.MU["+str(i)+"] = " + str(MU[i] - bssncon.MU[i]))
```

    Consistency check between BSSN_constraints tutorial and NRPy+ module: ALL SHOULD BE ZERO.
    H - bssncon.H = 0
    MU[0] - bssncon.MU[0] = 0
    MU[1] - bssncon.MU[1] = 0
    MU[2] - bssncon.MU[2] = 0

# Step 5: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-BSSN_constraints.pdf](Tutorial-BSSN_constraints.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-BSSN_constraints")
```

    Created Tutorial-BSSN_constraints.tex, and compiled LaTeX file to PDF file
        Tutorial-BSSN_constraints.pdf


+ This notebook is part of lecture 2 *Elimination with matrices* in the OCW MIT course 18.06 [1]
+ Created by me, Dr Juan H Klopper
    + Head of Acute Care Surgery
    + Groote Schuur Hospital
    + University Cape Town
    + Email me with your thoughts, comments, suggestions and corrections

Linear Algebra OCW MIT18.06 IPython notebook [2] study notes by Dr Juan H Klopper is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

+ [1] OCW MIT 18.06
+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org

```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

```python
from sympy import init_printing, Matrix, symbols, eye, Rational
from warnings import filterwarnings
```

```python
init_printing(use_latex = 'mathjax')
filterwarnings('ignore')
```

# Elimination

## A system of linear equations

+ Linear refers to the fact that each variable appears on its own (i.e. to the power 1) and is not transcendtal
+ A solution satisfies all of the equations at once
+ Consider the following linear set
$$ 1x+2y+1z=2 \\ 3x + 8y + 1z = 12 \\ 0x + 4y + 1z = 2 $$
+ A solution for *x*, *y*, and *z* could be as follows
$$ 1\left(2\right)+2\left(1\right)+1\left(-2\right)=2 \\ 3\left(2\right)+8\left(1\right)+1\left(-2\right)=12 \\ 0\left(2\right)+4\left(1\right)+1\left(-2\right)=2 $$
+ Since this is a set ( of three) equations that have a solution (solutions) for the variable in common, all left- and all right hand sides can be manipulated in certain ways
    + We could simply exchange the order of the equations (here equations 2 and 3 have been exchanged; row exchange)
    $$ 1x+2y+1z=2 \\ 0x + 4y + 1z = 2 \\ 3x + 8y + 1z = 12 $$
    + We could multiply both the left- and right-hand side of one of the equations with a scalar (here I multiply the first equation by 2)
    $$  2x+4y+2z=4 \\ 3x + 8y + 1z = 12 \\ 0x + 4y + 1z = 2 $$
    + Lastly, we can subtract a constant multiple of one equation from another
        + This serves an excellent purpose, as I can eliminate of one (or more) of the variables (give it a coefficient of 0)
        + Remember that we are trying to solve for all three equations and have three unknowns
        + We can most definitely struggle by doing this problem algebraically by substitution, but linear algebra makes it much easier
        + Here I have multiplies the first equation by 3 (both sides, so that we maintain integrity of the equation) and subtracted the left hand side of this new equation from the left-hand side of equation two and the new right-hand side of equation 1 from the right-hand side of equation two
        + This is quite legitimate, as the left- and right-hand sides are equal (it is an equation after all) and so, when subtracting from equation 2, we are still doing the same thing to the lfet-hand side as the right-hand side
        $$ 1x+2y+1z=2 \\ 0x + 2y - 2z = 6 \\ 0x + 4y + 1z = 2 $$
+ This has introduced a noice zero for me in the second equation
+ Let's go further and multiply equation 2 by 2 and subtract that from equation 3
$$ 1x+2y+1z=2 \\ 0x + 2y - 2z = 6 \\ 0x + 0y + 5z = -10 $$
+ Now let last equation is easy to solve for *z*
$$ z=-2 $$
+ Knowing this I can go back up to equation 2 and solve for *y*
$$ 2y+2(-2)=6 \\ y=1 $$
+ Finally up to equation 1
$$ x+2(1)+1(-2)=2 \\ x=2 $$

+ We need to have gone straight for substitution, indeed, we could have tried to get zeros above all our leading (non-zero) coefficients
+ Let's just clean up equation three by multiplying out by ⅕
$$ 1x+2y+1z=2 \\ 0x + 2y - 2z = 6 \\ 0x + 0y + 1z = -2 $$
+ Now we have to get rid of the -2*z* in equation 2 which we can do by multiplying equation 3 by -2 and subtracting from equations 2
$$ 1x+2y+1z=2 \\ 0x + 2y - 0z = 2 \\ 0x + 0y + 1z = -2 $$
+ Multiplying equation 2 by ½ gives us the following
$$ 1x+2y+1z=2 \\ 0x + 1y + 0z = 1 \\ 0x + 0y + 1z = -2 $$
+ Now we can do the same to get rid of the 1*z* in equation 1 (multiply equation 3 by 1 and subtracting from equation 1)
$$ 1x+2y+0z=4 \\ 0x + 1y + 0z = 1 \\ 0x + 0y + 1z = -2 $$
+ Now tow get rid of the 2*y* in equation 1, which is above our leading 1*y* in equation 2
+ Simple enough, we multiply equation 2 by 2 and subtract that from equation 1
$$ 1x+0y+0z=2 \\ 0x + 1y + 0z = 1 \\ 0x + 0y + 1z = -2 $$
+ The solution is now clear for *x*, *y*, and *z*

+ We need not rewrite all of the variables all the time
+ We can simply write the coefficients
$$ \begin{bmatrix} 1&2&1&2\\3&8&1&12\\0&4&1&2 \end{bmatrix} $$
+ This is called the augmented matrix (right-hand side is added)
    + A matrix has rows and columns (attcahed in position to our algebraic equation above; we simply omit the variables)
+ The left-upper entry is called the pivot
+ Our aim is to get everything below it to be a zero (as we did with the algebra)
+ We do exactely the same as we did above, which is multiply row 1 by 3 and subtract these new values from row 2
$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&4&1&2 \end{bmatrix} $$
+ Now 2 times row 2 subtracted from row 3
$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&0&5&-10 \end{bmatrix} $$
+ Multiply the last row with ⅕
$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&0&1&-2 \end{bmatrix} $$
+ This show 1*z* to equal -2
+ With this small matrix, it's easy to do back substitution as we did algebraically above
+ The first non-zero number in each row is the pivot (just like the upper-left entry)
+ The steps we have taken up to this point is called Gauss elimination and the form we end up with is row-echelon form
+ We could carry on and do the same sort of thing to get rid of all the non-zero entries above each pivot
+ This is called Gauss-Jordan elimination and the result is reduced row-echelon form (see the computer code below)
+ All of these steps are called elementary row operations
+ The only one we didn't do is row exchange
    + We reserve this so as not to have leading (in the pivot position) zeros

```python
A_augmented = Matrix([[1, 2, 1, 2], [3, 8, 1, 12], [0, 4, 1, 2]])
A_augmented
```

+ We can ask python™ to simply get the augmented matrix in reduced row-echelon form and read off the solutions

```python
A_augmented.rref() # The rref() method returns the reduced row-echelon form
```

+ So row one reads as follows
$$ 1x + 0y + 0z = 2 \\ x=2 $$

## Elimination matrices

+ Matrices can only be multiplied by each other if in order we have the first column size equal the second row size
+ Rows are usually called *m* and columns *n*
+ So, our augmented matrix above will be *m* × *n* = 3 × 4
+ Let's look at how matrices are multiplied by looking at two small matrices
$$ \begin{bmatrix} {a}_{11}&{a}_{12}\\{a}_{21}&{a}_{22} \end{bmatrix}\begin{bmatrix} {b}_{11}&{b}_{12}\\{b}_{21}&{b}_{22} \end{bmatrix} $$
+ The subscripts refer to row and column position, i.e. 21 means row 2 column 1
+ We see that we have a 2 × 2 matrix times a 2 × 2 matrix
    + The *inner* two values are the same (2 and 2), so this multiplication is allowed
    + The resultant matrix will have the size equal to the *outer* two values (first row and last columns); here also 2  and 2
+ So let's look at position 11 (row 1 and column 1)
    + To get this we take the entries in row 1 of the first matrix and multiply them by the entries in the first column of the second matrix
    + We do this element by element and add the multiplication of each set of separate elements tow each other
    + The python code below shows you exactly how this is done

```python
a11, a12, a21, a22, b11, b12, b21, b22 = symbols('a11 a12 a21 a22 b11 b12 b21 b22')
```

```python
A = Matrix([[a11, a12], [a21, a22]])
B = Matrix([[b11, b12], [b21, b22]])
A, B
```

```python
A * B
```

+ Let's constrain ourselves to the matrix of coefficients (this discards the right-hand side from the augmented matrix above)

```python
A = Matrix([[1, 2, 1], [3, 8, 1], [0, 4, 1]]) # I use the same computer variable above, which
# will change its value in the computer memory
A # A 3 by 3 matrix, which we call square
```

+ The identity matrix is akin to the number 1, i.e. multiplying by it leaves everything unchanged
+ It has 1's along what is called the main diagonal and 0's everywhere else

```python
I = eye(3) # Identity matrices are always square and the argument
# here is 3, so it is a 3 by 3 matrix
I # Note what the main diagonal is
```

+ Let's multiply this by A

```python
I * A # Nothing will change
```

+ To get rid of the leading 3 in row 2 (because we want a zero under the pivot 1 in row 1), we multiplied row 1 by 3 and subtracted that from row 2
+ Interestingly enough we can do something to this identity matrix that when multiplied by A will results in the first step we have above
+ Since we required to subtract 3 times the first row from the 2 (it's all about that 3 in row 2, column 1), we can do the following

```python
E21 = Matrix([[1, 0, 0], [-3, 1, 0], [0, 0, 1]])
E21 # 21 because we are working on row 2, column 1
```

+ That gives us the required 3 times row 1 and the negative shows that we subtract (add the negative)
+ It's a thing of beauty

```python
E21 * A
```

+ Just what we wanted
+ E1 is called the first elimination matrix

+ Let's do something to the identity matrix to get rif of the 4 in row 3 column 2
+ It would require 2 times row 2 subtracted from row 3
+ Look carefully at the positions

```python
E32 = Matrix([[1, 0, 0], [0, 1, 0], [0, -2, 1]])
E32
```

```python
E32 * (E21 * A)
```

+ Spot on!
+ We now have nice pivots (leading non-zeros), with nothing under them
+ As a tip, try not to get fractions involved
+ As far as the other two row operations are concerned, we can either exchange rows in the identity matrix or multiply the required row by a scalar constant

+ Look at what happens we multiply E2 and E1

```python
L_inv = E32 * E21
L_inv
```

+ Later we'll call this matrix the inverse of L
+ It is in triangular form, in this case lower triangular (note all the zeros above the main diagonal)

```python
L_inv * A # Later we'll call this result the matrix U
```

+ We now have the following
$$ {L}^{-1}{A}={U} $$

+ If we can get the inverse of the inverse of L we'll have the following
$$ {L}{L}^{-1}{A}={L}{U} $$
+ The inverse of a square matrix multiplied by itself gives the identity matrix
$$ {I}{A}={L}{U} \\ {A}={L}{U} $$

+ We can construct L from E32 and E21 above
$$ {E}_{21}^{-1}{E}_{32}^{-1}{E}_{32}{E}_{21}={E}_{21}^{-1}{E}_{32}^{-1}{U} \\ \therefore {E}_{21}^{-1}{E}_{32}^{-1}={L} $$

```python
E21.inv() # The inverse is easy to understand in words
# We just want to add 3 instead of subtracting 3
```

```python
E32.inv()
```

```python
E21.inv() * E32.inv()
```

+ This is exactly the inverse of our inverse of L above

```python
L_inv.inv()
```

+ This is called LU-decomposition of A
+ More about this in two chapter from now (I_05_LU_decomposition)

+ As an aside we can also do elementary column operation, but then we have to multiply on the right of A and not on the left as above

## Example problems

### Example problem 1

+ Solve the following linear set (set of linear equations)
$$ x-y-z+u=0 \\ 2x+2z=8 \\ -y-2z=-8 \\ 3x-3y-2z+4u=7 $$

#### Solution

```python
A_augm = Matrix([[1, -1, -1, 1, 0], [2, 0, 2, 0, 8], [0, -1, -2, 0, -8], [3, -3, -2, 4, 7]])
A_augm
```

```python
A_augm.rref()
```

+ Whoa! That was easy!
+ Let's take it a notch down and do some elementary matrices
+ First off, we want the matrix of coefficients

```python
A = Matrix([[1, -1, -1, 1], [2, 0, 2, 0], [0, -1, -2, 0], [3, -3, -2, 4]])
A
```

+ Now we need to get rid of the 2 in position row 2, column 1
+ We start by numbering the elementary matrix by this position and modifying the identity matrix

```python
E21 = Matrix([[1, 0, 0, 0], [-2, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]])
E21 * A
```

+ Now for position row 3, column 2
+ We have to use row 2 to do this
+ If we used row 1, we would introduce a non-zero into position row 3, column 1

```python
E32 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, Rational(1, 2), 1, 0], [0, 0, 0, 1]])
E32 * (E21 * A)
```

+ Now for the 3 in position row 4, column 1

```python
E41 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [-3, 0, 0, 1]])
E41 * (E32 * E21 * A)
```

+ Let's exchange rows 3 and 4

```python
Ee34 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0]])
Ee34 * E41 * E32 * E21 * A
```

+ Let's see where that leaves **b**, after all, what we do to the left, we must do to the right
$$ {Ee}_{34}\times{E}_{41}\times{E}_{32}\times{E}_{21}{A}{x}={Ee}_{34}\times{E}_{41}\times{E}_{32}\times{E}_{21}{b} $$

```python
b_vect = Matrix([[0], [8], [-8], [7]])
b_vect
```

```python
Ee34 * E41 * E32 * E21 * b_vect
```

+ Let's print them next to each other on the screen

```python
Ee34 * E41 * E32 * E21 * A, Ee34 * E41 * E32 * E21 * b_vect
```

+ So we can simply do back substitution
+ We note that -1*u* = -4 and thus *u* = 4
+ From here, we work our way back up
$$ -1(u)=-4 \quad \therefore \quad u=4 \\ 1(z)+1(4) = 7 \quad \therefore \quad z=3 \\  2(y) + 4(3) - 2(4) = 8 \quad \therefore \quad y=2 \\ 1(x)-1(2)-1(3)+1(4)=0 \quad \therefore x=1 $$

```python

```