# FY 2015 WIC Potential Eligibles Calculation

# Data Sources

This report calculates WIC Potential Eligibles for FY 2015. It was created by the Office of Program Decision Support (raa), FCHS, DSHS 1/20/2016.

Eligibles are calculated using a Modified CPS (Current Population Survey) Based Option. The CPS Based Option is described in:

Estimating Eligibility and Participation for the WIC Program:  Final Report (2003).  Eds. M. Ver Ploeg & D.M. Betson.  The National Academies Press, Washington, D.C., accessed from www.nap.edu on December 31, 2016.

This 2015 calculation follows the prodedure outlined in "CPS Based Option_updated 2011 estimates.doc", created by Jamie Clark on 7/21/2014 and last modified by Dorothy Mandell on 7/21/2014.

Data for children under five is from the Texas State Demographer (http://osd.texas.gov/Data/TPEPP/Projections/). Data is projected for 2015 with full 2000–2010 migration rate. Populations of children of 0, 1, 2, 3, and 4 years were returned for each Texas county.

Texas and county projections, Population Estimates and Projections Program, Texas State Data Center, Office of the State Demographer, Institute for Demographic and Socioeconomic Research, The University of Texas at San Antonio. Data retrieved by Rachel Asquith on 12/31/2015.

Estimates of population below 185% of Federal Poverty Level are from the U.S. Census Bureau.

U.S. Census Bureau; American Community Survey, 2014 American Community Survey 5-Year Estimates, Table B17024; generated by Rachel Asquith; using American FactFinder; <http://factfinder.census.gov>; (1/7/2016)

U.S. Census Bureau; American Community Survey, 2014 American Community Survey 5-Year Estimates, Table C17002; generated by Rachel Asquith; using American FactFinder; <http://factfinder.census.gov>; (1/7/2016)

# Tools

All calculations were completed using Python and performed and presented through a Jupyter, Ipython notebook:

Python Software Foundation. Python Language Reference, version 2.7. Available at http://www.python.org

Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org

Dependencies:

numpy 1.9.2

Stéfan van der Walt, S. Chris Colbert and Gaël Varoquaux. The NumPy Array: A Structure for Efficient Numerical Computation, Computing in Science & Engineering, 13, 22-30 (2011), DOI:10.1109/MCSE.2011.37 (publisher link)

pandas 0.16.2

Wes McKinney. Data Structures for Statistical Computing in Python, Proceedings of the 9th Python in Science Conference, 51-56 (2010) (publisher link)

# Weights

The subsequent outline is copied almost verbatim from the "CPS Based Option_updated 2011 estimates.doc"; 2011 adjustment factors were removed and small changes were made for clarity. The applied adjustment factors are declared in the code blocks and were updated from: http://www.fns.usda.gov/sites/default/files/ops/WICEligibles2013-Volume2.pdf. An additonal weight from this update was added to the 2011 methodology to adjust for nutritional risk for pregnatn women (Pregnant Women D).

The following five sub populations are factored into an overall estimation of the WIC eligible population:

•	Infants

•	Children

•	Pregnant women

•	Postpartum women

•	Breastfeeding women

The formulas for the five populations are:

\begin{equation}Infants: A*B*C*D*E*F\end{equation}
o	A:  The number of infants less than one year of age

o	B:  Adjustment factor to account for the undercount of infants (12 months/11 months)

o	C:  Proportion <185 Federal Poverty Level

o	D:  Adjustment factor to account for infants adjunctively eligible (Medicaid, TANF, and food stamps)

o	E:  Adjustment factor to adjust for variation in monthly income

o	F:  Adjustment factor to adjust for nutritional risk

\begin{equation}Children:  A*B*C*D*E*F\end{equation}
o	A:  Number of children aged 1-4 years old

o	B:  Adjustment factor to account for the undercount of infants (12 months/11 months)

o	C:  Proportion <185 Federal Poverty Level

o	D:  Adjustment factor to account for children adjunctively eligible (Medicaid, TANF, and food stamps)

o	E:  Adjustment factor to adjust for variation in monthly income

o	F:  Adjustment factor to adjust for nutritional risk

\begin{equation}Pregnant Women:  A*B*C*D\end{equation}
o	A:  Number of eligible infants calculated above

o	B:  Adjustment factor to adjust for variation in monthly income during pregnancy and first year postpartum

o	C:  Adjustment factor to account for fetal deaths, infant deaths and multiple births

o	D:  Adjustment factor to adjust for nutritional risk

\begin{equation}Postpartum Women:  A*B*C\end{equation}
o	A:  Number of eligible infants calculated above

o	B:  Adjustment factor to account for fetal deaths, infant deaths and multiple births

o	C:  Adjustment factor to estimate the number of women less than 6 months postpartum who do not breastfeed

\begin{equation}Breastfeeding Women:  A*B*C\end{equation}
o	A:  Number of eligible infants calculated above

o	B:  Adjustment factor to account for fetal deaths, infant deaths and multiple births

o	C:  Adjustment factor to estimate the number of women less than 12 months postpartum who do breastfeed

# Calculation

These first few cells import the necessary libraries, specify the filepath, and set up some formatting.

```python
import pandas as pd
import numpy as np
pd.set_option("display.precision", 4)  # Specifies display precision
```

Step 1: First specify the file paths

```python
def define_file_paths(acs_years):
    '''Create the filepaths for the input data
    Args:
        acs_years(int): 1 or 5
    Returns:
        filepaths(str)'''

    # Points to the CSV with data for children under 5
    county_under5 = fpath + 'TxSDC2014PopPrjctn_Full_Migration'
    state_under5 = county_under5 + '_State.csv'
    county_under5 += '.csv'

    # Points to data on income by ratio of FPL by age
    county_under185 = fpath + 'ACS_14_%sYR_B17024' % str(acs_years)
    state_under185 = county_under185 + '_State.csv'
    county_under185 += '.csv'

    return(county_under5, state_under5, county_under185, state_under185)
```

Step 2: Read, clean, and format the data for projections of children under 5.

The following code performs these actions:

1) Loads the data, selects the desired columns, cleans the input

2) Reshapes the data

3) Adds the total population under 5 and the total population betewen 1 and 4

4) Converts the "Area Code" from the Texas State Demographer projection data into a FIPS code

```python
def preprocess(df, FIPS_col, sum_col_name, sum_col_filter, drop_col_filter, cols_to_drop=[]):
    '''The following function processes the ACS ratio of FPL data frames. It renames and cleans the FIPS column.
    It sums all the estimate columns with FPL ratio groups. The it drops all the margin of error columsn, all the ratio
    group columns besides the sums, and any other specified drop columns.
    Args:
        df (pandas dataframe): dataframe with ACS ratio data with groups of interest
        FIPS_col (string): name of the FIPS column
        sum_col_name (string): desired name of the sum column, e.g. under 185% FPL
        sum_col_filter (string): sum columns with this string in the name
        drop_col_filter (string): drop columns with this string in the name
        cols_to_drop (list of strings): additional columns to drop
    Returns:
        cleaned up dataframe'''

    df = df.rename(columns={FIPS_col: 'FIPS'})
    df['FIPS'] = df['FIPS'].apply(lambda x: str(x).strip())

    cols_to_sum = [
        col for col in df.columns if drop_col_filter not in col and sum_col_filter in col]
    df[sum_col_name] = df[cols_to_sum].sum(axis=1, numeric_only=True)

    return df.drop(cols_to_drop + [col for col in df.columns if drop_col_filter in col] + cols_to_sum, axis=1, inplace=False)
```

Step 3: Create the dataframe

The following code performs these actions:

1) Creates the data frames

2) Sets up a filter for migration

3) Merges, filters, and reshapes the data

```python
def create_data_frame(fpath, migration, acs_years):
    # Create the data frames
    county_under5, state_under5, county_under185, state_under185 = define_file_paths(
        acs_years)

    # Set up a filter value for migration
    if migration == 'full':
        migration_filter = '2000-2010'
    elif migration == 'half':
        migration_filter = '0.5 2000-2010'
    else:
        migration_filter = '0 : No Migration'

    # Load the county level data
    county_under5_df = pd.read_csv(county_under5, header=0,
                                   thousands=None,
                                   dtype={'Age': np.str_})
    # And the state level data; clean the State Code
    state_under5_df = pd.read_csv(state_under5, header=0,
                                  thousands=None,
                                  dtype={'Age': np.str_}).dropna()
    state_under5_df['Area Code'] = 0

    # Merge or replace the county-level data
    if acs_years < 5:
        under5_df = state_under5_df
    else:
        under5_df = pd.concat(
            [county_under5_df, state_under5_df], axis=0, join='outer', ignore_index=True)

    # Filter by Migration
    under5_df = under5_df[under5_df['Migration Rate'] == migration_filter]
    # Format the Population
    under5_df['Total Pop'] = under5_df['Total Pop'].replace(
        regex=True, to_replace=r'\D', value=r'').astype(float)

    # Reshape the data
    under5_df = pd.pivot_table(
        under5_df, values='Total Pop', index=['Area Code', 'Area Name'], columns=['Age'])
    under5_df.columns.name = None
    under5_df = under5_df.reset_index()

    # Add the total population under 5 and the total population betewen 1 and 4
    one_to_four = ['1', '2', '3', '4']
    under5_df['total <5'] = under5_df.sum(
        axis=1, numeric_only=True)  # sums all numeric columns
    # sums only 1, 2, 3, 4
    under5_df[
        '1-4 years'] = under5_df[one_to_four].sum(axis=1, numeric_only=True)

    # Rename the columns
    replacements = dict(zip(one_to_four + ['0', 'Area Code'],
                            [str(col) + ' year' for col in under5_df[one_to_four].columns] + ['infants', 'FIPS']))
    under5_df = under5_df.rename(columns=replacements)

    # Converts the "Area Code" from the Texas State Demographer projection
    # data into a FIPS code
    under5_df['FIPS'] = under5_df['FIPS'].apply(
        lambda x: '48' + str(x).zfill(3))

    # Load and reshape the ACS data
    under185_df = pd.read_csv(county_under185, skiprows=1).iloc[:, :21]
    state_under185_df = pd.read_csv(state_under185, skiprows=1).iloc[:, :21]
    state_under185_df['Id2'] = 48000
    if acs_years < 5:
        under185_df = state_under185_df
    else:
        under185_df = pd.concat(
            [under185_df, state_under185_df], axis=0, join='outer', ignore_index=True)

    # Clean the data
    under185_df = preprocess(under185_df, FIPS_col='Id2', sum_col_name='Under 185% FPL <6',
                             sum_col_filter='Under 6 years: -', drop_col_filter='Error',
                             cols_to_drop=['Id'])

    # Merge both sets
    combined_df = pd.merge(under185_df, under5_df, how='inner', on='FIPS')
    combined_df['Prop Below 185% FPL <6'] = combined_df[
        'Under 185% FPL <6']*1./combined_df['Estimate; Under 6 years:']
    combined_df['Prop Below 185% FPL <6'].fillna(0, inplace=True)
    combined_df = combined_df[['FIPS', 'Geography', 'Estimate; Total:', 'Prop Below 185% FPL <6',
                               'infants', '1-4 years']]
    combined_df.columns = ['FIPS', 'County', 'Total Pop', 'Prop Below 185% FPL <6',
                           'infants', '1-4 years']

    # Return the data set
    return combined_df

```

Step 4: Apply the adjustment factors and calculate the final eligibles. Round the results to the nearest whole person.

The following code performs these actions:

1) Selects the desired columns

2) Calculates the eligibles

3) Calculates some totals

4) Rounds the calculated amount to the nearest individual

5) Sets the output format

```python
def calculate_eligibles(fpath, writer, migration='full', acs_years=5):
    '''Reads the data and calculates the eligibles. Writes results to worksheet
    Args:
        fpath(str): filepath for the input data
        write(pandas.ExcelWriter): where to write the output
        migration(str): 'full', 'half', or 'none'
        acs_years(int): 1 or 5
    Returns:
        None
        '''
    # Determine the sheet name for the output
    out_name = 'Potential Elig 2015' + migration + str(acs_years) + 'YR'

    # Get the data set
    combined_df = create_data_frame(fpath, migration, acs_years)

    # infant_A is population from infant column
    infant_B = 1
    # infant_C is proportion below 185% <6 from column
    infant_D = 1.2866
    infant_E = 1.16
    infant_F = 0.97

    # child_A is population from 1 to 4
    child_B = 1
    # child_C is proportion below 185% <6 from column
    child_D = 1.2906
    child_E = 1.02
    child_F = 0.99

    # PRG_A is eligible infants
    PRG_B = 0.533
    PRG_Adj_compl = 0.9961  # PRG_C (but also noBF_B, and BF_B)
    PRG_D = 0.97

    # noBF_A is eligible infants
    #noBF_B is PRG_Adj_compl
    noBF_C = 0.302

    # BF_A is eligible infants
    #BF_B is PRG_Adj_compl
    BF_C = 0.308

    # Create the eligible dataframe
    df_elig = combined_df.iloc[:, [0, 1, 2, 3]]

    # Apply the adjustments
    elig_infants = combined_df['infants']*infant_B*combined_df['Prop Below 185% FPL <6']*infant_D*infant_E*infant_F*1.0
    df_elig['Infants'] = elig_infants
    df_elig['Children'] = combined_df['1-4 years']*child_B * \
        combined_df['Prop Below 185% FPL <6']*child_D*child_E*child_F
    df_elig['Pregnant'] = elig_infants*PRG_B*PRG_Adj_compl*PRG_D
    df_elig['Postpartum'] = elig_infants*PRG_Adj_compl*noBF_C
    df_elig['Breastfeeding'] = elig_infants*PRG_Adj_compl*BF_C

    # Create a total column
    df_elig['Total'] = df_elig.iloc[:, 4:].sum(
        axis=1, skipna=True, level=None, numeric_only=True)

    # Create a State Total row
    if acs_years == 5:
        row_total = np.concatenate([['48000', 'State All, Texas'], df_elig.iloc[
                                   :-1, :].sum(numeric_only=True).astype(int)]).tolist()
        df_elig = df_elig.append(
            dict(zip(df_elig.columns.tolist(), row_total)), ignore_index=True)
    # Round
    df_elig.iloc[:, 4:] = df_elig.iloc[:, 4:].astype(float).apply(np.round)

    df_elig.to_excel(writer, sheet_name=out_name, index=False)
    workbook = writer.book
    worksheet = writer.sheets[out_name]
    num_format = workbook.add_format({'num_format': '#,##0'})

    # Set the column width and format
    worksheet.set_column('C:C', 18, num_format)
    worksheet.set_column('E:J', 18, num_format)
    worksheet.set_column('A:A', 9)
    worksheet.set_column('B:B', 27)
```

Step 5: Output the potential eligibles to an excel workbook to get the whole set.

```python
fpath = 'Data Files\\'  # Set the filepath for the folder with all the data
writer = pd.ExcelWriter('Potential Eligibles 2015 test.xlsx')
calculate_eligibles(fpath, writer)
writer.save()
```

    c:\anaconda\lib\site-packages\ipykernel\__main__.py:49: SettingWithCopyWarning:
    A value is trying to be set on a copy of a slice from a DataFrame.
    Try using .loc[row_indexer,col_indexer] = value instead

    See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

Potential Eligibles Calculation Example.ipynb


```python
# ORIGINAL RGB FILE
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import random
import pystan

from IPython.display import display

from matplotlib import rc
rc("font", family="serif", size=11)
```

First we will read in a file that contains the overall list of all the available datasets, with corresponding star IDs, $\nu_{max}$ and $\sigma_{\nu_{max}}$ values. From this we will take a random selection for our analysis.

```python
random.seed(121)
#output = pd.read_csv('../data/output_1000stars.csv', usecols=range(1,4))
output = pd.read_csv('../data/output_back_filesremoved.csv', usecols=['kic','numax', 'numax_err'])
output = output.loc[(np.abs(output['numax'] - 75) < 30.0)].reset_index(drop=True)
# Select a random number of stars from the file, recording their IDs
IDs = [random.choice(output['kic']) for i in range(6)]
IDs = [10802837,12003742,2283075,4760954,11358669,11521197]
# Create a list of the numax values corresponding to each ID
Numax = [(output.loc[(output['kic'] == IDs[i])]).iloc[0]['numax'] for i in range(len(IDs))]
Numax_err = [(output.loc[(output['kic'] == IDs[i])]).iloc[0]['numax_err'] for i in range(len(IDs))]
```

Using our list of selected stars, we can then pull the respective files containing the induvidual mode frequencies and errors. The different files will contain varying numbers of modes; in order to successfully parse the data through Stan, one must ensure that all the arrays are of the same length. To do this, for every dataset that contains less modes than the most complete dataset, we can produce additional modes based on the average frequency spacing $\left<\Delta\nu\right>$ for each star, ensuring to assign these points with a large uncertainty so that Stan does not use this as restrictive points during the fit (they are essentially ignored).

```python
modesID = [pd.read_csv('../data/rgbmodes/modes_'+str(IDs[i])+'.csv', usecols=['f0', 'f0_err', 'A0'])
           for i in np.arange(0,len(IDs),1)]
lenmodes = [len(modesID[i]) for i in range(len(IDs))]
maxmodes = max(lenmodes)
arr_n = np.zeros([len(IDs),maxmodes])
arr_freq = np.zeros([len(IDs),maxmodes])
arr_freqerr = np.zeros([len(IDs),maxmodes])

dnu_avgID = []
for i in np.arange(0,len(IDs),1):
    modesID[i] = modesID[i].sort_values(by=['f0'])
    modesID[i] = modesID[i].set_index(np.arange(0,len(modesID[i]),1))
    modesID[i]['dnu'] = (modesID[i]['f0'].diff(2).shift(-1))/2
    dnu_avg = (np.mean(modesID[i]['dnu']))
    dnu_avgID.append(dnu_avg)

    n_min = int(modesID[i]['f0'].min() / dnu_avg)
    n = np.arange(n_min, n_min+len(modesID[i]), 1)
    modesID[i].insert(loc=0, column='n', value=n)

    # Loop to ensure all arrays are the same length
    if lenmodes[i] < maxmodes:
        l = lenmodes[i]
        while l < maxmodes:
            newrow = {'n': int(np.max(modesID[i]['n'])+1),
                      'f0': np.max(modesID[i]['f0'])+dnu_avgID[i],
                      'f0_err': 100000}
            modesID[i] = modesID[i].append(newrow, ignore_index=True)
            l += 1

    arr_n[i,:] = modesID[i]['n']
    arr_freq[i,:] = modesID[i]['f0']
    arr_freqerr[i,:] = modesID[i]['f0_err']

    #display(modesID[i])

dnu_avgarr = np.asarray(dnu_avgID)

#Define start parameters
eps = []
for i in range(len(IDs)):
    epsilon1 = np.median((modesID[i].f0 % dnu_avg) / dnu_avg)
    eps.append(epsilon1)
epsilon = np.asarray(eps)

numax_obs = Numax
nmax = numax_obs/dnu_avgarr - epsilon
alpha = 0.015*dnu_avgarr**(-0.32)
A = 0.06*dnu_avgarr**(-0.88)
G = np.ones([len(IDs)])*3.08
tau = np.ones([len(IDs)])*10
phi = np.ones([len(IDs)])*1.71
```

The data for the different stars can be viewed on an Échelle Diagram.

```python
plt.figure(1)
for i in np.arange(0,len(IDs),1):
    plt.scatter(modesID[i]['f0'] % dnu_avgID[i], modesID[i]['f0'], label=r'ID = '+str(IDs[i]))

plt.ylabel('Frequency')
plt.xlabel(r'Mod. Freq. Spacing ('+ str(u"%.3f" % dnu_avg) +') $\mu$Hz')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.5)
#plt.savefig('seminar/manyrgbs.pdf')

plt.figure(2)
for i in np.arange(0,len(IDs),1):
    plt.scatter(modesID[i]['n'], modesID[i]['f0'], label=r'ID = '+str(IDs[i]))
plt.xlabel('Radial Order ($n$)')
plt.ylabel('Frequency')
plt.show()
```

With all our required data, we can parse the data into a stan model constructed to perform a hierarchical Bayesian model. Hyperpriors have been introduced on a number of parameters based on values in the Vrard (2014) paper.

The model follows the form:
\begin{equation}
    \left(n + \epsilon + \frac{\alpha}{2}\left(n-n_{max}\right)^2 + \frac{\mathcal{AG}}{2\pi}\sin\left(\frac{2\pi(n-n_{max})}{\mathcal{G}} + \phi \right)\right) e^{\frac{n}{\tau}}
\end{equation}
where
\begin{equation}
    n_{max} = \frac{\nu_{max}}{\Delta\nu} - \epsilon
\end{equation}

```python
code = '''
functions {
    real glitch(real n, real dnu, real numax, real epsilon, real alpha, real A, real G, real phi){
        real nmax = numax/dnu - epsilon;
        return (n + epsilon + alpha/2 * (n-nmax)^2 +
                A*G/(2*pi()) * sin((2*pi()*(n-nmax))/G + phi)) * dnu;
    }
}
data {
    int N;  //number of stars
    int M; //number of modes
    real n[N, M];
    real freq[N, M];
    real freq_err[N, M];
    real numax_obs[N];
    real numax_err[N];
    real dnu_guess[N];
}
parameters {
    real dnu[N];
    real numax[N];
    real<lower = -2.0*pi(), upper = 2.0*pi()> phi[N];
    //real<lower = 0> A[N];
    real G[N];
    //real<lower = 0> tau[N];
    real epsilon_std[N];
    real<lower=0> eps_std;
    real epsA;
    real epsB;
    real alpha_std[N];
    real<lower=0> al_std;
    real alA;
    real alB;
    real A_std[N];
    real<lower=0> A_err;
    real AA;
    real AB;
    //real G_std[N];
    //real<lower=0> G_err;
    //real GA;

}

transformed parameters {
    real epsilon[N];
    real alpha[N];
    real A[N];
    //real G[N];
    for (i in 1:N){
        epsilon[i] = epsilon_std[i] * eps_std + (epsA + epsB * log(dnu[i]));
        alpha[i] = alpha_std[i] * al_std + (alA * (dnu[i])^(-alB));
        A[i] = A_std[i] * A_err + (AA * (dnu[i])^(-AB));
        //G[i] = G_std[i] * G_err + GA;
    }
}

model {
    real mod[M];
    for (i in 1:N){
        for (j in 1:M){
            mod[j] = glitch(n[i,j], dnu[i], numax[i], epsilon[i], alpha[i], A[i], G[i], phi[i]);
            // mod[j] ~ normal(freq[i], freq_err[i]);
        }
        freq[i,:] ~ normal(mod, freq_err[i,:]);
        dnu[i] ~ normal(dnu_guess[i], dnu_guess[i]*0.001);
        //A[i] ~ lognormal(log(0.06*dnu[i]^(-0.88)), 0.4);
    }

    //epsilon ~ uniform(-1.0, 2.0);
    //nmax ~ normal(10, 4);
    //alpha ~ lognormal(log(0.015*dnu^(-0.32)), 0.3);

    G ~ normal(3.08, 0.65);

    numax ~ normal(numax_obs, numax_err);

    //tau ~ normal(50, 10);
    epsilon_std ~ normal(0, 1);
    eps_std ~ normal(0, 0.5);
    epsA ~ normal(0.601, 0.25);
    epsB ~ normal(0.632, 0.25);
    alpha_std ~ normal(0, 1);
    al_std ~ normal(0, 0.5);
    alA ~ normal(0.015, 0.005);
    alB ~ normal(0.32, 0.08);
    A_std ~ normal(0, 1);
    A_err ~ normal(0, 0.5);
    AA ~ normal(0.06, 0.02);
    AB ~ normal(0.88, 0.05);
    //G_std ~ normal(0, 1);
    //G_err ~ normal(0.65, 0.5);
    //GA ~ normal(3.08, 0.1);
}
'''
sm = pystan.StanModel(model_code=code)
```

    INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_24226729f4076e83170fb604bfce242e NOW.

```python
stan_data = {'N': len(IDs),
         'M': maxmodes,
         'n': arr_n,
         'freq': arr_freq,
         'freq_err': arr_freqerr,
         'numax_obs': Numax,
         'numax_err': Numax_err,
         'dnu_guess': dnu_avgarr
        }
start = {'dnu': dnu_avgarr,
         'numax': Numax,
         'eps_std': 0.01,
         'al_std': 0.01,
         'A_err': 0.01,
         #'G_err': 0.65,
         #'epsilon': epsilon,
         #'alpha': alpha,
         #'A': A,
         'G': G,
         'phi': phi,
         'epsA': 0.601,
         'epsB': 0.632,
         'alA': 0.015,
         'alB': 0.32,
         'AA': 0.06,
         'AB': 0.88,
         #'GA': 3.08
         #'tau': tau
    }
nchains = 4

fit = sm.sampling(data=stan_data, iter=5000, chains=nchains, init=[start for n in range(nchains)])
```

    WARNING:pystan:10000 of 10000 iterations saturated the maximum tree depth of 10 (100.0%)
    WARNING:pystan:Run again with max_treedepth larger than 10 to avoid saturation

```python
print(fit)
```

    Inference for Stan model: anon_model_24226729f4076e83170fb604bfce242e.
    4 chains, each with iter=5000; warmup=2500; thin=1;
    post-warmup draws per chain=2500, total post-warmup draws=10000.

                     mean se_mean     sd   2.5%    25%    50%    75%  97.5%  n_eff   Rhat
    dnu[1]           5.48  1.5e-4 4.7e-3   5.47   5.48   5.48   5.49   5.49    922    1.0
    dnu[2]           5.39  1.6e-4 4.9e-3   5.39   5.39   5.39    5.4    5.4   1016    1.0
    dnu[3]           6.45  1.9e-4 4.6e-3   6.44   6.45   6.45   6.45   6.46    599   1.01
    dnu[4]           4.75  1.3e-4 4.2e-3   4.74   4.75   4.75   4.75   4.76    991    1.0
    dnu[5]           8.43  1.7e-4 4.7e-3   8.42   8.43   8.43   8.44   8.44    806    1.0
    dnu[6]           6.17  1.4e-4 4.6e-3   6.16   6.16   6.17   6.17   6.18   1096    1.0
    numax[1]        57.26    0.01   0.35  56.58  57.03  57.27   57.5  57.93    595    1.0
    numax[2]        59.85    0.02    0.4  59.06  59.57  59.85  60.12  60.61    381   1.02
    numax[3]        67.68    0.01   0.33  67.04  67.45  67.68  67.89  68.33    495   1.01
    numax[4]        45.75    0.01   0.32   45.1  45.53  45.75  45.97  46.37    825    1.0
    numax[5]         98.1  9.6e-3   0.26   97.6  97.93   98.1  98.27  98.61    732   1.01
    numax[6]        67.64    0.01   0.29  67.09  67.45  67.64  67.84  68.22    639   1.01
    phi[1]           0.85  9.3e-3   0.22   0.41    0.7   0.86    1.0   1.28    560   1.01
    phi[2]          -0.04    0.01   0.24  -0.51   -0.2  -0.04   0.12   0.42    340   1.02
    phi[3]           1.56  6.3e-3   0.15   1.28   1.46   1.55   1.65   1.85    543   1.01
    phi[4]           0.14    0.01   0.31  -0.42  -0.07   0.12   0.34   0.81    600   1.01
    phi[5]           2.38  9.0e-3   0.22    1.9   2.25    2.4   2.54   2.77    598    1.0
    phi[6]           2.62  7.6e-3   0.18   2.26   2.51   2.63   2.74   2.96    536   1.01
    G[1]              2.7  9.4e-3   0.22    2.3   2.55   2.68   2.83   3.16    566   1.01
    G[2]             2.59  6.6e-3   0.13   2.32    2.5   2.59   2.68   2.84    401   1.01
    G[3]             2.67  2.8e-3   0.08   2.51   2.62   2.67   2.73   2.84    917    1.0
    G[4]              3.1  7.8e-3   0.19    2.8   2.97   3.07    3.2   3.54    564    1.0
    G[5]             2.81  5.2e-3   0.12   2.62   2.73    2.8   2.88   3.12    574    1.0
    G[6]             2.38  8.3e-3   0.16   2.13   2.26   2.36   2.47   2.73    353   1.01
    epsilon_std[1]  -0.92    0.04    0.4  -1.75  -1.19  -0.91  -0.65  -0.19    124   1.05
    epsilon_std[2]  -0.55    0.03   0.38  -1.31  -0.82  -0.55  -0.29   0.15    125   1.06
    epsilon_std[3]  -0.58    0.04   0.41  -1.39  -0.86  -0.57  -0.29   0.19    124   1.06
    epsilon_std[4]   1.43    0.04   0.73   0.15   0.88   1.39   1.93   2.96    332   1.02
    epsilon_std[5]  -0.56    0.04   0.47  -1.48  -0.88  -0.56  -0.23   0.35    128   1.06
    epsilon_std[6]  -0.75    0.04    0.4  -1.55  -1.03  -0.73  -0.47   0.02    125   1.06
    eps_std          0.43  7.4e-3   0.16   0.22   0.31   0.39   0.51   0.84    481   1.01
    epsA             0.31  9.5e-3   0.22  -0.12   0.16   0.31   0.45   0.73    523   1.01
    epsB             0.06  8.6e-3   0.15  -0.22  -0.04   0.05   0.16   0.36    304   1.03
    alpha_std[1]     0.85    0.03   0.79  -0.81   0.37   0.88   1.36   2.35    695    1.0
    alpha_std[2]     0.41    0.03    0.8  -1.26  -0.09   0.44   0.94   1.93    758   1.01
    alpha_std[3]     0.69    0.03   0.62   -0.4   0.26   0.64   1.07   2.03    607    1.0
    alpha_std[4]    -0.14    0.03   0.77  -1.65  -0.62  -0.12   0.34   1.43    568   1.01
    alpha_std[5]    -0.31    0.03   0.61   -1.6   -0.7  -0.28   0.11   0.79    480   1.01
    alpha_std[6]    -0.56    0.03    0.7  -2.03  -1.01  -0.53  -0.09   0.75    630   1.01
    al_std         2.1e-3  7.7e-5 1.6e-3 3.0e-4 1.1e-3 1.7e-3 2.6e-3 6.0e-3    414   1.01
    alA              0.02  1.2e-4 2.8e-3   0.02   0.02   0.02   0.02   0.03    551   1.01
    alB              0.28  2.8e-3   0.07   0.15   0.23   0.28   0.33   0.41    580    1.0
    A_std[1]         0.58    0.02   0.36   0.04   0.32   0.52   0.77   1.45    306   1.01
    A_std[2]         1.71    0.03   0.59   0.72   1.29   1.66   2.07   2.99    324   1.02
    A_std[3]         0.42    0.01   0.19   0.13   0.29    0.4   0.53   0.84    265   1.02
    A_std[4]         0.56    0.02   0.29    0.1   0.35   0.52   0.72   1.22    291   1.03
    A_std[5]         0.27  8.2e-3   0.15   0.03   0.17   0.25   0.35   0.59    316   1.03
    A_std[6]         0.65    0.03   0.45   0.04   0.34   0.56   0.87   1.79    310   1.01
    A_err            0.04  1.1e-3   0.02   0.02   0.03   0.04   0.05   0.09    291   1.01
    AA               0.07  8.2e-4   0.02   0.03   0.06   0.07   0.09   0.11    632   1.01
    AB               0.87  1.8e-3   0.05   0.77   0.83   0.87    0.9   0.97    819   1.01
    epsilon[1]       0.03  3.1e-4 8.9e-3   0.01   0.02   0.03   0.04   0.05    852    1.0
    epsilon[2]       0.17  3.3e-4   0.01   0.15   0.16   0.17   0.18   0.19   1055    1.0
    epsilon[3]       0.17  3.0e-4 7.5e-3   0.15   0.16   0.17   0.17   0.18    602   1.01
    epsilon[4]       0.92  3.0e-4 9.2e-3    0.9   0.91   0.92   0.92   0.94    942   1.01
    epsilon[5]       0.19  2.2e-4 6.4e-3   0.18   0.18   0.19   0.19    0.2    855    1.0
    epsilon[6]        0.1  2.5e-4 8.4e-3   0.09    0.1    0.1   0.11   0.12   1096    1.0
    alpha[1]         0.01  6.6e-5 1.7e-3   0.01   0.01   0.01   0.02   0.02    650   1.01
    alpha[2]         0.01  5.6e-5 1.5e-3   0.01   0.01   0.01   0.01   0.02    712    1.0
    alpha[3]         0.01  1.6e-5 6.3e-4   0.01   0.01   0.01   0.01   0.01   1514    1.0
    alpha[4]         0.01  3.4e-5 1.2e-3   0.01   0.01   0.01   0.01   0.02   1256   1.01
    alpha[5]         0.01  9.6e-6 5.0e-4   0.01   0.01   0.01   0.01   0.01   2725    1.0
    alpha[6]         0.01  3.6e-5 1.0e-3 9.5e-3   0.01   0.01   0.01   0.01    779    1.0
    A[1]             0.04  5.7e-4   0.01   0.02   0.03   0.04   0.04   0.06    530   1.01
    A[2]             0.08  3.5e-4   0.01   0.06   0.07   0.08   0.08    0.1    946    1.0
    A[3]             0.03  4.5e-5 2.8e-3   0.02   0.03   0.03   0.03   0.04   3798    1.0
    A[4]             0.04  2.1e-4 6.7e-3   0.03   0.03   0.04   0.04   0.05   1000    1.0
    A[5]             0.02  8.4e-5 2.8e-3   0.02   0.02   0.02   0.02   0.03   1067    1.0
    A[6]             0.04  8.9e-4   0.02   0.02   0.03   0.04   0.05   0.09    433    1.0
    lp__           -82.75    0.24   6.16 -95.67 -86.64 -82.41 -78.46 -71.75    632    1.0

    Samples were drawn using NUTS at Tue Feb 12 16:15:37 2019.
    For each parameter, n_eff is a crude measure of effective sample size,
    and Rhat is the potential scale reduction factor on split chains (at
    convergence, Rhat=1).

```python
fit.plot()
plt.show()
```

```python
#%matplotlib inline
def model(n,dnu,numax,eps,alpha,A,G,phi):#,tau):
    nmax = numax/dnu - eps
    s = (n + eps + ((alpha/2)*(nmax-n)**2) +
         (((A*G)/(2*np.pi)) * np.sin((2*np.pi*(n-nmax))/G + phi)))*dnu#*np.exp(-n/(tau))
    return s

no_osc = np.loadtxt('/home/daniel/Documents/y4project/Dan_notebooks/no_osc_models.csv', delimiter=',')

fig = plt.figure(1, figsize=(12,4))
ax  = fig.add_subplot(111)
ax.set_position([0.1,0.1,0.6,0.85])

import itertools
colours = itertools.cycle(('r', 'orange', 'olive', 'teal', 'sienna', 'indigo'))

params = np.zeros([len(IDs), 8])

for i in np.arange(0,len(IDs),1):
    # Iterate point colour
    colour = next(colours)
    pts = modesID[i].loc[(modesID[i]['f0_err'] < 10e3)]
    # Plot all frequencies, and errorbars for the 'true' data
    ax.scatter(modesID[i]['f0'] % dnu_avgID[i], modesID[i]['f0'],
                label='', zorder=-1, facecolors='none', edgecolors=colour)
    ax.scatter(pts['f0'] % dnu_avgID[i], pts['f0'],
                label=r'ID = '+str(IDs[i]), c=colour)
    ax.errorbar(pts['f0'] % dnu_avgID[i], pts['f0'], xerr=pts['f0_err'],
                        zorder=0, fmt="none", label='', c='k',
                        capsize=2, markersize=4, elinewidth=1)

    params[i] = [np.mean(fit['dnu'], axis=0)[i],
           np.mean(fit['numax'], axis=0)[i],
           np.mean(fit['epsilon'], axis=0)[i],
           np.mean(fit['alpha'], axis=0)[i],
           np.mean(fit['A'], axis=0)[i],
           np.mean(fit['G'], axis=0)[i],
           np.mean(fit['phi'], axis=0)[i],
                np.nan]

    # Plot the modelled fit
    g2 = model(arr_n[i], params[i,0], params[i,1], params[i,2], params[i,3],
               params[i,4], params[i,5], params[i,6])
    ax.plot(g2 % dnu_avgID[i], g2, c=colour)
    ax.plot(no_osc[i] % dnu_avgID[i], no_osc[i], linestyle=':', c=colour)

np.savetxt('no_tau_models.csv', params, delimiter=',')

plt.legend()
plt.ylabel('Frequency')
plt.xlabel(r'Mod. Freq. Spacing ($\mu$Hz)')
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
#plt.savefig('seminar/somergbseizure.pdf', bbox='tight_layout')
plt.show()
```

```python
%matplotlib inline
plt.figure(2)
for i in np.arange(0,len(IDs),1):
    plt.scatter(modesID[i]['n'], modesID[i]['f0'], label=r'ID = '+str(IDs[i]))
plt.xlabel('Radial Order ($n$)')
plt.ylabel('Frequency')
plt.legend()
plt.show()

fig, ax = plt.subplots(len(IDs))
for i in np.arange(0,len(IDs),1):
    ax[i].errorbar(modesID[i]['n'], modesID[i]['f0'] - g2, yerr=modesID[i]['f0_err']*10)
    #ax[i].set_xlim([6,20])
    #ax[i].set_ylim([-3.5, 3.5])
    ax[i].set_ylim([-15, 15])

plt.show()
```

```python
no_osc = np.loadtxt('/home/daniel/Documents/y4project/Dan_notebooks/no_osc_models.csv', delimiter=',')

fig = plt.figure(1, figsize=(12,4))
ax  = fig.add_subplot(111)
ax.set_position([0.1,0.1,0.6,0.85])

import itertools
colours = itertools.cycle(('r', 'orange', 'olive', 'teal', 'sienna', 'indigo'))

for i in np.arange(0,len(IDs),1):
    colour = next(colours)
    pts = modesID[i].loc[(modesID[i]['f0_err'] < 10e3)]
    ax.scatter(modesID[i]['f0'] % dnu_avgID[i], modesID[i]['f0'],
                label='', zorder=-1, facecolors='none', edgecolors=colour)
    ax.scatter(pts['f0'] % dnu_avgID[i], pts['f0'],
                label=r'ID = '+str(IDs[i]), c=colour)
    ax.errorbar(pts['f0'] % dnu_avgID[i], pts['f0'], xerr=pts['f0_err'],
                        zorder=0, fmt="none", label='', c='k',
                        capsize=2, markersize=4, elinewidth=1)

    # Plot the modelled fit
    g2 = model(arr_n[i],
           np.mean(fit['dnu'], axis=0)[i],
           np.mean(fit['numax'], axis=0)[i],
           np.mean(fit['epsilon'], axis=0)[i],
           np.mean(fit['alpha'], axis=0)[i],
           np.mean(fit['A'], axis=0)[i],
           np.mean(fit['G'], axis=0)[i],
           np.mean(fit['phi'], axis=0)[i],
           #np.mean(fit['tau'], axis=0)[i]
              )
    ax.plot(g2 % dnu_avgID[i], g2,
             c=colour)
    ax.plot(no_osc[i] % dnu_avgID[i], no_osc[i], linestyle=':', c=colour)

colours = itertools.cycle(('r', 'orange', 'olive', 'teal', 'sienna', 'indigo'))
fig, ax = plt.subplots(len(IDs), figsize=(5,14))
for i in np.arange(0,len(IDs),1):
    #ax[i].scatter(no_osc[i], (no_osc[i] % dnu_avgID[i]))
    #ax[i].scatter(osc[i], (osc[i] % dnu_avgID[i]))
    ax[i].scatter(no_osc[i], (no_osc[i] % dnu_avgID[i])-(osc[i] % dnu_avgID[i]), c=next(colours))
plt.xlabel(r'Freqency ($\mu$Hz)')
plt.show()
```

```python

```


#### New to Plotly?
Plotly's Python library is free and open source! [Get started](https://plotly.com/python/getting-started/) by downloading the client and [reading the primer](https://plotly.com/python/getting-started/).

You can set up Plotly to work in [online](https://plotly.com/python/getting-started/#initialization-for-online-plotting) or [offline](https://plotly.com/python/getting-started/#initialization-for-offline-plotting) mode, or in [jupyter notebooks](https://plotly.com/python/getting-started/#start-plotting-online).

We also have a quick-reference [cheatsheet](https://images.plot.ly/plotly-documentation/images/python_cheat_sheet.pdf) (new!) to help you get started!

#### Imports
The tutorial below imports [NumPy](http://www.numpy.org/), [Pandas](https://plotly.com/pandas/intro-to-pandas-tutorial/), [SciPy](https://www.scipy.org/) and [PeakUtils](http://pythonhosted.org/PeakUtils/).

```python
import plotly.plotly as py
import plotly.graph_objs as go
import plotly.figure_factory as ff

import numpy as np
import pandas as pd
import scipy
import peakutils
```

#### Tips
Our method for finding the area under any peak is to find the area from the `data values` to the x-axis, the area from the `baseline` to the x-axis, and then take the difference between them. In particular, we want to find the areas of these functions defined on the x-axis interval $I$ under the peak.

Let $T(x)$ be the function of the data, $B(x)$ the function of the baseline, and $Area$ the peak integration area between the baseline and the first peak. Since $T(x) \geq B(x)$ for all $x$, then we know that

$$
\begin{align}
A = \int_{I} T(x)dx - \int_{I} B(x)dx
\end{align}
$$

#### Import Data
For our example below we will import some data on milk production by month:

```python
milk_data = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/monthly-milk-production-pounds.csv')
time_series = milk_data['Monthly milk production (pounds per cow)']
time_series = np.asarray(time_series)

df = milk_data[0:15]

table = ff.create_table(df)
py.iplot(table, filename='milk-production-dataframe')
```

#### Area Under One Peak

```python
baseline_values = peakutils.baseline(time_series)

x = [j for j in range(len(time_series))]
time_series = time_series.tolist()
baseline_values = baseline_values.tolist()

rev_baseline_values = baseline_values[:11]
rev_baseline_values.reverse()
area_x = [0,1,2,3,4,5,6,7,8,9,10,11,10,9,8,7,6,5,4,3,2,1]
area_y = time_series[:11] + rev_baseline_values

trace = go.Scatter(
    x=x,
    y=time_series,
    mode='lines',
    marker=dict(
        color='#B292EA',
    ),
    name='Original Plot'
)

trace2 = go.Scatter(
    x=x,
    y=baseline_values,
    mode='markers',
    marker=dict(
        size=3,
        color='#EB55BF',
    ),
    name='Bassline'
)

trace3 = go.Scatter(
    x=area_x,
    y=area_y,
    mode='lines+markers',
    marker=dict(
        size=4,
        color='rgb(255,0,0)',
    ),
    name='1st Peak Outline'
)

first_peak_x = [j for j in range(11)]
area_under_first_peak = np.trapz(time_series[:11], first_peak_x) - np.trapz(baseline_values[:11], first_peak_x)
area_under_first_peak

annotation = go.Annotation(
    x=80,
    y=1000,
    text='The peak integration for the first peak is approximately %s' % (area_under_first_peak),
    showarrow=False
)

layout = go.Layout(
    annotations=[annotation]
)

trace_data = [trace, trace2, trace3]
fig = go.Figure(data=trace_data, layout=layout)
py.iplot(fig, filename='milk-production-peak-integration')
```

    /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/plotly/graph_objs/_deprecations.py:144: DeprecationWarning:

    plotly.graph_objs.Annotation is deprecated.
    Please replace it with one of the following more specific types
      - plotly.graph_objs.layout.Annotation
      - plotly.graph_objs.layout.scene.Annotation

```python
from IPython.display import display, HTML

display(HTML('<link href="//fonts.googleapis.com/css?family=Open+Sans:600,400,300,200|Inconsolata|Ubuntu+Mono:400,700" rel="stylesheet" type="text/css" />'))
display(HTML('<link rel="stylesheet" type="text/css" href="http://help.plot.ly/documentation/all_static/css/ipython-notebook-custom.css">'))

! pip install git+https://github.com/plotly/publisher.git --upgrade
import publisher
publisher.publish(
    'python-Peak-Integration.ipynb', 'python/peak-integration/', 'Peak Integration | plotly',
    'Learn how to integrate the area between peaks and bassline in Python.',
    title='Peak Integration in Python | plotly',
    name='Peak Integration',
    language='python',
    page_type='example_index', has_thumbnail='false', display_as='peak-analysis', order=4,
    ipynb= '~notebook_demo/121')
```

<link href="//fonts.googleapis.com/css?family=Open+Sans:600,400,300,200|Inconsolata|Ubuntu+Mono:400,700" rel="stylesheet" type="text/css" />

<link rel="stylesheet" type="text/css" href="http://help.plot.ly/documentation/all_static/css/ipython-notebook-custom.css">

    Collecting git+https://github.com/plotly/publisher.git
      Cloning https://github.com/plotly/publisher.git to /private/var/folders/tc/bs9g6vrd36q74m5t8h9cgphh0000gn/T/pip-req-build-iKhEAt
    Building wheels for collected packages: publisher
      Running setup.py bdist_wheel for publisher ... done
      Stored in directory: /private/var/folders/tc/bs9g6vrd36q74m5t8h9cgphh0000gn/T/pip-ephem-wheel-cache-1Y3Gor/wheels/99/3e/a0/fbd22ba24cca72bdbaba53dbc23c1768755fb17b3af0f33966
    Successfully built publisher
    Installing collected packages: publisher
      Found existing installation: publisher 0.11
        Uninstalling publisher-0.11:
          Successfully uninstalled publisher-0.11
    Successfully installed publisher-0.11

    /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/nbconvert.py:13: ShimWarning:

    The `IPython.nbconvert` package has been deprecated since IPython 4.0. You should import from nbconvert instead.

    /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/publisher/publisher.py:53: UserWarning:

    Did you "Save" this notebook before running this command? Remember to save, always save.

```python

```


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c) 2019 Daniel Koehn, based on (c)2018 L.A. Barba, G.F. Forsyth [CFD Python](https://github.com/barbagroup/CFDPython#cfd-python), (c)2014 L.A. Barba, I. Hawke, B. Knaepen [Practical Numerical Methods with Python](https://github.com/numerical-mooc/numerical-mooc#practical-numerical-methods-with-python), also under CC-BY.

```python
# Execute this cell to load the notebook's style sheet, then ignore it
from IPython.core.display import HTML
css_file = '../style/custom.css'
HTML(open(css_file, "r").read())
```

<link href="https://fonts.googleapis.com/css?family=Merriweather:300,300i,400,400i,700,700i,900,900i" rel='stylesheet' >
<link href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:300,300i,400,400i,700,700i" rel='stylesheet' >
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' >
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 800px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.5em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Source Sans Pro', sans-serif;
    line-height: 140%;
    font-size: 110%;
    width:680px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Merriweather', serif;
    font-style:regular;
    font-weight: bold;
    font-size: 250%;
    line-height: 100%;
    color: #004065;
    margin-bottom: 1em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Merriweather', serif;
    font-weight: bold;
    font-size: 180%;
    line-height: 100%;
    color: #0096d6;
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Merriweather', serif;
	font-size: 150%;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: regular;
    color: #008367;
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Merriweather', serif;
    font-weight: 300;
    font-size: 100%;
    line-height: 120%;
    text-align: left;
    width:500px;
    margin-top: 1em;
    margin-bottom: 2em;
    margin-left: 80pt;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Source Sans Pro', sans-serif;
    font-weight: regular;
    font-size: 130%;
    color: #e31937;
    font-style: italic;
    margin-bottom: .5em;
    margin-top: 1em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'Source Code Pro', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

# 2D Thermal Convection in the Earth Mantle: Introduction

In the modules 4-7 of this lecture, we decomposed the general equations for the conservation of mass, momentum and energy into different sub-problems:

- (non)-linear Advection
- linear Diffusion
- Boundary value problems (Poisson equations)

For each problem, we discussed different approaches to compute numerical solutions using the finite-difference approach. In the following modules we will assemble the individual problems to more complex differential equations related to earth science. We begin with 2D thermal convection in the earth mantle, which includes all problems mentioned above.

## Thermal Convection in the Earth Mantle

We already derived the governing equations for thermal convection in [lecture 3](https://danielkoehnsite.files.wordpress.com/2019/11/lect_03_simple_convection_lorenz_eqs-1.pdf) , when investigating the complex dynamics of the [Lorenz equations](https://github.com/daniel-koehn/Differential-equations-earth-system/tree/master/03_Lorenz_equations#lecture-3-simplified-convection-problem-the-lorenz-equations). Let's shortly revisit the basic results:

The full coupled set of scaled PDE's describe the coupling of incompressible 2D Navier Stokes flow with an advection-diffusion equation for temperature, and can be written in dimensionless form as,

$$
   \frac{1}{\mathrm{Pr}}\left[ \frac{\partial \omega}{\partial t} + \vec{v}\cdot\nabla\omega\right] = \nabla^2\omega + \mathrm{Ra}\frac{\partial T}{\partial x}
$$
$$
    \nabla^2 \psi = -\omega
$$
$$
    \frac{\partial T}{\partial t} + \vec{v}\cdot\nabla T = \nabla^2 T
$$

where

$$
    \vec{v}=(u,0,w) = \nabla\times\psi\hat{j}=(-\frac{\partial\psi}{\partial z}, 0,  \frac{\partial\psi}{\partial x})
$$

is the fluid velocity field (which in this form is exactly incompressible with $\nabla\cdot\vec{v}=0$).  $\psi$ is the "Streamfunction"  whose contours are tangent to the fluid trajectories at all times.  The vorticity,

$$
    \omega = \bf{\vec{\omega} \cdot \hat{j}} = (\nabla\times\vec{v}) \cdot \hat{j}
$$

measures the local rate of rotation of the fluid, and is driven by horizontal variations in temperature (actually density). $\hat{j}=(0,1,0)^T$ denotes the unit vector in y-direction.

Now, if we are dealing with mantle convection, the Prandtl number Pr becomes very large, actually it can be assumed to be inifinite. Therefore, the governing equations simplify to

\begin{equation}
\begin{split}
\nabla^2\omega &=  - \mathrm{Ra}\frac{\partial T}{\partial x}\\
\nabla^2 \psi &= -\omega\\
\frac{\partial T}{\partial t} &= \nabla^2 T - \vec{v}\cdot\nabla T
\end{split}
\tag{1}
\end{equation}

The differential equations (1) consist of different subproblems. For a given temperature field $T$, we have to solve the coupled **Poisson equations**

\begin{equation}
\begin{split}
\nabla^2\omega &=  - \mathrm{Ra}\frac{\partial T}{\partial x}\\
\nabla^2 \psi &= -\omega
\end{split}
\tag{2}
\end{equation}

in order to estimate the streamfunction $\psi$ and vorticity $\omega$. We already discussed the solution of a similar problem using the steepest descent method when trying to find [stationary solutions for the 2D thermal mantle convection problem](https://nbviewer.jupyter.org/github/daniel-koehn/Differential-equations-earth-system/blob/master/07_BVP_2D/05_BVP_2D_Thermal_Mantle_Convection.ipynb).

Therefore, the remaining problem is the 2D heat advection-diffusion problem from the conservation of energy equation

\begin{equation}
\frac{\partial T}{\partial t} = \nabla^2 T - \vec{v}\cdot\nabla T \notag
\end{equation}

or by inserting the velocity vector $\vec{v}=(u,0,w) =(-\frac{\partial\psi}{\partial z}, 0,  \frac{\partial\psi}{\partial x})$ using the streamfunction

\begin{equation}
\frac{\partial T}{\partial t} = \nabla^2 T + \frac{\partial\psi}{\partial z} \frac{\partial T}{\partial x} - \frac{\partial\psi}{\partial x} \frac{\partial T}{\partial z} \tag{3}
\end{equation}

Using finite-differences, problem (3) can be further decomposed into two well known subproblems using **operator splitting**:

- The 2D linear advection problem:

\begin{equation}
\frac{\tilde{T}^{n+1}-T^{n}}{\Delta t} - \frac{\partial\psi}{\partial z} \frac{\partial T}{\partial x} + \frac{\partial\psi}{\partial x} \frac{\partial T}{\partial z} = 0\tag{4}
\end{equation}

- The 2D heat conduction problem:

\begin{equation}
\frac{\partial \tilde{T}}{\partial t} = \nabla^2 \tilde{T} \tag{5}
\end{equation}

## How to solve the 2D time-dependent mantle convection problem

In summary, we can solve the 2D mantle convection problem using the following approach. Starting from an initial condition $T_0, \psi_0, \omega_0$, we compute at each time step ...

1. For a given temperature field $T$, solve the Poisson equations (2) to estimate the streamfunction $\psi$ and vorticity $\omega$, as described in the [2D mantle convection BVP notebook](https://nbviewer.jupyter.org/github/daniel-koehn/Differential-equations-earth-system/blob/master/07_BVP_2D/05_BVP_2D_Thermal_Mantle_Convection.ipynb)

2. Solve the 2D linear advection problem (4)

3. Solve the 2D heat conduction problem (5)

Let 's take a closer look on how to implement the 2D linear advection and heat conduction problems separately in the following notebooks, before assembling the final mantle convection modelling code.


# CS230 Deep learning

Name: Bicheng Wang
SUNETID: bichengw@stanford.edu
Signature: Bicheng Wang

## Question 1

(a) iv

(b) i, ii, iii, iv

(c) i, ii

(d) ii

(e) iv

(f) iv

(g) iv

(h) iv

## Question 2

(a)
The model already accurately predicts all training data, no error need to backpropagate.
The model cannot rightly backpropagate the training error.

(b)
The first one is we should avoid the false positive result, secondly, we also hope to push model performance.
Thus,
1.	Precision rate is one important metrics if we do not want to punish the good people.
2.	At the same time, if we also hope to ensure the city safety, we need also ensure the Recall rate not too low, So choose F1 score could ensure both precision and recall rate as high as possible.

(c)
The benefit: a. increase model invariance, like translation, rotation, scale invariance. b. reduces whole information but focus on experienced important feature, improve the calculation efficiency, and reduce the overfit possibility.

(d)
High variance of model means it highly depend on specific data and low generalization ability and learn more noisy to make the model overfit. Possible way like consider drop out, batch norm.

(e)
Small batch benefit like reduce calculation resource requirement in each batch train.
Disadvantage may have model convergence slow or not stable.

(f)
A. as softmax design for highly sensitive for higher value and the result as 0~1 possiblity, it ensure the max feature of final possibility result, B. Softmax easy to derive on backpropagation, especially combine with cross entropy, which often use in mutli-classification.

(g)
A. L1 is better on some case like as L1 sparse, it is a good way to select feature. B. L1 lower calculation performance than l2, as l2 good derive feature especially close to 0, and good at no-sparse vector calculation. C. l2 as MSE, the error > 1 outlier would be detecting more sensitive outlier, which also makes l2 has better performance than l1.

(h)
As downsampled the false class, model may not fully learn some part of false class feature. But no action needs to take on testing.

## Question 3

(i) HWC: 126, 126, 64, number of param: 1792

(ii) HWC: 128 128 16 param: 1216

(iii) HWC: 64 64 32 param: 416

(iv) Output HWC: 14 14 64 total param: 75648

(v) Output HWC: 13 13 16 total param: 14048

(vi) Output HWC: 2 2 32 total param: 8672

(vii) The (vi) model. 128 to 10, said layer param: 1290

## Question 4:

(i) The direvative func
\begin{equation}
\nabla_w \widetilde{J} = \nabla_w J + \alpha \cdot sign(w)
\end{equation}

(ii) the original func could convert to:

\begin{equation}
J(w; X, y) = J(w^*; X, y) + sum_i(\frac{1}{2}H_{i,i}(w_i - w_i^*)^2 + \alpha_i|w_i|
\end{equation}

And as the Hessian func we assume no correlation between features, and when gradient equal to zero, the derivative func by weight i could be:

\begin{equation}
H_{i,i}(w_i - w_i^*) + \alpha \cdot sign(w_i) = 0
\end{equation}

Then,

\begin{equation}
w_i = sign(w^*) \cdot (|w_i^*| - \frac{\alpha}{H_{i,i}})
\end{equation}

Given condition 1:
\begin{equation}
|w_i^*| - \frac{\alpha}{H_{i,i}} > 0
\end{equation}
The new weight i update by:
\begin{equation}
w_i = sign(w^*) \cdot ||w_i^*| - \frac{\alpha}{H_{i,i}}|
\end{equation}

Given condition 2:
\begin{equation}
|w_i^*| - \frac{\alpha}{H_{i,i}} <= 0
\end{equation}
The new weight i would be:
\begin{equation}
w_i = 0
\end{equation}

(iii)
When large weight, the l1 update weight with a fixed number, while l2 update weight with a fixed number multiply weight value itself, if weight value > 1, the gradient faster.
When small weight, the l1 update a fixed number, while l2 update step smaller, gradient slower.
When weight close to 0, as l2 update a linear value with weight itself, l2 would more smooth to close to 0. While the l1 may not stable highly depend on the update constant value itself.

## Question 5:

(i)

\begin{equation}
\nabla_{\theta_{d}} L (X; \theta_d, \theta_g) = \frac{1}{n} \sum_{i=1}^{n} (\frac{\nabla_{\theta_d} D(X^i)}{D(X^i)} - \frac{\nabla_{\theta_d}D(G(Z^i))}{1 - D(G(Z^i))})
\end{equation}

(ii)
According to the sigmoid derivation feature:

\begin{equation}
g^{'}(x) = g(x)(1-g(x))
\end{equation}

And combine with (i) function, the result(A, B are single simbols):

\begin{equation}
\frac{\partial L(\theta_d, \theta_g)}{\partial z_d^{L_d}} = \frac{1}{n} \sum_{i=1}^{n} (\frac{A^{'}}{A} + \frac{-B^{'}}{1-B})
\end{equation}

According to sigmoid, then:
\begin{equation}
\frac{\partial L(\theta_d, \theta_g)}{\partial z_d^{L_d}} = \frac{1}{n} \sum_{i=1}^{n} (1 - A - B)
\end{equation}

Final combine with layer l activation values:

\begin{equation}
\frac{\partial L(\theta_d, \theta_g)}{\partial z_d^{L_d}} = \frac{1}{n} \sum_{i=1}^{n} (1 - 2A_d^{L_d})
\end{equation}

(iii)

\begin{equation}
\frac{\partial L(\theta_d, \theta_g)}{\partial z_d^{l}} =  W_d^{l+1} \cdot  \frac{\partial L(\theta_d, \theta_g)}{\partial z_d^{l+1}} \cdot g_d^{'}(z_d^{l})
\end{equation}

(iv)

assume it is supplied post activation, g0 derivative is 1.
Then,
\begin{equation}
\frac{\partial L(\theta_d, \theta_g)}{\partial g(z_i; \theta_g)} =  W_d^{1} \cdot  \frac{\partial L(\theta_d, \theta_g)}{\partial z_d^{1}}
\end{equation}

(v)
\begin{equation}
\nabla_{\theta_{g}} L(\theta_d, \theta_g) = \frac{\partial L(\theta_d, \theta_g)}{\partial g(z_i; \theta_g)} \cdot \nabla_{theta_g}g(z_i; \theta_g)
\end{equation}

(vi)
Repeat{
 Calculate the thetaD on t iteration and update with function:
\begin{equation}
\theta_d^{t+1} := \theta_d^{t} - \alpha \cdot \nabla_{theta_d}L(\theta_d, \theta_g)
\end{equation}
}

(vii)
Repeat{
  Calculate the thetaG on t iteration and update with function:
\begin{equation}
\theta_g^{t+1} := \theta_g^{t} - \alpha \cdot \nabla_{theta_g}L(\theta_d, \theta_g)
\end{equation}
}

(viii)

\begin{equation}
\nabla_{\theta_{d}} L^{'}(\theta_d, \theta_g) = \frac{1}{n} \sum_{i=1}^{n} (\nabla_{\theta_d} D(X^i) + \nabla_{\theta_d} D(G(Z^i)) )
\end{equation}

## Question 6

```python
import numpy as np

h = 100
w = 80
input_channels = 3
output_channels = 5

key_kernel = (3,3,3)
q_kernel=(1,1,3)
value_kernel = (3,3,1)

key_conv = Conv2d(input_channels, output_channels, kernel_size=(3,3))
query_conv = Conv2d(input_channels, output_channels, kernel_size=1)
value_conv = Conv2d(input_channels, output_channels, kernel_size=(3,3))

def norm(X):
    norm = np.linalg.norm(an_array)
    return an_array/norm

def forward(X):
    key_out = key_conv(X)
    query_out = query_conv(X[1:-1][1:-1][:])
    value_out = value_conv(X)
    softmax_input = numpy.zeros(h, w, output_channels)
    hk, wk, ck = key_out.shape
    for hi in range(hk):
        for wi in range(wk):
            for ci in range (ck):
                softmax_input[hi, wi, ci] = np.multiply(key_out[hi,wi,ci], query_out[hi,hi,ci])
    softmax_output = softmax(softmax_input)
    out = numpy.zeros(ck)
    return np.einsum(out, softmax_out)

norm_X = norm(X)
output = forward(X)
```

```python

```


# Data to public database

# Purpose
In order to make this research reproducible the data used will be stored at [Mendeley Data](https://data.mendeley.com/):
* MDL model test roll decay with KVLCC2
* FNPF roll decay with KVLCC2
* Ikeda section results (From manual digitalization)

# Methodology
* Load
* Cut
* Fit
* save

# Setup

```python
# %load imports.py
"""
These is the standard setup for the notebooks.
"""

%matplotlib inline
%load_ext autoreload
%autoreload 2

from jupyterthemes import jtplot
jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)

import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
import os
import matplotlib.pyplot as plt
from collections import OrderedDict
#plt.style.use('paper')
from reports.paper_writing import save_fig

#import data
import copy
from mdldb.run import Run

from sklearn.pipeline import Pipeline
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from rolldecayestimators.direct_estimator_cubic import EstimatorQuadraticB, EstimatorCubic
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
import src.equations as equations
import rolldecayestimators.lambdas as lambdas
from rolldecayestimators.substitute_dynamic_symbols import lambdify
import rolldecayestimators.symbols as symbols
import sympy as sp

from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex

from sklearn.metrics import r2_score
from src.data import database
from mdldb import tables
import shipflowmotionshelpers.shipflowmotionshelpers as helpers
import src.visualization.visualize as visualize

```

    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 462 ('figure.figsize   : 5, 3   ## figure size in inches')
    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 463 ('figure.dpi       : 100        ## figure dots per inch')

```python
from copy import deepcopy
import joblib
from rolldecayestimators import measure
from src.helpers import get_ikeda, calculate_ikeda, get_estimator_variation, get_data_variation , get_variation, hatify
import scipy
from sklearn.pipeline import Pipeline
#from reports import mdl_results
```

## Load meta data from MDL

```python
def load():
    db = database.get_db()

    sql = """
    SELECT * from run
    INNER JOIN loading_conditions
    ON (run.loading_condition_id = loading_conditions.id)
    INNER JOIN models
    ON (run.model_number = models.model_number)
    INNER JOIN ships
    ON (run.ship_name = ships.name)
    WHERE run.model_number='M5057-01-A' and run.test_type='roll decay' and run.project_number=40178362;
    """
    df_rolldecays = pd.read_sql(sql=sql, con=db.engine)
    df_rolldecays['rho']=1000
    df_rolldecays['g']=9.81
    df_rolldecays=df_rolldecays.loc[:,~df_rolldecays.columns.duplicated()]
    df_rolldecays.set_index('id', inplace=True)

    df_rolldecays['ship_speed'].fillna(0, inplace=True)

    df_rolldecays=df_rolldecays.loc[[21337,21338,21340,]].copy()
    df_rolldecays['paper_name'] = np.arange(len(df_rolldecays)) + 1

    return df_rolldecays

df_rolldecays = load()
```

## Load data from Motions:

```python
file_paths = [
    '../data/external/kvlcc2_rolldecay_0kn',
    '../data/external/kvlcc2_rolldecay_15-5kn_const_large',
    '../data/external/kvlcc2_rolldecay_15-5kn_const_large2',
    '../data/external/kvlcc2_rolldecay_15-5kn_const_large_5deg',
    '../data/external/kvlcc2_rolldecay_15-5kn_const_large_ikeda',
    '../data/external/kvlcc2_rolldecay_15-5kn_ikeda_dev',
]
df_parameters = pd.DataFrame()
df_parameters =  helpers.load_parameters(file_path=file_paths)

mask_0 = (df_parameters['vm_s'].round(5) == 0)
df_parameters.loc[mask_0,'id'] = 21338  # MDL DB run id
df_parameters.loc[~mask_0,'id'] = 21340
```

```python
mask_visc = (df_parameters[['b4l','b4q']] > 0).any(axis=1)
df_parameters.loc[mask_visc]
```

      B
      IXY
      IXZ
      IYY
      IYZ
      IZZ
      KXX
      KXY
      KYY
      KZZ
      LPP
      S
      V
      XCG
      YCG
      ZCG
      accx
      acto
      b1c
      b1cr
      b1l
      b1q
      b2c
      b2cr
      b2l
      b2q
      b3c
      b3cr
      b3l
      b3q
      b4c
      b4cr
      b4l
      b4q
      b5c
      b5cr
      b5l
      b5q
      b6c
      b6cr
      b6l
      b6q
      bdens
      body
      clev
      clevel
      curv
      dens
      densi
      dofa
      dofactor
      dopa
      dopadding
      dopo
      dopower
      down
      downs
      downst
      dt
      file
      file_path_ts
      fn
      form
      free
      gravi
      heave
      heel
      hull
      inter
      k1nd
      k2nd
      k3nd
      k4nd
      k5nd
      k6nd
      kxx
      kyy
      leve
      level
      lpp
      m
      maxti
      maxtime
      mesh
      name
      npxp
      npyp
      nstep
      nthr
      pitch
      powe
      power
      refle
      reflen
      rn
      roll
      rud1
      rud2
      rud3
      rud4
      rud5
      rud6
      rudt
      side
      slim
      stre
      strength
      surge
      sway
      ta
      tf
      tfnhi
      tfnhigh
      tfnlo
      tfnlow
      titl
      title
      trim
      upst
      upstr
      vm_s
      wlin
      wlme
      yaw
      ymax
      ymin
      zcg
      conv
      encounters
      id

      kvlcc2_rolldecay_15-5kn_const_large_ikeda
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.025
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      6.07217
      2.74371
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      10.0
      10.0
      2.0
      2.0
      2.0
      2.0
      1.00
      -5.0
      1.00
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      0.142341
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.000000e+00
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      200.0
      200.0
      0.000000e+00
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3826600.0
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.00
      30.0
      0.1
      0.1
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      1.00
      5.0
      0.966976
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21340.0

      kvlcc2_rolldecay_15-5kn_ikeda_dev
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.050
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      6.07217
      2.74371
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00004
      1000.0
      1000.0
      0.0
      0.0
      2.0
      2.0
      1.0
      1.0
      0.25
      -2.0
      0.25
      NaN
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      0.142341
      0.20
      0.3
      9.80665
      0.0
      0.0
      0.002
      1.000000e-07
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      600.0
      600.0
      1.000000e-07
      TRAN
      24.0
      24.0
      30.0
      6.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3826600.0
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.25
      30.0
      1.0
      1.0
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      0.25
      2.0
      0.966976
      0.000001
      0.000001
      0.0
      2.0
      -2.0
      0.2735
      0.0001
      0.0
      21340.0

```python
df_parameters.loc[~mask_visc]
```

      B
      IXY
      IXZ
      IYY
      IYZ
      IZZ
      KXX
      KXY
      KYY
      KZZ
      LPP
      S
      V
      XCG
      YCG
      ZCG
      accx
      acto
      b1c
      b1cr
      b1l
      b1q
      b2c
      b2cr
      b2l
      b2q
      b3c
      b3cr
      b3l
      b3q
      b4c
      b4cr
      b4l
      b4q
      b5c
      b5cr
      b5l
      b5q
      b6c
      b6cr
      b6l
      b6q
      bdens
      body
      clev
      clevel
      curv
      dens
      densi
      dofa
      dofactor
      dopa
      dopadding
      dopo
      dopower
      down
      downs
      downst
      dt
      file
      file_path_ts
      fn
      form
      free
      gravi
      heave
      heel
      hull
      inter
      k1nd
      k2nd
      k3nd
      k4nd
      k5nd
      k6nd
      kxx
      kyy
      leve
      level
      lpp
      m
      maxti
      maxtime
      mesh
      name
      npxp
      npyp
      nstep
      nthr
      pitch
      powe
      power
      refle
      reflen
      rn
      roll
      rud1
      rud2
      rud3
      rud4
      rud5
      rud6
      rudt
      side
      slim
      stre
      strength
      surge
      sway
      ta
      tf
      tfnhi
      tfnhigh
      tfnlo
      tfnlow
      titl
      title
      trim
      upst
      upstr
      vm_s
      wlin
      wlme
      yaw
      ymax
      ymin
      zcg
      conv
      encounters
      id

      kvlcc2_rolldecay_0kn
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.050
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      50.0
      50.0
      2.0
      2.0
      2.0
      2.0
      1.0
      -5.0
      1.0
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.472020e-07
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.0
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      180.0
      180.0
      0.0
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.957280e+00
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      30.0
      0.50
      0.50
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      1.0
      5.0
      0.000001
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21338.0

      kvlcc2_rolldecay_15-5kn_const_large
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.025
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      10.0
      10.0
      2.0
      2.0
      2.0
      2.0
      1.0
      -5.0
      1.0
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.423410e-01
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.0
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      200.0
      200.0
      0.0
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.826600e+06
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      30.0
      0.10
      0.10
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      1.0
      5.0
      0.966976
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21340.0

      kvlcc2_rolldecay_15-5kn_const_large2
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.025
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      1.0
      1.0
      2.0
      2.0
      1.0
      1.0
      0.7
      -5.0
      0.7
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.423410e-01
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.0
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      200.0
      200.0
      0.0
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.826600e+06
      10.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.7
      30.0
      0.05
      0.05
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      0.7
      5.0
      0.966976
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21340.0

      kvlcc2_rolldecay_15-5kn_const_large_5deg
      0.853
      0.0
      0.0
      26.055
      0.0
      26.055
      0.341
      7.556
      1.177
      1.177
      4.706
      5.981
      0.993
      2.519
      0.0
      0.274
      0.025
      1.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      0.6
      0.3
      6.0
      6.0
      0.00002
      1000.0
      1000.0
      10.0
      10.0
      2.0
      2.0
      2.0
      2.0
      1.0
      -5.0
      1.0
      0.02
      ..
      C:\Dev\Prediction-of-roll-damping-using-fully-...
      1.423410e-01
      0.23
      0.3
      9.80665
      0.0
      0.0
      0.002
      0.0
      0.1
      0.1
      0.0
      0.0
      0.0
      0.1
      0.341185
      1.1765
      6.0
      6.0
      4.706
      993.42
      200.0
      200.0
      0.0
      TRAN
      40.0
      40.0
      30.0
      32.0
      0.0
      2.0
      2.0
      4.706
      4.706
      3.826600e+06
      5.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      30.0
      0.10
      0.10
      0.0
      0.0
      0.3059
      0.3059
      4.0
      4.0
      0.5
      0.5
      KVLCC2
      KVLCC2
      0.0
      1.0
      5.0
      0.966976
      0.000001
      0.000001
      0.0
      5.0
      -5.0
      0.2735
      NaN
      NaN
      21340.0

## Load time series from Motions

```python
def find_t_max(X, phi_max):

    Xs = X.copy()
    Xs.index = pd.TimedeltaIndex(X.index, unit='s')
    X_ = Xs.abs().resample('5S').max()
    X_.index = X_.index.total_seconds()
    mask = X_['phi'] < phi_max

    X_small = X_.loc[mask]
    if len(X_small) > 0:
        t_max = X_small.iloc[0].name
    else:
        t_max = X.index[-1]

    return t_max
```

```python
time_series = helpers.load_time_series(df_parameters=df_parameters)
time_series_raw = deepcopy(time_series)

phi_max = np.deg2rad(3.5)

for key,df in time_series.items():

    if df.mean().abs()['V1'] > 0.01:

        phi1d_limit = 10**-2
        index0 = (df['phi1d'].abs() > phi1d_limit).argmax()
        X = df.iloc[index0:].copy()
        t_max = find_t_max(X=X, phi_max=phi_max)
        mask = X.index <= t_max
        X = X.loc[mask]

        time_series[key] = X
```

## Save raw Motions datasets

```python
interesting = ['phi','phi1d','phi2d','V1']

save_runs = [
'kvlcc2_rolldecay_0kn',
'kvlcc2_rolldecay_15-5kn_const_large2',
'kvlcc2_rolldecay_15-5kn_ikeda_dev',
]

for key in save_runs:

    df_save = time_series[key][interesting].copy()
    save_file_name = 'fnpf_%s.csv' % key
    save_path = os.path.join('../data/processed/roll decay KVLCC2', save_file_name)
    df_save.to_csv(save_path)

df_parameters_save = df_parameters.loc[save_runs].copy()
save_path = os.path.join('../data/processed/roll decay KVLCC2', 'fnpf_parameters.csv')
df_parameters_save.to_csv(save_path)
```

```python
db = database.get_db()

fig,axes=plt.subplots(ncols=len(df_rolldecays))
for (id, row),ax in zip(df_rolldecays.iterrows(),axes):

    run = db.session.query(Run).get(int(id))
    run = database.load_run(run, save_as_example=False, prefer_hdf5=True)

    run.track_plot(ax)

plt.tight_layout()
```

## Save raw MDL datasets

```python
def save_run(run, save_dir='../data/processed/roll decay KVLCC2'):

    renamers = {
        'modelpos/xm' : 'x',
        'modelpos/ym' : 'y',
        'modelpos/zm' : 'z',
        'modelpos/yawm' : 'psi',
    }

    renamers_units = {
        'ModelPos/Xm' : 'x',
        'ModelPos/Ym' : 'y',
        'ModelPos/Zm' : 'z',
        'ModelPos/Yawm' : 'psi',
    }

    df = run.df.copy()
    df.rename(columns = renamers, inplace=True)
    units = dict(run.units)

    for old,new in renamers_units.items():
        units[new] = units.pop(old)
    units = pd.Series(units)

    interesting = ['x','y','z','phi','theta','psi']
    df_save = df[interesting].copy()
    units_save = units[interesting].copy()

    save_file_name = 'model_test_%s.csv' % run.id
    save_path = os.path.join(save_dir, save_file_name)
    df_save.to_csv(save_path)

    save_path_units = os.path.join(save_dir, 'model_test_units.csv')
    units_save.to_csv(save_path_units)
```

```python
for (id, row),ax in zip(df_rolldecays.iterrows(),axes):

    run = db.session.query(Run).get(int(id))
    run = database.load_run(run, save_as_example=False, prefer_hdf5=True)
    save_run(run)
```

    c:\dev\evaluation\signal_lab\mdl_to_evaluation.py:106: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
      df_.units = units


```python
%matplotlib inline
```

PyTorch 소개
***********************

Torch의 텐서(Tensor) 라이브러리 소개
======================================

모든 딥러닝은 2차원 이상으로 색인될 수 있는 행렬의 일반화인
텐서의 연산입니다. 이것이 무엇을 의미하지 나중에 정확히
알게 될 것입니다. 먼저, 텐서로 무엇을 할 수 있는지 살펴 봅시다.

```python
# Author: Robert Guthrie

import torch
import torch.autograd as autograd
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)
```

텐서 생성하기
~~~~~~~~~~~~~~~~

텐서는 파이썬 리스트에서 torch.Tensor() 함수로 생성될 수 있습니다.

```python
# torch.tensor(data) 는 주어진 데이터로 torch.Tensor 객체를 생성합니다.
V_data = [1., 2., 3.]
V = torch.tensor(V_data)
print(V)

# 행렬 생성
M_data = [[1., 2., 3.], [4., 5., 6]]
M = torch.tensor(M_data)
print(M)

# 2x2x2 크기의 3D 텐서 생성.
T_data = [[[1., 2.], [3., 4.]],
          [[5., 6.], [7., 8.]]]
T = torch.tensor(T_data)
print(T)
```

어쨌든 3D 텐서가 무엇입니까? 이렇게 생각해 보십시오. 만약 벡터가 있다면
벡터에 주소를 입력하면 스칼라를 줍니다. 만약 행렬을 가지고 있다면 행렬에
주소를 입력하면 벡터를 줍니다. 만약 3D 텐서를 가지고 있다면 텐서에 주소를
입력하면 행렬을 줍니다.

용어에 대한 주석:
이 튜토리얼에서 "텐서"를 언급 할 때 그것은 어떤 torch.Tensor 객체를 말합니다.
행렬과 벡터는 각각 차원이 1과 2인 torch.Tensors 의 특별한 케이스 입니다.
3D 텐서를 말할 때는 "3D 텐서"라고 명시적으로 사용하겠습니다.

```python
# Index into V and get a scalar (0 dimensional tensor)
print(V[0])
# Get a Python number from it
print(V[0].item())

# Index into M and get a vector
print(M[0])

# Index into T and get a matrix
print(T[0])
```

다른 데이터 유형의 텐서를 생성 할 수도 있습니다. 보시다시피 기본값은
Float입니다. 정수형의 텐서를 만들려면 torch.LongTensor ()를 사용하십시오.
더 많은 데이터 유형에 대해서는 설명서를 확인하십시오.
그러나 Float 및 Long이 가장 일반적입니다.

torch.randn ()을 사용하여 랜덤 데이터와 제공된 차원으로 텐서를
만들 수 있습니다.

```python
x = torch.randn((3, 4, 5))
print(x)
```

텐서로 작업하기
~~~~~~~~~~~~~~~~~~~~~~~

기대하는 방식으로 텐서로 작업할 수 있습니다.

```python
x = torch.tensor([1., 2., 3.])
y = torch.tensor([4., 5., 6.])
z = x + y
print(z)
```

사용 가능한 방대한 작업의 전체 목록은
`문서 <http://pytorch.org/docs/torch.html>`__ 를 참고하십시오. 단순한
수학적 연산 이상으로 확장됩니다.

나중에 사용하게 될 유용한 작업 중 하나는 연결입니다.

```python
# 기본으로 첫번째 축(가로 연결)을 따라 연결합니다.
x_1 = torch.randn(2, 5)
y_1 = torch.randn(3, 5)
z_1 = torch.cat([x_1, y_1])
print(z_1)

# 세로 연결:
x_2 = torch.randn(2, 3)
y_2 = torch.randn(2, 5)
# 두번째 변수는 연결될 축을 결정합니다.
z_2 = torch.cat([x_2, y_2], 1)
print(z_2)

# 텐서가 호환되지 않으면 Torch가 오류 메시지를 출력 합니다. 주석 처리를 제거하여 오류를 확인하십시오.
# torch.cat([x_1, x_2])
```

텐서 재구성
~~~~~~~~~~~~~~~~~

.view() 메서드를 사용해서 텐서를 재구성합니다.
이 메서드는 많은 신경망 구성 요소가 입력을 특정 모양으로 예상하기
때문에 많이 사용됩니다. 데이터를 구성 요소로 전달하기 전에 종종 모양을
변경해야합니다.

```python
x = torch.randn(2, 3, 4)
print(x)
print(x.view(2, 12))  # 가로 2 , 세로 12로 재구성
# 위와 같이 하나의 차원이 -1이면 그 것의 크기는 유추될 수 있습니다.
print(x.view(2, -1))
```

연산 그래프(Computation Graph)와 자동 미분(Automatic Differentiation)
=======================================================================

연산 그래프의 개념은 역전파 그라디언트를 직접 작성할 필요가 없게 해주기 때문에
효율적인 딥러닝 프로그래밍에 필수적입니다.
연산 그래프는 데이터를 결합하여 출력을 제공하는 방법의 간단한 특징입니다.
그래프는 어떤 연산과 어떤 매개 변수가 연관되는지를 완전하게 특정하기 때문에
도함수(derivative)를 계산하기에 충분한 정보를 포함합니다.
아마 모호하게 들릴지도 모르니, 근본적인 플래그``requires_grad`` 를 사용하는데
어떤 일이 일어나는지 살펴봅시다.

먼저 프로그래머 관점에서 생각해 보세요.
torch 에 무엇이 저장되있나요.
위에서 생성한 텐서의 객체는 무엇입니까? 분명히 데이터와 형태, 그리고
아마 몇몇 다른 것들 입니다. 그러나 우리가 두개의 텐서를 더할 때
출력 텐서를 얻습니다. 이 모든 출력 텐서는 그것의 데이터와 형태를 알고 있습니다.
하지만 그것이 두 텐서의 합이 었는지는 알지 못합니다.(파일에서 읽었을 수도
있고 다른 연산의 결과일 수도 있음)

만일 ``requires_grad=True`` 라면, 텐서 객체는 것이 어떻게 생성되었는지 추적합니다.
실제로 그것을 확인해 봅시다.

```python
# 텐서 공장 메소드에 ``requires_grad`` 플래그가 있습니다.
x = torch.tensor([1., 2., 3], requires_grad=True)

# requires_grad=True 으로 이전에 있었던 모든 작업을 여전히 할 수 있습니다.
y = torch.tensor([4., 5., 6], requires_grad=True)
z = x + y
print(z)

# 그러나 z 는 몇가지를 추가로 알고 있습니다.
print(z.grad_fn)
```

그래서 텐서는 무엇이 그들을 생성했는지를 알고 있습니다.
z 는 그것이 파일에서 읽어온 것이 아니고 곱셈 지수승 또는 다른 어떤 것의
결과가 아니라는 것을 알고 있습니다. 그리고 만약 z.grad_fn 를 따라 간다면
x 와 y 를 찾을 것 입니다.

그러나 그것이 기울기(gradient)를 계산하는데 어떻게 도움이 될까요?

```python
# z 의 모든 요소를 합해 봅시다.
s = z.sum()
print(s)
print(s.grad_fn)
```

이제 x의 첫번째 구성요소에 해당하는 이 합계의 도함수가 무엇입니까?
우리가 원하는 수식은 다음과 같습니다.

\begin{align}\frac{\partial s}{\partial x_0}\end{align}

그럼 s 는 텐서 z 의 합으로 생성되었다는 것을 알고 있습니다.
z 는 x + y 합이 었다는 것을 알고 있습니다. 따라서

\begin{align}s = \overbrace{x_0 + y_0}^\text{$z_0$} + \overbrace{x_1 + y_1}^\text{$z_1$} + \overbrace{x_2 + y_2}^\text{$z_2$}\end{align}

그리고 따라서 s 는 우리가 원하는 도함수가 1인 것을 결정하는데 충분한 정보를 가지고 있습니다.

물론 이것은 그 도함수를 실제로 계산법에 대한 도전에 대해 해설하고 있습니다.
여기서 핵심은 s 가 그것을 계산 할 수 있는 충분한 정보를 가지고 있다는 것 입니다.
실제로 Pytorch 개발자는 기울기를 계산하는 방법을 알고 sum() 과 + 연산을 프로그램하고
역전파 알고리즘을 실행합니다. 이 알고리즘에 대한 자세한 설명은 이 튜토리얼의
범위를 벗어납니다.

Pytorch가 기울기를 계산하게 하고, 우리가 옳았다는 것을 확인하십시오
( 만약 이 블럭을 여러번 실행한다면 기울기가 증가할 것입니다.
그것은 Pytorch 가 .grad 속성에 기울기를 누적하기 때문이고,
이것은 많은 모델에서 매우 편리합니다.

```python
# 어떤 변수에서 .backward() 호출은 그것에서 시작하는 역전파를 실행합니다.
s.backward()
print(x.grad)
```

아래 블록에서 진행되는 작업을 이해하는 것은 딥러닝의 성공적인
프로그래머가 되는데 중요합니다.

```python
x = torch.randn(2, 2)
y = torch.randn(2, 2)
# 기본적으로 사용자가 생성한 텐서는 ``requires_grad=False`` 로 설정됩니다.
print(x.requires_grad, y.requires_grad)
z = x + y
# 따라서 z 를 통하는 역전파를 계산할 수 없습니다.
print(z.grad_fn)

# ``.requires_grad_( ... )`` 는 기존 텐서의 ``requires_grad`` 플래그를 변경합니다.
# 입력 플래그는 주어지지 않을 경우 기본적으로 ``True`` 설정됩니다.
x = x.requires_grad_()
y = y.requires_grad_()
# 위에서 본대로 z 는 기울기를 계산하는데 충분한 정보를 가지고 있습니다.
z = x + y
print(z.grad_fn)
# 만일 연산의 어떤 입력이 ``requires_grad=True`` 를 가지면, 그 출력도 동일합니다.
print(z.requires_grad)

# 이제 z는 x와 y와 관련이 있는 연산 이력을 가집니다.
# 그것의 값만 취해서 그것의 이력에서 **분리** 할 수 있을 까요?
new_z = z.detach()

# ... new_z는 x와 y에 역전파하기 위한 정보를 가지고 있습니까?
# 아닙니다!
print(new_z.grad_fn)
# 어떻게 그럴 수 있을까요?  ``z.detach()`` 는 ``z`` 와 같은 저장 공간을 공유하지만
# 계산 이력을 잊어버린 텐서를 반환합니다. 그것은 그것이 어떻게 계산되었는지
# 아무 것도 모릅니다.
# 기본적으로 우리는 과거의 이력에서 Tensor를 분리했습니다.
```

requires_grad=True 를 가진 텐서를 ``with torch.no_grad():`` 으로
둘러싸서 이력 추적의 자동미분(autograd)을 멈출 수 있습니다.

```python
print(x.requires_grad)
print((x ** 2).requires_grad)

with torch.no_grad():
	print((x ** 2).requires_grad)
```


```python
from IPython.display import HTML

HTML('''

<form action="javascript:code_toggle()"><input type="submit" id="toggleButton" value="Show Code"></form>
<ul>
<li>The code is in Python. Matlab does not work with the interactive elements used in this notebook.
<li>The current version of ipywidgets shows only static images online.  Interactivity can only be viewed by downloading and running in a Jupyter Notebook server.  The next version of ipywidgets will permit online viewing of interactivity.
</ul>
''')
```

<form action="javascript:code_toggle()"><input type="submit" id="toggleButton" value="Show Code"></form>

The code is in Python. Matlab does not work with the interactive elements used in this notebook.

The current version of ipywidgets shows only static images online.  Interactivity can only be viewed by downloading and running in a Jupyter Notebook server.  The next version of ipywidgets will permit online viewing of interactivity.

# Interior Point Methods

Let's do a simple 1D problem, just for illustration:

\begin{align}
\text{minimize} &\quad f(x) = -x\\
\text{subject to} &\quad x \le 5
\end{align}

Clearly, the answer is $x^* = 5$, but let's see how it looks like with a (simplified) interior point approach.  First, let's write our constraint in a standard constraint form $c(x) \le 0$ and form a new objective.

$$c(x) = x - 5 \le 0$$

Thus,

\begin{align}
\pi(x) &= f(x) - \mu \log(-c(x)) \\
&= -x - \mu \log(5 - x)
\end{align}

First, let's just plot the penalty portion $-\mu\log(-c(x))$, starting with the case $\mu = 1$.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('fivethirtyeight')

x = np.linspace(0, 4.9999, 1000)

plt.figure()
plt.plot(x, -np.log(5 - x))
plt.xlim([0, 5.2])
plt.show()
```

We can see that it actus like a steep wall near $x=5$ with small effect elsewhere.  The next example is interactive, letting you set a $\mu$.  Notice that as $\mu$ becomes smaller and smaller the penalty more closely represents a perfect barrier.

```python
from ipywidgets import interact

@interact(mu=(0.1, 2.0, 0.1))
def logbarrier(mu):
    f = -mu*np.log(5 - x)
    plt.plot(x, f)
    plt.xlim([0, 5.1])
    plt.ylim([-2, 2])
    plt.xlabel('x')
    plt.ylabel('f')
    plt.show()
```

Let's now plot this objective plus the penalty, or in other words our new function to minimize.  We start with $\mu = 1.0$.  Also shown for reference is the line $f(x) = -x$ which is the objective function.  The constraint $x \le 5$ is like a wall at $x = 5$.  We see that the log function creates a barrier that approximates the wall.  As we decrease the valye of $\mu$, the barrier will increase in steepness and closer approximate the wall.

```python
plt.figure()
plt.plot(x, -x, 'k--')
mu = 1.0
f = -x - mu*np.log(5 - x)
plt.plot(x, f)
plt.xlim([0, 5.1])
plt.ylim([-5, 2])
plt.xlabel('x')
plt.ylabel('f')
plt.show()
```

We can add a slider to try a range of values of $\mu$.

```python
@interact(mu=(0.1, 2.0, 0.1))
def barrier(mu):
    f = -x - mu*np.log(5 - x)
    plt.plot(x, -x, 'k--')
    plt.plot(x, f)
    plt.xlim([0, 5.1])
    plt.ylim([-5, 2])
    plt.xlabel('x')
    plt.ylabel('f')
    idx = np.argmin(f)
    plt.plot(x[idx], f[idx], 'r*', ms=14)
    plt.show()
```

For those of you viewing this statically, rather than interactively, here are a few snapshots at $\mu = 1.0, 0.5, 0.25, \text{ and } 0.125$

```python
plt.figure()
plt.plot(x, -x, 'k--')

muvec = [1.0, 0.5, 0.25, 0.125]

for mu in muvec:

    f = -x - mu*np.log(5 - x)
    plt.plot(x, f)

plt.xlim([0, 5.1])
plt.ylim([-5, 2])
plt.xlabel('x')
plt.ylabel('f')
plt.show()
```

```python

```