# AIA Temperature Response Functions
In this notebook, we'll use the code already on my PR branch to first calculate the wavelength response functions. Then, we'll try to use CHIANTI and ChiantiPy to calculate the temperature response functions for a few ions.

We need to come up with a way to easily constrain how many wavelengths (or more precisely which ions) we need for each channel. We can certainly just search through every ion but this takes time!

```python
import json

import numpy as np
import h5py
import seaborn as sns
from scipy.interpolate import splev,splrep
import matplotlib.pyplot as plt
import astropy.units as u
from sunpy.instr import aia
import ChiantiPy.core as ch
import ChiantiPy.tools.data as ch_data

%matplotlib inline
```

## Wavelength Response

```python
response = aia.Response(ssw_path='/Users/willbarnes/Documents/Rice/Research/ssw/',
                        #channel_list=[131,171,193,211,304]
                       )
```

```python
response.calculate_wavelength_response(include_crosstalk=True)
```

```python
response.peek_wavelength_response()
```

```python
data = np.loadtxt('../aia_sample_data/aia_wresponse_raw.dat')
channels = sorted(list(response.wavelength_response.keys()))
ssw_results = {}
for i in range(len(channels)):
    ssw_results[channels[i]] = {'wavelength':data[:,0],
                                'response':data[:,i+1]}
```

```python
fig,axes = plt.subplots(3,3,figsize=(12,12))
for c,ax in zip(channels,axes.flatten()):
    #ssw
    ax.plot(ssw_results[c]['wavelength'],ssw_results[c]['response'],
            #color=response.channel_colors[c],
            label='ssw')
    #sunpy
    ax.plot(response.wavelength_response[c]['wavelength'],response.wavelength_response[c]['response'],
            #color=response.channel_colors[c],
            #marker='.',ms=6,markevery=5,
            label='SunPy',linestyle=':',alpha=0.95,lw=2)
    if c!=335 and c!=304:
        ax.set_xlim([c-20,c+20])
    #if c==335:
    #    ax.set_xlim([120,140])
    #if c==304:
    #    ax.set_xlim([80,100])
    ax.set_title('{} $\mathrm{{\mathring{{A}}}}$'.format(c),fontsize=20)
    ax.set_xlabel(r'$\lambda$ ({0:latex})'.format(response.wavelength_response[c]['wavelength'].unit),fontsize=20)
    ax.set_ylabel(r'$R_i(\lambda)$ ({0:latex})'.format(response.wavelength_response[c]['response'].unit),fontsize=20)
# contamination plots
#304
#ssw
ax = axes.flatten()[-2]
ax.plot(ssw_results[304]['wavelength'],ssw_results[304]['response'],
        #color=response.channel_colors[c],
        label='ssw')
#sunpy
ax.plot(response.wavelength_response[304]['wavelength'],response.wavelength_response[304]['response'],
        label='SunPy',linestyle='',alpha=0.95,lw=2,
        #color=response.channel_colors[c],
        marker='.',ms=8,markevery=2,
       )
ax.set_xlim([80,100])
ax.set_title('{} $\mathrm{{\mathring{{A}}}}$ contamination from 94'.format(304),fontsize=14)
ax.set_xlabel(r'$\lambda$ ({0:latex})'.format(response.wavelength_response[c]['wavelength'].unit),fontsize=20)
ax.set_ylabel(r'$R_i(\lambda)$ ({0:latex})'.format(response.wavelength_response[c]['response'].unit),fontsize=20)
#335
ax = axes.flatten()[-1]
ax.plot(ssw_results[335]['wavelength'],ssw_results[335]['response'],
        #color=response.channel_colors[c],
        label='ssw')
#sunpy
ax.plot(response.wavelength_response[335]['wavelength'],response.wavelength_response[335]['response'],
        label='SunPy',linestyle='',alpha=0.95,lw=2,
        #color=response.channel_colors[c],
        marker='.',ms=8,markevery=2,
       )
ax.set_xlim([120,140])
ax.set_title('{} $\mathrm{{\mathring{{A}}}}$ contamination from 131'.format(335),fontsize=14)
ax.set_xlabel(r'$\lambda$ ({0:latex})'.format(response.wavelength_response[c]['wavelength'].unit),fontsize=20)
ax.set_ylabel(r'$R_i(\lambda)$ ({0:latex})'.format(response.wavelength_response[c]['response'].unit),fontsize=20)
axes[0,0].legend(loc='best')
plt.tight_layout()
```

```python
fig,axes = plt.subplots(3,3,figsize=(12,12),sharey=True,sharex=True)
for c,ax in zip(channels,axes.flatten()):
    #ssw
    ax2 = ax.twinx()
    ssw_interp = ssw_results[c]['response']*response.wavelength_response[c]['response'].unit
    delta_response = np.fabs(response.wavelength_response[c]['response'] - ssw_interp)/(ssw_interp)
    ax.plot(response.wavelength_response[c]['wavelength'],delta_response,
            #color=response.channel_colors[c]
           )
    ax2.plot(response.wavelength_response[c]['wavelength'],response.wavelength_response[c]['response'],
            color='k',linestyle='--')
    ax.set_title('{} $\mathrm{{\mathring{{A}}}}$'.format(c),fontsize=20)
    ax.set_xlabel(r'$\lambda$ ({0:latex})'.format(response.wavelength_response[c]['wavelength'].unit),fontsize=20)
    ax.set_ylabel(r'$\frac{|\mathrm{SSW}-\mathrm{SunPy}|}{\mathrm{SSW}}$',fontsize=20)
    ax2.set_ylabel(r'$R_i(\lambda)$ ({0:latex})'.format(response.wavelength_response[c]['response'].unit))
    ax.set_ylim([-1.1,1.1])
plt.tight_layout()
```

```python
info_table = aia.response.aia_instr_properties_to_table([94,131,171,193,211,335],
                        ['/Users/willbarnes/Documents/Rice/Research/ssw/sdo/aia/response/aia_V6_all_fullinst.genx'])
```

## Temperature Response

Set a temperature and density range. This is the range of temperatures and densities over which the contribution function for each ion will be calculated. Note that this is **not** a grid of temperatures and densities, but rather a list of $T$ and $n$ pairs.

According to Boerner et al. (2012), we should use a constant pressure of $10^{15}$ cm$^{-3}$ K.

```python
temperature = np.logspace(5,8,50)*u.K
pressure = 1e15*u.K*u.cm**(-3)
density = pressure/temperature
```

The main question is: how exactly do we calculate the temperature response functions? According to Boerner et al. (2012), the response function $K$ for channel $i$ is given by,
$$
K_i(T) = \int_0^{\infty}\mathrm{d}\lambda\,G(\lambda,T)R_i(\lambda)
$$
where $K$ has units DN cm$^{-5}$ s$^{-1}$ pix$^{-1}$. So what is the right expression for $G(\lambda,n,T)$, the contribution function?

Let's divide the contribution function into a line emission part and a continuum part such that,
$$
G(\lambda,T) = G_{continuum}(\lambda,T) + \sum_XG_X(\lambda,T)
$$

where $X$ denotes an ion in the CHIANTI database. In this way, the response for each channel becomes,

$$
\begin{align}
K_i(T) &= \int_0^{\infty}\mathrm{d}\lambda\,\big(G_{continuum}(\lambda,T) + \sum_XG_X(\lambda,T)\big)R_i(\lambda) \\
       &= \int_0^{\infty}\mathrm{d}\lambda\,G_{continuum}(\lambda,T)R_i(\lambda) + \sum_X\int_0^{\infty}\mathrm{d}\lambda\,G_X(\lambda,T)R_i(\lambda)
\end{align}
$$

So there is a contribution from the continuum plus a contribution from each ion at every wavelength where there is a spectral line. In particular, the continuum includes three different contributions: free-free losses, free-bound losses, and two-photon losses,

$$
G_{continuum}(\lambda,T) = G_{ff}(\lambda,T) + G_{fb}(\lambda,T) + G_{tp}(\lambda,T)
$$

The expressions for these losses can be found in [Landi et al. (1999)](http://adsabs.harvard.edu/abs/1999A%26AS..135..339L). These are calculated and summed over each ion as well. Thus, the full expression for the temperature response is given by,

$$
K_i(T) = \int_0^{\infty}\mathrm{d}\lambda\,(G_{ff}(\lambda,T) + G_{fb}(\lambda,T)R_i(\lambda) + G_{tp}(\lambda,T)) + \sum_X\int_0^{\infty}\mathrm{d}\lambda\,G_X(\lambda,T)R_i(\lambda)
$$

Make a list of ions. Exactly what ions should be included is not really clear. Certainly, at least all ions of Fe. Really though, just look at all the ions in CHIANTI. Though, the user should have a chance to select which ions are included so that they can easily calculate the response functions, given only a few lines/ions.

```python
ion_list = (['fe_{}'.format(i) for i in np.arange(6,26)]
            + ['ca_{}'.format(i) for i in np.arange(10,20)])
```

Boerner et al. (2012) use the coronal abundances of Feldman and Widing (1993) and the ionization balances of Dere et al. (2009). We also want to make sure we are calculating all of the emissivities in units of photons rather than ergs as this makes it easier when multiplying by the instrument response function.

```python
ch_data.Defaults['flux'] = 'photon'
ch_data.Defaults['abundfile'] = 'sun_coronal_1992_feldman'
ch_data.Defaults['ioneqfile'] = 'chianti'
```

Now iterate through the ion list, calculating the emission and subsequently the contribution function at each stage and then interpolating the wavelength response function to the appropriate wavelengths.

Not that for the line emission, we are using the expression,

$$
G_X(\lambda,T) = \frac{1}{4\pi}\epsilon_X(\lambda,T)\mathrm{Ab}(X)\frac{N(X^{+m}}{N(X)}\frac{1}{n_e}
$$

which has units of photons cm$^{3}$ s$^{-1}$ sr$^{-1}$. The factor of $1/4\pi$ is included in the expression return by the `emiss()` method on the ChiantiPy `ion` object.

```python
temperature_responses = {k:np.zeros(len(temperature)) for k in response.wavelength_response}
```

```python
for ion in ch_data.MasterList:
    #if ion.split('_')[0] != 'fe':
    #    continue
    print('{}: Calculating contribution function for {}'.format(ch_data.MasterList.index(ion),ion))
    #declare ion object
    tmp = ch.ion(ion,temperature=temperature.value,eDensity=density.value,
                 abundance='sun_coronal_1992_feldman')
    #calculate emissivity
    tmp.emiss()
    em = tmp.Emiss['emiss'][np.argsort(tmp.Emiss['wvl']),:]
    wvl = np.sort(tmp.Emiss['wvl'])
    #calculate contribution function.
    gofnt = tmp.Abundance*em*tmp.IoneqOne/tmp.EDensity
    #iterate over channels
    for channel in response.wavelength_response:
        #print('Adding to channel {}'.format(channel))
        #interpolate response function to transitions
        rsp = splev(wvl,splrep(response.wavelength_response[channel]['wavelength'].value,
                               response.wavelength_response[channel]['response'].value))
        rsp = np.where(rsp<0,0,rsp)*response._channel_info[channel]['plate_scale'].value
        #weighted sum over wavelength
        #add to temperature response
        temperature_responses[channel] += np.dot(rsp,gofnt)
```

    0: Calculating contribution function for h_1
    1: Calculating contribution function for he_2
    2: Calculating contribution function for c_6
    3: Calculating contribution function for n_7
    4: Calculating contribution function for o_8
    5: Calculating contribution function for ne_10
    6: Calculating contribution function for na_11
    7: Calculating contribution function for mg_12
    8: Calculating contribution function for al_13
    9: Calculating contribution function for si_14
    10: Calculating contribution function for p_15
    11: Calculating contribution function for s_16
    12: Calculating contribution function for cl_17
    13: Calculating contribution function for ar_18
    14: Calculating contribution function for k_19
    15: Calculating contribution function for ca_20
    16: Calculating contribution function for fe_26
    17: Calculating contribution function for ni_28
    18: Calculating contribution function for he_1
    19: Calculating contribution function for c_5
    20: Calculating contribution function for n_6
    21: Calculating contribution function for o_7
    22: Calculating contribution function for ne_9
    23: Calculating contribution function for na_10
    24: Calculating contribution function for mg_11
    25: Calculating contribution function for al_12
    26: Calculating contribution function for si_13
    27: Calculating contribution function for p_14
    28: Calculating contribution function for s_15
    29: Calculating contribution function for cl_16
    30: Calculating contribution function for ar_17
    31: Calculating contribution function for k_18
    32: Calculating contribution function for ca_19
    33: Calculating contribution function for fe_25
    34: Calculating contribution function for ni_27
    35: Calculating contribution function for zn_29
    36: Calculating contribution function for c_5d
    37: Calculating contribution function for n_6d
    38: Calculating contribution function for o_7d
    39: Calculating contribution function for ne_9d
    40: Calculating contribution function for mg_11d
    41: Calculating contribution function for al_12d
    42: Calculating contribution function for si_13d
    43: Calculating contribution function for s_15d
    44: Calculating contribution function for ar_17d
    45: Calculating contribution function for ca_19d
    46: Calculating contribution function for fe_25d
    47: Calculating contribution function for ni_27d
    48: Calculating contribution function for c_4
    49: Calculating contribution function for n_5
    50: Calculating contribution function for o_6
    51: Calculating contribution function for ne_8
    52: Calculating contribution function for na_9
    53: Calculating contribution function for mg_10
    54: Calculating contribution function for al_11
    55: Calculating contribution function for si_12
    56: Calculating contribution function for p_13
    57: Calculating contribution function for s_14
    58: Calculating contribution function for ar_16
    59: Calculating contribution function for k_17
    60: Calculating contribution function for ca_18
    61: Calculating contribution function for ti_20
     Elvlc file missing for ti_21
    62: Calculating contribution function for cr_22
    63: Calculating contribution function for mn_23
     Elvlc file missing for mn_24
    64: Calculating contribution function for fe_24
    65: Calculating contribution function for co_25
     Elvlc file missing for co_26
    66: Calculating contribution function for ni_26
    67: Calculating contribution function for zn_28
    68: Calculating contribution function for c_4d
    69: Calculating contribution function for n_5d
    70: Calculating contribution function for o_6d
    71: Calculating contribution function for ne_8d
    72: Calculating contribution function for mg_10d
    73: Calculating contribution function for al_11d
    74: Calculating contribution function for si_12d
    75: Calculating contribution function for s_14d
    76: Calculating contribution function for ar_16d
    77: Calculating contribution function for ca_18d
    78: Calculating contribution function for fe_24d
    79: Calculating contribution function for ni_26d
    80: Calculating contribution function for c_3
    81: Calculating contribution function for n_4
    82: Calculating contribution function for o_5
    83: Calculating contribution function for ne_7
    84: Calculating contribution function for na_8
    85: Calculating contribution function for mg_9
    86: Calculating contribution function for al_10
    87: Calculating contribution function for si_11
    88: Calculating contribution function for p_12
    89: Calculating contribution function for s_13
    90: Calculating contribution function for cl_14
    91: Calculating contribution function for ar_15
    92: Calculating contribution function for k_16
    93: Calculating contribution function for ca_17
    94: Calculating contribution function for ti_19
    95: Calculating contribution function for cr_21
    96: Calculating contribution function for mn_22
    97: Calculating contribution function for fe_23
    98: Calculating contribution function for co_24
    99: Calculating contribution function for ni_25
    100: Calculating contribution function for zn_27
    101: Calculating contribution function for fe_23d
    102: Calculating contribution function for fe_22d
    103: Calculating contribution function for fe_21d
    104: Calculating contribution function for fe_20d
    105: Calculating contribution function for fe_19d
    106: Calculating contribution function for fe_18d
    107: Calculating contribution function for fe_17d
    108: Calculating contribution function for fe_16d
    109: Calculating contribution function for fe_15d
    110: Calculating contribution function for fe_14d
    111: Calculating contribution function for fe_13d
    112: Calculating contribution function for fe_12d
    113: Calculating contribution function for fe_11d
    114: Calculating contribution function for fe_10d
    115: Calculating contribution function for fe_9d
    116: Calculating contribution function for fe_8d
    117: Calculating contribution function for fe_7d
    118: Calculating contribution function for fe_6d
    119: Calculating contribution function for fe_5d
    120: Calculating contribution function for fe_4d
    121: Calculating contribution function for fe_3d
    122: Calculating contribution function for fe_2d
    123: Calculating contribution function for c_2
    124: Calculating contribution function for n_3
    125: Calculating contribution function for o_4
    126: Calculating contribution function for ne_6
    127: Calculating contribution function for na_7
    128: Calculating contribution function for mg_8
    129: Calculating contribution function for al_9
    130: Calculating contribution function for si_10
    131: Calculating contribution function for p_11
    132: Calculating contribution function for s_12
    133: Calculating contribution function for cl_13
    134: Calculating contribution function for ar_14
    135: Calculating contribution function for k_15
    136: Calculating contribution function for ca_16
    137: Calculating contribution function for ti_18
    138: Calculating contribution function for cr_20
    139: Calculating contribution function for mn_21
    140: Calculating contribution function for fe_22
    141: Calculating contribution function for co_23
    142: Calculating contribution function for ni_24
    143: Calculating contribution function for c_1
    144: Calculating contribution function for n_2
    145: Calculating contribution function for o_3
    146: Calculating contribution function for ne_5
    147: Calculating contribution function for na_6
    148: Calculating contribution function for mg_7
    149: Calculating contribution function for al_8
    150: Calculating contribution function for si_9
    151: Calculating contribution function for p_10
    152: Calculating contribution function for s_11
    153: Calculating contribution function for cl_12
    154: Calculating contribution function for ar_13
    155: Calculating contribution function for k_14
    156: Calculating contribution function for ca_15
    157: Calculating contribution function for ti_17
    158: Calculating contribution function for cr_19
    159: Calculating contribution function for mn_20
    160: Calculating contribution function for fe_21
    161: Calculating contribution function for co_22
    162: Calculating contribution function for ni_23
    163: Calculating contribution function for zn_25
     Elvlc file missing for zn_26
    164: Calculating contribution function for n_1
    165: Calculating contribution function for o_2
    166: Calculating contribution function for ne_4
    167: Calculating contribution function for na_5
    168: Calculating contribution function for mg_6
    169: Calculating contribution function for al_7
    170: Calculating contribution function for si_8
    171: Calculating contribution function for p_9
    172: Calculating contribution function for s_10
    173: Calculating contribution function for cl_11
    174: Calculating contribution function for ar_12
    175: Calculating contribution function for k_13
    176: Calculating contribution function for ca_14
    177: Calculating contribution function for ti_16
    178: Calculating contribution function for cr_18
    179: Calculating contribution function for mn_19
    180: Calculating contribution function for fe_20
    181: Calculating contribution function for co_21
    182: Calculating contribution function for ni_22
    183: Calculating contribution function for zn_24
    184: Calculating contribution function for o_1
    185: Calculating contribution function for ne_3
    186: Calculating contribution function for na_4
    187: Calculating contribution function for mg_5
    188: Calculating contribution function for al_6
    189: Calculating contribution function for si_7
    190: Calculating contribution function for p_8
    191: Calculating contribution function for s_9
    192: Calculating contribution function for cl_10
    193: Calculating contribution function for ar_11
    194: Calculating contribution function for k_12
    195: Calculating contribution function for ca_13
    196: Calculating contribution function for ti_15
    197: Calculating contribution function for cr_17
    198: Calculating contribution function for mn_18
    199: Calculating contribution function for fe_19
    200: Calculating contribution function for co_20
    201: Calculating contribution function for ni_21
    202: Calculating contribution function for zn_23
    203: Calculating contribution function for ne_2
    204: Calculating contribution function for na_3
    205: Calculating contribution function for mg_4
    206: Calculating contribution function for al_5
    207: Calculating contribution function for si_6
    208: Calculating contribution function for p_7
    209: Calculating contribution function for s_8
    210: Calculating contribution function for ar_10
    211: Calculating contribution function for k_11
    212: Calculating contribution function for ca_12
    213: Calculating contribution function for ti_14
    214: Calculating contribution function for cr_16
    215: Calculating contribution function for mn_17
    216: Calculating contribution function for fe_18
    217: Calculating contribution function for co_19
    218: Calculating contribution function for ni_20
    219: Calculating contribution function for na_2
    220: Calculating contribution function for mg_3
    221: Calculating contribution function for al_4
    222: Calculating contribution function for si_5
    223: Calculating contribution function for p_6
    224: Calculating contribution function for s_7
    225: Calculating contribution function for ar_9
    226: Calculating contribution function for k_10
    227: Calculating contribution function for ca_11
    228: Calculating contribution function for ti_13
    229: Calculating contribution function for cr_15
    230: Calculating contribution function for mn_16
    231: Calculating contribution function for fe_17
    232: Calculating contribution function for co_18
    233: Calculating contribution function for ni_19
    234: Calculating contribution function for zn_21
     Elvlc file missing for zn_22
    235: Calculating contribution function for mg_2
    236: Calculating contribution function for al_3
    237: Calculating contribution function for si_4
    238: Calculating contribution function for p_5
    239: Calculating contribution function for s_6
    240: Calculating contribution function for ar_8
    241: Calculating contribution function for k_9
    242: Calculating contribution function for ca_10
    243: Calculating contribution function for ti_12
    244: Calculating contribution function for cr_14
    245: Calculating contribution function for mn_15
    246: Calculating contribution function for fe_16
    247: Calculating contribution function for co_17
    248: Calculating contribution function for ni_18
    249: Calculating contribution function for zn_20
    250: Calculating contribution function for al_2
    251: Calculating contribution function for si_3
    252: Calculating contribution function for s_5
    253: Calculating contribution function for ar_7
    254: Calculating contribution function for ca_9
    255: Calculating contribution function for ti_11
    256: Calculating contribution function for cr_13
    257: Calculating contribution function for fe_15
    258: Calculating contribution function for ni_17
    259: Calculating contribution function for si_2
    260: Calculating contribution function for s_4
    261: Calculating contribution function for ca_8
    262: Calculating contribution function for fe_14
    263: Calculating contribution function for ni_16
    264: Calculating contribution function for s_3
    265: Calculating contribution function for cl_4
     Elvlc file missing for cl_5
    266: Calculating contribution function for ar_5
     Elvlc file missing for ar_6
    267: Calculating contribution function for k_6
     Elvlc file missing for k_7
    268: Calculating contribution function for ca_7
    269: Calculating contribution function for fe_13
    270: Calculating contribution function for ni_15
    271: Calculating contribution function for s_2
    272: Calculating contribution function for cl_3
    273: Calculating contribution function for ar_4
    274: Calculating contribution function for k_5
    275: Calculating contribution function for ca_6
    276: Calculating contribution function for fe_12
    277: Calculating contribution function for ni_14
    278: Calculating contribution function for s_1
    279: Calculating contribution function for cl_2
    280: Calculating contribution function for ar_3
    281: Calculating contribution function for ca_5
    282: Calculating contribution function for cr_9
     Elvlc file missing for cr_10
    283: Calculating contribution function for mn_10
     Elvlc file missing for mn_11
    284: Calculating contribution function for fe_11
    285: Calculating contribution function for ni_13
    286: Calculating contribution function for cr_8
    287: Calculating contribution function for mn_9
    288: Calculating contribution function for fe_10
    289: Calculating contribution function for ni_12
    290: Calculating contribution function for cr_7
    291: Calculating contribution function for mn_8
    292: Calculating contribution function for fe_9
    293: Calculating contribution function for ni_11
    294: Calculating contribution function for ca_2
     Elvlc file missing for ca_3
    295: Calculating contribution function for fe_8
    296: Calculating contribution function for fe_7
    297: Calculating contribution function for fe_6
    298: Calculating contribution function for fe_5
    299: Calculating contribution function for fe_4
    300: Calculating contribution function for fe_3
    301: Calculating contribution function for fe_2
    302: Calculating contribution function for ni_2
     Elvlc file missing for ni_3

```python
# ssw responses
precalculated_responses_data = np.loadtxt('../aia_sample_data/aia_tresponse_raw.dat')
precalc_channels = [94,131,171,193,211,304,335]
precalculated_responses = {c: precalculated_responses_data[:,i+1] for i,c in enumerate(precalc_channels)}
precalculated_responses['temperature'] = precalculated_responses_data[:,0]
# ssw responses with chiantifix and evenorm fix
precalculated_responses_data = np.loadtxt('../aia_sample_data/aia_tresponse_fix.dat')
precalculated_responses_fix = {c: precalculated_responses_data[:,i+1] for i,c in enumerate(precalc_channels)}
precalculated_responses_fix['temperature'] = precalculated_responses_data[:,0]
```

```python
channel_colors = {c: sns.color_palette('Set2',7)[i] for i,c in enumerate(response.wavelength_response)}
```

```python
fig,axes = plt.subplots(4,2,figsize=(15,30),sharex=True)
for channel,ax in zip(sorted(list(temperature_responses.keys())),axes.flatten()):
    ax.plot(temperature,temperature_responses[channel]/(0.83*(1./4./np.pi)),
             label=r'lines',
             color=channel_colors[channel])
    ax.plot(temperature,(temperature_responses[channel]/(0.83*(1./4./np.pi))
                          + continuum_contributions[channel]),
             linestyle=':',
             label='lines + continuum',
             color=channel_colors[channel])
    ax.set_xscale('log')
    ax.set_yscale('log')
    ax.set_ylim([1e-30,2e-24])
    ax.set_xlim([1e5,1e8])
    ax.set_title(r'{} $\AA$'.format(channel))

for i,pc in enumerate(precalc_channels):
    axes.flatten()[i].plot(10**precalculated_responses['temperature'],
                            precalculated_responses[pc],
                            linestyle='--',
                            label=r'SSW',
                            color=channel_colors[pc])
    axes.flatten()[i].plot(10**precalculated_responses_fix['temperature'],
                        precalculated_responses_fix[pc],
                        linestyle='-.',
                        label=r'SSW with chiantifix',
                        color=channel_colors[pc])
axes[0,1].legend(loc='best')
```

```python
continuum_contributions = {k:np.zeros(len(temperature)) for k in response.wavelength_response}
```

```python
wvl = response.wavelength_response[94]['wavelength'].value
for ion in ch_data.MasterList:
    #if ion.split('_')[0] != 'fe':
    #    continue
    print('{}: Calculating contribution function for {}'.format(ch_data.MasterList.index(ion),ion))
    tmp = ch.ion(ion,temperature=temperature.value,eDensity=density.value,abundance='sun_coronal_1992_feldman')
    #two photon emiss
    tmp.twoPhoton(wvl)
    if 'rate' in tmp.TwoPhoton:
        two_photon = tmp.TwoPhoton['rate']
    else:
        two_photon = tmp.TwoPhoton['emiss']
    #free-free
    tmp_cont = ch.continuum(ion,temperature.value,abundance='sun_coronal_1992_feldman')
    if tmp_cont.Ion > 1:
        tmp_cont.freeFree(wvl)
        if 'rate' in tmp_cont.FreeFree:
            free_free = tmp_cont.FreeFree['rate']
        else:
            free_free = np.zeros((len(temperature),len(wvl)))
    else:
        free_free = np.zeros((len(temperature),len(wvl)))
    #free-bound
    if tmp_cont.Ion > 1:
        tmp_cont.freeBound(wvl)
        if 'rate' in tmp_cont.FreeBound:
            free_bound = tmp_cont.FreeBound['rate']
        else:
            free_bound = np.zeros((len(temperature),len(wvl)))
    else:
        free_bound = np.zeros((len(temperature),len(wvl)))
    #add to channels
    for channel in response.wavelength_response:
        continuum_contributions[channel] += np.dot((two_photon + free_free + free_bound),
                                                   (response.wavelength_response[channel]['response'].value
                                                    *response._channel_info[channel]['plate_scale'].value))
```

    0: Calculating contribution function for h_1
    1: Calculating contribution function for he_2
    2: Calculating contribution function for c_6
    3: Calculating contribution function for n_7
    4: Calculating contribution function for o_8
    5: Calculating contribution function for ne_10
    6: Calculating contribution function for na_11
    7: Calculating contribution function for mg_12
    8: Calculating contribution function for al_13
    9: Calculating contribution function for si_14
    10: Calculating contribution function for p_15
    11: Calculating contribution function for s_16
    12: Calculating contribution function for cl_17
    13: Calculating contribution function for ar_18
    14: Calculating contribution function for k_19
    15: Calculating contribution function for ca_20
    16: Calculating contribution function for fe_26

    /Users/willbarnes/anaconda/envs/sunpy-aia-response/lib/python3.5/site-packages/ChiantiPy-0.7.dev417-py3.5.egg/ChiantiPy/core/Continuum.py:413: RuntimeWarning: invalid value encountered in multiply
      fbrate[0,itemp] = em[itemp]*(const.planck*const.light/(1.e-8*wvl))**5*const.verner*gIoneq[itemp]*ratg[0]*expf[0,itemp]*vCross/temperature[itemp]**1.5
    /Users/willbarnes/anaconda/envs/sunpy-aia-response/lib/python3.5/site-packages/numpy/ma/core.py:852: RuntimeWarning: invalid value encountered in greater_equal
      return umath.absolute(a) * self.tolerance >= umath.absolute(b)

    17: Calculating contribution function for ni_28
    18: Calculating contribution function for he_1
    19: Calculating contribution function for c_5
    20: Calculating contribution function for n_6
    21: Calculating contribution function for o_7
    22: Calculating contribution function for ne_9
    23: Calculating contribution function for na_10
    24: Calculating contribution function for mg_11
    25: Calculating contribution function for al_12
    26: Calculating contribution function for si_13
    27: Calculating contribution function for p_14
    28: Calculating contribution function for s_15
    29: Calculating contribution function for cl_16
    30: Calculating contribution function for ar_17
    31: Calculating contribution function for k_18
    32: Calculating contribution function for ca_19
    33: Calculating contribution function for fe_25
    34: Calculating contribution function for ni_27
    35: Calculating contribution function for zn_29
    36: Calculating contribution function for c_5d
     not doing 2 photon for c_5d
    37: Calculating contribution function for n_6d
     not doing 2 photon for n_6d
    38: Calculating contribution function for o_7d
     not doing 2 photon for o_7d
    39: Calculating contribution function for ne_9d
     not doing 2 photon for ne_9d
    40: Calculating contribution function for mg_11d
     not doing 2 photon for mg_11d
    41: Calculating contribution function for al_12d
     not doing 2 photon for al_12d
    42: Calculating contribution function for si_13d
     not doing 2 photon for si_13d
    43: Calculating contribution function for s_15d
     not doing 2 photon for s_15d
    44: Calculating contribution function for ar_17d
     not doing 2 photon for ar_17d
    45: Calculating contribution function for ca_19d
     not doing 2 photon for ca_19d
    46: Calculating contribution function for fe_25d
     not doing 2 photon for fe_25d
    47: Calculating contribution function for ni_27d
     not doing 2 photon for ni_27d
    48: Calculating contribution function for c_4
     not doing 2 photon for c_4
    49: Calculating contribution function for n_5
     not doing 2 photon for n_5
    50: Calculating contribution function for o_6
     not doing 2 photon for o_6
    51: Calculating contribution function for ne_8
     not doing 2 photon for ne_8
    52: Calculating contribution function for na_9
     not doing 2 photon for na_9
    53: Calculating contribution function for mg_10
     not doing 2 photon for mg_10
    54: Calculating contribution function for al_11
     not doing 2 photon for al_11
    55: Calculating contribution function for si_12
     not doing 2 photon for si_12
    56: Calculating contribution function for p_13
     not doing 2 photon for p_13
    57: Calculating contribution function for s_14
     not doing 2 photon for s_14
    58: Calculating contribution function for ar_16
     not doing 2 photon for ar_16
    59: Calculating contribution function for k_17
     not doing 2 photon for k_17
    60: Calculating contribution function for ca_18
     not doing 2 photon for ca_18
    61: Calculating contribution function for ti_20
     not doing 2 photon for ti_20
    62: Calculating contribution function for cr_22
     not doing 2 photon for cr_22
    63: Calculating contribution function for mn_23
     not doing 2 photon for mn_23
    64: Calculating contribution function for fe_24
     not doing 2 photon for fe_24
    65: Calculating contribution function for co_25
     not doing 2 photon for co_25
    66: Calculating contribution function for ni_26
     not doing 2 photon for ni_26
    67: Calculating contribution function for zn_28
     not doing 2 photon for zn_28
    68: Calculating contribution function for c_4d
     not doing 2 photon for c_4d
    69: Calculating contribution function for n_5d
     not doing 2 photon for n_5d
    70: Calculating contribution function for o_6d
     not doing 2 photon for o_6d
    71: Calculating contribution function for ne_8d
     not doing 2 photon for ne_8d
    72: Calculating contribution function for mg_10d
     not doing 2 photon for mg_10d
    73: Calculating contribution function for al_11d
     not doing 2 photon for al_11d
    74: Calculating contribution function for si_12d
     not doing 2 photon for si_12d
    75: Calculating contribution function for s_14d
     not doing 2 photon for s_14d
    76: Calculating contribution function for ar_16d
     not doing 2 photon for ar_16d
    77: Calculating contribution function for ca_18d
     not doing 2 photon for ca_18d
    78: Calculating contribution function for fe_24d
     not doing 2 photon for fe_24d
    79: Calculating contribution function for ni_26d
     not doing 2 photon for ni_26d
    80: Calculating contribution function for c_3
     not doing 2 photon for c_3
    81: Calculating contribution function for n_4
     not doing 2 photon for n_4
    82: Calculating contribution function for o_5
     not doing 2 photon for o_5
    83: Calculating contribution function for ne_7
     not doing 2 photon for ne_7
    84: Calculating contribution function for na_8
     not doing 2 photon for na_8
    85: Calculating contribution function for mg_9
     not doing 2 photon for mg_9
    86: Calculating contribution function for al_10
     not doing 2 photon for al_10
    87: Calculating contribution function for si_11
     not doing 2 photon for si_11
    88: Calculating contribution function for p_12
     not doing 2 photon for p_12
    89: Calculating contribution function for s_13
     not doing 2 photon for s_13
    90: Calculating contribution function for cl_14
     not doing 2 photon for cl_14
    91: Calculating contribution function for ar_15
     not doing 2 photon for ar_15
    92: Calculating contribution function for k_16
     not doing 2 photon for k_16
    93: Calculating contribution function for ca_17
     not doing 2 photon for ca_17
    94: Calculating contribution function for ti_19
     not doing 2 photon for ti_19
    95: Calculating contribution function for cr_21
     not doing 2 photon for cr_21
    96: Calculating contribution function for mn_22
     not doing 2 photon for mn_22
    97: Calculating contribution function for fe_23
     not doing 2 photon for fe_23
    98: Calculating contribution function for co_24
     not doing 2 photon for co_24
    99: Calculating contribution function for ni_25
     not doing 2 photon for ni_25
    100: Calculating contribution function for zn_27
     not doing 2 photon for zn_27
    101: Calculating contribution function for fe_23d
     not doing 2 photon for fe_23d
    102: Calculating contribution function for fe_22d
     not doing 2 photon for fe_22d
    103: Calculating contribution function for fe_21d
     not doing 2 photon for fe_21d
    104: Calculating contribution function for fe_20d
     not doing 2 photon for fe_20d
    105: Calculating contribution function for fe_19d
     not doing 2 photon for fe_19d
    106: Calculating contribution function for fe_18d
     not doing 2 photon for fe_18d
    107: Calculating contribution function for fe_17d
     not doing 2 photon for fe_17d
    108: Calculating contribution function for fe_16d
     not doing 2 photon for fe_16d
    109: Calculating contribution function for fe_15d
     not doing 2 photon for fe_15d
    110: Calculating contribution function for fe_14d
     not doing 2 photon for fe_14d
    111: Calculating contribution function for fe_13d
     not doing 2 photon for fe_13d
    112: Calculating contribution function for fe_12d
     not doing 2 photon for fe_12d
    113: Calculating contribution function for fe_11d
     not doing 2 photon for fe_11d
    114: Calculating contribution function for fe_10d
     not doing 2 photon for fe_10d
    115: Calculating contribution function for fe_9d
     not doing 2 photon for fe_9d
    116: Calculating contribution function for fe_8d
     not doing 2 photon for fe_8d
    117: Calculating contribution function for fe_7d
     not doing 2 photon for fe_7d
    118: Calculating contribution function for fe_6d
     not doing 2 photon for fe_6d
    119: Calculating contribution function for fe_5d
     not doing 2 photon for fe_5d
    120: Calculating contribution function for fe_4d
     not doing 2 photon for fe_4d
    121: Calculating contribution function for fe_3d
     not doing 2 photon for fe_3d
    122: Calculating contribution function for fe_2d
     not doing 2 photon for fe_2d
    123: Calculating contribution function for c_2
     not doing 2 photon for c_2
    124: Calculating contribution function for n_3
     not doing 2 photon for n_3
    125: Calculating contribution function for o_4
     not doing 2 photon for o_4
    126: Calculating contribution function for ne_6
     not doing 2 photon for ne_6
    127: Calculating contribution function for na_7
     not doing 2 photon for na_7
    128: Calculating contribution function for mg_8
     not doing 2 photon for mg_8
    129: Calculating contribution function for al_9
     not doing 2 photon for al_9
    130: Calculating contribution function for si_10
     not doing 2 photon for si_10
    131: Calculating contribution function for p_11
     not doing 2 photon for p_11
    132: Calculating contribution function for s_12
     not doing 2 photon for s_12
    133: Calculating contribution function for cl_13
     not doing 2 photon for cl_13
    134: Calculating contribution function for ar_14
     not doing 2 photon for ar_14
    135: Calculating contribution function for k_15
     not doing 2 photon for k_15
    136: Calculating contribution function for ca_16
     not doing 2 photon for ca_16
    137: Calculating contribution function for ti_18
     not doing 2 photon for ti_18
    138: Calculating contribution function for cr_20
     not doing 2 photon for cr_20
    139: Calculating contribution function for mn_21
     not doing 2 photon for mn_21
    140: Calculating contribution function for fe_22
     not doing 2 photon for fe_22
    141: Calculating contribution function for co_23
     not doing 2 photon for co_23
    142: Calculating contribution function for ni_24
     not doing 2 photon for ni_24
    143: Calculating contribution function for c_1
     not doing 2 photon for c_1
    144: Calculating contribution function for n_2
     not doing 2 photon for n_2
    145: Calculating contribution function for o_3
     not doing 2 photon for o_3
    146: Calculating contribution function for ne_5
     not doing 2 photon for ne_5
    147: Calculating contribution function for na_6
     not doing 2 photon for na_6
    148: Calculating contribution function for mg_7
     not doing 2 photon for mg_7
    149: Calculating contribution function for al_8
     not doing 2 photon for al_8
    150: Calculating contribution function for si_9
     not doing 2 photon for si_9
    151: Calculating contribution function for p_10
     not doing 2 photon for p_10
    152: Calculating contribution function for s_11
     not doing 2 photon for s_11
    153: Calculating contribution function for cl_12
     not doing 2 photon for cl_12
    154: Calculating contribution function for ar_13
     not doing 2 photon for ar_13
    155: Calculating contribution function for k_14
     not doing 2 photon for k_14
    156: Calculating contribution function for ca_15
     not doing 2 photon for ca_15
    157: Calculating contribution function for ti_17
     not doing 2 photon for ti_17
    158: Calculating contribution function for cr_19
     not doing 2 photon for cr_19
    159: Calculating contribution function for mn_20
     not doing 2 photon for mn_20
    160: Calculating contribution function for fe_21
     not doing 2 photon for fe_21
    161: Calculating contribution function for co_22
     not doing 2 photon for co_22
    162: Calculating contribution function for ni_23
     not doing 2 photon for ni_23
    163: Calculating contribution function for zn_25
     not doing 2 photon for zn_25
    164: Calculating contribution function for n_1
     not doing 2 photon for n_1
    165: Calculating contribution function for o_2
     not doing 2 photon for o_2
    166: Calculating contribution function for ne_4
     not doing 2 photon for ne_4
    167: Calculating contribution function for na_5
     not doing 2 photon for na_5
    168: Calculating contribution function for mg_6
     not doing 2 photon for mg_6
    169: Calculating contribution function for al_7
     not doing 2 photon for al_7
    170: Calculating contribution function for si_8
     not doing 2 photon for si_8
    171: Calculating contribution function for p_9
     not doing 2 photon for p_9
    172: Calculating contribution function for s_10
     not doing 2 photon for s_10
    173: Calculating contribution function for cl_11
     not doing 2 photon for cl_11
    174: Calculating contribution function for ar_12
     not doing 2 photon for ar_12
    175: Calculating contribution function for k_13
     not doing 2 photon for k_13
    176: Calculating contribution function for ca_14
     not doing 2 photon for ca_14
    177: Calculating contribution function for ti_16
     not doing 2 photon for ti_16
    178: Calculating contribution function for cr_18
     not doing 2 photon for cr_18
    179: Calculating contribution function for mn_19
     not doing 2 photon for mn_19
    180: Calculating contribution function for fe_20
     not doing 2 photon for fe_20
    181: Calculating contribution function for co_21
     not doing 2 photon for co_21
    182: Calculating contribution function for ni_22
     not doing 2 photon for ni_22
    183: Calculating contribution function for zn_24
     not doing 2 photon for zn_24
    184: Calculating contribution function for o_1
     not doing 2 photon for o_1
    185: Calculating contribution function for ne_3
     not doing 2 photon for ne_3
    186: Calculating contribution function for na_4
     not doing 2 photon for na_4
    187: Calculating contribution function for mg_5
     not doing 2 photon for mg_5
    188: Calculating contribution function for al_6
     not doing 2 photon for al_6
    189: Calculating contribution function for si_7
     not doing 2 photon for si_7
    190: Calculating contribution function for p_8
     not doing 2 photon for p_8
    191: Calculating contribution function for s_9
     not doing 2 photon for s_9
    192: Calculating contribution function for cl_10
     not doing 2 photon for cl_10
    193: Calculating contribution function for ar_11
     not doing 2 photon for ar_11
    194: Calculating contribution function for k_12
     not doing 2 photon for k_12
    195: Calculating contribution function for ca_13
     not doing 2 photon for ca_13
    196: Calculating contribution function for ti_15
     not doing 2 photon for ti_15
    197: Calculating contribution function for cr_17
     not doing 2 photon for cr_17
    198: Calculating contribution function for mn_18
     not doing 2 photon for mn_18
    199: Calculating contribution function for fe_19
     not doing 2 photon for fe_19
    200: Calculating contribution function for co_20
     not doing 2 photon for co_20
    201: Calculating contribution function for ni_21
     not doing 2 photon for ni_21
    202: Calculating contribution function for zn_23
     not doing 2 photon for zn_23
    203: Calculating contribution function for ne_2
     not doing 2 photon for ne_2
    204: Calculating contribution function for na_3
     not doing 2 photon for na_3
    205: Calculating contribution function for mg_4
     not doing 2 photon for mg_4
    206: Calculating contribution function for al_5
     not doing 2 photon for al_5
    207: Calculating contribution function for si_6
     not doing 2 photon for si_6
    208: Calculating contribution function for p_7
     not doing 2 photon for p_7
    209: Calculating contribution function for s_8
     not doing 2 photon for s_8
    210: Calculating contribution function for ar_10
     not doing 2 photon for ar_10
    211: Calculating contribution function for k_11
     not doing 2 photon for k_11
    212: Calculating contribution function for ca_12
     not doing 2 photon for ca_12
    213: Calculating contribution function for ti_14
     not doing 2 photon for ti_14
    214: Calculating contribution function for cr_16
     not doing 2 photon for cr_16
    215: Calculating contribution function for mn_17
     not doing 2 photon for mn_17
    216: Calculating contribution function for fe_18
     not doing 2 photon for fe_18
    217: Calculating contribution function for co_19
     not doing 2 photon for co_19
    218: Calculating contribution function for ni_20
     not doing 2 photon for ni_20
    219: Calculating contribution function for na_2
     not doing 2 photon for na_2
    220: Calculating contribution function for mg_3
     not doing 2 photon for mg_3
    221: Calculating contribution function for al_4
     not doing 2 photon for al_4
    222: Calculating contribution function for si_5
     not doing 2 photon for si_5
    223: Calculating contribution function for p_6
     not doing 2 photon for p_6
    224: Calculating contribution function for s_7
     not doing 2 photon for s_7
    225: Calculating contribution function for ar_9
     not doing 2 photon for ar_9
    226: Calculating contribution function for k_10
     not doing 2 photon for k_10
    227: Calculating contribution function for ca_11
     not doing 2 photon for ca_11
    228: Calculating contribution function for ti_13
     not doing 2 photon for ti_13
    229: Calculating contribution function for cr_15
     not doing 2 photon for cr_15
    230: Calculating contribution function for mn_16
     not doing 2 photon for mn_16
    231: Calculating contribution function for fe_17
     not doing 2 photon for fe_17
    232: Calculating contribution function for co_18
     not doing 2 photon for co_18
    233: Calculating contribution function for ni_19
     not doing 2 photon for ni_19
    234: Calculating contribution function for zn_21
     not doing 2 photon for zn_21
    235: Calculating contribution function for mg_2
     not doing 2 photon for mg_2
    236: Calculating contribution function for al_3
     not doing 2 photon for al_3
    237: Calculating contribution function for si_4
     not doing 2 photon for si_4
    238: Calculating contribution function for p_5
     not doing 2 photon for p_5
    239: Calculating contribution function for s_6
     not doing 2 photon for s_6
    240: Calculating contribution function for ar_8
     not doing 2 photon for ar_8
    241: Calculating contribution function for k_9
     not doing 2 photon for k_9
    242: Calculating contribution function for ca_10
     not doing 2 photon for ca_10
    243: Calculating contribution function for ti_12
     not doing 2 photon for ti_12
    244: Calculating contribution function for cr_14
     not doing 2 photon for cr_14
    245: Calculating contribution function for mn_15
     not doing 2 photon for mn_15
    246: Calculating contribution function for fe_16
     not doing 2 photon for fe_16
    247: Calculating contribution function for co_17
     not doing 2 photon for co_17
    248: Calculating contribution function for ni_18
     not doing 2 photon for ni_18
    249: Calculating contribution function for zn_20
     not doing 2 photon for zn_20
    250: Calculating contribution function for al_2
     not doing 2 photon for al_2
    251: Calculating contribution function for si_3
     not doing 2 photon for si_3
    252: Calculating contribution function for s_5
     not doing 2 photon for s_5
    253: Calculating contribution function for ar_7
     not doing 2 photon for ar_7
    254: Calculating contribution function for ca_9
     not doing 2 photon for ca_9
    255: Calculating contribution function for ti_11
     not doing 2 photon for ti_11
    256: Calculating contribution function for cr_13
     not doing 2 photon for cr_13
    257: Calculating contribution function for fe_15
     not doing 2 photon for fe_15
    258: Calculating contribution function for ni_17
     not doing 2 photon for ni_17
    259: Calculating contribution function for si_2
     not doing 2 photon for si_2
    260: Calculating contribution function for s_4
     not doing 2 photon for s_4
    261: Calculating contribution function for ca_8
     not doing 2 photon for ca_8
    262: Calculating contribution function for fe_14
     not doing 2 photon for fe_14
    263: Calculating contribution function for ni_16
     not doing 2 photon for ni_16
    264: Calculating contribution function for s_3
     not doing 2 photon for s_3
    265: Calculating contribution function for cl_4
     not doing 2 photon for cl_4
    266: Calculating contribution function for ar_5
     not doing 2 photon for ar_5
    267: Calculating contribution function for k_6
     not doing 2 photon for k_6
    268: Calculating contribution function for ca_7
     not doing 2 photon for ca_7
    269: Calculating contribution function for fe_13
     not doing 2 photon for fe_13
    270: Calculating contribution function for ni_15
     not doing 2 photon for ni_15
    271: Calculating contribution function for s_2
     not doing 2 photon for s_2
    272: Calculating contribution function for cl_3
     not doing 2 photon for cl_3
    273: Calculating contribution function for ar_4
     not doing 2 photon for ar_4
    274: Calculating contribution function for k_5
     not doing 2 photon for k_5
    275: Calculating contribution function for ca_6
     not doing 2 photon for ca_6
    276: Calculating contribution function for fe_12
     not doing 2 photon for fe_12
    277: Calculating contribution function for ni_14
     not doing 2 photon for ni_14
    278: Calculating contribution function for s_1
     not doing 2 photon for s_1
    279: Calculating contribution function for cl_2
     not doing 2 photon for cl_2
    280: Calculating contribution function for ar_3
     not doing 2 photon for ar_3
    281: Calculating contribution function for ca_5
     not doing 2 photon for ca_5
    282: Calculating contribution function for cr_9
     not doing 2 photon for cr_9
    283: Calculating contribution function for mn_10
     not doing 2 photon for mn_10
    284: Calculating contribution function for fe_11
     not doing 2 photon for fe_11
    285: Calculating contribution function for ni_13
     not doing 2 photon for ni_13
    286: Calculating contribution function for cr_8
     not doing 2 photon for cr_8
    287: Calculating contribution function for mn_9
     not doing 2 photon for mn_9
    288: Calculating contribution function for fe_10
     not doing 2 photon for fe_10
    289: Calculating contribution function for ni_12
     not doing 2 photon for ni_12
    290: Calculating contribution function for cr_7
     not doing 2 photon for cr_7
    291: Calculating contribution function for mn_8
     not doing 2 photon for mn_8
    292: Calculating contribution function for fe_9
     not doing 2 photon for fe_9
    293: Calculating contribution function for ni_11
     not doing 2 photon for ni_11
    294: Calculating contribution function for ca_2
     not doing 2 photon for ca_2
    295: Calculating contribution function for fe_8
     not doing 2 photon for fe_8
    296: Calculating contribution function for fe_7
     not doing 2 photon for fe_7
    297: Calculating contribution function for fe_6
     not doing 2 photon for fe_6
    298: Calculating contribution function for fe_5
     not doing 2 photon for fe_5
    299: Calculating contribution function for fe_4
     not doing 2 photon for fe_4
    300: Calculating contribution function for fe_3
     not doing 2 photon for fe_3
    301: Calculating contribution function for fe_2
     not doing 2 photon for fe_2
    302: Calculating contribution function for ni_2
     not doing 2 photon for ni_2

```python
plt.figure(figsize=(8,8))
for channel in continuum_contributions:
    plt.plot(temperature,
             continuum_contributions[channel],
             label=channel,color=channel_colors[channel])
plt.xscale('log')
plt.yscale('log')
plt.xlim([1e5,1e8])
plt.ylim([1e-32,1e-27])
plt.legend(loc='best')
```

Seems like the factor of $1/4\pi$ is missing from the contribution function, despite what the comments in the IDL code say...neglecting it gives good (almost exact!) agreement in the parts of the response function dominated by line emission

Factor of 0.83 is definitely missing.

Continuum calculation still too small.

TODO:

* re-calculate emission structure from IDL scripts
* re-calculate temperature response functions
* double-check continuum calculations, compare with CHIANTI-IDL results; continuum calculations still too small
* save emissivity info in HDF5 file, separated by ion, $N_{\lambda}\times N_{T}$ array for each ion, continuum contributions as well
* flow chart of how this code works might be helpful...

```python

```


```python
import numpy as np
import pandas as pd

import respy as rp

from python.auxiliary_setup import load_model_specs
from python.auxiliary_plots import plot_chatter_numagents_sim
from python.auxiliary_plots import plot_chatter_numagents_both
from python.auxiliary_plots import plot_criterion_params
from python.auxiliary_plots import plot_moments_choices
from python.auxiliary_plots import plot_moments_wage
from python.auxiliary_plots import plot_chatter
```

## Method of Simulated Moments Estimation (MSM)

This lecture heavily relies on [respy](https://respy.readthedocs.io/en/latest/index.html)'s method of simulated moments (MSM) interface. A guide to using the interface can be found [here](https://respy.readthedocs.io/en/latest/how_to_guides/msm.html).

Table of Contents

1  Introduction

2  Observed Data

3  Data Moments

4  Weighting Matrix

5  Criterion Function

6  Estimation Exercise

6.1  Estimation for one parameter

6.1.1  Simulated Moments

6.1.2  Optimization Procedure

6.1.2.1  Upper and Lower Bounds

6.1.2.2  Constraints

6.1.2.3  Optimize

6.2  Chatter in the Criterion Function

6.2.1  Changing the Simulation Seed

6.2.2  Multiple Simulation Seeds

6.2.3  Changing the Simulation Seed for increasing Sample Size of Simulated Agents

6.2.4  Changing the Simulation Seed for increasing Sample Size of Simulated and Observed Agents

6.3  Moving away from the Optimum

6.3.1  Retrieving the true Parameter Vector

6.4  Derivative-Based Optimization Algorithm

7  References

### Introduction

The method of simulated moments approach to estimating model parameters is to minimize a certain distance between observed moments and simulated moments with respect to the parameters that generate the simulated model.

Denote $X$ our observed data and $m(X)$ the vector of observed moments. To construct the criterion function, we use the parameter vector $\theta$ to simulate data from the  model $\hat{X}$. We can then calculate the simulated moments $m(\hat{X}| \theta)$.

The criterion function is then given by

\begin{equation}
\psi(\theta) = (m(X) - m(\hat{X}| \theta))'\Omega(m(X) - m(\hat{X}| \theta))
\end{equation}

where the difference between observed and simulated moments $(m(X) - m(\hat{X}| \theta))$ constitutes a vector of the dimension $1\times M$ with $1,..,M$ denoting the number of moments. The $M\times M$ weighting matrix is given by $\Omega$.

The MSM estimator is defined as the solution to

\begin{equation}
\hat{\theta}(\Omega) = \underset{\theta}{\operatorname{argmin}} \psi(\theta).
\end{equation}

The criterion function is thus a strictly positive scalar and the estimator depends on the choice of moments $m$ and the weighting matrix $\Omega$. The weighting matrix applies some scaling for discrepancies between the observed and simulated moments. If we use the identity matrix, each moment is given equal weight and the criterion function reduces to the sum of squared moment deviations.

Aside from the choice of moments and weighting matrix, some other important choices that influence the the estimation are the simulator itself and the algorithm and specifications for the optimization procedure. Many explanations for simulated method of moments estimation also feature the number of simulations as a factor that is to be determined for estimation. We can ignore this factor for now since we are working with a large simulated dataset.

In the following we will set up the data, moments and weighting matrix needed to construct the criterion function and subsequently try to estimating the parameters of the model using this MSM setup.

###  Observed Data

We generate our model and data using [respy](https://respy.readthedocs.io/en/latest/) and a simple Robinson Crusoe model. In this model, the agent, Robinson Crusoe, in each time period decides between two choice options: working (i.e. going fishing) or spending time in the hammock.

We can use respy to simulate the data for this exercise.

```python
params_true, options = load_model_specs()
```

```python
# Generate observed data from model.
simulate = rp.get_simulate_func(params_true, options)
data_obs = simulate(params_true)
```

Let's take a look at the model specifications.

```python
params_true
```

      value

      category
      name

      delta
      delta
      0.950

      wage_fishing
      exp_fishing
      0.070

      nonpec_fishing
      constant
      -0.100

      nonpec_hammock
      constant
      1.046

      shocks_sdcorr
      sd_fishing
      0.010

      sd_hammock
      0.010

      corr_hammock_fishing
      0.000

```python
options
```

    {'estimation_draws': 100,
     'estimation_seed': 100,
     'estimation_tau': 0.001,
     'interpolation_points': -1,
     'n_periods': 5,
     'simulation_agents': 1000,
     'simulation_seed': 132,
     'solution_draws': 100,
     'solution_seed': 456,
     'covariates': {'constant': '1'}}

```python
data_obs.head(10)
```

      Experience_Fishing
      Shock_Reward_Fishing
      Meas_Error_Wage_Fishing
      Shock_Reward_Hammock
      Meas_Error_Wage_Hammock
      Dense_Key
      Core_Index
      Choice
      Wage
      Discount_Rate
      ...
      Nonpecuniary_Reward_Fishing
      Wage_Fishing
      Flow_Utility_Fishing
      Value_Function_Fishing
      Continuation_Value_Fishing
      Nonpecuniary_Reward_Hammock
      Wage_Hammock
      Flow_Utility_Hammock
      Value_Function_Hammock
      Continuation_Value_Hammock

      Identifier
      Period

      0
      0
      0
      -0.035035
      1
      0.040965
      1
      0
      0
      fishing
      0.999650
      0.95
      ...
      -0.1
      0.999650
      0.899650
      4.740058
      4.042536
      1.046
      NaN
      1.046410
      4.733035
      3.880659

      1
      1
      0.074254
      1
      1.506491
      1
      1
      1
      fishing
      1.073305
      0.95
      ...
      -0.1
      1.073305
      0.973305
      4.043351
      3.231627
      1.046
      NaN
      1.061065
      3.907554
      2.996304

      2
      2
      -0.354560
      1
      1.185316
      1
      2
      2
      fishing
      1.146203
      0.95
      ...
      -0.1
      1.146203
      1.046203
      3.227408
      2.296006
      1.046
      NaN
      1.057853
      3.079072
      2.127598

      3
      3
      -0.109397
      1
      -0.785877
      1
      3
      3
      fishing
      1.232329
      0.95
      ...
      -0.1
      1.232329
      1.132329
      2.294755
      1.223607
      1.046
      NaN
      1.038141
      2.115558
      1.134123

      4
      4
      -1.063705
      1
      1.245234
      1
      4
      4
      fishing
      1.309130
      0.95
      ...
      -0.1
      1.309130
      1.209130
      1.209130
      0.000000
      1.046
      NaN
      1.058452
      1.058452
      0.000000

      1
      0
      0
      -0.713137
      1
      1.418950
      1
      0
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.992894
      0.892894
      4.733303
      4.042536
      1.046
      NaN
      1.060190
      4.746815
      3.880659

      1
      0
      -0.464134
      1
      -0.384774
      1
      1
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.995369
      0.895369
      3.741858
      2.996304
      1.046
      NaN
      1.042152
      3.877028
      2.984080

      2
      0
      0.066755
      1
      -0.844060
      1
      2
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      1.000668
      0.900668
      2.838283
      2.039595
      1.046
      NaN
      1.037559
      2.975174
      2.039595

      3
      0
      -0.748565
      1
      -1.348821
      1
      3
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.992542
      0.892542
      1.886280
      1.046040
      1.046
      NaN
      1.032512
      2.026250
      1.046040

      4
      0
      -0.130574
      1
      -0.539693
      1
      4
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.998695
      0.898695
      0.898695
      0.000000
      1.046
      NaN
      1.040603
      1.040603
      0.000000

10 rows × 21 columns

### Data Moments

For the setup of the estimation we first have to choose a set of moments that we will use to match the observed data and the simulated model. For this model we include two sets of moments:

1. The first set are Robinson's **choice frequencies** (choice frequencies here refers to the share of agents that have chosen a specific option) for each period.
2. The second set are moments that characterize the **wage distribution** for each period, i.e. the mean of the wage of all agents that have chosen fishing in a given period and the standard deviation of the wages.

We need a functions that compute the set of moments on the observed and simulated data.

```python
def calc_choice_frequencies(df):
    """Calculate choice frequencies"""
    return df.groupby("Period").Choice.value_counts(normalize=True).unstack()
```

```python
def calc_wage_distribution(df):
    """Calculate wage distribution."""
    return df.groupby(["Period"])["Wage"].describe()[["mean", "std"]]
```

```python
# Save functions in a dictionary to pass to the criterion function later on.
calc_moments = {
    "Choice Frequencies": calc_choice_frequencies,
    "Wage Distribution": calc_wage_distribution,
}
```

Respy's MSM interface additionally requires us to specify, how missings in the computed moments should be handled. Missing moments for instance can arise, when certain choice options aren't picked at all for some periods. We just specify a function that replaces missings with 0.

```python
def replace_nans(df):
    """Replace missing values in data."""
    return df.fillna(0)
```

Now we are ready to calculate the moments.

```python
moments_obs = {
    "Choice Frequencies": replace_nans(calc_moments["Choice Frequencies"](data_obs)),
    "Wage Distribution": replace_nans(calc_moments["Wage Distribution"](data_obs)),
}
```

```python
print("Choice Frequencies")
print(moments_obs["Choice Frequencies"])
print("\n Wage Distribution")
print(moments_obs["Wage Distribution"])
```

    Choice Frequencies
    Choice  fishing  hammock
    Period
    0         0.698    0.302
    1         0.698    0.302
    2         0.698    0.302
    3         0.698    0.302
    4         0.698    0.302

     Wage Distribution
                mean       std
    Period
    0       1.003189  0.008640
    1       1.072815  0.010737
    2       1.150106  0.012316
    3       1.234041  0.012406
    4       1.322711  0.013473

### Weighting Matrix

Next we specify a weighting matrix. It needs to be a square matrix with the same number of diagonal elements as there are moments. One option would be to use the identity matrix, but we use a weighting matrix that adjusts for the variance of each moment to improve the estimation process. The variances for the moments are constructed using a bootstrapping procedure.

```python
def get_weighting_matrix(data, calc_moments, num_boots, num_agents_msm):
    """ Compute weighting matrix for estimation with MSM."""
    # Seed for reproducibility.
    np.random.seed(123)

    index_base = data.index.get_level_values("Identifier").unique()

    # Create bootstrapped moments.
    moments_sample = list()
    for _ in range(num_boots):
        ids_boot = np.random.choice(index_base, num_agents_msm, replace=False)
        moments_boot = [calc_moments[key](data.loc[ids_boot, :]) for key in calc_moments.keys()]

        moments_boot = rp.get_flat_moments(moments_boot)

        moments_sample.append(moments_boot)

    # Compute variance for each moment and construct diagonal weighting matrix.
    moments_var = np.array(moments_sample).var(axis=0)
    weighting_matrix = np.diag(moments_var ** (-1))

    return np.nan_to_num(weighting_matrix)
```

```python
W = get_weighting_matrix(data_obs, calc_moments, 300, 500)
```

```python
pd.DataFrame(W)
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19

      0
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      1
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      2
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      3
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      4
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      5
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      6
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      7
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      8
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      9
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      10
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      9.466832e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      11
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      6.036166e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      12
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      4.698312e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      13
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      4.248616e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      14
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      3.566318e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      15
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      1.779242e+07
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      16
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      1.258966e+07
      0.000000e+00
      0.000000e+00
      0.000000e+00

      17
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      9.317930e+06
      0.000000e+00
      0.000000e+00

      18
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      8.904094e+06
      0.000000e+00

      19
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      1.067252e+07

### Criterion Function

We have collected the observed data for our model, chosen the set of moments we want to use for estimation and defined a weighting matrix based on these moments. We can now set up the criterion function to use for estimation.

As already discussed above, the criterion function is given by the weighted square product of the difference between observed moments $m(X)$ and simulated moments $m(\hat{X}| \theta)$. Trivially, if we have that $m(X) = m(\hat{X}| \theta)$, the criterion function returns a value of 0. Thus, the closer $\theta$ is to the real parameter vector, the smaller should be the value for the criterion function.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options, calc_moments, replace_nans, moments_obs, W
)
```

Criterion function at the true parameter vector:

```python
fval = criterion_msm(params_true)
fval
```

    0.0

We can plot the criterion function to examine its behavior around the minimum in more detail. The plots below show the criterion function at varying values of all parameters in the the paramter vector.

```python
plot_criterion_params(params_true, list(params_true.index), criterion_msm, 0.05)
```

This depiction for most parameters conceals the fact that the criterion function is not a smooth function of our parameter values. We can reveal this property if we 'zoom in' on the function far enough. The plots below show the criterion function for varying values of *delta* around the true minimum value of 0.95. We can see that the function exhibits small plateaus and is thus not completely smooth.

```python
for radius in [0.05, 0.01, 0.001, 0.0001]:
    plot_criterion_params(params_true, list(params_true.index)[0:1], criterion_msm, radius)
```

###  Estimation Exercise

In the following we will conduct a simulation exercise to estimate the parameter vector using our criterion function and weighting matrix. We will begin by simulating data using the new parameter vector and examine how the simulated moments differ from the observed ones. We will then use an optimizer to minimize the criterion function in order to retrieve the true parameter vector. Additionally, we will explore how the criterion function behaves if we change the simulation seed, move away from the true values and test the optimization for a derivative-based optimization algorithm.

#### Estimation for one parameter

For now, our candidate parameter vector will just differ in *delta* from the true parameters.

```python
params_cand = params_true.copy()
params_cand.loc["delta", "value"] = 0.93
```

##### Simulated Moments

We can now use our model to simulate data using the candidate parameter vector. We can see that the choice frequencies and wage distribution differ from the moments of the observed dataset.

```python
params = params_cand.copy()
simulate = rp.get_simulate_func(params, options)
df_sim = simulate(params)
```

```python
moments_sim = {
    "Choice Frequencies": replace_nans(calc_moments["Choice Frequencies"](df_sim)),
    "Wage Distribution": replace_nans(calc_moments["Wage Distribution"](df_sim)),
}
```

We can plot the moments to compare the choice frequencies for each period.

```python
plot_moments_choices(moments_obs, moments_sim)
```

The plots below show the mean and the standard deviation in the wages for each period.

```python
plot_moments_wage(moments_obs, moments_sim)
```

The criterion function value for the candidate parameter vector is not zero.

```python
fval = criterion_msm(params_cand)
fval
```

    7957.872609154357

##### Optimization Procedure

We will now use an optimization procedure to retrieve the true parameter vector. For the optimization we can use [estimagic](https://estimagic.readthedocs.io/en/latest/index.html). In order to minimize the criterion function we need estimagic's `minimize` function.

```python
from estimagic.optimization.optimize import minimize  # noqa: E402
```

We have verified above that the criterion function gives a value of 0 for the true parameter vector. Before we try different parameter specifications, we can check whether an optimizer recognizes the true vector as the minimum of our criterion function.

As the code below shows, the optimization algorithm recognizes the true parameter vector as the minimum of the criterion function as it returns a function value of 0 and the true parameter values.

```python
rslt = minimize(
    criterion=criterion_msm,
    params=params_true,
    algorithm="nlopt_bobyqa",
)
rslt[0]
```

    {'status': 'success',
     'fitness': 0.0,
     'x': [0.95, 0.07, -0.1, 1.046, 0.01, 0.01, 0.0],
     'n_evaluations': 125,
     'n_evaluations_jacobian': 0,
     'n_evaluations_hessian': 0,
     'n_constraints': 0,
     'n_constraints_equality': 0,
     'n_constraints_inequality': 0}

##### Upper and Lower Bounds

We can help the optimizer by specifying bounds for the parameters. Since we know the true parameters in the case of this model, we can just pick upper and lower bounds that are fairly close to the true values of the parameters to aid the optimizer in the search for the optimum. By default, the upper and lower bounds are set to $\infty$ and $-\infty$, so specifying upper and lower bounds substantially reduces the range of parameter values that the optimizer can potentially cover.

For optimization with estimagic, we can specify bounds by adding the columns *'lower'* and *'upper'* to the dataframe that contains the parameter values.

```python
params_cand["lower"] = [0.89, 0.066, -0.11, 1.04, 0, 0, 0]
params_cand["upper"] = [0.98, 0.072, -0.095, 1.055, 0.1, 0.1, 0.1]
params_cand
```

      value
      lower
      upper

      category
      name

      delta
      delta
      0.930
      0.890
      0.980

      wage_fishing
      exp_fishing
      0.070
      0.066
      0.072

      nonpec_fishing
      constant
      -0.100
      -0.110
      -0.095

      nonpec_hammock
      constant
      1.046
      1.040
      1.055

      shocks_sdcorr
      sd_fishing
      0.010
      0.000
      0.100

      sd_hammock
      0.010
      0.000
      0.100

      corr_hammock_fishing
      0.000
      0.000
      0.100

##### Constraints

Additionally we hold all other parameters fixed for now to aid the optimizer in finding the optimal value for *delta*.

```python
# Define base constraints to use for the rest of the notebook.
constr_base = [
    {"loc": "shocks_sdcorr", "type": "sdcorr"},
    {"loc": "delta", "type": "fixed"},
    {"loc": "wage_fishing", "type": "fixed"},
    {"loc": "nonpec_fishing", "type": "fixed"},
    {"loc": "nonpec_hammock", "type": "fixed"},
    {"loc": "shocks_sdcorr", "type": "fixed"},
]
```

```python
# Remove constraint for delta.
constr = constr_base.copy()
constr.remove({"loc": "delta", "type": "fixed"})
```

##### Optimize

We can now optimize the criterion function with respect to the parameter vector. The optimizer manages to reach a function value of 0 and finds an approximately correct *delta* for our model.

This exercise again reveals that we are dealing with a non-smooth criterion function. The optimizer does not return the exact value of 0.95 for *delta* because of the little plateaus we saw when zooming into the criterion function. As the plot shows, there is a small area around the true value for *delta* that also returns a function value of 0 and might thus be picked by the optimizer.

```python
rslt = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="nlopt_bobyqa",
    constraints=constr,
)
rslt[0]
```

    {'status': 'success',
     'fitness': 0.0,
     'x': [0.9499847421091535],
     'n_evaluations': 24,
     'n_evaluations_jacobian': 0,
     'n_evaluations_hessian': 0,
     'n_constraints': 0,
     'n_constraints_equality': 0,
     'n_constraints_inequality': 0}

#### Chatter in the Criterion Function

In this exercise we explore the sensitivity of the criterion function to the choice of simulation seed and number of agents.

##### Changing the Simulation Seed

As shown above, the optimizer manages to find a function value of exactly 0 for the true parameter vector. This is the case because respy controls the realization of random elements in the simulation process via a simulation seed. The model thus always simulates the exact same dataset for a given parameter vector. Our criterion function becomes exactly 0 at the true parameter vector because for this vector, the observed and simulated data are identical.

Changing the simulation seed results in simulated moments that, even for the true parameter vector, are never completely equal to the observed moments.

Let's estimate the model with a different simulation seed.

```python
options_new_seed = options.copy()
options_new_seed["simulation_seed"] = 400
```

We can see that the criterion function is not 0 anymore for the true parameter vector.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options_new_seed, calc_moments, replace_nans, moments_obs, W
)
criterion_msm(params_true)
```

    30.913298986679713

Our optimizer thus also does not return a function value of 0 for the true parameter vector.

```python
rslt_new_seed = minimize(
    criterion=criterion_msm,
    params=params_true,
    algorithm="nlopt_bobyqa",
    constraints=constr,
)
rslt_new_seed[0]
```

    {'status': 'success',
     'fitness': 30.913298986679713,
     'x': [0.95],
     'n_evaluations': 27,
     'n_evaluations_jacobian': 0,
     'n_evaluations_hessian': 0,
     'n_constraints': 0,
     'n_constraints_equality': 0,
     'n_constraints_inequality': 0}

Since the optimizer does not even recognize the true parameter vector, it is also not able to reach a criterion function  value of 0 for a different parameter vector.

```python
rslt_new_seed = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="nlopt_bobyqa",
    constraints=constr,
)

rslt_new_seed[0]
```

    {'status': 'success',
     'fitness': 31.713998229248002,
     'x': [0.9498764924810543],
     'n_evaluations': 25,
     'n_evaluations_jacobian': 0,
     'n_evaluations_hessian': 0,
     'n_constraints': 0,
     'n_constraints_equality': 0,
     'n_constraints_inequality': 0}

##### Multiple Simulation Seeds

The section above shows what happens to the criterion function if we change the seed for simulating the data. In the sections below we will repeat this exercise for multiple seeds and explore what happens if we increase the number of simulated as well as observed agents in our model.

```python
# List of seeds, includes true simulation seed of 132.
seeds = list(range(120, 140, 1))
```

```python
# Define other inputs for criterion function
criterion_kwargs = dict()
criterion_kwargs["params"] = params_true.copy()
criterion_kwargs["options"] = options
criterion_kwargs["calc_moments"] = calc_moments
criterion_kwargs["replace_nans"] = replace_nans
criterion_kwargs["moments_obs"] = moments_obs
criterion_kwargs["weighting_matrix"] = W
```

The plot below shows the criterion function for different simulation seeds, including the true one. We can seed that different seeds lead to different fits between the simulated and observed data, with some seeds performing worse than others. Only the true seed leads to a function value of 0.

```python
plot_chatter(seeds, criterion_kwargs)
```

##### Changing the Simulation Seed for increasing Sample Size of Simulated Agents

We now repeat this exercise for an increasing number of simulated agents. As the plot below shows, increasing the number of simulated agents reduces the chatter considerably but the criterion function remains consistently above zero. Only the simulated sample of 1000 agents at the true simulation seed reaches a function of zero, since, as discussed before, the simulated sample is identical to the observed one for this calibration.

```python
# Changing the number of agents.
num_agents = [500, 1000, 5000, 10000, 50000]
```

```python
plot_chatter_numagents_sim(seeds, num_agents, criterion_kwargs)
```

##### Changing the Simulation Seed for increasing Sample Size of Simulated *and* Observed Agents

We now increase not only the number of simulated agents but also the number of observed agents in our sample. Doing so decreases the chatter in the criterion function as before but also levels out the function around 0 for all simulation seeds. For large enough samples of observed and simulated agents the choice of simulation seed is thus irrelevant.

```python
plot_chatter_numagents_both(seeds, num_agents, calc_moments, replace_nans, criterion_kwargs)
```

*Note*: In contrast to the previous plot, we can see that the function reaches a value of 0 at the true simulation seed for all quantities of agents. This is the case because we now simultaneously increase the number of both simulated and observed agents while the observed sample remained untouched in the previous plot. Increasing the number of agents simultaneously for both groups creates identical samples for the true simulation seed at each quantity of agents.

#### Moving away from the Optimum

For the next exercise we explore what happens to the criterion function if we move away from the optimum. We do this in a controlled fashion by changing the parameter values for *delta* and *wage_fishing* and fixing the latter in the constraints. Doing so should lead *delta* to move away from its optimal value of 0.95 during optimization.

```python
params_cand.loc["wage_fishing", "value"] = 0.072
params_cand
```

      value
      lower
      upper

      category
      name

      delta
      delta
      0.930
      0.890
      0.980

      wage_fishing
      exp_fishing
      0.072
      0.066
      0.072

      nonpec_fishing
      constant
      -0.100
      -0.110
      -0.095

      nonpec_hammock
      constant
      1.046
      1.040
      1.055

      shocks_sdcorr
      sd_fishing
      0.010
      0.000
      0.100

      sd_hammock
      0.010
      0.000
      0.100

      corr_hammock_fishing
      0.000
      0.000
      0.100

The optimizer cannot retrieve the true parameter and does not reach a value of 0.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options_new_seed, calc_moments, replace_nans, moments_obs, W
)
rslt_wrong_fix = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="nlopt_bobyqa",
    constraints=constr,
)
rslt_wrong_fix[0]
```

    {'status': 'success',
     'fitness': 1019.176716250516,
     'x': [0.9247609208404869],
     'n_evaluations': 24,
     'n_evaluations_jacobian': 0,
     'n_evaluations_hessian': 0,
     'n_constraints': 0,
     'n_constraints_equality': 0,
     'n_constraints_inequality': 0}

##### Retrieving the true Parameter Vector

We now repeat the estimation with the new parameter vector and free up *wage_fishing* to retrieve the optimal values for both parameters.

The parameter for *wage_fishing* is still 0.072 since we fixed it for the prior estimation:

```python
params_cand.loc[:, "value"] = rslt_wrong_fix[1][["value"]]
params_cand
```

      value
      lower
      upper

      category
      name

      delta
      delta
      0.924761
      0.890
      0.980

      wage_fishing
      exp_fishing
      0.072000
      0.066
      0.072

      nonpec_fishing
      constant
      -0.100000
      -0.110
      -0.095

      nonpec_hammock
      constant
      1.046000
      1.040
      1.055

      shocks_sdcorr
      sd_fishing
      0.010000
      0.000
      0.100

      sd_hammock
      0.010000
      0.000
      0.100

      corr_hammock_fishing
      0.000000
      0.000
      0.100

We now free up *wage_fishing* in the constraints in addition to *delta*.

```python
# Adjust constraints to free up both delta and wage_fishing.
constr_u = constr_base.copy()
constr_u.remove({"loc": "delta", "type": "fixed"})
constr_u.remove({"loc": "wage_fishing", "type": "fixed"})
```

Freeing up the non-optimal *wage_fishing* improves the estimates. The criterion function value is much closer to 0 and the optimizer manages to retrieve the true parameter values quite closely.

```python
rslt_unfix = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="nlopt_bobyqa",
    constraints=constr_u,
)
rslt_unfix[0]
```

    {'status': 'success',
     'fitness': 15.680230382289366,
     'x': [0.9534870629022841, 0.06973568705354913],
     'n_evaluations': 150,
     'n_evaluations_jacobian': 0,
     'n_evaluations_hessian': 0,
     'n_constraints': 0,
     'n_constraints_equality': 0,
     'n_constraints_inequality': 0}

For easier comparison, we can compute the difference between the true and estimated value:

```python
deviation = params_true["value"] - rslt_unfix[1]["value"]
deviation
```

    category        name
    delta           delta                  -0.003487
    wage_fishing    exp_fishing             0.000264
    nonpec_fishing  constant                0.000000
    nonpec_hammock  constant                0.000000
    shocks_sdcorr   sd_fishing              0.000000
                    sd_hammock              0.000000
                    corr_hammock_fishing    0.000000
    Name: value, dtype: float64

#### Derivative-Based Optimization Algorithm

So far we have used only one optimization algorithm to estimate the parameter vector. The algorithm we used, BOBYQA (Bound Optimization by Quadratic Approximation), is a derivative-free optimization algorithm which works fairly well on our criterion function. As discussed above, our criterion function is a step function that contains plateaus for certain ranges of parameter values. An important implication of this property is that we cannot calculate proper derivatives and thus derivative-based optimization algorithms will not work to estimate the parameters.

To demonstrate this problem, we will now try to estimate the parameters using an optimization algorithm that does use derivatives during optimization.

```python
# Define candidate parameter vector and constraints.
params_cand = params_true.copy()
params_cand.loc["delta", "value"] = 0.93

params_cand["lower"] = [0.89, 0.066, -0.11, 1.04, 0, 0, 0]
params_cand["upper"] = [0.98, 0.072, -0.095, 1.055, 0.1, 0.1, 0.1]

constr = constr_base.copy()
constr.remove({"loc": "delta", "type": "fixed"})
```

We try the L-BFGS-B (Limited-Memory-Broyden-Fletcher-Goldfarb-Shanno with box constraints) algorithm. As shown below, L-BFGS-B fails and the optimizer returns the same parameter vector that we used as an input.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options_new_seed, calc_moments, replace_nans, moments_obs, W
)
rslt = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="scipy_L-BFGS-B",
    constraints=constr,
)
rslt[0]
```

    {'status': 'success',
     'message': b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL',
     'x': [0.93],
     'success': True,
     'fitness': 8160.8543221541695,
     'n_evaluations': 1,
     'jacobian': array([0.]),
     'hessian': None,
     'n_evaluations_jacobian': None,
     'n_evaluations_hessian': None,
     'n_iterations': 0,
     'max_constraints_violations': None,
     'hessian_inverse': <1x1 LbfgsInvHessProduct with dtype=float64>}

### References

* Adda, J., & Cooper, R. W. (2003). *Dynamic Economics: Quantitative Methods and Applications*. MIT press.

* Adda, J., Dustmann, C., & Stevens, K. (2017). The Career Costs of Children. *Journal of Political Economy*, 125(2), 293-337.

* Andrews, I., Gentzkow, M., & Shapiro, J. M. (2017). Measuring the Sensitivity of Parameter Estimates to Estimation Moments. *The Quarterly Journal of Economics*, 132(4), 1553-1592.

* Bruins, M., Duffy, J. A., Keane, M. P., & Smith Jr, A. A. (2018). Generalized Indirect Inference for Discrete Choice Models. *Journal of econometrics*, 205(1), 177-203.

* Davidson, R., & MacKinnon, J. G. (2004). *Econometric Theory and Methods (Vol. 5)*. New York: Oxford University Press.

* Evans, R. W. (2018, July 5). Simulated Method of Moments (SMM) Estimation. Retrieved November 30, 2019, from https://notes.quantecon.org/submission/5b3db2ceb9eab00015b89f93.

* Frazier, D. T., Oka, T., & Zhu, D. (2019). Indirect Inference with a Non-Smooth Criterion Function. *Journal of Econometrics*, 212(2), 623-645.

* Gourieroux, M., & Monfort, D. A. (1996). *Simulation-based econometric methods*. Oxford university press.

* McFadden, D. (1989). A Method of Simulated Moments for Estimation of Discrete Response Models without Numerical Integration. *Econometrica: Journal of the Econometric Society*, 995-1026.