## Notes on the Ammonia model ##

Exact equation from the [source code](https://github.com/pyspeckit/pyspeckit/blob/master/pyspeckit/spectrum/models/ammonia.py#L210):

            population_upperstate = lin_ntot * orthoparafrac * partition/(Z.sum())

            tau_dict[linename] = (population_upperstate /
                                  (1. + np.exp(-h*frq/(kb*tkin) ))*ccms**2 /
                                  (8*np.pi*frq**2) * aval *
                                  (1-np.exp(-h*frq/(kb*tex))) /
                                  (width/ckms*frq*np.sqrt(2*np.pi)) )

\begin{equation}
    \tau = N_{tot} g_{opr} Z_{upper} \frac{A_{ij} c^2}{8\pi\nu^2}
    \left(1-\exp{ \frac{-h \nu}{k_B T_{ex}} } \right)
    \left(1+\exp{\frac{-h \nu}{k_B T_K}}\right)
    \left((2\pi)^{1/2} \nu \sigma_\nu / c\right)^{-1}
\end{equation}

Equation 16 from Rosolowsky et al 2008:

$$N(1,1) = \frac{8 \pi k \nu_0^2}{h c^3} \frac{1}{A_{1,1}} \sqrt{2\pi}\sigma_\nu (T_{ex}-T_{bg})\tau$$

Rearranges to:

$$\tau = N(1,1) \frac{h c^3}{8\pi k \nu_0^2} A_{1,1} \frac{1}{\sqrt{2 \pi} \sigma_\nu} \left(T_{ex}-T_{bg}\right)^{-1}$$

Equation A4 of Friesen et al 2009:

$$N(1,1) = \frac{8\pi\nu^2}{c^2} \frac{g_1}{g_2} \frac{1}{A_{1,1}} \frac{1+\exp\left(-h\nu_0/k_B T_{ex}\right)}{1-\exp\left(-h \nu_0/k_B T_{ex}\right)} \int \tau(\nu) d\nu$$

Equation 98 of Mangum & Shirley 2015:

$$N_{tot} = \frac{3 h}{8 \pi \mu^2 R_i} \frac{J_u(J_u+1)}{K^2}
\frac{Q_{rot}}{g_J g_K g_I} \frac{\exp{E_u/k_B T_{ex}}}{\exp{h \nu/k_B T_{ex}} - 1}
\left[\frac{\int T_R dv}{f\left(J_\nu(T_{ex})-J_\nu{T_B}\right) }\right]$$

## From Scratch ##

$$\tau_\nu = \int \alpha_\nu ds$$
$$\alpha_\nu = \frac{c^2}{8\pi\nu_0^2} \frac{g_u}{g_l} n_l A_{ul} \left(1-\frac{g_l n_u}{g_u n_l}\right) \phi_\nu$$

Excitation temperature:
$$T_{ex} \equiv \frac{h\nu_0/k_b}{\ln \frac{n_l g_u}{n_u g_l} } $$

$\nu_0$ = rest frequency of the line

Rearranges to:
$$ \frac{n_l g_u}{n_u g_l} = \exp\left(\frac{h \nu_0}{k_B T_{ex}}\right)$$

Boltzman  distribution:
$$ \frac{n_u}{n_l} = \frac{g_u}{g_l} \exp\left(\frac{-h \nu_0}{k_B T}\right)$$
where T is a thermal equilibrium temperature

Rearranges to:
$$ 1-\frac{n_u g_l}{n_l g_u} = 1-\exp\left(\frac{-h \nu_0}{k_B T}\right)$$

Column Density
$$N_u \equiv \int n_u ds$$
$$N_l \equiv \int n_l ds$$

Starting to substitute previous equations into each other:

$$\tau_\nu d\nu= \alpha_\nu d\nu = \frac{c^2}{8\pi\nu_0^2} \frac{g_u}{g_l} n_l A_{ul} \left(1-\frac{g_l n_u}{g_u n_l}\right) \phi_\nu d\nu$$

$$\frac{g_u}{g_l}N_l = N_u\exp\left(\frac{h \nu_0}{k_B T_{ex}}\right)$$

First substitution is the Boltzmann distribution, with $T_{ex}$ for T
$$\int \tau_\nu d\nu = \int \frac{c^2}{8\pi\nu_0^2} \frac{g_u}{g_l} n_l A_{ul} \left[  1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right) \right] \phi_\nu d\nu $$

Second is the $N_l$ - $N_u$ relation:
$$\int \tau_\nu d\nu = \frac{c^2}{8\pi\nu_0^2}  A_{ul} N_u\left[\exp\left(\frac{h \nu_0}{k_B T_{ex}}\right)\right] \left[  1-\exp\left(\frac{-h \nu_0}{k_B T}\right) \right] \int \phi_\nu d\nu $$

Then some simplification:
$$\int \tau_\nu d\nu = \frac{c^2}{8\pi\nu_0^2}  A_{ul} N_u \left[  \exp\left(\frac{h \nu_0}{k_B T}\right) - 1 \right] \int \phi_\nu d\nu $$

$$A_{ul} = \frac{64\pi^4\nu_0^3}{3 h c^3} \left|\mu_{lu}\right|^2$$

Becomes, via some manipulation, equation 29 of Mangum & Shirley 2015:
$$N_u = \frac{3 h c}{8\pi^3 \nu \left|\mu_{lu}\right|^2} \left[\exp\left(\frac{h\nu}{k_B T_{ex}}\right) -1\right]^{-1} \int \tau_\nu d\nu$$
where I have used $T_{ex}$ instead of $T$ here because that is one of the substitutions invoked (quietly) in their derivation.  There is some sleight-of-hand regarding assuming $N_l = n_l$ that essentially assumes $T_{ex}$ is constant along the line of sight, but that is fine.

(Equation 30 is the same as this one, but with $dv$ instead of $d\nu$ units)

Solve for tau again (because that's what's implemented in the code):

$$\mathrm{"tau"} = \int \tau_\nu d\nu = N_u \frac{
c^2 A_{ul}}{8\pi\nu_0^2} \left[\exp\left(\frac{h\nu}{k_B T_{ex}}\right) -1\right] $$

The key difference from Erik's derivation is that this is $N_u$, but he has defined $N_{(1,1)}= N_u + N_l$.

So, we get $N_l$ the same way as above:
$$N_l = \frac{8\pi\nu_0^2}{c^2} \frac{g_l}{g_u} A_{ul}^{-1} \left[  1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right) \right]^{-1} \int \tau d\nu$$

$$N_l = \frac{3 h c}{8 \pi^3 \nu \left|\mu_{lu}\right|^2} \frac{g_l}{g_u}  \left[  1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right) \right]^{-1} \int \tau d\nu$$

Added together:
$$N_u + N_l = \frac{3 h c}{8 \pi^3 \nu \left|\mu_{lu}\right|^2} \frac{\frac{g_l}{g_u}  +\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)}{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} \int \tau d\nu$$

We can solve that back for tau, which is what Erik has done:

$$\int \tau d\nu = (N_u + N_l) \frac{8 \pi^3 \nu \left|\mu_{lu}\right|^2}{3 h c} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {\frac{g_l}{g_u}  +\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

$$=(N_u + N_l) \frac{g_u}{g_l}\frac{8 \pi^3 \nu \left|\mu_{lu}\right|^2}{3 h c} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

$$=(N_u + N_l) \frac{g_u}{g_l}\frac{A_{ul}c^2}{8\pi\nu_0^2} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

now identical to Erik's equation.

This is actually a problem, because $N_u$ is related to $N_{tot}$ via the partition function, but there is some double-counting going on if we try to relate $N_{(1,1)}$ to $N_{tot}$ with the same equation.

So, to reformulate the equations in pyspeckit using the appropriate values, we want to use both the partition function (calculated using $T_{kin}$) and $N_u$.

Eqn 31:

$$N_u = N_{tot} \frac{g_u}{Q_{rot}} \exp\left(\frac{-E_u}{k_B T_{kin}}\right)$$

is implemented correctly in pyspeckit:

    population_upperstate = lin_ntot * orthoparafrac * partition/(Z.sum())

where ``partition`` is

$$Z_i(\mathrm{para}) = (2J + 1) \exp\left[ \frac{ -h (B_0 J (J+1) + (C_0-B_0)J^2)}{k_B T_{kin}}\right]$$

$$Z_i(\mathrm{ortho}) = 2(2J + 1) \exp\left[ \frac{ -h (B_0 J (J+1) + (C_0-B_0)J^2)}{k_B T_{kin}}\right]$$

...so I'm assuming (haven't checked) that $E_u = h (B_0 J (J+1) + (C_0-B_0)J^2)$

Note that the leading "2" above cancels out in the Z/sum(Z), so it doesn't matter if it's right or not.  I suspect, though, that the 2 belongs in front of both the para and ortho states, but it should be excluded for the J=0 case.

### An aside by Erik Rosolowsky ###
(Note May 16, 2018: I believe this was incorporated into the above analysis)

EWR: The above equation is problematic because it relates the total column density to the $(J,J)$ state which is the equivalent of the $N_{(1,1)}$ term.  In the notation above $N_{(1,1)} = N_u + N_l$, so to get this right, you need to consider the inversion transition splitting on top of the total energy of the state so that
$$ E_u = h (B_0 J (J+1) + (C_0-B_0)J^2) + \Delta E_{\mathrm{inv}}, g_u = 1 $$
and
$$ E_l = h (B_0 J (J+1) + (C_0-B_0)J^2) - \Delta E_{\mathrm{inv}}, g_l = 1 $$
or, since the splitting is small compared to the rotational energy (1 K compared to > 20 K), then

$$Z_J \approx 2 (2J + 1) \exp\left[ \frac{ -h (B_0 J (J+1) + (C_0-B_0)J^2)}{k_B T_{\mathrm{rot}}}\right]$$

where the leading 2 accounts for the internal inversion states.  Since this 2 appears in all the terms, it cancels out in the sum.  Note that I have also changed the $T_{\mathrm{kin}}$ to $T_{\mathrm{rot}}$ since these two aren't the same and it is the latter which establishes the level populations.

Returning to the above, I would then suggest

$$N_{(J,J)} = N_{tot} \frac{Z_J}{\sum_j Z_j} $$

## Is the treatment of optical depth correct? ##
May 16, 2018: https://github.com/pyspeckit/pyspeckit/blob/725746f517e9bdcc22b83f4f9d6c9b8666e0a99e/pyspeckit/spectrum/models/ammonia.py

In this version, we [compute the optical depth](https://github.com/pyspeckit/pyspeckit/blob/725746f517e9bdcc22b83f4f9d6c9b8666e0a99e/pyspeckit/spectrum/models/ammonia.py#L318) with the code:

```
        for kk,nuo in enumerate(nuoff):
            tauprof_ = (tau_dict[linename] * tau_wts[kk] *
                        np.exp(-(xarr.value+nuo-lines[kk])**2 /
                               (2.0*nuwidth[kk]**2)))
            if return_components:
                components.append(tauprof_)
            tauprof += tauprof_

```

The total tau is normalized such that $\Sigma(\tau_{hf})_{hf} = \tau_{tot}$ for each line, i.e., the hyperfine $\tau$s sum to the tau value specified for the line.

The question Nico raised is, should we be computing the synthetic spectrum as $1-e^{\Sigma(\tau_{hf,\nu})}$ or $\Sigma(1-e^{\tau_{hf,\nu}})$?

The former is correct: we only have one optical depth per frequency bin.  It doesn't matter what line the optical depth comes from.

```python
# This is a test to show what happens if you add lines vs. computing a single optical depth per channel
from pyspeckit.spectrum.models.ammonia_constants import (line_names, freq_dict, aval_dict, ortho_dict,
                                voff_lines_dict, tau_wts_dict)
from astropy import constants
from astropy import units as u
import pylab as pl

linename = 'oneone'

xarr_v = (np.linspace(-25,25,1000)*u.km/u.s)
xarr = xarr_v.to(u.GHz, u.doppler_radio(freq_dict['oneone']*u.Hz))
tauprof = np.zeros(xarr.size)
true_prof = np.zeros(xarr.size)
width = 0.1
xoff_v = 0
ckms = constants.c.to(u.km/u.s).value

pl.figure(figsize=(12,12))
pl.clf()
for ii,tau_tot in enumerate((0.001, 0.1, 1, 10,)):
    tau_dict = {'oneone':tau_tot}

    voff_lines = np.array(voff_lines_dict[linename])
    tau_wts = np.array(tau_wts_dict[linename])

    lines = (1-voff_lines/ckms)*freq_dict[linename]/1e9
    tau_wts = tau_wts / (tau_wts).sum()
    nuwidth = np.abs(width/ckms*lines)
    nuoff = xoff_v/ckms*lines

    # tau array
    tauprof = np.zeros(len(xarr))
    for kk,nuo in enumerate(nuoff):
        tauprof_ = (tau_dict[linename] * tau_wts[kk] *
                    np.exp(-(xarr.value+nuo-lines[kk])**2 /
                           (2.0*nuwidth[kk]**2)))

        tauprof += tauprof_
        true_prof += (1-np.exp(-tauprof_))

    ax = pl.subplot(4,1,ii+1)
    ax.plot(xarr_v, 1 - np.exp(-tauprof), label=str(tau_tot), zorder=20, linewidth=1)
    ax.plot(xarr_v, true_prof, label=str(tau_tot), alpha=0.7, linewidth=2)
    ax.plot(xarr_v, true_prof-(1-np.exp(-tauprof)) - tau_tot/20, linewidth=1)
    pl.title(str(tau_tot))
```

# Below are numerical checks for accuracy #
Some numerical checks: How bad was the use of Tkin instead of Tex in the $\tau$ equation?

$$(N_u + N_l) \frac{g_u}{g_l}\frac{A_{ul}c^2}{8\pi\nu_0^2} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

```python
from astropy import units as u
from astropy import constants
freq = 23*u.GHz
def tau_wrong(tkin, tex):
    return (1-np.exp(-constants.h * freq/(constants.k_B*tkin)))/(1+np.exp(-constants.h * freq/(constants.k_B*tex)))
def tau_right(tex):
    return (1-np.exp(-constants.h * freq/(constants.k_B*tex)))/(1+np.exp(-constants.h * freq/(constants.k_B*tex)))
```

```python
tkin = np.linspace(5,40,101)*u.K
tex = np.linspace(5,40,100)*u.K
```

```python
grid = np.array([[tau_wrong(tk,tx)/tau_right(tx) for tx in tex] for tk in tkin])
```

```python
%matplotlib inline
import pylab as pl
```

```python
pl.imshow(grid, cmap='hot', extent=[5,40,5,40])
pl.xlabel("Tex")
pl.ylabel("Tkin")
pl.colorbar()
pl.contour(tex, tkin, grid, levels=[0.75,1,1/0.75], colors=['w','w','k'])
```

So the error could be 50%-700% over a somewhat reasonable range.  That's bad, and it affects the temperature estimates.  However, the effect on temperature estimates should be pretty small, since each line will be affected in the same way.  The biggest effect will be on the column density.

But, is this error at all balanced by the double-counting problem?

Because we were using the partition function directly, it's not obvious.  I was assuming that we were using the equation with $N_u$ as the leader, but we were using $N_u+N_l$.  i.e., I was using this equation:

$$\int \tau d\nu =(N_u + N_l) \frac{g_u}{g_l}\frac{A_{ul}c^2}{8\pi\nu_0^2} \frac{1-\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} {1  +\frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T_{ex}}\right)} $$

but with $N_u$ in place of $N_u + N_l$.

The magnitude of the error can therefore be estimated by computing $(N_u+N_l)/N_u = 1 + \frac{N_l}{N_u}$.

We can use the Boltzmann distribution to compute this error, then:
$$ \frac{n_u}{n_l} = \frac{g_u}{g_l}\exp\left(\frac{-h \nu_0}{k_B T}\right)$$

```python
def nunlnu_error(Tkin):
    return 1+np.exp(-constants.h * freq / (constants.k_B * Tkin))
```

```python
pl.plot(tkin.value, nunlnu_error(tkin))
```

So we were always off by a factor very close to 2.  The *relative* values of $\tau$ should never have been affected by this issue.

It will be more work to determine exactly how much the T_K and column estimates were affected.

# New work in May 2016: T_{rot} #

Comparing Trot and Tkin.  If we start with the equation that governs level populations,

$$N_u = N_{tot} \frac{g_u}{Q_{rot}} \exp\left(\frac{-E_u}{k_B T_{kin}}\right)$$

we get

$$N_u / N_l = \frac{g_u}{g_l} \exp\left(\frac{-E_u}{k_B T_{kin}} + \frac{E_l}{k_B T_{kin}}\right)$$

where we really mean $T_{rot}$ instead of $T_{kin}$ here as long as we're talking about just two levels.  This gives us a definition

$$T_{rot} = \left(\frac{E_l-E_u}{k_B}\right)\left[\ln\left(\frac{N_u g_l}{N_l g_u}\right)\right]^{-1}$$

which is the rotational temperature for a two-level system... which is just a $T_{ex}$, but governing non-radiatively-coupled levels.

So, for example, if we want to know $T_{rot}$ for the 2-2 and 1-1 lines at $n=10^4$ and $T_{kin}=20$ K:

```python
from pyradex import Radex
from astropy import constants, units as u
R = Radex(species='p-nh3', column=1e13, collider_densities={'pH2':1e4}, temperature=20)
tbl = R(collider_densities={'ph2': 1e4}, temperature=20, column=1e13)
```

    invalid value encountered in true_divide

```python
tbl[8:10]
```

Table length=2

Tex	tau	frequency	upperstateenergy	upperlevel	lowerlevel	upperlevelpop	lowerlevelpop	brightness	T_B

K		GHz	K					erg / (cm2 Hz s sr)	K

float64	float64	float64	float64	bytes6	bytes6	float64	float64	float64	float64

6.78936052483	0.0929414309528	23.6944955	1.14	01_01_	01_01_	0.393279209876	0.464987835394	6.17629086835e-17	0.358064352704

6.53340348878	0.0212672724409	23.7226333	42.32	02_02_	02_02_	0.064641582585	0.0769470693495	1.37491461717e-17	0.0795203378061

```python
# we're comparing the upper states since these are the ones that are emitting photons
trot = (u.Quantity(tbl['upperstateenergy'][8]-tbl['upperstateenergy'][9], u.K) *
        np.log((tbl['upperlevelpop'][9] * R.upperlevel_statisticalweight[8]) /
               (tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[9]))**-1
        )
```

```python
trot
```

$17.776914 \; \mathrm{K}$

```python
tbl['Tex'][8:10].mean()
```

    6.6613820068044447

## Pause here ##

$T_{rot} = 60$ K for $T_{kin}=25$ K?  That doesn't seem right.  Is it possible RADEX is doing something funny with level populations?

# ERIK I SOLVED IT #
I had left out the $^{-1}$ in the code.  Oops!

```python
dT_oneone = -(constants.h * u.Quantity(tbl['frequency'][8], u.GHz)/constants.k_B).to(u.K)
print("delta-T for 1-1_upper - 1-1_lower: {0}".format(dT_oneone))
tex = (dT_oneone *
        np.log((tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[8]) /
               (tbl['lowerlevelpop'][8] * R.upperlevel_statisticalweight[8]))**-1
        )
print("Excitation temperature computed is {0} and should be {1}".format(tex.to(u.K), tbl['Tex'][8]))
```

    delta-T for 1-1_upper - 1-1_lower: -1.1371568105216585 K
    Excitation temperature computed is 6.789343356695167 K and should be 6.789360524825584

## Moving on: comparison to Swift et al 2005 ##

Swift et al 2005 eqn A6

$$T_R = T_K \left[ 1 + \frac{T_K}{T_0} \ln \left[1+0.6\exp\left( -15.7/T_K \right)\right] \right]^{-1}$$

where $T_0=41.18$ K

```python
T0=tbl['upperstateenergy'][9]-tbl['upperstateenergy'][8]
T0
```

    41.18

```python
def tr_swift(tk, T0=T0):
    return tk*(1+tk/T0 * np.log(1+0.6*np.exp(-15.7/tk)))**-1
```

Note that the approximation "works" - gets something near 20 - for positive or negative values of T0 (but see below)

```python
tr_swift(20, T0=-41.18)
```

    22.662533151044173

```python
tr_swift(20, T0=41.18)
```

    17.897313974001626

```python
tr_swift(20, T0=41.5)
```

    17.911834634970091

```python
def trot_radex(column=1e13, density=1e4, tkin=20):
    tbl = R(collider_densities={'ph2': density}, temperature=tkin, column=column)
    trot = (u.Quantity(tbl['upperstateenergy'][8]-tbl['upperstateenergy'][9], u.K) *
        np.log((tbl['upperlevelpop'][9] * R.upperlevel_statisticalweight[8]) /
               (tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[9]))**-1
        )
    return trot
```

RADEX suggests that the *positive* T0 value is the correct one (the negative one appeared correct when incorrectly indexed statistical weights were being used)

```python
trot_radex(tkin=20)
```

    invalid value encountered in true_divide

$17.776914 \; \mathrm{K}$

```python
def tex_radex(column=1e13, density=1e4, tkin=20, lineno=8):
    """ used in tests below """
    tbl = R(collider_densities={'ph2': density}, temperature=tkin, column=column)
    return tbl[lineno]['Tex']
```

```python
%matplotlib inline
import pylab as pl
```

```python
cols = np.logspace(12,15)
trots = [trot_radex(column=c).to(u.K).value for c in cols]
pl.semilogx(cols, trots)
pl.hlines(tr_swift(20), cols.min(), cols.max(), color='k')
pl.xlabel("Column")
pl.ylabel("$T_{rot} (2-2)/(1-1)$")
```

```python
densities = np.logspace(3,9)
trots = [trot_radex(density=n).to(u.K).value for n in densities]
pl.semilogx(densities, trots)
pl.hlines(tr_swift(20), densities.min(), densities.max(), color='k')
pl.xlabel("Volume Density")
pl.ylabel("$T_{rot} (2-2)/(1-1)$")
```

This is the plot that really convinces me that the negative (black curve) value of T0 is the appropriate value to use for this approximation

```python
temperatures = np.linspace(5,40)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
pl.plot(temperatures, trots)
# wrong pl.plot(temperatures, tr_swift(temperatures, T0=-41.18), color='k')
pl.plot(temperatures, tr_swift(temperatures, T0=41.18), color='r')
pl.xlabel("Temperatures")
pl.ylabel("$T_{rot} (2-2)/(1-1)$")
```

```python
temperatures = np.linspace(5,40,50)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
pl.plot(temperatures, np.abs(trots-tr_swift(temperatures, T0=41.18))/trots)
pl.xlabel("Temperatures")
pl.ylabel("$(T_{rot}(\mathrm{RADEX}) - T_{rot}(\mathrm{Swift}))/T_{rot}(\mathrm{RADEX})$")
```

## Tests of cold_ammonia reproducing pyspeckit ammonia spectra ##

```python
from pyspeckit.spectrum.models.tests import test_ammonia
from pyspeckit.spectrum.models import ammonia
```

### Test 1: Use a constant excitatino temperature for all lines

```python
tkin = 20*u.K
trot = trot_radex(tkin=tkin)
print(trot)
```

    17.776914063182385 K

    invalid value encountered in true_divide

```python
spc = test_ammonia.make_synthspec(lte=False, tkin=None, tex=6.66, trot=trot.value, lines=['oneone','twotwo'])
spc.specfit.Registry.add_fitter('cold_ammonia',ammonia.cold_ammonia_model(),6)
spc.specfit(fittype='cold_ammonia', guesses=[23, 5, 13.1, 1, 0.5, 0],
            fixed=[False,False,False,False,False,True])

print("For Tkin={1} -> Trot={2}, pyspeckit's cold_ammonia fitter got:\n{0}".format(spc.specfit.parinfo, tkin, trot))
```

    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]
    For Tkin=20.0 K -> Trot=17.776914063182385 K, pyspeckit's cold_ammonia fitter got:
    Param #0        tkin0 =      19.8365 +/-         107.868   Range:[2.7315,inf)
    Param #1         tex0 =      6.65999 +/-         2277.76   Range:[2.7315,inf)
    Param #2        ntot0 =           13 +/-         107.383   Range:    [5,25]
    Param #3       width0 =          0.5 +/-          2.6139   Range:   [0,inf)
    Param #4      xoff_v0 = -2.45914e-05 +/-               0
    Param #5      fortho0 =            0 (fixed)  Range:     [0,1]

```python
spc.specfit(fittype='cold_ammonia', guesses=[22.80, 6.6, 13.1, 1, 0.5, 0],
            fixed=[False,False,False,False,False,True])
bestfit_coldammonia_temperature = spc.specfit.parinfo[0]
print("The best fit cold ammonia temperature is {0} for an input T_rot={1}".format(bestfit_coldammonia_temperature, trot))
```

    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]
    The best fit cold ammonia temperature is Param #0        tkin0 =      19.8365 +/-     5.72747e-05   Range:[2.7315,inf) for an input T_rot=17.776914063182385 K

### Test 2: Use a different (& appropriate) tex for each level in the input model spectrum

If we use the exact tex for each line in the input model, in principle, the resulting fitted temperature should be more accurate.  However, at present, it looks dramatically incorrect

```python
tex11 = tex_radex(tkin=tkin, lineno=8)
tex22 = tex_radex(tkin=tkin, lineno=9)
print("tex11={0}, tex22={1} for tkin={2}, trot={3}".format(tex11,tex22,tkin,trot))
```

    tex11=6.789360524825584, tex22=6.533403488783305 for tkin=20.0 K, trot=17.776914063182385 K

    invalid value encountered in true_divide

```python
spc = test_ammonia.make_synthspec(lte=False, tkin=None,
                                  tex={'oneone':tex11, 'twotwo':tex22},
                                  trot=trot.value,
                                  lines=['oneone','twotwo'])
spc.specfit.Registry.add_fitter('cold_ammonia',ammonia.cold_ammonia_model(),6)
spc.specfit(fittype='cold_ammonia', guesses=[23, 5, 13.1, 1, 0.5, 0],
            fixed=[False,False,False,False,False,True])

print("For Tkin={1} -> Trot={2}, pyspeckit's cold_ammonia fitter got:\n{0}"
      .format(spc.specfit.parinfo, tkin, trot))
print("The best fit cold ammonia temperature is {0} for an input T_rot={1}"
      .format(bestfit_coldammonia_temperature, trot))
```

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    For Tkin=20.0 K -> Trot=17.776914063182385 K, pyspeckit's cold_ammonia fitter got:
    Param #0        tkin0 =      19.5548 +/-         105.813   Range:[2.7315,inf)
    Param #1         tex0 =       6.7917 +/-         2369.76   Range:[2.7315,inf)
    Param #2        ntot0 =      12.9982 +/-         106.007   Range:    [5,25]
    Param #3       width0 =     0.500008 +/-           2.586   Range:   [0,inf)
    Param #4      xoff_v0 =  0.000212855 +/-               0
    Param #5      fortho0 =            0 (fixed)  Range:     [0,1]
    The best fit cold ammonia temperature is Param #0        tkin0 =      19.8365 +/-     5.72747e-05   Range:[2.7315,inf) for an input T_rot=17.776914063182385 K

### Test 3: compare cold_ammonia to "normal" ammonia model to see why they differ

In a previous iteration of the ammonia model, there was a big (and incorrect) difference between the synthetic spectra from ammonia and cold_ammonia.  This is now something of a regression test for that error, which turned out to be from yet another incorrect indexing of the degeneracy.

```python
tkin = 20*u.K
trot = trot_radex(tkin=tkin)
dT0=41.18
print(tkin * (1 + (tkin.value/dT0)*np.log(1 + 0.6*np.exp(-15.7/tkin.value)))**-1)
print("tkin={0} trot={1} tex11={2} tex22={3}".format(tkin, trot, tex11, tex22))

spc = test_ammonia.make_synthspec(lte=False, tkin=None,
                                  tex={'oneone':tex11, 'twotwo':tex22},
                                  trot=trot.value,
                                  lines=['oneone','twotwo'])
spc_666 = test_ammonia.make_synthspec(lte=False, tkin=None,
                                      tex=6.66,
                                      trot=trot.value,
                                      lines=['oneone','twotwo'])
# this one is guaranteed different because tex = trot
spc_cold = test_ammonia.make_synthspec_cold(tkin=tkin.value,
                                            lines=['oneone','twotwo'])
```

    17.897313974001626 K
    tkin=20.0 K trot=17.776914063182385 K tex11=6.789360524825584 tex22=6.533403488783305
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: Assuming tex=trot [pyspeckit.spectrum.models.ammonia]
    WARNING: No header given.  Creating an empty one.
    WARNING: Assuming tex=trot [pyspeckit.spectrum.models.ammonia]
    WARNING: No header given.  Creating an empty one.

```python
spc[0].plotter(linewidth=3, alpha=0.5)
spc_666[0].plotter(axis=spc[0].plotter.axis, clear=False, color='r', linewidth=1, alpha=0.7)
spc_cold[0].plotter(axis=spc[0].plotter.axis, clear=False, color='b', linewidth=1, alpha=0.7)
```

The red and black look too different to me; they should differ only by a factor of (tex11-6.66)/6.66 or so.  Instead, they differ by a factor of 5-6.

```python
spc[0].data.max(), spc_666[0].data.max()
```

    (0.14522383506633255, 0.14324536653850356)

```python
spc[1].plotter()
spc_666[1].plotter(axis=spc[1].plotter.axis, clear=False, color='r')
spc_cold[1].plotter(axis=spc[1].plotter.axis, clear=False, color='b')
```

## RADEX analysis: T_rot vs T_kin vs T_ex ##

```python
temperatures = np.linspace(5,40)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
tex11s = np.array([tex_radex(tkin=t, lineno=8) for t in temperatures])
tex22s = np.array([tex_radex(tkin=t, lineno=9) for t in temperatures])
pl.plot(trots, tex11s)
pl.plot(trots, tex22s)
#pl.plot(tr_swift(temperatures), color='k')
pl.ylabel("$T_{ex}$")
pl.xlabel("$T_{rot} (2-2)/(1-1)$")
```

Apparently there are some discreteness problems but the ratio changes very little.

```python
temperatures = np.linspace(5,40)
trots = [trot_radex(tkin=t).to(u.K).value for t in temperatures]
tex11s = np.array([tex_radex(tkin=t, lineno=8) for t in temperatures])
tex22s = np.array([tex_radex(tkin=t, lineno=9) for t in temperatures])
pl.plot(trots, tex11s/tex22s)
#pl.plot(tr_swift(temperatures), color='k')
pl.ylabel("$T_{ex} (2-2)/(1-1)$")
pl.xlabel("$T_{rot} (2-2)/(1-1)$")
```

## run pyspeckit tests

```python
from pyspeckit.spectrum.models.tests import test_ammonia
test_ammonia.test_ammonia_parlimits()
test_ammonia.test_ammonia_parlimits_fails()
test_ammonia.test_cold_ammonia()
test_ammonia.test_self_fit()
```

    WARNING: No header given.  Creating an empty one.

    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,50 [pyspeckit.spectrum.interactive]

    WARNING: gnorm=0.   wa2=[ 0.  0.  0.  0.  0.  0.] [pyspeckit.mpfit.mpfit]
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,50 [pyspeckit.spectrum.interactive]
    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,753 [pyspeckit.spectrum.interactive]

    tex > trot in the ammonia model.  This is unphysical and suggests that you may need to constrain tex.  See ammonia_model_restricted_tex.

## More extensive (& expensive) tests: recovered Tkin

### 1. Check the recovered temperature as a function of input temperature using RADEX to simulate "real" data

```python
temperatures = np.array((10,15,20,25,30,35,40))
recovered_tkin = {}
recovered_column = {}
for tkin in temperatures:

    tbl = R(collider_densities={'ph2': 1e4}, temperature=tkin, column=1e13)
    tex11 = tbl['Tex'][8]
    tex22 = tbl['Tex'][9]
    trot = (u.Quantity(tbl['upperstateenergy'][8]-tbl['upperstateenergy'][9], u.K) *
            np.log((tbl['upperlevelpop'][9] * R.upperlevel_statisticalweight[8]) /
                   (tbl['upperlevelpop'][8] * R.upperlevel_statisticalweight[9]))**-1
            )

    spc = test_ammonia.make_synthspec(lte=False, tkin=None,
                                      tex={'oneone':tex11, 'twotwo':tex22},
                                      trot=trot.value,
                                      lines=['oneone','twotwo'])
    spc.specfit.Registry.add_fitter('cold_ammonia',ammonia.cold_ammonia_model(),6)
    spc.specfit(fittype='cold_ammonia', guesses=[23, 5, 13.1, 1, 0.5, 0],
                fixed=[False,False,False,False,False,True])

    recovered_tkin[tkin] = spc.specfit.parinfo['tkin0'].value
    recovered_column[tkin] = spc.specfit.parinfo['ntot0'].value
```

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

    invalid value encountered in true_divide
    WARNING: No header given.  Creating an empty one.
    WARNING: No header given.  Creating an empty one.

    INFO: Creating spectra [pyspeckit.spectrum.classes]
    INFO: Concatenating data [pyspeckit.spectrum.classes]
    INFO: Left region selection unchanged.  xminpix, xmaxpix: 0,502 [pyspeckit.spectrum.interactive]

```python
pl.xlabel("$T_K$")
pl.ylabel("Fitted $T_K$ from cold_ammonia")
pl.plot(recovered_tkin.keys(), recovered_tkin.values(), 'o')
pl.plot(temperatures, temperatures)
```

```python
pl.xlabel("$T_K$")
pl.ylabel("$|T_K-T_{fit}|/T_K$")
inp = np.array(list(recovered_tkin.keys()), dtype='float')
rslt = np.array(list(recovered_tkin.values()), dtype='float')
pl.plot(inp, np.abs(rslt-inp)/rslt, 'o')
```

### 2. Check the recovery as a function of column density

```python
pl.xlabel("$N(NH_3)$")
pl.ylabel("Fitted $N(NH_3)$ from cold_ammonia")
pl.plot(recovered_column.keys(), recovered_column.values(), 'o')
pl.plot(temperatures, temperatures*0+13)
```

```python

```


## lab 04-1: multi-variable linear regression을 TensorFlow에서 구현하기

### Hypothesis using matrix

\begin{align}
H(x_1, x_2, x_3) = x_1w_1 + x_2w_2 + x_3w_3 \\
\end{align}

| x1 | x2 | x3  | Y   |
| -- | -- | --- | --- |
| 73 | 80 | 75  | 152 |
| 93 | 88 | 93  | 185 |
| 89 | 91 | 90  | 180 |
| 96 | 98 | 100 | 196 |
| 73 | 66 | 70  | 142 |

변수 3개(x1, x2, x3)을 이용하여 `Y`(결과)를 예측

```python
import tensorflow as tf

x1_data = [73., 93., 89., 96., 73.]
x2_data = [80., 88., 91., 98., 66.]
x3_data = [75., 93., 90., 100., 70.]
y_data = [152., 185., 180., 196., 142.]

# placeholder for a tensor that will be always fed.
x1 = tf.placeholder(tf.float32)
x2 = tf.placeholder(tf.float32)
x3 = tf.placeholder(tf.float32)

Y = tf.placeholder(tf.float32)

w1 = tf.Variable(tf.random_normal([1]), name='weight1')
w2 = tf.Variable(tf.random_normal([1]), name='weight2')
w3 = tf.Variable(tf.random_normal([1]), name='weight3')
b = tf.Variable(tf.random_normal([1]), name='bias')

hypothesis = x1 * w1 + x2 *w2 + x3 * w3 + b

# cost/Loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize. Need a very small learning rate for this data set
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)

# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())

for step in range(2001):
    cost_val, hy_val, _ = sess.run([cost, hypothesis, train],
                                  feed_dict={x1: x1_data,
                                             x2: x2_data,
                                             x3: x3_data,
                                             Y : y_data})
    if step % 10 == 0:
        print(step, "Cost:", cost_val, "\nPrediction:\n", hy_val)
```

    /anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      from ._conv import register_converters as _register_converters

    0 Cost: 7444.396
    Prediction:
     [73.21022  93.897156 89.57611  96.45675  73.962616]
    10 Cost: 2.2866924
    Prediction:
     [149.52972 185.61728 179.95482 194.87642 143.91982]
    20 Cost: 2.2078712
    Prediction:
     [149.76602 185.8911  180.2299  195.17554 144.12651]
    30 Cost: 2.1972535
    Prediction:
     [149.77211 185.88824 180.23238 195.17775 144.1222 ]
    40 Cost: 2.186692
    Prediction:
     [149.77747 185.88455 180.234   195.17906 144.11723]
    50 Cost: 2.1761773
    Prediction:
     [149.78282 185.88087 180.23561 195.18037 144.11229]
    60 Cost: 2.165716
    Prediction:
     [149.78815 185.87718 180.23721 195.18167 144.10735]
    70 Cost: 2.1553218
    Prediction:
     [149.79349 185.87354 180.23885 195.18298 144.10245]
    80 Cost: 2.145004
    Prediction:
     [149.79877 185.86989 180.24045 195.18427 144.09753]
    90 Cost: 2.1347032
    Prediction:
     [149.80406 185.86623 180.24205 195.18556 144.09264]
    100 Cost: 2.1244762
    Prediction:
     [149.80936 185.86261 180.24365 195.18684 144.08777]
    110 Cost: 2.1143155
    Prediction:
     [149.8146  185.859   180.24524 195.18813 144.0829 ]
    120 Cost: 2.1041794
    Prediction:
     [149.81985 185.85536 180.24683 195.18944 144.07805]
    130 Cost: 2.0941243
    Prediction:
     [149.82509 185.85178 180.24843 195.19072 144.07321]
    140 Cost: 2.084106
    Prediction:
     [149.8303  185.84819 180.25    195.192   144.06837]
    150 Cost: 2.0741415
    Prediction:
     [149.83553 185.8446  180.25159 195.19328 144.06357]
    160 Cost: 2.0642438
    Prediction:
     [149.84071 185.84103 180.25316 195.19453 144.05876]
    170 Cost: 2.0544047
    Prediction:
     [149.84589 185.83746 180.25471 195.1958  144.05399]
    180 Cost: 2.0445814
    Prediction:
     [149.85106 185.83391 180.25629 195.19708 144.0492 ]
    190 Cost: 2.0348637
    Prediction:
     [149.85619 185.83037 180.25786 195.19833 144.04443]
    200 Cost: 2.025165
    Prediction:
     [149.86133 185.82683 180.25941 195.1996  144.03969]
    210 Cost: 2.0155265
    Prediction:
     [149.86646 185.82332 180.26096 195.20087 144.03496]
    220 Cost: 2.0059502
    Prediction:
     [149.87154 185.8198  180.2625  195.20209 144.03021]
    230 Cost: 1.9963982
    Prediction:
     [149.87665 185.81628 180.26405 195.20337 144.02551]
    240 Cost: 1.986913
    Prediction:
     [149.88171 185.81277 180.26556 195.20459 144.0208 ]
    250 Cost: 1.9774872
    Prediction:
     [149.8868  185.80931 180.26712 195.20584 144.01613]
    260 Cost: 1.9681065
    Prediction:
     [149.89183 185.80582 180.26866 195.20708 144.01144]
    270 Cost: 1.9587536
    Prediction:
     [149.8969  185.80235 180.27019 195.20833 144.00679]
    280 Cost: 1.9494833
    Prediction:
     [149.9019  185.79889 180.27168 195.20955 144.00214]
    290 Cost: 1.9402473
    Prediction:
     [149.90692 185.79547 180.27322 195.2108  143.9975 ]
    300 Cost: 1.931061
    Prediction:
     [149.91193 185.79202 180.27472 195.212   143.99287]
    310 Cost: 1.9219244
    Prediction:
     [149.9169  185.78857 180.27623 195.21326 143.98827]
    320 Cost: 1.9128532
    Prediction:
     [149.92188 185.78517 180.27774 195.21446 143.98367]
    330 Cost: 1.9038101
    Prediction:
     [149.92682 185.78175 180.27925 195.21568 143.97906]
    340 Cost: 1.8948357
    Prediction:
     [149.93176 185.77837 180.28075 195.2169  143.9745 ]
    350 Cost: 1.885896
    Prediction:
     [149.93669 185.77498 180.28223 195.21812 143.96994]
    360 Cost: 1.87701
    Prediction:
     [149.9416  185.77158 180.2837  195.21931 143.9654 ]
    370 Cost: 1.8681513
    Prediction:
     [149.94652 185.76822 180.28522 195.22055 143.96085]
    380 Cost: 1.8593712
    Prediction:
     [149.9514  185.76486 180.2867  195.22176 143.95633]
    390 Cost: 1.850614
    Prediction:
     [149.95627 185.76149 180.28816 195.22295 143.9518 ]
    400 Cost: 1.8419205
    Prediction:
     [149.96114 185.75813 180.28963 195.22414 143.94731]
    410 Cost: 1.8332698
    Prediction:
     [149.96597 185.7548  180.2911  195.22534 143.94281]
    420 Cost: 1.8246619
    Prediction:
     [149.97081 185.75148 180.29257 195.22652 143.93832]
    430 Cost: 1.8161064
    Prediction:
     [149.97563 185.74817 180.29402 195.22772 143.93387]
    440 Cost: 1.8075842
    Prediction:
     [149.98044 185.74486 180.29547 195.2289  143.9294 ]
    450 Cost: 1.79911
    Prediction:
     [149.98524 185.74156 180.29694 195.23009 143.92496]
    460 Cost: 1.7906916
    Prediction:
     [149.99002 185.73825 180.29837 195.23126 143.92053]
    470 Cost: 1.7823124
    Prediction:
     [149.99478 185.73495 180.2998  195.23244 143.9161 ]
    480 Cost: 1.7739916
    Prediction:
     [149.99954 185.7317  180.30127 195.23363 143.91171]
    490 Cost: 1.7657068
    Prediction:
     [150.00429 185.72844 180.3027  195.23479 143.90732]
    500 Cost: 1.757473
    Prediction:
     [150.009   185.72519 180.30414 195.23595 143.90292]
    510 Cost: 1.7492632
    Prediction:
     [150.01373 185.72194 180.30557 195.23714 143.89856]
    520 Cost: 1.741095
    Prediction:
     [150.01845 185.71869 180.30699 195.2383  143.8942 ]
    530 Cost: 1.7329819
    Prediction:
     [150.02313 185.71547 180.30841 195.23946 143.88983]
    540 Cost: 1.7249329
    Prediction:
     [150.0278  185.71225 180.30981 195.24059 143.8855 ]
    550 Cost: 1.716905
    Prediction:
     [150.03247 185.70905 180.31123 195.24178 143.88118]
    560 Cost: 1.708923
    Prediction:
     [150.03712 185.70583 180.31264 195.2429  143.87686]
    570 Cost: 1.700997
    Prediction:
     [150.04175 185.70264 180.31404 195.24406 143.87256]
    580 Cost: 1.693092
    Prediction:
     [150.04639 185.69945 180.31544 195.24522 143.86827]
    590 Cost: 1.685236
    Prediction:
     [150.05101 185.69629 180.31683 195.24635 143.86398]
    600 Cost: 1.6774279
    Prediction:
     [150.0556  185.6931  180.31822 195.24748 143.85971]
    610 Cost: 1.6696533
    Prediction:
     [150.0602  185.68994 180.31963 195.24864 143.85545]
    620 Cost: 1.6619431
    Prediction:
     [150.06477 185.68681 180.32101 195.24976 143.85121]
    630 Cost: 1.6542494
    Prediction:
     [150.06935 185.68367 180.32239 195.2509  143.84698]
    640 Cost: 1.6466067
    Prediction:
     [150.0739  185.68053 180.32378 195.25203 143.84276]
    650 Cost: 1.6390197
    Prediction:
     [150.07841 185.67738 180.32513 195.25314 143.83855]
    660 Cost: 1.631459
    Prediction:
     [150.08293 185.67427 180.3265  195.25427 143.83434]
    670 Cost: 1.623929
    Prediction:
     [150.08746 185.67117 180.32788 195.25542 143.83015]
    680 Cost: 1.6164379
    Prediction:
     [150.09198 185.66808 180.32924 195.25653 143.82597]
    690 Cost: 1.6090024
    Prediction:
     [150.09645 185.66496 180.3306  195.25763 143.8218 ]
    700 Cost: 1.6016032
    Prediction:
     [150.10094 185.6619  180.33194 195.25874 143.81764]
    710 Cost: 1.5942478
    Prediction:
     [150.10538 185.65881 180.33328 195.25984 143.81348]
    720 Cost: 1.586923
    Prediction:
     [150.10986 185.65576 180.33464 195.26097 143.80937]
    730 Cost: 1.5796349
    Prediction:
     [150.1143  185.65271 180.336   195.26208 143.80524]
    740 Cost: 1.5724044
    Prediction:
     [150.11873 185.64967 180.33733 195.26318 143.80113]
    750 Cost: 1.5652078
    Prediction:
     [150.12314 185.6466  180.33867 195.26425 143.79703]
    760 Cost: 1.558038
    Prediction:
     [150.12755 185.64359 180.34    195.26537 143.79294]
    770 Cost: 1.5509208
    Prediction:
     [150.13194 185.64056 180.34134 195.26645 143.78886]
    780 Cost: 1.5438364
    Prediction:
     [150.1363  185.63756 180.34265 195.26756 143.78479]
    790 Cost: 1.5367777
    Prediction:
     [150.14069 185.63454 180.34398 195.26865 143.78073]
    800 Cost: 1.5297611
    Prediction:
     [150.14503 185.63152 180.34528 195.26971 143.77667]
    810 Cost: 1.5227896
    Prediction:
     [150.14938 185.62854 180.34659 195.27081 143.77264]
    820 Cost: 1.5158542
    Prediction:
     [150.15373 185.62556 180.34792 195.2719  143.76863]
    830 Cost: 1.508959
    Prediction:
     [150.15804 185.62257 180.34923 195.27296 143.7646 ]
    840 Cost: 1.5021021
    Prediction:
     [150.16235 185.61963 180.35052 195.27403 143.7606 ]
    850 Cost: 1.4952784
    Prediction:
     [150.16664 185.61665 180.35182 195.2751  143.7566 ]
    860 Cost: 1.4884869
    Prediction:
     [150.17093 185.61371 180.35312 195.27618 143.75262]
    870 Cost: 1.481741
    Prediction:
     [150.17519 185.61075 180.35442 195.27724 143.74864]
    880 Cost: 1.4750115
    Prediction:
     [150.17946 185.60782 180.35568 195.2783  143.74467]
    890 Cost: 1.4683406
    Prediction:
     [150.18372 185.6049  180.35698 195.27937 143.74074]
    900 Cost: 1.4616933
    Prediction:
     [150.18794 185.60197 180.35826 195.28044 143.73679]
    910 Cost: 1.4551
    Prediction:
     [150.19218 185.59909 180.35954 195.2815  143.73288]
    920 Cost: 1.4485308
    Prediction:
     [150.1964  185.59618 180.36081 195.28253 143.72896]
    930 Cost: 1.4419882
    Prediction:
     [150.20059 185.59326 180.36209 195.28358 143.72504]
    940 Cost: 1.4354874
    Prediction:
     [150.20477 185.59038 180.36334 195.28464 143.72113]
    950 Cost: 1.4290103
    Prediction:
     [150.20897 185.5875  180.36461 195.28568 143.71724]
    960 Cost: 1.4225893
    Prediction:
     [150.21313 185.58466 180.36588 195.28673 143.71336]
    970 Cost: 1.4162041
    Prediction:
     [150.21729 185.58179 180.36714 195.28777 143.7095 ]
    980 Cost: 1.4098374
    Prediction:
     [150.22144 185.57893 180.3684  195.2888  143.70564]
    990 Cost: 1.4035069
    Prediction:
     [150.22557 185.57608 180.36963 195.28984 143.7018 ]
    1000 Cost: 1.3972118
    Prediction:
     [150.2297  185.57324 180.3709  195.29088 143.69797]
    1010 Cost: 1.3909466
    Prediction:
     [150.23381 185.5704  180.37212 195.29189 143.69412]
    1020 Cost: 1.3847395
    Prediction:
     [150.2379  185.56758 180.37337 195.29292 143.69032]
    1030 Cost: 1.3785343
    Prediction:
     [150.24199 185.56476 180.37459 195.29396 143.68651]
    1040 Cost: 1.3723568
    Prediction:
     [150.24608 185.56194 180.37582 195.29497 143.6827 ]
    1050 Cost: 1.3662355
    Prediction:
     [150.25015 185.55914 180.37706 195.296   143.67892]
    1060 Cost: 1.3601433
    Prediction:
     [150.25421 185.55635 180.37828 195.29703 143.67516]
    1070 Cost: 1.3540882
    Prediction:
     [150.25826 185.55357 180.37952 195.29803 143.67139]
    1080 Cost: 1.348065
    Prediction:
     [150.26227 185.55078 180.38074 195.29904 143.66762]
    1090 Cost: 1.3420637
    Prediction:
     [150.26631 185.54802 180.38196 195.30005 143.66388]
    1100 Cost: 1.3361015
    Prediction:
     [150.27031 185.54526 180.38315 195.30107 143.66014]
    1110 Cost: 1.3301713
    Prediction:
     [150.27432 185.54251 180.38437 195.30206 143.65642]
    1120 Cost: 1.3242692
    Prediction:
     [150.2783  185.53976 180.38556 195.30307 143.6527 ]
    1130 Cost: 1.3184025
    Prediction:
     [150.28227 185.537   180.38676 195.30408 143.64899]
    1140 Cost: 1.3125622
    Prediction:
     [150.28625 185.53427 180.38797 195.30507 143.6453 ]
    1150 Cost: 1.3067654
    Prediction:
     [150.29018 185.53152 180.38914 195.30605 143.64159]
    1160 Cost: 1.3009866
    Prediction:
     [150.29414 185.52882 180.39034 195.30705 143.63792]
    1170 Cost: 1.2952467
    Prediction:
     [150.29808 185.52612 180.39154 195.30806 143.63426]
    1180 Cost: 1.2895386
    Prediction:
     [150.302   185.52342 180.39273 195.30904 143.6306 ]
    1190 Cost: 1.2838535
    Prediction:
     [150.30594 185.52072 180.39394 195.31003 143.62697]
    1200 Cost: 1.2782018
    Prediction:
     [150.30983 185.51802 180.3951  195.311   143.62332]
    1210 Cost: 1.2725726
    Prediction:
     [150.31374 185.51537 180.39627 195.312   143.61969]
    1220 Cost: 1.2669827
    Prediction:
     [150.31761 185.51268 180.39745 195.31299 143.61607]
    1230 Cost: 1.2614424
    Prediction:
     [150.32147 185.51003 180.39862 195.31395 143.61247]
    1240 Cost: 1.2559019
    Prediction:
     [150.32533 185.50735 180.39978 195.31491 143.60886]
    1250 Cost: 1.2504075
    Prediction:
     [150.3292  185.50471 180.40094 195.31589 143.60529]
    1260 Cost: 1.2449313
    Prediction:
     [150.33302 185.50206 180.4021  195.31688 143.6017 ]
    1270 Cost: 1.2394952
    Prediction:
     [150.33685 185.49942 180.40326 195.31783 143.59813]
    1280 Cost: 1.2340796
    Prediction:
     [150.34067 185.49678 180.40442 195.31879 143.59456]
    1290 Cost: 1.228692
    Prediction:
     [150.34448 185.49417 180.40556 195.31975 143.591  ]
    1300 Cost: 1.2233301
    Prediction:
     [150.34828 185.49153 180.4067  195.32071 143.58746]
    1310 Cost: 1.2180153
    Prediction:
     [150.35207 185.48895 180.40785 195.32169 143.58394]
    1320 Cost: 1.2127165
    Prediction:
     [150.35583 185.48634 180.40898 195.32262 143.5804 ]
    1330 Cost: 1.2074543
    Prediction:
     [150.35959 185.48375 180.41013 195.3236  143.57689]
    1340 Cost: 1.2021974
    Prediction:
     [150.36337 185.48116 180.41125 195.32455 143.5734 ]
    1350 Cost: 1.1970017
    Prediction:
     [150.3671  185.47856 180.41238 195.32545 143.56989]
    1360 Cost: 1.1917948
    Prediction:
     [150.37085 185.476   180.41351 195.32643 143.56639]
    1370 Cost: 1.1866564
    Prediction:
     [150.37456 185.47343 180.41463 195.32738 143.56293]
    1380 Cost: 1.1815315
    Prediction:
     [150.37827 185.47087 180.41577 195.32831 143.55945]
    1390 Cost: 1.1764206
    Prediction:
     [150.38199 185.46832 180.41687 195.32925 143.556  ]
    1400 Cost: 1.1713502
    Prediction:
     [150.38567 185.46577 180.41798 195.33018 143.55254]
    1410 Cost: 1.1663027
    Prediction:
     [150.38936 185.46323 180.4191  195.33112 143.5491 ]
    1420 Cost: 1.1612841
    Prediction:
     [150.39304 185.46071 180.42021 195.33205 143.54567]
    1430 Cost: 1.1562954
    Prediction:
     [150.3967  185.45818 180.42133 195.33298 143.54225]
    1440 Cost: 1.1513275
    Prediction:
     [150.40034 185.45566 180.42241 195.33391 143.53883]
    1450 Cost: 1.146389
    Prediction:
     [150.40399 185.45314 180.42351 195.33482 143.53543]
    1460 Cost: 1.1414802
    Prediction:
     [150.40762 185.45064 180.42462 195.33577 143.53204]
    1470 Cost: 1.1365807
    Prediction:
     [150.41125 185.44815 180.4257  195.33669 143.52864]
    1480 Cost: 1.1317335
    Prediction:
     [150.41487 185.44568 180.4268  195.33762 143.52528]
    1490 Cost: 1.1269041
    Prediction:
     [150.41846 185.44318 180.42789 195.3385  143.5219 ]
    1500 Cost: 1.1220871
    Prediction:
     [150.42206 185.44069 180.42897 195.33943 143.51854]
    1510 Cost: 1.1173115
    Prediction:
     [150.42564 185.43823 180.43005 195.34035 143.5152 ]
    1520 Cost: 1.11255
    Prediction:
     [150.42923 185.43578 180.43112 195.34125 143.51186]
    1530 Cost: 1.1078186
    Prediction:
     [150.4328  185.43332 180.4322  195.34216 143.50853]
    1540 Cost: 1.1031231
    Prediction:
     [150.43634 185.43086 180.43329 195.34306 143.5052 ]
    1550 Cost: 1.098439
    Prediction:
     [150.43988 185.4284  180.43434 195.34395 143.50188]
    1560 Cost: 1.0937725
    Prediction:
     [150.44342 185.42598 180.43541 195.34488 143.49857]
    1570 Cost: 1.0891542
    Prediction:
     [150.44695 185.42355 180.43648 195.34578 143.49529]
    1580 Cost: 1.0845461
    Prediction:
     [150.45045 185.42113 180.43753 195.34668 143.49199]
    1590 Cost: 1.0799593
    Prediction:
     [150.45398 185.41872 180.4386  195.34756 143.48871]
    1600 Cost: 1.0754064
    Prediction:
     [150.45746 185.41629 180.43962 195.34843 143.48543]
    1610 Cost: 1.0708756
    Prediction:
     [150.46095 185.4139  180.44069 195.34935 143.48218]
    1620 Cost: 1.0663693
    Prediction:
     [150.46443 185.4115  180.44176 195.35022 143.47891]
    1630 Cost: 1.0618893
    Prediction:
     [150.4679  185.4091  180.4428  195.35112 143.47568]
    1640 Cost: 1.05742
    Prediction:
     [150.47136 185.40672 180.44382 195.35199 143.47243]
    1650 Cost: 1.0530033
    Prediction:
     [150.4748  185.40434 180.44489 195.35287 143.46921]
    1660 Cost: 1.0485767
    Prediction:
     [150.47826 185.402   180.4459  195.35376 143.46599]
    1670 Cost: 1.0441878
    Prediction:
     [150.48167 185.39961 180.44695 195.35464 143.46277]
    1680 Cost: 1.039823
    Prediction:
     [150.48509 185.39725 180.44797 195.35551 143.45956]
    1690 Cost: 1.0354841
    Prediction:
     [150.48851 185.3949  180.449   195.35638 143.45638]
    1700 Cost: 1.0311816
    Prediction:
     [150.4919  185.39255 180.45003 195.35725 143.4532 ]
    1710 Cost: 1.026886
    Prediction:
     [150.4953  185.39023 180.45105 195.35812 143.45003]
    1720 Cost: 1.022605
    Prediction:
     [150.49869 185.3879  180.45207 195.35901 143.44685]
    1730 Cost: 1.0183613
    Prediction:
     [150.50204 185.38554 180.45306 195.35983 143.44368]
    1740 Cost: 1.0141401
    Prediction:
     [150.50542 185.38324 180.45412 195.36072 143.44054]
    1750 Cost: 1.0099428
    Prediction:
     [150.50876 185.38092 180.45511 195.36157 143.4374 ]
    1760 Cost: 1.0057514
    Prediction:
     [150.51212 185.37863 180.45612 195.36244 143.43425]
    1770 Cost: 1.0015968
    Prediction:
     [150.51546 185.37631 180.45712 195.3633  143.43114]
    1780 Cost: 0.99745846
    Prediction:
     [150.51878 185.37402 180.45813 195.36415 143.42801]
    1790 Cost: 0.9933399
    Prediction:
     [150.52211 185.37173 180.45914 195.365   143.4249 ]
    1800 Cost: 0.9892602
    Prediction:
     [150.5254  185.36945 180.46013 195.36584 143.4218 ]
    1810 Cost: 0.98517835
    Prediction:
     [150.5287  185.36717 180.4611  195.3667  143.41869]
    1820 Cost: 0.9811322
    Prediction:
     [150.53201 185.36491 180.46211 195.36755 143.41562]
    1830 Cost: 0.97710514
    Prediction:
     [150.5353  185.36264 180.4631  195.3684  143.41254]
    1840 Cost: 0.9731108
    Prediction:
     [150.53856 185.3604  180.4641  195.36923 143.40947]
    1850 Cost: 0.96912825
    Prediction:
     [150.54181 185.35814 180.46506 195.37007 143.4064 ]
    1860 Cost: 0.9651655
    Prediction:
     [150.54507 185.35588 180.46605 195.3709  143.40335]
    1870 Cost: 0.96122473
    Prediction:
     [150.54832 185.35365 180.46703 195.37175 143.40031]
    1880 Cost: 0.95731306
    Prediction:
     [150.55156 185.35144 180.468   195.37257 143.39728]
    1890 Cost: 0.95340216
    Prediction:
     [150.5548  185.34921 180.46898 195.37343 143.39424]
    1900 Cost: 0.9495268
    Prediction:
     [150.55801 185.34698 180.46996 195.37425 143.39122]
    1910 Cost: 0.94566953
    Prediction:
     [150.56123 185.34477 180.47092 195.37508 143.38821]
    1920 Cost: 0.94182956
    Prediction:
     [150.56442 185.34254 180.47188 195.3759  143.3852 ]
    1930 Cost: 0.938021
    Prediction:
     [150.56761 185.34035 180.47284 195.37672 143.3822 ]
    1940 Cost: 0.93422854
    Prediction:
     [150.57079 185.33815 180.47379 195.37755 143.37921]
    1950 Cost: 0.93045455
    Prediction:
     [150.57397 185.33597 180.47475 195.37836 143.37624]
    1960 Cost: 0.9266957
    Prediction:
     [150.57715 185.33379 180.47571 195.37918 143.37326]
    1970 Cost: 0.922966
    Prediction:
     [150.5803  185.33162 180.47667 195.38    143.3703 ]
    1980 Cost: 0.9192538
    Prediction:
     [150.58345 185.32944 180.47762 195.38081 143.36734]
    1990 Cost: 0.915562
    Prediction:
     [150.5866  185.32727 180.47856 195.38162 143.3644 ]
    2000 Cost: 0.9118868
    Prediction:
     [150.5897  185.3251  180.47948 195.38242 143.36143]

 <br/>

 ### Matrix
\begin{align}
(x_1 x_2 x_3) \cdot \begin{pmatrix}w_1\\\w_2\\\w_3\end{pmatrix} = (x_1w_1 + x_2w_2 + x_3w_3) \\
\end{align}

\begin{align}
H(X) = XW \\
\end{align}

```python
x_data = [
    [73., 80., 75.],
    [93., 88., 93.],
    [89., 91., 90.],
    [96., 98., 100.],
    [73., 66., 70.],
]
y_data = [
    [152.],
    [185.],
    [180.],
    [196.],
    [142.],
]

# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 3])
Y = tf.placeholder(tf.float32, shape=[None, 1])

W = tf.Variable(tf.random_normal([3, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')

# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/Loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)

# Launch the graph in a session.
sess = tf.Session()
# Initalizes global variables in the graph.
sess.run(tf.global_variables_initializer())

for step in range(2001):
    cost_val, hy_val, _ = sess.run([cost, hypothesis, train],
                                   feed_dict={X: x_data, Y: y_data})
    if step % 10 == 0:
        print(step, "Cost:", cost_val, "\nPrediction:\n", hy_val)
```

    0 Cost: 84.2611
    Prediction:
     [[147.19199]
     [172.82959]
     [172.4113 ]
     [187.48358]
     [131.04774]]
    10 Cost: 7.225972
    Prediction:
     [[154.9442 ]
     [182.16638]
     [181.601  ]
     [197.4919 ]
     [138.17336]]
    20 Cost: 7.187818
    Prediction:
     [[154.95755]
     [182.20158]
     [181.62575]
     [197.51979]
     [138.20421]]
    30 Cost: 7.150592
    Prediction:
     [[154.94751]
     [182.20862]
     [181.62276]
     [197.51749]
     [138.21355]]
    40 Cost: 7.1135874
    Prediction:
     [[154.93742]
     [182.21553]
     [181.61969]
     [197.51506]
     [138.22278]]
    50 Cost: 7.07672
    Prediction:
     [[154.92738]
     [182.22246]
     [181.61665]
     [197.51268]
     [138.23204]]
    60 Cost: 7.040091
    Prediction:
     [[154.91736]
     [182.22935]
     [181.61359]
     [197.51028]
     [138.24123]]
    70 Cost: 7.003637
    Prediction:
     [[154.90735]
     [182.23624]
     [181.61053]
     [197.5079 ]
     [138.2504 ]]
    80 Cost: 6.96741
    Prediction:
     [[154.89737]
     [182.24307]
     [181.60751]
     [197.50552]
     [138.25955]]
    90 Cost: 6.9313
    Prediction:
     [[154.88744]
     [182.24994]
     [181.60449]
     [197.50314]
     [138.2687 ]]
    100 Cost: 6.8954954
    Prediction:
     [[154.8775 ]
     [182.25673]
     [181.60147]
     [197.50078]
     [138.27779]]
    110 Cost: 6.859832
    Prediction:
     [[154.86761]
     [182.26353]
     [181.59846]
     [197.49841]
     [138.28687]]
    120 Cost: 6.8243675
    Prediction:
     [[154.85776]
     [182.27032]
     [181.59547]
     [197.49608]
     [138.29593]]
    130 Cost: 6.7890854
    Prediction:
     [[154.84792]
     [182.27708]
     [181.59248]
     [197.49373]
     [138.30496]]
    140 Cost: 6.753995
    Prediction:
     [[154.8381 ]
     [182.28383]
     [181.5895 ]
     [197.49138]
     [138.31396]]
    150 Cost: 6.7190948
    Prediction:
     [[154.82832]
     [182.29056]
     [181.5865 ]
     [197.48903]
     [138.32294]]
    160 Cost: 6.6843843
    Prediction:
     [[154.81857]
     [182.29727]
     [181.58356]
     [197.48674]
     [138.33192]]
    170 Cost: 6.6498604
    Prediction:
     [[154.80884]
     [182.30396]
     [181.5806 ]
     [197.48439]
     [138.34085]]
    180 Cost: 6.6155167
    Prediction:
     [[154.79912]
     [182.31062]
     [181.57764]
     [197.48206]
     [138.34975]]
    190 Cost: 6.5813804
    Prediction:
     [[154.78944]
     [182.31728]
     [181.5747 ]
     [197.47977]
     [138.35864]]
    200 Cost: 6.547391
    Prediction:
     [[154.7798 ]
     [182.32391]
     [181.57178]
     [197.47745]
     [138.36752]]
    210 Cost: 6.513591
    Prediction:
     [[154.77016]
     [182.33052]
     [181.56883]
     [197.47514]
     [138.37636]]
    220 Cost: 6.480005
    Prediction:
     [[154.76056]
     [182.33713]
     [181.56592]
     [197.47285]
     [138.38516]]
    230 Cost: 6.446599
    Prediction:
     [[154.75099]
     [182.3437 ]
     [181.563  ]
     [197.47057]
     [138.39395]]
    240 Cost: 6.413347
    Prediction:
     [[154.74144]
     [182.35027]
     [181.5601 ]
     [197.46828]
     [138.40273]]
    250 Cost: 6.380278
    Prediction:
     [[154.73192]
     [182.35681]
     [181.5572 ]
     [197.46599]
     [138.41147]]
    260 Cost: 6.347413
    Prediction:
     [[154.72243]
     [182.36336]
     [181.55434]
     [197.46373]
     [138.42018]]
    270 Cost: 6.3146825
    Prediction:
     [[154.71295]
     [182.36986]
     [181.55145]
     [197.46146]
     [138.4289 ]]
    280 Cost: 6.28218
    Prediction:
     [[154.70352]
     [182.37636]
     [181.5486 ]
     [197.45923]
     [138.43758]]
    290 Cost: 6.2498236
    Prediction:
     [[154.6941 ]
     [182.38283]
     [181.54572]
     [197.45694]
     [138.44621]]
    300 Cost: 6.2176538
    Prediction:
     [[154.68471]
     [182.38928]
     [181.54286]
     [197.4547 ]
     [138.45483]]
    310 Cost: 6.1856294
    Prediction:
     [[154.67534]
     [182.39572]
     [181.54002]
     [197.45245]
     [138.46346]]
    320 Cost: 6.1538086
    Prediction:
     [[154.66602]
     [182.40215]
     [181.53719]
     [197.45023]
     [138.47205]]
    330 Cost: 6.1221466
    Prediction:
     [[154.65668]
     [182.40855]
     [181.53436]
     [197.448  ]
     [138.48059]]
    340 Cost: 6.090661
    Prediction:
     [[154.6474 ]
     [182.41493]
     [181.53151]
     [197.44576]
     [138.48912]]
    350 Cost: 6.059315
    Prediction:
     [[154.63812]
     [182.42131]
     [181.52872]
     [197.44354]
     [138.49765]]
    360 Cost: 6.0281644
    Prediction:
     [[154.6289 ]
     [182.42767]
     [181.52592]
     [197.44135]
     [138.50616]]
    370 Cost: 5.997177
    Prediction:
     [[154.61967]
     [182.43399]
     [181.5231 ]
     [197.43912]
     [138.51462]]
    380 Cost: 5.9663796
    Prediction:
     [[154.61049]
     [182.4403 ]
     [181.52031]
     [197.43692]
     [138.52306]]
    390 Cost: 5.935727
    Prediction:
     [[154.60132]
     [182.44661]
     [181.51753]
     [197.43472]
     [138.53148]]
    400 Cost: 5.9052315
    Prediction:
     [[154.5922 ]
     [182.4529 ]
     [181.51474]
     [197.43253]
     [138.53989]]
    410 Cost: 5.874913
    Prediction:
     [[154.58307]
     [182.45917]
     [181.512  ]
     [197.43034]
     [138.54826]]
    420 Cost: 5.8447876
    Prediction:
     [[154.57399]
     [182.46541]
     [181.50923]
     [197.42816]
     [138.55661]]
    430 Cost: 5.814752
    Prediction:
     [[154.56491]
     [182.47165]
     [181.50647]
     [197.426  ]
     [138.56496]]
    440 Cost: 5.7849298
    Prediction:
     [[154.55588]
     [182.47786]
     [181.50371]
     [197.42381]
     [138.57326]]
    450 Cost: 5.755288
    Prediction:
     [[154.54689]
     [182.48407]
     [181.50099]
     [197.42168]
     [138.58156]]
    460 Cost: 5.725777
    Prediction:
     [[154.53789]
     [182.49025]
     [181.49826]
     [197.41951]
     [138.58981]]
    470 Cost: 5.696393
    Prediction:
     [[154.52892]
     [182.49641]
     [181.49553]
     [197.41734]
     [138.59807]]
    480 Cost: 5.667184
    Prediction:
     [[154.51996]
     [182.50255]
     [181.4928 ]
     [197.41519]
     [138.6063 ]]
    490 Cost: 5.6381435
    Prediction:
     [[154.51103]
     [182.50868]
     [181.4901 ]
     [197.41304]
     [138.61449]]
    500 Cost: 5.609261
    Prediction:
     [[154.50217]
     [182.51482]
     [181.48741]
     [197.4109 ]
     [138.62268]]
    510 Cost: 5.580563
    Prediction:
     [[154.49329]
     [182.52089]
     [181.48471]
     [197.40878]
     [138.63083]]
    520 Cost: 5.5519934
    Prediction:
     [[154.48445]
     [182.52698]
     [181.48201]
     [197.40666]
     [138.63896]]
    530 Cost: 5.5235696
    Prediction:
     [[154.47563]
     [182.53305]
     [181.47934]
     [197.40454]
     [138.64708]]
    540 Cost: 5.495291
    Prediction:
     [[154.46684]
     [182.53912]
     [181.47668]
     [197.40244]
     [138.65518]]
    550 Cost: 5.4672155
    Prediction:
     [[154.45807]
     [182.54514]
     [181.47401]
     [197.40033]
     [138.66324]]
    560 Cost: 5.4392586
    Prediction:
     [[154.44931]
     [182.55113]
     [181.47134]
     [197.39821]
     [138.67128]]
    570 Cost: 5.411414
    Prediction:
     [[154.44057]
     [182.55714]
     [181.46869]
     [197.3961 ]
     [138.6793 ]]
    580 Cost: 5.3837724
    Prediction:
     [[154.43188]
     [182.56313]
     [181.46605]
     [197.39401]
     [138.68732]]
    590 Cost: 5.356266
    Prediction:
     [[154.42322]
     [182.5691 ]
     [181.46344]
     [197.39194]
     [138.69531]]
    600 Cost: 5.3289275
    Prediction:
     [[154.41457]
     [182.57504]
     [181.4608 ]
     [197.38985]
     [138.70326]]
    610 Cost: 5.3016863
    Prediction:
     [[154.40593]
     [182.58098]
     [181.45818]
     [197.38776]
     [138.71121]]
    620 Cost: 5.274646
    Prediction:
     [[154.39732]
     [182.58688]
     [181.45555]
     [197.38568]
     [138.71912]]
    630 Cost: 5.247711
    Prediction:
     [[154.38873]
     [182.59279]
     [181.45294]
     [197.38362]
     [138.72702]]
    640 Cost: 5.220964
    Prediction:
     [[154.38019]
     [182.59868]
     [181.45035]
     [197.38158]
     [138.7349 ]]
    650 Cost: 5.1943107
    Prediction:
     [[154.37166]
     [182.60455]
     [181.44777]
     [197.3795 ]
     [138.74277]]
    660 Cost: 5.167825
    Prediction:
     [[154.36314]
     [182.61041]
     [181.44518]
     [197.37746]
     [138.7506 ]]
    670 Cost: 5.1414986
    Prediction:
     [[154.35466]
     [182.61624]
     [181.4426 ]
     [197.37541]
     [138.7584 ]]
    680 Cost: 5.1152854
    Prediction:
     [[154.34619]
     [182.62207]
     [181.44003]
     [197.37337]
     [138.7662 ]]
    690 Cost: 5.089203
    Prediction:
     [[154.33772]
     [182.62787]
     [181.43745]
     [197.37132]
     [138.77397]]
    700 Cost: 5.063308
    Prediction:
     [[154.32933]
     [182.63367]
     [181.4349 ]
     [197.3693 ]
     [138.78172]]
    710 Cost: 5.0375247
    Prediction:
     [[154.32092]
     [182.63943]
     [181.43236]
     [197.36728]
     [138.78946]]
    720 Cost: 5.011885
    Prediction:
     [[154.31255]
     [182.64519]
     [181.42981]
     [197.36525]
     [138.79716]]
    730 Cost: 4.986396
    Prediction:
     [[154.3042 ]
     [182.65094]
     [181.42729]
     [197.36325]
     [138.80486]]
    740 Cost: 4.9610047
    Prediction:
     [[154.29587]
     [182.65666]
     [181.42474]
     [197.36122]
     [138.81253]]
    750 Cost: 4.935779
    Prediction:
     [[154.28758]
     [182.6624 ]
     [181.42224]
     [197.35924]
     [138.82019]]
    760 Cost: 4.910701
    Prediction:
     [[154.2793 ]
     [182.66808]
     [181.41972]
     [197.35724]
     [138.82782]]
    770 Cost: 4.885743
    Prediction:
     [[154.27103]
     [182.67375]
     [181.41722]
     [197.35522]
     [138.83542]]
    780 Cost: 4.8609223
    Prediction:
     [[154.26279]
     [182.67941]
     [181.4147 ]
     [197.35324]
     [138.843  ]]
    790 Cost: 4.836232
    Prediction:
     [[154.25458]
     [182.68507]
     [181.41222]
     [197.35126]
     [138.85057]]
    800 Cost: 4.811656
    Prediction:
     [[154.24637]
     [182.6907 ]
     [181.40973]
     [197.34927]
     [138.85812]]
    810 Cost: 4.7872453
    Prediction:
     [[154.23822]
     [182.69633]
     [181.40726]
     [197.34732]
     [138.86566]]
    820 Cost: 4.7629747
    Prediction:
     [[154.23007]
     [182.70192]
     [181.40479]
     [197.34532]
     [138.87315]]
    830 Cost: 4.738786
    Prediction:
     [[154.22194]
     [182.70752]
     [181.40231]
     [197.34335]
     [138.88065]]
    840 Cost: 4.7147565
    Prediction:
     [[154.21382]
     [182.71307]
     [181.39984]
     [197.34138]
     [138.8881 ]]
    850 Cost: 4.6908727
    Prediction:
     [[154.20575]
     [182.71864]
     [181.39738]
     [197.33943]
     [138.89554]]
    860 Cost: 4.667103
    Prediction:
     [[154.1977 ]
     [182.72418]
     [181.39494]
     [197.33748]
     [138.90297]]
    870 Cost: 4.6434684
    Prediction:
     [[154.18967]
     [182.72972]
     [181.39252]
     [197.33556]
     [138.91039]]
    880 Cost: 4.6199403
    Prediction:
     [[154.18166]
     [182.73523]
     [181.39008]
     [197.33362]
     [138.91779]]
    890 Cost: 4.5965357
    Prediction:
     [[154.17366]
     [182.74074]
     [181.38765]
     [197.33168]
     [138.92516]]
    900 Cost: 4.5732937
    Prediction:
     [[154.16568]
     [182.7462 ]
     [181.38524]
     [197.32974]
     [138.9325 ]]
    910 Cost: 4.5501738
    Prediction:
     [[154.15771]
     [182.75166]
     [181.38283]
     [197.32782]
     [138.9398 ]]
    920 Cost: 4.527137
    Prediction:
     [[154.1498 ]
     [182.75713]
     [181.38042]
     [197.3259 ]
     [138.94713]]
    930 Cost: 4.50424
    Prediction:
     [[154.14189]
     [182.76256]
     [181.378  ]
     [197.32397]
     [138.95442]]
    940 Cost: 4.481469
    Prediction:
     [[154.13402]
     [182.768  ]
     [181.37563]
     [197.32208]
     [138.9617 ]]
    950 Cost: 4.458844
    Prediction:
     [[154.12616]
     [182.7734 ]
     [181.37323]
     [197.32016]
     [138.96893]]
    960 Cost: 4.436311
    Prediction:
     [[154.1183 ]
     [182.7788 ]
     [181.37085]
     [197.31825]
     [138.97617]]
    970 Cost: 4.4138947
    Prediction:
     [[154.11049]
     [182.78418]
     [181.36847]
     [197.31635]
     [138.98338]]
    980 Cost: 4.391627
    Prediction:
     [[154.10269]
     [182.78955]
     [181.3661 ]
     [197.31447]
     [138.99057]]
    990 Cost: 4.369443
    Prediction:
     [[154.09491]
     [182.79489]
     [181.36374]
     [197.31256]
     [138.99776]]
    1000 Cost: 4.347419
    Prediction:
     [[154.08714]
     [182.80023]
     [181.3614 ]
     [197.31068]
     [139.0049 ]]
    1010 Cost: 4.3255186
    Prediction:
     [[154.07944]
     [182.80556]
     [181.35905]
     [197.30882]
     [139.01204]]
    1020 Cost: 4.303708
    Prediction:
     [[154.0717 ]
     [182.81085]
     [181.3567 ]
     [197.30695]
     [139.01915]]
    1030 Cost: 4.282009
    Prediction:
     [[154.06401]
     [182.81615]
     [181.35439]
     [197.30507]
     [139.02626]]
    1040 Cost: 4.2604628
    Prediction:
     [[154.05634]
     [182.82141]
     [181.35205]
     [197.3032 ]
     [139.03333]]
    1050 Cost: 4.238987
    Prediction:
     [[154.04869]
     [182.82669]
     [181.34972]
     [197.30135]
     [139.04039]]
    1060 Cost: 4.2176805
    Prediction:
     [[154.04108]
     [182.83194]
     [181.34741]
     [197.2995 ]
     [139.04742]]
    1070 Cost: 4.196437
    Prediction:
     [[154.03345]
     [182.83716]
     [181.34511]
     [197.29764]
     [139.05446]]
    1080 Cost: 4.1753354
    Prediction:
     [[154.02586]
     [182.84239]
     [181.3428 ]
     [197.29582]
     [139.06146]]
    1090 Cost: 4.15436
    Prediction:
     [[154.01831]
     [182.8476 ]
     [181.34053]
     [197.29398]
     [139.06845]]
    1100 Cost: 4.133458
    Prediction:
     [[154.01074]
     [182.85277]
     [181.33821]
     [197.29211]
     [139.07541]]
    1110 Cost: 4.112689
    Prediction:
     [[154.00322]
     [182.85797]
     [181.33594]
     [197.2903 ]
     [139.08235]]
    1120 Cost: 4.092043
    Prediction:
     [[153.99571]
     [182.86311]
     [181.33365]
     [197.28847]
     [139.08928]]
    1130 Cost: 4.071485
    Prediction:
     [[153.98822]
     [182.86827]
     [181.33139]
     [197.28664]
     [139.09619]]
    1140 Cost: 4.0510683
    Prediction:
     [[153.98076]
     [182.8734 ]
     [181.32912]
     [197.28482]
     [139.10307]]
    1150 Cost: 4.0307446
    Prediction:
     [[153.97331]
     [182.87852]
     [181.32686]
     [197.283  ]
     [139.10994]]
    1160 Cost: 4.010518
    Prediction:
     [[153.96591]
     [182.88365]
     [181.32462]
     [197.28122]
     [139.11682]]
    1170 Cost: 3.9904258
    Prediction:
     [[153.95851]
     [182.88875]
     [181.32237]
     [197.2794 ]
     [139.12364]]
    1180 Cost: 3.9704216
    Prediction:
     [[153.95111]
     [182.89381]
     [181.32013]
     [197.2776 ]
     [139.13046]]
    1190 Cost: 3.950533
    Prediction:
     [[153.94376]
     [182.89888]
     [181.3179 ]
     [197.2758 ]
     [139.13727]]
    1200 Cost: 3.9307609
    Prediction:
     [[153.9364 ]
     [182.90393]
     [181.31567]
     [197.27403]
     [139.14404]]
    1210 Cost: 3.9110794
    Prediction:
     [[153.9291 ]
     [182.90898]
     [181.31345]
     [197.27223]
     [139.1508 ]]
    1220 Cost: 3.891503
    Prediction:
     [[153.92177]
     [182.91399]
     [181.31122]
     [197.27045]
     [139.15755]]
    1230 Cost: 3.8720424
    Prediction:
     [[153.91449]
     [182.919  ]
     [181.30902]
     [197.26868]
     [139.16428]]
    1240 Cost: 3.8526788
    Prediction:
     [[153.90724]
     [182.92401]
     [181.30684]
     [197.2669 ]
     [139.171  ]]
    1250 Cost: 3.8334281
    Prediction:
     [[153.9    ]
     [182.92899]
     [181.30463]
     [197.26514]
     [139.17769]]
    1260 Cost: 3.814282
    Prediction:
     [[153.89276]
     [182.93396]
     [181.30244]
     [197.26338]
     [139.18436]]
    1270 Cost: 3.7952247
    Prediction:
     [[153.88556]
     [182.93892]
     [181.30025]
     [197.26161]
     [139.19101]]
    1280 Cost: 3.7762802
    Prediction:
     [[153.87837]
     [182.94385]
     [181.29807]
     [197.25984]
     [139.19765]]
    1290 Cost: 3.7574482
    Prediction:
     [[153.8712 ]
     [182.94878]
     [181.29588]
     [197.2581 ]
     [139.20425]]
    1300 Cost: 3.7387176
    Prediction:
     [[153.86406]
     [182.95369]
     [181.29373]
     [197.25636]
     [139.21086]]
    1310 Cost: 3.720073
    Prediction:
     [[153.85695]
     [182.95862]
     [181.29158]
     [197.25464]
     [139.21745]]
    1320 Cost: 3.7015388
    Prediction:
     [[153.84984]
     [182.96349]
     [181.28941]
     [197.25288]
     [139.22401]]
    1330 Cost: 3.6830833
    Prediction:
     [[153.84276]
     [182.96838]
     [181.28728]
     [197.25116]
     [139.23058]]
    1340 Cost: 3.6647499
    Prediction:
     [[153.83571]
     [182.97325]
     [181.28516]
     [197.24944]
     [139.23712]]
    1350 Cost: 3.6465003
    Prediction:
     [[153.82866]
     [182.97809]
     [181.28299]
     [197.24771]
     [139.24364]]
    1360 Cost: 3.6283653
    Prediction:
     [[153.82162]
     [182.98291]
     [181.28085]
     [197.24597]
     [139.25012]]
    1370 Cost: 3.6103013
    Prediction:
     [[153.8146 ]
     [182.98775]
     [181.27873]
     [197.24426]
     [139.2566 ]]
    1380 Cost: 3.592366
    Prediction:
     [[153.80762]
     [182.99255]
     [181.27661]
     [197.24255]
     [139.26306]]
    1390 Cost: 3.5745187
    Prediction:
     [[153.80064]
     [182.99734]
     [181.27449]
     [197.24084]
     [139.2695 ]]
    1400 Cost: 3.5567505
    Prediction:
     [[153.79369]
     [183.00212]
     [181.27238]
     [197.23914]
     [139.27594]]
    1410 Cost: 3.5390935
    Prediction:
     [[153.78676]
     [183.0069 ]
     [181.27028]
     [197.23744]
     [139.28235]]
    1420 Cost: 3.5215306
    Prediction:
     [[153.77985]
     [183.01166]
     [181.26819]
     [197.23575]
     [139.28874]]
    1430 Cost: 3.504055
    Prediction:
     [[153.77295]
     [183.0164 ]
     [181.2661 ]
     [197.23405]
     [139.29512]]
    1440 Cost: 3.486671
    Prediction:
     [[153.76608]
     [183.02115]
     [181.26404]
     [197.23238]
     [139.3015 ]]
    1450 Cost: 3.4693997
    Prediction:
     [[153.75922]
     [183.02586]
     [181.26196]
     [197.2307 ]
     [139.30783]]
    1460 Cost: 3.4522095
    Prediction:
     [[153.75238]
     [183.03056]
     [181.25987]
     [197.229  ]
     [139.31415]]
    1470 Cost: 3.4351144
    Prediction:
     [[153.74556]
     [183.03526]
     [181.25781]
     [197.22736]
     [139.32047]]
    1480 Cost: 3.4181175
    Prediction:
     [[153.73875]
     [183.03993]
     [181.25575]
     [197.22568]
     [139.32675]]
    1490 Cost: 3.4011772
    Prediction:
     [[153.73196]
     [183.0446 ]
     [181.2537 ]
     [197.224  ]
     [139.33304]]
    1500 Cost: 3.3843827
    Prediction:
     [[153.72522]
     [183.04926]
     [181.25163]
     [197.22234]
     [139.33928]]
    1510 Cost: 3.3676388
    Prediction:
     [[153.71844]
     [183.0539 ]
     [181.24959]
     [197.22069]
     [139.34552]]
    1520 Cost: 3.3509986
    Prediction:
     [[153.71173]
     [183.05853]
     [181.24754]
     [197.21904]
     [139.35175]]
    1530 Cost: 3.3344681
    Prediction:
     [[153.70503]
     [183.06314]
     [181.24553]
     [197.21739]
     [139.35796]]
    1540 Cost: 3.3179955
    Prediction:
     [[153.69833]
     [183.06773]
     [181.24348]
     [197.21574]
     [139.36415]]
    1550 Cost: 3.3016143
    Prediction:
     [[153.69167]
     [183.07234]
     [181.24147]
     [197.21411]
     [139.37033]]
    1560 Cost: 3.2853363
    Prediction:
     [[153.68501]
     [183.07692]
     [181.23946]
     [197.21246]
     [139.37648]]
    1570 Cost: 3.269144
    Prediction:
     [[153.67839]
     [183.0815 ]
     [181.23747]
     [197.21086]
     [139.38264]]
    1580 Cost: 3.2530143
    Prediction:
     [[153.67177]
     [183.08604]
     [181.23544]
     [197.2092 ]
     [139.38876]]
    1590 Cost: 3.236993
    Prediction:
     [[153.66518]
     [183.09059]
     [181.23344]
     [197.2076 ]
     [139.39488]]
    1600 Cost: 3.221051
    Prediction:
     [[153.65858]
     [183.09512]
     [181.23145]
     [197.20598]
     [139.40097]]
    1610 Cost: 3.2052112
    Prediction:
     [[153.65202]
     [183.09962]
     [181.22946]
     [197.20435]
     [139.40704]]
    1620 Cost: 3.1894317
    Prediction:
     [[153.64548]
     [183.10413]
     [181.22748]
     [197.20273]
     [139.41312]]
    1630 Cost: 3.1737666
    Prediction:
     [[153.63895]
     [183.10863]
     [181.22551]
     [197.20113]
     [139.41914]]
    1640 Cost: 3.1581712
    Prediction:
     [[153.63245]
     [183.1131 ]
     [181.22354]
     [197.19952]
     [139.42519]]
    1650 Cost: 3.1426396
    Prediction:
     [[153.62593]
     [183.11757]
     [181.22157]
     [197.19792]
     [139.4312 ]]
    1660 Cost: 3.127199
    Prediction:
     [[153.61946]
     [183.12202]
     [181.21959]
     [197.19632]
     [139.4372 ]]
    1670 Cost: 3.1118674
    Prediction:
     [[153.613  ]
     [183.12646]
     [181.21764]
     [197.19472]
     [139.44316]]
    1680 Cost: 3.096592
    Prediction:
     [[153.60658]
     [183.13092]
     [181.2157 ]
     [197.19315]
     [139.44914]]
    1690 Cost: 3.081408
    Prediction:
     [[153.60016]
     [183.13533]
     [181.21375]
     [197.19156]
     [139.4551 ]]
    1700 Cost: 3.0663052
    Prediction:
     [[153.59375]
     [183.13974]
     [181.2118 ]
     [197.18999]
     [139.46103]]
    1710 Cost: 3.0513062
    Prediction:
     [[153.58736]
     [183.14412]
     [181.20987]
     [197.18842]
     [139.46693]]
    1720 Cost: 3.0363648
    Prediction:
     [[153.58101]
     [183.14851]
     [181.20796]
     [197.18684]
     [139.47285]]
    1730 Cost: 3.0215058
    Prediction:
     [[153.57466]
     [183.1529 ]
     [181.20604]
     [197.18529]
     [139.47874]]
    1740 Cost: 3.006731
    Prediction:
     [[153.56833]
     [183.15726]
     [181.20412]
     [197.18372]
     [139.4846 ]]
    1750 Cost: 2.9920075
    Prediction:
     [[153.562  ]
     [183.1616 ]
     [181.2022 ]
     [197.18214]
     [139.49046]]
    1760 Cost: 2.9774265
    Prediction:
     [[153.55571]
     [183.16592]
     [181.2003 ]
     [197.18059]
     [139.49629]]
    1770 Cost: 2.9628778
    Prediction:
     [[153.54942]
     [183.17024]
     [181.19838]
     [197.17905]
     [139.50212]]
    1780 Cost: 2.9484098
    Prediction:
     [[153.54317]
     [183.17456]
     [181.19649]
     [197.17749]
     [139.50793]]
    1790 Cost: 2.9340224
    Prediction:
     [[153.53691]
     [183.17886]
     [181.1946 ]
     [197.17593]
     [139.51372]]
    1800 Cost: 2.919725
    Prediction:
     [[153.53069]
     [183.18315]
     [181.19272]
     [197.17441]
     [139.5195 ]]
    1810 Cost: 2.9054844
    Prediction:
     [[153.52448]
     [183.18744]
     [181.19083]
     [197.17285]
     [139.52525]]
    1820 Cost: 2.8913445
    Prediction:
     [[153.51826]
     [183.19168]
     [181.18895]
     [197.17131]
     [139.53099]]
    1830 Cost: 2.8772464
    Prediction:
     [[153.51208]
     [183.19595]
     [181.18707]
     [197.16978]
     [139.53673]]
    1840 Cost: 2.8632743
    Prediction:
     [[153.50594]
     [183.20018]
     [181.18523]
     [197.16827]
     [139.54245]]
    1850 Cost: 2.8493297
    Prediction:
     [[153.49979]
     [183.20442]
     [181.18336]
     [197.16675]
     [139.54816]]
    1860 Cost: 2.8354833
    Prediction:
     [[153.49365]
     [183.20863]
     [181.1815 ]
     [197.16522]
     [139.55383]]
    1870 Cost: 2.8217072
    Prediction:
     [[153.48755]
     [183.21284]
     [181.17966]
     [197.16371]
     [139.55951]]
    1880 Cost: 2.8079991
    Prediction:
     [[153.48146]
     [183.21706]
     [181.17781]
     [197.16222]
     [139.56517]]
    1890 Cost: 2.7943757
    Prediction:
     [[153.47537]
     [183.22122]
     [181.17596]
     [197.16069]
     [139.5708 ]]
    1900 Cost: 2.7808175
    Prediction:
     [[153.46931]
     [183.2254 ]
     [181.17413]
     [197.1592 ]
     [139.57643]]
    1910 Cost: 2.7673526
    Prediction:
     [[153.46327]
     [183.22957]
     [181.1723 ]
     [197.15771]
     [139.58203]]
    1920 Cost: 2.7539055
    Prediction:
     [[153.45723]
     [183.23372]
     [181.17047]
     [197.1562 ]
     [139.58765]]
    1930 Cost: 2.7405834
    Prediction:
     [[153.45123]
     [183.23785]
     [181.16866]
     [197.1547 ]
     [139.59322]]
    1940 Cost: 2.7273293
    Prediction:
     [[153.44524]
     [183.24197]
     [181.16685]
     [197.15321]
     [139.59879]]
    1950 Cost: 2.7141159
    Prediction:
     [[153.43925]
     [183.24611]
     [181.16505]
     [197.15175]
     [139.60435]]
    1960 Cost: 2.700992
    Prediction:
     [[153.4333 ]
     [183.25021]
     [181.16324]
     [197.15027]
     [139.6099 ]]
    1970 Cost: 2.6879435
    Prediction:
     [[153.42735]
     [183.2543 ]
     [181.16145]
     [197.14879]
     [139.61542]]
    1980 Cost: 2.6749604
    Prediction:
     [[153.42142]
     [183.25839]
     [181.15965]
     [197.14731]
     [139.62091]]
    1990 Cost: 2.6620722
    Prediction:
     [[153.41551]
     [183.26245]
     [181.15785]
     [197.14584]
     [139.62639]]
    2000 Cost: 2.6492188
    Prediction:
     [[153.4096 ]
     [183.26651]
     [181.15607]
     [197.14436]
     [139.63187]]