- Title: Notebook with MD info in the first cell
- Slug: md-info-in-cell
- Date: 2100-12-31
- Tags: Test
- Author: Daniel Rodriguez

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur purus mi, sollicitudin ac justo a, dapibus ultrices dolor. Curabitur id eros mattis, tincidunt ligula at, condimentum urna. Morbi accumsan, risus eget porta consequat, tortor nibh blandit dui, in sodales quam elit non erat. Aenean lorem dui, lacinia a metus eu, accumsan dictum urna. Sed a egestas mauris, non porta nisi. Suspendisse eu lacinia neque. Morbi gravida eros non augue pharetra, condimentum auctor purus porttitor.

## Header 2

```python
a = 1
```

```python
a
```

    1

```python
b = 'pew'
```

```python
b
```

    'pew'

```python
%matplotlib inline
```

```python
import matplotlib.pyplot as plt
```

```python
from pylab import *
```

```python
x = linspace(0, 5, 10)
y = x ** 2
```

```python
figure()
plot(x, y, 'r')
xlabel('x')
ylabel('y')
title('title')
show()
```

```python
import numpy as np
```

```python
num_points = 130
y = np.random.random(num_points)
plt.plot(y)
```

This is some text, here comes some latex

```latex
%%latex
\begin{align}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0
\end{align}
```

\begin{align}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0
\end{align}

Apos?

```python
import re
```

```python
text = 'foo bar\t baz \tqux'
```

```python
re.split('\s+', text)
```

    ['foo', 'bar', 'baz', 'qux']

## Javascript plots

### plotly

```python
from plotly import __version__
from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
from plotly.graph_objs import Scatter, Figure, Layout
```

```python
init_notebook_mode(connected=True)
```

```python
iplot([{"x": [1, 2, 3], "y": [3, 1, 6]}])
```

### bokeh

```python
from bokeh.plotting import figure, output_notebook, show
```

```python
output_notebook()
```

    Loading BokehJS ...

```python
p = figure()
p.line([1, 2, 3, 4, 5], [6, 7, 2, 4, 5], line_width=2)
show(p)
```

```python

```


# Chi-Squared Distribution
***
## Definition
>The Chi-Squared distribution is a continous probability distribution focused on sample standard deviations and can (e.g.) "let you know whether two groups have significantly different opinions, which makes it a very useful statistic for survey research"  $ ^{[1]}$.

## Formula
The probability mass function of a Chi-Squared distributed random variable is defined as:
$$
 \begin{equation}
    f(x|k) =
    \begin{cases}
      \frac{x^{\frac{k}{2}-1}e^{-\frac{x}{2}}}{2^{\frac{k}{2}}\Gamma(\frac{k}{2})}, & \text{if}\ x>0 \\
      0, & \text{otherwise}
    \end{cases}
  \end{equation}
$$

where
$$ \Gamma(n) = (n-1)! $$

and $k$ denotes the degrees of freedom.

```python
# %load ../src/chi2/01_general.py
```

***
## Parameters

```python
# %load ../src/chi2/02_k.py
```

***
## Implementation in Python
Multiple Python packages implement the Chi-Squared distribution. One of those is the stats.poisson module from the scipy package. The following methods are only an excerpt. For a full list of features the [official documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.poisson.html) should be read.

### Random Variates
In order to generate a random sample from, the function `rvs` should be used.

```python
import numpy as np
from scipy.stats import chi2

# draw a single sample
np.random.seed(42)
print(chi2.rvs(df=4), end="\n\n")

# draw 10 samples
print(chi2.rvs(df=4, size=10), end="\n\n")
```

    4.78735877974

    [ 2.98892946  2.76456717  2.76460459  9.29942882  5.73341246  2.262156
      4.93962895  3.99792053  0.43182989  1.34248457]

### Probability Density Function
The probability mass function can be accessed via the `pdf` function. Like the rvs method, the pdf allows for adjusting the mean of the random variable:

```python
from scipy.stats import chi2

# additional imports for plotting purpose
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
plt.rcParams["figure.figsize"] = (14,7)

# likelihood of x and y
x = 1
y = 7
print("pdf(1) = {}\npdf(7) = {}".format(chi2.pdf(x=x, df=4), chi2.pdf(x=y, df=4)))

# continuous pdf for the plot
x_s = np.arange(15)
y_s = chi2.pdf(x=x_s, df=4)
plt.scatter(x_s, y_s, s=100);
```

### Cumulative Probability Density Function¶
The cumulative probability density function is useful when a probability range has to be calculated. It can be accessed via the `cdf` function:

```python
from scipy.stats import chi2

# probability of x less or equal 0.3
print("P(X <=3) = {}".format(chi2.cdf(x=3, df=4)))

# probability of x in [-0.2, +0.2]
print("P(2 < X <= 8) = {}".format(chi2.cdf(x=8, df=4) - chi2.cdf(x=2, df=4)))
```

    P(X <=3) = 0.4421745996289252
    P(2 < X <= 8) = 0.6441806878992138

***
## Infering $k$
Given a sample of datapoints it is often required to estimate the "true" parameters of the distribution. In the case of the Chi-Squared distribution this estimation is quite simple. $k$ can be derived by calculating the mean of the sample.

```python
# %load ../src/chi2/03_estimation.py
```

## Infering $k$ - MCMC
In addition to a "direct" inference, $k$ can also be estimated using Markov chain Monte Carlo simulation - implemented in Python's [PyMC3](https://github.com/pymc-devs/pymc3).

```python
# %load ../src/chi2/04_MCMC_estimation.py
```

***
[1] - [Practical Surveys. Understanding Chi Squared](http://practicalsurveys.com/reporting/chisquare.php)


# Statisical error analysis of line uncertainty

We want to find a mean and variance for the parameters of the Hough transform for line extraction in 2 dimensions as a function of the length between two points.

This will be a good starting point for comparing the relative accuracy of lines vs points.

```python
%matplotlib inline

```

```python
# Simulations
import numpy as np
import scipy
import matplotlib.pyplot as plt
from scipy.stats import norm

N = 50000
img_size = 1000 #Average side length
bins = 100

L = 500 #pixels (average length)

sigma = 5./np.sqrt(12) #5 pixel uncertainty, assumed uniform

theta = np.random.uniform(-np.pi*0.85, np.pi*0.85, N)
x1 = np.random.uniform(0, img_size, (N, 2))
dx = np.array([np.sin(-theta)*L, np.cos(-theta) * L]).transpose()
x2 = x1 + dx
ro = x1[:, 0]*np.cos(theta) + x1[:, 1] * np.sin(theta)

dtheta = np.zeros(N)
dro    = np.zeros(N)

for i in range(N):
    x1_measured    = np.random.multivariate_normal(x1[i], sigma*np.identity(2))
    x2_measured    = np.random.multivariate_normal(x2[i], sigma*np.identity(2))
    dx_measured    = x2_measured-x1_measured
    theta_measured = np.arctan2(-dx_measured[0], dx_measured[1])
    ro_measured    = x1_measured[0]*np.cos(theta_measured) + x1_measured[1] * np.sin(theta_measured)
    ro_measured_2    = x2_measured[0]*np.cos(theta_measured) + x2_measured[1] * np.sin(theta_measured)

    dtheta[i]      = theta[i]-theta_measured
    dro[i]         = ro[i] - ro_measured

```

```python
ans = np.histogram(dtheta, bins, density = True)
y_theta = ans[0]
x_theta = ans[1][:-1]

sig_theta = np.std(dtheta)
print(sig_theta)

plt.plot(x_theta,y_theta, "-b", x_theta, norm.pdf(x_theta, 0, sig_theta), "-r")
plt.xlabel("$\\Delta \\theta$")
plt.ylabel("$p(\\Delta \\theta)$")
plt.legend(["Simulation", "Approximation"])

```

```python
ans = np.histogram(dro/sigma, bins, range= (-5, 5), density = True)
y_ro = ans[0]
x_ro = ans[1][:-1]

sig_ro = np.std(dro/sigma)
print(sig_ro)

def double_exp_pdf(x, var):
    b = np.sqrt(var/2)
    return 1/(2*b)*np.exp(-np.abs(x)/b)

plt.plot(x_ro, y_ro, "-b", x_ro, double_exp_pdf(x_ro, sig_ro**2), "-r")

plt.xlabel("$\\frac{\\Delta r}{\\sigma}$")
plt.ylabel("$p(\\Delta r)$")
plt.legend(["Simulation", "Approximation"])

```

```python

```

```python

```

```python

```

```python

```

Want to find

https://stats.stackexchange.com/questions/3215/trigonometric-operations-on-standard-deviations

$$ \hat{\theta} = \tan^{-1}(\Delta Y/\Delta X) $$
$$ \theta = \theta_0 + \Delta \theta$$

$$ \Delta Y = \sigma_y \zeta + \mu_y$$
$$ \Delta X = \sigma_x \xi + \mu_x$$

where $\zeta, \xi$ are standard normal distributions. For simplicity we will assume that $\sigma_y = \sigma_x = \sigma$, $ \mu_y = \sin(\theta_0) L $, and $  \mu_x = \cos(\theta_0) L$ where $L$ is the distance

$$ P[\hat{\theta} \le \theta] = P[\tan^{-1}(\Delta Y/\Delta X) \le \theta_0 + \Delta \theta] = P[\Delta Y/\Delta X \le \tan(\Delta \theta + \theta_0)] $$

Let $q = \tan(\theta) =  \tan(\Delta \theta + \theta_0) $

$$ = P[\sigma \zeta + \mu_y \le q (\sigma_y \zeta + \mu_y)] $$

$$ = P[\frac{\sigma}{L} (\zeta - q \xi)  \le  q  \sin(\theta_0) - \cos(\theta_0)] $$

This is a difference of gaussians giving a new gaussian being smaller than a function of $\theta$ and $\theta_0$. Let $b(\theta) = q  \sin(\theta_0) - \cos(\theta_0) $ and $\sigma^*(\theta) = (\frac{\sigma}{L})^2(1 +  \tan(\theta)^2)$ . The expression then becomes:

$$ P[\hat{\theta} \le \theta] =  \int_{-\infty}^{b(\theta)} \mathcal{N}\left(z; 0, \sigma^*(\theta)  \right) dz $$

We have that

$$ p(\theta) = \frac{d(P[\hat{\theta} \le \theta])}{d\theta} = \mathcal{N}\left(b(\theta); 0, \sigma^*(\theta) \right) \cdot \frac{db(\theta)}{d\theta} + \int_{-\infty}^{b(\theta)} \frac{d\left(\mathcal{N}\left(z; 0, \sigma^*(\theta) \right)\right)}{d\theta} dz$$

This simplifies to:

$$ p(\theta) =\mathcal{N}\left(b(\theta); 0,\sigma^*(\theta)  \right) \cdot \left(\frac{db(\theta)}{d\theta} + \frac{d((\sigma^*(\theta))^{-2})}{d\theta}\right) +  \frac{1}{\sigma^*(\theta)}\frac{d\sigma^*(\theta) }{d\theta} \int_{-\infty}^{b(\theta)} \mathcal{N}\left(z; 0, \sigma^*(\theta) \right) dz$$

```python
import sympy as sp

from sympy import symbols, exp, init_printing, latex, tan, atan, cos, sin

init_printing()

```

```python
sig, theta, theta0, L = symbols("sigma theta theta_0 L")

mux = L * cos(theta0)
muy = L * sin(theta0)
```

```python

Z = (muy * (sig + 1) - mux * (sig + 1) * tan(theta - theta0))**2 / (2 * (sig**2 + sig**2 + tan(theta - theta0)**2))
```

```python
expr = Z.diff(theta).diff(theta).subs(theta, 0)

```

```python
expr.subs(theta0, 0)
```


```python
import math
import matplotlib.pyplot as plt
from sympy import symbols, diff
import re
import numpy as np
```

```python
# Data :
# Продаж солодкої води :

x = [i for i in range(1,37)]

y = [9,10,10,11,12,18,26,40,39,28,20,13,
     8,9,11,9,13,15,33,45,45,25,18,10,
     7,9,10,11,13,15,31,40,35,26,19,13]
```

```python
plt.plot(x, y)
plt.axis([0, 50, 0, 50])
plt.show()
```

```python
## Stage find regressions :

def Cov(a,b):
    a = list(a)
    b = list(b)

    avr_a = 0
    for i in range(len(a)):
        avr_a +=a[i]
    avr_a /=len(a)

    avr_b = 0
    for i in range(len(b)):
        avr_b += b[i]
    avr_b /= len(b)

    znamenik = 0
    for i in range(len(a)):
        znamenik += (a[i] - avr_a) * (b[i] - avr_b)

    chiselnik = len(a) - 1

    cov = 0

    cov = znamenik / chiselnik

    return cov

```

```python
print('кореляція = ', Cov(x,y))
```

    кореляція =  18.62857142857143

```python
def standart_error(a,b):
    # Дисперсії :
    avr_a = 0
    for i in range(len(a)):
        avr_a += a[i]
    avr_a /=len(a)

    avr_b = 0
    for i in range(len(b)):
        avr_b += b[i]
    avr_b /= len(b)

    d_a = 0 # dispersia
    for i in range(len(a)):
        d_a += math.pow((a[i] - avr_a),2)

    d_b = 0 # dispersia
    for i in range(len(b)):
        d_b += math.pow((b[i] - avr_b),2)

    # Стандартне відхилення :
    std_a = 0
    std_b = 0

    std_a = math.pow(d_a,(0.5))
    std_b = math.pow(d_b ,(0.5))

    return std_a,std_b

def rxy(a,b):
    std_a,std_b = standart_error(a,b)
    # cof_korell :
    cov = Cov(a,b)
    cof_korell = cov / (std_a * std_b)

    return cof_korell
print('Кофіцієнт кореляції :',rxy(x,y))
```

    Кофіцієнт кореляції : 0.00429572464444998

```python
print('Кофіцієнт детермінації : ',math.pow(rxy(x,y),(0.5)))
```

    Кофіцієнт детермінації :  0.06554177785542577

```python
def firstRegressor(a,b):
    std_a,std_b = standart_error(a,b)
    cov = Cov(a,b)

    # Регресор :
    first = (std_a / std_b) * cov
    return first
firstRegressor(x,y)
```

    16.688890243688178

```python
def secondRegressor(a,b):
    avr_a = 0
    for i in range(len(a)):
        avr_a += a[i]
    avr_a /= len(a)

    avr_b = 0
    for i in range(len(b)):
        avr_b += b[i]
    avr_b /= len(b)
    first = firstRegressor(a,b)

    second = avr_b - (first * avr_a)
    return second
secondRegressor(x,y)
```

    -289.1333583971202

```python

```

```python

```

```python
def minMNK(a,b):
    function = 0
    w0, w1 = symbols('x y', real=True)

    for i in range(len(a)):
        function +=(w1*a[i] + w0 - b[i]) ** 2

    #print(function)

    function1W0 = diff(function,w0)
    function2W1 = diff(function,w1)

    #print(function1W0)
   # print(function2W1)

    function1W0 = str(function1W0)
    function2W1 = str(function2W1)

    result1 = re.findall(r'\d+',function1W0)
    result2 = re.findall(r'\d+',function2W1)

    for i in range(len(result1)):
        result1[i] = int(result1[i])

    for i in range(len(result2)):
        result2[i] = int(result2[i])

    matrix = [result1[0:2],result2[0:2]]
    matrix_result = [result1[2],result2[2]]

    #print('\n')

    #for i in range(2):
       # print(matrix[i],'  \t=   ',matrix_result[i])

    #print(matrix)
    #print(matrix_result)
    resultAll = np.linalg.solve(matrix,matrix_result)

    new_w0, new_w1 = resultAll

    return new_w0, new_w1
minMNK(x,y)
```

    (16.506349206349206, 0.16782496782496775)

```python
def linerRegression(a,b,x):
    w0,w1 = minMNK(a,b)
    y = w0 + (w1*x)
    return y
```

```python
print('Результат лінійної регресі на 37 місяць : ', linerRegression(x,y,37))
```

    Результат лінійної регресі на 37 місяць :  22.715873015873015

```python
import pandas as pd
import math
import matplotlib.pyplot as plt
from statsmodels.tsa.arima_model import ARIMA
import statsmodels.api as sm
```

```python
data = pd.read_csv('full_external_temperatures.csv',header=0, parse_dates=[0], index_col=0, squeeze=True)
```

```python
x = []
temp_x = []
temp_x.extend(data['dateTime'])
for i in range(2000):
    x.append(temp_x[i])

x_data = []
for i in range(200):
    x_data.append(i)
```

```python
y = []
temp_y = []
temp_y.extend(data['data'])
for i in range(200):
    y.append(temp_y[i])
```

```python
test_y = []
temp_y = []
temp_y.extend(data['data'])
for i in range(2003):
    test_y.append(temp_y[i])
```

```python
# Автокорегресія :
def auto_reg(a,b,my_x,p=1):
    w0,w1 = minMNK(a,b)
    function = w0
    #print(w0)
    result_array = []
    for i in range(p):
        len_arr = len(a) - i -1

        w0,w1 = minMNK(a[0:len_arr],b[0:len_arr])
        #print('liner - ',res_liner_regr, 'b - ',b[len_arr])
        function += b[len_arr] * w1

        result_array.append(function)
    return result_array
auto_reg(x_data,y,p=5,my_x=201)
```

    [8.685538329367326,
     8.753450289568269,
     8.81563303068502,
     8.874470829217163,
     8.931993883482486]

```python
# Скільзьке середнє :
def SMA(b,p=3):
    SMA_res = []
    sum_b = 0
    num = 0
    for i in range(len(b) - p):
        for j in range(p):
            sum_b += b[i+j]
        SMA_res.append(sum_b / p)
        sum_b = 0
    return SMA_res
def MA_foresee(b,p):
    SMA_res = 0

    for i in range(p):
        sum_b = len(b) - i -1
        SMA_res += b[sum_b]
        SMA_res /= p
    return SMA_res

MA_foresee(y,2)
```

    8.254999999999999

```python
MA_arr = [0,0,0]
temp_arr = SMA(y)
for i in range(len(temp_arr)):
    MA_arr.append(temp_arr[i])
```

```python

```

```python
plt.plot(x_data,MA_arr,'r')
plt.plot(x, y,'g')
plt.axis([0, 50, 0, 50])
plt.show()
```

```python
def ARIMA(x,y,my_x,p,q,d):
    new_y = [0]
    new_y1 = []

    MA_plust_ar = 0

    if d == 0:
        MA_res = MA_foresee(y,q)
        AR_res = auto_reg(x_data,y,p=p,my_x=my_x)
        print("result MA = ", MA_res)
        print("result AR = ", AR_res)
        MA_plust_ar = MA_res + AR_res
    elif d == 1:
        for i in range(len(y)):
            num = y[i] - y[i-1]
            new_y.append(int(num))
            num = 0
        MA_res = MA_foresee(new_y,q)
        AR_res = auto_reg(x_data,new_y,p=p,my_x=my_x)
        print("result MA = ", MA_res)
        print("result AR = ", AR_res)
        MA_plust_ar = MA_res + AR_res

    elif d == 2:
        for i in range(len(y)):
            num = y[i] - y[i-1]
            new_y.append(int(num))
            num = 0
        for i in range(len(new_y)):
            num = new_y[i] - new_y[i-1]
            new_y1.append(num)
            num = 0
        arr_MA = SMA(new_y1,p)
        res_MA = 0
        for i in range(len(arr_MA)):
            res_MA += arrMA[i]
        arr_AR = auto_reg()

    return MA_plust_ar

ARIMA(x_data,y,my_x=100,p=1,q=1,d=0)
```

```python
print(ARIMA(x_data,y,my_x=201,p=2,q=6,d=0))
```

```python
test_y = []
temp_y = []
temp_y.extend(data['data'])
for i in range(2003):
    test_y.append(temp_y[i])
```

```python
print(test_y[201])
```

```python
# Скільзьке середнє :
def SMA(b,p=3):
    SMA_res = []
    sum_b = 0
    num = 0
    for i in range(len(b) - p):
        for j in range(p):
            sum_b += b[i+j]
        SMA_res.append(sum_b / p)
        sum_b = 0
    return SMA_res
def MA_foresee(b,p):
    SMA_res = 0

    for i in range(p):
        sum_b = len(b) - i -1
        SMA_res += b[sum_b]
        SMA_res /= p
    return SMA_res

def MAGrath(b,p=3):
    SMA_res = []
    sum_b = 0
    num = 0
    for i in range(len(b) - p):
        for j in range(p):
            sum_b += b[i+j]
        SMA_res.append(sum_b / p)
        sum_b = 0
    return SMA_res

MA_foresee(y,2)
```

```python
# Автокорегресія :
def auto_reg(a,b,p=1):
    w0,w1 = minMNK(a,b)
    function = w0
    #print(w0)
    result_array = []
    for i in range(70):
        len_arr = len(a) - i -1
        w0,w1 = minMNK(a[0:len_arr],b[0:len_arr])
        function += b[len_arr] * w1
        result_array.append(function)
    return result_array
arr_Reg = auto_reg(x_data,y,p=5)
print(len(arr_Reg))
print(arr_Reg)
```

```python
def ARIMA(p,q,d):
    new_y = []
    new_y1 = []

    MA_plust_ar = 0

    if d == 1:
        for i in range(len(y)):
            num = y[i] - y[i-1]
            #print(y[i])
            new_y.append(int(num))
            num = 0
        print(len(x_data))
        print(len(new_y))
        plt.plot(x_data, y,'r')
        plt.plot(x_data,new_y,'b')
        plt.axis([-50, 150, -50, 150])
        plt.show()

    elif d == 0 :
        arr_MA = [0,0,0]
        temp_MA = MAGrath(y)
        for i in range(len(temp_MA)):
            tr = temp_MA[i]
            arr_MA.append(tr)
        plt.plot(x_data,arr_MA,'r')
        plt.plot(x_data,y,'b')
        plt.axis([-10, 350, -10, 20])
        plt.show()
ARIMA(p=1,q=1,d=0)
```

```python

```

```python

```

```python

```


```python
import numpy as np
import cmath as cm
import matplotlib as mpl
from matplotlib import pyplot as plt
import scipy.integrate as integrate
from scipy.integrate import quad
import scipy.optimize as so
import scipy.special as ss
import sympy as sym
import math

h = 4.9                                                     #radial scalelength in kpc
z0 = .99                                                    #half-thickness in kpc
L0 = 5.0e9                                                  #units? value?

x = np.linspace(10, 50, 200)

Lrz = lambda r,z: L0*np.exp(-r/h)*(math.cosh(z/z0))**-2     #cosh in radians?

y = np.zeros(np.shape(x))
hr = lambda r: Lrz(r,z)
for r,z in enumerate(x):
    y[r] = quad(hr, 1, r)[0]
```

```python
#Take sqrt only if there will be a real result.
for i,j in enumerate(y):
    if j >= 0:
        v[i] = np.sqrt(j)
#v = np.sqrt(y)

fig = plt.figure(figsize=(9.0,8.0))                 #size of the plot
ax3 = plt.axes()
ax3.grid(True)
ax3.yaxis.grid(True,which='minor',linestyle='--')

 #functions
plt.plot(x, v, linestyle='-.', label='Velocity(r)')

plt.axis('tight');                                 #xmin, xmax, ymin, ymax
                                                   #or use: plt.xlim(10, 0) plt.ylim(1.2, -1.2);
                                                   #or to tighten the bounds: plt.axis('tight')
                                                   #or use: plt.axis('tight')
                                                   #plt.axis([0, 15000, 0.5e6, 2.5e6])
 #labeling plots
ax3.set_title('Luminosity Profile for Disk', fontsize=25)              #labeling axes
ax3.set_xlabel('Radius (kpc)', fontsize=14)
ax3.set_ylabel('Luminosity ($L_\odot$)', fontsize=25)

#plt.legend(fontsize=14);                                   #adding legend
#plt.savefig('Luminosity Profile for Disk.png')
```

```python

```


```python
from itertools import repeat
from sympy import *
```

```python
init_printing()
```

```python
z, x, t, n = symbols('z x t n', positive=True)
o = symbols('o0:50', commutative=False)
u = symbols('u0:50', commutative=True)
```

```python
lst = 1/(1-z)
```

```python
catalan = (1-sqrt(1-4*z))/(2*z)
catalan
```

```python
catalan.series(z, n=10)
```

```python
catalan.diff(z,3).ratsimp().series(z, n=10)
```

```python
integrate(lst, (z,0,z))
```

```python
integrate(catalan, (z,0,z)).series(z,n=10)
```

```python
lst = (1/(1-z))
lst
```

```python
lst.diff(z,4).series(z,n=10)
```

```python
5040/24
```

```python
((1/(1-t))*(1/(1-t**5))*(1/(1-t**10))).apart()
```

---

```python
def multiples(z, digits=set(range(10)),degree=3):
    return sum(o[i]*z**i for i in digits)**degree
```

```python
multiples(z)
```

```python
multiples(z).series(z,n=10)
```

```python
number = 3
eps = exp(2*pi*I/number)
eps
```

```python
assert 0 == sum(eps**i for i in range(number)).simplify()
```

```python
s = sum(multiples(z*eps**i) for i in range(number)).expand().collect(z)
```

```python
D = dict(zip(o, repeat(S.One)))
muls = sum((c/number).simplify()*z**i
           for i in range(40)
           for c in [s.coeff(z,i)]
           for d in [c.subs(D, simultaneous=True).radsimp()]
           if d.is_Integer)
```

```python
muls
```

```python
muls.subs(dict(zip(o, u)))
```

```python
muls.subs(dict(zip(o, repeat(1))))
```

```python
s = _.coeff(z,8)
s.radsimp()
```

```python
_.simplify()
```

```python
_.powsimp()
```

```python

```


Chapter 1
======
***

Bayesian inference is simply updating your beliefs after considering new evidence.
Bayesians interpret probability as a measure of confidence in an event occuring.
Conversely, a frequentist interprets probability as the frequency of events.

Consider flipping an unfair coin. It lands heads.

** Bayesian**: The data is fixed (the coin landed heads). The parameters (the probability that the coin landed heads) are random.

** Frequentist**: The parameters are fixed. The result is a random variable, a single trial sampling from a probability distribution with a fixed, unknown heads  probability

In the Bayesian approach, we start with a *prior* belief about the probability of event $A$, $P(A)$. Given some evidence $X$, we then update our belief as $P(A|X)$. This approach includes our prior belief in an event occuring. As more evidence is aquired, our prior belief is washed out.
Suppose we have a number of instances of evidence $N$. For $N \rightarrow \infty$, the Bayesian and frequentist results will align. Any inference made can be easily and objectively agreed upon. On the other hand, for small $N$ inference is muc more unstable - frequentist estimates have greter variance and larger confidence intervals. This is where Bayesian analyis excels; by starting with a prior and updating this prior, we preserve the uncertainty that reflects the instability of statistical inference from a small $N$ dataset.

### Our Bayesian framework

We have a prior $P(A)$. This prior is updated to consider new evidence $X$ via Bayes' Theorem:

\begin{align}
 P( A | X ) = & \frac{ P(X | A) P(A) } {P(X) } \\\\[5pt]
\end{align}

How our beliefs are updated is easily observed with the plots below.

```python
%matplotlib inline
from IPython.core.pylabtools import figsize
import numpy as np
from matplotlib import pyplot as plt
figsize(11, 9)

import scipy.stats as stats

dist = stats.beta
n_trials = [0, 1, 2, 3, 4, 5, 8, 15, 50, 500]
data = stats.bernoulli.rvs(0.5, size=n_trials[-1])
x = np.linspace(0, 1, 100)

# For the already prepared, I'm using Binomial's conj. prior.
for k, N in enumerate(n_trials):
    sx = plt.subplot(len(n_trials) / 2, 2, k + 1)
    plt.xlabel("$p$, probability of heads") \
        if k in [0, len(n_trials) - 1] else None
    plt.setp(sx.get_yticklabels(), visible=False)
    heads = data[:N].sum()
    y = dist.pdf(x, 1 + heads, 1 + N - heads)
    plt.plot(x, y, label="observe %d tosses,\n %d heads" % (N, heads))
    plt.fill_between(x, 0, y, color="#348ABD", alpha=0.4)
    plt.vlines(0.5, 0, 4, color="k", linestyles="--", lw=1)

    leg = plt.legend()
    leg.get_frame().set_alpha(0.4)
    plt.autoscale(tight=True)

plt.suptitle("Bayesian updating of posterior probabilities",
             y=1.02,
             fontsize=14)

plt.tight_layout()
```

_______

## Quick recap of Probability Distributions

We can consider 3 classes of random variables $Z$:

-   **$Z$ is discrete**.

Expected value $E(Z) = \Sigma x_i p_i$, for result $x$ and probability $p$.

Distribution of $Z$ is called the *probability mass function* - the probability that $Z$ takes the value $k$, $P(Z=k)$. The most common probability mass function is the *Poisson distribution*:

$$P(Z = k) =\frac{ \lambda^k e^{-\lambda} }{k!}, \; \; k=0,1,2, \dots $$

$\lambda$ parameterizes the Poisson distribution, describing the shape. Usefully, the expected value of the Poisson distribution is equal to $\lambda$:
$$ E[Z] = \lambda$$

-   **$Z$ is continuous**.

Expected value $E(Z) = \int x f(x) dx$ for probability density function $f(x)$

- **$Z$ is mixed**

### Worked example

>  You are given a series of daily text-message counts from a user of your system. The data, plotted over time, appears in the chart below. You are curious to know if the user's text-messaging habits have changed over time, either gradually or suddenly. How can you model this? (This is in fact my own text-message data. Judge my popularity as you wish.)

```python
figsize(12.5, 3.5)
count_data = np.loadtxt("data/txtdata.csv")
n_count_data = len(count_data)
plt.bar(np.arange(n_count_data), count_data, color="#348ABD")
plt.xlabel("Time (days)")
plt.ylabel("count of text-msgs received")
plt.title("Did the user's texting habits change over time?")
plt.xlim(0, n_count_data);
```

How should we model this? We have discrete data, so lets use a Poisson distribution. We denote day $i$'s text-message count by $C_i$,

$$ C_i \sim \text{Poisson}(\lambda)  $$

Now, we don't know what $\lambda$ is. Looking at the chart, we might suspect that at some point in the observations, $\lambda$ increases. We can state this mathematically,

$$
\lambda =
\begin{cases}
\lambda_1  & \text{if } t \lt \tau \cr
\lambda_2 & \text{if } t \ge \tau
\end{cases}
$$

We are now interested in inferring the unknowns $\lambda_1, \lambda_2$. What would be a good prior probability distribution for these?
$\lambda_{1,2}$, can be any positive number, so we want some form of continuous probability density function. Lets use the *exponential distribution*

$$f_Z(z | \alpha) = \alpha e^{-\alpha z }, \;\; z\ge 0$$

The expected value of random variable $z$, is

$$ E[Z] = \frac{1}{\alpha}$$

What about $\tau$? it is tricky to pick when $\tau$ might have occured. Instead we can assign a uniform prior. Our overall prior distribution on the unknown variables is getting messy; we want $\lambda$, which in turn is described by distributions of $\lambda_1, \lambda_2, \tau$. Furthermore, $\lambda_1, \lambda_2$ are both parameterized by $\alpha$.

We instead use PyMC, a Python library for performing Bayesian analysis.

### Introducing PyMC
We will model our problem using PyMC. This is known as *probabilistic programming*. It is probabilistic in the sense that we create probability models using programming variables.

Lets dive in:

```python
import pymc as pm

alpha = 1 / count_data.mean() #From our earlier defenition of the expected value
                              #of the exponential distribution

lambda_1 = pm.Exponential("lambda_1", alpha)
lambda_2 = pm.Exponential("lambda_2", alpha)
tau = pm.DiscreteUniform("tau", lower=0, upper=n_count_data)

```

We can then use these parameters to construct our structure of $\lambda$ itself.

```python
@pm.deterministic
def lambda_(tau=tau, lambda_1=lambda_1, lambda_2=lambda_2):
    out = np.zeros(n_count_data)
    out[:tau] = lambda_1  # lambda before tau is lambda1
    out[tau:] = lambda_2  # lambda after (and including) tau is lambda2
    return out
```

@pm.deterministic is a decorator that tells PyMC this is a deterministic function. That is, if the arguments were deterministic (which they are not), the output would be deterministic as well. Deterministic functions will be covered in Chapter 2.

Now lets create our $\lambda$ prior distribution:

```python
observation = pm.Poisson("obs", lambda_, value=count_data, observed=True)

model = pm.Model([observation, lambda_1, lambda_2, tau])
```

The code below is explained in Chapter 3. It is known as *Markov Chain Monte Carlo*, which returns thousands or random variables from the posterior distributions of $\lambda_1, \lambda_2$ and $\tau$.

```python
# Mysterious code to be explained in Chapter 3.
mcmc = pm.MCMC(model)
mcmc.sample(40000, 10000, 1)
```

     [-----------------100%-----------------] 40000 of 40000 complete in 10.0 sec

```python
lambda_1_samples = mcmc.trace('lambda_1')[:]
lambda_2_samples = mcmc.trace('lambda_2')[:]
tau_samples = mcmc.trace('tau')[:]
```

```python
figsize(12.5, 10)
# histogram of the samples:

ax = plt.subplot(311)
ax.set_autoscaley_on(False)

plt.hist(lambda_1_samples, histtype='stepfilled', bins=30, alpha=0.85,
         label="posterior of $\lambda_1$", color="#A60628", normed=True)
plt.legend(loc="upper left")
plt.title(r"""Posterior distributions of the variables
    $\lambda_1,\;\lambda_2,\;\tau$""")
plt.xlim([15, 30])
plt.xlabel("$\lambda_1$ value")

ax = plt.subplot(312)
ax.set_autoscaley_on(False)
plt.hist(lambda_2_samples, histtype='stepfilled', bins=30, alpha=0.85,
         label="posterior of $\lambda_2$", color="#7A68A6", normed=True)
plt.legend(loc="upper left")
plt.xlim([15, 30])
plt.xlabel("$\lambda_2$ value")

plt.subplot(313)
w = 1.0 / tau_samples.shape[0] * np.ones_like(tau_samples)
plt.hist(tau_samples, bins=n_count_data, alpha=1,
         label=r"posterior of $\tau$",
         color="#467821", weights=w, rwidth=2.)
plt.xticks(np.arange(n_count_data))

plt.legend(loc="upper left")
plt.ylim([0, .75])
plt.xlim([35, len(count_data) - 20])
plt.xlabel(r"$\tau$ (in days)")
plt.ylabel("probability");
```

So now we have some distributions of $\lambda_1, \lambda_2$ and $\tau$. These represent our belief in the possible values these parameters can take. We can now answer the question, what is the expected number of texts at day $t, 0 \leq t \leq 70$? This is equivalent to asking *what is the expected value of $\lambda$ at time $t$*?

```python
figsize(12.5, 5)
# tau_samples, lambda_1_samples, lambda_2_samples contain
# N samples from the corresponding posterior distribution
N = tau_samples.shape[0]
expected_texts_per_day = np.zeros(n_count_data)
for day in range(0, n_count_data):

    ix = day < tau_samples

    expected_texts_per_day[day] = (lambda_1_samples[ix].sum()
                                   + lambda_2_samples[~ix].sum()) / N

plt.plot(range(n_count_data), expected_texts_per_day, lw=4, color="#E24A33",
         label="expected number of text-messages received")
plt.xlim(0, n_count_data)
plt.xlabel("Day")
plt.ylabel("Expected # text-messages")
plt.title("Expected number of text-messages received")
plt.ylim(0, 60)
plt.bar(np.arange(len(count_data)), count_data, color="#348ABD", alpha=0.65,
        label="observed texts per day")

plt.legend(loc="upper left");
```

This analysis seems to support that the user's behaviour did change, rahter suddenly

### Exercises
1\.  Using `lambda_1_samples` and `lambda_2_samples`, what is the mean of the posterior distributions of $\lambda_1$ and $\lambda_2$?

```python
print (np.mean(lambda_1_samples))
print (np.mean(lambda_2_samples))
```

    17.7584487269
    22.7316017731

Do these seem reasonable from looking at the histograms? Yes.

2\.  What is the expected percentage increase in text-message rates? `hint:` compute the mean of `lambda_1_samples/lambda_2_samples`. Note that this quantity is very different from `lambda_1_samples.mean()/lambda_2_samples.mean()`.

```python
print (np.mean(lambda_2_samples/lambda_1_samples))
```

    1.28171650014

i.e. ~ a 28% increase

3\. What is the mean of $\lambda_1$ **given** that we know $\tau$ is less than 45.  That is, suppose we have been given new information that the change in behaviour occurred prior to day 45. What is the expected value of $\lambda_1$ now? (You do not need to redo the PyMC part. Just consider all instances where `tau_samples < 45`.)

```python
lambda_1_cropped = []
for i in range(len(tau_samples)):
    if tau_samples[i] < 45:
        #Only get the lambdas where tau < 45
        lambda_1_cropped.extend([lambda_1_samples[i]])

print (np.mean(lambda_1_cropped))
```

    17.7623110292

```python

```