# Basic Python Packages for Science
## The Aeropython’s guide to the Python Galaxy!

###### Siro Moreno Martín
###### Alejandro Sáez Mollejo

### 0. Introduction

#### Python in the Scientific environment

##### Principal Python Packages for scientific purposes

##### Anaconda & conda

http://conda.pydata.org/docs/intro.html

Conda is a package manager application that quickly installs, runs, and updates packages and their dependencies. The conda command is the primary interface for managing installations of various packages. It can query and search the package index and current installation, create new environments, and install and update packages into existing conda environments.

```python
from IPython.display import HTML
HTML('')
```

##### Main objectives of this workshop

* Provide you with a first insight into the principal Python tools & libraries used in Science:
    - conda.
    - Jupyter Notebook.
    - NumPy, matplotlib, SciPy
* Provide you with the basic skills to face basic tasks such as:
    -
* Show other common libraries:
    - Pandas, scikit-learn (some talks & workshops will focus on these packages)
    - SymPy
    - Numba ¿?

### 1. Jupyter Notebook

The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more.

It has been widely recognised as a great way to distribute scientific papers, because of the capability to have an integrated format with text and executable code, highly reproducible. Top level investigators around the world are already using it, like the team behind the Gravitational Waves discovery (LIGO), whose analysis was translated to an interactive dowloadable Jupyter notebook. You can see it here: https://github.com/minrk/ligo-binder/blob/master/GW150914_tutorial.ipynb

### 2. Using arrays: NumPy

#### ndarray object

| index     | 0     | 1     | 2     | 3     | ...   | n-1   | n  |
| ---------- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| value      | 2.1   | 3.6   | 7.8   | 1.5   | ...   | 5.4   | 6.3 |

* N-dimensional data structure.
* Homogeneously typed.
* Efficient!

A universal function (or ufunc for short) is a function that operates on ndarrays. It is a “vectorized function".

```python
import numpy as np
```

```python
my_list  = list(range(0,100000))
%timeit sum(my_list)
```

    1000 loops, best of 3: 1.54 ms per loop

```python
array = np.arange(0, 100000)
%timeit np.sum(array)
```

    10000 loops, best of 3: 97.8 µs per loop

#### Array creation

```python
one_dim_array = np.array([1, 2, 3, 4])
one_dim_array
```

    array([1, 2, 3, 4])

```python
two_dim_array = np.array([[1, 2, 3],
                                           [4, 5, 6],
                                           [7, 8, 9]])
two_dim_array
```

    array([[1, 2, 3],
           [4, 5, 6],
           [7, 8, 9]])

```python
two_dim_array.size
```

    9

```python
two_dim_array.shape
```

    (3, 3)

```python
two_dim_array.dtype
```

    dtype('int64')

```python
zeros_arr = np.zeros([3, 3])
ones_arr = np.ones([10])
eye_arr = np.eye(5)
```

```python
range_arr = np.arange(15)
range_arr
```

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])

```python
range_arr.reshape([3, 5])
```

    array([[ 0,  1,  2,  3,  4],
           [ 5,  6,  7,  8,  9],
           [10, 11, 12, 13, 14]])

```python
np.linspace(0, 10, 21)
```

    array([  0. ,   0.5,   1. ,   1.5,   2. ,   2.5,   3. ,   3.5,   4. ,
             4.5,   5. ,   5.5,   6. ,   6.5,   7. ,   7.5,   8. ,   8.5,
             9. ,   9.5,  10. ])

#### Basic slicing

```python
one_dim_array[0]
```

    1

```python
two_dim_array[-1, -1]
```

    9

`[start:stop:step]`

```python
my_arr = np.arange(100)
my_arr[0::2]
```

    array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32,
           34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66,
           68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98])

```python
chess_board = np.zeros([8, 8], dtype=int)

chess_board[0::2, 1::2] = 1
chess_board[1::2, 0::2] = 1

chess_board
```

    array([[0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0],
           [0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0],
           [0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0],
           [0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0]])

### 2. Drawing: Matplotlib

```python
%matplotlib inline
import matplotlib.pyplot as plt
```

```python
plt.matshow(chess_board, cmap=plt.cm.gray)
```

#### Operations & linalg

```python
# numpy functions
x = np.linspace(1, 10)
y = np.sin(x)
```

```python
plt.plot(x, y)
```

```python
y_2 = (1 + np.log(x)) ** 2
```

```python
# Our first plot
plt.plot(x, y_2, 'r-*')
```

```python
# Creating a 2d array
two_dim_array = np.array([[10, 25, 33],
                                           [40, 25, 16],
                                           [77, 68, 91]])

two_dim_array.T
```

    array([[10, 40, 77],
           [25, 25, 68],
           [33, 16, 91]])

```python
# matrix multiplication
two_dim_array @ two_dim_array
```

    array([[ 3641,  3119,  3733],
           [ 2632,  2713,  3176],
           [10497,  9813, 11910]])

```python
# matrix vector
one_dim_array = np.array([2.5, 3.6, 3.8])

two_dim_array @ one_dim_array
```

    array([ 240.4,  250.8,  783.1])

```python
# inv
np.linalg.inv(two_dim_array)
```

    array([[-0.05372256,  0.00140303,  0.01923512],
           [ 0.10898393,  0.07381761, -0.05250057],
           [-0.03598099, -0.05634759,  0.03394433]])

```python
# eigenvectors & eigenvalues
np.linalg.eig(two_dim_array)
```

    (array([ 133.6946629,  -17.266221 ,    9.5715581]),
     array([[-0.29580975, -0.74274264,  0.0661375 ],
            [-0.24477775,  0.65983255, -0.79576005],
            [-0.92335283,  0.1138173 ,  0.60198985]]))

#### Air quality data

```python
from IPython.display import HTML
HTML('')
```

##### Loading the data

```python
# Linux command
!head ./data/barrio_del_pilar-20160322.csv

# Windows
# !gc log.txt | select -first 10 # head
```

    Estación: Barrio del Pilar;;;;
    Fecha;Hora;CO;NO2;O3
    ;;mg/m³;µg/m³;µg/m³
    22/03/2016;01:00;0.2;14;73
    22/03/2016;02:00;0.2;10;77
    22/03/2016;03:00;0.2;9;75
    22/03/2016;04:00;0.2;3;81
    22/03/2016;05:00;0.2;3;81
    22/03/2016;06:00;0.2;6;79
    22/03/2016;07:00;0.2;24;59

```python
# loading the data
# ./data/barrio_del_pilar-20160322.csv
data1 = np.genfromtxt('./data/barrio_del_pilar-20160322.csv', skip_header=3, delimiter=';', usecols=(2,3,4))
data1
```

    array([[   0.2,   14. ,   73. ],
           [   0.2,   10. ,   77. ],
           [   0.2,    9. ,   75. ],
           [   0.2,    3. ,   81. ],
           [   0.2,    3. ,   81. ],
           [   0.2,    6. ,   79. ],
           [   0.2,   24. ,   59. ],
           [   0.3,   48. ,   37. ],
           [   0.3,   40. ,   43. ],
           [   0.3,   41. ,   44. ],
           [   0.3,   20. ,   68. ],
           [   0.3,   17. ,   74. ],
           [   0.2,   14. ,   84. ],
           [   0.3,   16. ,   88. ],
           [   0.3,   15. ,   94. ],
           [   0.4,   29. ,   81. ],
           [   0.3,   23. ,   82. ],
           [   0.3,   26. ,   81. ],
           [   0.3,   30. ,   75. ],
           [   0.4,   57. ,   39. ],
           [   0.4,   73. ,   17. ],
           [   0.4,   51. ,   42. ],
           [   0.4,   72. ,   16. ],
           [   0.4,   61. ,   28. ],
           [   0.3,   25. ,   62. ],
           [   0.3,   21. ,   64. ],
           [   0.3,   40. ,   39. ],
           [   0.4,   52. ,   19. ],
           [   0.4,   47. ,    8. ],
           [   0.4,   42. ,    8. ],
           [   0.5,   68. ,    8. ],
           [   0.6,   71. ,    9. ],
           [   0.9,   76. ,   10. ],
           [   0.7,   63. ,   29. ],
           [   0.3,   27. ,   66. ],
           [   0.3,   13. ,   88. ],
           [   0.2,   10. ,   92. ],
           [   0.3,   10. ,   98. ],
           [   0.3,   11. ,   99. ],
           [   0.3,   12. ,   99. ],
           [   0.2,   11. ,   98. ],
           [   0.2,    8. ,  101. ],
           [   0.2,   13. ,   92. ],
           [   0.2,   23. ,   79. ],
           [   0.5,   40. ,   56. ],
           [   0.6,   49. ,   43. ],
           [   0.5,   66. ,   25. ],
           [   0.4,   47. ,   44. ],
           [   0.3,   18. ,   76. ],
           [   0.3,   25. ,   64. ],
           [   0.3,   16. ,   77. ],
           [   0.3,   16. ,   59. ],
           [   0.3,   34. ,   31. ],
           [   0.3,   27. ,   33. ],
           [   0.3,   44. ,   17. ],
           [   0.4,   45. ,    9. ],
           [   0.5,   52. ,   22. ],
           [   0.4,   37. ,   53. ],
           [   0.3,   21. ,   73. ],
           [   0.3,   20. ,   76. ],
           [   0.3,   24. ,   76. ],
           [   0.4,   38. ,   71. ],
           [   0.3,   32. ,   78. ],
           [   0.3,   21. ,   89. ],
           [   0.2,   10. ,  105. ],
           [   0.3,   15. ,  102. ],
           [   0.3,   21. ,   93. ],
           [   0.3,   45. ,   63. ],
           [   0.4,   59. ,   47. ],
           [   0.4,   59. ,   44. ],
           [   0.7,   99. ,    9. ],
           [   0.6,   88. ,    9. ],
           [   0.8,   93. ,    9. ],
           [   0.9,   89. ,    9. ],
           [   0.8,   84. ,    8. ],
           [   0.5,   64. ,   10. ],
           [   0.4,   58. ,   11. ],
           [   0.5,   53. ,    9. ],
           [   0.4,   41. ,    8. ],
           [   0.5,   43. ,    9. ],
           [   0.5,   45. ,   13. ],
           [   0.6,   51. ,   25. ],
           [   0.5,   44. ,   40. ],
           [   0.4,   36. ,   59. ],
           [   0.4,   36. ,   68. ],
           [   0.3,   26. ,   84. ],
           [   0.3,   16. ,   98. ],
           [   0.3,   17. ,   97. ],
           [   0.3,   24. ,   89. ],
           [   0.3,   17. ,   99. ],
           [   0.3,   12. ,  100. ],
           [   0.3,   42. ,   61. ],
           [   0.4,   52. ,   44. ],
           [   0.5,   54. ,   39. ],
           [   0.5,   60. ,   28. ],
           [   0.6,   73. ,   13. ],
           [   0.5,   58. ,   23. ],
           [   0.4,   58. ,   16. ],
           [   0.5,   61. ,   10. ],
           [   0.5,   59. ,    9. ],
           [   0.4,   50. ,    9. ],
           [   0.3,   31. ,   10. ],
           [   0.4,   36. ,    9. ],
           [   0.6,   45. ,    9. ],
           [   0.5,   43. ,   18. ],
           [   0.5,   37. ,   24. ],
           [   0.5,   40. ,   38. ],
           [   0.4,   26. ,   59. ],
           [   0.3,   14. ,   67. ],
           [   0.3,   12. ,   64. ],
           [   0.3,   13. ,   62. ],
           [   0.2,   10. ,   63. ],
           [   0.2,    7. ,   58. ],
           [   0.2,    8. ,   53. ],
           [   0.2,   11. ,   51. ],
           [   0.3,   16. ,   47. ],
           [   0.2,   19. ,   42. ],
           [   0.3,   22. ,   38. ],
           [   0.3,   23. ,   36. ],
           [   0.3,   16. ,   43. ],
           [   0.2,    9. ,   49. ],
           [   0.2,    6. ,   48. ],
           [   nan,    nan,    nan],
           [   0.2,    4. ,   64. ],
           [   0.2,    3. ,   89. ],
           [   0.2,    4. ,   90. ],
           [   0.2,    3. ,   92. ],
           [   0.2,    6. ,   89. ],
           [   0.3,   11. ,   83. ],
           [   0.3,    9. ,   87. ],
           [   0.3,    8. ,   84. ],
           [   0.3,   10. ,   82. ],
           [   0.3,   10. ,   80. ],
           [   0.3,   12. ,   80. ],
           [   0.3,   12. ,   81. ],
           [   0.3,    8. ,   84. ],
           [   0.3,   10. ,   85. ],
           [   0.2,   10. ,   85. ],
           [   0.2,   14. ,   82. ],
           [   0.3,   18. ,   72. ],
           [   0.3,   28. ,   60. ],
           [   0.3,   30. ,   55. ],
           [   0.3,   21. ,   61. ],
           [   0.3,   16. ,   63. ],
           [   0.3,   12. ,   65. ],
           [   0.2,    9. ,   67. ],
           [   0.2,    5. ,   70. ],
           [   0.2,    5. ,   69. ],
           [   0.2,    6. ,   65. ],
           [   0.2,    7. ,   63. ],
           [   0.3,   16. ,   55. ],
           [   0.3,   30. ,   45. ],
           [   0.3,   38. ,   39. ],
           [   0.3,   37. ,   41. ],
           [   0.3,   29. ,   53. ],
           [   0.3,   27. ,   53. ],
           [   0.3,   27. ,   49. ],
           [   0.3,   23. ,   54. ],
           [   0.3,   22. ,   57. ],
           [   0.3,   19. ,   61. ],
           [   0.3,   17. ,   63. ],
           [   0.3,   22. ,   59. ],
           [   0.3,   27. ,   53. ],
           [   0.3,   29. ,   50. ],
           [   0.3,   34. ,   44. ],
           [   0.3,   33. ,   45. ],
           [   0.3,   26. ,   50. ],
           [   0.3,   19. ,   56. ],
           [   0.2,   11. ,   63. ],
           [   0.2,    8. ,   63. ],
           [   0.2,    9. ,   58. ],
           [   0.2,    6. ,   63. ],
           [   0.2,    5. ,   66. ],
           [   0.2,    7. ,   62. ],
           [   0.3,   18. ,   53. ],
           [   0.4,   38. ,   37. ],
           [   0.4,   49. ,   28. ],
           [   0.4,   45. ,   35. ],
           [   0.3,   34. ,   47. ],
           [   0.3,   24. ,   62. ],
           [   0.3,   24. ,   68. ],
           [   0.3,   28. ,   68. ],
           [   0.3,   23. ,   78. ],
           [   0.3,   21. ,   82. ],
           [   0.3,   17. ,   87. ],
           [   0.3,   23. ,   80. ],
           [   0.3,   28. ,   75. ],
           [   0.3,   29. ,   71. ],
           [   0.3,   46. ,   50. ],
           [   0.4,   66. ,   27. ],
           [   0.3,   51. ,   38. ],
           [   0.3,   42. ,   46. ]])

##### Dealing with missing values

```python
np.mean(data1, axis=0)
```

    array([ nan,  nan,  nan])

```python
np.nanmean(data1, axis=0)
```

    array([  0.33717277,  29.79581152,  55.47643979])

```python
# masking invalid data
data1 = np.ma.masked_invalid(data1)
np.mean(data1, axis=0)
```

    masked_array(data = [0.3371727748691094 29.79581151832461 55.47643979057592],
                 mask = [False False False],
           fill_value = 1e+20)

```python
data2 = np.genfromtxt('./data/barrio_del_pilar-20151222.csv', skip_header=3, delimiter=';', usecols=(2,3,4))
data2 = np.ma.masked_invalid(data2)
```

##### Plotting the data

** Maximum values ** from: http://www.mambiente.munimadrid.es/opencms/export/sites/default/calaire/Anexos/valores_limite_1.pdf

* NO2
    - Media anual: 40 µg/m3
    - **Media horaria: 200 µg/m3 **

```python
plt.plot(data1[:, 1], label='2016')
plt.plot(data2[:, 1], label='2015')

plt.legend()

plt.hlines(200, 0, 200, linestyles='--')
plt.ylim(0, 220)
```

```python
from IPython.display import HTML
HTML('')
```

* CO
    - **Máxima diaria de las medias móviles octohorarias: 10 mg/m³**

```python
# http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.convolve.html
def moving_average(x, N=8):
    return np.convolve(x, np.ones(N)/N, mode='same')
```

```python
plt.plot(moving_average(data1[:, 0]), label='2016')

plt.plot(moving_average(data2[:, 0]), label='2015')

plt.hlines(10, 0, 250, linestyles='--')
plt.ylim(0, 11)

plt.legend()
```

* O3
    - **Máxima diaria de las medias móviles octohorarias: 120 µg/m3**
    - Umbral de información. 180 µg/m3
    - Media horaria. Umbral de alerta. 240 µg/m3

```python
plt.plot(moving_average(data1[:, 2]), label='2016')
#plt.plot(data1[:, 2])

plt.plot(moving_average(data2[:, 2]), label='2015')
#plt.plot(data2[:, 2])

plt.hlines(180, 0, 250, linestyles='--')
plt.ylim(0, 190)

plt.legend()
```

### 4. Scientific functions: SciPy

```
scipy.linalg: ATLAS LAPACK and BLAS libraries

scipy.stats: distributions, statistical functions...

scipy.integrate: integration of functions and ODEs

scipy.optimization: local and global optimization, fitting, root finding...

scipy.interpolate: interpolation, splines...

scipy.fftpack: Fourier trasnforms

scipy.signal, scipy.special, scipy.io
```

#### Temperature data

Now, we will use some temperature data from the Spanish Ministry of Agriculture.

```python
HTML('')
```

The file contains data from 2004 to 2015 (included). Each row corresponds to a day of the year, so evey 365 lines contain data from a whole year*

Note1: 29th February has been removed for leap-years.
Note2: Missing values have been replaced with the immediately prior valid data.

These kind of events are better handled with Pandas!

```python
!head data/M01_Center_Finca_temperature_data_2004_2015.csv
```

    # mean; max; min
    3.49;10.87;-2.75
    5.10;10.88;-2.36
    8.77;11.86;3.80
    6.07;14.77;-1.36
    2.48;12.06;-3.68
    1.69;11.47;-3.81
    2.57;9.29;-3.81
    6.28;11.08;0.49
    11.66;16.24;8.76

```python
# Loading the data
temp_data = np.genfromtxt('data/M01_Center_Finca_temperature_data_2004_2015.csv',
                          skip_header=1,
                          delimiter=';')
```

```python
# Importing SciPy stats
import scipy.stats as st
```

```python
# Applying some functions: describe, mode, mean...
st.describe(temp_data)
```

    DescribeResult(nobs=4380, minmax=(array([ -4.21,  -0.04, -11.88]), array([ 30.59,  42.  ,  21.77])), mean=array([ 14.03399543,  21.22098858,   6.86335388]), variance=array([ 59.25648501,  76.46548678,  44.03205308]), skewness=array([ 0.07277802,  0.10781561, -0.0989195 ]), kurtosis=array([-1.09612434, -1.11634177, -0.94371617]))

```python
st.mode(temp_data)
```

    ModeResult(mode=array([[ 21.  ,  24.77,   0.  ]]), count=array([[10, 11, 53]]))

```python
np.mean(temp_data, axis=0)
```

    array([ 14.03399543,  21.22098858,   6.86335388])

```python
np.median(temp_data, axis=0)
```

    array([ 13.52,  20.62,   7.09])

We can also get information about **percentiles**!

```python
st.scoreatpercentile(temp_data, per=25, axis=0)
```

    array([  7.65 ,  13.665,   1.32 ])

```python
st.percentileofscore(temp_data[:,0], score=0)
```

    1.1415525114155249

```python
st.percentileofscore(temp_data[:,1], score=0)
```

    0.022831050228310501

```python
st.percentileofscore(temp_data[:,2], score=0)
```

    19.315068493150687

##### Rearranging the data

```python
temp_data2 = np.zeros([365, 3, 12])

for year in range(12):
    temp_data2[:, :, year] = temp_data[year*365:(year+1)*365, :]
```

```python
# Calculating mean of mean temp
mean_mean = np.mean(temp_data2[:, 0, :], axis=1)
# max of max
max_max = np.max(temp_data2[:, 1, :], axis=1)
# min of min
min_min = np.min(temp_data2[:, 2, :], axis=1)
```

##### Let's visualize data!

Using matplotlib styles http://matplotlib.org/users/whats_new.html#styles

```python
%matplotlib inline
plt.style.available
```

    ['seaborn-deep',
     'seaborn-ticks',
     'dark_background',
     'grayscale',
     'seaborn-muted',
     'ggplot',
     'seaborn-talk',
     'seaborn-paper',
     'seaborn-poster',
     'seaborn-bright',
     'seaborn-whitegrid',
     'classic',
     'fivethirtyeight',
     'seaborn-pastel',
     'bmh',
     'seaborn-white',
     'seaborn-darkgrid',
     'seaborn-dark',
     'seaborn-colorblind',
     'seaborn-notebook',
     'seaborn-dark-palette']

```python
plt.style.use('ggplot')
```

```python
days = np.arange(1, 366)

plt.fill_between(days, max_max, min_min, alpha=0.7)
plt.plot(days, mean_mean)
plt.xlim(1, 365)
```

Let's see if 2015 was a normal year...

```python
plt.plot(days, temp_data2[:,0,-1], lw=2)
plt.plot(days, mean_mean)
plt.xlim(1, 365)
```

```python
plt.fill_between(days, max_max, min_min, alpha=0.7)
plt.fill_between(days, temp_data2[:,1,-1], temp_data2[:,2,-1],
                 color='purple', alpha=0.5)
plt.plot(days, temp_data2[:,0,-1], lw=2)
plt.xlim(1, 365)
```

For example, lets represent a function over a 2D domain!

For this we will use the contour function, which requires some special inputs...

```python
#we will use numpy functions in order to work with numpy arrays
def funcion(x,y):
    return np.cos(x) + np.sin(y)
```

```python
# 0D: works!
funcion(3,5)
```

    -1.9489167712635838

```python
# 1D: works!
x = np.linspace(0,5, 100)
plt.plot(x, funcion(x,1))
```

In oder to plot the 2D function, we will need a grid.

For 1D domain, we just needed one 1D array containning the X position and another 1D array containing the value.

Now, we will create a grid, a distribution of points covering a surface. For the 2D domain, we will need:
- One 2D array containing the X coordinate of the points.
- One 2D array containing the Y coordinate of the points.
- One 2D array containing the function value at the points.

The three matrices must have the exact same dimensions, because each cell of them represents a particular point.

```python
#We can create the X and Y matrices by hand, or use a function designed to make ir easy:

#we create two 1D arrays of the desired lengths:
x_1d = np.linspace(0, 5, 5)
y_1d = np.linspace(-2, 4, 7)
#And we use the meshgrid function to create the X and Y matrices!
X, Y = np.meshgrid(x_1d, y_1d)
```

```python
X
```

    array([[ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ]])

```python
Y
```

    array([[-2., -2., -2., -2., -2.],
           [-1., -1., -1., -1., -1.],
           [ 0.,  0.,  0.,  0.,  0.],
           [ 1.,  1.,  1.,  1.,  1.],
           [ 2.,  2.,  2.,  2.,  2.],
           [ 3.,  3.,  3.,  3.,  3.],
           [ 4.,  4.,  4.,  4.,  4.]])

Note that with the meshgrid function we can only create rectangular grids

```python
#Using Numpy arrays, calculating the function value at the points is easy!
Z = funcion(X,Y)
```

```python
#Let's plot it!
plt.contour(X, Y, Z)
plt.colorbar()
```

We can try a little more resolution...

```python
x_1d = np.linspace(0, 5, 100)
y_1d = np.linspace(-2, 4, 100)
X, Y = np.meshgrid(x_1d, y_1d)
Z = funcion(X,Y)
plt.contour(X, Y, Z)
plt.colorbar()
```

The countourf function is simmilar, but it colours also between the lines. In both functions, we can manually adjust the number of lines/zones we want to differentiate on the plot.

```python
plt.contourf(X, Y, Z, np.linspace(-2, 2, 6),cmap=plt.cm.Spectral) #With cmap, a color map is specified
plt.colorbar()
```

```python
plt.contourf(X, Y, Z, np.linspace(-2, 2, 100),cmap=plt.cm.Spectral)
plt.colorbar()
```

```python
#We can even combine them!
plt.contourf(X, Y, Z, np.linspace(-2, 2, 100),cmap=plt.cm.Spectral)
plt.colorbar()
cs = plt.contour(X, Y, Z, np.linspace(-2, 2, 9), colors='k')
plt.clabel(cs)

```

These functions can be enormously useful when you want to visualize something.

And remember!

## Always visualize data!

Let's try it with **Real** data!

```python
time_vector = np.loadtxt('data/ligo_tiempos.txt')
frequency_vector =  np.loadtxt('data/ligo_frecuencias.txt')
intensity_matrix =  np.loadtxt('data/ligo_datos.txt')
```

The time and frequency vectors contain the values at which the instrument was reading, and the intensity matrix, the postprocessed strength measured for each frequency at each time.

We need again to create the 2D arrays of coordinates.

```python
time_2D, freq_2D = np.meshgrid(time_vector, frequency_vector)
```

```python
plt.figure(figsize=(10,6)) #We can manually adjust the sice of the picture
plt.contourf(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 200),cmap='bone')
plt.xlabel('time (s)')
plt.ylabel('Frequency (Hz)')
plt.colorbar()
```

Wow! What is that? Let's zoom into it!

```python

plt.figure(figsize=(10,6))
plt.contourf(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 200),cmap = plt.cm.Spectral)
plt.colorbar()
plt.contour(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 9), colors='k')
plt.xlabel('time (s)')
plt.ylabel('Frequency (Hz)')

plt.axis([9.9, 10.05, 0, 300])
```

# IPython Widgets

The IPython Widgets are interactive tools to use in the notebook. They are fun and very useful to quickly understand how different parameters affect a certain function.

This is based on a section of the PyConEs 14 talk by Kiko Correoso "Hacking the notebook": http://nbviewer.jupyter.org/github/kikocorreoso/PyConES14_talk-Hacking_the_Notebook/blob/master/notebooks/Using%20Interact.ipynb

```python
from ipywidgets import interact
```

```python
#Lets define a extremely simple function:
def ejemplo(x):
    print(x)
```

```python
interact(ejemplo, x =10) #Try changing the value of x to True, 'Hello' or ['hello', 'world']
```

    10

    <function __main__.ejemplo>

```python
#We can control the slider values with more precission:
interact(ejemplo, x = (9,10,0.1))
```

    9.0

    <function __main__.ejemplo>

If you want a dropdown menu that passes non-string values to the Python function, you can pass a dictionary. The keys in the dictionary are used for the names in the dropdown menu UI and the values are the arguments that are passed to the underlying Python function.

```python
interact(ejemplo, x={'one': 10, 'two': 20})
```

    10

    <function __main__.ejemplo>

Let's have some fun! We talked before about frequencys and waves. Have you ever learn about AM and FM modulation? It's the process used to send radio communications!

```python
x = np.linspace(-1, 7, 1000)

fig = plt.figure()

plt.subplot(211)#This allows us to display multiple sub-plots, and where to put them
plt.plot(x, np.sin(x))
plt.grid(False)
plt.title("Audio signal: modulator")

plt.subplot(212)
plt.plot(x, np.sin(50 * x))
plt.grid(False)
plt.title("Radio signal: carrier")
```

```python
#Am modulation simply works like this:
am_wave = np.sin(50 * x) * (0.5 + 0.5 * np.sin(x))
plt.plot(x, am_wave)
```

In order to interact with it, we will need to transform it into a function

```python
def am_mod (f_carr=50, f_mod=1, depth=0.5): #The default values will be the starting points of the sliders
    x = np.linspace(-1, 7, 1000)
    am_wave = np.sin(f_carr * x) * (1- depth/2 + depth/2 * np.sin(f_mod * x))

    plt.plot(x, am_wave)

```

```python
interact(am_mod,
        f_carr = (1,100,2),
        f_mod = (0.2, 2, 0.1),
        depth = (0, 1, 0.1))
```

#### Other options...

### 5. Other packages

#### Symbolic calculations with SymPy

SymPy is a Python package for symbolic math. We will not cover it in depth, but let's take a picure of the basics!

```python
# Importación
from sympy import init_session

init_session(use_latex='matplotlib') #We must start calling this function
```

    IPython console for SymPy 1.0 (Python 3.5.1-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at http://docs.sympy.org/1.0/

The basic unit of this package is the symbol. A simbol object has name and graphic representation, which can be different:

```python
coef_traccion = symbols('c_T')
coef_traccion
```

```python
w = symbols('omega')
W = symbols('Omega')
w, W
```

By default, SymPy takes symbols as complex numbers. That can lead to unexpected results in front of certain operations, like logarithms. We can explicitly signal that a symbol is real when we create it. We can also create several symbols at a time.

```python
x, y, z, t = symbols('x y z t', real=True)
x.assumptions0
```

    {'commutative': True,
     'complex': True,
     'hermitian': True,
     'imaginary': False,
     'real': True}

Expressions can be created from symbols:

```python
expr = cos(x)**2 + sin(x)**2
expr
```

```python
simplify(expr)
```

```python
#We can substitute pieces of the expression:
expr.subs(x, y**2)
```

```python
#We can particularize on a certain value:
(sin(x) + 3 * x).subs(x, pi)
```

```python
#We can evaluate the numerical value with a certain precission:
(sin(x) + 3 * x).subs(x, pi).evalf(25)
```

We can manipulate the expression in several ways. For example:

```python
expr1 = (x ** 3 + 3 * y + 2) ** 2
expr1
```

```python
expr1.expand()
```

We can derivate and integrate:

```python
expr = cos(2*x)
expr.diff(x, x, x)
```

```python
expr_xy = y ** 3 * sin(x) ** 2 + x ** 2 * cos(y)
expr_xy
```

```python
diff(expr_xy, x, 2, y, 2)
```

```python
int2 =  1 / sin(x)
integrate(int2)
```

```python
x, a = symbols('x a', real=True)

int3 = 1 / (x**2 + a**2)**2
integrate(int3, x)
```

We also have ecuations and differential ecuations:

```python
a, x, t, C = symbols('a, x, t, C', real=True)
ecuacion = Eq(a * exp(x/t), C)
ecuacion
```

```python
solve(ecuacion ,x)
```

```python
x = symbols('x')
f = Function('y')
ecuacion_dif = Eq(f(x).diff(x,2) + f(x).diff(x) + f(x), cos(x))
ecuacion_dif
```

```python
dsolve(ecuacion_dif, f(x))
```

#### Data Analysis with pandas

Pandas is a package that focus on data structures and data analysis tools. We will not cover it because the next workshop, by Kiko Correoso, will develop it in depth.

#### Machine Learning with scikit-learn

Scikit-learn is a very complete Python package focusing on machin learning, and data mining and analysis. We will not cover it in depth because it will be the focus of many more talks at the PyData.

##### A world of possibilities...

# Thanks for yor attention!

## Any Questions?

---

```python
# Notebook style
from IPython.core.display import HTML
css_file = './static/style.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Source+Sans+Pro|Josefin+Sans:400,700,400italic|Ubuntu+Condensed' rel='stylesheet' type='text/css'>

<style>

#notebook_panel { /* main background */
    background: #f7f7f7;
}

div.cell { /* set cell width */
    width: 900px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 950px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.7em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    font-family: 'Source Sans Pro', sans-serif;
    background-color: rgb(256,256,256);
    font-size: 110%;
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Josefin Sans', serif;
    line-height: 145%;
    font-size: 125%;
    font-weight: 500;
    width:750px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1, .text_cell_render h2, .text_cell_render h3,
.text_cell_render h4, .text_cell_render h5 {
    font-family: 'Ubuntu Condensed', sans-serif;
}

.text_cell_render h1 {    /*Use this for Title*/
    font-weight: 500;
    font-size: 38pt;
    line-height: 100%;
    color: #EE9141;
    text-align: center;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h2 {    /*Use this for Subtitle*/
    margin-top:16px;
    font-size: 32pt;
    font-weight: 500;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    text-align: center;
    font-style: regular;
    color: #459EBA;
}

.text_cell_render h3 {   /*Sections*/
    font-size: 30pt;
    font-weight: 300;
    text-align: left;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    font-style: regular;
    color:  #252525;
}

.text_cell_render h4 {    /*Subsections*/
    font-size: 28pt;
    font-weight: 200;
    text-align: left;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    font-style: regular;
    color:  #377e95;
}

.text_cell_render h5 {    /*Subsubsections*/
    font-size: 20pt;
    font-weight: 300;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
    color:  #e77615;
}

.text_cell_render h6 {    /*Author*/
    font-family: 'Ubuntu Condensed', sans-serif;
    font-weight: 100;
    font-size: 14pt;
    line-height: 100%;
    color: #252525;
    text-align: right;
    margin-bottom: 1px;
    margin-top: 3px;
}

.CodeMirror{
        font-family: 'Duru Sans', sans-serif;
        font-size: 100%;
}

</style>

workshops_materials/20160408_0900_Basic_Python_Packages_for_Science/Basic Python Packages for Science.ipynb


# Univariate Distributions

All the examples in the introduction focused exclusively on the Normal (or Gaussian) Distribution over a single
random variable, but lots of other univariate distributions exist.

The **Copulas** library supports several of them through the `Univariate` subclasses defined within the
`copulas.univariate` package:

* `copulas.univariate.BetaUnivariate`: Implements a Beta distribution.
* `copulas.univariate.GammaUnivariate`: Implements a Gamma distribution.
* `copulas.univariate.GaussianKDE`: Implements a Kernel-Density Estimate using a Gaussian kernel.
* `copulas.univariate.GaussianUnivariate`: Implements a Gaussian (or Normal) distribution.
* `copulas.univariate.TruncatedGaussian`: Implements a Truncated Gaussian distribution.

Let's explore an example of how to use a `Univariate` subclass.

## Univariate Usage Example

In this example we will be focusing on the `BetaUnivariate` class, which implements a [Beta Distribution](https://en.wikipedia.org/wiki/Beta_distribution).

This distribution is parameterized by two positive shape parameters, $α$ and $β$, which in our implementation are called $a$ and $b$.

Additionally, our implementation also uses the same `loc` and `scale` arguments as the underlying `scipy.stats.beta` model that we are using.

For this example, we will be using a simulated dataset that follows the beta distribution with parameters $a = 3$, $b = 1$, $loc = 4$ and $scale = 1$.

```python
import warnings
warnings.filterwarnings('ignore')
```

```python
from copulas.datasets import sample_univariate_beta

data = sample_univariate_beta()
```

```python
data.head()
```

    0    4.796025
    1    4.935189
    2    4.637677
    3    4.945320
    4    4.726815
    dtype: float64

```python
from copulas.visualization import hist_1d

hist_1d(data)
```

### Fitting the model

The first step to use our `BetaUnivariate` model is to fit it to the data by passing the `data` to
its `fit` method.

```python
from copulas.univariate import BetaUnivariate

beta = BetaUnivariate()
beta.fit(data)
```

After the model has been fitted, we can observe the parameters that it has aproximated from the data.

```python
beta._params
```

    {'loc': 4.095558923695414,
     'scale': 0.9044258869808379,
     'a': 2.552323190906777,
     'b': 0.9041091899047261}

### Sampling new data

Once the model is fitted, we can generate new data that follows a similar distribution.

To do this, we can call the `sample` method of our `beta` object, passing the number
of samples that we want to generate.

```python
sampled = beta.sample(1000)
```

We can compare the distribution of the real data to the sampled one by plotting both
histograms together.

```python
from copulas.visualization import compare_1d

compare_1d(data, sampled)
```

### Probability Density

The probability density of a Beta distribution is defined by

\begin{equation}
\frac{x^{\alpha -1}(1-x)^{\beta -1}}
     {B(\alpha ,\beta)}
\end{equation}

where $B(\alpha,\beta)$ is the Beta function defined as

\begin{equation}
B(\alpha, \beta) = \frac{\Gamma (\alpha )\Gamma (\beta )}
                        {\Gamma (\alpha +\beta )}
\end{equation}

and $\Gamma$ is the Gamma function.

The probability density can be computed for an array of data points using the `probability_density` method.

```python
probability_density = beta.pdf(sampled)
```

```python
probability_density[0:5]
```

    array([3.96772858, 1.67572128, 1.90369638, 3.51125542, 0.71732109])

If we plot the probability densities sorted by the sampled values we will get a better notion
of their behavior.

```python
import pandas as pd

pd.DataFrame({
    'data': sampled,
    'probability_density': probability_density
}).sort_values('data').set_index('data').plot()
```

### Cumulative Distribution

The cumulative distribution of Beta distribution is defined by:

\begin{equation}
p = F(x, a, b) = \frac{1}{B(a, b)}\int_{0}^{x}t^{a - 1}{(1 - t)}^{b - 1}dt
\end{equation}

where $B(\alpha,\beta)$ is the Beta function defined above.

The cumulative distribution can be computed for an array of data points using the
`cumulative_distribution` method.

```python
cumulative_distribution = beta.cumulative_distribution(sampled)
```

```python
cumulative_distribution[0:5]
```

    array([0.98185868, 0.4148939 , 0.49927947, 0.95037356, 0.11102835])

Like with probability densities, if we plot the cumulative distribution values
sorted by the input data, we will get a better notion of how it behaves

```python
pd.DataFrame({
    'data': sampled,
    'cumulative distribution': cumulative_distribution
}).sort_values('data').set_index('data').plot()
```

### to_dict and from_dict

All the `Univariate` classes implement a `to_dict` method that allows obtaining
all the parameters that define the distribution in a python dictionary.

```python
parameters = beta.to_dict()
```

```python
parameters
```

    {'loc': 4.095558923695414,
     'scale': 0.9044258869808379,
     'a': 2.552323190906777,
     'b': 0.9041091899047261,
     'type': 'copulas.univariate.beta.BetaUnivariate'}

This `parameters` dictionary can be later on passed to the `Univariate.from_dict` class method,
which will create an instance of our model with the same parameters as before.

```python
from copulas.univariate import Univariate

new_beta = Univariate.from_dict(parameters)
```

We can sample some more data again to see how it still behaves as expected.

```python
new_sampled = new_beta.sample(1000)

compare_1d(data, new_sampled)
```

## Selecting the best Univariate

A part from the specific `Univariate` subclasses, **Copulas** allows you to use the `Univariate` parent
class as a generic distribution.

### Fitting a generic Univariate

When this class is used directly and fitted to the data, it will automatically search for the `Univariate`
subclass that better fits to the data.

Let's see what happens when we fit it to the data that we were using above.

```python
from copulas.univariate import Univariate

univariate = Univariate()
univariate.fit(data)
```

In order to see which subclass has been chosen and with which parameters,
we can call the `to_dict` method.

```python
parameters = univariate.to_dict()
```

```python
parameters
```

    {'loc': 4.095558923695414,
     'scale': 0.9044258869808379,
     'a': 2.552323190906777,
     'b': 0.9041091899047261,
     'type': 'copulas.univariate.beta.BetaUnivariate'}

We can observe here how the `Univariate` class automatically selected the `BetaUnivariate`
as the best distribution for our data, and how it learned the same parameters as before.

Let's go a bit further and see what happens when we fit `Univariate` instances on data
with different distributions:

```python
from copulas.datasets import sample_univariates

data = sample_univariates()
```

```python
data.head()
```

      bernoulli
      bimodal
      uniform
      normal
      degenerate
      exponential
      beta

      0
      0.0
      11.399355
      0.498160
      1.496714
      0.37454
      3.469268
      4.796025

      1
      0.0
      10.924634
      2.802857
      0.861736
      0.37454
      6.010121
      4.935189

      2
      0.0
      10.059630
      1.927976
      1.647689
      0.37454
      4.316746
      4.637677

      3
      0.0
      9.353063
      1.394634
      2.523030
      0.37454
      3.912943
      4.945320

      4
      1.0
      -0.234153
      -0.375925
      0.765847
      0.37454
      3.169625
      4.726815

```python
synth_data = pd.DataFrame()
distributions = []

for column in data.columns:
    real_data = data[column]
    univariate = Univariate()
    univariate.fit(real_data)
    synth_data[column] = univariate.sample(len(real_data))
    distributions.append(univariate.to_dict()['type'])
```

```python
synth_data.head()
```

      bernoulli
      bimodal
      uniform
      normal
      degenerate
      exponential
      beta

      0
      0.126054
      1.950079
      0.833157
      1.154327
      0.37454
      3.014643
      4.990397

      1
      -0.216558
      13.461737
      2.131687
      -0.031510
      0.37454
      3.205208
      4.781987

      2
      0.158551
      1.011978
      0.274088
      1.563969
      0.37454
      4.097697
      4.982117

      3
      0.151797
      9.476947
      0.527291
      0.698796
      0.37454
      3.031567
      4.809642

      4
      1.023261
      0.982005
      1.266706
      0.890644
      0.37454
      5.713887
      4.724303

```python
distributions
```

    ['copulas.univariate.gaussian_kde.GaussianKDE',
     'copulas.univariate.gaussian_kde.GaussianKDE',
     'copulas.univariate.beta.BetaUnivariate',
     'copulas.univariate.gaussian_kde.GaussianKDE',
     'copulas.univariate.beta.BetaUnivariate',
     'copulas.univariate.gamma.GammaUnivariate',
     'copulas.univariate.beta.BetaUnivariate']

```python
compare_1d(data, synth_data)
```

### Recreating the Univariate

As we learned before, the `Univariate.from_dict` class method can be used to recreate our model.

```python
univariate = Univariate()
univariate.fit(data['beta'])
new_model = Univariate.from_dict(parameters)
```

Notice, though, how in this case the model will not be an instance of the class `Univariate` anymore,
but rather an instance of the corresponding subclass, `BetaUnivariate`.

```python
new_model.__class__
```

    copulas.univariate.beta.BetaUnivariate

### Univariate Families

In some cases, we might want to add some constraints in order to restrict the search to only
some `Univariate` distributions.

This can be achieved by creating and passing a list of candidate subclasses to our `Univariate` class.

For example, let's restrict the search over the `bimodal` column of our dataset restricting the
search to only the `BetaUnivariate`, the `GaussianUnivariate` and the `GammaUnivariate`.

```python
from copulas.univariate import GaussianUnivariate, GammaUnivariate

candidates = [BetaUnivariate, GaussianUnivariate, GammaUnivariate]

univariate = Univariate(candidates=candidates)
univariate.fit(data['bimodal'])

univariate.to_dict()
```

    {'loc': 6.830066099361434,
     'scale': 4.811082463652564,
     'type': 'copulas.univariate.gaussian.GaussianUnivariate'}

However, the **Copulas** `Univariate` subclasses are organized in families to make the
application of these constraints to the search easier.

The `Univariate` families are organized in ways:

* PARAMETRIC: Distributions can be either non parametric or parametric.
* BOUNDED: Distributions can be either unbounded, semi-bounded or bounded.

We can see these properties by examining the corresponding attributes of each `Univariate` subclass.

```python
from copulas.univariate import GaussianKDE, TruncatedGaussian

univariates = [
    BetaUnivariate,
    GammaUnivariate,
    GaussianKDE,
    GaussianUnivariate,
    TruncatedGaussian
]

pd.DataFrame([
    [uni.__name__, uni.PARAMETRIC, uni.BOUNDED]
    for uni in univariates
], columns=['Distribution', 'Parametric', 'Bounded'])
```

      Distribution
      Parametric
      Bounded

      0
      BetaUnivariate
      ParametricType.PARAMETRIC
      BoundedType.BOUNDED

      1
      GammaUnivariate
      ParametricType.PARAMETRIC
      BoundedType.SEMI_BOUNDED

      2
      GaussianKDE
      ParametricType.NON_PARAMETRIC
      BoundedType.UNBOUNDED

      3
      GaussianUnivariate
      ParametricType.PARAMETRIC
      BoundedType.UNBOUNDED

      4
      TruncatedGaussian
      ParametricType.PARAMETRIC
      BoundedType.BOUNDED

When searching for the best distribution, instead of building and passing the list
of candidates by hand, we can simply pass the `parametric` or `bounded` value
that we want use.

For example, let's search again for the best distribution over the `bimodal` column of
our dataset, but restricting the search to only `PARAMETRIC` and `BOUNDED` distributions.

```python
from copulas.univariate import ParametricType, BoundedType

univariate = Univariate(
    parametric=ParametricType.PARAMETRIC,
    bounded=BoundedType.BOUNDED
)
univariate.fit(data['bimodal'])

univariate.to_dict()
```

    {'a': -1.5932065697344446,
     'b': 1.2398855571316814e-14,
     'loc': 13.193107687054022,
     'scale': 10.315282059796969,
     'type': 'copulas.univariate.truncated_gaussian.TruncatedGaussian'}


# Non Linear Regression Analysis
> Non Linear Regression Analysis using Google Colab.

- toc: true
- badges: true
- comments: true
- categories: [Non Linear Regression Analysis, Python, Machine Learning]
- image: images/chart-preview.png

# Non Linear Regression Analysis
If the data shows a curvy trend, then linear regression will not produce very accurate results when compared to a non-linear regression because, as the name implies, linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example on python. In this notebook, we fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014.

## Importing required libraries

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

Though Linear regression is very good to solve many problems, it cannot be used for all datasets. First recall how linear regression, could model a dataset. It models a linear relation between a dependent variable y and independent variable x. It had a simple equation, of degree 1, for example y = $2x$ + 3.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
#plt.figure(figsize=(8,6))
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

Non-linear regressions are a relationship between independent variables $x$ and a dependent variable $y$ which result in a non-linear function modeled data. Essentially any relationship that is not linear can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like exponentials, logarithms, fractions, and others. For example: $$ y = \log(x)$$

Or even, more complicated such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic

$$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Exponential

An exponential function with base c is defined by $$ Y = a + b c^X$$ where b ≠0, c > 0 , c ≠1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

Y= np.exp(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Logarithmic

The response $y$ is a results of applying logarithmic map from input $x$'s to output variable $y$. It is one of the simplest form of __log()__: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be polynomial representation of the $x$'s. In general form it would be written as
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Sigmoidal/Logistic

$$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = 1-4/(1+np.power(3, X-2))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

# Non-Linear Regression example

For an example, we're going to try and fit a non-linear model to the datapoints corresponding to China's GDP from 1960 to 2014. We download a dataset with two columns, the first, a year between 1960 and 2014, the second, China's corresponding annual gross domestic income in US dollars for that year.

```python
import numpy as np
import pandas as pd

#downloading dataset
!wget -nv -O china_gdp.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv

df = pd.read_csv("china_gdp.csv")
df.head(10)
```

    2020-04-27 22:31:54 URL:https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv [1218/1218] -> "china_gdp.csv" [1]

      Year
      Value

      0
      1960
      5.918412e+10

      1
      1961
      4.955705e+10

      2
      1962
      4.668518e+10

      3
      1963
      5.009730e+10

      4
      1964
      5.906225e+10

      5
      1965
      6.970915e+10

      6
      1966
      7.587943e+10

      7
      1967
      7.205703e+10

      8
      1968
      6.999350e+10

      9
      1969
      7.871882e+10

### Plotting the Dataset ###
This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it decelerate slightly in the 2010s.

```python
plt.figure(figsize=(8,5))
x_data, y_data = (df["Year"].values, df["Value"].values)
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

### Choosing a model

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta_1(X-\beta_2)}}$$

$\beta_1$: Controls the curve's steepness,

$\beta_2$: Slides the curve on the x-axis.

### Building The Model
Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data
xdata =x_data/max(x_data)
ydata =y_data/max(y_data)
```

#### How we find the best parameters for our fit line?
we can use __curve_fit__ which uses non-linear least squares to fit our sigmoid function, to data. Optimal values for the parameters so that the sum of the squared residuals of sigmoid(xdata, *popt) - ydata is minimized.

popt are our optimized parameters.

```python
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
```

     beta_1 = 690.453017, beta_2 = 0.997207

Now we plot our resulting regression model.

```python
x = np.linspace(1960, 2015, 55)
x = x/max(x)
plt.figure(figsize=(8,5))
y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')
plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

## Practice
Can you calculate what is the accuracy of our model?

```python
# split data into train/test
msk = np.random.rand(len(df)) < 0.8
train_x = xdata[msk]
test_x = xdata[~msk]
train_y = ydata[msk]
test_y = ydata[~msk]

# build the model using train set
popt, pcov = curve_fit(sigmoid, train_x, train_y)

# predict using test set
y_hat = sigmoid(test_x, *popt)

# evaluation
print("Mean absolute error: %.2f" % np.mean(np.absolute(y_hat - test_y)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_hat - test_y) ** 2))
from sklearn.metrics import r2_score
print("R2-score: %.2f" % r2_score(y_hat , test_y) )
```

    Mean absolute error: 0.02
    Residual sum of squares (MSE): 0.00
    R2-score: 0.91


```python
from sympy.physics.mechanics import *
import sympy as sp
mechanics_printing(pretty_print=True)
```

```python
m, l, l01, l02, k1, k2 = sp.symbols(r'm l l_{0\,1} l_{0\,2} k_1 k_2')
t, g, h, w = sp.symbols('t g h w')
v = dynamicsymbols(r'\theta')
dv = dynamicsymbols(r'\theta', 1)
```

```python
x = l * sp.sin(v)
y = -l * sp.cos(v)

dx = x.diff(t)
dy = y.diff(t)
```

```python
V = m*g*y + sp.Rational(1,2)*k1*(sp.sqrt((x+w)**2+(y+l)**2)-l01)**2+sp.Rational(1,2)*k2*(sp.sqrt((x-w)**2+(y+l)**2)-l02)**2
T = sp.Rational(1,2)*m*(dx**2+dy**2)
L = T - V
```

```python
LM = LagrangesMethod(L, [v])
```

```python
soln = LM.form_lagranges_equations()
```

```python
solvedsoln = sp.solve(sp.Eq(soln[0]), v.diff(t,t))
```

```python
solvedsoln
```

```python
sp.latex(solvedsoln)
```

    '\\left [ \\frac{- g m \\sin{\\left (\\theta{\\left (t \\right )} \\right )} + \\frac{k_{1} l l_{0,1} \\sin{\\left (\\theta{\\left (t \\right )} \\right )}}{\\sqrt{- 2 l^{2} \\cos{\\left (\\theta{\\left (t \\right )} \\right )} + 2 l^{2} + 2 l w \\sin{\\left (\\theta{\\left (t \\right )} \\right )} + w^{2}}} - k_{1} l \\sin{\\left (\\theta{\\left (t \\right )} \\right )} + \\frac{k_{1} l_{0,1} w \\cos{\\left (\\theta{\\left (t \\right )} \\right )}}{\\sqrt{- 2 l^{2} \\cos{\\left (\\theta{\\left (t \\right )} \\right )} + 2 l^{2} + 2 l w \\sin{\\left (\\theta{\\left (t \\right )} \\right )} + w^{2}}} - k_{1} w \\cos{\\left (\\theta{\\left (t \\right )} \\right )} + \\frac{k_{2} l l_{0,2} \\sin{\\left (\\theta{\\left (t \\right )} \\right )}}{\\sqrt{- 2 l^{2} \\cos{\\left (\\theta{\\left (t \\right )} \\right )} + 2 l^{2} - 2 l w \\sin{\\left (\\theta{\\left (t \\right )} \\right )} + w^{2}}} - k_{2} l \\sin{\\left (\\theta{\\left (t \\right )} \\right )} - \\frac{k_{2} l_{0,2} w \\cos{\\left (\\theta{\\left (t \\right )} \\right )}}{\\sqrt{- 2 l^{2} \\cos{\\left (\\theta{\\left (t \\right )} \\right )} + 2 l^{2} - 2 l w \\sin{\\left (\\theta{\\left (t \\right )} \\right )} + w^{2}}} + k_{2} w \\cos{\\left (\\theta{\\left (t \\right )} \\right )}}{l m}\\right ]'

```python

```


# Symbolically Understanding Christoffel Symbol and Riemann Curvature Tensor using EinsteinPy

```python
import sympy
from einsteinpy.utils import christoffel
from pprint import pprint
```

### Defining the metric tensor for 3d spherical coordinates

```python
syms = sympy.symbols('r theta phi')
# define the metric for 3d spherical coordinates
metric = [[0 for i in range(3)] for i in range(3)]
metric[0][0] = 1
metric[1][1] = syms[0]**2
metric[2][2] = (syms[0]**2)*(sympy.sin(syms[1])**2)
```

### Calculating the christoffel symbols

```python
ch = christoffel.christoffels(metric, syms)
sympy.Matrix(ch)
```

$\displaystyle \left[\begin{matrix}\left[ 0, \  0, \  0\right] & \left[ 0, \  - r, \  0\right] & \left[ 0, \  0, \  - r \sin^{2}{\left(\theta \right)}\right]\\\left[ 0, \  \frac{1}{r}, \  0\right] & \left[ \frac{1}{r}, \  0, \  0\right] & \left[ 0, \  0, \  - \sin{\left(\theta \right)} \cos{\left(\theta \right)}\right]\\\left[ 0, \  0, \  \frac{1}{r}\right] & \left[ 0, \  0, \  \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}}\right] & \left[ \frac{1}{r}, \  \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}}, \  0\right]\end{matrix}\right]$

```python
ch[1][1][0]
```

$\displaystyle \frac{1}{r}$

### Calculating the Riemann Curvature tensor

```python
rm = christoffel.riemann_curvature_tensor(metric, syms)
sympy.Matrix(rm)
```

$\displaystyle \left[\begin{matrix}\left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right] & \left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right] & \left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right]\\\left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right] & \left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right] & \left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right]\\\left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right] & \left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right] & \left[ \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right], \  \left[ 0, \  0, \  0\right]\right]\end{matrix}\right]$

### Calculating the christoffel symbols for Kerr Spacetime Metric
  - The expressions are unsimplified

```python
kerr_ch = christoffel.kerr_christoffels()
sympy.Matrix(kerr_ch[0][0])
```

$\displaystyle \left[\begin{matrix}0\\\frac{R a r \left(\frac{2 R a r^{2} \sin^{2}{\left(\theta \right)}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2}} - \frac{R a \sin^{2}{\left(\theta \right)}}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}}\right) \sin^{2}{\left(\theta \right)}}{2 \left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right) \left(- \frac{R^{2} a^{2} r^{2} \sin^{4}{\left(\theta \right)}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2}} + \frac{\left(\frac{R r}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}} - 1\right) \left(- a^{2} \left(- R r + a^{2} + r^{2}\right) \sin^{2}{\left(\theta \right)} + \left(a^{2} + r^{2}\right)^{2}\right) \sin^{2}{\left(\theta \right)}}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}}\right)} + \frac{\left(- \frac{2 R r^{2}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2}} + \frac{R}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}}\right) \left(- a^{2} \left(- R r + a^{2} + r^{2}\right) \sin^{2}{\left(\theta \right)} + \left(a^{2} + r^{2}\right)^{2}\right) \sin^{2}{\left(\theta \right)}}{2 \left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right) \left(- \frac{R^{2} a^{2} r^{2} \sin^{4}{\left(\theta \right)}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2}} + \frac{\left(\frac{R r}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}} - 1\right) \left(- a^{2} \left(- R r + a^{2} + r^{2}\right) \sin^{2}{\left(\theta \right)} + \left(a^{2} + r^{2}\right)^{2}\right) \sin^{2}{\left(\theta \right)}}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}}\right)}\\\frac{R a^{2} r \left(- a^{2} \left(- R r + a^{2} + r^{2}\right) \sin^{2}{\left(\theta \right)} + \left(a^{2} + r^{2}\right)^{2}\right) \sin^{3}{\left(\theta \right)} \cos{\left(\theta \right)}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{3} \left(- \frac{R^{2} a^{2} r^{2} \sin^{4}{\left(\theta \right)}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2}} + \frac{\left(\frac{R r}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}} - 1\right) \left(- a^{2} \left(- R r + a^{2} + r^{2}\right) \sin^{2}{\left(\theta \right)} + \left(a^{2} + r^{2}\right)^{2}\right) \sin^{2}{\left(\theta \right)}}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}}\right)} + \frac{R a r \left(- \frac{2 R a^{3} r \sin^{3}{\left(\theta \right)} \cos{\left(\theta \right)}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2}} - \frac{2 R a r \sin{\left(\theta \right)} \cos{\left(\theta \right)}}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}}\right) \sin^{2}{\left(\theta \right)}}{2 \left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right) \left(- \frac{R^{2} a^{2} r^{2} \sin^{4}{\left(\theta \right)}}{\left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2}} + \frac{\left(\frac{R r}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}} - 1\right) \left(- a^{2} \left(- R r + a^{2} + r^{2}\right) \sin^{2}{\left(\theta \right)} + \left(a^{2} + r^{2}\right)^{2}\right) \sin^{2}{\left(\theta \right)}}{a^{2} \cos^{2}{\left(\theta \right)} + r^{2}}\right)}\\0\end{matrix}\right]$

### Calculating the simplified expressions

```python
kerr_ch_simplified = christoffel.simplify_christoffels(kerr_ch)
sympy.Matrix(kerr_ch_simplified[0][0])
```

$\displaystyle \left[\begin{matrix}0\\\frac{R \left(- a^{4} \cos^{2}{\left(\theta \right)} - a^{2} r^{2} \cos^{2}{\left(\theta \right)} + a^{2} r^{2} + r^{4}\right)}{2 \left(a^{2} \cos^{2}{\left(\theta \right)} + r^{2}\right)^{2} \left(- R r + a^{2} + r^{2}\right)}\\- \frac{2 R a^{2} r \sin{\left(2 \theta \right)}}{\left(a^{2} \cos{\left(2 \theta \right)} + a^{2} + 2 r^{2}\right)^{2}}\\0\end{matrix}\right]$


# Gate F

# Classification via Interference

$$\newcommand{\ket}[1]{\left|{#1}\right\rangle}$$
$$\newcommand{\bra}[1]{\left\langle{#1}\right|}$$
$$\newcommand{\braket}[1]{\left\langle{#1}\right\rangle}$$

## Cosine Classifier

Given a training point $(x_{b}, y_{b})$ at random, the probability of $y^{(\text{test})}$ and $y_{b}$ being equal is a function of the similarity between $x^{(\text{test})}$ and $x_{b}$.

Predict a similar target class for similar input features
\begin{align}\label{equation:cosine classifier}
    Pr\left(y^{(\text{test})} = y_{b}\right) = \frac{1}{2}+\frac{\left[d\left(x_{b}, x^{(\text{test})}\right)\right]^2}{2}
\end{align}

Final classification rule:
 \begin{align} \label{classification_rule}
    y^{(\text{test})} =
    \begin{cases} y_{b}, & \mbox{if } Pr\left(y^{(\text{test})} = y_{b}\right) > \frac{1}{2} \\ 1- y_{b}
                      & \mbox{otherwise }
    \end{cases}
\end{align}

## Cosine Classifier (II)

+ the prediction depends on the squared cosine distance between the two vectors

+ max dissimilarity occurs when training and test observations are orthogonal

        $\rightarrow$ uniform probability distribution in the two classes for $y^{(test)}$

+ the cosine classifier performs well only if training and test belongs to the same class

## Quantum Cosine Classifier

The quantum circuit that implements the cosine classifier encodes data into three different registers: the training vector $x^{(i)}$, the training label $y^{(i)}$ and the test point $x^{(test)}$. One more qubit is used to store the prediction.

The algorithm is made of three steps: **State Preparation**, **Swap test**, **Controlled-X**

```python
%matplotlib inline

import sys
sys.path.insert(1, '../')
from Utils import *
from modeling import *

x_train = [random.uniform(-1, 1), random.uniform(0, 1)]
x_train_norm = normalize_custom(x_train)
x_test = [random.uniform(-1, 1), random.uniform(0, 1)]
x_test_norm = normalize_custom(x_test)
class0 = [1,0]

qc = quantum_cosine_classifier(x_train_norm, x_test_norm, class0)
qc.draw(output='mpl', scale=.7)
```

### Step 1: State Preparation

Data are encoded into three different quantum registers. Since each set of data ($x$ and $y$) is stored separately, the state preparation routine can be performed independently for each qubit.

$$ \ket{\Phi_1} = \Big( S_{x_{b}} \otimes S_{x^{(test)}} \otimes S_{y_{b}} \otimes  \mathbb{1}  \Big) \ket{0} \ket{0} \ket{0} \ket{0}  = \ket{x_{b}} \ket{x^{(test)}} \ket{y_{b}} \ket{0}
$$

where $S_{(x)}$ is the routine which encodes in the amplitudes of a qubit a $2$-dimensional real vector $x$.

### Step 2: Execution of the swap test

In the second step, the swap-test transforms the amplitudes of the qubit $y^{(test)}$ as a function of the squared cosine distance:

\begin{align}
\ket{\Phi_2} & =
\big(\mathbb{1} \otimes \mathbb{1} \otimes \mathbb{1} \otimes H\big)
\big(\text{cswap} \otimes \mathbb{1} \otimes C \big)
\big(\mathbb{1} \otimes \mathbb{1} \otimes \mathbb{1} \otimes H\big)
\ket{x_{b}} \ket{x^{(test)}} \ket{y_{b}} \ket{0} \nonumber \\
   & = \frac{1}{2}
   \Big(
   \big( \ket{x_{b}} \ket{x^{(test)}} + \ket{x^{(test)}} \ket{x_{b}} \big)\ket{y_{b}} \ket{0} +
  \big(\ket{x_{b}} \ket{x^{(test)}} - \ket{x^{(test)}} \ket{x_{b}} \big)\ket{y_{b}} \ket{1} \Big)
\end{align}

where $H$ is the Hadamard gate, cswap is the controlled-swap operation which uses the fourth qubit (position of gate *C*) as control  qubit to swap $\ket{x_{b}}$ and $\ket{x^{(test)}}$.
After the execution of the swap test the probability to redout the basis state $\ket{0}$, that is the probability for the *test* observation to be classified in class $0$ is:
\begin{align}
    P(y^{(test)} = \left|0\right\rangle) =
    \frac{1}{2}+\frac{|\langle x_{b}|x^{(test)} \rangle|^2}{2}
\end{align}

### Step 3: Controlled Pauli-*X* gate

The third step consists in applying a controlled Pauli-$X$ rotation using the control qubit that encodes the label of the training vector. This implies that $y^{(test)}$ is left untouched if  $x_{b}$ belongs to the class $0$. Otherwise,
the amplitudes of the $y^{(test)}$ qubit are exchanged,
and the probability $P(y^{(test)} = 1)$ is higher as the similarity between the two vectors increases.

$$
\left|\Phi_3\right\rangle = \big(\mathbb{1} \otimes \mathbb{1} \otimes \text{C-X} \big) \left|\Phi_2\right\rangle
$$

Thus, the probability amplitudes of the two basis states of $y^{(test)}$ lead to a classification rule that is equivalent to the classic cosine classifier.

## Behaviour of Quantum Cosine Classifier

To study the behaviour of the quantum cosine classifier $2000$ random datasets are generated. Each dataset is made  of a training point $(x_{b}, y_{b})$ and a test point $x^{(test)}$.

```python
n_shots = 100
n=1000
x = []
x_err = []
P0 = []
P1 = []

for i in np.arange(n):
    '''Random generated dataset'''
    x_train = [random.uniform(-1, 1), random.uniform(0, 1)]
    x_train_norm = normalize_custom(x_train)
    x_test = [random.uniform(-1, 1), random.uniform(0, 1)]
    x_test_norm = normalize_custom(x_test)

    '''Compute cosine distance and append it to x'''
    d_cos = cosine_similarity([x_train], [x_test])[0][0]
    x.append(d_cos)

    '''If x_train belongs to class 0'''
    qc = quantum_cosine_classifier(x_train_norm, x_test_norm, [1,0])
    r = exec_simulator(qc, n_shots)
    P_q = r['0']/n_shots
    P0.append(P_q)
    d_cos_err = np.sqrt(2*P_q-1)
    x_err.append( d_cos_err )

P0 = np.array(P0)
x = np.array(x)
x_err = np.array(x_err)

order = x.argsort()

x = x[order[::-1]]
P0 = P0[order[::-1]]
x_err = x_err[order[::-1]]

err = [ abs(x1 - x2) for (x1, x2) in zip(abs(x), x_err)]
err = np.array(err)
P1 = 1-P0
```

```python
quantum_cos_random_data(x, P0, P1, err)
```

presentation/.ipynb_checkpoints/Gate F - Quantum Cosine Classifier-checkpoint.ipynb


# MCMC On Posteriors With Improper Priors

MCMC (Markov Chain Monte Carlo) requires a posterior probability distribution to sample from.
This means that one cannot sample from a likelihood, if one wishes to sample the parameters of a model. This is because the likelihood is a probability distribution of the *data*, conditioned on the parameters, rather than a probability distribution of the parameters. If our goal were to sample data sets, given a set of parameters we know to be true, the likelihood would be the right function to use.
However, most inference problems in science are concerned with sampling the parameters of a model, which presumably hold some information of interest about the underlying process that generated the data.

Even so, in many applications, researchers specify the likelihood and then use MCMC to infer the distribution of the parameters, without explicitly specifying the prior. But that doesn't mean the model is without prior. In fact, it cannot be in order for MCMC to work. The implicit assumption in this case is that the priors are flat (all values of the parameters are equally probable) and infinite. This runs the risk of making the posterior improper, that is, diverging, so that it does not integrate to 1. This is in general a bad property to have for a probability distribution, and for MCMC in particular. It's worth noting that this doesn't have to be true: if the likelihood function is well behaved, the posterior might still integrate to 1 even if the prior is improper. The key thing here is that this needs to be checked; for complex problems, this might be hard to do.

Here, I advocate that explicitly specifying the prior is always a good idea, and try to show some ways where not specifying priors can go very wrong.

```python
%matplotlib notebook
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib.cm as cmap

import numpy as np
import scipy.stats
```

## Exponential Decay of a Transient

This example is inspired by Steve Gull, via David MacKay.
Suppose an astronomical transient
occurs at time $t=0$, and has a brightness that
decays exponentially over time:

\begin{align}
\mathrm{Brightness}(t) &=
\left\{
\begin{array}{lr}
Ae^{-\frac{t}{L}}, & t \geq 0\\
0,                 & \mathrm{otherwise}.
\end{array}
\right.
\end{align}

Suppose we observe the object between times $t=t_{\rm min}$ and
$t=t_{\rm max}$, and we want to know the value of $L$, but don't
particularly care about $A$. We observe $N$ photons, and their
arrival times $\{t_1, t_2, ..., t_N\}$.
The probability distribution for the arrival times
$\{t_i\}$
given $L$ (and $N$, which we consider to be prior information)
is
\begin{align}
p(t_1, t_2, ..., t_N | L)
&\propto \prod_{i=1}^N e^{-t_i/L}.
\end{align}
This is the probability distribution for the data given the
parameters, which will give us the likelihood function once
we plug in the observed data. However, we need to normalise
it first, as the observed photon arrival times must be between
$t_{\rm min}$ and $t_{\rm max}$:
\begin{align}
p(t_1, t_2, ..., t_N | L)
&= \prod_{i=1}^N
\frac{e^{-t_i/L}}
     {\int_{t_{\rm min}}^{t_{\rm max}} e^{-t_i/L} dt_i}\\
&= \prod_{i=1}^N
\frac{e^{-t_i/L}}
     {L\left[e^{-t_{\rm min}/L} - e^{-t_{\rm max}/L}\right]}\\
&= L^{-N}\left[e^{-t_{\rm min}/L} - e^{-t_{\rm max}/L}\right]^{-N}
e^{-\frac{\sum_{i=1}^N t_i}{L}}.
\end{align}

Let's code up the likelihood for this problem:

```python
logmin = -100000000.0

class TransientLikelihood(object):

    def __init__(self, times, tmin, tmax):
        self.times = times
        self.tmin = tmin
        self.tmax = tmax
        self.n = times.shape[0]

    def likelihood(self, pars):
        #amp = pars[0] # amplitude is first parameter
        ll = pars[0] # decay time scale is first parameter

        like = np.log(ll**(-self.n)\
                *(np.exp(-self.tmin/ll) - np.exp(-self.tmax/ll))**(-self.n)\
                *np.exp(-self.times.sum()/ll))

        if not np.isfinite(like):
            return logmin
        else:
            return like

    def __call__(self, pars):
        return self.likelihood(pars)
```

Next step: let's make some data:

```python
np.random.seed(0)

ll = 10.3                     # True value of L
n = 20                        # Number of data points
t_min, t_max = 0.0, 15.0
t = []
for i in range(0, n):
    while True:
        tt = -ll*np.log(np.random.rand())
        if tt > t_min and tt < t_max:
            break
    t.append(tt)

t = np.array(t)
```

```python
plt.figure()
plt.hist(t, bins=100, color="k", alpha=0.5)
plt.xlabel("$t$", fontsize=16)
plt.xlim([0.0, 30.0])
plt.ylim([0, 6])
plt.title("Photon Arrival Times")
plt.show()

```

    <IPython.core.display.Javascript object>

What's the likelihood of the data given this model?

```python
llike = TransientLikelihood(t, t_min, t_max)
```

```python
llike([2])
```

    -57.044617751153282

let's plot the log-likelihood as a function of the parameter $L$:

```python
L = np.linspace(1./1000.0, 1000.0, 10001)

like_grid = np.array([llike([l]) for l in L])

plt.figure()
plt.plot(L, like_grid, "k")
plt.xlabel("$L$", fontsize=16)
plt.ylabel("Likelihood")
plt.title("Spot the problem...")
plt.ylim(-80, -10)
plt.show()

```

    <IPython.core.display.Javascript object>

Okay, so it looks like it plateaus to the right. Let's run MCMC:

```python
import emcee
```

```python
start_pars = np.array([200])
start_cov = np.diag(start_pars/100.0)
```

```python
nwalkers = 2
niter = 200
ndim = len(start_pars)
burnin = 50000
```

```python
p0 = np.array([np.random.multivariate_normal(start_pars, start_cov) for
               i in range(nwalkers)])

```

```python
# initialize the sampler
sampler = emcee.EnsembleSampler(nwalkers, ndim, llike, threads=4)

```

```python
pos, prob, state = sampler.run_mcmc(p0, burnin)
```

```python
sampler.chain.shape
```

    (2, 50000, 1)

```python
plt.figure()
plt.plot(sampler.chain[0,:])
#plt.yscale("log")
```

    <IPython.core.display.Javascript object>

    [<matplotlib.lines.Line2D at 0x1124ef0f0>]

That doesn't look like it's converging.

This problem actually gave me headaches during my PhD until
I thought harder about priors and realised why it was happening.
Consider an $N \times N$ image of a patch of sky. Without any noise,
we'd observe the *true* image, defined by the flux $f$ in each pixel:
\begin{align}
\textbf{true image} &= \left\{f_{ij}\right\}
\end{align}
The true total flux of the patch of sky is
\begin{align}
F &= \sum_{i=1}^N \sum_{i=1}^N f_{ij}.
\end{align}

However, due to noise, we observe a perturbed version of the
flux in each pixel. i.e. our data is noisy fluxes $D_{ij}$,
such that
\begin{align}
p(D_{ij} | f_{ij}) &\sim \textbf{Normal}\left(f_{ij}, \sigma^2\right)
\end{align}
Assuming $\sigma$ is known, the likelihood
we'd need to infer the $f$s from the $D$s
is therefore
\begin{align}
p\left(\{D_{ij}\} | \{f_{ij}\}\right)
&\propto \exp
\left[
  -\frac{1}{2\sigma^2}\sum_{i=1}^N\sum_{j=1}^N
                  \left(D_{ij} - f_{ij}\right)^2
\right].
\end{align}

With this likelihood, we can infer the $f$s
(the de-noised image) from the $D$s (the noisy data).
We know fluxes are non-negative, so $f_{ij} \geq 0$.

With an implicit uniform prior on the $f$s,
try MCMC sampling and using the chain of $f$s to
get a posterior for $F$. You should notice something
"wrong" with it, caused by the uniform prior.

#### Making some fake data

Let's start by simulating some data.

```python
def makeGaussian(size, amp, fwhm = 3, center=None):
    """ Make a square gaussian kernel.
    size is the length of a side of the square
    fwhm is full-width-half-maximum, which
    can be thought of as an effective radius.
    """

    x = np.arange(0, size, 1, float)
    y = x[:,np.newaxis]

    if center is None:
        x0 = y0 = size // 2
    else:
        x0 = center[0]
        y0 = center[1]

    return amp*np.exp(-4*np.log(2) * ((x-x0)**2 + (y-y0)**2) / fwhm**2)
```

```python
npix = 100
width = 3
pos = [50, 50]
amp = 10.0

image = makeGaussian(npix, amp, fwhm=width, center=pos)

```

```python
plt.figure()
plt.imshow(image)
```

    <IPython.core.display.Javascript object>

    <matplotlib.image.AxesImage at 0x11f777898>

```python
img_std = 0.1

image = np.random.normal(image, img_std)
```

```python
plt.figure()
plt.imshow(image)
```

    <IPython.core.display.Javascript object>

    <matplotlib.image.AxesImage at 0x115cd4048>

None, so we're cool. Let's now write down the Gaussian likelihood:

```python
class ImageLikelihood(object):

    def __init__(self, image, model, sigma):
        self.image = image
        self.npix = self.image.shape[0]
        self.model = model
        self.sigma = sigma

    def loglikelihood(self, pars):
        mean_model = self.model(np.exp(pars))

        loglike = -(0.5/self.sigma**2.)*np.sum((self.image-mean_model)**2.)

        if not np.isfinite(loglike):
            return logmin

        else:
            return loglike

    def __call__(self, pars):
        return self.loglikelihood(pars)
```

Now we can define our simple flat model:

```python
model = lambda c: makeGaussian(npix, c, fwhm=width, center=pos)
```

```python
mean_model = model(10)
```

And instantiate the likelihood function:

```python
llike = ImageLikelihood(image, model, img_std)
```

```python
llike([10])
```

    -123578335138.00447

Our grid of parameters:

```python
c_test = np.linspace(0, 50, 10000)
```

```python
llike_grid = []
for c in c_test:
    llike_grid.append(llike([c]))

llike_grid = np.array(llike_grid)
```

```python
np.all(np.isfinite(llike_grid))
```

    True

```python
np.all(np.isfinite(c_test))
```

    True

```python
llike_grid
```

    array([ -2.34406620e+04,  -2.34174957e+04,  -2.33942262e+04, ...,
            -6.71751372e+45,  -6.78503264e+45,  -6.85323021e+45])

```python
np.max(llike_grid)
```

    -2487.7037143312227

```python
np.min(llike_grid)
```

    -6.8532302097644669e+45

```python
plt.figure()
plt.plot(c_test, llike_grid)
#plt.ylim(-1e30, -2000)
```

    <IPython.core.display.Javascript object>

    [<matplotlib.lines.Line2D at 0x11b347390>]

```python

```

```python

```

```python

```

* Fit a straight line with the variance as a parameter with a flat prior
* photoionization grid, try to figure out ionization parameter, but emission lines have been reddened by dust, try to feed in an EB-V, end up with a decreasing exponential distribution from zero, can't place limit on colour access, requires some reparametrization of color access based on column density