# ***Introduction to Radar Using Python and MATLAB***
## Andy Harrison - Copyright (C) 2019 Artech House
<br/>

# Circular Array Antenna
***

The circular array is shown in Figure 3.22.  For this type of array, the radiating elements are placed on a circular path with spacing of $a \, \Delta\phi$, where $a$ is the radius, and $\Delta\phi$ is the angular difference between the elements.  The normalized electric field is written as

$$
	E(r, \theta, \phi) = \sum\limits_{n=1}^{N}a_n\, \frac{e^{-j k r_n}}{r_n} \hspace{0.5in} \text{(V/m)},
$$

where

$$
	r_n = \sqrt{r^2 + a^2 - 2\, a\, r\, \cos \psi_n} \hspace{0.5in} \text{(m)}.
$$

For phase terms, $r_n$ is approximated by

$$
	r_n \approx r - a\, \sin\theta\, \cos(\phi - \phi_n) \hspace{0.5in} \text{(m)},
$$

where $\phi_n$ is the angular position of the $n$th element, and is expressed as

\begin{equation}
	\phi_n = \frac{2\pi\, n}{N} \hspace{0.5in} \text{(rad)},
\end{equation}

and for amplitude $r_n \approx r$.  The coefficients, $a_n$, are represented by an amplitude and phase as

$$
	a_n = A_n\, e^{j \alpha_n},
$$

where $A_n$ is the amplitude and $\alpha_n$ is the phase.  The array factor for the circular array is now written as

$$
	{AF}(\theta, \phi) = \sum\limits_{n=1}^{N}A_n\, e^{j [k a \sin\theta\, \cos(\phi-\phi_n) + \alpha_n]} .
$$
***

Begin by getting the library path

```python
import lib_path
```

Set the operating frequency (Hz), the number of elements, the scan angle (theta - degrees), (phi - degrees), and the radius (m)

```python
frequency = 300e6

number_of_elements = 40.0

scan_angle_theta = 30.0

scan_angle_phi = 30.0

radius = 1.1
```

Set up the theta and phi arrays using the `meshgrid` and `linspace` routines from `scipy`

```python
from numpy import linspace, meshgrid, radians

from scipy.constants import pi

n = 360

m = int(n / 8)

theta, phi = meshgrid(linspace(0.0 * pi, 0.5 * pi, n), linspace(0.0, 2.0 * pi, n))
```

Set up the keyword args

```python
kwargs = {'number_of_elements': number_of_elements,

          'scan_angle_theta': radians(scan_angle_theta),

          'scan_angle_phi': radians(scan_angle_phi),

          'radius': radius,

          'frequency': frequency,

          'theta': theta,

          'phi': phi}
```

Calculate the array factor using the `circular_uniform` routines

```python
from Libs.antenna.array import circular_uniform

af = circular_uniform.array_factor(**kwargs)
```

Create the U-V coordinates for displaying the antenna pattern

```python
from numpy import sin, cos

uu = sin(theta) * cos(phi)

vv = sin(theta) * sin(phi)
```

Create the color plot of the array factor using the `matplotlib` routines

```python
from matplotlib import pyplot as plt

# Set the figure size

plt.rcParams["figure.figsize"] = (15, 10)

# Set up the axes

fig, axes1 = plt.subplots()

im = axes1.pcolor(uu, vv, abs(af), cmap="jet", shading = 'auto')

cbar = fig.colorbar(im, ax=axes1, orientation='vertical')

cbar.set_label("Normalized Electric Field (V/m)", size=10)

# Set the x- and y-axis labels

axes1.set_xlabel("U (sines)", size=12)

axes1.set_ylabel("V (sines)", size=12)

# Set the plot title

plt.title('Circular Array - Array Factor', size=14)
```

Create the contour plot using the `matplotlib` routines

```python
# Set the figure size

plt.rcParams["figure.figsize"] = (12, 12)

plt.contour(uu, vv, abs(af), 20, cmap="jet", vmin=-0.2, vmax=1.0)

plt.grid(linestyle=':', linewidth=0.5)

# Set the x- and y-axis labels

plt.xlabel("U (sines)", size=12)

plt.ylabel("V (sines)", size=12)

# Set the plot title

plt.title('Circular Array - Array Factor', size=14)
```

Create the line plot using the `matplotlib` routines

```python
from numpy import degrees, log10

plt.plot(degrees(theta[0]), 20.0 * log10(abs(af[m])), '', label='E Plane')

plt.plot(degrees(theta[0]), 20.0 * log10(abs(af[0])), '--', label='H Plane')

# Set the y axis limit

plt.ylim(-60, 5)

# Set the x and y axis labels

plt.xlabel("Theta (degrees)", size=12)

plt.ylabel("Array Factor (dB)", size=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Place the legend

plt.legend(loc='upper right', prop={'size': 10})

# Set the plot title

plt.title('Circular Array - Array Factor', size=14)
```


# Homework 5
## Due Date:  Tuesday, October 3rd at 11:59 PM

# Problem 1
We discussed documentation and testing in lecture and also briefly touched on code coverage.  You must write tests for your code for your final project (and in life).  There is a nice way to automate the testing process called continuous integration (CI).

This problem will walk you through the basics of CI and show you how to get up and running with some CI software.

### Continuous Integration
The idea behind continuous integration is to automate away the testing of your code.

We will be using it for our projects.

The basic workflow goes something like this:

1. You work on your part of the code in your own branch or fork
2. On every commit you make and push to GitHub, your code is automatically tested on a fresh machine on Travis CI. This ensures that there are no specific dependencies on the structure of your machine that your code needs to run and also ensures that your changes are sane
3. Now you submit a pull request to `master` in the main repo (the one you're hoping to contribute to). The repo manager creates a branch off `master`.
4. This branch is also set to run tests on Travis. If all tests pass, then the pull request is accepted and your code becomes part of master.

We use GitHub to integrate our roots library with Travis CI and Coveralls. Note that this is not the only workflow people use. Google git..github..workflow and feel free to choose another one for your group.

### Part 1:  Create a repo
Create a public GitHub repo called `cs207test` and clone it to your local machine.

**Note:** No need to do this in Jupyter.

```bash
%%bash
git clone https://github.com/xuwd11/cs207test
```

    Cloning into 'cs207test'...

### Part 2:  Create a roots library
Use the example from lecture 7 to create a file called `roots.py`, which contains the `quad_roots` and `linear_roots` functions (along with their documentation).

Also create a file called `test_roots.py`, which contains the tests from lecture.

All of these files should be in your newly created `cs207test` repo.  **Don't push yet!!!**

```python
%%file cs207test/roots.py
def linear_roots(a=1.0, b=0.0):
    """Returns the roots of a linear equation: ax+ b = 0.

    INPUTS
    =======
    a: float, optional, default value is 1
       Coefficient of linear term
    b: float, optional, default value is 0
       Coefficient of constant term

    RETURNS
    ========
    roots: 1-tuple of real floats
       Has the form (root) unless a = 0
       in which case a ValueError exception is raised

    EXAMPLES
    =========
    >>> linear_roots(1.0, 2.0)
    -2.0
    """
    if a == 0:
        raise ValueError("The linear coefficient is zero.  This is not a linear equation.")
    else:
        return ((-b / a))

def quad_roots(a=1.0, b=2.0, c=0.0):
    """Returns the roots of a quadratic equation: ax^2 + bx + c = 0.

    INPUTS
    =======
    a: float, optional, default value is 1
       Coefficient of quadratic term
    b: float, optional, default value is 2
       Coefficient of linear term
    c: float, optional, default value is 0
       Constant term

    RETURNS
    ========
    roots: 2-tuple of complex floats
       Has the form (root1, root2) unless a = 0
       in which case a ValueError exception is raised

    EXAMPLES
    =========
    >>> quad_roots(1.0, 1.0, -12.0)
    ((3+0j), (-4+0j))
    """
    import cmath # Can return complex numbers from square roots
    if a == 0:
        raise ValueError("The quadratic coefficient is zero.  This is not a quadratic equation.")
    else:
        sqrtdisc = cmath.sqrt(b * b - 4.0 * a * c)
        r1 = -b + sqrtdisc
        r2 = -b - sqrtdisc
        return (r1 / 2.0 / a, r2 / 2.0 / a)
```

    Overwriting cs207test/roots.py

```python
%%file cs207test/test_roots.py
import roots

def test_quadroots_result():
    assert roots.quad_roots(1.0, 1.0, -12.0) == ((3+0j), (-4+0j))

def test_quadroots_types():
    try:
        roots.quad_roots("", "green", "hi")
    except TypeError as err:
        assert(type(err) == TypeError)

def test_quadroots_zerocoeff():
    try:
        roots.quad_roots(a=0.0)
    except ValueError as err:
        assert(type(err) == ValueError)

def test_linearoots_result():
    assert roots.linear_roots(2.0, -3.0) == 1.5

def test_linearroots_types():
    try:
        roots.linear_roots("ocean", 6.0)
    except TypeError as err:
        assert(type(err) == TypeError)

def test_linearroots_zerocoeff():
    try:
        roots.linear_roots(a=0.0)
    except ValueError as err:
        assert(type(err) == ValueError)
```

    Overwriting cs207test/test_roots.py

### Part 3:  Create an account on Travis CI and Start Building

#### Part A:
Create an account on Travis CI and set your `cs207test` repo up for continuous integration once this repo can be seen on Travis.

#### Part B:
Create an instruction to Travis to make sure that

1. python is installed
2. its python 3.5
3. pytest is installed

The file should be called `.travis.yml` and should have the contents:
```yml
language: python
python:
    - "3.5"
before_install:
    - pip install pytest pytest-cov
script:
    - pytest
```

You should also create a configuration file called `setup.cfg`:
```cfg
[tool:pytest]
addopts = --doctest-modules --cov-report term-missing --cov roots
```

#### Part C:
Push the new changes to your `cs207test` repo.

At this point you should be able to see your build on Travis and if and how your tests pass.

### Part 4:  Coveralls Integration
In class, we also discussed code coverage.  Just like Travis CI runs tests automatically for you, Coveralls automatically checks your code coverage.  One minor drawback of Coveralls is that it can only work with public GitHub accounts.  However, this isn't too big of a problem since your projects will be public.

#### Part A:
Create an account on [`Coveralls`](https://coveralls.zendesk.com/hc/en-us), connect your GitHub, and turn Coveralls integration on.

#### Part B:
Update your the `.travis.yml` file as follows:
```yml
language: python
python:
    - "3.5"
before_install:
    - pip install pytest pytest-cov
    - pip install coveralls
script:
    - py.test
after_success:
    - coveralls
```

Be sure to push the latest changes to your new repo.

### Part 5:  Update README.md in repo
You can have your GitHub repo reflect the build status on Travis CI and the code coverage status from Coveralls.  To do this, you should modify the `README.md` file in your repo to include some badges.  Put the following at the top of your `README.md` file:

```
[](https://travis-ci.org/dsondak/cs207testing.svg?branch=master)

[](https://coveralls.io/github/dsondak/cs207testing?branch=master)
```

Of course, you need to make sure that the links are to your repo and not mine.  You can find embed code on the Coveralls and Travis CI sites.

---

# Problem 2
Write a Python module for reaction rate coefficients.  Your module should include functions for constant reaction rate coefficients, Arrhenius reaction rate coefficients, and modified Arrhenius reaction rate coefficients.  Here are their mathematical forms:
\begin{align}
  &k_{\textrm{const}}   = k \tag{constant} \\
  &k_{\textrm{arr}}     = A \exp\left(-\frac{E}{RT}\right) \tag{Arrhenius} \\
  &k_{\textrm{mod arr}} = A T^{b} \exp\left(-\frac{E}{RT}\right) \tag{Modified Arrhenius}
\end{align}

Test your functions with the following paramters:  $A = 10^7$, $b=0.5$, $E=10^3$.  Use $T=10^2$.

A few additional comments / suggestions:
* The Arrhenius prefactor $A$ is strictly positive
* The modified Arrhenius parameter $b$ must be real
* $R = 8.314$ is the ideal gas constant.  It should never be changed (except to convert units)
* The temperature $T$ must be positive (assuming a Kelvin scale)
* You may assume that units are consistent
* Document each function!
* You might want to check for overflows and underflows

**Recall:** A Python module is a `.py` file which is not part of the main execution script.  The module contains several functions which may be related to each other (like in this problem).  Your module will be importable via the execution script.  For example, suppose you have called your module `reaction_coeffs.py` and your execution script `kinetics.py`.  Inside of `kinetics.py` you will write something like:
```python
import reaction_coeffs
# Some code to do some things
# :
# :
# :
# Time to use a reaction rate coefficient:
reaction_coeffs.const() # Need appropriate arguments, etc
# Continue on...
# :
# :
# :
```
Be sure to include your module in the same directory as your execution script.

```python
%%file reaction_coeffs.py
import numpy as np

def const(k):
    '''Returen the constant reaction rate coefficient k_const.

    INPUTS
    =======
    k: float
       Rection rate coefficient.

    RETURNS
    ========
    k_const: float, except the following cases:
       If k <= 0, a ValueError exception will be raised;
       if k = float('inf'), an OverflowError exception will be raised.

    EXAMPLES
    =========
    >>> const(100)
    100
    '''
    if k <= 0:
        raise ValueError('The reaction rate coefficient must be positive.')
    if abs(k) == float('inf'):
        raise OverflowError
    return k

def arr(A, E, T, R=8.314):
    '''Returns the Arrhenius reaction rate coefficient.

    INPUTS
    =======
    A: float
       The Arrhenius prefactor.
    E: float
       The activation energy for the reaction (in the same unit as R*T).
    T: float
       The absolute temperature (in Kelvins).
    R: float, optional, default value is 8.314
       The universal gas constant.

    RETURNS
    ========
    k_arr: float, except the following cases:
       If A <= 0, a ValueError exception will be raised;
       if T <= 0, a ValueError exception will be raised;
       if R <= 0, a ValueError exception will be raised;
       if A*exp(-E/R/T) = float('inf'), an OverflowError exception will be raised.

    EXAMPLES
    =========
    >>> arr(10**7, 10**3, 10**2)
    3003549.0889639617
    '''
    if A <= 0:
        raise ValueError('The Arrhenius prefactor A must be positive.')
    if T <= 0:
        raise ValueError('The temperature T must be positive.')
    if R <= 0:
        raise ValueError('The ideal gas constant R must be positive.')
    if R != 8.314:
        print('Warning! The ideal gas constant R has been changed from the default value (8.314).')
    k_arr = A * np.exp(-E/R/T)
    if k_arr == float('inf'):
        raise OverflowError
    if k_arr == 0:
        print('Warning! An underflow error might occur.')
    return k_arr

def mod_arr(A, b, E, T, R=8.314):
    '''Returns the modified Arrhenius reaction rate coefficient.

    INPUTS
    =======
    A: float
       The Arrhenius prefactor.
    b: float
       The modified Arrhenius parameter. If b is not a float number, a conversion is attempted.
    E: float
       The activation energy for the reaction (in the same unit as R*T).
    T: float
       The absolute temperature (in Kelvins).
    R: float, optional, default value is 8.314
       The universal gas constant.

    RETURNS
    ========
    k_mod_arr: float, except the following cases:
       If A <= 0, a ValueError exception will be raised;
       if b cannot be converted to a float number, a ValueError exception will be raised;
       if T <= 0, a ValueError exception will be raised;
       if R <= 0, a ValueError exception will be raised;
       if A*exp(-E/R/T) = float('inf'), an OverflowError exception will be raised.

    EXAMPLES
    =========
    >>> mod_arr(10**7, 0.5, 10**3, 10**2)
    30035490.889639616
    '''
    if A <= 0:
        raise ValueError('The Arrhenius prefactor A must be positive.')
    try:
        b = float(b)
    except:
        raise ValueError('The modified Arrhenius parameter b must be real.')
    if T <= 0:
        raise ValueError('The temperature T must be positive.')
    if R <= 0:
        raise ValueError('The ideal gas constant R must be positive.')
    if R != 8.314:
        print('Warning! The ideal gas constant R has been changed from the default value (8.314).')
    k_mod_arr = A*T**b*np.exp(-E/R/T)
    if k_mod_arr == float('inf'):
        raise OverflowError
    if k_mod_arr == 0:
        print('Warning! An underflow error might occur.')
    return k_mod_arr
```

    Writing reaction_coeffs.py

```python
# Test
import reaction_coeffs
def test_mod_arr():
    assert reaction_coeffs.mod_arr(10**7, 0.5, 10**3, 10**2) == 30035490.889639616
test_mod_arr()
```

---

# Problem 3
Write a function that returns the **progress rate** for a reaction of the following form:
\begin{align}
  \nu_{A} A + \nu_{B} B \longrightarrow \nu_{C} C.
\end{align}
Order your concentration vector so that
\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    \left[A\right] \\
    \left[B\right] \\
    \left[C\right]
  \end{bmatrix}
\end{align}

Test your function with
\begin{align}
  \nu_{i}^{\prime} =
  \begin{bmatrix}
    2.0 \\
    1.0 \\
    0.0
  \end{bmatrix}
  \qquad
  \mathbf{x} =
  \begin{bmatrix}
    1.0 \\
    2.0 \\
    3.0
  \end{bmatrix}
  \qquad
  k = 10.
\end{align}

You must document your function and write some tests in addition to the one suggested.  You choose the additional tests, but you must have at least one doctest in addition to a suite of unit tests.

```python
def progress_rate_1(nu, x, k):
    '''Returns the progress rate for a reaction of the form: nu_A A + nu_B B -> nu_C C

    INPUTS
    =======
    nu: 3-element list or array
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of
       species A, B and C
    x: 3-element list or array
       Concentration vector which specifies the concentrations of species A, B and C
    k: float
       reaction rate coefficient

    RETURNS
    ========
    omega: float, except the following cases:
       If nu or x is not a 3-element list or array, a TypeError will be raised;
       if nu contains non-positive element(s), a ValueError will be raised;
       if x contains negative element(s), a ValueError will be raised;
       if k <= 0, a ValueError will be raised.

    EXAMPLES
    >>> progress_rate_1([2, 1, 0], [1, 2, 3], 10)
    20
    >>> progress_rate_1([1, 1, 0], [4, 2, 3], 10)
    80
    '''
    try:
        if len(nu) != 3:
            raise TypeError('nu must be a 3 element list or array.')
    except:
        raise TypeError('nu must be a 3 element list or array.')
    if not all([nu_ >= 0 for nu_ in nu]):
        raise ValueError('All elements in nu must be non-negative.')
    try:
        if len(x) != 3:
            raise TypeError('x must be a 3 element list or array.')
    except:
        raise TypeError('x must be a 3 element list or array.')
    if any([x_ < 0 for x_ in x]):
        raise ValueError('All elements in x must be non-negative.')
    if k <= 0:
        raise ValueError('k must be positive.')
    omega = k*x[0]**nu[0]*x[1]**nu[1]
    return omega
```

```python
# Tests
def test_progress_rate_1_types():
    try:
        progress_rate_1([1, 1], [1, 1, 1], 10)
    except TypeError as err:
        assert(type(err) == TypeError)

def test_progress_rate_1_values():
    try:
        progress_rate_1([-1, 1, 1], [1, 1, 1], 10)
    except ValueError as err:
        assert(type(err) == ValueError)

test_progress_rate_1_types()
test_progress_rate_1_values()
```

```python
# doctest
import doctest
doctest.testmod(verbose=True)
```

    Trying:
        progress_rate_1([2, 1, 0], [1, 2, 3], 10)
    Expecting:
        20
    ok
    Trying:
        progress_rate_1([1, 1, 0], [4, 2, 3], 10)
    Expecting:
        80
    ok
    4 items had no tests:
        __main__
        __main__.test_mod_arr
        __main__.test_progress_rate_1_types
        __main__.test_progress_rate_1_values
    1 items passed all tests:
       2 tests in __main__.progress_rate_1
    2 tests in 5 items.
    2 passed and 0 failed.
    Test passed.

    TestResults(failed=0, attempted=2)

---
# Problem 4
Write a function that returns the **progress rate** for a system of reactions of the following form:
\begin{align}
  \nu_{11}^{\prime} A + \nu_{21}^{\prime} B \longrightarrow \nu_{31}^{\prime\prime} C \\
  \nu_{12}^{\prime} A + \nu_{32}^{\prime} C \longrightarrow \nu_{22}^{\prime\prime} B + \nu_{32}^{\prime\prime} C
\end{align}
Note that $\nu_{ij}^{\prime}$ represents the stoichiometric coefficient of reactant $i$ in reaction $j$ and $\nu_{ij}^{\prime\prime}$ represents the stoichiometric coefficient of product $i$ in reaction $j$.  Therefore, in this convention, I have ordered my vector of concentrations as
\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    \left[A\right] \\
    \left[B\right] \\
    \left[C\right]
  \end{bmatrix}.
\end{align}

Test your function with
\begin{align}
  \nu_{ij}^{\prime} =
  \begin{bmatrix}
    1.0 & 2.0 \\
    2.0 & 0.0 \\
    0.0 & 2.0
  \end{bmatrix}
  \qquad
  \nu_{ij}^{\prime\prime} =
  \begin{bmatrix}
    0.0 & 0.0 \\
    0.0 & 1.0 \\
    2.0 & 1.0
  \end{bmatrix}
  \qquad
  \mathbf{x} =
  \begin{bmatrix}
    1.0 \\
    2.0 \\
    1.0
  \end{bmatrix}
  \qquad
  k = 10.
\end{align}

You must document your function and write some tests in addition to the one suggested.  You choose the additional tests, but you must have at least one doctest in addition to a suite of unit tests.

```python
import numpy as np

def progress_rate_2(nu_1, nu_2, x, k):
    '''Returns the progress rate for a system of reactions of the form:
       nu'_11 A + nu'_21 B -> nu''_31 C
       nu'_12 A + nu'_32 C -> nu''_22 B + nu''_32 C

    INPUTS
    =======
    nu_1: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of reactants.
       If nu_1 is not an array, a conversion is attempted.
    nu_2: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of products.
       If nu_2 is not an array, a conversion is attempted.
    x: 3-element list or array
       Concentration vector which specifies the concentrations of species A, B and C
    k: float
       reaction rate coefficient

    RETURNS
    ========
    omega: 2-element tuple
       Has the form (float, float) which corresponds to the progress rates of 2 reactions, except
       the following cases:
       If nu_1 or nu_2 cannot be converted to an array of shape (3, 2), a TypeError will be raised;
       if nu_1 or nu_2 contains negative element(s), a ValueError will be raised;
       if x is not a 3-element list or array, a TypeError will be raised;
       if x contains negative element(s), a ValueError will be raised;
       if k <= 0, a ValueError will be raised.

    EXAMPLES
    >>> progress_rate_2([[1, 2], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    (40, 10)
    >>> progress_rate_2([[1, 1], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 3], 10)
    (40, 90)
    '''
    try:
        nu_1 = np.array(nu_1)
        if nu_1.shape != (3, 2):
            raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    try:
        nu_2 = np.array(nu_2)
        if nu_2.shape != (3, 2):
            raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    if np.any(nu_1 < 0):
        raise ValueError('All elements in nu_1 must be non-negative.')
    if np.any(nu_2 < 0):
        raise ValueError('All elements in nu_2 must be non-negative.')
    try:
        if len(x) != 3:
            raise TypeError('x must be a 3 element list or array.')
    except:
        raise TypeError('x must be a 3 element list or array.')
    if any([x_ < 0 for x_ in x]):
        raise ValueError('All elements in x must be non-negative.')
    if k <= 0:
        raise ValueError('k must be positive.')
    omega = (k*np.prod([x[i]**nu_1[i, 0] for i in range(3)]), k*np.prod([x[i]**nu_1[i, 1] for i in range(3)]))
    return omega
```

```python
# Tests
def test_progress_rate_2_types():
    try:
        progress_rate_2([[1, 2], [2, 0], [0, 3, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    except TypeError as err:
        assert(type(err) == TypeError)

def test_progress_rate_2_values():
    try:
        progress_rate_2([[1, -1], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    except ValueError as err:
        assert(type(err) == ValueError)

test_progress_rate_2_types()
test_progress_rate_2_values()
```

```python
# doctest
import doctest
doctest.testmod(verbose=True)
```

    Trying:
        progress_rate_1([2, 1, 0], [1, 2, 3], 10)
    Expecting:
        20
    ok
    Trying:
        progress_rate_1([1, 1, 0], [4, 2, 3], 10)
    Expecting:
        80
    ok
    Trying:
        progress_rate_2([[1, 2], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    Expecting:
        (40, 10)
    ok
    Trying:
        progress_rate_2([[1, 1], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 3], 10)
    Expecting:
        (40, 90)
    ok
    6 items had no tests:
        __main__
        __main__.test_mod_arr
        __main__.test_progress_rate_1_types
        __main__.test_progress_rate_1_values
        __main__.test_progress_rate_2_types
        __main__.test_progress_rate_2_values
    2 items passed all tests:
       2 tests in __main__.progress_rate_1
       2 tests in __main__.progress_rate_2
    4 tests in 8 items.
    4 passed and 0 failed.
    Test passed.

    TestResults(failed=0, attempted=4)

---
# Problem 5
Write a function that returns the **reaction rate** of a system of irreversible reactions of the form:
\begin{align}
  \nu_{11}^{\prime} A + \nu_{21}^{\prime} B &\longrightarrow \nu_{31}^{\prime\prime} C \\
  \nu_{32}^{\prime} C &\longrightarrow \nu_{12}^{\prime\prime} A + \nu_{22}^{\prime\prime} B
\end{align}

Once again $\nu_{ij}^{\prime}$ represents the stoichiometric coefficient of reactant $i$ in reaction $j$ and $\nu_{ij}^{\prime\prime}$ represents the stoichiometric coefficient of product $i$ in reaction $j$.  In this convention, I have ordered my vector of concentrations as
\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    \left[A\right] \\
    \left[B\right] \\
    \left[C\right]
  \end{bmatrix}
\end{align}

Test your function with
\begin{align}
  \nu_{ij}^{\prime} =
  \begin{bmatrix}
    1.0 & 0.0 \\
    2.0 & 0.0 \\
    0.0 & 2.0
  \end{bmatrix}
  \qquad
  \nu_{ij}^{\prime\prime} =
  \begin{bmatrix}
    0.0 & 1.0 \\
    0.0 & 2.0 \\
    1.0 & 0.0
  \end{bmatrix}
  \qquad
  \mathbf{x} =
  \begin{bmatrix}
    1.0 \\
    2.0 \\
    1.0
  \end{bmatrix}
  \qquad
  k = 10.
\end{align}

You must document your function and write some tests in addition to the one suggested.  You choose the additional tests, but you must have at least one doctest in addition to a suite of unit tests.

```python
import numpy as np

def reaction_rate_1(nu_1, nu_2, x, k):
    '''Returns the reaction rates for species in a system of reactions of the form:
       nu'_11 A + nu'_21 B -> nu''_31 C
       nu'_32 C -> nu''_12 A + nu''_22 B

    INPUTS
    =======
    nu_1: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of reactants.
       If nu_1 is not an array, a conversion is attempted.
    nu_2: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of products.
       If nu_2 is not an array, a conversion is attempted.
    x: 3-element list or array
       Concentration vector which specifies the concentrations of species A, B and C
    k: float
       reaction rate coefficient

    RETURNS
    ========
    f: 3-element array
       Has the form (float, float, float) which corresponds to the reaction rates of species A, B, C,
       except the following cases:
       If nu_1 or nu_2 cannot be converted to an array of shape (3, 2), a TypeError will be raised;
       if nu_1 or nu_2 contains negative element(s), a ValueError will be raised;
       if x is not a 3-element list or array, a TypeError will be raised;
       if x contains negative element(s), a ValueError will be raised;
       if k <= 0, a ValueError will be raised.

    EXAMPLES
    >>> reaction_rate_1([[1, 0], [2, 0], [0, 2]], [[0, 1], [0, 2], [1, 0]], [1, 2, 1], 10)
    array([-30, -60,  20])
    >>> reaction_rate_1([[1, 0], [2, 0], [0, 1]], [[0, 1], [0, 2], [1, 0]], [1, 2, 3], 10)
    array([-10, -20,  10])
    '''
    try:
        nu_1 = np.array(nu_1)
        if nu_1.shape != (3, 2):
            raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    try:
        nu_2 = np.array(nu_2)
        if nu_2.shape != (3, 2):
            raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    if np.any(nu_1 < 0):
        raise ValueError('All elements in nu_1 must be non-negative.')
    if np.any(nu_2 < 0):
        raise ValueError('All elements in nu_2 must be non-negative.')
    try:
        if len(x) != 3:
            raise TypeError('x must be a 3 element list or array.')
    except:
        raise TypeError('x must be a 3 element list or array.')
    if any([x_ < 0 for x_ in x]):
        raise ValueError('All elements in x must be non-negative.')
    if k <= 0:
        raise ValueError('k must be positive.')
    omega = progress_rate_2(nu_1, nu_2, x, k)
    omega = np.array(omega).reshape((len(omega), 1))
    f = np.sum((np.dot(nu_2, omega) - np.dot(nu_1, omega)), axis=1)
    return f
```

```python
# Tests
def test_reaction_rate_1_types():
    try:
        reaction_rate_1([[1, 2], [2, 0], [0, 3, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    except TypeError as err:
        assert(type(err) == TypeError)

def test_reaction_rate_1_values():
    try:
        reaction_rate_1([[1, -1], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    except ValueError as err:
        assert(type(err) == ValueError)

test_reaction_rate_1_types()
test_reaction_rate_1_values()
```

```python
# doctest
import doctest
doctest.testmod(verbose=True)
```

    Trying:
        progress_rate_1([2, 1, 0], [1, 2, 3], 10)
    Expecting:
        20
    ok
    Trying:
        progress_rate_1([1, 1, 0], [4, 2, 3], 10)
    Expecting:
        80
    ok
    Trying:
        progress_rate_2([[1, 2], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    Expecting:
        (40, 10)
    ok
    Trying:
        progress_rate_2([[1, 1], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 3], 10)
    Expecting:
        (40, 90)
    ok
    Trying:
        reaction_rate_1([[1, 0], [2, 0], [0, 2]], [[0, 1], [0, 2], [1, 0]], [1, 2, 1], 10)
    Expecting:
        array([-30, -60,  20])
    ok
    Trying:
        reaction_rate_1([[1, 0], [2, 0], [0, 1]], [[0, 1], [0, 2], [1, 0]], [1, 2, 3], 10)
    Expecting:
        array([-10, -20,  10])
    ok
    8 items had no tests:
        __main__
        __main__.test_mod_arr
        __main__.test_progress_rate_1_types
        __main__.test_progress_rate_1_values
        __main__.test_progress_rate_2_types
        __main__.test_progress_rate_2_values
        __main__.test_reaction_rate_1_types
        __main__.test_reaction_rate_1_values
    3 items passed all tests:
       2 tests in __main__.progress_rate_1
       2 tests in __main__.progress_rate_2
       2 tests in __main__.reaction_rate_1
    6 tests in 11 items.
    6 passed and 0 failed.
    Test passed.

    TestResults(failed=0, attempted=6)

---
# Problem 6
Put parts 3, 4, and 5 in a module called `chemkin`.

Next, pretend you're a client who needs to compute the reaction rates at three different temperatures ($T = \left\{750, 1500, 2500\right\}$) of the following system of irreversible reactions:
\begin{align}
  2H_{2} + O_{2} \longrightarrow 2OH + H_{2} \\
  OH + HO_{2} \longrightarrow H_{2}O + O_{2} \\
  H_{2}O + O_{2} \longrightarrow HO_{2} + OH
\end{align}

The client also happens to know that reaction 1 is a modified Arrhenius reaction with $A_{1} = 10^{8}$, $b_{1} = 0.5$, $E_{1} = 5\times 10^{4}$, reaction 2 has a constant reaction rate parameter $k = 10^{4}$, and reaction 3 is an Arrhenius reaction with $A_{3} = 10^{7}$ and $E_{3} = 10^{4}$.

You should write a script that imports your `chemkin` module and returns the reaction rates of the species at each temperature of interest given the following species concentrations:

\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    H_{2}  \\
    O_{2}  \\
    OH     \\
    HO_{2} \\
    H_{2}O
  \end{bmatrix} =
  \begin{bmatrix}
    2.0 \\
    1.0 \\
    0.5 \\
    1.0 \\
    1.0
  \end{bmatrix}
\end{align}

You may assume that these are elementary reactions.

```python
%%file chemkin.py
import numpy as np
import reaction_coeffs

def progress_rate_1(nu, x, k):
    '''Returns the progress rate for a reaction of the form: nu_A A + nu_B B -> nu_C C

    INPUTS
    =======
    nu: 3-element list or array
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of
       species A, B and C
    x: 3-element list or array
       Concentration vector which specifies the concentrations of species A, B and C
    k: float
       reaction rate coefficient

    RETURNS
    ========
    omega: float, except the following cases:
       If nu or x is not a 3-element list or array, a TypeError will be raised;
       if nu contains non-positive element(s), a ValueError will be raised;
       if x contains negative element(s), a ValueError will be raised;
       if k <= 0, a ValueError will be raised.

    EXAMPLES
    >>> progress_rate_1([2, 1, 1], [1, 2, 3], 10)
    20
    >>> progress_rate_1([1, 1, 1], [4, 2, 3], 10)
    80
    '''
    try:
        if len(nu) != 3:
            raise TypeError('nu must be a 3 element list or array.')
    except:
        raise TypeError('nu must be a 3 element list or array.')
    if not all([nu_ > 0 for nu_ in nu]):
        raise ValueError('All elements in nu must be positive.')
    try:
        if len(x) != 3:
            raise TypeError('x must be a 3 element list or array.')
    except:
        raise TypeError('x must be a 3 element list or array.')
    if any([x_ < 0 for x_ in x]):
        raise ValueError('All elements in x must be non-negative.')
    if k <= 0:
        raise ValueError('k must be positive.')
    omega = k*x[0]**nu[0]*x[1]**nu[1]
    return omega

def progress_rate_2(nu_1, nu_2, x, k):
    '''Returns the progress rate for a system of reactions of the form:
       nu'_11 A + nu'_21 B -> nu''_31 C
       nu'_12 A + nu'_32 C -> nu''_22 B + nu''_32 C

    INPUTS
    =======
    nu_1: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of reactants.
       If nu_1 is not an array, a conversion is attempted.
    nu_2: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of products.
       If nu_2 is not an array, a conversion is attempted.
    x: 3-element list or array
       Concentration vector which specifies the concentrations of species A, B and C
    k: float
       reaction rate coefficient

    RETURNS
    ========
    omega: 2-element tuple
       Has the form (float, float) which corresponds to the progress rates of 2 reactions, except
       the following cases:
       If nu_1 or nu_2 cannot be converted to an array of shape (3, 2), a TypeError will be raised;
       if nu_1 or nu_2 contains negative element(s), a ValueError will be raised;
       if x is not a 3-element list or array, a TypeError will be raised;
       if x contains negative element(s), a ValueError will be raised;
       if k <= 0, a ValueError will be raised.

    EXAMPLES
    >>> progress_rate_2([[1, 2], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 1], 10)
    (40, 10)
    >>> progress_rate_2([[1, 1], [2, 0], [0, 2]], [[0, 0], [0, 1], [2, 1]], [1, 2, 3], 10)
    (40, 90)
    '''
    try:
        nu_1 = np.array(nu_1)
        if nu_1.shape != (3, 2):
            raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    try:
        nu_2 = np.array(nu_2)
        if nu_2.shape != (3, 2):
            raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    if np.any(nu_1 < 0):
        raise ValueError('All elements in nu_1 must be non-negative.')
    if np.any(nu_2 < 0):
        raise ValueError('All elements in nu_2 must be non-negative.')
    try:
        if len(x) != 3:
            raise TypeError('x must be a 3 element list or array.')
    except:
        raise TypeError('x must be a 3 element list or array.')
    if any([x_ < 0 for x_ in x]):
        raise ValueError('All elements in x must be non-negative.')
    if k <= 0:
        raise ValueError('k must be positive.')
    omega = (k*np.prod([x[i]**nu_1[i, 0] for i in range(3)]), k*np.prod([x[i]**nu_1[i, 1] for i in range(3)]))
    return omega

def reaction_rate_1(nu_1, nu_2, x, k):
    '''Returns the reaction rates for species in a system of reactions of the form:
       nu'_11 A + nu'_21 B -> nu''_31 C
       nu'_32 C -> nu''_12 A + nu''_22 B

    INPUTS
    =======
    nu_1: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of reactants.
       If nu_1 is not an array, a conversion is attempted.
    nu_2: array of shape (3, 2)
       Stoichiometric coefficient vector which specifies the stoichiometric coefficients of products.
       If nu_2 is not an array, a conversion is attempted.
    x: 3-element list or array
       Concentration vector which specifies the concentrations of species A, B and C
    k: float
       reaction rate coefficient

    RETURNS
    ========
    f: 3-element array
       Has the form (float, float, float) which corresponds to the reaction rates of species A, B, C,
       except the following cases:
       If nu_1 or nu_2 cannot be converted to an array of shape (3, 2), a TypeError will be raised;
       if nu_1 or nu_2 contains negative element(s), a ValueError will be raised;
       if x is not a 3-element list or array, a TypeError will be raised;
       if x contains negative element(s), a ValueError will be raised;
       if k <= 0, a ValueError will be raised.

    EXAMPLES
    >>> reaction_rate_1([[1, 0], [2, 0], [0, 2]], [[0, 1], [0, 2], [1, 0]], [1, 2, 1], 10)
    array([-30, -60,  20])
    >>> reaction_rate_1([[1, 0], [2, 0], [0, 1]], [[0, 1], [0, 2], [1, 0]], [1, 2, 3], 10)
    array([-10, -20,  10])
    '''
    try:
        nu_1 = np.array(nu_1)
        if nu_1.shape != (3, 2):
            raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_1 must be able to converted to a 3 X 2 array.')
    try:
        nu_2 = np.array(nu_2)
        if nu_2.shape != (3, 2):
            raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    except:
        raise TypeError('nu_2 must be able to converted to a 3 X 2 array.')
    if np.any(nu_1 < 0):
        raise ValueError('All elements in nu_1 must be non-negative.')
    if np.any(nu_2 < 0):
        raise ValueError('All elements in nu_2 must be non-negative.')
    try:
        if len(x) != 3:
            raise TypeError('x must be a 3 element list or array.')
    except:
        raise TypeError('x must be a 3 element list or array.')
    if any([x_ < 0 for x_ in x]):
        raise ValueError('All elements in x must be non-negative.')
    if k <= 0:
        raise ValueError('k must be positive.')
    omega = progress_rate_2(nu_1, nu_2, x, k)
    omega = np.array(omega).reshape((len(omega), 1))
    f = np.sum((np.dot(nu_2, omega) - np.dot(nu_1, omega)), axis=1)
    return f
```

    Writing chemkin.py

```python
import chemkin
import numpy as np

class reaction:
    def __init__(self, nu_1, nu_2, k_paras, specie_names=None):
        self.nu_1 = np.array(nu_1)
        self.nu_2 = np.array(nu_2)
        self.rate_coeffs = self.f_rate_coeffs(k_paras)
        if None == specie_names:
            self.specie_names = ['Specie{}'.format(i+1) for i in range(self.nu_1.shape[0])]
        else:
            self.specie_names = specie_names

    def f_rate_coeffs(self, k_paras):
        def rate_coeffs(T, R=8.314):
            k = []
            for k_para in k_paras:
                if 'k' in k_para:
                    k.append(k_para['k'])
                    continue
                elif 'b' in k_para:
                    k.append(chemkin.reaction_coeffs.mod_arr(k_para['A'], k_para['b'], k_para['E'], T, R))
                    continue
                else:
                    k.append(chemkin.reaction_coeffs.arr(k_para['A'], k_para['E'], T, R))
            return k
        return rate_coeffs

    def progress_rate(self, x, k, **kwargs):
        nu_1 = self.nu_1
        nu_2 = self.nu_2
        if 'nu_1' in kwargs:
            nu_1 = kwargs['nu_1']
        if 'nu_2' in kwargs:
            nu_2 = kwargs['nu_2']
        dim = nu_1.shape
        omega = [k[j]*np.prod([x[i]**nu_1[i, j] for i in range(dim[0])]) for j in range(dim[1])]
        return omega

    def reaction_rate(self, x, k, **kwargs):
        nu_1 = self.nu_1
        nu_2 = self.nu_2
        if 'nu_1' in kwargs:
            nu_1 = kwargs['nu_1']
        if 'nu_2' in kwargs:
            nu_2 = kwargs['nu_2']
        dim = nu_1.shape
        omega = np.array(self.progress_rate(x, k, nu_1=nu_1, nu_2=nu_2)).reshape(len(nu_1[1]), 1)
        f = np.sum((np.dot(nu_2, omega) - np.dot(nu_1, omega)), axis=1)
        return f

    def cal_reaction_rate(self, x, T, R=8.314):
        self.x = x
        self.T = T
        self.k = self.rate_coeffs(self.T, R)
        self.f = self.reaction_rate(self.x, self.k)
        return self.f

    def print_reaction_rate(self):
        print('At {} K ({}), the reaction rates are as follows:'\
              .format(self.T, ', '.join(['[{}] = {}'.format(specie_name, self.x[i])\
                                        for i, specie_name in enumerate(self.specie_names)])))
        print('\n'.join(['f({}) = {}'.format(specie_name, self.f[i])\
                        for i, specie_name in enumerate(self.specie_names)]))
        print()
```

```python
# Test
specie_names = ['H2', 'O2', 'OH', 'HO2', 'H2O']
nu_1 = [[2, 0, 0], [1, 0, 1], [0, 1, 0], [0, 1, 0], [0, 0, 1]]
nu_2 = [[1, 0, 0], [0, 1, 0], [2, 0, 1], [0, 0, 1], [0, 1, 0]]
x = [2, 1, 0.5, 1, 1]
k = [{'A':10**8, 'b':0.5, 'E':5*10**4}, {'k':10**4}, {'A':10**7, 'E':10**4}]
r = reaction(nu_1, nu_2, k, specie_names)
T_list = [750, 1500, 2500]
for T in T_list:
    r.cal_reaction_rate(x, T)
    r.print_reaction_rate()
```

    At 750 K ([H2] = 2, [O2] = 1, [OH] = 0.5, [HO2] = 1, [H2O] = 1), the reaction rates are as follows:
    f(H2) = -3607077.8728040676
    f(O2) = -5613545.183620796
    f(OH) = 9220623.056424864
    f(HO2) = 2006467.3108167283
    f(H2O) = -2006467.3108167283

    At 1500 K ([H2] = 2, [O2] = 1, [OH] = 0.5, [HO2] = 1, [H2O] = 1), the reaction rates are as follows:
    f(H2) = -281117620.76487046
    f(O2) = -285597559.2380457
    f(OH) = 566715180.0029161
    f(HO2) = 4479938.47317522
    f(H2O) = -4479938.47317522

    At 2500 K ([H2] = 2, [O2] = 1, [OH] = 0.5, [HO2] = 1, [H2O] = 1), the reaction rates are as follows:
    f(H2) = -1804261425.9632487
    f(O2) = -1810437356.938906
    f(OH) = 3614698782.902155
    f(HO2) = 6175930.975657232
    f(H2O) = -6175930.975657232

---
# Problem 7
Get together with your project team, form a GitHub organization (with a descriptive team name), and give the teaching staff access.  You can have has many repositories as you like within your organization.  However, we will grade the repository called **`cs207-FinalProject`**.

Within the `cs207-FinalProject` repo, you must set up Travis CI and Coveralls.  Make sure your `README.md` file includes badges indicating how many tests are passing and the coverage of your code.


# An introduction to the algorithms used in `glum`

Before continuing, please take a look at the [sklearn documentation](https://scikit-learn.org/stable/modules/linear_model.html#generalized-linear-regression) for a high-level intro to generalized linear models.

In addition, please take a look at [the tutorials.](tutorials/tutorials.rst)

For mathematical and algorithmic details, see below:

## What is a GLM?

This package is intended to fit L1 and L2-norm penalized generalized linear models (GLMs). What is a GLM?

A GLM is a linear model ($\eta = x^\top \beta$) wrapped in a transformation (link function) and equipped with a response distribution from an exponential family. The choice of link function and response distribution is very flexible. In a GLM, a predictive distribution for the response variable $Y$ is associated with a vector of observed predictors $x$.  The distribution has the form:

$$
\begin{align}
  p(y \, |\, x) &= m(y, \phi) \exp\left(\frac{\theta\, T(y) - A(\theta)}{\phi}\right)  & & \textrm{random component / distribution family} \\
  \theta &:= g(\eta) && \textrm{systematic component / link function} \\
  \eta &:= x^\top \beta && \textrm{parametric link component / linear predictor}
\end{align}
$$

Here $\beta$ are the parameters; $\phi$ is a parameter representing dispersion ("variance"); and $m$, $T$, $A$ are characterized by the user-specified model family; and $g$ is the **link function.** For background on the exponential family, and how common distributions can be represented in the form of the "random component" above, see [Wikipedia | Exponential family](https://en.wikipedia.org/wiki/Exponential_family#Table_of_distributions).

The expected mean of $Y$ depends on $x$ by composition of **linear response** $\eta$ and (inverse) link function, i.e.:

$$
E[y | \eta] := \mu = g^{-1}(\eta).
$$

Proving this involves use of [moment-generating functions and cumulants](https://en.wikipedia.org/wiki/Exponential_family#Moments_and_cumulants_of_the_sufficient_statistic).

Defining an offset of $\gamma$, we can write the linear predictor as $\eta = X^T \beta + \gamma$.

## Examples of GLMs

### Normal

Many of the "regression" models we commonly work with are GLMs. For example, the normal distribution with a linear link function is a simple GLM: $y | x \sim N(x^T \beta, \sigma^2)$. If we fit $\beta$ with maximum likelihood, this is least-squares regression. If we replace the linear link function with an exponential link function, we get a simple multiplicative model that is also a GLM: $y | x \sim N(e^{x^T \beta}, \sigma^2)$.

### Tweedie

In the insurance context, we usually work with a Tweedie distribution, which is a generalization of Poisson ($p=1$) and Gamma ($p=2$):

$$
\begin{align*}
\theta &= \left\{\begin{array}{ll}
\frac{\mu^{1-p}}{1-p} & \text{if } p \neq 1\\
\log \mu              & \text{if } p = 1
\end{array}\right.,\\
T(y) &= y, \\
A(\theta) &= \left\{\begin{array}{ll}
\frac{\mu^{2-p}}{2-p} & \text{if } p \neq 2\\
\log \mu              & \text{if } p = 2.
\end{array}\right.,
\end{align*}
$$

With $1 < p < 2$, the Tweedie distribution is a [compound Poisson-gamma](https://en.wikipedia.org/wiki/Compound_Poisson_distribution#Compound_Poisson_Gamma_distribution) distribution. $y$ is distributed as if a number $N$ is drawn from a Poisson distribution, and then $N$ draws are taken IID from a gamma distribution and added. This distribution might model the total amount of a claim in an insurance context: There are $N$ incidents, and each incident $i$ has an amount $X_i$, and the total amount is the sum of $X_i$.

## Objective function

To fit a GLM, we minimize the negative log-likelihood (or typically the unit deviance) subject to an elastic net constraint involving a mix of L1 and L2 penalty terms:

$$
\min_{\beta} \mathcal L + \alpha \rho ||\beta||_1  + \frac{\alpha (1-\rho)}{2} ||\beta||_2^2
$$

## Optimizers/solvers

There are three solvers implemented in `glum`:

* `lbfgs` - This solver uses the scipy `fmin_l_bfgs_b` optimizer to minimize L2-penalized GLMs. The L-BFGS solver does not work with L1-penalties. Because L-BFGS does not store the full Hessian, it can be particularly effective for very high dimensional problems with several thousand or more columns. For more details, see [the scipy documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_l_bfgs_b.html).
* `irls-cd` and `irls-ls` - These solvers are both based on Iteratively Reweighted Least Squares (IRLS). IRLS proceeds by iteratively approximating the objective function with a quadratic, then solving that quadratic for the optimal update. For purely L2-penalized settings, the `irls-ls` uses a least squares inner solver for each quadratic subproblem. For problems that have any L1-penalty component, the `irls-cd` uses a coordinate descent inner solver for each quadratic subproblem. The IRLS-LS and IRLS-CD implementations largely follow the algorithm described in `newglmnet` (see references below).

### Convexity
Our objective function will generally be convex, because the log-likelihoods of members of the exponential family are convex in their "natural parameterizations." The natural parameterization may not be the most obvious one. Example: The log-likelihood of the normal distribution is convex in one over the variance, which is its natural parameterization. It is not convex in the variance. We can generally assume we are solving convex problems.

An exception is multicollinearity (rank deficiency) in the design matrix, without an L2 component to the penalty. In that case, the problem will be only weakly convex and will have no unique miminum. This is not an arcane consideration, since we frequently generate rank deficiency by constructing multiple sets of one-hot encoded categorical variables. This can make evaluating different optimizers tricky, since they could converge to different equally good optima. The original `glmnet` paper suggests using at least a small L2 regularization component to remove "degeneracies and wild behavior."

### IRLS

We minimize the log likelihood using Iteratively Reweighted Least Squares (IRLS). IRLS can be justified by seeing it as taking a Newton step, but replacing the Hessian with the expected Hessian.

In the `irls-cd` and `irls-ls` solvers, the outer loop is an IRLS iteration that forms a quadratic approximation to the negative loglikelihood. That is, we find `w` and `z` so that the problem can be expressed as:

```
min sum_i w_i (z_i - x_i beta)^2 + penalty
```

We exit when either the gradient is small (`gradient_tol`) or the step size is small (`step_size_tol`). Both of these tolerances are user configurable.

Once we have formed this quadratic approximation, an "inner solver" finds the minimum of the quadratic. In the `irls-ls` solver, the inner solver is simply a direct least squares solve.

See the `glmintro` reference for an excellent discussion of IRLS in the context of GLMs.

### Coordinate Descent
With an L1 penalty, we use the `irls-cd` solver where the inner solver finds the minimum of the quadratic using coordinate descent. We exit the inner loop when the quadratic problem's gradient is small. The classic reference here is the `glmnet` paper.

However, coordinate descent is older than the glmnet paper, and is a simple idea. In a problem with data $y$ and $x$ and parameters "params", Coordinate Descent involves repeatedly optimizing one or several parameters while holding the rest fixed. Interestingly, Wikipedia [claims](https://en.wikipedia.org/wiki/Coordinate_descent#Applications) that coordinate descent is often overlooked among researchers because it is simple to implement, and they would rather work on something more interesting.

```python
"""
Cyclical coordinate descent with Newton-like steps and a
twice-differentiable objective function.
"""
def coordinate_descent_inner(y, x, params, grad_func, hess_func):
    for i, elt in enumerate(params):
        step = -grad_func(y, x, params) / hess_func(y, x, params)
        params[i] += step
    return params

def coordinate_descent(y, x, params, grad_func, hess_func,
                       convergence_func):
    while not convergence_func():
        params = coordinate_descent_inner(y, x, params, grad_func,
                                         hess_func)
    return params
```

In practice, the implementation is a bit more complicated when the objective function is not differentiable. See also the `coordinate_descent` reference below.

### Active set tracking

When penalizing with an L1-norm, it is common for many coefficients to be exactly zero. And, it is possible to predict during a given iteration which of those coefficients will stay zero. As a result, we track the "active set" consisting of all the coefficients that are either currently non-zero or likely to remain non-zero. We follow the outer loop active set tracking algorithm in the `newglmnet` reference. That paper refers to the same concept as "shrinkage", whereas the `glmnet` reference calls this the "active set". Currently, we have not yet implemented the inner loop active set tracking from the `newglmnet` reference.

## Matrix Types

Along with the GLM solvers, this package supports dense, sparse, categorical matrix types and mixtures of these types. Using the most efficient matrix representations massively improves performacne.

For more details, see the [README for tabmat](https://github.com/Quantco/tabmat)

* We support dense matrices via standard numpy arrays.
* We support sparse CSR and CSC matrices via standard `scipy.sparse` objects. However, we have extended these operations with custom matrix-vector and sandwich product routines that are optimized and parallelized. A user does not need to modify their code to take advantage of this optimization. If a `scipy.sparse.csc_matrix` object is passed in, it will be automatically converted to a `SparseMatrix` object. This operation is almost free because no data needs to be copied.
* We implement a `CategoricalMatrix` object that efficiently represents these matrices without nearly as much overhead as a normal CSC or CSR sparse matrix.
* Finally, `SplitMatrix` allows mixing different matrix types for different columns to minimize overhead.

## Standardization

Internal to our solvers, all matrix types are wrapped in a `tabmat.StandardizedMatrix` which offsets columns to have mean zero and standard deviation one without modifying the matrix data itself. This avoids situations where modifying a matrix to have mean zero would result in losing the sparsity structure. It also avoids ever needing to copy or modify the input data matrix. As a result, excess memory usage is very low in `glum`.

References
----------

`glmnet` - [Regularization Paths for Generalized Linear Models via Coordinate Descent](https://web.stanford.edu/~hastie/Papers/glmnet.pdf)

`newglmnet` - [An Improved GLMNET for L1-regularized LogisticRegression](https://www.csie.ntu.edu.tw/~cjlin/papers/l1_glmnet/long-glmnet.pdf)

`glmintro` - [Bryan Lewis on GLMs](https://bwlewis.github.io/GLM/)

`coordinate_descent` - [Coordinate Descent Algorithms](http://www.optimization-online.org/DB_FILE/2014/12/4679.pdf)

`glmbook` - [Generalized Linear Models](http://www.utstat.toronto.edu/~brunner/oldclass/2201s11/readings/glmbook.pdf), McCullagh and Nelder

```python

```


Welcome to the third post in my series on modeling the Coronavirus pandemic.  In the [first](http://www.davidketcheson.info/2020/03/17/SIR_model.html) and [second](http://www.davidketcheson.info/2020/03/19/SIR_Estimating_parameters.html) posts, we introduced the SIR model and used the available data to estimate the model parameters, in the absence of any mitigation (i.e. with no social distancing, quarantine, etc.).  In this post we'll put model and parameters together to see what they predict for the current pandemic.  We'll do this first assuming no mitigation, and then we'll make a very rough attempt to understand how mitigation may impact the predictions.

Remember, I am not an epidemiologist and I make no guarantees as to the accuracy of these predictions.  I don't recommend that you make precise plans based on the details of these predictions.  My goal is merely to show that with a bit of mathematics and common sense, we can have a reasonable idea of what the future holds.

# Predictions in the absence of mitigation

First, let's look at what we expect to happen without any mitigation.  In other words, this is a scenario in which schools and workplaces remain open, people continue to shake hands or kiss cheeks when greeting one another, and so forth.  **Note that the predictions here are NOT what we expect will actually happen, because we have intentionally ignored the effects of containment measures.**

Here are the basic assumptions leading to the predictions below:

  - The dynamics of the COVID-19 epidemic follow the SIR model, with parameters $\beta \approx 0.25$ and $\gamma \approx 0.05$.
  - No containment measures (like quarantine, closures, and social distancing) are implemented.
  - The number of confirmed cases at present is about 15 percent of the actual cases (this is a very rough guess based on some expert opinions).

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
from ipywidgets import interact, widgets
import matplotlib.dates as dates
from scipy.integrate import solve_ivp
plt.style.use('seaborn-poster')
matplotlib.rcParams['figure.figsize'] = (10., 6.)
```

```python
def SIR(beta=0.25, gamma=0.05, N = 7770000000, confirmed_fraction=0.15):
    """ Model the current outbreak using the SIR model."""

    du = np.zeros(3)
    u0 = np.zeros(3)

    def f(t,u):
        du[0] = -beta*u[1]*u[0]/N
        du[1] = beta*u[1]*u[0]/N - gamma*u[1]
        du[2] = gamma*u[1]
        return du

    # Initial values
    total_cases = 199258./confirmed_fraction
    u0[2] = 81972./confirmed_fraction+7956  # Initial recovered
    u0[1] = total_cases-u0[2] # Initial infected
    u0[0] = N - u0[1] - u0[2]

    #dt = 0.01
    T = 400
    times = np.arange(0,T)
    solution = solve_ivp(f,[0,T],u0,t_eval=times)
    S = solution.y[0,:]
    I = solution.y[1,:]
    R = solution.y[2,:]

    today = '19/03/2020'
    start = dates.datestr2num(today)
    mydates = np.arange(400)+start

    fig = plt.figure(figsize=(12,8))
    plt.plot_date(mydates,S/1.e9,'-',lw=3)
    plt.plot_date(mydates,I/1.e9,'-',lw=3)
    plt.plot_date(mydates,R/1.e9,'-',lw=3)
    ax = plt.gca()
    ax.xaxis.set_major_locator(dates.MonthLocator())
    ax.xaxis.set_major_formatter(dates.DateFormatter('%b'))
    fig.autofmt_xdate()
    plt.legend(['Susceptible','Infected','Recovered'])
    plt.ylabel('Individuals (in billions)')
    plt.xlim(737503,737903)
    plt.text(737803, 6,r'$\beta$ = {}, $\gamma$ = {}'.format(beta,gamma),fontsize=15,bbox=dict(facecolor='white', alpha=1.0))
```

```python
SIR(beta=0.25, gamma=0.05, confirmed_fraction=0.15);
```

There are several important things to notice here.

## When does the maximum number of infected individuals occur, and how high is it?

As we expect, there is an initial exponential growth of infection that eventually levels off and then decreases after most of the population has been infected.  Recall from the first post that we expect the infection peak to occur when the fraction of susceptible individuals is $\gamma/\beta = 0.05/0.25 = 0.2$; i.e., when four-fifths of the world population has already been infected (including those who have recovered).  With the current parameters, this peak occurs around the middle of May and results in almost 4 billion concurrent cases.  Of course, the vast majority of those cases would not need medical care; based on our estimate of 15% reporting and less than 20% of reported cases being severe, perhaps only about 3% of all cases would require medical attention.  Even so, that would mean a peak of 100 million cases requiring medical attention simultaneously, worldwide.

## How many people catch the virus?

In this scenario, almost everyone in the world is eventually infected; by day 400 only 54 million susceptible individuals remain.

## How long does the epidemic last?

After mid-May, the epidemic begins to trail off gradually, but it is not until about early November that the number of infected drops back to below 1 million.  Thus (in this scenario) we should expect the severe epidemic to last at least several months.  Of course, it would make sense for those who have already recovered from the virus -- which is most of the world population -- to go back to work/school in the early Fall,

# Variations on the average scenario

In the last post, we saw that there is significant uncertainty in the value of $\gamma$ and especially $\beta$.  How do these predictions change if we vary $\beta$?  We found values in the range $(0.2, 0.3)$, so let's look at what happens for each of the extremes of this interval:

```python
SIR(beta=0.2, gamma=0.05, confirmed_fraction=0.15)
```

With a smaller value of $\beta=0.2$, the virus spreads more slowly; the peak occurs near June 1st with just over 3 billion infected.  Again, almost the entire world catches the virus (eventually 150 million susceptibles remain), and the number of cases does not drop below 1 million until November.

```python
SIR(beta=0.3, gamma=0.05, confirmed_fraction=0.15)
```

With a larger values of $\beta=0.3$, the virus spreads more quickly; the peak occurs in early May with just over 4 billion infected.  The epidemic ends a bit sooner, with the number of cases dropping below 1 million some time in October.  Only 19 million people remain susceptible after a year.

As we can see, even with these fairly large changes in $\beta$, the general picture remains the same.  We can expect the epidemic to peak in May or June and last into the fall.

```python
interact(SIR,beta=widgets.FloatSlider(min=0.01,max=0.5,step=0.01,value=0.25,description=r'$\beta$'),
         gamma=widgets.FloatSlider(min=0.01,max=0.5,step=0.01,value=0.05,description=r'$\gamma$'),
         N = widgets.fixed(7770000000),
         confirmed_fraction=widgets.FloatSlider(min=0.01,max=1.0,step=0.01,value=0.15,description=r'Fraction C'));
```

# The effect of mitigation

As I write this, almost every country in the world is adopting measures to mitigate the spread of the virus.  This includes closing schools and workplaces, quarantining infected individuals and their contacts, and encouraging people to stay home.

For simplicity we can view all of these mitigation measures as having a single effect: increasing the average time between encounters, or in other words reducing $\beta$.    Remember that $\beta$ is the average number of people per day that a given individual has close contact with.  Since $\beta$ is a constant in our model but the mitigation techniques and their effectiveness may vary over time, we can incorporate mitigation by adding a new factor $q(t)\in[0,1]$ multiplying $\beta$:

\begin{align}
\frac{dS}{dt} & = -q(t)\beta I \frac{S}{N} \\
\frac{dI}{dt} & = q(t)\beta I \frac{S}{N}-\gamma I \\
\frac{dR}{dt} & = \gamma I
\end{align}

What is the meaning of $q(t)$?  If there were an absolute quarantine, with no human contact at all, we would have $q=0$, whereas if no measures are implemented then we would have $q=1$ (corresponding to the predictions above).  In the real world, $q$ will be somewhere between these extremes.

What is the correct value of $q$?  Frankly, I have no idea and I doubt that even the experts can say with confidence.  But we can hypothesize some values and explore their impact.  For simplicity, we'll assume that some value $q<1$ is achieved through mitigation measures starting now and lasting for the next $N_q$ days.  After $N_q$ days, these measures are lifted so that society (and $\beta$) returns to normal.

If we could maintain mitigation measures forever (i.e. $N_q=\infty$) then this mitigation would have exactly the same effect as reducing $\beta$.  As we saw above, smaller values of $\beta$ lead to a smaller infection peak, but a longer epidemic.

Of course, we do not expect mitigation to last forever; people must go back to work eventually.  This can have some surprising effects.

```python
def SIR_mitigated(beta=0.25, gamma=0.05, N = 7770000000, confirmed_fraction=0.15, q=0.5, Nq=180):
    """ Model the current outbreak using the SIR model."""

    du = np.zeros(3)
    u0 = np.zeros(3)

    def f(t,u):
        if t<Nq:
            qval = q
        else:
            qval = 1.
        du[0] = -qval*beta*u[1]*u[0]/N
        du[1] = qval*beta*u[1]*u[0]/N - gamma*u[1]
        du[2] = gamma*u[1]
        return du

    # Initial values
    total_cases = 199258./confirmed_fraction
    u0[2] = 81972./confirmed_fraction+7956  # Initial recovered
    u0[1] = total_cases-u0[2] # Initial infected
    u0[0] = N - u0[1] - u0[2]

    #dt = 0.01
    T = 400
    times = np.arange(0,T)
    solution = solve_ivp(f,[0,T],u0,t_eval=times,method='RK23')
    S = solution.y[0,:]
    I = solution.y[1,:]
    R = solution.y[2,:]

    today = '19/03/2020'
    start = dates.datestr2num(today)
    mydates = np.arange(T)+start

    fig = plt.figure(figsize=(12,8))
    plt.plot_date(mydates,S/1.e9,'-',lw=3)
    plt.plot_date(mydates,I/1.e9,'-',lw=3)
    plt.plot_date(mydates,R/1.e9,'-',lw=3)
    ax = plt.gca()
    ax.xaxis.set_major_locator(dates.MonthLocator())
    ax.xaxis.set_major_formatter(dates.DateFormatter('%b'))
    fig.autofmt_xdate()
    plt.legend(['Susceptible','Infected','Recovered'])
    plt.ylabel('Individuals (in billions)')
    plt.xlim(737503,737503+T)
    plt.text(737803, 6,r'$\beta$ = {}, $\gamma$ = {}'.format(beta,gamma),fontsize=15,bbox=dict(facecolor='white', alpha=1.0))
    plt.text(737803, 5.4,r'$q$ = {}, $N_q$ = {}'.format(q,Nq),fontsize=15,bbox=dict(facecolor='white', alpha=1.0))
```

As a first scenario, let's imagine that $q=1/2$; i.e., we are able to reduce the amount of human contact by 50%, and $N_q=180$ (about six months).

```python
SIR_mitigated(beta=0.25, gamma=0.05, confirmed_fraction=0.15, q=0.5, Nq=180)
```

We see that this mitigation makes a substantial difference.  Now the peak point of infection is delayed until late June, with less than 2 billion simultaneous cases at maximum.  Notice the bump in the number of cases around mid-September when the restrictions are relaxed.  Another effect of this mitigation is that the time in which there are some significant number of cases lasts much longer -- in this scenario there are still more than 1 million cases even 1 year from now.  It's still true that most of the world catches the virus eventually, but about 480 million susceptibles remain even after two years.

Next let's consider an even more successful mitigation: suppose that we cut human contact by three-fifths, so $q=0.4$, with measures again lasting $N_q$=180 days.

```python
SIR_mitigated(beta=0.25, gamma=0.05, confirmed_fraction=0.15, q=0.4, Nq=180)
```

Here the initial growth is even slower, and we see an interesting phenomenon: the infection appears to reach a peak around mid-September, right when restrictions are relaxed.  But the relaxing of restrictions allows the virus to suddenly spread much faster, leading to a higher peak in early October.  What's quite surprising and counterintuitive is that **even though we have stronger mitigation, the peak is actually higher in this case than in the previous case**.  Why?  It's because the very strong mitigation for 180 days means that at the end of that period there is still a large proportion of susceptible people, leading to stronger growth of the epidemic when mitigation ends.

Finally, if we have extremely strong mitigation and then suddenly remove all restrictions, we simply get a delayed version of the full-scale outbreak:

```python
SIR_mitigated(beta=0.25, gamma=0.05, confirmed_fraction=0.15, q=0.25, Nq=180)
```

This is because our mitigation was so effective that hardly anyone caught the virus and everyone was still susceptible after our 180-day mitigation ended.  Of course, the ideal scenario (in terms of minimizing infection) would be to maintain strong mitigation until a vaccine can be deployed.

# When should we let society go back to normal?

A natural question at this point is, when is it okay to let everyone go back to school and to work?  There are many possible answers, but perhaps a reasonable one can be obtained as follows.  Let's assume that our goal is to ensure that the number of infected $I(t)$ will not increase when we send everyone back to work; i.e., when we let $q$ go back up to 1.  We have

\begin{align}
\frac{dI}{dt} & = (q\beta \frac{S}{N}-\gamma) I.
\end{align}

To ensure that $dI/dt<0$ even with $q=1$, we need the susceptible fraction $S/N$ to have fallen to $\gamma/\beta$ -- the same condition we found previously for when the infection peak would occur (without mitigation).  Using the parameter values we found in the last post, this means we should wait until 80% of the population has been infected before returning to normal.  This is probably too cautious, since ending mitigation a bit sooner (but after the initial peak of infection) would only lead to a small subsequent rise, with a second infection peak smaller than the first:

```python
SIR_mitigated(beta=0.25,gamma=0.05,q=0.4,Nq=220)
```

Of course, other strategies are possible and could make sense, like sending people back to work as soon as they have tested positive and then recovered from the virus.  And more complicated mitigation strategies are possible (and likely) in which some restrictions are relaxed while others remain in place.  All of these will have to be weighed against the cost they impose on our lives.

In the [Jupyter notebook for this post]() there is an interactive widget that lets you experiment with the model, including all five of the parameters we have discussed.  The possible effects are quite interesting and we have only touched on the basics here.  I encourage you to try it out!

```python
interact(SIR_mitigated,beta=widgets.FloatSlider(min=0.01,max=0.5,step=0.01,value=0.25,description=r'$\beta$'),
         gamma=widgets.FloatSlider(min=0.01,max=0.5,step=0.01,value=0.05,description=r'$\gamma$'),
         N = widgets.fixed(7770000000),
         confirmed_fraction=widgets.FloatSlider(min=0.01,max=1.0,step=0.01,value=0.15,description=r'Fraction C'),
         q = widgets.FloatSlider(min=0.01, max=1.0, step=0.01, value=0.5),
         Nq = widgets.IntSlider(min=10, max=400, step=5, value=180));
```

# Further considerations

There are a number of potentially important factors that we have ignored here, including:

  - Because countries will have differing strategies, there may be significantly **earlier infection peaks in some countries and later peaks in others**.  This is likely to be the case if international travel restrictions remain in place for an extended period.
  - Possible **seasonal effects** on the spread of Coronavirus.  Currently there seems to be no way to know if COVID-19 will be seasonal like the flu.
  - Development of a **vaccine**.  As we have seen, significantly reducing the number of susceptible individuals can rapidly halt the spread of a disease, even if not every individual is vaccinated.  So if a vaccine appears and can be mass produced before the fall, it could susbstantially shorten the duration of the epidemic and reduce its impact.  On the other hand, it seems that a vaccine arriving after a year would be too late to have much impact.


```python
%pylab inline
from IPython.display import display, Math, Latex
from scipy import *
```

    Populating the interactive namespace from numpy and matplotlib

```python
arc = load('mnist.npz')

x_train = arc['arr_0']
y_train = arc['arr_1']
x_test  = arc['arr_2']
y_test  = arc['arr_3']

print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)
```

    (60000, 28, 28) (60000,)
    (10000, 28, 28) (10000,)

```python
##function given by prof.

def classify(z):
    all_distances = array([dist(x, z) for x in x_train])
    digit = y_train[argmin(all_distances)]
    return digit
```

```python
# Show image number 15, and write in the title what digit it should correspond to
N=15
imshow(x_train[N], cmap='gray_r')
_ = title('Hand written digit '+str(y_train[N]))
```

##                                    GENERAL GUIDELINES

1. Time all functions you construct, and try to make them run as fast as possible by precomputing anything that can be precomputed
2. Extra points are gained if you reduce the complexity of the given algorithms in any possible way, for example by exploiting linearity, etc.
3. If something takes too long to execute, make sure you time it on a smaller set of input data, and give estimates of how long it would take to run the full thing (without actually running it). Plot only the results you manage to run on your PC.

##                                                           ASSIGNMENT 1

Implement the following distance functions.
1. Dinfty
2. D_one
3. D_two

```python

def d_infty(a,b):
    return norm((b-a).reshape(-1),inf)

def d_one(a,b):
    return norm((b-a).reshape(-1),1)

def d_two(a,b):
    return norm((b-a).reshape(-1),2)

```

#### COMMENT OF ASSIGNMENT 1

funtion that I used:
1. norm(x, ord): This function is able to return one of eight different matrix norms, or one of an infinite number of vector norms , depending on the value of the ord parameter (in our case ord= inf,1,2)
2. reshape(): Gives a new shape to an array without changing its data.

## Assignment 2

Write a function that, given a number N, and a distance function dist, computes the distance matrix D of shape (N,N) between the first N entries of x_train:
\begin{equation}
 D[i,j] = dist(x_train[i], x_train[j])
\end{equation}

erforming the minimum number of operations (i.e., avoid computing a distance if it has already been computed before, i.e., keep in mind that dist(a,b) = dist(b,a)).

```python
def distance(N,dist):
    D = zeros((N,N))

    for i in range(N):
        for j in range(i+1,N):
            D[i][j] = D[j][i] = dist(x_train[i],x_train[j])

    return D

A= distance(4,d_infty)
print(A)

```

    [[0.         1.         1.         0.99215686]
     [1.         0.         1.         1.        ]
     [1.         1.         0.         1.        ]
     [0.99215686 1.         1.         0.        ]]

## Asiignment 3

Compute and plot the three distance matrices
1. Dinfty
2. D1
3. D2

for the first 100 images of the training set, using the function imshow applied to the three matrices

```python
# define the 3 matrices

Dinfty = distance(100,d_infty)
D1 = distance(100,d_one)
D2 = distance(100,d_two)

```

```python
matshow(Dinfty)
_ = title('plot of the d_infty distance matrix')
```

```python
matshow(D1,cmap='Blues_r')
_ = title('plot of the D-one matrix')
```

```python
matshow(D2,cmap='Blues_r')
_ = title('plot of the D-two matrix')
```

## ASSIGNMENT 4

Using only a distance matrix, apply the algorithm described above and compute the efficency of the algorithm, i.e., write a function that:

Given a distance matrix with shape (N,N), constructed on the first N samples of the x_train set, count the number of failures of the leave one out strategy, i.e.,

1. set error_counter to zero

2. for every line i of the matrix:

    1. find the index j (different from i) for which D[i,k] >= D[i,j] for all k different from i and j.

    2. if y_train[j] is different from y_train[i], increment by one error_counter.

3. return the error: error_counter/N.

4. apply the function above to the 3 different distance matrices you computed before

```python
# Funzione che riceve in input una matrice distanza e ritorna un numero che rappresenta l'errore
#Nell'ass.4 chiede di prendere una riga della matrice delle distanze, trovare l'indice per cui c'é
#il valore minimo della disranza e vedere se é le y_train sono diverse
#(ovvero se i numeri corrispondenti sono diversi)

#tanto le matrici sono uguali nelle dimensioni comuni
def performance(N,dist_matrix):
    D = dist_matrix[0:N,0:N] # considero solo le righe e le colonne che mi interessano
    error_counter = 0
    for i in range(N):
        if i!=N-1:
            D[i,i] = D[i,i+1] + 100  # faccio in modo che non prenda elemento diagonale
        else:
            D[i,i] = D[i,i-1] + 100
        minimo = argmin(D[i,:]) #trova l'indice dove c'è l'elemento minimo
        if (y_train[i]!=y_train[minimo]):
            error_counter = error_counter + 1
    return error_counter/N

##questo algoritmo è molto dispendioso, perché ogni volta crea una matrice delle distanze
##l'algoritmo di prima invece lo riceve in input e prende la sottomatrice che gli serve
def performance2(N,dist):
    D = distance(N,dist)
    error_counter = 0
    for i in range(N):
        if i!=N-1:
            D[i,i] = D[i,i+1] + 100  # faccio in modo che non prenda elemento diagonale
        else:
            D[i,i] = D[i,i-1] + 100
        minimo = argmin(D[i,:]) #trova l'indice dove c'è l'elemento minimo
        if (y_train[i]!=y_train[minimo]):
            error_counter = error_counter + 1
    return error_counter/N

#qualcosa non va..introduco un errore? secondo me ci starebbe
```

## ASSIGNMENT 5

Run the algorithm implemented above for N=100,200,400,800,1600 on the three different distances, and plot the three error rate as a function of N (i.e., compute the distance matrix, and compute the efficiency associated to the distance matrix).

```python
from timeit import default_timer as timer
Size = [100,200,400,800,1600]
errors = [] #array con gli
start = timer()
D_I = distance(1600,d_infty)
D_1 = distance(1600,d_one)
D_2 = distance(1600,d_two)

errors = zeros((5,3))

for i in range(5):
    errors[i][0] = performance(Size[i],D_I)
    errors[i][1] = performance(Size[i],D_1)
    errors[i][2] = performance(Size[i],D_2)
print(errors)
end = timer()
t = end -start
print("tempo trascorso per l'intero procedimento: ",t)

```

    [[0.58     0.17     0.17    ]
     [0.52     0.145    0.135   ]
     [0.4425   0.15     0.135   ]
     [0.4      0.145    0.12875 ]
     [0.369375 0.1025   0.09375 ]]
    tempo trascorso per l'intero procedimento:  48.811145658999976

```python
##verifico che l'algoritmo performance2 è molto più lento del primo

err = zeros((5,3))
inizio = timer()
for i in range(5):
    err[i][0] = performance2(Size[i],d_infty)
    err[i][1] = performance2(Size[i],d_one)
    err[i][2] = performance2(Size[i],d_two)

print(err)
fine = timer()
tt = fine - inizio
print("tempo trascorso per l'intero procedimento: ",tt)

```

    [[0.58     0.17     0.17    ]
     [0.52     0.145    0.135   ]
     [0.4425   0.15     0.135   ]
     [0.4      0.145    0.12875 ]
     [0.369375 0.1025   0.09375 ]]
    tempo trascorso per l'intero procedimento:  64.28401658300004

```python
for i in range(3):  # tre volte--> una per ogni tipo di distanza
    plot(Size,errors.T[i],)
title("efficiency plot for our three different distance")
legend(["d_infty","d_one","d_two"])
grid()
```

```python

```

## ASSIGNMENT 6

In principle, it should be possible to decrease the error by using a better norm. From the table above, it is clear that the L2 distance works better than the L1 distance, which works better than the Linfty distance.

However, none of these distances exploit the fact that the image is a two-dimensional object, and that there is information also in the neighboring information of the pixels.

One way to exploit this, is to interpret the image as a continuous function with values between zero and one, defined on a square domain \Omega=[0,27]x[0,27].
\begin{equation*}
f:\Omega \longrightarrow \mathbb{R}
\end{equation*}

1. Implement a function that computes an approximation of the  H1 norm distance on the renormalized images. Given two images  f1 e f2.

    1. Compute
    \begin{equation*}
    a = \frac{f_{1}}{\int_{\Omega} f_{1} }
    \end{equation*}

    \begin{equation*}
    b = \frac{f_{2}}{\int_{\Omega} f_{2} }
    \end{equation*}
    2. Define the  H1 distance as

    \begin{equation*}
     d_{H1}(f_{1},f_{2}) = \sqrt{\int_{\Omega}| \nabla(a-b)|^{2} + (a-b)^{2}}
    \end{equation*}

    3. Compute the distance matrix and the efficiency for this distance for N=100,200,400,800,1600

```python
#function which returns a and b---> normalized function!
from scipy import integrate
def normalization(f1,f2):
    a = f1/sum(f1)    #fa la somma dei valori pixel per pixel---matrice 28X28
    b = f2/sum(f2)
    return a,b

def h1_distance(f1,f2):
    a,b = normalization(f1,f2)
    z = a-b
    func = absolute(gradient(z))**2 + z**2
    res = sqrt(sum(func))
    return res

Size = [100,200,400,800,1600]
#creo una dict di matrici che definiscono la distanza!
print("inizio calcolo del dict")
start = timer()
matrici = {}
for i in Size:
    matrici[i] = distance(i,h1_distance)

end = timer()
print("fine calcolo del dict")
print(end-start)

#creo array di efficiency
print("inizio con efficienza")
inizio = timer()
H = distance(1600,h1_distance)
efficienza = []
for i in Size:
    efficienza.append(performance(i,H))
fine = timer()
print("fine con efficienza")
print(efficienza)
print("tempo necessario")
print(fine - inizio)

#compute distance matrix for N = 10

```

    inizio calcolo del dict
    fine calcolo del dict
    228.16818850899995
    inizio con efficienza
    fine con efficienza
    [0.22, 0.15, 0.1375, 0.11625, 0.09375]
    tempo necessario
    170.18338032299994

## Assignment 7

An even better improvement on the previous distance function is given by the following algorithm

- Given two images $f1$ and $f2$:
    - Compute $$a = \frac{f_1}{\int_\Omega f_1}$$, $$b=\frac{f_2}{\int_\Omega f_2}$$
    - Solve
    $$
    -\Delta \phi = a - b \qquad \text{ in } \Omega
    $$
    $$
    \phi = 0 \text{ on } \partial\Omega
    $$
    - Define the *Monge Ampere* distance
    $$
    d_{MA}(f_1,f_2) = \int_\Omega (a+b)|\nabla \phi|^2
    $$

- Compute the distance matrix and the efficiency for this distance for N=100,200,400,800,1600

```python

from scipy.sparse import *
from scipy.sparse.linalg import *
from scipy.sparse import csc_matrix

```

```python
from timeit import default_timer as timer
def laplace(N):
    A = diags([-1,-1,4,-1,-1], [-N,-1, 0, 1,N], shape=(N*N,N*N)).toarray()
    for i in range(1,N):
        A[i*N,(i*N)-1] = 0
        A[(i*N)-1,i*N] = 0
    return A

# voglio risolvere delta(phi) = b-a

def LU(N):
    L = laplace(N)
    LAP = csc_matrix(L) # compact representation of sparse matrices--> witout this there would be a warning
    return splu(LAP)

A = LU(len(x_train[0]) - 2 )

def resolve(immagine):
    immagine = immagine[1:-1,1:-1]
    lun = len(immagine)
    u = immagine.reshape(-1)
    v = A.solve(u)
    v = v.reshape(lun,lun)
    sol = zeros((28,28))
    sol[1:-1,1:-1]=v

    return sol

```

```python
## distanza di Monge ampere
from timeit import default_timer as timer

Size = [100,200,400,800,1600]

def grad_norm(f):
    grad = gradient(f)
    return  grad[0]**2 + grad[1]**2

def D_MA(f1,f2):
    a = f1/sum(f1)
    b = f2/sum(f2)
    phi = resolve(a-b)
    func = (a+b)*grad_norm(phi)
    return sum(func)
print("inizio operazione")
start = timer()
ampere = distance(1600,D_MA)
end = timer()

print(end - start)
eff = []
for i in Size:
    eff.append(performance(i,ampere))
end = timer()
print("fine operazione")
print(end - start)

```

    inizio operazione
    267.89958388900004
    fine operazione
    267.91211425899996

```python
print(eff)
plot(Size,eff)
title("efficiency plot for monge-ampere distance")
grid()

```

```python

```