# `bsym` – a basic symmetry module

`bsym` is a basic Python symmetry module. It consists of some core classes that describe configuration vector spaces, their symmetry operations, and specific configurations of objects withing these spaces. The module also contains an interface for working with [`pymatgen`](http://pymatgen.org) `Structure` objects, to allow simple generation of disordered symmetry-inequivalent structures from a symmetric parent crystal structure.

API documentation is [here](http://bsym.readthedocs.io).

## Configuration Spaces, Symmetry Operations, and Groups

The central object described by `bsym` is the **configuration space**. This defines a vector space that can be occupied by other objects. For example; the three points $a, b, c$ defined by an equilateral triangle,

which can be described by a length 3 vector:

\begin{pmatrix}a\\b\\c\end{pmatrix}

If these points can be coloured black or white, then we can define a **configuration** for each different colouring (0 for white, 1 for black), e.g.

with the corresponding vector

\begin{pmatrix}1\\1\\0\end{pmatrix}

A specific **configuration** therefore defines how objects are distributed within a particular **configuration space**.

The symmetry relationships between the different vectors in a **configuration space** are described by **symmetry operations**. A **symmetry operation** describes a transformation of a **configuration space** that leaves it indistinguishable. Each **symmetry operation** can be describes as a matrix that maps the vectors in a **configuration space** onto each other, e.g. in the case of the equiateral triangle the simplest **symmetry operation** is the identity, $E$, which leaves every corner unchanged, and can be represented by the matrix

\begin{equation}
E=\begin{pmatrix}1 & 0 & 0\\0 & 1 & 0 \\ 0 & 0 & 1\end{pmatrix}
\end{equation}

For this triangular example, there are other **symmetry operations**, including reflections, $\sigma$ and rotations, $C_n$:

In this example reflection operation, $b$ is mapped to $c$; $b\to c$, and $c$ is mapped to $b$; $b\to c$.

The matrix representation of this **symmetry operation** is

\begin{equation}
\sigma_\mathrm{a}=\begin{pmatrix}1 & 0 & 0\\0 & 0 & 1 \\ 0 & 1 & 0\end{pmatrix}
\end{equation}

For the example rotation operation, $a\to b$, $b\to c$, and $c\to a$, with matrix representation

\begin{equation}
C_3=\begin{pmatrix}0 & 0 & 1\\ 1 & 0 & 0 \\ 0 & 1 & 0\end{pmatrix}
\end{equation}

Using this matrix and vector notation, the effect of a symmetry operation on a specific **configuration** can be calculated as the [matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication#Square_matrix_and_column_vector) of the **symmetry operation** matrix and the **configuration** vector:

In matrix notation this is represented as

\begin{equation}
\begin{pmatrix}0\\1\\1\end{pmatrix} = \begin{pmatrix}0 & 0 & 1\\ 1 & 0 & 0 \\ 0 & 1 &
0\end{pmatrix}\begin{pmatrix}1\\1\\0\end{pmatrix}
\end{equation}

or more compactly

\begin{equation}
c_\mathrm{f} = C_3 c_\mathrm{i}.
\end{equation}

The set of all symmetry operations for a particular **configuration space** is a **group**.

For an equilateral triangle this group is the $C_{3v}$ [point group](https://en.wikipedia.org/wiki/Point_group), which contains six symmetry operations: the identity, three reflections (each with a mirror plane bisecting the triangle and passing through $a$, $b$, or $c$ respectively) and two rotations (120° clockwise and counterclockwise).

\begin{equation}
C_{3v} = \left\{ E, \sigma_\mathrm{a}, \sigma_\mathrm{b}, \sigma_\mathrm{c}, C_3, C_3^\prime \right\}
\end{equation}

## Modelling this using `bsym`

### The `SymmetryOperation` class

In `bsym`, a **symmetry operation** is represented by an instance of the `SymmetryOperation` class. A `SymmetryOperation` instance can be initialised from the matrix representation of the corresponding **symmetry operation**.

For example, in the trigonal **configuration space** above, a `SymmetryOperation` describing the identify, $E$, can be created with

```python
from bsym import SymmetryOperation
```

```python
SymmetryOperation([[ 1, 0, 0 ],
                   [ 0, 1, 0 ],
                   [ 0, 0, 1 ]])
```

    SymmetryOperation
    label(---)
    array([[1, 0, 0],
           [0, 1, 0],
           [0, 0, 1]])

Each `SymmetryOperation` has an optional `label` attribute. This can be set at records the matrix representation of the **symmetry operation** and an optional label. We can provide the label when creating a `SymmetryOperation`:

```python
SymmetryOperation([[ 1, 0, 0 ],
                   [ 0, 1, 0 ],
                   [ 0, 0, 1 ]], label='E' )
```

    SymmetryOperation
    label(E)
    array([[1, 0, 0],
           [0, 1, 0],
           [0, 0, 1]])

or set it afterwards:

```python
e = SymmetryOperation([[ 1, 0, 0 ],
                       [ 0, 1, 0 ],
                       [ 0, 0, 1 ]])
e.label = 'E'
e
```

    SymmetryOperation
    label(E)
    array([[1, 0, 0],
           [0, 1, 0],
           [0, 0, 1]])

Or for $C_3$:

```python
c_3 = SymmetryOperation( [ [ 0, 0, 1 ],
                           [ 1, 0, 0 ],
                           [ 0, 1, 0 ] ], label='C3' )
c_3
```

    SymmetryOperation
    label(C3)
    array([[0, 0, 1],
           [1, 0, 0],
           [0, 1, 0]])

#### Vector representations of symmetry operations

The matrix representation of a **symmetry operation** is a [permutation matrix](https://en.wikipedia.org/wiki/Permutation_matrix). Each row maps one position in the corresponding **configuration space** to one other position. An alternative, condensed, representation for each **symmetry operation** matrix uses vector notation, where each element gives the row containing `1` in the equivalent matrix column. e.g. for $C_3$ the vector mapping is given by $\left[2,3,1\right]$, corresponding to the mapping $1\to2$, $2\to3$, $3\to1$.

```python
c_3_from_vector = SymmetryOperation.from_vector( [ 2, 3, 1 ], label='C3' )
c_3_from_vector
```

    SymmetryOperation
    label(C3)
    array([[0., 0., 1.],
           [1., 0., 0.],
           [0., 1., 0.]])

The vector representation of a `SymmetryOperation` can be accessed using the `as_vector()` method.

```python
c_3.as_vector()
```

    [2, 3, 1]

#### Inverting symmetry operations

For every **symmetry operation**, $A$, there is an **inverse** operation, $A^{-1}$, such that

\begin{equation}
A \cdot A^{-1}=E.
\end{equation}

For example, the inverse of $C_3$ (clockwise rotation by 120°) is $C_3^\prime$ (anticlockwise rotation by 120°):

```python
c_3     = SymmetryOperation.from_vector( [ 2, 3, 1 ], label='C3' )
c_3_inv = SymmetryOperation.from_vector( [ 3, 1, 2 ], label='C3_inv' )

print( c_3, '\n' )
print( c_3_inv, '\n' )
```

    SymmetryOperation
    label(C3)
    array([[0., 0., 1.],
           [1., 0., 0.],
           [0., 1., 0.]])

    SymmetryOperation
    label(C3_inv)
    array([[0., 1., 0.],
           [0., 0., 1.],
           [1., 0., 0.]])

The product of $C_3$ and $C_3^\prime$ is the identity, $E$.

```python
c_3 * c_3_inv
```

    SymmetryOperation
    label(---)
    array([[1., 0., 0.],
           [0., 1., 0.],
           [0., 0., 1.]])

`c_3_inv` can also be generated using the `.invert()` method

```python
c_3.invert()
```

    SymmetryOperation
    label(---)
    array([[0, 1, 0],
           [0, 0, 1],
           [1, 0, 0]])

The resulting `SymmetryOperation` does not have a label defined. This can be set directly, or by chaining the `.set_label()` method, e.g.

```python
c_3.invert( label= 'C3_inv')
```

    SymmetryOperation
    label(C3_inv)
    array([[0, 1, 0],
           [0, 0, 1],
           [1, 0, 0]])

```python
c_3.invert().set_label( 'C3_inv' )
```

    SymmetryOperation
    label(C3_inv)
    array([[0, 1, 0],
           [0, 0, 1],
           [1, 0, 0]])

### The `SymmetryGroup` class

A `SymmetryGroup` is a collections of `SymmetryOperation` objects. A `SymmetryGroup` is not required to contain _all_ the symmetry operations of a particular **configuration space**, and therefore is not necessarily a complete mathematical group.

For convenience `bsym` has `PointGroup` and `SpaceGroup` classes, that are equivalent to the `SymmetryGroup` parent class.

```python
from bsym import PointGroup
```

```python
# construct SymmetryOperations for C_3v group
e       = SymmetryOperation.from_vector( [ 1, 2, 3 ], label='e' )
c_3     = SymmetryOperation.from_vector( [ 2, 3, 1 ], label='C_3' )
c_3_inv = SymmetryOperation.from_vector( [ 3, 1, 2 ], label='C_3_inv' )
sigma_a = SymmetryOperation.from_vector( [ 1, 3, 2 ], label='S_a' )
sigma_b = SymmetryOperation.from_vector( [ 3, 2, 1 ], label='S_b' )
sigma_c = SymmetryOperation.from_vector( [ 2, 1, 3 ], label='S_c' )
```

```python
c3v = PointGroup( [ e, c_3, c_3_inv, sigma_a, sigma_b, sigma_c ] )
```

```python
c3v
```

    PointGroup
    e	[1, 2, 3]
    C_3	[2, 3, 1]
    C_3_inv	[3, 1, 2]
    S_a	[1, 3, 2]
    S_b	[3, 2, 1]
    S_c	[2, 1, 3]

### The `ConfigurationSpace` class

A `ConfigurationSpace` consists of a set of objects that represent the **configuration space** vectors, and the `SymmetryGroup` containing the relevant **symmetry operations**.

```python
from bsym import ConfigurationSpace
```

```python
c = ConfigurationSpace( objects=['a', 'b', 'c' ], symmetry_group=c3v )
```

```python
c
```

    ConfigurationSpace
    ['a', 'b', 'c']
    e	[1, 2, 3]
    C_3	[2, 3, 1]
    C_3_inv	[3, 1, 2]
    S_a	[1, 3, 2]
    S_b	[3, 2, 1]
    S_c	[2, 1, 3]

### The `Configuration` class

A `Configuration` instance describes a particular **configuration**, i.e. how a set of objects are arranged within a **configuration space**. Internally, a `Configuration` is represented as a vector (as a `numpy` array).
Each element in a configuration is represented by a single digit non-negative integer.

```python
from bsym import Configuration

conf_1 = Configuration( [ 1, 1, 0 ] )
conf_1
```

    Configuration([1 1 0])

The effect of a particular **symmetry operation** acting on a **configuration** can now be calculated using the `SymmetryOperation.operate_on()` method, or by direct multiplication, e.g.

```python
c1 = Configuration( [ 1, 1, 0 ] )
c_3 = SymmetryOperation.from_vector( [ 2, 3, 1 ] )
c_3.operate_on( c1 )
```

    Configuration([0 1 1])

```python
c_3 * conf_1
```

    Configuration([0 1 1])

## Finding symmetry-inequivalent permutations.

A common question that comes up when considering the symmetry properties of arrangements of objects is: how many ways can these be arranged that are not equivalent by symmetry?

As a simple example of solving this problem using `bsym` consider four equivalent sites arranged in a square.

```python
c = ConfigurationSpace( [ 'a', 'b', 'c', 'd' ] ) # four vector configuration space
```

This `ConfigurationSpace` has been created without a `symmetry_group` argument. The default behaviour in this case is to create a `SymmetryGroup` containing only the identity, $E$.

```python
c
```

    ConfigurationSpace
    ['a', 'b', 'c', 'd']
    E	[1, 2, 3, 4]

We can now calculate all symmetry inequivalent arrangements where two sites are occupied and two are unoccupied, using the `unique_configurations()` method. This takes as a argument a `dict` with the numbers of labels to be arranged in the **configuration space**. Here, we use the labels `1` and `0` to represent occupied and unoccupied sites, respectively, and the distribution of sites is given by `{ 1:2, 0:2 }`.

```python
c.unique_configurations( {1:2, 0:2} )
```

    [Configuration([0 0 1 1]),
     Configuration([0 1 0 1]),
     Configuration([0 1 1 0]),
     Configuration([1 0 0 1]),
     Configuration([1 0 1 0]),
     Configuration([1 1 0 0])]

Because we have not yet taken into account the symmetry of the **configuration space**, we get

\begin{equation}
\frac{4\times3}{2}
\end{equation}

unique configurations (where the factor of 2 comes from the occupied sites being indistinguishable).

The configurations generated by `unique_configurations` have a `count` attribute that records the number of *symmetry equivalent* configurations of each case:

In this example, each configuration appears once:

```python
[ uc.count for uc in c.unique_configurations( {1:2, 0:2} ) ]
```

    [1, 1, 1, 1, 1, 1]

We can also calculate the result when all symmetry operations of this **configuration space** are included.

```python
# construct point group
e        = SymmetryOperation.from_vector( [ 1, 2, 3, 4 ], label='E' )
c4       = SymmetryOperation.from_vector( [ 2, 3, 4, 1 ], label='C4' )
c4_inv   = SymmetryOperation.from_vector( [ 4, 1, 2, 3 ], label='C4i' )
c2       = SymmetryOperation.from_vector( [ 3, 4, 1, 2 ], label='C2' )
sigma_x  = SymmetryOperation.from_vector( [ 4, 3, 2, 1 ], label='s_x' )
sigma_y  = SymmetryOperation.from_vector( [ 2, 1, 4, 3 ], label='s_y' )
sigma_ac = SymmetryOperation.from_vector( [ 1, 4, 3, 2 ], label='s_ac' )
sigma_bd = SymmetryOperation.from_vector( [ 3, 2, 1, 4 ], label='s_bd' )
c4v = PointGroup( [ e, c4, c4_inv, c2, sigma_x, sigma_y, sigma_ac, sigma_bd ] )

# create ConfigurationSpace with the c4v PointGroup.
c = ConfigurationSpace( [ 'a', 'b', 'c', 'd' ], symmetry_group=c4v )
c
```

    ConfigurationSpace
    ['a', 'b', 'c', 'd']
    E	[1, 2, 3, 4]
    C4	[2, 3, 4, 1]
    C4i	[4, 1, 2, 3]
    C2	[3, 4, 1, 2]
    s_x	[4, 3, 2, 1]
    s_y	[2, 1, 4, 3]
    s_ac	[1, 4, 3, 2]
    s_bd	[3, 2, 1, 4]

```python
c.unique_configurations( {1:2, 0:2} )
```

    [Configuration([0 0 1 1]), Configuration([0 1 0 1])]

```python
[ uc.count for uc in c.unique_configurations( {1:2, 0:2 } ) ]
```

    [4, 2]

Taking symmetry in to account, we now only have two unique configurations: either two adjacent site are occupied (four possible ways), or two diagonal sites are occupied (two possible ways):

The `unique_configurations()` method can also handle non-binary site occupations:

```python
c.unique_configurations( {2:1, 1:1, 0:2} )
```

    [Configuration([0 0 1 2]), Configuration([0 1 0 2])]

```python
[ uc.count for uc in c.unique_configurations( {2:1, 1:1, 0:2 } ) ]
```

    [8, 4]

## Working with crystal structures using `pymatgen`

One example where the it can be useful to identify symmetry-inequivalent arrangements of objects in a vector space, is when considering the possible arrangements of disordered atoms on a crystal lattice.

To solve this problem for an arbitrary crystal structure, `bsym` contains an interface to [`pymatgen`](http://pymatgen.org) that will identify symmetry-inequivalent atom substitutions in a given `pymatgen` `Structure`.

As an example, consider a $4\times4$ square-lattice supercell populated by lithium atoms.

```python
from pymatgen import Lattice, Structure
import numpy as np
```

```python
# construct a pymatgen Structure instance using the site fractional coordinates
coords = np.array( [ [ 0.0, 0.0, 0.0 ] ] )
atom_list = [ 'Li' ]
lattice = Lattice.from_parameters( a=1.0, b=1.0, c=1.0, alpha=90, beta=90, gamma=90 )
parent_structure = Structure( lattice, atom_list, coords ) * [ 4, 4, 1 ]
parent_structure.cart_coords.round(2)
```

    array([[ 0.,  0.,  0.],
           [-0.,  1.,  0.],
           [-0.,  2.,  0.],
           [-0.,  3.,  0.],
           [ 1.,  0.,  0.],
           [ 1.,  1.,  0.],
           [ 1.,  2.,  0.],
           [ 1.,  3.,  0.],
           [ 2.,  0.,  0.],
           [ 2.,  1.,  0.],
           [ 2.,  2.,  0.],
           [ 2.,  3.,  0.],
           [ 3.,  0.,  0.],
           [ 3.,  1.,  0.],
           [ 3.,  2.,  0.],
           [ 3.,  3.,  0.]])

We can use the `bsym.interface.pymatgen.unique_structure_substitutions()` function to identify symmetry-inequivalent structures generated by substituting at different sites.

```python
from bsym.interface.pymatgen import unique_structure_substitutions
```

```python
print( unique_structure_substitutions.__doc__ )
```

        Generate all symmetry-unique structures formed by substituting a set of sites in a `pymatgen` structure.

        Args:
            structure (pymatgen.Structure): The parent structure.
            to_substitute (str): atom label for the sites to be substituted.
            site_distribution (dict): A dictionary that defines the number of each substituting element.
            verbose (bool): verbose output.
            atol      (Optional [float]):       tolerance factor for the ``pymatgen`` `coordinate mapping`_ under each symmetry operation. Default=1e-5.
            show_progress (opt:default=False): Show a progress bar.
                                               Setting to `True` gives a simple progress bar.
                                               Setting to `"notebook"` gives a Jupyter notebook compatible progress bar.

        Returns:
            (list[Structure]): A list of Structure objects for each unique substitution.

        Notes:
            The number of symmetry-equivalent configurations for each structure
            is stored in the `number_of_equivalent_configurations` attribute.

            If the parent structure was previously generated using this function
            (as part of a sequence of substitutions) the full configuration
            degeneracy of each symmetry inequivalent configuration is stored in
            the `full_configuration_degeneracy` attribute. If the parent structure
            is a standard Pymatgen Structure object, `number_of_equivalent_configurations`
            and `full_configuration_degeneracy` will be equal.

        .. _coordinate mapping:
            http://pymatgen.org/pymatgen.util.coord_utils.html#pymatgen.util.coord_utils.coord_list_mapping_pbc

As a trivial example, when substituting one Li atom for Na, we get a single unique structure

```python
unique_structures = unique_structure_substitutions( parent_structure, 'Li', { 'Na':1, 'Li':15 } )
len( unique_structures )
```

    1

```python
na_substituted = unique_structures[0]
```

This Li$\to$Na substitution breaks the symmetry of the $4\times4$ supercell.

If we now replace a second lithium with a magnesium atom, we generate five symmetry inequivalent structures:

```python
unique_structures_with_Mg = unique_structure_substitutions( na_substituted, 'Li', { 'Mg':1, 'Li':14 } )
len( unique_structures_with_Mg )
```

    5

```python
[ s.number_of_equivalent_configurations for s in unique_structures_with_Mg ]
```

    [4, 2, 4, 4, 1]

`number_of_equivalent_configurations` only lists the number of equivalent configurations found when performing the second substitution, when the list of structures `unique_structures_with_Mg` was created. The full configuration degeneracy relative to the initial empty 4×4 lattice can be queried using `full_configuration_degeneracy`.

```python
[ s.full_configuration_degeneracy for s in unique_structures_with_Mg ]
```

    [64, 32, 64, 64, 16]

```python
# Check the squared distances between the Na and Mg sites in these unique structures are [1, 2, 4, 5, 8]
np.array( sorted( [ s.get_distance( s.indices_from_symbol('Na')[0],
                                    s.indices_from_symbol('Mg')[0] )**2 for s in unique_structures_with_Mg ] ) )
```

    array([1., 2., 4., 5., 8.])

This double substitution can also be done in a single step:

```python
unique_structures = unique_structure_substitutions( parent_structure, 'Li', { 'Mg':1, 'Na':1, 'Li':14 } )
```

```python
len(unique_structures)
```

    5

```python
np.array( sorted( [ s.get_distance( s.indices_from_symbol('Na')[0],
                                    s.indices_from_symbol('Mg')[0] ) for s in unique_structures ] ) )**2
```

    array([1., 2., 4., 5., 8.])

```python
[ s.number_of_equivalent_configurations for s in unique_structures ]
```

    [64, 32, 64, 64, 16]

Because both substitutions were performed in a single step, `number_of_equivalent_configurations` and `full_configuration_degeneracy` now contain the same data:

```python
[ s.full_configuration_degeneracy for s in unique_structures ]
```

    [64, 32, 64, 64, 16]

## Constructing `SpaceGroup` and `ConfigurationSpace` objects using `pymatgen`

The `bsym.interface.pymatgen` module contains functions for generating `SpaceGroup` and `ConfigurationSpace` objects directly from `pymatgen` `Structure` objects.

```python
from bsym.interface.pymatgen import ( space_group_symbol_from_structure,
                                      space_group_from_structure,
                                      configuration_space_from_structure )
```

Documentation:

- [`space_group_symbol_from_structure`](http://bsym.readthedocs.io/en/latest/api/interface/pymatgen.html#bsym.interface.pymatgen.space_group_symbol_from_structure)
- [`space_group_from_structure`](http://bsym.readthedocs.io/en/latest/api/interface/pymatgen.html#bsym.interface.pymatgen.space_group_from_structure)
- [`configuration_space_from_structure`](http://bsym.readthedocs.io/en/latest/api/interface/pymatgen.html#bsym.interface.pymatgen.configuration_space_from_structure)

```python
coords = np.array( [ [ 0.0, 0.0, 0.0 ],
                     [ 0.5, 0.5, 0.0 ],
                     [ 0.0, 0.5, 0.5 ],
                     [ 0.5, 0.0, 0.5 ] ] )
atom_list = [ 'Li' ] * len( coords )
lattice = Lattice.from_parameters( a=3.0, b=3.0, c=3.0, alpha=90, beta=90, gamma=90 )
structure = Structure( lattice, atom_list, coords )
```

```python
space_group_symbol_from_structure( structure )
```

    'Fm-3m'

```python
space_group_from_structure( structure )
```

    SymmetryGroup
    None	[1, 2, 3, 4]
    None	[3, 4, 1, 2]
    None	[1, 4, 3, 2]
    None	[2, 3, 4, 1]
    None	[4, 3, 2, 1]
    None	[3, 2, 1, 4]
    None	[2, 1, 4, 3]
    None	[4, 1, 2, 3]
    None	[4, 1, 3, 2]
    None	[3, 2, 4, 1]
    None	[2, 4, 1, 3]
    None	[1, 3, 2, 4]
    None	[1, 4, 2, 3]
    None	[3, 1, 4, 2]
    None	[2, 3, 1, 4]
    None	[4, 2, 3, 1]
    None	[3, 1, 2, 4]
    None	[3, 4, 2, 1]
    None	[2, 4, 3, 1]
    None	[2, 1, 3, 4]
    None	[1, 3, 4, 2]
    None	[1, 2, 4, 3]
    None	[4, 2, 1, 3]
    None	[4, 3, 1, 2]

```python
configuration_space_from_structure( structure )
```

    ConfigurationSpace
    [1, 2, 3, 4]
    None	[1, 2, 3, 4]
    None	[3, 4, 1, 2]
    None	[1, 4, 3, 2]
    None	[2, 3, 4, 1]
    None	[4, 3, 2, 1]
    None	[3, 2, 1, 4]
    None	[2, 1, 4, 3]
    None	[4, 1, 2, 3]
    None	[4, 1, 3, 2]
    None	[3, 2, 4, 1]
    None	[2, 4, 1, 3]
    None	[1, 3, 2, 4]
    None	[1, 4, 2, 3]
    None	[3, 1, 4, 2]
    None	[2, 3, 1, 4]
    None	[4, 2, 3, 1]
    None	[3, 1, 2, 4]
    None	[3, 4, 2, 1]
    None	[2, 4, 3, 1]
    None	[2, 1, 3, 4]
    None	[1, 3, 4, 2]
    None	[1, 2, 4, 3]
    None	[4, 2, 1, 3]
    None	[4, 3, 1, 2]

## Progress bars

`bsym.ConfigurationSpace.unique_configurations()` and `bsym.interface.pymatgen.unique_structure_substitutions()` both accept optional `show_progress` arguments, which can be used to display progress bars (using `tqdm`(https://tqdm.github.io).

Setting `show_progress=True` will give a simple progress bar. If you are running `bsym` in a Jupyter notebook, setting `show_progress="notebook"` will give you a progress bar as a notebook widget.

(note, the widget status is not saved with this notebook, and may not display correctly on GitHub or using nbviewer)

In the example below, we find all unique configurations for the pseudo-ReO3 structured TiOF2 in a 2×2×2 supercell.

```python
a = 3.798 # lattice parameter

coords = np.array( [ [ 0.0, 0.0, 0.0 ],
                     [ 0.5, 0.0, 0.0 ],
                     [ 0.0, 0.5, 0.0 ],
                     [ 0.0, 0.0, 0.5 ] ] )
atom_list = [ 'Ti', 'X', 'X', 'X' ]
lattice = Lattice.from_parameters( a=a, b=a, c=a, alpha=90, beta=90, gamma=90 )
unit_cell = Structure( lattice, atom_list, coords )

parent_structure = unit_cell * [ 2, 2, 2 ]
unique_structures = unique_structure_substitutions( parent_structure, 'X', { 'O':8, 'F':16 },
                                                    show_progress='notebook' )
```

    HBox(children=(IntProgress(value=0, max=735471), HTML(value='')))

```python
%load_ext version_information
%version_information bsym, numpy, jupyter, pymatgen, tqdm
```

Software	Version

Python	3.7.0 64bit [Clang 10.0.0 (clang-1000.10.44.2)]

IPython	7.0.1

OS	Darwin 18.2.0 x86_64 i386 64bit

bsym	1.1.1

numpy	1.15.2

jupyter	1.0.0

pymatgen	2018.10.18

tqdm	4.28.1

Sun Feb 10 12:08:49 2019 GMT


# Extended Kalman filter for Abkowitz model
An Extended Kalman filter with an Abkowitz model as the predictor will be developed.

```python
%load_ext autoreload
%autoreload 2

import warnings
warnings.filterwarnings('ignore')

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from numpy.linalg import inv
import sympy as sp

import src.visualization.book_format as book_format
book_format.set_style()
from src.substitute_dynamic_symbols import lambdify
from sympy import Matrix
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify
from sympy.physics.vector.printing import vpprint, vlatex
from src.data import mdl
from src.kalman_filter import extended_kalman_filter
from src.models.vmm import ModelSimulator
from src.symbols import *
from src.models import vmm_abkowitz as vmm
import src.prime_system as prime_system
```

```python
#format the book
import src.visualization.book_format as book_format
book_format.set_style()
```

## Load model

```python
model = ModelSimulator.load('../models/model_VCT_abkowitz.pkl')
```

### Run a zigzag simulation with the model

```python
u0_=2
angle_deg = 35
result = model.zigzag(u0=u0_, angle=angle_deg)
```

```python
result.track_plot();
result.plot(compare=False);
```

```python
x_0,x_01d = sp.symbols('x_0 \dot{x_0}')
y_0,y_01d = sp.symbols('y_0 \dot{y_0}')
u,u1d = sp.symbols('u \dot{u}')
v,v1d = sp.symbols('v \dot{v}')
delta = sp.symbols('delta')
psi,psi1d = sp.symbols('\Psi \dot{\Psi}')
r,r1d = sp.symbols('r \dot{r}')

```

```python
from src.symbols import dynamicsymbols
subs = [

    (dynamicsymbols('u').diff(),u1d),
    (dynamicsymbols('v').diff(),v1d),
    (dynamicsymbols('r').diff(),r1d),

    (dynamicsymbols('u'),u),
    (dynamicsymbols('v'),v),
    (dynamicsymbols('r'),r),

    (dynamicsymbols('delta'),delta),

]

X_eq = vmm.X_eq.subs(subs)
Y_eq = vmm.Y_eq.subs(subs)
N_eq = vmm.N_eq.subs(subs)
```

```python
subs_prime = [

    (m,m/prime_system.df_prime.mass.denominator),
    (I_z,I_z/prime_system.df_prime.inertia_moment.denominator),
    (x_G,x_G/prime_system.df_prime.length.denominator),

    (u, u/sp.sqrt(u**2+v**2)),
    (v, v/sp.sqrt(u**2+v**2)),
    (r, r/(sp.sqrt(u**2+v**2))/L),

    (u1d, u1d/((u**2+v**2)/L)),
    (v1d, v1d/((u**2+v**2)/L)),
    (r1d, r1d/((u**2+v**2)/(L**2))),

]
```

```python
X_eq_SI = X_eq.subs(subs_prime)
Y_eq_SI = Y_eq.subs(subs_prime)
N_eq_SI = N_eq.subs(subs_prime)
```

```python
A_SI, b_SI = sp.linear_eq_to_matrix([X_eq_SI, Y_eq_SI, N_eq_SI], [u1d, v1d, r1d])
```

```python
A_SI
```

```python
A_inv = A_SI.inv()
S = sp.symbols('S')
eq_S=sp.Eq(S,-sp.fraction(A_inv[1,1])[1])
```

```python
eq_S
```

```python
acceleration = sp.matrices.MutableDenseMatrix([u1d,v1d,r1d])
acceleartion_eq = sp.Eq(acceleration,A_inv * b_SI)

acceleartion_eq
```

```python
u1d_function = sp.Function(r'\dot{u}')(u,v,r,delta)
v1d_function = sp.Function(r'\dot{v}')(u,v,r,delta)
r_function = sp.Function(r'\dot{r}')(u,v,r,delta)

eq_u1d_function = sp.Eq(u1d_function, acceleartion_eq.rhs[0])
eq_v1d_function = sp.Eq(v1d_function, acceleartion_eq.rhs[1])
eq_r_function = sp.Eq(r_function, acceleartion_eq.rhs[2])

display(eq_u1d_function)
display(eq_v1d_function)
display(eq_r_function)

```

```python
eq_acceleartion_function = sp.Eq(acceleration,sp.matrices.MutableDenseMatrix([u1d_function,v1d_function,r_function]))
display(eq_acceleartion_function)
```

### Simulation model

```python
x, x1d = sp.symbols(r'\vec{x} \dot{\vec{x}}')  # State vector
h = sp.symbols('h')
u_input = sp.symbols(r'u_{input}')  # input vector
w_noise = sp.symbols(r'w_{noise}')  # input vector
```

```python
f = sp.Function('f')(x,u_input,w_noise)
eq_system = sp.Eq(x1d, f)
eq_system
```

Where the state vector $\vec{x}$:

```python
eq_x = sp.Eq(x, sp.UnevaluatedExpr(Matrix([x_0, y_0, psi, u, v, r])))
eq_x
```

```python
eq_x0_1d = sp.Eq(x_01d,u)
eq_y0_1d = sp.Eq(y_01d,v)
eq_psi_1d = sp.Eq(psi1d,r)

display(eq_x0_1d)
display(eq_y0_1d)
display(eq_psi_1d)

```

```python
eq_f =sp.Eq(f,
sp.UnevaluatedExpr(
Matrix([
    eq_x0_1d.rhs,
    eq_y0_1d.rhs,
    eq_psi_1d.rhs,
    eq_u1d_function.lhs,
    eq_v1d_function.lhs,
    eq_r_function.lhs,

]
))
)
display(eq_f)

```

```python
f_explicit = Matrix([
    eq_x0_1d.rhs,
    eq_y0_1d.rhs,
    eq_psi_1d.rhs,
    eq_u1d_function.rhs,
    eq_v1d_function.rhs,
    eq_r_function.rhs,

]
)

```

```python
eq_f.rhs.doit().jacobian(eq_x.rhs.doit())
```

```python
Ad = sp.eye(6,6) + Matrix([
    eq_x0_1d.rhs,
    eq_y0_1d.rhs,
    eq_psi_1d.rhs,
    eq_u1d_function.rhs,
    eq_v1d_function.rhs,
    eq_r_function.rhs,

]
).jacobian(eq_x.rhs.doit())*h
```

```python
from src.parameters import df_parameters
p = df_parameters["symbol"]
subs = {value: key for key, value in p.items()}
```

```python
lambda_f = lambdify(f_explicit.subs(subs))
```

```python
lambda_f
```

```python
output = run(lambda_f, inputs=result.result, **model.ship_parameters, **model.parameters, h=h_)
```

```python
output.shape
```

```python
def simulate(E, ws, t, us):

    simdata = []
    x_=np.deg2rad(np.zeros((6,1)))

    for u_,w_ in zip(us,ws):

        u = x_[3]
        v = x_[4]
        r = x_[5]
        delta = u_

        x_=x_ + h_*run(lambda_f, u=u, v=v, r=r, delta=delta, **model.ship_parameters, **model.parameters,)

        simdata.append(x_.flatten())

    simdata = np.array(simdata)
    df = pd.DataFrame(simdata, columns=["x0","y0","psi","u","v","r"], index=t)
    df['delta'] = us

    return df
```

```python
np.random.seed(42)
df = result.result.iloc[0:100]
N_ = len(df)
E = np.array([[0, 1]]).T
process_noise = 0
ws = process_noise*np.random.normal(size=N_)

us = df['delta'].values
t_ = df.index
h_ = t_[1]-t_[0]

df2 = simulate(E=E, ws=ws, t=t_, us=us)
```

```python

```


# Idéer til R&D endogen vækst modeller med et dynamisk element

```python

```

```python
import matplotlib.pyplot as plt
from scipy import optimize
import numpy as np
import sympy as sm
sm.init_printing(use_unicode=True)
```

```python
def plot(x,y):
    fig = plt.figure()
    ax = fig.add_subplot(1,1,1)
    ax.plot(x,y)
```

# mikro problemet
Optimal arbejdsfordeling

I hver periode, t, skal den enkelte repræsentative forbruger vælge hvor meget han vil arbejde med produktion og arbejde med forskning. Produktion bruges til forbrug i periode t og kapitalopsparing som bidrag til fremtidig produktion. Forskning øger produktiviten i fremtidige perioder .

```python
#valgmuligheder:
# andelen af mulig arbejdstid som bruges på forskning:
sr = sm.symbols('s_Rt')
# I starten antages kapitalopspsparingsraten at være konstant
sk = sm.symbols('s_Kt')
```

```python
# Ud fra de valg nogle udfald
# Først defineres variable
l = sm.symbols('L_{t}')
ly = sm.symbols('L_{Yt}')
la = sm.symbols('L_{At}')
alpha = sm.symbols('alpha')
k =sm.symbols('K_{t}')
k1 = sm.symbols('K_{t+1}')
y = sm.symbols('Y_{t}')
c = sm.symbols('C')
ct = sm.symbols('C_{t}')
a = sm.symbols('A_{t}')
a1 = sm.symbols('A_{t+1}')
r = sm.symbols('r')

#Greek:
rho = sm.symbols('rho')
beta = sm.symbols('beta')
alpha = sm.symbols('alpha')
phi = sm.symbols('phi')
delta = sm.symbols('delta')
theta = sm.symbols('theta')

t = sm.symbols('t')

# Functions:
uc = sm.Function('u')(ct)
u = sm.Function('U')(c)
```

Nyttefunktion:

```python
sm.Eq(uc,(ct**(1-theta)-1)/(1-theta))
```

```python
hello = sm.Limit((ct**(1-theta)-1)/(1-theta),theta,1)
```

```python
hello
```

```python
hello.doit()
```

```python
sm.diff(ct**(1-rho),rho)
```

```python
sm.Eq(u,sm.Sum(beta**t*(ct**(1-theta)-1)/(1-theta),(t,1,3)))
```

*** Value of capital leftover capital ***

```python
k/(1+r)
```

```python
sm.Eq(y,a*k**alpha*(l*(1-sr))**(1-alpha))
```

```python
mpl = sm.Derivative(a*k**alpha*(l*(1-sr))**(1-alpha),k)
```

```python
sm.Eq(r,mpl)
```

```python
sm.Eq(r,mpl.doit())
```

```python
a*alpha*((k/l*(1-sr))**(alpha-1))
```

Betingelser i produktion:

```python
sm.Eq(c,sk*y)
```

Tidshorisonts betingelser

```python
sm.Eq(a1,a*(1+rho*(l*sr)**phi))
```

```python
sm.Eq(k1,sk*y+(1-delta)*k)
```

```python
sm.init_printing(use_unicode=False)
```

```python
def utility(c,theta):
    ''' Utility function for consumption including the limit case for rho->1 '''
    if c<0:
        raise error
    if rho == 1:
        return np.log(c)
    else:
        return (c**(1-theta)-1)/(1-theta)

def total_utility(c, beta, theta, k):
    '''
    Sums utility for c for multiple years
    c is an array
    '''
    t_u = 0
    for i,ct in enumerate(c):
        t_u += beta**i*utility(ct,theta)
    t_u += beta**(len(c))*utility(ct,theta)
    return t_u

def prod(k,l,alpha,b):
    return b*k**alpha*l**(1-alpha)
```

```python
def tot_ut_sk(sk,t, k0, b, beta, alpha, theta, delta, detail=False):
    '''
    Returns utitlity for  given period
    depending on initial amount of kapital, k0,
    and a fixed savingsrate, sk, for all periods
    '''
    k_short = np.empty(t)
    c_short = np.empty(t)
    y_short = np.empty(t)
    k_short[0] = k0
    for i in range(t):
        y_short[i] = prod(k_short[i],l,alpha,b)
        c_short[i]=(1-sk)*y_short[i]
        if i+1<t:
            k_short[i+1]=sk*y_short[i]+(1-delta)*k_short[i]
    if detail:
        return k_short,c_short,y_short
    else:
        return total_utility(c_short,beta,theta,k_short[-1])

def opt_sk(t, b, beta, delta, alpha, theta, k0):
    '''Finds optimal fixed savings rate, sk,
    for maximal total utility '''
    obj = lambda sk: -tot_ut_sk(sk,t,k0, b, beta, alpha, theta, delta)
    res = optimize.minimize_scalar(obj, alpha, method='bounded',bounds=[0,1])
    return res.x
```

```python
theta = 0.5
alpha = 1/3
delta = 0.05
```

```python
t = 1000
b = 10
k0 = 10
l = 1
beta = 0.9
```

```python
obj = lambda sk: -tot_ut_sk(sk,t, k0, b, beta,alpha, theta, delta)
sk0 = [0.3]
res = optimize.minimize(obj, sk0, method='Nelder-Mead')
print(f'Optimal sk = {res.x[0]:.4f}, with total utility = {-res.fun:.4f}')
```

    Optimal sk = 0.2419, with total utility = 84.1670

```python
obj = lambda sk: -tot_ut_sk(sk,t, k0, b, beta,alpha, theta, delta)
sk0 = [0.3]
res = optimize.minimize_scalar(obj, sk0, method='bounded',bounds=[0,1])
print(f'Optimal sk = {res.x:.4f}, with total utility = {-res.fun:.4f}')
```

    Optimal sk = 0.2420, with total utility = 84.1670

```python
n = 100
ks = np.linspace(0.1,200,n)
sks = np.zeros(n)
for i,k in enumerate(ks):
    sks[i] = opt_sk(t, b, beta,delta,alpha,theta,k)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(ks,sks)
```

# Hvad hvis forskellige opsparingsrater kan vælges hvert år?

```python
def tot_ut_multiple_sks(sks, k0, b, beta, theta, alpha, delta, vector=False):
    '''
    Finds total utitilty for a set of years with a savingsrate for each year
    '''
    t = len(sks)
    k_short = np.empty(t)
    c_short = np.empty(t)
    y_short = np.empty(t)
    k_short[0] = k0
    for i in range(t):
        y_short[i] = prod(k_short[i],l,alpha,b)
        c_short[i]=(1-sks[i])*y_short[i]
        if i+1<t:
            k_short[i+1]=sks[i]*y_short[i]+(1-delta)*k_short[i]
    if vector==False:
        return total_utility(c_short, beta, theta, k_short[-1])
    elif vector==True:
        return total_utility(c_short, beta, theta, k_short[-1]), y_short, c_short, k_short
```

```python
theta = 0.9
alpha = 1/3
delta = 0.05
beta = 1
b = 100
k0 = 20
l = 1
```

```python
sks = [0.4, 0.3201, 0.2988, 0.9, 0.]
tot_ut_multiple_sks(sks, k0, b, beta, theta, alpha,delta)
```

```python
t = 200
obj = lambda sks: -tot_ut_multiple_sks(sks, k0, b, beta, theta, alpha, delta)
sks0 = np.full(t,0.3)
sks0[-1] = 0
bounds = np.full((t,2),[0,1])

res = optimize.minimize(obj, sks0, method='SLSQP', bounds=bounds)

print(f'Optimal sk')
np.set_printoptions(precision=4,suppress=True)
print(res.x)
print('Total utility:')
print(f'{-res.fun:.4f}')
```

    Optimal sk
    [0.6413 0.6466 0.6271 0.6044 0.583  0.5634 0.5458 0.5299 0.5154 0.5023
     0.4904 0.4794 0.4693 0.4601 0.4516 0.4437 0.4365 0.4297 0.4234 0.4177
     0.4122 0.4072 0.4026 0.3982 0.3941 0.3903 0.3868 0.3835 0.3804 0.3775
     0.3747 0.3722 0.3698 0.3676 0.3655 0.3635 0.3617 0.36   0.3584 0.3569
     0.3555 0.3541 0.3529 0.3517 0.3506 0.3495 0.3485 0.3476 0.3468 0.346
     0.3453 0.3445 0.3439 0.3432 0.3426 0.3421 0.3415 0.341  0.3406 0.3401
     0.3397 0.3393 0.3389 0.3386 0.3383 0.338  0.3377 0.3374 0.3372 0.3369
     0.3367 0.3365 0.3362 0.3361 0.3359 0.3357 0.3355 0.3354 0.3352 0.3351
     0.3349 0.3348 0.3347 0.3346 0.3345 0.3343 0.3342 0.3341 0.334  0.3339
     0.3338 0.3337 0.3336 0.3336 0.3334 0.3333 0.3332 0.3332 0.3331 0.333
     0.3329 0.3328 0.3327 0.3326 0.3325 0.3324 0.3322 0.3321 0.332  0.3319
     0.3318 0.3316 0.3315 0.3313 0.3312 0.331  0.3308 0.3307 0.3305 0.3303
     0.33   0.3298 0.3296 0.3294 0.3291 0.3288 0.3285 0.3282 0.3278 0.3274
     0.3271 0.3266 0.3262 0.3257 0.3252 0.3247 0.3241 0.3236 0.3229 0.3222
     0.3215 0.3207 0.3199 0.319  0.3181 0.3171 0.3161 0.3149 0.3137 0.3124
     0.3111 0.3096 0.308  0.3063 0.3046 0.3026 0.3006 0.2984 0.296  0.2934
     0.2907 0.2877 0.2846 0.2811 0.2774 0.2735 0.2692 0.2645 0.2595 0.2541
     0.2482 0.2417 0.2347 0.2271 0.2187 0.2095 0.1993 0.188  0.1757 0.1619
     0.1465 0.1291 0.1094 0.0869 0.0614 0.0319 0.     0.     0.     0.
     0.     0.     0.     0.     0.     0.     0.     0.     0.     0.    ]
    Total utility:
    2189.2619

```python
def optimal_sks(t, b, beta, delta, alpha, theta, k0, first=True):
    obj = lambda sks: -tot_ut_multiple_sks(sks, k0, b, beta, theta, alpha, delta)
    sks0 = np.full(t,0.3)
    sks0[-1] = 0
    bounds = np.full((t,2),[0,1])
    res = optimize.minimize(obj, sks0, method='SLSQP', bounds=bounds)
    if res.success == False:
        print('Optimization was sadly not succesfull')
    elif first:
        return res.x[0]
    else:
        return res.x
```

```python
theta = 0.9
alpha = 1/3
delta = 0.05
beta = 1
b = 2
beta = 0.99
k0 = 40
l = 1
t = 100
```

```python
opt_sk(t, b, beta, delta, alpha, theta, k0)
```

```python
sks = optimal_sks(t, b, beta, delta, alpha, theta, k0,first=False)
```

```python
total, y, c, k = tot_ut_multiple_sks(sks, k0, b, beta, theta, alpha, delta, vector=True)
```

```python
sks_alt = [alpha for i in list(range(t))]
total_alt, y_alt, c_alt, k_alt = tot_ut_multiple_sks(sks_alt, k0, b, beta, theta, alpha, delta, vector=True)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(list(range(t)),sks)
ax.plot(list(range(t)),sks_alt)
```

```python

```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(list(range(t)),k)
ax.plot(list(range(t)),k_alt)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(list(range(t)),c)
ax.plot(list(range(t)),c_alt)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(list(range(t)), [utility(ct,theta)for ct in c])
ax.plot(list(range(t)), [utility(ct,theta)for ct in c_alt])
```

```python
ks = np.linspace(0.1,100,100)
sks1 = np.empty(100)

for i, k in enumerate(ks):
    sks1[i] = optimal_sks(t, b, beta, delta, alpha, theta, k)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(ks,sks1)
```

```python
ks = np.linspace(1,600,50)
sks1 = np.empty(50)

for i, k in enumerate(ks):
    sks1[i] = optimal_sks(t, b, beta, delta, alpha, theta, k)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(ks,sks1)
```

# Need for speed

```python
import time
```

```python

```

# Makro sammensætning

```python
sm.init_printing(use_unicode=True)
```

```python
#valgmuligheder:
# andelen af mulig arbejdstid som bruges på forskning:
sr = sm.symbols('s_Rt')
# I starten antages kapitalopspsparingsraten at være konstant
sk = sm.symbols('s_Kt')

# Ud fra de valg nogle udfald
# Først defineres variable
l = sm.symbols('L_{t}')
alpha = sm.symbols('alpha')
k =sm.symbols('K_{t}')
k1 = sm.symbols('K_{t+1}')
y = sm.symbols('Y_{t}')
c = sm.symbols('C')
ct = sm.symbols('C_{t}')
a = sm.symbols('A_{t}')
a1 = sm.symbols('A_{t+1}')
r = sm.symbols('r')

#Greek:
rho = sm.symbols('rho')
beta = sm.symbols('beta')
alpha = sm.symbols('alpha')
phi = sm.symbols('phi')
delta = sm.symbols('delta')
lambd = sm.symbols('lambda')
theta = sm.symbols('theta')

t = sm.symbols('t')

# Functions:
uc = sm.Function('u')(ct)
u = sm.Function('U')(c)
```

```python
sm.Eq(y,a*k**alpha*(l*(1-sr))**(1-alpha))
```

```python
sm.Eq(k1,sk*y+(1-delta)*k)
```

```python
sm.Eq(a1,a+rho*a**phi*(l*sr)**lambd)
```

```python
# parameters:
alpha = 1/3
delta = 0.05
rho = 0.9
phi = 0.1
lambd = 0.1
n=0

# fixed parameters
sr = 0.05
sk = 0.2

# starting values:
l0 = 100
k0 = 1000
a0 = 1000
```

```python
# funktiions
def prod(a,k,l,sr,alpha):
    return k**alpha*(a*(1-sr)*l)**(1-alpha)
def capitalakku(a,k,l,sr,sk,alpha,delta):
    return prod(a,k,l,sr,alpha)*sk+(1-delta)*k
def research(a,sr,l,rho,phi):
    return a+rho*a**phi*(l*sr)**lambd
```

```python
def solowwalk(k0,a0, l0, n, sr, sk, alpha, delta, rho, phi, t):
    k_path=np.array([k0])
    a_path=np.array([a0])
    l_path = [(1+n)**i for i in list(range(end))]

    for i in range(1,t):
        k_plus = capitalakku(a_path[i-1],k_path[i-1],l_path[i-1],sr,sk,alpha,delta)
        a_plus = research(a_path[i-1],sr,l_path[i-1],rho,phi)

        np.append(k_path, k_plus)
        np.append(a_path, a_plus)
    k_tilde_path = k_path/(a_path*l_path)
    return a_path, k_path, k_tilde_path
```

```python
t=10000
```

```python
plot(range(t),np.log(a_path))
```

```python
end = 10000
ks = np.empty(end)
ys = np.empty(end)
ls = [(1+n)**i for i in list(range(end))]
ks[0] = k0
ats = np.zeros(end)
ats[0] = a0
margin = 1e-3
ys[0] = prod(ats[0],ks[0],ls[0],sr,alpha)
ks[1]   = capitalakku(ats[0],ks[0],ls[0],sr,sk,alpha,delta)
ats[1] = research(ats[0],sr,ls[0],rho,phi)
```

```python

t = 1
while True:
    ys[t] = prod(ats[t],ks[t],ls[t],sr,alpha)
    if -margin<ys[t]-ys[t-1]<margin:
        k_tilde = ks[t]/(ats[t]*ls[t])
        y_tilde = ys[t]/(ats[t]*ls[t])
        print(f'Succesful convergens')
        print(f'Periode = {t}')
        print(f'y_tilde = {y_tilde:.4f}')
        print(f'k_tilde = {k_tilde:.4f}')
        break
    ks[t+1]   = capitalakku(ats[t],ks[t],ls[t],sr,sk,alpha,delta)
    ats[t+1] = research(ats[t],sr,ls[t],rho,phi)
    t += 1
    if t+1 == end:
        print('no convergens')
        print(t)
        break
```

```python
ks_small = ks/ls
```

```python
plot(range(t),ks_small[:t])
```

Simulation

```python
t = 1000
ys = np.zeros(t)
ks = np.zeros(t)
ls = [(1+n)**i for i in list(range(t))]
ks[0] = k0
ats = np.zeros(t)
ats[0] = a0
```

```python
for i in list(range(t)):
    ys[i] = prod(ats[i],ks[i],ls[i],sr,alpha)
    if i+1<t:
        ks[i+1]   = capitalakku(ats[i],ks[i],ls[i],sr,sk,alpha,delta)
        ats[i+1] = research(ats[i],sr,ls[i],rho,phi)

```

```python
y_small = ys/ls
y_tildes = y_small/ats
log_ys = np.log(ys)
ks_small = ks/ls
ks_tilde = ks_small/ats
log_ats = np.log(ats)
log_ls = np.log(ls)
log_ks = np.log(ks)
log_ks_small = np.log(ks_small)
```

```python
plot(range(t),ks_small)
```

```python
plot(range(t),log_ats)
plot(range(t),log_ls)
plot(range(t),log_ks)
plot(range(t),log_ys)
```

# Makro med mikro-valg

```python
rho = 0.5
alpha = 1/3
delta = 0.05
beta = 0.5
b = 10
l = 1
t= 50
k0 = 10
l = 1
beta = 0.9
```

```python
opt_sk(t, b, beta,delta,alpha,rho,k0)
```

```python
t_store = 25
c = np.empty(t_store)
k = np.empty(t_store)
k[0] = 1
y = np.empty(t_store)
sks = np.empty(t_store)
```

```python
for i in range(t_store):
    k0 = k[i]
    sks[i] = opt_sk(t,b, beta,delta,alpha,rho,k0)
    if i+1<t_store:
        k[i+1] = sks[i]*prod(k[i],l,alpha,b)-delta*k[i]
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(range(t_store),sks)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(range(t_store),k)
```

```python
def capakku(k1,l,sk,delta,alpha,b):
    return sk*prod(k1,l,alpha,b)-delta*k1
```

```python
t=10
alpha = 1/3
delta = 0.05
sk_fix = 1/3
n= 0.01
```

```python
ks = np.empty(t)
ys = np.empty(t)
l = 1
ks[0]= 5.6
```

```python
for i in range(t):
    ys[i]=prod(ks[i],l,alpha,b)
    if i+1<t:
        ks[i+1] = capakku(ks[i],l,sk_fix,delta,alpha,b)
```

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(range(t),ys)
```


```python
from sympy.physics.units import *
from sympy import *

# Rounding:
import decimal
from decimal import Decimal as DX
def iso_round(obj, pv, rounding=decimal.ROUND_HALF_EVEN):
    import sympy
    """
    Rounding acc. to DIN EN ISO 80000-1:2013-08
    place value = Rundestellenwert
    """
    assert pv in set([
        # place value   #  round to:
        0.1,            #  1st digit after decimal
        0.01,           #  2nd
        0.001,          #  3rd
        0.0001,         #  4th
        0.00001,        #  5th
        0.000001,       #  6th
        0.0000001,      #  7th
        0.00000001,     #  8th
        0.000000001,    #  9th
        0.0000000001,   # 10th
        ])
    try:
        tmp = DX(str(float(obj)))
        obj = tmp.quantize(DX(str(pv)), rounding=rounding)
    except:
        for i in range(len(obj)):
            tmp = DX(str(float(obj[i])))
            obj[i] = tmp.quantize(DX(str(pv)), rounding=rounding)
    return obj

# LateX:
kwargs = {}
kwargs["mat_str"] = "bmatrix"
kwargs["mat_delim"] = ""
# kwargs["symbol_names"] = {FB: "F^{\mathsf B}", }

# Units:
(k, M, G ) = ( 10**3, 10**6, 10**9 )
(mm, cm, deg) = ( m/1000, m/100, pi/180)
Newton = kg*m/s**2
Pa     = Newton/m**2
MPa    = M*Pa
GPa    = G*Pa
kN     = k*Newton

half = S(1)/2

# ---

E, A, l, F = var("E, A, l, F")

c = E*A/l

F = var("F")
c1,c2,c3,c4,c5 = var("c1,c2,c3,c4,c5")
u1,u2,u3,u4 = var("u1,u2,u3,u4")
F1,F2,F3,F4 = var("F1,F2,F3,F4")

u1 = 0
F2 = 0
F3 = 0
F4 = F

sub_list = [
    (c1, c),
    (c2, c),
    (c3, c),
    (c4, 2*c),
    (c5, c),
    ]

u = Matrix([u1,u2,u3,u4])
f = Matrix([F1,F2,F3,F4])
pprint("\nK / c:")
K = Matrix([
    [ c1, -c1, 0, 0 ],
    [ -c1, c1+c2+c3+c4, -c2-c3, -c4 ],
    [0,-c2-c3,c2+c3+c5,-c5],
    [0,-c4,-c5,c5+c4]
    ])

K = K.subs(sub_list)
tmp = K
tmp /= E*A/l
pprint(tmp)

pprint("\n3x3-submatrix / c:")
K33 = tmp[1:,1:]
pprint(K33)
pprint("\nInverse / (1/c):")
K33i = K33.inv()
pprint(K33i)

pprint("\nSolution:")
sol = solve(Eq(K*u,f),[u2,u3,u4,F1], dict=True)
sol = sol[0]
for x in sol:
    pprint("\n")
    tmp = str(x)+":"
    pprint(tmp)
    tmp = sol[x]
    pprint(tmp)

# K / c:
# ⎡1   -1  0   0 ⎤
# ⎢              ⎥
# ⎢-1  5   -2  -2⎥
# ⎢              ⎥
# ⎢0   -2  3   -1⎥
# ⎢              ⎥
# ⎣0   -2  -1  3 ⎦
#
# 3x3-submatrix / c:
# ⎡5   -2  -2⎤
# ⎢          ⎥
# ⎢-2  3   -1⎥
# ⎢          ⎥
# ⎣-2  -1  3 ⎦
#
# Inverse / (1/c):
# ⎡1   1     1  ⎤
# ⎢             ⎥
# ⎢1  11/8  9/8 ⎥
# ⎢             ⎥
# ⎣1  9/8   11/8⎦
#
# Solution:
#
# u2:
# F⋅l
# ───
# A⋅E
#
# u3:
# 9⋅F⋅l
# ─────
# 8⋅A⋅E
#
# u4:
# 11⋅F⋅l
# ──────
# 8⋅A⋅E
#
# F1:
# -F

```


```python
import matplotlib.pyplot as plt
from mpl_toolkits import mplot3d
import numpy as np
import sympy as sym
```

```python
def f(X, n, q, b, c):
    Z = np.zeros(len(X))
    for i in range(len(X)):
        for j in range(int(n)):
            for k in range(int(n)):
                Z[i] += q[j][k]*X[i][j]*X[i][k]

        for j in range(int(n)):
            Z[i] += b[j]*X[i][j]
    return Z
```

```python
def f_mesh(X, Y, q, b, c):
    Z = np.zeros(len(X))
    Z = q[0][0]*X*X + q[0][1]*X*Y + q[1][0]*Y*X + q[1][1]*Y*Y + b[0]*X + b[1]*Y + c
    return Z
```

```python
def f2(X, Y, n, q, b, c):
    Z = q[0][0]*X*X + q[0][1]*X*Y + q[1][0]*Y*X + q[1][1]*Y*Y + b[0]*X + b[1]*Y + int(c[0])
    return Z
```

```python
def rmsprop(X_new, X1, Y1, Z1, n, q, b, c, alpha=0.10, beta=0.9, precision=0.0001, max_iter=200):
    X_old = np.zeros((1, 2))
    X_new = np.zeros((1, 2))
    dfr = np.zeros((1, 2))

    X_new[0][0] = 4.9
    X_new[0][1] = 4.9
    i = 0
    Xs = np.zeros((max_iter, 2))
    Ys = np.zeros(max_iter)

    S = np.zeros((max_iter + 1, 2))

    x, y = sym.symbols('x y')
    df1 = sym.diff(f2(x, y, n, q, b, c), x)
    df2 = sym.diff(f2(x, y, n, q, b, c), y)

    while np.all(abs(X_new - X_old)) > precision and max_iter > i:
        Xs[i] = X_new
        Ys[i] = f2 (X_new[0][0], X_new[0][1], n, q, b, c)

        X_old = X_new

        dfr[0][0] = df1.evalf(subs={x: X_old[0][0], y: X_old[0][1]})
        dfr[0][1] = df2.evalf(subs={x: X_old[0][0], y: X_old[0][1]})

        i += 1
        S[i] = beta * S[i-1] + (1 - beta) * np.power(dfr, 2)
        X_new = X_new - alpha * dfr/np.sqrt(S[i])

        alpha *= 0.99

    print("Finished with {} step".format(i))
    if (i < max_iter):
        Xs[i] = X_new
        Ys[i] = f2(X_new[0][0], X_new[0][1], n, q, b, c)

        for j in range(max_iter - 1, i, -1):
            Xs = np.delete(Xs, j, axis=0)
            Ys = np.delete(Ys, j, axis=0)
    return Xs, Ys

```

```python
X1 = np.arange(-5, 5, 0.1)
Y1 = np.arange(-5, 5, 0.1)
Z1 = np.zeros(len(X1))

X_new = np.zeros((100,2))

for i in range(len(X1)):
    X_new[i][0] = X1[i]
    X_new[i][1] = Y1[i]

Z1 = f(X_new, n, q, b, c)

x_list, y_list = rmsprop(X_new, X1, Y1, Z1, n, q, b, c)
```

    Finished with 200 step

```python
n = input("Enter power of your function: ")
q = []
for i in range(int(n)):
    q.append(input("Enter the function's coefficient matrix q's row q[{}]: ".format(i)).split())
b = input("Enter the function's coefficient b matrix: ").split()
c = input("Enter the function's constant c: ")

for i in range(int(n)):
    q[i] = list(map(float, q[i]))
b = list(map(float, b))
c = list(map(float, c))
```

    Enter power of your function: 2
    Enter the function's coefficient matrix q's row q[0]: 1 0
    Enter the function's coefficient matrix q's row q[1]: 0 2
    Enter the function's coefficient b matrix: 0 0
    Enter the function's constant c: 0

```python
X1, Y1 = np.meshgrid(X1, Y1)
Z1 = f_mesh(X1, Y1, q, b, c)

X, Y = zip(*x_list)
Z = y_list

ax = plt.subplots(nrows=1, ncols=1, figsize=(10,10))
cs = plt.contour(X1, Y1, Z1)
plt.clabel(cs, inline=1, fontsize=10)
colors = ['b', 'g', 'm', 'c', 'orange']
for j in range(1, len(X)):
    ax[1].annotate('', xy=(X[j], Y[j]), xytext=(X[j-1], Y[j-1]),
                arrowprops={'arrowstyle': '->', 'color': 'r', 'lw': 1},
                va='center', ha='center')
ax[1].scatter(X, Y, s=40, lw=0)
ax[1].set_xlabel('X')
ax[1].set_ylabel('Y')
ax[1].set_title('Minimizing function')
```

```python

```


```python
# Load modules
from __future__ import print_function
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
import statsmodels.tsa.api as smtsa
```

```python

# Function to plot signal, ACF and PACF
def plotds(xt, nlag=30, fig_size=(12, 10)):
    if not isinstance(xt, pd.Series):
         xt = pd.Series(xt)
    plt.figure(figsize=fig_size)
    layout = (2, 2)

    # Assign axes
    ax_xt = plt.subplot2grid(layout, (0, 0), colspan=2)
    ax_acf= plt.subplot2grid(layout, (1, 0))
    ax_pacf = plt.subplot2grid(layout, (1, 1))

    # Plot graphs
    xt.plot(ax=ax_xt)
    ax_xt.set_title('Time Series')
    plot_acf(xt, lags=50, ax=ax_acf)
    plot_pacf(xt, lags=50, ax=ax_pacf)
    plt.tight_layout()
    return None
```

```python
# Number of samples
n = 600

# Generate AR(1) dataset
ar = np.r_[1, -0.6]
ma = np.r_[1, 0]
ar1_data = smtsa.arma_generate_sample(ar=ar, ma=ma, nsample=n)
plotds(ar1_data)
plt.savefig('../plots/ch4/P_01.png', format='png', dpi=300)
```

```python
# Generate AR(2) dataset
ar = np.r_[1, 0.6, 0.7]
ma = np.r_[1, 0]
ar2_data = smtsa.arma_generate_sample(ar=ar, ma=ma, nsample=n)
plotds(ar2_data)

```

```python
# Generate AR(3) dataset
ar = np.r_[1, 0.6, 0.7, 0.5]
ma = np.r_[1, 0]
ar3_data = smtsa.arma_generate_sample(ar=ar, ma=ma, nsample=n)
plotds(ar3_data)
```

```python
# Build AR(1) model
ar1model = smtsa.ARMA(ar1_data.tolist(), order=(1, 0))
ar1=ar1model.fit(maxlag=30, method='mle', trend='nc')
ar1.summary()

```

ARMA Model Results

  Dep. Variable:	         y	          No. Observations:  	    600

  Model:	            ARMA(1, 0)	      Log Likelihood     	 -859.185

  Method:	               mle	         S.D. of innovations	   1.012

  Date:	          Fri, 07 Feb 2020	   AIC                	 1722.370

  Time:	              10:15:02	       BIC                	 1731.164

  Sample:	                0	          HQIC               	 1725.793

     	        coef	     std err	      z	      P>|z|	  [0.025	    0.975]

  ar.L1.y	     0.9206	     0.016	    58.747	  0.000	     0.890	     0.951

Roots

    	               Real	           Imaginary	          Modulus	          Frequency

  AR.1	            1.0863	           +0.0000j	            1.0863	            0.0000

```python
# Build MA(3) model
ar3 = smtsa.ARMA(ar3_data.tolist(), order=(3, 0)).fit(
    maxlag=30, method='mle', trend='nc')
ar3.summary()
```

ARMA Model Results

  Dep. Variable:	         y	          No. Observations:  	    600

  Model:	            ARMA(3, 0)	      Log Likelihood     	 -851.727

  Method:	               mle	         S.D. of innovations	   0.999

  Date:	          Fri, 07 Feb 2020	   AIC                	 1711.453

  Time:	              10:15:03	       BIC                	 1729.041

  Sample:	                0	          HQIC               	 1718.300

     	        coef	     std err	      z	      P>|z|	  [0.025	    0.975]

  ar.L1.y	    -0.5564	     0.036	   -15.556	  0.000	    -0.626	    -0.486

  ar.L2.y	    -0.6723	     0.032	   -20.983	  0.000	    -0.735	    -0.610

  ar.L3.y	    -0.4837	     0.036	   -13.521	  0.000	    -0.554	    -0.414

Roots

    	               Real	           Imaginary	          Modulus	          Frequency

  AR.1	            0.0674	           -1.1624j	            1.1644	           -0.2408

  AR.2	            0.0674	           +1.1624j	            1.1644	            0.2408

  AR.3	           -1.5248	           -0.0000j	            1.5248	           -0.5000

\begin{equation}
X_t = \phi \epsilon_{t-1} + \epsilon_t
\end{equation}

\begin{equation}
X_t = \phi_1 \epsilon_{t-1} + \phi_2 \epsilon_{t-2}  + \epsilon_t
\end{equation}

\begin{equation}
X_t = \phi_1 \epsilon_{t-1} + \phi_2 \epsilon_{t-2}+...  + \phi_p \epsilon_{t-p}+ \epsilon_t
\end{equation}

```python

```


```python

import numpy as np
from matplotlib import pyplot as plt
from jax import numpy as jnp
from jax import grad, vmap, jit, random, lax, ops
from matplotlib.widgets import Slider, Button, RadioButtons
from ipywidgets import interact, widgets
from sympy import *
init_printing()
```

# Problems with exact solutions

### Damped harmonic oscillator

$$\ddot{x} + 2\beta\dot{x} + \omega_0^2 x=0$$

$$x(t) = A(t)\sin(\omega t + \phi)$$

```python
x, t, beta, w, w_0 = symbols("x t beta omega omega_0",real=True)
A = Function("A")(t)
xa = A*sin(w*t)
eq = (diff(diff(xa, t), t) + 2*beta*diff(xa, t) + w_0*xa).expand()
eq.coeff(sin(w*t)), eq.coeff(cos(w*t))
```

$$[\ddot{A} + 2\beta \dot{A} + (\omega_0^2 - \omega^2)A]\sin(\omega t +\phi) = 0$$

$$[2\omega(\dot{A} + \beta A)]\cos(\omega t +\phi) = 0$$

$$A(t) = A_0 e^{-\beta t}$$

$$\omega = \sqrt{\omega_0^2 - \beta^2}$$

```python
def plot(b):
  xaf =  lambda t: jnp.exp(-b*t)*jnp.sin(jnp.sqrt(1-b**2)*t)
  ts = np.linspace(0, 20, 1000)
  plt.figure(figsize=(12, 6))
  plt.plot(ts, xaf(ts))
  plt.plot(ts, np.zeros_like(ts), "black")
  plt.grid()
  plt.xlabel("t", fontsize=14)
  plt.ylabel("$x/A_0$", fontsize=14)

interact(plot, b=widgets.FloatSlider(value=0.5, min=0.0, max=0.9))
```

    interactive(children=(FloatSlider(value=0.5, description='b', max=0.9), Output()), _dom_classes=('widget-inter…

    <function __main__.plot>

## Driven oscillator

$$\ddot{x} + 2\beta\dot{x} + \omega_0^2 x= a_0 \sin(\omega t)$$

$$x(t) = A\sin(\omega t + \phi)$$

$$A = \frac{a_0}{\sqrt{(\omega_0^2 - \omega^2)^2 + 4\beta^2 \omega^2}}$$

$$\tan(\phi) = \frac{2\beta\omega}{\omega^2 - \omega_0^2}$$

```python
 def plot1(w, b):
  print(w)
  w_0 = 1.0
  dfun =lambda w: np.arctan(2*b*w/(w**2-w_0**2))
  Afun = lambda w: 1/jnp.sqrt((w**2-w_0**2)**2 + 4*b**2*w**2)
  xaf =  lambda t: jnp.sin(w*t + dfun(w))*Afun(w)

  ts = np.linspace(0, 40, 1000)
  plt.figure(figsize=(18, 6))
  plt.subplot(131)
  plt.plot(ts, np.sin(w*ts), label="a/a_0")
  plt.plot(ts, xaf(ts), label="x(t)/a_0")
  plt.plot(ts, np.zeros_like(ts), "black")
  plt.grid()
  plt.xlabel("t", fontsize=14)
  plt.legend(fontsize=14)
  plt.subplot(132)
  ws = np.linspace(0.1, 2.0, 200)
  plt.plot(ws, Afun(ws))
  plt.plot([w], Afun(w), "rx")
  plt.xlabel("$\omega$", fontsize=14)
  plt.ylabel("$A$", fontsize=14)
  plt.grid()
  plt.subplot(133)
  ws = np.linspace(0.0, 2.0, 200)
  plt.plot(ws, dfun(ws))
  plt.plot([w], dfun(w), "rx")
  plt.xlabel("$\omega$", fontsize=14)
  plt.ylabel("$\phi$", fontsize=14)
  plt.grid()

interact(plot1, b=widgets.FloatSlider(value=0.25, min=0.01, max=0.5, step=0.01), w=widgets.FloatSlider(value=0.1, min=0.0, max=2.0, step=0.04999))
None
```

    interactive(children=(FloatSlider(value=0.1, description='w', max=2.0, step=0.04999), FloatSlider(value=0.25, …

```python

```

# Linear chain

$$\ddot{u}_i = \omega_0^2(u_{i+1} + u_{i-1} - 2 u_i)$$

Written as matrix equation:

$$\ddot{u} = \mathbf{D} u$$

$$u_i(t) = A(q)e^{jiqa}e^{j\omega t}$$

Eigenvalue problem. Periodic boundary condition: $$A_0 = 1 = A_N = e^{jqaN} \implies q = k\cdot\frac{2 \pi}{a N}$$

$$\omega^2 = 2\omega_0^2|\sin(q a/2)|$$

```python
def plot2():
  plt.figure(figsize=(10,10))
  plt.grid()
  N = 100
  qs = np.arange(-50, 50)*2*np.pi/N
  plt.plot(qs, 2*np.abs(np.sin(qs/2)))
  plt.xlabel(r"$q\cdot a$", fontsize=14)
  plt.ylabel(r"$\frac{\omega}{\omega_0}$", fontsize=14)
plot2()
```

```python

```

## Schrödinger equation in finite square potential well

$$H\psi = E\psi$$

$$\frac{-\hbar^2}{2 m}\psi'' - V(x)=-|E|\psi$$

Three equations for the three regions. $\psi$ and $\psi'$ must match on the boundary. This gives a transcendent equation

```python
def plot3(E):
  plt.figure(figsize=(16, 8))
  plt.subplot(121)
  plt.xlabel('x', fontsize=14)
  xs = np.linspace(-1.0, 1.0, 200)
  V = lambda x: -1.0*(np.abs(x)<0.5)
  plt.plot(xs, V(xs))
  kap = np.sqrt(2*np.abs(E))
  k = np.sqrt(-2*(-1 - np.abs(E)))
  A = 1.0
  D = A*np.exp(-kap*0.5)/np.cos(k*0.5)
  n1, n2 = xs<-0.5, xs>0.5
  plt.plot(xs, A*np.exp(kap*xs)*n1 + D*np.cos(k*xs)*((~n1) & (~n2)) + A*np.exp(-kap*xs)*n2)
  plt.grid()

interact(plot3, E = widgets.FloatSlider(-0.2, min=-3.5, max=-0.0, step=0.01))
```

    interactive(children=(FloatSlider(value=-0.2, description='E', max=-0.0, min=-3.5, step=0.01), Output()), _dom…

    <function __main__.plot3>

## Quantum Harmonic Oscillator

$$V(x) = \frac{1}{2}m\omega^2x^2$$

$$-\frac{\hbar^2}{2m}\frac{\partial^2\psi}{\partial x^2} + \frac{1}{2}m\omega^2x^2\psi = E\psi$$

1. Dimensionless: $$\frac{\partial^2\psi}{\partial \xi^2} = (\xi^2 - k)\psi$$

2. Solve asymptotic behaviour: $\xi \to \infty$

$$\psi = h(\xi)e^{\xi^2/2}$$

3. Equation for the polinomial:

$$\frac{d^2 h}{d \xi^2} - 2 \xi \frac{dh}{d\xi} + (k-1)h = 0$$

4. Write $h$ as series: $h=\sum_{i=0}^\infty a_i\xi^i \implies$ recursion:

$$ a_{i+2} = a_i\frac{2i+1-k}{(i+1)(i+2)}$$

5. For $\psi$ to remain normalizable both the even and the odd recursion must terminate at some i: $$2n + 1 = k = \frac{2E}{\hbar \omega}$$

6. Name the the polynomials resulting from this recursion after Hermite:

$$\psi_n = H_n(\xi)e^{\xi^2/2}$$

## QHI algebraic solution

1. Ladder operators:
$$\hat{a}_\pm=\frac{1}{\sqrt{2 m}}(\hat{p}\pm im\omega\hat{x})$$

2. Their commutator:

$$[\hat{a}_-, \hat{a}_+] = \hbar \omega$$

3. Write Schrödiner-equation with them:

$$\left(\hat{a}_+\hat{a}_- + \frac{\hbar\omega}{2}\right)\psi = E\psi$$

4. They make new solutions:

$$\left(\hat{a}_+\hat{a}_- + \frac{\hbar\omega}{2}\right)\hat{a}_+\psi = \hat{a}_+\left(\hat{a}_-\hat{a}_+ + \frac{\hbar\omega}{2}\right)\psi  = \hat{a}_+\left(\hat{a}_+\hat{a}_- + \hbar\omega+ \frac{\hbar\omega}{2}\right)\psi= \hat{a}_+
(E + \hbar \omega)\psi$$

5. To avoid negative energy solutions:

$$ \hat{a}_-\psi_0 = 0 \implies \psi_0=Ce^{-x^2/2}, \ E_0=\frac{\hbar\omega}{2}$$

6. All other states:

$$\psi_n = (\hat{a}_+)^n\psi_0$$

#Rotator (rigid rotor)

A $\psi(\theta, \phi)$ wavefuction. No potential. Describes a freely rotating object

- angular momentum operator: $\hat{L_i} = \epsilon_{ijk} \hat{x}_j \hat{p}_k$

- commutes with Hamiltonian: $[\hat{L}_i, \hat{H}] = 0 $ -> common eigenstates

- magnitde op: $[\hat{L}, \hat{L^2}] =  0 = [\hat{L}, \hat{H}]$

We look for the common egienstates of $\hat{H}$, $\hat{L}$ and one chosen $\hat{L}_i$ ($\hat{L}_z$). (Or simply write the Laplacian arising from the kinetic term in Sch in spherical coordiantes and call it a day:

$$\Delta = \frac{1}{r^2}\frac{\partial}{\partial r}(r^2\frac{\partial}{\partial r}) - \frac{\hat{L}^2}{\hbar^2r^2}$$)

1. This can be solved by sperataing the variables $\theta$ an $\phi$.

2. The $\phi$-dependent part is: $\Phi(\phi) = (2\pi)^{-1/2}e^{im\phi}$

  - The cyclic boundary condition fixes $m \in \mathbb{Z}$

3. The $\theta$-dep perts are the associaated Legendere polynomials indexed by $l\in\mathbb{N}$. $|m|\leq l$

4. $E\propto l(l+1)\hbar^2$

The solution are the spherical harmonics:

$$\hat{L}_z Y_{lm} = m\hbar Y_{lm}$$

$$\hat{L}^2 Y_{lm} = l(l+1)\hbar^2 Y_{lm}$$

```python
from matplotlib import cm, colors
from mpl_toolkits.mplot3d import Axes3D
from scipy.special import sph_harm

phi = np.linspace(0, np.pi, 100)
theta = np.linspace(0, 2*np.pi, 100)
phi, theta = np.meshgrid(phi, theta)

x = np.sin(phi) * np.cos(theta)
y = np.sin(phi) * np.sin(theta)
z = np.cos(phi)

def plot3(l, m):
  fcolors = sph_harm(m, l, theta, phi).real
  fmax, fmin = fcolors.max(), fcolors.min()
  fcolors = (fcolors - fmin)/(fmax - fmin)

  fig = plt.figure(figsize=plt.figaspect(1.))
  ax = fig.add_subplot(111, projection='3d')
  ax.plot_surface(x, y, z,  rstride=1, cstride=1, facecolors=cm.seismic(fcolors))

  #ax.set_axis_off()
  plt.show()

interact(plot3, l=widgets.IntSlider(1, 0, 5), m=widgets.IntSlider(0, 0, 5))
None
```

    interactive(children=(IntSlider(value=1, description='l', max=5), IntSlider(value=0, description='m', max=5), …

```python

```


# Stastistical Learning Theory and Excess Risk Decomposition

Sreyas Mohan, CDS

```python
import numpy as np
import matplotlib.pyplot as plt
from functools import partial
from sklearn import linear_model
from scipy.stats import norm
import matplotlib.mlab as mlab
%matplotlib inline
```

#### Generic Utility Functions

```python
def plot_prediction_function(f_list = None, label_list = None, data = None, alpha = 0.9, include_data_x = False):

    plt.figure(figsize=(20,10))

    if data is not None:
        x_min, x_max = np.min(data[:, 0]), np.max(data[:, 0])
    else:
        x_min, x_max = -5.0, 5.0;

    if ( (f_list is not None) and (label_list is not None) ):
        x_array = np.arange(x_min, x_max, 0.1);
        if include_data_x:
            x_array = np.concatenate([x_array, data[:, 0], data[:, 0]+0.001, data[:, 0]-0.001])
            x_array = np.sort(x_array)

        for f, label in zip(f_list, label_list):
            f_y = f(x_array);
            plt.plot(x_array, f_y, label=label);

    if data is not None:
        plt.scatter(data[:, 0], data[:, 1], alpha = alpha )

    if label_list is not None:
        plt.legend()
    plt.show()
```

## Generative Model for Data

We will work on a regression problem in this lab. The input, action and output space will be $\mathcal{R}$.  The output $y$ is related to the input $x$ as $ y = g(x) = ax^2 + bx + c $ where $a,b$ and $c$ are sampled from random variables which follows gaussian distribution with parameters $\mu_a, \sigma_a$,  $\mu_b, \sigma_b$ and  $\mu_c, \sigma_c$ respectively. (Basically, given $x$,  $y$ depends on $3$ random numbers). Assume that $X$ is sampled from $\mathcal{N}(\mu_x, \sigma_x)$. For the purposes of this lab, let's set $\mu_a = 1, \mu_b = 2, \mu_c = 3, \mu_x = 0$ and $\sigma_a = \sigma_b = \sigma_c = \sigma_x = 1$.

#### Utility functions for sampling

```python
mu_a = 1;
sigma_a = 1;

mu_b = 2;
sigma_b = 1;

mu_c = 3;
sigma_c = 1;

mu_x = 0;
sigma_x = 1;

assert( (mu_x == 0) and (sigma_x == 1))
```

```python
## assumes g is a polynomial. Takes in a coeff list, where coeff_list[i] is the coefficient of x^i and x to evaluate it at

def template_g(coeff_list, x):
    ans = 0;
    for i, coeff in enumerate(coeff_list):
        ans += coeff * (x**i)
    return ans
```

```python
### generates one sample function g

def sample_g():
    a = np.random.randn() * sigma_a + mu_a;
    b = np.random.randn() * sigma_b + mu_b;
    c = np.random.randn() * sigma_c + mu_c;

    return partial(template_g, [c, b, a] )

```

```python
## give one (x,y) sample

def get_one_x_y_sample():
    x = np.random.randn()*sigma_x + mu_x;
    g = sample_g()
    return x, g(x)
```

```python
## gives a matrix with first column x and second column y

def generate_n_samples(n = 1000):
    matrix = np.zeros([n, 2]);
    for i in range(n):
        matrix[i] = get_one_x_y_sample();
    return matrix
```

### Visualizing samples from $\mathcal{P}_{X \times Y}$

```python
data_cloud =  generate_n_samples(1000)
plot_prediction_function(data = data_cloud)
```

### Visualizing samples of $Y$ for fixed values of $X$

```python
def y_for_fixed_x( x_list = np.arange(-4, 4.5, 0.5), n_sample_per_x = 10):
    data = np.zeros([len(x_list)*n_sample_per_x, 2]);
    for i, x in enumerate(x_list):
        for j in range(n_sample_per_x):
            g = sample_g();
            data[n_sample_per_x*i + j, 0] = x;
            data[n_sample_per_x*i + j, 1] = g(x);
    return data
```

```python
plot_prediction_function(data = y_for_fixed_x(n_sample_per_x=50) )
```

## Exercise:

Recall that the risk of a function $f$ wrt to loss $l$, $R(f) = E[l((f(x), y)]$ and the bayes optimal function $f^* = \underset{f}{\operatorname{argmin}}R(f)$

(Hint: for $X  \sim \mathcal{N}(0, 1)$,  $E[X^4] = 3$)

1. If we minimize l2 loss, what is the bayes optimal function $f^*(x)$ for the model described above? Does your answer depend on the distribution assumed on $X$?
2. What is the risk assosciated with $f^*(x)$, $R(f)$?
3. Once you have mathematical expressions for 1 and 2, fill in the python functions below for the bayes prediction function and bayes risk.

## Answers:
Remember that $R(f) = E[(f(x) - E[y|x])^2 ] + E[(y - E[y|x])^2] $
1. For l2 loss, $f^*(x) = E[Y|X=x] = \mu_a x^2 + \mu_b x + \mu_c $. No, independent of distribution on $X$.
2. From the decomposition of l2 loss above,
$$ \begin{align}
        R(f^*(x)) &= E[(y - E[y|x])^2] \\
                  &= E_x[ E_{y|x} [ (y - E[y|x])^2 ] ] \\
                  &= E_x[ \sigma_a^2 x^4 + \sigma_b^2 x^2 + \sigma_c^2 ] \\
                  &= \sigma_a^2 (3) + \sigma_b^2 ( \sigma_x^2 + \mu_x^2) + \sigma_c^2
   \end{align} $$

```python
## variable f_star assiged to bayes optimal function
## bayes risk with numerical value of bayes risk
f_star = partial(template_g, [mu_c, mu_b, mu_a] )
bayes_risk = sigma_a**2 * 3  + sigma_b**2 * ( sigma_x**2 + mu_x**2) + sigma_c**2;
print(bayes_risk)
```

    5

```python
plot_prediction_function([f_star], ['$f^*$'], data=data_cloud)
```

```python
plot_prediction_function([f_star], ['$f^*$'], data=y_for_fixed_x(n_sample_per_x=100) )
```

## Estimating the Risk $R(f)$ using monte-carlo

```python
## For Students to Fill in
## use get_one_x_y_sample()

def estimate_risk(f, n_try = int(1e5) ):
    sum = 0;
    for i in range(n_try):
        x, y = get_one_x_y_sample()
        sum += (f(x) - y)**2
    return sum/n_try
```

```python
estimate_risk(f_star)
```

    4.946726925737005

# Empirical Risk

We never usually know $\mathcal{P}_{X, Y}$ and we work with finite samples drawn from the distribution. With $\mathcal{D}_n = ( (x_1, y_1), (x_2, y_2), \dots , (x_n, y_n) )$ be $n$ iid data points, the empirical risk of $f$ with respect to loss $l$ on dataset $\mathcal{D}_n$ is defined as $$ \hat{R}_n(f) = \frac{1}{n} \sum_{i=1}^{n} l( f(x_i), y_i) $$

Have we used the expression of $ \hat{R}_n(f) $ for anything till now?

Answer:
We estimated risk in monte carlo simulation using the same expression.

### Exercise:

1. Is $\hat{R}_n(f)$ or $R(f)$ a random variable? If so, what is the mean of the random variable and what is it's distribution?
2. Can $R(f) \geq  \hat{R}_n(f)$?

Answers:

1. $\hat{R}_n(f)$ is random variable with the mean $R(f)$. Approximately gaussian using CLT.
2. Yes

```python
def empirical_risk(f, sample_matrix):
    fy_array = f(sample_matrix[:, 0]);
    risk = np.mean((fy_array - sample_matrix[:, 1]) ** 2)
    return risk
```

#### Checking the distribution of $\hat{R}_n(f)$

```python
n = 1000;
emp_rsk = np.zeros(n)
for i in range(n):
    emp_rsk[i] = empirical_risk(f_star, generate_n_samples(5000))

print( np.mean(emp_rsk) )
```

    5.024611956635761

```python
datos = emp_rsk
plt.figure(figsize=(20,10))
# best fit of data
(mu, sigma) = norm.fit(datos)

# the histogram of the data
n, bins, patches = plt.hist(datos, 75, density=True, facecolor='green', alpha=0.75)

# add a 'best fit' line
y = mlab.normpdf( bins, mu, sigma)
l = plt.plot(bins, y, 'r--', linewidth=2)

#plot
plt.xlabel('Smarts')
plt.ylabel('Probability')
plt.title(r'$\mathrm{Histogram\ of\ \hat{R}(f^*):}\ \mu=%.3f,\ \sigma=%.3f$' %(mu, sigma))
plt.grid(True)

plt.show()

```

## Empirical Risk Minimization

Let's fix a training data with $n = 100$ points

```python
data = generate_n_samples(n = 100);
```

### Visualizing Data

```python
plot_prediction_function([f_star], ['$f^*$'], data)
```

### Exercise
One way to achieve $\hat{R}_n(f) = 0 $ is to memorize the data. For the sake of simplicity let's assume that the $\mathcal{D}_n$ we are working with has no duplicate values for $x$. The function $f(x)$ returns the corresponding $y$ from $\mathcal{D}_n$ otherwise $f(x)$ returns $0$. What is the risk, $R(f)$, for this function?

Answer:

$$ \begin{align}
        R(f) &= E[ (Y - 0) ^2 ] \\
             &= E_{X} E_{Y|X=x} E[ Y^2 ] \\
             &= E_{X} ( var(Y|X=x) + E[Y|X=x]^2 ) \\
             &= E_{X} ( \sigma_a^2 X^4 + \sigma_b^2 X^2 + \sigma_c^2 + (\mu_a X^2 + \mu_b X + \mu_c )^2 ) \\
             &= E_{X} ( (\sigma_a^2 + \mu_a^2) X^4 + (\alpha)X^3 + (\sigma_b^2 + \mu_b^2 + 2 \mu_c \mu_a ) X^2 + \beta X + \sigma_c^2 + \mu_c^2 \\
             &= 3(\sigma_a^2 + \mu_a^2) + (\sigma_b^2 + \mu_b^2 + 2 \mu_c \mu_a ) + \sigma_c^2 + \mu_c^2
   \end{align} $$

```python
risk_memorized_function = 3*(sigma_a**2 + mu_a**2) + (sigma_b**2 + mu_b**2 + 2*mu_c*mu_a) + sigma_c**2 + mu_c**2
print(risk_memorized_function)
```

    27

### Estimating the risk of memorized function numerically

```python
def f_memorized(x_array):
    scalar = False
    if not isinstance(x_array,(list,np.ndarray)):
        scalar = True
        x_array = np.array([x_array]);

    result = np.zeros_like(x_array).astype(float)
    for i, x in enumerate(x_array):
        found = False
        for x_i, y_i in data:
            if x==x_i:
                result[i] = y_i;
                found = True
        if not found:
            result[i] = 0.0

    if scalar:
        return result[0]
    return result
```

```python
linear_f_hat_risk = estimate_risk( f_memorized, n_try=int(1e3) )
print('Risk: ', linear_f_hat_risk)
linear_f_hat_empirical_risk = empirical_risk(f_memorized, data)
print('Empirical Risk: ', linear_f_hat_empirical_risk)
```

    Risk:  27.339779203876198
    Empirical Risk:  0.0

```python
plot_prediction_function([f_star, f_memorized], ['$f^*$', 'memorized'], data, include_data_x=True)
```

### Constrained Risk Minimization

Like we saw in the previous section, (unconstrained) empirical risk minimization is (almost) useless in practise. We achieved perfect $0$ for empirical risk but we have a increase in the actual risk, which is the quantity we care about.

#### Linear (Affine) Hypothesis Space

## Exercise
For $f(x) = \alpha x + \beta$
1. Calculate an expression for risk of $f(x)$
2. Find $\alpha^*, \beta^*$ which minimises $R(f)$.

Answer:
1.

$$ \begin{align}
        R(f) &= E[ (Y - \alpha X - \beta )^2 ] \\
             &= E_{X} E_{Y|X=x} E[ (Y - \alpha x - \beta )^2 ] \\
             &= E_{X} ( var( (Y - \alpha x - \beta )|X=x) + E[(Y - \alpha x - \beta )|X=x]^2 ) \\
             &= E_{X} ( \sigma_a^2 X^4 + \sigma_b^2 X^2 + \sigma_c^2 + (\mu_a X^2 + (\mu_b - \alpha) X + (\mu_c - \beta) )^2 ) \\
             &= E_{X} ( (\sigma_a^2 + \mu_a^2) X^4 + (something)X^3 + (\sigma_b^2 + (\mu_b - \alpha)^2 + 2 (\mu_c - \beta) \mu_a ) X^2 + (something) X + \sigma_c^2 + (\mu_c - \beta)^2 \\
             &= 3(\sigma_a^2 + \mu_a^2) + (\sigma_b^2 + (\mu_b - \alpha)^2 + 2 (\mu_c - \beta) \mu_a ) + \sigma_c^2 + (\mu_c - \beta)^2
     \end{align} $$

 2.

 To find $\alpha^*, \beta^*$
 $$  \frac{d}{d \alpha}R(f) = 2(\alpha - \mu_b)  = 0 $$
 Therefore, $\alpha^* = \mu_b$
 $$  \frac{d}{d \beta}R(f) = -2 \mu_a + 2(\beta - \mu_c) = 0 $$
 $\beta^* = \mu_a + \mu_c$

```python
### risk for linear function (alphax + beta)
## students fill in

def linear_function_risk(alpha, beta):
    return 3*(sigma_a**2 + mu_a**2) + sigma_b**2 + (mu_b-alpha)**2 + 2*(mu_c - beta)*mu_a + sigma_c**2 + (mu_c - beta)**2;

```

```python
### Estimating Risk
linear_f_star = partial(template_g, [mu_a + mu_c, mu_b]);

mc_linear_f_star_risk = estimate_risk( linear_f_star)
print('MC Risk: ', mc_linear_f_star_risk)

linear_f_star_risk = linear_function_risk(mu_b, mu_a + mu_c)
print('Risk :' , linear_f_star_risk)
```

    MC Risk:  7.067073491179056
    Risk : 7

### Visualizing Prediction Function $\ \ f^*_\mathcal{H}$

```python
plot_prediction_function([linear_f_star, f_star], ['$f^*_\mathcal{H}$', '$f^*$'], data, alpha = 1)
```

### Constrained Empirical Risk Minimization

#### $\hat{\alpha}, \hat{\beta}$ which minimises $\hat{R}(f)$ and estimate the risk using monte carlo method.

```python
## Fitting Linear Regression on Data
reg = linear_model.LinearRegression(fit_intercept=True).fit(data[:, 0:1], data[:, 1])
```

```python
reg.intercept_, reg.coef_
```

    (3.9819081006768657, array([1.81626746]))

```python
linear_f_hat = partial(template_g, [reg.intercept_, reg.coef_[0]])
```

```python
linear_f_hat_risk = linear_function_risk(reg.coef_[0], reg.intercept_)
print('Risk :' , linear_f_hat_risk)
mc_linear_f_hat_risk = estimate_risk( linear_f_hat )
print('MC Risk: ', mc_linear_f_hat_risk)
linear_f_hat_empirical_risk = empirical_risk(linear_f_hat, data)
print('Empirical Risk: ', linear_f_hat_empirical_risk)
```

    Risk : 7.034084964233141
    MC Risk:  6.8096723870970095
    Empirical Risk:  5.294274858620543

### Visualizing Prediction Function $\ \ \hat{f}_\mathcal{H}$

```python
plot_prediction_function([linear_f_star, linear_f_hat, f_star], ['$f^*_\mathcal{H}$', '$\hat{f}_\mathcal{H}$', '$f^*$'], data, alpha = 1)
```

## Error Decomposition

Recall that:

Approximation Error for $\mathcal{H}$ = $R(f^*_{\mathcal{H}}) - R(f^*) $

Estimation Error of $\ \hat{f}_\mathcal{H}$ = $R(\hat{f}_{\mathcal{H}}) - R(f_{\mathcal{H}})$

Excess Risk of f = $R(f) - R(f^*)$

### Exercise:
1. From the values we calculated above, what is the approximation error for linear hypothesis space? Is approximation error a random variable?
2. What is the estimation error of $\ \hat{f}_\mathcal{H}$? Is estimation error a random variable?
3. What is the excess risk of  $\ \hat{f}_\mathcal{H}$? Is this a random variable?

#### Answers:

1. Approximation Error is not random.
2. Estimation Error is random.
3. Excess risk of $\ \hat{f}_\mathcal{H}$ is random.

### Estimation Error and Excess risk of $\hat{f}_\mathcal{H}$

```python
def estimation_error_and_excess_risk(data, risk_fh, risk_f_star):
    reg = linear_model.LinearRegression(fit_intercept=True).fit(data[:, 0:1], data[:, 1]);
    linear_f_hat_risk = linear_function_risk(reg.coef_[0], reg.intercept_)

    return linear_f_hat_risk - risk_fh, linear_f_hat_risk - risk_f_star
```

```python
n_try = 1000;
estimation_error_array = np.zeros(n_try);
excess_risk_array = np.zeros(n_try)
for i in range(n_try):
    estimation_error_array[i], excess_risk_array[i] = estimation_error_and_excess_risk( generate_n_samples(n = 100), linear_f_star_risk, bayes_risk)
```

```python
_ = plt.hist(estimation_error_array, bins = 100)
```

```python
_ = plt.hist(excess_risk_array, bins = 100)
```

### Optimization Error

Since we were optimizing for L2 loss over a linear hypothesis space, we found the best possible $\hat{f}_\mathcal{H}$ (upto numerical error) using the closed form expression for linear regression. What if we use SGD instead to find $f$ and stop iterations prematurely?

```python
reg_sgd = linear_model.SGDRegressor(max_iter=3,
                                    fit_intercept = True,
                                    penalty = 'none', #No Regularization
                                    tol = None)
```

```python
reg_sgd.fit(data[:, 0:1], data[:, 1])
```

    SGDRegressor(alpha=0.0001, average=False, early_stopping=False, epsilon=0.1,
           eta0=0.01, fit_intercept=True, l1_ratio=0.15,
           learning_rate='invscaling', loss='squared_loss', max_iter=3,
           n_iter=None, n_iter_no_change=5, penalty='none', power_t=0.25,
           random_state=None, shuffle=True, tol=None, validation_fraction=0.1,
           verbose=0, warm_start=False)

```python
reg_sgd.intercept_, reg_sgd.coef_
```

    (array([2.44107929]), array([1.06487941]))

```python
linear_f_tilde = partial(template_g, [reg_sgd.intercept_[0], reg_sgd.coef_[0]])
```

```python
linear_f_tilde_risk = linear_function_risk(reg_sgd.coef_[0], reg_sgd.intercept_[0])
print('Risk :' , linear_f_tilde_risk)
mc_linear_f_tilde_risk = estimate_risk( linear_f_tilde )
print('MC Risk: ', linear_f_tilde_risk)
linear_f_tilde_empirical_risk = empirical_risk(linear_f_tilde, data)
print('Empirical Risk: ', linear_f_tilde_empirical_risk)
```

    Risk : 10.304684305947845
    MC Risk:  10.304684305947845
    Empirical Risk:  8.211548790813369

### Visualizing $\ \ \ \tilde{f_\mathcal{H} }$

```python
plot_prediction_function([linear_f_star, linear_f_hat, linear_f_tilde, f_star],
                         ['$f^*_\mathcal{H}$', '$\hat{f}_\mathcal{H}$', '$\tilde{f_\mathcal{H}}$', '$f^*$'],
                         data, alpha = 1)
```

### Exercise:
What is the optimization error? How do you expect it to change with the number of iterations of gradient descent?

Answer: As number if iterations increase, optimization error will reduce.

# Additional Questions

1. Suppose we use another hypothesis space $\mathcal{H}' \subset \mathcal{H}$, how do you expect the approximation error to change?
2. Suppose we increase the number of data points $n$ in the sample to calculate $\ \hat{f}_\mathcal{H}$. How do we expect estimation error to change?
2. Repeat everything we did for linear hypothesis space for constant, quadratic and cubic hypothesis space for $n=500, 1000$ as well - does your results match with what you expected in 1 and 2?


# Hermitian matrices (via sympy)

Recall that for a $N\times N$ Hermitian matrix, we have the properties

\begin{aligned}
A &= A^{\dag} \text{ or } \\
a_{ij} &= a_{ji}^* \end{aligned}

So for $i,j \in \lbrace 1, 2 , \dots N \rbrace$, $N^2$ entries, $\frac{ N(N-1) }{2}$ off diagonal complex entries, $N$  *real*, diagonal entries.

Given upper triangular entries $a_{ij}$, $i<j$,
Consider $\lbrace (i,j) | i< j \text{ and } i,j \in \lbrace 1 , 2 \dots N \rbrace \rbrace \to \lbrace k | k \in \lbrace 1 , 2 , \dots , \frac{N (N-1) }{2} \rbrace \rbrace$

\begin{aligned}
\begin{matrix}
12 & 13 \\
 & 23 \end{matrix}  &
\begin{matrix}
1 & 2 \\
 & 3 \end{matrix}
\end{matrix}
\end{aligned}

$\begin{aligned}
\begin{matrix}
12 & 13 & 14 \\
& 23 & 24 \\
& & 34 \end{matrix} \qquad \, & \qquad \, \begin{matrix}
1 & 2 & 3 \\
& 4 & 5 \\
& & 6 \end{matrix} \end{aligned}$

$\begin{aligned}
\begin{matrix}
12 & 13 & 14 & 15 \\
& 23 & 24 & 25 \\
& & 34 & 35 \\
& & & 45 \end{matrix} \qquad \, & \qquad \, \begin{matrix}
1 & 2 & 3 & 4 \\
& 5 & 6 & 7 \\
& & 8 & 9 \\ & & & 10 \end{matrix} \end{aligned}$

$\forall \, i \lbrace 1, 2, \dots N - 1 \rbrace$,
consider
\begin{gathered}
N-1 + N - 2 + \dots + N - (i-1) = (i-1)N + - \sum_{k=1}^{i-1} k = (i-1)N - \frac{ (i-1)i }{2} = \frac{ (i-1)(2N-i) }{2}
\end{gathered}

and so

\begin{equation}
\begin{gathered} \lbrace (i,j) | i< j \text{ and } i,j \in \lbrace 1, 2, \dots N \rbrace \rbrace \to \lbrace k | k \in \lbrace 1,  2, \dots , \frac{N(N-1)}{2} \rbrace \rbrace \\
    (i,j) \mapsto k = \frac{ (i-1)(2N-i) }{2} + j -i
\end{gathered}
\end{equation}

```python
import sympy
```

```python
from sympy import diag, Matrix, Symbol, symbols, exp, I, integrate, oo, latex, pretty_print, sqrt, pi
```

```python
from sympy.abc import x
```

```python
from sympy import Rational as Rat
```

```python
N = 4; print N*(N-1)/2
```

    6

```python
entriesz = symbols("z1:"+str(N*(N-1)/2 + 1), complex=True)
```

```python
entriesz[0]
```

    z1

```python
len(entriesz)
```

    6

```python
realentriesd = symbols("x1:"+str(N+1), real=True)
```

```python
entriesz[0].conjugate()
```

    conjugate(z0)

```python
range(1,N+1)
```

    [1, 2, 3, 4]

```python
[entriesz[k].conjugate() for k in range(3,1,-1)]
```

    [conjugate(z4), conjugate(z3)]

```python
entriesz
```

    (z1, z2, z3, z4, z5, z6)

```python
entriesz_list = list( entriesz)
uppertrimat_H = []
for i in range(1,N):
    row = []
    for j in range(1,i+1):
        row.append( Rat(0) )
    for j in range(i+1,N+1):
        row.append( entriesz_list.pop(0))
    uppertrimat_H.append(row)

uppertrimat_H.append( [ Rat(0) for j in range(1,N+1)])
```

```python
uppertrimat_H
```

    [[0, z1, z2, z3], [0, 0, z4, z5], [0, 0, 0, z6], [0, 0, 0, 0]]

```python
uppertrimat_H = Matrix( uppertrimat_H)
```

```python
uppertrimat_H
```

    Matrix([
    [0, z1, z2, z3],
    [0,  0, z4, z5],
    [0,  0,  0, z6],
    [0,  0,  0,  0]])

```python
uppertrimat_H.T
```

    Matrix([
    [ 0,  0,  0, 0],
    [z1,  0,  0, 0],
    [z2, z4,  0, 0],
    [z3, z5, z6, 0]])

```python
entriesz_list_conj = [ z.conjugate() for z in entriesz]
lowertrimat_H = []
for i in range(1,N):
    row = []
    for j in range(1,i+1):
        row.append( Rat(0) )
    for j in range(i+1,N+1):
        row.append( entriesz_list_conj.pop(0))
    lowertrimat_H.append(row)

lowertrimat_H.append( [ Rat(0) for j in range(1,N+1)])
```

```python
lowertrimat_H
```

    [[0, conjugate(z1), conjugate(z2), conjugate(z3)],
     [0, 0, conjugate(z4), conjugate(z5)],
     [0, 0, 0, conjugate(z6)],
     [0, 0, 0, 0]]

```python
lowertrimat_H = Matrix(lowertrimat_H).T
```

```python
lowertrimat_H
```

    Matrix([
    [            0,             0,             0, 0],
    [conjugate(z1),             0,             0, 0],
    [conjugate(z2), conjugate(z4),             0, 0],
    [conjugate(z3), conjugate(z5), conjugate(z6), 0]])

```python
diag( *realentriesd )
```

    Matrix([
    [x1,  0,  0,  0],
    [ 0, x2,  0,  0],
    [ 0,  0, x3,  0],
    [ 0,  0,  0, x4]])

```python
HermitianMatrix = diag( *realentriesd ) + uppertrimat_H + lowertrimat_H
```

```python
HermitianMatrix
```

    Matrix([
    [           x1,            z1,            z2, z3],
    [conjugate(z1),            x2,            z4, z5],
    [conjugate(z2), conjugate(z4),            x3, z6],
    [conjugate(z3), conjugate(z5), conjugate(z6), x4]])

```python
def HermitianMatrixCreator( N, MatName="" ):
    """
    HermitianMatrixCreator( N )
    Creates and returns a NxN Hermitian matrix;
    with the optional argument, you can add further notation to each entry to denote which matrix it belongs to
    """
    if MatName == "":
        entriesz        = symbols("z1:"+str(N*(N-1)/2 + 1), complex=True)
        realentriesdiag = symbols("x1:"+str(N+1), real=True)
    else:
        entriesz        = symbols(MatName+"z1:"+str(N*(N-1)/2 + 1), complex=True)
        realentriesdiag = symbols(MatName+"x1:"+str(N+1), real=True)

    entriesz_list = list( entriesz)
    uppertrimat_H = []
    for i in range(1,N):
        row = []
        for j in range(1,i+1):
            row.append( Rat(0) )
        for j in range(i+1,N+1):
            row.append( entriesz_list.pop(0))
        uppertrimat_H.append(row)

    uppertrimat_H.append( [ Rat(0) for j in range(1,N+1)])
    uppertrimat_H = Matrix( uppertrimat_H)

    entriesz_list_conj = [ z.conjugate() for z in entriesz]
    lowertrimat_H = []
    for i in range(1,N):
        row = []
        for j in range(1,i+1):
            row.append( Rat(0) )
        for j in range(i+1,N+1):
            row.append( entriesz_list_conj.pop(0))
        lowertrimat_H.append(row)

    lowertrimat_H.append( [ Rat(0) for j in range(1,N+1)])
    lowertrimat_H = Matrix(lowertrimat_H).T

    HermitianMatrix = diag( *realentriesdiag ) + uppertrimat_H + lowertrimat_H
    return HermitianMatrix;
```

```python
HermitianMatrixCreator(5)
```

    Matrix([
    [           x1,            z1,            z2,             z3,  z4],
    [conjugate(z1),            x2,            z5,             z6,  z7],
    [conjugate(z2), conjugate(z5),            x3,             z8,  z9],
    [conjugate(z3), conjugate(z6), conjugate(z8),             x4, z10],
    [conjugate(z4), conjugate(z7), conjugate(z9), conjugate(z10),  x5]])

```python
A = HermitianMatrixCreator(5,"A"); A
```

    Matrix([
    [           Ax1,            Az1,            Az2,             Az3,  Az4],
    [conjugate(Az1),            Ax2,            Az5,             Az6,  Az7],
    [conjugate(Az2), conjugate(Az5),            Ax3,             Az8,  Az9],
    [conjugate(Az3), conjugate(Az6), conjugate(Az8),             Ax4, Az10],
    [conjugate(Az4), conjugate(Az7), conjugate(Az9), conjugate(Az10),  Ax5]])

```python
A.trace()
```

    Ax1 + Ax2 + Ax3 + Ax4 + Ax5

For

$
\langle x^{2n} \rangle = \frac{1}{ \sqrt{2 \pi } } \int_{-\infty}^{\infty} \exp{ - \frac{x^2}{2}} x^{2n} dx = (2n-1)!! = \frac{(2n)!}{2^n n!}
$

```python
n = Symbol("n", integer=True, positive=True)
```

```python
pretty_print( integrate(exp(-x**2/2)*x**2, (x, -oo, oo) ) )
```

    √2⋅√π

$ n = 2 $

```python
pretty_print( integrate(Rat(1)/sqrt( Rat(2) * pi ) * exp(-x**2/2)*x**(2*n), (x, -oo, oo) ) )
```

        n - 1/2
    √2⋅2       ⋅Γ(n + 1/2)
    ──────────────────────
              √π

```python

```


# A Model of Prevalence for COVID-19

**This worksheet presents hypothetical mathematical models of COVID -- and it is too early to draw conclusions about which represents the current epidemic. In all cases you should follow the advice of public health authorities to stay at home (for everyone who can) and self isolate if you have any symptoms. For authoritative advice see: https://www.nhs.uk/conditions/coronavirus-covid-19/**

Author

    George Danezis
    University College London
    Twitter: @gdanezis
    Web: http://www0.cs.ucl.ac.uk/staff/G.Danezis/

Code and Data, as well as Jupyter notebook available here:
https://github.com/gdanezis/COVID-Prevalence

I use here the time series of reported outcomes from COVID-19, namely recoveries and deaths in different countries, to estimate the prevalence of the virus, as well as to project its growth. Under most scenarios a significant fraction of the population will be infected in the next 4-6 weeks, unless the latest public heath measures lower its growth. However, there are scenarios that explain the current apparent high association with fatalities, through association rather than causation. So it is possible that COVID-19 is highly infectious (particularly within hospitals) but does not cause significant fatalities (CFR < 0.1%). Of course other scenarios with a CFR of about 1% (high) are also possible.

```python
import sys
from analytical import *
```

# The model

We are trying to infer 4 key variables
* Prevalence $p$.
* Testing rate $f$ of mild or asympromatic carriers.
* Total infected population $R_p$.

We are given as data:
* The number of deaths $D$ that tested positive with COVID.
* The number of recoveries $R$ that tested positive with COVID.
* The total population $P$.

We have to make assumptions about:
* The Case Fatality Rate (CFR) due to COVID $\text{CFR}_c$.
* The CFR due to other reasons with similar symptoms as COVID $\text{CFR}_o$. We set it to $0.01/100$ (Annual mortality of 1% * 1/12 months * ~1/10 serious conditions mau be confused.)
* The increase in risk to be infected with COVID if in a serious condition $\mu$. We usually set $\mu=10$, namely being in a serious medical condition leading to death (for other reasons) also exposes a patient to the equavalent of x10 to catch COVID -- due to hospital infection rates.

The system of equations we need to solve are:
* The hospital prevalence $p_h$:
\begin{equation}
h_p = 1 - (1 - p)^{\mu}
\end{equation}
* The actual deaths due to COVID. All deaths minus the ones due to other causes that still tested positive for COVID:
\begin{equation}
D_a = \max{(D - p_h \cdot \text{CFR}_o \cdot P, 1)}
\end{equation}
* The total infections is the population is actual infected and deaths due to Covid
\begin{equation}
p = (R_p + D) / P
\end{equation}
* The observed recoveries are a fraction of the actual one based on testing rates.
\begin{equation}
R = f \cdot R_p
\end{equation}
* The definition of the Case Fatality Rate is, the number of deaths due to the virus, devided by the number of infections (sum of all dead with virus, and those recovered).
\begin{equation}
\text{CFR}_c = \frac{D_a}{D + R_p}
\end{equation}

This system of equaltions is non-linear but we can solve it numericaly using iterative methods.

### Limitations

* The Testing rate $f$ assumes totaly random testing in the population that is not a fatality. However most countries do use some symptoms or at least self-selection as a gate for testing. Therefore a lower rate of testing can justify the observed recovered cases and the reported rate of testing may be a factor of 5-10 lower than the one reported here. (depending of how well symptoms guide testing).
* We assume all Deaths either caused or associated with COVID are tested, and reported in $D$. However, it is not clear that health authorities are testing dead people, and many cases resulting in fatalities may not have been reported.
* The Case Fatality Rate $\text{CFR}_c$ measure the fatalities **caused** by COVID-19, rather than the ones merely associated with COVID. The raw data about recoveries and deaths can only be used directly to estimate the latter (association) since it is not clear whether a fatality is due to COVID or something else (but the patient also tested positive). As a result the CFR we estimate can be much lower than other studies, since a lot of deaths may simply be due to other causes (evidenced by high comorbidity, and potentially already high prevalence in some places).

# Results for different $\text{CFR}_c$

### Discussion for CFR=1%

A CFR of around 1%-2% is an estimate that was feared early on from experiences in China and elsewhere. However, given this CFR the prevalence in South Korea is so small that the resting rate should be close to 31%. In face we know that about 1-in-170 people have been tested there (huge, but not 31%), weakening the evidence for such a CFR. Other testing rates also seem an order of magniture off. Comorbidity figures are much lower than those reported from Italy.

```python
# Since we measure prevalence based on outcomes, the figures lag by about 20 days.
CFR_covid = 0.01 # CFR medium high: 1%
hospital_infection_mult = 5.0
make_table(populations, CFR_covid, hospital_infection_mult, flx=sys.stdout)
```

    Assumptions: COVID CFR:  1.00% In Hospital factor: x5.0
    Country           Prev      CFR  Testing   Comorb.    Infected
    --------------------------------------------------------------
    Japan            0.00%    1.00%    5.94%    4.76%        3,958
    USA              0.02%    1.00%    0.61%    4.76%       48,721
    Germany          0.01%    1.00%    3.91%    4.76%       11,591
    Italy            0.96%    1.00%    1.30%    4.68%      573,208
    Spain            0.45%    1.00%    1.61%    4.72%      208,073
    Belgium          0.07%    1.00%    4.84%    4.76%        8,293
    Switzerland      0.13%    1.00%    1.18%    4.75%       11,121
    Iran             0.21%    1.00%    4.90%    4.74%      170,794
    Korea, South     0.02%    1.00%   30.27%    4.76%       10,460
    United Kingdom   0.05%    1.00%    0.43%    4.76%       31,571
    Netherlands      0.12%    1.00%    0.01%    4.75%       20,074
    France           0.12%    1.00%    2.71%    4.75%       81,054

### Discussion for CFR=0.1%

A CFR of 0.1% is on the low side, and lower than one estimated by most studies. In fact it would put COVID-19 on par with seasonal viruses in terms of fatality rate. Such a CFR would require Italy and Spain to have had a single digit percentage of their populations infected in early March, which means that by now (end of March) about 50% of the population must have had COVID-19 (if the increase is at a similar rate, see projection section below). The testing rate for South Korea, and others, is still too large (1-in-20 rather than 1-in-170).

```python
# Since we measure prevalence based on outcomes, the figures lag by about 20 days.
CFR_covid = 0.001 # CFR low: 0.1%
hospital_infection_mult = 5.0
make_table(populations, CFR_covid, hospital_infection_mult, flx=sys.stdout)
```

    Assumptions: COVID CFR:  0.10% In Hospital factor: x5.0
    Country           Prev      CFR  Testing   Comorb.    Infected
    --------------------------------------------------------------
    Japan            0.02%    0.10%    0.84%   33.32%       27,962
    USA              0.11%    0.10%    0.09%   33.29%      344,391
    Germany          0.10%    0.10%    0.55%   33.29%       81,931
    Italy            7.00%    0.10%    0.18%   30.30%    4,229,650
    Spain            3.22%    0.10%    0.22%   31.92%    1,500,361
    Belgium          0.52%    0.10%    0.68%   33.10%       58,780
    Switzerland      0.92%    0.10%    0.17%   32.92%       79,031
    Iran             1.50%    0.10%    0.69%   32.67%    1,218,232
    Korea, South     0.14%    0.10%    4.28%   33.27%       73,959
    United Kingdom   0.34%    0.10%    0.06%   33.18%      223,499
    Netherlands      0.83%    0.10%    0.00%   32.96%      142,571
    France           0.86%    0.10%    0.38%   32.95%      575,753

### Discussion for CFR=0.001

This is a negligible Case Fatality rate, and as a result most deaths with COVID are due to other reasons rather than the COVID virus. As a result the comorbidity rates are high (>90%) which is compatible with what was observed in Italy. This scenario would mean that Italy has long reached now the >60% herd immunity threshold, and we should be seeing the tail end of the epidemic soon.

```python
# Since we measure prevalence based on outcomes, the figures lag by about 20 days.
CFR_covid_low = 0.0001 # CFR very low: 0.01%
hospital_infection_mult = 10.0
make_table(populations, CFR_covid_low, hospital_infection_mult, flx=sys.stdout)
```

    Assumptions: COVID CFR:  0.01% In Hospital factor: x10.0
    Country           Prev      CFR  Testing   Comorb.    Infected
    --------------------------------------------------------------
    Japan            0.03%    0.01%    0.62%   90.90%       38,186
    USA              0.14%    0.01%    0.06%   90.86%      472,266
    Germany          0.14%    0.01%    0.40%   90.86%      112,317
    Italy           16.65%    0.01%    0.07%   83.43%    10,066,389
    Spain            5.30%    0.01%    0.14%   88.79%    2,471,375
    Belgium          0.72%    0.01%    0.49%   90.64%       82,301
    Switzerland      1.32%    0.01%    0.12%   90.41%      113,058
    Iran             2.22%    0.01%    0.47%   90.06%    1,799,738
    Korea, South     0.20%    0.01%    3.12%   90.84%      101,616
    United Kingdom   0.47%    0.01%    0.04%   90.73%      310,069
    Netherlands      1.18%    0.01%    0.00%   90.46%      202,948
    France           1.23%    0.01%    0.27%   90.44%      820,873

# Projection forward

Since the data used to estimate prevalence relates to outcomes, we know that the estimation lags behind by the amount of typical time it takes to have an outcome. From studies we consider this to be about 20 days. We therefore build a projection of the prevalence based on a simple model:
* The increase in prevalence follows the difference equation. Its solution is a logistic curve, and 0.6 represents 60% of the population being infected (after which herd immunity kicks in).
\begin{equation}
dp = (p \cdot r - p) \cdot (0.6 - p)
\end{equation}
* The rate of growth $r$ is computed based on the 3 previous days of growth.
* We assume a low CFR (0.01%).

The resulting projections are:

The inflence of $\text{CFR}_c$ is not dramatic, but only moves prevance rates by 2 weeks. For the United Kingdom the plot a range of $\text{CFR}_c$ in $[1\%-0.01\%]$

The same plot for Italy:

```python

```