.. _nb_subset_selection:

## Subset Selection Problem

A genetic algorithm can be used to approach subset selection problems by defining custom operators. In general, a metaheuristic algorithm might not be the ultimate goal to implement in a real-world scenario; however, it might be useful to investigate patterns or characteristics of possible well-performing subsets.
Let us consider a simple toy problem where we have to select numbers from a list. For every solution, exactly ten numbers have to be selected that their sum is minimized.
For the subset selection problem, a binary encoding can be used where **one** indicates a number is picked. In our problem formulation, the list of numbers is represented by $L$ and the binary encoded variable by $x$.

\begin{align}
\begin{split}
\min f(x) & = & \sum_{k=1}^{n} L_k \cdot x_k\\[2mm]
\text{s.t.} \quad g(x)  & = & (\sum_{k=1}^{n} x_k - 10)^2\\[2mm]
\end{split}
\end{align}

As shown above, the equality constraint is handled by ensuring $g(x)$ can only be zero if exactly ten numbers are chosen.
The problem can be implemented as follows:

```python
import numpy as np
from pymoo.core.problem import ElementwiseProblem

class SubsetProblem(ElementwiseProblem):
    def __init__(self,
                 L,
                 n_max
                 ):
        super().__init__(n_var=len(L), n_obj=1, n_constr=1)
        self.L = L
        self.n_max = n_max

    def _evaluate(self, x, out, *args, **kwargs):
        out["F"] = np.sum(self.L[x])
        out["G"] = (self.n_max - np.sum(x)) ** 2

# create the actual problem to be solved
np.random.seed(1)
L = np.array([np.random.randint(100) for _ in range(100)])
n_max = 10
problem = SubsetProblem(L, n_max)

```

The customization requires writing custom operators in order to solve this problem efficiently. We recommend considering the feasibility directly in the evolutionary operators because otherwise, most of the time, infeasible solutions will be processed.
The sampling creates a random solution where the subset constraint will always be satisfied.
The mutation randomly removes a number and chooses another one. The crossover takes the values of both parents and then randomly picks either the one from the first or from the second parent until enough numbers are picked.

```python
from pymoo.core.crossover import Crossover
from pymoo.core.mutation import Mutation
from pymoo.core.sampling import Sampling

class MySampling(Sampling):

    def _do(self, problem, n_samples, **kwargs):
        X = np.full((n_samples, problem.n_var), False, dtype=bool)

        for k in range(n_samples):
            I = np.random.permutation(problem.n_var)[:problem.n_max]
            X[k, I] = True

        return X

class BinaryCrossover(Crossover):
    def __init__(self):
        super().__init__(2, 1)

    def _do(self, problem, X, **kwargs):
        n_parents, n_matings, n_var = X.shape

        _X = np.full((self.n_offsprings, n_matings, problem.n_var), False)

        for k in range(n_matings):
            p1, p2 = X[0, k], X[1, k]

            both_are_true = np.logical_and(p1, p2)
            _X[0, k, both_are_true] = True

            n_remaining = problem.n_max - np.sum(both_are_true)

            I = np.where(np.logical_xor(p1, p2))[0]

            S = I[np.random.permutation(len(I))][:n_remaining]
            _X[0, k, S] = True

        return _X

class MyMutation(Mutation):
    def _do(self, problem, X, **kwargs):
        for i in range(X.shape[0]):
            X[i, :] = X[i, :]
            is_false = np.where(np.logical_not(X[i, :]))[0]
            is_true = np.where(X[i, :])[0]
            X[i, np.random.choice(is_false)] = True
            X[i, np.random.choice(is_true)] = False

        return X
```

After having defined the operators a genetic algorithm can be initialized.

```python
from pymoo.algorithms.soo.nonconvex.ga import GA
from pymoo.optimize import minimize

algorithm = GA(
    pop_size=100,
    sampling=MySampling(),
    crossover=BinaryCrossover(),
    mutation=MyMutation(),
    eliminate_duplicates=True)

res = minimize(problem,
               algorithm,
               ('n_gen', 60),
               seed=1,
               verbose=True)

print("Function value: %s" % res.F[0])
print("Subset:", np.where(res.X)[0])

```

    Compiled modules for significant speedup can not be used!
    https://pymoo.org/installation.html#installation

    To disable this warning:
    from pymoo.config import Config
    Config.show_compile_hint = False

    ===========================================================================
    n_gen |  n_eval |   cv (min)   |   cv (avg)   |     fopt     |     favg
    ===========================================================================
        1 |     100 |  0.00000E+00 |  0.00000E+00 |  2.58000E+02 |  4.43940E+02
        2 |     200 |  0.00000E+00 |  0.00000E+00 |  1.85000E+02 |  3.49800E+02
        3 |     300 |  0.00000E+00 |  0.00000E+00 |  1.59000E+02 |  3.01110E+02
        4 |     400 |  0.00000E+00 |  0.00000E+00 |  1.49000E+02 |  2.60290E+02
        5 |     500 |  0.00000E+00 |  0.00000E+00 |  1.45000E+02 |  2.25590E+02
        6 |     600 |  0.00000E+00 |  0.00000E+00 |  1.28000E+02 |  1.97060E+02
        7 |     700 |  0.00000E+00 |  0.00000E+00 |  1.00000E+02 |  1.75580E+02
        8 |     800 |  0.00000E+00 |  0.00000E+00 |  1.00000E+02 |  1.57490E+02
        9 |     900 |  0.00000E+00 |  0.00000E+00 |  9.80000E+01 |  1.41920E+02
       10 |    1000 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  1.28920E+02
       11 |    1100 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  1.17730E+02
       12 |    1200 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  1.09730E+02
       13 |    1300 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  1.03210E+02
       14 |    1400 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  9.73500E+01
       15 |    1500 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  9.37400E+01
       16 |    1600 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  9.04500E+01
       17 |    1700 |  0.00000E+00 |  0.00000E+00 |  6.10000E+01 |  8.60100E+01
       18 |    1800 |  0.00000E+00 |  0.00000E+00 |  5.70000E+01 |  8.21100E+01
       19 |    1900 |  0.00000E+00 |  0.00000E+00 |  5.50000E+01 |  7.90200E+01
       20 |    2000 |  0.00000E+00 |  0.00000E+00 |  5.50000E+01 |  7.72200E+01
       21 |    2100 |  0.00000E+00 |  0.00000E+00 |  5.30000E+01 |  7.44300E+01
       22 |    2200 |  0.00000E+00 |  0.00000E+00 |  5.30000E+01 |  7.25200E+01
       23 |    2300 |  0.00000E+00 |  0.00000E+00 |  4.90000E+01 |  7.06400E+01
       24 |    2400 |  0.00000E+00 |  0.00000E+00 |  4.60000E+01 |  6.82500E+01
       25 |    2500 |  0.00000E+00 |  0.00000E+00 |  4.60000E+01 |  6.49500E+01
       26 |    2600 |  0.00000E+00 |  0.00000E+00 |  4.60000E+01 |  6.21700E+01
       27 |    2700 |  0.00000E+00 |  0.00000E+00 |  4.40000E+01 |  5.90200E+01
       28 |    2800 |  0.00000E+00 |  0.00000E+00 |  4.40000E+01 |  5.76400E+01
       29 |    2900 |  0.00000E+00 |  0.00000E+00 |  4.40000E+01 |  5.61500E+01
       30 |    3000 |  0.00000E+00 |  0.00000E+00 |  4.40000E+01 |  5.44400E+01
       31 |    3100 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.31800E+01
       32 |    3200 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.27300E+01
       33 |    3300 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.21800E+01
       34 |    3400 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.17700E+01
       35 |    3500 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.12400E+01
       36 |    3600 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.06200E+01
       37 |    3700 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.02900E+01
       38 |    3800 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  5.00100E+01
       39 |    3900 |  0.00000E+00 |  0.00000E+00 |  4.10000E+01 |  4.95300E+01
       40 |    4000 |  0.00000E+00 |  0.00000E+00 |  3.90000E+01 |  4.88000E+01
       41 |    4100 |  0.00000E+00 |  0.00000E+00 |  3.90000E+01 |  4.85000E+01
       42 |    4200 |  0.00000E+00 |  0.00000E+00 |  3.90000E+01 |  4.82400E+01
       43 |    4300 |  0.00000E+00 |  0.00000E+00 |  3.90000E+01 |  4.81500E+01
       44 |    4400 |  0.00000E+00 |  0.00000E+00 |  3.90000E+01 |  4.77300E+01
       45 |    4500 |  0.00000E+00 |  0.00000E+00 |  3.90000E+01 |  4.75700E+01
       46 |    4600 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.68200E+01
       47 |    4700 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.64700E+01
       48 |    4800 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.62800E+01
       49 |    4900 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.59900E+01
       50 |    5000 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.58000E+01
       51 |    5100 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.55800E+01
       52 |    5200 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.54500E+01
       53 |    5300 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.53900E+01
       54 |    5400 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.51800E+01
       55 |    5500 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.50300E+01
       56 |    5600 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.48000E+01
       57 |    5700 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.47100E+01
       58 |    5800 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.45800E+01
       59 |    5900 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.44300E+01
       60 |    6000 |  0.00000E+00 |  0.00000E+00 |  3.70000E+01 |  4.43300E+01
    Function value: 37.0
    Subset: [ 5  9 12 36 37 40 47 52 68 99]

Finally, we can compare the found subset with the optimum known simply through sorting:

```python
opt = np.sort(np.argsort(L)[:n_max])
print("Optimal Subset:", opt)
print("Optimal Function Value: %s" % L[opt].sum())
```

    Optimal Subset: [ 5  9 12 31 36 37 47 52 68 99]
    Optimal Function Value: 36


#Softmax from First Principles

## Language barriers between humans and autonomous systems

If our goal is to help humans and autnomous systems communicate, we need to speak in a common language. Just as humans have verbal and written languages to communicate ideas, so have we developed mathematical languages to communicate information. Probability is one of those languages and, thankfully for us, autonomous systems are pretty good at describing probabilities, even if humans aren't. This document shows one technique for translating a human language (English) into a language known by autonomous systems (probability).

Our translator is something called the **SoftMax classifier**, which is one type of probability distribution that takes discrete labels and translates them to probabilities. We'll show you the details on how to create a softmax model, but let's get to the punchline first: we can decompose elements of human language to represent a partitioning of arbitrary state spaces.

Say, for instance, we'd like to specify the location of an object in two dimensional cartesian coordinates. Our state space is all combinations of *x* and *y*, and we'd like to translate human language into some probability that our target is at a given combination of *x* and *y*. One common tactic humans use to communicate position is range (near, far, next to, etc.) and bearing (North, South, SouthEast, etc.). This already completely partitions our *xy* space: if something is north, it's not south; if it's east, it's not west; and so on.

A softmax model that translates range and bearing into probability in a state space is shown below:

Assuming that *next to* doesn't require a range, we see seventeen different word combinations we can use to describe something's position: two ranges (*nearby* and *far*) for each cardinal and intercardinal direction (eight total), and then one extra label for *next to*. This completely partitions our entire state space $\mathbb{R}^2$.

This range and bearing language is, by its natue, inexact. If I say, "That boat is far north.", you don't have a deterministic notion of exactly where the boat is -- but you have a good sense of where it is, and where it is not. We can represent that sense probabilistically, such that the probability of a target existing at a location described by a range and bearing label is nonzero over the entire state space, but that probability is very small if not in the area most associated with that label.

What do we get from this probabilistic interpretation of the state space? We get a two-way translation between humans and autonomous systems to describe anything we'd like. If our state space is one-dimensional relative velocity (i.e. the derivative of range without bearing), I can say, "She's moving really fast!", to give the autonomous system a probability distribution over my target's velocity with an expected value of, say, 4 m/s. Alternatively, if my autnomous system knows my target's moving at 0.04352 m/s, it can tell me, "Your target is moving slowly." Our labeled partitioning of the state space (that is, our classifier) is the mechanism that translates for us.

## Softmax model construction
The [SoftMax model](http://en.wikipedia.org/wiki/Softmax_function) goes by many names: normalized exponential, multinomial logistic function, log-linear model, sigmoidal function. We use the SoftMax function to develop a classification model for our state space:

$$
P(D=i \vert \mathbf{x}) = \frac{e^{\mathbf{w}_i^T \mathbf{x}}}{\sum_{k=i}^M e^{\mathbf{w}_k^T\mathbf{x}}}
$$

Where $D = i$ is our random variable of class labels instantiated as class $i$, $\mathbf{x}$ is our state vector, $\mathbf{w}_i$ is a vector of parameters (or weights) associated with our class $i$, and $M$ is the total number of classes. The state vector $\mathbf{x}$ traditionally includes a constant bias term.

Note that a label is a set of words associated with a class (i.e. *far northwest*) whereas a class is a single probability distribution over the entire state space. The terms are sometimes used interchangeably.

Several key factors come out of the SoftMax equation:
- The probabilities of all classes for any given point $\mathbf{x}$ sum to 1.
- The probability any single class for any given point $\mathbf{x}$ is bounded by 0 and 1.
- The space can be partitioned into an arbitrary number of classes (with some restrictions about those classes - more on this later).
- The probability of one class for a given point $\mathbf{x}$ is determined by that class' weighted exponential sum of the state vector *relative* to the weighted exponential sums of other classes.
- Since the probability of a class is conditioned on $\mathbf{x}$, we can apply estimators such as [Maximum Likelihood](http://en.wikipedia.org/wiki/Maximum_likelihood) to learn SoftMax models.
- $P(D=i \vert \mathbf{x})$ is convex for any arbitrary $\mathbf{x}$ [citation needed].

For any two classes, we can take the ratio of their probabilities to determine the **odds** of one class instead of the other:

$$
L(i,j) =\frac{P(D=i \vert \mathbf{x})}{P(D=j \vert \mathbf{x})} =
\frac{\frac{e^{\mathbf{w}_i^T \mathbf{x}}}{\sum_{k=i}^M e^{\mathbf{w}_k^T\mathbf{x}}}}{\frac{e^{\mathbf{w}_j^T \mathbf{x}}}{\sum_{k=i}^M e^{\mathbf{w}_k^T\mathbf{x}}}} = \frac{e^{\mathbf{w}_i^T \mathbf{x}}}{e^{\mathbf{w}_j^T\mathbf{x}}}
$$

When $L(i,j)=1$, the two classes have equal probability. This doesn't give us a whole lot of insight until we take the **log-odds** (the logarithm of the odds):

$$
L_{log}(i,j) =
\log{\frac{P(D=i \vert \mathbf{x})}{P(D=j \vert \mathbf{x})}} = \log{\frac{e^{\mathbf{w}_i^T \mathbf{x}}}{e^{\mathbf{w}_j^T\mathbf{x}}}} = \mathbf{w}_i^T\mathbf{x} - \mathbf{w}_j^T\mathbf{x} = (\mathbf{w}_i - \mathbf{w}_j)^T\mathbf{x}
$$

When $L_{log}(i,j) = \log{L(i,j)} = \log{1} = 0$, we have equal probability between the two classes, and we've also stumbled upon the equation for an n-dimensional affine hyperplane dividing the two classes:

$$
\begin{align}
0 &= (\mathbf{w}_i - \mathbf{w}_j)^T\mathbf{x} \\
  &= (w_{i,x_1} - w_{j,x_1})x_1 + (w_{i,x_2} - w_{j,x_2})x_2 + \dots + (w_{i,x_n} - w_{j,x_n})x_n
\end{align}
$$

This follows from the general definition of an Affine Hyperplane (that is, an n-dimensional flat plane):

$$
a_1x_1 + a_2x_2 + \dots + a_nx_n - b = 0
$$

This gives us a general formula for the division of class boundaries -- that is, we can specify the class boundaries directly, rather than specifying the weights leading to those class boundaries.

### Example
Let's take a step back and look at an example. Suppose I'm playing Pac-Man, and I want to warn our eponymous hero of a ghost approaching him. Let's restrict my language to the four intercardinal directions: NE, SE, SW and NW. My state space is $\mathbf{x} = \begin{bmatrix}1 & x & y\end{bmatrix}^T$, which includes one bias term and one term for each cartesian direction in $\mathbb{R}^2$.

In this simple problem, we can expect our weights to be something along the lines of:

$$
\begin{align}
\mathbf{w}_{NE} &= \begin{bmatrix}0 & 1 & 1 \end{bmatrix}^T \\
\mathbf{w}_{SE} &= \begin{bmatrix}0 & 1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{SW} &= \begin{bmatrix}0 & -1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{NW} &= \begin{bmatrix}0 & -1 & 1 \end{bmatrix}^T
\end{align}
$$

If we run these weights in our SoftMax model, we get the following results:

```
import numpy as np
from cops_and_robots.robo_tools.fusion.softmax import SoftMax
%matplotlib inline

labels = ['SW','NW','SE','NE']
weights = np.array([[0, -1, -1],
                    [0, -1, 1],
                    [0, 1, -1],
                    [0, 1, 1],
                   ])
pacman = SoftMax(weights,class_labels=labels)
pacman.plot(title='Pac-Man Bearing Model')
```

Which is along the right path, but needs to be shifted down to Pac-Man's location. Say Pac-Man is approximately one quarter of the map south from the center point, we can update our model accordingly (assuming a $10m \times 10m$ space):

$$
\begin{align}
\mathbf{w}_{NE} &= \begin{bmatrix}2.5 & 1 & 1 \end{bmatrix}^T \\
\mathbf{w}_{SE} &= \begin{bmatrix}-2.5 & 1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{SW} &= \begin{bmatrix}-2.5 & -1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{NW} &= \begin{bmatrix}2.5 & -1 & 1 \end{bmatrix}^T
\end{align}
$$

```
weights = np.array([[-2.5, -1, -1],
                    [2.5, -1, 1],
                    [-2.5, 1, -1],
                    [2.5, 1, 1],
                   ])
pacman = SoftMax(weights,class_labels=labels)
pacman.plot(title='Pac-Man Bearing Model')
```

Looking good! Note that we'd get the same answer had we used the following weights:

$$
\begin{align}
\mathbf{w}_{NE} &= \begin{bmatrix}0 & 1 & 1 \end{bmatrix}^T \\
\mathbf{w}_{SE} &= \begin{bmatrix}-5 & 1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{SW} &= \begin{bmatrix}-5 & -1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{NW} &= \begin{bmatrix}0 & -1 & 1 \end{bmatrix}^T
\end{align}
$$

Because the class boundaries are defined by the *relative differences*.

One other thing we can illustrate with this example: how would the SoftMax model shift if we multiplied all our weights by 10? When we use:

$$
\begin{align}
\mathbf{w}_{NE} &= \begin{bmatrix}25 & 10 & 10 \end{bmatrix}^T \\
\mathbf{w}_{SE} &= \begin{bmatrix}-25 & 10 & -10 \end{bmatrix}^T \\
\mathbf{w}_{SW} &= \begin{bmatrix}-25 & -10 & -10 \end{bmatrix}^T \\
\mathbf{w}_{NW} &= \begin{bmatrix}25 & -10 & 10 \end{bmatrix}^T
\end{align}
$$

We get:

```
weights = np.array([[-25, -10, -10],
                    [25, -10, 10],
                    [-25, 10, -10],
                    [25, 10, 10],
                   ])
pacman = SoftMax(weights,class_labels=labels)
pacman.plot(title='Pac-Man Bearing Model')
```

Why does this increase in slope happen? We'll let you ponder this question for now and get back to it later.

## Specification from Normals
We've seen how we can create a simple SoftMax distribtion by specifying each class's parameters, but we also saw that the log-odds between two classes generates a hyperplane when one class equals the other. If we wanted to chop up the state space, couldn't we simply specify those hyperplanes instead?

First, we need to investigate the relationships between the weights and the normals. Take our first attempt at the intercardinal bearing Pac-Man problem: we have four classes, and each class shares a boundary with another class over an equiprobable region. In the case of *NE* and *SE*, for instance, we have a line dividing north from south as our boundary. In the case of *NE* and *SW*, we have a point at the cardinal center as our boundary. Let's see if we can calculate these.

Recall our weights:

$$
\begin{align}
\mathbf{w}_{NE} &= \begin{bmatrix}0 & 1 & 1 \end{bmatrix}^T \\
\mathbf{w}_{SE} &= \begin{bmatrix}0 & 1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{SW} &= \begin{bmatrix}0 & -1 & -1 \end{bmatrix}^T \\
\mathbf{w}_{NW} &= \begin{bmatrix}0 & -1 & 1 \end{bmatrix}^T
\end{align}
$$

And the definition of our class boundaries:

$$
\begin{align}
0 &= (\mathbf{w}_i - \mathbf{w}_j)^T\mathbf{x} \\
  &= (w_{i,x_1} - w_{j,x_1})x_1 + (w_{i,x_2} - w_{j,x_2})x_2 + \dots + (w_{i,x_n} - w_{j,x_n})x_n
\end{align}
$$

From the equation of a plane, recall that the **normal vector** can be taken from the coefficients for the non-constant terms. We can define a normal vector for each class boundary:

$$
\begin{align}
\mathbf{n}_{NW,NE} &= (\mathbf{w}_{NE} - \mathbf{w}_{NW})^T\mathbf{x} \\
&= (0 - 0) + (1 + 1)x + (1 - 1)y = 2x \\
\mathbf{n}_{NE,SE} &= (0 - 0) + (1 - 1)x + (-1 - 1)y = -2y \\
\mathbf{n}_{SE,SW} &= (0 - 0) + (-1 - 1)x + (-1 - 1)y = -2x \\
\mathbf{n}_{SW,NW} &= (0 - 0) + (-1 + 1)x + (1 + 1)y = 2y
\end{align}
$$

But these aren't the only class boundaries -- diagonally positioned classes also have normals:

$$
\begin{align}
\mathbf{n}_{NW,SE} &= (0 - 0) + (1 + 1)x + (-1 - 1)y = 2x - 2y \\
\mathbf{n}_{NE,SW} &= (0 - 0) + (-1 - 1)x + (-1 - 1)y = -2x -2y \\
\end{align}
$$

So we have a map of the normal vectors:

```
import matplotlib.pyplot as plt
x = np.arange(-5,5,0.1)
n_NWNE = 0  # Vertical Line
n_NESE = 0 * x # Horizontal Line
n_SESW = 0 # Vertical Line
n_SWNW = 0 * x # Horizontal Line
n_NWSE = -x
n_SWNE = x

plt.plot(x, n_NESE, 'g-', label="NESE", lw=3)
plt.plot(x, n_SWNW, 'y--', label="SWNW", lw=3)
plt.plot(x, n_NWSE, 'k-', label="NWSE", lw=3)
plt.plot(x, n_SWNE, 'r-', label="SWNE", lw=3)
plt.axvline(color='blue', label="NWNE", lw=3)
plt.axvline(color='pink', ls='--', label="SESW", lw=3)

plt.grid()
plt.xlabel('x [m]')
plt.ylabel('y [m]')
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.title('Normals to Intercardinal Spaces')
plt.legend(loc='lower center', bbox_to_anchor=(-0.2, -0.175, 1.4, -0.075),
            mode='expand', borderaxespad=0., ncol=6)
```

Using this visualization as a guide, we notice that the sum of all normal vectors is zero in each dimension. Will this always be true? Let's generalize:

$$

$$

```
### Example
```

```
weights = np.array([[0, -1, 1],
                    [2, 0, 0],
                    [0, 1, -1],
                   ])
simple = SoftMax(weights)
simple.plot()
```

## Learning weights from data

### Example

## Prior class boundaries

### Example

## Using symmetry to develop models from sparse data

### Examples

a

a

a

```

```

```

```

```
from IPython.core.display import HTML

# Borrowed style from Probabilistic Programming and Bayesian Methods for Hackers
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Roboto:500,300,300italic,400' rel='stylesheet' type='text/css'>


```python
import numpy as np
import sympy as sp
```

```python
a,b,c,d = sp.symbols('a, b, c, d')
w = sp.symbols('w')
s = sp.symbols('s')
h = sp.symbols('h')
A = sp.Matrix([[0, w], [-w, 0]])
I2 = sp.Matrix([[1, 0], [0, 1]])
B = sp.Matrix([[a,b],[c,d]])
B.inv()
```

    Matrix([
    [1/a + b*c/(a**2*(d - b*c/a)), -b/(a*(d - b*c/a))],
    [          -c/(a*(d - b*c/a)),      1/(d - b*c/a)]])

```python
As = (s*I2-A)
As
```

    Matrix([
    [s, -w],
    [w,  s]])

```python
Asinv = As.inv()
sp.pretty_print(Asinv)
```

    ⎡          2                ⎤
    ⎢1        w           w     ⎥
    ⎢─ - ───────────  ──────────⎥
    ⎢s      ⎛     2⎞    ⎛     2⎞⎥
    ⎢     2 ⎜    w ⎟    ⎜    w ⎟⎥
    ⎢    s ⋅⎜s + ──⎟  s⋅⎜s + ──⎟⎥
    ⎢       ⎝    s ⎠    ⎝    s ⎠⎥
    ⎢                           ⎥
    ⎢     -w              1     ⎥
    ⎢  ──────────       ──────  ⎥
    ⎢    ⎛     2⎞            2  ⎥
    ⎢    ⎜    w ⎟           w   ⎥
    ⎢  s⋅⎜s + ──⎟       s + ──  ⎥
    ⎣    ⎝    s ⎠           s   ⎦

```python
sp.LaplaceTransform?
```

```python
A*A*A
```

    Matrix([
    [   0, -w**3],
    [w**3,     0]])

```python
As =
```


# Linear-time

## Pseudo-Code
The "Linear-time" maximum sub-array algorithm is described by the following pseudo-code:
~~~~
LINEAR-TIME-MAX-SUBARRAY(A[1, ..., N])
	maximum sum = -Infinity
	sum ending here = -Infinity
	for i from 1 to N
		ending here high index = j
		if ending here sum > 0
			ending here sum = ending here sum + A[i]
		else
			ending here low index = j
			ending here sum = A[i]

		if ending here sum > maximum sum
			maximum sum = ending here sum
			start index = ending here low index
			end index = ending here high index

	return maximum sum, A[start index, ..., end index]
~~~~

## Theoretical Run-time Analysis
The $i$ loop runs from $1$ to $N$. Inside the loop are constant time operations. We can compute the number of iterations of these constant time operations as:

\begin{equation}
\begin{split}
\sum_{i=1}^N \Theta(1) & = N\cdot \Theta(1) \\
& = \Theta(N)
\end{split}
\end{equation}

Thus, the theoritical run-time is $\Theta(N)$.

## Experimental Analysis
For a series of array sizes $N$, 10 random arrays were generated and run through the "Linear-time" algorithm. The CPU clock time was recorded for each of the 10 random array inputs, and an average run time was computed. Below is the plot of average run time versus $N$ for the "Linear-time" algorithm.

A curve fit was applied to the average run time data, which resulted in the following fit equation as a function of $x$ standing in for $N$:

$$y = 2.04735 * 10^{-7} * x - 1.4449$$

The best fit curve fits the plotted data extremely well, showing that the runtimes reflect a linear trend. The observed linear trend in the data matches with the theoretically derived runtime of $\theta(n)$.

Based on the average run time data curve fit, we would expect the "Linear-time" algorithm to be able to process the following number of elements in the given amount of time:

|    Time    | Max Input Size |
|:----------:|:--------------:|
| 10 seconds |     55901100   |
| 30 seconds |    153588000   |
| 60 seconds |    300119000   |


# The basic solow model and extension with human capital

Imports for use in project:

```python
import numpy as np
from scipy import optimize
import sympy as sm
from types import SimpleNamespace
import matplotlib.pyplot as plt
import ipywidgets as widgets
from types import SimpleNamespace
import warnings

# Autoreload modules when code is run
%load_ext autoreload
%autoreload 2

# Import our python file
import modelproject as mp
```

# Basic Solow model

**Write out the model in equations here.**

Make sure you explain well the purpose of the model and comment so that other students who may not have seen it before can follow.

## Analytical solution

If your model allows for an analytical solution, you should provide here.

You may use Sympy for this. Then you can characterize the solution as a function of a parameter of the model.

To characterize the solution, first derive a steady state equation as a function of a parameter using Sympy.solve and then turn it into a python function by Sympy.lambdify. See the lecture notes for details.

## Numerical solution

To find a numerical solution for the steady state of capital we use the Solow-equation:
$$ k_{t+1} - k_t = \frac{1}{1+n} \left[sBk_t^\alpha - (n+\delta) k_t \right] $$
And the fact that in steady state we have: $$ k_{t+1}=k_t=k^* $$

This gives us that in steady state it must follow that:
$$ 0 = \frac{1}{1+n} \left[sBk^{*\alpha} - (n+\delta) k^* \right]$$
$$\Leftrightarrow 0 = sBk^{*\alpha} - (n+\delta) k^* \tag{1} $$

As plausible values for our parameters we look at table A pages 349-351 in the book: **Introducing Advanced Macroeconomics - Growth and business cycles** by *Peter Birch Sørensen and Hans Jørgen Whitta-Jacobsen*, second edition. We choose values for Denmark.

```python
# Plausible parameter values as found in table A
par = SimpleNamespace()
par.alpha = 1/3
par.s = 0.333
par.B = 1
par.n = 0.008
par.delta = 0.01
```

We construct an algorithm to find the steady state of capital per worker and use it with the plausible values.

```python
# Algorithm to find the root of a function on the interval [a,b] with the bisect method
def bisection(f,a,b,tol=1e-8):
    """
    Args:
    f      (function): Function you wish to find root of
    a      (float)   : Lower bound
    b      (float)   : Upper bound
    tol    (float)   : Tolerance on solution

    Returns:
    The x value that solves equation f(x) = 0 for a <= x <= b.

    """

    # Test inputs
    if f(a)*f(b) >= 0:
        print("Bisection method fails.") # If f(a) and f(b) both have same sign it will not have a root on given interval
        return None

    # Step 1: initialize
    a_n = a
    b_n = b

    # Step 2-4:
    while True:

        # Step 2: midpoint and associated value
        m_n = (a_n+b_n)/2 # The midpoint of interval is found
        f_m_n = f(m_n) # The function value of the midpoint (m) is found

        # Step 3: Check if solution is found and if not determine sub-interval for evaluation in next iteration
        if abs(f_m_n) < tol: # If the function value of the midpoint is close enough to 0 we have found our solution
            return m_n
        elif f(a_n)*f_m_n < 0: # If f(a) and f(m) have different signs then m is the new upper bound
            a_n = a_n
            b_n = m_n
        elif f(b_n)*f_m_n < 0: # If f(b) and f(m) have different signs then m is the new lower bound
            a_n = m_n
            b_n = b_n
        else:
            print("Bisection method fails.")
            return None
    # Step 4: results
    return (a_n + b_n)/2

# Use the algorithm to find the steady state of capital (k*)
f = lambda k: par.s*par.B*k**par.alpha - (par.n + par.delta)*k # Equation (1) is the one we wish to solve
print(f'The steady state of capital found with algorithm is : k* = {bisection(f,0.1,100,1e-8):.3f}')
```

    The steady state of capital found with algorithm is : k* = 79.572

We can also solve the problem by using a built in optimizer from the scipy package that uses the bisection method.

```python
print(f'The steady state of capital found by using built in optimizer is :k* = {mp.ss_bas(par.n, par.s, par.B, par.alpha, par.delta):.3f}')
```

    The steady state of capital found by using built in optimizer is :k* = 79.572

Unsurprisingly the results are the same.

From equation (1) we can see that in order for the model to converge to a steady state of capital then the stability condition of $$ n + \delta > 0 $$ must be fulfilled. If this condition is met then the model will converge for any starting values.

## Further analysis

We plot the basic solow model with interactive widgets to examine how changes in parameters affect the model. We also find the steady states of: capital per worker (k*), output per worker (y*) and consumption per worker (c*).

```python
widgets.interact(mp.ss_bas_plot,
                alpha = widgets.FloatSlider(description = r'$\alpha$', min = 0.01, max = 0.6, step = 0.005, value = 0.3, readout_format='.3f'),
                delta = widgets.FloatSlider(description = r'$\delta$', min = 0, max = 0.1, step = 0.005, value = 0.01, readout_format='.3f'),
                s = widgets.FloatSlider(description = '$s$', min = 0.01, max = 0.7, step = 0.005, value = 0.4, readout_format='.3f'),
                n = widgets.FloatSlider(description ='$n$', min = -0.05, max = 0.05, step = 0.005, value = 0.01, readout_format='.3f'),
                B = widgets.fixed(1), #B is assumed to be the same across countries
                k_max = widgets.IntSlider(description='k_max', min = 1, max = 1000, step = 10, value = 100)) # Changes the size of x-axis to zoom
```

    interactive(children=(FloatSlider(value=0.01, description='$n$', max=0.05, min=-0.05, readout_format='.3f', st…

    <function modelproject.ss_bas_plot(n, s, B, alpha, delta, k_max)>

From figure 1 and the interactive widgets we can see the ceteris paribus effects of changes in the different parameters. We find that: n is positively correlated with k*, y* and c*, s is positively correlated with k* and y* but negatively correlated with c*, $\alpha$ is positively correlated with k*, y* and c* while $\delta$ is negatively correlated with k*, y* and c*.

In order to ensure convergance we must have that $ n + \delta > 0 $

# Solow model with human capital

**Write out the model in equations here.**

Make sure you explain well the purpose of the model and comment so that other students who may not have seen it before can follow.

```python
# Plausible parameter values as found in table A
par.s_H = 0.1
par.s_K = 0.2
par.g = 0.02
par.n = 0.01
par.alpha = 1/3
par.phi = 1/3
par.delta = 0.01
```

## Analytical solution

If your model allows for an analytical solution, you should provide here.

You may use Sympy for this. Then you can characterize the solution as a function of a parameter of the model.

To characterize the solution, first derive a steady state equation as a function of a parameter using Sympy.solve and then turn it into a python function by Sympy.lambdify. See the lecture notes for details.

## Numerical solution

The equations for the steady states of physical capital per capita and human capital per capita can be written as

\\[ 0 = \left(\dfrac{s_k^{1-\varphi}s_h^{\varphi}}{n+g+\delta+ng}\right)^{\frac{1}{1-\alpha-\varphi}} - \tilde{k}^{\ast} \\]

\\[ 0 = \left(\dfrac{s_k^{\alpha}s_h^{1-\alpha}}{n+g+\delta+ng}\right)^{\frac{1}{1-\alpha-\varphi}} - \tilde{h}^{\ast} \\]

```python
warnings.filterwarnings("ignore", category=RuntimeWarning) # Some values of parameters give a runtimewarning, this is ignored

# We define a function containing our h- and k-functions, and a vector x = [h,k]
obj = lambda x: [mp.h_func(x[1],par.s_H,par.s_K,par.g,par.n,par.alpha,par.phi,par.delta,x[0]),mp.k_func(x[0],par.s_H,par.s_K,par.g,par.n,par.alpha,par.phi,par.delta,x[1])]

# We solve the vector functions, have nonlinear system of equation and therefore use broyden1 method
sol = optimize.root(obj,[1,1],method = 'broyden1')

# The numerical solution is found
num_sol1 = sol.x

print(f'Numerical solution is: k* = {num_sol1[1]:.3f}, h* = {num_sol1[0]:.3f}')
```

    Numerical solution is: k* = 61.572, h* = 30.786

```python
par.incr = 0.01 # Define a variable called incr to examine the effects of increase in human capital

# Solve the functions, first with original value of human capital and then with increased human capital
k_vec, h_vec_DeltaK0, h_vec_DeltaH0 = mp.solve_ss(par.s_H,par.s_K,par.g,par.n,par.alpha,par.phi,par.delta)
k1_vec, h1_vec_DeltaK0, h1_vec_DeltaH0 = mp.solve_ss(par.s_H+par.incr,par.s_K,par.g,par.n,par.alpha,par.phi,par.delta)

# Find steady state values for new s_H

# # We define a function containing our h- and k-functions, and a vector x = [h,k] for increased human capital
obj1 = lambda x: [mp.h_func(x[1],par.s_H+par.incr,par.s_K,par.g,par.n,par.alpha,par.phi,par.delta,x[0]),mp.k_func(x[0],par.s_H+par.incr,par.s_K,par.g,par.n,par.alpha,par.phi,par.delta,x[1])]

# We solve the vector functions, have nonlinear system of equation and therefore use broyden1 method
sol1 = optimize.root(obj1,[1,1],method = 'broyden1')

# The numerical solution is found for increased human capital
num_sol2 = sol1.x

# Create the plot
fig2 = plt.figure(figsize=(7,7))
ax = fig2.add_subplot(1,1,1)
ax.plot(k_vec,h_vec_DeltaK0, label=r'$\Delta \tilde{k}=0$', c='lime')
ax.plot(k_vec,h_vec_DeltaH0, label=r'$\Delta \tilde{h}=0$', c='aqua')
ax.plot(k1_vec,h1_vec_DeltaH0, label=r'$\Delta \tilde{h_1}=0$', c='teal')
ax.set_xlabel(r'$\tilde{k}$')
ax.set_ylabel(r'$\tilde{h}$')
ax.legend()

# We mark the steady state
plt.scatter(sol.x[1],sol.x[0],color='black',s=80,zorder=3)
plt.scatter(sol1.x[1],sol1.x[0],color='black',s=80,zorder=3)

# Lines are drawn to mark ss-value on the axes
plt.axvline(sol.x[1],ymax=sol.x[0]/60,color='gray',linestyle='--')
plt.axhline(sol.x[0],xmax=sol.x[1]/80,color='gray',linestyle='--')
plt.axvline(sol1.x[1],ymax=sol1.x[0]/60,color='black',linestyle='--')
plt.axhline(sol1.x[0],xmax=sol1.x[1]/80,color='black',linestyle='--')

# Text is added to the plot
ax.text(0.15,sol.x[0]+1 , r'$\tilde{h}^*$', fontsize=12)
ax.text(sol.x[1]+1,0.15 , r'$\tilde{k}^*$', fontsize=12)
ax.text(0.15,sol1.x[0]+1 , r'$\tilde{h_1}^*$', fontsize=12)
ax.text(sol1.x[1]+1,0.15 , r'$\tilde{k_1}^*$', fontsize=12)

#The axis limits are chosen
ax.set(xlim=(0, 80), ylim=(0, 60))

ax.set_title('Figure 2. Phase diagram');

print(f'Numerical solution for original s_H is: k* = {num_sol1[1]:.3f}, h* = {num_sol1[0]:.3f}')
print(f'Numerical solution for increased s_H is: k* = {num_sol2[1]:.3f}, h* = {num_sol2[0]:.3f}')
```

Figure 2 shows the phase diagram and the steady state is found where the nullclines intersect. We can see that an increase in $s_H$ increases the steady states of both $\tilde{h}$ and $\tilde{k}$.

# Conclusion

Add concise conclusion.


# Space Science with Python - Part 1
## Distance between Earth and the Sun

```python
# Import the SPICE module
import spiceypy
```

```python
# We want to determine the position of our home planet with respect to the Sun.
# The datetime shall be set as "today" (midnight). SPICE requires the Ephemeris Time (ET);
# thus, we need to convert a UTC datetime string to ET.

import datetime

# get today's date
DATE_TODAY = datetime.datetime.today()
print(DATE_TODAY)
# convert the datetime to a string, replacing the time with midnight
DATE_TODAY = DATE_TODAY.strftime('%Y-%m-%dT00:00:00')
print(DATE_TODAY)

```

    2021-10-18 18:00:57.686419
    2021-10-18T00:00:00

```python
# For this tutorial the Git repository contains already the necessary kernel. We need to load it first.
spiceypy.furnsh('../_kernels/lsk/naif0012.tls')
```

```python
# Let's re-try our first time conversion command
ET_TODAY_MIDNIGHT = spiceypy.utc2et(DATE_TODAY)
```

```python
# It works! How does the value look like?
print(ET_TODAY_MIDNIGHT)
```

    687787269.1823821

## Compute the state of the Earth with respect to the Sun

Can we compute now the position and velocity (so called state) of the Earth
with respect to the Sun? We use the following function to determine the
state vector and the so called light time (travel time of the light between
the Sun and our home planet). Positions are always given in km, velocities
in km/s and times in seconds

targ : Object that shall be pointed at

et : The ET of the computation

ref : The reference frame. Here, it is ECLIPJ2000 (so Medium article)

obs :  The observer respectively the center of our state vector computation

```python
# Load a so called spk to load positional information:
spiceypy.furnsh('../_kernels/spk/de432s.bsp')
```

```python
# Let's try the computation
EARTH_STATE_WRT_SUN, EARTH_SUN_LIGHT_TIME = spiceypy.spkgeo(targ=399, et=ET_TODAY_MIDNIGHT, \
                                                            ref='ECLIPJ2000', obs=10)
```

```python
# The state vector is 6 dimensional: x,y,z in km and the corresponding velocities in km/s
print('State vector of the Earth w.r.t. the Sun for "today" (midnight):\n', EARTH_STATE_WRT_SUN)
```

    State vector of the Earth w.r.t. the Sun for "today" (midnight):
     [ 1.35596428e+08  6.18986888e+07 -3.08968785e+03 -1.28556950e+01
      2.69750187e+01 -1.63129751e-03]

```python
# The (Euclidean) distance should be around 1 AU. Why "around"? Well the Earth revolves the Sun in a slightly
# non-perfect circle (elliptic orbit). First, we compute the distance in km.
import math
EARTH_SUN_DISTANCE = math.sqrt(EARTH_STATE_WRT_SUN[0]**2.0 \
                             + EARTH_STATE_WRT_SUN[1]**2.0 \
                             + EARTH_STATE_WRT_SUN[2]**2.0)
```

```python
# Convert the distance in astronomical units (1 AU)
# Instead of searching for the "most recent" value, we use the default value in SPICE. This way, we can easily
# compare our results with the results of others.
EARTH_SUN_DISTANCE_AU = spiceypy.convrt(EARTH_SUN_DISTANCE, 'km', 'AU')

# Cool, it works!
print('Current distance between the Earth and the Sun in AU:', EARTH_SUN_DISTANCE_AU)
```

    Current distance between the Earth and the Sun in AU: 0.9963811342608239

## Orbital speed of the Earth
For this, we need the equation to determine the orbital speed. We assume that the Sun's mass is greater than the mass
of the Earth and we assume that our planet is moving on an almost circular orbit. The orbit velocity $v_{\text{orb}}$ can be approximated with, where $G$ is the gravitational constant, $M$ is the mass of the Sun and $r$ is the distance between the Earth and the Sun:
\begin{align}
    v_{\text{orb}}\approx\sqrt{\frac{GM}{r}}
\end{align}

```python
# First, we compute the actual orbital speed of the Earth around the Sun
EARTH_ORB_SPEED_WRT_SUN = math.sqrt(EARTH_STATE_WRT_SUN[3]**2.0 \
                                  + EARTH_STATE_WRT_SUN[4]**2.0 \
                                  + EARTH_STATE_WRT_SUN[5]**2.0)

# It's around 30 km/s
print('Current orbital speed of the Earth around the Sun in km/s:', EARTH_ORB_SPEED_WRT_SUN)
```

    Current orbital speed of the Earth around the Sun in km/s: 29.88177586346213

```python
# Now let's compute the theoretical expectation. First, we load a pck file that contain miscellanoeus information,
# like the G*M values for different objects

# First, load the kernel
spiceypy.furnsh('../_kernels/pck/gm_de431.tpc')
_, GM_SUN = spiceypy.bodvcd(bodyid=10, item='GM', maxn=1)

# Now compute the orbital speed
V_ORB_FUNC = lambda gm, r: math.sqrt(gm/r)
EARTH_ORB_SPEED_WRT_SUN_THEORY = V_ORB_FUNC(GM_SUN[0], EARTH_SUN_DISTANCE)

# Print the result
print('Theoretical orbital speed of the Earth around the Sun in km/s:', EARTH_ORB_SPEED_WRT_SUN_THEORY)
```

    Theoretical orbital speed of the Earth around the Sun in km/s: 29.838731960667126

```python
print('Difference between actual and theoretical speed km/s:', EARTH_ORB_SPEED_WRT_SUN - EARTH_ORB_SPEED_WRT_SUN_THEORY)
```

    Difference between actual and theoretical speed km/s: 0.043043902795005806

```python

```


```python
%matplotlib inline

from typing import List

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import scipy.linalg
import scipy.ndimage
import scipy.optimize
import scipy.special
import sklearn.datasets

from chmp.ds import mpl_set, get_color_cycle, Loop
```

```python
# helper for gradient checking
def approximate_gradient(x, func, eps=1e-5):
    res = np.zeros(x.size)

    for i in range(x.size):
        d = np.zeros(x.size)
        d[i] = eps

        res[i] = (func(x + d) - func(x - d)) / (2 * eps)

    return res
```

# Linear Models for classification

## Discriminant Functions

### Fischer's Linear Discriminant

```python
n_samples = 200

y = np.random.binomial(1, p=0.5, size=n_samples)
mu = np.asarray([[-1, 0.5], [1, 0]])
sigma = np.asarray([
    [[+0.65, +0.35],
     [+0.35, +0.65]],
    [[+0.75, +0.35],
     [+0.35, +0.75]],
])

x = mu[y] + np.einsum('ij...,ij...->i...', sigma[y], np.random.normal(size=(n_samples, 2)))

simple_w = np.mean(x[y == 0], axis=0) - np.mean(x[y == 1], axis=0)
simple_w = simple_w / (simple_w @ simple_w) ** 0.5

delta_0 = x[y == 0] - np.mean(x[y == 0], axis=0, keepdims=True)
delta_1 = x[y == 1] - np.mean(x[y == 1], axis=0, keepdims=True)
S_w = delta_0.T @ delta_0 + delta_1.T @ delta_1

fischer_w = np.linalg.pinv(S_w) @ simple_w
fischer_w = fischer_w / (fischer_w @ fischer_w) ** 0.5
```

```python
c0, c1 = get_color_cycle(2)

plt.figure(figsize=(12, 4))
plt.subplot(1, 3, 1)
plt.scatter(x[y == 0, 0], x[y == 0, 1], color=c0, alpha=0.4, marker='.')
plt.scatter(x[y == 1, 0], x[y == 1, 1], color=c1, alpha=0.4, marker='.')

plt.subplot(1, 3, 2)
plt.hist(x[y == 0] @ simple_w, alpha=0.4, range=(-3, +3), bins=21, color=c0)
plt.hist(x[y == 1] @ simple_w, alpha=0.4, range=(-3, +3), bins=21, color=c1)

plt.subplot(1, 3, 3)
plt.hist(x[y == 0] @ fischer_w, alpha=0.4, range=(-3, +3), bins=21, color=c0)
plt.hist(x[y == 1] @ fischer_w, alpha=0.4, range=(-3, +3), bins=21, color=c1)
pass
```

### Perceptron Algorithm

```python
n_samples = 200

y = np.random.binomial(1, p=0.5, size=n_samples)
mu = np.asarray([[-1, 0.5], [1, 0]])
sigma = np.asarray([
    [[+0.65, +0.35],
     [+0.35, +0.65]],
    [[+0.75, +0.35],
     [+0.35, +0.75]],
])

x = mu[y] + np.einsum('ij...,ij...->i...', sigma[y], np.random.normal(size=(n_samples, 2)))
x = np.concatenate([np.ones(n_samples)[:, None], x], axis=1)

y = 2 * y - 1
```

```python
w = np.random.normal(loc=1e-2, size=3)
errors: List[float] = []

for loop, _ in Loop.over(range(10)):
    idx = np.arange(n_samples)
    np.random.shuffle(idx)
    for i in idx:
        if y[i] * np.sign(x[i, :] @ w):
            w += y[i] * x[i, :]

        errors += [np.mean(y * np.sign(x @ w) + 1) / 2]
    print(f'{loop}'.ljust(20), end='\r')
```

    [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿ 0.20s / 0.20s]

```python
plt.plot(errors, '.', alpha=0.1)
plt.plot(scipy.ndimage.gaussian_filter(errors, 20), '-')
mpl_set(ylabel='Accuracy', xlabel='Iteration')
```

## Probabilistic Generative Models

Assume probability of data given class $p(x|C_k)$. This results in the posterior:

$$
\begin{align}
    p(C_k|x)
        &= \frac{p(x|C_k) p(C_k)}{\sum_k p(x|C_k) p(C_k)} \\
        &= \frac{e^{a_k(x)}}{\sum_k e^{a_k(x)}}
\end{align}
$$

with $a_k(x) = \log p(x|C_k)  + \log p(C_k)$. For two classes this can be written as:

$$
\begin{align}
    p(C_1|x) &= \sigma(a(x)) \\
    a(x) &= c_1(x) - c_0(x)
\end{align}
$$

For example for continuous features $p(x|C_k)$ could be modeled as a Gaussian:

$$
    a(x) =
        -\frac{1}{2} (x - \mu_1)^T \Sigma_1 (x - \mu_1) +
        \frac{1}{2} (x - \mu_2)^T \Sigma_2 (x - \mu_2) +
        \frac{1}{2} \log \frac{|\Sigma_1|}{|\Sigma_2|} +
        \log \frac{p(C_1)}{p(C_0)}
$$

To fit, parametrize $p(C_1) = \pi, p(C_0) = 1 - \pi$ and optimize the joint loglikelihood $p(x|C_k)p(C_k)$. Then use Bayes theorem to obtain class probabilities.

To reduce the number of parameters for discrete features use Naive Bayes approximation:

$$
\begin{align}
    p(\{x\}|C_k)
        &= \sum_i p(x_i|C_k) &\; \\
        &= \sum_i \mu_{ik}^{x_i} (1 - \mu_{ik})^{1 - x_i} &&\text{for binary $x_i$}
\end{align}
$$

This results in

$$
\begin{align}
    a(\{ x \}) &=
        \sum_i x_i \log \frac{\mu_{i1} (1 - \mu_{i0})}{\mu_{i0} (1 - \mu_{i1})} +
        \sum_i \log \frac{1 - \mu_{i0}}{1 - \mu_{i1}} +
        \log \frac{p(C_1)}{p(C_0)}
\end{align}
$$

## Probabilistic Discriminative Models

Instead of modelling the data and using Bayes theorem. Approximate the class probabilities directly by using the functional form.

### Logistic Regression

Take $y \in \{-1, +1\}$. Then, the negative loglikelihood is given by:

$$
\begin{align}
    NLL
        &= -\sum_n \log \sigma(y_n x_n^T w) + \frac{1}{2} \lambda w^T w
\\
    \frac{\partial}{\partial w_\alpha} NLL
        &= -\sum_n \sigma(-y_n x_n^T w) y_n x_{n\alpha} + \lambda w_\alpha
\\
    \frac{\partial^2}{\partial w_\alpha \partial w_\beta} NLL
        &= \sum_n \sigma(y_n x_n^T w) \sigma(-y_n x_n^T w) x_{n\alpha} x_{n\beta} + \lambda
\end{align}
$$

```python
def fit_logistic_regression(x, y, reg=0.0):
    N, M = x.shape
    y = 2 * y - 1

    w0 = np.random.uniform(low=-1e-2, high=+1e-2, size=M)

    def loss(w):
        return -np.mean(np.log(sigmoid(y * (x @ w)))) + (0.5 * reg / N) * (w @ w)

    def grad(w):
        return (
            -np.mean(sigmoid(-y * (x @ w))[:, None] * y[:, None] * x, axis=0) +
            reg / N * w
        )

    res = scipy.optimize.minimize(loss, w0, jac=grad, method='BFGS')
    if not res.success:
        raise ValueError(f'could not fit: {res.message}')

    return res.x

def sigmoid(x):
        p = x > 0
        s = 1.0 / (1.0 + np.exp(-np.abs(x)))
        return p * s + (1 - p) * (1 - s)
```

```python
# do not shuffle to see posterior structure more clearly
x, y = sklearn.datasets.make_classification(random_state=42, shuffle=False)
w_logit = fit_logistic_regression(x, y)
w_logit_reg = fit_logistic_regression(x, y, reg=1e-3)

print('likelihood (no reg)', np.prod(sigmoid((2 * y - 1) * (x @ w_logit))))
print('likelihood (w/ reg)', np.prod(sigmoid((2 * y - 1) * (x @ w_logit_reg))))
```

    likelihood (no reg) 0.999652286703
    likelihood (w/ reg) 0.979098237265

### Probit regression

Assume noisy threshold model

$$
\begin{align}
    p(t = 1 | x)
        &= \mathbb{E}_\theta \left[ w^T x > \theta \right] \\
        &= \int_{-\infty}^{+\infty}\mathrm{d}\theta\; p(\theta) \mathbb{1}\left[ w^T x > \theta \right] \\
        &= \int_{-\infty}^{+w^T x}\mathrm{d}\theta\; p(\theta)
\end{align}
$$

This model is called probit regression for $\theta \sim \mathcal{N}(0, 1)$.

```python
def fit_probit_regression(x, y):
    N, M = x.shape
    y = 2 * y - 1

    w0 = np.random.uniform(low=-1e-2, high=+1e-2, size=M)

    def loss(w):
        return -np.mean(np.log(probit(y * (x @ w))))

    def grad(w):
        arg = y * (x @ w)
        scalar = probit_grad(arg) / probit(arg) * y
        return -np.mean(scalar[:, None] * x, axis=0)

    res = scipy.optimize.minimize(loss, w0, jac=grad, method='BFGS')
    if not res.success:
        raise ValueError(f'could not fit: {res.message}')

    return res.x

def probit(u):
    # NOTE the definition of the error function between Bishop and scipy differ
    return 0.5 * (1 + scipy.special.erf(u / np.sqrt(2)))

def probit_grad(u):
    return np.exp(-u ** 2.0 / 2) / (np.sqrt(2 * np.pi))
```

```python
u = np.linspace(-5, +5, 100)

plt.figure(figsize=(8, 4))
plt.subplot(1, 2, 1)
plt.plot(u, probit(u), label='probit function')
plt.plot(u, sigmoid(u), label='sigmoid function')
plt.legend(loc='best')

plt.subplot(1, 2, 2)
plt.plot(u, np.asarray([approximate_gradient(np.asarray([x]), probit) for x in u]), label='probit deriv.')
plt.plot(u, probit_grad(u), ls='--', c='k')

plt.plot(u, np.asarray([approximate_gradient(np.asarray([x]), sigmoid) for x in u]), label='sigmoid deriv.')
plt.plot(u, sigmoid(u) * (1 - sigmoid(u)), ls='--', c='k')

plt.legend(loc='best')
pass
```

```python
w_probit = fit_probit_regression(x, y)
print('probit likelihood', np.prod(probit((2 * y - 1) * (x @ w_probit))))
```

    probit likelihood 0.999933414634

```python
pd.DataFrame({'probit': w_probit, 'logit': w_logit}).plot.bar()
plt.title('Comparision Probit / Logistic regression coefficients')
pass
```

## Laplace Approximation

Talyor expand the log probability $p(z)$ around its maximum $\partial_{z} \log p(z) \big|_{z_0} = 0$.
This results in:

$$
\begin{align}
    \log p(z)
    &\approx
        \log p(z_0) +
        \frac{1}{2} (z - z_0)_\alpha (z - z_0)_\beta \;
        \left[ \frac{\partial^2}{\partial z_\alpha \partial z_\beta} \log p(z) \right]_{z=z_0}
\\
    &=
        \log p(z_0) - \frac{1}{2} (z - z_0)^T A (z - z_0)
\\
    &\approx
        \mathcal{N}(z|z_0, A^{-1})
\\
    A_{\alpha\beta} &= -\left[ \frac{\partial^2}{\partial z_\alpha \partial z_\beta} \log p(z) \right]_{z=z_0}
\end{align}
$$

Note: As a result of the central limit theorem, the posterior distribution for a model is expected to become increasingly better approximated by a Gaussian as the number of observed data points is increased, and so we would expect the Laplace approximation to be most useful in situations where the number of data points is relatively large. (p. 216).

The same approach can be used to estimate the normalization constant of the data likelihood:

$$
\begin{align}
    \log p(\mathcal{D})
    &=
        \log p(\mathcal{D}|\theta) + \log p(\theta) -
        \log \int \mathrm{d}\theta\;  p(\mathcal{D}|\theta) p(\theta)
\\
    &\approx
        \log p(\mathcal{D}|\theta_\mathrm{MAP}) + \log p(\theta_\mathrm{MAP}) +
        \frac{M}{2} \log 2\pi - \frac{1}{2} \log |A|
\\
    A_{\alpha\beta} &=
        -\left[
            \frac{\partial^2}{\partial \theta_\alpha \partial \theta_\beta}
                \log p(\mathcal{D}|\theta) p(\theta)
        \right]_{\theta = \theta_\mathrm{MAP}}
\end{align}
$$

## Bayesian Logistic Regression

$$
\begin{align}
    p(w) &= \mathcal{N}(w|0, \sigma_0^2)
\\
    p(y|w, x) &= \sigma(y w^T x)
\\
    \log p(w|\left\{y, x \right\})
    &=
        \sum_n \log \sigma (y_n w^T x_n) -
        \frac{1}{2} \sigma_0^{-2} w^T w +
        \mathrm{const}
\\
    &\approx
        \mathcal{N}(w|w_\mathrm{map}, A^{-1})
\end{align}
$$

Identify $\sigma_0^{-2} = \lambda$

```python
N, M = x.shape
```

```python
prior_var = 100
w_map = fit_logistic_regression(x, y, reg=1.0 / prior_var)
```

```python
arg = (2 * y - 1) * (x @ w_map)
hessian = 1 / prior_var + np.sum(
    (sigmoid(arg) * sigmoid(-arg))[:, None, None] * x[:, None, :] * x[:, :, None],
    axis=0
)
```

```python
posterior_var = np.linalg.pinv(hessian)

# NOTE: imaginary values can appear to due to numerical inaccuracies
posterior_std = np.real(scipy.linalg.sqrtm(posterior_var))

# draw 10 posterior samples
posterior_sample = w_map[None, :] + np.random.normal(size=[10, M]) @ posterior_std
```

```python
plt.imshow(posterior_var, origin='lower')
plt.xticks(range(0, 20, 2))
plt.yticks(range(0, 20, 2))
plt.title('posterior variance')
pass
```

```python

```

```python

```


# Algorithms Exercise 2

## Imports

```python
%matplotlib inline
from matplotlib import pyplot as plt
import seaborn as sns
import numpy as np
```

## Peak finding

Write a function `find_peaks` that finds and returns the indices of the local maxima in a sequence. Your function should:

* Properly handle local maxima at the endpoints of the input array.
* Return a Numpy array of integer indices.
* Handle any Python iterable as input.

```python
def find_peaks(a):
    """Find the indices of the local maxima in a sequence."""
    # YOUR CODE HERE
    #I always start with an empty list k.
    k=[]
    for i in range(0, len(a)):
        #Check to see if the number in index i is greater than the numbers in the adjacent indicies, whilst being in range of the list.
        if (i==len(a)-1 or a[i]>a[i+1]) and a[i]>a[i-1]:
            k.append(i)
    return np.array(k)
```

```python
p1 = find_peaks([2,0,1,0,2,0,1])
assert np.allclose(p1, np.array([0,2,4,6]))
p2 = find_peaks(np.array([0,1,2,3]))
assert np.allclose(p2, np.array([3]))
p3 = find_peaks([3,2,1,0])
assert np.allclose(p3, np.array([0]))
```

Here is a string with the first 10000 digits of $\pi$ (after the decimal). Write code to perform the following:

* Convert that string to a Numpy array of integers.
* Find the indices of the local maxima in the digits of $\pi$.
* Use `np.diff` to find the distances between consequtive local maxima.
* Visualize that distribution using an appropriately customized histogram.

```python
from sympy import pi, N
pi_digits_str = str(N(pi, 10001))[2:]
```

```python
# YOUR CODE HERE
h = pi_digits_str
j=[]
for i in h:
    j.append(int(i))
n = np.array(j)
v = find_peaks(n)
m = np.diff(v)

f = plt.figure(figsize=(10,6))
plt.hist(m, bins=20)
plt.ylabel('Distance between maxima')
plt.xlabel('Index of maxima')
m
```

```python
assert True # use this for grading the pi digits histogram
```


### Mathematical equations in Latex

\begin{equation}
% % sign is used to write comments in latex
% \\ sign is used to enter new line
x = y+z\\
f(x) = x^2\\
f(x) = x_1 + x_2 + x_3 + .......... + x_n\\
f(x) = \sum_{i=1}^{n}{x_i}\\
f(x) = \int_{i=1}^{n}{x_i}
\\
X=
\begin{cases}
5, \text{if x is divisible by 5}
\\
10, \text{if x is divisible by 10}
\\
-1, \text{otherwise}
\end{cases}
\\
x=
\frac{\substack{\sum_{i=1}^{n}{x_i}}}
{\substack{\sum_{i=20}^{50} {x_j}}}
\end{equation}

Let's begin with a formula $e^{i\pi}+1=0$ But we can also do

$$ e = \lim_{n\to\infty} \left(1+\frac{1}{n}\right)^n =
    \lim_{n\to\infty}\frac{n}{\sqrt[n]{n!}}$$

```python

```

```python

```


```python
from sympy import *
from sympy.utilities.lambdify import lambdify, implemented_function
from functools import lru_cache
import sys
import math
init_printing()
n = symbols('n',integer=True)

```

```python
initial_positive_conditions = {0:0, 1:1, 2:1}

def makePositiveTribonacci(maxsize = None):
    @lru_cache(maxsize)
    def calcPositiveTribonacci(n):
        return calcPositiveTribonacci(n-1)  + calcPositiveTribonacci(n-2) + calcPositiveTribonacci(n-3) if n not in initial_positive_conditions else initial_positive_conditions[n]
    return calcPositiveTribonacci

positiveTribonacci = makePositiveTribonacci(None)
PT = implemented_function(Function('T'), positiveTribonacci)

```

```python
initial_negative_conditions = {0:0, -1:0, -2:1}
def makeNegativeTribonacci(maxsize=None):
    @lru_cache(maxsize=maxsize)
    def calcNegativeTribonacci(n):
        return calcNegativeTribonacci(n+3)  - calcNegativeTribonacci(n+2) - calcNegativeTribonacci(n+1) if n not in initial_negative_conditions else initial_negative_conditions[n]
    return calcNegativeTribonacci
negativeTribonacci = makeNegativeTribonacci(maxsize = None)
NT = implemented_function(Function('NT'), negativeTribonacci)
lambda_negative_tribonacci = lambdify(n, NT(n))
[(n*-1, lambda_negative_tribonacci(n*-1)) for n in range (0,11)]
```

```python
def tribonacci(n):
    if n>=0:
        return positiveTribonacci(n)
    elif n<0:
        return negativeTribonacci(n)
T = implemented_function(Function('T'), tribonacci)
lambda_tribonacci = lambdify(n, T(n))
[(n, lambda_tribonacci(n)) for n in range(-40, 40)]
```

```python
limit = 8
k = symbols('k', integer = True)
q = Product(T(k), (k,n,limit))
p = Sum(q/T(k), (k,n,limit))
t = T(n)-T(n-1)+1
calc_t = Lambda([n], t.doit())
calc_p = Lambda([n], p.doit())
calc_q = Lambda([n], q.doit())
calc_expression = Lambda([n], ((t.doit() * p.doit())-q.doit()).doit())
print(type(N(calc_expression(36), 0)))
(int(calc_t(36).evalf())*int(calc_p(36).evalf())-int(calc_q(36).evalf()))
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```


```python
%matplotlib inline
```

```python
from IPython.display import display

import numpy
import matplotlib
from numpy import linspace
from sympy import lambdify, init_printing
from sympy import symbols, pi, sqrt, exp, sin

from matplotlib import pyplot

matplotlib.rcParams['axes.grid'] = True
matplotlib.rcParams['grid.color'] = 'k'
matplotlib.rcParams['grid.linestyle'] = ':'
matplotlib.rcParams['grid.linewidth'] = 0.5

init_printing()
```

# Logistic function

$$ f(x) = \frac{L}{1 + e^{-k(x-x_0)}} $$

where
- $L$ is the maximum value of the function
- $x_0$ is the midpoint
- $k$ is the stepness of the curve at the midpoint
- $e$ is the natural logarithm

We will simplify now the function by setting $L=1$, $k=1$ and $x_0=0$.

$$ f(x) = \frac{1}{1 + e^{-x}} $$

```python
from sympy import simplify

s_x = symbols('x')

logistic = 1/(1 + exp(-s_x))
display(logistic)
```

```python
logistic_f = lambdify((s_x), logistic, "numpy")

a = linspace(-5,5,100)
pyplot.plot(a, logistic_f(a))
```

# Samples

Lets assume that the logistic regression defines a Bernoulli random variable $Y$ for each variable $X$. This can be interpreted as the posterior probabilities

$$ P(Y|X) = f(x) = \frac{1}{1 + e^{-x}} $$

```python
N = 1000
# x = numpy.random.randn(N, 1)
x = (numpy.random.random(N) - 0.5)*4
y = numpy.zeros((N, 1))
for i in range(N):
    y[i] = numpy.random.binomial(1, logistic_f(x[i]), 1)

pyplot.scatter(x, y, alpha=0.1)
```

```python
import sklearn
from sklearn.linear_model import LogisticRegression
# TODO change the implementation from Sklearn for my own

LR = LogisticRegression()
LR.fit(x.reshape(-1,1), y)

a = linspace(-5,5,100).reshape(-1,1)
pyplot.plot(a, logistic_f(a), label='logistic')
pyplot.plot(a, LR.predict_proba(a)[:,1], label='Logistic Regression')
pyplot.legend()
```

# Estimation problems

We may be interested on solving different problems

1. Classification problem: only interested on the most probable class (mostly applied in AI)
2. Class probability estimation: approximate the conditional posterior probability (mostly applied in statistics)

## Classification

Interested in reducing the **classification error** but difficult to use as an optimization method because of the properties of this type of functions. For that reason, it is common to use **surrogate functions**.

Some examples of surrogate functions for classification are:

- Log-loss: $L(y|q) = -\log(q^y(1-q)^{1-y} = -y \log(q) - (1-y) \log(1-q)$
- Squared error loss: $L(y|q) = (y-q)^2 = y(1-q)^2 + (1-y)q^2$

Log-loss is the negative log-likelihood of the Bernoulli model.

These losses are Fisher consistent estimates of $\eta$ because

$$\eta = argmin_{q\in [0,1]} E_y L(y|q) \text{, for }y \sim Bernoulli(\eta)$$

And are know as **proper scoring rules**

# TODO understand why the previous definition is correct given that

sometimes the argmin is not able to find the true conditional posterior probabilities

```python
# TODO understand why the previous definition is correct given that

sinoidal = sin(s_x)/2+0.5
sinoidal_f = lambdify((s_x), sinoidal, "numpy")

pyplot.plot(a, sinoidal_f(a))

x = (numpy.random.random(N)-0.5)*10
for i in range(N):
    y[i] = numpy.random.binomial(1, sinoidal_f(x[i]), 1)

pyplot.scatter(x, y, alpha=0.1)

LR = LogisticRegression()
LR.fit(x.reshape(-1,1), y)

a = linspace(-5,5,100).reshape(-1,1)
pyplot.plot(a, LR.predict_proba(a)[:,1], label='Logistic Regression')
pyplot.legend()
```

```python

```

```python

```


# Monkeys with Typewriters

The premise is: a monkey randomly pushes the keys of a typewriter for an infinite length of time. At some point it will have typed any possible sequence of letters, including Shakespeare's collected works. Perhaps even Goethe's.

## Proof that Immortal Monkeys are Literary Geniuses

I have mixed feelings about the fact that I have arrived at this point in life. Also: aren't lobsters immortal? Can they type? If so, should they receive every Nobel Prize?

Once again, inspired by https://mast.queensu.ca/~stat455/. The key is to construct a lower bound on the probability by looking at events that happen independently and therefore make for easy calculations.

Let $M$ be the set of $m$ keys on the typewriter. Let $\{N|N\in M^n\}$ be the set of possible strings of length $n$, and let $S$ be Shakespeare's works of length $s$.

Every $N\in\{N\}$ has $n-s+1$ consecutive overlapping substrings of length $s$. Let $A_i$ denote the event that $S$ is contained in the $i$th substring. The event $E_n$ that Shakespeare's works are in one or more of the substrings of $N$ is:

\begin{equation}
E_n = \bigcup_{i=1}^{n-s+1} A_i
\end{equation}

The difficulty here is that the probabilities of overlapping substrings containing $S$ are not independent. However, we can select a set of up to $\lfloor \frac{n}{s}\rfloor$ non-overlapping substrings from the expection for $E_n$ and define the event $E_n'$ that none of the selected substrings contain $S$:

\begin{equation}
E_n' = \bigcup_{i=1}^{\lfloor\frac{n}{s}\rfloor} A_{1+j+i\cdot s} = \left[\bigcap_{i=1}^{\lfloor\frac{n}{s}\rfloor} A^c_{1+j+i\cdot s}\right]^c
\end{equation}

Where $j$ can be chosen arbitrarily from $j\in (0,...,n\ \mathrm{mod}\ s)$ and De Morgan's rule was used to transform unions to intersections. Now clearly $E_n' \subset E_n$ for any choice of $j$ ($j$ has the function of shifting), and therefore $p(E_n')\leq p(E_n) \leq 1$. Since the events $A_{1+j+i\cdot s}$ are independent for any $i,j$, $A^c_{1+j+i\cdot s}$ are also independent. Consequently:

\begin{equation}
p(E_n') = 1-p\left(\bigcap_{i=1}^{\lfloor\frac{n}{s}\rfloor} A^c_{1+j+i\cdot s}\right) = 1-\prod_{i=1}^{\lfloor\frac{n}{s}\rfloor}p\left(A^c_{1+j+i\cdot s}\right)
\end{equation}

The probability of a particular substring not containing S is $p\left(A^c_{i}\right) = \frac{1}{m^s} \leq 1$. Then, as $n$ approaches infinity:

\begin{equation}
\lim_{n \rightarrow \infty} p(E_n')= \lim_{n \rightarrow \infty} 1-\prod_{i=1}^{\lfloor\frac{n}{s}\rfloor}\frac{1}{m^s}=1 \leq p(E_n) \leq 1
\end{equation}

## How long it will take

What's dissatisfying about this is of course that it doesn't say how long one might have to wait until Shakespeare is typed out!

In the other notebook (```Coinflips - Expected Flips until Sequence of n Heads```) I calculated how long it take on average to flip a coin until a sequence of $n$ heads appears. The calculation is easy because there is no overlap between a wrong flip at the end of a sequence and the beginning of the sequence -- the moment tails is flipped, the process starts over. This does not necessarily have to be the case with Shakespeare. However unlikely, whenever part of the beginning of the sequence is repeated somewhere within the sequence, then there is a small chance that the monkey does not start over from the beginning if he hits a wrong key.

Example: 'to be or not to be t[hat]'

Instead of a 't', the monkey types an 'o': '[to be or not ]to be o'

At that point, despite the mistake, the monkey has a higher chance of typing out the full sequence, because it's already typed the first 7 letters correctly: 'to be o'.

Ignoring these shortcuts makes it easy to calculate an upper bound on the expected number of key strokes that it takes the monkey until it has typed Shakespeare for the first time:

\begin{equation}
\mathbb{E}[n] = \left[\frac{m-1}{m}\sum_{k=0}^{s-1}\frac{k+1}{m^{k}} + \frac{1}{m^s}s\right]\left[1-\frac{m-1}{m}\sum_{k=0}^{s-1}\frac{1}{m^{k}}\right]^{-1}
\end{equation}

Google tells me that a standard American typewriter has 44 keys and 88 characters. I will assume that hitting the right keys is enough and pretend that capslock doesn't work and isn't necessary to type Shakespeare. Then $m=44$. (Considering the problem with capslock is very interesting).

According to https://www.opensourceshakespeare.org/views/plays/plays_numwords.php there are 835,997 words in all of Shakespeare's plays. Google also tells me that the average English word has 4.7 letters (which I doubt is true for Shakespeare, but whatever). So call it 4,000,000 characters.

Evaluating this expression will run into numerical issues within a first sentence. Simulating will be impossible. Which is to say, I have to do what I didn't do last time, and simplify it. Using the expression for the finite geometric series $\sum_{i=0}^{s-1}\frac{1}{m^k} = \frac{m}{m^s}\frac{m^s-1}{m-1}$.

Simplifying the expression "slightly":

\begin{equation}
\mathbb{E}[n] = m^s + s - 1 + \frac{m-1}{m}\sum_{i=0}^{s-1}\frac{k}{m^k}
\end{equation}

I don't know an expression for the finite sum $\sum_{i=0}^{s-1}\frac{k}{m^k}$ but below is a digression on the infinite sum. The relevant part is that the $m^s$ term easily dominates the expression, so that:

\begin{equation}
\mathbb{E}[n] \approx m^s = e^{s \ln m} \approx e^{15,136,758} \approx 10^{6573810}
\end{equation}

Fortunately that's just an upper bound!

(If you are worried that the sum might blow up into something huge, keep reading... I found an expression for the infinite sum, which must be larger than the finite sum, and the values are pretty small.)

### Digression below:

An expression for the infinite sum $\sum_{i=0}^{\infty}\frac{k}{m^k}$ can be found thanks to this monster from Gradsteyn and Rhyzik (http://fisica.ciens.ucv.ve/~svincenz/TISPISGIMR.pdf), a book that was introduced to me by the same professor who had me up all night coding in FORTRAN.

In G&R, the asterisk denotes a new result. Fancy. You noticed correctly that I was too lazy to type that out. In that expression, replace $b=m$ and $a=1$. Then:

\begin{equation}
\sum_{i=0}^{\infty}\frac{k}{m^k} = 0 + \sum_{i=1}^{\infty}\frac{k}{m^k} = \frac{1}{(m-1)^2}
\end{equation}

But that can't be right! For example, if $m=2$, then $\sum_{i=0}^{\infty}\frac{k}{m^k} = \frac{1}{2} + \frac{2}{4} + \frac{3}{8} + ... \geq 1$, but $\frac{1}{(2-1)^2} = 1$.

Days later, I receive my nerd oscar. Dream big, everybody:

```python
import numpy as np
import matplotlib.pyplot as plt

def factorial(x):
    if x == 0:
        return 1
    else:
        res = 1
        for i in range(1,x+1):
            res *= i
        return res

def GnR(a,b):
    """
    Corrected Gradshteyn & Rhyzik
    """
    prefactor = b**a/(b-1)**(a+1)

    def jsum(i):
        return np.sum(
            [(-1)**j * factorial(a+1) * (i - j)**a /(factorial(j) * factorial(a+1-j)) for j in range(0,i+1)]
        )

    def isum(a):
        return np.sum([(1/b**(a-i))*jsum(i) for i in range(1,a+1)])

    return prefactor*isum(a)

def series(a,b,N=100):
    """
    approximate the series using the first N terms
    """
    s = 0
    for i in range(N+1):
        s += (i**a)/(b**i)
    return s

mm = list(range(2,50))

plt.figure(figsize=(12,8))

aa = [1,2,3,4]
for a in aa:
    plt.semilogy(mm,[GnR(a,m) for m in mm],'.',markersize=12)
    plt.semilogy(mm,[series(a,m) for m in mm],'-')

plt.legend(np.hstack([['G&R a = %i' % a,'Apprx. a = %i' % a] for a in aa]))

_ = plt.title('Corrected Gradhsteyn & Rhyzik vs. Approximation')
```

Probability - Monkeys with Typewriters.ipynb