# Hands On Scipy Tutorial
**Tutors:** Axel Donath

This hands-on tutorial gives an introduction and overview of the [Scipy](http://www.scipy.org/) Python package. It based on the [excellent Scipy tutorial](https://github.com/Asterics2020-Obelics/School2019/blob/master/scipy/scipy_general.ipynb) by Maximilian Noethe and Kai Brügge from the past Asterics software summer school 2019.

## Preface
We recommend to follow this tutorial by **executing the code cells on your local machine**, along with the tutor. Every sub-topic we will cover in this tutorial will be concluded by a few **exercises** with different levels of difficulty** (*easy*, *advanced* and *hard*). The exercises will not be discussed during the course, however we provide a **sample solution** for all exercises in the [solutions folder](astropy_hands_on_solutions.ipynb) you can look at **after the course**. You can always ask for help on the exercises on the corresponding Slack channel **#d05-fri11-scipy**.

The estimated time for this tutorial is ~1.5 hours. We have marked some of the sections that deal with more advanced topics as "optional".

We're happy to receive any **feedback or questions** on the tutorial via mail to *axel.donath@mpi-hd.mpg.de* or using the
repository's [issue tracker](https://github.com/escape2020/school2021/issues).

The SciPy framework builds on top of the low-level NumPy framework for multidimensional arrays, and provides a large number of higher-level scientific algorithms. Some of the topics that SciPy covers are:

* Special functions ([scipy.special](http://docs.scipy.org/doc/scipy/reference/special.html))
* Integration and ODE  ([scipy.integrate](http://docs.scipy.org/doc/scipy/reference/integrate.html))
* Optimization ([scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html))
* Interpolation ([scipy.interpolate](http://docs.scipy.org/doc/scipy/reference/interpolate.html))
* Fourier Transforms ([scipy.fftpack](http://docs.scipy.org/doc/scipy/reference/fftpack.html))
* Signal Processing ([scipy.signal](http://docs.scipy.org/doc/scipy/reference/signal.html))
* Linear Algebra ([scipy.linalg](http://docs.scipy.org/doc/scipy/reference/linalg.html))
* Sparse Eigenvalue Problems ([scipy.sparse](http://docs.scipy.org/doc/scipy/reference/sparse.html))
* Statistics ([scipy.stats](http://docs.scipy.org/doc/scipy/reference/stats.html))
* Multi-dimensional image processing ([scipy.ndimage](http://docs.scipy.org/doc/scipy/reference/ndimage.html))
* File IO ([scipy.io](http://docs.scipy.org/doc/scipy/reference/io.html))

Each submodule in SciPy  provides a number of functions and classes that can be used to solve problems in their respective domain.

## 0. Setup

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

rng = np.random.default_rng(42)
```

## 1. Linear algebra

The linear algebra module contains a lot of matrix related functions, including linear equation solving, eigenvalue solvers, matrix functions (for example matrix-exponentiation), a number of different decompositions (SVD, LU, cholesky), etc.

Detailed documetation is available at: http://docs.scipy.org/doc/scipy/reference/linalg.html

Here we will look at how to use some of these functions:

### 1.1 Linear equation systems

Linear equation systems on the matrix form

$A x = b$

where $A$ is a matrix and $x,b$ are vectors can be solved like:

```python
from scipy import linalg
```

```python
A = (np.arange(100) + rng.normal(0, 10, size=(100))).reshape(10, 10)
b = np.arange(10)
A.shape, b.shape
```

    ((10, 10), (10,))

```python
plt.matshow(A)
plt.grid(False) # turn off the grid for this plot
```

```python
x = linalg.solve(A, b)
x
```

    array([ 0.0298923 ,  0.00868663,  0.0258469 ,  0.01367623,  0.01353663,
            0.02030362, -0.05241789,  0.05267412,  0.01826446, -0.03321952])

```python
# check whether we found the correct x. All entries should be close to zero.
np.dot(A, x) - b
```

    array([-2.22044605e-16,  0.00000000e+00, -4.44089210e-16, -8.88178420e-16,
           -8.88178420e-16, -8.88178420e-16, -2.66453526e-15, -8.88178420e-16,
           -4.44089210e-15,  0.00000000e+00])

We can also do the same with

$A X = B$

where $A, B, X$ are matrices:

```python
A = rng.uniform(size=(5, 5))
B = rng.uniform(size=(5, 5))
f, [ax1, ax2] = plt.subplots(1, 2)
ax1.matshow(A)
ax2.matshow(B)
ax1.grid(False)
ax2.grid(False)
ax1.set_title("Matrix A")
ax2.set_title("Matrix B");
```

```python
X = linalg.solve(A, B)
X
```

    array([[ 4.26661683,  0.97303145,  0.21291271,  4.38475543,  3.60122584],
           [-0.52762202, -0.69149307,  0.15805858,  0.11202386,  0.02839992],
           [-1.65294528, -0.62130048,  0.64404986, -2.65960636, -1.57717235],
           [ 4.4992729 ,  1.95322224, -0.05940088,  5.89156964,  6.56577378],
           [-5.3781515 , -0.8935471 , -0.18052208, -7.24626002, -7.87467682]])

```python
# check
linalg.norm(np.dot(A, X) - B)
```

    1.1666631218766921e-15

### 1.2 Eigenvalues and eigenvectors

The eigenvalue problem for a matrix $A$:

$\displaystyle A v_n = \lambda_n v_n$

where $v_n$ is the $n$th eigenvector and $\lambda_n$ is the $n$th eigenvalue.

To calculate eigenvalues of a matrix, use the `eigvals` and for calculating both eigenvalues and eigenvectors, use the function `eig`. Both return complex numbers.

```python
A = rng.uniform(size=(4, 4))

eigen_values = linalg.eigvals(A)
eigen_values
```

    array([ 2.03197191+0.j        ,  0.32055586+0.j        ,
           -0.25719529+0.28106828j, -0.25719529-0.28106828j])

```python
values, vectors = linalg.eig(A)
values, vectors
```

    (array([ 2.03197191+0.j        ,  0.32055586+0.j        ,
            -0.25719529+0.28106828j, -0.25719529-0.28106828j]),
     array([[-0.40772097+0.j        , -0.66263393+0.j        ,
              0.69076099+0.j        ,  0.69076099-0.j        ],
            [-0.59942711+0.j        ,  0.1547345 +0.j        ,
              0.05160407+0.2579825j ,  0.05160407-0.2579825j ],
            [-0.34154935+0.j        ,  0.61135759+0.j        ,
              0.05395196-0.46534462j,  0.05395196+0.46534462j],
            [-0.5981595 +0.j        , -0.40399925+0.j        ,
             -0.45963856+0.15135145j, -0.45963856-0.15135145j]]))

There are also more specialized solvers, like the `eigh` for Hermitian matrices.

### 1.3 Exercise:  Simple Linear Algebra

1. Create a 10x10 matrix $A$ with random entries

2. Create a symmetric matrix from $A$ using its transpose (use A.T to transpose) and then plot the new matrix using `plt.matshow`

3. Use `linalg.eigh` to get the eigenvalues and vectors.

```python
# %load solutions/simple_linalg_solution.py
```

# 2. Special Functions

A large number of mathematical special functions are important for many computional physics problems. SciPy provides implementations of a very extensive set of special functions. For details, see the list of functions in the reference documention at http://docs.scipy.org/doc/scipy/reference/special.html#module-scipy.special.

- Airy functions
- Elliptic functions and integrals
- Bessel functions
- Gamma functions
- etc....

## 2.1 Polynomials

The module contains a large number of functions for evaluating polynomials and calculating their roots.

The following plot shows just a few of them.

```python
from scipy import special

functions = {
    "Chebyshev": special.eval_chebys,
    "Laguerre": special.eval_laguerre,
    "Legendre": special.eval_legendre,
    "Chebyshev II": special.eval_chebyu
}

fig, axes = plt.subplots(2, 2, sharex=True, sharey=True, figsize=(12, 8))

x = np.linspace(-2., 2., 200)

for name, ax in zip(functions, axes.flat):
    f = functions[name]
    for n in range(5):
        ax.plot(x, f(n, x), label=f"Order {n}")

    ax.set_title(name)
    ax.set_ylim([-3, 3])

plt.legend();
```

## 2.2 Statistical Functions

The module also contains a host of useful functions for statistical computations. for example the so called 'error function' `scipy.special.erf`

\begin{align}
\operatorname {erf}(x)&={\frac {1}{\sqrt {\pi }}}\int _{-x}^{x}e^{-t^{2}},dt \\ &={\frac {2}{\sqrt {\pi }}}\int _{0}^{x}e^{-t^{2}}\,dt.
\end{align}

Which allows you to get the Gaussian probability distribution
$$
\phi(z) = \frac 1 2 \left(1 + erf\left(\frac{z}{sqrt(2)}\right) \right)
$$

The module `scipy.stats` contains nicer methods to handle statistical methods. More on that later.

```python
x = np.linspace(-3, 3)
plt.plot(x, special.erf(x), label='$erf$')
plt.plot(x, 0.5 * (1 + special.erf(x / np.sqrt(2))), label='$phi$')
plt.xlabel('$x$')
plt.legend();
```

## 2.3 Bessel Functions
To demonstrate the typical usage of special functions we will look in more detail at the Bessel functions.
The `scipy.special` module includes a large number of Bessel-functions
Here we will use the functions $j_n$ and $y_n$, which are the Bessel functions
of the first and second kind and real-valued order.

Bessel functions are a family of solutions to Bessel’s differential equation with real or complex order alpha:
$$
x^2 \frac{d^2 y}{dx^2} +x  \frac{dy}{dx} +(x^2−α^2)y=0
$$

```python
from scipy import special

x = np.linspace(0, 10, 100)

for n in range(4):
    plt.plot(x, special.jn(n, x), label=f'$J_{n}(x)$')

plt.legend();
```

Among other uses, these functions arise in wave propagation problems such as the vibrational modes of a thin drum head. Here is an example of a circular drum head anchored at the edge. See https://en.wikipedia.org/wiki/Vibrations_of_a_circular_membrane

```python
# fix the radius of the drum
DRUM_RADIUS = 1

def drumhead_height(n, k, radius, theta, t):
    '''Solution to the Drumhead problem with a fixed membrane on the edge of the drum
    See https://en.wikipedia.org/wiki/Vibrations_of_a_circular_membrane
    '''
    kth_zero = special.jn_zeros(n, k)[-1] # fix the drum on the outter edge to be zero.
    return np.cos(t) * np.cos(n * theta) * special.jn(n, radius * kth_zero / DRUM_RADIUS)

# create two coordinate vectors containing all combinations of radius and theta we want to plot
r = np.linspace(0, DRUM_RADIUS, 50)
t = np.linspace(0, 2 * np.pi, 50)
radius, theta = np.meshgrid(r, t) # this creates all combinations we are interested in

x = radius * np.cos(theta)
y = radius * np.sin(theta)
```

```python
from itertools import product

fig = plt.figure(figsize=(12, 12))

idx = 1

for n in range(3):
    for k in range(1, 4):
        ax = fig.add_subplot(3, 3, idx, projection='3d')
        z = drumhead_height(n=n, k=k, radius=radius, theta=theta, t=0.5)
        ax.plot_surface(x, y, z, cmap='YlGnBu')
        ax.set_title(f"n={n}, k={k}", fontsize=14)
        ax.axis('off')
        idx += 1

```

In the `scripts` folder you find an example script that creates the following animation:

```python
from IPython.display import Video
Video('scripts/animation_membrane.mp4')
```

## 3. Ordinary Differential Equations

### 3.1 Zombie outbreak

To demonstrate the usefulness of SciPy lets try to predict what will happen to humanity in case of a zombie outbreak.  This example comes courtesy of Christopher Campo.

We will show that, in case of a Zombie outbreak, humanity is inevitably doomed.
As shown by [Phillip Munz et al](http://mysite.science.uottawa.ca/rsmith43/Zombies.pdf) we can model a simple outbreak scenario like so.

\begin{align}
\frac{S}{dt} &= B - \beta S Z - \delta S \\
\frac{Z}{dt} &= \beta S Z + \gamma R - \alpha S Z \\
\frac{R}{dt} &= \delta S + \alpha S Z - \gamma R
\end{align}

Where $S$ is the number of susceptible humans, $Z$ is the number of zombies and $R$ is the number of removed persons which have died either by natural causes or zombie attack. The human birthrate is assumed to be constant and modeled by $B$

Susceptibles can become zombies through an encounter with a zombie ($\beta$). Natural causes of human deaths are parameterized by $\delta$. Humans in the removed class can be resurrected and become zombies ($\gamma$).
Zombies can enter the removed class by cutting their heads of or removing their brains ($\alpha$)

The question I'm trying to answer is

> __Can Humanity survive the zombie Apocalypse?__

Since I don't know how to solve differential equations, I'm going to use SciPy instead.

Now we use SciPy to solve this system of differential equations. The function `solve_ivp` does the entire job for us.

If we go to [the SciPy documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.solve_ivp.html#scipy.integrate.solve_ivp) about this function we will read the following.

>This function numerically integrates a system of ordinary differential equations given an initial value:
>
> $y^{\prime} = f(t, y)$
>
>$y(t_0) = y_0$
>
>Here $t$ is a one-dimensional independent variable (time), $y(t)$ is an __n-dimensional vector-valued function__ (state), and an __n-dimensional vector-valued function f(t, y) determines the differential equations__. The goal is to find y(t) approximately satisfying the differential equations, given an initial value y(t0)=y0.

So we start by defining some starting values for the problem and the model function $f(t, y)$.

```python
from scipy.integrate import solve_ivp

# Type `solve_ivp?` to get inline documentation about the method in your notebook.
```

```python
# fixed model parameters
B = 0.1 # birth rate
delta = 0.0001 # rate of natural casues of death
beta = 0.00016 # transmission coefficent
gamma = 0.0001 # resurrection rate
alpha = 0.0001 # rate of destroyed zombies

# initial conditions
S0 = 5000.              # initial population
Z0 = 0                 # initial zombie population
R0 = 0                 # initial dead population
y0 = [S0, Z0, R0]     # initial condition vector

def f_model(t, y):
    Si, Zi, Ri = y

    # the model equations (see Munz et al. 2009)
    f0 = B - beta * Si * Zi - delta * Si
    f1 = beta * Si * Zi + gamma * Ri - alpha * Si * Zi
    f2 = delta * Si + alpha * Si * Zi - gamma * Ri
    return [f0, f1, f2]

```

Now we solve the equation and output the result

```python
t_start, t_end = 0, 150  # 200 days of zombie apocalypse
solution = solve_ivp(f_model, (t_start, t_end), y0, t_eval=np.linspace(t_start, t_end, 200))
```

```python
s = solution.y[0, :]
z = solution.y[1, :]
plt.plot(solution.t, s, label='survivors')
plt.plot(solution.t, z, label='zombies')
plt.xlabel('t / days')
plt.ylabel('Population')
plt.legend()
```

### 3.2 Exercise: Save the world.

The ODE defined above is a very pessimistic model. There is one unstable equilibrium state if $\beta = 0$.
Your task is to save the world by adding a the notion of a _cure_ to the model above. The cure
can move individuals from the $Z$ to the $S$ group.

1. Introduce a new constant $\rho$ which models the effectivness of the cure.
2. Add a term to the ODE to model the cure.
3. Solve the ODE using `solve_ivp`

```python
# %load solutions/solution_zombies.py
```

A more complex case of coupled differential equations is given again in the `scripts` folder, which creates the following animation:

```python
Video("scripts/animation_pendulum.mp4")
```

## 4. Numerical Integration

### 4.1 Quadrature

Numerical evaluation of a function of the type

$\displaystyle \int_a^b f(x) dx$

is called *numerical quadrature*, or simply *quadature*. SciPy provides a series of functions for different kind of quadrature, for example the `quad`, `dblquad` and `tplquad` for single, double and triple integrals, respectively.

```python
from scipy.integrate import quad, dblquad, tplquad
```

The `quad` function takes a large number of optional arguments, which can be used to fine-tune the behaviour of the function.

The basic usage is as follows:

```python
# define a simple function for the integrand
def f(x):
    return x ** 2
```

```python
x_lower = 0 # the lower limit of x
x_upper = 1 # the upper limit of x

val, abserr = quad(func=f, a=x_lower, b=x_upper)

print(f'integral value={val} , absolute error={abserr}' )

x = np.linspace(-1.2, 1.2, 200)
plt.plot(x, f(x))
plt.fill_between(x, f(x), alpha=0.3, where=(x > 0) & (x < 1))
```

If we need to pass extra arguments to integrand function we can use the `args` keyword argument:

```python
def integrand(x, n):
    return special.jn(n, x)

x_lower = 0  # the lower limit of x
x_upper = 10 # the upper limit of x

val, abserr = quad(integrand, x_lower, x_upper, args=(3,))
print(val, abserr)
```

    0.7366751370811073 9.389126882496403e-13

For simple functions we can use a lambda function (name-less function) instead of explicitly defining a function for the integrand:

```python
val, abserr = quad(lambda x: np.exp(-x ** 2), -np.Inf, np.Inf)
print(f'numerical= {val}, {abserr}')
print(f'analytical solution = {np.sqrt(np.pi)}')
```

    numerical= 1.7724538509055159, 1.4202636780944923e-08
    analytical solution = 1.7724538509055159

As show in the example above, we can also use 'Inf' or '-Inf' as integral limits.

Higher-dimensional integration works in the same way:

```python
def integrand(x, y):
    return np.exp(-x**2 - y**2)

x, y = np.meshgrid(np.linspace(-2, 2, 100), np.linspace(-2, 2, 100))
z = integrand(x, y)

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1, projection='3d')
ax.plot_surface(x, y, z, cmap='YlGnBu')

# the integral boundarys can be functions when integrating in more than one dimensions
val, abserr = dblquad(integrand, -2, 2, lambda x: -2, lambda x: 2)

print(f'numerical solution= {val}, {abserr}')
```

Note how we had to pass lambda functions for the limits for the y integration, since these in general can be functions of x.

### 4.2 Exercise II:  Integrals and Special Functions

Find the circumference of an ellipse. The formula for calculating circumference $C$ is as follows.

$$
\frac{C}{4a} =  E(\epsilon)=\int _{0}^{\pi /2}{\sqrt {1-\epsilon^{2}\sin ^{2}\theta }}\ d\theta
$$

where $\epsilon$ is the eccentricity of the ellipse given by the length and width of the ellipse.

$$
\epsilon = \sqrt{1 - \frac{b^2}{a^2}}
$$

compare the solution found using `quad` with the appropriate function from the `scipy.special` package.

```python
# %load solutions/solution_2.py
```

## 5. Signal Processing

SciPy offers a large amount of utilities to handle time series data. However the methods in the `scipy.signal` are useful for much more than just time series data.

### 5.1 Convolution

The convolution of two continuos functions is defined by a simple integral transform.

\begin{aligned}
(f*g)(t)&\,{\stackrel {\mathrm {def} }{=}}\ \int _{-\infty }^{\infty }f(\tau )g(t-\tau )\,d\tau \\&=\int _{-\infty }^{\infty }f(t-\tau )g(\tau )\,d\tau .
\end{aligned}

Convolution happens whenever data gets smeared out by a detector or filter of any kind.

```python
from scipy import signal

# create two rectangular signals
f = np.repeat([0., 1., 0.], 120)
g = np.repeat([0., 1., 0.], 20)

result = signal.convolve(f, g, mode='same')  # same output shape as input, see docs
```

```python
fig, (ax_orig, ax_win, ax_filt) = plt.subplots(3, 1, sharex=True, figsize=(6, 6))

ax_orig.plot(f, color='gray')
ax_orig.set_title('Original pulse')
ax_orig.margins(0, 0.1)

ax_win.plot(g, color='gray')
ax_win.arrow(70, 0.5, 50.0, 0.0, width=0.08, head_length=8)
ax_win.set_title('Filter impulse response')

ax_filt.plot(result)
ax_filt.set_title('Filtered signal');
```

The naive algorithm for producing discrete convolutions is not very efficient. It takes $\mathcal{O}(n^2)$ steps to get a result. A more efficient implementation is the FFT based alogrithm. It is based on the *convolution theorem*

> The convolution theorem states that
>
 $$
 \mathcal{F}\{f*g\}=k\cdot {\mathcal {F}}\{f\}\cdot {\mathcal {F}}\{g\}
 $$
>
> where
> $\mathcal {F}\{f\}$ denotes the Fourier transform of $f$, and $\mathcal {F}\{g\}$ the Fourier transform of $g$.
>
> https://en.wikipedia.org/wiki/Convolution

SciPy uses a heuristic to automagically find the [fastest convolution method](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.choose_conv_method.html).

### 5.2 Signal Windows

SciPy conveniently provides a handful of common signal windows ready to be used for convolution or other filtering operations. All of them can be found in `scipy.signal.window` but their are also available in the `scipy.signal` name space.

```python
windows = {
    "Nutall": signal.nuttall(150),
    "Hannings": signal.hann(150),
    "Cosine": signal.cosine(150),
    "Triangle": signal.triang(150),
    "Blackman": signal.blackman(150)
}

for name, window in windows.items():
    plt.plot(window, label=name)

plt.legend();
```

### 5.3 Correlation

The correlation of two functions $f$ and $g$ looks similar to a convolution

> For continuous functions $f$ and $g$, the cross-correlation is defined as:
>
$$
(f\star g)(\tau )\ {\stackrel {\mathrm {def} }{=}}\int _{-\infty }^{\infty }f^{*}(t)\ g(t+\tau )\,dt,
$$
>
> where $f^{*}$ denotes the complex conjugate of  $f$, and $\tau$ is the displacement, also known as lag, although a positive value of $\tau$ actually means that $g(t+\tau)$ leads $f(t)$.

> *https://en.wikipedia.org/wiki/Cross-correlation*

The result of the correlation is a function of the delay $\tau$. This is useful for finding the offset in shifted signals.

```python
offset = 0.25
t = np.linspace(-1, 1, 600)
f = signal.gausspulse(t, fc=3)
g = signal.gausspulse( t + offset, fc=3)

result = signal.correlate(f, g, mode='same')
offset = t[np.argmax(result)]
```

```python
grid = plt.GridSpec(2, 2, wspace=0.1, hspace=0.4, width_ratios=[1, 1])
ax1 = plt.subplot(grid[0, 0])
ax2 = plt.subplot(grid[1, 0])
ax3 = plt.subplot(grid[:, 1], )

ax1.plot(t, f, color='gray')
ax1.set_title('$f$')
ax1.set_yticks([])
ax1.set_xlabel('t')

ax2.plot(t, g, color='gray')
ax2.set_title('$g$')
ax2.set_yticks([])
ax2.set_xlabel('t')

ax3.plot(t, result)
ax3.axvline(offset, color='xkcd:tangerine')
ax3.set_title('$(f\star g)(\\tau )$')
ax3.set_yticks([])
ax3.set_xlabel('τ');
```

### 5.4 Exercise III:  Signal Filtering

If we ever find an alien civilization we will have to communicate with them using binary encoded signals recorded by large radio telescopes.

Assuming the aliens use binary signals with 0s and 1s.
Given the clock speed and the sample length with which the signal was created. Can you decode the following signal?

The plot below shows the shape of an example signal without any noise containing the message [0, 1 0]

```python
sample_length = 64
plt.plot(np.repeat([0, 1, 0], sample_length))
plt.text(25, 0.5, '0')
plt.text(100, 0.5, '1')
plt.text(175, 0.5, '0')
```

```python
alien_signal = np.loadtxt('data/alien_signal.txt')
sample_length = 128

# let's sample the signal in the middle of each period
clock = np.arange(sample_length // 2, len(alien_signal), sample_length)
plt.plot(alien_signal)

for c in clock:
    plt.axvline(c, color='gray', lw=1.5)
```

```python
# %load solutions/solution_3.py
```

### 5.5 Fourier transform

Fourier transforms are one of the universal tools in computational physics, which appear over and over again in different contexts. SciPy provides functions for accessing the classic [FFTPACK](http://www.netlib.org/fftpack/) library, which is an efficient and well tested FFT library written in FORTRAN. The SciPy API has a few additional convenience functions, but overall the API is closely related to the original FORTRAN library.

To use the `fftpack` module in a python program, include it using:

```python
from scipy import fftpack
```

To demonstrate how to do a fast Fourier transform with SciPy, we will create some artificial data first.

```python
# Number of sample points
n_samples = 1000
length = 10
sample_size = length / n_samples

# sample spacing
x = np.linspace(0.0, length, n_samples)

freqs = [8, 3, 32]
amps = [1, 0.5, 0.25]

y = np.zeros(n_samples)
for a, f in zip(amps, freqs):
    y += a * np.sin(2 * np.pi * f * x)

y += rng.normal(0, 0.75, len(y))
plt.plot(x, y, lw=2)
```

```python
yf = fftpack.fft(y)
xf = fftpack.fftfreq(n_samples, sample_size)
plt.plot(xf, 2.0 / n_samples * np.abs(yf))
plt.xlim(0, 0.5 / sample_size)
```

As expected, we now see a peaks in the spectrum that match the true frequencies in the signal.

## 6. Statistics

The `scipy.stats` module contains a large number of statistical distributions, statistical functions and tests. For a complete documentation of its features, see http://docs.scipy.org/doc/scipy/reference/stats.html.

Below we instantiate an object modeling a random variable following the normal distribution.

### 6.1 Continuous Distributions

```python
from scipy import stats

mean = 0
sigma = 1
X = stats.norm(loc=mean, scale=sigma)
```

All distributions in this submodule extend either `scipy.stats.rv_continuous` or `scipy.stats.rv_discrete`.
Below is a list of functions supported by most continuous distribution objects

```
rvs(*args, **kwds) 	Random variates of given type.
pdf(x, *args, **kwds) 	Probability density function at x of the given RV.
logpdf(x, *args, **kwds) 	Log of the probability density function at x of the given RV.
cdf(x, *args, **kwds) 	Cumulative distribution function of the given RV.
logcdf(x, *args, **kwds) 	Log of the cumulative distribution function at x of the given RV.
sf(x, *args, **kwds) 	Survival function (1 - cdf) at x of the given RV.
logsf(x, *args, **kwds) 	Log of the survival function of the given RV.
ppf(q, *args, **kwds) 	Percent point function (inverse of cdf) at q of the given RV.
isf(q, *args, **kwds) 	Inverse survival function (inverse of sf) at q of the given RV.
moment(n, *args, **kwds) 	n-th order non-central moment of distribution.
stats(*args, **kwds) 	Some statistics of the given RV.
entropy(*args, **kwds) 	Differential entropy of the RV.
expect([func, args, loc, scale, lb, ub, …]) 	Calculate expected value of a function with respect to the distribution.
median(*args, **kwds) 	Median of the distribution.
mean(*args, **kwds) 	Mean of the distribution.
std(*args, **kwds) 	Standard deviation of the distribution.
var(*args, **kwds) 	Variance of the distribution.
interval(alpha, *args, **kwds) 	Confidence interval with equal areas around the median.
__call__(*args, **kwds) 	Freeze the distribution for the given arguments.
fit(data, *args, **kwds) 	Return MLEs for shape (if applicable), location, and scale parameters from data.
fit_loc_scale(data, *args) 	Estimate loc and scale parameters from data using 1st and 2nd moments.
nnlf(theta, x) 	Return negative loglikelihood function.

```

```python
N = 1200
measurements = X.rvs(size=N)
x = np.linspace(-3.5, 3.5, 200)

f, [ax1, ax2] = plt.subplots(1, 2, figsize=(12, 5))
ax1.hist(measurements, bins=40, density=True)
ax1.plot(x, X.pdf(x), color='red')

ax2.hist(measurements, bins=40, cumulative=True, density=True)
ax2.plot(x, X.cdf(x), color='red');
```

### 6.2 Discrete Distributions

Discrete functions work in a very similar manner.

```
rvs(*args, **kwargs) 	Random variates of given type.
pmf(k, *args, **kwds) 	Probability mass function at k of the given RV.
logpmf(k, *args, **kwds) 	Log of the probability mass function at k of the given RV.
cdf(k, *args, **kwds) 	Cumulative distribution function of the given RV.
logcdf(k, *args, **kwds) 	Log of the cumulative distribution function at k of the given RV.
sf(k, *args, **kwds) 	Survival function (1 - cdf) at k of the given RV.
logsf(k, *args, **kwds) 	Log of the survival function of the given RV.
ppf(q, *args, **kwds) 	Percent point function (inverse of cdf) at q of the given RV.
isf(q, *args, **kwds) 	Inverse survival function (inverse of sf) at q of the given RV.
moment(n, *args, **kwds) 	n-th order non-central moment of distribution.
stats(*args, **kwds) 	Some statistics of the given RV.
entropy(*args, **kwds) 	Differential entropy of the RV.
expect([func, args, loc, lb, ub, …]) 	Calculate expected value of a function with respect to the distribution for discrete distribution.
median(*args, **kwds) 	Median of the distribution.
mean(*args, **kwds) 	Mean of the distribution.
std(*args, **kwds) 	Standard deviation of the distribution.
var(*args, **kwds) 	Variance of the distribution.
interval(alpha, *args, **kwds) 	Confidence interval with equal areas around the median.
__call__(*args, **kwds) 	Freeze the distribution for the given arguments.
```

```python
# create a (discreet) random variable with poissionian distribution
Y = stats.poisson(3.5)
```

```python
n = np.arange(0, 12)

fig, axes = plt.subplots(3,1, sharex=True, figsize=(8, 8))

# plot the probability mass function (PMF)
axes[0].plot(n, Y.pmf(n), 'o')

# plot the cumulative distribution function (CDF)
axes[1].step(n, Y.cdf(n))

# plot histogram of 1000 random realizations of the stochastic variable X
axes[2].hist(Y.rvs(size=200), bins=np.arange(13) - 0.5, density=True, lw=2, edgecolor='w');
axes[2].set_xticks(n);
```

The objects support easy access to some of its statistical properties:

```python
print(X.mean(), X.std(), X.var()) # normal distribution
print(Y.mean(), Y.std(), Y.var()) # poission distribution
```

    0.0 1.0 1.0
    3.5 1.8708286933869707 3.5

## 7. Optimization

Optimization (finding minima or maxima of a function) is a large field in mathematics, and optimization of complicated functions or in many variables can be rather complicated. Here we will only look at a few very simple cases. For a more detailed introduction to optimization with SciPy see: http://scipy-lectures.github.com/advanced/mathematical_optimization/index.html

To use the optimization module in scipy first include the `optimize` module:

```python
from scipy import optimize
```

### 7.1 Minimization

Let's first look at how to find the minima of a simple function of a single variable:

```python
def f(x):
    return 4.5*x**3 + (x-4)**2 + 0.75*x**4 - 20

x = np.linspace(-6, 3, 100)
plt.plot(x, f(x));
```

We can use the `minimize_scalar` function to find the minima of a scalar function. It provides wrapper around many optimization algorithms you can choose from using the `method` keyword.

```python
result = optimize.minimize_scalar(f)

plt.plot(x, f(x));
plt.axvline(result.x, color='xkcd:orange')

# add some boundaries
result = optimize.minimize_scalar(f, bracket=[-6, -2])
plt.axvline(result.x, color='xkcd:dark red')
```

### 7.2 Minimization in More Dimensions.

Below you'll find the infamous *Rosenbrock* function in two dimensions. It's very popular for benchmarking global optimization problems. Its solutions are well known and easily parametrized.

```python
def rosenbrock_2d(x, y, a=1, b=1):
    return (a - x)**2 + b*(y - x**2)**2

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1, projection='3d')

x, y = np.meshgrid(np.linspace(-2 ,2, 100), np.linspace(-3. ,5, 100))
ax.plot_surface(x, y, rosenbrock_2d(x, y), cmap='magma_r')
```

```python
result = optimize.minimize(lambda x: rosenbrock_2d(x[0], x[1]), x0=[5, 2])
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1, projection='3d')
ax.view_init(20, 30)

x, y = np.meshgrid(
    np.linspace(-2 ,2, 100),
    np.linspace(-3. ,5, 100)
)

ax.plot_surface(x, y, rosenbrock_2d(x, y), cmap='magma_r', alpha=0.6)

x, y, z = result.x[0], result.x[1], 20
ax.quiver(x, y, z, 0, 0, -1, length=20, arrow_length_ratio=0.04)

result
```

### 7.3 Least Squares Minimization

Lets presume a linear weighted combination of variables:

$$
f(x)= \hat{y} =  \hat{\beta}_0 + \sum_{j=1}^p x_j \hat{\beta}_j
$$
where $f:\mathbb{R}^{p} \to \mathbb{R}$.

When we include a 1 as the first entry into our sample $x$ e.g. $x = (1, x_1, x_2, \ldots, x_p)$ we can rewrite
$f$ in matrix form

$$
f(x)= \hat{y} =  x^T \mathbf{\beta}
$$

where $\beta = (\beta_0, \beta_1, \beta_2, \ldots, \beta_p)$.

How do you find those weights? Like before we choose a loss function and try to opimize it.
In this case we choose a loss function called the residual sum of squares (RSS).
We calculate it over all samples $x_i$ in a matrix $\mathbf{X}$.

$$L(\beta) = RSS(\mathbf{\beta}) = \sum_{i=1}^N (y_i - x_i^T \beta)^2 $$

Here $x_i$ is a row in $\mathbf{X}$, hence the transpose.

We can now rewrite the loss function in matrix form:

$$
RSS(\beta) = (\mathbf{y} - \mathbf{X} \beta)^T (\mathbf{y} - \mathbf{X} \beta )
$$

Now we optimize the loss function just like we would any other function, by differentiating with respect to $\beta$ and setting the result equals to zero.

$$
 \mathbf{X}^T (\mathbf{y} - \mathbf{X} \beta ) \stackrel{!}{=} 0
$$

Solving for $\beta$ leads to

$$
\hat{\beta} = (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{y}
$$

We just performed  __Linear Least Squares__ regression. Below we fit some noisy a polynomial data.

```python
def f(x, beta=[4.5, 1, 0.75]):
    return beta[0]*x**3 + beta[1]*(x-4)**2 + beta[2]*x**4

x = np.linspace(-6, 3, 100)
data = f(x) + rng.normal(0, 35, size=x.shape)

# build the answer matrix
A = np.vstack([x ** 3, (x - 4) ** 2, x ** 4]).T

# find the linear least squares solution for the parameters
result = optimize.lsq_linear(A, data)

plt.plot(x, data, '.')
plt.plot(x, f(x), color='gray', label='truth')
plt.plot(x, f(x, beta=result.x), color='crimson', label='fit')
plt.legend();
```

### 7.4 Curve Fitting

For more general cases where the parameters are not linear there is the more convenient function `curve_fit`

```python
from scipy import stats

def f(x, scale=5, loc=4, ):
    return (1/(scale))**2*x**3 + (1/(2*loc))*x**2

x = np.linspace(-3,  1.5, 100)
data = f(x) + rng.normal(0, 0.15, size=x.shape)

popt, pcov = optimize.curve_fit(f, x, data, p0=[5, 4, ])

plt.plot(x, data, '.')
plt.plot(x, f(x), color='gray', label='truth')
plt.plot(x, f(x, *popt), color='crimson', label='fit')

pcov
```

### 7.5 Exercise:  Curve Fitting

Use `numpy.loadtxt` to load the file called `munich_temperatures_average.txt` and fit a model to it using `optimize.curve_fit`.

```python
# %load solutions/solution_6.py
```

### 7.6 Root Finding

To find the solution for a function of the form $f(x) = 0$ we can use the `fsolve` or the `root` function. It requires one or more initial guesses:

```python
def f(x):
    return 4.5 * x **3 + (x - 4) ** 2 + 0.75 * x ** 4 - 20

x = np.linspace(-6, 3, 100)

plt.plot(x, f(x))
plt.axhline(color='black')

result = optimize.root(f, [-6, -3, 0, 2])

for r in result.x:
    plt.axvline(r, color='xkcd:tangerine')
```

## 8. Interpolation

Interpolation is simple and convenient in scipy: The `interp1d` function, when given arrays describing X and Y data, returns and object that behaves like a function that can be called for an arbitrary value of x (in the range covered by X), and it returns the corresponding interpolated y value:

```python
from scipy import interpolate

def f(x):
    return np.sin(x)
```

```python
n = [0, 1, 2, 3, 3.5, 4, 6, 6.5, 7, 9]
x = np.linspace(0, 9, 100)

y_measured = f(n)

linear_interpolation = interpolate.interp1d(n, y_measured)

cubic_interpolation = interpolate.interp1d(n, y_measured, kind='cubic')
# this returns a new function we can call for values of x
cubic_interpolation
```

    <scipy.interpolate.interpolate.interp1d at 0x1195d1400>

```python
plt.plot(n, y_measured, 's', label='noisy data', color='black')
plt.plot(x, f(x), c='gray', lw=2, label='true function')
plt.plot(x, linear_interpolation(x), label='linear interpolation')
plt.plot(x, cubic_interpolation(x), label='cubic interpolation')
plt.legend();
```

## 9. SciPy IO Module

## 9.1 Wave files

SciPy provides some useful methods for reading file from MatLab, NetCDF or WAV data. Below is an example which uses the `scipy.io.wavfile` module to write a file containing the sound of hydrogen atoms.

```python
from scipy.io import wavfile
from scipy import signal

rate = 44100 #44.1 khz
duration = 10 # in sec

def rydberg(n, m):
    return (1 / (n ** 2) - 1 / (m ** 2))

def lyman(m):
    """Lyman series"""
    return rydberg(1, m)

def balmer(m):
    """Balmer series"""
    return rydberg(2, m)

def sound(frequency, time):
    return 2 ** 12 * np.sin(2 * np.pi * 440 * frequency * time)

t = np.linspace(0, duration, rate * duration)
hydrogen_sound = np.sum(
    [sound(lyman(i), t) + sound(balmer(i), t) for i in range(2, 12)], axis=0) / 6

plt.plot(t[:10000], hydrogen_sound[:10000])

wavfile.write('data/hydrogen.wav', rate, hydrogen_sound.astype(np.int16))
```

```python
%%HTML
<audio controls="controls" style="width:600px" >
  <source src="hydrogen.wav" type="audio/wav" />
</audio>
```

<audio controls="controls" style="width:600px" >
  <source src="hydrogen.wav" type="audio/wav" />
</audio>

### 9.2 Exercise:  Signal Processing and IO.

Load the file `synth_sound.wav` and plot a the frequency spectrum vs time.

```python
# %load solutions/solution_4.py
```


```python
%load_ext autoreload
%autoreload 2
%matplotlib inline

import sys
import pylab as pl
sys.path.insert(0, '..')

import pymola.parser
import pymola.backends.sympy.generator as generator
import pylab as pl
```

```python
ast = pymola.parser.parse('''
model Aircraft
    parameter Real c = 0.1;
    Real x(start=10);
    Real y(start=100);
equation
    der(x) = -x + y;
    der(y) = -c*y;
end Aircraft;
''')
```

```python
src = generator.generate(ast, 'Aircraft')
print(src)
```

    # do not edit, generated by pymola

    from __future__ import print_function, division
    import sympy
    import sympy.physics.mechanics as mech
    from pymola.backends.sympy.runtime import OdeModel
    from sympy import sin, cos, tan

    class Aircraft(OdeModel):

        def __init__(self):

            super(Aircraft, self).__init__()

            # states
            x, y = mech.dynamicsymbols('x, y')
            self.x = sympy.Matrix([x, y])
            self.x0 = {
                x : 10.0,
                y : 100.0,
                }

            # variables
            self.v = sympy.Matrix([])

            # constants
            self.c = sympy.Matrix([])
            self.c0 = {
                }

            # parameters
            c = sympy.symbols('c')
            self.p = sympy.Matrix([c])
            self.p0 = {
                c : 0.1,
                }

            # inputs
            self.u = sympy.Matrix([])
            self.u0 = {
                }

            # outputs
            self.y = sympy.Matrix([])

            # equations
            self.eqs = [
                (x).diff(self.t) - (- x + y),
                (y).diff(self.t) - (- c * y),
                ]

            self.compute_fg()

```python
exec(src)
ac = Aircraft()
res = ac.simulate()
res
```

    {'t': array([ 0.  ,  0.01,  0.02, ...,  9.98,  9.99, 10.  ]),
     'u': array([], shape=(1001, 0), dtype=float64),
     'x': array([[10.0000000000000, 100.000000000000],
            [10.895012161296487, 99.90005005160832],
            [11.780129353073226, 99.8001999349418],
            ...,
            [40.952635133614756, 36.8615936557411],
            [40.91174521009363, 36.82475048674022],
            [40.87089575475432, 36.78794414249291]], dtype=object),
     'y': array([], shape=(1001, 0), dtype=float64)}

```python
pl.plot(res['t'], res['x'])
```

```python
ss = ac.linearize()
ss[0]
```

    matrix([[-1. ,  1. ],
            [ 0. , -0.1]])

```python
ac.linearize_symbolic()
```

    [Matrix([
     [-1,  1],
     [ 0, -c]]), Matrix(0, 0, []), Matrix(0, 0, []), Matrix(0, 0, [])]

```python
pl.matrix(ss[0]).astype(float)
```

    matrix([[-1. ,  1. ],
            [ 0. , -0.1]])


```python
import numpy as np
import matplotlib.pyplot as plt
import sympy as senpai
from sympy import print_latex
```

```python
# Definiujemy obiekty biblioteki SymPy
t, y, s, w, j = senpai.symbols('t y s w j')
```

```python
Ks = senpai.Matrix([[1,2,3], [4,5,6],[7,8,9]])
print_latex(Ks)
print("K(s)=")
Ks
```

    \left[\begin{matrix}1 & 2 & 3\\4 & 5 & 6\\7 & 8 & 9\end{matrix}\right]
    K(s)=

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\4 & 5 & 6\\7 & 8 & 9\end{matrix}\right]$

```python
a = senpai.Matrix([[1,2,0], [4,5,1]])
print_latex(a)
print("a=")
a
```

    \left[\begin{matrix}1 & 2 & 0\\4 & 5 & 1\end{matrix}\right]
    a=

$\displaystyle \left[\begin{matrix}1 & 2 & 0\\4 & 5 & 1\end{matrix}\right]$

```python
a*Ks
```

$\displaystyle \left[\begin{matrix}9 & 12 & 15\\31 & 41 & 51\end{matrix}\right]$

```python
K = s**4 + 2*s**3 + 3*s**2 + 10*s + 8
K
```

$\displaystyle s^{4} + 2 s^{3} + 3 s^{2} + 10 s + 8$

```python
senpai.factor(K)
```

$\displaystyle \left(s + 1\right) \left(s + 2\right) \left(s^{2} - s + 4\right)$

```python
print_latex(senpai.solve(K,s))
```

    \left[ -2, \  -1, \  \frac{1}{2} - \frac{\sqrt{15} i}{2}, \  \frac{1}{2} + \frac{\sqrt{15} i}{2}\right]

$\left[ -2, \  -1, \  \frac{1}{2} - \frac{\sqrt{15} i}{2}, \  \frac{1}{2} + \frac{\sqrt{15} i}{2}\right]$

```python
plt.scatter([-2, -1, 0.5, 0.5], [0, 0, -np.sqrt(15)/5, np.sqrt(15)/5])
# plt.grid(True)
plt.axes()
# plt.axes(rect, projection=None, polar=False, **kwargs)
# plt.axes(ax)
plt.show()
```

```python
type(senpai.Float(senpai.re(im)))
```

    sympy.core.numbers.Float

```python

```

$\displaystyle k$

```python
-np.sqrt(15)/5
```

    -0.7745966692414834


# Introduction

Several pieces of the puzzle have come together lately to really demonstrate the power of the scientific python software packages to handle complex dynamic and controls problems (i.e. IPython notebooks, matplotlib animations, python-control, and our software packages: sympy.physics.mechanics and PyDy).

This [blog post by Wolfram](http://blog.wolfram.com/2011/03/01/stabilized-n-link-pendulum/) demonstrates Mathematica's ability to symbolically derive the equations of motion for the n-link pendulum and stabilize it with an LQR controller. This blog post inspired us to replicate the example with all free and open source software.

In this example problem, we derive the equations of motion of an n-link pendulum on a laterally sliding cart and then develop a controller to stabilize it. Balancing a single inverted pendulum is a classic problem that is often a student's first experience with non-linear dynamics and control. The problem here is extended to a general n-link pendulum in which the equations of motion quickly get messy with greater than 2 links.

The diagram below shows the general description of the problem.

```python
from IPython.display import SVG
SVG(filename='n-pendulum-with-cart.svg')
```

Setup
===

This example depends on the following software:

- IPython
- NumPy
- SciPy
- SymPy >= 0.7.6
- matplotlib
- python-control
- avconv

The easiest way to install the Python packages it is to use conda:

```
$ conda install ipython-notebook, numpy, scipy, sympy, matplotlib
$ conda install -c https://conda.binstar.org/cwrowley control
```

`avconv` should be installed as per its recommended procedure for your operating system.

Equations of Motion
===================

We'll start by generating the equations of motion for the system with SymPy **[mechanics](http://docs.sympy.org/dev/modules/physics/mechanics/index.html)**. The functionality that mechanics provides is much more in depth than Mathematica's functionality. In the Mathematica example, Lagrangian mechanics were implemented manually with Mathematica's symbolic functionality. **mechanics** provides an assortment of functions and classes to derive the equations of motion for arbitrarily complex (i.e. configuration constraints, nonholonomic motion constraints, etc) multibody systems in a very natural way. First we import the necessary functionality from SymPy.

```python
import sympy as sm
import sympy.physics.mechanics as me
```

We can enable mathematical rendering of the resulting equations in the notebook with the following command.

```python
me.init_vprinting()
```

Now specify the number of links, $n$. I'll start with 5 since the Wolfram folks only showed four.

```python
n = 5
```

**mechanics** will need the generalized coordinates, generalized speeds, and the input force which are all time dependent variables and the bob masses, link lengths, and acceleration due to gravity which are all constants. Time, $t$, is also made available because we will need to differentiate with respect to time.

```python
q = me.dynamicsymbols('q:{}'.format(n + 1))  # Generalized coordinates
u = me.dynamicsymbols('u:{}'.format(n + 1))  # Generalized speeds
f = me.dynamicsymbols('f')                   # Force applied to the cart

m = sm.symbols('m:{}'.format(n + 1))         # Mass of each bob
l = sm.symbols('l:{}'.format(n))             # Length of each link
g, t = sm.symbols('g t')                     # Gravity and time
```

Now we can create and inertial reference frame $I$ and define the point, $O$, as the origin.

```python
I = me.ReferenceFrame('I')  # Inertial reference frame
O = me.Point('O')           # Origin point
O.set_vel(I, 0)             # Origin's velocity is zero
```

Secondly, we define the define the first point of the pendulum as a particle which has mass. This point can only move laterally and represents the motion of the "cart".

```python
P0 = me.Point('P0')                 # Hinge point of top link
P0.set_pos(O, q[0] * I.x)           # Set the position of P0
P0.set_vel(I, u[0] * I.x)           # Set the velocity of P0
Pa0 = me.Particle('Pa0', P0, m[0])  # Define a particle at P0
```

Now we can define the $n$ reference frames, particles, gravitational forces, and kinematical differential equations for each of the pendulum links. This is easily done with a loop.

```python
frames = [I]                              # List to hold the n + 1 frames
points = [P0]                             # List to hold the n + 1 points
particles = [Pa0]                         # List to hold the n + 1 particles
forces = [(P0, f * I.x - m[0] * g * I.y)] # List to hold the n + 1 applied forces, including the input force, f
kindiffs = [q[0].diff(t) - u[0]]          # List to hold kinematic ODE's

for i in range(n):
    Bi = I.orientnew('B' + str(i), 'Axis', [q[i + 1], I.z])   # Create a new frame
    Bi.set_ang_vel(I, u[i + 1] * I.z)                         # Set angular velocity
    frames.append(Bi)                                         # Add it to the frames list

    Pi = points[-1].locatenew('P' + str(i + 1), l[i] * Bi.x)  # Create a new point
    Pi.v2pt_theory(points[-1], I, Bi)                         # Set the velocity
    points.append(Pi)                                         # Add it to the points list

    Pai = me.Particle('Pa' + str(i + 1), Pi, m[i + 1])        # Create a new particle
    particles.append(Pai)                                     # Add it to the particles list

    forces.append((Pi, -m[i + 1] * g * I.y))                  # Set the force applied at the point

    kindiffs.append(q[i + 1].diff(t) - u[i + 1])              # Define the kinematic ODE:  dq_i / dt - u_i = 0
```

With all of the necessary point velocities and particle masses defined, the `KanesMethod` class can be used to derive the equations of motion of the system automatically.

```python
kane = me.KanesMethod(I, q_ind=q, u_ind=u, kd_eqs=kindiffs)  # Initialize the object
fr, frstar = kane.kanes_equations(forces, particles)         # Generate EoM's fr + frstar = 0
```

The equations of motion are quite long as can been seen below. This is the general nature of most non-simple mutlibody problems. That is why a SymPy is so useful; no more mistakes in algebra, differentiation, or copying hand written equations. Note that `trigsimp` can take quite a while to complete for extremely large expressions. Below we print $\tilde{M}$ and $\tilde{f}$ from $\tilde{M}\dot{u}=\tilde{f}$ to show the size of the expressions.

```python
sm.trigsimp(kane.mass_matrix)
```

$\tilde{M}$ is a function of the constant parameters and the configuration.

```python
me.find_dynamicsymbols(kane.mass_matrix)
```

```python
sm.trigsimp(kane.forcing)
```

$\tilde{f}$ is a function of the constant parameters, configuration, speeds, and the applied force.

```python
me.find_dynamicsymbols(kane.forcing)
```

Simulation
==========

Now that the symbolic equations of motion are available we can simulate the pendulum's motion. We will need some more SymPy functionality and several NumPy functions, and most importantly the integration function from SciPy, `odeint`.

```python
import numpy as np
from numpy.linalg import solve
from scipy.integrate import odeint
```

First, define some numeric values for all of the constant parameters in the problem.

```python
arm_length = 1. / n                          # The maximum length of the pendulum is 1 meter
bob_mass = 0.01 / n                          # The maximum mass of the bobs is 10 grams
parameters = [g, m[0]]                       # Parameter definitions starting with gravity and the first bob
parameter_vals = [9.81, 0.01 / n]            # Numerical values for the first two
for i in range(n):                           # Then each mass and length
    parameters += [l[i], m[i + 1]]
    parameter_vals += [arm_length, bob_mass]
```

Mathematica has a really nice `NDSolve` function for quickly integrating their symbolic differential equations. We make use of SymPy's lambdify function to do something similar, i.e. to create functions that will evaluate the "full" mass matrix, $M$, and "full" forcing vector, $f$ from $M\dot{x} = f(x, r, t)$ as a NumPy function.

```python
dynamic = q + u    # Make a list of the states
dynamic.append(f)  # Add the input force

M_func = sm.lambdify(dynamic + parameters, kane.mass_matrix_full) # Create a callable function to evaluate the mass matrix
f_func = sm.lambdify(dynamic + parameters, kane.forcing_full)     # Create a callable function to evaluate the forcing vector
```

To integrate the ODE's we need to define a function that returns the derivatives of the states given the current state and time.

```python
def right_hand_side(x, t, args):
    """Returns the derivatives of the states.

    Parameters
    ----------
    x : ndarray, shape(2 * (n + 1))
        The current state vector.
    t : float
        The current time.
    args : ndarray
        The constants.

    Returns
    -------
    dx : ndarray, shape(2 * (n + 1))
        The derivative of the state.

    """
    r = 0.0                                  # The input force is always zero
    arguments = np.hstack((x, r, args))      # States, input, and parameters
    dx = np.array(solve(M_func(*arguments),  # Solving for the derivatives
                  f_func(*arguments))).T[0]

    return dx
```

Now that we have the right hand side function, the initial conditions are set such that the pendulum is in the vertical equilibrium and a slight initial rate is set for each speed to ensure the pendulum falls. The equations can then be integrated with SciPy's `odeint` function given a time series.

```python
x0 = np.hstack((0.0,                              # q0
                np.pi / 2 * np.ones(len(q) - 1),  # q1...qn+1
                1e-3 * np.ones(len(u))))          # u0...un+1
```

```python
t = np.linspace(0.0, 10.0, num=500)  # Time vector
```

```python
x = odeint(right_hand_side, x0, t, args=(parameter_vals,))  # Numerical integration
```

Plotting
========

The results of the simulation can be plotted with matplotlib. First, load the plotting functionality.

```python
import matplotlib.pyplot as plt
```

```python
%matplotlib inline
```

```python
from IPython.core.pylabtools import figsize
figsize(8.0, 6.0)
```

The coordinate trajectories are plotted below.

```python
lines = plt.plot(t, x[:, :x.shape[1] / 2])
lab = plt.xlabel('Time [sec]')
leg = plt.legend(dynamic[:x.shape[1] / 2])
```

And the generalized speed trajectories.

```python
lines = plt.plot(t, x[:, x.shape[1] / 2:])
lab = plt.xlabel('Time [sec]')
leg = plt.legend(dynamic[x.shape[1] / 2:])
```

Animation
=========

matplotlib now includes very nice animation functions for animating matplotlib plots. First we import the necessary functions for creating the animation.

```python
from matplotlib import animation
from matplotlib.patches import Rectangle
```

The following function was modeled from Jake Vanderplas's [post on matplotlib animations](http://jakevdp.github.com/blog/2012/08/18/matplotlib-animation-tutorial/).

```python
def animate_pendulum(t, states, length, filename=None):
    """Animates the n-pendulum and optionally saves it to file.

    Parameters
    ----------
    t : ndarray, shape(m)
        Time array.
    states: ndarray, shape(m,p)
        State time history.
    length: float
        The length of the pendulum links.
    filename: string or None, optional
        If true a movie file will be saved of the animation. This may take some time.

    Returns
    -------
    fig : matplotlib.Figure
        The figure.
    anim : matplotlib.FuncAnimation
        The animation.

    """
    # the number of pendulum bobs
    numpoints = states.shape[1] / 2

    # first set up the figure, the axis, and the plot elements we want to animate
    fig = plt.figure()

    # some dimesions
    cart_width = 0.4
    cart_height = 0.2

    # set the limits based on the motion
    xmin = np.around(states[:, 0].min() - cart_width / 2.0, 1)
    xmax = np.around(states[:, 0].max() + cart_width / 2.0, 1)

    # create the axes
    ax = plt.axes(xlim=(xmin, xmax), ylim=(-1.1, 1.1), aspect='equal')

    # display the current time
    time_text = ax.text(0.04, 0.9, '', transform=ax.transAxes)

    # create a rectangular cart
    rect = Rectangle([states[0, 0] - cart_width / 2.0, -cart_height / 2],
                     cart_width, cart_height, fill=True, color='red',
                     ec='black')
    ax.add_patch(rect)

    # blank line for the pendulum
    line, = ax.plot([], [], lw=2, marker='o', markersize=6)

    # initialization function: plot the background of each frame
    def init():
        time_text.set_text('')
        rect.set_xy((0.0, 0.0))
        line.set_data([], [])
        return time_text, rect, line,

    # animation function: update the objects
    def animate(i):
        time_text.set_text('time = {:2.2f}'.format(t[i]))
        rect.set_xy((states[i, 0] - cart_width / 2.0, -cart_height / 2))
        x = np.hstack((states[i, 0], np.zeros((numpoints - 1))))
        y = np.zeros((numpoints))
        for j in np.arange(1, numpoints):
            x[j] = x[j - 1] + length * np.cos(states[i, j])
            y[j] = y[j - 1] + length * np.sin(states[i, j])
        line.set_data(x, y)
        return time_text, rect, line,

    # call the animator function
    anim = animation.FuncAnimation(fig, animate, frames=len(t), init_func=init,
            interval=t[-1] / len(t) * 1000, blit=True, repeat=False)

    # save the animation if a filename is given
    if filename is not None:
        anim.save(filename, fps=30, writer="avconv", codec='libx264')
```

Now we can create the animation of the pendulum. This animation will show the open loop dynamics.

```python
animate_pendulum(t, x, arm_length, filename="open-loop.mp4")
```

```python
from IPython.display import HTML
html = \
"""

"""
HTML(html)
```

Controller Design
=================

The n-link pendulum can be balanced such that all of the links are inverted above the cart by applying the correct lateral force to the cart. We can design a full state feedback controller based from a linear model of the pendulum about its upright equilibrium point. We'll start by specifying the equilibrium point and parameters in dictionaries. We make sure to use SymPy types in the equilibrium point to ensure proper cancelations in the linearization.

```python
equilibrium_point = [sm.S(0)] + [sm.pi / 2] * (len(q) - 1) + [sm.S(0)] * len(u)
equilibrium_dict = dict(zip(q + u, equilibrium_point))
equilibrium_dict
```

The `KanesMethod` class has method that linearizes the forcing vector about generic state and input perturbation vectors. The equilibrium point and numerical constants can then be substituted in to give the linear system in this form: $M\dot{x}=F_Ax+F_Br$. The state and input matrices, $A$ and $B$, can then be computed by left side multiplication by the inverse of the mass matrix: $A=M^{-1}F_A$ and $B=M^{-1}F_B$.

```python
M, F_A, F_B, r = kane.linearize(new_method=True, op_point=equilibrium_dict)
```

```python
sm.simplify(M)
```

```python
sm.simplify(F_A)
```

```python
sm.simplify(F_B)
```

Now the numerical $A$ and $B$ matrices can be formed. First substitute numerical parameter values into $M$, $F_A$, and $F_B$.

```python
parameter_dict = dict(zip(parameters, parameter_vals))
parameter_dict
```

```python
M_num = sm.matrix2numpy(M.subs(parameter_dict), dtype=float)
F_A_num = sm.matrix2numpy(F_A.subs(parameter_dict), dtype=float)
F_B_num = sm.matrix2numpy(F_B.subs(parameter_dict), dtype=float)
```

```python
A = np.linalg.solve(M_num, F_A_num)
B = np.linalg.solve(M_num ,F_B_num)
```

```python
print(A)
```

    [[  0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   1.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   1.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        1.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   1.00000000e+00   0.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   1.00000000e+00   0.00000000e+00]
     [  0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00
        0.00000000e+00   0.00000000e+00   0.00000000e+00   1.00000000e+00]
     [ -0.00000000e+00   4.90500000e+01  -0.00000000e+00   4.62592927e-15
       -1.15648232e-15  -3.61400724e-17  -0.00000000e+00  -0.00000000e+00
       -0.00000000e+00  -0.00000000e+00  -0.00000000e+00  -0.00000000e+00]
     [ -0.00000000e+00   4.90500000e+02  -1.96200000e+02  -0.00000000e+00
       -0.00000000e+00   5.42101086e-16  -0.00000000e+00  -0.00000000e+00
       -0.00000000e+00  -0.00000000e+00  -0.00000000e+00  -0.00000000e+00]
     [ -0.00000000e+00  -2.45250000e+02   3.92400000e+02  -1.47150000e+02
       -0.00000000e+00  -2.03287907e-15  -0.00000000e+00  -0.00000000e+00
       -0.00000000e+00  -0.00000000e+00  -0.00000000e+00  -0.00000000e+00]
     [  0.00000000e+00  -4.33680869e-14  -1.96200000e+02   2.94300000e+02
       -9.81000000e+01   7.22801448e-15   0.00000000e+00   0.00000000e+00
       -0.00000000e+00  -0.00000000e+00  -0.00000000e+00  -0.00000000e+00]
     [  0.00000000e+00   2.16840434e-14   0.00000000e+00  -1.47150000e+02
        1.96200000e+02  -4.90500000e+01   0.00000000e+00   0.00000000e+00
       -0.00000000e+00  -0.00000000e+00  -0.00000000e+00  -0.00000000e+00]
     [ -0.00000000e+00  -0.00000000e+00  -0.00000000e+00  -0.00000000e+00
       -9.81000000e+01   9.81000000e+01  -0.00000000e+00  -0.00000000e+00
       -0.00000000e+00  -0.00000000e+00  -0.00000000e+00  -0.00000000e+00]]

```python
print(B)
```

    [[  0.00000000e+00]
     [  0.00000000e+00]
     [  0.00000000e+00]
     [  0.00000000e+00]
     [  0.00000000e+00]
     [  0.00000000e+00]
     [  5.00000000e+02]
     [  2.50000000e+03]
     [ -1.73472348e-13]
     [  3.46944695e-13]
     [ -1.73472348e-13]
     [ -0.00000000e+00]]

Now that we have a linear system, the python-control package can be used to design an optimal controller for the system.

```python
import control
from numpy.linalg import matrix_rank
```

First we can check to see if the system is, in fact, controllable. The rank of the controllability matrix must be equal to the number of rows in $A$, but the `matrix_rank` algorithm is numerically ill conditioned and for certain values of $n$ this will fail, as seen below for $n=5$. Nevertheless, the system is controllable, no matter the number of links.

```python
matrix_rank(control.ctrb(A, B)) == A.shape[0]
```

    False

So now we can compute the optimal gains with a linear quadratic regulator. I chose identity matrices for the weightings for simplicity.

```python
K, X, E = control.lqr(A, B, np.ones(A.shape), 1);
```

The gains can now be used to define the required input during simulation to stabilize the system. The input $r$ is simply the gain vector multiplied by the error in the state vector from the equilibrium point, $r(t)=K(x_{eq} - x(t))$.

```python
def right_hand_side(x, t, args):
    """Returns the derivatives of the states.

    Parameters
    ----------
    x : ndarray, shape(2 * (n + 1))
        The current state vector.
    t : float
        The current time.
    args : ndarray
        The constants.

    Returns
    -------
    dx : ndarray, shape(2 * (n + 1))
        The derivative of the state.

    """
    r = np.dot(K, equilibrium_point - x)    # The controller
    arguments = np.hstack((x, r, args))     # States, input, and parameters
    dx = np.array(solve(M_func(*arguments), # Solving for the derivatives
                        f_func(*arguments))).T[0]

    return dx
```

Now we can simulate and animate the system to see if the controller works.

```python
x0 = np.hstack((0,
                np.pi / 2 * np.ones(len(q) - 1),
                1 * np.ones(len(u))))
t = np.linspace(0.0, 10.0, num=500)
x = odeint(right_hand_side, x0, t, args=(parameter_vals,))
```

The plots show that we seem to have a stable system.

```python
lines = plt.plot(t, x[:, :x.shape[1] / 2])
lab = plt.xlabel('Time [sec]')
leg = plt.legend(dynamic[:x.shape[1] / 2])
```

```python
lines = plt.plot(t, x[:, x.shape[1] / 2:])
lab = plt.xlabel('Time [sec]')
leg = plt.legend(dynamic[x.shape[1] / 2:])
```

```python
animate_pendulum(t, x, arm_length, filename="closed-loop.mp4")
```

```python
from IPython.display import HTML
html = \
"""

"""
HTML(html)
```

The video clearly shows that the controller can balance all $n$ of the pendulum links. The weightings in the lqr design can be tweaked to give different performance if needed.

This example shows that the free and open source scientific Python tools for dynamics are easily comparable in ability and quality to a commercial package such as Mathematica.

The IPython notebook for this example can be downloaded from https://github.com/pydy/pydy/tree/master/examples/npendulum. You can try out different $n$ values. I've gotten the equations of motion to compute for an open loop simulation of 10 links. My computer ran out of memory when I tried to compute for $n=50$. The controller weightings and initial conditions will probably have to be adjusted for better performance for $n>5$, but it should work.

```python
%install_ext http://raw.github.com/jrjohansson/version_information/master/version_information.py
%load_ext version_information
%version_information numpy, sympy, scipy, matplotlib, control
```

    Installed version_information.py. To use it, type:
      %load_ext version_information

<table><tr><th>Software</th><th>Version</th></tr><tr><td>Python</td><td>2.7.10 64bit [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]</td></tr><tr><td>IPython</td><td>3.1.0</td></tr><tr><td>OS</td><td>Linux 3.13.0 54 generic x86_64 with debian jessie sid</td></tr><tr><td>numpy</td><td>1.9.2</td></tr><tr><td>sympy</td><td>0.7.7.dev</td></tr><tr><td>scipy</td><td>0.15.1</td></tr><tr><td>matplotlib</td><td>1.4.3</td></tr><tr><td>control</td><td>0.7.0</td></tr><tr><td colspan='2'>Thu Jun 11 17:15:19 2015 PDT</td></tr></table>


# **Project 2: Model Analysis Project**

**By GWM442 and GXJ882**

## **Introduction**

In this project we consider the consumer's choice problem with two goods and a firm's maximization problem. We start by solving the consumer's problem for two different consumers, one with Cobb-Douglas preferences and the other with logarithmic preferences. Next, we find the social optimum for the two consumers. Lastly, we solve the firm's maximization problem, where the reader can choose between the settings of a competitive market or a monopolistic market. This project is made as an interactive helping tool, such that the user can adjust the parameters to the user's function.

## **Importing necessary tools**

We start by importing the tools, which we are going to use in this project.

```python
import ipywidgets as widgets #For making interactive widgets for the income and prices
import matplotlib.pyplot as plt # For making plots
import numpy as np # This is used, when we make graphs
from scipy import optimize # This is used for solving the consumer's problem
from sympy import * # For some methods of optimization
import warnings # We import this feature because we want to avoid printing a warning (This is purely done for estetic reasons)
warnings.filterwarnings("ignore", category=RuntimeWarning)
```

## **Consumer problems**

We solve the consumer problems for both consumers simultaneously. The consumer on the left will be the one with Cobb-Douglas preferences and the one on the rigth is the consumer with the logarithmic preferences.

We start by implementing the price of good 1 and good 2 as widgets. There are two ways to interact with the widgets for each variable. They are connected and will move simultaneously, allowing for either setting the variables using the slider or writing a specific value using the text-widget.

```python
# Price on good 1

# We only describe the code for the prices, as it is repeated for all variables.
# First we print a headline "Price on good 1", where the first part is used to make the text bold. We also write P1 in a parenthesis,
# as this is the variable name which we are going to use.
print('\033[1m' + 'Price on good 1 (P1)')
# Now we create the text-widget, where we set the starting value to 5.
P1 = widgets.FloatText(
value=5)
# Next we create a slide-widget (using floats, such that decimals can be used). Here we set the same starting value,
# we set the steps to 1 and we define the minimum value to 0 and the maximum value to 100.
# This can of cause be adjusted for the problem settings.
P1b = widgets.FloatSlider(
value=5,
step=1,
min=1,
max=100.0,)
# Next we print both widgets
display(P1,P1b)
# And lastly we link the two widgets together, so one can change either widget.
linkp1 = widgets.jslink((P1, 'value'), (P1b, 'value'))

# Price on good 2
print('\033[1m' + 'Price on good 2 (P2)')
P2 = widgets.FloatText(
value=15)
P2b = widgets.FloatSlider(
value=5,
step=1,
min=1,
max=100.0,)
display(P2,P2b)

linkp2 = widgets.jslink((P2, 'value'), (P2b, 'value'))
```

    Price on good 1 (P1)

    FloatText(value=5.0)

    FloatSlider(value=5.0, min=1.0, step=1.0)

    Price on good 2 (P2)

    FloatText(value=15.0)

    FloatSlider(value=5.0, min=1.0, step=1.0)

### **Setting the income level**

```python
# Income consumer 1
# This code is similar to the one for prices
print('\033[1m' + 'Income for consumer 1 (I1)')
I1 = widgets.FloatText(
value=100)
I1b = widgets.FloatSlider(
value=100,
step=1,
min=1,
max=1000,)
display(I1,I1b),

linkI1 = widgets.jslink((I1, 'value'), (I1b, 'value'))

# Income consumer 2
print('\033[1m' + 'Income for consumer 2 (I2)')
I2 = widgets.FloatText(
value=100)
I2b = widgets.FloatSlider(
value=100,
step=1,
min=1,
max=1000,)
display(I2,I2b)

linkI2 = widgets.jslink((I2, 'value'), (I2b, 'value'))

```

    Income for consumer 1 (I1)

    FloatText(value=100.0)

    FloatSlider(value=100.0, max=1000.0, min=1.0, step=1.0)

    Income for consumer 2 (I2)

    FloatText(value=100.0)

    FloatSlider(value=100.0, max=1000.0, min=1.0, step=1.0)

Next we plot the budget constraint. This shows us what the consumer is able to afford given the income. To do this we first need to define the budget constraint function, which is
\\[
\begin{eqnarray*}
I(p_{1},p_{2}) & = & x_{1} p_{1}+x_{2} p_{2}
\end{eqnarray*}
\\]
And then isolate for $x_2$

```python
# Define the income function (This is done by isolating x2)
def x2_func(x1,I,P1,P2): # We define x2 as a function of x1
    return (I-x1*P1)/P2 # The function is derived by isolating x2 in the budget constraint

# We then make a vector of x_1 values which we are going to use in our functions
N=500
shape_tuple = (N,N)
x1_values = np.empty(shape_tuple) # First we create an empty array

# Next we define the maximum value of x1, which is set so x1 is in the budget constraint for both consumers.
if I1.value>I2.value:    # This if-else statement ensures that the x-vector is large enough to solve the problem of the consumers.
    x_max1=I1.value/P1.value
else:
    x_max1=I2.value/P1.value
# And the vector is made by a double for loop.
for i in range(N): # 0,1,...,N-1
    for j in range(N): # 0,1,...,N-1
        x1_values[i,j] = (i/(N-1))*x_max1 # in [0,x_max1]

# Here we illustrate the results.
def Figure_1():
    fig = plt.figure(figsize=(15,5)) # creates the figure
    ax = fig.add_subplot(1,2,1) # We add a subplot and place it in the top left corner
    ax.plot(x1_values,x2_func(x1_values,I1.value,P1.value,P2.value),'b'); # creates surface plot in the axis
    ax.set_xlabel('$x_1$') # adds labels for the x-axis
    ax.set_ylabel('$x_2$') # adds labels for the y-axis
    ax.set_title("Budget constraint for Consumer 1") # adds title for the first plot
    ax.set_ylim(0,I1.value/P2.value) #sets the bound of the y-axis in the first plot
    ax.set_xlim(0,I1.value/P1.value) #sets the bound of the x-axis in the first plot
    ay = fig.add_subplot(1,2,2) # We add a second plot and place it in the top right corner
    ay.plot(x1_values,x2_func(x1_values,I2.value,P1.value,P2.value),'b'); # creates surface plot in the axis
    ay.set_xlabel('$x_1$') # adds labels
    ay.set_ylabel('$x_2$') # adds labels
    ay.set_title("Budget constraint for Consumer 2") # add title
    ay.set_ylim(0,I2.value/P2.value) # Sets a boundary for the y-axis
    ay.set_xlim(0,I2.value/P1.value) # Sets a boundary for the x-axis
Figure_1()
```

The results above are based on your choice of income, but given the same income values these should be identical. They illustrate the combinations of goods that the consumers can afford given thier income and the prices on the goods.

### **Utility function**

After setting the price levels and income levels, we define the utility functions.

Cobb-Douglass:
\\[
u_1(x_1,x_2) = x_1^{\alpha}x_2^{1-\alpha}
\\]
Logarithmic:
\\[
u_2(x_1,x_2) = \beta log(x_1)+(1-\beta) log(x_2)
\\]

These functions' parameters can easily be altered to fit your own problem using the widgets, which allows you to change the values of \\(\alpha\\) and \\(\beta\\).

Note that the subscript on *u* represents the consumer, 1 being the one with Cobb-Douglass preferences and 2 being the one with logarithmic preferences.

```python
# Consumer 1
def c1_u_func(x1,x2,alpha): # We define a Cobb-Douglass utility function
    return (x1**alpha)*(x2**(1-alpha))

print('\033[1m' + 'Consumer 1:')
print('\033[1m' + 'Alpha (\u03B1)') # We use this code \u03B1 and similar ones to print greek letters. This particular one is for alpha
alpha = widgets.FloatText(
value=0.5)
alphab = widgets.FloatSlider(
value=0.5,
step=0.1,
min=0.1,
max=1,)
display(alpha,alphab)

mylink = widgets.jslink((alpha, 'value'), (alphab, 'value'))

# Consumer 2

def c2_u_func(x1,x2,beta): # We define a logarithmic utility function
    return beta*np.log(x1)+(1-beta)*np.log(x2)

print('\033[1m' + 'Consumer 2:')
print('\033[1m' + 'Beta (\u03B2)')
beta = widgets.FloatText(
value=0.5)
betab = widgets.FloatSlider(
value=0.5,
step=0.1,
min=0.1,
max=1,)
display(beta,betab)

mylink = widgets.jslink((beta, 'value'), (betab, 'value'))
```

    Consumer 1:
    Alpha (α)

    FloatText(value=0.5)

    FloatSlider(value=0.5, max=1.0, min=0.1)

    Consumer 2:
    Beta (β)

    FloatText(value=0.5)

    FloatSlider(value=0.5, max=1.0, min=0.1)

We set the utility values so we can plot the indifference curves.

```python
# Widget for Consumer 1's utility
print('\033[1m' + 'Utility for consumer 1 (u1)')
u1 = widgets.FloatText(
value=10)
u1b = widgets.FloatSlider(
value=10,
step=0.1,
min=0.1,
max=10,)
display(u1,u1b)
mylink = widgets.jslink((u1, 'value'), (u1b, 'value'))

# Widget for Consumer 2's utility
print('\033[1m' + 'Utility for consumer 2 (u2)')
u2 = widgets.FloatText(
value=1)
u2b = widgets.FloatSlider(
value=1,
step=0.1,
min=0.1,
max=3,)
display(u2,u2b)
mylink = widgets.jslink((u2, 'value'), (u2b, 'value'))
```

    Utility for consumer 1 (u1)

    FloatText(value=10.0)

    FloatSlider(value=10.0, max=10.0, min=0.1)

    Utility for consumer 2 (u2)

    FloatText(value=1.0)

    FloatSlider(value=1.0, max=3.0, min=0.1)

Next we plot the utility functions using the values chosen above.

```python
# Next we define our 2 utility functions (Where we have isolated for x2)
def c1_x2_func2(x1,u,alpha):
    return u**(1/(1-alpha))*(x1**(-alpha/(1-alpha)))
def c2_x2_func2(x1,u,beta):
    return np.exp((u-beta*np.log(x1))/(1-beta))

def Figure_2():
    fig = plt.figure(figsize=(15,5)) # creates the figure
    ax = fig.add_subplot(1,2,1) # creates subplot
    ax.plot(x1_values,c1_x2_func2(x1_values,u1.value,alpha.value),'b'); # creates surface plot in the axis
    ax.set_xlabel('$x_1$') # add labels
    ax.set_ylabel('$x_2$') # add labels
    ax.set_title("Utility for Consumer 1") # add title
    ax.set_ylim([0, 20]) # here we limit the y axis, which is needed due to the values going towards negative infinity, when x1 goes towards 0.
    ay = fig.add_subplot(1,2,2) # creates subplot
    ay.plot(x1_values,c2_x2_func2(x1_values,u2.value,beta.value),'b'); # creates surface plot in the axis, but this time we use a different method due to the fact that we are using numpy functions (exp and log).
    ay.set(xlabel='$x_1$', ylabel='$x_2$',
       title="Utility for Consumer 2", xlim=[0,I2.value/P1.value], ylim=[0,I2.value/P2.value]) # We also need to set our labels and title in a different way.
Figure_2()
```

The figures illustrate the combination of goods to obtain the set utility levels from before. By default $u_1=10$ and $u_2=1$ in our widgets. This means that every point on the graph represent bundles of good 1 and 2, which corresponds to these utility levels.

### **Solution**

Now we solve the consumer problem for both types. This is done by maximizing the utility with regards to the budget constraint.
\\[
\begin{eqnarray*}
\max_{x_{1},x_{2}}u(x_{1},x_{2})\\
\text{s.t.}\\
p_{1}x_{1}+p_{2}x_{2} & \leq & I,\,\,\,p_{1},p_{2},I>0\\
x_{1},x_{2} & \geq & 0
\end{eqnarray*}
\\]

```python
# Consumer 1
def c1(x1,I,P1,P2,alpha): # First we define a function, which we will use to find the optimal value of x1.
    x2 = (I-P1*x1)/P2
# We return the negative function of our utility and then minimizes this, which is the same as maximizing the utility. The reason for this is that scipy use minimization.
    return -c1_u_func(x1,x2,alpha)

# Consumer 2
def c2(x1,I,P1,P2,beta):
    x2 = (I-P1*x1)/P2
    return -c2_u_func(x1,x2,beta)

# Next we use our optimizing function to maximize the utility (And thereby minimizing the 2 functions above.
# Here we define x1 to be between 0 and the maximum number of good 1 you can buy for your income
solver_c1 = optimize.minimize_scalar(c1, method='bounded', bounds=(0,I1.value/P1.value), args=(I1.value,P1.value,P2.value,alpha.value))
solver_c2 = optimize.minimize_scalar(c2, method='bounded', bounds=(0,I2.value/P1.value), args=(I2.value,P1.value,P2.value,beta.value))
# We then define our answers for both consumers

# Consumer 1
c1_x1 = solver_c1.x
c1_x2 = (I1.value-P1.value*c1_x1)/P2.value
c1_u = c1_u_func(c1_x1,c1_x2,alpha.value)

# Consumer 2
c2_x1 = solver_c2.x
c2_x2 = (I2.value-P1.value*c2_x1)/P2.value
c2_u = c2_u_func(c2_x1,c2_x2,beta.value)
```

This provides the following results:

```python
# These lines print our results

# Consumer 1
print('\033[1m' + 'Consumer 1:') # The first bit makes the text bold and then writes Consumer 1
print('\033[0m' + f'x1 = {c1_x1:.1f}') # The first but makes the text unbold and then prints the x1 value
print(f'x2 = {c1_x2:.1f}') # Prints the x2 value
print(f'u  = {c1_u:.2f}') # Prints the utility
print(f'I-p1*x1-p2*x2 = {I1.value-P1.value*c1_x1-P2.value*c1_x2:.2f}') # Prints a calculation to see whether the whole income has been used, such that income - consumption = 0

print ('')

# Consumer 2 (The following code is the same as for consumer 1)
print('\033[1m' + 'Consumer 2:')
print('\033[0m' + f'x1 = {c2_x1:.1f}')
print(f'x2 = {c2_x2:.1f}')
print(f'u  = {c2_u:.2f}')
print(f'I-p1*x1-p2*x2 = {I2.value-P1.value*c2_x1-P2.value*c2_x2:.2f}')
```

    Consumer 1:
    x1 = 10.0
    x2 = 3.3
    u  = 5.77
    I-p1*x1-p2*x2 = 0.00

    Consumer 2:
    x1 = 10.0
    x2 = 3.3
    u  = 1.75
    I-p1*x1-p2*x2 = 0.00

We can compare this answer to the one, we get from solving the problem manually.

**Consumer 1:**
\\[
\begin{eqnarray*}
x_1^{\ast} &=& \alpha \frac{I}{p_1} \\
x_2^{\ast} &=& (1-\alpha) \frac{I}{p_2}\\
u_1^{\ast} &=& u_{1}(x_1^{\ast},x_2^{\ast})
\end{eqnarray*}
\\]

**Consumer 2:**
\\[
\begin{eqnarray*}
x_1^{\ast} &=& \beta \frac{I}{p_1} \\
x_2^{\ast} &=& (1-\beta) \frac{I}{p_2}\\
u_2^{\ast} &=& u_{2}(x_1^{\ast},x_2^{\ast})
\end{eqnarray*}
\\]

```python
# Here we use our formulas to check if the answers is correct

# Consumer 1
print('\033[1m' + 'Consumer 1:')
print('\033[0m' + f'x1 = {alpha.value*I1.value/P1.value:.1f}') # Prints the x1 value
print(f'x2 = {(1-alpha.value)*I1.value/P2.value:.1f}') # Prints the x2 value
print(f'u  = {c1_u_func(alpha.value*I1.value/P1.value,(1-alpha.value)*I1.value/P2.value,alpha.value):.2f}') # Prints the utility

print ('')

# Consumer 2 (The code is the same as above)
print('\033[1m' + 'Consumer 2:')
print('\033[0m' + f'x1 = {beta.value*I2.value/P1.value:.1f}')
print(f'x2 = {(1-beta.value)*I2.value/P2.value:.1f}')
print(f'u  = {c2_u_func(beta.value*I2.value/P1.value,(1-beta.value)*I2.value/P2.value,beta.value):.2f}')
```

    Consumer 1:
    x1 = 10.0
    x2 = 3.3
    u  = 5.77

    Consumer 2:
    x1 = 10.0
    x2 = 3.3
    u  = 1.75

We see that the manual solution gives the same result as the numerical optimizer.
We also see that given the same income the consumption of the goods are identical, which is in accordance with the theory, as logarithmic utility can be written on the form of a Cobb-Douglas utility function, for which the consumption of goods depends on income, prices and the preferences illustrated by the exponents, $\alpha$ and $\beta$. However, one must note that even though the good combinations are identical (for the same parameters) the utility obtained are not.

We visualize our results by plotting the budget constraints against the utility functions.

```python
# Here we make a plot of the solution
def Figure_3():
    fig = plt.figure(figsize=(15,5)) # Start the plot and set the figure size
    ax = fig.add_subplot(1,2,1) # Adding the plot for consumer 1
    ax.plot(x1_values,c1_x2_func2(x1_values,c1_u,alpha.value),'b'); # Insert the optimal utility function
    ax.plot(x1_values,x2_func(x1_values,I1.value,P1.value,P2.value),'b'); # Insert the budget constraint
    ax.set_xlabel('$x_1$') # Set label
    ax.set_ylabel('$x_2$') # Set label
    ax.set_title("Solution for Consumer 1") # Set titel
    ax.set_ylim([0, I1.value/P2.value]) # Set boundaries of y-axis
    ax.set_xlim([0,I1.value/P1.value]) # Set boundaries of x-axis

    # The next line marks the solution, which is where the utility and budget constraint touches
    ax.plot(c1_x1,c1_x2, color='red', marker='o', linestyle='dashed', linewidth=5, markersize=10, fillstyle='none') # creates a red circle where the budget is tangent to the indifference curve

    # Next we do the same for Consumer 2 in a plot left of the one above
    ay = fig.add_subplot(1,2,2)
    ay.plot(x1_values,c2_x2_func2(x1_values,c2_u,beta.value),'b');
    ay.plot(x1_values,x2_func(x1_values,I2.value,P1.value,P2.value),'b')
    ay.set(xlabel='$x_1$', ylabel='$x_2$',
       title="Solution for Consumer 2", xlim=[0,I2.value/P1.value], ylim=[0,20])
    ay.plot(c1_x1, c1_x2, color='red', marker='o', linestyle='dashed', linewidth=5, markersize=10, fillstyle='none') # creates the red circle for consumer 2.
Figure_3()
```

We see that the utility function touches the budget constraint (shown by the red circle) at the results found before. This is illustrating where the budget constraint is tangent to the utility function, which is the optimal choice of the consumer. This is because this point is the highest indifference curve (the one providing the most utility) that can be reached given the budget contraint.

## **Social optimum**

First we define the necessary aggregated variables

```python
# We start by summing the values for income and utility to create social values and functions.
soc_I=I1.value+I2.value
def u_func(x1,x2,alpha,beta):
    return c1_u_func(x1,x2,alpha)+c2_u_func(x1,x2,beta)
```

And then we solve for the social optimum:

\\[
\begin{eqnarray*}
\max_{x_{1},x_{2}}u_1(x_{1},x_{2})+u_{2}(x_{1},x_{2})\\
\text{s.t.}\\
p_{1}x_{1}+p_{2}x_{2} & \leq & I_{1}+I_{2},\,\,\,p_{1},p_{2},I_{1},I_{2}>0\\
x_{1},x_{2} & \geq & 0
\end{eqnarray*}
\\]

**Notice:** We do not plot the graphs since isolateing \\(x_2\\) isn't possible for costumer 2.

```python
# We start by creating the functions which we are going to optimize. Here we return the u_func and contrain it to the budget constraint
def c1_and_c2(x1,I,P1,P2,alpha,beta):
    x2 = (I-P1*x1)/P2
    return -u_func(x1,x2,alpha,beta)

# Solve the problem by scipy
solver = optimize.minimize_scalar(c1_and_c2, method='bounded', bounds=(0,soc_I/P1.value), args=(soc_I,P1.value,P2.value,alpha.value,beta.value))

# Set values
soc_x1 = solver.x
soc_x2 = (soc_I-P1.value*soc_x1)/P2.value
soc_u = u_func(soc_x1,soc_x2,alpha.value,beta.value)

# Print values
print('\033[1m' + 'Social optimum:')
print('\033[0m' + f'x1 = {soc_x1:.1f}')
print(f'x2 = {soc_x2:.1f}')
print(f'u  = {soc_u:.2f}')
print(f'I-p1*x1-p2*x2 = {soc_I-P1.value*soc_x1-P2.value*soc_x2:.2f}')
```

    Social optimum:
    x1 = 20.0
    x2 = 6.7
    u  = 13.99
    I-p1*x1-p2*x2 = 0.00

To check whether this answer is correct we are going to solve the social optimum problem using sympy.

```python
# Define all the three variables
socialx1, socialx2, sociallambda = symbols('socialx1 socialx2 sociallambda')

# Write up the Lagrange function, which is L(x1,x2,lambda)=u_consumer1(x1,x2)+u_consumer2(x1,x2)-lambda(I1+I2-p1*x1-p2*x2)
eq_L = socialx1**alpha.value*socialx2**(1-alpha.value)+beta.value*log(socialx1)+(1-beta.value)*log(socialx2)-sociallambda*(soc_I-P1.value*socialx1-P2.value*socialx2)

# Differentiate the lagrange function with regards to our three variables
eq_dLdx1 = eq_L.diff(socialx1)
eq_dLdx2 = eq_L.diff(socialx2)
eq_dLdlambda = eq_L.diff(sociallambda)

# Maximize the three differentials with regards to x1,x2 and lambda and creating af dictionary
solution_set = solve([eq_dLdx1,eq_dLdx2,eq_dLdlambda], socialx1,socialx2,sociallambda, dict=True)
solution_set_dict=solution_set[0]

# We then divide the dictionary to get the optimal x1 and x2 and then insert these into the combined utility function
soc2_x1=solution_set_dict[socialx1]
soc2_x2=solution_set_dict[socialx2]
soc2_u=soc2_x1**alpha.value*soc2_x2**(1-alpha.value)+beta.value*log(soc2_x1)+(1-beta.value)*log(soc2_x2)

# And lastly we print the three values, x1,x2 and u.
print('\033[1m' + 'Social optimum:')
print('\033[0m' + f'x1 = {soc2_x1:.1f}')
print(f'x2 = {soc2_x2:.1f}')
print(f'u  = {soc2_u:.2f}')
```

    Social optimum:
    x1 = 20.0
    x2 = 6.7
    u  = 13.99

We can then compare this result to our earlier results from solving the consumer problems individually. We see that the changes is given by:

```python
# Print to see the changes between the individual results and the social result
print('\033[1m' + 'Changes:')
print('\033[0m' + f'\u0394x1 = {soc_x1-c1_x1-c2_x1:.1f}')
print(f'\u0394x2 = {soc_x2-c1_x2-c2_x2:.1f}')
print(f'\u0394u = {soc_u-c1_u-c2_u:.2f}')
```

    Changes:
    Δx1 = 0.0
    Δx2 = 0.0
    Δu = 6.47

The higher utility is due to allocating of the goods, such that consumer 1 consumes more of the collective resources, as he has a higher utility gained from consumption of the goods.

## **Firms maximization problem**

 ### **Determine the competitive situation**

First we need to select the competitive situation of the problem as this has influence on the firms price setting.

```python
# Create a widget with 2 posible outcomes: Competitive or Monopolistic
competition=widgets.RadioButtons(
    options=['Competitive', 'Monopolistic'],
    value='Competitive',
    description='Competition:',
    disabled=False
)
competition
```

    RadioButtons(description='Competition:', options=('Competitive', 'Monopolistic'), value='Competitive')

### **Set demand according to the competition**

```python
# We make an if statement to check the competition and if it is competitive we make a widget of the price and else we make a demand func.

# Start of if statement
if ('Competitive'==competition.value): # This is the start of an if statement, which checks which form of competition was choosen.

    # Then we print a short text
    print('Since you have picked the competitive market, you need to choose the price, p, at which the other firms sell their goods')
    print('')
    # And then we make and print the widget
    print('\033[1m' + 'Price on goods (P)')
    P = widgets.FloatText(
    value=15)
    Pb = widgets.FloatSlider(
    value=5,
    step=1,
    min=1,
    max=100.0,)
    display(P,Pb)
    linkP = widgets.jslink((P, 'value'), (Pb, 'value'))

# Start of else statement
else:

    # We again start with a short text
    print('Since you have picked monopolistic competition, we need to define the demand function of the consumer. Here we use a linear function (D(x)=b-a*x), where the slope, a, and the constant term, b, are determined by the widgets below.')
    print('')

    # And the we make the demand function
    # First we determine a
    print('\033[1m' + 'Slope of demand function (a)')
    a = widgets.FloatText(
    value=5)
    aa = widgets.FloatSlider(
    value=5,
    step=0.5,
    min=1,
    max=10.0,)
    display(a,aa)
    linka = widgets.jslink((a, 'value'), (aa, 'value'))

    # Then we determine b
    print('\033[1m' + 'Constant term (b)')
    b = widgets.FloatText(
    value=200)
    bb = widgets.FloatSlider(
    value=200,
    step=1,
    min=1,
    max=500.0,)
    display(b,bb)
    linkb = widgets.jslink((b, 'value'), (bb, 'value'))
# Note that we don't define the demand function yet since the user need to be able to change the variables first. Therefore, we define the demand function, when solving the problem
```

    Since you have picked the competitive market, you need to choose the price, p, at which the other firms sell their goods

    Price on goods (P)

    FloatText(value=15.0)

    FloatSlider(value=5.0, min=1.0, step=1.0)

### **Define the Total Cost function**

Next we define our total cost function, which we define as a quadratic form. (We have included total cost, as it is needed to determine whether the firm wants to enter the market or not)

\\[
TC(x) = c x^2+FC
\\]

```python
# Determine c as a widget
print('\033[1m' + 'Slope, c')
c = widgets.FloatText(
value=0.5)
cb = widgets.FloatSlider(
value=0.5,
step=0.1,
min=0,
max=10.0,)
display(c,cb)
linkb = widgets.jslink((c, 'value'), (cb, 'value'))

# Determine fixed costs as a widget
print('\033[1m' + 'Fixed costs (FC)')
FC = widgets.FloatText(
value=10)
FCb = widgets.FloatSlider(
value=10,
step=1,
min=0,
max=100.0,)
display(FC,FCb)
linkb = widgets.jslink((FC, 'value'), (FCb, 'value'))
```

    Slope, c

    FloatText(value=0.5)

    FloatSlider(value=0.5, max=10.0)

    Fixed costs (FC)

    FloatText(value=10.0)

    FloatSlider(value=10.0, step=1.0)

```python
def TC(x,c,FC): # defining the TC function
    return c*x**2+FC
x = Symbol('x')
def MC(x,c): # defining the MC function
    return 2*c*x

# Here we create a vector of prices, which we are going to use in our functions. It the same as the privous one with x1.
p_x_max=50 # Here set the capacity limit at 50
shape_tuple = (N,N)
p_x_values = np.empty(shape_tuple)
for i in range(N):
    for j in range(N):
        p_x_values[i,j] = (i/(N-1))*p_x_max

# And lastly we plot our Total cost function
def Figure_7():
    fig = plt.figure(figsize=(8,5))
    ax = fig.add_subplot(1,1,1)
    ax.plot(p_x_values,TC(p_x_values,c.value,FC.value), 'b');
    ax.set_xlabel('$x$')
    ax.set_ylabel('$TC$')
    ax.set_title('Total cost')
Figure_7()
```

We see that given our definition of the cost function, the MC is increasing in x, as TC increases more than 1:1 in x.

### **Solution**

```python
# We start by printing a few words, which we  are going to use a the start of our conclusion
print('We now solve the maximization problem by maximizing the profit of the firm.')

# Next we start an if statement to chech for the competition and print another line depending on that
if ('Competitive'==competition.value):
    print('And since you have choosen the competitive market. The profit function of the firm is \u03C0=px-TC(x)')
elif ('Monopolistic'==competition.value):
    print('And since you have choosen the monopolistic market. The profit function of the firm is \u03C0=D(x)p-TC(x)')

# Next we start the same if statement but now we will solve the problem depending on the competition

# The first the the competitive market
if ('Competitive'==competition.value):
    # Here we start by defining the function that we are going to optimize
    def pi_competitive(x,P,c,FC):
        return -(P*x-TC(x,c,FC)) # Here we define pi as the profit times -1, so that we can use scipy optimize
    # Next we use scipy optimize to solve our problem
    solver4 = optimize.minimize_scalar(pi_competitive, method='bounded', bounds=(0,p_x_max), args=(P.value,c.value,FC.value))

    # We define our answers as
    x_opt1 = solver4.x
    price1 = P.value
    pi1=P.value*x_opt1-TC(x_opt1,c.value,FC.value)
    TC1 = TC(x_opt1,c.value,FC.value)
    REV1  = P.value*x_opt1

    # And then print the results
    print('The results are:')
    print(f'x = {x_opt1:.1f}')
    print(f'Price = {P.value:.1f}')
    print(f'\u03C0 = {pi1:.1f}')
    print(f'Total cost  = {TC1:.1f}')
    print(f'Revenue  = {REV1:.1f}')

    # We also chekcs to see whether the firm wants to produce anything. This is done in the folowing if statement
    if pi1>-FC.value and pi1<0: # So if the profit is negative but above the fixed cost we still want to produce but we still print a line commenting this
        print('')
        print('Although the profit is negative, the firm will still produce since they must cover some of the fixed costs')
    elif pi1<-FC.value: # If the profit is below the negative value of the fixed costs the firm should not produce
        print('Since the profit is less than the fixed costs, the firm would not produce anything, as this will lower their profits')

    # Next we print the next part of our conclusion, which is going to introduce the figure
    print('')
    print('We show this result by plotting the marginal cost against the marginal revenue. The marginal costs is the first derivative')
    print('of the total cost function. This means MC(x)=2cx. The marginal revenue is equal to the price level, p. This gives us the following plot.')
    print('It shows that for MR=MC, the firm maximizes its profit.')

    # Next we define the figure, which shows us our result
    def Figure_8():
        fig = plt.figure(figsize=(15,5))
        # We start by adding a plot for MR and MC
        ax = fig.add_subplot(1,2,1)
        ax.plot(p_x_values,MC(p_x_values,c.value),'b'); # We plot our marginal cost
        ax.hlines(y=price1, xmin=0, xmax=p_x_max) # This code plots the marginal revenue, which is constant at the price level. So here we plot an horisontal line.
        ax.set_xlabel('$x$') # Add label
        ax.set_ylabel('$p$') # Add label
        ax.set_title('Marginal cost and Marginal revenue') # Add title

        # The next line mark the solution
        ax.plot(x_opt1, P.value, color='red', marker='o', linestyle='dashed', linewidth=5, markersize=10, fillstyle='none')

        # Next we add a plot for the profit
        ay = fig.add_subplot(1,2,2)
        ay.plot(p_x_values,TC(p_x_values,c.value,FC.value),'b');
        ay.hlines(y=price1, xmin=0, xmax=p_x_max, color='blue') # This code plots a the price level, which is costant in a competitive market
        ay.set_xlabel('$x$')
        ay.set_ylabel('$p$')
        ay.set_title('Total cost and Price')
        ay.set_xlim(0,20)
        ay.set_ylim(0,100)
    Figure_8()

# Next we do the same for the monopolistic competition
elif ('Monopolistic'==competition.value):
    def pi_monopolistic(x,b,a,c,FC):
        return -((b-a*x)*x-TC(x,c,FC)) # Here we define pi as the profit times -1, so that we can use scipy optimize

    solver5 = optimize.minimize_scalar(pi_monopolistic, method='bounded', bounds=(0,p_x_max), args=(b.value,a.value,c.value,FC.value))

    def MR2(x,b,a): # Here we define the marginal revenue which isnt constant as it was in the competitive market
        return b-2*a*x

    # We define our answers as
    x_opt2 = solver5.x
    price2 = b.value-a.value*x_opt2
    pi2=(b.value-a.value*x_opt2)*x_opt2-TC(x_opt2,c.value,FC.value)
    TC2 = TC(x_opt2,c.value,FC.value)
    REV2  = (b.value-a.value*x_opt2)*x_opt2
    print('The results are:')
    print(f'x = {x_opt2:.1f}')
    print(f'Price = {price2:.1f}')
    print(f'\u03C0 = {pi2:.1f}')
    print(f'Total cost = {TC2:.1f}')
    print(f'Revenue = {REV2:.1f}')

    if pi2>-FC.value and pi2<0:
        print('')
        print('Although the profit is negative, the firm will still produce since they must cover some of the fixed costs')
    elif pi2<-FC.value and pi2<0:
        print('Since the profit is less than the fixed costs, the firm would not produce anything, as this will lower their profits')

    print('')
    print('We show this result by plotting the marginal cost against the marginal revenue. The marginalcosts is the first derivative')
    print('of the total cost. This means MC(x)=2cx. The marginal revenue is equal to MR(x)=b-2ax.')
    print('When plotting this, we notice that when MR=MC, the firm maximizes its profit.')
    print('And as they have monopolly, they find the optimal value of x, when MR=MC, and sets the optimal price on the demand function according to this.')

    def Figure_9():
        fig = plt.figure(figsize=(15,5)) # creates the figure
        ax = fig.add_subplot(1,2,1)
        ax.plot(p_x_values,b.value-a.value*p_x_values, 'b' );
        ax.plot(p_x_values,MC(p_x_values,c.value), 'r');
        ax.plot(p_x_values,MR2(p_x_values,b.value,a.value), 'r');
        ax.plot(x_opt2, MR2(x_opt2,b.value,a.value), color='blue', marker='*', linestyle='dashed', linewidth=5, markersize=10, fillstyle='full')
        ax.plot(x_opt2, b.value-a.value*x_opt2, color='red', marker='o', linestyle='dashed', linewidth=5, markersize=10, fillstyle='full')
        ax.vlines(x=x_opt2, ymin=0, ymax=price2) # This code plots a vertical line at the optimal value of x
        ax.set_xlabel('$x$')
        ax.set_ylabel('$p$')
        ax.set_ylim(0,b.value)
        ax.set_xlim(0,b.value/a.value)
        ax.set_title('Marginal cost and Marginal revenue')
        ay = fig.add_subplot(1,2,2)
        ay.plot(p_x_values,TC(p_x_values,c.value,FC.value),'b');
        ay.plot(p_x_values,b.value-a.value*p_x_values,'b')
        ay.set_xlabel('$x$')
        ay.set_ylabel('$p$')
        ay.set_title('Total cost and Demand function')
        ay.set_ylim(0,b.value)
        ay.set_xlim(0,b.value/a.value)
    Figure_9()
```

```python

```