# Mandelbrot Polynomials and Matrices

```{image} ../Figures/Mandelbrot/explosion32comp.gif
:height: 400px
:alt: Exploding Mandelbrot 32
:align: center
```

This unit uses some ideas of the late [Benoit Mandelbrot](https://en.wikipedia.org/wiki/Benoit_Mandelbrot) and some known facts about the so-called _Mandelbrot Set_, together with some of our own ideas, to help you to learn the following:

1. A bit more Python programming, including more about _iteration_, _recursion_, _loop invariants_, _automatic differentiation_, _symbolic computation_, the use of _numerical libraries_ from NumPy, and a bit more about graphics.
2. A bit more about polynomials: the _cubic formula_ (we won't need the quartic formula, and even our use of the cubic formula is a bit contrived; but we think it's fun), the _cost of numerical solution of polynomials_ (we will point you to the current numerical champion polynomial solver, [MPSolve](https://en.wikipedia.org/wiki/MPSolve)), and the rather necessary-to-know notions of _numerical stability_ and _conditioning_.  Most of these topics are not taught as thoroughly as they should be, in one's first numerical analysis course; so this material here should strengthen the results when you do encounter them.
3. The surprising use of _eigenvalues_ to find roots of polynomials.  The first treatment of eigenvalues (typically in a Linear Algebra course, even though the problem is not, strictly speaking, linear) usually runs the other way around, and defines the eigenvalues of a matrix in terms of the _characteristic polynomial_ of the matrix. Indeed, we will teach you a concept not in the textbooks: namely, the concept of a _minimal height companion matrix_ and we will show you one such for the Mandelbrot polynomials.
4. A bit of _dynamical systems_, especially about _iteration_ and _composition_.
5. An excellent approximate formula for the _largest magnitude root_ of the Mandelbrot polynomial.
6. An honest-to-goodness analytic solution to the Mandelbrot iteration (this is a very new result, published only in 2021), valid for all $c$ _outside_ the Mandelbrot set.
7. That we don't know everything about Mandelbrot polynomials and matrices, and that you might be able to answer some open questions.

## The Mandelbrot Set and Mandelbrot Polynomials

The [Wikipedia article on the Mandelbrot Set](https://en.wikipedia.org/wiki/Mandelbrot_set) is pretty comprehensive, and readable. We'll sum up the definition here, and try to flesh it out a bit.  The definition is based on the following, apparently very simple, function:
\begin{equation*}
f(z) = z^2 + c
\end{equation*}
This is apparently only a simple quadratic in $z$, and depends linearly on the constant $c$.  We now consider _what happens when we repeatedly apply this function_, starting at 0 (an apparently simple point).  That is, put
\begin{equation*}
z_0 = 0
\end{equation*}
and then define $z_k$ for $k=1$, $k=2$, $k=3$, and so on in turn by the iterative formula (or recursive formula, or _dynamical_ formula)
\begin{equation*}
z_{n+1} = f(z_n) = z_n^2 + c .
\end{equation*}
This gives us $z_1 = z_0^2 + c = 0^2 + c = c$, and then subsequently $z_2 = z_1^2 + c = c^2 + c$, and then $z_3 = (c^2+c)^2 + c = c^4 + 2c^3 + c^2 + c$, and so on.

Several important things are going on here: this looks simple, and in one sense it is, but we will see some surprising things from this.  This is an example of a _nonlinear dynamical system_ (perhaps the "most famous" example of a nonlinear dynamical system).  We are repeatedly composing the function $f(z) = z^2+c$ with itself: $z_2 = f(c)$ and $z_3 = f(f(c))$ and $z_4 = f(f(f(c))$. Sometimes repeated composition is written with a power in brackets: $f^{(n)}(c)$ means $f(f(f(\cdots(f(c))\cdots)))$ where there are $n$ copies of $f$ used; this is where the pernicious notation $f^{(-1)}(c)$ comes from, by the way. We will try to avoid this confusing notation.  Several things can be deduced about repeated composition.  For instance, the result of composing two polynomials together is another polynomial.  If $f(c)$ is of degree $m$ and $g(c)$ is of degree $m$ then $f(g(c))$ is of degree $mn$, and so is $g(f(c))$.  If we were working with non-polynomials then we would also have to worry about matching up the domains and ranges, but polynomials are great because they always give a finite result output for any finite value input, and so composition always works.

We have written $z_3$ out above as an explicit degree $4$ polynomial (in the variable $c$), but if we actually want to compute a numerical value of $z_3$ given a numerical value for $c$ (say, $c=-1.2$) then it turns out to be better in several ways to just use the iterative formula itself and not the explicit polynomial. This is true even though we could write that explicit polynomial somewhat more efficiently in what is known as [Horner form](https://en.wikipedia.org/wiki/Horner%27s_method): $z_3 = c\cdot(1 + c\cdot(1 + c\cdot(2 + c)))$, which can be evaluated using only three floating-point multiplications and three floating-point additions. Using the iterative formula directly, though, there is no work to compute $z_0$, there is no floating-point work to compute $z_1 = c$, there is one multiplication and one addition to compute $z_2$, and one more multiplication and one more addition to compute $z_3$, showing that $z_3$ can be computed in only two "flops" instead of three.  Note: one "flop" is _defined to be_ one floating-point multiplication (or division) together with one floating-point addition (or subtraction).  It's not a fine-tuned measure; it was meant for use in older analyses of the cost of computation.  On modern architectures, the notion is not really all that helpful; but you can see here that no matter what, the cost is less if we use the iteration directly.  This advantage only increases as the iteration proceeds: to compute $z_4$ you only need one more "flop," i.e. $3$ flops, whereas from its polynomial form you need $2^3 = 8$ flops (even ignoring the work that has to be done simply to write the polynomial out).  For $z_{30}(c)$ the iteration takes only 29 flops; but the explicit polynomial is degree $2^{29}$ and would take $2^{29}$ flops to evaluate (actually, it's worse: we would have to use multiple precision to deal with the big integer coefficients).  Thus, somehow, the iteration has _compressed_ a very high degree polynomial into a very efficient box.

Another important thing to notice in the above is that by leaving $c$ symbolic, we have turned Mandelbrot's iteration formula (or recursion formula) into some kind of generator for certain polynomials.  These are called (naturally enough) _Mandelbrot polynomials_.  We list the first few here, or at least their coefficients.

```python
import numpy as np
from numpy.polynomial import Polynomial as Poly
N = 8

z0 = Poly([0])
print( z0 )
c = Poly([0,1]) # We can do "symbolic polynomial arithmetic" just with these coefficient vectors in Python
z1 = z0*z0 + c
print( z1 )
z2 = z1*z1 + c
print( z2 )
z3 = z2*z2 + c
print( z3 )
z4 = z3*z3 + c
print( z4 )
z5 = z4*z4 + c
print( z5 )
```

    0.0
    0.0 + 1.0 x**1
    0.0 + 1.0 x**1 + 1.0 x**2
    0.0 + 1.0 x**1 + 1.0 x**2 + 2.0 x**3 + 1.0 x**4
    0.0 + 1.0 x**1 + 1.0 x**2 + 2.0 x**3 + 5.0 x**4 + 6.0 x**5 + 6.0 x**6 +
    4.0 x**7 + 1.0 x**8
    0.0 + 1.0 x**1 + 1.0 x**2 + 2.0 x**3 + 5.0 x**4 + 14.0 x**5 + 26.0 x**6 +
    44.0 x**7 + 69.0 x**8 + 94.0 x**9 + 114.0 x**10 + 116.0 x**11 +
    94.0 x**12 + 60.0 x**13 + 28.0 x**14 + 8.0 x**15 + 1.0 x**16

Translating that Python notation for polynomials back into something standard, we have
\begin{align}
z_0 &= 0 \\
z_1 &= c \\
z_2 &= c + c^2 \\
z_3 &= c + c^2 + 2c^3 + c^4\\
z_4 &= c + c^2 + 2c^3 + 5c^4 + 6c^5 + 6c^6 + 4c^7 + c^8 \\
z_5 &= c + c^2 + 2c^3 + 5c^4 + 14c^5 + 26c^6 + 44c^7 + 69c^8 + 94c^9 + 114c^{10} + 116c^{11} + 94c^{12} + 60c^{13} + 28c^{14} + 8c^{15} + c^{16}
\end{align}

The Mandelbrot set is _defined to be_ the set of $c$ for which this iteration remains bounded for all $n$: that is, everything _except_ those values of $c$ for which $|z_n| \to \infty$.  That's a weird sort of definition, and you have to first reassure yourself that there really are values of $c$ for which the $z_n$ do _not_ remain bounded.  Indeed, almost the first thing you would try was $c=1$: then $z_0 = 0$, $z_1 = 1$, $z_2 = 2$, $z_3 = 5$, $z_4 = 26$, and so on; each succeeding number is one more than the square of the previous, and this goes to infinity rapidly, in some sense.

Let us write a short program to evaluate Mandelbrot polynomials of arbitrary order.  This code shows several features: how to define a procedure (function); how to set a type for a parameter (more of a hint than a requirement, sadly); the use of keyword arguments; and the use of comments.

```python
# This short program will evaluate Mandelbrot polynomials at a given point,
# or else it will compute the coefficient vector of a Mandelbrot polynomial.
# Which it will do depends on the type of the argument "c."
def Mandelbrot(n: int, c):
    z = 0*c # try to inherit the type of c
    for i in range(n):
        z = z*z + c
    return(z)

z = Mandelbrot( 4, Poly([0, 1]) ) # The above code works with Polys (defined in previous cell)
print( z )

z = Mandelbrot( 4, -1.2 ) # The code also works with numeric input
print( z )

z = Mandelbrot( 21, 1.0j ) # Will work with complex numbers also
print( z )

z = Mandelbrot( 30, -1.2 ) # Since it is iterative, we can ask for high iteration numbers on numerical input
print( z )

# Don't try that with c = Poly([0,1]) though; the result would be a vector of length half a billion or so

zbig = Mandelbrot( 8, Poly( [0, 1] ) ) # n=8 is ghastly already; n=9 is twice as ghastly
print( zbig )

# If you forget the ordering, you can use keyword arguments ("kwargs")
zforgot = Mandelbrot( c=Poly([0,1]), n=5 )
print( zforgot )
```

    0.0 + 1.0 x**1 + 1.0 x**2 + 2.0 x**3 + 5.0 x**4 + 6.0 x**5 + 6.0 x**6 +
    4.0 x**7 + 1.0 x**8
    0.10507775999999969
    -1j
    0.17381269635430296
    0.0 + 1.0 x**1 + 1.0 x**2 + 2.0 x**3 + 5.0 x**4 + 14.0 x**5 + 42.0 x**6 +
    132.0 x**7 + 429.0 x**8 + 1302.0 x**9 + 3774.0 x**10 + 10652.0 x**11 +
    29538.0 x**12 + 80812.0 x**13 + 218324.0 x**14 + 582408.0 x**15 +
    1534301.0 x**16 + 3993030.0 x**17 + 10269590.0 x**18 + 26108844.0 x**19 +
    65626918.0 x**20 + 163107044.0 x**21 + 400844588.0 x**22 +
    974083128.0 x**23 + 2340595778.0 x**24 + 5560968284.0 x**25 +
    13062923500.0 x**26 + 30336029592.0 x**27 + 69640352964.0 x**28 +
    158015533208.0 x**29 + 354347339496.0 x**30 + 785248461712.0 x**31 +
    1719477330477.0 x**32 + 3720187393990.0 x**33 + 7952125694214.0 x**34 +
    16792863663700.0 x**35 + 35031835376454.0 x**36 + 72188854953372.0 x**37 +
    146932182777116.0 x**38 + 295372837865192.0 x**39 +
    586400982013486.0 x**40 + 1149605839249820.0 x**41 +
    2225301467579844.0 x**42 + 4252710138415640.0 x**43 +
    8022825031835276.0 x**44 + 1.493886254800156e+16 x**45 +
    2.74522110625734e+16 x**46 + 4.977884824296494e+16 x**47 +
    8.905447314769734e+16 x**48 + 1.5716052351565462e+17 x**49 +
    2.7355172158080035e+17 x**50 + 4.6954064603904256e+17 x**51 +
    7.946434187602729e+17 x**52 + 1.3257523767902403e+18 x**53 +
    2.1800537744427668e+18 x**54 + 3.5327112592255063e+18 x**55 +
    5.640327912922026e+18 x**56 + 8.870996681171367e+18 x**57 +
    1.3741246529612442e+19 x**58 + 2.095927615188073e+19 x**59 +
    3.1472438318100877e+19 x**60 + 4.651494458339957e+19 x**61 +
    6.764924725333256e+19 x**62 + 9.679171961159197e+19 x**63 +
    1.3621049366959063e+20 x**64 + 1.88481251186354e+20 x**65 +
    2.563862282500011e+20 x**66 + 3.427436298110824e+20 x**67 +
    4.501599369559943e+20 x**68 + 5.8070677903005857e+20 x**69 +
    7.355370500364918e+20 x**70 + 9.144707579144349e+20 x**71 +
    1.1155975817333279e+21 x**72 + 1.3349570927521004e+21 x**73 +
    1.566365198635996e+21 x**74 + 1.801452751402956e+21 x**75 +
    2.0299665953207943e+21 x**76 + 2.2403538973044617e+21 x**77 +
    2.420609646335252e+21 x**78 + 2.559320275988284e+21 x**79 +
    2.6467918122462075e+21 x**80 + 2.6761185429789726e+21 x**81 +
    2.644036970936308e+21 x**82 + 2.5514255916439574e+21 x**83 +
    2.40335441894389e+21 x**84 + 2.2086534878322607e+21 x**85 +
    1.9790454080732723e+21 x**86 + 1.727958521630465e+21 x**87 +
    1.4691893415965517e+21 x**88 + 1.2156044111615272e+21 x**89 +
    9.780579233191513e+20 x**90 + 7.646558443405197e+20 x**91 +
    5.804305658425433e+20 x**92 + 4.274173538740883e+20 x**93 +
    3.0506058020522374e+20 x**94 + 2.1083592136150558e+20 x**95 +
    1.4096018354614477e+20 x**96 + 9.107194359314248e+19 x**97 +
    5.679679982609651e+19 x**98 + 3.4150590308701282e+19 x**99 +
    1.977232248195698e+19 x**100 + 1.1008161481780605e+19 x**101 +
    5.88491770051913e+18 x**102 + 3.0161914185066383e+18 x**103 +
    1.4795944964627553e+18 x**104 + 6.934349554985458e+17 x**105 +
    3.098816487096832e+17 x**106 + 1.317607701576062e+17 x**107 +
    5.318195959195802e+16 x**108 + 2.0324543852025936e+16 x**109 +
    7333879739219600.0 x**110 + 2490875091238112.0 x**111 +
    793548088258508.0 x**112 + 236221241425176.0 x**113 +
    65418624260840.0 x**114 + 16771945556496.0 x**115 +
    3958458557608.0 x**116 + 854515874096.0 x**117 + 167453394320.0 x**118 +
    29524775520.0 x**119 + 4634116312.0 x**120 + 639097008.0 x**121 +
    76185104.0 x**122 + 7685024.0 x**123 + 637360.0 x**124 + 41696.0 x**125 +
    2016.0 x**126 + 64.0 x**127 + 1.0 x**128
    0.0 + 1.0 x**1 + 1.0 x**2 + 2.0 x**3 + 5.0 x**4 + 14.0 x**5 + 26.0 x**6 +
    44.0 x**7 + 69.0 x**8 + 94.0 x**9 + 114.0 x**10 + 116.0 x**11 +
    94.0 x**12 + 60.0 x**13 + 28.0 x**14 + 8.0 x**15 + 1.0 x**16

One advantage of using Polynomials from NumPy is that they "know how to find their own roots".  Let's test that out; the Mandelbrot polynomials turn out to be tough customers, though (which is why they are used as test problems for [MPSolve](https://en.wikipedia.org/wiki/MPSolve), not at all coincidentally—we learned about them first from the MPSolve test suite).

```python
N = 5
p = Mandelbrot( N, Poly([0,1]))
rts = p.roots()
# print( rts )
import matplotlib.pyplot as plt
x = [rt.real for rt in rts]
y = [rt.imag for rt in rts]
rootplot = plt.figure( figsize=(8,8) )
plt.scatter( x, y, s=15, marker=".")
plt.xlim(-2.5,0.5)
plt.ylim(-1.5,1.5)
plt.show()
```

```python
# Great.  Are those roots correct?
residuals = [p(rt) for rt in rts]
# print( residuals ) shows them all, and it's a confusing mess
absresiduals = [abs(r) for r in residuals ]
print( max(absresiduals) )
```

    3.251288986100655e-11

That's, that's ok; it looks like we have in every case found the exact root of a polynomial different from $z_5(c)$ by no more than $3\cdot 10^{-11}$.  Wait, what?  Well, putting in a computed root back to the polynomial (assuming it evaluates correctly) gives us a residual, say $r$.  That is $p(c^*) = r$, some (hopefully small) complex number.  But then $c^*$ is the _exact_ root of the following polynomial: $p(c) - r$.

Two questions: Did Python evaluate the polynomial well? That is, are the residuals accurate?  And second, what happens to a root if we slightly change a polynomial?  We've just wandered into numerical analysis territory.  Let's first compare the built-in evaluation of a polynomial by its coefficients to our iterative code.

```python
r1 = rts[0]
residual1byPython = p( r1 )
residual1byIteration = Mandelbrot( n=N, c=r1 )
print( "Using n = ", N, "\nFor the computed root ", r1, "The residual by Python is ", residual1byPython,
      "\nwhile the residual by iteration is ", residual1byIteration )
```

    Using n =  5
    For the computed root  (-1.9854242530544686+0j) The residual by Python is  (3.251288986100655e-11+0j)
    while the residual by iteration is  (2.8044677691241304e-11+0j)

We see some minor difference in that computation.  But it seems nothing to worry about.  Perhaps we should try a larger N, just to be sure?

```python
N = 6
p = Mandelbrot( N, Poly([0,1]))
rts = p.roots()
residuals = [p(rt) for rt in rts]
# print( residuals ) shows them all, and it's a confusing mess
absresiduals = [abs(r) for r in residuals ]
print( max(absresiduals) )
r1 = rts[0]
residual1byPython = p( r1 )
residual1byIteration = Mandelbrot( n=N, c=r1 )
print( "Using n = ", N, "and degree d = ", p.degree(), "\nFor the computed root ", r1, "The residual by Python is ", residual1byPython,
      "\nwhile the residual by iteration is ", residual1byIteration )
x = [rt.real for rt in rts]
y = [rt.imag for rt in rts]
rootplot = plt.figure( figsize=(8,8) )
plt.scatter( x, y, s=15, marker=".")
plt.xlim(-2.5,0.5)
plt.ylim(-1.5,1.5)
plt.show()
```

Ooh, that's not very good.  Try n=7 and see it get much worse; n=8 and worse yet (ridiculously worse). The issue seems to either be Python's polynomial rootfinder (we kind of thought it would be bad, but were hoping it was better than that), or its evaluation of the polynomial from its coefficients.  Of course the two things are connected.

```python
N = 7
p = Mandelbrot( N, Poly([0,1]))
rts = p.roots()
residuals = [p(rt) for rt in rts]
# print( residuals ) shows them all, and it's a confusing mess
absresiduals = [abs(r) for r in residuals ]
print( max(absresiduals) )
r1 = rts[0]
residual1byPython = p( r1 )
residual1byIteration = Mandelbrot( n=N, c=r1 )
print( "Using n = ", N, "and degree d = ", p.degree(), "\nFor the computed root ", r1, "The residual by Python is ", residual1byPython,
      "\nwhile the residual by iteration is ", residual1byIteration )
x = [rt.real for rt in rts]
y = [rt.imag for rt in rts]
rootplot = plt.figure( figsize=(8,8) )
plt.scatter( x, y, s=15, marker=".")
plt.xlim(-2.5,0.5)
plt.ylim(-1.5,1.5)
plt.show()
```

The fact that the roots near $-2$ have _exploded_ (see the fireworks GIF at the beginning) can be explained using what are known as _pseudozeros_.

Instead of pursuing that now, though, let us do something different. Namely, let us introduce an approximate formula for the _largest magnitude root_:
\begin{equation*}
\rho_k \approx -2 + \frac32 \pi^2 4^{-k} + O(4^{-2k}) .
\end{equation*}
This formula was published in 2013 by one of us (RMC) and his then-student Piers W. Lawrence, and with the help of another of us (NJC).  We (the present authors, including EYSC) discuss it a bit in our paper [Some Facts and Conjectures on Mandelbrot Polynomials](https://doi.org/10.5206/mt.v1i1.14037) but for now let's just use it.

```python
import math
rho = lambda k: -2 + 3*math.pi**2*4**(-k)/2
print( rho(5) - -1.9854242530543296 ) # Was -1.9854242530543296 above
print( rho(6) - -1.9963773629263977 ) # Was -1.9963773629263977+0j above
print( 4**6*Mandelbrot( n=6, c=rho(6) ) ) # Like 4 times 4 (pattern observed experimentally)
print( 4**7*Mandelbrot( n=7, c=rho(7) ) ) # Like 4 times 5
print( 4**8*Mandelbrot( 8, rho(8) )) # Like 4 times 6
print( 4**9*Mandelbrot( 9, rho(9) ))  # Like 4 times 7
print( 4**13*Mandelbrot( 13, rho(13) )) # Like 4 times 11
```

    -0.00011831862376210367
    -8.279993125359653e-06
    16.828183511695897
    20.712561469499633
    24.591600405416102
    28.468529149366077
    44.025735571980476

Now we see something interesting (to a numerical analyst).  The largest root (computed by Python) is actually better than its residual indicates!  We also see something interesting to an analyst: there is apparently a pattern in the residual: the experiments above predict that
\begin{equation*}
z( \rho_k ) = \frac{4(k-2)}{4^k} + \cdots
\end{equation*}
and if this were true in general then this would be enough to deduce that there really is a root nearby: one Newton correction from here will help as we will see below (but this turns out to be tricky because the _second_ derivatives are so _large_ in this area; this fact interferes with the theory of Newton iteration, as you may eventually see in your numerical analysis class).

Now let us see a nifty way to compute derivatives of the iteration: just differentiate the loop!  But we have to be careful to do it in the right order (this is quite weird).

```python
def MandelbrotWithDerivative(n: int, c):
    dz = 0*c
    z = 0*c # try to inherit the type of c
    for i in range(n):
        dz = 2*z*dz + 1 # A bit iffy to add constants to Polynomials in Python, but maybe it works. Yes! It does!
        z = z*z + c
    return(z,dz)
print( MandelbrotWithDerivative( 3, -1.2 ) )
print( MandelbrotWithDerivative( 3, Poly([0,1])))

```

    (-1.1423999999999999, 0.32800000000000007)
    (Polynomial([0., 1., 1., 2., 1.], domain=[-1.,  1.], window=[-1.,  1.]), Polynomial([1., 2., 6., 4.], domain=[-1.,  1.], window=[-1.,  1.]))

The output there is a bit hard to read, but it _looks_ like it worked; the derivative of $z_3 = c + c^2 + 2c^3 + c^4$ is indeed $1 + 2c + 6c^2 + 4c^3$.  We don't really know what all that "domain" and "window" stuff is, but let's just use the numerical part: if we plug in $c=-1.2$ into $1 + 2c + 6c^2 + 4c^3$ we should get $0.328$.  Let's try it and see.

```python
pd = Poly( [1,2,6,4])
print( pd(-1.2))
```

    0.3280000000000002

That looks good.  On this machine at the time of writing, the final digits are different (2 vs 7 from before) but it's convincing.  That code computes both $z(c)$ and $z'(c)$, with the same loop. So we can now take a Newton step. Or two.

```python
pdp = MandelbrotWithDerivative( 6, rho(6) )
improved = rho(6) - pdp[0]/pdp[1]
print( rho(6), pdp[0], pdp[1], improved, Mandelbrot(6,improved) )
pdp2 = MandelbrotWithDerivative( 6, improved )
better = improved - pdp2[0]/pdp2[1]
print( better, Mandelbrot(6,better ))
```

    -1.996385642919523 0.004108443240160131 -432.51765820326005 -1.9963761440166177 2.7235905697420293e-06
    -1.9963761377111966 1.2387868508767497e-12

Good!  Now we can (maybe) check Python's rootfinder for larger $N$.  Let's try $N=8$.

```python
N = 8
z = Mandelbrot( N, Poly([0,1]))
print( z.degree() )
rts = z.roots()
print( rts[0] )  # Hopelessly wrong (can't be smaller than -2)
rho8 = rho(8)
print( z( rho8 ) ) # Disaster has struck the polynomial evaluation
print( Mandelbrot(n=N, c=rho8)) # The iteration is better; *much* better
```

    128
    (-3.668564273119099+0j)
    -1.9744758160196454e+34
    0.0003752380432955338

We see that the built-in Python rootfinder has failed hopelessly, already at $N=8$ (when there are only $128$ roots to compute). There is a technical issue here: this polynomial is what is known as _ill-conditioned_. That means that it is _sensitive to changes in its input_.  We could pursue that, with the notion of _pseudozeros_ (and that's what leads to the fireworks GIF that we opened this section with), but for the moment we will try another tack: instead of working with Mandelbrot polynomials, we will work with Mandelbrot _matrices_, which turn out to be much better behaved.

```{prf:remark}
:label: eigenvalue_remark

It may seem silly, and wasteful*, to solve polynomials by finding eigenvalues of matrices.  Against that, we actually have software for eigenvalue problems that is _really good_ in the sense that it is robust, accurate, and efficient.  So we are "reducing the problem to a previously solved problem" which is like recycling, and so is not wasteful in that sense.  However, at some point the wastefulness (we store $d^2$ numbers, or sometimes even more, to attack a polynomial of degree $d$) will catch up to us.  But it turns out that this "wasteful" approach gets us _farther_ than trying to solve the polynomials directly does.
```

_Any_ guesses as to how NumPy's "roots" command works?  —Heh.

Let $\mathbf{M}_1 = [c]$ be the $1$-by-$1$ matrix whose determinant is $z_1(c)$. Now define
\begin{equation*}
    \mathbf{M}_{k+1} =
    \begin{bmatrix}
    \mathbf{M}_k & -c \mathbf{e}_1\mathbf{e}_{d_k}^T \\
    \mathbf{e}_1\mathbf{e}_{d_k}^T & \mathbf{M}_k
    \end{bmatrix} .
\end{equation*}
Here the $\mathbf{e}_j$'s are the unit elementary vectors of dimension $d_k=2^{k-1}$; all they do here is put a $1$ in the top right corner of the relevant blocks. The next two matrices are then
\begin{equation*}
    \left[\begin{array}{cc}
c  & -c
\\
 1 & c
\end{array}\right]
\end{equation*}
and
\begin{equation*}
    \left[\begin{array}{cccc}
c  & -c  & 0 & \mathbf{-{c}}
\\
 1 & c  & 0 & 0
\\
 0 & \mathbf{1} & c  & -c
\\
 0 & 0 & 1 & c
\end{array}\right]
\end{equation*}
where we have bold-faced the two new entries that connect the two copies of $\mathbf{M}_2$ that lie in the upper left corner and bottom right corner.

We can write this matrix as $\mathbf{A} + c\mathbf{B}$ where
\begin{equation*}
\mathbf{A} = \begin{bmatrix}
0 & 0 & 0 & 0 \\
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 1 & 0
\end{bmatrix}
\end{equation*}
and
\begin{equation*}
\mathbf{B} = \begin{bmatrix}
1 & -1 & \phantom{-}0 & -1 \\
0 & \phantom{-}1 & \phantom{-}0 & \phantom{-}0 \\
0 & \phantom{-}0 & \phantom{-}1 & -1\\
0 & \phantom{-}0 & \phantom{-}0 & \phantom{-}1
\end{bmatrix} .
\end{equation*}
Finding values of $c$ which make the determinant of $\mathbf{A} + c \mathbf{B}$ equal to zero is an example of what is called a _generalized_ eigenvalue problem, and [SciPy's linalg routine eigvals](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eigvals.html#scipy.linalg.eigvals) is set up do do this already, except that it solves problems of the form $\det( \mathbf{A} - \lambda \mathbf{B} ) = 0$ so we will have to change the sign of our matrix $\mathbf{B}$.

The matrix $\mathbf{A_k}$ is analogous at every dimension $k$: it is zero except for the subdiagonal which is all $1$s.  The matrix $\mathbf{B}_k$ is very simply constructed from two copies of the matrix $\mathbf{B}_{k-1}$ (if $k>1$), with an additional $-1$ tucked into the upper right corner.  We might as well build $-\mathbf{B}_k$ instead of building $\mathbf{B}_k$ while we are at it.  We will do this _recursively_.  See the code below.

```python
from scipy.linalg import eigvals, tri

def getMinusBmatrix(n: int):
    d = 2**(n-1)
    if n==1:
        minusB = -np.ones([1,1])
        return( minusB )
    else:
        zeroBlock = np.zeros( [d//2,d//2] )
        smallerB  = getMinusBmatrix( n-1 )  # This is a recursive program: it calls itself.
        minusB = np.bmat( 'smallerB, zeroBlock; zeroBlock, smallerB')
        minusB[0,-1] = 1 # Glue the two blocks together with a 1 in the upper right corner
        return( minusB )

def getAmatrix(n: int):
    d = 2**(n-1)
    A = np.zeros([d,d])
    for i in range(1,d):
        A[i,i-1] = 1
    return A
```

```python
N = 2
A = getAmatrix( N  )
B = getMinusBmatrix( N )
print( N, A , "\n", B )
N = 3
A = getAmatrix( N  )
B = getMinusBmatrix( N )
print( N, A, "\n", B )
N = 4
A = getAmatrix( N  )
B = getMinusBmatrix( N )
print( N, A, "\n", B )
```

    2 [[0. 0.]
     [1. 0.]]
     [[-1.  1.]
     [ 0. -1.]]
    3 [[0. 0. 0. 0.]
     [1. 0. 0. 0.]
     [0. 1. 0. 0.]
     [0. 0. 1. 0.]]
     [[-1.  1.  0.  1.]
     [ 0. -1.  0.  0.]
     [ 0.  0. -1.  1.]
     [ 0.  0.  0. -1.]]
    4 [[0. 0. 0. 0. 0. 0. 0. 0.]
     [1. 0. 0. 0. 0. 0. 0. 0.]
     [0. 1. 0. 0. 0. 0. 0. 0.]
     [0. 0. 1. 0. 0. 0. 0. 0.]
     [0. 0. 0. 1. 0. 0. 0. 0.]
     [0. 0. 0. 0. 1. 0. 0. 0.]
     [0. 0. 0. 0. 0. 1. 0. 0.]
     [0. 0. 0. 0. 0. 0. 1. 0.]]
     [[-1.  1.  0.  1.  0.  0.  0.  1.]
     [ 0. -1.  0.  0.  0.  0.  0.  0.]
     [ 0.  0. -1.  1.  0.  0.  0.  0.]
     [ 0.  0.  0. -1.  0.  0.  0.  0.]
     [ 0.  0.  0.  0. -1.  1.  0.  1.]
     [ 0.  0.  0.  0.  0. -1.  0.  0.]
     [ 0.  0.  0.  0.  0.  0. -1.  1.]
     [ 0.  0.  0.  0.  0.  0.  0. -1.]]

```python
N = 8
A = getAmatrix( N  )
B = getMinusBmatrix( N )
# Now compute the generalized eigenvalues
E = eigvals( A, B )
x = [e.real for e in E]
y = [e.imag for e in E ]
residuals = [ Mandelbrot(N,e) for e in E]
absresiduals = [abs(r) for r in residuals ]
print( max(absresiduals) )
rootplot = plt.figure( figsize=(8,8) )
plt.scatter( x, y, s=15, marker=".")
plt.xlim(-2.5,0.5)
plt.ylim(-1.5,1.5)
plt.show()
```

This shows that (at least for $N=8$) solving the (generalized) eigenvalue problem is _much_ more accurate than trying to solve the explicit Mandelbrot polynomial at the same dimension.

Now it turns out that the matrix method will also (eventually) run into trouble, at about $N=17$ (when RMC's computer tries to store all the zeros in the $d$ by $d$ matrix, and runs out of memory); the dimension of the matrix for $N=14$ is already $16384$ by $16384$, using about sixteen thousand times too much memory, and so this isn't that surprising.

Here are some results of running this code on a 2019 Microsoft Surface Pro (1.3GHz, 4 cores; not that the 4 cores are specially used by Python).  We report the Mandelbrot index, the dimension, the time taken in seconds, and the maximum absolute value of the residual.  We see that in all the cases we get accurate roots, but we also see that as the dimension doubles the cost increases by _more_ than a factor of $2^3=8$, consistently; we do not know why.  We estimated that running this for $N=14$ would take about $3$ hours (it took $2$ hours and $22$ minutes), and would return all roots to about single precision accuracy, which it did (we are working in double precision, of course).  We could use a supposedly faster language (Julia, for instance: except that, ironically, Python seems to be faster for generalized eigenvalues) to speed the computation up, but the _condition number_ of these eigenvalues also seems to be growing; this means the accuracy will diminish.  Another approach seems needed: see MPSolve, and see [Eunice Chan's Master's thesis](https://ir.lib.uwo.ca/etd/4028/).  Also, _generalized_ eigenvalue code is (quite a bit!) slower than code for the simple eigenvalue problem $\mathbf{A}\mathbf{x} = \lambda \mathbf{x}$ and we could convert our generalized eigenvalue problem to a simple one by postmultiplying by $\mathbf{B}^{-1}$ (we can even do that analytically, and this works _very_ well: see the exercises) but for now we will step away from the keyboard, happy with accurately computing the roots of the degree $8192$ Mandelbrot polynomial by means of a generalized eigenvalue problem.

| n  | d    | time(s) | slowdown | max residual       |
|----|------|---------|----------|--------------------|
| 9 | 256  | 0.06     |          | $2.8\cdot 10^{-11}$                   |
| 10 | 512  | 0.6     | 10.       | $1.8\cdot 10^{-10}$                   |
| 11 | 1024 | 6       | 10.       | $1.6\cdot 10^{-9}$                   |
| 12 | 2048 | 78      | 13.       | $3.5\cdot 10^{-9}$ |
| 13 | 4096 | 850     | 11.     | $2.0\cdot 10^{-8}$ |
| 14 | 8192 | 8512    | 10.       | $4.0\cdot 10^{-7}$ |

```{prf:remark}
:label: memory_remark

If we are more careful with memory and use what are known as _sparse_ matrices, then we can go a lot farther.  Using sparse matrices, and special algorithms for finding eigenvalues of sparse matrices, Piers Lawrence was able to take this approach up to about a half a million eigenvalues ($n=19$); it was a bit awkward to make sure that we got all the eigenvalues once and only once, but it worked.  Eunice Chan got up over two million eigenvalues with a different method (homotopy continuation), but the champion is still MPSolve; Dario Bini and Leonard Robol used their software to compute over four million roots of Mandelbrot polynomials.
```

```python
import time
start = time.time()
N=9
A = getAmatrix( N  )
B = getMinusBmatrix( N )
# print( A, "\n", B )
built = time.time()
E = eigvals( A, B )
end = time.time()
x = [e.real for e in E]
y = [e.imag for e in E ]
residuals = [ Mandelbrot(N,e) for e in E]
absresiduals = [abs(r) for r in residuals ]
print( max(absresiduals) )
print( "Time taken was ", end-built, "seconds to find d=", 2**(N-1),
      "\ngeneralized eigenvalues of matrices that took ", built-start, "seconds to build" )
```

    1.4007750515077078e-10
    Time taken was  0.06600117683410645 seconds to find d= 256
    generalized eigenvalues of matrices that took  0.0019373893737792969 seconds to build

```python
rootplot = plt.figure( figsize=(8,8) )
plt.scatter( x, y, s=0.5, marker=".")
plt.xlim(-2.5,0.5)
plt.ylim(-1.5,1.5)
plt.show()
```

The following is the resulting image from the case $N=14$:

```{image} ../Figures/Mandelbrot/MandelbrotZ14.png
:height: 400px
:alt: Our best eigenvalue image of the roots of a Mandelbrot polynomial of degree 8192
:align: center
```

## Exercises
1. Write down as many questions as you can, about this section.
2. Read the Wikipedia entry on the Mandelbrot set, and the entry on Mandelbrot.  Mandelbrot's _educational_ books and papers are amazing, and amazingly clearly written.  If you don't come back from this exercise, well, at least you left going in a good direction.
3. Prove by induction that the degree of the $n$th Mandelbrot polynomial is $2^{n-1}$, and that the leading coefficient is $1$.
4. Prove that $c=-2$ and $c=0$ are both in the Mandelbrot set. Show that all zeros of Mandelbrot polynomials give _periodic orbits_ under the Mandelbrot iteration and are thus in the Mandelbrot set.
5. Is $i = \sqrt{-1}$ in the Mandelbrot set?
6. Prove that the Mandelbrot polynomials are _unimodal_.  At the time we write this, this question is open: solve it, and you could publish a paper with your proof ([Maple Transactions](https://www.mapletransactions.org) would be a good place).  The word "unimodal" just means that the size of the coefficients increases to a peak, and then decays again.  As in $z_4$, the peak might be attained by two coefficients, not just one.  Seriously, we don't know how to prove this.  We think it's true, though.
7. Solve $z_3 = 0$ by hand, as follows.  First, divide out the visible root, $c=0$, to get the cubic equation $0 = 1 + c + 2c^2 + c^3$.  Then put $c = \xi - 2/3$ so $c^3 = (\xi -2/3)^3 = \xi^3 - 2\xi^2 + 4\xi/9 - 8/27$ which will transform the equation into a cubic in $\xi$ of the form $0 = q + p\xi + \xi^3$.  Then put $\xi = u + v$ (introducing two new variables seems like the opposite of progress, but trust us, it will help).  This transforms the equation to $(u+v)^3 + p(u+v) + q = 0$, or $u^3 + 3uv(u+v) + p(u+v) + q = 0$.  Show that if you can find $u$ and $v$ that simultaneously solve $3uv + p = 0$ and $u^3 + v^3 + q = 0$ then you can solve the original equation.  Do so.  Your formula at the end should give you three, and only three, roots. Write out your answer explicitly, and compare it with (say) the answer from Wolfram Alpha. There is an analogous formula for quartics, but even teaching this cubic formula is considered cruel and inhumane; well, it would be if you were to be tested on it.  But maybe a [public competition for solving cubics](https://en.wikipedia.org/wiki/Cubic_equation) might be entertaining for the spectators, as it seems to have been in 1530.  Quite amazingly, at least to him, RMC had recent recourse to this method in order to write a very fast program to evaluate the cube roots of an equation that depended on a parameter, except that the equation was of the form $x^3 + mx^2 + n=0$ (which was apparently the type that defeated Fior in the contest—but just solve for $1/x$ instead, and that puts it into the previous form).  All modern computer algebra systems know how to solve cubics and quartics in terms of radicals; quintics cannot, in general, be solved in terms of radicals, but they can be solved in terms of _elliptic functions_.  Now there's a rabbit hole for you.
8. Prove by induction that the code MandelbrotWithDerivative computes the correct derivative of every Mandelbrot polynomial. Rather, that it _would_ compute the correct derivative if the underlying arithmetic were exact. You might need what is known as a _loop invariant_: something that is true before the loop starts, and is true at the start of every iteration, and at the end of every iteration (though not necessarily true after each intermediate step of the iteration). But this loop is simple enough that a direct induction also works.
9. Are the zeros of the derivatives of the Mandelbrot polynomials in the Mandelbrot set? We think so, but we don't know.
10. "A good numerical method gives you the exact solution to a nearby problem".  This is an informal definition of _numerical stability_.  An _algorithm_ is "numerically stable" if it gives you the exact solution, not to the problem you were trying to solve (say the polynomial $z(c) = 0$) but instead to a _nearby_ problem, say $z(c) - 10^{-16}i = 0$.  Some problems, however, are _sensitive to changes_ (in the vernacular of numerical analysis, _ill-conditioned_).  Consider the polynomial
$z(c) = \sum_{k=0}^d a_k c^k$
and a _perturbed_ (changed) polynomial
$z(c) + \Delta(c) = \sum_{k=0}^d a_k(1+s_k) c^k$
where the coefficients $a_k$ have all been changed by small relative amounts $s_k$.  Suppose further that all $|s_k| \le t$, some tiny number. By using the triangle inequality, show that
$
| \Delta(c) | \le K(c) t
$
where
$
K(c) = \sum_{k=0}^d |a_k| |c|^k
$
This number $K(c)$ serves as a kind of _condition number_ for the polynomial: the larger it is, the more sensitive the polynomial is.  Plot the condition numbers (on a log scale!) of the Mandelbrot polynomials $z_6(c)$ and $z_7(c)$ on $-2 \le c \le 0$.
11. __Pseudozeros__.  The zeros of $z(c) - 10^{-k}\exp(i\theta)$ for $k=6$, say, represent a _curve_ in $c$ space (let $\theta$ vary from $-\pi$ to $\pi$ so the tiny perturbation $10^{-6}\exp(i\theta)$ turns a full circle in the complex plane). As $k$ is taken larger, one expects these curves to surround the zeros of $z(c)$ itself.  Graph some pseudozeros of a Mandelbrot polynomial.  Animate them.
12. Write $\mathbf{C} = \mathbf{A}\mathbf{B}^{-1}$ and show that the generalized eigenvalues of $\mathbf{A}-\lambda\mathbf{B}$ are the simple eigenvalues of $\mathbf{C} - \lambda\mathbf{I}$.  Find $\mathbf{C}$ for the Mandelbrot matrices above, and code it up and compare the times to compute simple eigenvalues versus generalized eigenvalues. How high a degree can you get to?  We stopped at $N=16$, where the computation took about an hour and a half, and the residual was about $4.0e-5$.  The current record is held by [MPSolve](https://numpi.dm.unipi.it/software/mpsolve) with $k=23$, so over $4$ million roots (using multiple precision); we don't think eigenvalue methods can get that high, but we think [homotopy methods](https://ir.lib.uwo.ca/etd/4028/) can, although they haven't yet.
13. Other families of polynomials like the Mandelbrot polynomials can be defined: Fibonacci--Mandelbrot polynomials defined by $q_{n+1}(z) = z q_n(z) q_{n-1}(z) + 1$ and $q_0(z)=0$ and $q_1(z) = 1$, for instance; or Fibonacci--Narayana polynomials.  Define your own polynomials by recurrence relation, and find your own matrices whose eigenvalues are their roots, and draw their eigenvalues.  We have only done a few of these, and there are infinitely many to choose from.  It's possible your results will be completely new (and therefore publishable).
14. The paper [Some Facts and Conjectures on Mandelbrot Polynomials](https://doi.org/10.5206/mt.v1i1.14037) contains a formula for the _Mandelbrot Generating Function_ MGF(k,c), which _analytically solves_ the Mandelbrot iteration $z_{n+1} = z_n^2 + c$, at least for $c$ _outside_ the Mandelbrot set.  This means that we could (say) take half an iteration—and find $z_{1/2}(c)$ or $z_{3/2}(c)$.  As with [Stand-Up Math's Video on Complex Fibonacci Numbers](https://youtu.be/ghxQA3vvhsk) these are likely to be _complex numbers_.  So far as we know, no-one (not even us) has computed these.  Do so, for (say) $c=1$ or $c=2$.  Plot the results.  This, also, might be publishable (again, [Maple Transactions](https://www.mapletransactions.org) might be a good place).
15. A [companion matrix](https://en.wikipedia.org/wiki/Companion_matrix) for a polynomial (say $P(\lambda)$) is simply a matrix $C$ whose eigenvalues are the roots of $P(\lambda)$.  Every matrix has a characteristic polynomial whose roots are its eigenvalues; what's interesting is that this can be run the other way (it's a bit more complicated because "Frobenius companion matrices" as in the link above have special properties).  But companion matrices are not unique: any matrix similar to a companion matrix $\mathbf{S}\mathbf{C}\mathbf{S}^{-1}$ is also a companion matrix for that same $P(\lambda)$.  Suppose that you have a polynomial with _integer_ coefficients.  Then the Frobenius companion matrix has (those same) integers as entries, together with ones and zeros.  Therefore, out of all integer companion matrices for $P(\lambda)$, there must be one with _minimal height_: the height of the matrix is the size of the largest absolute value of any entry in the matrix.  Mandelbrot matrices as defined above have height $1$, which is the smallest possible for integer companions, and are therefore "minimal height".  Note that the size of the coefficients of the polynomial are exponential in the degree! So the height of the Frobenius companion matrix of Mandelbrot polynomials is likewise exponential.  Not much is known about [minimal height companion matrices](https://link.springer.com/article/10.1007/s11786-018-0364-2) (see also [our first paper](https://journals.uwyo.edu/index.php/ela/article/view/1781)); we do not have an algorithm for finding one for a given integer polynomial, for instance. We _believe_ that minimal height companion matrices are generally better-conditioned as far as their eigenvalues, but we have no proof.  Find a minimal height companion matrix for (say) $\lambda^3 + 5\lambda^2 + 3\lambda + 1$.  Find a general algorithm for computing minimal height companions.  Prove that they're useful.  If you succeed at this problem, you should certainly publish your results; a good venue might be the [Electronic Journal of Linear Algebra](https://journals.uwyo.edu/index.php/ela).


# Computational Models for Complex Systems Project

## Implementation of a SIRVSD model with different age goups

### The model
$$
\begin{equation}
    \begin{cases}
    \frac{\textit{dS}_j}{\textit{dt}} = \phi \textit{R}_j - \eta_j (t) \textit{S}_j + \rho \textit{V}_j - \textit{S}_j \sum_{k=1}^M \beta_{j,k} \textit{I}_k \\[4pt]
    \frac{\textit{dI}_j}{\textit{dt}} = - \gamma \textit{I}_j - \mu_j \textit{I}_j + \textit{S}_j \sum_{k=1}^M \beta_{j,k} \textit{I}_k \\[4pt]
    \frac{\textit{dR}_j}{\textit{dt}} = \gamma \textit{I}_j - \phi \textit{R}_j & \hspace{10pt} \forall j \in \left(0, M\right] \\[4pt]
    \frac{\textit{dV}_j}{\textit{dt}} = \eta_j (t) \textit{S}_j - \rho \textit{V}_j \\[4pt]
    \frac{\textit{dD}_j}{\textit{dt}} = \mu_j \textit{I}_j \\[4pt]
    \end{cases}
\end{equation}
$$

with $M$ = 4 age groups:
- Children (0-9 years)
- Teenagers (10-19 years)
- Adults (20-69 years)
- Senior Citizens (70+ years)

### Transition Schema

### Model assumptions
- Constant population and each compartment is normalized (S+I+R+V+D = 1).
- We don't consider vital dynamics (reproduction, "natural" death, migration), only the deaths caused by disease.
- Some coefficients are the same for all age group (only transmission, vaccination and mortality coefficients are different).
- We consider symmetric matrix for the transmission coefficients (heterogeneous contacts).
- Both recovery and vaccination immunity are not forever, but it ensures 100\% protection from infection.
- Just one injection of the vaccine is considered.
- Disease is transmitted by personal contacts between individuals of I and S classes (horizontal transmission).
- Contacts between individuals are random, the number of infections is proportional to both I and S.

### Definition of parameters
In our system, we have a lot of different coefficients:
- $\phi$ is the $\textit{transfer coefficient}$ for loss of immunity from Recovered.
- $\rho$ is the $\textit{transfer coefficient}$ for loss of immunity from Vaccinated.
- $\beta_{j, k}$ is the $\textit{infection coefficient}$, computed as the average number of contacts per person per time, multiplied by the probability of disease transmission in a contact between a susceptible subject of group $j$ and an infected subject of group $k$. <br>
We define also the entire $\textit{infection coefficient matrix}$ $\beta$ (or $\textit{contact matrix}$):
$$\beta = \begin{bmatrix}
        \beta_{1, 1} & \cdots & \beta_{1, M}\\
        \vdots & \ddots & \vdots \\
        \beta_{M, 1} & \cdots & \beta_{M, M}
\end{bmatrix}
$$
- $\gamma$ is the $\textit{recovery coefficient}$ of each infected subject.
- $\mu_j$ is the $\textit{mortality coefficient}$, different for each age group.

The last coefficient is $\eta_j(t)$, a time-dependent $\textit{vaccination coefficient}$, defines as follows:
$$
    \begin{equation}
        \eta_j (t) =
        \begin{cases}
            0 &\quad\text{if } t < t_{\textit{vacc}_j}\\
            \eta_j &\quad\text{otherwise}
        \end{cases}
    \end{equation}
$$

where $t_{\textit{vacc}_j}$ defines the starting day of the vaccination period.

```python
import numpy as np
from scipy.integrate import odeint, solve_ivp
import matplotlib.pyplot as plt
```

```python
def sirvd_solver(t,beta_matrix,gamma,mu_group,phi, rho,eta_group,x0,start_vaccination):
    """Wrapper function to compute ODEs using different APIs (currently only with methods of SciPy)

    Args:
        t (np.ndarray): simulation time
        beta_matrix (np.ndarray): infection coefficient for each group
        gamma (float): recovery coefficient (same for all group)
        mu_group (list): mortality coefficient for each group (case fatality rate ISS report January 2021)
        phi (float): transfer coefficient for loss of immunity from recovered (six months of immunity and same for all group)
        rho (float): transfer coefficient for loss of immunity from vaccinated (nine months of immunity and same for all group)
        eta_group (list): vaccination coefficient for each group
        x0 (list): initial conditions
        start_vaccination (list): day of start of the vaccination period for each group
    """
    def assign_vaccination_coefficient(t, eta, start_vaccination):
        """Auxiliary function to assign time-dependent vaccination coefficient eta

        Args:
            t (float): scalar representing the current timestamp
            eta (float): vaccination coefficient
            start_vaccination (int): starting day of vaccination

        Returns:
            float: eta for a specific timestamp (0 or eta)
        """
        if t < start_vaccination or start_vaccination == -1: # -1 means no vaccination for a specific age group
            return 0
        else:
            return eta

    def sirvd(t,x,beta_matrix,gamma,mu_group,phi,rho,eta_group,start_vaccination,assign_vaccination_coefficient):
        """
        Function called by solve_ivp (or odeint) to compute the derivative of x at t.
        """
        n_groups = len(start_vaccination) # or any other "_group" parameter
        derivatives_matrix = np.zeros((5,4)) # save all derivatives in a 2-D array
        n_infectious = [x[j+n_groups] for j in range(0,n_groups)] # list of the number of infectious measured for each group
        for j in range(0,n_groups):
            s = x[j] # Susceptible
            # i = n_infectious[j]
            i = x[j+n_groups] # Infectious
            r = x[j+n_groups*2] # Recovered
            v = x[j+n_groups*3] # Vaccinated
            d = x[j+n_groups*4] # Deceased
            eta = assign_vaccination_coefficient(t,eta_group[j],start_vaccination[j]) # time-dependent parameter
            derivatives_matrix[0][j] = phi*r - eta*s + rho*v - s*np.dot(beta_matrix[j],n_infectious) # dsdt
            derivatives_matrix[1][j] = s*np.dot(beta_matrix[j],n_infectious) - gamma*i - mu_group[j]*i # didt
            derivatives_matrix[2][j] = gamma*i - phi*r # drdt
            derivatives_matrix[3][j] = eta*s - rho*v # dvdt
            derivatives_matrix[4][j] = mu_group[j]*i # dddt
        return derivatives_matrix.reshape(-1) # return all measurements with a 1-D array

    # odeint solve a system of ordinary differential equations using lsoda from the FORTRAN library odepack.
    # y = odeint(sirvd,x0,t,tfirst=True,args=(beta_matrix,gamma,mu_group,phi,rho,eta_group,start_vaccination,assign_vaccination_coefficient,))
    # return y
    # for new code, use scipy.integrate.solve_ivp to solve a differential equation (SciPy documentation).
    sol = solve_ivp(sirvd,[t[0],t[-1]],x0,t_eval=t,args=(beta_matrix,gamma,mu_group,phi,rho,eta_group,start_vaccination,assign_vaccination_coefficient,))
    return sol.y.T
```

### Case Study: COVID-19 pandemic
### Definition of constants
We considered these groups:
- Children (0-9 years)
- Teenagers (10-19 years)
- Adults (20-69 years)
- Senior Citizens (70+ years)

In the following initial conditions and parameters, we define an indexing like this:

    S_0_GROUP = [x, y, w, y]
        - x is the intitial number of susceptible in "Children" group
        - y is the intitial number of susceptible in "Teenagers" group
        - w is the intitial number of susceptible in "Adults" group
        - z is the intitial number of susceptible in "Senior Citizens" group

We can change the value of the costants, according to the experiments we want to run and the plots we want to analyze.

```python
group_dict = {
    "children": 0,
    "teenagers": 1,
    "adults": 2,
    "senior": 3
}
START = 0 # observation starting day
END = 365 # observation ending day
START_VACCINATION_GROUP = [-1, -1, -1, -1] # day of start of the vaccination period (-1 means no vaccination)

# initial conditions
S_0_GROUP = [0.99, 0.99, 0.99, 0.99] # Susceptible
I_0_GROUP = [0.01, 0.01, 0.01, 0.01] # Infectious
R_0_GROUP = [0, 0, 0, 0] # Recovered
V_0_GROUP = [0, 0, 0, 0] # Vaccinated
D_0_GROUP = [0, 0, 0, 0] # Deceased

# model parameters
beta_matrix = np.array([[0.05,0.003,0.04,0.005],[0.003,0.09,0.07,0.007],[0.04,0.07,0.09,0.02],[0.005,0.007,0.02,0.03]]) # infection coefficient for each group
gamma = 1/15 # recovery coefficient (same for all group)
mu_group = [0.00009, 0.00005, 0.00688, 0.15987] # mortality coefficient for each group (case fatality rate ISS report January 2021)
phi = 1/180 # transfer coefficient for loss of immunity from recovered (six months of immunity and same for all group)
rho = 1/270 # transfer coefficient for loss of immunity from vaccinated (nine months of immunity and same for all group)
eta_group = [0.003, 0.003, 0.003, 0.01] # vaccination coefficient for each group

t = np.linspace(START,END,END-START+1) # setting the simulation time and the number of points
```

### Function call with same vaccination strategy

```python
results_dict = {}
x_0 = [*S_0_GROUP, *I_0_GROUP, *R_0_GROUP, *V_0_GROUP, *D_0_GROUP] # unpacking list operator
y = sirvd_solver(t, beta_matrix, gamma, mu_group, phi, rho, eta_group, x_0, START_VACCINATION_GROUP)
_, n_total_column = y.shape
n_groups = len(group_dict) # number of age groups
n_compartments = int(n_total_column/n_groups) # number of compartments of the model
for group_name, group_id in group_dict.items():
    # select the right columns (the compartments) for each age group
    results_dict[group_name] = y[:,[group_id+n_groups*j for j in range(0,n_compartments)]]
```

### Function call with combination of different vaccination strategy

```python
"""vaccination_dict = {
        "no_vaccination": 0,
        "vaccination_strategy_ascending_order": 1,
        "vaccination_strategy_descending_order": 2,
        "vaccination_strategy_same_time": 3
    }
    results_dict = {}
    x_0 = [*S_0_GROUP, *I_0_GROUP, *R_0_GROUP, *V_0_GROUP, *D_0_GROUP] # unpacking list operator
    for vacc_name, vacc_id in vaccination_dict.items():
        results_dict[vacc_name] = {}
        if(vacc_id == 1): # ascending order
            START_VACCINATION_GROUP = [0, 30, 60, 90]
        elif(vacc_id == 2): # descending order
            START_VACCINATION_GROUP = [90, 60, 30, 0]
        elif(vacc_id == 3): # same time
            START_VACCINATION_GROUP = [0, 0, 0, 0]
            eta_group = [0.0025, 0.0025, 0.0025, 0.0025]
        y = sirvd_solver(t, beta_matrix, gamma, mu_group, phi, rho, eta_group, x_0, START_VACCINATION_GROUP)
        _, n_total_column = y.shape
        n_groups = len(group_dict) # number of age groups
        n_compartments = int(n_total_column/n_groups) # number of compartments of the model
        for group_name, group_id in group_dict.items():
            # select the right columns (the compartments) for each age group
            results_dict[vacc_name][group_name] = y[:,[group_id+n_groups*j for j in range(0,n_compartments)]]"""
```

### References
- [Compartmental models in epidemiology — Wikipedia, the free encyclopedia](https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology)
- [Modeling Infectious Diseases in Humans and Animals](https://homepages.warwick.ac.uk/~masfz/ModelingInfectiousDiseases/Chapter3/Program_3.3/index.html)
- [Influence of nonlinear incidence rates upon the behavior of SIRS epidemiological models](https://link.springer.com/content/pdf/10.1007/BF00276956.pdf)
- [Analysis of COVID-19 Data with PRISM: Parameter Estimation and SIR Modelling](https://www.springerprofessional.de/en/analysis-of-covid-19-data-with-prism-parameter-estimation-and-si/18929902)
- [Use of a Modified SIRD Model to Analyze COVID-19 Data](https://pubs.acs.org/doi/pdf/10.1021/acs.iecr.0c04754)
- [Global  results  for  an  SIRS  model  with  vaccination and isolation](https://www.sciencedirect.com/science/article/abs/pii/S1468121810000763)
- [Mathematical  models  of  contact patterns between age groups for predicting the spread of infectious diseases](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4002176/pdf/nihms-570528.pdf)
- [A statistical methodology for data-driven partitioning of infectious disease incidence into age-groups](https://arxiv.org/pdf/1907.03441.pdf)
- [Lab24 - Coronavirus in Italia, i dati e la mappa](https://lab24.ilsole24ore.com/coronavirus/#)
- [SIR Modelling with data fitting in Python](https://github.com/Unipisa/SIR-covid)
- [Matplotlib Documentation](https://matplotlib.org/stable/contents.html)

notebooks/Multi-age structured SIRVSD model.ipynb


Before you turn this problem in, make sure everything runs as expected. First, restart the kernel (in the menubar, select Kernel → Restart) and then run all cells (in the menubar, select Cell → Run All).

Make sure you fill in any place that says YOUR CODE HERE or "YOUR ANSWER HERE", as well as your name and collaborators below:

```python
NAME = ""
COLLABORATORS = ""
```

# **Floating Point Representation**

### **8 bit Floating Point Representation**

We will implement an 8 bit Floating point number system in this part, where First bit is a sign bit, next three bits are exponents, and the rest 4 bits are mantissas (significands).

Numbers are divided into two categories: normals and subnormals.

For normals the encoding is:
\begin{equation}
(-1)^{sign} \times 2^{(e-3)} \times 1.fraction
\end{equation}

For subnormals the encoding is:
\begin{equation}
(-1)^{sign} \times 2^{(1-3)} \times 0.fraction
\end{equation}

Example:
\begin{aligned}
    0 001 0000 &= 2^{(1-3)} \times 1.0 = 0.25\\
    0 010 0000 &= 2^{(2-3)} \times 1.0 = 0.5\\
    0 011 0000 &= 2^{(3-3)} \times 1.0 = 1.0\\
    0 011 0100 &= 2^{(3-3)} \times 1.25 = 1.25
\end{aligned}

If the exponent is 0 then the number is considered subnormal.

Example:
\begin{aligned}
    0 000 0000 &= 2^{-2} \times 0   = 0.0\\
    0 000 1000 &= 2^{-2} \times 0.5 = 0.125\\
    0 000 1100 &= 2^{-2} \times 0.75 = 0.1875
\end{aligned}

If the exponent is 7 then and mantissa is zero then the number is considered infinity (inf), otherwise it is not-a-number (nan).

Example:
\begin{aligned}
    0 111 0000 &= +\infty\\
    1 111 0000 &= -\infty\\
    X 111 XXXX &= NaN
\end{aligned}

```python
import math
import itertools
import matplotlib.pyplot as plt
import numpy as np
```

```python
class Float8():

    def __init__(self, bitstring):
        '''Constructor
        takes a 8-bit string of 0's and 1's as input and stores the sub-strings
        accordingly.
        Usage: Float8('00011110')
        '''

        # Make sure the input consists of exactly 8-bits.
        assert(len(bitstring)==8)

        # Assign the sign bit
        self.sign = bitstring[0:1]

        # Assign the exponent part
        self.exponent = bitstring[1:4]

        # Assign the mantissa
        self.mantissa = bitstring[4:]

        # YOUR CODE HERE

        self.val = self.calculate_value()

    def __str__(self):
        return f'Sign bit value: {self.sign}\n' + \
            f'Exponent value: {self.exponent}\n' + \
            f'Mantissa value: {self.mantissa}\n' + \
            f'Floating value: {self.val}\n'

    def tobitstring(self):
        return self.sign + self.exponent + self.mantissa

    def toformattedstring(self):
        return ' '.join([self.sign, self.exponent, self.mantissa])

    def calculate_value(self):
        '''Calculate the value of the number from bits'''

        # Initialize with zero
        val = 0.0

        # YOUR CODE HERE
        if self.exponent == '111':
            if self.mantissa == '0000':
                if self.sign == '0':
                    val = math.inf
                else:
                    val = -math.inf
            else:
                val = math.nan

        elif self.exponent == '000':
            for index, i in enumerate(self.mantissa):
                val = val + ((float(i)) * (2 ** (-index - 1)))
            val = val * (2 ** (-2))
            j = int(self.sign, 2)
            val = ((-1) ** j) * val

        else:
            k = int(self.exponent, 2)
            val = 1.0
            for index, i in enumerate(self.mantissa):
                val = val + ((float(i)) * (2 ** (-index - 1)))
            val = val * (2 ** (k - 3))
            j = int(self.sign, 2)
            val = ((-1) ** j) * val

        return val
```

```python
data = [ '00000000', '00000001', '00001001', '00010000',
         '00010001', '00011000', '00011011', '00100000',
         '00101101', '00110000', '00110101', '01000011',
         '01000000', '01010000', '01011100', '01100000',
         '01110111', '01110000', '10000000', '10000001',
         '11110001', '11110000', '10110001', '10111101',
         '11100000', '11101011', '11010000', '11000000']
result = ['(0, 1)', '(1, 64)', '(9, 64)', '(1, 4)', '(17, 64)', '(3, 8)', '(27, 64)',
          '(1, 2)', '(29, 32)', '(1, 1)', '(21, 16)', '(19, 8)', '(2, 1)', '(4, 1)',
          '(7, 1)', '(8, 1)', 'nan', 'inf', '(0, 1)', '(-1, 64)', 'nan', '-inf',
          '(-17, 16)', '(-29, 16)', '(-8, 1)', '(-27, 2)', '(-4, 1)', '(-2, 1)']

test = [Float8(x).val for x in data]

for index in range(len(test)):
    d = test[index]
    try:
        test[index] = str(d.as_integer_ratio())
    except Exception:
        test[index] = str(d)

np.testing.assert_array_equal(test, result)
```

## **16 Bit Floating Point Representation**

Over here, we will implement a 16 bit Floating point number system, where the first bit is a sign bit, next four bits are exponents, and the rest 11 bits are mantissas (significands).

Numbers are divided into two categories: normals and subnormals.

For normals the encoding is:
\begin{equation}
(-1)^{sign} \times 2^{(e-7)} \times 1.fraction
\end{equation}

For subnormals the encoding is:
\begin{equation}
(-1)^{sign} \times 2^{(1-7)} \times 0.fraction
\end{equation}

Example:
\begin{aligned}
    0 0101 00000000000 &= 2^{(5-7)} \times 1.0  = 0.25\\
    0 0110 00000000000 &= 2^{(6-7)} \times 1.0  = 0.5\\
    0 0111 00000000000 &= 2^{(3-7)} \times 1.0  = 1.0\\
    0 0111 01000000000 &= 2^{(3-7)} \times 1.25 = 1.25
\end{aligned}

If the exponent is 0 then the number is considered subnormal.

Example:
\begin{aligned}
    0 0000 00000000000 &= 2^{-6} \times 0    = +0.0\\
    0 0000 10000000000 &= 2^{-6} \times 0.5  = +0.0078125\\
    0 0000 11000000000 &= 2^{-6} \times 0.75 = +0.01171875
\end{aligned}

If the exponent is 15 (all 1's) then and mantissa is zero then the number is considered infinity (inf), otherwise it is not-a-number (nan).

Example:
\begin{aligned}
    0 1111 00000000000 &= +\infty\\
    1 1111 00000000000 &= -\infty\\
    X 1111 XXXXXXXXXXX &= NaN
\end{aligned}
## **Instructions**

In this task, you need to implement the Float16() class. Some parts of the class are already written for you. You only need to modify __init__(self, bitstring) and calculate_value(self) fuctions, where it says # YOUR CODE HERE. You may ignore raise NotImplementedError().

```python
class Float16():

    def __init__(self, bitstring):
        '''Constructor
        takes a 16-bit string of 0's and 1's as input and stores the sub-strings
        accordingly.
        Usage: Float16('0001111000011110')
        '''

        # Make sure the input consists of exactly 16-bits.
        assert(len(bitstring)==16)

        # Assign the sign bit
        self.sign = bitstring[0:1]

        # Assign the exponent part
        self.exponent = bitstring[1:5]

        # Assign the mantissa
        self.mantissa = bitstring[5:]

        # YOUR CODE HERE

        self.val = self.calculate_value()

    def __str__(self):
        return f'Sign bit value: {self.sign}\n' + \
            f'Exponent value: {self.exponent}\n' + \
            f'Mantissa value: {self.mantissa}\n' + \
            f'Floating value: {self.val}\n'

    def tobitstring(self):
        return self.sign + self.exponent + self.mantissa

    def toformattedstring(self):
        return ' '.join([self.sign, self.exponent, self.mantissa])

    def calculate_value(self):
        '''Calculate the value of the number from bits'''

        # Initialize with zero
        val = 0.0

        # YOUR CODE HERE
        if self.exponent == '1111':
            if self.mantissa == '00000000000':
                if self.sign == '0':
                    val = math.inf
                else:
                    val = -math.inf
            else:
                val = math.nan

        elif self.exponent == '0000':
            for index, i in enumerate(self.mantissa):
                val = val + ((float(i)) * (2 ** (-index - 1)))
            val = val * (2 ** (-6))
            j = int(self.sign, 2)
            val = ((-1) ** j) * val

        else:
            k = int(self.exponent, 2)
            val = 1.0
            for index, i in enumerate(self.mantissa):
                val = val + ((float(i)) * (2 ** (-index - 1)))
            val = val * (2 ** (k - 7))
            j = int(self.sign, 2)
            val = ((-1) ** j) * val

        return val
```

```python
# 28 test cases for 16-bit floating points

data = [ '0011100000000010', '0100000000000000', '1100000000000000', '0100010000000000',
         '1100010000000000', '0100100000000000', '1100100000000000', '0100101000000000',
         '1100101000000000', '0100110000000000', '1100110000000000', '0101101110000000',
         '0010010000000000', '0000000000000001', '0000011111111111', '0000100000000000',
         '0111011111111111', '0000000000000000', '1000000000000000', '0111100000000000',
         '1111100000000000', '0111100000000001', '0111110000000001', '0111111111111111',
         '0010101010101011', '0100010010010001', '0011100000000000', '0011100000000001']
result = ['(1025, 1024)', '(2, 1)', '(-2, 1)', '(3, 1)', '(-3, 1)', '(4, 1)', '(-4, 1)',
           '(5, 1)', '(-5, 1)', '(6, 1)', '(-6, 1)', '(23, 1)', '(3, 16)', '(1, 131072)',
           '(2047, 131072)', '(1, 64)', '(4095, 16)', '(0, 1)', '(0, 1)', 'inf', '-inf',
           'nan', 'nan', 'nan', '(2731, 8192)', '(3217, 1024)', '(1, 1)', '(2049, 2048)']

test = [Float16(x).val for x in data]

for index in range(len(test)):
    d = test[index]
    try:
        test[index] = str(d.as_integer_ratio())
    except Exception:
        test[index] = str(d)

np.testing.assert_array_equal(test, result)
```

```python

```


# Common engineering numeric problems
This notebook aims to collect a very brief summary of simple ways to solve common engineering problems in Python

For a far more comprehensive discussion of chemical engineering problems solved in Python, see the [Kitchin Group pycse file](http://kitchingroup.cheme.cmu.edu/pycse/pycse.html).

# Table lookup/interpolation

## 1d: numpy.interp

Problem: find P where T=130.

T	P

100	200

200	500

300	1000

```python
import numpy

T = [100, 200, 300]
P = [200, 500, 1000]

numpy.interp(150, T, P)
```

## 2d: scipy.interpolate.interp2d

Problem: Find the viscosity from this table for $T=230$ and $P=800$

T\P	200	500	1000

100	5	4	2

200	3	3	2

300	2	2	1

```python
import scipy.interpolate

T = [100, 200, 300]
P = [200, 500, 1000]

visc = [[5, 4, 2],
        [3, 3, 2],
        [2, 2, 1]]
```

```python
interpolator = scipy.interpolate.interp2d(T, P, visc)
interpolator(230, 800)
```

    array([ 2.1])

# Plotting

Plotting is normally handled by the matplotlib library. The pyplot interface aims for rough Matlab equivalence

```python
import matplotlib.pyplot as plt
```

The notebook allows for inline graphics

```python
%matplotlib inline
```

```python
plt.plot(T, P)
plt.xlabel('T')
plt.ylabel('P')
```

```python
plt.contour(T, P, visc)
```

# Algebra (manipulation of equations)

The sympy library supplies the ability to manipulate equations.

```python
import sympy
sympy.init_printing()  # This will make the expressions be printed in a more pretty way
```

```python
x = sympy.Symbol('x')
```

```python
equation = sympy.expand((x + 1)**4)
equation
```

```python
y = sympy.Symbol('y')
```

```python
equation.subs(x, y + 1)
```

# Systems of linear equations
Systems of linear equations can be solved directly without iteration. In both cases we reason via matrix algebra.

$$\begin{align}
2a + 3b + 4c &= 5 \\
2b + c &= 1 \\
a + b + c &= 1 \\
\end{align}$$

## Analytic solution
### sympy.solve

```python
a, b, c = sympy.symbols('a, b, c')
```

We state the equations in the form $f(a, b, c) = 0$.

```python
sympy.solve([2*a + 3*b + 4*c - 5,
             2*b - c - 1,
             a + b + c - 1], [a, b, c])
```

### scipy.linalg.solve

```python
A = numpy.matrix([[2, 3, 4],
                  [0, 2, -1],
                  [1, 1, 1]])
c = numpy.matrix([5, 1, 1]).T  # .T for transpose
```

Direct calculation via inverse:

```python
A.I*c
```

    matrix([[-1.],
            [ 1.],
            [ 1.]])

For larger matrices, it can be faster to use a dedicated solver:

```python
scipy.linalg.solve(A, c)
```

    array([[-1.],
           [ 1.],
           [ 1.]])

# Laplace transforms
## sympy.laplace_transform

```python
t, s = sympy.symbols('t, s')
```

```python
sympy.laplace_transform(sympy.sin(t), t, s)
```

    (1/(s**2 + 1), 0, True)

## sympy.inverse_laplace_transform

```python
sympy.inverse_laplace_transform(1/(s + 1), s, t)
```

# Systems of nonlinear equations
## Analytic solution (sometimes):
### sympy.solve

```python
sympy.solve([a**2 + b,
             a/b + 1], [a, b])
```

## Numeric solution (often)
### scipy.optimize.fsolve

```python
import scipy.optimize
```

```python
def eqs(x):
    a, b = x
    return [a**2 + b,
            a/b + 1]
```

```python
scipy.optimize.fsolve(eqs, [2, -1])
```

    array([ 1., -1.])

# Systems of ODEs (IVP)
## scipy.integrate.odeint

```python
def sys(x, t):
    a, b = x
    return [-2*a, -a - 2*b]
```

```python
times = numpy.linspace(0, 5)
```

```python
scipy.integrate.odeint?
```

```python
solution = scipy.integrate.odeint(sys, [1, 1], times)
```

```python
plt.plot(times, solution)
```

# Numerical integration (aka quadrature)
## Given a function
### scipy.integrate.quad

Let's say we're trying to integrate the surface under the curve $f(x) = \sin(x)/x$ between 0.5 and 1

```python
def f(x):
    return numpy.sin(x)/x
```

```python
x = numpy.linspace(0.01, 10)
```

```python
plt.plot(x, f(x))
```

```python
integral, error = scipy.integrate.quad(f, 0.5, 1)
print(integral)
```

    0.45297565232411635

## Given smooth data
### Trapezoidal rule: scipy.signal.trapz

```python
xdata = numpy.linspace(0.5, 1)
ydata = f(xdata)
```

```python
scipy.integrate.trapz(ydata, xdata)  # note it's _not_ x, y
```

### Simpson's rule: scipy.signal.simps

```python
scipy.integrate.simps(ydata, xdata)
```

# Numerical derivatives
## Given smooth data
### central difference estimate: numpy.gradient

```python
x = numpy.linspace(0, 2*numpy.pi)
```

```python
y = numpy.sin(x)
```

```python
derivative_estimate = numpy.gradient(y, x[1])
```

```python
plt.plot(x, y, x, derivative_estimate)
```

## Given noisy data
### Savitzky-Golay scipy.signal.savgol_filter

That's fine for smooth data, but what if you have some noise added?

```python
y_noise = y + numpy.random.randn(len(y))*0.1
```

```python
plt.plot(x, y_noise)
```

```python
derivative_estimate = numpy.gradient(y_noise, x[1])
```

```python
plt.plot(x, y_noise, x, derivative_estimate)
```

That's clearly not great.

```python
import scipy.signal
```

```python
smooth_derivative_estimate = scipy.signal.savgol_filter(y, 5, 2, deriv=1, delta=x[1])
```

```python
plt.plot(x, y_noise, x, smooth_derivative_estimate)
```

That's more like it!

# Optimisation
## Nonlinear
### scipy.optimize.minimize

Let's minimise $\sin(x)/x$

```python
plt.plot(x, f(x))
```

By inspection, we can see the minimum lies between 4 and 5.

```python
scipy.optimize.minimize(f, 4.5)
```

          fun: -0.21723362821071318
     hess_inv: array([[ 4.61057157]])
          jac: array([ -4.67523932e-07])
      message: 'Optimization terminated successfully.'
         nfev: 12
          nit: 3
         njev: 4
       status: 0
      success: True
            x: array([ 4.49340729])

# Statistical analysis
## Regression

Simple polynomial regression

```python
x = numpy.linspace(1, 4, 20)
```

```python
y = f(x)
```

```python
plt.plot(x, y, '.')
```

```python
quadratic = numpy.polyfit(x, y, 2)
```

```python
smoothx = numpy.linspace(1, 4, 1000)
```

```python
plt.plot(x, y, '.')
plt.plot(smoothx, numpy.polyval(quadratic, smoothx))
```

Useful references/Common problems solved in Python.ipynb


```python
from sympy import symbols, cos, sin, pi, simplify, pprint, tan, expand_trig, sqrt, trigsimp, atan2
from sympy.matrices import Matrix
```

```python
def pose(theta, alpha, a, d):
  # returns the pose T of one joint frame i with respect to the previous joint frame (i - 1)
  # given the parameters:
  # theta: theta[i]
  # alpha: alpha[i-1]
  # a: a[i-1]
  # d: d[i]

  r11, r12 = cos(theta), -sin(theta)
  r23, r33 = -sin(alpha), cos(alpha)
  r21 = sin(theta) * cos(alpha)
  r22 = cos(theta) * cos(alpha)
  r31 = sin(theta) * sin(alpha)
  r32 = cos(theta) * sin(alpha)
  y = -d * sin(alpha)
  z = d * cos(alpha)

  T = Matrix([
    [r11, r12, 0.0, a],
    [r21, r22, r23, y],
    [r31, r32, r33, z],
    [0.0, 0.0, 0.0, 1]
  ])

  T = simplify(T)

  return T
```

```python
# get the pose (homogenous transforms) of each joint wrt to previous joint

q1, q2, q3, q4, q5, q6= symbols('q1:7')

d90 = pi / 2

T01 = pose(q1, 0, 0, 0.75)
T12 = pose(q2 - d90, -d90, 0.35, 0)
T23 = pose(q3, 0, 1.25, 0)
T34 = pose(q4, -d90, -0.054, 1.5)
T45 = pose(q5, d90, 0, 0)
T56 = pose(q6, -d90, 0, 0)
T6g = pose(0, 0, 0, 0.303)
```

```python
# From the poses, get the rotation of joint 3 wrt to the base frame and the transpose
# We will need this later

T03 = simplify(T01 * T12 * T23)
R03 = T03[:3, :3]
R03T = R03.T

print("R03 = ")
print()
print(R03)

print("R03.T = ")
print()
print(R03T)
```

    R03 =

    Matrix([
    [sin(q2 + q3)*cos(q1), cos(q1)*cos(q2 + q3), -sin(q1)],
    [sin(q1)*sin(q2 + q3), sin(q1)*cos(q2 + q3),  cos(q1)],
    [        cos(q2 + q3),        -sin(q2 + q3),        0]])
    R03.T =

    Matrix([
    [sin(q2 + q3)*cos(q1), sin(q1)*sin(q2 + q3),  cos(q2 + q3)],
    [cos(q1)*cos(q2 + q3), sin(q1)*cos(q2 + q3), -sin(q2 + q3)],
    [            -sin(q1),              cos(q1),             0]])

```python
# From the poses, get the rotation of joint 6 wrt to the joint 3
# We will need this later

T36 = simplify(T34 * T45 * T56)
R36 = T36[:3, :3]

print("R36 = ")
print()
print(R36)
```

    R36 =

    Matrix([
    [-sin(q4)*sin(q6) + cos(q4)*cos(q5)*cos(q6), -sin(q4)*cos(q6) - sin(q6)*cos(q4)*cos(q5), -sin(q5)*cos(q4)],
    [                           sin(q5)*cos(q6),                           -sin(q5)*sin(q6),          cos(q5)],
    [-sin(q4)*cos(q5)*cos(q6) - sin(q6)*cos(q4),  sin(q4)*sin(q6)*cos(q5) - cos(q4)*cos(q6),  sin(q4)*sin(q5)]])

```python
# rotation matrices in x, y, z axes

def rotx(q):

  sq, cq = sin(q), cos(q)

  r = Matrix([
    [1., 0., 0.],
    [0., cq,-sq],
    [0., sq, cq]
  ])

  return r

def roty(q):

  sq, cq = sin(q), cos(q)

  r = Matrix([
    [ cq, 0., sq],
    [ 0., 1., 0.],
    [-sq, 0., cq]
  ])

  return r

def rotz(q):

  sq, cq = sin(q), cos(q)

  r = Matrix([
    [cq,-sq, 0.],
    [sq, cq, 0.],
    [0., 0., 1.]
  ])

  return r
```

```python
# the yaw, pitch roll is given wrt to the URDF frame
# We must convert this to gripper frame by performing
# a rotation of 180 degrees ccw about the z axis and then
# a rotation of 90 degrees cw about the new y axis

# This is the transpose of the rotation of the urdf frame wrt to gripper frame and its transpose
# ( which is strangely the same) which is important later

Rgu = (rotz(pi) * roty(-pi/2)).T
RguT = Rgu.T
pprint(RguT)
print(Rgu == RguT)
```

    ⎡ 0    0    1⎤
    ⎢            ⎥
    ⎢ 0   -1.0  0⎥
    ⎢            ⎥
    ⎣1.0   0    0⎦
    True

```python
#  euler_R is the composite rotation matrix of the following
# a rotation of alpha in the z axis
# a rotation of beta in the new y axis
# a rotation of gamma in the new x axis
# this will be useful later

alpha, beta, gamma = symbols('alpha beta gamma', real = True)
euler_R = rotz(alpha) * roty(beta) * rotx(gamma)
```

```python
def get_wrist_center(gripper_point, R0g, dg = 0.303):

  # get the coordinates of the wrist center wrt to the base frame (xw, yw, zw)
  # given the following info:
  # the coordinates of the gripper (end effector) (x, y, z)
  # the rotation of the gripper in gripper frame wrt to the base frame (R0u)
  # the distance between gripper and wrist center dg which is along common z axis
  xu, yu, zu = gripper_point

  nx, ny, nz = R0g[0, 2], R0g[1, 2], R0g[2, 2]
  xw = xu - dg * nx
  yw = yu - dg * ny
  zw = zu - dg * nz

  return xw, yw, zw
```

```python
# This is given position and orientation of the gripper wrt to URDFrame
px, py, pz = 0.49792, 1.3673, 2.4988
roll, pitch, yaw = 0.366, -0.078, 2.561
```

```python
gripper_point = px, py, pz

# This is the rotation of the gripper in URDF wrt to base frame
R0u_eval = euler_R.evalf(subs = {alpha: yaw, beta: pitch, gamma: roll})

# R0g * Rgu = R0u
R0g_eval = R0u_eval * RguT

# calculate wrist center
wrist_center = get_wrist_center(gripper_point, R0g_eval, dg = 0.303)
print("wrist_center", wrist_center)

# evaluated R0g
print("evaluated R0g:")
pprint(R0g_eval)
```

    wrist_center (0.750499428337951, 1.20160389781975, 2.47518995758694)
    evaluated R0g:
    ⎡0.257143295038827   0.48887208255965   -0.833595473062543⎤
    ⎢                                                         ⎥
    ⎢0.259329420712765  0.796053601157403    0.54685182237706 ⎥
    ⎢                                                         ⎥
    ⎣0.93092726749696   -0.356795110642117  0.0779209320563015⎦

```python
def get_hypotenuse(a, b):
  # calculate the longest side given the two shorter sides of a right triangle using pythagorean theorem
  return sqrt(a*a + b*b)

def get_cosine_law_angle(a, b, c):
  # given all sides of a triangle a, b, c
  # calculate angle gamma between sides a and b  using cosine law

  cos_gamma = (a*a + b*b - c*c) / (2*a*b)
  sin_gamma = sqrt(1 - cos_gamma * cos_gamma)
  gamma = atan2(sin_gamma, cos_gamma)

  return gamma

def get_first_three_angles(wrist_center):
  # given the wrist center which a tuple of 3 numbers x, y, z
  # (x, y, z) is the wrist center point wrt base frame
  # return the angles q1, q2, q3 for each respective joint
  # given geometry of the kuka kr210

  x, y, z  = wrist_center

  a1, a2, a3 = 0.35, 1.25, -0.054
  d1, d4 = 0.75, 1.5
  l = 1.50097168527591 #get_hypotenuse(d4, -a3)
  phi = 1.53481186671284 # atan2(d4, -a3)

  x_prime = get_hypotenuse(x, y)
  mx = x_prime -  a1
  my = z - d1
  m = get_hypotenuse(mx, my)
  alpha = atan2(my, mx)

  gamma = get_cosine_law_angle(l, a2, m)
  beta = get_cosine_law_angle(m, a2, l)

  q1 = atan2(y, x)
  q2 = pi/2 - beta - alpha
  q3 = -(gamma - phi)

  return q1, q2, q3

  return q1, q2, q3
```

```python
j1, j2, j3 = get_first_three_angles(wrist_center)

print("q1:", j1.evalf())
print("q2:", j2.evalf())
print("q3:", j3.evalf())
```

    q1: 1.01249809363771
    q2: -0.275800363737724
    q3: -0.115686651053751

```python
'''
Recall that from our simplification earlier, R36 equals the following:

Matrix([
[-sin(q4)*sin(q6) + cos(q4)*cos(q5)*cos(q6), -sin(q4)*cos(q6) - sin(q6)*cos(q4)*cos(q5), -sin(q5)*cos(q4)],
[                           sin(q5)*cos(q6),                           -sin(q5)*sin(q6),          cos(q5)],
[-sin(q4)*cos(q5)*cos(q6) - sin(q6)*cos(q4),  sin(q4)*sin(q6)*cos(q5) - cos(q4)*cos(q6),  sin(q4)*sin(q5)]])

From trigonometry we can get q4, q5, q6 if we know numerical values of all cells of matrix R36
'''

def get_last_three_angles(R):

  sin_q4 = R[2, 2]
  cos_q4 =  -R[0, 2]

  sin_q5 = sqrt(R[0, 2]**2 + R[2, 2]**2)
  cos_q5 = R[1, 2]

  sin_q6 = -R[1, 1]
  cos_q6 = R[1, 0]

  q4 = atan2(sin_q4, cos_q4)
  q5 = atan2(sin_q5, cos_q5)
  q6 = atan2(sin_q6, cos_q6)

  return q4, q5, q6
```

```
 - R0g = R03 * R36 * R6g
 - frame of joint 6 is the same orientation of gripper frame so  R6g = I
 - R03.T * R0g = R03.T * R03 * R36 * I
 ---> R36 = R03.T * R0g

 Recall we have this expression earlier for R03T:
   Matrix([
    [sin(q2 + q3)*cos(q1), sin(q1)*sin(q2 + q3),  cos(q2 + q3)],
    [cos(q1)*cos(q2 + q3), sin(q1)*cos(q2 + q3), -sin(q2 + q3)],
    [            -sin(q1),              cos(q1),             0]])

 Recall we also have evaluated R0g earlier.
   Matrix([
     [0.257143295038827, 0.488872082559650, -0.833595473062543],
     [0.259329420712765, 0.796053601157403, 0.546851822377060],
     [0.930927267496960, -0.356795110642117, 0.0779209320563015]])

  We also have solved for q1, q2, q3 earlier:
    q1: 1.01249809363771
    q2: -0.275800363737724
    q3: -0.115686651053748

  So we can actually evaluate for R36 because we have numerical values for
    R03.T and R0g
```

```python
R03T_eval = R03T.evalf(subs = {q1: j1.evalf(), q2: j2.evalf(), q3: j3.evalf()})
R36_eval = R03T_eval * R0g_eval

j4, j5, j6 = get_last_three_angles(R36_eval)

print("q1:", j1.evalf())
print("q2:", j2.evalf())
print("q3:", j3.evalf())
print("q4:", j4.evalf())
print("q5:", j5.evalf())
print("q6:", j6.evalf())
```

    q1: 1.01249809363771
    q2: -0.275800363737724
    q3: -0.115686651053751
    q4: 1.63446527240323
    q5: 1.52050002599430
    q6: -0.815781306199679


```python
%matplotlib inline
from matplotlib import pyplot as plt
import numpy as np
```

### Extracting information about sequence quality and enrichment

### Enrichment

+ Often want to compare two datasets (tissue 1 vs. tissue 2; -drug vs. +drug; etc.)
+ Done by taking ratio of counts for sequences between data sets

$$f_{seq} = \frac{C_{seq}}{C_{total}}$$

The normalized frequency of a sequence $f_{seq}$ is determined by the number of counts of that sequence relative to all counts in the data set.

The enrichment of the sequence in dataset 2 vs. dataset 1 is given by:

$$E_{seq} = \frac{f_{seq,2}}{f_{seq,1}}$$

where $f_{seq,1}$ and $f_{seq,2}$ are the normalized frequencies of the sequence in dataset 1 and 2.

### How do we decide which sequences are high enough quality to include?

### For each cluster, you get a sequence of colors representing the sequence

+ Some bases are read well, others are ambiguous.

### The "Phred" score measures confidence in the base "call":

\begin{equation}
Q = -10log_{10}(p) \qquad \qquad \text{$(1)$}
\end{equation}

where $p$ is the probability that the call is **wrong**.

By rearranging Eq. $(1)$ above, we get:
\begin{equation}
p = 10^{- \frac{Q}{10}} \qquad \qquad \text{$(2)$}
\end{equation}

+ Create a plot of Q vs. p.  Is a high "Q" good or bad?

```python

```

#### Phred scores are encoded in last line:

```
@SRR001666.1 071112_SLXA-EAS1_s_7:5:1:817:345 length=60
GGGTGATGGCCGCTGCCGATGGCGTCAAATCCCACCAAGTTACCCTTAACAACTTAAGGG
+SRR001666.1 071112_SLXA-EAS1_s_7:5:1:817:345 length=60
IIIIIIIIIIIIIIIIIIIIIIIIIIIIII9IG9ICIIIIIIIIIIIIIIIIIIIIDIII
```

### Encoding goes like

|Letter | ASCII | $Q$ | $p$     |
|:-----:|:-----:|:---:| -------:|
|  `!`  | 33    | 0   | 1.00000 |
|  `"`  | 34    | 1   | 0.79433 |
|  `#`  | 35    | 2   | 0.63096 |
|  ...  | ...   | ... | ...     |
|  `J`  | 74    | 41  | 0.00008 |
|  `K`  | 75    | 42  | 0.00006 |

### python `chr` command converts integer ASCII to character

```python
print(chr(33))
print(chr(34))
print(chr(35))
print("...")
print(chr(74))
print(chr(75))
```

    !
    "
    #
    ...
    J
    K

### Can create dictionary that converts letters to quality scores

Recall that
\begin{equation}
p = 10^{- \frac{Q}{10}} \qquad \qquad \text{$(2)$}
\end{equation}

```python
Q_dict = {}
p_dict = {}
for i in range(33,76):
    Q_dict[chr(i)] = i-33
    p_dict[chr(i)] = 10**(-(Q_dict[chr(i)])/10.)

p_dict["K"]
```

    6.309573444801929e-05

### Example

$$p_{correct} = \prod_{i=1}^{L} (1-p_{incorrect})$$

where $i$ indexes along sequence and $L$ is sequence length.

```python
qual_string = "IIIIIIIIIIIIIIIIIIIIIIIIIIIIII9IG9ICIIIIIIIIIIIIIIIIIIIIDIII"

p_correct = 1.0
for q in qual_string:
    p_correct = p_correct*(1-p_dict[q])

print(p_correct)
```

    0.9857512130454114

### Modify this code to pull out the $p_{correct}$ each the sequence

```python
import gzip

get_line = False
seqs = {}
with gzip.open("files/example.fastq.gz") as f:
    for l in f:
        l_ascii = l.decode("ascii")
        if l_ascii[0] == "@":
            get_line = True
            continue
        if get_line:
            try:
                seqs[l_ascii.strip()] += 1
            except KeyError:
                seqs[l_ascii.strip()] = 1
            get_line = False
```

### Does sequence quality vary along the length of your reads?

```python

```

```python

```


# Example Of Analyzing a Nonlinear Biochemical System

```python
import sympy
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sympy import init_session
init_session()
```

    IPython console for SymPy 1.7 (Python 3.6.9-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at https://docs.sympy.org/1.7/

```python
Integral(sqrt(1/x),x)
```

```python
da, A, a, x, k1, k2, db, b, B, k3, k4 = symbols('da A a x k1 k2 db b B k3 k4')
da = -(A - a)*x*k1+ k2*a
db = -k3*(B-b)*a - b*k4*(A-a)
solve([da, db], [a, b])
```

```python

```

```python
from sympy import solve

xdd, z, dxx, xZ, x, g, zdd, ydd, yZ, y, Ldy, zZ, zdd, Ldz = symbols("xdd z dxx xZ x g zdd ydd yZ y Ldy zZ zdd Ldz")

E1 = z * xdd + (xZ - x) * (g + zdd)
E2 = z * ydd + (yZ - y) * (g + zdd) - Ldy
E3 = -y * xdd + x * ydd - zZ * (g + zdd) + Ldz

sols = solve([E1, E2, E3], [xdd, ydd, Ldy])

print ("xdd = ", (sols[xdd]).factor())
print ("ydd = ", (sols[ydd]).factor())
print ("Ldy = ", (sols[Ldy]).factor())
```

    xdd =  (g + zdd)*(x - xZ)/z
    ydd =  (-Ldz*z + g*x*y - g*xZ*y + g*z*zZ + x*y*zdd - xZ*y*zdd + z*zZ*zdd)/(x*z)
    Ldy =  (-Ldz*z + g*x*yZ - g*xZ*y + g*z*zZ + x*yZ*zdd - xZ*y*zdd + z*zZ*zdd)/x


# Numerical Solution of the Helmholtz Equation with Robin Boundary Conditions using the Finite Element Method

This notebook illustrates the numerical solution of the wave equation for an harmonic excitation and Robin boundary conditions using the [Finite Element Method](https://en.wikipedia.org/wiki/Finite_element_method) (FEM). The method aims at an approximate solution by subdividing the area of interest into smaller parts with simpler geometry, linking these parts together and applying methods from the calculus of variations to solve the problem numerically. The FEM is a well established method for the numerical approximation of the solution of partial differential equations (PDEs). The solutions of PDEs are often known analytically only for rather simple geometries. FEM based simulations allow to gain insights into other more complex cases.

## Problem Statement

The inhomogeneous [Helmholtz equation](https://en.wikipedia.org/wiki/Helmholtz_equation) is given as

\begin{equation}
\Delta P(\mathbf{x}, \omega) + \frac{\omega^2}{c^2} P(\mathbf{x}, \omega) = - Q(\mathbf{x}, \omega) .
\end{equation}

We aim for a numerical solution of the Helmholtz equation on the domain $V$ with respect to the homogeneous Robin boundary condition

\begin{equation}
V_n(\mathbf{x}, \omega) + \frac{1}{Z(\mathbf{x}, \omega)} P(\mathbf{x}, \omega) = 0 \qquad \text{for } x \in \partial V ,
\end{equation}

where $V_n(\mathbf{x}, \omega)$ denotes the particle velocity in inward normal direction to the boundary $\partial V$ of $V$ and $Z(\mathbf{x}, \omega)$ the acoustic impedance of the boundary.
The particle velocity can be linked to the pressure using the Euler equation

\begin{equation}
-\mathrm{j} \omega \rho_0 V_n(\mathbf{x}, \omega) = \frac{\partial}{\partial n} P(\mathbf{x}, \omega) ,
\end{equation}

where $\rho_0$ denotes the static density of air.
Introducing this into the Robin boundary equation above yields

\begin{equation}
\frac{\partial}{\partial n} P(\mathbf{x}, \omega) - \mathrm{j} \underbrace{\frac{\omega \rho_0}{Z}}_{\sigma} P(\mathbf{x}, \omega) = 0 \qquad \text{for } x \in \partial V .
\end{equation}

The medium impedance of air for free-field propagation is $Z_0 = \rho_0 c$, hence $\sigma_0 = \frac{\omega}{c}$ in this case. Free-field conditions can be simulated by matching the impedance of the boundary to $Z_0$.

## Variational Formulation

Starting from the [variational formulation of the Helmholtz equation](FEM_Helmholtz_equation_2D.ipynb#Variational-Formulation) (before application of Green's first theorem)

\begin{equation}
{-} \int_V \nabla P(\mathbf{x}, \omega) \cdot \nabla V(\mathbf{x}, \omega) \mathrm{d}x  + \int_{\partial V} V(\mathbf{x}, \omega) \frac{\partial}{\partial n}  P(\mathbf{x}, \omega) \mathrm{d}s + \frac{\omega^2}{c^2} \int_V P(\mathbf{x}, \omega) V(\mathbf{x}, \omega) \mathrm{d}x = -\int_V Q(\mathbf{x}, \omega) V(\mathbf{x}, \omega) \mathrm{d}x
\end{equation}

and introducing the Robin boundary condition into the second integral yields

\begin{equation}
{-} \int_V \nabla P(\mathbf{x}, \omega) \cdot \nabla V(\mathbf{x}, \omega) \mathrm{d}x  + \mathrm{j} \sigma \int_{\partial V} V(\mathbf{x}, \omega) P(\mathbf{x}, \omega) \mathrm{d}s + \frac{\omega^2}{c^2} \int_V P(\mathbf{x}, \omega) V(\mathbf{x}, \omega) \mathrm{d}x = -\int_V Q(\mathbf{x}, \omega) V(\mathbf{x}, \omega) \mathrm{d}x .
\end{equation}

It is common to express this integral equation in terms of the bilinear $a(P, V)$ and linear $L(V)$ forms

\begin{equation}
a(P, V) = \frac{\omega^2}{c^2} \int_V P(\mathbf{x}, \omega) V(\mathbf{x}, \omega) \mathrm{d}x - \int_V \nabla P(\mathbf{x}, \omega) \cdot \nabla V(\mathbf{x}, \omega) \mathrm{d}x + \mathrm{j} \sigma \int_{\partial V} V(\mathbf{x}, \omega) P(\mathbf{x}, \omega) \mathrm{d}s , \end{equation}

\begin{equation}
L(V) = -\int_V Q(\mathbf{x}, \omega) V(\mathbf{x}, \omega) \mathrm{d}x ,
\end{equation}

where

\begin{equation}
a(P, V) = L(V) .
\end{equation}

Computational implementations of the FEM (like FEniCS) may not be able to handle complex numbers.
In this case the problem can be split into two coupled problems with respect to the real and imaginary part. By introducing $P(\mathbf{x}, \omega) = P_r(\mathbf{x}, \omega) + \mathrm{j} P_i(\mathbf{x}, \omega)$ and $V(\mathbf{x}, \omega) = V_r(\mathbf{x}, \omega) + \mathrm{j} V_i(\mathbf{x}, \omega)$ and identifying the real and imaginary parts of the bilinear and linear forms, we get

\begin{equation}
a_r = \int_V \left( \frac{\omega^2}{c^2} V_r(\mathbf{x}, \omega) P_r(\mathbf{x}, \omega) -  \frac{\omega^2}{c^2} V_i(\mathbf{x}, \omega) P_i(\mathbf{x}, \omega) - \nabla P_r(\mathbf{x}, \omega) \cdot \nabla V_r(\mathbf{x}, \omega) + \nabla P_i(\mathbf{x}, \omega) \cdot \nabla V_i(\mathbf{x}, \omega) \right) \mathrm{d}x -
\sigma \cdot \int_{\partial V} \left( V_i(\mathbf{x}, \omega) P_r(\mathbf{x}, \omega) + V_r(\mathbf{x}, \omega) P_i(\mathbf{x}, \omega) \right) \mathrm{d}s,
\end{equation}

\begin{equation}
a_i = \int_V \left( \frac{\omega^2}{c^2} V_r(\mathbf{x}, \omega) P_i(\mathbf{x}, \omega) +  \frac{\omega^2}{c^2} V_i(\mathbf{x}, \omega) P_r(\mathbf{x}, \omega) - \nabla P_i(\mathbf{x}, \omega) \cdot \nabla V_r(\mathbf{x}, \omega) + \nabla P_r(\mathbf{x}, \omega) \cdot \nabla V_i(\mathbf{x}, \omega) \right) \mathrm{d}x +
\sigma \cdot \int_{\partial V} \left( V_r(\mathbf{x}, \omega) P_r(\mathbf{x}, \omega) - V_i(\mathbf{x}, \omega) P_i(\mathbf{x}, \omega) \right) \mathrm{d}s
\end{equation}

for the bilinear form.

## Numerical Solution

The numerical solution of the variational problem is based on [FEniCS](https://fenicsproject.org/), an open-source framework for numerical solution of PDEs.
Its high-level Python interface `dolfin` is used in the following to define the problem and compute the solution.
The implementation is based on the variational formulation derived above.
It is common in the FEM to denote the solution of the problem by $u$ and the test function by $v$.
The definition of the problem in FEniCS is very close to the mathematical formulation of the problem.

For the subsequent examples the solution of inhomogeneous wave equation for a point source $Q(\mathbf{x}) = \delta(\mathbf{x}-\mathbf{x_s})$ at position $\mathbf{x_s}$ is computed using the FEM.
A function is defined for this purpose, as well as for the plotting of the resulting sound field.

```python
import dolfin
import mshr
import matplotlib.pyplot as plt
%matplotlib inline

def Helmholtz_Robin(mesh, frequency, xs, sigma=dolfin.Constant(0), c=343):

    # squared wavenumber
    k2 = dolfin.Constant(2*dolfin.pi*frequency/c)**2

    # define function space
    V = dolfin.VectorFunctionSpace(mesh, "CG", 1, dim=2)

    # define variational problem
    (u_r, u_i) = dolfin.TrialFunction(V)
    (v_r, v_i) = dolfin.TestFunction(V)

    a_r = ( k2 * dolfin.inner(u_r,v_r) - k2 * dolfin.inner(u_i,v_i) - dolfin.inner(dolfin.grad(u_r), dolfin.grad(v_r)) + dolfin.inner(dolfin.grad(u_i), dolfin.grad(v_i)) ) * dolfin.dx  - sigma*dolfin.inner(u_r, v_i) * dolfin.ds - sigma*dolfin.inner(u_i, v_r) * dolfin.ds
    a_i = ( k2 * dolfin.inner(u_r,v_i) + k2 * dolfin.inner(u_i,v_r) - dolfin.inner(dolfin.grad(u_r), dolfin.grad(v_i)) - dolfin.inner(dolfin.grad(u_i), dolfin.grad(v_r)) ) * dolfin.dx  + sigma*dolfin.inner(u_r, v_r) * dolfin.ds - sigma*dolfin.inner(u_i, v_i) * dolfin.ds
    L_r = dolfin.Constant(0) * v_r * dolfin.dx
    L_i = dolfin.Constant(0) * v_i * dolfin.dx

    a = a_r + a_i
    L = L_r + L_i
    A, b = dolfin.assemble_system(a, L)

    # define inhomogenity
    delta = dolfin.PointSource(V.sub(0), xs, -1)  # negative amplitude accounts for -Q(x,w) in inhomogeneous wave equation
    delta.apply(b)

    # compute solution
    u = dolfin.Function(V)
    dolfin.solve(A, u.vector(), b)
    (u_r, u_i) = dolfin.split(u)

    return u_r

def plot_soundfield(u):
    '''plots solution of FEM-based simulation'''
    fig = plt.figure(figsize=(10,10))
    fig = dolfin.plot(u)
    plt.title(r'$P(\mathbf{x}, \omega)$')
    plt.xlabel(r'$x$ in m')
    plt.ylabel(r'$y$ in m')
    plt.colorbar(fig, fraction=0.038, pad=0.04);
```

### Sound Field in a Rectangular Room

The two-dimensional sound field in a rectangular room (rectangular plate) with homogeneous Robin boundary conditions is computed for the frequency $f=1000$ Hz and source position $x_s = (1.2,3.2)$ m.

```python
f = 1000  # frequency
xs = dolfin.Point(1.2, 3.2)  # source position

# define geometry and mesh
mesh = dolfin.RectangleMesh(dolfin.Point(0, 0), dolfin.Point(5, 4), 200, 200, "right/left")
```

First, the case of sound-hard (Neumann) boundary conditions with $\sigma = 0$ ($Z \to \infty$) is considered.

```python
# compute solution for sigma=0
u = Helmholtz_Robin(mesh, f, xs, sigma=dolfin.Constant(0))

# plot sound field
plot_soundfield(u)
plot_soundfield(abs(u))
plt.title(r'$|P(\mathbf{x}, \omega)|$');
```

Now the case of a matched boundary (free-field propagation) with $\sigma = \frac{\omega}{c}$ ($Z = Z_0$) is considered.

```python
# compute solution for free-field propogation
u = Helmholtz_Robin(mesh, f, xs, sigma=dolfin.Constant(2*dolfin.pi*f/343))

# plot sound field
plot_soundfield(u)
plot_soundfield(abs(u))
plt.title(r'$|P(\mathbf{x}, \omega)|$');
```

The last example shows the simulation result for sound-soft (Dirichlet) boundary conditions with $\sigma \to \infty$ ($Z=0$).

```python
# compute solution for very large sigma
u = Helmholtz_Robin(mesh, f, xs, sigma=dolfin.Constant(1e15))

# plot sound field
plot_soundfield(u)
plot_soundfield(abs(u))
plt.title(r'$|P(\mathbf{x}, \omega)|$');
```

The effect of the homogeneous Dirichlet boundary condition (zero pressure at the boundary) can be observed conveniently by inspecting the magnitude of the sound field close to the boundary.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT).