# Working with Symbolic Expressions

## 0.1 Finding an exact derivative with a computer algebra system

## 0.2 Doing symbolic algebra in Python

```python
from math import sin
def f(x):
    return (3*x**2 + x) * sin(x)
```

# 1 Modeling algebraic expressions

## 1.1 Breaking an expression into pieces

## 1.2 Building an expression tree

## 1.3 Translating the expression tree to Python

```python
class Power():
    def __init__(self,base,exponent):
        self.base = base
        self.exponent = exponent
```

```python
class Number():
    def __init__(self,number):
        self.number = number

class Variable():
    def __init__(self,symbol):
        self.symbol = symbol
```

This represents $x^2$:

```python
Power(Variable("x"),Number(2))
```

    <__main__.Power at 0x2d030c86710>

```python
class Product():
    def __init__(self, exp1, exp2):
        self.exp1 = exp1
        self.exp2 = exp2
```

This represents $3x^2$:

```python
Product(Number(3),Power(Variable("x"),Number(2)))
```

    <__main__.Product at 0x2d030c86dd8>

```python
class Sum():
    def __init__(self, *exps): #<1>
        self.exps = exps

class Function(): #<2>
    def __init__(self,name):
        self.name = name

class Apply(): #<3>
    def __init__(self,function,argument):
        self.function = function
        self.argument = argument

f_expression = Product( #<4>
                Sum(
                    Product(
                        Number(3),
                        Power(
                            Variable("x"),
                            Number(2))),
                    Variable("x")),
                Apply(
                    Function("sin"),
                    Variable("x")))
```

This represents $\cos(x^3 + -5)$:

```python
Apply(Function("cos"),Sum(Power(Variable("x"),Number("3")), Number(-5)))
```

    <__main__.Apply at 0x2d030ca1390>

## 1.4 Exercises

**Exercise:** Draw the expression $\ln(y^z)$ as a tree built out of elements and combinators from this section.

**Exercise:** Translate the expression from the previous exercise to Python code.  Write it both as a Python function and as a data structure built from elements and combinators.

**Solution:** Here's the ordinary Python function

```python
from math import log
def f(y,z):
    return log(y**z)
```

Here's the data structure:

```python
Apply(Function("ln"), Power(Variable("y"), Variable("z")))
```

    <__main__.Apply at 0x2d030ca17f0>

**Exercise:** Implement a “Quotient” combinator representing one expression divided by another.  How do you represent the following expression? $$\frac{a+b}{2}$$

```python
class Quotient():
    def __init__(self,numerator,denominator):
        self.numerator = numerator
        self.denominator = denominator
```

Here's the representation of $(a+b)/2$:

```python
Quotient(Sum(Variable("a"),Variable("b")),Number(2))
```

    <__main__.Quotient at 0x2d030ca1a20>

**Exercise:** Implement a `Difference` combinator representing one expression subtracted from another.  How can you represent the expression $b^2 - 4ac$?

**Solution:**

```python
class Difference():
    def __init__(self,exp1,exp2):
        self.exp1 = exp1
        self.exp2 = exp2
```

$b^2 - 4ac$ is then represented by:

```python
Difference(
    Power(Variable('b'),Number(2)),
    Product(Number(4),Product(Variable('a'), Variable('c'))))
```

    <__main__.Difference at 0x2d030ca8240>

**Exercise:** Implement a `Negative` combinator, representing the negation of an expression.  For example, the negation of $x^2 + y$  is $-(x^2 + y)$.  Represent the latter expression in code using your new combinator.

```python
class Negative():
    def __init__(self,exp):
        self.exp = exp
```

$-(x^2 + y)$ is represented by:

```python
Negative(Sum(Power(Variable("x"),Number(2)),Variable("y")))
```

    <__main__.Negative at 0x2d030ca83c8>

**Exercise:** Add a Function called `"sqrt"` representing a square root, and use it to encode the following formula:

$$\frac{-b \pm \sqrt{b^2 - 4ac}}{2a}$$

```python
A = Variable('a')
B = Variable('b')
C = Variable('c')
Sqrt = Function('sqrt')
```

```python
Quotient(
    Sum(
        Negative(B),
        Apply(
            Sqrt,
            Difference(
                Power(B,Number(2)),
                Product(Number(4), Product(A,C))))),
    Product(Number(2), A))
```

    <__main__.Quotient at 0x2d030ca8d30>

**Mini-project:** Create an abstract base class called Expression and make all of the elements and combinators inherit from it.  For instance, class Variable() should become class Variable(Expression).  Then, overload the Python arithmetic operations +, -, *, and / so they produce Expression objects.  For instance, the code 2 * Variable(“x”) + 3 should yield: Sum(Product(Number(2), Variable(“x”)), Number(3)).

**Solution:** see "expressions.py" file, and section 2.2 and beyond below.

# 2 Putting a symbolic expression to work

## 2.1 Finding all the variables in an expression

```python
def distinct_variables(exp):
    if isinstance(exp, Variable):
        return set(exp.symbol)
    elif isinstance(exp, Number):
        return set()
    elif isinstance(exp, Sum):
        return set().union(*[distinct_variables(exp) for exp in exp.exps])
    elif isinstance(exp, Product):
        return distinct_variables(exp.exp1).union(distinct_variables(exp.exp2))
    elif isinstance(exp, Power):
        return distinct_variables(exp.base).union(distinct_variables(exp.exponent))
    elif isinstance(exp, Apply):
        return distinct_variables(exp.argument)
    else:
        raise TypeError("Not a valid expression.")
```

```python
distinct_variables(Variable("z"))
```

    {'z'}

```python
distinct_variables(Number(3))
```

    set()

```python
distinct_variables(f_expression)
```

    {'x'}

## 2.2 Evaluating an expression

```python
from abc import ABC, abstractmethod

class Expression(ABC):
    @abstractmethod
    def evaluate(self, **bindings):
        pass
```

Note: we are redefining these classes now.

```python
class Number(Expression):
    def __init__(self,number):
        self.number = number
    def evaluate(self, **bindings):
        return self.number

class Variable(Expression):
    def __init__(self,symbol):
        self.symbol = symbol
    def evaluate(self, **bindings):
        try:
            return bindings[self.symbol]
        except:
            raise KeyError("Variable '{}' is not bound.".format(self.symbol))

class Product(Expression):
    def __init__(self, exp1, exp2):
        self.exp1 = exp1
        self.exp2 = exp2
    def evaluate(self, **bindings):
        return self.exp1.evaluate(**bindings) * self.exp2.evaluate(**bindings)
```

```python
Product(Variable("x"), Variable("y")).evaluate(x=2,y=5)
```

    10

```python
import math
from math import sin, cos, log

_function_bindings = {
    "sin": math.sin,
    "cos": math.cos,
    "ln": math.log
}

class Apply(Expression):
    def __init__(self,function,argument):
        self.function = function
        self.argument = argument
    def evaluate(self, **bindings):
        return _function_bindings[self.function.name](self.argument.evaluate(**bindings))
```

From the text: *... Similarly, we can add an “evaluate” method to the Sum, Power, Difference, or Quotient combinators....*

```python
class Sum(Expression):
    def __init__(self, *exps):
        self.exps = exps
    def evaluate(self, **bindings):
        return sum([exp.evaluate(**bindings) for exp in self.exps])

class Power(Expression):
    def __init__(self,base,exponent):
        self.base = base
        self.exponent = exponent
    def evaluate(self, **bindings):
        return self.base.evaluate(**bindings) ** self.exponent.evaluate(**bindings)

class Difference(Expression):
    def __init__(self,exp1,exp2):
        self.exp1 = exp1
        self.exp2 = exp2
    def evaluate(self, **bindings):
        return self.exp1.evaluate(**bindings) - self.exp2.evaluate(**bindings)

class Quotient(Expression):
    def __init__(self,numerator,denominator):
        self.numerator = numerator
        self.denominator = denominator
    def evaluate(self, **bindings):
        return self.numerator.evaluate(**bindings) / self.denominator.evaluate(**bindings)
```

Redefine `f_expression` in light of the new class definitions

```python
f_expression = Product( #<4>
                Sum(
                    Product(
                        Number(3),
                        Power(
                            Variable("x"),
                            Number(2))),
                    Variable("x")),
                Apply(
                    Function("sin"),
                    Variable("x")))
```

```python
f_expression.evaluate(x=5)
```

    -76.71394197305108

```python
from math import sin
def f(x):
    return (3*x**2 + x) * sin(x)

f(5)
```

    -76.71394197305108

## 2.3 Expanding an expression

```python
class Expression(ABC):
    @abstractmethod
    def evaluate(self, **bindings):
        pass
    @abstractmethod
    def expand(self):
        pass

    # Printing expressions legibly in REPL (See first mini project in 2.4)
    @abstractmethod
    def display(self):
        pass
    def __repr__(self):
        return self.display()
```

```python
class Sum(Expression):
    def __init__(self, *exps):
        self.exps = exps
    def evaluate(self, **bindings):
        return sum([exp.evaluate(**bindings) for exp in self.exps])
    def expand(self):
        return Sum(*[exp.expand() for exp in self.exps])
    def display(self):
        return "Sum({})".format(",".join([e.display() for e in self.exps]))

class Product(Expression):
    def __init__(self, exp1, exp2):
        self.exp1 = exp1
        self.exp2 = exp2
    def evaluate(self, **bindings):
        return self.exp1.evaluate(**bindings) * self.exp2.evaluate(**bindings)
    def expand(self):
        expanded1 = self.exp1.expand()
        expanded2 = self.exp2.expand()
        if isinstance(expanded1, Sum):
            return Sum(*[Product(e,expanded2).expand() for e in expanded1.exps])
        elif isinstance(expanded2, Sum):
            return Sum(*[Product(expanded1,e) for e in expanded2.exps])
        else:
            return Product(expanded1,expanded2)
    def display(self):
        return "Product({},{})".format(self.exp1.display(),self.exp2.display())

class Difference(Expression):
    def __init__(self,exp1,exp2):
        self.exp1 = exp1
        self.exp2 = exp2
    def evaluate(self, **bindings):
        return self.exp1.evaluate(**bindings) - self.exp2.evaluate(**bindings)
    def expand(self):
        return self
    def display(self):
        return "Difference({},{})".format(self.exp1.display(), self.exp2.display())

class Quotient(Expression):
    def __init__(self,numerator,denominator):
        self.numerator = numerator
        self.denominator = denominator
    def evaluate(self, **bindings):
        return self.numerator.evaluate(**bindings) / self.denominator.evaluate(**bindings)
    def expand(self):
        return self
    def display(self):
        return "Quotient({},{})".format(self.numerator.display(),self.denominator.display())

class Negative(Expression):
    def __init__(self,exp):
        self.exp = exp
    def evaluate(self, **bindings):
        return - self.exp.evaluate(**bindings)
    def expand(self):
        return self
    def display(self):
        return "Negative({})".format(self.exp.display())

class Number(Expression):
    def __init__(self,number):
        self.number = number
    def evaluate(self, **bindings):
        return self.number
    def expand(self):
        return self
    def display(self):
        return "Number({})".format(self.number)

class Power(Expression):
    def __init__(self,base,exponent):
        self.base = base
        self.exponent = exponent
    def evaluate(self, **bindings):
        return self.base.evaluate(**bindings) ** self.exponent.evaluate(**bindings)
    def expand(self):
        return self
    def display(self):
        return "Power({},{})".format(self.base.display(),self.exponent.display())

class Variable(Expression):
    def __init__(self,symbol):
        self.symbol = symbol
    def evaluate(self, **bindings):
        return bindings[self.symbol]
    def expand(self):
        return self
    def display(self):
        return "Variable(\"{}\")".format(self.symbol)

class Function():
    def __init__(self,name,make_latex=None):
        self.name = name
        self.make_latex = make_latex
    def latex(self,arg_latex):
        if self.make_latex:
            return self.make_latex(arg_latex)
        else:
            return " \\operatorname{{ {} }} \\left( {} \\right)".format(self.name, arg_latex)

class Apply(Expression):
    def __init__(self,function,argument):
        self.function = function
        self.argument = argument
    def evaluate(self, **bindings):
        return _function_bindings[self.function.name](self.argument.evaluate(**bindings))
    def expand(self):
        return Apply(self.function, self.argument.expand())
    def display(self):
        return "Apply(Function(\"{}\"),{})".format(self.function.name, self.argument.display())
```

```python
Y = Variable('y')
Z = Variable('z')
A = Variable('a')
B = Variable('b')
Product(Sum(A,B),Sum(Y,Z))
```

    Product(Sum(Variable("a"),Variable("b")),Sum(Variable("y"),Variable("z")))

```python
Product(Sum(A,B),Sum(Y,Z)).expand()
```

    Sum(Sum(Product(Variable("a"),Variable("y")),Product(Variable("a"),Variable("z"))),Sum(Product(Variable("b"),Variable("y")),Product(Variable("b"),Variable("z"))))

```python
f_expression = Product( #<4>
                Sum(
                    Product(
                        Number(3),
                        Power(
                            Variable("x"),
                            Number(2))),
                    Variable("x")),
                Apply(
                    Function("sin"),
                    Variable("x")))
```

```python
f_expression.expand()
```

    Sum(Product(Product(Number(3),Power(Variable("x"),Number(2))),Apply(Function("sin"),Variable("x"))),Product(Variable("x"),Apply(Function("sin"),Variable("x"))))

## 2.4 Exercises

**Exercise:** Write a function `contains(expression, variable)` which checks whether the given expression contains any occurence of the specified variable.

```python
def contains(exp, var):
    if isinstance(exp, Variable):
        return exp.symbol == var.symbol
    elif isinstance(exp, Number):
        return False
    elif isinstance(exp, Sum):
        return any([contains(e,var) for e in exp.exps])
    elif isinstance(exp, Product):
        return contains(exp.exp1,var) or contains(exp.exp2,var)
    elif isinstance(exp, Power):
        return contains(exp.base, var) or contains(exp.exponent, var)
    elif isinstance(exp, Apply):
        return contains(exp.argument, var)
    else:
        raise TypeError("Not a valid expression.")
```

**Exercise:** Write a “distinct_functions” function which takes an expression as an argument and returns the distinct, named functions like “sin” or “ln” that appear in the expression.

```python
def distinct_functions(exp):
    if isinstance(exp, Variable):
        return set()
    elif isinstance(exp, Number):
        return set()
    elif isinstance(exp, Sum):
        return set().union(*[distinct_functions(exp) for exp in exp.exps])
    elif isinstance(exp, Product):
        return distinct_functions(exp.exp1).union(distinct_functions(exp.exp2))
    elif isinstance(exp, Power):
        return distinct_functions(exp.base).union(distinct_functions(exp.exponent))
    elif isinstance(exp, Apply):
        return set([exp.function.name]).union(distinct_functions(exp.argument))
    else:
        raise TypeError("Not a valid expression.")
```

**Exercise:** Write a function contains_sum which takes an expression and returns True if it contains a Sum and returns False otherwise.

```python
def contains_sum(exp):
    if isinstance(exp, Variable):
        return False
    elif isinstance(exp, Number):
        return False
    elif isinstance(exp, Sum):
        return True
    elif isinstance(exp, Product):
        return contains_sum(exp.exp1) or contains_sum(exp.exp2)
    elif isinstance(exp, Power):
        return contains_sum(exp.base) or contains_sum(exp.exponent)
    elif isinstance(exp, Apply):
        return contains_sum(exp.argument)
    else:
        raise TypeError("Not a valid expression.")
```

**NOTE:** For the rest of the mini-projects, consult "expressions.py".

# 3 Finding the derivative of a function

For the rest of the notebook, I'll use the complete implementations from `expressions.py` so I don't have to re-implement every time.

```python
from expressions import *
```

```python
Product(Power(Variable("x"),Number(2)),Apply(Function("sin"),Variable("y")))
```

$$x ^ { 2 } \operatorname{ sin } \left( y \right)$$

## 3.1 Derivatives of powers

## 3.2 Derivatives of transformed functions

## 3.3 Derivatives of some special functions

## 3.4 Derivatives of products and compositions

## 3.5 Exercises

# 4 Taking derivatives automatically

## 4.1 Implementing a derivative method for expressions

```python
Sum(Variable("x"),Variable("c"),Number(1)).derivative(Variable("x"))
```

$$1 + 0 + 0$$

## 4.2 Implementing the product rule and chain rule

```python
Product(Variable("c"),Variable("x")).derivative(Variable("x"))
```

$$c\cdot 1$$

```python
Apply(Function("sin"),Power(Variable("x"),Number(2))).derivative(x)
```

$$1\cdot 2x ^ { 1 } \operatorname{ cos } \left( x ^ { 2 } \right)$$

## 4.3 Implementing the power rule

```python
f_expression = Product( #<4>
                Sum(
                    Product(
                        Number(3),
                        Power(
                            Variable("x"),
                            Number(2))),
                    Variable("x")),
                Apply(
                    Function("sin"),
                    Variable("x")))
```

```python
f_expression.derivative(x)
```

$$\left( 3\cdot 1\cdot 2x ^ { 1 } + 1 \right) \operatorname{ sin } \left( x \right) + \left( 3x ^ { 2 } + x \right)\cdot 1 \operatorname{ cos } \left( x \right)$$

## 4.4 Exercises

# 5 Integrating functions symbolically

## 5.1 Integrals as antiderivatives

## 5.2 Introducing the SymPy library

```python
from sympy import *
from sympy.core.core import *
Mul(Symbol('y'),Add(3,Symbol('x')))
```

    y*(x + 3)

```python
y = Symbol('y')
```

```python
x = Symbol('x')
```

```python
y*(3+x)
```

    y*(x + 3)

```python
y*(3+x).subs(x,1)
```

    4*y

```python
(x**2).diff(x)
```

    2*x

```python
(3*x**2).integrate(x)
```

    x**3

## 5.3 Exercises

**Exercise:** What is the integral of $f(x) = 0$?  Confirm your answer with SymPy, remembering that SymPy does not automatically include a constant of integration.

```python
Integer(0).integrate(x)
```

    0

**Exercise:** What is the integral of $x\cdot \cos(x)$?  Hint: look at the derivative of $x\sin(x)$.  Confirm your answer with SymPy.

```python
(x*cos(x)).integrate(x)
```

    x*sin(x) + cos(x)

**Exercise:** What is the integral of $x^2$?  Confirm your answer with SymPy.

```python
(x**2).integrate(x)
```

    x**3/3


(vector_calculus)=
# Vector calculus
```{index} Vector calculus
```

## Vector functions

**Vector** is a quantity that is described with magnitude and direction. A _vector function_ describes a vector whole value is a function of some independent variable.

For example, if $f(t)$ represents a real function of $t$, then vector function of $t$ would be:
\begin{equation}
\mathbf{F}(t)=[f_x(t), f_y(t), f_z(t)].
\end{equation}

We can visualise a vector in Cartesian (x, y, z) cooridante system as an arrow beginning at the origin and ending at point specified by the vector:

```python
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np

# Create plot
fig = plt.figure(figsize=(7,5))

# Add subplot with 3D axes projection
ax = fig.add_subplot(111, projection='3d')

ax.quiver(0, 0, 0, 4, 5, 7,
          color="blue")

ax.scatter(4, 5, 7, c="black")

ax.text(4.5, 5.5, 7, "(4, 5, 7)", fontsize=14)
ax.text(2, 1, 5, "$\mathbf{r}$", fontsize=14, color="blue")

ax.set_xlabel("x")
ax.set_ylabel("y")
ax.set_zlabel("z")

ax.set_xlim(0,10)
ax.set_ylim(0,10)
ax.set_zlim(0,10)

plt.show()
```

Vector (4, 5, 7) can be represented with _position vector_ $\mathbf{r}$ plotted on the graph.

**Example:**

Describe the shape of the geometric representation of the vector function given by
\begin{equation}
\mathbf{F}(t)=[t+(t/10)\sin{t}, t+(t/10)\cos{t}].
\end{equation}

```python
t = np.linspace(0,100, 301)

def F(t):
    return np.array([t+t/10*np.sin(t), t+(t/10.)*np.cos(t)])

t1 = np.linspace(0, 30, 200)
t2 = np.linspace(0, 50, 200)
t3 = np.linspace(0, 90, 200)
```

```python
fig, axes = plt.subplots(1, 3, figsize=(10,3), sharey=True)

ax1 = axes[0]
ax2 = axes[1]
ax3 = axes[2]

ax1.plot(F(t1)[0], F(t1)[1], label="$t=30$ s", color="blue")
ax2.plot(F(t2)[0], F(t2)[1], label="$t=50$ s", color="blue")
ax3.plot(F(t3)[0], F(t3)[1], label="$t=90$ s", color="blue")

ax1.quiver(0, 0, F(t1)[0,-1], F(t1)[1,-1],
           scale=1, scale_units="xy", width=0.02)

ax2.quiver(0, 0, F(t2)[0,-1], F(t2)[1,-1],
           scale=1, scale_units="xy", width=0.02)

ax3.quiver(0, 0, F(t3)[0,-1], F(t3)[1,-1],
           scale=1, scale_units="xy", width=0.02)

for ax in axes:
    ax.set_xlim(0,100)
    ax.set_ylim(0,100)
    ax.legend(loc="upper left")

ax1.set_ylabel("y")
ax2.set_xlabel("x")

plt.show()
```

### Differentiation
```{index} Vector differentiation
```
To differentiate vector functions, we need to take derivatives of its components:
\begin{equation}
\frac{d\mathbf{F}}{dt}=\Big(\frac{df_x}{dt}, \frac{df_y}{dt}, \frac{df_z}{dt}\Big).
\end{equation}

**Example:**

Let's find the derivative of $\mathbf{F}(t)=[t+(t/10)\sin{t}, t+(t/10)\cos{t}]$ with respect to $t$. We will differentiate each component to get:

\begin{equation}\frac{d\mathbf{F}}{dt}=\Big[\Big(1+\frac{1}{10}\sin{t}+\frac{t}{10}\cos{t}\Big), \Big(1+\frac{1}{10}\cos{t}-\frac{t}{10}\sin{t}\Big) \Big].
\end{equation}

We can also use [SymPy](https://www.sympy.org/en/index.html) package for Python that allows symbolic mathematics. With this package we can get analytical solutions to derivatives in this example. At first we can create separate derivatives:

```{index} SymPy: differentiation
```

```python
import sympy as sym
from IPython.display import display
from sympy import init_printing

init_printing(use_latex='mathjax')

# Create a symbol t variable will go by
t = sym.Symbol("t")

# Differentiate Fx with respect to t
print('dfx/dt =')
display(sym.diff(t+(t/10)*sym.sin(t), t))

# Differentiate Fy with respect to t
print('dfy/dt =')
display(sym.diff(t+(t/10)*sym.cos(t), t))
```

    dfx/dt =

$$\frac{t \cos{\left (t \right )}}{10} + \frac{\sin{\left (t \right )}}{10} + 1$$

    dfy/dt =

$$- \frac{t \sin{\left (t \right )}}{10} + \frac{\cos{\left (t \right )}}{10} + 1$$

**How can we represent vector differentiation geometrically?**

For that, we will use the position vector $\mathbf{r}$.  Letting $\mathbf{F}(t)=\mathbf{r}(t):$

$$\frac{d\mathbf{F}}{dt}=\frac{d\mathbf{r}}{dt}=\lim_{\delta t \rightarrow 0}\frac{\mathbf{r}(t+\delta t)-\mathbf{r}(t)}{\delta t}=\Big(\frac{dx}{dt}, \frac{dy}{dt}, \frac{dz}{dt}\Big).$$

Let's find tangent $\mathbf{t}(t)$ to the curve created by $\mathbf{r}(t)=[10+5\sin{t}, 10+5\cos{t}]$ as $t$ goes from 0 to $2\pi$.

```python
t = sym.Symbol("t")

print('dx/dt =')
display(sym.diff(10+5*sym.sin(t)))

print('dy/dt =')
display(sym.diff(10+5*sym.cos(t)))
```

    dx/dt =

$$5 \cos{\left (t \right )}$$

    dy/dt =

$$- 5 \sin{\left (t \right )}$$

```python
def r(t):
    return np.array([10+5*np.sin(t), 10+5*np.cos(t)])

def dr(t):
    return np.array([5*np.cos(t), -5*np.sin(t)])
```

Unit tangent $\hat{\mathbf{t}}$ to the curve $C$ can be defined as:
\begin{equation}
\hat{\mathbf{t}}=\frac{d\mathbf{r}/dt}{|d\mathbf{r}/dt|}.
\end{equation}

Unit tangent is a vector in the same direction as vector derivative but its length is one.

```python
t = np.linspace(3,4.8,100)

plt.figure(figsize=(5,5))

plt.plot(r(t)[0], r(t)[1], color="black",
        lw=2)

plt.quiver(0, 0, r(3.8)[0], r(3.8)[1],
           color="blue", scale=1,
           scale_units="xy", width=0.01)

plt.quiver(0, 0, r(4)[0], r(4)[1],
           color="blue", scale=1,
           scale_units="xy", width=0.01)

# Calculate dr(4) unit vector
# linalg.norm gets the length of a vector
x = (dr(4)/np.linalg.norm(dr(4)))[0]
y = (dr(4)/np.linalg.norm(dr(4)))[1]

# Plot unit vector exaggerated
plt.quiver(r(4)[0], r(4)[1],
           x, y,
           scale=8, color="red", width=0.01)

plt.text(4.5, 3, r"$\mathbf{r}(t)$", color="blue",
         fontsize=16)

plt.text(1, 5, r"$\mathbf{r}(t+\delta(t))$", color="blue",
         fontsize=16)

plt.text(6, 8, r"$C$", color="black", fontsize=16)
plt.text(3.5, 7, r"$d\mathbf{r}/dt$", color="red", fontsize=16)

plt.xlim(0,11)
plt.ylim(0,11)

plt.xlabel('x')
plt.ylabel('y')

plt.show()
```

```{admonition} Differentiation rules

If vectors $\mathbf{a}$, $\mathbf{b}$ and scalar $\lambda$ are functions of $t$, we can then differentiate:

\\[
\frac{d}{dt}(\mathbf{a+b})=\frac{d\mathbf{a}}{dt}+\frac{d\mathbf{b}}{dt},\\]
\\[\frac{d}{dt}(\lambda\mathbf{a})=\frac{d\lambda}{dt}\mathbf{a}+\frac{d\mathbf{a}}{dt}\lambda,\\]
\\[\frac{d}{dt}(\mathbf{a\cdot b})=\frac{d\mathbf{a}}{dt}\cdot\mathbf{b}+\frac{d\mathbf{b}}{dt}\cdot\mathbf{a}=\frac{d}{dt}(\mathbf{b\cdot a}),\\]
\\[\frac{d}{dt}(\mathbf{a\times b})=\frac{d\mathbf{a}}{dt}\times\mathbf{b}+\frac{d\mathbf{b}}{dt}\times\mathbf{a}\neq \frac{d}{dt}(\mathbf{b\times a}).\\]
```

## Scalar and vector fields
```{index} Scalar and vector fields
```
**Scalar field** is a scalar quantity that varies with position and can be written as $\Omega(x,y,z)=\Omega(\mathbf{r})$. Scalar fields are for example temperature and density. 3D scalar fields can be represented as iso-surfaces in 3D or contour plots in 2D for specific $z$ value.

For example, let's plot a scalar field:

$$f(x,y,z)=ze^{-xy}$$

```python
# Create x and y values between 0 and 1
x = np.linspace(0, 1, 100)
y = np.linspace(0, 1, 100)

# Create 2D grid ourt of x and y
X, Y = np.meshgrid(x, y)
Z = 5

# Create a contour plot
plt.contour(X, Y, Z*np.exp(-X*Y))

plt.colorbar(label="f(x, y, %g)" % Z)

plt.title(r"$f(x, y, z) = ze^{-xy}$")
plt.xlabel("x")
plt.ylabel("y")

plt.gca().set_aspect("equal")

plt.show()
```

```python
from mpl_toolkits.mplot3d import Axes3D

# Create plot
fig = plt.figure(figsize=(7,5))

# Add subplot with 3D axes projection
ax = fig.add_subplot(111, projection='3d')

colours = ["lightsalmon", "peachpuff", "lightcyan", "lightskyblue"]

# Plot surface
for idx, z in enumerate(range(3,10,2)):
    ax.plot_surface(X, Y, z*np.exp(-X*Y), color=colours[idx])

# Add extras
ax.set_xlabel("x")
ax.set_ylabel("y")
ax.set_zlabel("z")

ax.set_title("Iso-surfaces for z = 3, 5, 7, 9")

# Display
plt.show()
```

**Vector field** is a vector quantity that varies as a function of position and can be written as $\mathbf{F}(x,y,z)=\mathbf{F}(\mathbf{r})$. A vector field is for example ocean currents or wind field.

One way to visualise vector field is to use a _quiver plot_, where at each location we can draw an arrow with length and direction based on magnitude and direction of the field at that location. Quiver plots in Python can be plotted using 2D array of locations and x and y components of vector field at each location:

```{index} Quiver plot
```

```python
# Locations
x = np.linspace(0, 2*np.pi, 20)
y = np.linspace(0, 2*np.pi, 20)

X, Y = np.meshgrid(x,y)

# x and y components of the field
U = np.sin(X)**2
V = np.cos(Y)**2
```

```python
plt.figure(figsize=(5,5))
plt.quiver(X, Y, U, V, scale=20,
          width=0.005)

plt.xlabel("x")
plt.ylabel("y")

plt.show()
```

Vector fields can also be expressed with _streamline plots_. Vector field is tangent to the streamlines:
```{index} Streamline plot
````

```python
plt.figure(figsize=(5,5))

plt.quiver(X, Y, U, V, scale=10, width=0.01)

plt.streamplot(X, Y, U, V, color="red", linewidth=2)

plt.xlabel("x")
plt.ylabel("y")

plt.xlim(0,3)
plt.ylim(0,3)

plt.show()
```

### Derivatives
#### Gradient of a scalar field
```{index} Gradient of a scalar field
```
We can take derivatives of a scalar field in three directions $x$, $y$ and $z$. Three derivatives then create a vector field known as _gradient_:

$$\nabla\Omega = \Big(\frac{\partial\Omega}{\partial x}, \frac{\partial \Omega}{\partial y}, \frac{\partial\Omega}{\partial z}\Big).$$

The $\nabla$ symbol is known as "nabla" or "del" and it is defined as:

$$\nabla = \Big(\frac{\partial}{\partial x}, \frac{\partial}{\partial y}, \frac{\partial }{\partial z}\Big).$$

The vector $\nabla\Omega$ represents the rate of change of $\Omega$ in space.

Let's use SymPy to find gradient of a scalar field

$$\Omega=x^2+xy+y^2.$$

```python
from sympy.vector import gradient, CoordSys3D

R = CoordSys3D(' ')

Omega = R.x**2+R.x*R.y+R.y**2

# Print omega
display(Omega)

# Print grad omega
display(gradient(Omega))
```

$$\mathbf{{x}_{ }}^{2} + \mathbf{{x}_{ }} \mathbf{{y}_{ }} + \mathbf{{y}_{ }}^{2}$$

$$(2 \mathbf{{x}_{ }} + \mathbf{{y}_{ }})\mathbf{\hat{i}_{ }} + (\mathbf{{x}_{ }} + 2 \mathbf{{y}_{ }})\mathbf{\hat{j}_{ }}$$

The gradient becomes a vector in $\hat{\mathbf{i}}$ (x) and $\hat{\mathbf{j}}$ (y) directions.

Let's create a contour plot of scalar field $\Omega$ and quiver plot of $\nabla\Omega$.

```python
plt.rcParams.update({'font.size': 14})
plt.figure(figsize=(7,7))

x = np.linspace(0,10,100)
y = np.linspace(0,10,100)

X, Y = np.meshgrid(x, y)

Omega = X**2+X*Y+Y**2

# Create contour plot of scalar field
plt.contour(X, Y, Omega, 20, cmap="jet",
           zorder=1, linewidths=2)

plt.colorbar(label=r"$\Omega=x^2+xy+y^2$",
             fraction=0.046, pad=0.04)

x = np.linspace(0,10,10)
y = np.linspace(0,10,10)

X, Y = np.meshgrid(x, y)

# Create quiver plot of the gradient
plt.quiver(X, Y, 2*X+Y, X+2*Y,
           label=r"$\nabla\Omega$", zorder=2)

plt.legend(loc="lower left")

plt.xlabel("x")
plt.ylabel("y")

plt.gca().set_aspect("equal")

plt.show()
```

Scalar field $\Omega$ will vary at different rate in different directions. The _directional derivative_ of a unit vector $\hat{\mathbf{a}}$ is:
\begin{equation}
\frac{\partial \Omega }{\partial a}=\hat{\mathbf{a}}\cdot\nabla\Omega.
\end{equation}

For example, we are given scalar field $\Omega = x^2yz+4xz^2$ and we would like to know its derivative in the direction of $\mathbf{a}=(2,-1,-1)$ at point $P=(1,-2,-1)$.

First, we need to find $\hat{\mathbf{a}}$:

```python
a = np.array([2,-1,-1])

a_len = np.linalg.norm(a)

a_hat = a/a_len

print(a)
print(a_hat)
```

    [ 2 -1 -1]
    [ 0.81649658 -0.40824829 -0.40824829]

Through SymPy we can find the gradient of $\Omega$:
```{index} SymPy: gradient
```

```python
R = CoordSys3D(' ')

Omega = R.x**2*R.y*R.z+4*R.x*R.z**2

# Print omega
display(Omega)

# Print grad omega
display(gradient(Omega))
```

$$\mathbf{{x}_{ }}^{2} \mathbf{{y}_{ }} \mathbf{{z}_{ }} + 4 \mathbf{{x}_{ }} \mathbf{{z}_{ }}^{2}$$

$$(2 \mathbf{{x}_{ }} \mathbf{{y}_{ }} \mathbf{{z}_{ }} + 4 \mathbf{{z}_{ }}^{2})\mathbf{\hat{i}_{ }} + (\mathbf{{x}_{ }}^{2} \mathbf{{z}_{ }})\mathbf{\hat{j}_{ }} + (\mathbf{{x}_{ }}^{2} \mathbf{{y}_{ }} + 8 \mathbf{{x}_{ }} \mathbf{{z}_{ }})\mathbf{\hat{k}_{ }}$$

The directional derivative is a dot product between unit vector $\hat{\mathbf{a}}$ and $\nabla\Omega$:

$$\hat{\mathbf{a}}\cdot\nabla\Omega=0.816\times(2xyz+4z^2)-0.408\times(x^2z)-0.408\times(x^2y+8xz).$$

Now, we can evaluate it at point $P$:

```python
P = np.array([1,-2,1])

print(a_hat[0]*(2*P[0]*P[1]*P[2]+4*P[2]**2)+a_hat[1]*P[0]**2*P[2]+a_hat[2]*(P[0]**2*P[1]+8*P[0]*P[2]))
```

    -2.8577380332470415

#### The divergence of a vector field
```{index} Divergence of a vector field
```
The _divergence_ of a vector field $\mathbf{F}=(f_x, f_y, fz)$ is:

$$\text{div}\mathbf{F}=\nabla\cdot\mathbf{F}=\frac{\partial f_x}{\partial x}+\frac{\partial f_y}{\partial y}+\frac{\partial z}{\partial z}.$$

For example, take divergence of a vector field $\mathbf{F}=(x^2,3y,x^3)$:

```{index} SymPy: divergence
```

```python
from sympy.vector import divergence
R = CoordSys3D(' ')
v1 = R.x**2*R.i+ 3*R.y*R.j+R.x**3*R.k
v1
```

$$(\mathbf{{x}_{ }}^{2})\mathbf{\hat{i}_{ }} + (3 \mathbf{{y}_{ }})\mathbf{\hat{j}_{ }} + (\mathbf{{x}_{ }}^{3})\mathbf{\hat{k}_{ }}$$

```python
divergence(v1)
```

$$2 \mathbf{{x}_{ }} + 3$$

If divergence is applied to the flow velocity of some fluid, the divergence represents the net amount of fluid entering or leaving a particular point. The divergence is zero for incompressible fluids. Positive divergence implies that density of the fluid decreases at that location, while negative divergence suggests it is increasing.

If we look at the field $\mathbf{F}(x,y)=(x,y)$, its divergence is:

```python
v1 = R.x*R.i+ R.y*R.j
divergence(v1)
```

$$2$$

If that field indicated flow velocity, the positive divergence would mean flow increasing outward, which can indeed be seen in a quiver plot:

```python
x = np.linspace(-6, 6, 10)
y = np.linspace(-6, 6, 10)

X, Y = np.meshgrid(x, y)

plt.quiver(X, Y, X, Y)
plt.gca().set_aspect("equal")

plt.xlabel("x")
plt.ylabel("y")

plt.show()
```

#### Curl of a vector field
```{index} Curl of a vector field
```

_Curl_ of a vector field is defined as:

$$\text{curl}\mathbf{F}=\nabla\times\mathbf{F}=\Big(\frac{\partial f_z}{\partial y}-\frac{\partial f_y}{\partial z}\Big)\hat{\mathbf{i}}+\Big(\frac{\partial f_x}{\partial z}-\frac{\partial f_z}{\partial x}\Big)\hat{\mathbf{j}}+\Big(\frac{\partial f_y}{\partial x}-\frac{\partial f_x}{\partial y}\Big)\hat{\mathbf{k}}.$$

The result is another vector field. For example, let's take divergence of a vector field $\mathbf{F}=(z,x,y)$:

$$\nabla\times\mathbf{F}=(1-0)\hat{\mathbf{i}}+(1-0)\hat{\mathbf{j}}+(1-0)\hat{\mathbf{k}}=\hat{\mathbf{i}}+\hat{\mathbf{j}}+\hat{\mathbf{k}}.$$

To check our answer we can use SymPy:
```{index} SymPy: curl
```

```python
from sympy.vector import curl
R = CoordSys3D(' ')
v1 = R.z*R.i+ R.x*R.j+R.y*R.k
curl(v1)
```

$$\mathbf{\hat{i}_{ }} + \mathbf{\hat{j}_{ }} + \mathbf{\hat{k}_{ }}$$

#### Curl of a vector field that is a gradient of a scalar field

If we take a gradient of a scalar field $\Omega$ and take a curl of that field, the identity states:

$$\nabla\times(\nabla\Omega)=0.$$

Curl of a gradient field is always zero. We can check what SymPy returns:

```python
Omega = R.x**2*R.y*R.z+4*R.x*R.z**2
curl(gradient(Omega))
```

$$\mathbf{\hat{0}}$$

#### Laplacian
```{index} Laplacian for scalar/vector fields
```
Laplacian is defined as:

$$\nabla^2 = \nabla\cdot\nabla=\frac{\partial^2}{\partial x^2}$$

Laplacian can operate on both scalar and vector fields. For a vector field, it is defined as:

$$\nabla^2\mathbf{F}=(\nabla^2f_x,\nabla^2f_y,\nabla^2f_z).$$

If we take $\Omega = x^2yz+4*xz^2$, then Laplacian would be:

```python
divergence(gradient(Omega))
```

$$8 \mathbf{{x}_{ }} + 2 \mathbf{{y}_{ }} \mathbf{{z}_{ }}$$

If we take $\mathbf{F}=x^2yz\hat{\mathbf{i}}+xyz^2\hat{\mathbf{j}}+(xy-2y^2+2x^2z)\hat{\mathbf{k}}$, then we can calculate $\nabla^2f_x,\nabla^2f_y,\nabla^2f_z$:

```python
F = R.x**2*R.y*R.z*R.i+R.x*R.y*R.z**2*R.j +(R.x*R.y-2*R.y**2+2*R.x**2*R.z)*R.k
F
```

$$(\mathbf{{x}_{ }}^{2} \mathbf{{y}_{ }} \mathbf{{z}_{ }})\mathbf{\hat{i}_{ }} + (\mathbf{{x}_{ }} \mathbf{{y}_{ }} \mathbf{{z}_{ }}^{2})\mathbf{\hat{j}_{ }} + (2 \mathbf{{x}_{ }}^{2} \mathbf{{z}_{ }} + \mathbf{{x}_{ }} \mathbf{{y}_{ }} - 2 \mathbf{{y}_{ }}^{2})\mathbf{\hat{k}_{ }}$$

The $\nabla^2f_x$ is:

```python
divergence(gradient(R.x**2*R.y*R.z))
```

$$2 \mathbf{{y}_{ }} \mathbf{{z}_{ }}$$

The $\nabla^2f_y$:

```python
divergence(gradient(R.x*R.y*R.z**2))
```

$$2 \mathbf{{x}_{ }} \mathbf{{y}_{ }}$$

The $\nabla^2f_z$:

```python
divergence(gradient((R.x*R.y-2*R.y**2+2*R.x**2*R.z)))
```

$$4 \mathbf{{z}_{ }} - 4$$


These questions are so classic that they appear so many times that it's worth it to dedicate a Hall of Fame for them.

# Classic Problem 1

## Control the Temperature - How much wood is needed?

This question basically utilizes the bisection method.

This is a collection, don't run it, because some variables are not defined.

```python
import numpy as np

offset = 375
intervals = [(0, max_wood)]

while abs(get_temperature((intervals[-1][0] + intervals[-1][1]) / 2) - 375) >= epsilon:
    mid = (intervals[-1][0] + intervals[-1][1]) / 2
    if get_temperature(mid) - 375 > 0:
        intervals.append((intervals[-1][0], mid))
    else:
        intervals.append((mid, intervals[-1][1]))

weight = (intervals[-1][0] + intervals[-1][1]) / 2
```

# Interesting Problem 1

## Sparse Matrix-vector multiplication

This question only appeared once, but I think it's fun.

```python
import numpy as np

Ax = np.zeros(shape[0])

indexes = list(A)
for i in range(len(indexes)):
    for k in A[indexes[i]]:
        Ax[indexes[i]] += A[indexes[i]][k] * x[k]
```

# Classic Problem 2

## Estimate the amount of clay

This question utilizes the Monte Carlo Method, fun to work on.

Nevertheless, remember to read the conditions carefully so that you don't miss any of them.

```python
import numpy as np
from random import random

n = 100000
count = 0

for i in range(n):
    x = random()*10-5
    y = random()*10-5
    z = random()*2
    if z < f(x, y) and x**2+y**2 < 25:
        count += 1

volume = count / n * 10 * 10 * 2
```

# Classic Problem 3

## Solve linear system using SVD

So this question basically only allows you to multiply $U$ on the right and $V$ on the left. We will assume that A is invertible, which saves a lot of troubles.

Let's try to justify it with analytic expression:

$Ax=b$

$A^{-1}Ax = A^{-1}b$

$x = V\Sigma^{-1}U^Tb$

$x = V\Sigma^{-1}(b^TU)^T$

Therefore, we can conclude that the following code works:

```python
import numpy as np
x = left_multiply_with_V(np.diag(1/sigma)@right_multiply_with_U(b).T)
```

# Interesting Problem 2

## Newton's Method - Newton's Method

This is Newton's method used for ND solve. At first I think my scripts are not directly applicable, but then I found a way to transform the problem into what I need.

```python
import numpy as np
import numpy.linalg as la
import sympy as sp

# These functions are from the newton_nd_solve script
def subs_all…
def calculated_jacobian…
def newton_nd_step…

f_strs = ['16*x**4+16*y**4+z**4-16', 'x**2+y**2+z**2-3', 'x**3-y']
x_prev = [1, 1, 1]
var_str = 'x y z'
res = []
for i in range(5):
    x_prev = [1, 1, 1]
    for j in range(i+1):
        jac, x_prev = newton_nd_step(x_prev, f_strs, var_str)
    res.append(np.array(x_prev))
guesses = np.array(res)
```


```python
import sympy as sy
sy.init_printing()
```

```python
M, m, g, l, u, w, z, theta, thetadot = sy.symbols('M,m,g,l,u,w,z,theta, thetadot')
```

```python
Mm = sy.Matrix([[M+m, -m*l*sy.cos(theta)], [-m*l*sy.cos(theta), m*l**2]])
Mm
```

```python
G = sy.Matrix([[m*l*sy.sin(theta)*thetadot**2],[m*g*l*sy.sin(theta)]])
G
```

```python
F = sy.Matrix([[u+w],[l*w]])
F
```

```python
acc = Mm.inv()*(F-G)
sy.simplify(acc.subs({w:0}))
```

```python
sy.simplify(sy.sin(theta+sy.pi))
```

```python
sy.simplify(sy.cos(theta+sy.pi))
```

```python
sy.simplify(sy.cos(-theta))
```

```python
theta2 = -theta+sy.pi
```

```python
sy.simplify(sy.cos(theta2))
```

## With Lagrangian mechanics

```python
from sympy.physics.mechanics import *
q1,q2 = dynamicsymbols('q1,q2')  # q1=z, q2=theta
q1d,q2d = dynamicsymbols('q1,q2',1)

zpdot = q1d -l*sy.cos(q2)*q2d
ypdot = l*sy.sin(q2)*q2d

T = 0.5*M*q1d**2 + 0.5*m*(zpdot**2 + ypdot**2)
sy.simplify(sy.expand(T))
```

```python
yp = -l*sy.cos(q2)
U = m*g*yp
L = T-U
L
```

```python
LM=LagrangesMethod(L, [q1,q2])
```

```python
sy.simplify(LM.form_lagranges_equations())
```

```python
sy.simplify(LM.mass_matrix)
```

```python
sy.simplify(LM.forcing)
```

```python
sy.simplify(LM.mass_matrix.inv()*LM.forcing)

```

```python

```


# Curse of Dimensionality

* The number of points needed to densely populate a space grows quickly with dimensionality. If you add informative, discriminating features - more features can be helpful - but you will need more and more points to fully understand the space.

* Things do not always behave as you would expect in high dimensions - this is part of the Curse of Dimensionality.

## Unit Porcupine:  The unit hyper-sphere inscribed within the unit hyper-cube.

* Consider a sphere with radius $r$ in $D$ dimensions
\begin{equation}
S = \left\{ \mathbf{x} \left| \sum_{i=1}^D x_i^2 \le r^2 \right. \right\} \nonumber
\end{equation}
It's volume is:
\begin{equation}
v_D(r) = \frac{r^D \pi^{\frac{D}{2}}}{\Gamma(\frac{d}{2} + 1)} \nonumber
\end{equation}
where $\Gamma(n) = \int_0^\infty e^{-x}x^{n-1}dx$.

  So, for $D = 1$: $v_1(r) = \frac{r \pi^{1/2}}{\Gamma(1/2 + 1)} = 2r$

  $D = 2$: $v_2(r) = \frac{r^2 \pi}{\Gamma(2)} = \pi r^2$

   $D = 3$: $v_3(r) = \frac{r^2 \pi^{3/2}}{\Gamma(3/2 + 1)} = \frac{4}{3}\pi r^3$

* Consider a hypercube with radius $r$.  It's volume is $(2r)^D$.

So, for $D = 1$: $v_{1,c} = 2r$

$D = 2$: $v_{2,c} = 4r^2$

$D = 3$: $v_{3,c} = 8r^3$
* Take the case where the hyper-sphere is inscribed within the unit hyper-cube.  What happens to the relative volume of the sphere and cube as $D$ increases?

\begin{eqnarray}
\frac{Vol(Sphere)}{Vol(Cube)} &=& \frac{r^D\pi^{\frac{D}{2}}}{\Gamma(\frac{D}{2} +1)(2r)^D}\\ \nonumber
&=& \frac{\pi^{\frac{D}{2}}}{2^D\Gamma(\frac{D}{2} + 1)}
\end{eqnarray}
Note: The $r$ dropped out, relative volume depends only on dimension.

```python
#Import needed python libraries
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
import math

#The Unit Porcupine Plot
#The unit hyper-sphere inscribed within the unit hyper-cube.

fig = plt.figure()
d = range(1,100)
V = [math.pi**(i/2)/(2**i*math.gamma(i/2 + 1)) for i in d]
ax = fig.add_subplot(*[1,2,1])
ax.plot(d, V)
ax.set_title('Ratio of Volume')

dCorner = [math.sqrt(d) for d in range(1,10000)]
ax = fig.add_subplot(*[1,2,2])
ax.plot(range(1,10000), dCorner)
ax.set_title('Distance to Corner')

plt.show();
```

## Volume of space between two spheres with slightly different radii in high dimensions

* $Vol_{crust} = Vol_{S_1} - Vol_{S_2}$ where radius of $S_1$ is greater than the radius of $S_2$
\begin{eqnarray}
Vol_{crust} &=& Vol_{S_1} - Vol_{S_2} \nonumber\\
&=& \left[ 1 - \frac{Vol_{S_2}}{Vol_{S_1}}\right]Vol_{S_1} \nonumber \\
&=& \left[ 1 - \frac{\frac{(a - \epsilon)^D \pi^{\frac{D}{2}}}{\Gamma(\frac{D}{2}+1)}}{\frac{a^D \pi^{\frac{D}{2}}}{\Gamma(\frac{D}{2}+1)}}\right]Vol_{S_1} \nonumber \\
&=& \left[ 1 - \frac{a^D ( 1 - \frac{\epsilon}{a})^D}{a^D}\right]Vol_{S_1}   \nonumber \\
&=&  \left[ 1 - \left( 1 - \frac{\epsilon}{a}\right)^D \right]Vol_{S_1}\nonumber
\end{eqnarray}

* What happens as $D$ increases?

```python
#Crust volume between spheres with epsilon different radii and increasing dimensionality
a = 1
eps = 0.001
D = np.arange(1,10000)
RatioVol = [1-(1-eps/a)**d for d in D]
fig = plt.figure()
plt.plot(D, RatioVol)
plt.title('Ratio of Volume of Crust to Bigger Sphere')
plt.show()
```

## Radius needed to cover the same percentage volume with growing dimensionality:

* Volume of unit line, square, cube, hyper-cube:  $s^D = 1^D$
* Side of a cube covering some percentage of the area: say, 10% would be $r^D = 1/10$, $r = (1/10)^{(1/D)}$
* What happens as D increases?

```python
'''Illustrate that average distance between uniform random points increases when dimensionality
increases and number of data points held constant. Illustrates that you need more and more
points to characterize the unit cube as you go up in dimensionality.'''

from mpl_toolkits.mplot3d import Axes3D
import scipy.spatial.distance as sc
import textwrap

N = 100

x1 = np.random.uniform(0,1,(N,1)) #generate N uniform random data points in the unit interval
x2 = np.random.uniform(0,1,(N,2)) #generate N uniform random data points in the unit square
x3 = np.random.uniform(0,1,(N,3)) #generate N uniform random data points in the unit cube
y1 = np.mean(sc.pdist(x1, 'euclidean')) #compute avg euclidean distance between points
y2 = np.mean(sc.pdist(x2, 'euclidean'))
y3 = np.mean(sc.pdist(x3, 'euclidean'))
z1 = np.mean(sc.pdist(x1, 'cityblock')) #compute l1 distance between points
z2 = np.mean(sc.pdist(x2, 'cityblock'))
z3 = np.mean(sc.pdist(x3, 'cityblock'))

#plot results
fig = plt.figure()
ax = fig.add_subplot(*[1,3,1])
ax.scatter(x1, np.zeros((N,1)))
myTitle = 'Avg. L2 Dist.: ' + str("%.2f"%y1) + ' Avg. L1 Dist.: ' + str("%.2f"%z1);
ax.set_title("\n".join(textwrap.wrap(myTitle, 20)))

ax = fig.add_subplot(*[1,3,2])
ax.scatter(x2[:,0], x2[:,1])
myTitle = 'Avg. L2 Dist.: ' + str("%.2f"%y2) + ' Avg. L1 Dist.: ' + str("%.2f"%z2);
ax.set_title("\n".join(textwrap.wrap(myTitle, 20)))

ax = fig.add_subplot(*[1,3,3], projection='3d')
ax.scatter(x3[:,0], x3[:,1], x3[:,2])
myTitle = 'Avg. L2 Dist.: ' + str("%.2f"%y3) + ' Avg. L1 Dist.: ' + str("%.2f"%z3);
ax.set_title("\n".join(textwrap.wrap(myTitle, 20)))
plt.show();
```

```python
#Percentage Volume Illustration (p=.1)

p = 0.1
fig = plt.figure()
x = np.arange(0,1,.05)

# 1D
ax = fig.add_subplot(*[1,3,1])
ax.plot(x, np.zeros(x.size))
z = np.arange(0,p,.01)
ax.plot(z, np.zeros(z.size), 'r')

# 2D
ax = fig.add_subplot(*[1,3,2])
[xx,yy] = np.meshgrid(x,x)
z = np.arange(0,math.sqrt(p),0.01)
[zx,zy] = np.meshgrid(z,z)
ax.scatter(xx, yy)
ax.scatter(zx, zy, color='red')

# 3D
ax = fig.add_subplot(*[1,3,3], projection='3d')
[xx,yy,zz] = np.meshgrid(x,x,x)
z = np.arange(0,math.sqrt(p),0.01)
[zx,zy,zz2] = np.meshgrid(z,z,z)
ax.scatter(xx, yy, zz)
ax.scatter(zx, zy, zz2, color='red')
plt.show()
```

# Principal Components Analysis

* So, a major take away from the Curse of Dimensionality discussion is that when we are in high dimensional spaces, much of the space is empty and the data lives at the surface.  Given this, it makes sense to use a lower-dimensional *manifold* representation of the data.

* A very common approach (and one of the simplest approaches) to dimensionality reduction is  Principal Components Analysis (PCA).  PCA  takes data from sensor coordinates to data centric coordinates using linear projections (i.e., it is assuming that the informative components of the data lies on a linear manifold.)

* PCA uses a linear transformation to minimize the redundancy of the resulting transformed data (by ending up with data that is uncorrelated).

* PCA finds the directions of maximum variance in high-dimensional data and projects it onto a new subspace with equal or fewer dimensions than the original one.

* Without loss of generality, let's assume the input data has zero mean.
\begin{equation}
\mathbf{y} = \mathbf{A}\mathbf{x}
\end{equation}

The correlation matrix of $\mathbf{y}$ is:
\begin{eqnarray}
R_y &=& E[\mathbf{y}\mathbf{y}^T]\\
&=& E[\mathbf{A}\mathbf{x}\mathbf{x}^T\mathbf{A}^T]\\
&=&\mathbf{A}R_x\mathbf{A}^T
\end{eqnarray}

 If we are given $N$ data vectors, $\{ \mathbf{x}_1, \ldots, \mathbf{x}_N \}$, we can estimate $R_x$ as
 \begin{equation}
 R_x \approx \frac{1}{n} \sum_{k=1}^n \mathbf{x}_k \mathbf{x}_k^T
 \end{equation}
 * This is a symmetric matrix, so, it's eigenvectors are mutually orthogonal.
 * So, if we choose $\mathbf{A}$ to have columns equal to the orthonomal eigenvectors of $R_x$, the $R_y$ is diagonal.
 \begin{eqnarray}
 \mathbf{R}_y = \left[ \begin{array}{c} \mathbf{e_1}^T \\ \mathbf{e_2}^T \\ \vdots \\ \mathbf{e_D}^T\end{array}\right]\mathbf{R}_x\left[ \mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_D \right] & = & \left[ \begin{array}{c c c c}\mathbf{e}_1^T\mathbf{R}_x\mathbf{e}_1 & \mathbf{e}_1^T\mathbf{R}_x\mathbf{e}_2 & \ldots & \mathbf{e}_1^T\mathbf{R}_x\mathbf{e}_D \\ \mathbf{e}_2^T\mathbf{R}_x\mathbf{e}_1 & \mathbf{e}_2^T\mathbf{R}_x\mathbf{e}_2 & \ldots & \mathbf{e}_2^T\mathbf{R}_x\mathbf{e}_D \\ \vdots & & \ddots & \vdots \\ \mathbf{e}_D^T\mathbf{R}_x\mathbf{e}_1 & \mathbf{e}_D^T\mathbf{R}_x\mathbf{e}_D & \ldots & \mathbf{e}_D^T\mathbf{R}_x\mathbf{e}_D \end{array}\right]\\
 & = & \left[ \begin{array}{c c c c} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_2 & \cdots & 0 \\ \vdots & & \ddots & \\ 0 & 0 & \cdots & \lambda_D \end{array}\right]
\end{eqnarray}
where $\mathbf{e}_i \in \mathbb{R}^{D\times1}$ and $\mathbf{R}_x \in \mathbb{R}^{D\times D}$.

* Note: Given that $\mathbf{e}_i$ is an eigenvector of $\mathbf{R}_x$, we know that $\lambda_i\mathbf{e}_i = \mathbf{R}_x\mathbf{e}_i$.  So, $\mathbf{e}_i^T\mathbf{R}_x\mathbf{e}_i = \mathbf{e}_i^T\left( \lambda_i \mathbf{e}_i \right) = \lambda_i$ using the fact that $\mathbf{e}_i$ is normalized (i.e., $\left\|\mathbf{e}_i\right\|_2^2$ = 1)

* Similarly, Given that $\mathbf{e}_i$ and $\mathbf{e}_j$ are orthogonal eigenvectors of $\mathbf{R}_x$, we know that $\lambda_i\mathbf{e}_i = \mathbf{R}_x\mathbf{e}_i$.  So, $\mathbf{e}_j^T\mathbf{R}_x\mathbf{e}_i = \mathbf{e}_j^T\left( \lambda_i \mathbf{e}_i \right) = 0$

## Given a symmetric matrix, the eigenvectors of distinct eigenvalues are orthogonal:

Let $\mathbf{A}$ be an $l\times l$ symmetric matrix, $\mathbf{A}^T = \mathbf{A}$.  Then the eigenvectors corresponding to distinct eigenvalues are orthogonal.  Let $\lambda_i \ne \lambda_j$ be two such eigenvalues.  The definitions we have:
 \begin{eqnarray}
 \mathbf{A} \mathbf{v}_i = \lambda_i\mathbf{v}_i \\
 \mathbf{A} \mathbf{v}_j = \lambda_j\mathbf{v}_j
 \end{eqnarray}

 By multiplying the first eigenvalue equation on the left by $\mathbf{v}_j^T$ and by the transpose of the second equation on the right by $\mathbf{v}_i$, we get:
\begin{eqnarray}
  \mathbf{v}_j^T \mathbf{A} \mathbf{v}_i =  \mathbf{v}_j^T \lambda_i\mathbf{v}_i \\
 \left(\mathbf{A} \mathbf{v}_j\right)^T\mathbf{v}_i = \left(\lambda_j\mathbf{v}_j\right)^T\mathbf{v}_i\\
   \mathbf{v}_j^T\mathbf{A}\mathbf{v}_i = \lambda_j\mathbf{v}_j^T\mathbf{v}_i\\
   \mathbf{v}_j^T \mathbf{A} \mathbf{v}_i - \mathbf{v}_j^T \mathbf{A} \mathbf{v}_i = 0 = \left(\lambda_i - \lambda_j\right)\mathbf{v}_j^T\mathbf{v}_i
\end{eqnarray}

Thus, $\mathbf{v}_j^T\mathbf{v}_i = 0$.

* The eigenvectors can be interpreted as an orthogonal axis defined by the data.
* Since all data is noisy, we can concentrate on the axes corresponding to the largest eigenvectors if you are interested in preserving the variance of the data.
* However, when using PCA within a classification problem, it is much more difficult because we are interested in discriminability (not necessarily variance).

## Principal Components Analysis - Maximal Variance Formulation

* PCA is a linear transformation
* PCA minimizes the redundancy of the resulting transformed data (by ending up data that is uncorrelated), minimizes the mean squared error between original and transformed/reduced data, and maximizes the retained variance of the data.

* Consider a data set of observations $\left\{ \mathbf{x}_n \right\}_{n=1}^N$ and $\mathbf{x}_n \in \mathbb{R}^D$. We want to maximize the variance of the projected data.

* Let us first consider reducing dimensionality to $M = 1$.  Let us define the projection as a vector $\mathbf{u}_1$ where $\mathbf{u}_1^T\mathbf{u}_1 = 1$.  Then, each projected data point into 1-D would be $y_n = \mathbf{u}_1^T\mathbf{x}_n$

* The mean of the sample data is $\bar{\mathbf{x}} = \frac{1}{N}\sum_{n=1}^N\mathbf{x}_n$ and the mean of the projected data is $\mathbf{u}_1^T\bar{\mathbf{x}}$

* The variance of projected data is:
\begin{eqnarray}
\frac{1}{N} \sum_{n=1}^N \left\{ \mathbf{u}_1^T\mathbf{x}_n - \mathbf{u}_1^T\bar{\mathbf{x}} \right\}^2 & = & \frac{1}{N} \sum_{n=1}^N  \left( \mathbf{u}_1^T\mathbf{x}_n - \mathbf{u}_1^T\bar{\mathbf{x}} \right)\left( \mathbf{u}_1^T\mathbf{x}_n - \mathbf{u}_1^T\bar{\mathbf{x}} \right)^T\\
& = & \frac{1}{N} \sum_{n=1}^N  \left( \mathbf{u}_1^T\mathbf{x}_n - \mathbf{u}_1^T\bar{\mathbf{x}} \right) \left( \mathbf{x}_n^T\mathbf{u}_1 - \bar{\mathbf{x}}^T\mathbf{u}_1 \right)\\
& = & \frac{1}{N} \sum_{n=1}^N  \mathbf{u}_1^T\mathbf{x}_n\mathbf{x}_n^T\mathbf{u}_1 - \mathbf{u}_1^T\mathbf{x}_n\bar{\mathbf{x}}^T\mathbf{u}_1 - \mathbf{u}_1^T\bar{\mathbf{x}}\mathbf{x}_n^T\mathbf{u}_1 + \mathbf{u}_1^T\bar{\mathbf{x}}\bar{\mathbf{x}}^T\mathbf{u}_1\\
& = & \mathbf{u}_1^T \left( \frac{1}{N} \sum_{n=1}^N \mathbf{x}_n\mathbf{x}_n^T - \mathbf{x}_n\bar{\mathbf{x}}^T - \bar{\mathbf{x}}\mathbf{x}_n^T+ \bar{\mathbf{x}}\bar{\mathbf{x}}^T\right)\mathbf{u}_1\\
& = & \mathbf{u}_1^T  \left( \frac{1}{N} \sum_{n=1}^N  \left(\mathbf{x}_n - \bar{\mathbf{x}}\right)\left(\mathbf{x}_n - \bar{\mathbf{x}}\right)^T \right) \mathbf{u}_1\\
& = & \mathbf{u}_1^T\mathbf{S}\mathbf{u_1}
\end{eqnarray}

* Now, we can maximize the projected variance with respect to $\mathbf{u}_1$ while constraining $\mathbf{u}_1^T\mathbf{u}_1 = 1$.  We will do this using a Lagrange multiplier:
\begin{equation}
L = \mathbf{u}_1^T\mathbf{S}\mathbf{u_1} + \lambda_1\left(1 - \mathbf{u}_1^T\mathbf{u}_1\right)
\end{equation}

* By taking the derivative of the Lagrangian and setting it equal to zero, we get:
\begin{equation}
\mathbf{S}\mathbf{u}_1 = \lambda_1\mathbf{u_1}
\end{equation}

* We can multiply the left side by $\mathbf{u}_1^T$ and get:
	\begin{equation}
	\mathbf{u_1}^T\mathbf{S}\mathbf{u_1} = \lambda_1
	\end{equation}

* So the variance of the projected data is equal to the eigenvalue of the covariance matrix of the sample data along the direction of the eigenvector used for dimensionality reduction.
* We can incrementally add new eigenvector directions (ordered by maximal eigenvalue/variance) to project into an $M$ dimensional space where $1\leq M \leq D$

## PCA for Minimization of Mean Squared Error

* We can also look at PCA as a minimization of mean squared error.
* Consider $\mathbf{x}\in R^n$ and an orthogonal basis $\mathbf{a}$:

 \begin{equation}
 \hat{\mathbf{x}} = \sum_{i=1}^m y_i\mathbf{a}_i
 \end{equation}
 where $m < n$.
 \begin{equation}
 y_j = \mathbf{x}^T\mathbf{a}_j
 \end{equation} where $\mathbf{A}^T\mathbf{A}=\mathbf{I}$

We want to minimize the residual error:
\begin{equation}
\epsilon = \mathbf{x} - \hat{\mathbf{x}} = \sum_{j=m+1}^n y_j \mathbf{a}_j
\end{equation}
The objective we will use is the mean square residual:
\begin{eqnarray}
J &=& E\{ \|\epsilon\|^2_2\}\\
&=& E\left\{\left( \sum_{i=m+1}^n y_i \mathbf{a}_i^T\right)\left( \sum_{i=m+1}^n y_i \mathbf{a}_i\right) \right\}\\
&=&\sum_{j=m+1}^n E \{y_j^2\}\\
&=&\sum_{j=m+1}^n E \{(\mathbf{a}_j^T\mathbf{x})(\mathbf{x}^T\mathbf{a}_j)\}\\
&=& \sum_{j=m+1}^n \mathbf{a}_j^T E\{\mathbf{x}\mathbf{x}^T\}\mathbf{a}_j\\
&=& \sum_{j=m+1}^n \mathbf{a}_j^T R_x\mathbf{a}_j
\end{eqnarray}
Minimize the error and incorporate Lagrange parameters for $\mathbf{A}^T\mathbf{A}=\mathbf{I}$:
\begin{eqnarray}
\frac{\partial J}{\partial \mathbf{a}_j} &=& 2(R_x\mathbf{a}_j - \lambda_j\mathbf{a}_j) = 0 \text{ for }j = m+1 \ldots n\\
R_x\mathbf{a}_j &=& \lambda_j\mathbf{a}_j
\end{eqnarray}
So, the sum of the error is the sum of the eigenvalues of the unused eigenvectors.  So, we want to select the eigenvectors with the $m$ largest values.

## Reading Assignment

* Section 1.4
* Section 12.1
* Appendix C
* (and Appendix E for Lagrange Optimization discussed in the previous lecture)

```python

```

Lecture12_CurseDim/Curse of Dimensionality and Principal Components Analysis.ipynb


# 3.4.2 The Lasso

The lasso is a shrinkage method, which is estimated by (3.51):
$$
\begin{equation}
\hat{\beta}^{lasso} = \underset{\beta}{argmin}\sum_{i=1}^N \left( y_i-\beta_0-\sum_{j=1}^p{x_{ij}\beta_j} \right)^2\\
\text{ subject to } \sum_{j=1}^p|\beta_j| \le t
\end{equation}
$$

we can re-parametrize the constant $\beta_0$ by standartizing the predictors; the solution for $\beta_0$ is $\overline{y}$.

We can also write the lasso in the equivalent *Lagragngian form* (3.52):

$$\hat{\beta}^{lasso} = \underset{\beta}{argmin} \left\{
\frac{1}{2}\sum_{i=1}^N \left( y_i-\beta_0-\sum_{j=1}^p{x_{ij}\beta_j} \right)^2
+\lambda\sum_{j=1}^p|\beta_j|
\right\}$$

The lasso solution is nonlinear in the $y_i$ and it is a quadratic programming problem.


```python
# some imports
from IPython.display import Math
from sympy import sin, cos, pi
from sympy.abc import x,y

from sympde.core import Constant
from sympde.calculus import grad, dot, inner, cross, rot, curl, div
from sympde.calculus import laplace, hessian, bracket, D, conv
from sympde.calculus import jump, avg, minus, plus, Dn

from sympde.topology import (dx, dy, dz)
from sympde.topology import ScalarFunctionSpace, VectorFunctionSpace
from sympde.topology import ScalarField, VectorField
from sympde.topology import ProductSpace
from sympde.topology import Domain, Square
from sympde.topology import element_of, elements_of
from sympde.expr import BilinearForm, LinearForm, integral, find
from sympde.expr import TerminalExpr
from sympde.printing.latex import latex
```

```python
DIM = 2
domain = Domain('Omega', dim=DIM)
```

```python
domain = Square('Omega')
```

```python
V = ScalarFunctionSpace('V', domain)
```

```python
v = element_of(V, name='v')
u = element_of(V, name='u')
```

```python
a = BilinearForm((v,u), integral(domain, dot(grad(v), grad(u))))
```

```python
Math(latex(a))
```

$\displaystyle \left( v, u \right) \mapsto \int_{\Omega} \nabla{u} \cdot \nabla{v} ~d\mathbf{x}$

```python
b = LinearForm(v, integral(domain, cos(x+y)*v))
```

```python
Math(latex(b))
```

$\displaystyle v \mapsto \int_{\Omega} v \cos{\left (x + y \right )} ~d\mathbf{x}$

```python
Math(latex(jump(u)))
```

$\displaystyle [\![ u ]\!]$

```python
Math(latex(avg(u)))
```

$\displaystyle \{ u \}$

```python
Math(latex(minus(u)))
```

$\displaystyle u_{-}$

```python
Math(latex(Dn(u)))
```

$\displaystyle \nabla u \cdot \mathbf{n}$

```python
Math(latex(jump(Dn(u))))
```

$\displaystyle [\![ \nabla u \cdot \mathbf{n} ]\!]$

```python
Math(latex(minus(Dn(u))))
```

$\displaystyle \nabla{u_{-}} \cdot \mathbf{n}_{-}$

```python
Math(latex(plus(Dn(u))))
```

$\displaystyle \nabla{u_{+}} \cdot \mathbf{n}_{+}$

```python
a = BilinearForm((v,u), integral(domain, dot(grad(v), grad(u)) + jump(u)*avg(v) - avg(u)*jump(v)))
```

```python
Math(latex(a))
```

$\displaystyle \left( v, u \right) \mapsto \int_{\Omega} - \{ u \} [\![ v ]\!] ~d\mathbf{x} + \int_{\Omega} \{ v \} [\![ u ]\!] ~d\mathbf{x} + \int_{\Omega} \nabla{u} \cdot \nabla{v} ~d\mathbf{x}$

```python
Math(latex(laplace(u)))
```

$\displaystyle \nabla^2u$

```python
Math(latex(hessian(u)))
```

$\displaystyle \mathrm{hess}( u )$

```python
Math(latex(conv(u, v)))
```

$\displaystyle u \ast v$

```python
W = VectorFunctionSpace('W', domain)
```

```python
w = element_of(W, name='w')
```

```python
Math(latex(D(w)))
```

$\displaystyle \mathrm{D}\left( \mathbf{w} \right)$

```python
l = LinearForm(v, integral(domain.boundary, cos(x+y)*v))
```

```python
Math(latex(l))
```

$\displaystyle v \mapsto \int_{\Gamma_1} v \cos{\left (x + y \right )} ~ds + \int_{\Gamma_2} v \cos{\left (x + y \right )} ~ds + \int_{\Gamma_3} v \cos{\left (x + y \right )} ~ds + \int_{\Gamma_4} v \cos{\left (x + y \right )} ~ds$

```python
equation = find(u, forall=v, lhs=a(u,v), rhs=l(v))
```

```python
Math(latex(equation))
```

$\displaystyle \mbox{find} ~u \in V ~\mbox{such that}\\\int_{\Omega} - \{ v \} [\![ u ]\!] ~d\mathbf{x} + \int_{\Omega} \{ u \} [\![ v ]\!] ~d\mathbf{x} + \int_{\Omega} \nabla{u} \cdot \nabla{v} ~d\mathbf{x} = \int_{\Gamma_1} \left ( v\right ) \cos{\left (x + y \right )} ~ds + \int_{\Gamma_2} \left ( v\right ) \cos{\left (x + y \right )} ~ds + \int_{\Gamma_3} \left ( v\right ) \cos{\left (x + y \right )} ~ds + \int_{\Gamma_4} \left ( v\right ) \cos{\left (x + y \right )} ~ds,\quad \forall~ v \in V$

### Multiple domains

```python
A = Square(r'A')
B = Square(r'B')

domain = A.join(B, name = 'Omega',
            bnd_minus = A.get_boundary(axis=0, ext=1),
            bnd_plus  = B.get_boundary(axis=0, ext=-1))

I = domain.interfaces
```

```python
V = ScalarFunctionSpace('V', domain, kind=None)
```

```python
u, v = elements_of(V, names='u, v')
```

```python
# Nitsch
kappa = Constant('kappa')
expr_I = ( - jump(u) * jump(Dn(v))
           + kappa * jump(u) * jump(v)
           + plus(Dn(u)) * minus(v)
           + minus(Dn(u)) * plus(v) )
a = BilinearForm((u,v), integral(domain, dot(grad(u),grad(v)))
                      + integral(I,      expr_I))
```

```python
Math(latex(a))
```

$\displaystyle \left( u, v \right) \mapsto \int_{\Omega} \nabla{u} \cdot \nabla{v} ~d\mathbf{x} + \int_{A|B} - [\![ u ]\!] [\![ \nabla v \cdot \mathbf{n} ]\!] ~ds + \int_{A|B} \left(\nabla{u_{-}} \cdot \mathbf{n}_{-}\right) v_{+} ~ds + \int_{A|B} \left(\nabla{u_{+}} \cdot \mathbf{n}_{+}\right) v_{-} ~ds + \int_{A|B} \kappa [\![ u ]\!] [\![ v ]\!] ~ds$

```python
expr = TerminalExpr(a)
```

```python
code = r'\\'.join(latex(e) for e in expr)
Math(code)
#print(code)
```

$\displaystyle \textbf{target}~A|B\\- \kappa u_{-} v_{+} + \left(\nabla{v_{+}} \cdot \mathbf{n}_{+}\right) u_{-}\\\textbf{target}~A|B\\- \kappa v_{-} u_{+} + \left(\nabla{v_{-}} \cdot \mathbf{n}_{-}\right) u_{+}\\\textbf{target}~\Gamma_2\\\kappa u v - u \left(\nabla{v} \cdot \mathbf{n}_{-}\right)\\\textbf{target}~\Gamma_1\\\kappa u v - u \left(\nabla{v} \cdot \mathbf{n}_{+}\right)\\\textbf{target}~A\\\partial_x u_{A} \partial_x v_{A} + \partial_y u_{A} \partial_y v_{A}\\\textbf{target}~B\\\partial_x u_{B} \partial_x v_{B} + \partial_y u_{B} \partial_y v_{B}$

```python

```


```python
from sympy import solve, symbols, plot, simplify
```

```python
f = (x-h)**2 + k - (x**2-4*x+7)
```

```python
x, h, k = symbols('x h k')
solve( f, [h,k] )
```

    [(2, 3)]

```python
f2 = ((x-2)**2 + 3).expand()
```

```python
    plot(f2, (x, -3, 3))
```

```python
plot(f2)
```

```python
x, r, s = symbols('x r s', real=True)
```

```python
f3 = x**12 + 6*x + 13 - ((x - r)**2 + s)
solve( f3, [r,s] )
```

    [(r, x**12 + 6*x - (r - x)**2 + 13)]

```python
plot(x**2-4*x+7)
```

```python
plot(x**2-20)
```

```python
plot(3*x**2+9*x-30)
```

```python
simplify(42/100)
```

$\displaystyle 0.42$

```python
plot(x**3)
```

```python

```


# Estacionaridade

Um fenômeno aleatório pode ser tal que sua densidade de probabilidade varia com o tempo - $p(x,t)$.

```python
# importar as bibliotecas necessárias
import numpy as np # arrays
import matplotlib.pyplot as plt # plots
from scipy.stats import norm
from scipy import signal
plt.rcParams.update({'font.size': 14})
import IPython.display as ipd # to play signals
import sounddevice as sd
```

# Um conjunto de ruídos brancos

Vamos gerar um sinal cuja densidade de probabilidade varia com o tempo. Faremos isso, a partir de uma distribuição normal, mas

\begin{equation}
p(x, t) = \mathcal{N}(\mu_x(t), \sigma_x(t)) = \frac{1}{\sqrt{2\pi}\sigma_x(t)}\mathrm{e}^{-\frac{1}{2\sigma_{x}^{2}(t)}(x(t)-\mu_x(t))^2}
\end{equation}
em que $\mu_x(t)$ é a média e $\sigma_{x}(t)$ é o desvio padrão (ambos podem variar com o tempo).

## Exemplo 1 - Não estacionário

```python
# Tempo
fs = 44100
time = np.arange(0,2, 1/fs)

# parâmetros de um fenômeno, cuja p(x) varia com o tempo
mu_x = np.sin(2*np.pi*0.4*time)    # média
mod_x = np.sin(2*np.pi*0.5*time)   # desvio padrão

# sinal
xt = np.random.normal(loc = mu_x, scale = mod_x**2, size=len(time))

# plot signal
plt.figure(figsize = (10, 3))
plt.plot(time, xt, linewidth = 1, color = 'b')
plt.axvline(0.5, color='lightcoral',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(0.80, color='grey',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(1.75, color='green',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.ylabel(r'$x(t)$')
plt.xlim((0, time[-1]))
plt.ylim((-4, 4))
plt.xlabel('tempo [s]')
plt.tight_layout()

# plot p(x,t)
x = np.arange(-6, 6, 0.01)
px1 = norm.pdf(x, loc= mu_x[int(0.50*fs)], scale = mod_x[int(0.50*fs)]**2)
px2 = norm.pdf(x, loc= mu_x[int(0.80*fs)], scale = mod_x[int(0.80*fs)]**2)
px3 = norm.pdf(x, loc= mu_x[int(1.75*fs)], scale = mod_x[int(1.75*fs)]**2)

plt.figure(figsize = (10, 3))
plt.subplot(1,3,1)
plt.plot(px1, x, 'lightcoral', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,2)
plt.plot(px2, x, 'grey', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,3)
plt.plot(px3, x, 'green', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.tight_layout()
```

## Exemplo 2 - Estacionário com respeito a média

```python
# Tempo
fs = 44100
time = np.arange(0,2, 1/fs)

# parâmetros de um fenômeno, cuja p(x) varia com o tempo
mu_x = 0.0    # média
mod_x = np.sin(2*np.pi*0.5*time)   # desvio padrão

# sinal
xt = np.random.normal(loc = mu_x, scale = mod_x**2, size=len(time))

# plot signal
plt.figure(figsize = (10, 3))
plt.plot(time, xt, linewidth = 1, color = 'b')
plt.axvline(0.5, color='lightcoral',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(0.80, color='grey',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(1.75, color='green',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.ylabel(r'$x(t)$')
plt.xlim((0, time[-1]))
plt.ylim((-4, 4))
plt.xlabel('tempo [s]')
plt.tight_layout()

# plot p(x,t)
x = np.arange(-6, 6, 0.01)
px1 = norm.pdf(x, loc= mu_x, scale = mod_x[int(0.50*fs)]**2)
px2 = norm.pdf(x, loc= mu_x, scale = mod_x[int(0.80*fs)]**2)
px3 = norm.pdf(x, loc= mu_x, scale = mod_x[int(1.75*fs)]**2)

plt.figure(figsize = (10, 3))
plt.subplot(1,3,1)
plt.plot(px1, x, 'lightcoral', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,2)
plt.plot(px2, x, 'grey', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,3)
plt.plot(px3, x, 'green', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.tight_layout()

ipd.Audio(xt, rate=fs) # load a NumPy array
```

## Exemplo 3 - Estacionário com respeito a variância

```python
# Tempo
fs = 44100
time = np.arange(0,2, 1/fs)

# parâmetros de um fenômeno, cuja p(x) varia com o tempo
mu_x = np.sin(2*np.pi*0.4*time)    # média
sigma_x = 1.0   # desvio padrão

# sinal
xt = np.random.normal(loc = mu_x, scale = sigma_x, size=len(time))

# plot signal
plt.figure(figsize = (10, 3))
plt.plot(time, xt, linewidth = 1, color = 'b')
plt.axvline(0.5, color='lightcoral',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(0.80, color='grey',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(1.75, color='green',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.ylabel(r'$x(t)$')
plt.xlim((0, time[-1]))
plt.ylim((-4, 4))
plt.xlabel('tempo [s]')
plt.tight_layout()

# plot p(x,t)
x = np.arange(-6, 6, 0.01)
px1 = norm.pdf(x, loc= mu_x[int(0.50*fs)], scale = sigma_x)
px2 = norm.pdf(x, loc= mu_x[int(0.80*fs)], scale = sigma_x)
px3 = norm.pdf(x, loc= mu_x[int(1.75*fs)], scale = sigma_x)

plt.figure(figsize = (10, 3))
plt.subplot(1,3,1)
plt.plot(px1, x, 'lightcoral', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,2)
plt.plot(px2, x, 'grey', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,3)
plt.plot(px3, x, 'green', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.tight_layout()

```

## Exemplo 4 - Estacionário com respeito a média e a variância

```python
# Tempo
fs = 44100
time = np.arange(0,2, 1/fs)

# parâmetros de um fenômeno, cuja p(x) varia com o tempo
mu_x = 0.0    # média
sigma_x = 1.0   # desvio padrão

# sinal
xt = np.random.normal(loc = mu_x, scale = sigma_x, size=len(time))

# plot signal
plt.figure(figsize = (10, 3))
plt.plot(time, xt, linewidth = 1, color = 'b')
plt.axvline(0.5, color='lightcoral',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(0.80, color='grey',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.axvline(1.75, color='green',linestyle = '--', linewidth = 2, alpha = 0.8)
plt.ylabel(r'$x(t)$')
plt.xlim((0, time[-1]))
plt.ylim((-4, 4))
plt.xlabel('tempo [s]')
plt.tight_layout()

# plot p(x,t)
x = np.arange(-6, 6, 0.01)
px1 = norm.pdf(x, loc= mu_x, scale = sigma_x)
px2 = norm.pdf(x, loc= mu_x, scale = sigma_x)
px3 = norm.pdf(x, loc= mu_x, scale = sigma_x)

plt.figure(figsize = (10, 3))
plt.subplot(1,3,1)
plt.plot(px1, x, 'lightcoral', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,2)
plt.plot(px2, x, 'grey', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.subplot(1,3,3)
plt.plot(px3, x, 'green', linewidth = 2)
plt.xlabel(r'$p(x)$ [-]')
plt.ylabel(r'x [-]')
plt.tight_layout()

ipd.Audio(xt, rate=fs) # load a NumPy array
```

Aula 52 - estacionaridade/estacionaridade.ipynb


```python
from functools import partial
import jax
from jax import grad, jit, lax, random, vmap
import jax.numpy as jnp
import jax.scipy as scipy
import matplotlib.pyplot as plt
```

```python
plt.rcParams["figure.figsize"] = (16,10)
plt.rcParams["font.size"] = 24
```

```python
SEED = 11 * 2 * 2021
key = random.PRNGKey(SEED)
key, subkey = random.split(key)
```

    WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

```python
from jax.experimental.ode import odeint

@jit
def SIR_step(u, t, θ):
    # Susceptible, Infectious wild type, Infectious variant, Recovered wild type, Recovered variant
    S, I_wt, I_v, R_wt, R_v, Inc_wt, Inc_v =  (u[0], u[1], u[2], u[3], u[4], u[5], u[6])

    # Transmissibility, recovery rate
    # eta_T is added variant transmissibility in susceptibles,
    # eta_E is escape in recovered wt hosts
    # eta_gam is the ratio between recovery rates
    beta_wt, gam_wt, eta_T, eta_E, eta_gam  = (θ[0], θ[1], θ[2], θ[3], θ[4])

    lam_wt = beta_wt * S * I_wt # Wild-type transmission in naive
    lam_vs = (beta_wt * eta_T) * S * I_v # Variant transmission in naive
    lam_vr = (beta_wt * eta_T) * (eta_E * R_wt) * I_v # Variant transmission in wt recovered

    recov_wt = gam_wt * I_wt
    recov_v = gam_wt * eta_gam * I_v

    # Defining differential equations
    dS = -lam_wt - lam_vs
    dI_wt = lam_wt - recov_wt
    dI_v = lam_vs + lam_vr - recov_v
    dR_wt = recov_wt - lam_vr
    dR_v = recov_v

    # New compartments for capturing cummulative incidence
    dIncidence_wt = lam_wt
    dIncidence_v = lam_vs + lam_vr

    return jnp.stack([dS, dI_wt, dI_v, dR_wt, dR_v, dIncidence_wt, dIncidence_v])

# Defining model parameters
R0_wt = 1.4 # Basic reproduction number in wild type
R0_v = 1.4 # Basic reproduction number in variant

gen_wt = 3.8 # Average length of infection (days): 1 / gam_wt
gen_v = 3.8 # (...):  1 / gam_v
eta_E = 0.99 # Escape fraction

# Rest of quantities can be computed from the above
gam_wt =  1 / gen_wt
beta_wt = R0_wt * gam_wt

gam_v = 1 / gen_v
beta_v = R0_v * gam_v

eta_gam = gam_v / gam_wt
eta_T = beta_v / beta_wt
```

```python
#  Defining parameters and initial conditions
true_θ = (beta_wt, gam_wt, eta_T, eta_E, eta_gam)

# Simulate in two parts
## Initial dynamics without variant
u0 = jnp.array([0.999, 0.001, 0.0, 0.0, 0.0, 0.0, 0.0])
t0 = jnp.arange(70.)
Y0 = odeint(SIR_step, u0, t0, true_θ)

## Adding in second variant
u1 = Y0[-1,:] + jnp.array([0.0, 0.0, 1e-4, 0.0, 0.0, 0.0, 0.0])
t1 = jnp.arange(80.)
Y1 = odeint(SIR_step, u1 , t1, true_θ)

# Combining
t = jnp.hstack([t0[:-1], t0[-1] + t1])
Y = jnp.vstack([Y0[:-1,:], Y1])
```

```python
# Getting Rt for each variant
# We computed these analytically

# R0 * S
Rt_wt = R0_wt * Y[:,0]

# R0 * S + R0 * eta_E * R_wt
Rt_v = (R0_v * Y[:,0]) + (R0_v * eta_E * Y[:,3])
```

```python
fig = plt.figure(figsize=(12., 10.), constrained_layout=True)
spec = fig.add_gridspec(ncols=1, nrows=2)

ax0 = fig.add_subplot(spec[0, 0])
ax1 = fig.add_subplot(spec[1, 0])

# Plotting infectious individuals
ax0.plot(t, Y[:, 1], label = "Wildtype", color = "green")
ax0.plot(t, Y[:, 2], label = "Immune Escaping Variant", color = "purple")
ax0.set_ylabel("Percent infectious")

# Plotting effective reproduction numbers
ax1.plot(t, Rt_wt, label = "Rt_w", color = "green")
ax1.plot(t[t > t0[-1]], Rt_v[t > t0[-1]], label = "Rt_v", color = "purple")
ax1.hlines(y=1.0, xmin = t[0], xmax = t[-1], color = "grey", linestyle = "--")
ax1.set_ylabel("Effective Reproduction Number")
```

```python
# Simulating fake data based on cummulative incidence

def simulate_incidence(key, N_pop, Y):
    key1, key2, key3 = random.split(key, 3)

    EInc_wt = jnp.clip(jnp.diff(N_pop * Y[:, -2], prepend=0.), a_min = 0.0)
    EInc_v = jnp.clip(jnp.diff(N_pop * Y[:, -1], prepend=0.), a_min = 0.0)

    T =  len(EInc_wt)

    Inc_wt = random.poisson(key1, EInc_wt, shape= (T,))
    Inc_v = random.poisson(key2, EInc_v, shape= (T,))
    return key3, Inc_wt, Inc_v

N_pop = 20_000
key, Inc_wt, Inc_v = simulate_incidence(key, N_pop, Y)
```

```python
def simulate_sequencing(key, Nsamples, Y):
    key1, key2 = random.split(key, 2)

    frac_v = jnp.divide(Y[:, 1][:,None], (Y[:,1] + Y[:,2])[:,None])
    T = len(frac_v)
    samples = random.bernoulli(key, p=frac_v, shape= (T, Nsamples)).sum(axis=1)
    return key, samples

N_samples = 100
key, samples = simulate_sequencing(key, N_samples, Y)
```

```python
fig = plt.figure(figsize=(12., 8.5), constrained_layout=True)
spec = fig.add_gridspec(ncols=1, nrows=2)

ax0 = fig.add_subplot(spec[0, 0])
ax1 = fig.add_subplot(spec[1, 0])

# Plotting observed incidence
ax0.bar(t, Inc_wt, label = "Wildtype", color = "grey")
ax0.bar(t, Inc_v, label = "Immune Escaping Variant", color = "grey", bottom=Inc_wt)
ax0.set_ylabel("Observed incidence")

# Plotting effective reproduction numbers
ax1.bar(t, samples / N_samples, color = "green")
ax1.bar(t, 1-samples/ N_samples,  color = "purple", bottom=samples/ N_samples)
ax1.set_ylabel("Variant Frequency")
```

# A More General Model

Let's consider a model with $V$ variants which differ from the base strain in terms of $\eta_{T}$, $\Sigma_{E}$, $\eta_{G}$. $\eta_{T}$ and $\eta_{T}$ are vectors with first component equal to 1, and $\Sigma_{E}$ is an escape matrix with diagonal elements 0.

\begin{align}
\frac{dS}{dt} &= - \beta_{\text{wt}} S \sum_{v=1}^{V} \eta_{T}^{v} I_{v}\\
\frac{dI_{v}}{dt} &= \beta_{\text{wt}} \left( \eta_{T}^{v} S I_{v} + \sum_{u=1}^{V} \Sigma_{E}^{u,v} R_{u} \right) I_{v} - \gamma_{\text{wt}} \eta_{G}^{v} I_{v} \\
\frac{dR_{v}}{dt} &= \gamma_{\text{wt}} \eta_{G}^{v} I_{v} - \left(\beta_{\text{wt}}\sum_{u=1}^V \Sigma_{E}^{v,u} I_{u} \right) R_{v}
\end{align}

We could also write this with matrix products and transposes.

This means that the vector of $R_{0}$ is going to become

$$
R^{v}_{0} = \frac{\eta_{T}^{v} \beta_{\text{wt}} }{\eta_{G}^{v} \gamma_{\text{wt}}}.
$$

Further, we have that

$$
R^{v}_{t} = R^{v}_{0} \left( S + \sum_{v=1}^{V} \Sigma_{E}^{u, v} R_{u} \right)
$$

```python
## How do variants move from different compartments?
```

```python
# Can we design a mutational path here, could we then simplify this path and infer it from data?
```


# Toy compartmental model for COVID19 in Australia

Table of Contents

1  Context/Outline

2  Python set up

3  Assumptions - model/set-up

4  The model

4.1  The ODE model

4.2  Starting points

4.3  Use an ODE solver

4.4  Plot and describe

4.5  plot comparative

4.6  Run the model

## Context/Outline

This implements a simple SEIRV model, where the population is compartmentalised into five mutually exclusive compartments:

Model States:
*   S - susceptible population
*   E - exposed population
*   I - infectious unquarantined population
*   R - recovered/DEAD population
*   V - vaccinated population (that never had COVID)

Model factors:
*   $\alpha$ - 1/exposure period
*   $\beta$ - r0/infectious period
*   $\gamma$ - 1/infectious period
*   $\delta$ - proportion of the population vaccinated every day

Ordinary differential equations
$$\begin{align}
\frac{dS}{dt} & = -\beta SI - \delta \ (where: v < cap)\\
\frac{dE}{dt} & = \beta SI - \alpha E \\
\frac{dI}{dt} & = \alpha E - \gamma I \\
\frac{dR}{dt} & = \gamma I \\
\frac{dV}{dt} & = \delta \ (where: v < cap) \\
\end{align} $$

Where:
$$\begin{align}
\frac{d(S+E+I+R+V)}{dt} & = 0 \\
S+E+I+R+V & = 1 \\
\end{align}$$

## Python set up

```python
# imports
import numpy as np
import pandas as pd
import matplotlib
import matplotlib.pyplot as plt
from pathlib import Path
from io import StringIO
import scipy.integrate

#pandas
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999

# plotting
plt.style.use('ggplot')

# where to save the plot
CHART_DIRECTORY = '../charts'
Path(CHART_DIRECTORY).mkdir(parents=True, exist_ok=True)
CHART_DIRECTORY += '/!SIR-!ODE-'
```

## Assumptions - model/set-up

```python
N_DAYS = 200                # days - model period
STEP_SIZE = .01             # fraction of days
H = int(1 / STEP_SIZE)      # steps per day
POPULATION = 8_166_000      # NSW POPULATION
e_period = 2                # exposure period - days
                            # to the point of being infectious,
                            # so less than to the point of having
                            # symptoms
i_period = 12               # fully infectious period - days

# Other
case_fatality_rate = 0.016  # proportion of those infected
VACCINATION_MAX_OUT = 0.8   # proportion of the population that will get vaccinated
W = 0.4                     # proportion of vaccinated/recovered who get infected ???
```

## The model

### The ODE model

```python
def SEIR_model(y, t, alpha, beta, gamma, delta):
    s, e, i, r, v = y

    # an unusual vacination function
    vaxxedΔ = delta
    if v > VACCINATION_MAX_OUT:
        vaxxedΔ = 0

    ds_dt = (-beta * s * i) - vaxxedΔ
    de_dt = (beta * s * i) + (beta * v * i * W) - (alpha * e)
    di_dt = (alpha * e) - (gamma * i)
    dr_dt = (gamma * i) -
    dv_dt = vaxxedΔ - (beta * v * i * W)

    return ds_dt, de_dt, di_dt, dr_dt, dv_dt
```

### Starting points

```python
def get_parameters(r0):
    # y - starting points - ensure sum to one
    y = np.zeros(5) # S E I R V
    y[1] = 2_000 / POPULATION # e
    y[2] = 10_000 / POPULATION # i
    y[3] = 0.0 # r
    y[4] = 0.4 # v
    y[0] = 1.0 - y[1] - y[2] - y[3] - y[4] # s

    # t - times slices
    t = np.linspace(0, N_DAYS, N_DAYS*H)

    # parameters
    beta = r0 / i_period
    alpha = 1 / e_period
    gamma = 1 / i_period
    delta = 90_000 / POPULATION # proportion of the population vaxxed every day
    print(f'alpha: {alpha}, beta: {beta}, gamma: {gamma}, delta: {delta}')

    return y, t, alpha, beta, gamma, delta
```

### Use an ODE solver

```python
def solve(r0):

    y0, t, alpha, beta, gamma, delta = get_parameters(r0)

    solution = scipy.integrate.odeint(SEIR_model, y0,
        t, args=(alpha, beta, gamma, delta))

    # put the solution into a DataFrame
    solution = pd.DataFrame(solution, columns = ['susceptible',
            'exposed', 'infectious', 'removed', 'vaccinated'])
    solution.index = solution.index / H # return index to days

    # calculate deaths
    solution['dead'] = solution['removed'] * case_fatality_rate
    solution['recovered'] = solution['removed'] - solution['dead']
    del solution['removed']

    return solution
```

### Plot and describe

```python
def plot_solution(solution, r0):
    ax = solution.plot(lw=2)
    t = f'COVID19: toy SEIR ODE solution, ' '$R_{eff}=' f'{r0}$'
    ax.set_title(t)
    ax.set_xlabel('Day')
    ax.set_ylabel('Population Proportion')
    fig = ax.figure
    fig.set_size_inches(8, 4)
    fig.tight_layout(pad=1)
    fig.savefig(f'{CHART_DIRECTORY}-{t}.png', dpi=125)
    plt.show()
    plt.close()

    # key results
    print('Susceptible at end: '+
          f'{int(solution.susceptible.iloc[-1]*POPULATION):,} '+
          f'({np.round(solution.susceptible.iloc[-1]*100, 1)}%)')
    print('Resistant at end: '+
          f'{int(solution.recovered.iloc[-1]*POPULATION):,} '+
          f'({np.round(solution.recovered.iloc[-1]*100, 1)}%)')
    print('Dead at end: '+
          f'{int(solution.dead.iloc[-1]*POPULATION):,} '+
          f'({np.round(solution.dead.iloc[-1]*100, 1)}%)')
```

### plot comparative

```python
def plot_comparative(data, title):

    data = data * POPULATION
    ax = data.plot(lw=2)
    ax.set_title(title)
    ax.set_xlabel('Day')
    ax.set_ylabel('the New South Welsh')
    fig = ax.figure
    fig.set_size_inches(8, 4)
    fig.tight_layout(pad=1)
    fig.savefig(f'{CHART_DIRECTORY}-{title}.png', dpi=125)
    plt.show()
    plt.close()
```

### Run the model

```python
deaths = pd.DataFrame()
cases = pd.DataFrame()
for r_eff in [2.4, 2.2, 2.0, 1.8, 1.6, 1.4]:
    print ('---------------')
    solution = solve(r_eff)
    plot_solution(solution, r_eff)
    label = '$R_{eff}=' f'{r_eff}$'
    deaths[label] = solution['dead']
    cases[label] = (
        solution['exposed'] + solution['infectious'] +
        solution['recovered'] + solution['dead']
    )

print ('---------------')
plot_comparative(deaths, 'Total Deaths')
plot_comparative(deaths.diff(H), 'Daily New Deaths')
plot_comparative(cases, 'Total Cases (both reported and unreported)')
plot_comparative(cases.diff(H), 'Daily New Cases (both reported and unreported)')

```

The End


# Rabi model fitting

```python
import numpy as np
import matplotlib.pyplot as plt
import scipy.stats as stats
from scipy.optimize import curve_fit
%matplotlib inline
```

## The model

According to the Rabi model, the probability of being in the excited state, $p_e$ is given by,

\begin{equation}
    P_e = a_0 + a_1 \frac{\Omega^2}{W^2} \left[ 1 - \left( e^{-\frac{t}{T_1}} \cos \left( 2\pi Wt + \phi \right) \right) \right] + a_2 \left( 1 - e^{-\frac{t}{T_{decay}}} \right)
\end{equation}
where $W = \sqrt{\Omega^2 + \delta^2}$, $\Omega$ is the Rabi frequency, $\delta$ is the detuning, $T_1$ is the relaxational coherence time, and $t$ is time.

This gives a form which looks like the plot below.

```python
def rabiModel(time, rabiFreq, T1, Tdec, phi, a0, a1, a2, detuning=0.0):
    phi_deg = phi*(np.pi/180)
    W = np.sqrt(rabiFreq**2 + detuning**2)
    ampl = rabiFreq**2 / W**2
    osci = np.cos(2*np.pi*W*time + phi_deg)
    expo = np.exp(-time/T1)
    decay = np.exp(-time/Tdec)
    return a0 + a1*ampl*(1 - (expo * osci)) + a2 * (1 - decay)
```

## Plot example model

```python
time_start = 0.0
time_end = 7.0
time_steps = 1000
time_fit = np.linspace(time_start,time_end,time_steps)
rabiFreq = 3
T1 = 2
Tdec = 20
phi = -65
a0 = 0
a1 = 0.5
a2 = -1.0

p_e_fit = rabiModel(time_fit, rabiFreq, T1, Tdec, phi, a0, a1, a2, detuning=0)
plt.plot(time_fit, p_e_fit, 'g-', label='$P_e (\Delta = 0)$')

plt.plot(time_fit, np.exp(-time_fit/T1), 'r--', label='$e^{-t/T_1}$')
plt.xlabel("Time, ($\mu s$)")
plt.ylabel("Prob. excited state, $P_e$")
plt.title("Rabi model")
plt.legend()
plt.ylim([0.0, 1.0])
plt.grid()
```

## Plot data and initial guess

```python
d0,d1,d2,d3,d4,d5,d6,d7,d8,d9 = np.loadtxt('SR080317_026.dat',delimiter="\t",unpack=True)
time_exp = d1*1e6
p_e_exp = d4-min(d4)
p_e_exp = p_e_exp/max(p_e_exp)

cropNum = 170
time_exp = time_exp[1:len(time_exp)-cropNum]
p_e_exp = p_e_exp[1:len(p_e_exp)-cropNum]
```

```python
time_start = 0.0
time_end = 6.0
time_steps = 1000
time_fit = np.linspace(time_start,time_end,time_steps)
# Initial guess
rabiFreq = 3
T1 = 2
Tdec = 20
phi = -65
a0 = 0
a1 = 0.5
a2 = -1.0

p_e_fit = rabiModel(time_fit, rabiFreq, T1, Tdec, phi, a0, a1, a2, detuning=0)
plt.plot(time_fit, p_e_fit, 'g-', label='$P_e (\Delta = 0)$')

plt.plot(time_exp,p_e_exp, 'b-', label='data')
plt.xlabel("Time, ($\mu s$)")
plt.ylabel("Prob. excited state, $P_e$")
plt.title("Initial guess")
plt.legend()
plt.ylim([0.0, 1.0])
plt.grid()
```

## Curve fitting

```python
guess = [rabiFreq, T1, Tdec, phi, a0, a1, a2]
popt,pcov = curve_fit(rabiModel, time_exp, p_e_exp, p0=guess)

perr = np.sqrt(np.diag(pcov))
params = ['rabiFreq', 'T1', 'Tdec', 'phi', 'a0', 'a1', 'a2']

for idx in range(len(params)):
    print( "The fitted value of ", params[idx], " is ", popt[idx], " with error ", perr[idx] )

p_e_fit = rabiModel(time_fit,*popt)
plt.plot(time_fit, p_e_fit, 'g-', label='$P_e (\Delta = 0)$')
plt.plot(time_exp,p_e_exp, 'b-', label='data')
plt.xlabel("Time, ($\mu s$)")
plt.ylabel("Prob. excited state, $P_e$")
plt.title("Rabi model fit")
plt.legend()
plt.ylim([0.0, 1.0])
plt.grid()
```

```python

```


# Symbolically Understanding Christoffel Symbol and Riemann Curvature Tensor using EinsteinPy

```python
import sympy
from einsteinpy.symbolic import MetricTensor, ChristoffelSymbols, RiemannCurvatureTensor

sympy.init_printing()  # enables the best printing available in an environment
```

### Defining the metric tensor for 3d spherical coordinates

```python
syms = sympy.symbols('r theta phi')
# define the metric for 3d spherical coordinates
metric = [[0 for i in range(3)] for i in range(3)]
metric[0][0] = 1
metric[1][1] = syms[0]**2
metric[2][2] = (syms[0]**2)*(sympy.sin(syms[1])**2)
# creating metric object
m_obj = MetricTensor(metric, syms)
m_obj.tensor()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & r^{2} & 0\\0 & 0 & r^{2} \sin^{2}{\left(\theta \right)}\end{matrix}\right]$

### Calculating the christoffel symbols

```python
ch = ChristoffelSymbols.from_metric(m_obj)
ch.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & 0 & 0\\0 & - r & 0\\0 & 0 & - r \sin^{2}{\left(\theta \right)}\end{matrix}\right] & \left[\begin{matrix}0 & \frac{1}{r} & 0\\\frac{1}{r} & 0 & 0\\0 & 0 & - \sin{\left(\theta \right)} \cos{\left(\theta \right)}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & \frac{1}{r}\\0 & 0 & \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}}\\\frac{1}{r} & \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}} & 0\end{matrix}\right]\end{matrix}\right]$

```python
ch.tensor()[1,1,0]
```

### Calculating the Riemann Curvature tensor

```python
# Calculating Riemann Tensor from Christoffel Symbols
rm1 = RiemannCurvatureTensor.from_christoffels(ch)
rm1.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\end{matrix}\right]$

```python
# Calculating Riemann Tensor from Metric Tensor
rm2 = RiemannCurvatureTensor.from_metric(m_obj)
rm2.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\end{matrix}\right]$

### Calculating the christoffel symbols for Schwarzschild Spacetime Metric
  - The expressions are unsimplified

```python
syms = sympy.symbols("t r theta phi")
G, M, c, a = sympy.symbols("G M c a")
# using metric values of schwarschild space-time
# a is schwarzschild radius
list2d = [[0 for i in range(4)] for i in range(4)]
list2d[0][0] = 1 - (a / syms[1])
list2d[1][1] = -1 / ((1 - (a / syms[1])) * (c ** 2))
list2d[2][2] = -1 * (syms[1] ** 2) / (c ** 2)
list2d[3][3] = -1 * (syms[1] ** 2) * (sympy.sin(syms[2]) ** 2) / (c ** 2)
sch = MetricTensor(list2d, syms)
sch.tensor()
```

$\displaystyle \left[\begin{matrix}- \frac{a}{r} + 1 & 0 & 0 & 0\\0 & - \frac{1}{c^{2} \left(- \frac{a}{r} + 1\right)} & 0 & 0\\0 & 0 & - \frac{r^{2}}{c^{2}} & 0\\0 & 0 & 0 & - \frac{r^{2} \sin^{2}{\left(\theta \right)}}{c^{2}}\end{matrix}\right]$

```python
# single substitution
subs1 = sch.subs(a,0)
subs1.tensor()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & - \frac{1}{c^{2}} & 0 & 0\\0 & 0 & - \frac{r^{2}}{c^{2}} & 0\\0 & 0 & 0 & - \frac{r^{2} \sin^{2}{\left(\theta \right)}}{c^{2}}\end{matrix}\right]$

```python
# multiple substitution
subs2 = sch.subs([(a,0), (c,1)])
subs2.tensor()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & -1 & 0 & 0\\0 & 0 & - r^{2} & 0\\0 & 0 & 0 & - r^{2} \sin^{2}{\left(\theta \right)}\end{matrix}\right]$

```python
sch_ch = ChristoffelSymbols.from_metric(sch)
sch_ch.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & \frac{a}{2 r^{2} \left(- \frac{a}{r} + 1\right)} & 0 & 0\\\frac{a}{2 r^{2} \left(- \frac{a}{r} + 1\right)} & 0 & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}- \frac{a \left(\frac{a c^{2}}{2 r} - \frac{c^{2}}{2}\right)}{r^{2}} & 0 & 0 & 0\\0 & \frac{a \left(\frac{a c^{2}}{2 r} - \frac{c^{2}}{2}\right)}{c^{2} r^{2} \left(- \frac{a}{r} + 1\right)^{2}} & 0 & 0\\0 & 0 & \frac{2 r \left(\frac{a c^{2}}{2 r} - \frac{c^{2}}{2}\right)}{c^{2}} & 0\\0 & 0 & 0 & \frac{2 r \left(\frac{a c^{2}}{2 r} - \frac{c^{2}}{2}\right) \sin^{2}{\left(\theta \right)}}{c^{2}}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & \frac{1}{r} & 0\\0 & \frac{1}{r} & 0 & 0\\0 & 0 & 0 & - \sin{\left(\theta \right)} \cos{\left(\theta \right)}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & 0 & \frac{1}{r}\\0 & 0 & 0 & \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}}\\0 & \frac{1}{r} & \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}} & 0\end{matrix}\right]\end{matrix}\right]$

### Calculating the simplified expressions

```python
simplified = sch_ch.simplify()
simplified
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & \frac{a}{2 r \left(- a + r\right)} & 0 & 0\\\frac{a}{2 r \left(- a + r\right)} & 0 & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}\frac{a c^{2} \left(- a + r\right)}{2 r^{3}} & 0 & 0 & 0\\0 & \frac{a}{2 r \left(a - r\right)} & 0 & 0\\0 & 0 & a - r & 0\\0 & 0 & 0 & \left(a - r\right) \sin^{2}{\left(\theta \right)}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & \frac{1}{r} & 0\\0 & \frac{1}{r} & 0 & 0\\0 & 0 & 0 & - \frac{\sin{\left(2 \theta \right)}}{2}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & 0 & \frac{1}{r}\\0 & 0 & 0 & \frac{1}{\tan{\left(\theta \right)}}\\0 & \frac{1}{r} & \frac{1}{\tan{\left(\theta \right)}} & 0\end{matrix}\right]\end{matrix}\right]$

docs/source/examples/Symbolic Module - Christoffel Symbols and Riemann Curvature Tensor.ipynb


# LaTeX math equation

$\LaTeX$ math symbol
https://www.colorado.edu/physics/phys4610/phys4610_sp15/PHYS4610_sp15/Home_files/LaTeXSymbols.pdf
$$
a_2\\
a_{17}
$$
$$
a_2
\\
a_{17}
$$

---
$$
a^2\\
a^{17^{3}}
$$
$$
a^2\\
a^{17^{3}}
$$

---
$$
\bar x
\\
\hat x
\\
\overline{xyz}
$$
$$
\bar x
\\
\hat x
\\
\overline{xyz}
$$

---
$$
\alpha \beta \sigma \Sigma \pi
$$
$$
\alpha \beta \sigma \Sigma \pi
$$

---
$$
\sqrt{10}
\\
\sqrt[3]{64}
$$
$$
\sqrt{10}
\\
\sqrt[3]{64}
$$

---
$$
A\cdot B
\\
A\times B
\\
A\div B
$$
$$
A\cdot B
\\
A\times B
\\
A\div B
$$

---
$$
\begin{align}
x &= \sum_{i=1}^{12}(14-i)N_i \pmod{11}
\\
N_{13} &=
\begin{cases}
1-x, \mbox{if }x \le 1 \\
11-x, \mbox{if }x > 1
\end{cases}
\end{align}
$$
$$
\begin{align}
x&=\sum_{i=1}^{12}(14-i)N_i \pmod{11}
\\
N_{13}&=
\begin{cases}
1-x, \mbox{if }x \le 1 \\
11-x, \mbox{if }x > 1
\end{cases}
\end{align}
$$

```python

```


# Test results

# Question 1

Let us consider the two-bus system. The two buses are connected to each other through a line of impedance $R+jX$. We are working in sinusoidal steady-state and the voltage phasors of bus 1 and 2 are respectively equal to $\bar{V}_1 = V_1\angle \theta_1$ and $\bar{V}_2 = V_2\angle \theta_2$.

Q1.a An apparent power $S_{12} = P_{12} + jQ_{12}$ flows from bus 1 to bus 2, derive its expression. Only the variables $\bar{V}_1$, $\bar{V}_2$, $R$ and $X$ should appear in the final expression. Hint: using Ohm's Law, express the current $\bar{I}$ with respect to $\bar{V}_1$ and $\bar{V}_2$.

Answer :
\begin{align}
S_{12} &= \bar{V_1}\bar{I}_{12}^*\\
       &= \bar{V_1}\left(\frac{\bar{V_1}-\bar{V_2}}{jX+R}\right)^*
\end{align}
One can write :
$$Y = \frac{1}{jX+R} = y e^{j\eta}$$
with
$$y = \sqrt{\left(\frac{R}{R^2+X^2}\right)^2+\left(\frac{X}{R^2+X^2}\right)^2} \text{   and  } \eta = \tan^{-1}\left(\frac{-X}{R}\right)$$
thus we have :
\begin{align}
S_{12} &= \left(\bar{V_1}\bar{V_1}^*-\bar{V_1}\bar{V_2}^*\right)y e^{-j\eta}\\
       &= y\left(V_1^2 e^{j(-\eta)}-V_1 V_2 e^{j(\theta_1-\theta_2-\eta)}\right)
\end{align}

Q1.b Starting from the expression $S_{12} = y\left(V_1^2 e^{j(-\eta)}-V_1 V_2 e^{j(\theta_1-\theta_2-\eta)}\right)$ where $ y e^{j\eta} = \frac{1}{jX+R}$, derive the expressions of $P_{12}$ and $Q_{12}$.

Answer :
$$P_{12} = Re\{S_{12}\} = y V_1^2 \cos(\eta) - y V_1 V_2 \cos(\theta_1-\theta_2-\eta)$$
$$Q_{12} = Im\{S_{12}\} = -y V_1^2 \sin(\eta) - y V_1 V_2 \sin(\theta_1-\theta_2-\eta)$$

Q1.c Suppose that you have a line inductance per km $l = 10^{-2}$ H/km and a line resistance per km $r = 0.03\Omega$/km. Assume a line length $L=$10km. The system is operated at a frequency $f=50$Hz. Finally, suppose $\bar{V}_1 = 380\angle 0^{\circ{}}$ kV and $\bar{V}_2 = 375 \angle 5^{\circ{}}$ kV. Calculate the values of $P_{12}$ and $Q_{12}$ in MW and in Mvar, respectively.

```python
import numpy as np

L_per_km = 1e-2          #H/km
R_per_km = 3e-2          #Ohm/km
f = 50                   #Hz
V1 = 380e3               #V
theta1 = 0               #deg
V2 = 375e3               #V
theta2 = 5               #deg
line_length = 10         #km

X = L_per_km*f*2*np.pi*line_length
R = R_per_km*line_length

eta = np.arctan(-X/R)
y = np.sqrt((R/(R**2+X**2))**2+(X/(R**2+X**2))**2)

P12 = y*V1**2*np.cos(eta)-y*V1*V2*np.cos((theta1-theta2)*np.pi/180-eta)
Q12 = -y*V1**2*np.sin(eta)-y*V1*V2*np.sin((theta1-theta2)*np.pi/180-eta)
print(f'Active power P12 : {round(P12/1e6,3)} MW, Reactive power Q12 : {round(Q12/1e6,3)} Mvar')
```

    Active power P12 : -394.553 MW, Reactive power Q12 : 81.507 Mvar

Q1.d Most of the time, in transmission systems, the line resistance is neglected as the ratio $\frac{X}{R}>>1$. Now, assume a line resistance of 0$\Omega$ and calculate $P_{12}$ and $Q_{12}$.

```python
import numpy as np

L_per_km = 1e-2         #H/km
R_per_km = 0          #Ohm/km
f = 50                   #Hz
V1 = 380e3               #V
theta1 = 0               #deg
V2 = 375e3               #V
theta2 = 5               #deg
line_length = 10         #km

X = L_per_km*f*2*np.pi*line_length
R = R_per_km*line_length

eta = -np.pi/2
y = np.sqrt((R/(R**2+X**2))**2+(X/(R**2+X**2))**2)

P12 = y*V1**2*np.cos(eta)-y*V1*V2*np.cos((theta1-theta2)*np.pi/180-eta)
Q12 = -y*V1**2*np.sin(eta)-y*V1*V2*np.sin((theta1-theta2)*np.pi/180-eta)
print(f'Active power P12 : {round(P12/1e6,3)} MW, Reactive power Q12 : {round(Q12/1e6,3)} Mvar')
```

    Active power P12 : -395.331 MW, Reactive power Q12 : 77.739 Mvar

Q1.e As you may have seen, the impact of neglecting the line resistance stays small and the assumption can be considered as valid. Write the expressions of $P_{12}$, $Q_{12}$ as well as $P_{21}$,$Q_{21}$ (active and reactive power flowing from bus 2 to bus 1) if the line resistance is neglected. Only the variables $V_1$, $V_2$, $\theta_1$, $\theta_2$ and $X$ should appear in the final expressions.

Answer :
$$P_{12} = \frac{V_1V_2 \sin(\theta_1-\theta_2)}{X}$$
$$Q_{12} = \frac{V_1^2-V_1V_2\cos(\theta_1-\theta_2)}{X}$$
$$P_{21} = \frac{V_2V_1 \sin(\theta_2-\theta_1)}{X}$$
$$Q_{21} = \frac{V_2^2-V_2V_1\cos(\theta_2-\theta_1)}{X}$$

# Question 2

Q2. Consider the circuit shown below. The following parameters are given :
$\bar{V} = 120\angle 0$, $C = 200\mu F$, $L = 30 mH$, $R = 5\Omega$. The circuit is operated at a frequency $f=50Hz$.

Q2.1 Find the total apparent, active and reactive powers consumed by the circuit. Finally, compute the power factor and indicate if it is leading or lagging.

```python
import numpy as np
import cmath

#values given
V = cmath.rect(120,0) #V
C = 200e-6             #F
L = 30e-3              #H
R = 5                  #Ohms
f = 50                 #Hz

#find the reactances
omega = 2*np.pi*f      #rad/s
Xc = -1/(omega*C)      #Ohms
Xl = omega*L           #Ohms

#find the eqivalent impedance
Zeq = 1/(1/(1j*Xc)+1/(R+1j*Xl))
print(f'Zeq : {cmath.polar(Zeq)} Ohms')
#find the current flowing through the circuit
I = V/Zeq
print(f'Total current : {cmath.polar(I)} A')
#compute the apparent power
S = V*np.conj(I)
P = np.real(S)
Q = np.imag(S)

#calculate the power factor
cosPhi = P/(abs(S))
#or phi = angleV-angleI => cos(phi)

#print the results
print(f'Apparent power  : {S} VA, active power {P} W, reactive power {Q} var')
print(f'Power factor : {cosPhi}')

```

    Zeq : (20.724560478502138, 0.4266481389796722) Ohms
    Total current : (5.7902313597664765, -0.4266481389796722) A
    Apparent power  : (632.5419669350566+287.5348336123867j) VA, active power 632.5419669350566 W, reactive power 287.5348336123867 var
    Power factor : 0.9103579339539083

Q2.2 Assuming that we want a power factor equals 1, what would be the value of the capacitor knowing that the others parameters are kept the same?

```python
import numpy as np
import cmath

#values given
V = cmath.rect(120,0)  #V
cosPhi = 1             #
L = 30e-3              #H
R = 5                  #Ohms
f = 50                 #Hz

#find the reactances
omega = 2*np.pi*f      #rad/s
Xl = omega*L           #Ohms

#the apparent power is equal to the active power consumed by the circuit because cosPhi = 1
#thus, the power consumed by the resistor
#real part of the apparent power consumed by branch R+jXl = active power consumed by the circuit = apparent power consumed by the circuit
S = np.real(V*np.conj(V/(R+1j*Xl)))
print(f'S = {S} VA')
#find the total current
I = np.conj(abs(S)/V)
print(f'I = {cmath.polar(I)} A')
#find the equivalent impedance
Zeq = V/I
print(f'Zeq = {cmath.polar(Zeq)} Ohms')
#calculate the reactance Xc
#Zeq = 1/(1/(-jXc)+1/(R+jXl)) => Zeq/(R+jXl)-1 = Zeq/(-jXc) => j/Xc = 1/(R+jXl) - 1/Zeq => Xc = j/(-1/(R+jXl)+1/Zeq)
Xc = 1j/(-1/(R+1j*Xl)+1/Zeq)
print(f'Xc = {Xc} Ohms')
C = 1/(Xc*omega)
print(f'C = {C*1e6} uF')
```

    S = 632.5419669350566 VA
    I = (5.271183057792139, -0.0) A
    Zeq = (22.765287921960844, 0.0) Ohms
    Xc = (12.0773603456343+0j) Ohms
    C = (263.559152889607+0j) uF