# day23: Introduction to autograd for gradient descent

# Objectives

We will practice:

* Using the **autograd** Python package to compute gradients of functions
* Using gradients from autograd to do a basic linear regression

# Outline

* [Part 1: Autograd for scalar input, scalar output functions](#part1)
* [Part 2: Autograd for vector input, scalar output functions](#part2)
* [Part 3: Using autograd with simple gradient descent procedure](#part3)
* [Part 4: Using autograd to solve linear regression](#part4)
* [Part 5: Using autograd with functions that take dicts and other data structures](#part5)

# Takeaways

* Automatic differentiation is a powerful idea that has made experimenting with different models and loss functions far easier than it was even 8 years ago.
* The Python package `autograd` is a wonderfully simple tool that makes this work with numpy/scipy

* `autograd` works by a super-smartly implemented version of the backpropagation dynamic programming we've already discussed from Unit 3
* * Basically, after doing a "forward" pass to evaluate the function, we do a "reverse" pass through the computation graph and compute gradients via the chain rule.
* * This general purpose method is called [reverse-mode differentiation](https://github.com/HIPS/autograd/blob/master/docs/tutorial.md#reverse-mode-differentiation)

* `autograd` does NOT do symbolic math!
* * e.g. It does not simplify `ag_np.sqrt(ag_np.square(x))` as `x`. It will use the chain rule on all nested functions that the user specifies.
* `autograd` does NOT do numerical approximations to gradients.
* * e.g. It does not estimate gradients by perturbing inputs slightly

* In Part 5, we see how we can define losses in terms of dictionaries, which let us define complicated models with many different parameters. We'll exploit this in Project C for matrix factorization with many parameters

# Limitations

FYI There are some things that autograd *cannot* handle that you should be aware of.

Make sure any loss function you define that you want to differentiate does not do any of these things:

* Do not use assignment to elements of arrays, like `A[0] = x` or `A[1] = y`
* * Instead, compute entries individually and then stack them together.
* * Like this: `x = ...; y = ...; A = ag_np.hstack([x, y])`
* Do not rely on implicit casting of lists to arrays, like `A = ag_np.sum([x, y])`
* * use `A = ag_np.sum(ag_np.array([x, y]))` instead.
* Do not use A.dot(B) notation
* * Instead, use `ag_np.dot(A, B)`
* Avoid in-place operations (such as `a += b`)
* * Instead, use a = a + b

# Further Reading

Check out these great resources

* Official tutorial for the autograd package: https://github.com/HIPS/autograd/blob/master/docs/tutorial.md
* Short list of what autograd *can* and *cannot* do: https://github.com/HIPS/autograd/blob/master/docs/tutorial.md#supported-and-unsupported-parts-of-numpyscipy

```python
## Import numpy
import numpy as np
import pandas as pd
import copy
```

```python
## Import autograd
import autograd.numpy as ag_np
import autograd
```

```python
# Import plotting libraries
import matplotlib
import matplotlib.pyplot as plt

%matplotlib inline
plt.style.use('seaborn') # pretty matplotlib plots

import seaborn as sns
sns.set('notebook', font_scale=1.25, style='whitegrid')
```

# PART 1: Using autograd.grad for univariate functions

Suppose we have a mathematical function of interest $f(x)$.

For now, we'll assume this function has a scalar input and scalar output. This means:

* $x \in \mathbb{R}$
* $f(x) \in \mathbb{R}$

We can ask: what is the derivative (aka *gradient*) of this function:

$$
g(x) \triangleq \frac{\partial}{\partial x} f(x)
$$

Instead of computing this gradient by hand via calculus/algebra, we can use `autograd` to do it for us.

First, we need to implement the math function $f(x)$ as a **Python function** `f`.

The Python function `f` needs to satisfy the following requirements:
* INPUT 'x': scalar float
* OUTPUT 'f(x)': scalar float
* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy

### From numpy to autograd's wrapper of numpy

You might be used to importing numpy as `import numpy as np`, and then using this shorthand for `np.cos(0.0)` or `np.square(5.0)` etc.

For autograd to work, you need to instead use **autograd's** provided numpy wrapper interface:

`from autograd.numpy as ag_np`

The `ag_np` module has the same API as `numpy`. So for example, you can call

* `ag_np.cos(0.0)`
* `ag_np.square(5.0)`
* `ag_np.sum(a_N)`
* `ag_np.mean(a_N)`
* `ag_np.dot(u_NK, v_KM)`

Or almost any other function you usually would use with `np`

**Summary:** Make sure your function `f` produces a scalar and only uses functions within the `ag_np` wrapper

### Example: f(x) = x^2

$$
f(x) = x^2
$$

```python
def f(x):
    return ag_np.square(x)
```

```python
f(0.0)
```

    0.0

```python
f(1.0)
```

    1.0

```python
f(2.0)
```

    4.0

### Computing gradients with autograd

Given a Python function `f` that meets our requirements and evaluates $f(x)$, we want a Python function ``g` that computes the gradient $g(x) \triangleq \frac{\partial}{\partial x}$

We can use `autograd.grad` to create a Python function `g`

```
g = autograd.grad(f) # create function g that produces gradients of input function f
```

The symbol `g` is now a **Python function** that takes the same input as `f`, but produces the derivative at a given input.

```python
g = autograd.grad(f)
```

```python
# 'g' is just a function.
# You can call it as usual, by providing a possible scalar float input

g(0.0)
```

    0.0

```python
g(1.0)
```

    2.0

```python
g(2.0)
```

    4.0

```python
g(3.0)
```

    6.0

### Discussion 1a: Do you agree that the printed values above are correct? Why or why not?

```python
# TODO discuss
```

### Plot to demonstrate the gradient function  side-by-side with original function

```python
# Input values evenly spaced between -5 and 5
x_grid_G = np.linspace(-5, 5, 100)

fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)
subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-')
subplot_grid[0,0].set_title('f(x) = x^2')

subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-')
subplot_grid[0,1].set_title('gradient of f(x)');
```

### Exercise 1b:

Consider the decaying periodic function below. Can you compute its derivative using autograd and plot the result?

$$
f(x) = e^{-x/10} * cos(x)
$$

```python
def f(x):
    return ag_np.exp(-x/10) * ag_np.cos(x)

g = autograd.grad(f) # TODO define g as gradient of f, using autograd's `grad`

# TODO plot the result
x_grid_G = np.linspace(-10, 10, 500)
fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)
subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-');
subplot_grid[0,0].set_title('f(x) = x^2');

subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-');
subplot_grid[0,1].set_title('gradient of f(x)');
```

# PART 2: Using autograd.grad for functions with multivariate input

Now, imagine the input $x$ could be a vector of size D.

Our mathematical function $f(x)$ will map each input vector to a scalar.

We want the gradient function

\begin{align}
g(x) &\triangleq \nabla_x f(x)
\\
&= [
    \frac{\partial}{\partial x_1} f(x)
    \quad \frac{\partial}{\partial x_2} f(x)
    \quad \ldots \quad \frac{\partial}{\partial x_D} f(x)  ]
\end{align}

Instead of computing this gradient by hand via calculus/algebra, we can use autograd to do it for us.

First, we implement math function $f(x)$ as a **Python function** `f`.

The Python function `f` needs to satisfy the following requirements:
* INPUT 'x': numpy array of float
* OUTPUT 'f(x)': scalar float
* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy

### Worked Example 2a

Let's set up a function that is defined as the inner product of the input vector x with some weights $w$

We assume both $x$ and $w$ are $D$ dimensional vectors

$$
f(x) = \sum_{d=1}^D x_d w_d
$$

Define the fixed weights

```python
D = 2

w_D = np.asarray([1., 2.,])
```

Define the function `f` using `ag_np` wrapper functions only

```python
def f(x_D):
    return ag_np.dot(x_D, w_D) # dot product is just inner product in this case
```

Use `autograd.grad` to get the gradient function `g`

```python
g = autograd.grad(f)
```

Try putting in the all-zero vector

```python
x_D = np.zeros(D)

print("x_D", x_D)
print("f(x_D) = %.3f" % (f(x_D)))
```

    x_D [0. 0.]
    f(x_D) = 0.000

Compute the gradient wrt that all-zero vector

```python
g(x_D)
```

    array([1., 2.])

Try another input vector

```python
x_D = np.asarray([1., 2.])

print("x_D", x_D)
print("f(x_D) = %.3f" % (f(x_D)))
```

    x_D [1. 2.]
    f(x_D) = 5.000

Compute the gradient wrt the vector [1, 2, 3]

```python
g(x_D)
```

    array([1., 2.])

### Discussion 2b: Does this gradient computation agree with what you expect?

```python
# TODO discuss
```

### Exercise 2c:

Let's set up a function that is just the sum-of-squares penalty on the input vector x

$$
f(x) = \sum_{d=1}^D x_d^2
$$

Define the function `f` using `ag_np` wrapper functions only

```python
def f(x_D):
    return ag_np.sum(x_D * x_D) # TODO define sum-of-squares function f via calls to ag_np functions
```

Use `autograd.grad` to get the gradient function `g`

```python
g = autograd.grad(f)
```

Try out an input vector

```python
x_D = np.asarray([1., 2.])

print("x_D", x_D)
print("f(x_D) = %.3f" % (f(x_D)))
```

    x_D [1. 2.]
    f(x_D) = 5.000

Compute the gradient.

```python
g(x_D)
```

    array([2., 4.])

### Discussion 2d: Mathematically, should the gradient of sum-of-squares function be when x is all zero? Does your function `g` agree?

TODO try out feeding `np.zeros(D)` into your `g` function and discuss if you get what you expect

```python
g(np.zeros(D))
```

    array([0., 0.])

# Part 3: Using autograd gradients within gradient descent to solve multivariate optimization problems

### Helper function: basic gradient descent

Here's a very simple function that will perform many gradient descent steps to optimize a given function.

```python
def run_many_iters_of_gradient_descent(f, g, init_x_D=None, n_iters=100, step_size=0.001):
    ''' Run many iterations of GD

    Args
    ----
    f : python function (D,) to float
        Maps vector x_D to scalar loss
    g : python function, (D,) to (D,)
        Maps vector x_D to gradient g_D
    init_x_D : 1D array, shape (D,)
        Initial value for the input vector
    n_iters : int
        Number of gradient descent update steps to perform
    step_size : positive float
        Step size or learning rate for GD

    Returns
    -------
    x_D : 1D array, shape (D,)
        Best value of input vector for provided loss f found via this GD procedure
    history : dict
        Contains history of this GD run useful for plotting diagnostics
    '''
    # Copy the initial parameter vector
    x_D = copy.deepcopy(init_x_D)

    # Create data structs to track the per-iteration history of different quantities
    history = dict(
        iter=[],
        f=[],
        x_D=[],
        g_D=[])

    for iter_id in range(n_iters):
        if iter_id > 0:
            x_D = x_D - step_size * g(x_D)

        history['iter'].append(iter_id)
        history['f'].append(f(x_D))
        history['x_D'].append(x_D)
        history['g_D'].append(g(x_D))
    return x_D, history
```

### Worked Example 3a: Minimize f(x) = sum(square(x))

It's easy to figure out that the vector with smallest L2 norm (smallest sum of squares) is the all-zero vector.

Here's a quick example of showing that using gradient functions provided by autograd can help us solve the optimization problem:

$$
\min_x  \sum_{d=1}^D x_d^2
$$

```python
def f(x_D):
    return ag_np.sum(ag_np.square(x_D))

g = autograd.grad(f)

# Initialize at x_D = [6, 4, -3, -5]
D = 4
init_x_D = np.asarray([6.0, 4.0, -3.0, -5.0])
```

```python
opt_x_D, history = run_many_iters_of_gradient_descent(f, g, init_x_D, n_iters=1000, step_size=0.01)
```

```python
# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations
# Expected result: f goes to zero. all x values goto zero.

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
for d in range(D):
    subplot_grid[0,0].plot(history['iter'], np.vstack(history['x_D'])[:,d], label='x[%d]' % d);
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('x_d')
subplot_grid[0,0].legend(loc='upper right')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('f(x)');
```

### Exercise 3b: Minimize the 'trid' function

Given a 2-dimensional vector $x = [x_1, x_2]$, the trid function is:

$$
f(x) = (x_1-1)^2 + (x_2-1)^2 - x_1 x_2
$$

Background and Picture: <https://www.sfu.ca/~ssurjano/trid.html>

Can you use autograd + gradient descent to find the optimal value $x^*$ that minimizes $f(x)$?

You can initialize your gradient descent at [+1.0, -1.0]

```python
def f(x_D):
    return ag_np.power(x_D[0] - 1, 2) + ag_np.power(x_D[1] - 1, 2) - (x_D[0] * x_D[1]) # TODO

g = autograd.grad(f) # TODO
# Initialize at x_D = [6, 4, -3, -5]
D = 4
init_x_D = np.asarray([6.0, 4.0, -3.0, -5.0])
```

```python
# TODO call run_many_iters_of_gradient_descent() with appropriate args

opt_x_D, history = run_many_iters_of_gradient_descent(f, g, init_x_D, n_iters=1000, step_size=0.01)
```

```python
# TRID example
# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations
# Expected result: ????

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
for d in range(D):
    subplot_grid[0,0].plot(history['iter'], np.vstack(history['x_D'])[:,d], label='x[%d]' % d);
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('x_d')
subplot_grid[0,0].legend(loc='upper right')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('f(x)');
```

# Part 4: Solving linear regression with gradient descent + autograd

We observe $N$ examples $(x_n, y_n)$ consisting of D-dimensional 'input' vectors $x_n$ and scalar outputs $y_n$.

Consider the multivariate linear regression model for making a prediction given any input vector $x_i \in \mathbb{R}^D$:

\begin{align}
\hat{y}(x_i) = w^T x_i
\end{align}

One way to train weights would be to just compute the weights that minimize mean squared error

\begin{align}
\min_{w \in \mathbb{R}^D}  \sum_{n=1}^N (y_n - x_n^T w )^2
\end{align}

### Toy Data for linear regression task

We'll generate data that comes from an idealized linear regression model.

Each example has D=2 dimensions for x.

* The first dimension is weighted by +4.2.

* The second dimension is weighted by -4.2

```python
N = 100
D = 2
sigma = 0.1

true_w_D = np.asarray([4.2, -4.2])
true_bias = 0.1

train_prng = np.random.RandomState(0)
x_ND = train_prng.uniform(low=-5, high=5, size=(N,D))
y_N = np.dot(x_ND, true_w_D) + true_bias + sigma * train_prng.randn(N)
```

### Toy Data Visualization: Pairplots for all possible (x_d, y) combinations

You can clearly see the slopes of the lines:
* x1 vs y plot: slope is around +4
* x2 vs y plot: slope is around -4

```python
sns.pairplot(
    data=pd.DataFrame(np.hstack([x_ND, y_N[:,np.newaxis]]), columns=['x1', 'x2', 'y']));
```

```python
# Define the optimization problem as an AUTOGRAD-able function wrt the weights w_D
def calc_squared_error_loss(w_D):
    return ag_np.sum(ag_np.square(ag_np.dot(x_ND, w_D) - y_N))
```

```python
# Test the *loss function* at the known "ideal" initial point

calc_squared_error_loss(true_w_D)
```

    1.6882674607128603

```python
# Createa an all-zero weight array to use as our initial guess

init_w_D = np.zeros(2)
```

```python
# Test the *loss function* at that all-zero initial point

calc_squared_error_loss(init_w_D)
```

    30431.701153286307

```python
# Use autograd.grad to build the gradient function

calc_grad_wrt_w = autograd.grad(calc_squared_error_loss)
```

```python
# Test the gradient function at that same initial point

calc_grad_wrt_w(init_w_D)
```

    array([-7148.8368846 ,  7344.46400842])

### Discussion 4a: Is the gradient pointing in the right direction? Why or why not?

TODO discuss

### Run gradient descent

Use the code below to run GD on our simple regression problem

```python
# Because the gradient's magnitude is very large, use very small step size
opt_w_D, history = run_many_iters_of_gradient_descent(
    calc_loss, calc_grad_wrt_w, init_w_D,
    n_iters=400, step_size=0.00001,
    )
```

```python
# LinReg worked example
# Make plots of how w_D parameter values evolve over iterations, and function values evolve over iterations
# Expected result: x

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
for d in range(D):
    subplot_grid[0,0].plot(history['iter'], np.vstack(history['x_D'])[:,d], label='w[%d]' % d);
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('w_d')
subplot_grid[0,0].legend(loc='upper right')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('-1 * log p(y | w, x)');
```

### Discussion 4b: Do these trace plots indicate we have converged to good weight vector values? Why or why not?

TODO discuss

```python

```

# Part 5: Autograd for functions of data structures of arrays

#### Useful Fact: autograd can take derivatives with respect to DATA STRUCTURES of parameters

This can help us when it is natural to define models in terms of several parts (e.g. NN layers).

We don't need to turn our many model parameters into one giant weights-and-biases vector. We can express our thoughts more naturally.

### Demo 1: gradient of a LIST of parameters

```python
def f(w_list_of_arr):
    return ag_np.sum(ag_np.square(w_list_of_arr[0])) + ag_np.sum(ag_np.square(w_list_of_arr[1]))

g = autograd.grad(f)
```

```python
w_list_of_arr = [np.zeros(3), np.arange(5, dtype=np.float64)]

print("Type of the gradient is: ")
print(type(g(w_list_of_arr)))

print("Result of the gradient is: ")
g(w_list_of_arr)
```

### Demo 2: gradient of DICT of parameters

```python
def f(dict_of_arr):
    return ag_np.sum(ag_np.square(dict_of_arr['weights'])) + ag_np.sum(ag_np.square(dict_of_arr['bias']))
g = autograd.grad(f)
```

```python
dict_of_arr = dict(weights=np.arange(5, dtype=np.float64), bias=4.2)

print("Type of the gradient is: ")
print(type(g(dict_of_arr)))

print("Result of the gradient is: ")
g(dict_of_arr)
```

### Exercise 5a: Try to implement gradient descent for linear regression with weights and bias parameters

The above example only uses weights on the dimensions of each $x$ vector , and thus can only learn linear models that pass through the origin.

Can you instead optimize a model that includes a **bias** parameter $b>0$?

The predictions look like:

\begin{align}
\hat{y}(x_i) = w^T x_i + b
\end{align}

The training problem is:

\begin{align}
\min_{w \in \mathbb{R}^D,b \in \mathbb{R}} \sum_{n=1}^N (y_n - w^T x_n - b)^2
\end{align}

Use this format of parameter dictionary

```python
init_param_dict = dict(
    w_D=np.zeros(D),
    b=1.23)
```

Exercise 5a TODO: Define a function called `calc_loss` that computes our training loss given a parameter dictionary

```python
def calc_loss(param_dict):
    w_D = param_dict['w_D'] # Unpack weight array
    b = param_dict['b'] # Unpack bias scalar

    return 0.0 # TODO fix me

```

Exercise 5a TODO: build function `g` that can compute gradients

```python
g = None # TODO fix me
```

Given: use the implementation below of gradient descent on a parameter dictionary representation

```python
def run_many_iters_of_gradient_descent_for_param_dict(f, g, init_param_dict=None, n_iters=100, step_size=0.001):
    ''' Run many iterations of GD

    Args
    ----
    f : python function of dict to float
        Maps dict of arrays to scalar loss
    g : python function of dict to dict
        Maps dict of arrays to gradient dict of arrays
    init_param_dict : dict
        Initial values for the input parameters
    n_iters : int
        Number of gradient descent update steps to perform
    step_size : positive float
        Step size or learning rate for GD

    Returns
    -------
    opt_param_dict : dict
        Best value of parameter dict for provided loss f found via this GD procedure
    history : dict
        Contains history of this GD run useful for plotting diagnostics
    '''
    # Copy the initial parameter dict
    param_dict = copy.deepcopy(init_param_dict)

    # Create data structs to track the per-iteration history of different quantities
    history = dict(
        iter=[],
        f=[],
        param_dict=[],
        grad_dict=[])

    for iter_id in range(n_iters):
        if iter_id > 0:
            grad_dict = g(param_dict)
            for key in param_dict.keys():
                p_arr = param_dict[key] # current param array
                g_arr = grad_dict[key]  # current gradient array
                p_arr = p_arr - step_size * g_arr
                param_dict[key] = p_arr # store as latest value in dictionary

        history['iter'].append(iter_id)
        history['f'].append(f(param_dict))
        history['param_dict'].append(param_dict)
        history['grad_dict'].append(g(param_dict))
    return param_dict, history
```

TODO Run gradient descent to see what you get

```python

```


# Conservative SDOF - Harmonic Balance (3)

- Good for strongly and weakly NL systems
- Requires significant knowledge about system studied
- Similar to Galerkin method, but in time

## Overview

Assume an harmonic expansion for $x(t)$

$$
\begin{gather*}
     x(t) = \sum^N_{m = 0} A_m \cos\left( m\omega t + m \beta_0\right)
\end{gather*}
$$

substitute this into the EOM and enforce zero projected residual, i.e. taking the inner product with orthogonal assumption.

$$
\begin{gather*}
    \left< EOM_{LHS},\phantom{-} \cos\left(m\omega t + m\beta_0\right) \right> = 0\\
    m = 0, 1, 2, \cdots, N
\end{gather*}
$$

which is

$$
\begin{gather*}
    \int_0^{\frac{2\pi}{\omega} } \left( EOM_{LHS}\bullet \cos\left(m\omega t + m\beta_0\right) \right)dt
\end{gather*}
$$

Then we solve for $A_0, A_2, A_3, \cdots, A_n$ and $\omega$ in terms of $A_1$ and other system parameters. In this example we solve the following equation

$$
\begin{gather*}
    \ddot{x} + \omega_0^2 x + \alpha_2 x^2 + \alpha_3 x^3 = 0.
\end{gather*}
$$

## Assumed Harmonic Expansion - up to and including $\cos\left(2\left(\omega t + \beta \right)\right)$ and $\sin\left(2\left(\omega t + \beta \right)\right) $

### $\sin\phi$ term turns out having a zero coefficient

Assume a harmonic expansion for $x(t)$ - will use this to satisfy the cubic equation, so include harmonic terms at twice the linear natrual frequency

Note: phi = omega * t + beta0

```python
import sympy as sp
from sympy.simplify.fu import TR4, TR7, TR8, TR11
import numpy as np
from itertools import combinations_with_replacement
from math import factorial
```

```python
M = 2  # order of harmonic expansion cosine
K = 1  # order of harmonic expansion sine

Ai = sp.symbols('A_(0:' + str(M+K+1) + ')', real=True)  # coefficient A
t, w, w0 = sp.symbols('t omega omega_0', real=True, positive=True)
b0 = sp.Symbol('beta_0')

# harmonic expansion of x(t)
phi = sp.Symbol('phi')
xh = sp.Function('x_h')(phi)
xh = sum([Ai[i] * sp.cos(i * phi) for i in range(M+1)]) + Ai[3] * sp.sin(2 * phi)
xh
```

$\displaystyle A_{0} + A_{1} \cos{\left(\phi \right)} + A_{2} \cos{\left(2 \phi \right)} + A_{3} \sin{\left(2 \phi \right)}$

```python
# Defining the EOM
N = 3  # order of powered terms of alpha
ai = sp.symbols('alpha_(2:' + str(N+1) + ')')
x = sp.Function('x')(phi)

# EOM
EOM = w**2 * sp.diff(x, phi, 2) + w0**2 * x + sum([ai[i-2] * x**i for i in range(2, N+1)])
EOM = sp.Eq(EOM, 0)
EOM
```

$\displaystyle \alpha_{2} x^{2}{\left(\phi \right)} + \alpha_{3} x^{3}{\left(\phi \right)} + \omega^{2} \frac{d^{2}}{d \phi^{2}} x{\left(\phi \right)} + \omega_{0}^{2} x{\left(\phi \right)} = 0$

```python
# Substitute x(t) for the harmonic expansion of x(t) into EOM
err = EOM.lhs.subs(x, xh).doit().expand()
err = TR7(err).expand()
err = TR8(err).expand()
err
```

$\displaystyle A_{0}^{3} \alpha_{3} + 3 A_{0}^{2} A_{1} \alpha_{3} \cos{\left(\phi \right)} + 3 A_{0}^{2} A_{2} \alpha_{3} \cos{\left(2 \phi \right)} + 3 A_{0}^{2} A_{3} \alpha_{3} \sin{\left(2 \phi \right)} + A_{0}^{2} \alpha_{2} + \frac{3 A_{0} A_{1}^{2} \alpha_{3} \cos{\left(2 \phi \right)}}{2} + \frac{3 A_{0} A_{1}^{2} \alpha_{3}}{2} + 3 A_{0} A_{1} A_{2} \alpha_{3} \cos{\left(\phi \right)} + 3 A_{0} A_{1} A_{2} \alpha_{3} \cos{\left(3 \phi \right)} + 3 A_{0} A_{1} A_{3} \alpha_{3} \sin{\left(\phi \right)} + 3 A_{0} A_{1} A_{3} \alpha_{3} \sin{\left(3 \phi \right)} + 2 A_{0} A_{1} \alpha_{2} \cos{\left(\phi \right)} + \frac{3 A_{0} A_{2}^{2} \alpha_{3} \cos{\left(4 \phi \right)}}{2} + \frac{3 A_{0} A_{2}^{2} \alpha_{3}}{2} + 3 A_{0} A_{2} A_{3} \alpha_{3} \sin{\left(4 \phi \right)} + 2 A_{0} A_{2} \alpha_{2} \cos{\left(2 \phi \right)} - \frac{3 A_{0} A_{3}^{2} \alpha_{3} \cos{\left(4 \phi \right)}}{2} + \frac{3 A_{0} A_{3}^{2} \alpha_{3}}{2} + 2 A_{0} A_{3} \alpha_{2} \sin{\left(2 \phi \right)} + A_{0} \omega_{0}^{2} + \frac{3 A_{1}^{3} \alpha_{3} \cos{\left(\phi \right)}}{4} + \frac{A_{1}^{3} \alpha_{3} \cos{\left(3 \phi \right)}}{4} + \frac{3 A_{1}^{2} A_{2} \alpha_{3} \cos{\left(2 \phi \right)}}{2} + \frac{3 A_{1}^{2} A_{2} \alpha_{3} \cos{\left(4 \phi \right)}}{4} + \frac{3 A_{1}^{2} A_{2} \alpha_{3}}{4} + \frac{3 A_{1}^{2} A_{3} \alpha_{3} \sin{\left(2 \phi \right)}}{2} + \frac{3 A_{1}^{2} A_{3} \alpha_{3} \sin{\left(4 \phi \right)}}{4} + \frac{A_{1}^{2} \alpha_{2} \cos{\left(2 \phi \right)}}{2} + \frac{A_{1}^{2} \alpha_{2}}{2} + \frac{3 A_{1} A_{2}^{2} \alpha_{3} \cos{\left(\phi \right)}}{2} + \frac{3 A_{1} A_{2}^{2} \alpha_{3} \cos{\left(3 \phi \right)}}{4} + \frac{3 A_{1} A_{2}^{2} \alpha_{3} \cos{\left(5 \phi \right)}}{4} + \frac{3 A_{1} A_{2} A_{3} \alpha_{3} \sin{\left(3 \phi \right)}}{2} + \frac{3 A_{1} A_{2} A_{3} \alpha_{3} \sin{\left(5 \phi \right)}}{2} + A_{1} A_{2} \alpha_{2} \cos{\left(\phi \right)} + A_{1} A_{2} \alpha_{2} \cos{\left(3 \phi \right)} + \frac{3 A_{1} A_{3}^{2} \alpha_{3} \cos{\left(\phi \right)}}{2} - \frac{3 A_{1} A_{3}^{2} \alpha_{3} \cos{\left(3 \phi \right)}}{4} - \frac{3 A_{1} A_{3}^{2} \alpha_{3} \cos{\left(5 \phi \right)}}{4} + A_{1} A_{3} \alpha_{2} \sin{\left(\phi \right)} + A_{1} A_{3} \alpha_{2} \sin{\left(3 \phi \right)} - A_{1} \omega^{2} \cos{\left(\phi \right)} + A_{1} \omega_{0}^{2} \cos{\left(\phi \right)} + \frac{3 A_{2}^{3} \alpha_{3} \cos{\left(2 \phi \right)}}{4} + \frac{A_{2}^{3} \alpha_{3} \cos{\left(6 \phi \right)}}{4} + \frac{3 A_{2}^{2} A_{3} \alpha_{3} \sin{\left(2 \phi \right)}}{4} + \frac{3 A_{2}^{2} A_{3} \alpha_{3} \sin{\left(6 \phi \right)}}{4} + \frac{A_{2}^{2} \alpha_{2} \cos{\left(4 \phi \right)}}{2} + \frac{A_{2}^{2} \alpha_{2}}{2} + \frac{3 A_{2} A_{3}^{2} \alpha_{3} \cos{\left(2 \phi \right)}}{4} - \frac{3 A_{2} A_{3}^{2} \alpha_{3} \cos{\left(6 \phi \right)}}{4} + A_{2} A_{3} \alpha_{2} \sin{\left(4 \phi \right)} - 4 A_{2} \omega^{2} \cos{\left(2 \phi \right)} + A_{2} \omega_{0}^{2} \cos{\left(2 \phi \right)} + \frac{3 A_{3}^{3} \alpha_{3} \sin{\left(2 \phi \right)}}{4} - \frac{A_{3}^{3} \alpha_{3} \sin{\left(6 \phi \right)}}{4} - \frac{A_{3}^{2} \alpha_{2} \cos{\left(4 \phi \right)}}{2} + \frac{A_{3}^{2} \alpha_{2}}{2} - 4 A_{3} \omega^{2} \sin{\left(2 \phi \right)} + A_{3} \omega_{0}^{2} \sin{\left(2 \phi \right)}$

Note there are terms with frequency content as high as $6\phi$

To find $A_0$, $\omega$, $A_2$, we project the error back on the assumed expansion and set the projections to zero

Note: same as setting the coefficients in front of each $\cos(n\phi)$ to zero for $n = 0, 1, 2$

```python
proj_eqn = {}
for i in range(M+1):
    proj_eqn['eq'+str(i)] = sp.integrate(sp.expand_trig(err * sp.cos(i * phi)), (phi, 0, 2*sp.pi))

proj_eqn['eq'+str(3)] = sp.integrate(sp.expand_trig(err * sp.sin(2 * phi)), (phi, 0, 2*sp.pi))
```

```python
proj_eqn['eq0']
```

$\displaystyle 2 \pi A_{0}^{3} \alpha_{3} + 2 \pi A_{0}^{2} \alpha_{2} + 3 \pi A_{0} A_{1}^{2} \alpha_{3} + 3 \pi A_{0} A_{2}^{2} \alpha_{3} + 3 \pi A_{0} A_{3}^{2} \alpha_{3} + 2 \pi A_{0} \omega_{0}^{2} + \frac{3 \pi A_{1}^{2} A_{2} \alpha_{3}}{2} + \pi A_{1}^{2} \alpha_{2} + \pi A_{2}^{2} \alpha_{2} + \pi A_{3}^{2} \alpha_{2}$

```python
proj_eqn['eq1']
```

$\displaystyle 3 \pi A_{0}^{2} A_{1} \alpha_{3} + 3 \pi A_{0} A_{1} A_{2} \alpha_{3} + 2 \pi A_{0} A_{1} \alpha_{2} + \frac{3 \pi A_{1}^{3} \alpha_{3}}{4} + \frac{3 \pi A_{1} A_{2}^{2} \alpha_{3}}{2} + \pi A_{1} A_{2} \alpha_{2} + \frac{3 \pi A_{1} A_{3}^{2} \alpha_{3}}{2} - \pi A_{1} \omega^{2} + \pi A_{1} \omega_{0}^{2}$

```python
proj_eqn['eq2']
```

$\displaystyle 3 \pi A_{0}^{2} A_{2} \alpha_{3} + \frac{3 \pi A_{0} A_{1}^{2} \alpha_{3}}{2} + 2 \pi A_{0} A_{2} \alpha_{2} + \frac{3 \pi A_{1}^{2} A_{2} \alpha_{3}}{2} + \frac{\pi A_{1}^{2} \alpha_{2}}{2} + \frac{3 \pi A_{2}^{3} \alpha_{3}}{4} + \frac{3 \pi A_{2} A_{3}^{2} \alpha_{3}}{4} - 4 \pi A_{2} \omega^{2} + \pi A_{2} \omega_{0}^{2}$

```python
proj_eqn['eq3']
```

$\displaystyle 3 \pi A_{0}^{2} A_{3} \alpha_{3} + 2 \pi A_{0} A_{3} \alpha_{2} + \frac{3 \pi A_{1}^{2} A_{3} \alpha_{3}}{2} + \frac{3 \pi A_{2}^{2} A_{3} \alpha_{3}}{4} + \frac{3 \pi A_{3}^{3} \alpha_{3}}{4} - 4 \pi A_{3} \omega^{2} + \pi A_{3} \omega_{0}^{2}$

if $A_1$ is small neglect terms with $A_0^2, A_2^2, A_0A_2, A_1^2A_0, A_0^3, A_1^2A_2, \cdots$

This is because there is the assumption of
$$
\begin{gather*}
    A_1 > A_i \phantom{---} i = 0, 2, 3, \cdots
\end{gather*}
$$

equation 0 tells us that $A_0$ is proportional to $A_1$ squared - We have a constant term multiplying $A_0$ (in addition to others) and no $A_1$ terms but only $A_1^2$ terms.

```python
deglist = list(map(int, sp.degree_list(proj_eqn['eq0'], gens=Ai)))
A1_deg = deglist[1]
removing_combos = []
Ai_ = tuple(filter(lambda A: A != Ai[1], Ai))

# A1 ** n -> n can be up to O(A1**4) which is square of A1**2
keep_terms = []
for i in range(A1_deg+1, A1_deg**2+1):
    temp = []
    for j in range(i):
        temp.append(Ai[1])
    keep_terms.append(tuple(temp))

temp = list(combinations_with_replacement(Ai_, A1_deg))
for t in temp:
    if t not in keep_terms:
        removing_combos.append(t)

for i in range(A1_deg+1, max(deglist)+1):
    temp = list(combinations_with_replacement(Ai, i))
    for t in temp:
        if t not in keep_terms:
            removing_combos.append(t)

eq_copy = proj_eqn['eq0']
for term in removing_combos:
    eq_copy = eq_copy.subs(np.prod(term), 0)

eq_copy = sp.Eq(eq_copy, 0)

eq0_new = eq_copy
eq0_ordered = eq0_new
epsilon = sp.Symbol('epsilon')
for A in Ai_:
    eq0_ordered = eq0_ordered.subs(A, epsilon**A1_deg * A)
eq0_ordered = sp.collect(eq0_ordered.subs(Ai[1], epsilon * Ai[1]).lhs, epsilon)
eq0_ordered = sp.Eq(eq0_ordered, 0)

eq_copy = sp.Eq(Ai[0], sp.solve(eq_copy, Ai[0])[0])
eq_copy
```

$\displaystyle A_{0} = - \frac{A_{1}^{2} \alpha_{2}}{2 \omega_{0}^{2}}$

equation 2 tells $A_2$ is also proportional to $A_1$ squared

```python
deglist = list(map(int, sp.degree_list(proj_eqn['eq2'], gens=Ai)))
A1_deg = deglist[1]
Ai_ = tuple(filter(lambda A: A != Ai[1], Ai))

for i in range(A1_deg+1, max(deglist)+1):
    temp = list(combinations_with_replacement(Ai, i))
    for t in temp:
        if t not in removing_combos:
            if t not in keep_terms:
                removing_combos.append(t)

eq_copy = proj_eqn['eq2']
for term in removing_combos:
    eq_copy = eq_copy.subs(np.prod(term), 0)

eq_copy = sp.Eq(eq_copy, 0)

eq2_new = eq_copy
eq2_ordered = eq2_new
for A in Ai_:
    eq2_ordered = eq2_ordered.subs(A, epsilon**A1_deg * A)
eq2_ordered = sp.collect(eq2_ordered.subs(Ai[1], epsilon * Ai[1]).lhs, epsilon)
eq2_ordered = sp.Eq(eq2_ordered, 0)

eq_copy = sp.Eq(Ai[2], sp.solve(eq_copy, Ai[2])[0])
eq_copy
```

$\displaystyle A_{2} = \frac{A_{1}^{2} \alpha_{2}}{2 \left(4 \omega^{2} - \omega_{0}^{2}\right)}$

equation 3 tells us $A_3$ is also proportional to $A_1^2$

```python
deglist = list(map(int, sp.degree_list(proj_eqn['eq3'], gens=Ai)))
A1_deg = deglist[1]
Ai_ = tuple(filter(lambda A: A != Ai[1], Ai))

for i in range(A1_deg+1, max(deglist)+1):
    temp = list(combinations_with_replacement(Ai, i))
    for t in temp:
        if t not in removing_combos:
            if t not in keep_terms:
                removing_combos.append(t)

eq_copy = proj_eqn['eq3']
for term in removing_combos:
    eq_copy = eq_copy.subs(np.prod(term), 0)

eq_copy = sp.Eq(eq_copy, 0)

eq3_new = eq_copy
eq3_ordered = eq3_new
for A in Ai_:
    eq3_ordered = eq3_ordered.subs(A, epsilon**A1_deg * A)
eq3_ordered = sp.collect(eq3_ordered.subs(Ai[1], epsilon * Ai[1]).lhs, epsilon)
eq3_ordered = sp.Eq(eq3_ordered, 0)

eq_copy = sp.Eq(Ai[3], sp.solve(eq_copy, Ai[3])[0])
eq_copy
```

$\displaystyle A_{3} = 0$

```python
eq_copy = proj_eqn['eq1']
for term in removing_combos:
    eq_copy = eq_copy.subs(np.prod(term), 0)

eq_copy = sp.Eq(eq_copy, 0)
eq1_new = eq_copy
eq1_ordered = eq1_new
for A in Ai_:
    eq1_ordered = eq1_ordered.subs(A, epsilon**A1_deg * A)
eq1_ordered = sp.collect(eq1_ordered.subs(Ai[1], epsilon * Ai[1]).lhs, epsilon)
eq1_ordered = sp.Eq(eq1_ordered, 0)
```

To facilitate the solution, we order the coefficients using epsilon and then ignore all terms quadratic and higher in $A_0$ and $A_2$ - this means we keep terms up to tand including $O(\epsilon^3)$ -> small amplitude assumption

$A_1$ is ordered at $\epsilon$ and $A_0$ and $A_2$ are order at $\epsilon^2$

```python
# Eq0
eq0_ordered
```

$\displaystyle \epsilon^{2} \left(2 \pi A_{0} \omega_{0}^{2} + \pi A_{1}^{2} \alpha_{2}\right) = 0$

```python
# Eq1
eq1_ordered
```

$\displaystyle \epsilon^{3} \left(2 \pi A_{0} A_{1} \alpha_{2} + \frac{3 \pi A_{1}^{3} \alpha_{3}}{4} + \pi A_{1} A_{2} \alpha_{2}\right) + \epsilon \left(- \pi A_{1} \omega^{2} + \pi A_{1} \omega_{0}^{2}\right) = 0$

```python
# Eq2
eq2_ordered
```

$\displaystyle \epsilon^{2} \left(\frac{\pi A_{1}^{2} \alpha_{2}}{2} - 4 \pi A_{2} \omega^{2} + \pi A_{2} \omega_{0}^{2}\right) = 0$

```python
# Eq 3
eq3_ordered
```

$\displaystyle \epsilon^{2} \left(- 4 \pi A_{3} \omega^{2} + \pi A_{3} \omega_{0}^{2}\right) = 0$

```python
A0_sol = sp.solve(eq0_ordered, Ai[0])[0]
A0_sol
```

$\displaystyle - \frac{A_{1}^{2} \alpha_{2}}{2 \omega_{0}^{2}}$

```python
A2_sol = sp.solve(eq2_ordered, Ai[2])[0]
A2_sol
```

$\displaystyle \frac{A_{1}^{2} \alpha_{2}}{2 \left(4 \omega^{2} - \omega_{0}^{2}\right)}$

```python
A3_sol = sp.solve(eq3_ordered, Ai[3])[0]
A3_sol
```

$\displaystyle 0$

```python
eq1_ordered_update = eq1_ordered.subs([
    (Ai[0], A0_sol), (Ai[2], A2_sol), (Ai[3], A3_sol)
])
omegaSol = sp.solve(eq1_ordered_update, w)
```

```python
omegaSol[0]
```

$\displaystyle - \frac{\sqrt{2} \sqrt{- 4 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} + 3 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{2} + 5 \omega_{0}^{4} - \sqrt{16 A_{1}^{4} \alpha_{2}^{4} \epsilon^{4} - 24 A_{1}^{4} \alpha_{2}^{2} \alpha_{3} \epsilon^{4} \omega_{0}^{2} + 9 A_{1}^{4} \alpha_{3}^{2} \epsilon^{4} \omega_{0}^{4} - 16 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{6} + 9 \omega_{0}^{8}}}}{4 \omega_{0}}$

```python
omegaSol[1]
```

$\displaystyle \frac{\sqrt{2} \sqrt{- 4 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} + 3 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{2} + 5 \omega_{0}^{4} - \sqrt{16 A_{1}^{4} \alpha_{2}^{4} \epsilon^{4} - 24 A_{1}^{4} \alpha_{2}^{2} \alpha_{3} \epsilon^{4} \omega_{0}^{2} + 9 A_{1}^{4} \alpha_{3}^{2} \epsilon^{4} \omega_{0}^{4} - 16 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{6} + 9 \omega_{0}^{8}}}}{4 \omega_{0}}$

```python
omegaSol[2]
```

$\displaystyle - \frac{\sqrt{2} \sqrt{- 4 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} + 3 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{2} + 5 \omega_{0}^{4} + \sqrt{16 A_{1}^{4} \alpha_{2}^{4} \epsilon^{4} - 24 A_{1}^{4} \alpha_{2}^{2} \alpha_{3} \epsilon^{4} \omega_{0}^{2} + 9 A_{1}^{4} \alpha_{3}^{2} \epsilon^{4} \omega_{0}^{4} - 16 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{6} + 9 \omega_{0}^{8}}}}{4 \omega_{0}}$

```python
omegaSol[3]
```

$\displaystyle \frac{\sqrt{2} \sqrt{- 4 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} + 3 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{2} + 5 \omega_{0}^{4} + \sqrt{16 A_{1}^{4} \alpha_{2}^{4} \epsilon^{4} - 24 A_{1}^{4} \alpha_{2}^{2} \alpha_{3} \epsilon^{4} \omega_{0}^{2} + 9 A_{1}^{4} \alpha_{3}^{2} \epsilon^{4} \omega_{0}^{4} - 16 A_{1}^{2} \alpha_{2}^{2} \epsilon^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \epsilon^{2} \omega_{0}^{6} + 9 \omega_{0}^{8}}}}{4 \omega_{0}}$

We are keeping up to $O(\epsilon^4)$ only - use this to get our final expression for $\omega$

```python
# The built in sympy series command takes forever (bad algorithm) so we just implement our original Taylor series function
def taylor(expr, var, upto):
    res = expr.subs(var, 0)
    dexpr = sp.diff(expr, var)
    for n in range(1,upto+1):
        res += dexpr.subs(var, 0) / factorial(n) * var**n
        dexpr = sp.diff(dexpr, var)
    return res + sp.O(var**upto)
```

```python
omega1 = taylor(omegaSol[0], epsilon, 4)
omega1
```

$\displaystyle - \frac{\omega_{0}}{2} - \frac{\epsilon^{2} \left(- 4 A_{1}^{2} \alpha_{2}^{2} + 3 A_{1}^{2} \alpha_{3} \omega_{0}^{2} - \frac{- 16 A_{1}^{2} \alpha_{2}^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \omega_{0}^{6}}{6 \omega_{0}^{4}}\right)}{8 \omega_{0}^{3}} + O\left(\epsilon^{4}\right)$

```python
omega2 = taylor(omegaSol[1], epsilon, 4)
omega2
```

$\displaystyle \frac{\omega_{0}}{2} + \frac{\epsilon^{2} \left(- 4 A_{1}^{2} \alpha_{2}^{2} + 3 A_{1}^{2} \alpha_{3} \omega_{0}^{2} - \frac{- 16 A_{1}^{2} \alpha_{2}^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \omega_{0}^{6}}{6 \omega_{0}^{4}}\right)}{8 \omega_{0}^{3}} + O\left(\epsilon^{4}\right)$

```python
omega3 = taylor(omegaSol[2], epsilon, 4)
omega3
```

$\displaystyle - \omega_{0} - \frac{\epsilon^{2} \left(- 4 A_{1}^{2} \alpha_{2}^{2} + 3 A_{1}^{2} \alpha_{3} \omega_{0}^{2} + \frac{- 16 A_{1}^{2} \alpha_{2}^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \omega_{0}^{6}}{6 \omega_{0}^{4}}\right)}{16 \omega_{0}^{3}} + O\left(\epsilon^{4}\right)$

```python
omega4 = taylor(omegaSol[3], epsilon, 4)
omega4
```

$\displaystyle \omega_{0} + \frac{\epsilon^{2} \left(- 4 A_{1}^{2} \alpha_{2}^{2} + 3 A_{1}^{2} \alpha_{3} \omega_{0}^{2} + \frac{- 16 A_{1}^{2} \alpha_{2}^{2} \omega_{0}^{4} + 18 A_{1}^{2} \alpha_{3} \omega_{0}^{6}}{6 \omega_{0}^{4}}\right)}{16 \omega_{0}^{3}} + O\left(\epsilon^{4}\right)$

Note: we get a spurious solution - response at $0.5 \omega_0$ !


# Numeri

In questa pagina introdurremo diversi insiemi di numeri, facendosi aiutare da un sistema di calcolo simbolico:

```python
from sympy import *
```

```python
init_printing()
```

```python
x = symbols('x')
```

```python
x**2
```

## $\mathbb{N}$

L'insieme dei Naturali $\mathbb{N} = \lbrace 0, 1, 2, 3, 4, ..., \infty \rbrace$ e' stato definito *per risolvere il problema del conteggio* di un insieme arbitrario.

```python
eq = Eq(x + 3, 2, evaluate=False)
eq
```

Quando leggi la scrittura matematica $x + 3 = 2$, questa significa la seguente cosa: voglio trovare l'insieme $\mathcal{X}$ tale che $\mathcal{X} = \lbrace x \in \mathbb{N} : x + 3 = 2 \rbrace$. In questo caso, $\mathcal{X} = \emptyset$ perche' nessun naturale $x$ soddisfa l'equazione.

Allo stesso modo, per l'equazione

```python
Eq(x**2 + 3*x -1, 0, evaluate = False)
```

puoi leggere la scrittura matematica $x^2 + 3x -1 = 0$, questa significa la seguente cosa: voglio trovare l'insieme $\mathcal{X}$ tale che $\mathcal{X} = \lbrace x \in \mathbb{N} : x^2 + 3x -1 = 0 \rbrace$. Comunque, qui siamo gia avanti per capire la gerarchia dei numeri, solo per applicare il concetto precedente.

## $\mathbb{Z}$

Dato che $x = -1$ non ha soluzioni in $\mathbb{N}$, si definisce l'insieme $\mathbb{Z}$ dei numeri interi definito come $\mathbb{Z} = \lbrace -\infty, ..., -2, -1, 0, 1, 2, ..., \infty\rbrace$. Infatti, risolvendo l'equazione rispetto a $x$,

```python
solve(eq, [x], dict=True)
```

Quando leggi la scrittura matematica $x + 3 = 2$, questa significa la seguente cosa: voglio trovare l'insieme $\mathcal{X}$ tale che $\mathcal{X} = \lbrace x \in \mathbb{Z} : x + 3 = 2 \rbrace$. In questo caso, $\mathcal{X} = \lbrace -1 \rbrace$.

**Intermezzo sulla costruzione degli insiemi.**

Lo schema per definire un insieme $\mathcal{X}$ e' questo:

<center>$\mathcal{X} = \lbrace x\in\mathcal{Y}: p(x) \text{ e' vero } \rbrace$</center>

dove $\mathcal{Y}$ e' un insieme gia definito e $p$ e' un predicato, ovvero una espressione logica su $x$ che puo' essere o vera o falsa. Poi puoi scegliere un nome al posto di $\mathcal{X}$ a tua scelta. Importante: **quando uso questa scrittura, ottengo e definisco sempre un insieme**.

## $\mathbb{Q}$

Adesso supponiamo di voler risolvere

```python
eq = Eq(3*x, -2, evaluate=False)
eq
```

Ha significato $\mathcal{X} = \lbrace x \in \mathbb{N} : 3x = -2 \rbrace$ ? La risposta e' *si*, in questo caso $\mathcal{X}=\emptyset$.

Ha significato $\mathcal{X} = \lbrace x \in \mathbb{Z} : 3x = -2 \rbrace$ ? La risposta e' *si*, in questo caso $\mathcal{X}=\emptyset$.

Dato che $3x = -2$ non ha soluzioni ne in $\mathbb{N}$ ne in $\mathbb{Z}$, si definisce l'insieme $\mathbb{Q}$ dei numeri razionali definito come $\mathbb{Q} = \left\lbrace z \in\mathbb{Z} \wedge y \in\mathbb{Z}\setminus\lbrace0\rbrace: y\,x = z\right\rbrace$. Infatti, risolvendo l'equazione rispetto a $x$,

```python
solve(eq, [x], dict=True)
```

dove in questo caso sia $y=3$ sia $z=-2$ sono in $\mathbb{Z}$.

## $\mathbb{R}$

Adesso supponiamo di voler risolvere

```python
eq = Eq(x**2, 3, evaluate=False)
eq
```

Ha significato $\mathcal{X} = \lbrace x \in \mathbb{N} : x^2 = 3 \rbrace$ ? La risposta e' *si*, in questo caso $\mathcal{X}=\emptyset$.

Ha significato $\mathcal{X} = \lbrace x \in \mathbb{Z} : x^2 = 3 \rbrace$ ? La risposta e' *si*, in questo caso $\mathcal{X}=\emptyset$.

Ha significato $\mathcal{X} = \lbrace x, y \in \mathbb{Z} : \left({{x}\over{y}}\right)^2 = 3 \rbrace$ ?

```python
tentativi = list(map(S, range(-5, 5)))
tentativi
```

```python
equazioni = [Eq((x/y)**2, S(3), evaluate=False) for x in tentativi for y in tentativi if y]
equazioni
```

```python
list(map(solve, equazioni))
```

Ha significato $\mathcal{X} = \lbrace x, y \in \mathbb{Z} : \left({{x}\over{y}}\right)^2 = 3 \rbrace$ ? La risposta e' *si*, in questo caso $\mathcal{X}=\emptyset$.

Dato che $x^2 = 3$ non ha soluzioni ne in $\mathbb{N}$ ne in $\mathbb{Z}$ ne in $\mathbb{Q}$, si definisce l'insieme $\mathbb{R}$ dei numeri reali definito come $\mathbb{R} = \lbrace y,z \in\mathbb{Z}: x^y = z\rbrace$. Infatti, risolvendo l'equazione rispetto a $x$,

```python
solve(eq, [x], dict=True)
```

dove in questo caso sia $y=2$ sia $z=3$ sono in $\mathbb{Z}$.

```python
0.3 == 3/10
```

    True

```python
#0.47368754355678678678678678678678678...(678)*... is in Q
```

```python
pi
```

```python
list(filter(lambda r: r.is_real, solve(Eq(x**5, 10))))
```


## 1D linear operator with a zero parameter

We have shown that parameter estimation works well with positive parameters. In some complex problems, one might include extra terms in the predicted function, whose parameter equals to 0 in the end. To check if the framework works in such case, we construct the following example:

\begin{align*}
\mathcal{L}_x^\phi &:= \phi_1 \cdot + \phi_2\frac{d}{dx}\cdot + \phi_3\frac{d^2}{dx^2}\cdot \\
u(x) &= sin(x) \\
f(x) &= \mathcal{L}_x^\phi u(x)\\
&=\phi_1 sin(x) + \phi_2 cos(x) - \phi_3 sin(x) \\
&=(\phi_1- \phi_3)sin(x) + \phi_2 cos(x) \\
x &\in [0, 1] \\
\end{align*}

#### Simulate data
We assume $\phi_1 = 2$ and $\phi_3 = 5$, and $cos(x)$ is the extra term with $\phi_2 = 0$, then the function f is given by:

\begin{align*}
f(x) &= -3sin(x)
\end{align*}

```python
import numpy as np
import sympy as sp
from scipy.optimize import minimize
import matplotlib.pyplot as plt
```

```python

def get_simulated_data(n, phi1, phi2, phi3):
    x = np.random.rand(n)
    y_u = np.sin(x)
    y_f = phi1*np.sin(x) + phi2*np.cos(x) - phi3*np.sin(x)
    return(x, y_u, y_f)
```

```python
(x, y_u, y_f) = get_simulated_data(20, 2.0, 0.0, 5.0)
f, (ax1, ax2) = plt.subplots(ncols=2, nrows=1, sharey=True, figsize=(10,3))
f.suptitle("Input and Output for the operator")
ax1.plot(x, y_u, 'o')
ax1.set(xlabel= "x", ylabel= "u(x)")
ax2.plot(x, y_f, 'ro')
ax2.set(xlabel= "x", ylabel= "f(x)")
```

#### Evaluate kernels
Corresponding kernels are defined as follows:

$k _ { u u } \left( x _ { i } , x _ { j } ; \theta \right) = \theta \exp \left( - \frac { 1 } { 2 l } \left( x _ { i } - x _ { j } \right) ^ { 2 } \right)$

$\left. \begin{array} { l } { k _ { f f } \left( x _ { i } , x _ { j } ; \theta , \phi _ { 1 } , \phi _ { 2 } , \phi _ { 3 } \right) } \\ { = \mathcal { L } _ { x _ { i } } ^ { \phi } \mathcal { L } _ { x _ { j } } ^ { \phi } k _ { u u } \left( x _ { i } , x _ { j } ; \theta \right) } \\ { = \mathcal { L } _ { x _ { i } } ^ { \phi } \left( \phi _ { 1 } k _ { u u } + \phi _ { 2 } \frac { \partial } { \partial x _ { j } } k _ { u u } + \phi _ { 3 } \frac { \partial ^ { 2 } } { \partial x _ { j } ^ { 2 } } k _ { u u } \right) } \\ { = \left( \phi _ { 1 } k _ { u u } + \phi _ { 2 } \frac { \partial } { \partial x _ { i } } k _ { u u } + \phi _ { 3 } \frac { \partial ^ { 2 } } { \partial x _ { i } ^ { 2 } } k _ { u u } \right) \left( \phi _ { 1 } k _ { u u } + \phi _ { 2 } \frac { \partial } { \partial x _ { j } } k _ { u u } + \phi _ { 3 } \frac { \partial ^ { 2 } } { \partial x _ { j } ^ { 2 } } k _ { u u } \right) } \end{array} \right.$

$\left. \begin{array} { l } { k _ { f u } \left( x _ { i } , x _ { j } ; \theta , \phi _ { 1 } , \phi _ { 2 } , \phi _ { 3 } \right) } \\ { = \mathcal { L } _ { x _ { i } } ^ { \phi } k _ { u u } \left( x _ { i } , x _ { j } ; \theta \right) } \\ { = \phi _ { 1 } k _ { u u } + \phi _ { 2 } \frac { \partial } { \partial x _ { i } } k _ { u u } + \phi _ { 3 } \frac { \partial ^ { 2 } } { \partial x _ { i } ^ { 2 } } k _ { u u } } \end{array} \right.$

$\left. \begin{array} { l } { k _ { u f } \left( x _ { i } , x _ { j } ; \theta , \phi _ { 1 } , \phi _ { 2 } , \phi _ { 3 } \right) } \\ { = \mathcal { L } _ { x _ { i } } ^ { \phi } k _ { u u } \left( x _ { i } , x _ { j } ; \theta \right) } \end{array} \right.$

The kernels apply to the general 1D system with three parameters.

```python
x_i, x_j, theta, phi1, phi2, phi3 = sp.symbols('x_i x_j theta phi1 phi2 phi3')
kuu_sym = sp.exp(theta*(-1)/(2)*((x_i - x_j)**2))
kuu_fn = sp.lambdify((x_i, x_j, theta), kuu_sym, "numpy")
def kuu(x, theta):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kuu_fn(x[i], x[j], theta)
    return k
```

$k_{ff}(x_i,x_j;\theta,\phi_1, \phi_2,\phi_3) \\
= \mathcal{L}_{x_i}^\phi \mathcal{L}_{x_j}^\phi k_{uu}(x_i, x_j; \theta) \\
= \mathcal{L}_{x_i}^\phi \left( \phi_1 k_{uu} + \phi_2 \frac{\partial}{\partial x_j}k_{uu} +\phi_3 \frac{\partial^2}{\partial x_j^2}k_{uu} \right) \\
=( \phi_1 k_{uu} + \phi_2 \frac{\partial}{\partial x_i}k_{uu} +\phi_3 \frac{\partial^2}{\partial x_i^2}k_{uu})( \phi_1 k_{uu} + \phi_2 \frac{\partial}{\partial x_j}k_{uu} +\phi_3 \frac{\partial^2}{\partial x_j^2}k_{uu})  $

```python
kff_sym = phi1**2*kuu_sym + phi1*phi2*sp.diff(kuu_sym, x_j) + phi1*phi3*sp.diff(kuu_sym, x_j, x_j) \
+ phi1*phi2*sp.diff(kuu_sym, x_i) + phi2**2*sp.diff(kuu_sym, x_i, x_j) + phi2*phi3* sp.diff(kuu_sym, x_i,x_j, x_j)\
+ phi1*phi3*sp.diff(kuu_sym, x_i, x_i)+phi2*phi3* sp.diff(kuu_sym, x_i,x_i, x_j)+phi3*phi3* sp.diff(kuu_sym,x_i, x_i,x_j, x_j)
kff_fn = sp.lambdify((x_i, x_j, theta, phi1,phi2,phi3), kff_sym, "numpy")
def kff(x, theta, phi1, phi2,phi3):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kff_fn(x[i], x[j], theta, phi1, phi2, phi3)
    return k
```

$k_{fu}(x_i,x_j;\theta,\phi_1, \phi_2,\phi_3) \\
= \mathcal{L}_{x_i}^\phi k_{uu}(x_i, x_j; \theta) \\
= \phi_1 k_{uu} + \phi_2 \frac{\partial}{\partial x_i}k_{uu}+\phi_3 \frac{\partial^2}{\partial x_i^2}k_{uu}$

```python
kfu_sym = phi1*kuu_sym + phi2*sp.diff(kuu_sym, x_i)+ phi3*sp.diff(kuu_sym, x_i, x_i)
kfu_fn = sp.lambdify((x_i, x_j, theta, phi1, phi2,phi3), kfu_sym, "numpy")
def kfu(x1, x2, theta, phi1, phi2,phi3):
    k = np.zeros((x1.size, x2.size))
    for i in range(x1.size):
        for j in range(x2.size):
            k[i,j] = kfu_fn(x1[i], x2[j], theta, phi1, phi2,phi3)
    return k
```

$k_{uf}(x_i,x_j;\theta,\phi_1, \phi_2,\phi_3) \\
= \mathcal{L}_{x_j}^\phi k_{uu}(x_i, x_j; \theta) $

```python
def kuf(x1, x2, theta, phi1, phi2,phi3):
    return kfu(x1,x2,theta,phi1, phi2,phi3).T
```

#### step 3: define negative log marginal likelihood

$K = \begin{bmatrix}
k_{uu}(X_u, X_u; \theta) + \sigma_u^2I & k_{uf}(X_u, X_f; \theta, \phi) \\
k_{fu}(X_f, X_u; \theta, \phi) & k_{ff}(X_f, X_f; \theta, \phi) + \sigma_f^2I
\end{bmatrix}$

For simplicity, assume $\sigma_u = \sigma_f$.

$\mathcal{NLML} = \frac{1}{2} \left[ log|K| + y^TK^{-1}y + Nlog(2\pi) \right]$

where $y = \begin{bmatrix}
y_u \\
y_f
\end{bmatrix}$

```python
def nlml(params, x1, x2, y1, y2, s):
    params = np.exp(params)
    K = np.block([
        [kuu(x1, params[0]) + s*np.identity(x1.size), kuf(x1, x2, params[0], params[1], params[2], params[3])],
        [kfu(x1, x2, params[0], params[1], params[2], params[3]), kff(x2, params[0], params[1], params[2], params[3])\
         + s*np.identity(x2.size)]
    ])
    y = np.concatenate((y1, y2))
    val = 0.5*(np.log(abs(np.linalg.det(K))) + np.mat(y) * np.linalg.inv(K) * np.mat(y).T)
    return val.item(0)
```

```python
nlml((1,2,0,5), x, x, y_u, y_f, 1e-7)

```

    18709.10525217124

#### Optimize hyperparameters

```python
m = minimize(nlml, np.random.rand(4), args=(x, x, y_u, y_f, 1e-7), method="Nelder-Mead")
m
```

     final_simplex: (array([[ -1.47605862, -16.16826005,  -5.06061408,   1.10230253],
           [ -1.47605905, -16.16827103,  -5.06061592,   1.10230254],
           [ -1.47605916, -16.16832294,  -5.06062467,   1.1023025 ],
           [ -1.47605891, -16.16827003,  -5.06061574,   1.10230253],
           [ -1.47605875, -16.16832427,  -5.06062497,   1.10230249]]), array([-272.74267315, -272.74267314, -272.74267313, -272.7426731 ,
           -272.7426731 ]))
               fun: -272.7426731514816
           message: 'Optimization terminated successfully.'
              nfev: 460
               nit: 261
            status: 0
           success: True
                 x: array([ -1.47605862, -16.16826005,  -5.06061408,   1.10230253])

```python

phi1 = np.exp(m.x[1]) ## phi1

```

```python
phi2 = np.exp(m.x[2]) ## phi2
```

```python
phi3 = np.exp(m.x[3]) ## phi3
```

$f(x) = \phi_1sin(x) + \phi_2 cos(x)- \phi_3sin(x)  $

$f(x) = (\phi_1- \phi_3)sin(x) + \phi_2 cos(x)  $

where $\phi_1- \phi_3$ should be around 2.0, and $\phi_2$ around 5.0

```python
phi = [phi1 -phi3, phi2]
```

```python
phi # [phi1 - phi3, phi2]
```

    [-3.0110910877279604, 0.006341664007503534]

| Parameter | Value |
|-----------|-------|
| $\phi_1 - \phi_3$  |-3.0001|
| $\phi_2$  |0.18e-05 |

The parameter estimation is very close to our prediction. The linear operator can be applied to most 1D linear PDE problems, which is quite powerful. While dealing with some specific problems, we can add more terms in the form of the transformation, then the parameter estimation with Gaussian Processes determines which of these terms are redundant.


# *FEniCS tutorial:* Diffusion of a Gaussian hill in two dimensions

In this demo, we solve the two-dimensional diffusion equation with Dirichlet
bounadary conditions $u_D$ and source term $f = 0$. The initial condition,
$u_0$, we define as a Gaussian hill.

$$
\begin{align}
  u'   &= \nabla^2 u + f \quad\text{in the unit square} \\
  u    &= u_D  \hphantom{u+f}\quad\text{on the boundary} \\
  u    &= u_0  \hphantom{u+f}\quad\;\text{at $t = 0$} \\
\end{align}
$$
with
$$
\begin{align}
  u_D  &= 0 \\
  u_0  &= \exp\bigl(-a(x^2 + y^2)\bigr) \\
  f    &= 0
\end{align}
$$

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import time as tm

# import os
# import sys
# import re
# from IPython.display import Image
# from IPython.core.interactiveshell import InteractiveShell
# InteractiveShell.ast_node_interactivity = "all"
```

```python
from fenics import *
```

```python
def minmax(it):
    min = max = None
    for val in it:
        if min is None or val < min:
            min = val
        if max is None or val > max:
            max = val
    return min, max
```

```python
## set up the problem and define the main simulation function, evolve()

# simulation parameters
# -- time step
T = 0.25          # total simulation time
n_steps = 50      # number of time steps
dt = T / n_steps  # size of time step
nip = 5           # number of intervals between plots
# -- mesh density
nx = ny = 30      # mesh density along axes

# create mesh and define function space
mesh = RectangleMesh(Point(-2, -2), Point(2, 2), nx, ny)
V = FunctionSpace(mesh, 'P', 1)

# define boundary condition
def boundary(x, on_boundary):
    return on_boundary

bc = DirichletBC(V, Constant(0), boundary)

# define initial value
u_0 = Expression('exp(-a*pow(x[0], 2) - a*pow(x[1], 2))',
                 degree=2, a=10)
u_n = interpolate(u_0, V)

u_min = 0.
u_max = 1.

# define variational problem
u = TrialFunction(V)
v = TestFunction(V)

f = Constant(0)
F = u*v*dx + dt*dot(grad(u), grad(v))*dx - (u_n + dt*f)*v*dx
a, L = lhs(F), rhs(F)

# define time-evolution function
def evolve():

    # report initial state
    yield u_n

    # time-stepping
    u = Function(V)
    t = 0
    for n in range(1, n_steps + 1):

        # update current time
        t += dt

        # compute solution
        solve(a == L, u, bc)

        # report current state
        if n % nip == 0:
            yield u

        # update previous solution
        u_n.assign(u)
```

```python
n_rows = 3
n_cols = 5
fig_wd = 15
# default sizing here yields unit aspect ratio
plt.figure(figsize = (fig_wd, fig_wd * n_rows // n_cols))

idx = 0
for u in evolve():
    idx += 1
    plt.subplot(n_rows, n_cols, idx)
    plot(u, vmin=u_min, vmax=u_max)
```

```python
# ranges
xmin, xmax = minmax(mesh.coordinates()[:,0])
ymin, ymax = minmax(mesh.coordinates()[:,1])

# default sizing here yields unit aspect ratio
plt.figure(figsize = (8,5))

tol = 3.e-3  # avoid hitting points outside the domain
xv = np.linspace(xmin * (1 - tol), xmax * (1 - tol), 51)
pts = [(x_, 0) for x_ in xv]

idx = 0
for u in evolve():
    idx += 1
    ux = np.array([u(pt) for pt in pts])
    plt.plot(xv, ux, lw=1)
plt.plot(xv, ux, 'k', lw=1)
plt.show()
```

```python
mesh
```

```python

```


Computational Seismology

 SBP-SAT finite difference method for the 2D acoustic wave equation in velocity-pressure form

This notebook is based on the paper [Dynamic earthquake rupture simulations on nonplanar faults embedded in 3D geometrically complex, heterogeneous Earth models](https://pangea.stanford.edu/~edunham/publications/Duru_Dunham_FD3d_JCP16.pdf), and on the theory of summation-by-parts (SBP) finite difference methods and weak implementation of boundary conditions using the simultaneous-approximation-term (SAT).

##### Authors:
* Kenneth Duru

---

## Basic Equations ##

Consider the 2D acoustic wave equation in a heterogeneous fluid medium

\begin{align}
\frac{1}{\lambda(x,y)}\frac{\partial p(x, y,t)}{\partial t} -\frac{\partial u(x, y,t)}{\partial x} -\frac{\partial v(x, y,t)}{\partial y} & = \frac{1}{\lambda(x,y)}F(x,y,t),  \\
\rho(x,y)\frac{\partial u(x, y,t)}{\partial t} -\frac{\partial p(x,y,t)}{\partial x} & = 0,\\
\rho(x,y)\frac{\partial v(x,y,t)}{\partial t} -\frac{\partial p(x,y,t)}{\partial y} & = 0,
\end{align}

with $\rho(x,y)$ the density, $\lambda(x,y)$ the bulk modulus and $(x,y) \in [0, L_x]\times [0, L_y]$.  The acoustic wave propagates at the wavespeed $c = \sqrt{\lambda/\rho}$. At the boundaries $ x = 0, x = L_x$, $ y = 0, y = L_y$ we pose the general well-posed linear boundary conditions

\begin{equation}
\begin{split}
B_{0x}(u, p, Z, r_0): =\frac{Z}{2}\left({1-r_0}\right){u} -\frac{1+r_0}{2} {p} = 0,  \quad \text{at} \quad x = 0, \\
 B_{Lx}(u, p, Z, r_n): =\frac{Z}{2} \left({1-r_n}\right){u} + \frac{1+r_n}{2}{p} = 0, \quad \text{at} \quad  x = L_x\\
 B_{0y}(v, p, Z, r_0): =\frac{Z}{2}\left({1-r_0}\right){v} -\frac{1+r_0}{2} {p} = 0,  \quad \text{at} \quad y = 0, \\
 B_{Ly}(v, p, Z, r_n): =\frac{Z}{2} \left({1-r_n}\right){v} + \frac{1+r_n}{2}{p} = 0, \quad \text{at} \quad  y = L_y\\
 \end{split}
\end{equation}

with  the acoustic wave impedance $Z = \rho c$, the reflection coefficients $r_0$, $r_n$ being real numbers and  $|r_0|, |r_n| \le 1$.

Note that,  while  $r_j = -1$ yields soft wall, $r_j = 0$  yields  an absorbing boundary, and  with $r_j = 1$  we have a hard wall boundary condition.

In the coming analysis, we will set $F(x,y,t) =0$ to simplify the algebra. Introduce the mechanical energy defined by
\begin{equation}
E(t) = \int_0^{L_y}\int_0^{L_x}{\left(\frac{\rho(x, y)}{2} \left(u^2(x, y, t) + v^2(x, y, t)\right) + \frac{1}{2\lambda(x,y)}p^2(x, y, t)\right) dxdy},
\end{equation}

where $E(t)$ is the sum of the kinetic energy and the strain energy.
We have

\begin{equation}
\frac{d E(t)}{dt} = -\int_0^{L_y}\left(u(0, y, t)p(0, y, t) - u(L_x, y, t)p(L_x, y, t)\right)dy
                    -\int_0^{L_x}\left(v(x, 0, t)p(x, 0, t) - v(x, L_y, t)p(x, L_y, t)\right)dx \le 0.
\end{equation}

From the boundary conditions, it is easy to check that $u(0, y, t)p(0, y, t) \ge 0$,  $v(x, 0, t)p(x, 0, t) \ge 0$ and $u(L_x, y, t)p(L_x, y, t) \le 0$,  $v(x, L_y, t)p(x, L_y, t) \le 0$, for all $|r_0|, |r_n| \le 1$. This energy loss through the boundaries is what the numerical method should emulate.

1) Discretize the spatial domain:
\begin{align}
x_i = (i-1)\Delta{x}, \quad i = 1, 2, \dots, N_x, \quad \Delta{x} = \frac{L_x}{N_x-1}, \quad y_j = (j-1)\Delta{y}, \quad j = 1, 2, \dots, N_y, \quad \Delta{y} = \frac{L_y}{N_y-1}.
\end{align}

Denote a nodal approximation of the solution on the grid by $\mathbf{v}(t) = [v_{i,j}(t)]$, where $v_{i,j}(t) \approx v(x_i, y_j, t)$.

2) Introduce $\mathbf{D}$, a one space dimensional finite difference matrix satisfying the summation-by-parts property:

\begin{align}
\mathbf{D} = \mathbf{H}^{-1}\mathbf{Q}, \quad \mathbf{Q} + \mathbf{Q} = \left(\boldsymbol{e}_{N}\boldsymbol{e}_{N}^T -\boldsymbol{e}_{1}\boldsymbol{e}_{1}^T\right), \quad \mathbf{H}^T = \mathbf{H} > 0,
\end{align}

where, $\boldsymbol{e}_{0} = [1, 0, \dots,  0 ]^T, \quad  \boldsymbol{e}_{L} = [ 0, 0, \dots,  1 ]^T$ and $\mathbf{H}$ defines a dicrete norm. We consider only diagonal norm SBP operators with $H_{jj} = h_j > 0$, and define the quadrature rule

\begin{equation}
 \sum_{i = 1}^{N} f(x_j)h_j \approx \int_{0}^{L}f(x) dx, \quad h_j = \alpha_j \Delta{x},
\end{equation}
where $\alpha_j > 0$ are dimensionless quadrature weights.

The operator can be easily extended to multiple dimensions using Kronecker products:
\begin{equation}
\mathbf{D}_x = \left(\mathbf{D}\otimes \mathbf{I}\right), \quad \mathbf{D}_y = \left(\mathbf{I}\otimes \mathbf{D}\right),
\end{equation}
where $\mathbf{I}$ is the identity matrix.

The second order accurate SBP operator for first derivative is:
\begin{align}
\left(\mathbf{D}_x\mathbf{v}\right)_{i,j} = \frac{v_{i+1,j}-v_{i-1, j}}{2 \Delta{x}}, \quad i = 2, 3, \cdots N_x-1, \quad
\left(\mathbf{D}_x\mathbf{v}\right)_{1,j} = \frac{v_{2,j}-v_{1,j}}{\Delta{x}},\quad
\left(\mathbf{D}_x\mathbf{v}\right)_{N_xj} = \frac{v_{N_x,j}-v_{N_x-1,j}}{\Delta{x}}.
\end{align}

\begin{align}
\left(\mathbf{D}_y\mathbf{v}\right)_{i,j} = \frac{v_{i,j+1}-v_{i, j-1}}{2 \Delta{y}}, \quad j = 2, 3, \cdots N_y-1, \quad
\left(\mathbf{D}_y\mathbf{v}\right)_{i,j} = \frac{v_{i,2}-v_{i,1}}{\Delta{y}},\quad
\left(\mathbf{D}_y\mathbf{v}\right)_{i,N_y} = \frac{v_{i, N_y}-v_{i, N_y-1}}{\Delta{y}}.
\end{align}

Note that the interior stencils are centered, with second order accuracy and the boundary stencils are one-sided and first order accurate.

Higher order SBP operators can be found in the book: High Order Difference Methods for Time Dependent PDE, by B. Gustafsson. In this notebook we implement SBP operators with interior accuracy 2, 4 and 6. The implementation of the spatial derivative operators can be found in the file first_derivative_sbp_operators.py

To construct a stable  semi-discrete approximation we replace the spatial derivatives by the SBP operators, and add the boundary conditions as SAT-terms with special penalty weights having:

\begin{align}
\frac{1}{\lambda_{i,j}}\frac{d p_{i,j}(t)}{d t} - \left(\mathbf{D}_x \mathbf{u}\right)_{ij} - \left(\mathbf{D}_y \mathbf{v}\right)_{ij} & = \frac{1}{Z_{i,j}h_1^{(x)}}SAT^{(1x)}_{i,j} -  \frac{1}{Z_{i,j}h_{N_x}^{(x)}}SAT^{(Nx)}_{i,j} + \frac{1}{Z_{i,j}h_1^{(y)}}SAT^{(1y)}_{i,j} -  \frac{1}{Z_{i,j}h_{N_y}^{(y)}}SAT^{(Ny)}_{i,j} + F_{i,j}(t),  \\
\rho_{i,j} \frac{d u_{i,j}(t)}{d t} - \left(\mathbf{D}_x \mathbf{p}\right)_{ij} & = -\frac{1}{h_1^{(x)}}SAT^{(1x)}_{i,j} -  \frac{1}{h_{N_x}^{(x)}}SAT^{(Nx)}_{i,j},\\
\rho_{i,j} \frac{d v_{i,j}(t)}{d t} - \left(\mathbf{D}_y \mathbf{p}\right)_{ij} & =  -\frac{1}{h_1^{(y)}}SAT^{(1y)}_{i,j} -  \frac{1}{h_{N_y}^{(y)}}SAT^{(Ny)}_{i,j},
\end{align}

where

\begin{align}
h_i^{(x)} = \alpha_i \Delta{x}, \quad h_j^{(y)} = \alpha_j \Delta{y},
\end{align}

and

\begin{align}
 SAT^{(1x)}_{i,j} =  \left \{
\begin{array}{rl}
B_{0x}\left(u_{1,j}(t), p_{1,j}(t), Z_{1,j}, r_0\right), \quad i = 1\\
  0, \quad i \ne 1
\end{array} \right.
\quad
SAT^{(Nx)}_{i,j} =  \left \{
\begin{array}{rl}
B_{Lx}\left(u_{N_x,j}(t), p_{N_x,j}(t), Z_{N_x,j}, r_n\right), \quad i = N_x\\
  0, \quad i \ne N_x
\end{array} \right.
\end{align}

\begin{align}
 SAT^{(1y)}_{i,j} =  \left \{
\begin{array}{rl}
B_{0y}\left(v_{i,1}(t), p_{i,1}(t), Z_{i,1}, r_0\right), \quad j = 1\\
  0, \quad j \ne 1
\end{array} \right.
\quad
SAT^{(Ny)}_{i,j} =  \left \{
\begin{array}{rl}
B_{Ly}\left(v_{i, N_y}(t), p_{i, N_y}(t), Z_{i, N_y}, r_n\right), \quad j = N_y\\
  0, \quad j \ne N_y
\end{array} \right.
\end{align}

Numerical implementation of the spatial derivatives and the boundary conditions are realized in the file rate2D.py.

Approximate the mechanical energy by the above quadrature rule, having
\begin{align}
\mathcal{E}( t) = \sum_{i=1}^{N_x}\sum_{j=1}^{N_y}\left(\frac{1}{2}\rho_{i,j}\left(u_{i,j}^2(t) + v_{i,j}^2(t) \right)+ \frac{1}{2 \lambda_{i,j}}p_{i,j}^2(t)\right)h_i^{(x)}h_j^{(y)} > 0.
\end{align}

The semi-discrete approximation satisfies the energy equation:
\begin{align}
\frac{d \mathcal{E}( t)}{d t} = BT_{x}(t) + BT_{y}(t) \le 0,
\end{align}

where the boundary terms $BT_{\xi}$, $\xi = x, y$, are given by

\begin{align}
&BT_{x}(t) = -\frac{1}{2}\sum_{j =1}^{N_y}\left(\left(1-r_0\right)Z_{1,j}u_{1,j}^2(t) + \frac{\left(1+r_0\right)}{Z_{1,j}}p_{1,j}^2(t) +
\left(1-r_n\right)Z_{N_x,j}u_{N_x,j}^2(t) + \frac{\left(1+r_n\right)}{Z_{N_x,j}}p_{N_x,j}^2(t)\right)h_j^{(y)} \le 0, \\
&BT_{y}(t) = -\frac{1}{2}\sum_{i =1}^{N_x}\left(\left(1-r_0\right)Z_{i,1}v_{i,1}^2(t) + \frac{\left(1+r_0\right)}{Z_{i,1}}p_{i,1}^2(t) +
\left(1-r_n\right)Z_{i, N_y}v_{i, N_y}^2(t) + \frac{\left(1+r_n\right)}{Z_{i, N_y}}p_{i, N_y}^2(t)\right)h_i^{(x)} \le 0.
\end{align}

3) The discrete mechanical energy can never grow in time, and thus the semidiscrete numerical approximation is asymptotically stable.

4)  Time integration can be performed using any stable time stepping scheme. This notebook implements the fourth order accurate Runge-Kutta method.

Waves are generated by adding a 'point' source to the pressure equation with

\begin{equation}
F(x,y,t)  = \frac{1}{2\pi\sqrt{\delta_x\delta_y}}e^{ -\left(\frac{(x - x_{0})^2}{2\delta_x^2}+ \frac{(y - y_{0})^2}{2\delta_y^2}\right)}f(t),  \quad \delta_x = 2\Delta{x},  \quad \delta_y = 2\Delta{y}.
\end{equation}

```python
# Parameters initialization and plotting the simulation
# Import necessary routines
%matplotlib notebook
import numpy as np
import math
import matplotlib.pyplot as plt
import RK4_2D

#plt.switch_backend("TkAgg")          # plots in external window
plt.switch_backend("nbagg")           # plots within this notebook
```

In order to show a less time consuming solution the spatial and temporal resolution has been decreased. In the case you would like to increase it, you can modify the grid points in x and y (mantaining their proportions), the snapshot frequency and the model type.

```python
# Initialization parameters
Lx = 50.0     # length of the domain (x-axis)
Ly = 50.0     # width of the domain (y-axis)
Tend = 25.0   # final time [s]

Nx = 100       # grid points in x (orgiginally set to 200)
Ny = 100       # grid points in y (orgiginally set to 200)

dx = Lx/Nx    # spatial step in x
dy = Ly/Ny    # spatial step in y

isnap = 5     # snapshot frequency (orgiginally set to 10)
nf = 3        # number of fields

cp = 1.418    # compresional wave speed [m/s]
rho = 1.0     # density [g/cm^3]

# source parameters
x0 = 25.0        # [km]
y0 = 7.5         # [km]
t0 = 0.7         # [s]
T =  0.1149      # [s]
M0 = 1000        # [MPa]

source_type = 'Gaussian'

source_parameter = [x0, y0, t0, T, M0, source_type]

# extract bulk modulus
lam = rho*cp**2

dt = 0.5/cp*dx    # Time step
nt = int(Tend/dt)+1 # number of time steps

order = 6        # spatial order of accuracy

# Model type, available are "homogeneous", "fault_zone",
# "surface_low_velocity_zone", "random", "topography",
# "slab"

model_type = "homogeneous"

# Initialize velocity model
Mat = np.zeros((Nx, Ny, 2))

if model_type == "homogeneous":
    Mat[:,:,0] += rho
    Mat[:,:,1] += lam

elif model_type == "random":
    pert = 0.25
    r_rho = 2.0 * (np.random.rand(Nx, Ny) - 0.5) * pert
    r_lambda = 2.0 * (np.random.rand(Nx, Ny) - 0.5) * pert

    Mat[:,:,0] += rho*(1.0 + r_rho)
    Mat[:,:,1] += lam*(1.0 + r_lambda)

# Initialize arrays for fields
F = np.zeros((Nx, Ny, nf))                # p=F(:,:,1), u=F(:,:,2), v=F(:,:,3)

# Initialize arrays for the mesh
X = np.zeros((Nx, Ny))
Y = np.zeros((Nx, Ny))

# generate a Cartesian mesh
for i in range(0, Nx):
    for j in range(0, Ny):
        X[i, j] = i*dx
        Y[i,j] = j*dy
```

```python
# Simulation
# initialize plot
v = 5.0
p = np.sqrt(F[:,:,0]**2)

image = plt.imshow(p.transpose(), aspect='auto',extent=[0,Lx,Ly,0]
          ,cmap = 'seismic', vmin=1.5, vmax=+v, animated=True,
          interpolation='none')

# Receiver locations
rx = np.array([15, 20, 25, 30, 35])
ry = np.array([0, 0, 0, 0, 0])

irx = np.array([15, 20, 25, 30, 35])
iry = np.array([0, 0, 0, 0, 0])

for i in range(len(rx)):
    irx[i] = (np.ceil(rx[i]*dx))
    iry[i] = (np.ceil(ry[i]*dy))

seisvx = np.zeros((len(irx), nt))
seisvy = np.zeros((len(irx), nt))

# Plot the receivers
for x, z in zip(rx, ry):
    plt.text(x, z, '+')

plt.text(x0, y0, 'o')
plt.colorbar()
plt.xlabel('x [km]')
plt.ylabel('y [km]')

plt.show()

# Boundary reflection coefficients
r = np.array([0.,0.,1.,0.])

# required for seismograms
ir = np.arange(len(irx))

# Time-stepping
for it in range(nt):

    t = it*dt

    #4th order Runge-Kutta
    RK4_2D.acoustics_RK4_2D(F, F, Mat, X, Y, t, nf, Nx, Ny, dx, dy, dt, order, r, source_parameter)

    #extract absolute pressure fields for visualization
    p = np.sqrt(F[:,:,0]**2)

    # update time
    t = it*dt

    # Plot every isnap-th iteration
    if it % isnap == 0:                    # you can change the speed of the plot by increasing the plotting interval
        plt.title("time: %.2f" % t)
        image.set_data(p.transpose())
        plt.gcf().canvas.draw()
        #plt.show()
        #print(it)

    # Save seismograms
    seisvx[ir, it] = F[(irx[ir]), (iry[ir]), 1]
    seisvy[ir, it] = F[(irx[ir]), (iry[ir]), 2]

plt.ioff()
plt.show()

plt.figure(figsize=(10, 10))
plt.subplot(211)
ymax = seisvx.ravel().max()
time = np.arange(nt) * dt
for ir in range(len(seisvx)):
    plt.plot(time, seisvx[ir, :] + ymax * ir)
    plt.xlabel('time [s]')
    plt.ylabel('u [m/s]')

plt.subplot(212)
ymax = seisvy.ravel().max()
for ir in range(len(seisvy)):
    plt.plot(time, seisvy[ir, :] + ymax * ir)
    plt.xlabel('time [s]')
    plt.ylabel('v [m/s]')

plt.show()

```

    <IPython.core.display.Javascript object>

    <IPython.core.display.Javascript object>

```python

```


Unit 1.1

Basics linear algebra, probability, calculus

IST 718 – Big Data Analytics

Daniel E. Acuna

http://acuna.io

# Scalars

- Represented by Greek letters $\alpha, \beta, \gamma$
- Represent numbers
- $\alpha = 0.1, \beta = 1^{-10}$

# Notation and simple matrix algebra

- We let $\textbf X$ denote a $n \times p$ matrix whose $(i,j)$th element is $x_{ij}$. That is,

$$\textbf X = \begin{bmatrix} x_{11}&x_{12}&\cdots&x_{1p}\\ x_{21}&x_{22}&\cdots&x_{2p}\\
\vdots&\vdots&\ddots&\vdots\\ x_{n1}&x_{n2}&\cdots&x_{np}\\ \end{bmatrix}$$

- $\textbf X$ can be visualize as a spreadsheet of numbers with $n$ rows and $p$ columns.

- The rows of $\textbf X$ can be written as $x_1, x_2,...,x_n$. Here $x_i$ is a vector of lenght $p$, containing the $p$ variable measurements for the $i$th observation. That is,

$$x_i = \begin{pmatrix} x_{i1}\\ x_{i2} \\ \vdots\\ x_{ip}\\ \end{pmatrix}$$

- IMPORTANT: vectors are by default represented as columns.

- The columns of $\textbf X$ can written as $x_1, x_2,...,x_p$. Each is a vector of length $n$. That is,

$${\scriptsize \textbf X_j} = \begin{pmatrix} x_{1j}\\ x_{2j} \\ \vdots\\ x_{nj}\\ \end{pmatrix}$$

$\quad$ Using the previous notation, the matrix $\textbf X$ can be written as

$\qquad\qquad$
$\textbf X = \begin{pmatrix} {\scriptsize \textbf X_1}&{\scriptsize \textbf X_2}&\cdots&{\scriptsize \textbf X_p}\\ \end{pmatrix}$
$\qquad$or$\qquad$
$\textbf X = \begin{pmatrix} x_1^T\\ x_2^T \\ \vdots\\ x_n^T\\ \end{pmatrix}$

The $\;^T\;$ notation denotes the *transpose* of a matrix or vector. So, for example,

$$\textbf X^T = \begin{bmatrix} x_{11}&x_{21}&\cdots&x_{n1}\\ x_{12}&x_{22}&\cdots&x_{n2}\\
\vdots&\vdots&\ddots&\vdots\\ x_{1p}&x_{2p}&\cdots&x_{np}\\ \end{bmatrix}$$

while

$$x_i^T = \begin{pmatrix} x_{i1}&x_{i2}&\cdots&x_{ip}\\ \end{pmatrix}$$

- We use $y_i$ to denote the $i$th observation of the variable on wish we wish to make predictions. Hence we write the set of all $n$ observations in *vector form* as

$$\textbf y = \begin{pmatrix} y_1\\ y_2 \\ \vdots\\ y_n\\ \end{pmatrix}$$

- Then our observed data consist of $\{(x_1,y_1), (x_2,y_2),...,(x_n,y_n)\}$, where each $x_i$ is a vector of length $p$.
- If $p=1$, then $x_i$ is simply a scalar.

- In this course, a vector of length $n$ will always be denoted in *lower case bold;* e.g.

$$\textbf a = \begin{pmatrix} a_1\\ a_2 \\ \vdots\\ a_n\\ \end{pmatrix}$$

- However, vectors that are not of length $n$ (e.g., $x_i$) will be denoted in *lower case*. The same rule applies to scalars (e.g., $a$).

- Matrices will be denoted using *bold capitals*, such as **$\textbf X$**
- Random variables will be denoted using capitals, e.g. $A$

# Matrix
- Sometimes, we can define a matrix by its components as follows
$\textbf A=(f(i,j))_{ij}$ where $f(i, j)$ is a function of $i$ and $j$.
- **For example**, define the matrix
$$\textbf X =
\begin{bmatrix}
1 & 1 & \cdots & 1\\
0 & 1 & \cdots & 1\\
\vdots&\vdots&\ddots&\vdots\\ 0& 0 & \cdots & 1\\
\end{bmatrix}
$$
using a function

# Matrix operations
- Scalar times matrix: $\alpha \textbf A=(\alpha \times a_{ij} )_{ij}$
- Matrix addition: $\textbf A + \textbf B $ (add each element one at a time)
- Matrix multiplication: $\textbf A \textbf B$ ($\text{#cols}_A = \text{#rows}_B$)
$$\textbf A \textbf B=\left(\sum_{z} a_{iz} b_{zj}\right)_{ij}$$
- Matrix transposition: make rows the columns
$$\textbf{A}^T=(a_{ij} )_{ji}$$
- Many operations can be easily written as matrices

# Special matrices and properties

- Identity matrix (diagonal values are 1, everything else is 0)
- $I = \begin{bmatrix} 1&\cdots&0\\ \vdots&\ddots&\vdots\\ 0&\cdots&1\\ \end{bmatrix}$
- Matrix inverse: $AA^{−1}=I$
- Matrix addition is commutative: $A + B = B + A$
- Matrix multiplication is NOT commutative: $AB \neq BA$
- $(AB)^T=B^T A^T$
- Other matrix properties https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf

# Dimension

To indicate that an object is:
- a scalar, we will use the notation $a \in \mathbb{R}$
- a vector of length $n$, we will use $\textbf a \in \mathbb{R^n}$
- a vector of length $k$, we will use $a \in \mathbb{R^k}$
- a $r \times s$ matrix, we will use $\textbf A \in \mathbb{R}^{r \times s}$

# Interpreting graphs (1)
- Equation of the line: intercept, slope
  - Interpreting intercept and slope

- Application:
  -  Model 1: $\widehat{income}=f(age)= 20000+5000 \times age$
  - The unit of the intercept is different from the unit of the slope
  - Using matrix notation to make predictions for $age = \{20, 25, 40\}$

  - Represent model as a vector $b = \begin{pmatrix} 20000\\ 5000\\ \end{pmatrix}$
  - Represent data as matrix $X =?$
  - Making predictions: $X \times b$

# Interpreting graphs (2)

- Model 2:
$$\widehat{income}=f(age)= 20000+5000 \times age + 10000 \times education$$
- Represent model 2 as a matrix?

# Linear models (3)

- Model:
$$y=b_0+\sum{b_j x_j}$$
- Parameters of the model $b$
- Data: set of features $X$ and outputs or targets $y$
- One of the simplest models

# Learning as optimization
- Let’s assume a simple model where we are trying to predict income
$$\widehat{income} = f() = b_0$$
- This model does not take any features or inputs
- We would like to find the $b_0$ to predict well the following data $income = \{30000, 40000, 30000\}$

- We usually define a **loss function** and a common loss function is squared or quadratic error
$$ (\widehat{income} − income)^2 $$
- How do we find the right parameters for the model?

# Learning as optimization (2)
- Define the loss as a function of the model's parameters and we try to minize it
$$\widehat{\Theta} = \arg \min_\Theta L(\Theta)$$
- How would this loss function look like for the model $\widehat{income} = b_0$, data $income = \{30000, 40000, 30000\}$, and squared loss?
- How to optimize it?

# Optimization
- We can find a minimum or maximum of a function by looking at the slope
- Finding the minimum of a function:
$$\frac{df(x)}{dx}=0$$

- In multiple dimensions it is called a gradient:
$$g = {\begin{pmatrix} \frac{df(x_1)}{dx_1} \frac{df(x_2)}{dx_2} \cdots \frac{df(x_p)}{dx_p}\\ \end{pmatrix}}^T$$

# Derivatives
- Definition of the derivative:
$$\frac{df(x)}{d(x)} \approx \lim\limits_{\Delta x \to 0} {\frac {f(x+\Delta x)-f(x)}{\Delta x}}$$
- This means: the infinitesimal change in the function as the change is taken to zero
- Take as examples:
  - $𝑓_1(x) = a + xb$
  - $𝑓_2(x) = x^2$

# Optimization for model fitting
- Model: $\widehat{age}=b$
- Data: $ages = \{20, 25, 40\}$
- Function to minimize with quadratic errors?
- Optimal value for $b$?

# Other common derivation rules
- Chain rule:
$${\textstyle \frac{dg(f(x)}{dx}= \frac{dg(f)}{f}\frac{df(x)}{x}}$$
- Exercise, combine the following  rules:

  (1) $\frac{d(cf(x))}{dx}= c\frac{df(x)}{dx}$

  (2) $\frac{d(f(x)+g(x))}{dx}=\frac{d(f(x))}{dx}+\frac{g(g(x))}{dx}$

  (3) $\frac{d(x^n)}{dx}=nx^{n-1}$

  to solve ${\textstyle \frac{d(5x-\mu)^3}{d𝑥}}$

# Common properties
- ${\displaystyle \frac{d(e^x)}{dx}= e^x}$

- ${\displaystyle \frac{d(\log(x))}{dx}=\frac{1}{x}}$

- A common prediction function for probability values is the sigmoid:
$$\sigma(z)=\frac{1}{1+e^{−z}}$$

- Use the properties learned before to calculate
${\displaystyle \frac{d(\sigma(z))}{𝑧}}$

# A more complicated loss function
- Logistic regression has a loss function called *cross-entropy*:
$$l(z) = -y\log(\sigma(z)) - (1−y)\log(1−\sigma(z))$$
- Calculate
${\; \displaystyle \frac{dl(z)}{dz}}$

# Probability
- There are some phenomena which are not certain and therefore need a set of tools to still work with them
- Probability deals with the likelihood or chance that an event will occur, and can deal with these phenomena
- Several interpretation of what a likelihood is
  - Frequency: the relative frequency of how many times an event occurs if the same conditions are repeated many times. E.g., if I flip a coin 5M times, what is the relative frequency of heads?
  - Subjective definition: subjects have beliefs about the probability of an outcome which must be updated with certain consistent rules. E.g.: I may assume I will see heads 50% of the time, but I will update my belief if I see tails 10 times in a row

# Experiments and events
- An experiment is any process for which the outcome is unknown
- E.g.,:
  - Experiment to estimate, out of 10 coin tosses, the number of times heads will be obtained
  - If a spark job is running on 100 computers,  estimate the probability that the job will finish successfully if all computers must finish without error
  - If I estimate that the average age of my data is 30 years, how likely is it to see someone 60 years old in the future?

# Set theory
- The collection of all possible outcomes in an experiment is called  *sample space*
- For example:
  - The *sample space* of an experiment with a dice could be
$S = \{1, 2, 3, 4, 5, 6\}$
  - The event $A$ that an even number is obtained is defined by
 $A = \{2, 4, 6\}$
- Operations of set theory:
  - Union
  - Intersection
  - Complement

# Probability
- Axioms:
  1. Probability of any event is greater or equal to zero
  $p(A) \geq 0$
  2. If an event $S$ is certain to occur, then
  $p(S) = 1$
  3. The probability of an infinite number of independent events $A, B, C,\ldots$ is the sum of the probability of each event
  $p(A) + p(B) + p(C) + \ldots$

- Any function that follows Axioms 1, 2, and 3 is a *probability distribution*

# Some derived properties (1)
- For event $A$,
$p(\lnot{A}) = 1 - p(A)$, proof?

- For any two events $A$ and $B$,
$p(A \cup B) = p(A) + p(B) - p(A \cap B)$

- Conditional probability (probability of an event knowing that another event is certain)
$p(A \mid B) = P(A \cap B) / P(B)$

# Some exercises
1. A ball is selected from an urn with red, blue, and green balls. If the probability of red is $^1/_5$ and blue is $^2/_5$, what is the probability of getting a green ball?

1. You toss 2 die
   - What is the probability that sum of the die is 4?
   - If I pay you one dollar for a 1 or 6. How much money are you expected to receive?
1. A friend tells you that she has two children. You see one children and it is a girl.
   - What is the probability that the other child is **also** a girl?
   - What is the probability that the other child is a girl?

# Random variables and probability distributions
- A random variable is a real-valued function that is defined on a sample space of an experiment
- For example, a function defined over the number of heads after 5 tosses is a random variable. For sample $s = HHTTH$, the random variable would be $X(s) = 3$
- The distribution of a random variable $X$ is the probability of the events underlying the random variable

# Discrete and continuous random variables
- If the random variable $X$ can take on a finite number of $k$ different values $x_1, ... x_k$ or, an infinite sequence of them, $X$ is a *discrete random variable*
- Random variables that can take on every value on an interval are *continuous random variables*

# Discrete probability distribution
- Describes the probability of each real value $x$ of a discrete random variable
$$ p(X = x)$$
sometimes denoted simply as $p(x)$
- The set of points such that $\{ x\mid p(x) > 0 \}$ is denoted the *support* the probability distribution
- The sum of all events must sum up to 1: $\sum_x p(x) = 1$
- $p$ is also call a *probability mass function*

# Example of discrete probability distributions
- Bernoulli distribution (probability of tossing head)
$$p(X = H) = p(H) = \theta $$
and since the probability of all events must sum up to one $p(H) + p(T) = 1$ then $$p(T) = 1 - \theta$$
- This can be compactly represented as
$$p(x) = \theta^x (1-\theta)^{1-x}$$
if we consider heads as 1 and tails as 0.

# Example of discrete probability distributions (2)
- Uniform distribution between integers $a$ and $b$ would be
$$p(x) = \begin{cases}
\frac{1}{b - a + 1} & a \leq x \leq b\\
0 & \text{o.w.}
\end{cases}$$

# Continous probability distribution
- Defines probabilities for bounded closed intervals $[a,b]$
$$p(a \leq X \leq b) = \int_a^b p(x) dx$$

- $p(x) \geq 0$ for all $x$
- $\int_{-\infty}^{\infty}p(x) = 1$
- A single point in a continuous distribution has probability 0
- $p$ is called a *probability density function*

# Example of a continous distribution
- Uniform distribution on an interval
$$ p(x) = \begin{cases}
\frac{1}{b - a} & a \leq x \leq b\\
0 & \text{o.w.}
\end{cases}
$$

# Example of a continous distribution (2)
- Sometimes we define probabilities without worrying about whether they sum up to 1
$$ p(x) \propto \begin{cases}
4 x & 0 \leq x \leq 1\\
0 & \text{o.w.}
\end{cases}$$
- How to properly define the previous probability distribution? Hint: Use the fact that $\int p(x) = 1$

# Example of continous distribution (3)
- Gaussian distribution
$$p(x) = \frac{1}{\sqrt{2\pi} \sigma} \exp\left(-\frac{1}{2 \sigma^2}(x-\mu)^2\right)$$
- $\mu$ is called the mean and $\sigma$ is called the standard deviation.

# Common statistics
- Expectation: A *fancy average*
$$E[f(x)] = \sum_x p(x) f(x) \quad E[f(x)] = \int_x p(x) f(x) dx $$
- Variance: *Spread*
$$Var[f(x)] = E[ (f(x) - E[f(x)])^2 ]$$
- Covariance: *Co-spread*
$$Cov(f(x), g(y)) = E[(f(x) - E[f(x)])(g(y) - E[g(y)])]$$

# Be careful: transformations of probability distributions
$$\begin{align}
p(x) &\propto \begin{cases}
1 & -\frac{1}{2} \leq x \leq \frac{1}{2}\\
0 & \text{o.w.}
\end{cases}\\
y &= x^2
\end{align}$$
- what is $E[x]$?
- what is $E[y]$?

# Joint distributions
- The joint distribution of a set of random variables
$$p(X_1 \in C_1, X_2 \in C_2, \dots, X_k \in C_k)$$
can be read as the probability that the random variables are simultaneously in the intervals $C_1, \dots, C_k$

# Marginal probability

- From a simple example distribution
$$p(X_1 \in C_1, X_2 \in C_2)$$
we can obtain the following
$$p(X_1 \in C_1) = \sum_{x_2 \in C_2} p(X_1 = x_1, X_2 = x_2)$$
for a discrete distribution, and
$$p(X_1 \in C_1) = \int_{x_2 \in C_2} p(X_1 = x_1, X_2 = x_2) dx_2$$
for a continous distribution.
- This can be generalized for many variables

# Conditional probability

- If we did not have uncertainty about the value of random variable $X_2$, we write
$$p(X_1 \in C_1 \mid X_2 \in C_2) = \frac{p(X_1 \in C_1 , X_2 \in C_2)}{p(X_2 \in C_2)} $$

# Independence
- If two random events are independent (they don't depend on each other), their joint probability can be expressed as the factor of their distributions
$$p(X_1 \in C_1,  X_2 \in C_2) = p(X_1 \in C_1) p(X_2 \in C_2)$$

# Some exercises
- You toss 2 die
   - If I pay you one dollar for each and 1 or 6. What is expected value you are expected to receive?


# TP3 - Ejercicio 4

## Consigna

Se desea obtener el valor de la resistencia de canal de un transistor LDMOS [RD15VHF1](https://www.mitsubishielectric.com/semiconductors/content/product/highfrequency/siliconrf/discrete/rd15hvf1.pdf), para ello se  intercala entre el terminal positivo de la fuente y DRAIN un amperímetro. El terminal de SOURCE se conecta al negativo de la fuente y en paralelo con la fuente, un voltímetro para medir la tensión de salida. Se hicieron mediciones simultaneas de tensión y corriente, obteniéndose para VGS=4V la siguiente tabla de resultados:

| N | $V_{i}$ | $I_{i}$ |
|---|---------|---------|
| 1 | 10.11V  | 2.51A   |
| 2 | 10.21V  | 2.52A   |
| 3 | 10.10V  | 2.52A   |
| 4 | 10.00V  | 2.49A   |
| 5 | 10.19V  | 2.49A   |

A su vez se especifica la resolución y precisión de los instrumentos utilizados durante la medición,

> **Voltímetro DC:** Rango 20V, Resolución 10mV, Precisión $\pm$ (0.5% de lectura + 2 digitos)

> **Amperímetro DC:** Rango 10V, Resolución 10mA, Precisión $\pm$ (2.5% de lectura + 5 digitos)

### Resistencias internas

> **Voltímetro DC:** $R_{v} = 1M\Omega$

> **Amperímetro DC:** Se calucla la resistencia de shunt como el cociente entre el menor rango del voltímetro sobre el rango utilizado del amperímetro $R_{a} = \frac{200mV}{10A} = 20m\Omega$

### Circuito utilizado

# Resolución

## Interpretación del problema

En primera instancia el problema consiste en considerar las resistencias internas de los instrumentos de medición para luego poder calcular el error de método y considerar una posible corrección en la medida.

### Circuito con resistencias internas

Debemos considerar que estamos en el caso de corriente bien medida, por estar el amperímetro más próximo al circuito a medir. El diagrama con resistencias internas tomará la forma:

De acá podemos sacar las ecuaciones:

\begin{equation}
    \left\{
    V_{i} = V_{DS} + V_{A} \atop
    I_{i} = I_{D}
    \right.
\end{equation}

donde $V_{A}$ es la caida de tensión sobre la resistencia $R_{a}$

### Ecuaciones a utilizar

Debemos considerar que la medición de la resistencia interna del canal va a seguir la siguiente expresión,

\begin{equation}
    R_{i} = \dfrac{V_{i}}{I_{i}} = \dfrac{V_{DS} + V_{A}}{I_{D}} = \dfrac{V_{DS}}{I_{D}} + \dfrac{V_{A}}{I_{D}} = R_{MOS} + R_{a}
\end{equation}

Por lo que finalmente podemos considerar que nuestro valor de interes será el siguiente:

\begin{equation}
     R_{MOS} = R_{i} - R_{a}
\end{equation}

#### Ecuación del error de método

Nosotros sabemos que la expresión del error de método sigue la forma,

\begin{equation}
    e_{m} = \dfrac{X_{i} - X}{X}
\end{equation}

Por lo que si lo queremos aplicar a nuestro caso será:

\begin{equation}
    e_{m} = \dfrac{R_{i} - R_{MOS}}{R_{MOS}} = \dfrac{R_{i} - (R_{i} - R_{a})}{R_{i} - R_{a}}
\end{equation}

Finalmente,

\begin{equation}
    e_{m} = \dfrac{R_{a}}{R_{i} - R_{a}}
\end{equation}

# Análisis numérico

Como de costumbre, lo primero que debemos hacer es incluir los módulos más importantes.

```python
import numpy as np
from scipy import signal
from matplotlib import pyplot as plt
# Módulos para Jupyter (mejores graficos!)
import warnings
warnings.filterwarnings('ignore')
plt.rcParams['figure.figsize'] = [12, 4]
plt.rcParams['figure.dpi'] = 150 # 200 e.g. is really fine, but slower
```

Ahora comenzamos a calcular los valores correspondientes a las mediciones.

```python
# Cargamos la tabla de valores
Vi = np.array([10.11, 10.21, 10.10, 10.00, 10.19])
Ii = np.array([2.51, 2.52, 2.52, 2.49, 2.49])
N = np.size(Vi)

V_mean = np.mean(Vi)
I_mean = np.mean(Ii)

V_std = np.std(Vi)
I_std = np.std(Ii)

# Calculamos

print("V_mean: %0.2fV - STD: %0.2fV" % (V_mean,V_std))
print("I_mean: %0.2fA - STD: %0.2fA" % (I_mean,I_std))
```

    V_mean: 10.12V - STD: 0.07V
    I_mean: 2.51A - STD: 0.01A

Ahora que conocemos los valores medios de $V_{i}$ y $I_{i}$, pasamos a calcular la resistencia medida. Recordemos que la expresión de calculo es,

\begin{equation}
    R_{i} = \dfrac{V_{i}}{I_{i}}
\end{equation}

```python
# Calculamos la resistencia medida
Ri = V_mean / I_mean
print("Ri: %0.2fΩ" % (Ri))
```

    Ri: 4.04Ω

## Incertidumbre

Ahora que conocemos los valores comunes, debemos calcular la incertidumbre de $R_{i}$.

### Tipo A
Calculamos el desvío estándar experimental de la media como:
\begin{equation}
     u_{i}(x_{i}) = \sqrt{S^{2}(\bar{X_{i}})}
\end{equation}

```python
# Corriente
I_ui = I_std/np.sqrt(N)

# Tensión
V_ui = V_std/np.sqrt(N)

print("I_ui: %0.3fA" % (I_ui))
print("V_ui: %0.3fV" % (V_ui))
```

    I_ui: 0.006A
    V_ui: 0.033V

### Tipo B

Cargamos las constantes provenientes de los datos de los fabricantes del instrumento.

> **Voltímetro DC:** Rango 20V, Resolución 10mV, Precisión $\pm$ (0.5% de lectura + 2 digitos)

> **Amperímetro DC:** Rango 10V, Resolución 10mA, Precisión $\pm$ (2.5% de lectura + 5 digitos)

```python
## Constantes del instrumento
const_decimales = 2      # Cantidad de decimales que representa el instrumento

# Voltimetro
V_error_porcentual = 0.5 # Error porcentual del instrumento de medición
V_error_cuenta = 2       # Error en cuentas del instrumento de medición

# Amperimetro
I_error_porcentual = 2.5 # Error porcentual del instrumento de medición
I_error_cuenta = 5       # Error en cuentas del instrumento de medición
```

**Calculamos el error porcentual total del dispositivo de medición como:**
\begin{equation}
     e_{\%T} = e_{\%} + \dfrac{e_{cuenta}\cdot 100\%}{\bar{X_{i}}(10^{cte_{Decimales}})}
\end{equation}

```python
# Corriente
I_error_porcentual_total = I_error_porcentual + (I_error_cuenta*100)/(I_mean * (10**const_decimales ))

# Tensión
V_error_porcentual_total = V_error_porcentual + (V_error_cuenta*100)/(V_mean * (10**const_decimales ))
```

**El error absoluto se representa como:**
\begin{equation}
     \Delta X = e_{\%T} \dfrac{\bar{X_{i}}}{100\%}
\end{equation}

```python
# Corriente
delta_I = I_error_porcentual_total * I_mean/100

# Tension
delta_V = V_error_porcentual_total * V_mean/100
```

**Finalmente la incertidumbre tipo B queda:**
\begin{equation}
     u_{j}(x_{i}) = \sqrt{Var(x_{i})} = \dfrac{\Delta X}{\sqrt{3}}
\end{equation}

donde recordamos que suponemos una distribución uniforme en el dispositivo de medición.

```python
# Corriente
I_uj = delta_I / np.sqrt(3)

# Tensión
V_uj = delta_V / np.sqrt(3)

print("I_uj: %0.3fA" % (I_uj))
print("V_uj: %0.3fV" % (V_uj))
```

    I_uj: 0.065A
    V_uj: 0.041V

**Calculamos la incertidumbre conjunta para cada medición directa**

\begin{equation}
     u_{c}^{2}(x_{i}) = u_{i}^{2}(x_{i}) + u_{j}^{2}(x_{i})
\end{equation}

```python
# Corriente
I_uc = np.sqrt(I_ui**2 + I_uj**2)

# Tensión
V_uc = np.sqrt(V_ui**2 + V_uj**2)

print("I_uc: %0.3fA" % (I_uc))
print("V_uc: %0.3fV" % (V_uc))
```

    I_uc: 0.065A
    V_uc: 0.053V

**Calculamos la incertidumbre conjunta para la medición indirecta**

Tener en cuenta que es necesario hallar la correlación entre las medidas

\begin{equation}
     u_{c}^{2}(Y) = \sum_{i=1}^{N} (\dfrac{\partial f}{\partial x_{i}})^{2} \cdot u_{c}^{2}(x_{i}) + 2 \sum_{i=1}^{N-1} \sum_{j = i+1}^{N} \dfrac{\partial f}{\partial x_{i}} \dfrac{\partial f}{\partial x_{j}} r(x_{i},x_{j})u(x_{i})u(x_{j})
\end{equation}

Calculamos las derivadas parciales:

```python
dR_dV = 1 / I_mean
dR_dI = (-V_mean/(I_mean**2))
```

Calculamos el coeficiente de correlación

```python
VI_corr = np.corrcoef(Vi,Ii)[0,1]
print("VI_corr: %0.3f" % (VI_corr))
```

    VI_corr: 0.344

Ahora implementamos el calculo de correlación para la incertidumbre combinada

```python
Ri_uc = np.sqrt( (dR_dV**2*V_uc**2) + (dR_dI**2*I_uc**2) + (2* (dR_dV*dR_dI*VI_corr*V_uc*I_uc)))
print("Ri_uc: %0.3fΩ" % (Ri_uc))
```

    Ri_uc: 0.100Ω

**Expandimos la incertidumbre combinada**

Considerando que existen al menos 4 variables aleatorias involucradas en la expresión de la incertidumbre combinada de $R_{i}$, podemos considerar que la distribución de $u_{c}(R_{i})$ se asemeja a una gaussiana lo suficiente. Por tanto el $K_{95\%} = 2$.

\begin{equation}
     U_{c}(Y) = K_{95\%} \cdot u_{c}(Y)
\end{equation}

```python
Ri_Uc = 2 * Ri_uc
print("Ri_Uc: %0.3fΩ" % (Ri_Uc))
```

    Ri_Uc: 0.200Ω

## Error de método

Ahora utilizaremos la expresión del error de método para calcular su valor y compararlo con la incertidumbre combinada de $R_{i}$. Recordemos que la expresión es,

\begin{equation}
    e_{m} = \dfrac{R_{a}}{R_{i} - R_{a}}
\end{equation}

donde $R_{a} = \frac{200mV}{10A} = 20m\Omega$

```python
Ra = 20e-3 # Resistencia del amperímetro

# Error de método
error_metodo = Ra / (Ri - Ra)

print("e_m: %0.3f" % (error_metodo))
```

    e_m: 0.005

Ahora debemos comparar. Si la incertidumbre combinada es al menos 10 veces mayor al error de método, se puede despreciar el $e_{m}$. Sino es necesario aplicar una corrección a la medición del $R_{i}$.

```python
relacion = (Ri_Uc/Ri)/np.abs(error_metodo)
print("La incertidumbre combinada porcentual es %0.3f veces el error de método porcentual" % (relacion))
```

    La incertidumbre combinada porcentual es 9.953 veces el error de método porcentual

Dado que la incertidumbre combinada no es ni 10 veces mayor al error de método, debemos aplicar una corrección. Para ello modificaremos el valor de Ri mediante la siguiente expresión.

\begin{equation}
    e_{m} = \dfrac{\Delta R_{i}}{R_{i}} = \dfrac{R_{i} - \bar{R}}{R_{i}}
\end{equation}

por tanto,

\begin{equation}
    \bar{R} = \dfrac{R_{i}}{1 + e_{m}}
\end{equation}

### Aplicando la corrección

```python
R_final = Ri / (1 + error_metodo)

print("El valor medido Ri es: %0.3fΩ. El valor corregido es: %0.3fΩ." % (Ri,R_final))
```

    El valor medido Ri es: 4.039Ω. El valor corregido es: 4.019Ω.

# Expresando el resultado

```python
print("La resistencia del canal del NMOS es: (%0.3f +/- %0.3f)Ω @ 95%% de confianza." % (R_final,Ri_Uc))
```

    La resistencia del canal del NMOS es: (4.019 +/- 0.200)Ω @ 95% de confianza.

Error de Metodo/Ej4/.ipynb_checkpoints/Ejericio 4-checkpoint.ipynb


| |Pierre Proulx, ing, professeur|
|:---|:---|
|Département de génie chimique et de génie biotechnologique |** GCH200-Phénomènes d'échanges I **|

### Section 10.3, Conduction de la chaleur avec une source nucléaire

>Cet exemple est assez complexe, il sera très utile tant sur le plan de l'application des bilans thermiques que dans l'utilisation de ** SYMPY ** pour effectuer le travail de solution mathématiques et des manipulations algébriques. Les développements faits dans Transport Phenomena sont à étudier en parallèlle.
La figure suivante extraite de Transport Phenomena illustre le problème à traiter. Utilisez le livre pour suivre les développements en détail, vous constaterez facilement que les solutions effectuées avec ** SYMPY ** génèrent initialement des noms de constantes d'intégration différentes, on les renommera dans le ** NOTEBOOK ** pour garder la similarité entre les expressions utilisées dans le texte de référence et ce document, afin de rendre les comparaisons plus faciles.

Cet exemple peut servir de base à tout autre problème de conduction de chaleur en présence de génération de chaleur, que ce soit par réaction chimique ou biochimique, source électrique ou magnétique.

Voyons dans ce qui suit un résumé de l'analyse qui est faite dans Transport Phenomena. Vous verrez dans ce qui suivra qu'en utilisant le solver ** SOLVE ** de ** SYMPY ** , on posera les 4 conditions aux limites simultanément, pas une à la fois comme dans le livre. On profite de la force du calcul symbolique de ** SYMPY **

http://pierreproulx.espaceweb.usherbrooke.ca/images/GCH200_Ch10_3_resume.pdf

>Maintenant regardons comment on procède en utilisant le calcul formel de ** SYMPY ** pour effectuer les manipulations, intégrales, solutions d'équations différentielles et recherche des constantes provenant des conditions aux limites. On terminera en tracant un profil de température dans les deux milieux.

```python
#
# Pierre Proulx
#
# Préparation de l'affichage et des outils de calcul symbolique
#
import sympy as sp
from IPython.display import *
sp.init_printing(use_latex=True)
```

```python
# Paramètres, variables et fonctions
S_n,r,b,R_F,R_C,S_n0,delta_r,T_0=sp.symbols('S_n,r,b,R_F,R_C,S_n0,delta_r,T_0')
k_C,k_F=sp.symbols('k_C,k_F')
q_F=sp.symbols('q_F')           # Notation de Transport Phenomena
T_F=sp.Function('T_F')(r)
q_C=sp.symbols('q_C')
T_C=sp.Function('T_C')(r)
S_n=S_n0*(1+b*(r/R_C)**2)
display(S_n)
```

```python
#
# Résultat du bilan dans la sphère fissible d'uranium, solution
#
eqF=(sp.diff(q_F(r)*r**2,r)-S_n*r**2)
# On substitue la loi de Fourier pour trouver le profil de température
eq_TF=eqF.subs(q_F(r),-k_F*sp.diff(T_F,r))
display(eq_TF)
T_F=sp.dsolve(eq_TF,T_F)
display( T_F)
```

```python
T_F=T_F.rhs # Voyez ci-dessous pourquoi on utilise .rhs (right hand side) de l'expression
display(T_F)
q_F=-k_F*sp.diff(T_F,r)
```

```python
#
# Bilan dans l'aluminium, la solution est plus simple évidemment car il n'y a pas de source
#
eqC=sp.diff(q_C(r)*r**2,r)
eq_TC=eqC.subs(q_C(r),-k_C*sp.diff(T_C,r))
T_C=sp.dsolve(eq_TC,T_C)
T_C=T_C.rhs
q_C=-k_C*sp.diff(T_C,r)
display(T_F)
display(q_F)
display(T_C)
display(q_C)
```

```python
#
# Les noms des constantes doivent être changés car sympy a réutilisé les noms C1 et C2 dans les 2 régions.
# Remarquez en plus que la nomenclature 1 et 2 est inversée par rapport au livre.
#
# Pour comparaison, on utilisera les mêmes noms que dans Transport Phenomena, ainsi
#  C_1 de l'uranium devient C_2^(F), C_1 de l'alu devient C_2^(F), etc...
#
T_F=T_F.subs(sp.symbols('C1'),sp.symbols('C_2^(F)'))
T_F=T_F.subs(sp.symbols('C2'),sp.symbols('C_1^(F)'))
q_F=q_F.subs(sp.symbols('C2'),sp.symbols('C_1^(F)'))
T_C=T_C.subs(sp.symbols('C1'),sp.symbols('C_2^(C)'))
T_C=T_C.subs(sp.symbols('C2'),sp.symbols('C_1^(C)'))
q_C=q_C.subs(sp.symbols('C2'),sp.symbols('C_1^(C)'))
#
display(q_F)
display(q_C)
display(T_F)
display(T_C)
```

On peut directement ces 4 équations à 4 inconnues en posant les 4 conditions aux limites comme ci-dessous. On n'aura pas de manipulations autres que de poser les conditions, effectuer la solution et ensuite substituer dans les solutions générales (ci-haut).

### Attention, ici la fonction ** SOLVE ** de SYMPY retourne une liste ([ ]) qui contient un seul élément, notre dictionnaire contenant la solution. Il faut alors sortir le dictionnaire contenant la solutionde la liste en référant à  l'élément [0]. Regardez-bien dessous. Notez que ** SOLVE** ne retourne pas toujours une liste, seulement quand il détecte qu'il pourrait y avoir des solutions multiples, quand il voit des équations non-linéaires.

```python
condition_0=sp.Eq(q_F.subs(r,0),0)                     # flux nul au centre
condition_1=sp.Eq(q_F.subs(r,R_F)-q_C.subs(r,R_F),0)   # flux égaux à l'interface
condition_2=sp.Eq(T_C.subs(r,R_C)-T_0,0)               # température égale à T_0 en r=R_C
condition_3=sp.Eq(T_F.subs(r,R_F)-T_C.subs(r,R_F),0)   # températures égales à l'interface
conditions=(condition_0,condition_1,condition_2,condition_3)
constantes=(sp.symbols('C_1^(C)'),sp.symbols('C_2^(C)'),sp.symbols('C_1^(F)'),sp.symbols('C_2^(F)'))
constantes=sp.solve(conditions,constantes,dict=True)
display(constantes[0])
```

```python
T_F=T_F.subs(constantes[0])
q_F=q_F.subs(constantes[0])
T_C=T_C.subs(constantes[0])
q_C=q_C.subs(constantes[0])
display(T_F)
display(q_F)
display(T_C)
display(q_C)
```

```python
### Comparer avec le profil montré sur la figure 10.3-1 en utilisant des valeurs estimées au PIF!
dico={'k_F':22,'k_C':140,'b':100,'R_F':0.02,'R_C':0.025,'S_n0':1e7,'T_0':300}
T_Fplot=T_F.subs(dico)
T_Cplot=T_C.subs(dico)
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize']=10,8
p = sp.plot((T_Fplot,(r,0,0.02)),(T_Cplot,(r,0.02,0.03))
            ,legend=True,xlabel='r',ylabel='T [K]'
            , title='Carburant nucléaire et coquille', show=False)
p[0].line_color = 'red'
p[0].label='Uranium'
p[1].line_color = 'blue'
p[1].label='Aluminium'
p.show()
```