# The Harder Way: C Code generation, Custom Printers, and CSE [1 hour]

One of the most common low level programming languages in use is C. Compiled C code can be optimized for execution speed for many different computers. Python is written in C as well as many of the vectorized operations in NumPy and numerical algorithms in SciPy. It is often necessary to translate a complex mathematical expression into C for optimal execution speeds and memory management. In this notebook you will learn how to automatically translate a complex SymPy expression into C, compile the code, and run the program.

We will continue examining the complex chemical kinetic reaction ordinary differential equation introduced in the previous lesson.

## Learning Objectives

After this lesson you will be able to:

- use a code printer class to convert a SymPy expression to compilable C code
- use an array compatible assignment to print valid C array code
- subclass the printer class and modify it to provide custom behavior
- utilize common sub expression elimination to simplify and speed up the code execution

Import SymPy and enable mathematical printing in the Jupyter notebook.

```python
import sympy as sym
```

```python
sym.init_printing()
```

# Ordinary Differential Equations

The previously generated ordinary differential equations that describe chemical kinetic reactions are loaded below. These expressions describe the right hand side of this mathematical equation:

$$\frac{d\mathbf{y}}{dt} = \mathbf{f}(\mathbf{y}(t))$$

where the state vector $\mathbf{y}(t)$ is made up of 14 states, i.e. $\mathbf{y}(t) \in \mathbb{R}^{14}$.

Below the variable `rhs_of_odes` represents $\mathbf{f}(\mathbf{y}(t))$ and `states` represents $\mathbf{y}(t)$.

From now own we will simply use $\mathbf{y}$ instead of $\mathbf{y}(t)$ and assume an implicit function of $t$.

```python
from scipy2017codegen.chem import load_large_ode
```

```python
rhs_of_odes, states = load_large_ode()
```

## Exercise [2 min]

Display the expressions (`rhs_of_odes` and `states`), inspect them, and find out their types and dimensions. What are some of the characteristics of the equations (type of mathematical expressions, linear or non-linear, etc)?

### Double Click For Solution

<!--

rhs_of_odes
type(rhs_of_odes)
rhs_of_odes.shape
# rhs_of_odes is a 14 x 1 SymPy matrix of expressions. The expressions are
# long multivariate polynomials.
states
type(states)
states.shape
# states is a 14 x 1 SymPy matrix of symbols

The equations are nonlinear equations of the states. There are 14 equations and 14 states. The coefficients in the equations are various floating point numbers.

-->

```python
# write your solution here
```

# Compute the Jacobian

As has been shown in the previous lesson the Jacobian of the right hand side of the differential equations is often very useful for computations, such as integration and optimization. With:

$$\frac{d\mathbf{y}}{dt} = \mathbf{f}(\mathbf{y})$$

the Jacobian is defined as:

$$\mathbf{J}(\mathbf{y}) = \frac{\partial\mathbf{f}(\mathbf{y})}{\partial\mathbf{y}}$$

SymPy can compute the Jacobian of matrix objects with the `Matrix.jacobian()` method.

## Exercise [3 min]

Look up the Jacobian in the SymPy documentation then compute the Jacobian and store the result in the variable `jac_of_odes`. Inspect the resulting Jacobian for dimensionality, type, and the symbolic form.

### Double Click For Solution

<!--

jac_of_odes = rhs_of_odes.jacobian(states)
type(jac_of_odes)
jac_of_odes.shape
jac_of_odes

The Jacobian is a 14 x 14 SymPy matrix and contains 196 expressions which are linear functions of the state variables.

-->

```python
# write your answer here
```

# C Code Printing

The two expressions are large and will likely have to be excuted many thousands of times to compute the desired numerical values, so we want them to execute as fast as possible. We can use SymPy to print these expressions as C code.

We will design a double precision C function that evaluates both $\mathbf{f}(\mathbf{y})$ and $\mathbf{J}(\mathbf{y})$ simultaneously given the values of the states $\mathbf{y}$. Below is a basic template for a C program that includes such a function, `evaluate_odes()`. Our job is to populate the function with the C version of the SymPy expressions.

```C
#include <math.h>
#include <stdio.h>

void evaluate_odes(const double state_vals[14], double rhs_result[14], double jac_result[196])
{
      // We need to fill in the code here using SymPy.
}

int main() {

    // initialize the state vector with some values
    double state_vals[14] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14};
    // create "empty" 1D arrays to hold the results of the computation
    double rhs_result[14];
    double jac_result[196];

    // call the function
    evaluate_odes(state_vals, rhs_result, jac_result);

    // print the computed values to the terminal
    int i;

    printf("The right hand side of the equations evaluates to:\n");
    for (i=0; i < 14; i++) {
        printf("%lf\n", rhs_result[i]);
    }

    printf("\nThe Jacobian evaluates to:\n");
    for (i=0; i < 196; i++) {
        printf("%lf\n", jac_result[i]);
    }

    return 0;
}

```

Instead of using the `ccode` convenience function you learned earlier let's use the underlying code printer class to do the printing. This will allow us to modify the class to for custom printing further down.

```python
from sympy.printing.ccode import C99CodePrinter
```

All printing classes have to be instantiated and then the `.doprint()` method can be used to print SymPy expressions. Let's try to print the right hand side of the differential equations.

```python
printer = C99CodePrinter()
```

```python
print(printer.doprint(rhs_of_odes))
```

In this case, the C code printer does not do what we desire. It does not support printing a SymPy Matrix (see the first line of the output). In C, on possible representation of a matrix is an array type. The array type in C stores contigous values, e.g. doubles, in a chunk of memory. You can declare an array of doubles in C like:

```C
double my_array[10];
```

The word `double` is the data type of the individual values in the array which must all be the same. The word `my_array` is the variable name we choose to name the array and the `[10]` is the syntax to declare that this array will have 10 values.

The array is "empty" when first declared and can be filled with values like so:

```C
my_array[0] = 5;
my_array[1] = 6.78;
my array[2] = my_array[0] * 12;
```

or like:

```C
my_array = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
```

It is possible to declare multidimensional arrays in C that could map more directly to the indices of our two dimensional matrix, but in this case we will map our two dimensional matrix to a one dimenasional array using C contingous row ordering.

The code printers are capable of dealing with this need through the `assign_to` keyword argument in the `.doprint()` method but we must define a SymPy object that is appropriate to be assigned to. In our case, since we want to assign a Matrix we need to use an appropriately sized Matrix symbol.

```python
rhs_result = sym.MatrixSymbol('rhs_result', 14, 1)
```

```python
print(rhs_result)
```

```python
print(rhs_result[0])
```

```python
print(printer.doprint(rhs_of_odes, assign_to=rhs_result))
```

Notice that we have proper array value assignment and valid lines of C code that can be used in our function.

## Excercise [5 min]

Print out valid C code for the Jacobian matrix.

### Double Click For Solution

<!---
jac_result = sym.MatrixSymbol('jac_result', 14, 14)

print(jac_result)

print(printer.doprint(jac_of_odes, assign_to=jac_result))
-->

```python
# write your answer here
```

# Changing the Behavior of the Printer

The SymPy code printers are relatively easy to extend. They are designed such that if you want to change how a particularly SymPy object prints, for example a `Symbol`, then you only need to modify the `_print_Symbol` method of the printer. In general, the code printers have a method for every SymPy object and also many builtin types. Use tab completion with `C99CodePrinter._print_` to see all of the options.

Once you find the method you want to modify, it is often useful to look at the existing impelementation of the print method to see how the code is written.

```python
C99CodePrinter._print_Symbol??
```

Below is a simple example of overiding the `Symbol` printer method. Note that you should use the `self._print()` method instead of simply returning the string so that the proper printer, `self._print_str()`, is dispatched. This is most important if you are printing non-singletons, i.e. expressions that are made up of multiple singletons.

```python
C99CodePrinter._print_str??
```

```python
class MyCodePrinter(C99CodePrinter):
    def _print_Symbol(self, expr):
        return self._print("No matter what symbol you pass in I will always print:\n\nNi!")
```

```python
my_printer = MyCodePrinter()
```

```python
theta = sym.symbols('theta')
theta
```

```python
print(my_printer.doprint(theta))
```

## Exercise [10 min]

One issue with our current code printer is that the expressions use the symbols `y0, y1, ..., y13` instead of accessing the values directly from the arrays with `state_vals[0], state_vals[1], ..., state_vals[13]`. We could go back and rename our SymPy symbols to use brackets, but another way would be to override the `_print_Symbol()` method to print these symbols as we desire. Modify the code printer so that it prints with the proper array access in the expression.

### Double Click For Solution: Subclassing

<!--

The following solution examines the symbol and if it is a state variable it overrides the printer, otherwise it uses the parent class to print the symbol as a fall back.

class MyCodePrinter(C99CodePrinter):
    def _print_Symbol(self, symbol):
        if symbol in states:
            idx = list(states).index(symbol)
            return self._print('state_vals[{}]'.format(idx))
        else:
            return super()._print_Symbol(symbol)

my_printer = MyCodePrinter()

print(my_printer.doprint(rhs_of_odes, assign_to=rhs_result))

-->

### Double Click For Solution: Exact replacement

<!--
Another option is to replace the symbols with `MatrixSymbol` elements. Notice that the C printer assumes that a 2D matrix will get mapped to a 1D C array.

state_vals = sym.MatrixSymbol('state_vals', 14, 1)
state_array_map = dict(zip(states, state_vals))
print(state_array_map)
print(printer.doprint(rhs_of_odes.xreplace(state_array_map), assign_to=rhs_result))

-->

```python
# write your answer here
```

## Bonus Exercise

*Do this exercise if you finish the previous one quickly.*

It turns out that calling `pow()` for low value integer exponents executes slower than simply expanding the multiplication. For example `pow(x, 2)` could be printed as `x*x`. Modify the CCodePrinter `._print_Pow` method to expand the multiplication if the exponent is less than or equal to 4. You may want to have a look at the source code with `printer._print_Pow??`

Note that a `Pow` expression has an `.exp` for exponent and `.base` for the item being raised. For example $x^2$ would have:

```python
expr = x**2
expr.base == x
expr.exp == 2
```

### Double Click for Solution

<!--

printer._print_Pow??

class MyCodePrinter(C99CodePrinter):
    def _print_Pow(self, expr):
        if expr.exp.is_integer and expr.exp > 0 and expr.exp <= 4:
            return '*'.join([self._print(expr.base) for i in range(expr.exp)])
        else:
            return super()._print_Pow(expr)

my_printer = MyCodePrinter()

x = sym.Symbol('x')

my_printer.doprint(x)

my_printer.doprint(x**2)

my_printer.doprint(x**4)

my_printer.doprint(x**5)

my_printer.doprint(x**1.5)

-->

```python
# write your answer here
```

# Common Subexpression Elimination

If you look carefully at the expressions in the two matrices you'll see repeated expressions. These are not ideal in the sense that the computer has to repeat the exact same calculation multiple times. For large expressions this can be a major issue. Compilers, such as gcc, can often eliminate common subexpressions on their own when different optimization flags are invoked but for complex expressions the algorithms in some compilers do not do a thorough job or compilation can take an extremely long time. SymPy has tools to perform common subexpression elimination which is both thorough and reasonably efficient. In particular if gcc is run with the lowest optimization setting `-O0` `cse` can give large speedups.

For example if you have two expressions:

```python
a = x*y + 5
b = x*y + 6
```

you can convert this to these three expressions:

```python
z = x*y
a = z + 5
b = z + 6
```

and `x*y` only has to be computed once.

The `cse()` function in SymPy returns the subexpression, `z = x*y`, and the simplified expressions: `a = z + 5`, `b = z + 6`.

Here is how it works:

```python
sm.cse?
```

```python
sub_exprs, simplified_rhs = sym.cse(rhs_of_odes)
```

```python
for var, expr in sub_exprs:
    sym.pprint(sym.Eq(var, expr))
```

`cse()` can return a number of simplified expressions and to do this it returns a list. In our case we have 1 simplified expression that can be accessed as the first item of the list.

```python
type(simplified_rhs)
```

```python
len(simplified_rhs)
```

```python
simplified_rhs[0]
```

You can find common subexpressions among multiple objects also:

```python
jac_of_odes = rhs_of_odes.jacobian(states)

sub_exprs, simplified_exprs = sym.cse((rhs_of_odes, jac_of_odes))
```

```python
for var, expr in sub_exprs:
    sym.pprint(sym.Eq(var, expr))
```

```python
simplified_exprs[0]
```

```python
simplified_exprs[1]
```

# Exercise [15min]

Use common subexpression elimination to print out C code for your two arrays such that:

```C
double x0 = first_sub_expression;
...
double xN = last_sub_expression;

rhs_result[0] = expressions_containing_the_subexpressions;
...
rhs_result[13] = ...;

jac_result[0] = ...;
...
jac_result[195] = ...;
```

The code you create can be copied and pasted into the provided template above to make a C program. *Refer back to the introduction to C code printing above.*

To give you a bit of help we will first introduce the `Assignment` class. The printers know how to print variable assignments that are defined by an `Assignment` instance.

```python
from sympy.printing.codeprinter import Assignment

print(printer.doprint(Assignment(theta, 5)))
```

The following code demonstrates a way to use `cse()` to simplify single matrix objects. Note that we use `ImmutableDenseMatrix` because all dense matrics are internally converted to this type in the printers. Check the type of your matrices to see.

```python
class CMatrixPrinter(C99CodePrinter):
    def _print_ImmutableDenseMatrix(self, expr):
        sub_exprs, simplified = sym.cse(expr)
        lines = []
        for var, sub_expr in sub_exprs:
            lines.append('double ' + self._print(Assignment(var, sub_expr)))
        M = sym.MatrixSymbol('M', *expr.shape)
        return '\n'.join(lines) + '\n' + self._print(Assignment(M, expr))
```

```python
p = CMatrixPrinter()
print(p.doprint(jac_of_odes))
```

Now create a custom printer that uses `cse()` on the two matrices simulatneously so that subexpressions are not repeated. *Hint: think about how the list printer method, `_print_list(self, list_of_exprs)`, might help here.*

### Double Click For Solution

<!--

class CMatrixPrinter(C99CodePrinter):

    def _print_list(self, list_of_exprs):
        # NOTE : The MutableDenseMatrix is turned in an ImmutableMatrix inside here.
        if all(isinstance(x, sym.ImmutableMatrix) for x in list_of_exprs):
            sub_exprs, simplified_exprs = sym.cse(list_of_exprs)
            lines = []
            for var, sub_expr in sub_exprs:
                ass = Assignment(var, sub_expr.xreplace(state_array_map))
                lines.append('double ' + self._print(ass))
            for mat in simplified_exprs:
                lines.append(self._print(mat.xreplace(state_array_map)))
            return '\n'.join(lines)
        else:
            return super()._print_list(list_of_exprs)

    def _print_ImmutableDenseMatrix(self, expr):
        if expr.shape[1] > 1:
            M = sym.MatrixSymbol('jac_result', *expr.shape)
        else:
            M = sym.MatrixSymbol('rhs_result', *expr.shape)
        return self._print(Assignment(M, expr))

p = CMatrixPrinter()
print(p.doprint([rhs_of_odes, jac_of_odes]))

-->

```python
# write your answer here
```

## Bonus Exercise: Compile and Run the C Program

Below we provide you with a template for the C program described above. You can use it by passing in a string like:

```python
c_template.format(code='the holy grail')
```

Use this template and your code printer to create a file called `run.c` in the working directory.

To compile the code there are several options. The first is gcc (the GNU C Compiler). If you have Linux, Mac, or Windows (w/ mingw installed) you can use the Jupyter notebook `!` command to send your command to the terminal. For example:

```ipython
!gcc run.c -lm -o run
```

This will compile `run.c`, link against the C math library with `-lm` and output, `-o`, to a file `run` (Mac/Linux) or `run.exe` (Windows).

On Mac and Linux the program can be executed with:

```ipython
!./run
```

and on Windows:

```ipython
!run.exe
```

Other options are using the clang compiler or Windows `cl` compiler command:

```ipython
!clang run.c -lm -o run
!cl run.c -lm
```

### Double Click For Solution

<!--

c_program = c_template.format(code=p.doprint([rhs_of_odes, jac_of_odes]))
print(c_program)

with open('run.c', 'w') as f:
    f.write(c_program)

-->

```python
c_template = """\
#include <math.h>
#include <stdio.h>

void evaluate_odes(const double state_vals[14], double rhs_result[14], double jac_result[196])
{{
    // We need to fill in the code here using SymPy.
{code}
}}

int main() {{

    // initialize the state vector with some values
    double state_vals[14] = {{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}};
    // create "empty" 1D arrays to hold the results of the computation
    double rhs_result[14];
    double jac_result[196];

    // call the function
    evaluate_odes(state_vals, rhs_result, jac_result);

    // print the computed values to the terminal
    int i;
    printf("The right hand side of the equations evaluates to:\\n");
    for (i=0; i < 14; i++) {{
        printf("%lf\\n", rhs_result[i]);
    }}
    printf("\\nThe Jacobian evaluates to:\\n");
    for (i=0; i < 196; i++) {{
        printf("%lf\\n", jac_result[i]);
    }}

    return 0;
}}\
"""
```

```python
# write your answer here
```


# Quadratic Program (QP) Tutorial

For instructions on how to run these tutorial notebooks, please see the [README](https://github.com/RobotLocomotion/drake/blob/master/tutorials/README.md).

## Important Note
Please refer to [mathematical program tutorial](./mathematical_program.ipynb) for constructing and solving a general optimization program in Drake.

A (convex) quadratic program (QP) is a special type of convex optimization. Its cost function is a convex quadratic function. Its constraints are linear, same as the constraints in linear program. A (convex) quadratic program has the following form
\begin{align}
\min_x 0.5 x^TQx + b^Tx + c\\
\text{s.t } Ex \leq f
\end{align}
where `Q` is a positive semidefinite matrix.

A quadratic program can be solved by many different solvers. Drake supports some solvers including OSQP, SCS, Gurobi, Mosek, etc. Please see our [Doxygen page]( https://drake.mit.edu/doxygen_cxx/group__solvers.html) for a complete list of supported solvers. Note that some commercial solvers (such as Gurobi and Mosek) are not included in the pre-compiled Drake binaries, and therefore not on Binder/Colab.

Drake's API supports multiple functions to add quadratic cost and linear constraints. We briefly go through some of the functions in this tutorial. For a complete list of functions, please check our [Doxygen](https://drake.mit.edu/doxygen_cxx/classdrake_1_1solvers_1_1_mathematical_program.html).

There are many applications of quadratic programs in robotics, for example, we can solve differential inverse kinematics problem as a QP, see [DifferentialInverseKinematics](https://drake.mit.edu/doxygen_cxx/namespacedrake_1_1manipulation_1_1planner.html#ab53fd2e1578db60ceb43b754671ae539) for more details. For more examples, check out [Underactuated Robotics code repo](https://github.com/RussTedrake/underactuated)

## Add quadratic cost

### AddQuadraticCost function

The easist way to add a quadratic cost is to call the `AddQuadraticCost` function. In the following code snippet, we first construct a program with 2 decision variables, and then show how to call the `AddQuadraticCost` function.

```python
from pydrake.solvers.mathematicalprogram import MathematicalProgram, Solve
import numpy as np

# Create an empty MathematicalProgram named prog (with no decision variables,
# constraints or costs)
prog = MathematicalProgram()
# Add two decision variables x[0], x[1].
x = prog.NewContinuousVariables(2, "x")
```

We can call `AddQuadraticCost(expression)` to add the quadratic cost, where `expression` is a symbolic expression representing a quadratic cost.

```python
# Add a symbolic quadratic expression as the quadratic cost.
cost1 = prog.AddQuadraticCost(x[0]**2 + 2*x[0]*x[1] + x[1]**2 + 3*x[0] + 4)
# The newly added cost is returned as cost1
print(cost1)
# The newly added cost is stored inside prog.
print(prog.quadratic_costs()[0])
```

If we call `AddQuadraticCost` again, the total cost of `prog` is the summation of all the added costs. You can see that `prog.quadratic_costs()` has two entries. And the total cost of `prog` is `cost1 + cost2`.

```python
# Add another quadratic cost to prog.
cost2 = prog.AddQuadraticCost(x[1]*x[1] + 3)
print(f"The number of quadratic costs in prog: {len(prog.quadratic_costs())}")
```

If you know the coefficients of the quadratic cost `Q, b, c`, you could also add the cost without using the symbolic expression, as shown in the following code snippet

```python
# Add the cost x[0]*x[0] + x[0]*x[1] + 1.5*x[1]*x[1] + 2*x[0] + 4*x[1] + 1
cost3 = prog.AddQuadraticCost(Q=[[2, 1], [1, 3]], b=[2, 4], c=1, vars=x)
print(f"cost 3 is {cost3}")
```

### AddQuadraticErrorCost
You could also add the quadratic cost
\begin{align}
(x - x_{desired})^TQ(x-x_{desired})
\end{align}
to the program by calling `AddQuadraticErrorCost`. Here is the code example

```python
# Adds the cost (x - [1;2])' * Q * (x-[1;2])
cost4 = prog.AddQuadraticErrorCost(Q=[[1, 2],[2, 6]], x_desired=[1,2], vars=x)
print(f"cost4 is {cost4}")
```

### AddL2NormCost
You could also add the quadratic cost
\begin{align}
|Ax-b|^2
\end{align}
which is the squared L2 norm of the vector `Ax-b` to the program by calling `AddL2NormCost`. Here is the code example

```python
# Adds the squared norm of (x[0]+2*x[1]-2, x[1] - 3) to the program cost.
cost5 = prog.AddL2NormCost(A=[[1, 2], [0, 1]], b=[2, 3], vars=x)
print(f"cost5 is {cost5}")
```

## Add linear cost

You could also add linear costs to a quadratic program. For an introduction on the different APIs on adding a linear cost, please refer to our [linear programming tutorial](./linear_program.ipynb). Here is an example

```python
# Adds a linear cost to the quadratic program
cost6 = prog.AddLinearCost(x[0] + 3 * x[1] + 1)
print(f"cost6 is {cost6}")
print(f"Number of linear costs in prog: {len(prog.linear_costs())}")
```

## Add linear constraints

To add linear constraints into quadratic program, please refer to the section `Add linear constraints` in our [linear programming tutorial](./linear_program.ipynb). Here is a brief example

```python
constraint1 = prog.AddLinearConstraint(x[0] + 3*x[1] <= 5)
# Adds the constraint 1 <= x[0] <= 5 and 1 <= x[1] <= 5
constraint2 = prog.AddBoundingBoxConstraint(1, 5, x)
```

## A complete code example
Here we show a complete example to construct and solve a QP

```python
prog = MathematicalProgram()
x = prog.NewContinuousVariables(3, "x")
prog.AddQuadraticCost(x[0] * x[0] + 2 * x[0] + 3)
# Adds the quadratic cost on the squared norm of the vector
# (x[1] + 3*x[2] - 1, 2*x[1] + 4*x[2] -4)
prog.AddL2NormCost(A = [[1, 3], [2, 4]], b=[1, 4], vars=[x[1], x[2]])

# Adds the linear constraints.
prog.AddLinearEqualityConstraint(x[0] + 2*x[1] == 5)
prog.AddLinearConstraint(x[0] + 4 *x[1] <= 10)
# Sets the bounds for each variable to be within [-1, 10]
prog.AddBoundingBoxConstraint(-1, 10, x)

# Solve the program.
result = Solve(prog)
print(f"optimal solution x: {result.GetSolution(x)}")
print(f"optimal cost: {result.get_optimal_cost()}")
```

For more details on quadratic programming, you could refer to

[Quadratic Programming wiki](https://en.wikipedia.org/wiki/Quadratic_programming)

[Numerical Optimization by J. Nocedal and S.Wright](http://www.apmath.spbu.ru/cnsa/pdf/monograf/Numerical_Optimization2006.pdf)


$\Huge{\text{Linear Regression}}$

In this session, we shall learn about the basic concepts of linear regression.

Let us first consider a sample data, which will be useful for our study.

```python
#First, we import the required packages
import pandas as pd #the pandas library is useful for data processing
import matplotlib.pyplot as plt #the matplotlib library is useful for plotting purposes

# The following python directive helps to plot the graph in the notebook directly
%matplotlib inline
```

Now let us consider some open source data sets available in the internet.
The first data set measures the mathscores of humans who have some drug concentration in their body.

```python
#Let us explore the data and the description

#Get the data from the web
drug_vs_mathscore_data = pd.read_csv('http://users.stat.ufl.edu/~winner/data/lsd.dat', delim_whitespace=True, index_col=False, header=None)

#print the first few rows of the data
drug_vs_mathscore_data.head()

```

      0
      1

      0
      1.17
      78.93

      1
      2.97
      58.20

      2
      3.26
      67.47

      3
      4.69
      37.47

      4
      5.83
      45.65

```python
#if you wish to print the entire data, then use the following
drug_vs_mathscore_data

```

      0
      1

      0
      1.17
      78.93

      1
      2.97
      58.20

      2
      3.26
      67.47

      3
      4.69
      37.47

      4
      5.83
      45.65

      5
      6.00
      32.92

      6
      6.41
      29.97

What do the numbers in the above data mean? Let us understand the data by seeing its description.

```python
import urllib.request  #this package is useful for accessing text files over internet
drug_vs_mathscore_data_description_file = urllib.request.urlopen('http://users.stat.ufl.edu/~winner/data/lsd.txt')

for line in drug_vs_mathscore_data_description_file:
  decoded_line = line.decode("utf-8")
  print(decoded_line)
```

    Dataset:  lsd.dat

    Source: Wagner, Agahajanian, and Bing (1968). Correlation of Performance

    Test Scores with Tissue Concentration of Lysergic Acid Diethylamide in

    Human Subjects. Clinical Pharmacology and Therapeutics, Vol.9 pp635-638.

    Description: Group of volunteers was given LSD, their mean scores on

    math exam and tissue concentrations of LSD were obtained at n=7 time points.

    Variables/Columns

    Tissue Concentration    1-4

    Math Score             8-12

Having known the data description, let us insert the descriptions into the data now.

```python
drug_vs_mathscore_data.columns = ['Tissue concentration', 'Math score']

#check by printing the data again
drug_vs_mathscore_data.head()
```

      Tissue concentration
      Math score

      0
      1.17
      78.93

      1
      2.97
      58.20

      2
      3.26
      67.47

      3
      4.69
      37.47

      4
      5.83
      45.65

Seeing the data as mere numbers might not be interesting. So, let us use some graphical ways to visualize the data.

```python
#We will plot a scatter plot of the math score vs tissue concentration

plt.scatter(drug_vs_mathscore_data['Tissue concentration'], drug_vs_mathscore_data['Math score'])
#plt.scatter(drug_vs_mathscore_data.iloc[:,0], drug_vs_mathscore_data.iloc[:,1])
plt.title("Tissue drug concentration vs Math scores")
plt.xlabel("Tissue concentration")
plt.ylabel("Math scores")
plt.show()
```

We see that there is a negative trend in the data. That is, as the tissue drug concentration increases, the math score decreases.

$\large{\text{Question:}}$ Is the trend linear (or) at least linear looking?

$\large{\text{Another Question:}}$ What do we mean by a linear (or) linear looking trend between the two variables tissue concentration and math scores?

In the next 2 figures, we shall illustrate the linear relationship between two variables $x$ and $y$ using a line. We assume $x$ to be independent (or) predictor variable and $y$ as the dependent (or) response variable.

$\large{\text{Linear relationship between }} x \text{ and } y \text{ captured by a straight line with positive slope }$.

```python
from urllib.request import urlopen
from PIL import Image

img = Image.open(urlopen('https://github.com/balamurugan-palaniappan-CEP/AIML_CEP_2021/raw/main/images/line_pic.png'))
img
```

In this figure, we see that the line is represented by $y=mx+c$, where $m=\tan(\theta)$ is the slope of the line and $c$ is the intercept made on the $y$-axis. $\textbf{Note}$ that since $\theta$ is acute, the line has a positive slope.

$\large{\text{Linear relationship between }} x \text{ and } y \text{ captured by a straight line with negative slope }$.

```python
img_2 = Image.open(urlopen('https://github.com/balamurugan-palaniappan-CEP/AIML_CEP_2021/raw/main/images/line_pic_neg_slope.png'))
img_2
```

In this figure too, we see that the line is represented by $y=mx+c$, where $m=\tan(\theta)$ is the slope of the line and $c$ is the intercept made on the $y$-axis. $\textbf{Note}$ that since $\theta$ is obtuse, the line has a negative slope.

Hence when we look for $\large{\text{linear trend}}$ (or) $\large{\text{linear relationship}}$, we look for the relationship to be approximated by a straight line with a positive or negative slope.

$\Large{\text{Probabilistic interpretation}}$

In probability terms, let us assume that $X$ and $Y$ are two real-valued random variables, where $Y$ denotes the response random variable and $X$ denotes the predictor random variable.

Then we assume that the expected value of $Y$ given some observation of $X=x$ is represented as:
$
E[Y|X=x] = \beta_1 x + \beta_0.
$.

However when we observe a data point $(x^i, y^i)$ as a realization of the pair $(X,Y)$, it is possible that the observed value $y^i$ of the response random variable $Y$ is not necessarily equal to the expected value $E[Y|X=x^i]=\beta_1 x^i + \beta_0$.

In such case, we assume that the discrepancy between the observed value $y^i$ and the expected value $E[Y|X=x^i]$ is captured by an error $e^i$ given by:

$
\begin{align}
e^i &= y^i - E[Y|X=x^i] \\
\implies e^i&= y^i - (\beta_1 x^i + \beta_0).
\end{align}
$

Now we can assume that the data set $D$ comprises multiple realizations of the random variable pair $(X,Y)$.

Thus when we the data set $D$ contains $n$ observations of the form $\{(x^1,y^1), (x^2,y^2), \ldots, (x^n,y^n)\}$, then we can compute the errors as:
$\begin{align}
e^i = y^i - (\beta_1 x^i + \beta_0), \forall i \in \{1,2,\ldots,n\}.
\end{align}
$

```python
img_points_errors = Image.open(urlopen('https://github.com/balamurugan-palaniappan-CEP/AIML_CEP_2021/raw/main/images/line_with_points_errors.png'))
img_points_errors
```

$\large{\text{How to estimate the parameters } \beta_1, \beta_0?}$

One way to estimate the values of $\beta_0, \beta_1$ is by minimizing the sum of squared errors given by:
$
\begin{align}
\min \sum_{i=1}^{n} (e^i)^2
\end{align}
$
which can be equivalently written as the following optimization problem:
$
\begin{align}
\min_{\beta_0, \beta_1} \sum_{i=1}^{n} [y^i - (\beta_1 x^i + \beta_0) ]^2.
\end{align}
$

$\textbf{Note}$ that the optimization problem is called the ordinary least squares (OLS) problem, and the term $\sum_{i=1}^{n} [y^i - (\beta_1 x^i + \beta_0) ]^2$ is called the OLS objective function and we will denote the OLS objective by:

$
\begin{align}
L(\beta_0, \beta_1) = \sum_{i=1}^{n} [y^i - (\beta_1 x^i + \beta_0) ]^2.
\end{align}
$

$\Large{\text{Solving the OLS optimization problem}}:$

Recall from calculus that to solve

$
\begin{align}
\min_{\beta_0, \beta_1} L(\beta_0, \beta_1) = \sum_{i=1}^{n} [y^i - (\beta_1 x^i + \beta_0) ]^2.
\end{align}
$,
we find the partial derivatives with respect to $\beta_0, \beta_1$, and equate them to zero. Thus we would have:

$
\begin{align}
\frac{\partial }{\partial \beta_0} L(\beta_0, \beta_1) &= 0 \\
\implies \frac{\partial } {\partial \beta_0} \sum_{i=1}^{n} [y^i - (\beta_1 x^i + \beta_0) ]^2 &= 0 \\
\implies  -2 \sum_{i=1}^{n} [y^i - \beta_1 x^i - \beta_0) ] &= 0  \\
\implies \beta_0 n + \beta_1 \sum_{i=1}^{n} x^i &= \sum_{i=1}^{n} y^i.
\end{align}
$,

and

$
\begin{align}
\frac{\partial }{\partial \beta_1} L(\beta_0, \beta_1) &= 0 \\
\implies \frac{\partial } {\partial \beta_1} \sum_{i=1}^{n} [y^i - (\beta_1 x^i + \beta_0) ]^2 &= 0 \\
\implies  -2 \sum_{i=1}^{n} x^i[y^i - \beta_1 x^i - \beta_0) ] &= 0  \\
\implies \beta_0 \sum_{i=1}^{n} x^i + \beta_1 \sum_{i=1}^{n} (x^i)^2 &= \sum_{i=1}^{n} x^i y^i.
\end{align}
$.

Thus we have a system of 2 linear equations in $\beta_0, \beta_1$ given by:

$
\begin{align}
\beta_0 n + \beta_1 \sum_{i=1}^{n} x^i &= \sum_{i=1}^{n} y^i. \\
\beta_0 \sum_{i=1}^{n} x^i + \beta_1 \sum_{i=1}^{n} (x^i)^2 &= \sum_{i=1}^{n} x^i y^i.
\end{align}
$.

This system of linear equations is called $\textbf{normal equations}$.

$\Large{\text{Exercise:}}$

Show that solving this system gives us the following estimates for $\beta_0$ and $\beta_1$:

$\beta_0 = \bar{y} - \beta_1 \bar{x} \\
\beta_1 = \frac{\Sigma_{XY}}{\Sigma_{XX}}
$

where

$
\begin{align}
\bar{x} &= \frac{1}{n} \sum_{i=1}^{n}x^i, \\
\bar{y} &= \frac{1}{n} \sum_{i=1}^{n}y^i, \\
\Sigma_{XX} &= \sum_{i=1}^{n} (x^i - \bar{x})^2 \\
\Sigma_{XY} &=  \sum_{i=1}^{n} (x^i - \bar{x})(y^i - \bar{y}) \\
\end{align}
$

$\Large{\text{Computing } \beta_0, \beta_1 \text{ for the Math Score vs Tissue Drug Concentration data set}}:$

```python
#Let us now compute beta_0 and beta_1 from the math score vs tissue concentration data set
n = len(drug_vs_mathscore_data.index) #number of data points in the data set
print('number of data points in the data set:',n)
```

    number of data points in the data set: 7

```python
#First let us compute x_bar and y_bar
x_bar = 0
y_bar = 0

for i in range(n): #access each row from the data set
  x_bar += float(drug_vs_mathscore_data.iat[i,0]) #access element at Tissue concentration column
  y_bar += float(drug_vs_mathscore_data.iat[i,1]) #access element at math scores column
x_bar /= n
y_bar /= n
print('x bar:',x_bar, 'y bar:',y_bar)
```

    x bar: 4.332857142857143 y bar: 50.08714285714286

```python
#Then we will compute Sigma_XX and Sigma_XY
sigma_xx = 0
sigma_xy = 0

for i in range(n): #access each row from the data set
  sigma_xx += (float(drug_vs_mathscore_data.iat[i,0]) - x_bar)**2 #computing (x^i - x_bar)^2
  sigma_xy += (float(drug_vs_mathscore_data.iat[i,0]) - x_bar) * (float(drug_vs_mathscore_data.iat[i,1])-y_bar) #computing (x^i - x_bar)(y^i-y_bar)
print('sigma_xx:',sigma_xx, 'sigma_xy:',sigma_xy)
```

    sigma_xx: 22.474942857142857 sigma_xy: -202.48724285714292

```python
#Now we can compute beta_0 and beta_1
beta_1 = sigma_xy/sigma_xx
beta_0 = y_bar - beta_1* x_bar

print('beta_0:', beta_0, 'beta_1', beta_1)

```

    beta_0: 89.12387376799306 beta_1 -9.009466415296783

$\Large{\text{Plotting the regression line}}$

Having computed $\beta_0$ and $\beta_1$, we will now plot the line $y=\beta_1 x + \beta_0$ along with the points in the data set.

```python
#compute the left and right limits of the x-axis
x_min = 9999
x_max = -9999

for i in range(n): #access each row from the data set
  x_i = float(drug_vs_mathscore_data.iat[i,0]) #access element at Tissue concentration column
  if x_i > x_max:
    x_max = x_i
  if x_i < x_min:
    x_min = x_i

x_min = x_min - 1
x_max = x_max + 1

print('x_min:',x_min,'x_max:',x_max)

```

    x_min: 0.16999999999999993 x_max: 7.41

```python
import numpy as np
x = np.linspace(x_min,x_max,100) #creates a series of points in x axis
y = beta_1*x+beta_0
plt.plot(x, y, '-r', label='regression line')
plt.scatter(drug_vs_mathscore_data['Tissue concentration'], drug_vs_mathscore_data['Math score'])

plt.title("Tissue drug concentration vs Math scores Regression")
plt.xlabel("Tissue concentration")
plt.ylabel("Math scores")
plt.legend(loc='upper right')
#plt.grid()
plt.show()
```

$\Large{\text{Residual Plot}}$

Sometimes it would be useful to plot the error (or) $\textbf{residual}$ $e^i$ versus the fitted values $\hat{y}^i=\beta_1 x^i + \beta_0$.

```python
e_i_residuals_list = []
for i in range(n): #access each row from the data set
  x_i = float(drug_vs_mathscore_data.iat[i,0]) #access element at Tissue concentration column
  y_i = float(drug_vs_mathscore_data.iat[i,1]) #access element at Math scores column

  y_pred_i = beta_1 * x_i + beta_0 #compute the prediction obtained using the regression coefficients
  e_i = y_i - y_pred_i #compute the difference between the actual observation y_i and prediction y_pred_i

  e_i_residuals_list.append(e_i) #append the value of e_i to the list

#plot the residuals e_i against the actual observations y_i
plt.scatter(drug_vs_mathscore_data['Math score'], e_i_residuals_list,color='r')
plt.title("Residual plot")
plt.xlabel("Math score responses")
plt.ylabel("Residuals")
plt.grid()
plt.show()
```

Note that the residual plot helps to check the variance in the errors $e^i$.

$\Large{\text{Correlation between the variables}}$

It is useful to understand how the response variable is correlated with the predictor variable.

One quantity useful to measure the correlation between the response variable and the predictor variable is the $\textbf{sample correlation}$ given by:

$
\begin{align}
\text{corr}(X,Y) = \frac{\Sigma_{XY}}{(n-1)s_X s_Y}
\end{align}
$
where

$
\begin{align}
\text{recall } \Sigma_{XY} &= \sum_{i=1}^{n} (x^i - \bar{x}) (y^i - \bar{y}), \\
s_X &= \sqrt{\frac{1}{n-1} \Sigma_{XX} } = \sqrt{\frac{1}{n-1} \sum_{i=1}^{n} (x^i-\bar{x})^2 } \\
s_Y &= \sqrt{\frac{1}{n-1} \Sigma_{YY} } = \sqrt{\frac{1}{n-1} \sum_{i=1}^{n} (y^i-\bar{y})^2 }
\end{align}
$

$\text{Note}$ that $s_X$ and $s_Y$ are sample standard deviations of $X$ and $Y$ respectively.

$\textbf{Exercise:}$ Verify that sample correlation can be equivalently written as

$
\begin{align}
\text{corr}(X,Y) = \frac{\Sigma_{XY}}{\sqrt{\Sigma_{XX}\Sigma_{YY}}}
\end{align}
$
where recall that:

$
\begin{align}
\Sigma_{XX} &= \sum_{i=1}^{n} (x^i - \bar{x})^2 \\
\Sigma_{YY} &= \sum_{i=1}^{n} (y^i - \bar{y})^2
\end{align}
$.

It is also known that $-1 \leq \text{corr}(X,Y) \leq 1$.

Let us now compute the sample correlation.

```python
#Note that sigma_XY and sigma_XX have already been computed.
#Hence we will now compute sigma_YY

#Also note that y_bar is computed before

sigma_yy = 0
for i in range(n): #access each row from the data set
  y_i = float(drug_vs_mathscore_data.iat[i,1]) #access element at Math scores column
  sigma_yy += (y_i-y_bar)**2

print('sigma_yy:', sigma_yy)

#Then we will compute the sample correlation
sample_correlation = sigma_xy/(np.sqrt(sigma_xx * sigma_yy))
print('sample correlation:',sample_correlation)
```

    sigma_yy: 2078.183342857143
    sample correlation: -0.9369284768740831

$\Large{\text{Coefficient of multiple determination}}$

Consider the term

$
\begin{align}
\Sigma_{YY} = \sum_{i=1}^{n} (y^i - \bar{y})^2.
\end{align}
$
This term $\Sigma_{YY}$ measures the $\textbf{observed total variation of the response}$, without considering the effect of predictor variables.

When we have access to a predictor variable, then the $\textbf{unexplained variation of response variable given the predictor variable}$ is given by the sum of squared residuals:

$
\begin{align}
L = \sum_{i=1}^{n} (e^i)^2.
\end{align}
$

Define the term:

$
\begin{align}
\text{SSD} = \Sigma_{YY}- L
\end{align}
$

as the difference between observed total variation of the response and the unexplained variation given the predictor. Hence this can be considered as $\textbf{variability explained by the regression on the predictor variable}$.

Assuming $\Sigma_{YY}\neq 0$, consider:

$
\begin{align}
\frac{\text{SSD}}{\Sigma_{YY}} = 1- \frac{L}{\Sigma_{YY}}.
\end{align}
$
Here the term $\frac{\text{SSD}}{\Sigma_{YY}}$ denotes the proportion of observed variability in the response variable which is explained by the regression on the predictor variable. From the equality, this proportion of observed variability given the predictor is simply $1$ minus the proportion of unexplained variability.

This term $\frac{\text{SSD}}{\Sigma_{YY}}$ denoting the proportion of observed variability which is explained by the predictor variable has a special name called $\textbf{coefficient of multiple determination}$ and is denoted by $R^2$.

It is known that $0\leq R^2 \leq 1$.

Let us compute $R^2$.

```python
#Note that sum of squared residuals needs to be computed

sum_sq_residuals = 0
for i in range(n): #access each row from the data set
  x_i = float(drug_vs_mathscore_data.iat[i,0]) #access element at Tissue concentration column
  y_i = float(drug_vs_mathscore_data.iat[i,1]) #access element at Math scores column
  y_pred_i = beta_1*x_i + beta_0
  sum_sq_residuals += (y_i-y_pred_i)**2

print('sum of squared residuals:', sum_sq_residuals)

#Then we will compute the R^2 quantity
R_sq = 1.0-sum_sq_residuals/sigma_yy
print('R^2:',R_sq)
```

    sum of squared residuals: 253.88132880967103
    R^2: 0.8778349707775888

$\Large{\text{References:}}$

1.   Olive J. David, $\textit{Linear Regression}$. Springer International Publishing,  2017.
2.   Sanford Weisberg. $\textit{Applied Linear Regression}$. John Wiley $\&$ Sons, Inc., Hoboken, New Jersey, 4-th Edition, 2014.

```python

```


```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
```

# Aprendizaje no supervisado

En esta lección se estudian dos algoritmos de aprendizaje no supervisado clásicos y ampliamente utilizados: PCA como representante de los algoritmos de reducción de dimensionalidad y KMeans como representando de los algoritmos de agrupamiento/clustering

## Principal Component Analysis (PCA)

A continuación se presentan los fundamentos teóricos tras el algoritmo PCA. En seguida veremos como utilizar la implementación de PCA de la librería `sklearn` para hacer reducción de dimensionalidad en base de datos con atributos continuos

### Formalismo matemático

PCA es un procedimiento estadístico que busca una **transformación ortogonal** que **maximice la varianza** de los datos

Matemáticamente, podemos escribir un conjunto de datos $\{x_i\}$ con $i=1,2,\ldots, N$ y $x_i \in \mathbb{R}^D$, como una matriz $X \in \mathbb{R}^{N\times D}$

La transformación está representada por una matriz $W \in \mathbb{R}^{D\times D}$ y los datos transformados se obtienen mediante una multiplicación matricial

$$
X' = X W
$$

Una transformación es ortogonal si cumple $W^T W = I$ es decir que la transformación traspuesta es equivalente a su inversa. Para encontrar una transformación que maximice la varianza de los datos debemos obtener primero la matriz de correlación de $X$ definida por

$$
C = \frac{1}{N} X^T X
$$

donde $C \in \mathbb{R}^{D\times D}$ y donde asumimos que la media de $X$ es cero. Notemos que la matriz de correlación de los datos transformados es $\frac{1}{N} X'^T X' = \frac{1}{N} W^T X^T X W = W^T C W$ con lo que podemos escribir la siguiente función objetivo

$$
\max_W W^T C W \text{ sujeto a } W^T W = I
$$

Usando *multiplicadores de Lagrange* para incluir la restricción en el objetivo y derivando e igualando a cero obtemos lo siguiente

$$
\begin{align}
\frac{d}{dW} W^T C W + \Lambda(I- W^T W) &= 0 \nonumber \\
(C - \Lambda) W &= 0 \nonumber
\end{align}
$$

donde $\Lambda = \lambda I$ y $\lambda = (\lambda_1, \lambda_2, \ldots, \lambda_D)$

Esto se conoce como el problema de los valores $\lambda$ y vectores propios $W$ de la matriz $C$.

### Ejemplo

Sean los siguientes datos bidimensionales

```python
X = np.random.multivariate_normal([0, 0], [[0.5, -0.7], [-0.7, 1]], size=1000)
```

Usaremos PCA para encontrar los ejes coordenados de máxima varianza y graficarlos

Para resolver el problema de valores propios usaremos `np.linalg.eigh` que recibe una matriz cuadrada y retorna sus valores y vectores propios

```python
import scipy.linalg
# Restamos la media
X_ = X - np.mean(X, axis=0, keepdims=True)
# Calculamos la covarianza
C = np.dot(X_.T, X_)/len(X_)
# Calculamos los valores y vectores propios de la covarianza
L, W = scipy.linalg.eigh(C)
# Proyectamos
U = np.dot(X, W)

# Visualización de datos y proyección
arrow_args = {'width': 0.05, 'length_includes_head': True, 'alpha': 0.5}
fig, ax = plt.subplots(1, 2, figsize=(6, 3), dpi=120,
                       tight_layout=True, sharex=True, sharey=True)
ax[0].scatter(X[:, 0], X[:, 1], s=10);
for i, c in enumerate(['g', 'r']):
    ax[0].arrow(0, 0, W[i, 0], W[i, 1], color=c, **arrow_args)
ax[0].set_aspect('equal');
ax[1].set_aspect('equal');
ax[0].set_xlim([-3.5, 3.5])
ax[1].scatter(U[:, 0], U[:, 1])
ax[1].spines['bottom'].set_color('g')
ax[0].set_ylabel('Atributo 1')
ax[0].set_xlabel('Atributo 2')
ax[1].set_ylabel('Componente principal 1')
ax[1].set_xlabel('Componente principal 2')
ax[1].spines['left'].set_color('r')
ax[0].set_title('Datos originales')
ax[1].set_title('Datos proyectados');
```

**Discusión:**

- El eje rojo acumula un 99.5% de la varianza
- El eje verde es ortogonal al rojo
- Los nuevos ejes están decorrelacionados c/r a los originales

### Reducción de dimensionalidad con PCA

Una aplicación típica de PCA es la reducción de dimensionalidad

Recordemos

- La matriz $W$ tiene las mismas dimensiones que $C$
- Las columnas de $W$ son los vector propios
- Cada vector propio tiene un valor propio asociado

Considerar que

> El valor propio $\lambda_i$ asociado a la columna $i$ de $W$ corresponde a la "cantidad de varianza" de dicha columna

Para obtener una proyección que disminuya la dimensionalidad necesitamos una matriz $\widehat W \in \mathbb{R}^{D\times \hat D}$. Podemos obtener $\widehat W$ uniendo un subconjunto de las columnas de $W$. En particular nos interesa proyectar a los vectores propios de mayor varianza.

- Para tareas de visualización de datos podemos retener los 2 o 3 componentes principales de mayor varianza
- Para tareas más generales de reducción de dimensionalidad o decorrelación de atributos debemos encontrar una dimensión $\hat D < D$ apropiada
- Un criterio típico es ordenar los vectores propios de mayor a menor y retener aquellos que acumulen un 90% de la varianza
- **Importante:** Si proyectamos con $\widehat W$ la transformación no es invertible porque estamos "descartando" información. Es una compresión de tipo *lossy*

### PCA con `scikit-learn`

PCA está incluido en el módulo [`sklearn.decomposition`](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.decomposition)

El constructor y sus argumentos más importantes son

```python
sklearn.decomposition.PCA(n_components=None, # Cantidad (int) o porcentaje de varianza (float) a retener
                          copy=True, # Si es falso, la data transformada reemplaza la original
                          whiten=False, # Reescala los datos para que tengan igual dispersión
                          ...
                         )
```

Los métodos más importantes de la clase `PCA` son

- `fit(X)`: Calcula la matriz de correlación y los vectores y valores propios
- `transform(X)`: Retorna la proyección de $X$
- `fit_transform(X)`: `fit` y `transform` en un solo paso
- `inverse_transform(hatX)`: Retorna los datos al espacio original (con pérdidas si $\hat D < D$)
- `get_covariance()`: Retorna la matriz de covarianza

Los atributos más importantes son

- `components_`: Los componentes principales (vectores propios)
- `singular_values_`: Los valores propios
- `explained_variance_ratio_`: El porcentaje de varianza asociado a cada vector propio

### Ejemplo

Base de datos con **cuatro atributos numéricos** asociados a las características de un conjunto de 150 flores del género Iris separadas en 3 clases

A continuación se presenta un gráfico de dispersión para visualizar las relaciones entre los atributos

```python
import sklearn.datasets

iris_set = sklearn.datasets.load_iris()
X = iris_set.data
Y = iris_set.target

fig, ax = plt.subplots(3, 3, figsize=(8, 5), tight_layout=True, sharex=True, sharey=True)
for i in range(3):
    for j in range(i, 3):
        for y in range(3):
            ax[i, j].scatter(X[Y==y, i], X[Y==y, j+1], s=10, alpha=0.5)

ax[0 ,0].set_ylabel('Sepal length')
ax[1 ,0].set_ylabel('Sepal width')
ax[2 ,0].set_ylabel('Petal length')
ax[2 ,0].set_xlabel('Sepal width')
ax[2 ,1].set_xlabel('Petal length')
ax[2 ,2].set_xlabel('Petal width');
```

Para hacer una visualización más concisa podemos usar PCA para reducir la dimensión de los datos de 4 a 2

Además podemos estudiar la contribución de cada atributo a los nuevos ejes

```python
import sklearn.decomposition

pca = sklearn.decomposition.PCA(n_components=2)
hatX = pca.fit_transform(X)
hatW = pca.components_

display("Vectores propios de los componentes más relevantes", hatW)
display("Valores propios", pca.singular_values_)
display("Porcentaje de varianza de cada vector propio", pca.explained_variance_ratio_)

fig, ax = plt.subplots(1, 2, figsize=(7, 3), dpi=120, tight_layout=True)
for y, name in enumerate(iris_set.target_names):
    ax[0].scatter(hatX[Y==y, 0], hatX[Y==y, 1], s=10, label=name)
ax[0].legend();
for ax_ in ax:
    ax_.set_ylabel('Vector propio 2'); ax_.set_xlabel('Vector propio 1')
ax[1].plot([0, 0], [-1 ,1], 'k--', alpha=0.5)
ax[1].plot([-0.5, 2], [0, 0],  'k--', alpha=0.5)
for i, name in enumerate(iris_set.feature_names):
    ax[1].arrow(0, 0, hatW[0, i], hatW[1, i], color='b', **arrow_args)
    ax[1].text(hatW[0, i], hatW[1, i], name)
```

**Análisis**

La figura de la izquierda son los datos de iris proyectados en los componentes principales más importantes. la figura de la derecha muestra la contribución de los atributos originales a cada uno de los vectores propios

- Podemos notar que el tipo de flor, es decir las clases de Iris, pueden separarse en el eje del vector propio 1
- Podemos ver también que las variables que tienen que ver con el sépalo están alineadas con el vector propio 2 (ángulo menor)
- En cambio las variables que tienen que ver con el pétalo están alineadas con el vector propio 1
- La clase de la flor tiene mayor relación con el pétalo que con el sépalo

## Clustering con algoritmo K-means

El objetivo general de los métodos de clustering es agrupar los datos en conjuntos (clusters) tal que:

- Los elementos dentro de un mismo cluster sean similares
- Los elementos dentro de un mismo cluster sean distintos a los elementos pertenecientes a otro cluster

esto nos permite revelar estructuras escondidas en los datos y así entenderlos mejor (análisis exploratorio).

Los algoritmos de clustering requieren definir un criterio para medir similitud entre los ejemplos. Lo más típico para datos numéricos es usar métricas de distancia como la [distancia euclidiana](https://es.wikipedia.org/wiki/Distancia_euclidiana) o la [distancia de manhattan/taxista](https://es.wikipedia.org/wiki/Geometr%C3%ADa_del_taxista)

Una vez definida la noción de distancia a utilizar se debe definir un criterio de agrupación. Dependiendo de la combinación entre estas definiciones surgen distintas familias de algoritmos de clustering

- Algoritmos basados en representantes (centroides)
- Algoritmos basados en conectividad y jerarquías
- Algoritmos basados en modelos de densidad

En esta lección nos enfocaremos en un algoritmo clásico de clustering basado en representates: K-medias o *K-means*.

### Formalismo matemático

El algoritmo de K-means requiere que el usuario especifique a priori el número de clusters a detectar, es decir que $K$, el número de clusters es un hiperparámetro del algoritmo. El algoritmo K-means es bastante simple y se resume en el siguiente diagrama esquemático

Detalles:

- Los centroides $\{c_j\}_{j=1,\ldots,K}$ son los centros de los clusters.
- En el paso 2, el cluster asociado a la muestra $i$ se calcula usando la distancia euclidiana

$$
\begin{align}
y_i &= \text{arg}\min_{j=1,\ldots,K} \| x_i - c_j\|_2^2  \nonumber \\
&= \text{arg}\min_{j=1,\ldots,K} \sqrt{ \sum_{d=1}^D (x_{id} - c_{jd})^2}  \nonumber
\end{align}
$$

- En el paso 3 los el centroide $j$ se actualiza como

$$
c_j = \frac{\sum_{i=1}^N \mathbb{1}(y_i = j) x_i}{\sum_{i=1}^N \mathbb{1}(y_i = j)}
$$

donde la función indicadora se define como

$$
\mathbb{1}(y_i = j) = \begin{cases} 1 & \text{si } y_i = j \\ 0 & \text{si } y_i \neq j \end{cases}
$$

es decir que el centroide es el promedio de las muestras asociadas a si mismo

### K-means con `scikit-learn`

K-means está incluido en el módulo [`sklearn.cluster`](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.cluster)

El constructor y sus argumentos más importantes son

```python
sklearn.cluster.KMeans(n_clusters=8, # El número de grupos/clusters
                       init='k-means++', #El algoritmo de inicialización puede ser random o k-means++
                       n_init=10, # El número de inicializaciones que se prueban
                       max_iter=300, # El número máximo de iteraciones
                       tol=0.0001, # Si la diferencia entre los centroides en distintas iteraciones es menor que este valor, el algoritmo se detiene
                       ...
                      )
```

Los métodos más importantes de la clase [`KMeans`](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans) son

- `fit(X)`: Entrena y obtiene los centroides
- `predict(X)`: Retorna el índice del cluster más cercano para cada dato
- `fit_predict(X)`: `fit` y `predict` en un solo paso

Los atributos más importantes son

- `cluster_centers_`: Los centroides de los clusters
- `inertia_`: La suma de errores cuadrados

### Detalles prácticos

- K-means es sensible a los centroides iniciales. Es recomendable probar varias veces con distintas inicializaciones
- La correctitud de los clusters depende fuertemente de $K$. Se recomienda probar con distinto $K$ y evaluar los resultados usando el método del codo o el coeficiente de silueta. Veremos un ejemplo de esto más adelante
- K-means es un algoritmo rápido y que escala bien a datos masivos. Scikit-learn ofrece [`MinibatchKMeans`](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.MiniBatchKMeans.html) para esos casos

**Variantes de K-means**

- Si los datos tiene outliers fuertes se puede reemplazar el promedio en el paso 3 de KMeans por la mediana, obteníendose el algoritmo de las k-medianas
- Si es un requisito que los centroides sean siempre ejemplos del dataset se puede usar el algoritmo de los [k-medoides](https://es.wikipedia.org/wiki/K-medoids)

**Debilidades de KMeans**

K-means asume que los clusters son

- de "forma esférica"
- de igual densidad.

Si esto no se cumple puede ser mejor utilizar algoritmos más generales (y más costosos) como [Modelos de mezcla de gaussianas](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.mixture) o [DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html#sklearn.cluster.DBSCAN). La siguiente figura de scikit-learn muestra el desempeño de distintos algoritmos de clustering ante datos con distinta geometría

Las filas corresponde a datos provenientes de distribuciones

- concéntricas
- con formas no lineales
- de distinto dispersión (densidad y tamaño)
- gaussiana pero no esférica (anisotrópicos)
- gaussian esférico
- uniforme

### Ejemplo

A modo de ejemplo se realiza un clustering con KMeans sobre el dataset Iris.

Para escoger el mejor número de clusters se utiliza el coeficiente de silueta implementado en `sklearn.metrics`

```python
import sklearn.metrics
import sklearn.cluster

iris_set = sklearn.datasets.load_iris()
X = iris_set.data

fig, ax = plt.subplots(4, 2, figsize=(5, 6), dpi=120, tight_layout=True)

for k, n_clusters in enumerate(range(2, 6)):
    # Clustering con kmeans
    kmeans = sklearn.cluster.KMeans(n_clusters=n_clusters)
    labels = kmeans.fit_predict(X)
    # Score de silueta promedio y por ejemplo
    score_promedio = sklearn.metrics.silhouette_score(X, labels)
    score_ejemplos = sklearn.metrics.silhouette_samples(X, labels)
    y_lower = 10
    for cluster in np.unique(labels):
        ax[k, 1].scatter(hatX[labels==cluster, 0], hatX[labels==cluster, 1], s=10)

        scores_cluster_sorted = np.sort(score_ejemplos[labels==cluster])
        y_upper = y_lower + len(scores_cluster_sorted)
        ax[k, 0].fill_betweenx(np.arange(y_lower, y_upper), 0, scores_cluster_sorted, alpha=0.7)
        ax[k, 0].axvline(score_promedio, ls='--', c='k')
        ax[k, 0].set_title(f'K: {n_clusters}')
        ax[k, 0].set_xlim([0, 1])
        y_lower = y_upper + 10
    ax[-1, 0].set_xlabel('Coeficiente de silueta')
```

**Análisis**

La figura de la izquierda muestra los coeficientes de silueta de los ejemplos asociados a cada uno de los clusters. La linea punteada negra corresponde al coeficiente de silueta promedio.

La figura de la derecha muestra un gráfico de dispersión de los datos con dimensionalidad reducida (PCA) donde el color corresponde a los clusters detecteados por KMeans

Tanto el caso de $K=4$ como $K=5$ presentan algunos clusters cuyos coeficientes de silueta son menores que el promedio. Por otro lado el caso $K=2$ presenta alta disparidad den el tamaño de los clusters. En el caso $K=3$ los clusters son de un tamaño más uniforme y todos los clusters tienen ejemplos que superan el coeficiente promedio.

Otra forma de guiar la selección del número de clusters es visualizar el decaimiento de la función de costo (suma de errores cuadráticos) en función del número de clusters

```python
sse = []
n_clusters_test = range(2, 10)
for n_clusters in n_clusters_test:
    kmeans = sklearn.cluster.KMeans(n_clusters=n_clusters)
    kmeans.fit(X)
    sse.append(kmeans.inertia_)

fig, ax = plt.subplots(figsize=(5, 3), dpi=120, tight_layout=True)
ax.plot(n_clusters_test, sse)
ax.set_ylabel('Suma de errores\ncuadráticos')
ax.set_xlabel('K');
```

En este caso el mayor decaimiento en error cuadrático ocurre cuando pasamos de $K=2$ a $K=3$, lo cual indica que $K=3$ es una buena elección.

## ¿Qué aprendimos en esta lección?

- A proyectar datos usando PCA
- A agrupar datos usando KMeans

```python

```


# Design of Digital Filters

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Design of Recursive Filters by the Bilinear Transform

Various techniques have been developed to derive digital realizations of analog systems. For instance the [impulse invariance](https://en.wikipedia.org/wiki/Impulse_invariance) method, the [matched Z-transform](https://en.wikipedia.org/wiki/Matched_Z-transform_method) and the [bilinear transform](https://en.wikipedia.org/wiki/Bilinear_transform). The following section introduces the bilinear transform and its application to the digital realization of analog systems and recursive filter design.

### The Bilinear Transform

The bilinear transform is used to map the transfer function $H(s) = \mathcal{L}\{ h(t) \}$ of a continuous system to the transfer function $H(z) = \mathcal{Z}\{ h[k] \}$ of a discrete system approximating the continuous system. The transform is designed such that if $H(s)$ is a [rational function](https://en.wikipedia.org/wiki/Rational_function) in $s$ then $H(z)$ is a rational function in $z^{-1}$. The coefficients of the powers of $z^{-1}$ are then the coefficients of the digital system.

Assuming ideal sampling with sampling interval $T$, the frequency variable $s$ of the Laplace transformation is linked to the frequency variable $z$ of the $z$-transform by

\begin{equation}
z = \mathrm{e}^{\,s T}
\end{equation}

For sampled signals the resulting mapping from the $s$-plane into the $z$-plane is shown in the following illustration:

The shading indicates how the different areas are mapped. The imaginary axis $s = \mathrm{j}\,\omega$ is mapped onto the unit circle $z = \mathrm{e}^{\,\mathrm{j}\,\Omega}$, representing the frequency response of the continuous and discrete system. The left half-plane of the $s$-plane is mapped into the unit circle of the $z$-plane.

For the desired mapping of $H(s)$ to $H(z)$ we need the inverse of above equation. It is given as

\begin{equation}
s = \frac{1}{T} \cdot \ln (z)
\end{equation}

However, when introduced into a rational transfer function $H(s)$ this non-linear mapping would not result in the desired rational transfer function $H(z)$. In order to achieve the desired mapping, $\ln(z)$ is expanded into the power series

\begin{equation}
\ln(z) = 2 \left( \frac{z-1}{z+1} + \frac{(z-1)^3}{3(z+1)^3} + \frac{(z-1)^5}{5(z+1)^5} + \dots \right)
\end{equation}

Using only the linear term as approximation of $\ln(z)$ yields the bilinear transform

\begin{equation}
s = \frac{2}{T} \cdot \frac{z - 1}{z + 1}
\end{equation}

and its inverse

\begin{equation}
z = \frac{2 + s T}{2 - s T}
\end{equation}

It worthwhile noting that this mapping rule is a special case of a [conformal map](https://en.wikipedia.org/wiki/Conformal_map).

Let's consider the mapping of the frequency response $H(\mathrm{j}\,\omega) = H(s) \big\vert_{s = \mathrm{j}\,\omega}$ of a continuous system to the frequency response $H_d(\mathrm{e}^{\,\mathrm{j}\,\omega T}) = H_d(z)\big\vert_{z = \mathrm{e}^{\,\mathrm{j}\,\Omega}}$ of a discrete system. Introducing the bilinear transform into the continuous system to yield its discrete counterpart results in

\begin{equation}
H_d(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = H \left( \frac{2}{T} \cdot \frac{z - 1}{z + 1} \right) \Bigg\vert_{z = \mathrm{e}^{\,\mathrm{j}\,\Omega}} = H \Big( \mathrm{j} \; \underbrace{\frac{2}{T} \cdot \tan(\frac{\Omega}{2})}_{\omega} \Big)
\end{equation}

The imaginary axis $s=\mathrm{j}\,\omega$ of the $s$-plane is mapped onto the unit circle $\mathrm{e}^{\,\mathrm{j}\,\Omega}$ of the $z$-plane. Note, that for sampled signals the mapping between the continuous frequency axis $\omega$ and the frequency axis $\Omega$ of the discrete system is $\Omega = \omega T$. However, for the bilinear transform the mapping is non-linear

\begin{align}
\omega &= \frac{2}{T} \cdot \tan(\frac{\Omega}{2}) \\
\Omega &= 2 \arctan(\frac{\omega T}{2})
\end{align}

In the following, this is illustrated for $T=1$

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

om = np.linspace(-10, 10, 200)
Om = 2*np.arctan(om*1/2)

plt.figure(figsize=(10, 4))
plt.plot(om, Om, label=r'$2 \cdot \arctan(\frac{\omega T}{2})$')
plt.plot(om, om, 'k--', label=r'$\omega T$')
plt.xlabel(r'$\omega$')
plt.ylabel(r'$\Omega$')
plt.axis([-10, 10, -np.pi, np.pi])
plt.legend(loc=2)
plt.grid()
```

It is evident that the frequency axis deviates from the linear mapping $\Omega = \omega T$, especially for high frequencies. The frequency response of the digital filter $H_d(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ therefore deviates from the desired continuous frequency response $H(\mathrm{j}\,\omega)$. This is due to the first-order approximation of the mapping from the $s$-plane to the $z$-plane. The effect is known as *frequency warping*. It can be considered explicitly in the filter design stage, as shown in the examples.

Besides this drawback, the bilinear transform has a number of benefits:

* Stability and minimum-phase of the continuous filter is preserved. This is due to mapping of the left half-space of the $s$-plane into the unit-circle of the $z$-plane.
* The order of the continuous filter is preserved. This is due to the linear mapping rule.
* No aliasing distortion as for instance observed for the [impulse invariance method](https://en.wikipedia.org/wiki/Impulse_invariance).

The application of the bilinear transform to the design of digital filters is discussed in the following.

### Design of Digital Filter

We aim at designing a digital filter $H_d(z)$ that approximates a given continuous prototype $H(s)$ using the bilinear transform. For instance, the transfer function $H(s)$ may result from the analysis of an analog circuit or filter design technique. The transfer function $H_d(z)$ of the digital filter is then given by

\begin{equation}
H_d(z) = H(s) \Big\vert_{s = \frac{2}{T} \cdot \frac{z - 1}{z + 1}}
\end{equation}

The coefficients of the digital filter are derived by representing the numerator and denominator of $H_d(z)$ as polynomials with respect to $z^{-1}$. For instance, for a continuous system of second order (second order section)

\begin{equation}
H(s) = \frac{\beta_0 + \beta_1 s + \beta_2 s^2}{\alpha_0 + \alpha_1 s + \alpha_2 s^2}
\end{equation}

the bilinear transform results in

\begin{equation}
H_d(z) = \frac{(\beta_2 K^2 - \beta_1 K + \beta_0) \, z^{-2} + (2 \beta_0 - 2 \beta_2 K^2) \,  z^{-1} + (\beta_2 K^2 + \beta_1 K + \beta_0)}
{(\alpha_2 K^2 - \alpha_1 K + \alpha_0) \, z^{-2} + (2 \alpha_0 - 2 \alpha_2 K^2) \,  z^{-1} + (\alpha_2 K^2 + \alpha_1 K + \alpha_0)}
\end{equation}
where $K = \frac{2}{T}$.

As outlined in the previous section, the frequency response of the digital filter $H_d(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ will differ for high frequencies from the desired analog frequency response $H(\mathrm{j}\,\omega)$. For the design of a digital filter from an analog prototype, this can be coped for by replacing corner frequencies with

\begin{equation}
\omega_\text{cw} = \frac{2}{T} \cdot \tan \left(\frac{\omega_\text{c} T}{2} \right)
\end{equation}

where $\omega_\text{cw}$ denotes a *warped* corner frequency $\omega_\text{c}$. This techniques is known as *pre-warping*.

### Examples

The following two examples illustrate the digital realization of an analog system and the design of a recursive filter, respectively.

#### Digital realization of an analog system

A second-order lowpass filter can be realized by the following passive circuit

where $x(t)$ denotes the input and $y(t)$ the output signal (voltage). Analysis of the circuit reveals its transfer function as

\begin{equation}
H(s) = \frac{Y(s)}{X(s)} = \frac{1}{LC \, s^2 + RC \, s + 1}
\end{equation}

Introducing this into the bilinear transform of a second order section (SOS) given in the previous section yields

\begin{equation}
H_d(z) = \frac{T^2 \, z^{-2} + 2 T^2 \, z^{-1} + T^2}{(4 LC - 2 TRC + T^2) \, z^{-2} + (-8 LC + 2 T^2) \, z^{-1} + (4 LC - 2 TRC + T^2)}
\end{equation}

In the following, the frequency response of the analog filter and its digital realization is compared numerically.

```python
import scipy.signal as sig

fs = 44100  # sampling frequency
fc = 1000  # corner frequency of the lowpass

# coefficients of analog lowpass filter
Qinf = 0.8
sinf = 2*np.pi*fc
C = 1e-6
L = 1/(sinf**2*C)
R = sinf*L/Qinf

B = [0, 0, 1]
A = [L*C, R*C, 1]

# cofficients of digital filter
T = 1/fs
b = [T**2, 2*T**2, T**2]
a = [(4*L*C+2*T*R*C+T**2), (-8*L*C+2*T**2), (4*L*C-2*T*R*C+T**2)]

# compute frequency responses
Om, Hd = sig.freqz(b, a, worN=1024)
tmp, H = sig.freqs(B, A, worN=fs*Om)

# plot results
f = Om*fs/(2*np.pi)
plt.figure(figsize=(10, 4))
plt.semilogx(f, 20*np.log10(np.abs(H)),
             label=r'$|H(j \omega)|$ of analog filter')
plt.semilogx(f, 20*np.log10(np.abs(Hd)),
             label=r'$|H_d(e^{j \Omega})|$ of digital filter')
plt.xlabel(r'$f$ in Hz')
plt.ylabel(r'dB')
plt.axis([100, fs/2, -70, 3])
plt.legend()
plt.grid()
```

**Exercise**

* Increase the corner frequency `fc` of the analog filter. What effect does this have on the deviations between the analog filter and its digital representation?

Solution: Increasing the corner frequency leads to larger deviations for the higher frequencies as the width of the stop-band of the filter decreases. These deviations may even lead to a different corner/[cut-off frequency](https://en.wikipedia.org/wiki/Cutoff_frequency) as the attenuation in the transition region between the pass- and the stop-band changes.

#### Design of digital filters from analog prototypes

The design of analog filters is a topic with a long lasting history. Consequently, many analog filter designs are known. For instance

* [Butterworth filters](https://en.wikipedia.org/wiki/Butterworth_filter)
* [Chebyshev_filters](https://en.wikipedia.org/wiki/Chebyshev_filter)
* [Cauer filters](https://en.wikipedia.org/wiki/Elliptic_filter)
* [Bessel filters](https://en.wikipedia.org/wiki/Bessel_filter)

The properties of these designs are well documented and therefore digital realizations are of interest. These can be achieved by the bilinear transform. In Python, the [`scipy.signal`](http://docs.scipy.org/doc/scipy/reference/signal.html) package provides implementations of various analog filter design techniques, as well as an implementation of the bilinear transform.

The design of a Butterworth bandpass using pre-warping is illustrated in the following.

```python
omc = 2*np.pi*np.array([5000, 6000])  # corner frequencies of bandpass
N = 2  # order of filter

# pre-warping of corner frequencies
omcp = 2*fs*np.tan(omc/(2*fs))

# design of analog filters with and without pre-warping
B, A = sig.butter(N, omc, btype='bandpass', analog=True)
Bp, Ap = sig.butter(N, omcp, btype='bandpass', analog=True)

# bilinear transform of analog prototypes
b, a = sig.bilinear(B, A, fs)
bp, ap = sig.bilinear(Bp, Ap, fs)

# compute frequency responses
Om, Hdp = sig.freqz(bp, ap, worN=1024)
Om, Hd = sig.freqz(b, a, worN=1024)
tmp, H = sig.freqs(B, A, worN=fs*Om)

# plot results
np.seterr(divide='ignore')
f = Om*fs/(2*np.pi)
plt.figure(figsize=(12, 8))
plt.semilogx(f, 20*np.log10(np.abs(H)),
             label=r'$|H(j \omega)|$ of analog prototype')
plt.semilogx(f, 20*np.log10(np.abs(Hd)),
             label=r'$|H_d(e^{j \Omega})|$ of digital filter without pre-warping')
plt.semilogx(f, 20*np.log10(np.abs(Hdp)),
             label=r'$|H_d(e^{j \Omega})|$ of digital filter with pre-warping')
plt.xlabel(r'$f$ in Hz')
plt.ylabel(r'dB')
plt.axis([100, fs/2, -70, 3])
plt.legend()
plt.grid()
```

**Exercise**

* What is improved by pre-warping?
* Change the corner frequencies `omc` of the analog prototype and examine the deviations from the analog prototype. When is pre-warping beneficial and when not?

Solution: The location and width of the pass-band of the digital filter is closer to the desired pass-band when pre-warping is used in the design process. Pre-warping is especially beneficial if the desired pass-band is located at higher frequencies, as the deviations caused by the bilinear transformation are less prominent for lower frequencies.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples*.


```python
from math import *
#Semi-Major Axis
a = 10424.1
#Semi-Minor Axis
b = 9579.42
#Eccentricity
ecc = sqrt(1 - (b**2)/(a**2))
print(ecc)
```

    0.39433084466244295

```python
#Circumference
def ellipse_circum(acc=5):
    temp = []
    S_a = 2*pi*a
    S_b = 1
    for x in range(2, acc):
        S_n_a = 2/sqrt(pi)
        S_n_b = (ecc**(2*x-2) * x)/(4*x**2 - 8*x + 3)
        S_n_c = (gamma(0.5 + x))/(factorial(x))
        temp.append(-(S_n_a*S_n_b*S_n_c)/2)

    print(temp)

    return S_a*(S_b + sum(temp))

```

```python
ellipse_circum(10)
```

    [-0.03887420376304893, -0.0015112037182110469, -0.00011749368253842683, -1.1418683389674473e-05, -1.242898229428654e-06, -1.4495003708262662e-07, -1.7709425727750508e-08, -2.237429428868252e-09]

    62842.91149982586

```python
from sympy import *
```

```python
n, A, B, E = Symbol('n'),Symbol('A'),Symbol('B'),Symbol('E')
```

```python
#Circumference
def ellipse_circum_sympy(acc=5):
    temp = []
    S_a = 2*pi*A
    S_b = 1
    for x in range(2, acc):
        S_n_a = 2/sqrt(pi)
        S_n_b = (sqrt(1 - (B**2)/(A**2))**(2*x-2) * x)/(4*x**2 - 8*x + 3)
        S_n_c = (gamma(0.5 + x))/(factorial(x))
        temp.append(-(S_n_a*S_n_b*S_n_c)/2)

    print(temp)

    return S_a*(S_b + sum(temp))
```

```python
#N=3
factor(ellipse_circum_sympy(3))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi)]

$\displaystyle \frac{2.0 \sqrt{\pi} \left(- 0.443113462726379 A^{2} + 1.0 \sqrt{\pi} A^{2} + 0.443113462726379 B^{2}\right)}{A}$

```python
simplify(ellipse_circum_sympy(3))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi)]

$\displaystyle - 0.886226925452758 \sqrt{\pi} A + 2 \pi A + \frac{0.886226925452758 \sqrt{\pi} B^{2}}{A}$

```python
#N=4
factor(ellipse_circum_sympy(4))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi), -0.110778365681595*(1 - B**2/A**2)**2/sqrt(pi)]

$\displaystyle \frac{2.0 \sqrt{\pi} \left(- 0.553891828407974 A^{4} + 1.0 \sqrt{\pi} A^{4} + 0.664670194089569 A^{2} B^{2} - 0.110778365681595 B^{4}\right)}{A^{3}}$

```python
simplify(ellipse_circum_sympy(4))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi), -0.110778365681595*(1 - B**2/A**2)**2/sqrt(pi)]

$\displaystyle \frac{2 \pi A^{4} + 0.886226925452758 \sqrt{\pi} A^{2} \left(- A^{2} + B^{2}\right) - 0.22155673136319 \sqrt{\pi} \left(A^{2} - B^{2}\right)^{2}}{A^{3}}$

```python
#N=5
factor(ellipse_circum_sympy(5))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi), -0.110778365681595*(1 - B**2/A**2)**2/sqrt(pi), -0.0553891828407974*(1 - B**2/A**2)**3/sqrt(pi)]

$\displaystyle \frac{2.0 \sqrt{\pi} \left(- 0.609281011248771 A^{6} + 1.0 \sqrt{\pi} A^{6} + 0.830837742611961 A^{4} B^{2} - 0.276945914203987 A^{2} B^{4} + 0.0553891828407974 B^{6}\right)}{A^{5}}$

```python
simplify(ellipse_circum_sympy(5))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi), -0.110778365681595*(1 - B**2/A**2)**2/sqrt(pi), -0.0553891828407974*(1 - B**2/A**2)**3/sqrt(pi)]

$\displaystyle \frac{1.0 \left(- 1.21856202249754 \pi A^{6} + 2.0 \pi^{\frac{3}{2}} A^{6} + 1.66167548522392 \pi A^{4} B^{2} - 0.553891828407974 \pi A^{2} B^{4} + 0.110778365681595 \pi B^{6}\right)}{\sqrt{\pi} A^{5}}$

```python
simplify(ellipse_circum_sympy(50))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi), -0.110778365681595*(1 - B**2/A**2)**2/sqrt(pi), -0.0553891828407974*(1 - B**2/A**2)**3/sqrt(pi), -0.0346182392754984*(1 - B**2/A**2)**4/sqrt(pi), -0.0242327674928489*(1 - B**2/A**2)**5/sqrt(pi), -0.0181745756196366*(1 - B**2/A**2)**6/sqrt(pi), -0.0142800237011431*(1 - B**2/A**2)**7/sqrt(pi), -0.0116025192571787*(1 - B**2/A**2)**8/sqrt(pi), -0.00966876604764896*(1 - B**2/A**2)**9/sqrt(pi), -0.00821845114050161*(1 - B**2/A**2)**10/sqrt(pi), -0.00709775325770594*(1 - B**2/A**2)**11/sqrt(pi), -0.0062105341004927*(1 - B**2/A**2)**12/sqrt(pi), -0.00549393401197431*(1 - B**2/A**2)**13/sqrt(pi), -0.00490529822497706*(1 - B**2/A**2)**14/sqrt(pi), -0.00441476840247935*(1 - B**2/A**2)**15/sqrt(pi), -0.00400088386474691*(1 - B**2/A**2)**16/sqrt(pi), -0.00364786470021042*(1 - B**2/A**2)**17/sqrt(pi), -0.00334387597519289*(1 - B**2/A**2)**18/sqrt(pi), -0.00307988576662503*(1 - B**2/A**2)**19/sqrt(pi), -0.00284889433412815*(1 - B**2/A**2)**20/sqrt(pi), -0.00264540188169043*(1 - B**2/A**2)**21/sqrt(pi), -0.00246503357157517*(1 - B**2/A**2)**22/sqrt(pi), -0.0023042705125594*(1 - B**2/A**2)**23/sqrt(pi), -0.00216025360552443*(1 - B**2/A**2)**24/sqrt(pi), -0.00203063838919297*(1 - B**2/A**2)**25/sqrt(pi), -0.00191348617443184*(1 - B**2/A**2)**26/sqrt(pi), -0.0018071813869634*(1 - B**2/A**2)**27/sqrt(pi), -0.00171036809837608*(1 - B**2/A**2)**28/sqrt(pi), -0.00162190078294283*(1 - B**2/A**2)**29/sqrt(pi), -0.00154080574379569*(1 - B**2/A**2)**30/sqrt(pi), -0.00146625062716041*(1 - B**2/A**2)**31/sqrt(pi), -0.00139752012901227*(1 - B**2/A**2)**32/sqrt(pi), -0.00133399648678444*(1 - B**2/A**2)**33/sqrt(pi), -0.00127514370060277*(1 - B**2/A**2)**34/sqrt(pi), -0.00122049468486265*(1 - B**2/A**2)**35/sqrt(pi), -0.00116964073966004*(1 - B**2/A**2)**36/sqrt(pi), -0.00112222287183599*(1 - B**2/A**2)**37/sqrt(pi), -0.0010779246005793*(1 - B**2/A**2)**38/sqrt(pi), -0.00103646596209548*(1 - B**2/A**2)**39/sqrt(pi), -0.000997598488516904*(1 - B**2/A**2)**40/sqrt(pi), -0.000961100982839456*(1 - B**2/A**2)**41/sqrt(pi), -0.000926775947738047*(1 - B**2/A**2)**42/sqrt(pi), -0.000894446554212301*(1 - B**2/A**2)**43/sqrt(pi), -0.000863954058045972*(1 - B**2/A**2)**44/sqrt(pi), -0.00083515558944444*(1 - B**2/A**2)**45/sqrt(pi), -0.000807922255006034*(1 - B**2/A**2)**46/sqrt(pi), -0.000782137502186693*(1 - B**2/A**2)**47/sqrt(pi), -0.000757695705243359*(1 - B**2/A**2)**48/sqrt(pi)]

$\displaystyle - 1.62849166690928 \sqrt{\pi} A + 2.0 \pi A + \frac{6.91018483181943 \sqrt{\pi} B^{2}}{A} - \frac{54.1297811825855 \sqrt{\pi} B^{4}}{A^{3}} + \frac{497.993986879787 \sqrt{\pi} B^{6}}{A^{5}} - \frac{4001.73739456972 \sqrt{\pi} B^{8}}{A^{7}} + \frac{27389.6692783883 \sqrt{\pi} B^{10}}{A^{9}} - \frac{160603.060768731 \sqrt{\pi} B^{12}}{A^{11}} + \frac{815369.385441251 \sqrt{\pi} B^{14}}{A^{13}} - \frac{3621599.02033489 \sqrt{\pi} B^{16}}{A^{15}} + \frac{14202349.0993525 \sqrt{\pi} B^{18}}{A^{17}} - \frac{49558723.4361617 \sqrt{\pi} B^{20}}{A^{19}} + \frac{154897828.229042 \sqrt{\pi} B^{22}}{A^{21}} - \frac{436071059.905673 \sqrt{\pi} B^{24}}{A^{23}} + \frac{1110974884.92891 \sqrt{\pi} B^{26}}{A^{25}} - \frac{2571701122.52064 \sqrt{\pi} B^{28}}{A^{27}} + \frac{5427176162.00906 \sqrt{\pi} B^{30}}{A^{29}} - \frac{10471386264.1989 \sqrt{\pi} B^{32}}{A^{31}} + \frac{18516248082.1486 \sqrt{\pi} B^{34}}{A^{33}} - \frac{30066859981.0127 \sqrt{\pi} B^{36}}{A^{35}} + \frac{44907827852.1526 \sqrt{\pi} B^{38}}{A^{37}} - \frac{61777050365.8459 \sqrt{\pi} B^{40}}{A^{39}} + \frac{78351380951.8045 \sqrt{\pi} B^{42}}{A^{41}} - \frac{91686024010.1983 \sqrt{\pi} B^{44}}{A^{43}} + \frac{99038623037.1032 \sqrt{\pi} B^{46}}{A^{45}} - \frac{98775222443.9193 \sqrt{\pi} B^{48}}{A^{47}} + \frac{90953837483.0539 \sqrt{\pi} B^{50}}{A^{49}} - \frac{77303902596.8339 \sqrt{\pi} B^{52}}{A^{51}} + \frac{60611445641.962 \sqrt{\pi} B^{54}}{A^{53}} - \frac{43805544804.8726 \sqrt{\pi} B^{56}}{A^{55}} + \frac{29150695272.45 \sqrt{\pi} B^{58}}{A^{57}} - \frac{17836272870.0923 \sqrt{\pi} B^{60}}{A^{59}} + \frac{10016986667.3918 \sqrt{\pi} B^{62}}{A^{61}} - \frac{5152586891.90741 \sqrt{\pi} B^{64}}{A^{63}} + \frac{2421355518.43481 \sqrt{\pi} B^{66}}{A^{65}} - \frac{1036357168.77697 \sqrt{\pi} B^{68}}{A^{67}} + \frac{402527132.220621 \sqrt{\pi} B^{70}}{A^{69}} - \frac{141262456.025781 \sqrt{\pi} B^{72}}{A^{71}} + \frac{44559649.2165737 \sqrt{\pi} B^{74}}{A^{73}} - \frac{12554876.6038276 \sqrt{\pi} B^{76}}{A^{75}} + \frac{3135583.56738952 \sqrt{\pi} B^{78}}{A^{77}} - \frac{687645.383607891 \sqrt{\pi} B^{80}}{A^{79}} + \frac{130861.753218966 \sqrt{\pi} B^{82}}{A^{81}} - \frac{21284.7429934463 \sqrt{\pi} B^{84}}{A^{83}} + \frac{2900.08262947641 \sqrt{\pi} B^{86}}{A^{85}} - \frac{321.978870827449 \sqrt{\pi} B^{88}}{A^{87}} + \frac{27.9771902741453 \sqrt{\pi} B^{90}}{A^{89}} - \frac{1.78449828074458 \sqrt{\pi} B^{92}}{A^{91}} + \frac{0.0743030627077358 \sqrt{\pi} B^{94}}{A^{93}} - \frac{0.00151539141048672 \sqrt{\pi} B^{96}}{A^{95}}$

```python
factor(simplify(ellipse_circum_sympy(100)))
```

    [-0.664670194089569*(2/3 - 2*B**2/(3*A**2))/sqrt(pi), -0.110778365681595*(1 - B**2/A**2)**2/sqrt(pi), -0.0553891828407974*(1 - B**2/A**2)**3/sqrt(pi), -0.0346182392754984*(1 - B**2/A**2)**4/sqrt(pi), -0.0242327674928489*(1 - B**2/A**2)**5/sqrt(pi), -0.0181745756196366*(1 - B**2/A**2)**6/sqrt(pi), -0.0142800237011431*(1 - B**2/A**2)**7/sqrt(pi), -0.0116025192571787*(1 - B**2/A**2)**8/sqrt(pi), -0.00966876604764896*(1 - B**2/A**2)**9/sqrt(pi), -0.00821845114050161*(1 - B**2/A**2)**10/sqrt(pi), -0.00709775325770594*(1 - B**2/A**2)**11/sqrt(pi), -0.0062105341004927*(1 - B**2/A**2)**12/sqrt(pi), -0.00549393401197431*(1 - B**2/A**2)**13/sqrt(pi), -0.00490529822497706*(1 - B**2/A**2)**14/sqrt(pi), -0.00441476840247935*(1 - B**2/A**2)**15/sqrt(pi), -0.00400088386474691*(1 - B**2/A**2)**16/sqrt(pi), -0.00364786470021042*(1 - B**2/A**2)**17/sqrt(pi), -0.00334387597519289*(1 - B**2/A**2)**18/sqrt(pi), -0.00307988576662503*(1 - B**2/A**2)**19/sqrt(pi), -0.00284889433412815*(1 - B**2/A**2)**20/sqrt(pi), -0.00264540188169043*(1 - B**2/A**2)**21/sqrt(pi), -0.00246503357157517*(1 - B**2/A**2)**22/sqrt(pi), -0.0023042705125594*(1 - B**2/A**2)**23/sqrt(pi), -0.00216025360552443*(1 - B**2/A**2)**24/sqrt(pi), -0.00203063838919297*(1 - B**2/A**2)**25/sqrt(pi), -0.00191348617443184*(1 - B**2/A**2)**26/sqrt(pi), -0.0018071813869634*(1 - B**2/A**2)**27/sqrt(pi), -0.00171036809837608*(1 - B**2/A**2)**28/sqrt(pi), -0.00162190078294283*(1 - B**2/A**2)**29/sqrt(pi), -0.00154080574379569*(1 - B**2/A**2)**30/sqrt(pi), -0.00146625062716041*(1 - B**2/A**2)**31/sqrt(pi), -0.00139752012901227*(1 - B**2/A**2)**32/sqrt(pi), -0.00133399648678444*(1 - B**2/A**2)**33/sqrt(pi), -0.00127514370060277*(1 - B**2/A**2)**34/sqrt(pi), -0.00122049468486265*(1 - B**2/A**2)**35/sqrt(pi), -0.00116964073966004*(1 - B**2/A**2)**36/sqrt(pi), -0.00112222287183599*(1 - B**2/A**2)**37/sqrt(pi), -0.0010779246005793*(1 - B**2/A**2)**38/sqrt(pi), -0.00103646596209548*(1 - B**2/A**2)**39/sqrt(pi), -0.000997598488516904*(1 - B**2/A**2)**40/sqrt(pi), -0.000961100982839456*(1 - B**2/A**2)**41/sqrt(pi), -0.000926775947738047*(1 - B**2/A**2)**42/sqrt(pi), -0.000894446554212301*(1 - B**2/A**2)**43/sqrt(pi), -0.000863954058045972*(1 - B**2/A**2)**44/sqrt(pi), -0.00083515558944444*(1 - B**2/A**2)**45/sqrt(pi), -0.000807922255006034*(1 - B**2/A**2)**46/sqrt(pi), -0.000782137502186693*(1 - B**2/A**2)**47/sqrt(pi), -0.000757695705243359*(1 - B**2/A**2)**48/sqrt(pi), -0.000734500938756317*(1 - B**2/A**2)**49/sqrt(pi), -0.000712465910593628*(1 - B**2/A**2)**50/sqrt(pi), -0.000691511030870286*(1 - B**2/A**2)**51/sqrt(pi), -0.000671563597287489*(1 - B**2/A**2)**52/sqrt(pi), -0.000652557080383126*(1 - B**2/A**2)**53/sqrt(pi), -0.000634430494816928*(1 - B**2/A**2)**54/sqrt(pi), -0.000617127844958285*(1 - B**2/A**2)**55/sqrt(pi), -0.000600597634825473*(1 - B**2/A**2)**56/sqrt(pi), -0.000584792433909014*(1 - B**2/A**2)**57/sqrt(pi), -0.000569668491652746*(1 - B**2/A**2)**58/sqrt(pi), -0.000555185394407337*(1 - B**2/A**2)**59/sqrt(pi), -0.000541305759547154*(1 - B**2/A**2)**60/sqrt(pi), -0.00052799496218124*(1 - B**2/A**2)**61/sqrt(pi), -0.000515220890515565*(1 - B**2/A**2)**62/sqrt(pi), -0.000502953726455671*(1 - B**2/A**2)**63/sqrt(pi), -0.000491165748491866*(1 - B**2/A**2)**64/sqrt(pi), -0.0004798311542959*(1 - B**2/A**2)**65/sqrt(pi), -0.000468925900789175*(1 - B**2/A**2)**66/sqrt(pi), -0.00045842755972673*(1 - B**2/A**2)**67/sqrt(pi), -0.0004483151870857*(1 - B**2/A**2)**68/sqrt(pi), -0.00043856920475775*(1 - B**2/A**2)**69/sqrt(pi), -0.000429171293227226*(1 - B**2/A**2)**70/sqrt(pi), -0.000420104294074539*(1 - B**2/A**2)**71/sqrt(pi), -0.000411352121281319*(1 - B**2/A**2)**72/sqrt(pi), -0.000402899680433073*(1 - B**2/A**2)**73/sqrt(pi), -0.000394732795018889*(1 - B**2/A**2)**74/sqrt(pi), -0.000386838139118511*(1 - B**2/A**2)**75/sqrt(pi), -0.000379203175846435*(1 - B**2/A**2)**76/sqrt(pi), -0.000371816100992284*(1 - B**2/A**2)**77/sqrt(pi), -0.000364665791357817*(1 - B**2/A**2)**78/sqrt(pi), -0.000357741757344694*(1 - B**2/A**2)**79/sqrt(pi), -0.000351034099394481*(1 - B**2/A**2)**80/sqrt(pi), -0.000344533467924213*(1 - B**2/A**2)**81/sqrt(pi), -0.000338231026437794*(1 - B**2/A**2)**82/sqrt(pi), -0.000332118417526268*(1 - B**2/A**2)**83/sqrt(pi), -0.000326187731499013*(1 - B**2/A**2)**84/sqrt(pi), -0.000320431477413736*(1 - B**2/A**2)**85/sqrt(pi), -0.000314842556296055*(1 - B**2/A**2)**86/sqrt(pi), -0.000309414236359916*(1 - B**2/A**2)**87/sqrt(pi), -0.000304140130058326*(1 - B**2/A**2)**88/sqrt(pi), -0.000299014172810152*(1 - B**2/A**2)**89/sqrt(pi), -0.000294030603263316*(1 - B**2/A**2)**90/sqrt(pi), -0.000289183944967767*(1 - B**2/A**2)**91/sqrt(pi), -0.000284468989343293*(1 - B**2/A**2)**92/sqrt(pi), -0.000279880779837756*(1 - B**2/A**2)**93/sqrt(pi), -0.00027541459718077*(1 - B**2/A**2)**94/sqrt(pi), -0.000271065945646337*(1 - B**2/A**2)**95/sqrt(pi), -0.000266830540245613*(1 - B**2/A**2)**96/sqrt(pi), -0.000262704294777898*(1 - B**2/A**2)**97/sqrt(pi), -0.000258683310674154*(1 - B**2/A**2)**98/sqrt(pi)]

$\displaystyle \frac{2.65151575361855 \cdot 10^{25} \left(- 6.30419546805028 \cdot 10^{-26} \sqrt{\pi} A^{196} + 7.54285543003312 \cdot 10^{-26} \pi A^{196} + 3.72876386665758 \cdot 10^{-25} \sqrt{\pi} A^{194} B^{2} - 6.02816825109642 \cdot 10^{-24} \sqrt{\pi} A^{192} B^{4} + 1.15740830421051 \cdot 10^{-22} \sqrt{\pi} A^{190} B^{6} - 1.96346051607141 \cdot 10^{-21} \sqrt{\pi} A^{188} B^{8} + 2.87101559905552 \cdot 10^{-20} \sqrt{\pi} A^{186} B^{10} - 3.64096978243859 \cdot 10^{-19} \sqrt{\pi} A^{184} B^{12} + 4.04907848332731 \cdot 10^{-18} \sqrt{\pi} A^{182} B^{14} - 3.99171653814684 \cdot 10^{-17} \sqrt{\pi} A^{180} B^{16} + 3.52210282777663 \cdot 10^{-16} \sqrt{\pi} A^{178} B^{18} - 2.80470609390844 \cdot 10^{-15} \sqrt{\pi} A^{176} B^{20} + 2.03007298225754 \cdot 10^{-14} \sqrt{\pi} A^{174} B^{22} - 1.34382005021178 \cdot 10^{-13} \sqrt{\pi} A^{172} B^{24} + 8.1786955671351 \cdot 10^{-13} \sqrt{\pi} A^{170} B^{26} - 4.59781166141854 \cdot 10^{-12} \sqrt{\pi} A^{168} B^{28} + 2.3972038731258 \cdot 10^{-11} \sqrt{\pi} A^{166} B^{30} - 1.16332050859149 \cdot 10^{-10} \sqrt{\pi} A^{164} B^{32} + 5.27123125283347 \cdot 10^{-10} \sqrt{\pi} A^{162} B^{34} - 2.23650811727363 \cdot 10^{-9} \sqrt{\pi} A^{160} B^{36} + 8.90785594362186 \cdot 10^{-9} \sqrt{\pi} A^{158} B^{38} - 3.33816191323163 \cdot 10^{-8} \sqrt{\pi} A^{156} B^{40} + 1.17940633450065 \cdot 10^{-7} \sqrt{\pi} A^{154} B^{42} - 3.93592579071729 \cdot 10^{-7} \sqrt{\pi} A^{152} B^{44} + 1.24276381488542 \cdot 10^{-6} \sqrt{\pi} A^{150} B^{46} - 3.71837577592047 \cdot 10^{-6} \sqrt{\pi} A^{148} B^{48} + 1.05571517948175 \cdot 10^{-5} \sqrt{\pi} A^{146} B^{50} - 2.84788325566079 \cdot 10^{-5} \sqrt{\pi} A^{144} B^{52} + 7.30777590131826 \cdot 10^{-5} \sqrt{\pi} A^{142} B^{54} - 0.000178565978679614 \sqrt{\pi} A^{140} B^{56} + 0.000415897772484281 \sqrt{\pi} A^{138} B^{58} - 0.000924138948689647 \sqrt{\pi} A^{136} B^{60} + 0.00196067977902848 \sqrt{\pi} A^{134} B^{62} - 0.003974850325838 \sqrt{\pi} A^{132} B^{64} + 0.00770509447777828 \sqrt{\pi} A^{130} B^{66} - 0.0142906164041322 \sqrt{\pi} A^{128} B^{68} + 0.0253739805966539 \sqrt{\pi} A^{126} B^{70} - 0.0431536360147317 \sqrt{\pi} A^{124} B^{72} + 0.0703303612502218 \sqrt{\pi} A^{122} B^{74} - 0.109888104788504 \sqrt{\pi} A^{120} B^{76} + 0.164667489693064 \sqrt{\pi} A^{118} B^{78} - 0.236735571416325 \sqrt{\pi} A^{116} B^{80} + 0.326625229819212 \sqrt{\pi} A^{114} B^{82} - 0.43259572176572 \sqrt{\pi} A^{112} B^{84} + 0.55012446368456 \sqrt{\pi} A^{110} B^{86} - 0.671847405361891 \sqrt{\pi} A^{108} B^{88} + 0.788099653031252 \sqrt{\pi} A^{106} B^{90} - 0.888071204813286 \sqrt{\pi} A^{104} B^{92} + 0.961416824794434 \sqrt{\pi} A^{102} B^{94} - 1.0 \sqrt{\pi} A^{100} B^{96} + 0.999368819692825 \sqrt{\pi} A^{98} B^{98} - 0.959595959595959 \sqrt{\pi} A^{96} B^{100} + 0.885264997087944 \sqrt{\pi} A^{94} B^{102} - 0.784606598427273 \sqrt{\pi} A^{92} B^{104} + 0.668008277331252 \sqrt{\pi} A^{90} B^{106} - 0.546268451088641 \sqrt{\pi} A^{88} B^{108} + 0.428996141405391 \sqrt{\pi} A^{86} B^{110} - 0.323472476635629 \sqrt{\pi} A^{84} B^{112} + 0.234129589516426 \sqrt{\pi} A^{82} B^{114} - 0.162627044712084 \sqrt{\pi} A^{80} B^{116} + 0.108370912020221 \sqrt{\pi} A^{78} B^{118} - 0.0692572089003179 \sqrt{\pi} A^{76} B^{120} + 0.0424307138121173 \sqrt{\pi} A^{74} B^{122} - 0.0249098224150077 \sqrt{\pi} A^{72} B^{124} + 0.0140064372893529 \sqrt{\pi} A^{70} B^{126} - 0.00753914408721937 \sqrt{\pi} A^{68} B^{128} + 0.00388241194616735 \sqrt{\pi} A^{66} B^{130} - 0.00191156924067018 \sqrt{\pi} A^{64} B^{132} + 0.000899259146772462 \sqrt{\pi} A^{62} B^{134} - 0.000403882947945846 \sqrt{\pi} A^{60} B^{136} + 0.000173037759354774 \sqrt{\pi} A^{58} B^{138} - 7.06556030746676 \cdot 10^{-5} \sqrt{\pi} A^{56} B^{140} + 2.74689448772956 \cdot 10^{-5} \sqrt{\pi} A^{54} B^{142} - 1.01567864362728 \cdot 10^{-5} \sqrt{\pi} A^{52} B^{144} + 3.56758922721418 \cdot 10^{-6} \sqrt{\pi} A^{50} B^{146} - 1.18886844536233 \cdot 10^{-6} \sqrt{\pi} A^{48} B^{148} + 3.75331353488886 \cdot 10^{-7} \sqrt{\pi} A^{46} B^{150} - 1.12082654967446 \cdot 10^{-7} \sqrt{\pi} A^{44} B^{152} + 3.16050063493639 \cdot 10^{-8} \sqrt{\pi} A^{42} B^{154} - 8.39924610426022 \cdot 10^{-9} \sqrt{\pi} A^{40} B^{156} + 2.09930363002553 \cdot 10^{-9} \sqrt{\pi} A^{38} B^{158} - 4.92313107575956 \cdot 10^{-10} \sqrt{\pi} A^{36} B^{160} + 1.08043870399692 \cdot 10^{-10} \sqrt{\pi} A^{34} B^{162} - 2.2124500470145 \cdot 10^{-11} \sqrt{\pi} A^{32} B^{164} + 4.21326741337263 \cdot 10^{-12} \sqrt{\pi} A^{30} B^{166} - 7.4335877160659 \cdot 10^{-13} \sqrt{\pi} A^{28} B^{168} + 1.20986620815608 \cdot 10^{-13} \sqrt{\pi} A^{26} B^{170} - 1.80747726051877 \cdot 10^{-14} \sqrt{\pi} A^{24} B^{172} + 2.46425044089065 \cdot 10^{-15} \sqrt{\pi} A^{22} B^{174} - 3.04510947338631 \cdot 10^{-16} \sqrt{\pi} A^{20} B^{176} + 3.38281062554817 \cdot 10^{-17} \sqrt{\pi} A^{18} B^{178} - 3.34501385878227 \cdot 10^{-18} \sqrt{\pi} A^{16} B^{180} + 2.90817791620194 \cdot 10^{-19} \sqrt{\pi} A^{14} B^{182} - 2.18856107141118 \cdot 10^{-20} \sqrt{\pi} A^{12} B^{184} + 1.39671029143242 \cdot 10^{-21} \sqrt{\pi} A^{10} B^{186} - 7.34985219919777 \cdot 10^{-23} \sqrt{\pi} A^{8} B^{188} + 3.06192673071564 \cdot 10^{-24} \sqrt{\pi} A^{6} B^{190} - 9.46832709596296 \cdot 10^{-26} \sqrt{\pi} A^{4} B^{192} + 1.93200200344952 \cdot 10^{-27} \sqrt{\pi} A^{2} B^{194} - 1.95121081457749 \cdot 10^{-29} \sqrt{\pi} B^{196}\right)}{A^{195}}$

```python

```


# Symbolic computation of the asymptotic expansion for a disk with constant $\varepsilon_{\mathsf{c}}$

The codes below are associated to the article:

- C. Carvalho and Z. Moitier, _Asymptotics for metamaterial cavities and their effect on scattering_ [[arXiv](https://arxiv.org/abs/2010.07583), [HAL](https://hal.archives-ouvertes.fr/hal-02965993)]

## Zoïs Moitier, Camille Carvalho (2021)

_Karlsruhe Institute of Technology, Germany_

_University of California Merced, USA_

```python
import sympy as sy
from IPython.display import display

from core import *
```

```python
order = 1
q = order - 1
```

```python
h, η = sy.symbols("h η", real=True, positive=True)
σ = sy.symbols("σ", real=True)
```

```python
Λn = sy.symbols(f"Λ_0:{order}", real=True)
Λ = formal_expan(Λn, h, order)

φn = sy.symbols(f"φ_0:{order}", cls=sy.Function)
φ = formal_expan(apply_var(φn, σ), h, order)

ψn = sy.symbols(f"ψ_0:{order}", cls=sy.Function)
ψ = formal_expan(apply_var(ψn, σ), h, order)
```

```python
def dσ_neg(expr):
    return expr.diff(σ)

def dσ_pos(expr):
    return expr.diff(σ)
```

```python
iε = inv_ε_expan((η,), σ, h, order)
f_neg, 𝓛_neg = op_disk(iε, dσ_neg, σ, h, order)
f_pos, 𝓛_pos = op_disk(1, dσ_pos, σ, h, order)
```

## $\mathcal{L}^- \varphi = \Lambda \varphi$

```python
display(
    sy.Eq(
        𝓛_neg.subs(f_neg(σ), φ).doit().expand().coeff(h, q),
        (Λ * φ).expand().coeff(h, q),
    )
)
```

$\displaystyle - \frac{\operatorname{φ_{0}}{\left(σ \right)}}{η^{2}} + \frac{\frac{d^{2}}{d σ^{2}} \operatorname{φ_{0}}{\left(σ \right)}}{η^{2}} = Λ_{0} \operatorname{φ_{0}}{\left(σ \right)}$

## $\mathcal{L}^+ \varphi = \Lambda \varphi$

```python
display(
    sy.Eq(
        𝓛_pos.subs(f_pos(σ), ψ).doit().expand().coeff(h, q),
        (Λ * ψ).expand().coeff(h, q),
    )
)
```

$\displaystyle \operatorname{ψ_{0}}{\left(σ \right)} - \frac{d^{2}}{d σ^{2}} \operatorname{ψ_{0}}{\left(σ \right)} = Λ_{0} \operatorname{ψ_{0}}{\left(σ \right)}$

## $[u] = 0$

```python
display(sy.Eq(φ.subs(σ, 0).coeff(h, q), ψ.subs(σ, 0).coeff(h, q)))
```

$\displaystyle \operatorname{φ_{0}}{\left(0 \right)} = \operatorname{ψ_{0}}{\left(0 \right)}$

## $ [\varepsilon^{-1} \partial u] = 0 $

```python
display(
    sy.Eq(
        -(η ** (-2)) * dσ_neg(φ).expand().coeff(h, q).subs(σ, 0),
        dσ_pos(ψ).subs(σ, 0).coeff(h, q),
    )
)
```

$\displaystyle - \frac{\left. \frac{d}{d σ} \operatorname{φ_{0}}{\left(σ \right)} \right|_{\substack{ σ=0 }}}{η^{2}} = \left. \frac{d}{d σ} \operatorname{ψ_{0}}{\left(σ \right)} \right|_{\substack{ σ=0 }}$


# Linear 2D solution

## Init symbols for *sympy*

```python
from sympy import *
from geom_util import *
from sympy.vector import CoordSys3D
import matplotlib.pyplot as plt
import sys
sys.path.append("../")

%matplotlib inline

%reload_ext autoreload
%autoreload 2
%aimport geom_util
```

```python
# Any tweaks that normally go in .matplotlibrc, etc., should explicitly go here
%config InlineBackend.figure_format='retina'
plt.rcParams['figure.figsize'] = (12, 12)

plt.rc('text', usetex=True)

plt.rc('font', family='serif')

init_printing()
```

```python
N = CoordSys3D('N')
alpha1, alpha2, alpha3 = symbols("alpha_1 alpha_2 alpha_3", real = True, positive=True)
A,K,rho = symbols("A K rho")
```

```python
B=Matrix([[0, 1/(A*(K*alpha3 + 1)), 0, 0, 0, 0, 0, 0, K/(K*alpha3 + 1), 0, 0, 0], [0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 1/(A*(K*alpha3 + 1)), 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], [-K/(K*alpha3 + 1), 0, 0, 0, 0, 0, 0, 0, 0, 1/(A*(K*alpha3 + 1)), 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]])
B
```

```python
E=zeros(6,9)
E[0,0]=1
E[1,4]=1
E[2,8]=1
E[3,1]=1
E[3,3]=1
E[4,2]=1
E[4,6]=1
E[5,5]=1
E[5,7]=1
E
```

```python
mu = Symbol('mu')
la = Symbol('lambda')
C_tensor = getIsotropicStiffnessTensor(mu, la)
C = convertStiffnessTensorToMatrix(C_tensor)
C
```

```python
S=B.T*E.T*C*E*B*A*(1+alpha3*K)
S
```

```python
M=Matrix([[rho, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, rho, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, rho, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])
M=M*A*(1+alpha3*K)
M
```

## Cartesian coordinates

```python
import fem.geometry as g
import fem.model as m
import fem.material as mat
import fem.solver as s
import fem.mesh as me
import plot

stiffness_matrix_func = lambdify([A, K, mu, la, alpha3], S, "numpy")
mass_matrix_func = lambdify([A, K, rho, alpha3], M, "numpy")

def stiffness_matrix(material, geometry, x1, x2, x3):
    A,K = geometry.get_A_and_K(x1,x2,x3)
    return stiffness_matrix_func(A, K, material.mu(), material.lam(), x3)

def mass_matrix(material, geometry, x1, x2, x3):
    A,K = geometry.get_A_and_K(x1,x2,x3)
    return mass_matrix_func(A, K, material.rho, x3)

def generate_layers(thickness, layers_count, material):
    layer_top = thickness / 2
    layer_thickness = thickness / layers_count
    layers = set()
    for i in range(layers_count):
        layer = m.Layer(layer_top - layer_thickness, layer_top, material, i)
        layers.add(layer)
        layer_top -= layer_thickness
    return layers

def solve(geometry, thickness, linear, N_width, N_height):
    layers_count = 1
    layers = generate_layers(thickness, layers_count, mat.IsotropicMaterial.steel())
    model = m.Model(geometry, layers, m.Model.FIXED_BOTTOM_LEFT_RIGHT_POINTS)
    mesh = me.Mesh.generate(width, layers, N_width, N_height, m.Model.FIXED_BOTTOM_LEFT_RIGHT_POINTS)
    lam, vec = s.solve(model, mesh, stiffness_matrix, mass_matrix)
    return lam, vec, mesh, geometry

width = 2
curvature = 0.8
thickness = 0.05

corrugation_amplitude = 0.05
corrugation_frequency = 20

geometry = g.General(width, curvature, corrugation_amplitude, corrugation_frequency)

N_width = 50
N_height = 4

lam, vec, mesh, geometry = solve(geometry, thickness, False, N_width, N_height)
results = s.convert_to_results(lam, vec, mesh, geometry)

results_index = 0

plot.plot_init_and_deformed_geometry_in_cartesian(results[results_index], 0, width, -thickness / 2, thickness / 2, 0, geometry.to_cartesian_coordinates)

to_print = 20
if (len(results) < to_print):
    to_print = len(results)

for i in range(to_print):
    print(results[i].rad_per_sec_to_Hz(results[i].freq))

```