# KW-Distance: Two alternatives LP Models
In this notebook, we write a basic Linear Programming (LP) model to approximate the Kantorovich-Wasserstein distance of order 1 between a pair of discrete measures, such as, for instance, a pair of gray scale images.

In order to assess computationally the deviance of our model from the optimal solution, we implement first a standard LP model defined on a bipartite graph, which has a quadratic number of variables. Later, we show how to implement our compact model.

In the following, for the easy of notation, we consider discrete measures with $N$ support points defined as sum of Diracs as follows:

$$\mu = \sum_{i=1,\dots,N} \mu_i \delta(x_i)$$

where $\mu_i$ is the quantity of mass located at position $x_i$, and $\delta(x_i)=1$ only at position $x_i$, and it is equal to zero, otherwise.

### Exact Bipartite Model
We begin with given the standard bipartite LP model. Given two discrete measutes $\mu$ and $\nu$ defined on 2-dimensional histograms (i.e., a regular grid) with $N=n \times n$ bins, we define a bipartite graph $G=(V \cup W, E)$ as follows. We have a node $v \in V$ for each support point of the first measure $\mu$ and a node $w \in W$ for each support point of the second measure $\nu$. Note that we need to consider only support points a strictly positive quantity of mass (we can ignore all the bins without mass). Then, we add an edge $\{i,j\} \in E$ with cost $c_{ij} = d_{ij} = \sqrt{||x_i - x_j||}$, whenever both $\mu_i>0$ and $\nu_j>0$.

Given the bipartite graph $G=(V \cup W, E)$, we can write the following LP model:

$$\begin{align}
\min \quad & \sum_{ij \in E} c_{ij} \pi_{ij} \\
\mbox{s.t.} \quad
& \sum_{ij \in E} \pi_{ij} = \mu_i, & \forall i \in V \\
& \sum_{ij \in E} \pi_{ij} = \nu_j, & \forall j \in W \\
& \pi_{ij} \geq 0, & \forall \{i,j\} \in E.
\end{align}$$

The variable $\pi_{ij}$ are the decision variables that indicate the quantity of mass to be moved from position $x_i$ to $x_j$.

### Installing PYOMO and GLPK
We show next how to implement the previous model using the [Pyomo](https://pyomo.readthedocs.io/en/stable/index.html) optimization modeling language and the [GLPK](https://www.gnu.org/software/glpk/) open source MILP solver.

```python
import shutil
import sys
import os.path

if not shutil.which("pyomo"):
    !pip install -q pyomo
    assert(shutil.which("pyomo"))

if not (shutil.which("glpk") or os.path.isfile("glpk")):
    if "google.colab" in sys.modules:
        !apt-get install -y -qq glpk-utils
    else:
        try:
            !conda install -c conda-forge glpk
        except:
            pass
```

From Pyomo, we use the following procedures for writing our LP models:

```python
from pyomo.environ import ConcreteModel, Var, Objective, Constraint, SolverFactory
from pyomo.environ import RangeSet, ConstraintList, NonNegativeReals
```

### Creating random 2D discrete measures
In order to store a random discrete measure, with support $N$ points randomly located over the space of a square grid of size $M \times M$, we use the following class

```python
import numpy as np
# Support function to normalize a numpy vector
Normalize = lambda x: x/sum(x)

# We define a data type for 2D-histograms as defined before
class Measure2D(object):
    def __init__(self, N, M=32, seed=13):
        """ default c'tor: N random points """
        # Fix the seed for debugging
        np.random.seed(seed)
        # N random weights
        self.W = Normalize(np.random.uniform(0, 1, size=N))
        # N random support points
        x = set()
        while len(x) < N:
            x.add((np.random.randint(1, M),
                   np.random.randint(1, M)))
        self.X = list(x)
        # Map point to weight
        self.D = {}
        for i in range(N):
            self.D[self.X[i]] = self.W[i]
```

And to create two random measures over a grid of size $32 \times 32$, the first with 300 support points, and the second with 200 support points:

```python
# Create two random measures
GridSize = 32

Mu = Measure2D(300, GridSize, seed=13)
Nu = Measure2D(200, GridSize, seed=14)
```

Later on, we need a cost function between a pair of support points defined in $\mathbb{R}^2$.

```python
from math import sqrt
# Euclidean distance in the plane
Cost = lambda x, y: sqrt((x[0] - y[0])**2 + (x[1] - y[1])**2)
```

## Solving the Bipartite model
Given two discrete measures $\mu$ and $\nu$, we can use an implicit definition of the bipartite graph, by defining directly the following function, whichi implements and solves LP model described before.

```python
# Others useful libraries
from time import time

# Second, we write a function that implements the model, solves the LP,
# and returns the KW distance along with an optimal transport plan.
def BipartiteDistanceW1_L2(Mu, Nu):
    t0 = time()
    # Main Pyomo model
    model = ConcreteModel()
    # Parameters
    model.I = RangeSet(len(Mu.X))
    model.J = RangeSet(len(Nu.X))
    # Variables
    model.PI = Var(model.I, model.J, within=NonNegativeReals)
    # Objective Function
    model.obj = Objective(
        expr=sum(model.PI[i,j] * Cost(Mu.X[i-1], Nu.X[j-1]) for i,j in model.PI))

    # Constraints on the marginals
    model.Mu = Constraint(model.I,
                          rule = lambda m, i: sum(m.PI[i,j] for j in m.J) == Mu.W[i-1])
    model.Nu = Constraint(model.J,
                          rule = lambda m, j: sum(m.PI[i,j] for i in m.I) == Nu.W[j-1])

    # Solve the model
    sol = SolverFactory('glpk').solve(model)

    # Get a JSON representation of the solution
    sol_json = sol.json_repn()
    # Check solution status
    if sol_json['Solver'][0]['Status'] != 'ok':
        return None
    if sol_json['Solver'][0]['Termination condition'] != 'optimal':
        return None

    return model.obj(), time()-t0
```

In order to compute the distance between the two models, we have only to call our function with the two discrete meausures randomly defined before.

```python
# Compute distance and runtime
distA, runtimeA = BipartiteDistanceW1_L2(Mu, Nu)
```

```python
print("Optimal distance: {}, runtime: {}".format(distA, runtimeA))
```

    Optimal distance: 2.3510932706221777, runtime: 10.579878091812134

In the following, we look at a different LP model which approximately solves an equivalent problem.

## Approximate LP model
In the this section, we show how we can solve the very same problem by using a likely smaller LP model. Note that for very samll instances, the difference might be invisible, but as the input measure scale up in size, the difference become quickly relevant.

The main idea is to exploit the cost structure of Kantorovich-Wasserstein distance, which permit to formulate the same problem on a flow network instead of using a bipartite graph. We can however prove that under given assumptions, the two problem formulations are equivalent.

Given two discrete measutes $\mu$ and $\nu$ defined on 2-dimensional histograms (i.e., a regular grid) with $N=n \times n$ bins, we define an uncapacitated network flow $G=(V, E, b)$ as follows. We have a node $v \in V$ for each bin, with support point $x_v=(i,j)$ with $0 \leq i,j \leq n-1$. Each node $v \in V$ has a flow balance $b_v = \mu_v - \nu_v$: if $b_v > 0$ then node $v$ is a source node; if $b_v < 0$ node $v$ is a sink node, otherwise, whenever $b_v=0$ node $v$ is a transit node. In addition, we have and edge $\{i,j\} \in E$ with cost $c_{ij} = d_{ij} = \sqrt{||x_i - x_j||}$, whenever a specific condition is verified between the pair of support points $x_i$ and $x_j$. The way we specify this condition is the core of our contribution in [1]. For the moment, suppose we have a link between any pair of bins.

At this point, if we look for a minimum flow from the source nodes to the sink nodes, we find the minimum cost of moving the first measure into the second at minimum cost. That is, we are solving a problem equivalent to the LP defined on the bipartite graph.

### Building the auxiliary Flow Network
The first task, given a value of the parameter $L$, is to build the flow network which implements an equivalent problem as an uncapacitated network flow problem. The fundamental rule for building such network is to add a link between a pair of location $(i,j)$ and $(i+v,j+w)$ if only if $(v,w)$ are co-primes.

Thus, first we define a function that compute the set of coprimes number from 0 to L.

```python
# Given a value of the parameter L, build the corrisponding co-primes set
def CoprimesSet(L):
    from numpy import gcd
    Cs = []
    for v in range(-L, L+1):
        for w in range(-L, L+1):
            if (not (v == 0 and w == 0)) and gcd(v, w) == 1:
                Cs.append((v, w))
    return Cs
```

For instance, if we want to build a small set of coprimes locations, we could use $L=3$.

```python
L = 3
Cs = CoprimesSet(L)
print(Cs)
```

    [(-3, -2), (-3, -1), (-3, 1), (-3, 2), (-2, -3), (-2, -1), (-2, 1), (-2, 3), (-1, -3), (-1, -2), (-1, -1), (-1, 0), (-1, 1), (-1, 2), (-1, 3), (0, -1), (0, 1), (1, -3), (1, -2), (1, -1), (1, 0), (1, 1), (1, 2), (1, 3), (2, -3), (2, -1), (2, 1), (2, 3), (3, -2), (3, -1), (3, 1), (3, 2)]

Using the set of coprimes pair, we can build our *small* flow network as follows.

```python
# Import the graph library NetworkX
import networkx as nx

# Build a Network using a precomputed set of pair of coprimes numbers
def BuildGridNetwork(N, Coprimes):
    def ID(x,y):
        return x*N+y

    G = nx.DiGraph()

    for i in range(N):
        for j in range(N):
            G.add_node(ID(i,j), pos=(i,j))

    for i in range(N):
        for j in range(N):
            for (v, w) in Coprimes:
                if i + v >= 0 and i + v < N and j + w >= 0 and j + w < N:
                    G.add_edge(ID(i,j), ID(i+v, j+w),
                               weight=sqrt(pow(v, 2) + pow(w, 2)))

    return G
```

We add also a function to plot a network in the plane

```python
# Plot a grid network (nodes must have coordinates position labels)
def PlotGridNetwork(G, name=""):
    import matplotlib.pyplot as plt

    plt.figure(3,figsize=(8, 8))
    plt.axis('equal')
    pos = nx.get_node_attributes(G, 'pos')
    nx.draw(G, pos, font_weight='bold', node_color='blue',
            arrows=True, arrowstyle='->',  arrowsize=15, width=1, node_size=200)

    # If a name is specified, save the plot in a file
    if name:
        plt.savefig("grid_{}.png".format(name), format="PNG")
```

Let start we a couple of small example, to get an idea about the networks that are built.

```python
L = 2
Cs = CoprimesSet(L)
G1 = BuildGridNetwork(8, Cs)
PlotGridNetwork(G1)
```

Note that the degree of every node is limited, and much smaller than the case where every node (location) is connected with every other possible locations, as in the bipartite graphs.

Let us try with a large value of $L$.

```python
L = 3
Cs = CoprimesSet(L)
G1 = BuildGridNetwork(8, Cs)
PlotGridNetwork(G1)
```

## Solving the Flow Problem
Given a grid (flow) network $G$, and a pair of discrete measures defined over the grid, we can build the following LP model.

```python
def ApproximateDistanceW1_L2(Mu, Nu, G):
    t0 = time()
    # Number of egdes
    m = len(G.edges())
    # Main Pyomo model
    model = ConcreteModel()
    # Parameters
    model.E = RangeSet(m)
    # Variables
    model.PI = Var(model.E, within=NonNegativeReals)
    # Map edges to cost
    C = np.zeros(m)
    M = {}
    for e, (i, j) in enumerate(G.edges()):
        C[e] = G.edges[i,j]['weight']
        M[i,j] = e+1

    # Objective Function
    model.obj = Objective(expr=sum(model.PI[e] * C[e-1] for e in model.PI))

    # Flow balance constraints (using marginals balance at each location)
    model.Flow = ConstraintList()
    for v in G.nodes():
        Fs = [M[w] for w in G.out_edges(v)]
        Bs = [M[w] for w in G.in_edges(v)]

        # Compute flow balance value at given node position
        x = G.nodes[v]['pos']
        b = Mu.D.get(x, 0.0) - Nu.D.get(x, 0.0)

        # Flow balance constraint
        model.Flow.add(expr = sum(model.PI[e] for e in Fs) - sum(model.PI[e] for e in Bs) == b)

    # Solve the model
    sol = SolverFactory('glpk').solve(model)

    # Get a JSON representation of the solution
    sol_json = sol.json_repn()
    # Check solution status
    if sol_json['Solver'][0]['Status'] != 'ok':
        return None
    if sol_json['Solver'][0]['Termination condition'] != 'optimal':
        return None

    return model.obj(), (time()-t0)
```

At this point, we can compute the distance between the same pair of discrete measures $Mu$ and $Nu$ defined before, but using our new LP model.

```python
# We build a flow network of size 32x32, using L=3
L = 3
Cs = CoprimesSet(L)
G = BuildGridNetwork(GridSize, Cs)

# Compute distance and runtime with the approximate model
distB, runtimeB = ApproximateDistanceW1_L2(Mu, Nu, G)

# ... and to compare with previous solution
print("LB   Full = {:.5}, LB   Apx = {:.5}".format(distA, distB))
print("Time Full = {:.5}, Time Apx = {:.5}".format(runtimeA, runtimeB))
```

    LB   Full = 2.3511, LB   Apx = 2.3558
    Time Full = 10.58, Time Apx = 6.1425

Note that in this case, the difference in running time is limited, and it is dominanted by the time for building the model (Pyomo is not very fast with this respect, despite its flexibility).

The approximate model is much more efficent for **dense** discrete measures, such as, for instance, images, where every point of a regular grid has a weight strictly positive.

Let us look at the following example.

```python
# Create two random measures
GridSize = 32
N = 900
Mu = Measure2D(N, GridSize, seed=13)
Nu = Measure2D(N, GridSize, seed=14)

# Compute distance and runtime
distA, runtimeA = BipartiteDistanceW1_L2(Mu, Nu)

# We build a flow network of size 32x32, using L=3
L = 3
Cs = CoprimesSet(L)
G = BuildGridNetwork(GridSize, Cs)

# Compute distance and runtime with the approximate model
distB, runtimeB = ApproximateDistanceW1_L2(Mu, Nu, G)

# ... and to compare with the previous solution optimal valur and runtime
print("LB   Full = {:.5}, LB   Apx = {:.5}".format(distA, distB))
print("Time Full = {:.5}, Time Apx = {:.5}".format(runtimeA, runtimeB))
```

    LB   Full = 0.63854, LB   Apx = 0.63878
    Time Full = 199.16, Time Apx = 7.8611

For additional details and more extensive computational experiments, we refer to the following paper.

### References
1. Bassetti, F., Gualandi, S. and Veneroni, M., 2020. [*On the Computation of Kantorovich--Wasserstein Distances Between Two-Dimensional Histograms by Uncapacitated Minimum Cost Flows*](https://epubs.siam.org/doi/abs/10.1137/19M1261195). **SIAM Journal on Optimization**, 30(3), pp.2441-2469.


```python
import pandas as pd
import numpy as np
import altair as alt
```

```python
import sympy as sp
```

```python
p2 = sp.symbols('p2')
p3 = sp.symbols('p3')
p4 = sp.symbols('p4')

sigma = sp.Matrix([[1,p2,p3,p4],[p2,1,p4,p3],[p3,p4,1,p2],[p4,p3,p2,1]]).T
sigma
```

$\displaystyle \left[\begin{matrix}1 & p_{2} & p_{3} & p_{4}\\p_{2} & 1 & p_{4} & p_{3}\\p_{3} & p_{4} & 1 & p_{2}\\p_{4} & p_{3} & p_{2} & 1\end{matrix}\right]$

```python
sigma.eigenvects()
```

    [(-p2 - p3 + p4 + 1,
      1,
      [Matrix([
       [ 1],
       [-1],
       [-1],
       [ 1]])]),
     (-p2 + p3 - p4 + 1,
      1,
      [Matrix([
       [-1],
       [ 1],
       [-1],
       [ 1]])]),
     (p2 - p3 - p4 + 1,
      1,
      [Matrix([
       [-1],
       [-1],
       [ 1],
       [ 1]])]),
     (p2 + p3 + p4 + 1,
      1,
      [Matrix([
       [1],
       [1],
       [1],
       [1]])])]


# Statistical parameters using probability density function
### Given probability density function, $p(x)$

$ p = 2x/b^2$, $0 < x < b$

### The mean value of $x$ is estimated analytically:
$\overline{x} = \int\limits_0^b x\, p(x)\, dx = \int\limits_0^b 2x^2/b^2 = \left. 2x^3/3b^2\right|_0^b =2b^3/3b^2 = 2b/3$

### the median
median: $ \int\limits_0^m p(x)\,dx = 1/2 = \int\limits_0^m 2x/b^2\,dx = \left. x^2/b^2 \right|_0^m = m^2/b^2 = 1/2$, $m = b/\sqrt(2)$

### the second moment
second moment: $x^{(2)} = \int\limits_0^b x^2\, p(x)\, dx = \int\limits_0^b 2x^3/b^2 = \left. x^4/2b^2\right|_0^b =b^4/2b^2 = b^2/2$

### the variance is the second moment less the squared mean value
$var(x) = x^{(2)} - \overline{x}^2 = b^2/2 - 4b^2/9 = b^2/18$

```python
def p(x,b):
    return 2*x/(b**2)

```

```python
b = 2
x = linspace(0,b,200)
y = p(x,b)
```

```python
plot(x,y)
xlabel('$x$')
ylabel('$p(x)$')
```

```python
# approximate using the numerical integration
print trapz(y*x,x)
print 2.*b/3
```

    1.33335016793
    1.33333333333

```python
print trapz(y*x**2,x)
print b^2/18
```

    2.00005050378
    2

```python
import sympy
```

```python
sympy.var('x,b,p,m')
p = 2*x/b**2
print p
```

    2*x/b**2

```python
sympy.integrate(p*x,(x,0,b))
```

    2*b/3

```python
sympy.integrate(p*x**2,(x,0,b))
```

    b**2/2

```python
sympy.integrate(p,(x,0,m))
```

    m**2/b**2

```python
sympy.solve(m**2/b**2 - 0.5,m)
```

    [-0.707106781186548*b, 0.707106781186548*b]

```python

```


This derives the equations of motion for the compound pendulum with a base attached to a vertical spring using both Kane's method and Langrange's method and compares the results of both.

```python
from sympy import symbols, solve, cos
from sympy.physics.mechanics import dynamicsymbols, ReferenceFrame, Point, Particle
from sympy.physics.mechanics import RigidBody, inertia, KanesMethod, init_vprinting, Lagrangian, LagrangesMethod
```

```python
init_vprinting()
```

```python
x, theta = dynamicsymbols('x, theta')
v, omega = dynamicsymbols('v, omega')
m, M, I, k, l, g = symbols('m, M, I, k, l, g')
```

```python
N = ReferenceFrame('N')
A = N.orientnew('A', 'Axis', (theta, N.z))
```

```python
O = Point('O')
a = O.locatenew('a', x * N.x)
b = a.locatenew('b', l * A.x)
```

# Kane's Method

```python
A.set_ang_vel(N, omega * N.z)
```

```python
O.set_vel(N, 0)
a.set_vel(N, v * N.x)
b.v2pt_theory(a, N, A)
```

```python
block = Particle('block', a, m)
bar = RigidBody('bar', b, A, M, (inertia(A, 0, 0, I), b))
```

```python
kane = KanesMethod(N, (x, theta), (v, omega), kd_eqs=(x.diff() - v, theta.diff() - omega))
fr, frstar = kane.kanes_equations(((a, -k * x * N.x + m * g * N.x), (b, M * g * N.x)), (block, bar))
```

```python
fr_plus_frstar = fr + frstar
fr_plus_frstar.simplify()
fr_plus_frstar
```

```python
rhs = kane.rhs()
rhs.simplify()
rhs
```

This gives the two equilibrium points:

```python
eq_points = solve(fr_plus_frstar.subs({omega.diff(): 0, v.diff(): 0, omega: 0, v: 0}), x, theta)
eq_points
```

Linearized equations of motion about the stable equilibrium point.

```python
A_mat, _, _ = kane.linearize(op_point=eq_points[0], new_method=True, A_and_B=True)
states = kane.q.col_join(kane.u)
states.diff() - A_mat * states
A_mat * states
```

# Lagrange's Method

```python
A.set_ang_vel(N, theta.diff() * N.z)
a.set_vel(N, x.diff() * N.x)
b.v2pt_theory(a, N, A)
```

```python
b.vel(N).magnitude()
```

```python
bar.kinetic_energy(N) + block.kinetic_energy(N)
```

```python
block.potential_energy = k * x**2 / 2 - m * g * x
bar.potential_energy = -M * g * (x + l * cos(theta))
```

```python
L = Lagrangian(N, block, bar)
L
```

```python
L.diff(theta.diff()).simplify().diff(dynamicsymbols._t)
```

```python
lagrange = LagrangesMethod(L, (x, theta))
```

```python
lagranges_equations = lagrange.form_lagranges_equations()
lagranges_equations.simplify()
lagranges_equations
```

# Compare

```python
should_be_zero = fr_plus_frstar.subs({omega: theta.diff(), v: x.diff()}) + lagranges_equations
should_be_zero.simplify()
should_be_zero
```


###### Content under Creative Commons Attribution license CC-BY 4.0, code under BSD 3-Clause License © 2021 Lorena A. Barba, Pi-Yueh Chuang, Tingyu Wang

# Logistic regression

In Lesson 1 of this module, you learned about fitting a line to data (linear regression) using the method of gradient descent to find the model parameters (slope and $y$-intercept). But what if the observational data looks nothing like it has a linear relationship?

A major class of problems deals with binary classification, that is, the data belong to one of two categories.
Typically, we code the two categories with $0$ and $1$. For example, is an email spam, or not spam? Is a credit-card transaction fraudulent, or legitimate?
In these settings, often the data correspond to numbers between $0$ and $1$ that represent some _probability_ (e.g., that the email be spam or not), and one applies a decision boundary to assign each item to a class.

One effective way to build a model representing this situation is logistic regression. The output of the model is a number between $0$ and $1$, for inputs in the whole real line.
With binary data (zeros and ones), we might imagine that a good model could look like a step function. But this function is not differentiable. Since we want to use gradient descent, we adopt a continuous function that approximates the step function.

## Logistic function

We can build a model that will output a value between zero and one by making a non-linear transformation of the linear regression.
This is achieved with the _logistic function_:

$$ \sigma(z) = \frac{1}{1+e^{-z}}$$

With the data consisting of two arrays, $x, y$, the model is a composition: $z= wx + b$ and $\sigma(z)$ is the output.
The task is finding $w$ and $b$ so that $\sigma(z)$ is closest to $y$ for all data points.

Let's play with this function using SymPy. We'll need NumPy and Matplotlib later, so might as well load all our libraries now.

```python
import sympy
import numpy

from matplotlib import pyplot
%matplotlib inline
```

```python
z = sympy.Symbol('z', real=True)

logistic = 1/(1+ sympy.exp(-z))
logistic
```

$\displaystyle \frac{1}{1 + e^{- z}}$

```python
sympy.plotting.plot(logistic);
```

That's a well-groomed $S$-shaped function: it's called a _sigmoid_ curve. Notice that when $z=0$ it takes the value $0.5$, and it approaches zero on the left, and one on the right.

Let's generate some synthetic data to play with. (We take this example from the SciPy 2019 tutorial by Eric Ma [1]).
Our goal is to use gradient descent to find the model parameters $w$ and $b$ that best fit the data, in some sense that we need to discover.

```python
# synthetic data
x_data = numpy.linspace(-5, 5, 100)
w = 2
b = 1
numpy.random.seed(0)
z_data = w * x_data + b + numpy.random.normal(size=len(x_data))
y_data = 1 / (1+ numpy.exp(-z_data))

pyplot.scatter(x_data, y_data, alpha=0.4);
```

Above, we chose the parameters $w$ and $b$ for the model, and used them to get the intermediate variable $z$ adding some random noise to make our synthetic data look more "realistic."
The call to `numpy.random.seed()` makes our added noise reproducible, i.e., you always get the same pseudo-random numbers from the subsequent call to `numpy.random.normal()`.
(Let's not get side-tracked into a discussion about pseudo-random number generation, and leave that for another tutorial.)

We can apply a decision boundary now to assign the data to the two classes. Be sure to read the documentation of [`numpy.where()`](https://numpy.org/doc/stable/reference/generated/numpy.where.html) to understand the code below, noting that after the logical condition we specify the values to assign if the condition is `True` or `False`.

```python
y_data = numpy.where(y_data >= 0.5, 1, 0)
pyplot.scatter(x_data, y_data, alpha=0.4);
```

## Logistic loss function

To use gradient descent, we need a loss function that we can optimize with respect to the parameters.
If we were to use a mean-square-error loss function, like in linear regression, we'd run into a few problems. Let's discuss.

Taking the derivatives to optimize would involve $\sigma^{\prime}(z)$, so let's look at that.

```python
lprime = logistic.diff(z)
lprime
```

$\displaystyle \frac{e^{- z}}{\left(1 + e^{- z}\right)^{2}}$

```python
sympy.plotting.plot(lprime);
```

The derivative of the logistic function takes very small values at the long tails on each side of $z=0$.
If our loss function has a $\sigma^{\prime}(z)$ factor (coming from the chain rule), this would lead to _slow learning_.
We'd like to work with a better loss function, that avoids this problem, and we build one below by integration. (For a more detailed discussion, we recommend Chapter 3 of Michael Nielsen's free ebook [2]).

It's important to note also that our prediction model is a nonlinear function, composed with the linear model, and the square-error would lead to a non-convex loss function that can have local minima, and make gradient descent fail.
Here's an example posted on Stackoverflow in answer to this very [question](https://math.stackexchange.com/questions/2381724/logistic-regression-when-can-the-cost-function-be-non-convex). Consider just three data points, and a model with no intercept, $z = wx$: $(-1, 2), (-20, -1), (-5, 5)$. What would a square-error loss function look like? We can plot it using SymPy.

```python
badloss = (1/(1+ sympy.exp(-z))-2)**2 + \
          (1/(1+ sympy.exp(-20*z))+1)**2  + \
          (1/(1+ sympy.exp(-5*z))-5)**2
badloss
```

$\displaystyle \left(-5 + \frac{1}{1 + e^{- 5 z}}\right)^{2} + \left(-2 + \frac{1}{1 + e^{- z}}\right)^{2} + \left(1 + \frac{1}{1 + e^{- 20 z}}\right)^{2}$

```python
sympy.plotting.plot(badloss, xlim=(-1,1));
```

You can see a local minimum, where gradient descent could get stuck!

But let's go back to inspecting the derivative of the logistic function.
We note that $\sigma^{\prime}(z)$ has the same expression in the denominator as $\sigma(z)$, but squared. Let's play around with this...

```python
lprime/logistic
```

$\displaystyle \frac{e^{- z}}{1 + e^{- z}}$

OK, we can try to express this in terms of $\sigma(z)$: add and subtract $1$ to the numerator and factor:

$$\frac{e^{-z}}{1+e^{-z}} = \frac{1+e^{-z}-1}{1+e^{-z}} = 1 - \frac{1}{1+ e^{-z}} = 1 - \sigma(z)$$

We get this interesting property for the derivative of the logistic function:

$$\sigma^{\prime}(z) = \sigma(z) (1-\sigma(z))$$

Following Nielsen [2], let's call $\sigma(z)=a$, representing the predicted value by the model where $z=wx +b$.
The deviation between the model output and the observational data at each point is measured by $|y-a|$.
A square-error loss function, as used for linear regression, is $(y-a)^2$, giving derivatives with respect to the parameters that are proportional to the error:
the larger the error, the larger the step taken in gradient descent.
That's a nice property.

Seeking a loss function $L$ with the same behavior, we would like, for example:
$\frac{\partial L}{\partial b} = (y-a)$.
Using the chain rule,
$$\frac{\partial L}{\partial b} = \frac{\partial L}{\partial a}\frac{\partial a}{\partial z} = \frac{\partial L}{\partial a} \sigma^{\prime}(z) = \frac{\partial L}{\partial a} \, a(1-a)$$

using the property of $\sigma^{\prime}$ in the last step.

Our desired behavior for $L$ is that this derivative be $(y-a)$, giving:

$$ \frac{\partial L}{\partial a} = \frac{(y-a)}{a(1-a)}$$

This lets us construct $L$ by integration:

```python
a, y = sympy.symbols('a y', real=True)
```

```python
dLda = (y-a)/a/(1-a)
dLda
```

$\displaystyle \frac{- a + y}{a \left(1 - a\right)}$

```python
L = sympy.integrate(dLda, a)
L
```

$\displaystyle y \log{\left(a \right)} + \left(1 - y\right) \log{\left(a + \frac{1 - 2 y}{2 y - 1} \right)}$

```python
sympy.simplify(L)
```

$\displaystyle y \log{\left(a \right)} - \left(y - 1\right) \log{\left(a - 1 \right)}$

(Plus a constant of integration.) We note that since $a<1$ (and the logarithm is only defined for positive real numbers) we need:

```python
L = -y*sympy.log(a) + (y-1)*sympy.log(1-a)
L
```

$\displaystyle - y \log{\left(a \right)} + \left(y - 1\right) \log{\left(1 - a \right)}$

```python
sympy.simplify(L.diff(a))
```

$\displaystyle \frac{- a + y}{a \left(a - 1\right)}$

The derivative is the same expression we started with.
This is the _logistic loss function_ for a single data point:

$$L = -y \log(a) + (y-1) \log(1-a)$$

Let's inspect this loss function. If the true value is $y=1$, we get $L= -\log(a)$.
Plotting in the range of interest shows that the gradient is largest when the model output is furthest away from the true value.
This is a nice property.

```python
sympy.plotting.plot(-sympy.log(a), xlim=(0,1));
```

If $y=0$: $L= - \log(1-a)$. The plot again shows that the gradient is largest when the model output is furthest from the true value.

```python
sympy.plotting.plot(-sympy.log(1-a), xlim=(0,1));
```

## Find the parameters using `autograd`

With the logistic loss function, we're ready to use gradient descent in an optimization loop.
Above, we defined the `logistic` SymPy expression as a function of the symbol `z`, so let's re-define it to include the composition with $wx+b$.
We could then define the loss function symbolically, as we did for linear regression; have a look at this:

```python
logistic
```

$\displaystyle \frac{1}{1 + e^{- z}}$

```python
w, b, x, y = sympy.symbols('w b x y')
logistic = 1/(1+ sympy.exp(-w*x-b)) # redefined with the composition

Loss = -y*sympy.log(logistic) - (1-y)*sympy.log(1-logistic)
Loss
```

$\displaystyle - y \log{\left(\frac{1}{e^{- b - w x} + 1} \right)} - \left(1 - y\right) \log{\left(1 - \frac{1}{e^{- b - w x} + 1} \right)}$

So far, we can still use SymPy to get derivatives of the loss function with respect to the parameters. But with more complicated models, finding symbolic derivatives could take a long time.

Have a look at the derivative of the logistic loss with respect to the parameter $b$:

```python
Loss.diff(b)
```

$\displaystyle - \frac{y e^{- b - w x}}{e^{- b - w x} + 1} - \frac{\left(y - 1\right) e^{- b - w x}}{\left(1 - \frac{1}{e^{- b - w x} + 1}\right) \left(e^{- b - w x} + 1\right)^{2}}$

Although we can use symbolic differentiation, we later need to convert the resulting expression to a Python function that can be called and evaluated at many data inputs. Maybe this is not the best approach.

There's a better way! It's called _automatic differentiation_: the idea is to algorithmically obtain derivatives of numeric functions written in computer code.
It sounds like magic, but it can be done by a combination of the chain rule, symbolic rules of differentiation for elementary operations, and a numeric evaluation trace of the elementary derivatives.

The details of how automatic differentiation works are fascinating, but here we skip ahead to say that software libraries are freely available to do this!
For Python programs, the [`autograd`](https://github.com/hips/autograd) package was specifically developed for gradient-based optimization.
It is described thus:

> Autograd can automatically differentiate native Python and NumPy code. It can handle a large subset of Python's features, including loops, ifs, recursion and closures, and it can even take derivatives of derivatives of derivatives.

(The package is no longer in active development, as a new version was folded into the [JAX](https://github.com/google/jax) project, out of Google. It is still maintained, though, and just as useful as ever!)

To install the package, you can run the following command (either on the terminal or in a Jupyter code cell):

```
pip install autograd
```

Or, if you installed your Python environment via [Anaconda](https://www.anaconda.com/products/individual), you can run this command:
```
conda install -c conda-forge autograd
```

Once the package is installed in your working environment, you can import it in the usual Pythonic way.
The main function you will need is `autograd.grad()`, which takes a scalar-valued Python function as argument, and returns another function that evaluates to its derivative.
It's what we use in the optimization loop to perform gradient descent.

In addition, `autograd.numpy` is a wrapper to the NumPy library. This allows you to call your favorite NumPy methods with `autograd` keeping track of every operation so it can give you the derivative (via the chain rule).

```python
# import the autograd-wrapped version of numpy
from autograd import numpy
```

```python
# import the gradient calculator
from autograd import grad
```

It's important to realize that we now need to design our program in a _functional_ style: defining Python functions for the logistic and loss functions, so that we may apply `autograd.grad()` in the optimization loop.
Study the code below.

```python
# note: the namespace numpy is the autograd wrapper to NumPy

def logistic(z):
    '''The logistic function'''
    return 1 / (1 + numpy.exp(-z))

def logistic_model(params, x):
    '''A prediction model based on the logistic function composed with wx+b
    Arguments:
       params: array(w,b) of model parameters
       x :  array of x data'''
    w = params[0]
    b = params[1]
    z = w * x + b
    y = logistic(z)
    return y

def log_loss(params, model, x, y):
    '''The logistic loss function
    Arguments:
       params: array(w,b) of model parameters
       model:  the Python function for the logistic model
       x, y:   arrays of input data to the model'''
    y_pred = model(params, x)
    return -numpy.mean(y * numpy.log(y_pred) + (1-y) * numpy.log(1 - y_pred))
```

```python
# get a function to compute the gradient of the logistic loss
gradient = grad(log_loss)
```

```python
type(gradient)
```

    function

##### Note:

> The argument list of the function returned by `grad()` will be the same as the argument list of the loss function that we input to it.
> The default setting of `grad()` is to return the derivative(s) with respect to the first argument, in this case, the two-element array `params`. This gives us the two derivatives (with respect to $w$ and $b$) at once.

Let's now make a random starting guess for the two model parameters:

```python
numpy.random.seed(0)
params = numpy.random.rand(2)
print(params)
```

    [0.5488135  0.71518937]

With values assigned to the two parameters, this is how you compute the corresponding derivatives—we'll need to do this in each iteration of the optimization loop:

```python
gradient(params, logistic_model, x_data, y_data)
```

    array([-0.42734877,  0.08274066])

Now we optimize! Notice that in this optimization loop we chose to use a `while` statement, instead of `for`.
We set both a maximum number of iterations, and an exit criterion based on the norm of the gradient being very small.
Already you should be thinking about these choices we make:

- The gradient is multiplied by a small step of $0.01$: this is called the _learning rate_. How do we choose this value?
- The `while` loop exits when the norm of the gradient is $0.001$: how do we know if this is a good choice?

Finally, we plot the synthetic data we created above, and the logistic regression curve corresponding to the parameters found in the optimization loop.

```python
max_iter = 5000
i = 0
descent = numpy.ones(len(params))

while numpy.linalg.norm(descent) > 0.001 and i < max_iter:

    descent = gradient(params, logistic_model, x_data, y_data)
    params = params - descent * 0.01
    i += 1

print(f'Optimized value of w is {params[0]:.3f} vs. true value: 2')
print(f'Optimized value of b is {params[1]:.3f} vs. true value: 1')
print(f'Exited after {i} iterations')

pyplot.scatter(x_data, y_data, alpha=0.4)
pyplot.plot(x_data, logistic_model(params, x_data), '-r');
```

It worked! We found a logistic curve that approximates the data quite well.
This curve represents the _prediction model_ for the continuous data between $0$ and $1$.
In a classification task, the next step is to assign a decision threshold for the two classes; a natural choice is $0.5$.

```python
def decision_boundary(y):
    return 1 if y >= .5 else 0
```

This function will code as $0$ or $1$ each data point. We will want to apply the function element-wise to all elements in an array, for which we can use [`numpy.vectorize()`](https://numpy.org/doc/stable/reference/generated/numpy.vectorize.html).
Be sure to read the documentation of this very useful NumPy function that saves you from having to write a for-loop!

The decision rule is what turns logistic regression into a classification tool. Check it out!

```python
decision_boundary = numpy.vectorize(decision_boundary)
```

```python
def classify(predictions):
    '''
    Argument:
    predictions, an array of values between 0 and 1

    Returns:
    classified, an array of 0 and 1 values'''

    return decision_boundary(predictions).flatten()

pyplot.scatter(x_data, y_data, alpha=0.4,
               label='true value')
pyplot.scatter(x_data, classify(logistic_model(params, x_data)), alpha=0.4,
               label='prediciton')

pyplot.legend();
```

And this is the end of Lesson 2 of our module. We spent some time deriving the logistic loss function, where most sources will just give it to you without explanation.
We find it more satisfying this way, but you can certainly skip it.

The key ingredients we have added in this lesson are the ideas of:
- using composition of functions in our model
- using automatic differentiation to get the gradients for the optimization loop

In the next lesson, we add the idea of multiple independent variables (a.k.a. _features_) affecting our model, leading to **multiple linear regression.**

## What we've learned

- The logistic function approximates a step function while being differentiable. That's nice.
- Logistic regression applies composition of the logistic function with a linear model.
- We can use logistic regression in binary classification problems.
- Using the square-error in this case leads to a non-convex loss function, with local minima.
- The logistic loss function is best here for optimizing with gradient descent.
- Automatic differentiation gives us derivatives of numeric functions written in computer code.
- We pick some values in the optimization loop that we need to be careful with (e.g., learning rate).

## References

1. Eric Ma, "Deep Learning Fundamentals: Forward Model, Differentiable Loss Function & Optimization," SciPy 2019 tutorial. [video on YouTube](https://youtu.be/JPBz7-UCqRo) and [archive on GitHub](https://github.com/ericmjl/dl-workshop/releases/tag/scipy2019).
2. Michael A. Nielsen, "Neural Networks and Deep Learning" (2015), Determination Press, http://neuralnetworksanddeeplearning.com

```python
# Execute this cell to load the notebook's style sheet, then ignore it
from IPython.core.display import HTML
css_file = '../style/custom.css'
HTML(open(css_file, "r").read())
```

<link href="https://fonts.googleapis.com/css?family=Merriweather:300,300i,400,400i,700,700i,900,900i" rel='stylesheet' >
<link href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:300,300i,400,400i,700,700i" rel='stylesheet' >
<link href='https://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' >
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 800px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.5em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Source Sans Pro', sans-serif;
    line-height: 140%;
    font-size: 110%;
    width:680px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Merriweather', serif;
    font-style:regular;
    font-weight: bold;
    font-size: 250%;
    line-height: 100%;
    color: #004065;
    margin-bottom: 1em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Merriweather', serif;
    font-weight: bold;
    font-size: 180%;
    line-height: 100%;
    color: #0096d6;
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Merriweather', serif;
	font-size: 150%;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: regular;
    color: #008367;
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Merriweather', serif;
    font-weight: 300;
    font-size: 100%;
    line-height: 120%;
    text-align: left;
    width:500px;
    margin-top: 1em;
    margin-bottom: 2em;
    margin-left: 80pt;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Source Sans Pro', sans-serif;
    font-weight: regular;
    font-size: 130%;
    color: #e31937;
    font-style: italic;
    margin-bottom: .5em;
    margin-top: 1em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'Source Code Pro', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>


Solution to: [Day 8: Pearson Correlation Coefficient II](https://www.hackerrank.com/challenges/s10-mcq-7/problem)

Table of Contents

- Table of Contents
- Solution
    - Write into regression form
    - Pearon's Coefficient Formula
    - Multiply equations
    - Direction

```javascript
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')
```

    <IPython.core.display.Javascript object>

# Solution

## Write into regression form

\begin{equation}
\large
y = -2 + -\frac{3}{4} x
\end{equation}

and

\begin{equation}
\large
x = -\frac{7}{4} + - \frac{3}{4} y
\end{equation}

Thus:
- b1 = $-\frac{3}{4}$
- b2 = $-\frac{3}{4}$

## Pearon's Coefficient Formula

- p = pearson coefficient
- x_sd = standard deviation of x
- y_sd = standard deviation of y

\begin{equation}
\large
p = b1 \frac {x\_sd}{y\_sd}
\end{equation}

and

\begin{equation}
\large
p = b2 \frac {x\_sd}{y\_sd}
\end{equation}

## Multiply equations

\begin{equation}
\large
p^{2} =
-\frac{3}{4} *
-\frac{3}{4}
\end{equation}

\begin{equation}
\large
p^{2} =
\frac{9}{16}
\end{equation}

\begin{equation}
\large
p =
\frac{3}{4}
\end{equation}

## Direction

As both original lines are negative, they are negatively correlated by definition.

Answer:

\begin{equation}
\large
p =
- \frac{3}{4}
\end{equation}


```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
```

# Array basics
## Construction/Accessing

```python
# Array sequences
array_sequence = np.array([1, 0, 0])
array_multi = np.array([[1, 0],[0, 1]])
array_multi2 = np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
array_complex = np.array([1 + 0j, 2, 3 + 1j])

# shape attribute
array_multi.shape

# slice
seq = np.array(range(25))
seq[0:6]
seq[:6]
seq[5:10]
seq[20:]

# Quick reverse
seq[::-1]

#Reshape
seq.reshape((5,5))
seq = seq.reshape((5,5))
seq[0, 0]
seq[4, 4]
seq[0:3, 0:3]
seq[:, 1] # Select column
seq[1, :] # Select row
```

    (2, 2)

```python
# Built-in array creation routines
# Linear space
this_arange = np.arange(0, 1, 0.05) # Does not include endpoint
this_linspace = np.linspace(0, 1, 21) # Does include endpoint
this_logspace = np.logspace(0, 1, 21)

# Zeroes, ones, random
np.zeros(2)
np.zeros((2,2))
np.zeros((3, 3, 3))

np.ones((3, 3, 3))

np.random.random((3, 3, 3))
# We'll come back to this in visualization
# plt.imshow(np.random.random((10, 10)), interpolation="nearest")

# Element-wise arithmetic
this_linspace + 2
this_linspace ** 2
```

    array([ 0.    ,  0.0025,  0.01  ,  0.0225,  0.04  ,  0.0625,  0.09  ,
            0.1225,  0.16  ,  0.2025,  0.25  ,  0.3025,  0.36  ,  0.4225,
            0.49  ,  0.5625,  0.64  ,  0.7225,  0.81  ,  0.9025,  1.    ])

## Exercise: create a for loop to double the input array

```python
def list_double(input_list):
    new = []
    for entry in input_list:
        new.append(entry*2)
    return new

def array_double(input_array):
    return input_array*2

input_list = range(1000)
input_array = np.array(input_list)

%timeit list_double(input_list)
%timeit array_double(input_array)
```

    10000 loops, best of 3: 115 µs per loop
    The slowest run took 24.71 times longer than the fastest. This could mean that an intermediate result is being cached.
    1000000 loops, best of 3: 1.26 µs per loop

## Exercise: create a for loop to construct the same array as a list, in 2-D

```python
def make_array_list():
    x = [0 + 0.01*i for i in range(101)]
    y = [0 + 0.01*j for j in range(101)]
    z = [[0.0 for k in range(101)] for l in range(101)]
    for i in range(101):
        for j in range(101):
            z[i][j] = x[i]**2 + y[j]**2
    return z

def make_array_numpy():
    x = np.arange(0, 1.01, 0.01)
    y = np.arange(0, 1.01, 0.01)
    X, Y = np.meshgrid(x, y)
    Z = X**2 + Y**2
    return Z

assert (np.array(make_array_list()) == make_array_numpy()).all()

%timeit np.array(make_array_list())
%timeit make_array_numpy()
```

    100 loops, best of 3: 3.04 ms per loop
    10000 loops, best of 3: 73.1 µs per loop

## Meshgrids

```python
xrange = np.arange(0, 1.01, 0.01)
yrange = np.arange(0, 1.01, 0.01)

X, Y = np.meshgrid(xrange, yrange)
# Different than imshow
plt.pcolor(X, Y, X**2 + Y**2)
plt.colorbar()
plt.show()
```

## Boolean masking

```python
# Boolean arithmetic
x = np.linspace(0, 2, 101)
c = np.zeros(101)

x < 1.5
x > 1.0
np.logical_and(x < 1.5, x > 1.0)

# masking
x[x < 1.5]
x[np.logical_and(x < 1.5, x > 1.0)]
c[np.logical_and(x < 1.5, x > 1.0)] = 1
plt.plot(x, c)
```

## Exercise: create a sawtooth function

```python

```

## Plotting

* matlab like functionality with pylab and pyplot

```python
plt.plot([0, 1], [0, 1])
```

```python
plt.plot(np.random.random(30), 'k-o') # color, linestyle, markerstyle
```

```python
# Bar chart histogram
hist, bins = np.histogram(np.random.random(10000))
plt.bar(bins[:-1], hist, width=np.diff(bins)[0])
plt.xlabel("bins")
plt.ylabel("counts")
```

```python
# plt.plot and plt.bar implicitly create figures and axes to plot on
# can do explicitly
fig = plt.figure()
ax = fig.add_axes([0.1, 0.1, 0.9, 0.9])
ax.plot(np.random.random(10), 'k-', label="Data")
ax.set_xlabel("x axis")
ax.set_ylabel("y axis")
ax.legend(loc = "best")
ax.set_title("This is a plot!")
```

```python
# subplots
fig = plt.figure()
ax = fig.add_subplot(121) # similar to matlab, (rows, columns, ordinal)
ax2 = fig.add_subplot(122)

hist, bins = np.histogram(np.random.uniform(size=1000))
ax.bar(bins[:-1], hist, np.diff(bins))
hist2, bins2 = np.histogram(np.random.normal(0.5, 0.15, size=1000))
ax2.bar(bins2[:-1], hist2, np.diff(bins2))
ax2.set_xlim([0,1])
```

## Example: 1-D convection

Very basic model of 1-D convection, PDE is:

\begin{equation}
\frac{\partial c}{\partial t} + a \frac{\partial c}{\partial x} = 0
\end{equation}

We can discretize the derivatives using the following:

\begin{equation}
\frac{c_i^{n + 1} - c_i^n}{\Delta t} + a \frac{c_i^{n} - c_{i-1}^n}{\Delta x} = 0
\end{equation}

and solve for $c^{n+1}_i$:
\begin{equation}
c_i^{n+1} = c_i^n - a\frac{\Delta t}{\Delta x} (c_i^n - c_{i-1}^n)
\end{equation}

Let's assume an initial distribution of c = 1 in the interval where 0.5 < x < 1 and c = 0 elsewhere in our domain with x from 0 to 2.

```python
x = np.linspace(0, 2, 101)
dx = np.diff(x)[0]
c = np.zeros(101)
c[np.logical_and(x < 1.0, x > 0.5)] = 1
plt.plot(x, c, 'k-')
plt.ylim(-0.1, 1.1)
```

```python
def conc_evolve(timesteps, dt, c_init, x, a):
    c = c_init.copy()
    dx = x[1] - x[0]
    for n in range(timesteps):
        cn = c.copy()
        for i in range(1, len(c)):
            c[i] = cn[i] - a * dt / dx * (cn[i] - cn[i-1])

    return c

# exercise, get rid of the forloop.
def conc_evolve_vectorized(timesteps, dt, c_init, x, a):
    c = c_init.copy()
    dx = np.diff(x)[0]
    for n in range(timesteps):
        c[1:] = c[1:] - a * dt / dx * (np.diff(c))
    return c

plt.plot(x, conc_evolve(50, 0.01, c, x, 1))
plt.plot(x, conc_evolve_vectorized(50, 0.01, c, x, 1))
```

## Linear convection in 2-dimensions

```python
from mpl_toolkits.mplot3d import Axes3D

x = y = np.linspace(0, 2, 201)
X, Y = np.meshgrid(x, y)
C = np.zeros(X.shape)
mask = np.logical_and(np.logical_and(X >= 0.5, X <= 1.0),
                      np.logical_and(Y >= 0.5, Y <= 1.0))
C[mask] = 1.0

fig = plt.figure(figsize = (11,7))
ax = fig.gca(projection='3d')

def conc_evolve_2d(timesteps, dt, C_init, X, Y, a):
    C = C_init.copy()
    for n in range(timesteps): ##loop across number of time steps
        Cn = C.copy()
        row, col = C.shape
        dx = dy = np.diff(X, axis=1)[0][0]
        for j in range(1, row):
            for i in range(1, col):
                C[j,i] = Cn[j, i] - (a*dt/dx*(Cn[j,i] - Cn[j,i-1]))-(a*dt/dy*(Cn[j,i]-Cn[j-1,i]))
    return C

ax.plot_surface(X, Y, conc_evolve_2d(100, 0.001, C, X, Y, a=1))
```

```python

```


# ルンゲ・クッタ法の誤差

```python
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
sns.set('notebook', 'whitegrid', 'dark', font_scale=2, rc={"lines.linewidth": 2, 'grid.linestyle': '--'})
```

## オイラー法

$x'=f(t,x)$の初期値問題に対して漸化式
$$
    x_{n+1} = x_n + h f(t_n, x_n)
$$
によって近似解を計算する

```python
def Euler(t, x, f, h):
    return x + h * f(t, x)
```

$x'=f(t,x)$の右辺の定義

```python
def func(t,x):
    return x
```

$x'=x, x(0)=1$に対してオイラー法で$t=1$における近似解を計算し，真の解$x(1)=e$との誤差を調べる

```python
x0 = 1.0 # 初期値
a = 0.0 # 初期時刻
b = 1.0 # 最終時刻
Steps = [2**n for n in range(4,17)]
H = [(b - a) / N for N in Steps]
```

```python
# いろんな刻み幅でx(1)の近似を計算する
Err = []
for n, N in enumerate(Steps):
    h = H[n]
    t = a
    x = x0
    for n in range(N):
        x = Euler(t, x, func, h)
        t = a + (n + 1) * h
    err = abs(x - np.exp(1))
    Err.append(err)
```

```python
fig = plt.figure(figsize=(8,4))
ax = fig.add_subplot(111)
ax.set_xscale('log')
ax.set_yscale('log')
ax.set_xlabel("$h$")
ax.set_ylabel("error")
ax.plot(H, Err, '-o', label="Euler")
ax.plot(H, H, '-', label="$h$")
ax.legend()
```

刻み幅$h$のときの誤差を$e_h$とおき，
$$
p_h := \log_2 \frac{e_{h}}{e_{2h}}
$$
を観察する．
1次精度なら$p_h$は$1$に近いはず．

```python
P1 = [np.log2(Err[k]/Err[k+1]) for k in range(0, len(Err)-1)]
fig = plt.figure(figsize=(8,4))
ax = fig.add_subplot(111)
ax.set_xscale('log')
ax.set_yscale('linear')
ax.plot(H[0:-1], P1, '-ob')
```

## ホイン法

\begin{align}
    k_1 &= f(t_n, x_n),\\
    k_2 &= f(t_n + h, x_n + h k_1),\\
    x_{n+1} &= x_n + \frac{h}{2} (k_1 + k_2)
\end{align}

```python
def Heun(t, x, f, h):
    k1 = f(t, x)
    k2 = f(t + h, x + h * k1)
    return x + 0.5 * h * (k1 + k2)
```

```python
Err2 = []
for n, N in enumerate(Steps):
    h = H[n]
    t = a
    x = x0
    for n in range(N):
        x = Heun(t, x, func, h)
        t = a + (n + 1) * h
    err = abs(x - np.exp(1))
    Err2.append(err)
```

```python
fig = plt.figure(figsize=(8,4))
ax = fig.add_subplot(111)
ax.set_xscale('log')
ax.set_yscale('log')
ax.set_xlabel("$h$")
ax.set_ylabel("error")
ax.plot(H, Err, '-o', label="Euler")
ax.plot(H, H, '-', label="$h$")
ax.plot(H, Err2, '-o', label="Heun")
ax.plot(H, np.array(H)**2, '-', label="$h^2$")
ax.legend()
```

```python
P2 = [np.log2(Err2[k]/Err2[k+1]) for k in range(0, len(Err)-1)]
fig = plt.figure(figsize=(8,4))
ax = fig.add_subplot(111)
ax.set_xscale('log')
ax.set_yscale('linear')
ax.plot(H[0:-1], P2, '-ob')
```

## 古典的ルンゲ・クッタ法
$$
\begin{aligned}
k_1 &= f(t_n, x_n),\\
k_2 &= f(t_n + \tfrac{h}{2}, x_n + \tfrac{h}{2}k_1),\\
k_3 &= f(t_n + \tfrac{h}{2}, x_n + \tfrac{h}{2}k_2),\\
k_4 &= f(t_n + h, x_n + h k_3),\\
x_{n+1} &= x_n + \frac{h}{6}(k_1 + 2 k_2 + 2 k_3 + k_4)
\end{aligned}
$$

```python
def RK4(t, x, f, h):
    k1 = f(t, x)
    k2 = f(t+0.5*h, x+0.5*h*k1)
    k3 = f(t+0.5*h, x+0.5*h*k2)
    k4 = f(t+h, x+h*k3)
    return x + h * (k1 + 2 * k2 + 2 * k3 + k4) / 6
```

```python
Err3 = []
for n, N in enumerate(Steps):
    h = H[n]
    t = a
    x = x0
    for n in range(N):
        x = RK4(t, x, func, h)
        t = a + (n + 1) * h
    err = abs(x - np.exp(1))
    Err3.append(err)
```

```python
fig = plt.figure(figsize=(8,6))
ax = fig.add_subplot(111)
ax.set_xscale('log')
ax.set_yscale('log')
ax.set_xlabel("$h$")
ax.set_ylabel("error")
ax.plot(H, Err, '-o', label="Euler")
ax.plot(H, H, '-', label="$h$")
ax.plot(H, Err2, '-x', label="Heun")
ax.plot(H, np.array(H)**2, '--', label="$h^2$")
ax.plot(H, Err3, '-v', label="RK4")
ax.plot(H, np.array(H)**4, '-.', label="$h^4$")
ax.legend()
# plt.savefig("rk_errors.pdf", bbox_inches='tight')
```

```python
P3 = [np.log2(Err3[k]/Err3[k+1]) for k in range(0, len(Err3)-1)]
fig = plt.figure(figsize=(8,4))
ax = fig.add_subplot(111)
ax.set_xscale('log')
ax.set_yscale('linear')
ax.plot(H[0:-1], P3, '-ob')
```


# Ecuación de la elasticidad lineal

## Ecuación diferencial parcial

Escribimos el problema de deformación elastica de un cuerpo con dominio geométrico $\Omega$ como
\begin{equation}
\rho \ddot{\boldsymbol{u}} + \rho \eta \dot{\boldsymbol{u}} - \boldsymbol{\nabla}\cdot\boldsymbol{\sigma} = \boldsymbol{f}\hbox{ in }\Omega,
\end{equation}

definiendo $\boldsymbol{\sigma}$ como el *tensor de esfuerzos*, $\boldsymbol{f}$ como la *fuerza de cuerpo por unidad de volumen* y $\boldsymbol{u}$ como el *vector de desplazamientos*. Además, las derivadas temporales se denotan aquí con un punto sobre la variable.

Restringiendo el problema a materiales isotrópicos, podemos definir el tensor de esfuerzos y su relación con las deformaciones como
\begin{align}
\boldsymbol{\sigma} &= \lambda\,\hbox{tr}\,(\boldsymbol{\varepsilon}) \boldsymbol{I} + 2\mu\boldsymbol{\varepsilon}, \\
\boldsymbol{\varepsilon} &= \frac{1}{2}\left(\boldsymbol{\nabla} \boldsymbol{u} + (\boldsymbol{\nabla} \boldsymbol{u})^{\top}\right),
\end{align}
donde $\boldsymbol{\varepsilon}$ es el *tensor simetrico de la tasa de deformación*, $\boldsymbol{I}$ denota el *tensor identidad*, $\mathrm{tr}$ denota la *traza* en un tensor, $\lambda$ y $\mu$
son los *parametros de Lamé* (propiedades del material) y $\rho$ es la densidad.

Si combinamos las definiciones de esfuerzo y deformación, podemos escribir el esfuerzo como
\begin{equation}
\boldsymbol{\boldsymbol{\sigma}} = \lambda(\boldsymbol{\nabla}\cdot \boldsymbol{u})\boldsymbol{I} + \mu(\boldsymbol{\nabla} \boldsymbol{u} + (\boldsymbol{\nabla} \boldsymbol{u})^{\top})
\end{equation}

## Formulación variacional

La formulación variacional del problema de deformación elastica la definimos como la integral sobre el dominio $\Omega$ del producto interno entre la ecuación y una función vectorial de test $\boldsymbol{v}\in \hat{V}$, donde $\hat{V}$ es el espacio de la funcion de test.

\begin{equation}
\int_\Omega \rho \ddot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x} + \int_\Omega \rho \eta \dot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x} -\int_\Omega (\boldsymbol{\nabla}\cdot\boldsymbol{\sigma}) \cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} = \int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x},
\end{equation}

Como $\boldsymbol{\nabla}\cdot\boldsymbol{\sigma}$ contiene derivadas de segundo orden en $\boldsymbol{u}$, debilitamos esta parte de la ecuación integrando por partes:

\begin{equation}
-\int_\Omega (\boldsymbol{\nabla}\cdot\boldsymbol{\sigma}) \cdot \boldsymbol{v} \ \mathrm{d}\boldsymbol{x}
= \int_\Omega \boldsymbol{\sigma} : \boldsymbol{\nabla} \boldsymbol{v} \ \mathrm{d}\boldsymbol{x} - \int_{\partial\Omega} (\boldsymbol{\sigma}\cdot \boldsymbol{n})\cdot \boldsymbol{v} \ \mathrm{d}\boldsymbol{s},
\end{equation}
donde $\boldsymbol{n}$ es vector normal en el contorno.

$\boldsymbol{\sigma}\cdot \boldsymbol{n}$ es conocido como la *tracción* (el esfuerzo en el contorno) y puede ser usado para prescribir una condicion de contorno de tipo Neumann o Robin de forma $\boldsymbol{\sigma}\cdot \boldsymbol{n} = \boldsymbol{T}$.

Combinando las 2 ecuaciones anteriores podemos escribir el problema variacional como

\begin{equation}
\int_\Omega \rho \ddot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x} + \int_\Omega \rho \eta \dot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x} + \int_\Omega \boldsymbol{\sigma} : \boldsymbol{\nabla} \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} =
\int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} + \int_{\partial\Omega} \boldsymbol{T}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{s}.
\end{equation}

Donde usando la definición de $\boldsymbol{\sigma}$ podemos escribir la ecuación variacional con $\boldsymbol{u}$ como incognita.

### Parte simetrica de  $\boldsymbol{\nabla} \boldsymbol{v}$

Ya que el producto de un tensor simetrico con un anti-simetrico es cero, y sabiendo que el tensor de esfuerzos $\boldsymbol{\sigma}$ es simetrico, podemos reemplazar en la ecuacion anterior $\boldsymbol{\nabla} \boldsymbol{v}$ por su parte simétrica $\boldsymbol{\epsilon}(\boldsymbol{v})$, dando como resultado

\begin{equation}
\int_\Omega \rho \ddot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x} + \int_\Omega \rho \eta \dot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x} + \int_\Omega \boldsymbol{\sigma} : \boldsymbol{\epsilon}(\boldsymbol{v})\ \mathrm{d}\boldsymbol{x} = \int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} + \int_{\partial\Omega} \boldsymbol{T}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{s}
\end{equation}

### Estableciendo las condiciones de contorno

Ahora consideremos cómo hacer cumplir las condiciones de contorno.
Para los contornos con condiciones de Dirichlet, aplicaremos las condiciones de manera fuerte. Para estos puntos, las funciones de test asociadas con los nodos de Dirichlet se desvanecen.

Para condiciones de tracción en los contornos, aplicaremos la forma variacional.
Similarmente a lo realizado con la ecuación de Poisson, requerimos que el espacio funcional de test correspondiente a
las funciones $ \boldsymbol{v} $ desaparezcan a lo largo de $ \partial \Omega $ para los puntos interiores.
Entonces, la integral de contorno anterior no tiene efectos para los puntos en
$ \partial \Omega \setminus \partial \Omega_T $.

### Forma variacional
En resumen, el problema variacional es el siguiente: encontrar $\boldsymbol{u}$ en un espacio funcional vectorial $\hat{V}$, tal que
\begin{equation}
m(\boldsymbol{u},\boldsymbol{v}) + c(\boldsymbol{u},\boldsymbol{v}) + a(\boldsymbol{u},\boldsymbol{v}) = L(\boldsymbol{v})\quad\forall \boldsymbol{v}\in\hat{V},
\end{equation}
donde
\begin{align}
m(\boldsymbol{u},\boldsymbol{v}) &= \int_\Omega \rho \ddot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x},\\
c(\boldsymbol{u},\boldsymbol{v}) &= \int_\Omega \rho \eta \dot{\boldsymbol{u}} \mathrm{d}\boldsymbol{x},\\
a(\boldsymbol{u},\boldsymbol{v}) &= \int_\Omega\sigma(\boldsymbol{u}) :\varepsilon(\boldsymbol{v})\ \mathrm{d}\boldsymbol{x},\\
L(\boldsymbol{v}) &= \int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} + \int_{\partial\Omega_T} \boldsymbol{T}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{s},
\end{align}

## Implementación en FEniCS

Modelamos una barra 3D empotrada en un extremo y con una carga aplicada en el otro. Adicionalmente incluimos el peso de la barra.

La carga y el peso pueden ser facilmente modeladas sumando al lado derecho de la ecuacion el termino $\boldsymbol{f}=(0,-\rho g + s,0)$, en donde $g$ es la magnitud de la aceleracion por gravedad y $s(t,x)$ es una distribución de carga cualquiera. La barra es un paralelepipedo de longitud $L$ y de sección transversal $W \times H$. Fijamos el desplazamiento en la parte empotrada ($x=0$) como $\boldsymbol{u}=(0,0,0)$, y el resto de la barra la dejamos libre de tracciones $\boldsymbol{T} = 0$. Entonces, el lado derecho de la forma variacional es de la forma
$$L(\boldsymbol{v}) = \int_\Omega \boldsymbol{f}\cdot \boldsymbol{v} \mathrm{d}\boldsymbol{x}$$

### Importar paquetes

Importamos los paquetes de solucion (**dolfin**), de mallado (**mshr**), de cálculo numérico (**numpy**) para realizar operaciones numericas de proceso y posproceso, así como **matplotlib** para ver los resultados directamente durante la ejecución del programa

```python
from dolfin import *
from mshr import *
import numpy as np
import matplotlib.pyplot as plt
from tqdm import tqdm
```

### Dominio geométrico, propiedades, malla y subdominios

Definimos el dominio computacional $\Omega$, definimos las propiedades, generamos la malla y creamos subdominios para aplicar la condición de contorno Dirichlet y la carga $s(t)$.

```python
# Dominio geometrico y malla
L = 1; W = 0.1; H =0.04

mesh = BoxMesh(Point(0, 0, 0), Point(L, W, H), 60, 10, 5)
x = SpatialCoordinate(mesh)

# Propiedades del material
# Constantes de Lame
E  = 1000.0
nu = 0.3
mu    = Constant(E / (2.0*(1.0 + nu)))
lmbda = Constant(E*nu / ((1.0 + nu)*(1.0 - 2.0*nu)))
# Densidad
rho = Constant(1.0)
# Coeficiente de amortiguacion
eta = Constant(0.1)

#Subdominios
# Izquierda
def left(x, on_boundary):
    return near(x[0], 0.) and on_boundary
# Derecha
def right(x, on_boundary):
    return near(x[0], 1.) and on_boundary
```

### Generación de los espacios funcionales sobre la malla

Una vez creada la malla, podemos crear un espacio funcional de elementos finitos $V$ sobre dicho dominio geométrico discreto:

```python
# Espacio de funciones para desplazamiento, velocidad y aceleraciones
V = VectorFunctionSpace(mesh, "Lagrange", 2)
# Espacio de funciones para esfuerzos
Vsig = TensorFunctionSpace(mesh, "DG", 0)
```

En este caso, tenemos un espacio funcional que es vectorial. Esto se puede entender como un arreglo de espacios funcionales escalares: uno para cada componente dimensional de la deformación del sólido elástico.

Usamos un elemento de tipo Lagrangeano (Lagrange) con un orden de interpolacion de primer orden. Adicionalmente creamos un espacio funcional discontinuo (DG) con el que representaremos los esfuerzos en el postproceso.

### Definición del problema variacional

La incógnita principal de este problema es un campo vectorial $ \boldsymbol{u} $ y no un campo escalar. Por lo que necesitamos trabajar con un espacio funcional vectorial.

```python
u = TrialFunction(V)
v = TestFunction(V)
```

Con `u = TrialFunction(V)` definimos `u` como un espacio funcional de elementos finitos vectorial con tres componentes: uno para cada dimensión de este problema tridimensional.

### Formas variacionales como funciones

Definimos las formas variacionales $m(\boldsymbol{u},\boldsymbol{v})$, $c(\boldsymbol{u},\boldsymbol{v})$, $a(\boldsymbol{u},\boldsymbol{v})$ y $L(\boldsymbol{v})$, la funcion para calcular los esfuerzos $\boldsymbol{\sigma}(\boldsymbol{u})$ (`sigma`) y una función de proyecccion local para representar el tensor de esfuerzos en el postproceso.

Todas estas funciones las definimos como funciones de python, usando las funciones intrinsecas de FEniCS (`inner`, `grad`, `sym`).

```python
# Tensor de esfuerzos
def sigma(u):
    return 2.0*mu*sym(grad(u)) + lmbda*tr(sym(grad(u)))*Identity(len(u))

# Matriz de masa
def mmat(u,v):
    return rho*inner(u, v)*dx

# Matriz de rigidez elastica
def kmat(u, v):
    return inner(sigma(u), sym(grad(v)))*dx

# Amortiguacion de Rayleigh
def cmat(u, u_):
    return eta*mmat(u, v)

# Proyeccion local
def local_project(v, V, u=None):
    dv = TrialFunction(V)
    v_ = TestFunction(V)
    a_proj = inner(dv, v_)*dx
    b_proj = inner(v, v_)*dx
    solver = LocalSolver(a_proj, b_proj)
    solver.factorize()
    if u is None:
        u = Function(V)
        solver.solve_local_rhs(u)
        return u
    else:
        solver.solve_local_rhs(u)
        return
```

### Definición de condiciones de contorno

Especificamos la condición de contorno de Dirichlet $u=(0,0,0)$ en el subdominio izquierdo (usando la funcion `DirichletBC`) y definimos el area donde aplicamos la carga $s(t)$ (lado derecho).

```python
# Condicion de contorno izquierda (Dirichlet)
zero = Constant((0.0, 0.0, 0.0))
bc = DirichletBC(V, zero, left)

# Condicion de contorno derecha
boundary_subdomains = MeshFunction("size_t", mesh, mesh.topology().dim() - 1)
boundary_subdomains.set_all(0)
force_boundary = AutoSubDomain(right)
force_boundary.mark(boundary_subdomains, 3)
dss = ds(subdomain_data=boundary_subdomains)
```

### Definicion de las fuerzas de cuerpo

Definimos la fuerza de cuerpo $\boldsymbol{f}$ compuesta por el peso de la barra $\rho g$ y la función temporal en el lado derecho $s(t)$

```python
# Definicion de las cargas
k = 0.5
s0 = 1.
s = Expression(("0", "s0*sin(k*pi*t)","0"), t=0, k=k, s0=s0, degree=0)
g = Constant((0.,-1,0.))
```

### Integrador temporal

Usamos el método de Newmark como integrador temporal para las derivadas temporales del problema. Creamos funciones para actualizar los campos de desplazamiento, velocidad y aceleracion en cada paso de tiempo. Tambien definimos campos vectoriales `u_old`, `v_old` y `a_old`, que vamos a usar en la integracion temporal.

Tambien definimos el tiempo total de simulación y el paso de tiempo.

```python
T  = 10.0 # tiempo total
dt = 0.5 # paso de tiempo
Nt = int(T/dt)
ti = np.linspace(0, T, Nt)

# Campos para el paso de tiempo anterior
u_old = Function(V, name="Desplazamiento")
v_old = Function(V, name="Velocidad")
a_old = Function(V, name="Aceleracion")

# Metodo de Newmark (punto medio)
gamma   = Constant(0.5)
beta    = Constant((gamma+0.5)**2/4.)

# Aceleracion
# a = 1/(2*beta)*((u - u0 - v0*dt)/(0.5*dt*dt) - (1-2*beta)*a0)
def update_a(u, u_old, v_old, a_old, ufl=True):
    if ufl:
        dt_ = dt
        beta_ = beta
    else:
        dt_ = float(dt)
        beta_ = float(beta)
    return (u-u_old-dt_*v_old)/beta_/dt_**2 - (1-2*beta_)/2/beta_*a_old

# Velocidad
# v = dt * ((1-gamma)*a0 + gamma*a) + v0
def update_v(a, u_old, v_old, a_old, ufl=True):
    if ufl:
        dt_ = dt
        gamma_ = gamma
    else:
        dt_ = float(dt)
        gamma_ = float(gamma)
    return v_old + dt_*((1-gamma_)*a_old + gamma_*a)

# Actualizar campos
def update_fields(u, u_old, v_old, a_old):
    """Update fields at the end of each time step."""

    # Vectores de referencia
    u_vec, u0_vec  = u.vector(), u_old.vector()
    v0_vec, a0_vec = v_old.vector(), a_old.vector()

    # Actualizacion de velocidad y aceleracion
    a_vec = update_a(u_vec, u0_vec, v0_vec, a0_vec, ufl=False)
    v_vec = update_v(a_vec, u0_vec, v0_vec, a0_vec, ufl=False)

    # Actualizacion de desplazamiento
    v_old.vector()[:], a_old.vector()[:] = v_vec, a_vec
    u_old.vector()[:] = u.vector()
```

### Definicion del problema variacional

Usando todos los ingredientes anteriores, escribimos el problema variacional y exprezamos el problema en forma matricial $\boldsymbol{K} \boldsymbol{u} = \boldsymbol{b}$

```python
# Ecuacion de elasticidad en forma residual
a_new = update_a(u, u_old, v_old, a_old, ufl=True)
v_new = update_v(a_new, u_old, v_old, a_old, ufl=True)
res = mmat(a_new,v) + cmat(v_new,v) \
       + kmat(u,v) - rho*dot(s,v)*dss(3) - dot(rho*g,v)*dx
# Ensamble del sistema de ecuaciones
B_form = lhs(res)
L_form = rhs(res)
K, b = assemble_system(B_form, L_form, bc)
```

    Calling FFC just-in-time (JIT) compiler, this may take some time.
    Calling FFC just-in-time (JIT) compiler, this may take some time.

### Arrays de postproceso e impresion de datos

Creamos arrays de numpy para guardar algunos resultados y abrimos el archivo de **XDMF** donde vamos a almacenar los datos de postproceso

```python
# Inicializacion de variables de salida
u_tip = np.zeros((Nt,)) # desplazamiento en lado derecho de la barra
E_damp = 0
energia = np.zeros((Nt, 2))

# Creacion de archivos de salida (ParaView)
xdmf_file = XDMFFile("bar.xdmf")
xdmf_file.parameters["flush_output"] = True
xdmf_file.parameters["functions_share_mesh"] = True
xdmf_file.parameters["rewrite_function_mesh"] = False
```

## Solución del problema

En esta sección finalmente resolvemos el problema. Iniciamos definiendo nuestro campo vectorial para la solución `u` y uno donde calcularemos el tensor de esfuerzos `sig`.
Usamos el ciclo `for` para iterar sobre todos los pasos de tiempo, y en este bucle evaluamos la fuerza $s(t)$ en el paso de tiempo actual, ensamblamos la forma $L(v)$ y solucionamos el problema.
Finalmente, actualizamos los campos de desplazamiento, velocidad, aceleración y esfuerzo, y guardamos los datos necesarios para el postproceso.

```python
u = Function(V, name="Desplazamiento")
sig = Function(Vsig, name="sigma")

# Propiedades del solver
solver = LUSolver(K, "mumps")
solver.parameters["symmetric"] = True

# Bucle temporal
for i in tqdm(range(Nt),desc='Time loop'):

    t = ti[i]

    # Fuerza evaluada en t_{n+1}=t_{n+1}-dt
    s.t = t

    # Solucionar desplazamientos
    b = assemble(L_form)
    bc.apply(b)
    solver.solve(K, u.vector(), b)

    # Actualizar campos
    update_fields(u, u_old, v_old, a_old)

    # Calcular esfuerzos
    local_project(sigma(u), Vsig, sig)

    # Guardar solucion (vizualizacion)
    xdmf_file.write(u_old, t)
    xdmf_file.write(v_old, t)
    xdmf_file.write(a_old, t)
    xdmf_file.write(sig, t)

    # Guardar solucion (desplazamiento y energia)
    u_tip[i] = u(1., 0.05, 0.02)[1]
    E_elas = assemble(0.5*kmat(u_old, u_old))
    E_kin = assemble(0.5*mmat(v_old, v_old))
    energia[i, :] = np.array([E_elas, E_kin])
```

    Time loop: 100%|██████████| 20/20 [01:31<00:00,  4.59s/it]

## Post-proceso

### Visualización y postprocesamiento interno de la solución

Usando el array `u_tip`, podemos graficar el desplazamiento en el lado derecho de la barra

```python
# Imprimir desplazamiento en x=(1,0.05,0.02)
plt.figure()
plt.plot(ti, u_tip)
plt.xlabel("Time")
plt.ylabel("Desplazamiento")
plt.savefig('images/tip.png',format='png',dpi=400)
plt.show()
```

Usando las soluciones de desplazamiento `u_old` y velocidad `v_old`, podemos definir la energia elastica como $E_e = \int_\Omega \frac{1}{2} \boldsymbol{\sigma}(\boldsymbol{u}):\boldsymbol{\epsilon}(\boldsymbol{u}) dx$ y la energia cinetica como $E_k = \int_\Omega \frac{1}{2} \rho \dot{\boldsymbol{u}} \cdot\dot{ \boldsymbol{u}} dx$

```python
# Imprimir energia
plt.figure()
plt.plot(ti, energia)
plt.legend(("elastica", "cinetica", "amortiguacion"))
plt.xlabel("Time")
plt.ylabel("Energia")
plt.savefig('images/energy.png',format='png',dpi=400)
plt.show()
```

**Reconocimiento**: Este cuaderno fue adaptado de [The FEniCS Tutorial Volume I] (https://fenicsproject.org/pub/tutorial/sphinx1/) elaborado por Hans Petter Langtangen y Anders Logg, publicado bajo licencia CC Attribution 4.0.


> https://www.hebergementwebs.com/tutorial-de-sympy/sympy-guia-rapida

## Sympy con graficas

```python
import matplotlib
```

```python
from sympy.plotting import plot
```

```python
from sympy import *
```

```python
x = Symbol ( 'x')
```

```python
x
```

$\displaystyle x$

```python
plot (x ** 2, line_color = 'fuchsia')
```

```python
plot (x ** 2, line_color = '#e30052')
```

```python
plot (x ** 2)
```

```python
plot (sin (x), cos (x), (x, -pi, pi))
```

```python
plot ((sin (x), (x, -2 * pi, 2 * pi)), (cos (x), (x , -pi, pi)))
```

```python
plot ((sin (x), (x, -2 * pi, 2 * pi)), (cos (x), (x, - pi, pi) ), line_color = 'green', title = 'Ejemplo grafico con SymPy en DIGITALEXA')
```

```python
##https://www.cfm.brown.edu/people/dobrush/am33/SymPy/part1.html
```

## Graficas en 3D

> La funcion **plot3d ()** genera un grafico tridimensional. **plot3d (expr, xrange, yrange, kwarg s)**

```python
from sympy.plotting import plot3d
```

```python
from sympy import *
```

```python
x = Symbol('x')
```

```python
y = Symbol('y')
```

```python
x, y
```

    (x, y)

```python
x
```

$\displaystyle x$

```python
y
```

$\displaystyle y$

### El siguiente ejemplo dibuja un grafico de superficie 3D

```python
plot3d (x * y, (x, -10,10), (y, -10,10))
```

```python
plot3d (x * y, x / y, (x, -5, 5), (y, -5, 5))
```

```python
plot3d(x*y, -x*y, (x, -5, 5), (y, -5, 5))
```

```python
 plot3d((x**2 + y**2, (x, -5, 5), (y, -5, 5)),
...     (x*y, (x, -3, 3), (y, -3, 3)))
```

> La funcion **plot3d_parametric_line ()** representa un grafico lineal parametrico en **3 dimensiones.**

```python
from sympy.plotting import plot3d_parametric_line
```

```python
plot3d_parametric_line (cos (x), sin (x), x, (x, -5, 5))
```

 ### Para dibujar una grafica de superficie parametrica, use la funcion plot3d_parametric_surface ().

> plot3d_parametric_surface (xexpr, yexpr, zexpr, rangex, rangey, kwargs)

```python
##https://www.cfm.brown.edu/people/dobrush/am33/SymPy/part1.html
```

```python
from sympy import symbols, cos, sin
from sympy.plotting import plot3d_parametric_surface
u, v = symbols('u v')
```

```python
plot3d_parametric_surface(cos(u + v), sin(u - v), u - v,(u, -5, 5), (v, -5, 5))
```

```python
u, v
```

    (u, v)

```python
u
```

$\displaystyle u$

```python
v
```

$\displaystyle v$

```python
from sympy import plot_implicit, cos, sin, symbols, Eq, And
x, y = symbols('x y')
```

```python
# Using mesh grid with number of points as input.
```

```python
p5 = plot_implicit(Eq(x**2 + y**2, 5),
...         (x, -5, 5), (y, -2, 2),
...         adaptive=False, points=400)
```

```python
# Plotting regions.
```

```python
p6 = plot_implicit(y > x**2)
```

```python
# Plotting Using boolean conjunctions.
```

```python
p7 = plot_implicit(And(y > x, y > -x))
```

```python

```


```python
import symbolic_modern_robotics as smr
import sympy as s
s.init_printing()
```

```python
# unit rotation axis
omega_hat = s.Matrix([[0],[0],[1]])
theta = s.pi/4
# 3-vector form  of angular velocity
omega = omega_hat*theta
omega
```

$\displaystyle \left[\begin{matrix}0\\0\\\frac{\pi}{4}\end{matrix}\right]$

```python
# get Unit axis of Rotation and theta from 3-vector of exponential coordinates
smr.AxisAng3(omega)
```

$\displaystyle \left( \left[\begin{matrix}0\\0\\1.0\end{matrix}\right], \  0.785398163397448\right)$

```python
# get so3(skew-symmetric matrix) from 3-vector of exponential coordinates
so3 = smr.VecToso3(omega)
so3
```

$\displaystyle \left[\begin{matrix}0 & -0.785398163397448 & 0\\0.785398163397448 & 0 & 0\\0 & 0 & 0\end{matrix}\right]$

```python
# get 3-vector of exponential coordinates from so3(skew-symmetric matrix)
omega = smr.so3ToVec(so3)
omega
```

$\displaystyle \left[\begin{matrix}0\\0\\0.785398163397448\end{matrix}\right]$

```python
# get SO3(Rotation matrix) from so3(skew-symmetric matrix)
R = smr.MatrixExp3(so3)
R
```

$\displaystyle \left[\begin{matrix}0.707106781186548 & -0.707106781186547 & 0\\0.707106781186547 & 0.707106781186548 & 0\\0 & 0 & 1.0\end{matrix}\right]$

```python
# get so3(skew-symmetric matrix) from SO3(Rotation matrix)
so3 = smr.MatrixLog3(R)
so3
```

$\displaystyle \left[\begin{matrix}0 & -0.785398163397448 & 0\\0.785398163397448 & 0 & 0\\0 & 0 & 0\end{matrix}\right]$

```python
# 6-vector form twist, V
V = s.Matrix([0,0,0,1,2,3])
V
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\1\\2\\3\end{matrix}\right]$

```python
# get se3(twist 4x4 matrix) from 6-vector spatial velocity, V
se3 = smr.VecTose3(V)
se3
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 1.0\\0 & 0 & 0 & 2.0\\0 & 0 & 0 & 3.0\\0 & 0 & 0 & 0\end{matrix}\right]$

```python
V = smr.se3ToVec(se3)
```

```python
S, theta_dot = smr.AxisAng6(V)
S, theta_dot
```

$\displaystyle \left( \left[\begin{matrix}0\\0\\0\\0.267261241912424\\0.534522483824849\\0.801783725737273\end{matrix}\right], \  3.74165738677394\right)$

```python
# get 6-vector spatial velocity, V from se3(twist 4x4 matrix)
V = smr.se3ToVec(se3)
V
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\1.0\\2.0\\3.0\end{matrix}\right]$

```python
T = smr.MatrixExp6(se3)
T
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 1.0\\0 & 1 & 0 & 2.0\\0 & 0 & 1 & 3.0\\0 & 0 & 0 & 1\end{matrix}\right]$

```python
se3 = smr.MatrixLog6(T)
se3
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 1.0\\0 & 0 & 0 & 2.0\\0 & 0 & 0 & 3.0\\0 & 0 & 0 & 0\end{matrix}\right]$

```python
help(smr.so3ToVec)
```

    Help on function so3ToVec in module symbolic_modern_robotics:

    so3ToVec(so3mat)
        ###### Rotation Matrix
        # Calculate 3-vector from so3

```python

```

```python

```


# we have
P_mean = 100
P_std = 15
S_mean = 108
Sample_no = 36

H0 : mu = 100

H1 : mu != 100

let say , alpha = 0.05

\begin{align}
Z = (X - mu) / (std / \sqrt{n}
\end{align}
\begin{align}
Z_{0.05} = 1.96
\end{align}
\begin{align}
Z = (108 - 100) / (15 / \sqrt{36} = 3.2
\end{align}

since Z > Z_{0.05} i.e. 3.2 > 1.96 , we will reject the null hypothesis at 5% level of significance.

let P1 be the probability for Repulicans first state and P2 be the Probabiity for Repulicans for second state.
hence, P1 = 52% = 0.52
       P2 = 47% = 0.47

sample size = 100

so the difference in proportion between two state be p

P = P1 - P2
  = 0.52 - 0.47
  = 0.05

calculating the variance for Republicans.

Pvariance   = P1(1-P1)/sample_size + P2(1-P2)/sample_size
      = 0.52(1-0.52)/100 + 0.47(1-0.47)/100
      = (0.52 * 0.48 + 0.47 * 0.53)/100
      = 0.00498

std = sqrt(0.004987) = 0.071
the difference between the first and second state of Republicans need to less than 0 in orer to achieve the greater percentage in second state.
hence the the valud of the difference is aproxmetly equal to 0 so p ~ 0

hence,

 Z = 0 - 0.05/ 0.071 = -0.704
 using Z score table , the value for -704 = 0.239
 thus the probability for greater percentage of Republicans in second state than in first state is 23.9 %

We have ,

X = 1100
mu = 1026
std = 209
hence calculating the Z-score value

Z = (1100-1026)/209 = 0.354

hence, te score is 35.4 5 standard deviation above the mean score in SAT.

using the z-score table for z(0.354)  = 0.636. ie. 63.6% chnage that the score is more than the average test takers.

```python

```


## RIHAD VARIAWA, Data Scientist - Who has fun LEARNING, EXPLORING & GROWING
## Systems of Equations
Imagine you are at a casino, and you have a mixture of £10 and £25 chips. You know that you have a total of 16 chips, and you also know that the total value of chips you have is £250. Is this enough information to determine how many of each denomination of chip you have?

Well, we can express each of the facts that we have as an equation. The first equation deals with the total number of chips - we know that this is 16, and that it is the number of £10 chips (which we'll call ***x*** ) added to the number of £25 chips (***y***).

The second equation deals with the total value of the chips (£250), and we know that this is made up of ***x*** chips worth £10 and ***y*** chips worth £25.

Here are the equations

\begin{equation}x + y = 16 \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

Taken together, these equations form a *system of equations* that will enable us to determine how many of each chip denomination we have.

## Graphing Lines to Find the Intersection Point
One approach is to determine all possible values for x and y in each equation and plot them.

A collection of 16 chips could be made up of 16 £10 chips and no £25 chips, no £10 chips and 16 £25 chips, or any combination between these.

Similarly, a total of £250 could be made up of 25 £10 chips and no £25 chips, no £10 chips and 10 £25 chips, or a combination in between.

Let's plot each of these ranges of values as lines on a graph:

```python
%matplotlib inline
from matplotlib import pyplot as plt

# Get the extremes for number of chips
chipsAll10s = [16, 0]
chipsAll25s = [0, 16]

# Get the extremes for values
valueAll10s = [25,0]
valueAll25s = [0,10]

# Plot the lines
plt.plot(chipsAll10s,chipsAll25s, color='blue')
plt.plot(valueAll10s, valueAll25s, color="orange")
plt.xlabel('x (£10 chips)')
plt.ylabel('y (£25 chips)')
plt.grid()

plt.show()
```

Looking at the graph, you can see that there is only a single combination of £10 and £25 chips that is on both the line for all possible combinations of 16 chips and the line for all possible combinations of £250. The point where the line intersects is (10, 6); or put another way, there are ten £10 chips and six £25 chips.

### Solving a System of Equations with Elimination
You can also solve a system of equations mathematically. Let's take a look at our two equations:

\begin{equation}x + y = 16 \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

We can combine these equations to eliminate one of the variable terms and solve the resulting equation to find the value of one of the variables. Let's start by combining the equations and eliminating the x term.

We can combine the equations by adding them together, but first, we need to manipulate one of the equations so that adding them will eliminate the x term. The first equation includes the term ***x***, and the second includes the term ***10x***, so if we multiply the first equation by -10, the two x terms will cancel each other out. So here are the equations with the first one multiplied by -10:

\begin{equation}-10(x + y) = -10(16) \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

After we apply the multiplication to all of the terms in the first equation, the system of equations look like this:

\begin{equation}-10x + -10y = -160 \end{equation}
\begin{equation}10x + 25y = 250 \end{equation}

Now we can combine the equations by adding them. The ***-10x*** and ***10x*** cancel one another, leaving us with a single equation like this:

\begin{equation}15y = 90 \end{equation}

We can isolate ***y*** by dividing both sides by 15:

\begin{equation}y = \frac{90}{15} \end{equation}

So now we have a value for ***y***:

\begin{equation}y = 6 \end{equation}

So how does that help us? Well, now we have a value for ***y*** that satisfies both equations. We can simply use it in either of the equations to determine the value of ***x***. Let's use the first one:

\begin{equation}x + 6 = 16 \end{equation}

When we work through this equation, we get a value for ***x***:

\begin{equation}x = 10 \end{equation}

So now we've calculated values for ***x*** and ***y***, and we find, just as we did with the graphical intersection method, that there are ten £10 chips and six £25 chips.

You can run the following Python code to verify that the equations are both true with an ***x*** value of 10 and a ***y*** value of 6.

```python
x = 10
y = 6
print ((x + y == 16) & ((10*x) + (25*y) == 250))
```

    True

```python

```

AI Professional/2 - Essential Mathematics For Artificial Intelligence/DAT256x/Module01/01-03-Systems of Equations.ipynb


# Contravariant & Covariant indices in Tensors (Symbolic)

```python
import sympy
from einsteinpy.symbolic import ChristoffelSymbols, RiemannCurvatureTensor
from einsteinpy.symbolic.predefined import Schwarzschild

sympy.init_printing()
```

### Analysing the schwarzschild metric along with performing various operations

```python
sch = Schwarzschild()
sch.tensor()
```

$\displaystyle \left[\begin{matrix}1 - \frac{r_{s}}{r} & 0 & 0 & 0\\0 & - \frac{1}{c^{2} \left(1 - \frac{r_{s}}{r}\right)} & 0 & 0\\0 & 0 & - \frac{r^{2}}{c^{2}} & 0\\0 & 0 & 0 & - \frac{r^{2} \sin^{2}{\left(\theta \right)}}{c^{2}}\end{matrix}\right]$

```python
sch_inv = sch.inv()
sch_inv.tensor()
```

$\displaystyle \left[\begin{matrix}\frac{r}{r - r_{s}} & 0 & 0 & 0\\0 & \frac{c^{2} \left(- r + r_{s}\right)}{r} & 0 & 0\\0 & 0 & - \frac{c^{2}}{r^{2}} & 0\\0 & 0 & 0 & - \frac{c^{2}}{r^{2} \sin^{2}{\left(\theta \right)}}\end{matrix}\right]$

```python
sch.order
```

```python
sch.config
```

    'll'

### Obtaining Christoffel Symbols from Metric Tensor

```python
chr = ChristoffelSymbols.from_metric(sch_inv) # can be initialized from sch also
chr.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & \frac{r_{s}}{2 r^{2} \left(1 - \frac{r_{s}}{r}\right)} & 0 & 0\\\frac{r_{s}}{2 r^{2} \left(1 - \frac{r_{s}}{r}\right)} & 0 & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}- \frac{r_{s} \left(- \frac{c^{2}}{2} + \frac{r_{s} c^{2}}{2 r}\right)}{r^{2}} & 0 & 0 & 0\\0 & \frac{r_{s} \left(- \frac{c^{2}}{2} + \frac{r_{s} c^{2}}{2 r}\right)}{r^{2} c^{2} \left(1 - \frac{r_{s}}{r}\right)^{2}} & 0 & 0\\0 & 0 & \frac{2 r \left(- \frac{c^{2}}{2} + \frac{r_{s} c^{2}}{2 r}\right)}{c^{2}} & 0\\0 & 0 & 0 & \frac{2 r \left(- \frac{c^{2}}{2} + \frac{r_{s} c^{2}}{2 r}\right) \sin^{2}{\left(\theta \right)}}{c^{2}}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & \frac{1}{r} & 0\\0 & \frac{1}{r} & 0 & 0\\0 & 0 & 0 & - \sin{\left(\theta \right)} \cos{\left(\theta \right)}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & 0 & \frac{1}{r}\\0 & 0 & 0 & \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}}\\0 & \frac{1}{r} & \frac{\cos{\left(\theta \right)}}{\sin{\left(\theta \right)}} & 0\end{matrix}\right]\end{matrix}\right]$

```python
chr.config
```

    'ull'

### Changing the first index to covariant

```python
new_chr = chr.change_config('lll') # changing the configuration to (covariant, covariant, covariant)
new_chr.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & \frac{r_{s}}{2 r^{2}} & 0 & 0\\\frac{r_{s}}{2 r^{2}} & 0 & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}- \frac{r_{s}}{2 r^{2}} & 0 & 0 & 0\\0 & \frac{r_{s}}{2 c^{2} \left(r - r_{s}\right)^{2}} & 0 & 0\\0 & 0 & \frac{r}{c^{2}} & 0\\0 & 0 & 0 & \frac{r \sin^{2}{\left(\theta \right)}}{c^{2}}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & - \frac{r}{c^{2}} & 0\\0 & - \frac{r}{c^{2}} & 0 & 0\\0 & 0 & 0 & \frac{r^{2} \sin{\left(2 \theta \right)}}{2 c^{2}}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & 0 & - \frac{r \sin^{2}{\left(\theta \right)}}{c^{2}}\\0 & 0 & 0 & - \frac{r^{2} \sin{\left(2 \theta \right)}}{2 c^{2}}\\0 & - \frac{r \sin^{2}{\left(\theta \right)}}{c^{2}} & - \frac{r^{2} \sin{\left(2 \theta \right)}}{2 c^{2}} & 0\end{matrix}\right]\end{matrix}\right]$

```python
new_chr.config
```

    'lll'

### Any arbitary index configuration would also work!

```python
new_chr2 = new_chr.change_config('lul')
new_chr2.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & \frac{r_{s}}{2 r \left(r - r_{s}\right)} & 0 & 0\\\frac{r_{s} c^{2} \left(- r + r_{s}\right)}{2 r^{3}} & 0 & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}- \frac{r_{s}}{2 r \left(r - r_{s}\right)} & 0 & 0 & 0\\0 & - \frac{r_{s}}{2 r \left(r - r_{s}\right)} & 0 & 0\\0 & 0 & - \frac{1}{r} & 0\\0 & 0 & 0 & - \frac{1}{r}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & r - r_{s} & 0\\0 & \frac{1}{r} & 0 & 0\\0 & 0 & 0 & - \frac{1}{\tan{\left(\theta \right)}}\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0 & 0\\0 & 0 & 0 & \left(r - r_{s}\right) \sin^{2}{\left(\theta \right)}\\0 & 0 & 0 & \frac{\sin{\left(2 \theta \right)}}{2}\\0 & \frac{1}{r} & \frac{1}{\tan{\left(\theta \right)}} & 0\end{matrix}\right]\end{matrix}\right]$

### Obtaining Riemann Tensor from Christoffel Symbols and manipulating it's indices

```python
rm = RiemannCurvatureTensor.from_christoffels(new_chr2)
rm[0,0,:,:]
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 0\\0 & - \frac{r_{s}}{r^{2} \left(r - r_{s}\right)} & 0 & 0\\0 & 0 & \frac{r_{s}}{2 r} & 0\\0 & 0 & 0 & \frac{r_{s} \sin^{2}{\left(\theta \right)}}{2 r}\end{matrix}\right]$

```python
rm.config
```

    'ulll'

```python
rm2 = rm.change_config("uuuu")
rm2[0,0,:,:]
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 0\\0 & - \frac{r_{s} c^{4}}{r^{3}} & 0 & 0\\0 & 0 & \frac{r_{s} c^{4}}{2 r^{4} \left(r - r_{s}\right)} & 0\\0 & 0 & 0 & \frac{r_{s} c^{4}}{2 r^{4} \left(r - r_{s}\right) \sin^{2}{\left(\theta \right)}}\end{matrix}\right]$

```python
rm3 = rm2.change_config("lulu")
rm3[0,0,:,:]
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 0\\0 & \frac{r_{s} c^{2}}{r^{3}} & 0 & 0\\0 & 0 & - \frac{r_{s} c^{2}}{2 r^{3}} & 0\\0 & 0 & 0 & - \frac{r_{s} c^{2}}{2 r^{3}}\end{matrix}\right]$

```python
rm4 = rm3.change_config("ulll")
rm4.simplify()
rm4[0,0,:,:]
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 0\\0 & - \frac{r_{s}}{r^{2} \left(r - r_{s}\right)} & 0 & 0\\0 & 0 & \frac{r_{s}}{2 r} & 0\\0 & 0 & 0 & \frac{r_{s} \sin^{2}{\left(\theta \right)}}{2 r}\end{matrix}\right]$

#### It is seen that `rm` and `rm4` are same as they have the same configuration

docs/source/examples/Symbolic Module - Tensor Indexing.ipynb