# Summary - KNN Algorithm
> Training process and useful points to know about KNN
- toc: true
- badges: true
- comments: true
- image: https://helloacm.com/wp-content/uploads/2016/03/2012-10-26-knn-concept.png
- author: Uday Paila
- categories: [KNN]

## Terminology
$N$ - Number of data points
$X_q$ - query data points
$X_{qn}$ - nth query data point
$X$ - input train data
$D$ - dimensionality of data
$C$ - Number of classes
$C_i$ - i^th class
$N_k$ - K nearst neighbors
$m$ - Number of epochs in SGD

This blog was originally published in [this](https://applied-ai-course.github.io/blog/knn/2020/08/10/KNN-Summary.html) link.

## Algorithm
- Given a query point $X_{qn}$, find the $k$ nearest points in $X$. (You can use any distance metric)
- Count which class has maximum points in those $k$ nearest points and classify given query point $X_{qn}$ belongs to the same class. We can give weightage to the nearest points based on the distance (any function of distance).
- Probability of belonging to the specific class in a classification scenario is

\begin{align}
P(Y=C_i|X_q) = \frac{1}{K}\sum_{i\epsilon N_k}I(y_i==C_i) \\I(true) = 1, I(false) = 0
\end{align}

- If we use KNN to solve the regression problem, we can get the average of nearest points and gives as output.

\begin{align}
Y_{X_q} = \frac{1}{K}\sum_{i\epsilon N_k}y_i
\end{align}

## Useful points to know
- There is no need for any specific process to work with multi-class classification because it works based on nearest neighbors.
- There is no need for training if we want to use brute force search to get k-nearest neighbors. If we want to use Tree-based/LSH based/graph-based searching, we have to create a corresponding Tree/LSH forest/graph in the training time.

- Testing time complexity is $O(ND)$ if we use brute force search, it is very huge if we have more data points to train. If we need less time complexity go for Tree/LSH/graph-based neighbor search. Many tree algorithms give complexity of $O(D*log(N))$.

>Important: You can check the benchmarking of NN searching algorithms in [this](https://arxiv.org/pdf/1807.05614.pdf) paper and  [this](https://github.com/erikbern/ann-benchmarks) GitHub

- It works based on distance measure so scaling the features is very important.

- Choosing a distance metric is very crucial. If we require a rotation-invariant distance metric then Euclidean distance is one of the best choices. But in high dimensions, a curious phenomenon arises: the ratio between the nearest and farthest points approaches 1, i.e. the points essentially become uniformly distant from each other. This phenomenon can be observed for a wide variety of distance metrics, but it is more pronounced for the Euclidean metric than, say, the Manhattan distance metric. You can read more about this in [this](https://bib.dbvis.de/uploadedFiles/155.pdf) paper.

>Tip: High-dimensional spaces tend to be extremely sparse, which means that every point is far away from virtually every other point, and hence pairwise distances tend to be uninformative so before applying nearest-neighbor classification it is a good idea to plot a histogram of pairwise distances of a data to see if they are sufficiently varied.

- High-dimensional spaces may not give better results so using one-hot vectors may not be useful if have more categorical variables. so, try with mean encoding/frequency-based encoding or use dimensionality reduction techniques/feature selection to get the lower dimensionality data.

- This is not a linear model, so you can classify the non-linear data.

- An increase in the feature value not always leads to an increase or decrease in the target outcome/probability so, it is not a monotone model.

- It won't consider the interaction between the features. We have to create the interaction features if we need it. More interaction features may lead to less interpretability.

## Hyperparameters
- K is the Hyperparameter in K-NN.
- The low value of K will give more variance to the model. increasing the K, reduces the variance, and increases the bias of the model.
- You can check the decision surface of KNN for different K values in the below image.

## Interpretability
- There is a lack of global model interpretability because the model is inherently local and there are no global weights or structures explicitly learned.
- We can get interpretability based on the data we have i.e. datapoint $x_1$ is similar to the datapoint $x_2$ and $x_1$ caused $y$ so we predict that $x_2$ will cause $y$ as well.  So, we can get the k nearest neighbors and we can analyze/interpret those data points.
- If we have thousands of features, it is very difficult to analyze and get interpretability.

## References
1. https://stackexchange.com/
2. https://arxiv.org/pdf/1807.05614.pdf
3. https://bib.dbvis.de/uploadedFiles/155.pdf
4. https://helloacm.com/wp-content/uploads/2016/03/2012-10-26-knn-concept.png

```python

```


Computational Seismology

Finite Element Method - 1D Elastic Wave Equation

Seismo-Live: http://seismo-live.org

##### Authors:
* David Vargas ([@dvargas](https://github.com/davofis))
* Heiner Igel ([@heinerigel](https://github.com/heinerigel))

## Basic Equations
This notebook presents a finite element code for the 1D elastic wave equation. Additionally, a solution using finite difference scheme is given for comparison.

The problem of solving the wave equation

\begin{equation}
\rho(x) \partial_t^2 u(x,t) = \partial_x (\mu(x) \partial_x u(x,t)) + f(x,t)
\end{equation}

using the finite element method is done after a series of steps performed on the above equation.

1) We first obtain a weak form of the wave equation by integrating over the entire physical domain $D$ and at the same time multiplying by some basis $\varphi_{i}$.

2) Integration by parts and implementation of the stress-free boundary condition is performed.

3) We approximate our unknown displacement field $u(x, t)$ by a sum over space-dependent basis functions $\varphi_i$ weighted by time-dependent coefficients $u_i(t)$.

\begin{equation}
u(x,t) \ \approx \ \overline{u}(x,t) \ = \ \sum_{i=1}^{n} u_i(t) \ \varphi_i(x)
\end{equation}

4) Utilize the same basis functions used to expand $u(x, t)$ as test functions in the weak form, this is the Galerkin principle.

5) We can turn the continuous weak form into a system of linear equations by considering the approximated displacement field.

\begin{equation}
\mathbf{M}^T\partial_t^2 \mathbf{u} + \mathbf{K}^T\mathbf{u} = \mathbf{f}
\end{equation}

6) For the second time-derivative, we use a standard finite-difference approximation. Finally, we arrive at the explicit time extrapolation scheme.

\begin{equation}
\mathbf{u}(t + dt) = dt^2 (\mathbf{M}^T)^{-1}[\mathbf{f} - \mathbf{K}^T\mathbf{u}] + 2\mathbf{u} - \mathbf{u}(t-dt).
\end{equation}

where $\mathbf{M}$ is known as the mass matrix, and $\mathbf{K}$ the stiffness matrix.

7) As interpolating functions, we choose interpolants such that $\varphi_{i}(x_{i}) = 1$ and zero elsewhere. Then, we transform the space coordinate into a local system. According to $\xi = x − x_{i}$ and $h_{i} = x_{i+1} − x_{i}$, we have:

\begin{equation}
 \varphi_{i}(\xi) =
  \begin{cases}
    \frac{\xi}{h_{i-1}} + 1  & \quad \text{if} \quad -h_{i-1} \le \xi \le 0\\
    1 + \frac{\xi}{h_{i}}    & \quad \text{if} \quad 0 \le \xi \le h_{i}\\
    0  & \quad elsewhere\\
  \end{cases}
\end{equation}

with the corresponding derivatives

\begin{equation}
 \partial_{\xi}\varphi_{i}(\xi) =
  \begin{cases}
    \frac{1}{h_{i-1}}  & \quad \text{if} \quad -h_{i-1} \le \xi \le 0\\
   -\frac{1}{h_{i}}    & \quad \text{if} \quad 0 \le \xi \le h_{i}\\
    0  & \quad elsewhere\\
  \end{cases}
\end{equation}

The figure on the left-hand side illustrates the shape of $\varphi_{i}(\xi)$ and $\partial_{\xi}\varphi_{i}(\xi)$ with varying $h$.

Code implementation starts with the initialization of a particular setup of our problem. Then, we define the source that introduces perturbations following by initialization of the mass and stiffness matrices. Finally, time extrapolation is done.

```python
# Import all necessary libraries, this is a configuration step for the exercise.
# Please run it before the simulation code!
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation

# Show the plots in the Notebook
plt.switch_backend("nbagg")
```

### 1. Initialization of setup

```python
# Initialization of setup
# ---------------------------------------------------------------
# Basic parameters
nt    = 2000    # Number of time steps
vs    = 3000    # Wave velocity [m/s]
ro0   = 2500    # Density [kg/m^3]
nx    = 1000    # Number of grid points
isx   = 500     # Source location [m]
xmax  = 10000.  # Maximum length
eps   = 0.5     # Stability limit
iplot = 20      # Snapshot frequency

dx = xmax/(nx-1)           # calculate space increment
x  = np.arange(0, nx)*dx   # initialize space coordinates
x  = np.transpose(x)

h = np.diff(x)  # Element sizes [m]

# parameters
ro = x*0 + ro0
mu = x*0 + ro*vs**2

# time step from stabiity criterion
dt = 0.5*eps*dx/np.max(np.sqrt(mu/ro))
# initialize time axis
t   = np.arange(1, nt+1)*dt

# ---------------------------------------------------------------
# Initialize fields
# ---------------------------------------------------------------
u    = np.zeros(nx)
uold = np.zeros(nx)
unew = np.zeros(nx)

p    = np.zeros(nx)
pold = np.zeros(nx)
pnew = np.zeros(nx)
```

### 2. Source time function

In 1D the propagating signal is an integral of the source time function. As we look for a Gaussian waveform, we initialize the source time function $f(t)$ using the first derivative of a Gaussian function.

\begin{equation}
f(t) = -\dfrac{2}{\sigma^2}(t - t_0)e^{-\dfrac{(t - t_0)^2}{\sigma^2}}
\end{equation}

#### Exercise 1
Initialize a source time function called 'src'.  Use $\sigma = 20 dt$ as Gaussian width, and time shift $t_0 = 3\sigma$. Then, visualize the source in a given plot.

```python
#################################################################
# INITIALIZE THE SOURCE TIME FUCTION HERE!
#################################################################

# Source vector
f = np.zeros(nx); f[isx:isx+1] = f[isx:isx+1] + 1.

#################################################################
# PLOT THE SOURCE TIME FUNCTION HERE!
#################################################################
```

### 3. The Mass Matrix
Having implemented the desired source, now we initialize the mass and stiffness matrices. In general, the mass matrix is given

\begin{equation}
M_{ij} = \int_{D} \rho \varphi_i \varphi_j dx = \int_{D_{\xi}} \rho \varphi_i \varphi_j d\xi
\end{equation}

next, the defined basis are introduced and some algebraic treatment is done to arrive at the explicit form of the mass matrix

#### Exercise 2
Implement the mass matrix

\begin{equation}
M_{ij} = \frac{\rho h}{6}
 \begin{pmatrix}
   \ddots  &    &    &    & 0\\
   1 & 4 &  1 &    &  \\
     & 1 &  4 &  1 &  \\
     &   &  1 &  4 & 1\\
   0 &   &    &    &  \ddots
 \end{pmatrix}
\end{equation}

Compute the inverse mass matrix and display your result to visually inspect how it looks like

```python
#################################################################
# IMPLEMENT THE MASS MATRIX HERE!
#################################################################

#################################################################
# COMPUTE THE INVERSE MASS MATRIX HERE!
#################################################################

#################################################################
# DISPLAY THE INVERSE MASS MATRIX HERE!
#################################################################

```

### 4. The Stiffness matrix
On the other hand, the general form of the stiffness matrix is

\begin{equation}
K_{ij} = \int_{D} \mu \partial_x\varphi_i \partial_x\varphi_j dx = \int_{D_{\xi}} \mu \partial_\xi\varphi_i \partial_\xi\varphi_j d\xi
\end{equation}

at this point, the defined basis are introduced. Again, with the help of some algebraic treatment, we arrive at the explicit form of the stiffness matrix

#### Exercise 3
Implement the stiffness matrix

\begin{equation}
K_{ij} = \frac{\mu}{h}
 \begin{pmatrix}
   \ddots  &    &    &    & 0\\
  -1 & 2 & -1 &    &  \\
     &-1 &  2 & -1 &  \\
     &   & -1 &  2 & -1\\
   0 &   &    &    &  \ddots
 \end{pmatrix}
\end{equation}

Display the stiffness matrix to visually inspect how it looks like

```python
#################################################################
# IMPLEMENT THE STIFFNESS MATRIX HERE!
#################################################################

#################################################################
# DISPLAY THE STIFFNESS MATRIX HERE!
#################################################################

```

### 5. Finite differences matrices
We implement a finite difference scheme in order to compare with the finite elements solution.

#### Exercise 4
Implement the finite differences matrices $M$ and $D$. Where $M$ is a diagonal mass matrix containing the inverse densities, and differentiation matrix

\begin{equation}
D_{ij} = \frac{\mu}{dt^2}
 \begin{pmatrix}
   -2 & 1  &    &    &  \\
    1 & -2 & 1  &    &  \\
      &  & \ddots  &  & \\
      &   & 1 &  -2 & 1\\
      &   &    &  1 & -2
 \end{pmatrix}
\end{equation}

Display both matrices to visually inspect how they look like

```python
#################################################################
# INITIALIZE FINITE DIFFERENCES HERE!
#################################################################

#################################################################
# DISPLAY THE DIFFERENCES MATRICES HERE!
#################################################################

```

### 6. Finite element solution

Finally we implement the finite element solution using the computed mass $M$ and stiffness $K$ matrices together with a finite differences extrapolation scheme

\begin{equation}
\mathbf{u}(t + dt) = dt^2 (\mathbf{M}^T)^{-1}[\mathbf{f} - \mathbf{K}^T\mathbf{u}] + 2\mathbf{u} - \mathbf{u}(t-dt).
\end{equation}

```python
# Initialize animated plot
# ---------------------------------------------------------------
plt.figure(figsize=(12,4))

line1 = plt.plot(x, u, 'k', lw=1.5, label='FEM')
line2 = plt.plot(x, p, 'r', lw=1.5, label='FDM')
plt.title('Finite elements 1D Animation', fontsize=16)
plt.ylabel('Amplitude', fontsize=12)
plt.xlabel('x (m)', fontsize=12)

plt.ion()   # set interective mode
plt.show()

# ---------------------------------------------------------------
# Time extrapolation
# ---------------------------------------------------------------
for it in range(nt):
    # --------------------------------------
    # Finite Element Method
    unew = (dt**2) * Minv @ (f*src[it] - K @ u) + 2*u - uold
    uold, u = u, unew

    # --------------------------------------
    # Finite Difference Method
    pnew = (dt**2) * Mf @ (D @ p + f/dx*src[it]) + 2*p - pold
    pold, p = p, pnew

    # --------------------------------------
    # Animation plot. Display both solutions
    if not it % iplot:
        for l in line1:
            l.remove()
            del l
        for l in line2:
            l.remove()
            del l
        line1 = plt.plot(x, u, 'k', lw=1.5, label='FEM')
        line2 = plt.plot(x, p, 'r', lw=1.5, label='FDM')
        plt.legend()
        plt.gcf().canvas.draw()
```

```python

```


```python
%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import pathlib
import os
import pwd
import figformat
fig_width,fig_height,params=figformat.figure_format(fig_width=3.4,fig_height=3.4)
mpl.rcParams.update(params)
#mpl.rcParams.keys()
#help(figformat)
def get_username():
    return pwd.getpwuid(os.getuid())[0]
user = get_username()

run_dir  = pathlib.Path(rf"/Users/{user}/GitHub/LEC/examples/") #path to TABLES

print(f"run_dir: {run_dir}")
```

    run_dir: /Users/StevE/GitHub/LEC/examples

### The function $q(\chi_e)$ for $\chi_e\ll 1$
\begin{equation}
    q(\chi_e\ll 1)\approx 1-\frac{55}{16}\sqrt{3}\chi + 48\chi^2 \nonumber
\end{equation}

```python
def func(x):
    f=1-(55/16)*np.sqrt(3)*x + 48*x**2
    return f
x=np.arange(0.001,10,0.001)
```

### The function $q(\chi_e)$ for $\chi_e\gg 1$
\begin{equation}
    q(\chi_e\gg 1)\approx\frac{48}{243}\Gamma(\frac{2}{3})\chi^{-4/3}
    \left[ 1 -\frac{81}{16\Gamma(2/3)}(3\chi)^{-2/3} \right] \nonumber
\end{equation}

```python
def func2(y):
    f2=(48/243)*1.354*2.08*y**(-4/3)*(1-(81/16)*0.73855*(3*y)**(-2/3))
    return f2
y=np.arange(0.1,100,0.001)
```

### Extract $\chi_e$, $P_\mathrm{Q}$, $P_\mathrm{C}$, and $q(\chi_e)$

```python
Xi,Prad,Pc,g = np.loadtxt('P_rad.dat',unpack=True,usecols=[0,1,2,3],dtype=np.float)
```

```python
fig, ax = plt.subplots()
ax.plot(x,func(x), "b", ls='--')
ax.plot(y,func2(y),"g", ls='-.')

ax.plot(Xi, g, color='black')
ax.set_xlim(0.001,100)
ax.set_ylim(0.001,3)
ax.set_yscale('log')
ax.set_xscale('log')
ax.set_xlabel("$\chi_e$")
ax.set_ylabel("$q(\chi_e)$")
ax.minorticks_on()

ax2 = ax.twinx()
ax2.plot(Xi,Prad,'-y',color='red',label='$P_Q$')
ax2.plot(Xi,Pc,'--',color='red',label='$P_Q$')
ax2.set_ylim(1,1e7)
ax2.set_yscale('log')
ax2.set_ylabel("$P_\mathrm{rad} (W)$",color='red')
ax2.minorticks_on()
ax2.tick_params(axis='y',labelcolor='red')
ax2.tick_params(which='major',color='red')
ax2.tick_params(which='minor',color='red')
ax2.spines['right'].set_color('red')

eq1 = r"\begin{eqnarray*}" + \
      r"g(\chi_e\ll 1)&\approx&1-\frac{55}{16}\sqrt{3}\chi \\&+& 48\chi^2" + \
      r"\end{eqnarray*}"
#ax.text(0.35, 0.4, eq1, {'color': 'b', 'fontsize': 15}, va="top", ha="right")
eq2 = r"\begin{eqnarray*}" + \
      r"&&g(\chi_e\gg 1)\approx\frac{48}{243}\Gamma(\frac{2}{3})\chi^{-4/3} \\" + \
      r"&& \times \{ 1 -\frac{81}{16\Gamma(2/3)}(3\chi)^{-2/3} \}" + \
      r"\end{eqnarray*}"
#ax.text(1, 0.01, eq2, {'color': 'g', 'fontsize': 15}, va="top", ha="right")
fig = plt.gcf()
fig.set_size_inches(fig_width, fig_width/1.618)
fig.tight_layout()
plt.show()
```

    <IPython.core.display.Javascript object>

```python
fig.savefig(rf"/Users/StevE/GitHub/LEC/docs/source/figures/qchi.png",format='png',dpi=600,transparent=True, bbox_inches='tight')
```

```python

```


# RL Exercise 1 - Markov Decision Processes

**GOAL:** The goal of the exercise is to introduce the Markov Decision Process abstraction and to show how to use Markov Decision Processes in Python.

**The key abstraction in reinforcement learning is the Markov decision process (MDP).** An MDP models sequential interactions with an external environment. It consists of the following:
- a **state space**
- a set of **actions**
- a **transition function** which describes the probability of being in a state $s'$ at time $t+1$ given that the MDP was in state $s$ at time $t$ and action $a$ was taken
- a **reward function**, which determines the reward received at time $t$
- a **discount factor** $\gamma$

More details are available [here](https://en.wikipedia.org/wiki/Markov_decision_process).

**NOTE:** Reinforcement learning algorithms are often applied to problems that don't strictly fit into the MDP framework. In particular, situations in which the state of the environment is not fully observed lead to violations of the MDP assumption. Nevertheless, RL algorithms can be applied anyway.

## Policies

A **policy** is a function that takes in a **state** and returns an **action**. A policy may be stochastic (i.e., it may sample from a probability distribution) or it can be deterministic.

The **goal of reinforcement learning** is to learn a **policy** for maximizing the cumulative reward in an MDP. That is, we wish to find a policy $\pi$ which solves the following optimization problem

\begin{equation}
\arg\max_{\pi} \sum_{t=1}^T \gamma^t R_t(\pi),
\end{equation}

where $T$ is the number of steps taken in the MDP (this is a random variable and may depend on $\pi$) and $R_t$ is the reward received at time $t$ (also a random variable which depends on $\pi$).

A number of algorithms are available for solving reinforcement learning problems. Several of the most widely known are [value iteration](https://en.wikipedia.org/wiki/Markov_decision_process#Value_iteration), [policy iteration](https://en.wikipedia.org/wiki/Markov_decision_process#Policy_iteration), and [Q learning](https://en.wikipedia.org/wiki/Q-learning).

## RL in Python

The `gym` Python module provides MDP interfaces to a variety of simulators. For example, the CartPole environment interfaces with a simple simulator which simulates the physics of balancing a pole on a cart. The CartPole problem is described at https://gym.openai.com/envs/CartPole-v0. This example fits into the MDP framework as follows.
- The **state** consists of the position and velocity of the cart as well as the angle and angular velocity of the pole that is balancing on the cart.
- The **actions** are to decrease or increase the cart's velocity by one unit.
- The **transition function** is deterministic and is determined by simulating physical laws.
- The **reward function** is a constant 1 as long as the pole is upright, and 0 once the pole has fallen over. Therefore, maximizing the reward means balancing the pole for as long as possible.
- The **discount factor** in this case can be taken to be 1.

More information about the `gym` Python module is available at https://gym.openai.com/.

```python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import gym
import numpy as np
```

The code below illustrates how to create and manipulate MDPs in Python. An MDP can be created by calling `gym.make`. Gym environments are identified by names like `CartPole-v0`. A **catalog of built-in environments** can be found at https://gym.openai.com/envs.

```python
env = gym.make('CartPole-v0')
```

    [2017-09-18 22:56:07,434] Making new env: CartPole-v0

Reset the state of the MDP by calling `env.reset()`. This call returns the initial state of the MDP.

```python
state = env.reset()
```

The `env.step` method takes an action (in the case of the CartPole environment, the appropriate actions are 0 or 1, for moving left or right). It returns a tuple of four things:
1. the new state of the environment
2. a reward
3. a boolean indicating whether the simulation has finished
4. a dictionary of miscellaneous extra information

```python
# Simulate taking an action in the environment. Appropriate actions for
# the CartPole environment are 0 and 1 (for moving left and right).
action = 0
state, reward, done, info = env.step(action)
```

A **rollout** is a simulation of a policy in an environment. It alternates between choosing actions based (using some policy) and taking those actions in the environment.

The code below performs a rollout in a given environment. It takes **random actions** until the simulation has finished and returns the cumulative reward.

```python
def random_rollout(env):
    state = env.reset()

    done = False
    cumulative_reward = 0

    # Keep looping as long as the simulation has not finished.
    while not done:
        # Choose a random action (either 0 or 1).
        action = np.random.choice([0, 1])

        # Take the action in the environment.
        state, reward, done, _ = env.step(action)

        # Update the cumulative reward.
        cumulative_reward += reward

    # Return the cumulative reward.
    return cumulative_reward

reward = random_rollout(env)
print(reward)
reward = random_rollout(env)
print(reward)
```

    33.0
    11.0

**EXERCISE:** Copy the `random_rollout` function from above and modify it to implement the `rollout_policy` function below, which should take an environment *and* a policy. The *policy* is a function that takes in a *state* and returns an *action*. The main difference is that instead of choosing a **random action**, the action should be chosen **with the policy** (as a function of the state).

```python
def rollout_policy(env, policy):
    # EXERCISE: Fill out this function by copying the 'random_rollout' function
    # and then modifying it to choose the action using the policy.
    state = env.reset()

    done = False
    cumulative_reward = 0

    # Keep looping as long as the simulation has not finished.
    while not done:
        # Choose a random action (either 0 or 1).
        action = policy(state)

        # Take the action in the environment.
        state, reward, done, _ = env.step(action)

        # Update the cumulative reward.
        cumulative_reward += reward

    # Return the cumulative reward.
    return cumulative_reward

def sample_policy(state):
    return 0 if state[0] < 0 else 1

reward = rollout_policy(env, sample_policy)
print(reward)
reward = rollout_policy(env, sample_policy)
print(reward)
```

    9.0
    10.0

```python

```

```python

```

```python

```


# 4. 2-D Anisotropic Diffusion

In this tutorial, we consider the following anisotropic diffusion problem

$$
	a(v, u) = \int_{\Omega} \kappa_{\parallel} (\textbf{b} \cdot \nabla v) (\textbf{b} \cdot \nabla u) + \kappa_{I} \nabla v \cdot \nabla u ~ d\Omega, \quad \forall u,v \in \mathcal{V}_h
$$

where $\mathbf{b} = \frac{\textbf{B}}{\|\mathbf{B}\|}$ while $\textbf{B}$ denotes the magnetic field, $\Omega$ is our 2D computational domain and $\mathcal{V}_h \subset H^1(\Omega)$.
We are interested in highly anisotropic configurations with $\displaystyle \frac{\kappa_{\parallel}}{\kappa_{I}} \simeq 10^6 \gg 1$.

```python
from numpy import linspace, zeros, meshgrid, pi

import matplotlib.pyplot as plt
%matplotlib inline

from IPython.display import Math

from sympy.core.containers import Tuple
from sympy import symbols
from sympy import Symbol
from sympy import Lambda
from sympy import IndexedBase
from sympy import Function

from gelato.glt import glt_symbol
from gelato.glt import glt_lambdify
from gelato.glt import glt_approximate_eigenvalues
from gelato.calculus   import (Dot, Cross, Grad, Curl, Rot, Div)
from gelato.calculus   import (dx, dy)
from gelato.calculus   import Constant
from gelato.fem.assembly import assemble_matrix
from gelato.fem.utils    import compile_kernel
from gelato.fem.utils    import compile_symbol

from spl.fem.splines import SplineSpace
from spl.fem.tensor  import TensorFemSpace
```

```python
x,y = symbols('x y')

u = Symbol('u')
v = Symbol('v')

k_par = Constant('k_par')
k_perp = Constant('k_perp')

b1 = Constant('b1')
b2 = Constant('b2')
b = Tuple(b1, b2)

a = Lambda((x,y,v,u), k_perp * Dot(Grad(v), Grad(u)) + k_par * Dot(b, Grad(v)) * Dot(b, Grad(u)))

```

```python
# ...  create a finite element space
p1  = 3 ; p2  = 3
ne1 = 32 ; ne2 = 32

grid_1 = linspace(0., 1., ne1+1)
grid_2 = linspace(0., 1., ne2+1)

V1 = SplineSpace(p1, grid=grid_1)
V2 = SplineSpace(p2, grid=grid_2)

V = TensorFemSpace(V1, V2)
# ...
```

```python
from IPython.display import Math
from gelato.printing.latex import glt_latex
from sympy import latex

symbol = glt_symbol(a, space=V, evaluate=False)

Math(latex(symbol))
#Math(glt_latex(symbol))
```

$$b_{1}^{2} k_{par} m_{2} s_{1} - 2 b_{1} b_{2} k_{par} a_{1} a_{2} + b_{2}^{2} k_{par} m_{1} s_{2} + k_{perp} m_{1} s_{2} + k_{perp} m_{2} s_{1}$$

```python
d_constants = {'b2': 1., 'k_perp': 1.}
d_types = {'b1': 'double', 'k_par': 'double'}
```

```python
kernel_a = compile_kernel('kernel_a', a, V,
                          d_constants=d_constants,
                          d_args=d_types,
                          backend='fortran')
```

```python
d_args = {'b1': 1.165939762441386085001227002067,
          'k_par': 1.e5}
```

```python
M = assemble_matrix(V, kernel_a, args=d_args).tocsr()
```

```python
# import eigenvalue solver
from scipy.linalg import eig

M = M.todense()
w, v = eig(M)
wr = w.real
wr.sort()
```

```python
symbol_a = compile_symbol('symbol_a', a, V,
                          d_constants=d_constants,
                          d_args=d_types,
                          backend='fortran')
```

```python
# ... example of symbol evaluation
[n1,n2] = [W.nbasis for W in V.spaces]

t1 = linspace(-pi,pi, n1)
t2 = linspace(-pi,pi, n2)
x1 = linspace(0.,1., n1)
x2 = linspace(0.,1., n2)
e = zeros((n1, n2), order='F')
symbol_a(x1,x2,t1,t2, e, d_args['k_par'], d_args['b1'])
t = e.ravel()
t.sort()
# ...
```

```python
plt.plot(t, "+b", label="glt symbol")
plt.plot(wr, "xr", label="eigenvalues")
plt.legend(loc=2);
```

```python
# ... compute the symbol f_a / f_c
nx = 100 ; ny = 100
t1 = linspace(-pi,pi, nx)
t2 = linspace(-pi,pi, ny)

x = linspace(0.,1.,nx)
y = linspace(0.,1.,ny)

f = zeros((nx, ny), order='F')
symbol_a(x,y,t1,t2, f, d_args['k_par'], d_args['b1'])
# ...

# ... plot
T1,T2 = meshgrid(t1,t2)
plt.contourf(T1, T2, f)
plt.axis("equal")
plt.colorbar();
# ...
```

Let's now create the symbol for linear B-Splines

```python
# ...  create a finite element space
W1 = SplineSpace(degree=1, grid=grid_1)
W2 = SplineSpace(degree=1, grid=grid_2)

W = TensorFemSpace(W1, W2)
# ...

symbol_p1 = compile_symbol('symbol_p1', a, W,
                           d_constants=d_constants,
                           d_args=d_types,
                           backend='fortran')
```

```python
# ... compute the symbol f_a / f_c
nx = 100 ; ny = 100
t1 = linspace(-pi,pi, nx)
t2 = linspace(-pi,pi, ny)

x = linspace(0.,1.,nx)
y = linspace(0.,1.,ny)

f_a = zeros((nx, ny), order='F')
symbol_a(x,y,t1,t2, f_a, d_args['k_par'], d_args['b1'])

f_p1 = zeros((nx, ny), order='F')
symbol_p1(x,y,t1,t2, f_p1, d_args['k_par'], d_args['b1'])

f = f_a/f_p1
# ...

# ... plot
T1,T2 = meshgrid(t1,t2)
plt.contourf(T1, T2, f)
plt.axis("equal")
plt.colorbar();
# ...
```

```python
# asymptotic expansion
t1 = Symbol('t1')
t2 = Symbol('t2')
symbol_p   = glt_symbol(a, space=V, evaluate=True)
symbol_lin = glt_symbol(a, space=W, evaluate=True)
```

```python
expr = symbol_p.expr/symbol_lin.expr
#Math(latex(expr))
```

```python
e = expr.subs({Symbol('t2'): - b1/b2*t1})
```

```python
e = e.subs({b1: 1.165939762441386085001227002067, b2: 1., k_perp: 1.})

es = []
ks = [1.e1, 1.e2, 1.e3, 1.e4, 1.e5, 1.e6, 1.e7]
for k in ks:
    enew = e.subs({k_par: k})
    es.append(enew)
```

```python
from sympy import lambdify
fs = []
for e in es:
    f = lambdify((t1),e,modules=['numpy'])
    fs.append(f)
```

```python
from numpy import log
t = linspace(-pi,pi,400)
for k,f in zip(ks,fs):
    plt.plot(t, f(t), label='$k_{parallel} = 1.e^{' + '{}'.format(int(log(k)/log(10))) + '}$')
plt.legend(loc=2);
```

```python
# zoom around 0
from numpy import log
t = linspace(-pi/16,pi/16,400)
for k,f in zip(ks,fs):
    plt.plot(t, f(t), label='$k_{parallel} = 1.e^{' + '{}'.format(int(log(k)/log(10))) + '}$')
plt.legend(loc=2);
```

```python

```

## Variable coefficient case

```python
# ... define the weak formulation
x,y = symbols('x y')

u = Symbol('u')
v = Symbol('v')

b0 = Function('b0')
b1 = Function('b1')

k_par = Constant('k_par')
k_perp = Constant('k_perp')

a = Lambda((x,y,v,u),
            k_par * (b0(x,y)*dx(v) + b1(x,y)*dy(v))
                  * (b0(x,y)*dx(u) + b1(x,y)*dy(u))
          + k_perp * Dot(Grad(v), Grad(u)))
# ...

# ... user defined function
def b0(x,y):
    from numpy import sqrt

    r = sqrt(x**2+y**2)
    b = -y/r

    return b

def b1(x,y):
    from numpy import sqrt

    r = sqrt(x**2+y**2)
    b = x/r

    return b
# ...

# ... create an interactive pyccel context
from pyccel.epyccel import ContextPyccel

context = ContextPyccel(name='context_2')
context.insert_function(b0, ['double', 'double'],
                        kind='function', results=['double'])
context.insert_function(b1, ['double', 'double'],
                        kind='function', results=['double'])

context.compile()
# ...

# ...  create a finite element space
p1  = 3 ; p2  = 3
ne1 = 32 ; ne2 = 32

grid_1 = linspace(0., 1., ne1+1)
grid_2 = linspace(0., 1., ne2+1)

V1 = SplineSpace(p1, grid=grid_1)
V2 = SplineSpace(p2, grid=grid_2)

V = TensorFemSpace(V1, V2)
# ...

# ...
d_constants = {'k_perp': 1.}
d_types = {'k_par': 'double'}

kernel_f90 = compile_kernel('kernel_2', a, V,
                            context=context,
                            d_constants=d_constants,
                            d_args=d_types,
                            backend='fortran')
# ...
```

    pyccel: [semantic]
     |warning| Duplicated import  (sqrt)

    pyccel: [codegen]
     |warning| Duplicated import  (sqrt)

    pyccel: [semantic]
     |warning| Duplicated import  (sqrt)

    pyccel: [semantic]
     |warning| Duplicated import  (sqrt)

    pyccel: [codegen]
     |warning| Duplicated import  (sqrt)

```python
d_args = {'k_par': 1.e5}
M = assemble_matrix(V, kernel_f90, args=d_args).tocsr()
```

```python
# import eigenvalue solver
from scipy.linalg import eig

M = M.todense()
w, v = eig(M)
wr = w.real
wr.sort()
```

```python
symbol_a = compile_symbol('symbol_2', a, V,
                          context=context,
                          d_constants=d_constants,
                          d_args=d_types,
                          backend='fortran')
```

    pyccel: [semantic]
     |warning| Duplicated import  (sqrt)

    pyccel: [semantic]
     |warning| Duplicated import  (sqrt)

    pyccel: [codegen]
     |warning| Duplicated import  (sqrt)

```python
# ... example of symbol evaluation
[n1,n2] = [W.nbasis for W in V.spaces]

t1 = linspace(-pi,pi, n1)
t2 = linspace(-pi,pi, n2)
x1 = linspace(0.,1., n1)
x2 = linspace(0.,1., n2)
e = zeros((n1, n2), order='F')
symbol_a(x1,x2,t1,t2, e, d_args['k_par'])
t = e.ravel()
t.sort()
# ...
```

```python
plt.plot(t, "+b", label="glt symbol")
plt.plot(wr, "xr", label="eigenvalues")
plt.legend(loc=2);
```

```python
# ... compute the symbol f_a / f_c
nx = 100 ; ny = 100
t1 = linspace(-pi,pi, nx)
t2 = linspace(-pi,pi, ny)

x = linspace(0.,1.,nx)
y = linspace(0.,1.,ny)

f = zeros((nx, ny), order='F')
symbol_a(x,y,t1,t2, f, d_args['k_par'])
# ...

# ... plot
T1,T2 = meshgrid(t1,t2)
plt.contourf(T1, T2, f)
plt.axis("equal")
plt.colorbar();
# ...
```

```python
# ...  create a finite element space
W1 = SplineSpace(degree=1, grid=grid_1)
W2 = SplineSpace(degree=1, grid=grid_2)

W = TensorFemSpace(W1, W2)
# ...

symbol_p1 = compile_symbol('symbol_p1_2', a, W,
                           context=context,
                           d_constants=d_constants,
                           d_args=d_types,
                           backend='fortran')
```

    pyccel: [semantic]
     |warning| Duplicated import  (sqrt)

    pyccel: [semantic]
     |warning| Duplicated import  (sqrt)

    pyccel: [codegen]
     |warning| Duplicated import  (sqrt)

```python
# ... compute the symbol f_a / f_c
nx = 200 ; ny = 200
t1 = linspace(-pi,pi, nx)
t2 = linspace(-pi,pi, ny)

x = linspace(0.,1.,nx)
y = linspace(0.,1.,ny)

f_a = zeros((nx, ny), order='F')
symbol_a(x,y,t1,t2, f_a, d_args['k_par'])

f_p1 = zeros((nx, ny), order='F')
symbol_p1(x,y,t1,t2, f_p1, d_args['k_par'])

f = f_a/f_p1
# ...

# ... plot
T1,T2 = meshgrid(t1,t2)
plt.contourf(T1, T2, f)
plt.axis("equal")
plt.colorbar();
# ...
```

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:600px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    h2 {
        font-family: 'Fenix', serif;
    }
    h3{
		font-family: 'Fenix', serif;
        margin-top:12px;
        margin-bottom: 3px;
       }
	h4{
		font-family: 'Fenix', serif;
       }
    h5 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    div.text_cell_render{
        font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 135%;
        font-size: 120%;
        width:600px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/
    .text_cell_render h1 {
        font-weight: 200;
        font-size: 50pt;
		line-height: 100%;
        color:#054BCD;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #054BCD;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

```python

```


###  \begin{equation} \texttt{Tiefe neuronale Netzwerke} \\ \Tiny \texttt{von Alexander Mroß} \end{equation}

#### Pakete

```julia
using LinearAlgebra, PyPlot
```

#### Cost-Funktion

```julia
function Cost(X,W2,W3,W4,b2,b3,b4,y)
    costvec = zeros(10,1);
    for i in collect(1:10)
        a2 = sigma(X[:,i],W2,b2);
        a3 = sigma(a2,W3,b3);
        a4 = sigma(a3,W4,b4);
        costvec[i] = norm(y[:,i] - a4,2);
    end
    costval = 0.5*norm(costvec,2)^2;
    return costval
end;
```

#### Sigmoid- bzw. S-Funktion (geglättete Treppenfunktion)

```julia
function sigma(x,W,b)
# sigma(z) = 1/(1+exp(-z))
y = 1 ./(1 .+exp.(-(W*x+b)));
return y
end;
```

#### Gesamt-Funktion $F$ zum Testen

```julia
function F(x,W2,b2,W3,b3,W4,b4)
    z=sigma(sigma(sigma(x,W2,b2),W3,b3),W4,b4)
    return z
end;
```

#### Plot-Funktion

```julia
function plots(X,N,W2,b2,W3,b3,W4,b4,lineplot,linspace,counter)

    figure()
    plot([0 1],[0 1])
    for i in 1:(N+1)
        for j in 1:(N+1)
            v = [(j-1)/N;(i-1)/N];
            z = F(v,W2,b2,W3,b3,W4,b4);
            # Zeichnen der Linie
            if (z[2]>z[1]) && (v[1]<lineplot[i])
                lineplot[i] = v[1];
            end
            # Ausfüllen des grauen Bereichs
            if z[1]>z[2]
                plot(v[1],v[2],"o",color="grey")
            end
        end
    end

    # Approximation durch Parabel mittels Least-squares
    #A = [ones(N+1,1) linspace linspace.^2]
    #koeff = (transpose(A)*A)\transpose(A)*lineplot
    #fx = koeff[1]*ones(N+1)+koeff[2]*linspace+koeff[3]*linspace.^2;

    # Plots
    plot(X[1,1:5] ,X[2,1:5], "o",color="red", label="Erfolg")
    plot(X[1,6:10],X[2,6:10],"x",color="blue",label="Niete")
    plot(0,1,color="grey",label="Trennlinie") # Trennlinie grau für Legende
    #plot(fx,linspace,color="purple",label="Trennlinie") # Trennlinie Approx lila
    #plot(lineplot,0:1/N:1,color="lightgreen",label="Trennlinie") # Trennlinie stückweise hellgrün
    xlabel("Anzahl Durchläufe = "*string(counter))
    legend(bbox_to_anchor=[0.75,0.985],loc=2,borderaxespad=0) # mit Trennlinie
    #savefig("Iter-N="*string(N)*"-C="*string(counter)*".png")

end; # der Plot Funktion
```

#### "Das neuronale Netzwerk"

```julia
function DNN()

    # (x1,x2)-Paare auf "Karte" [0,1]^2
    X = [0.1 0.3 0.1 0.6 0.4 0.6 0.5 0.9 0.4 0.7;
         0.1 0.4 0.5 0.9 0.2 0.3 0.6 0.2 0.4 0.6];

    # y = Ziel des Trainingssets / F(x) nah an [1,0] falls in Kat. A, F(x) nah an [0,1] falls in Kat. B
    y = [1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0;
         0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0];

    # Initialisieren der Gewichte und Schwellenwerte
    W2 = 0.5*randn(2,2); W3 = 0.5*randn(3,2); W4 = 0.5*randn(2,3);
    b2 = 0.5*randn(2,1); b3 = 0.5*randn(3,1); b4 = 0.5*randn(2,1);

    # Vorwärts- und Rückwärtspropagation + Allokation weiterer Variablen
    eta = 0.05; # Lernrate
    Niter = 5*10^5; # Anzahl Iterationen
    savecost = zeros(Niter); # enthält alle Werte der Cost-Funktion
    newcost = 0.0;
    k = 0; #für zufä.Trpkt.

    # Für die Plots
    N = 40; # Anzahl Unterteilungen zum Messen der Linie
    lineplot = ones(N+1);# Vektor zum Plotten der Linie
    linspace = collect(0:1/N:1);

    for counter = 1:Niter

        # zufälliger Trainingspunkt von 1 bis 10 (stoch. Grad.)
        k = Int64(ceil(10*rand()));
        # Variante analog Permutation (stabiler?!)
        # k = (counter%10)+1;
        # Vorwärts // X[:,k] = a_1, entspricht Trainingspunkt x_k (siehe auch unten bei Grad.vf.)
        a2 = sigma(X[:,k],W2,b2);
        a3 = sigma(a2,W3,b3);
        a4 = sigma(a3,W4,b4);
        # Rückwärts (backpropagation)
        delta4 = a4.*(1 .-a4).*(a4-y[:,k]);
        delta3 = a3.*(1 .-a3).*(W4'*delta4);
        delta2 = a2.*(1 .-a2).*(W3'*delta3);
        # Gradientenverfahren
        W2 = W2 - eta*delta2*X[:,k]';
        W3 = W3 - eta*delta3*a2';
        W4 = W4 - eta*delta4*a3';
        b2 = b2 - eta*delta2;
        b3 = b3 - eta*delta3;
        b4 = b4 - eta*delta4;
        # savecost befüllen mit Cost-Werten
        newcost = Cost(X,W2,W3,W4,b2,b3,b4,y);
        savecost[counter] = newcost;

        # Plots
        if counter in [45000 70000 90000 Niter]
            plots(X,N,W2,b2,W3,b3,W4,b4,lineplot,linspace,counter)
            lineplot = ones(N+1)
        end # der Plots

    end # von for-Schleife Niter

    figure()
    semilogy(1:10^4:Niter,savecost[1:10^4:Niter])
    ylim(1e-4, 1e1)
    #xlabel(L"\mathrm{Anzahl\ Iterationen}\ \ [\times 10^4]")
    xlabel("Anzahl Iterationen")
    ylabel("Cost-Funktion")
    #savefig("Iter-N="*string(N)*"-Cost.png")
    #savefig("Iter-N="*string(N)*"-Cost.pdf")

end; # von function DNN
```

```julia
@time DNN()
```

#### Add-Ons

```julia
# Code zum Erzeugen des Standardbilds mit grüner Beispiellösung
#=
x1 = [0.1,0.3,0.1,0.6,0.4,0.6,0.5,0.9,0.4,0.7];
x2 = [0.1,0.4,0.5,0.9,0.2,0.3,0.6,0.2,0.4,0.6];
a=collect(0:0.01:1);
a1=2;
a2=0.43;
a3=0.35;
b=a1*(a.-a2).^2 .+a3;
figure()
plot(x1[1:5],x2[1:5],"o",color="red",label="Erfolg")
plot(x1[6:10],x2[6:10],"x",color="blue",label="Niete")
plot([0;1],[0;1],"x",color="white")
plot(b,a,color="lightgreen")
legend(bbox_to_anchor=[0.81,0.985],loc=2,borderaxespad=0)
savefig("Iter-Pic0Linie.pdf")
=#
```

```julia
# Zeittest
@time begin
# Test
end
```

      0.000001 seconds (3 allocations: 144 bytes)

```julia

```


## Histograms of Oriented Gradients (HOG)

As we saw with the ORB algorithm, we can use keypoints in images to do keypoint-based matching to detect objects in images. These type of algorithms work great when you want to detect objects that have a lot of consistent internal features that are not affected by the background. For example, these algorithms work well for facial detection because faces have a lot of consistent internal features that don’t get affected by the image background, such as the eyes, nose, and mouth. However, these type of algorithms don’t work so well when attempting to do more general object recognition, say for example, pedestrian detection in images. The reason is that people don’t have consistent internal features, like faces do, because the body shape and style of every person is different (see Fig. 1). This means that every person is going to have a different set of internal features, and so we need something that can more generally describe a person.

Fig. 1. - Pedestrians.

One option is to try to detect pedestrians by their contours instead.  Detecting objects in images by their contours (boundaries) is very challenging because we have to deal with the difficulties brought about by the contrast between the background and the foreground. For example, suppose you wanted to detect a pedestrian in an image that is walking in front of a white building and she is wearing a white coat and black pants (see Fig. 2). We can see in Fig. 2, that since the background of the image is mostly white, the black pants are going to have a very high contrast, but the coat, since it is white as well, is going to have very low contrast. In this case, detecting the edges of pants is going to be easy but detecting the edges of the coat is going to be very difficult. This is where **HOG** comes in. HOG stands for **Histograms of Oriented Gradients** and it was first introduced by Navneet Dalal and Bill Triggs in 2005.

Fig. 2. - High and Low Contrast.

The HOG algorithm works by creating histograms of the distribution of gradient orientations in an image and then normalizing them in a very special way. This special normalization is what makes HOG so effective at detecting the edges of objects even in cases where the contrast is very low. These normalized histograms are put together into a feature vector, known as the HOG descriptor, that can be used to train a machine learning algorithm, such as a Support Vector Machine (SVM), to detect objects in images based on their boundaries (edges). Due to its great success and reliability, HOG has become one of the most widely used algorithms in computer vison for object detection.

In this notebook, you will learn:

* How the HOG algorithm works
* How to use OpenCV to create a HOG descriptor
* How to visualize the HOG descriptor.

# The HOG Algorithm

As its name suggests, the HOG algorithm, is based on creating histograms from the orientation of image gradients. The HOG algorithm is implemented in a series of steps:

1. Given the image of particular object, set a detection window (region of interest) that covers the entire object in the image (see Fig. 3).

2. Calculate the magnitude and direction of the gradient for each individual pixel in the detection window.

3. Divide the detection window into connected *cells* of pixels, with all cells being of the same size (see Fig. 3). The size of the cells is a free parameter and it is usually chosen so as to match the scale of the features that want to be detected. For example, in a 64 x 128 pixel detection window, square cells 6 to 8 pixels wide are suitable for detecting human limbs.

4. Create a Histogram for each cell, by first grouping the gradient directions of all pixels in each cell into a particular number of orientation (angular) bins; and then adding up the gradient magnitudes of the gradients in each angular bin (see Fig. 3). The number of bins in the histogram is a free parameter and it is usually set to 9 angular bins.

5. Group adjacent cells into *blocks* (see Fig. 3). The number of cells in each block is a free parameter and all blocks must be of the same size. The distance between each block (known as the stride) is a free parameter but it is usually set to half the block size, in which case you will get overlapping blocks (*see video below*).  The HOG algorithm has been shown empirically to work better with overlapping blocks.

6. Use the cells contained within each block to normalize the cell histograms in that block (see Fig. 3). If you have overlapping blocks this means that most cells will be normalized with respect to different blocks (*see video below*). Therefore, the same cell may have several different normalizations.

7. Collect all the normalized histograms from all the blocks into a single feature vector called the HOG descriptor.

8. Use the resulting HOG descriptors from many images of the same type of object to train a machine learning algorithm, such as an SVM, to detect those type of objects in images. For example, you could use the HOG descriptors from many images of pedestrians to train an SVM to detect pedestrians in images. The training is done with both positive a negative examples of the object you want detect in the image.

9. Once the SVM has been trained, a sliding window approach is used to try to detect and locate objects in images.  Detecting an object in the image entails finding the part of the image that looks similar to the HOG pattern learned by the SVM.

Fig. 3. - HOG Diagram.

Vid. 1. - HOG Animation.

# Why The HOG Algorithm Works

As we learned above, HOG creates histograms by adding the magnitude of the gradients in particular orientations in localized portions of the image called *cells*.  By doing this we guarantee that stronger gradients will contribute more to the magnitude of their respective angular bin, while the effects of weak and randomly oriented gradients resulting from noise are minimized. In this manner the histograms tell us the dominant gradient orientation of each cell.

### Dealing with contrast

Now, the magnitude of the dominant orientation can vary widely due to variations in local illumination and the contrast between the background and the foreground.

To account for the background-foreground contrast differences, the HOG algorithm tries to detect edges locally.  In order to do this, it defines groups of cells, called **blocks**, and normalizes the histograms using this local group of cells. By normalizing locally, the HOG algorithm can detect the edges in each block very reliably; this is called **block normalization**.

In addition to using block normalization, the HOG algorithm also uses overlapping blocks to increase its performance. By using overlapping blocks, each cell contributes several independent components to the final HOG descriptor, where each component corresponds to a cell being normalized with respect to a different block. This may seem redundant but, it has been shown empirically that by normalizing each cell several times with respect to different local blocks, the performance of the HOG algorithm increases dramatically.

### Loading Images and Importing Resources

The first step in building our HOG descriptor is to load the required packages into Python and to load our image.

We start by using OpenCV to load an image of a triangle tile. Since, the `cv2.imread()` function loads images as BGR we will convert our image to RGB so we can display it with the correct colors. As usual we will convert our BGR image to Gray Scale for analysis.

```python
import cv2
import numpy as np
import matplotlib.pyplot as plt

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Load the image
image = cv2.imread('./images/triangle_tile.jpeg')

# Convert the original image to RGB
original_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Convert the original image to gray scale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Print the shape of the original and gray scale images
print('The original image has shape: ', original_image.shape)
print('The gray scale image has shape: ', gray_image.shape)

# Display the images
plt.subplot(121)
plt.imshow(original_image)
plt.title('Original Image')
plt.subplot(122)
plt.imshow(gray_image, cmap='gray')
plt.title('Gray Scale Image')
plt.show()
```

# Creating The HOG Descriptor

We will be using OpenCV’s `HOGDescriptor` class to create the HOG descriptor. The parameters of the HOG descriptor are setup using the `HOGDescriptor()` function. The parameters of the `HOGDescriptor()` function and their default values are given below:

`cv2.HOGDescriptor(win_size = (64, 128),
                  block_size = (16, 16),
                  block_stride = (8, 8),
                  cell_size = (8, 8),
                  nbins = 9,
                  win_sigma = DEFAULT_WIN_SIGMA,
                  threshold_L2hys = 0.2,
                  gamma_correction = true,
                  nlevels = DEFAULT_NLEVELS)`

Parameters:

* **win_size** – *Size*
Size of detection window in pixels (*width, height*). Defines the region of interest. Must be an integer multiple of cell size.

* **block_size** – *Size*
Block size in pixels (*width, height*). Defines how many cells are in each block. Must be an integer multiple of cell size and it must be smaller than the detection window. The smaller the block the finer detail you will get.

* **block_stride** – *Size*
Block stride in pixels (*horizontal, vertical*). It must be an integer multiple of cell size. The `block_stride` defines the distance between adjecent blocks, for example, 8 pixels horizontally and 8 pixels vertically. Longer `block_strides` makes the algorithm run faster (because less blocks are evaluated) but the algorithm may not perform as well.

* **cell_size** – *Size*
Cell size in pixels (*width, height*). Determines the size fo your cell. The smaller the cell the finer detail you will get.

* **nbins** – *int*
Number of bins for the histograms. Determines the number of angular bins used to make the histograms. With more bins you capture more gradient directions. HOG uses unsigned gradients, so the angular bins will have values between 0 and 180 degrees.

* **win_sigma** – *double*
Gaussian smoothing window parameter. The performance of the HOG algorithm can be improved by smoothing the pixels near the edges of the blocks by applying a Gaussian spatial window to each pixel before computing the histograms.

* **threshold_L2hys** – *double*
L2-Hys (Lowe-style clipped L2 norm) normalization method shrinkage. The L2-Hys method is used to normalize the blocks and it consists of an L2-norm followed by clipping and a renormalization. The clipping limits the maximum value of the descriptor vector for each block to have the value of the given threshold (0.2 by default). After the clipping the descriptor vector is renormalized as described in *IJCV*, 60(2):91-110, 2004.

* **gamma_correction** – *bool*
Flag to specify whether the gamma correction preprocessing is required or not. Performing gamma correction slightly increases the performance of the HOG algorithm.

* **nlevels** – *int*
Maximum number of detection window increases.

As we can see, the  `cv2.HOGDescriptor()`function supports a wide range of parameters. The first few arguments (`block_size, block_stride, cell_size`, and `nbins`) are probably the ones you are most likely to change. The other parameters can be safely left at their default values and you will get good results.

In the code below, we will use the `cv2.HOGDescriptor()`function to set the cell size, block size, block stride, and the number of bins for the histograms of the HOG descriptor. We will then use `.compute(image)`method to compute the HOG descriptor (feature vector) for the given `image`.

```python
# Specify the parameters for our HOG descriptor

# Cell Size in pixels (width, height). Must be smaller than the size of the detection window
# and must be chosen so that the resulting Block Size is smaller than the detection window.
cell_size = (6, 6)

# Number of cells per block in each direction (x, y). Must be chosen so that the resulting
# Block Size is smaller than the detection window
num_cells_per_block = (2, 2)

# Block Size in pixels (width, height). Must be an integer multiple of Cell Size.
# The Block Size must be smaller than the detection window
block_size = (num_cells_per_block[0] * cell_size[0],
              num_cells_per_block[1] * cell_size[1])

# Calculate the number of cells that fit in our image in the x and y directions
x_cells = gray_image.shape[1] // cell_size[0]
y_cells = gray_image.shape[0] // cell_size[1]

# Horizontal distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (x_cells - num_cells_per_block[0]) / h_stride = integer.
h_stride = 1

# Vertical distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (y_cells - num_cells_per_block[1]) / v_stride = integer.
v_stride = 1

# Block Stride in pixels (horizantal, vertical). Must be an integer multiple of Cell Size
block_stride = (cell_size[0] * h_stride, cell_size[1] * v_stride)

# Number of gradient orientation bins
num_bins = 9

# Specify the size of the detection window (Region of Interest) in pixels (width, height).
# It must be an integer multiple of Cell Size and it must cover the entire image. Because
# the detection window must be an integer multiple of cell size, depending on the size of
# your cells, the resulting detection window might be slightly smaller than the image.
# This is perfectly ok.
win_size = (x_cells * cell_size[0] , y_cells * cell_size[1])

# Print the shape of the gray scale image for reference
print('\nThe gray scale image has shape: ', gray_image.shape)
print()

# Print the parameters of our HOG descriptor
print('HOG Descriptor Parameters:\n')
print('Window Size:', win_size)
print('Cell Size:', cell_size)
print('Block Size:', block_size)
print('Block Stride:', block_stride)
print('Number of Bins:', num_bins)
print()

# Set the parameters of the HOG descriptor using the variables defined above
hog = cv2.HOGDescriptor(win_size, block_size, block_stride, cell_size, num_bins)

# Compute the HOG Descriptor for the gray scale image
hog_descriptor = hog.compute(gray_image)
```

    The gray scale image has shape:  (250, 250)

    HOG Descriptor Parameters:

    Window Size: (246, 246)
    Cell Size: (6, 6)
    Block Size: (12, 12)
    Block Stride: (6, 6)
    Number of Bins: 9

# Number of Elements In The HOG Descriptor

The resulting HOG Descriptor (feature vector), contains the normalized histograms from all cells from all blocks in the detection window concatenated in one long vector. Therefore, the size of the HOG feature vector will be given by the total number of blocks in the detection window, multiplied by the number of cells per block, times the number of orientation bins:

\begin{equation}
\mbox{total_elements} = (\mbox{total_number_of_blocks})\mbox{ } \times \mbox{ } (\mbox{number_cells_per_block})\mbox{ } \times \mbox{ } (\mbox{number_of_bins})
\end{equation}

If we don’t have overlapping blocks (*i.e.* the `block_stride`equals the `block_size`), the total number of blocks can be easily calculated by dividing the size of the detection window by the block size. However, in the general case we have to take into account the fact that we have overlapping blocks. To find the total number of blocks in the general case (*i.e.* for any `block_stride` and `block_size`), we can use the formula given below:

\begin{equation}
\mbox{Total}_i = \left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right)\left( \frac{\mbox{window_size}_i}{\mbox{block_size}_i} \right) - \left [\left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right) - 1 \right]; \mbox{  for  } i = x,y
\end{equation}

Where Total$_x$, is the total number of blocks along the width of the detection window, and Total$_y$, is the total number of blocks along the height of the detection window. This formula for Total$_x$ and Total$_y$, takes into account the extra blocks that result from overlapping. After calculating Total$_x$ and Total$_y$, we can get the total number of blocks in the detection window by multiplying Total$_x$  $\times$ Total$_y$. The above formula can be simplified considerably because the `block_size`, `block_stride`, and `window_size`are all defined in terms of the `cell_size`. By making all the appropriate substitutions and cancelations the above formula reduces to:

\begin{equation}
\mbox{Total}_i = \left(\frac{\mbox{cells}_i - \mbox{num_cells_per_block}_i}{N_i}\right) + 1\mbox{  }; \mbox{  for  } i = x,y
\end{equation}

Where cells$_x$ is the total number of cells along the width of the detection window, and cells$_y$, is the total number of cells along the height of the detection window. And $N_x$ is the horizontal block stride in units of `cell_size` and $N_y$ is the vertical block stride in units of `cell_size`.

Let's calculate what the number of elements for the HOG feature vector should be and check that it matches the shape of the HOG Descriptor calculated above.

```python
# Calculate the total number of blocks along the width of the detection window
tot_bx = np.uint32(((x_cells - num_cells_per_block[0]) / h_stride) + 1)

# Calculate the total number of blocks along the height of the detection window
tot_by = np.uint32(((y_cells - num_cells_per_block[1]) / v_stride) + 1)

# Calculate the total number of elements in the feature vector
tot_els = (tot_bx) * (tot_by) * num_cells_per_block[0] * num_cells_per_block[1] * num_bins

# Print the total number of elements the HOG feature vector should have
print('\nThe total number of elements in the HOG Feature Vector should be: ',
      tot_bx, 'x',
      tot_by, 'x',
      num_cells_per_block[0], 'x',
      num_cells_per_block[1], 'x',
      num_bins, '=',
      tot_els)

# Print the shape of the HOG Descriptor to see that it matches the above
print('\nThe HOG Descriptor has shape:', hog_descriptor.shape)
print()
```

    The total number of elements in the HOG Feature Vector should be:  40 x 40 x 2 x 2 x 9 = 57600

    The HOG Descriptor has shape: (57600, 1)

# Visualizing The HOG Descriptor

We can visualize the HOG Descriptor by plotting the histogram associated with each cell as a collection of vectors. To do this, we will plot each bin in the histogram as a single vector whose magnitude is given by the height of the bin and its orientation is given by the angular bin that its associated with. Since any given cell might have multiple histograms associated with it, due to the overlapping blocks, we will choose to average all the histograms for each cell to produce a single histogram for each cell.

OpenCV has no easy way to visualize the HOG Descriptor, so we have to do some manipulation first in order to visualize it. We will start by reshaping the HOG Descriptor in order to make our calculations easier.  We will then compute the average histogram of each cell and finally we will convert the histogram bins into vectors. Once we have the vectors, we plot the corresponding vectors for each cell in an image.

The code below produces an interactive plot so that you can interact with the figure. The figure contains:
* the grayscale image,
* the HOG Descriptor (feature vector),
* a zoomed-in portion of the HOG Descriptor, and
* the histogram of the selected cell.

**You can click anywhere on the gray scale image or the HOG Descriptor image to select a particular cell**. Once you click on either image a *magenta* rectangle will appear showing the cell you selected. The Zoom Window will show you a zoomed in version of the HOG descriptor around the selected cell; and the histogram plot will show you the corresponding histogram for the selected cell. The interactive window also has buttons at the bottom that allow for other functionality, such as panning, and giving you the option to save the figure if desired. The home button returns the figure to its default value.

**NOTE**: If you are running this notebook in the Udacity workspace, there is around a 2 second lag in the interactive plot. This means that if you click in the image to zoom in, it will take about 2 seconds for the plot to refresh.

```python
%matplotlib notebook

import copy
import matplotlib.patches as patches

# Set the default figure size
plt.rcParams['figure.figsize'] = [9.8, 9]

# Reshape the feature vector to [blocks_y, blocks_x, num_cells_per_block_x, num_cells_per_block_y, num_bins].
# The blocks_x and blocks_y will be transposed so that the first index (blocks_y) referes to the row number
# and the second index to the column number. This will be useful later when we plot the feature vector, so
# that the feature vector indexing matches the image indexing.
hog_descriptor_reshaped = hog_descriptor.reshape(tot_bx,
                                                 tot_by,
                                                 num_cells_per_block[0],
                                                 num_cells_per_block[1],
                                                 num_bins).transpose((1, 0, 2, 3, 4))

# Print the shape of the feature vector for reference
print('The feature vector has shape:', hog_descriptor.shape)

# Print the reshaped feature vector
print('The reshaped feature vector has shape:', hog_descriptor_reshaped.shape)

# Create an array that will hold the average gradients for each cell
ave_grad = np.zeros((y_cells, x_cells, num_bins))

# Print the shape of the ave_grad array for reference
print('The average gradient array has shape: ', ave_grad.shape)

# Create an array that will count the number of histograms per cell
hist_counter = np.zeros((y_cells, x_cells, 1))

# Add up all the histograms for each cell and count the number of histograms per cell
for i in range (num_cells_per_block[0]):
    for j in range(num_cells_per_block[1]):
        ave_grad[i:tot_by + i,
                 j:tot_bx + j] += hog_descriptor_reshaped[:, :, i, j, :]

        hist_counter[i:tot_by + i,
                     j:tot_bx + j] += 1

# Calculate the average gradient for each cell
ave_grad /= hist_counter

# Calculate the total number of vectors we have in all the cells.
len_vecs = ave_grad.shape[0] * ave_grad.shape[1] * ave_grad.shape[2]

# Create an array that has num_bins equally spaced between 0 and 180 degress in radians.
deg = np.linspace(0, np.pi, num_bins, endpoint = False)

# Each cell will have a histogram with num_bins. For each cell, plot each bin as a vector (with its magnitude
# equal to the height of the bin in the histogram, and its angle corresponding to the bin in the histogram).
# To do this, create rank 1 arrays that will hold the (x,y)-coordinate of all the vectors in all the cells in the
# image. Also, create the rank 1 arrays that will hold all the (U,V)-components of all the vectors in all the
# cells in the image. Create the arrays that will hold all the vector positons and components.
U = np.zeros((len_vecs))
V = np.zeros((len_vecs))
X = np.zeros((len_vecs))
Y = np.zeros((len_vecs))

# Set the counter to zero
counter = 0

# Use the cosine and sine functions to calculate the vector components (U,V) from their maginitudes. Remember the
# cosine and sine functions take angles in radians. Calculate the vector positions and magnitudes from the
# average gradient array
for i in range(ave_grad.shape[0]):
    for j in range(ave_grad.shape[1]):
        for k in range(ave_grad.shape[2]):
            U[counter] = ave_grad[i,j,k] * np.cos(deg[k])
            V[counter] = ave_grad[i,j,k] * np.sin(deg[k])

            X[counter] = (cell_size[0] / 2) + (cell_size[0] * i)
            Y[counter] = (cell_size[1] / 2) + (cell_size[1] * j)

            counter = counter + 1

# Create the bins in degress to plot our histogram.
angle_axis = np.linspace(0, 180, num_bins, endpoint = False)
angle_axis += ((angle_axis[1] - angle_axis[0]) / 2)

# Create a figure with 4 subplots arranged in 2 x 2
fig, ((a,b),(c,d)) = plt.subplots(2,2)

# Set the title of each subplot
a.set(title = 'Gray Scale Image\n(Click to Zoom)')
b.set(title = 'HOG Descriptor\n(Click to Zoom)')
c.set(title = 'Zoom Window', xlim = (0, 18), ylim = (0, 18), autoscale_on = False)
d.set(title = 'Histogram of Gradients')

# Plot the gray scale image
a.imshow(gray_image, cmap = 'gray')
a.set_aspect(aspect = 1)

# Plot the feature vector (HOG Descriptor)
b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
b.invert_yaxis()
b.set_aspect(aspect = 1)
b.set_facecolor('black')

# Define function for interactive zoom
def onpress(event):

    #Unless the left mouse button is pressed do nothing
    if event.button != 1:
        return

    # Only accept clicks for subplots a and b
    if event.inaxes in [a, b]:

        # Get mouse click coordinates
        x, y = event.xdata, event.ydata

        # Select the cell closest to the mouse click coordinates
        cell_num_x = np.uint32(x / cell_size[0])
        cell_num_y = np.uint32(y / cell_size[1])

        # Set the edge coordinates of the rectangle patch
        edgex = x - (x % cell_size[0])
        edgey = y - (y % cell_size[1])

        # Create a rectangle patch that matches the the cell selected above
        rect = patches.Rectangle((edgex, edgey),
                                  cell_size[0], cell_size[1],
                                  linewidth = 1,
                                  edgecolor = 'magenta',
                                  facecolor='none')

        # A single patch can only be used in a single plot. Create copies
        # of the patch to use in the other subplots
        rect2 = copy.copy(rect)
        rect3 = copy.copy(rect)

        # Update all subplots
        a.clear()
        a.set(title = 'Gray Scale Image\n(Click to Zoom)')
        a.imshow(gray_image, cmap = 'gray')
        a.set_aspect(aspect = 1)
        a.add_patch(rect)

        b.clear()
        b.set(title = 'HOG Descriptor\n(Click to Zoom)')
        b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
        b.invert_yaxis()
        b.set_aspect(aspect = 1)
        b.set_facecolor('black')
        b.add_patch(rect2)

        c.clear()
        c.set(title = 'Zoom Window')
        c.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 1)
        c.set_xlim(edgex - cell_size[0], edgex + (2 * cell_size[0]))
        c.set_ylim(edgey - cell_size[1], edgey + (2 * cell_size[1]))
        c.invert_yaxis()
        c.set_aspect(aspect = 1)
        c.set_facecolor('black')
        c.add_patch(rect3)

        d.clear()
        d.set(title = 'Histogram of Gradients')
        d.grid()
        d.set_xlim(0, 180)
        d.set_xticks(angle_axis)
        d.set_xlabel('Angle')
        d.bar(angle_axis,
              ave_grad[cell_num_y, cell_num_x, :],
              180 // num_bins,
              align = 'center',
              alpha = 0.5,
              linewidth = 1.2,
              edgecolor = 'k')

        fig.canvas.draw()

# Create a connection between the figure and the mouse click
fig.canvas.mpl_connect('button_press_event', onpress)
plt.show()
```

    The feature vector has shape: (57600, 1)
    The reshaped feature vector has shape: (40, 40, 2, 2, 9)
    The average gradient array has shape:  (41, 41, 9)

    <IPython.core.display.Javascript object>

# Understanding The Histograms

Let's take a look at a couple of snapshots of the above figure to see if the histograms for the selected cell make sense. Let's start looking at a cell that is inside a triangle and not near an edge:

Fig. 4. - Histograms Inside a Triangle.

In this case, since the triangle is nearly all of the same color there shouldn't be any dominant gradient in the selected cell. As we can clearly see in the Zoom Window and the histogram, this is indeed the case. We have many gradients but none of them clearly dominates over the other.

Now let’s take a look at a cell that is near a horizontal edge:

Fig. 5. - Histograms Near a Horizontal Edge.

Remember that edges are areas of an image where the intensity changes abruptly. In these cases, we will have a high intensity gradient in some particular direction. This is exactly what we see in the corresponding histogram and Zoom Window for the selected cell.  In the Zoom Window, we can see that the dominant gradient is pointing up, almost at 90 degrees, since that’s the direction in which there is a sharp change in intensity. Therefore, we should expect to see the 90-degree bin in the histogram to dominate strongly over the others. This is in fact what we see.

Now let’s take a look at a cell that is near a vertical edge:

Fig. 6. - Histograms Near a Vertical Edge.

In this case we expect the dominant gradient in the cell to be horizontal, close to 180 degrees, since that’s the direction in which there is a sharp change in intensity. Therefore, we should expect to see the 170-degree bin in the histogram to dominate strongly over the others. This is what we see in the histogram but we also see that there is another dominant gradient in the cell, namely the one in the 10-degree bin. The reason for this, is because the HOG algorithm is using unsigned gradients, which means 0 degrees and 180 degrees are considered the same. Therefore, when the histograms are being created, angles between 160 and 180 degrees, contribute proportionally to both the 10-degree bin and the 170-degree bin. This results in there being two dominant gradients in the cell near the vertical edge instead of just one.

To conclude let’s take a look at a cell that is near a diagonal edge.

Fig. 7. - Histograms Near a Diagonal Edge.

To understand what we are seeing, let’s first remember that gradients have an *x*-component, and a *y*-component, just like vectors. Therefore, the resulting orientation of a gradient is going to be given by the vector sum of its components. For this reason, on vertical edges the gradients are horizontal, because they only have an x-component, as we saw in Figure 4. While on horizontal edges the gradients are vertical, because they only have a y-component, as we saw in Figure 3. Consequently, on diagonal edges, the gradients are also going to be diagonal because both the *x* and *y* components are non-zero.  Since the diagonal edges in the image are close to 45 degrees, we should expect to see a dominant gradient orientation in the 50-degree bin. This is in fact what we see in the histogram but, just like in Figure 4., we see there are two dominant gradients instead of just one. The reason for this is that when the histograms are being created, angles that are near the boundaries of bins, contribute proportionally to the adjacent bins.  For example, a gradient with an angle of 40 degrees, is right in the middle of the 30-degree and 50-degree bin. Therefore, the magnitude of the gradient is split evenly into the 30-degree and 50-degree bin. This results in there being two dominant gradients in the cell near the diagonal edge instead of just one.

Now that you know how HOG is implemented, in the workspace you will find a notebook named *Examples*. In there, you will be able set your own paramters for the HOG descriptor for various images. Have fun!


[Sebastian Raschka](http://www.sebastianraschka.com)

- [Link to the containing GitHub Repository](https://github.com/rasbt/algorithms_in_ipython_notebooks)
- [Link to this IPython Notebook on GitHub](https://github.com/rasbt/algorithms_in_ipython_notebooks/blob/master/ipython_nbs/geometry/points_in_hybercube.ipynb)

```
%load_ext watermark
```

```
%watermark
```

    30/06/2014 21:28:32

    CPython 3.4.1
    IPython 2.1.0

    compiler   : GCC 4.2.1 (Apple Inc. build 5577)
    system     : Darwin
    release    : 13.2.0
    machine    : x86_64
    processor  : i386
    CPU cores  : 2
    interpreter: 64bit

[More information](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/ipython_magic/watermark.ipynb) about the `watermark` magic command extension.

# Counting points inside a hypercube

Given a vector of coordinates, we want to determine how many points lie inside a hypercube (a hypercube is the more general concept of a 3-dimensional cube in $d$ dimensions):

\begin{equation} \begin{pmatrix}  x_i\\ x_{i+1} \\ ... \\ x_{d} \end{pmatrix} \end{equation}

E.g., in a 3-dimensional case in a kartesian coordinate system:

\begin{equation} \begin{pmatrix}  x\\ y\\ z \end{pmatrix} \end{equation}

To approach this problem more mathematically, we would use the following equation to count the samples $k_n$ within this hypercube, where $\phi$ is our so-called *window function*:

$\phi(\pmb u) = \Bigg[ \begin{array}{ll} 1 & \quad |u_j| \leq 1/2 \; ;\quad \quad j = 1, ..., d \\
0 & \quad otherwise \end{array}$

for a hypercube of unit length 1 centered at the coordinate system's origin. What this function basically does is assigning a value 1 to a sample point if it lies within 1/2 of the edges of the hypercube, and 0 if lies outside (note that the evaluation is done for all dimensions of the sample point).

If we extend on this concept, we can define a more general equation that applies to hypercubes of any length $h_n$ that are centered at $\pmb x$:

$k_n = \sum\limits_{i=1}^{n} \phi \bigg( \frac{\pmb x - \pmb x_i}{h_n} \bigg)$

where $\pmb u = \bigg( \frac{\pmb x - \pmb x_i}{h_n} \bigg)$

## Implementation

```
def inside_hypercube(x_vec, unit_len=1):
    """ Function to check if a point lies inside a hypercube."""
    for ele in x_vec:
        if abs(ele) > (unit_len/2):
            return False
    return True
```

## Example 3D-hypercubes

So let us visualize such an simple example: a typical 3-dimensional unit hypercube ($h_1 = 1$) and 10 sample points, where 3 of them lie within the hypercube (red triangles), and the other 7 outside (black dots).

```
samples_3d = [
            [0, 0, 0], [0.2, 0.2, 0.2], [0.1, -0.1, -0.3],
            [-1.2,0.3,-0.3], [0.8,-0.82,-0.9], [1, 0.6, -0.7],
            [0.8,0.7,0.2], [0.7,-0.8,-0.45], [-0.3, 0.6, 0.9],
            [0.7,-0.6,-0.8]
            ]
```

```
%matplotlib inline
```

```
inside = []
outside = []
for vec in samples_3d:
    if inside_hypercube(vec, unit_len=1):
        inside.append(vec)
    else:
        outside.append(vec)
```

```
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np
from itertools import product, combinations
fig = plt.figure(figsize=(7,7))
ax = fig.gca(projection='3d')
ax.set_aspect("equal")

for row in inside:
    ax.scatter(row[0], row[1], row[2], color="r", s=50, marker='^')

for row in outside:
    ax.scatter(row[0], row[1], row[2], color="k", s=50)

# Plot Cube centered at the coordinate origin X=0, Y=0, Z=0
h = [-0.5, 0.5]
for s, e in combinations(np.array(list(product(h,h,h))), 2):
    if np.sum(np.abs(s-e)) == h[1]-h[0]:
        ax.plot3D(*zip(s,e), color="g")

ax.set_xlim(-1.5, 1.5)
ax.set_ylim(-1.5, 1.5)
ax.set_zlim(-1.5, 1.5)

plt.show()
```


# Lecture 5

## Differentiation III:

### Exponentials and Partial Differentiation

```python
import numpy as np
import sympy as sp
sp.init_printing()

##################################################
##### Matplotlib boilerplate for consistency #####
##################################################
from ipywidgets import interact
from ipywidgets import FloatSlider
from matplotlib import pyplot as plt

%matplotlib inline

from IPython.display import set_matplotlib_formats
set_matplotlib_formats('svg')

global_fig_width = 10
global_fig_height = global_fig_width / 1.61803399
font_size = 12

plt.rcParams['axes.axisbelow'] = True
plt.rcParams['axes.edgecolor'] = '0.8'
plt.rcParams['axes.grid'] = True
plt.rcParams['axes.labelpad'] = 8
plt.rcParams['axes.linewidth'] = 2
plt.rcParams['axes.titlepad'] = 16.0
plt.rcParams['axes.titlesize'] = font_size * 1.4
plt.rcParams['figure.figsize'] = (global_fig_width, global_fig_height)
plt.rcParams['font.sans-serif'] = ['Computer Modern Sans Serif', 'DejaVu Sans', 'sans-serif']
plt.rcParams['font.size'] = font_size
plt.rcParams['grid.color'] = '0.8'
plt.rcParams['grid.linestyle'] = 'dashed'
plt.rcParams['grid.linewidth'] = 2
plt.rcParams['lines.dash_capstyle'] = 'round'
plt.rcParams['lines.dashed_pattern'] = [1, 4]
plt.rcParams['xtick.labelsize'] = font_size
plt.rcParams['xtick.major.pad'] = 4
plt.rcParams['xtick.major.size'] = 0
plt.rcParams['ytick.labelsize'] = font_size
plt.rcParams['ytick.major.pad'] = 4
plt.rcParams['ytick.major.size'] = 0
##################################################
```

## Wake Up Exercise

Find $\displaystyle y' = \frac{{\rm d}y}{{\rm d}x}$ when $y$ is given by:

1. $y=5x^2$
2. $y=\root 4\of x$
3. $y=x+{1\over\sqrt{x^3}}$
4. $y=\sqrt{6x^4+2}$
5. $y={x\over 3x+2}$
6. $y=x^2\sin x$

## Examples of applying chain rule to the exponential function.

1. $y=e^{-ax}$. Let $u=-ax\Rightarrow\frac{{\rm d}u}{{\rm d}x}=-a$. Thus $y=e^u$ and

$$\frac{{\rm d}y}{{\rm d}u}=e^u~~\Rightarrow~~\frac{{\rm d}y}{{\rm d}x}=\frac{{\rm d}y}{{\rm d}u}\times\frac{{\rm d}u}{{\rm d}x}=e^u\times
(-a)=-ae^{-ax}.$$

2. $\displaystyle y = e^{x^2}$. Then, letting $u = x^2$:

$$\frac{{\rm d}}{{\rm d}x}e^{x^2}=\frac{{\rm d}y}{{\rm d}x}=\frac{{\rm d}y}{{\rm d}u}\times\frac{{\rm d}u}{{\rm d}x}=e^u\cdot 2x =
e^{x^2}\cdot 2x.$$

An important generalization:

$\frac{{\rm d}}{{\rm d}x}e^{f(x)}=e^{f(x)}f'(x)$ for any function $f(x)$.

## Example with the natural logarithm.

1. $y=\ln(a-x)^2=2\ln(a-x)=2\ln u$. Let $u=(a-x)$:

$$\Rightarrow {{\rm d}u\over {\rm d}x}=-1~~{\rm and~~~~~}{{\rm d}y\over {\rm d}u}={2\over u}~~~
{\rm Thus~~~~}{{\rm d}y\over {\rm d}x}={2\over u}\times (-1)={-2\over a-x}$$

This also generalises:

$$\frac{{\rm d}}{{\rm d}x}\ln(f(x)) = {f'(x)\over f(x)}$$

## The Derivative of $a^x$:

By the properties of logarithms and indices we have
$\displaystyle a^x = \left({e^{\ln a}}\right)^x=e^{\left({x\cdot\ln a}\right)}$.

Thus, as we saw above we have:

$$\frac{{\rm d}}{{\rm d}x}a^x
   = \frac{{\rm d}}{{\rm d}x}e^{\left({x\cdot\ln a}\right)}
   = e^{\left({x\cdot\ln a}\right)}\frac{{\rm d}}{{\rm d}x}{\left({x\cdot\ln a}\right)}
   =a^x\cdot\ln a$$

Similarly, in general:

$$\frac{{\rm d}}{{\rm d}x}a^{f(x)} = a^{f(x)}\cdot \ln a\cdot f'(x)$$

## Sympy Example

Lets try and use Sympy to prove the rule:
$$\frac{{\rm d}}{{\rm d}x}a^{f(x)} = a^{f(x)}\cdot \ln a\cdot f'(x)$$

```python
x, a = sp.symbols('x a') # declare the variables x and a
f = sp.Function('f')     # declare a function dependent on another variable
sp.diff(a**f(x),x)       # write the expression we wish to evaluate
```

## The Derivative of $\log_a x\,\,$:

Recall the conversion formula $\displaystyle \log_a x = {{\ln x}\over {\ln a}}$
and note that $\ln a$ is a constant.  Thus:

$$\frac{{\rm d}}{{\rm d}x}\log_a x
  = \frac{{\rm d}}{{\rm d}x}\left({1\over{\ln a}}\cdot\ln x\right)
  = \left({1\over{\ln a}}\right)\cdot\frac{{\rm d}}{{\rm d}x}\ln x
  = \left({1\over{\ln a}}\right)\cdot{1\over {x}}
  = {1\over{x\cdot\ln a}}$$

In general:

$$\displaystyle \frac{{\rm d}}{{\rm d}x}\log_a f(x) = {{f'(x)} \over {f(x){(\ln a)}}}$$

## Sympy Example

Lets try and use Sympy again to prove the rule:
$$\frac{{\rm d}}{{\rm d}x}\log_a f(x) = {{f'(x)} \over {f(x){(\ln a)}}}$$

```python
sp.diff(sp.log(f(x),a),x)
```

## Further examples:

1.  Product Rule:  Let $\displaystyle y = x^2\,e^x$.  Then:

$${{dy\over dx}}={d\over dx}x^2e^x={d\over dx}x^2\cdot e^x+x^2\cdot{d\over dx}e^x = (2x + x^2)e^x$$

2.  Quotient Rule:  Let $\displaystyle y = {{e^x}\over x}$.  Then:

$${{dy\over dx}}={{{{d\over dx}e^x}\cdot x - e^x\cdot {d\over dx}x}\over {x^2}}={{e^x\cdot x - e^x\cdot 1\over {x^2}}}={{x - 1}\over x^2}e^x$$

3. Chain Rule: $\displaystyle y = e^{x^2}$.  Then, letting $f(x) = x^2$:

$$\frac{{\rm d}}{{\rm d}x}e^{x^2} = e^{f(x)}f'(x) = e^{x^2}\cdot 2x$$

4. $\displaystyle y=\ln (x^2 + 1)$.  Then, letting $f(x) = x^2+1$:

$$\frac{{\rm d}}{{\rm d}x}\ln(x^2+1) = {f'(x)\over f(x)} = {2x\over {x^2+1}}$$

5. $\displaystyle {{\rm d}\over {\rm d}x}2^{x^3}=2^{x^3}\cdot\ln 2\cdot 3x^2$

6. $\displaystyle {{\rm d}\over {\rm d}x}10^{x^2+1}=10^{x^2+1}\cdot\ln 10\cdot 2x$

7. $\displaystyle \frac{{\rm d}}{{\rm d}x}\log_{10}(7x+5)={7\over {(7x+5)\cdot \ln10}}$

8. $\displaystyle \frac{{\rm d}}{{\rm d}x}\log_2(3^x+x^4)={{3^x\cdot\ln3 + 4x^3}\over{\ln 2\cdot(3^x+x^4)}}$

## Functions of several variables: Partial Differentiation

**Definition:**  Given a function $z=f(x,y)$ of two variables $x$ and $y$, the **partial derivative of $z$ with respect to $x$** is the function obtained by differentiating $f(x,y)$ with respect to $x$, holding $y$ constant.

We denote this using $\partial$ (the "curly" delta, sometimes pronounced "del") as shown below:

$$\frac{\partial z}{\partial x}=\frac{\partial}{\partial x}f(x,y) = f_x(x,y)$$

## Example 1

$f(x,y)=z=x^2-2y^2$

$$f_x={\partial z\over \partial x}=2x\qquad\mbox{and}\qquad f_y={\partial z\over \partial y}=-4y$$

## Example 2

Let $z=3x^2y+5xy^2$. Then the partial derivative of $z$ with respect to $x$, holding $y$ fixed, is:

\begin{align*}
\frac{\partial z}{\partial x}&=\frac{\partial}{\partial x}\,\left(3x^2y+5xy^2\right) \\
&=3y\cdot 2x + 5y^2\cdot 1 \\
&=6xy+5y^2
\end{align*}

while the partial of $z$ with respect to $y$ holding $x$ fixed is:

\begin{align*}
\frac{\partial z}{\partial y}&=\frac{\partial}{\partial y}\,\left(3x^2y+5xy^2\right)\,
 =3x^2\cdot 1 + 5x\cdot 2y = 3x^2+10xy
\end{align*}

## Sympy example

In the previous slide we had:

$$\frac{\partial}{\partial x}\,\left(3x^2y+5xy^2\right)\, = 6xy+5y^2$$

Lets redo this in Sympy:

```python
x, y = sp.symbols('x y')
sp.diff(3*x**2*y + 5*x*y**2,x)
```

## Higher-Order Partial Derivatives:

Given $z = f(x,y)$ there are now four distinct possibilities for the
second-order partial derivatives.

(a)  With respect to $x$ twice:

$$\frac{\partial}{\partial x}\left(\frac{\partial z}{\partial x}\right)
    =\frac{\partial^2z}{\partial x^2}
    =z_{xx}$$

(b)  With respect to $y$ twice:

$$\frac{\partial}{\partial y}\left(\frac{\partial z}{\partial y}\right)
    =\frac{\partial^2z}{\partial y^2}
    =z_{yy}$$

(c)  First with respect to $x$, then with respect to $y$:

$$\frac{\partial}{\partial y}\left(\frac{\partial z}{\partial x}\right)
    =\frac{\partial^2z}{\partial y\partial x}
    =z_{xy}$$

(d)  First with respect to $y$, then with respect to $x$:

$$\frac{\partial}{\partial x}\left(\frac{\partial z}{\partial y}\right)
    =\frac{\partial^2z}{\partial x\partial y}
    =z_{yx}$$

## Example

(LaPlace's Equation for Equilibrium Temperature Distribution on a Copper Plate.)

Let $T(x,y)$ give the temperature at the point $(x,y)$.

According to a result of the French mathematician Pierre LaPlace (1749 - 1827), at every point $(x,y)$ the second-order partials of $T$ must satisfy the equation

$$T_{xx} + T_{yy} = 0$$

The function $T(x,y)=y^2-x^2$ satisfies LaPlace's equation:

First with respect to $x$:

$$T_x(x,y)=0-2x=-2x\qquad\mbox{so}\qquad T_{xx}(x,y)=-2$$

Then with respect to $y$:

$$T_y(x,y)=2y-0=2y\qquad\mbox{so}\qquad T_{yy}(x,y)=2$$

Finally:

$$T_{xx}(x,y)+T_{yy}(x,y) = 2 + (-2) = 0$$

which proves the result.

The function $z=x^2y - xy^2$ does *not* satisfy LaPlace's equation (and so
cannot be a model for thermal equilibrium).

First note that

$$z_x = 2xy - y^2$$

$$z_{xx}=2y$$

and

$$z_y = x^2 - 2xy$$

$$z_{yy} =-2x$$

Therefore:

$$z_{xx}+z_{yy}=2y-2x\ne 0$$

We can verify this in Sympy like so:

```python
T1 = y**2 - x**2
sp.diff(T1, x, x) + sp.diff(T1, y, y)
```

and for the second function:

```python
T2 = x**2*y - x*y**2
sp.diff(T2, x, x) + sp.diff(T2, y, y)
```

## A Note on the Mixed Partials $f_{xy}$ and $f_{yx}$:

If all of the partials of $f(x,y)$ exist, then $f_{xy}=f_{yx}$ for all $(x,y)$.

### Example:

Let $z = x^2y^3+3x^2-2y^4$.  Then $z_x=2xy^3+6x$ and $z_y = 3x^2y^2-8y^3$.

Taking the partial of $z_x$ with respect to $y$ we get

$$z_{xy}=\frac{\partial}{\partial y}\left(2xy^3+6x\right)=6xy^2$$

Taking the partial of $z_y$ with respect to $x$ we get the same thing:

$$z_{yx}=\frac{\partial}{\partial x}\left(3x^2y^2-8y^3\right)=6xy^2$$

So the operators ${\partial \over \partial x}$ and ${\partial \over \partial y}$ are **commutative**:

$${\rm~i.e.~~~~}~{\partial\over \partial x}\biggr({\partial z\over \partial y}\biggl)~~~~
={\partial\over \partial y}\biggr({\partial z\over \partial
x}\biggl)$$


| | Pierre Proulx, ing, professeur|
|:---|:---|
|Département de génie chimique et de génie biotechnologique |** GCH200-Phénomènes d'échanges I **|

### Section 2.2, écoulement d'un film de fluide Newtonien sur un plan incliné.

> Dans cette section il est important de revoir les concepts de flux de quantité de mouvement vus au chapitre 1.  Voici un résumé des points qui seront utilisés dans la section 2.2:

http://pierreproulx.espaceweb.usherbrooke.ca/images/GCH200_Ch1_resume.pdf

### Film d'un fluide newtonien, section 2.2 Transport Phenomena

>Le développement fait dans Transport Phenomena sera répété ici en développant les solutions avec le calculateur formel sympy et en traçant la solution avec sympy.plot.

>>Un fluide Newtonien s'écoule sous l'effet de la gravité sur un plan incliné:

>>Le bilan fait sur le film de fluide est schématisé ainsi:

>Plusieurs des termes montrés sur la figure ci-haut sont nuls, :

$
\begin{equation*}
  \boxed{ v_x=0 \quad  v_y=0 \quad \tau_{zz}=0 \quad \tau_{yz}=0 }
  \end{equation*}
$

>et de plus, on a que:

$
\begin{equation*}
  \boxed{ \rho v_z v_z \quad  et \quad p }
  \end{equation*}
$

>ne varient pas en fonction de z, ce qui fera que

$
\begin{equation*}
  \boxed{\phi_{zz}=constante }
  \end{equation*}
$

>Dans le traitement qui sera fait dans sympy on utilisera $\phi_{zz} = C $.

```python
#
# Pierre Proulx
#
# Préparation de l'affichage et des outils de calcul symbolique
#
import sympy as sp               # pour utiliser sympy, on lui donne l'alias sp
from IPython.display import *   # pour utiliser display qui formate les équations
sp.init_printing(use_latex=True) # pour utiliser LaTeX, formattage de grande qualité
```

### On pourrait aussi utiliser
> sp.init_printing(use_latex=False)
### Le résultat sera le même, mais le formattage des équations sera beaucoup moins élégant,

```python
#
# définition des variables symboliques
#
x,delta_x,L,W,rho,g,beta,mu,delta=sp.symbols('x,delta_x,L,W,rho,g,beta,mu,delta')
tau_xz=sp.symbols('tau_xz')
phi_xz=sp.symbols('phi_xz')
phi_zz=sp.symbols('phi_zz')
C1,C2=sp.symbols('C1,C2')
```

#### On pose le bilan:

```python
# Bilan des forces
dV=W*L*delta_x
dAx=L*W
dAz=W*delta_x
bilan=dAx*(phi_xz(x)-phi_xz(x+delta_x))+dAz*(phi_zz(0)-phi_zz(L))+dV*rho*g*sp.cos(beta)
bilan=bilan/(L*W*delta_x)
# mais phi_zz est le même en z=0 et en z=L, donc le terme de phi_zz(0) - phi_zz(L) =0
bilan=bilan.subs((phi_zz(0)-phi_zz(L)),0)
display(bilan)
```

> *Dans le bilan obtenu ci-haut on prend la limite quand $\delta x \rightarrow 0 $ On le fait avec **sympy** mais notez bien  que ce sont des manipulations de calcul algébrique qu'on ferait facilement à la main. Suivez bien les développements en faisant les mêmes sur une feuille de papier à côté. Vous obtiendrez le même résultat, même si votre notation peut être un peu différente et si vos manipulations ne sont pas exactement les mêmes. L'utilité du calculateur symbolique deviendra de plus en plus évidente au fur et à mesure que les problèmes deviendront plus complexes. *

```python
eq0=sp.Limit(bilan,delta_x,0)    # ces deux fonctions sont différentes, la première ne fait que poser la limite
eq1=sp.limit(bilan,delta_x,0)    # alors que la deuxième fait la limite au point demandé
display(eq0,eq1)    # on n'aurait pas besoin de la première ligne, c'est seulement pour montrer le développement.
```

```python
eq1=sp.limit(bilan,delta_x,0).doit()    # Ensuite, en faisant (doit()), on effectue vraiment la limite au point demandé
display(eq1)
```

```python
eq1=eq1.subs(phi_xz(x),tau_xz(x))    # remplacer phi par tau car la convection est nulle
display(eq1)
```

On insère ensuite la loi de Newton pour remplacer $\tau_{xz}$

```python
#
# Loi de Newton
vz=sp.Function('v_z')(x)
newton=-mu*sp.Derivative(vz,x)
#
# On la substitue dans le bilan
eq2=sp.Eq(eq1.subs(tau_xz(x),newton))
display(eq2)
```

```python
# La solution est faite avec la fonction dsolve de sympy
eq3=sp.dsolve(eq2,vz)
display(eq3)
```

>> Conditions aux limites,
>> en x = $\delta$

>>> ${v_{z}}{\left (\delta \right )} = C_{1} + C_{2} (\delta) - \frac{g \rho (\delta)^{2}}{2 \mu} \cos{\left (\beta \right )}$  $=0$

>> et en x=0

>>>$\frac  {{dv_{z}}}{dx} =  C_{2} - 2 \frac{g \rho (0)}{2 \mu} \cos{\left (\beta \right )}$  $=0$

> exercice facile à faire à la main, afin de nous préparer à des problèmes plus complexes à venir, voyons comment le résoudre avec **sympy**

```python
#
# Le rhs est la partie de droite de l'équation, c'est la partie qui nous intéresse
eq4=eq3.rhs
display('eq3 et eq4 (qui est eq3.rhs)',eq3,eq4)
#
# Pose et solution des 2 équations de conditions aux limites pour C1 et C2
#   la forme générale:
#   sp.solve([equation1, equation2, ...],('variable1, variable2,...'))
#
condition_1=sp.Eq(eq4.diff(x).subs(x,0    ),0) # L'équation 4 dérivée, quand x=0, est égale à 0
condition_2=sp.Eq(eq4.subs(x,delta)        ,0) # l'équation 4 quand x=delta est égale à 0
#
display('Les deux conditions aux limites',condition_1,condition_2)
constantes=sp.solve([condition_1,condition_2],('C1,C2') )                           # pour trouver C1 et C2
display('les constantes C1 et C2',constantes)
vz=eq4.subs(constantes)
display('Le profil est donc, en substituant C1 et C2 dans eq4',vz)
vzp=eq3.subs(constantes)
display('ou dans eq3',vzp)
```

```python
# Et le profil est obtenu.
display(vz.simplify())
#
# Qu'on peut maintenant tracer, en donnant des valeurs numériques bien choisies aux paramètres
#
dico={'beta':sp.pi/4,'delta':0.001,'g':9.81,'rho':1000,'mu':0.001}
vzmax=vz.subs(x,0).subs(dico).evalf(4)

import matplotlib.pyplot as plt
%matplotlib inline
plt.rcParams['figure.figsize'] = 6,4
sp.plot(vz.subs(dico)/vzmax,(x,0,0.001),title='Profil de vitesse parabolique',
       ylabel='V/Vmax',xlabel='r');
```

### À partir du profil de vitesses on peut calculer:

* La force que le fluide exerce sur la plaque:

$
\begin{equation*}
  \boxed{ F = -\mu \bigg [\frac {dv_z}{dx}\bigg ]_{x=\delta} WL}
  \end{equation*}
$
* Le débit volumétrique

$
\begin{equation*}
  \boxed{ Q =\int_0 ^{W} \int_0 ^{\delta} v_z dx dy }
  \end{equation*}
$

* La vitesse moyenne

$
\begin{equation*}
  \boxed{ v_{z_{moyen}} = \frac {\int_0 ^{W} \int_0 ^{\delta} v_z dx dy }
                        {\int_0 ^{W} \int_0 ^{\delta} dx dy     }}
  \end{equation*}
$

### Utilisons sympy

```python
force=-mu*vz.diff(x).subs(x,delta)*L*W
display(force)
y=sp.symbols('y')
display(sp.integrate(vz,(x,0,delta),(y,0,W)))
```

>>> Quel est le poids du film de liquide?

```python
debit=sp.integrate(vz,(x,0,delta))*W
display(debit)
```

```python
vzmoyen=debit/(delta*W)
display(vzmoyen)
```

```python
vzmax=vz.subs(x,0)
display(vzmax)
```

```python
display(vzmoyen/vzmax)
```