# Exercises

## Exercise 5: Compute the deflection of a cable with 2 P1 elements

Solve the problem for $u$ in [Exercise 2: Compute the deflection of a cable with sine functions](#fem:deq:exer:tension:cable)
using two P1 linear elements. Incorporate the condition $u(0)=0$
by two methods: 1) excluding the unknown at $x=0$, 2) keeping the unknown
at $x=0$, but modifying the linear system.

**Solution.**
From [Exercise 2: Compute the deflection of a cable with sine functions](#fem:deq:exer:tension:cable),
the Galerkin method, after integration by parts, reads

$$
(u',v')=-(1,v)\quad\forall v\in V{\thinspace .}
$$

We have two elements,
$\Omega^{(0)}=[0,\frac{1}{2}]$ and $\Omega^{(1)}=[\frac{1}{2},1]$.

**Method 1: Excluding the unknown at $x=0$.**
Since $u(0)=0$, we exclude the value at $x=0$ as degree of freedom in
the linear system. (There is no need for any boundary function.)
The expansion reads $u=c_0{\varphi}_1(x) + c_1{\varphi}_2(x)$.
The element matrix has then only one entry in the first element,

$$
\tilde A^{(0)} = \frac{1}{h}(1){\thinspace .}
$$

From element 1 we get the usual element matrix

$$
\tilde A^{(1)} = \frac{1}{h}
\left(\begin{array}{rr}
1 & -1\\
-1 & 1
\end{array}\right){\thinspace .}
$$

The element vector in element 0 becomes

$$
\tilde b^{(0)} = \frac{h}{2}(-1),
$$

while the second element gives a contribution

$$
\tilde b^{(1)} = \frac{h}{2}
\left(\begin{array}{c}
-1 \\
-1
\end{array}\right){\thinspace .}
$$

Assembling the contributions gives

$$
\frac{1}{h}
\left(\begin{array}{cc}
2 & -1\\
-1 & 1
\end{array}\right)
\left(\begin{array}{c}
c_0 \\
c_1
\end{array}\right)
=
- \frac{h}{2}
\left(\begin{array}{c}
2 \\
1
\end{array}\right){\thinspace .}
$$

Note that $h=\frac{1}{2}$.
Solving this system yields

$$
c_0 = -\frac{3}{8},\quad c_1=-\frac{1}{2}\quad\Rightarrow\quad u=-\frac{3}{8}{\varphi}_1(x)-\frac{1}{2}{\psi}_2(x){\thinspace .}
$$

Evaluating the exact solution for $x=\frac{1}{2}$ and $x=1$, we get $3/8$
and $1/2$, respectively, a result which shows
that the numerical solution with P1 is exact at the three node points.
The difference between the numerical and exact solution is that the
numerical solution varies linearly over the two elements while the
exact solution is quadratic.

**Method 2: Modifying the linear system.**
Now we let $c_i$ correspond to the value at node $x_{i}$, i.e.,
all known Dirichlet values become part of the linear system.
The expansion is now simply $u=\sum_{i=0}^2c_i{\varphi}_i(x)$, with
three unknowns $c_0$, $c_1$, and $c_2$.
Now the element matrix in the first and second element are equal.
The same is true for the element vectors.
Assembling yields

$$
\frac{1}{h}
\left(\begin{array}{ccc}
1 & -1 & 0\\
-1 & 2 & -1\\
0 & -1 & 1
\end{array}\right)
\left(\begin{array}{c}
c_0 \\
c_1\\
c_2
\end{array}\right)
=
- \frac{h}{2}
\left(\begin{array}{c}
1\\
2\\
1
\end{array}\right){\thinspace .}
$$

The next step is to modify the linear system to implement the
Dirichlet condition $c_0=0$. We first multiply by $h=\frac{1}{2}$ and replace the
first equation by $c_0=0$:

$$
\left(\begin{array}{ccc}
1 & 0 & 0\\
-1 & 2 & -1\\
0 & -1 & 1
\end{array}\right)
\left(\begin{array}{c}
c_0 \\
c_1\\
c_2
\end{array}\right)
=
- \left(\begin{array}{c}
0\\
\frac{1}{4}\\
\frac{1}{8}
\end{array}\right){\thinspace .}
$$

We see that the remaining $2\times 2$ system is identical to the one
previously solved, and the solution is the same.

$$
u = 0{\varphi}_0(x) - \frac{3}{8}{\varphi}_1(x) - \frac{1}{2}{\psi}_2(x){\thinspace .}
$$

Filename: `cable_2P1`.

## Exercise 6: Compute the deflection of a cable with 1 P2 element

Solve the problem for $u$ in [Exercise 2: Compute the deflection of a cable with sine functions](#fem:deq:exer:tension:cable)
using one P2 element with quadratic basis functions.

**Solution.**
The P2 basis functions on a reference element $[-1,1]$ are

$$
\begin{align*}
{\tilde{\varphi}}_0(X) &= \frac{1}{2} (X-1)X
\\
{\tilde{\varphi}}_1(X) &= 1 - X^2
\\
{\tilde{\varphi}}_2(X) &= \frac{1}{2} (X+1)X
\end{align*}
$$

The element matrix and vector are easily calculated by some lines with
`sympy`:

```python
import sympy as sym
X, h = sym.symbols('X h')
half = sym.Rational(1, 2)
psi = [half*(X-1)*X, 1-X**2, half*(X+1)*X]
dpsi_dX = [sym.diff(psi[r], X) for r in range(len(psi))]

# Element matrix
# (2/h)*dpsi_dX[r]*(2/h)*dpsi_dX[s]*h/2
import numpy as np
d = 2
# Use a numpy matrix with general objects to hold A
A = np.empty((d+1, d+1), dtype=object)
for r in range(d+1):
    for s in range(d+1):
        integrand = dpsi_dX[r]*dpsi_dX[s]*2/h
        A[r,s] = sym.integrate(integrand, (X, -1, 1))
print(A)

# Element vector
# f*psi[r]*h/2, f=1
d = 2
b = np.empty(d+1, dtype=object)
for r in range(d+1):
    integrand = -psi[r]*h/2
    b[r] = sym.integrate(integrand, (X, -1, 1))
print(b)
```

The formatted element matrix and vector output becomes

        [[7/(3*h) -8/(3*h) 1/(3*h)]
         [-8/(3*h) 16/(3*h) -8/(3*h)]
         [1/(3*h) -8/(3*h) 7/(3*h)]]
        [-h/6 -2*h/3 -h/6]

or in mathematical notation:

$$
\tilde A^{(e)}=\frac{1}{3h}
\left(\begin{array}{ccc}
7 & -8 & 1\\
-8 & 16 & -8\\
1 & -8 & 7
\end{array}\right),\quad
\tilde b^{(e)} = - \frac{h}{6}
\left(\begin{array}{c}
1\\
4\\
1
\end{array}\right){\thinspace .}
$$

**Method 1: Excluding the unknown at $x=0$.**
The expansion is $u=c_0{\varphi}_1(x) + c_1{\varphi}_2(x)$. The element matrix
corresponding to the first element excludes contributions associated with
the unknown at the left node, i.e., we exclude row and column 0. In the
element vector, we exclude the first entry.

$$
\tilde A^{(0)}=\frac{1}{3h}
\left(\begin{array}{cc}
16 & -8\\
-8 & 7
\end{array}\right),\quad
\tilde b^{(e)} = - \frac{h}{6}
\left(\begin{array}{c}
4\\
1
\end{array}\right){\thinspace .}
$$

Now, $h=1$.
The solution of the linear system

$$
\frac{1}{3h}
\left(\begin{array}{cc}
16 & -8\\
-8 & 7
\end{array}\right)
\left(\begin{array}{c}
c_1\\
c_2
\end{array}\right)
=
- \frac{h}{6}
\left(\begin{array}{c}
4\\
1
\end{array}\right)
$$

is $c_1=3/8$ and $c_2=1/2$. As for P1 elements in [Exercise 5: Compute the deflection of a cable with 2 P1 elements](#fem:deq:exer:cable:2P1), the values at the nodes are exact, but
this time the variation between the nodes is quadratic, i.e., exact.
One P2 element produces the complete, exact solution.

**Method 2: Modifying the linear system.**
This time the expansion reads $u=\sum_{i=0}^2 c_i{\varphi}_i(x)$ with
three unknowns $c_0$, $c_1$, and $c_2$. The linear system consists
of the complete $3\times 3$ element matrix and the corresponding
element vector:

$$
\frac{1}{3h}
\left(\begin{array}{ccc}
7 & -8 & 1\\
-8 & 16 & -8\\
1 & -8 & 7
\end{array}\right)
\left(\begin{array}{c}
c_0\\
c_1\\
c_2
\end{array}\right)
=
- \frac{h}{6}
\left(\begin{array}{c}
1\\
4\\
1
\end{array}\right){\thinspace .}
$$

The boundary condition is incorporated by replacing the first equation
by $c_0=0$, but prior to taking that action, we multiply by $3h$ and
insert $h=1$.

$$
\left(\begin{array}{ccc}
1 & 0 & 0   \\
-8 & 16 & -8\\
1 & -8 & 7
\end{array}\right)
\left(\begin{array}{c}
c_0\\
c_1\\
c_2
\end{array}\right)
=
\left(\begin{array}{c}
0\\
- 2\\
- \frac{1}{2}
\end{array}\right){\thinspace .}
$$

Realizing that $c_0=0$, which means we can remove the first column of
the system, shows that the equations are the same as above and hence
that the solution is identical.

Filename: `cable_1P2`.

## Exercise 7: Compute the deflection of a cable with a step load

We consider the deflection of a tension cable as described in
[Exercise 2: Compute the deflection of a cable with sine functions](#fem:deq:exer:tension:cable): $w''=\ell$, $w(0)=w(L)=0$.
Now the load is discontinuous:

$$
\ell (x) =\left\lbrace\begin{array}{ll}
\ell_1, & x < L/2,\\
\ell_2, & x \geq L/2
\end{array}\right.\quad x\in [0,L]
{\thinspace .}
$$

This load is not symmetric
with respect to the midpoint $x=L/2$ so the solution loses its symmetry.
Scaling the problem by introducing

$$
\bar x = \frac{x}{L/2},\quad u = \frac{w}{w_c},\quad\bar\ell = \frac{\ell - \ell_1}{\ell_2 - \ell_1}{\thinspace .}
$$

This leads to a scaled problem on $[0,2]$ (we rename $\bar x$ as $x$
for convenience):

$$
u'' = \bar\ell(x) = \left\lbrace\begin{array}{ll}
1, & x < 1,\\
0, & x \geq 1
\end{array}\right.
\quad x\in (0,1),\quad u(0)=0,\ u(2)=0
{\thinspace .}
$$

**a)**
Find the analytical solution of the problem.

**Hint.**
Integrate the equation separately for $x < 1$ and $x>1$. Use
the conditions that $u$ and $u'$ must be continuous at $x=1$.

**Solution.**
For $x < 1$ we get $u_1(x) = C_1x + C_2$, and the boundary condition
$u_1(0)=0$ implies $C_2=0$. For $x>1$ we get $u_2(x)=\frac{1}{2} x^2 + C_3x + C_4$.
Continuity of $u'(1)$ leads to

$$
C_1 = 1 + C_3,
$$

and continuity of $u(1)$ means

$$
C_1 = \frac{1}{2} + C_3 + C_4,
$$

while the condition $u_2(2)=0$ gives the third equation we need:

$$
2 + 2C_3 + C_4 = 0{\thinspace .}
$$

We use `sympy` to solve them:

```python
from sympy import symbols, Rational, solve
C1, C3, C4 = symbols('C1 C3 C4')
solve([C1 - 1 - C3,
```

Then

$$
u(x) = \left\lbrace\begin{array}{ll}
-\frac{1}{4}x, & x\leq 1,\\
\frac{1}{2} x^2 - \frac{5}{4}x + \frac{1}{2}, & x\geq 1
\end{array}\right.
$$

**b)**
Use ${\psi}_i = \sin((i+1)\frac{\pi x}{2})$,
$i=0,\ldots,N$ and the Galerkin method to find an approximate
solution $u=\sum_j c_j{\psi}_j$.
Plot how fast the coefficients $c_j$ tend to zero (on a log scale).

**Solution.**
The Galerkin formulation of the problem becomes

$$
(u',v') = -(\bar\ell, v) = \left\lbrace\begin{array}{ll}
0, & x\leq 1,\\
-(1,v), & x\geq 1
\end{array}\right.\quad\forall v\in V{\thinspace .}
$$

A requirement is that $v(0)=v(2)=0$ because of the boundary conditions
on $u$. The chosen basis functions
fulfill this requirement for any integer $i$. Inserting
$u=\sum_{j=0}^N c_j{\psi}_j$ and $v={\psi}_i$, $i=0,\ldots,N$,
gives as usual the linear system $\sum_j A_{i,j}c_j = b_i$, $i=0,\ldots,N$,
where

$$
A_{i,j} = (i+1)(j+1)\frac{\pi^2}{4}\int_0^2 \cos((i+1)\frac{\pi x}{2})
\cos((j+1)\frac{\pi x}{2})dx{\thinspace .}
$$

The cosine functions are orthogonal on $[0,2]$ so $A_{i,j}=0$ for
$i\neq j$, while $A_{i,i}$ is computed (e.g., by `sympy`) as in
[Exercise 2: Compute the deflection of a cable with sine functions](#fem:deq:exer:tension:cable), part e. The result is

$$
A_{i,i} = (i+1)^2\frac{\pi^2}{4}{\thinspace .}
$$

The right-hand side is

$$
b_i =
-\int_1^2 \sin((i+1)\frac{\pi x}{2})dx
= \frac{2}{\pi (i+1)}(\cos((i+1)\pi) - \cos((i+1)\pi/2)){\thinspace .}
$$

(Trying to do the integral in `sympy` gives a complicated expression that
needs discussion - it
is easier to do all calculations by hand.)
We have that $\cos((i+1)\pi = -1$ for $i$ even and
$\cos((i+1)\pi = 1$ for $i$ odd, while $\cos((i+1)\pi/2)$ is
discussed in
[Exercise 2: Compute the deflection of a cable with sine functions](#fem:deq:exer:tension:cable), part d. The values
of $\cos((i+1)\pi) - \cos((i+1)\pi/2)$ can be summarized in the following
table:

$i\hbox{ mod } 4 = 0$	 $(i-1)\hbox{ mod } 4 = 0$	 $(i-2)\hbox{ mod } 4 = 0$	 $(i-3)\hbox{ mod } 4 = 0$

   $-1 -0$                  	    $1 - (-1)$                   	    $-1 - 0$                     	    $1-1$

The following function computes the approximate solution:

```python
def sine_solution(x, N):
    from numpy import pi, sin
    s = 0
    u = [] # u[i] is the solution for N=i
    for i in range(N+1):
        if i % 4 == 0:
            cos_min_cos = -1
        elif (i-1) % 4 == 0:
            cos_min_cos = 2
        elif (i-2) % 4 == 0:
            cos_min_cos = -1
        elif (i-1) % 4 == 0:
            cos_min_cos = 0

        b_i = 2/(pi*(i+1))*cos_min_cos
        A_ii = (i+1)**2*pi**2/4
        c_i = b_i/A_ii
        s += c_i*sin((i+1)*x*pi/2)
        u.append(s.copy())
    return u
```

The exact solution is a function defined in a piecewise way. Below we make an implementation that works both for array and scalar arguments:

```python
def exact_solution(x):
    if isinstance(x, np.ndarray):
        return np.where(x < 1, -1./4*x, 0.5*x**2 - 5./4*x + 0.5)
    else:
        return -1./4*x if x < 1 else 0.5*x**2 - 5./4*x + 0.5
```

Now we can make a plot of the exact solution and approximate solutions for
various $N$:

```python
def plot_sine_solution():
    x = np.linspace(0, 2, 101)
    u = sine_solution(x, N=20)
    plt.figure()
    x = np.linspace(0, 2, 101)
    plt.plot(x, exact_solution(x), '--')
    N_values = 0, 1, 5
    for N in 0, 1, 5, 10:
        plt.plot(x, u[N])
    plt.legend(['exact'] + ['N=%d' % N for N in N_values])
    plt.savefig('tmp2.png');  plt.savefig('tmp2.pdf')
```

**c)**
Solve the problem with P1 finite elements.
Plot the solution for $N_e=2,4,8$ elements.

**Solution.**
The element matrices and vectors are as for the well-known model
problem $u''=1$, except that the element vectors vanish for all
elements in $[0,1]$. The following function defines a uniform mesh
of P1 elements and runs a finite element algorithm where we use
ready-made/known formulas for the element matrix and vector:

```python
def P1_solution():
    plt.figure()
    from fe1D import mesh_uniform, u_glob
    N_e_values = [2, 4, 8]
    d = 1
    legends = []
    for N_e in N_e_values:
        vertices, cells, dof_map = mesh_uniform(
            N_e=N_e, d=d, Omega=[0,2], symbolic=False)
        h = vertices[1] - vertices[0]
        Ae = 1./h*np.array(
            [[1, -1],
             [-1, 1]])
        N = N_e + 1
        A = np.zeros((N, N))
        b = np.zeros(N)
        for e in range(N_e):
            if vertices[e] >= 1:
                be = -h/2.*np.array(
                    [1, 1])
            else:
                be = h/2.*np.array(
                    [0, 0])
            for r in range(d+1):
                for s in range(d+1):
                    A[dof_map[e][r], dof_map[e][s]] += Ae[r,s]
                b[dof_map[e][r]] += be[r]
        # Enforce boundary conditions
        A[0,:] = 0; A[0,0] = 1; b[0] = 0
        A[-1,:] = 0; A[-1,-1] = 1; b[-1] = 0
        c = np.linalg.solve(A, b)

        # Plot solution
        print(('c:', c))
        print(('vertices:', vertices))
        print(('cells:', cells))
        print(('len(cells):', len(cells)))
        print(('dof_map:', dof_map))
        xc, u, nodes = u_glob(c, vertices, cells, dof_map)
        plt.plot(xc, u)
        legends.append('$N_e=%d$' % N_e)
    plt.plot(xc, exact_solution(xc), '--')
    legends.append('exact')
    plt.legend(legends, loc='lower left')
    plt.savefig('tmp3.png'); plt.savefig('tmp3.pdf')
```

Filename: `cable_discont_load`.

## Exercise 8: Compute with a non-uniform mesh

**a)**
Derive the linear system for the problem $-u''=2$ on $[0,1]$, with
$u(0)=0$ and $u(1)=1$, using P1 elements and a *non-uniform* mesh. The
vertices have coordinates $x_{0}=0 < x_{1} <\cdots <
x_{N_n-1}=1$, and the length of cell number $e$ is $h_e = x_{e+1}
-x_{e}$.

**Solution.**
The element matrix and vector for this problem is given by
([71](#fem:deq:1D:ex1:Ab:elm)). The change in this exercise is that
$h$ is not a constant element length, but varying with the element
number $e$. We therefore write

$$
\tilde A^{(e)} =\frac{1}{h_e}\left(\begin{array}{rr}
1 & -1\\
-1 & 1
\end{array}\right),\quad
\tilde b^{(e)} = h_e\left(\begin{array}{c}
1\\
1
\end{array}\right){\thinspace .}
$$

Assembling such element matrices yields

$$
\left(
\begin{array}{ccccccccc}
h_0^{-1} & -h_0^{-1} & 0 &\cdots & \cdots & \cdots & \cdots & \cdots & 0 \\
-h_0^{-1} & h_0^{-1}+h_1^{-1} & -h_1^{-1} & \ddots &   & &  & &  \vdots \\
0 & -h_1^{-1} & h_1^{-1} + h_2^{-1} & -h_2^{-1} &
\ddots & &  &  & \vdots \\
\vdots & \ddots &  & \ddots & \ddots & 0 &  & & \vdots \\
\vdots &  & \ddots & \ddots & \ddots & \ddots & \ddots & & \vdots \\
\vdots & &  & 0 & -h_{i-1}^{-1} & h_{i-1}^{-1} + h_i^{-1} & -h_i^{-1} & \ddots & \vdots \\
\vdots & & &  & \ddots & \ddots & \ddots &\ddots  & 0 \\
\vdots & & & &  &\ddots  & \ddots &\ddots  & -h_{N_e}^{-1} \\
0 &\cdots & \cdots &\cdots & \cdots & \cdots  & 0 & -h_{N_e}^{-1} & h_{N_e}^{-1}
\end{array}
\right)
$$

The element vectors assemble to

$$
\left(
\begin{array}{c}
h_0 \\
h_0 + h_1\\
\vdots\\
\vdots \\
\vdots \\
h_{i-1} + h_i\\
\vdots \\
\vdots\\
h_{N_e}
\end{array}
\right)
$$

**b)**
It is of interest to compare the discrete equations for the finite
element method in a non-uniform mesh with the corresponding discrete
equations arising from a finite difference method. Go through the
derivation of the finite difference formula $u''(x_i) \approx [D_x D_x
u]_i$ and modify it to find a natural discretization of $u''(x_i)$ on
a non-uniform mesh. Compare the finite element and difference
discretizations

**Solution.**
Using the definition of the centered, 2nd-order finite difference approximation
to $u''$ we can set up

$$
[D_xD_x u]_i = [D_x(D_x u)]_i = \frac{\frac{u_{i+1}-u_i}{x_{i+1}-x_i} -
\frac{u_{i}-u_{i-1}}{x_{i}-x_{i-1}}}{x_{i+1/2} - x_{i-1/2}}{\thinspace .}
$$

Now,

$$
x_{i+1/2} - x_{i-1/2} = \frac{1}{2} (x_i - x_{i-1}) + \frac{1}{2}(x_{i+1}-x_i) = \frac{1}{2}(x_{i+1}-x_{i-1}){\thinspace .}
$$

We then get the difference equation

$$
u''(x_i)\approx \frac{2}{h_i + h_{i-1}}\left(
\frac{u_{i+1}-u_i}{h_{i}} - \frac{u_{i}-u_{i-1}}{h_{i-1}}\right) = 2{\thinspace .}
$$

The factor 2 on either side cancels.

Looking at the finite element equations in a), the equation for a general
row $i$ reads

$$
\frac{1}{h_{i-1}}c_{i-1} - (\frac{1}{h_{i-1}} + \frac{1}{h_{i}})c_i
+ \frac{1}{h_{i}}c_{i+1} = h_{i-1} + h_i{\thinspace .}
$$

Replacing $c_i$ by $u_i$ (assuming we keep unknowns at all nodes) and
rearranging gives

$$
-\frac{1}{h_{i-1}}(u_i - u_{i-1}) + \frac{1}{h_{i}}(u_{i+1}-u_i)
 = h_{i-1} + h_i{\thinspace .}
$$

Dividing by the right-hand side gives

$$
-\frac{1}{h_{i-1} + h_i}\left(\frac{1}{h_{i-1}}(u_i - u_{i-1}) - \frac{1}{h_{i}}(u_{i+1}-u_i)\right) = 1{\thinspace .}
$$

This is the same difference equation as we have in the finite difference
method.

Filename: `nonuniform_P1`.

## Problem 9: Solve a 1D finite element problem by hand

The following scaled 1D problem is a very simple, yet relevant, model
for convective transport in fluids:

$$
\begin{equation}
u' = \epsilon u'' ,\quad u(0)=0,\ u(1)=1,\ x\in [0,1]
{\thinspace .}
\label{_auto47} \tag{119}
\end{equation}
$$

**a)**
Find the analytical solution to this problem.
(Introduce $w=u'$, solve the first-order differential equation for $w(x)$,
and integrate once more.)

**b)**
Derive the variational form of this problem.

**c)**
Introduce a finite element mesh with uniform partitioning.
Use P1 elements and compute the element matrix and vector for
a general element.

**d)**
Incorporate the boundary conditions and
assemble the element contributions.

**e)**
Identify the resulting linear system as a finite difference discretization
of the differential equation using

$$
[D_{2x}u = \epsilon D_xD_x u]_i {\thinspace .}
$$

**f)**
Compute the numerical solution and plot it together with the exact solution
for a mesh with 20 elements and
$\epsilon=10, 1, 0.1, 0.01$.

Filename: `convdiff1D_P1`.

## Exercise 10: Investigate exact finite element solutions

Consider

$$
-u''(x)=x^m,\quad x\in (0,L),\quad u'(0)=C,\ u(L)=D,
$$

where $m\geq 0$ is an integer, and $L$, $C$, and $D$ are given numbers.
Utilize a mesh with two (non-uniform) elements: $\Omega^{(0)}=[0,3]$ and
$\Omega^{(0)}=[3,4]$.
Plot the exact solution and the finite element solution for
polynomial degree $d=1,2,3,4$ and $m=0, 1, 2, 3, 4$. Find values of $d$ and $m$
that make the finite element solution exact at the nodes in the mesh.

**Hint.**
Use the `mesh_uniform`, `finite_element1D`, and `u_glob2` functions
from the `fe1D.py` module.

**Solution.**
The `model2` function from the section [Simple model problems and their solutions](#fem:deq:1D:models:simple)
can find the exact solution by `model2(x**m, L, C, D)`.
We fix, for simplicity, the values of $L$, $C$, and $D$ as
$L=4$, $C=5$, and $D=2$.
After calculating a symbolic solution, we can convert the expression
to a Python function with `sympy.lambdify`.
For each $d$ value we then create a uniform mesh and displace the
vertex with number 1 to the value 3.
The various functions for specifying the element matrix and vector
entries are as given in the section [Utilizing a sparse matrix](#fem:deq:1D:code:fe_sparse),
since the model problem is the same. Our code then becomes

```python
from u_xx_f_sympy import model2, x
import sympy as sym
import numpy as np
from fe1D import finite_element1D, mesh_uniform, u_glob
import matplotlib.pyplot as plt

C = 5
D = 2
L = 4

m_values = [0, 1, 2, 3, 4]
d_values = [1, 2, 3, 4]
for m in m_values:
    u = model2(x**m, L, C, D)
    print(('\nm=%d, u: %s' % (m, u)))
    u_exact = sym.lambdify([x], u)

    for d in d_values:
        vertices, cells, dof_map = mesh_uniform(
            N_e=2, d=d, Omega=[0,L], symbolic=False)
        vertices[1] = 3  # displace vertex
        essbc = {}
        essbc[dof_map[-1][-1]] = D

        c, A, b, timing = finite_element1D(
            vertices, cells, dof_map,
            essbc,
            ilhs=lambda e, phi, r, s, X, x, h:
            phi[1][r](X, h)*phi[1][s](X, h),
            irhs=lambda e, phi, r, X, x, h:
            x**m*phi[0][r](X),
            blhs=lambda e, phi, r, s, X, x, h: 0,
            brhs=lambda e, phi, r, X, x, h:
            -C*phi[0][r](-1) if e == 0 else 0,
            intrule='GaussLegendre')

        # Visualize
        # (Recall that x is a symbol, use xc for coordinates)
        xc, u, nodes = u_glob(c, vertices, cells, dof_map)
        u_e = u_exact(xc)
        print(('Max diff at nodes, d=%d:' % d, \
              np.abs(u_exact(nodes) - c).max()))
        plt.figure()
        plt.plot(xc, u, 'b-', xc, u_e, 'r--')
        plt.legend(['finite elements, d=%d' %d, 'exact'],
                   loc='lower left')
        figname = 'tmp_%d_%d' % (m, d)
        plt.savefig(figname + '.png'); plt.savefig(figname + '.pdf')
```

First we look at the numerical solution at the nodes:

        m=0, u: -x**2/2 + 5*x - 10
        Max diff at nodes, d=1: 2.22044604925e-16
        Max diff at nodes, d=2: 3.5527136788e-15
        Max diff at nodes, d=3: 1.7763568394e-15
        Max diff at nodes, d=4: 2.46913600677e-13

        m=1, u: -x**3/6 + 5*x - 22/3
        Max diff at nodes, d=1: 8.881784197e-16
        Max diff at nodes, d=2: 1.7763568394e-15
        Max diff at nodes, d=3: 7.9936057773e-15
        Max diff at nodes, d=4: 3.01092484278e-13

        m=2, u: -x**4/12 + 5*x + 10/3
        Max diff at nodes, d=1: 3.10862446895e-15
        Max diff at nodes, d=2: 0.084375
        Max diff at nodes, d=3: 0.0333333333333
        Max diff at nodes, d=4: 5.20472553944e-13

        m=3, u: -x**5/20 + 5*x + 166/5
        Max diff at nodes, d=1: 1.35555555556
        Max diff at nodes, d=2: 0.3796875
        Max diff at nodes, d=3: 0.185714285714
        Max diff at nodes, d=4: 0.0254255022334

        m=4, u: -x**6/30 + 5*x + 1778/15
        Max diff at nodes, d=1: 4.8
        Max diff at nodes, d=2: 1.4428125
        Max diff at nodes, d=3: 0.719047619047
        Max diff at nodes, d=4: 0.16865583147

We observe that all elements are capable of computing the exact values
at the nodes for $m=0$ and $m=1$. With $m=0$, the solution is quadratic
in $x$, and P2, P3, and P4 will be exact. It is more of a surprise that
also the P1 elements are exact in this case.
A peculiar feature is that P1
elements are also exact at the nodes $m=2$, but not P2 and P3 elements
(the solution goes like $x^4$ so it is not surprising that P2 and P3
elements give a numerical error also at the nodes).
Clearly, P4 elements produce the exact solution for $m=4$ since $u$
is a polynomial of degree 4. For larger $m$ values we have
discrepancy between the numerical and exact values at the nodes.

% for m in [0, 1, 2, 3, 4]:

**Plots for m=${m}.**

% endfor

Filename: `u_xx_xm_P1to4`.

## Exercise 11: Compare finite elements and differences for a radially symmetric Poisson equation

We consider the Poisson problem in a disk with radius $R$ with
Dirichlet conditions at the boundary.
Given that the solution is radially symmetric and hence dependent only on
the radial coordinate ($r=\sqrt{x^2+y^2}$), we can reduce the problem
to a 1D Poisson equation

$$
\begin{equation}
-\frac{1}{r}\frac{d}{dr}\left( r\frac{du}{dr}\right) = f(r),\quad r\in (0,R),\
u'(0)=0,\ u(R)=U_R
{\thinspace .}
\label{fem:deq:exer:1D:Poisson:polar:eq} \tag{120}
\end{equation}
$$

**a)**
Derive a variational form of ([120](#fem:deq:exer:1D:Poisson:polar:eq))
by integrating over the whole disk, or posed equivalently: use
a weighting function $2\pi r v(r)$ and integrate $r$ from $0$ to $R$.

**b)**
Use a uniform mesh partition with P1 elements and show what the
resulting set of equations becomes. Integrate the matrix entries
exact by hand, but use a Trapezoidal rule to integrate the $f$ term.

**c)**
Explain that an intuitive
finite difference method applied to ([120](#fem:deq:exer:1D:Poisson:polar:eq))
gives

$$
\frac{1}{r_i}\frac{1}{h^2}\left( r_{i+\frac{1}{2}}(u_{i+1}-u_i) -
r_{i-\frac{1}{2}}(u_{i}-u_{i-1})\right) = f_i,\quad i=rh
{\thinspace .}
$$

For $i=0$ the factor $1/r_i$ seemingly becomes problematic. One must always
have $u'(0)=0$, because of the radial symmetry, which implies
$u_{-1}=u_1$, if we allow introduction of a fictitious value $u_{-1}$.
Using this $u_{-1}$ in the difference equation for $i=0$ gives

$$
\begin{align*}
&\frac{1}{r_0}\frac{1}{h^2}\left( r_{\frac{1}{2}}(u_{1}-u_0) -
r_{-\frac{1}{2}}(u_{0}-u_{1})\right) = \\
& \qquad
\frac{1}{r_0}\frac{1}{2h^2}\left( (r_0 + r_1)(u_{1}-u_0) -
(r_{-1} + r_0)(u_{0}-u_{1})\right) \approx
2(u_1-u_0),
\end{align*}
$$

if we use $r_{-1}+r_1\approx 2r_0$.

Set up the complete set of equations for the finite difference method
and compare to the finite element method in case a Trapezoidal rule
is used to integrate the $f$ term in the latter method.

Filename: `radial_Poisson1D_P1`.

## Exercise 12: Compute with variable coefficients and P1 elements by hand

Consider the problem

$$
\begin{equation}
-\frac{d}{dx}\left( {\alpha}(x)\frac{du}{dx}\right) + \gamma u = f(x),
\quad x\in\Omega=[0,L],\quad u(0)={\alpha},\ u'(L)=\beta{\thinspace .}
\label{fem:deq:1D:model4} \tag{121}
\end{equation}
$$

We choose ${\alpha}(x)=1+x^2$. Then

$$
\begin{equation} u(x) = {\alpha} + \beta(1+L^2)\tan^{-1}(x),
\label{_auto48} \tag{122}
\end{equation}
$$

is an exact solution if $f(x) = \gamma u$.

Derive a variational formulation and compute general expressions for the
element matrix and vector in an arbitrary element, using P1 elements
and a uniform partitioning of $[0,L]$. The right-hand side
integral is challenging and can be computed by a numerical integration
rule. The Trapezoidal rule ([fem:approx:fe:numint1:trapez](#fem:approx:fe:numint1:trapez))
gives particularly simple expressions.
Filename: `atan1D_P1`.

## Exercise 13: Solve a 2D Poisson equation using polynomials and sines

The classical problem of applying a torque to the ends of a rod
can be modeled by a Poisson equation defined in the cross section $\Omega$:

$$
-\nabla^2 u = 2,\quad (x,y)\in\Omega,
$$

with $u=0$ on $\partial\Omega$. Exactly the same problem arises for
the deflection of a membrane with shape $\Omega$ under a constant load.

For a circular cross section one can readily
find an analytical solution. For a rectangular cross section the analytical
approach ends up with a sine series. The idea in this exercise is to
use a single basis function to obtain an approximate answer.

We assume for simplicity that the cross section is the unit square:
$\Omega = [0,1]\times [0,1]$.

**a)**
We consider the basis
${\psi}_{p,q}(x,y) = \sin((p+1)\pi x)\sin (q\pi y)$, $p,q=0,\ldots,n$.
These basis functions fulfill the Dirichlet condition.
Use a Galerkin method and $n=0$.

**b)**
The basis function involving sine functions are orthogonal.
Use this property in the Galerkin method
to derive the coefficients $c_{p,q}$ in a
formula $u=\sum_p\sum_q c_{p,q}{\psi}_{p,q}(x,y)$.

**c)**
Another possible basis is
${\psi}_i(x,y) = (x(1-x)y(1-y))^{i+1}$, $i=0,\ldots,N$.
Use the Galerkin method to compute the solution for $N=0$.
Which choice of a single basis function is best,
$u\sim x(1-x)y(1-y)$ or $u\sim \sin(\pi x)\sin(\pi y)$?
In order to answer the question,
it is necessary to search the web or the literature for an accurate
estimate of the maximum $u$ value at $x=y=1/2$.

Filename: `torsion_sin_xy`.

## Exercise 14: Solve a 3D Laplace problem with FEniCS

Solve the problem in the section [Implementation in 2D and 3D via FEniCS](#fem:varform:fenics) in 3D.

**Hint.**
Use `BoxMesh` as starting point. Consult the FEniCS tutorial [[ftut1]](#ftut1)
if necessary.

Filename: `borehole_fenics3D`.

## Exercise 15: Solve a 1D Laplace problem with FEniCS

Solve the problem in the section [Implementation in 2D and 3D via FEniCS](#fem:varform:fenics) in 1D, using
the radial formulation $(ru')'=0$.

**Hint.**
Use `IntervalMesh` for generating the mesh and introduce a stretching if
desired. Consult the FEniCS tutorial [[ftut1]](#ftut1)
so you can extract the solution in an array
and make your own curve plot of it.

This problem can be solved without markers, see the section on
multiple Dirichlet conditions in the tutorial [[ftut1]](#ftut1) (it is even
easier to solve by saying that the solution at the boundary
obeys a linear function from $u_a$ to $u_b$ and use this as the only
Dirichlet condition).

Filename: `borehole_fenics1D`.


```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import control
from ipywidgets import interact

```

# Error de Estado Estacionario

En el bucle de control, el error se calcula como la diferencia entre la referencia y la variable controlada.

Los cambios en la señal de referencia provocan errores durante los transitorios y pueden generar un error estacionario.

Considerando un sistema de control con realimentación unitaria y función de transferencia de lazo abierto $G(s) = G_c(s)G_a(s)G_p(s)$:

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^N(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)}$$

Este sistema tiene:
- $c$ ceros definidos por $T_{c1}$, $T_{c2}$, $T_{c3}$, ..., $T_{cc}$
- $N$ polos en $0$
- $p$ polos definidos por $T_{p1}$, $T_{p2}$, $T_{p3}$, ..., $T_{pp}$

La función de transferencia de lazo cerrado desde la referencia hacia la variable controlada es:

$$\frac{C(s)}{R(s)} = \frac{G(s)}{1 + G_(s)}$$

La función de transferencia de lazo cerrado desde la referencia hacia el error es:

$$\frac{E(s)}{R(s)} = \frac{R(s)-C(s)}{R(s)} = 1- \frac{C(s)}{R(s)} = 1 - \frac{G(s)}{1 + G(s)} = \frac{1}{1+G(s)}$$

El **error de estado estacionario** puede calcularse como:

$$E_{ee} = \lim_{t \rightarrow \infty} e(t) = \lim_{s \rightarrow 0} sE(s) = \lim_{s \rightarrow 0}\frac{sR(s)}{1+G(s)}$$

## Error de estado estacionario ante una entrada escalón (posición)

Dada esta señal como referencia:

$$R(s) = \frac{1}{s}$$

y

$$E_{ee} = \lim_{s \rightarrow 0}\frac{s \frac{1}{s} }{1+G(s)} = \frac{1}{1+ \lim_{s \rightarrow 0} G(s) }$$

**Ejemplo**

Suponga:

$$G(s) = \frac{1}{5s + 1}$$

Entonces:

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{1}{5s + 1}}_{1} }$$

Si se introduce una ganancia $K_p$ (controlador proporcional):

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{K_p}{5s + 1}}_{K_p} }$$

```python
# Se define la función de transferencia
G = control.tf(1, [5,1])
G*2
```

$$\frac{2}{5 s + 1}$$

Análisis $E_{ee}$

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{K_p}{5s + 1}}_{K_p} } = \frac{1}{1+K_p}$$

```python
ts = np.linspace(0, 10, 2000)

Kp = [0.5,0.75,1,2,3,5,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(GLC, ts)
    e.append(1/(1+k))
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot([ts[0],ts[-1]], [1,1])
plt.legend(etiq)
plt.grid(True)
print(e)
```

¿Qué pasa si el sistema tiene una dinámica integradora?

$$G(s) = \frac{1}{s(5s + 1)}$$

```python
# Se define la función de transferencia
G = control.tf(1, [5,1,0])
G*2
```

$$\frac{2}{5 s^2 + s}$$

Análisis $E_{ee}$

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{1}{s(5s + 1)}}_{\infty} } = 0$$

```python
ts = np.linspace(0, 50, 2000)

Kp = [0.5,0.75,1,2,3,5,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(GLC, ts)
    e.append(1/(1+k))
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot([ts[0],ts[-1]], [1,1])
plt.legend(etiq)
plt.grid(True)
print(e)
```

Observe que la presencia de una dinámica integradora hace que el sistema en lazo cerrado tenga error de estado estable nulo. La cantidad de polos integradores determina el **tipo** del sistema:

### Sistemas tipo cero

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \frac{1}{1+ \lim_{s \rightarrow 0} G(s) } \\
& = \frac{1}{1+ \lim_{s \rightarrow 0} K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} } \\
& = \frac{1}{1+ K }
\end{align}

### Sistemas tipo uno

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \frac{1}{1+ \lim_{s \rightarrow 0} G(s) } \\
& = \frac{1}{1+ \lim_{s \rightarrow 0} K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} } \\
& = 0
\end{align}

### Sistemas tipo dos

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \frac{1}{1+ \lim_{s \rightarrow 0} G(s) } \\
& = \frac{1}{1+ \lim_{s \rightarrow 0} K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} } \\
& = 0
\end{align}

## Error de estado estacionario ante una entrada rampa (velocidad)
$$R(s) = \frac{1}{s^2} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0}\frac{sR(s)}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{s\frac{1}{s^2}}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{1}{s+sG(s)} \\
&= \lim_{s \rightarrow 0} \frac{1}{sG(s)}
\end{align}

### Sistemas tipo cero

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{sK\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \infty
\end{align}

**Ejemplo**

Suponga:

$$G(s) = \frac{1}{5s + 1}$$

Entonces:

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{1}{5s + 1}}_{1} }$$

Si se introduce una ganancia $K_p$ (controlador proporcional):

$$E_{ee} =\frac{1}{1+ \underbrace{\lim_{s \rightarrow 0} \frac{K_p}{5s + 1}}_{K_p} }$$

```python
# Se define la función de transferencia
G = control.tf(1, [5,1])
G
```

$$\frac{1}{5 s + 1}$$

```python
ts = np.linspace(0, 50, 2000)

Kp = [0.5,0.75,1,2,3,5,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(control.tf(1,[1,0])*GLC, ts)
    e.append(1/(1+k))
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot([ts[0],ts[-1]], [0,ts[-1]])
plt.legend(etiq)
plt.grid(True)
print(e)
```

### Sistemas tipo uno

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{p3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{sK\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \frac{1}{K}
\end{align}

```python
# Se define la función de transferencia
G = control.tf(1, [5,1,0])
G
```

$$\frac{1}{5 s^2 + s}$$

```python
ts = np.linspace(0, 50, 2000)

Kp = [0.5,1,5,10]
etiq = []
e = []
for k in Kp:

    GLC = control.feedback(k*G,1)
    _, yLC = control.step_response(control.tf(1,[1,0])*GLC, ts)
    e.append(1/(1+k))
    plt.plot(ts, yLC)
    etiq.append('k = '+str(k))
etiq.append('referencia')
plt.plot([ts[0],ts[-1]], [0,ts[-1]])
plt.legend(etiq)
plt.grid(True)
print(e)
```

### Sistemas tipo dos

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{sK\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \frac{s}{K} = 0
\end{align}

```python
## Desarrolle su propio ejemplo

```

## Error de estado estacionario ante una entrada parábola (aceleración)

$$R(s) = \frac{1}{s^3} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0}\frac{sR(s)}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{s\frac{1}{s^3}}{1+G(s)} \\
&=  \lim_{s \rightarrow 0}\frac{1}{s^2+s^2G(s)} \\
&= \lim_{s \rightarrow 0} \frac{1}{s^2G(s)}
\end{align}

### Sistemas tipo cero

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{s^2G(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{s^2K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \infty
\end{align}

```python
## Desarrolle su propio ejemplo

```

### Sistemas tipo uno

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{p3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{s^2G(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{s^2K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& =  \infty
\end{align}

```python
## Desarrolle su propio ejemplo

```

### Sistemas tipo dos

$$G(s) = K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{c3}s+1)...(T_{pp}s+1)} $$

\begin{align}
E_{ee} &= \lim_{s \rightarrow 0} \frac{1}{sG(s)} \\
& = \lim_{s \rightarrow 0} \frac{1}{s^2K\frac{(T_{c1}s+1)(T_{c2}s+1)(T_{c3}s+1)...(T_{cc}s+1)}{s^2(T_{p1}s+1)(T_{p2}s+1)(T_{p3}s+1)...(T_{pp}s+1)} }\\
& = \frac{1}{K}
\end{align}

```python
## Desarrolle su propio ejemplo

```

## Tabla

|error   | $$\frac{1}{s}$$|$$\frac{1}{s^2}$$|$$\frac{1}{s^3}$$|
|--------|----------------|-----------------|-----------------|
| tipo 0 |$$\frac{1}{1+K}$$ |$$\infty$$     | $$\infty$$      |
| tipo 1 |$$0$$           |$$ \frac{1}{K}$$ | $$\infty$$      |
| tipo 2 |$$0$$           |$$ 0 $$          | $$ \frac{1}{K}$$|

```python

```


## SurfinPy

#### Tutorial 1 - Generating a phase diagram as a function of chemical potential and surface energy at 0 K

In this tutorial we will learn how to generate a basic phase diagram from DFT energies. This example will consider a series of surfaces that contain differing amounts of surface oxygen and adsorbed water species.
The physical quantity that is used to define the stability of a surface with a given composition is its surface energy $\gamma$ (J $m^{-2}$). Going forward in this tutorial we will use the example of water adsorbing on to defective Ti$O_2$ surfaces.

\begin{align}
\gamma_{Surf} & = \frac{1}{2A} \Bigg( E_{TiO_2}^{slab} - \frac{nTi_{slab}}{nTi_{Bulk}} E_{TiO_2}^{Bulk} \Bigg) - \Gamma_O \mu_O - \Gamma_{H_2O} \mu_{H_2O} ,
\end{align}

where A is the surface area, $E_{TiO_2}^{slab}$ is the DFT energy of the slab, $nTi_{Slab}$ is the number of cations in the slab, $nTi_{Bulk}$ is the number of cations in the bulk unit cell, $E_{TiO_2}^{Bulk}$ is the DFT energy of the bulk unit cell and

\begin{align}
\Gamma_O & = \frac{1}{2A} \Bigg( nO_{Slab} - \frac{nO_{Bulk}}{nTi_{Bulk}}nTi_{Slab}  \Bigg) ,
\end{align}

\begin{align}
\Gamma_{H_2O} & = \frac{nH_2O}{2A} ,
\end{align}

where $nO_{Slab}$ is the number of anions in the slab, $nO_{Bulk}$ is the number of anions in the bulk and $nH_2O$ is the number of adsorbing water molecules. $\Gamma_O$ / $\Gamma_{H_2O}$ is the excess oxygen / water at the surface and $\mu_O$ / $\mu_{H_2O}$ is the oxygen / water chemcial potential. Clearly $\Gamma$ and $\mu$ will only matter when the surface is non stoichiometric.

So now lets work through an example

```python
import matplotlib.pyplot as plt
from surfinpy import mu_vs_mu
from surfinpy import data
```

The first thing to do is input the data that we have generated from our DFT simulations. The input data needs to be contained within a `surfinpy.data` object. First we have created a `surfinpy.data.ReferenceDataSet` object for the bulk data (reference data), where `cation` is the number of cations, `anion` is the number of anions, `energy` is the DFT energy and `funits` is the number of formula units.

```python
bulk = data.ReferenceDataSet(cation = 1, anion = 2, energy = -780.0, funits = 4)
```

Next we create the surface `surfinpy.data.DataSet` objects - one for each surface or "phase". `cation` is the number of cations, `x` is in this case the number of oxygen species (corresponding to the X axis of the phase diagram), `y` is the number of in this case water molecules (corresponding to the Y axis of our phase diagram), `area` is the surface area, `energy` is the DFT energy, `label` is the label for the surface (appears on the phase diagram) and finally `nSpecies` is the number of adsorbin species.

```python
pure = data.DataSet(cation = 24, x = 48, y = 0, area = 60.0, energy = -575.0, label = "0.00 $TiO_2$", nspecies = 1)
H2O = data.DataSet(cation = 24, x = 48, y = 2, area = 60.0, energy = -612.0, label = "0.16 $TiO_2$", nspecies = 1)
H2O_2 = data.DataSet(cation = 24, x = 48, y = 4, area = 60.0, energy = -640.0, label = "0.32 $TiO_2$", nspecies = 1)
H2O_3 = data.DataSet(cation = 24, x = 48, y = 8, area = 60.0, energy = -676.0, label = "0.64 $TiO_2$", nspecies = 1)
Vo = data.DataSet(cation = 24, x = 46, y = 0, area = 60.0, energy = -558.0, label = "0.00 $TiO_1.9$", nspecies = 1)
H2O_Vo_1 = data.DataSet(cation = 24, x = 46, y = 2, area = 60.0, energy = -594.0, label = "0.00 $TiO_1.9$", nspecies = 1)
H2O_Vo_2 = data.DataSet(cation = 24, x = 46, y = 4, area = 60.0, energy = -624.0, label = "0.16 $TiO_1.9$", nspecies = 1)
H2O_Vo_3 = data.DataSet(cation = 24, x = 46, y = 6, area = 60.0, energy = -640.0, label = "0.32 $TiO_1.9$", nspecies = 1)
H2O_Vo_4 = data.DataSet(cation = 24, x = 46, y = 8, area = 60.0, energy = -670.0, label = "0.64 $TiO_1.9$", nspecies = 1)
```

Next we need to create a list of our data. Don't worry about the order, surfinpy will sort that out for you.

```python
data = [pure, Vo, H2O, H2O_Vo_1, H2O_2, H2O_Vo_2, H2O_3, H2O_Vo_3,  H2O_Vo_4]
```

We now need to generate our X and Y axis, or more appropriately, our chemical potential values. Again these exist in a dictionary. 'Range' corresponds to the range of chemcial potential values to be considered and 'Label' is the axis label.
```
    deltaX = {'Range': Range of Chemical Potential,
              'Label': Species Label}

```

```python
deltaX = {'Range': [ -12, -6],  'Label': 'O'}
deltaY = {'Range': [ -19, -12], 'Label': 'H_2O'}
```

And finally we can generate our plot using these 4 variables of data.

```python
system, SE = mu_vs_mu.calculate(data, bulk, deltaX, deltaY)
ax = system.plot_phase(cbar_title="$H_2O$ $/$ $nm^2$", figsize=(6, 5))
plt.savefig("../../../docs/source/Figures/Surfaces_1.png", dpi=600)
```

This plot is a good start in that the relative stability of the surfaces has been evaluated. However the chemical potential values are essentially meaningless and can be dependent on the pseudopotentials. The first thing that we can do is add in the DFT energy of species X/Y, this will give us a 0K phase diagram.

```python
Zero_K, SE = mu_vs_mu.calculate(data, bulk, deltaX, deltaY, x_energy=-4.54, y_energy=-14.84)
ax = Zero_K.plot_phase(colourmap="RdYlBu", cbar_title="$H_2O$ $/$ $nm^2$", figsize=(6, 5))
plt.savefig("../../../docs/source/Figures/Surfaces_2.png", dpi=600)
```

```python

```


# Exercises for Machine Learning with Python, Lecture 1: *Linear Least Squares Regression*

## Before you begin the exercises:

The datafiles are text files stored here (also try and download them to your own computer, open and see what they contain):

Features: https://www.dropbox.com/s/pf2sfiy9l86xhww/boston_features.txt

Prices: https://www.dropbox.com/s/j7flze0oe86pr6o/boston_prices.txt

*   Start by downloading the dataset:

```
# Download the features and target (the prices) to Google Colab using !wget
!wget -O boston_features.txt https://www.dropbox.com/s/pf2sfiy9l86xhww/boston_features.txt
!wget -O boston_prices.txt https://www.dropbox.com/s/j7flze0oe86pr6o/boston_prices.txt
```

    --2020-05-13 15:34:07--  https://www.dropbox.com/s/pf2sfiy9l86xhww/boston_features.txt
    Resolving www.dropbox.com (www.dropbox.com)... 162.125.82.1, 2620:100:6032:1::a27d:5201
    Connecting to www.dropbox.com (www.dropbox.com)|162.125.82.1|:443... connected.
    HTTP request sent, awaiting response... 301 Moved Permanently
    Location: /s/raw/pf2sfiy9l86xhww/boston_features.txt [following]
    --2020-05-13 15:34:08--  https://www.dropbox.com/s/raw/pf2sfiy9l86xhww/boston_features.txt
    Reusing existing connection to www.dropbox.com:443.
    HTTP request sent, awaiting response... 302 Found
    Location: https://ucfcce27fc0a2ecd38bd118e07c6.dl.dropboxusercontent.com/cd/0/inline/A3o5VSYBFQREn6bIHoMp8eCHzv-dlsHUxOhSzn0e-mGjJGXBIOBRccuEbsU007U_y6mY9lRWkNQOcGfZ-DeO-7KzLELA6md-IqkcjPs3uhHf7EpGEcnaMMu7NAhKw9gb2qs/file# [following]
    --2020-05-13 15:34:08--  https://ucfcce27fc0a2ecd38bd118e07c6.dl.dropboxusercontent.com/cd/0/inline/A3o5VSYBFQREn6bIHoMp8eCHzv-dlsHUxOhSzn0e-mGjJGXBIOBRccuEbsU007U_y6mY9lRWkNQOcGfZ-DeO-7KzLELA6md-IqkcjPs3uhHf7EpGEcnaMMu7NAhKw9gb2qs/file
    Resolving ucfcce27fc0a2ecd38bd118e07c6.dl.dropboxusercontent.com (ucfcce27fc0a2ecd38bd118e07c6.dl.dropboxusercontent.com)... 162.125.82.6, 2620:100:6032:6::a27d:5206
    Connecting to ucfcce27fc0a2ecd38bd118e07c6.dl.dropboxusercontent.com (ucfcce27fc0a2ecd38bd118e07c6.dl.dropboxusercontent.com)|162.125.82.6|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 164450 (161K) [text/plain]
    Saving to: ‘boston_features.txt’

    boston_features.txt 100%[===================>] 160.60K  --.-KB/s    in 0.05s

    2020-05-13 15:34:09 (3.43 MB/s) - ‘boston_features.txt’ saved [164450/164450]

    --2020-05-13 15:34:11--  https://www.dropbox.com/s/j7flze0oe86pr6o/boston_prices.txt
    Resolving www.dropbox.com (www.dropbox.com)... 162.125.82.1, 2620:100:6032:1::a27d:5201
    Connecting to www.dropbox.com (www.dropbox.com)|162.125.82.1|:443... connected.
    HTTP request sent, awaiting response... 301 Moved Permanently
    Location: /s/raw/j7flze0oe86pr6o/boston_prices.txt [following]
    --2020-05-13 15:34:11--  https://www.dropbox.com/s/raw/j7flze0oe86pr6o/boston_prices.txt
    Reusing existing connection to www.dropbox.com:443.
    HTTP request sent, awaiting response... 302 Found
    Location: https://uc63c23951c29d2f4243330fd490.dl.dropboxusercontent.com/cd/0/inline/A3oMs8OaL6sBmtAZvWsZc5w7zgsOycuEFjBIN9ybJ7O0yGEQNTh2zDhrAo0bL1c1Y9SbS0o7vU4Fx8cVzgGAkkMNB_2ESIui9_YaJ6gaUNUiaScThF_-p8wGvq76PNYqpCU/file# [following]
    --2020-05-13 15:34:11--  https://uc63c23951c29d2f4243330fd490.dl.dropboxusercontent.com/cd/0/inline/A3oMs8OaL6sBmtAZvWsZc5w7zgsOycuEFjBIN9ybJ7O0yGEQNTh2zDhrAo0bL1c1Y9SbS0o7vU4Fx8cVzgGAkkMNB_2ESIui9_YaJ6gaUNUiaScThF_-p8wGvq76PNYqpCU/file
    Resolving uc63c23951c29d2f4243330fd490.dl.dropboxusercontent.com (uc63c23951c29d2f4243330fd490.dl.dropboxusercontent.com)... 162.125.82.6, 2620:100:6032:6::a27d:5206
    Connecting to uc63c23951c29d2f4243330fd490.dl.dropboxusercontent.com (uc63c23951c29d2f4243330fd490.dl.dropboxusercontent.com)|162.125.82.6|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 12650 (12K) [text/plain]
    Saving to: ‘boston_prices.txt’

    boston_prices.txt   100%[===================>]  12.35K  --.-KB/s    in 0s

    2020-05-13 15:34:12 (105 MB/s) - ‘boston_prices.txt’ saved [12650/12650]

This time we will not be using a Pandas DataFrames, because we will code the Linear Least Squares Regression ourselves using Numpy.

*   Now, load the txt files into NumPy arrays:

```
import numpy as np

features = np.loadtxt("boston_features.txt")
prices = np.loadtxt("boston_prices.txt")
```

As you can see below, the features is a matrix contatining 506 rows (one for each house), each with 13 entries (one for each feature). Additionally, there are 506 prices (again, one for each house).

```
print(features.shape)
print(prices.shape)
```

    (506, 13)
    (506,)

```
print(features)
print(prices)
```

    [[1.58760e-01 0.00000e+00 1.08100e+01 ... 1.92000e+01 3.76940e+02
      9.88000e+00]
     [4.54192e+00 0.00000e+00 1.81000e+01 ... 2.02000e+01 3.74560e+02
      7.79000e+00]
     [7.52601e+00 0.00000e+00 1.81000e+01 ... 2.02000e+01 3.04210e+02
      1.93100e+01]
     ...
     [2.68380e-01 0.00000e+00 9.69000e+00 ... 1.92000e+01 3.96900e+02
      1.41000e+01]
     [1.09590e-01 0.00000e+00 1.19300e+01 ... 2.10000e+01 3.93450e+02
      6.48000e+00]
     [1.54450e-01 2.50000e+01 5.13000e+00 ... 1.97000e+01 3.90680e+02
      6.86000e+00]]
    [21.7 25.  13.  23.6 21.7 35.2 19.3 14.1 18.9 13.6 19.6 42.8 25.   9.7
     20.7 19.3 37.6 13.1 21.7 16.5 29.4 18.3 15.6 48.3 36.2 21.4 15.2 18.2
     17.8 19.6 25.  18.  32.4 50.  29.8 18.6 23.9 10.5 19.3 11.9  6.3 14.4
     23.8 29.  22.   8.5 24.5 36.  43.5 16.5 20.4 22.5 23.1 37.  20.3  7.4
     11.7 24.5 23.  25.  28.  21.4 20.3 13.8 20.  33.8 13.5 18.8 21.  23.2
     26.4 20.1 14.6 17.1 17.8 27.5 15.  18.9 23.  20.5 19.1 12.  19.2 19.1
     16.2 27.1 12.7 22.  19.3  7.2 21.7 17.4 20.2 35.1 50.  17.8 31.6 16.1
     15.  21.9 13.9 19.4 23.1 24.3 22.8 10.4 13.4 25.2 15.6 21.7 20.6 20.1
     18.7 12.7 11.7 19.8 24.7 23.7 20.9 46.  30.3 23.6 27.5 50.  10.2 22.
     23.1 24.3 17.9 18.9 18.5 21.2 22.7 43.8 18.5 21.2 22.9 50.   5.  21.8
     23.9 23.1 18.5 21.5 17.2 28.6 15.1 24.1 21.   8.7 21.7 31.5 14.4 21.7
     10.4 19.5 15.7 16.4 20.9 19.6 17.8 17.3 50.  12.8 25.  30.1 10.8 15.6
     14.6 19.4 28.7 23.2 18.7 23.5  7.2 22.6 10.9 23.9  9.5 16.1 24.3 23.7
     21.1 21.2 21.9 22.  28.1 35.4  8.1 14.1 12.7 15.4 23.2 12.1 34.7  7.
     24.  17.6 13.4 29.9 28.4 20.6 50.  44.  13.4 21.5 48.5 32.2 26.5 21.2
     22.6 17.7 13.2 12.3 46.7 41.3 11.8 21.8 23.1 19.4 18.8 32.  17.8 22.6
     43.1 22.5 17.2 16.6 14.8 27.1 22.  25.3 13.1 19.4 21.4 13.1  8.3 19.6
     19.8 29.1 33.4 24.  37.2 22.8 38.7 29.1 19.7 24.4 23.2 22.5  8.5 34.9
     15.  27.5 20.4 23.3 29.6  8.8 33.3 10.5 20.1 19.5 31.6 13.3 16.1 15.6
     18.1 14.3 34.6 22.2 13.3 17.  15.6 33.1 23.8 22.7 25.1 35.4 13.1 22.6
     23.7 12.5 20.  20.3 19.1 20.4 14.9 20.3 31.7 17.1 32.7 24.4 17.5 24.8
     23.1 11.5 24.2 18.9  8.4 22.9 14.9 33.2 31.5 13.5 50.  19.  42.3 14.5
     44.8 26.7 22.4 17.5 13.4 28.7 12.6 19.8 21.4 19.3 24.1 21.  23.3 23.8
     21.1 50.  16.3 36.4 30.7 24.4 23.  23.1 16.7 14.5 26.4 36.2 20.6  8.4
     16.2 20.1 10.2 17.1 19.4 33.4 32.  19.6 27.5 18.7 21.2 23.9 22.2 34.9
     18.6 18.2 39.8 26.6 23.3 15.4 50.  11.  20.5 19.4 18.4 37.3 21.6 20.
     11.3 50.  26.6 27.  18.4 11.9 33.  30.5 17.5 19.5 31.1 26.2 14.3 21.4
     24.8 13.9 15.2 14.9 25.  13.8 10.2 33.1 48.8 20.6 17.2 11.8 17.4  5.
     21.9 36.5 13.3 50.  23.7  8.3 24.8 23.9 25.  27.9 32.9 50.  19.1  8.8
     50.  33.2 29.  50.  41.7 28.4 24.6 20.5 25.  22.9 22.2 17.4 20.8 16.8
     19.9 21.6 19.7 22.3 24.1 24.7 19.2  7.2 50.  34.9 22.8 13.8 22.6 20.7
     15.2 10.9 14.1 20.  20.1 19.   9.6 31.2 28.7 16.8 28.5 18.5 19.5 36.1
     14.2 22.8 30.8  7.5 16.7 24.5 20.8 22.9  5.6 22.  45.4 20.4 19.9 22.2
     16.   7.  28.2 24.8 19.9 23.4 22.1 23.8 13.6 22.2 26.6 29.6 23.4 24.4
     50.  32.5 18.4 13.8 23.  30.1 31.  24.7 20.6 18.2 37.9 20.8 20.  20.2
     15.3 14.5 14.  29.8 13.8 30.1 22.4 27.9 20.6 16.6 22.3 19.9 24.6 18.3
     22.  23.3]

Beside the housing prices, the dataset contains and information about location, etc (see below), and the prices are units of in USD $1000.

```
Index   Description
-----------------------------------------------------------------------
   0    per capita crime rate by town
   1    proportion of residential land zoned for lots over 25,000 sq.ft.
   2    proportion of non-retail business acres per town
   3    Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
   4    nitric oxides concentration (parts per 10 million)
   5    average number of rooms per dwelling
   6    proportion of owner-occupied units built prior to 1940
   7    weighted distances to five Boston employment centres
   8    index of accessibility to radial highways
   9    full-value property-tax rate per $10,000
  10    pupil-teacher ratio by town
  11    1000(Bk - 0.63)^2 where Bk is the proportion of African Americans by town
  12    % lower status of the population
```

## Exercise 1.1: Exploratory Data Analysis

Before we begin the regression, lets have a look at the data. As in the lecture, we will plot the correlation between our features using Seaborn.

Since we loaded in the data as a Numpy Array, we have to create a Pandas DataFrame for Seaborn. This can be done with the code below:

```
#Pandas DataFrames
import pandas as pd

# Make a Pandas dataframe from the features
df = pd.DataFrame(features[:,:5])

# NOTE: ONLY TAKES THE FIRST 5 COLUMNS/FEATURES
# features[:,:5] is Numpy slice notation for "take all rows, and first five colums"

# Print the type of the dataframe, just to make sure we coverted correctly
print(type(df))
```

    <class 'pandas.core.frame.DataFrame'>

Note that since there are 13 features, the correlation plot would contain 169 plots. This would be quite slow in Google Colab. For speed reasons, we only loaded the first 5 featues into the pandas dataframe for plotting.

### Question 1.1.1:

*   Just like in the lecture, make an `sns.pairplot()` using Seaborn of the dataframe we just made.
*   (Optional) If you want, change the style of the plot as shown during the lecture. You can see some customization options in the documentation: https://seaborn.pydata.org/generated/seaborn.pairplot.html

```
import seaborn as sns
import matplotlib.pyplot as plt

# Plot the pairplot of the Pandas DataFrame
sns.pairplot(df)
```

### Question 1.1.2:

One thing you might note from the pair-correlation plot is that some of the features look odd. However, these features still work well for linear regression.

*    Can you explain why the correlations between Feature 3 and everything else look odd?

**Give your answer:**

## Exercise 1.2: Training and Test sets

In order to train and evaluate our linear regression model, we first have to divide the dataset into two parts: A training set and a test set.

### Question 1.2.1:
A common practice is to have the training set be 70% of the full set and test set 30%.

There are 506 houses in our dataset, so divide your data into 354 in the training set and 152 in the test set.

You can use Numpy's slice notation for this, as introducted in the first lectures. For example, to take the first five *rows* from an array you can use the following notation:
```
data_first_five_rows = data[:5]
```

To complete this Question, do the following task:
*    Divide both of the numpy arrays `features` and `prices` into two parts.
*    Use, for example, the Numpy slice notation to achieve this.
*    Make sure that the same house in never in both the test and training set at the same time! Other than that, you are free to choose which rows to the test set and which go to the training set.

```
# Make the two sets of features below
features_train = features[:354]
features_test = features[354:]

# Make the two sets of prices below
prices_train = prices[:354]
prices_test = prices[354:]
```

## Exercise 1.3: Regression Fit

In the linear least squares regression, we solve an equation that looks like this:

$$ \mathbf{y} = \mathbf{X} \mathbf{\alpha}$$

Where $\mathbf{y}$ is a vector containing the target labels (for example housing prices), $\mathbf{X}$ is a matrix where the each row contain features (in our case the features for a given house in Boston) for each house. Finally, $\alpha is a vector containing the (unknown) regression coefficients.

Fortunately, Numpy has built-in capabilites for solving linear least squares regression. This is done using the function which performs the "least squares" fit:

```
alpha, residual, rank, singular_values = np.linalg.lstsq(X, y)
```

As you can see, the function actually returns four items, the first one being the regression coefficients, and the next three are the residuals of the fit, the rank of the matrix problem, and finally the singular values of the matrix.

For this exercise we only need to use the first one, but feel free to print and look at the three other items as well.

You can see the documentation for `np.linalg.lstsq()` here: https://numpy.org/doc/stable/reference/generated/numpy.linalg.lstsq.html

### Question 1.3.1:

*    Just like in the code above, use `np.linalg.lstsq()` to fit the alpha regression-coefficients.

**Hint:** Use the `features_train` instead of `X` and `prices_train` instead of `y`.

**Note:** *Google Colab uses a version of Numpy which might give you a warning abour "rcond", which is not a problem.*

```
# Get the alpha coefficients
alpha, residual, rank, singular_values = np.linalg.lstsq(features_train, prices_train)

print(alpha)
```

    [-0.07637644  0.04115234 -0.02369435  2.54631417  0.04041712  6.10926255
     -0.01919318 -0.87295404  0.11448186 -0.01157476 -0.35137899  0.00886255
     -0.35929319]

    /usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:2: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
    To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.

## Exercise 1.4: Making predictions

Now that you have obtained the alpha coefficients in the previous question, it is time to use them to make predictions.

Remember how we are approximating the price as a linear sum of the weighted features for a given house:

\begin{equation}
f(\mathbf{x}) = x_1 \alpha_1 + x_2 \alpha_2 + \dots + x_n \alpha_n
\end{equation}
or written in vector notation:
\begin{equation}
f(\mathbf{x}) = \mathbf{x} \cdot \mathbf{\alpha}
\end{equation}

where $\mathbf{x}$ is the vector containing all the features for the house and $\mathbf{\alpha}$ is the vector of regression coefficient you obtained by fitting the features to the prices of the training set.

### Question 1.4.1:

With the alpha regression coefficients you found in the previous exercise, you can use your linear least squares "machine" to predict the price of house that are not in your training set.

As in the equation above, this is done by taking the dot product between the feature vector for a given house and the regression coefficients.

Remember, that the features for each house are stored in the rows of the "features" numpy arrays you created in Question 1.2.1.

*    Calculate the price of all the houses in the test set. That is, take the dot product between the regression coefficients and every row-vector of the `features_test` array.

```
# Calculate the prices of the houses in the test set
#, i.e. the dot product between "features_test" and "alpha"
prices_test_predicted = np.matmul(features_test, alpha)

print(features_test_predicted)
```

    [27.08891037 19.40768589 40.73604442 14.74809218 20.47140433 22.12963816
     17.05747905 34.9702988  24.73185704 15.56529702 12.39884119 39.47063809
     26.34010844 28.61534347 22.11361184  3.18403021 26.81028117 29.09876667
     18.02380111 21.17361141 31.58247512 26.04626572 17.78535658 19.79125809
     25.79350711 18.89456458 20.62019168 17.26913808 22.17780792 15.96777779
      5.25537392 31.83600946 42.52094819 18.79530044 16.28459705 14.39752525
     17.50229859  9.0630825  23.69766619 34.76964504 14.92175461 29.42299817
     12.68160434 11.4095486  25.7283809  29.78209888 27.53029616 16.43034755
     32.05528062 44.44065123 15.40174248  1.21258613 35.17710926 35.93682876
     30.56069276 21.6886704  39.78606429 29.72725493 26.13632751 20.21009359
     25.8010759  21.61063321 24.40576075 19.57518593 16.05095174 20.67048149
     16.76494214 24.63346547 20.76028385 24.28298716 24.66337165 25.75692176
     21.80124907 11.74836368 41.98719941 33.14918355 24.39680145  4.74327622
     23.36998542 24.57347741 18.3432605  14.36578646 18.5252126  19.8870277
     16.3526096  22.28521677 16.13198458 29.8903321  31.30195767 19.71691996
     33.29908759 20.45076518 17.35093391 33.99351059 17.42632737 26.18499206
     31.13949275 15.73894941 19.74703888 21.12887375 21.90239835 27.44500634
     12.79628345 26.46402109 38.11585401 22.75966775 20.39941232 24.555675
     18.4844471   8.61226014 31.6148623  29.58540095 18.2835844  23.97817612
     25.6524885  25.41707772 13.87415922 24.39300916 29.18818841 27.79792975
     23.25900244 25.36008706 16.60072053 29.060445   18.97941877 -1.63033123
     27.78769895 27.72044381 34.21961564 24.84762105 20.09799795 17.55782539
     33.15523117 15.25499585 21.50440107 22.50788458 18.98674393 19.43606724
     16.70870136 25.44730446 12.73039663 27.88586237 21.97523425 31.48189927
     23.32963543 18.15092789 25.61402465 19.69604668 26.75515522 19.15657774
     28.17345794 22.77691171]

*Hint:* Save the predicted prices for the test set in a list or numpy array, which you can use in question 1.4.2.

### Question 1.4.2:
Since we already know the true price of the houses in ou dataset, we can now compare the true prices of the house

Use for example `plt.scatter()` as we did in the introductory lectures.

*    Plot the correlation between the true prices stored in the `prices_test` and the predicted prices.

Does it look like there is some correlation?

```
# Write the code to plot the correlation between "prices_test" and the prices
# you predicted in Question 1.4.1.

plt.scatter(prices_test, prices_test_predicted)
plt.show()
```

### Question 1.4.3:

One common measure of the prediction error is the "root mean squared error" (RMSE) between the true values and the predicted values. The RMSE is calculated as follows:

\begin{equation}
\mathrm{RMSE} = \sqrt{\frac{1}{N}\sum_{i=0}^{N} \left( y_i^\mathrm{true} - y_i^\mathrm{predicted}\right)^2}
\end{equation}

*    In this question, calculate the RMSE for your test set, i.e. the RMSE between the true prices and the prices you predicted in Question 1.4.1.

```
# Calculate the RMSE
error = prices_test - prices_test_predicted
rmse = np.sqrt(np.mean(error**2))

print("RMSE [x $1,000] = ", rmse)
```

    RMSE [x $1,000] =  5.662741307904864

## Exercise 1.5: Linear Regression with Scikit-learn

As we saw in the lecture, the Python library Scikit-learn has capabilities for almost any type of macine learning you can think of.

It also does Linear Regression. Below is the a code example to use linear regression with sklearn.

```
# Import the machine
from sklearn.linear_model import LinearRegression

# Make a linear regression machine identically to our fitting in Numpy
machine = LinearRegression(fit_intercept=False)

# Fit the machine using the training features and training labels
machine.fit(x_train, y_train)
```

After the machine has been fitted, you can use the built-in `predict()` method to make predictions on new feature maps:

```
# Predict y-values using x_test features
y_test_predicted = machine.predict(x_test)
```

### Question 1.5.1:

*    In this question, use Scikit-learn to train a `LinearRegression` machine on the housing dataset. Use the same training and test data which you used in the previous example.

**Hint:** You use the above code snippets to solve this question.

```
# Implement the linear regression with Scikit-learn

from sklearn.linear_model import LinearRegression

machine = LinearRegression(fit_intercept=False)

machine.fit(features_train, prices_train)

prices_test_sklearn = machine.predict(features_test)

print(prices_test_sklearn)
```

    [27.08891037 19.40768589 40.73604442 14.74809218 20.47140433 22.12963816
     17.05747905 34.9702988  24.73185704 15.56529702 12.39884119 39.47063809
     26.34010844 28.61534347 22.11361184  3.18403021 26.81028117 29.09876667
     18.02380111 21.17361141 31.58247512 26.04626572 17.78535658 19.79125809
     25.79350711 18.89456458 20.62019168 17.26913808 22.17780792 15.96777779
      5.25537392 31.83600946 42.52094819 18.79530044 16.28459705 14.39752525
     17.50229859  9.0630825  23.69766619 34.76964504 14.92175461 29.42299817
     12.68160434 11.4095486  25.7283809  29.78209888 27.53029616 16.43034755
     32.05528062 44.44065123 15.40174248  1.21258613 35.17710926 35.93682876
     30.56069276 21.6886704  39.78606429 29.72725493 26.13632751 20.21009359
     25.8010759  21.61063321 24.40576075 19.57518593 16.05095174 20.67048149
     16.76494214 24.63346547 20.76028385 24.28298716 24.66337165 25.75692176
     21.80124907 11.74836368 41.98719941 33.14918355 24.39680145  4.74327622
     23.36998542 24.57347741 18.3432605  14.36578646 18.5252126  19.8870277
     16.3526096  22.28521677 16.13198458 29.8903321  31.30195767 19.71691996
     33.29908759 20.45076518 17.35093391 33.99351059 17.42632737 26.18499206
     31.13949275 15.73894941 19.74703888 21.12887375 21.90239835 27.44500634
     12.79628345 26.46402109 38.11585401 22.75966775 20.39941232 24.555675
     18.4844471   8.61226014 31.6148623  29.58540095 18.2835844  23.97817612
     25.6524885  25.41707772 13.87415922 24.39300916 29.18818841 27.79792975
     23.25900244 25.36008706 16.60072053 29.060445   18.97941877 -1.63033123
     27.78769895 27.72044381 34.21961564 24.84762105 20.09799795 17.55782539
     33.15523117 15.25499585 21.50440107 22.50788458 18.98674393 19.43606724
     16.70870136 25.44730446 12.73039663 27.88586237 21.97523425 31.48189927
     23.32963543 18.15092789 25.61402465 19.69604668 26.75515522 19.15657774
     28.17345794 22.77691171]

### Question 1.5.2:

*    In this question, make a scatter plot between the between housing prices you predicted with Scikit-learn and the true prices.
*    Additionally, calculate the RMSE for prices you predicted with Scikit-learn.

**Hint:** If everything went well, you should have gotten the exact same plot and RMSE as you found in Questions 1.4.2 and 1.4.3.

```
# Make a scatter plot for  the housing prices you predicted with
# Scikit-learn and the true prices for the test set

plt.scatter(prices_test, prices_test_sklearn)
plt.show()
```

```
# Calculate the RMSE for the housing prices you predicted with
# Scikit-learn and the true prices for the test set

# Calculate the RMSE
error = prices_test - prices_test_sklearn
rmse = np.sqrt(np.mean(error**2))

print("RMSE [x $1,000] = ", rmse)
```

    RMSE [x $1,000] =  5.662741307904864